Master UCSI BootCamp vAPTOS 3110 PDF

Lesson 1
Introduction to Cisco’s
UCS Platform
Overview
In this lesson you will learn about Cisco’s Unified Computing System Platform.
You will understand new trends in the datacenter and understand how Cisco has
designed the UCS hardware and software to be at the forefront of these trends.
Objectives
The specific objectives of this lesson are to enable you to perform the following
tasks:
• Introduce Data Server Trends
• Introduce the UCS system hardware, and address how it is addressing
these trends
• Introduce the UCS management hardware and software
Introduction to Unified
Computing System
Platform
© 2009 Cisco Systems Inc. All rights reserved. UCS Technical Training – Overview
In this lesson you will learn about Cisco’s Unified Computing System Platform. You will
understand new trends in the datacenter and understand how Cisco has designed the UCS
hardware and software to be at the forefront of these trends.
Upon completing this lesson you will be able to:
• Introduce Data Server Trends

• Introduce the UCS system hardware, and address how it is addressing these trends
• Introduce the UCS management hardware and software
1-1 UCSI Boot Camp © Cisco Systems, Inc.

Scalability: The server evolution
Scale Up Scale Out Scale In
Monolithic servers Commoditized servers Bladed servers
Large number of CPUs Few CPUs Multi-core CPUs
Proprietary platform X86 platform X86 platform
Proprietary OS Commoditized OS Commoditized OS
Many apps per server 1 App / server
1 App / virtual machine
The 90’s Early 2000’s Now

High cost Servers under-utilized Management complexity
Large failure domain Power & cooling Limited scale
Scalability: The server evolution
The first servers were designed as large rack mounted systems that could at times be quite
large in size (thus being termed “Big Iron”). At this time the components were expensive
so a single large system that could service large numbers of users was the most cost
effective means to provide services to users.
The next change in servers came with commoditized servers. These were smaller servers
that cost far less than the monolithic server, but they also had far fewer processing
resources thus limiting them to a single service or application per physical server.
Today the push for servers is to do more with less. Servers are being designed into
smaller form factors utilizing shared resources. Additionally the advent of virtualization
software, many servers are being consolidated onto single physical computing resource.
While this aids in many of the challenges faced by the datacenter this does not solve all
challenges
© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-2

Scalability: Changes in server design
Redundant power and networking
power
LAN/SAN/
serial
Scalability: Changes in server design
One of the major changes was the move from single self contained servers to multiple
smaller bladed servers in a single enclosure. Larger servers require more space and also
had to have its own power supplies, serial console connectivity, networking, and SAN
cables. Individually this may not seem like a problem but if we have to provide identical
infrastructure for 100 of these servers, the result is a complex and messy infrastructure to
try and manage and maintain.
By putting the compute resources of a server in a bladed form factor within a enclosure
we gain the following advantages:
• Maximum use of physical space

• Shared power distribution
• Shared networking and storage access
• More efficient cooling

Converged Network Fabrics
With the increase in servers has come the massive increase in the amount of cables in the
data center. There have been a number of companies who have found over the course of
years that their raised floors are completely filled from bottom to top with cabling. This is
exacerbated because network caballing is not limited to Ethernet, but also includes SAN
as well. Depending on the server, it may have dual Ethernet connections and dual SAN
connection for protection. That is 4 network cables per server not to mention other cable
connections like power and console cables. A standard rack could contain as many as 45
servers, which means as many as 180 network cables.
Recently Cisco has introduced into the market products that combine Ethernet and Fibre
Channel over the same cables. The Nexus switching line uses 10 gigabit Ethernet to
encapsulate Fibre Channel frames within Ethernet frames which results in the ability to
reduce your cabling needs in half. The resulting savings helps with space, power, and
cooling in the data center.

CPU and Memory Density
single core core core
core
CPU core core core
single socket, single core single socket, dual core single socket, quad core
addressable
memory
CPU and Memory Density
Today server processor are multi-core. Offering multiple processing cores in the same
space as previously used for single-core processors. In addition to this many vendors
have also created memory controllers that have the ability to address large amounts of
RAM. While the blade sizes have remained the same, the processing and memory has
increased significantly allowing blades to run processing and memory intensive
application.
Terminology:
• Socket refers officially to the physical chip receptacle on the mother board. It is used
informally to refer to the physical CPU module or chop itself (the unit of ordering, or
the FRU).
• Core is still a physical processing element embedded in the module (socket). An OS

running on a server will always see the number of processors or CPU’s as equal to the
number of cores or more.

The reason an OS could see more processors than existing cores is hyperthreading.
CPU’s have the ability to have each core emulate two or more processors. These
emulated processors are called threads indistinguishable from physical processors to the
CPU.
On the Nehalem architecture on the Cisco B-series blades, a “hyperthreading” setting is

enabled by default, and it doubles the number of processors visible to an OS. Therefore,
by default, an OS running in the blades will see 8 processors (threads) per socket.
The word CPU, all by itself, is almost always ambiguous. Do you mean “physical
module” (socket), or physical processor (core), or processor visible to the OS (core or
thread). You just have to agree what you are talking about when you use the word CPU.
An OS will always report the number or processors at the “lowest granularity”, ie

“processors visible to the OS”. So this will always be equivalent to cores or threads.
Software licensing is sometimes “per socket” (for example, VMware ESX, and Cisco
Nexus1000v). Sometimes it is “per core” (for example, Oracle DB)
There is nothing that prevents it from being “per thread”, but this author can’t think of
any examples.

Statelessness
WWN WWN
MAC MAC
BIOS BIOS
firmware firmware
Statelessness
State in a server includes the behavior and identity elements that make that server unique
compared to all others. These items include:
• Identity
• WWN
• MAC
• UUID
• Behavior
• Boot order
• Firmware versions
• QoS
A failed server moving its operating system and application to another server would
require that the new server personality changes be manually changed by network admin,
SAN admin, and server admin in order for the software to be able to run correctly.

In this second example, if the servers have the ability to be stateless personality settings
of the first server can be applied to the second server automatically at failover, thus
allowing the operating system and application to start without administrative
intervention.

Virtualization
virtual machines
hypervisor
Virtualization
This technology has allowed companies to more easily consolidate servers in the data
center. With the commoditized server model, one physical server was needed for an
application. With virtualization many servers, (each running in independent virtual
machines), can run on a single physical server. This is possible through the creation (in
software) of virtual hardware that can function like a server made of physical hardware.
There are a number of advantages to using virtualization including better use of
computing resources, higher server densities, and seamless server migrations.

Centralized Management
Server Orchestrators / Manager of Manager
Chassis Mgr Chassis Mgr Chassis Mgr
Network Mgr Network Mgr Network Mgr
Server Mgr Server Mgr Server Mgr
Vendor A Vendor B Vendor C
Centralized Management
As server sprawl has increased the challenges of managing a diverse environment has
become more difficult. To solve this both server vendors and third party companies have
developed software to help with many of the aspects of the systems management.
This does however present new challenges as it does create a myriad of software layers
and integrations in order to get it to work properly. Providing an easy integration of
vendor server management software and infrastructure management software is essential.

UCS Hardware
Components
© 2009 Cisco Systems, Inc. All rights reserved. UCS Technical Training – Overview
In this section, we will introduce each of the hardware components that comprise a UCS
system. Specifically, the learning objectives are
• Describe logical and physical perspectives of UCS

• Describe the blades
• Describe the mezzanine cards
• Describe the hard disk drives
• Describe the fabric interconnects
• Describe the expansion modules
• Describe the chassis
• Describe the fabric extender
• Describe the backplane

Logical View of UCS
SAN LAN
Logical View of UCS
The graphic depicts a fully-populated UCS system, which includes the following:
• 2 fabric interconnects
• 1-40 chassis
• 2 fabric extenders (aka IOM) per chassis
• 1-8 blades per chassis
Note that the fabric interconnects have uplink connectivity to both the LAN and SAN
networks. Also note that support for up to 40 chassis requires use of the 40-port 6140. In
an HA configuration using 2 interconnects, both switches may be configured to actively
switch traffic simultaneously, thereby improving throughput.

Front View of UCS
2 x power supplies redundant

fabric
2 x fan modules interconnect
4 x half-width
blades
chassis
2 x full-
width blades
4 x power supplies
Front View of UCS
Note that the power supplies for both the switch and chassis are accessible via the front.
Also note that a combination of half-width and full-width blades is supported in the same
chassis. The fan modules for the switch are accessible in the front, while fan modules for
the chassis are accessed from the rear.
The diagram shows the most typical topology, with a redundant fabric interconnect. The
same two fabric interconnects connect down to multiple chassis to form a UCS system.
At time of writing the blade options are:
• Half-width blade: B200

• Full-width blade: B250
As shown in the diagram, you are free to mix and match half-width and full-width blades
in the same chassis.

Chassis
The internal name for the chassis is Santa Clara. The chassis is 32” deep and 6RU tall.
The blades and power supplies plug in from the front. The same chassis may be used for
up to 8 half-width blades, 4 full-width blades, or some combination of half- and full-
width blades. For the half-width blades, a vertical divider is used to separate horizontally
adjacent blades. For the full-width blades, this divider must be removed.
Power
There are a maximum of 4 x 2500W hot-pluggable power supplies per chassis. The
actual number of power supplies required depends on the system’s hardware
configuration and desired power redundancy. They can be configured as non-redundant,
N+1, or N+N (grid) redundant. They are single-phase 220V, IEC320-C 19. The power
supplies provide a 550W budget per slot for up to 8 half-width blades and 1100W budget
per slot for up to 4 full-width blades. While this may seem like “overkill”, the design
supports future growth and power budget requirements with no service disruption.
Cooling
The chassis cools from front to rear with a high-efficiency, high-reliability flow-through
airflow design (63% of a fully populated chassis is air). The fan modules are sized to
support all blades running at full power budget. There are 8 hot-plug dual fan modules
per chassis with status indicators on each module.
UCS feature support:

UCS features supported by the chassis include the ability to scale up to as many as 20
chassis (using the 20-port interconnect) or 40 chassis (using the 40-port interconnect).

Rear View of UCS
20/40 x fabric/border expansion
ports module bay
1 or 2
2 x cluster ports
2 x power entry
console port
1 x management port
2x fabric 8 x fan
extenders modules
4 x 10GE SFP+
fabric ports
4 x power entry
Rear View of UCS
All the cabling for a UCS for both the interconnects and the chassis (including power,
Ethernet, console, and Fiber Channel) are plugged into the rear. In such a redudant setup
as shown , the “left” FEX connects to the “A” switch and the “right” FEX connects to the
“B” switch. Note that the interconnect shown in the slide is the 20-port 6120.

Blade Overview
Differences
Type Form Factor Number of Number of

DIMM Slots Mezzanine Slots
Half-width ½ chassis width 12 1
Full-width full chassis width 48 2
Similarities
Type On Board CPU Sockets Internal Drives

Management
Half-width BMC 2 2
Full-width BMC 2 2
Blade Overview
There are two types of blades available – half-width and full-width blades. Three factors
distinguish these blades: the form factor, the number of DIMM slots, and the number of
mezzanine card slots. The blades have a common (x64 Nehalem) architecture and share
many features such as being hot-swappable in the chassis, having 2 CPU sockets, 2
internal SAS disk drive bays, and a baseboard management controller (BMC). Only one
CPU is required for normal system operation. If only one CPU is installed, then it must
go in the first socket. The CPUs must be identical on the same blade but can be mixed
between blades in the same chassis. The baseboard management controller (BMC) is a
microcontroller on the motherboard that provides “lights out” hardware status and
configurability to the system management software over dedicated Ethernet links
between the BMC and each FEX. The BMC uses the IPMI (Intelligent Platform
Management Interface) protocol over the I2C (inter-integrated circuit) serial bus to
manage devices on the baseboard. The BMC is also responsible for providing remote
KVM access to the end user through dedicated management IP addresses for each blade
exposed on the fabric-interconnect management port and NAT’-ed into the BMC.

Choice of blades provides increased scalability to the consumer with regard to:
• Number of CPUs to populate (1-2)

• Number of disk drives to populate (1-2)
• Number of mezzanine adapters to populate (1-2)
• Amount of memory to install (depending on blade, up to 384GB)
Another UCS feature supported for the blades is statelessness, where the following blade
personality elements are relocatable between blades:
• Identity elements such as MACs, WWNs, and UUIDs

• Behavior elements such as BIOS version and QoS settings
At the time of writing the B250 supports two mezzanine slots, althought the two cards
must be identical.

Half-Width Blade
Mezzanine Card
Memory Slots
Drive Bays
CPU Sockets
Half-Width Blade
The half-width blade (referred to internally as the Gooding blade) contains 2 SFF drive
bays that support SAS/SAT disks in a hardware RAID-0 or RAID-1 configuration. It has
a 550W power and cooling budget. The chassis can hold up to 8 half-width blades. The
motherboard contains two CPU sockets and 12 DDR3 DIMM memory sockets. There is
a single mezzanine adapter slot.

Memory Configurations: (B200)
Maximum B/W
10.6GB/s 10.6GB/s
E5550
CPU CPU and
above
Performance
Balanced
E5520
8.5 GB/s
CPU CPU
8.5 GB/s
and
above
Memory Configurations: B200
Maximum B/W:
1333 speeds require a 1333-capable CPU, as shown, AND will only be achieved with 1
DPC (Dimm Per Channel) --- ie you could a speed vs capacity tradeoff with 1333 (max
48GB RAM on whole blade server actually running at 1333)
• DDR3 1333MHz across 3 channels

• 1 DPC (6 DIMMs)
• Max Capacity – 48 GB

Balanced Performance:
If both slots on a channel or populated (“2 DPC”) we can run at a max speed of 1066 (the
1066 is really MTS – mega-transfers-per second)
• DDR3 1066 MHz across 3 channels

• Up to 2 DPC (12 DIMMS)
• Max Capacity: 96 GB
You can mix and match SIZES but could the best performance, in descending order, with:
• same size across whole server

• same size across whole CPU
• same size across a bank

Full-Width Blade
Mezzanine Cards
Memory Slots
Drive Bays
CPU Sockets
Full Width Blade
The full-width blade (referred to internally as the Ventura blade) has 2 SFF drive bays. It
has a 1100W power and cooling budget. The chassis can hold up to 4 full-width blades.
The motherboard contains two CPU sockets and 48 DDR3 DIMM memory sockets,
effectively quadrupling the memory capacity of the Gooding blade. There are 2
mezzanine adapter slots, effectively doubling the I/O bandwidth and providing
redundancy.

Memory Configurations: Full-width Blade (B250)
Memory ASIC
Memory ASIC
8.5GB/s 8.5GB/s
CPU CPU
• Memory ASIC controls 4 physical DIMM

• Presents to CPU as 1 DIMM (up to 32GB)
Memory Configurations: Full Width
The unique memory ASIC on B250 is able to present 4 physical DIMM slots to the CPU
as one “virtual DIMM”, up to 32GB. The CPU cannot tell the difference between the
B200 and B250 architectures. This “virtualization” of a memory slot is fully supported
by Intel.

Adapter Offerings
Virtualization Compatibility Cost
VM I/O Virtualization Existing Driver Cost Effective

and Consolidation Stacks 10GbE LAN access
10GbE/FCoE 10GbE/FCoE
Eth Eth Eth

FC FC
vNICs
0 1 2 3 127 10GbE FC For hosts that

need LAN
access
PCIe x16
PCIe Bus
Adapter Offerings
The 3 types of adapters (aka mezzanine cards) available in a UCS blade include Palo,
Menlo, and Oplin.
Palo
The Palo adapter supports up to 128 Ethernet vNICs (or FC vHBAs) running at 500
KIOPs in both initiator and target mode. The Palo adapter is a converged network adapter
(CNA) with dual 10GE ports and dual FC ports to the backplane. The Palo ASIC
provides failover between the redundant Ethernet links. No multipathing software is
required on the host OS for this functionality.
Menlo
The Menlo adapter is a converged network adapter (CNA) with 2 host-side 10GE ports
and 2 FC ports to the backplane. The 2 network ports can run either native Ethernet or
FCoE protocols and can be configured for failover. This failover is performed by the
Menlo ASIC and does not require multipathing software on the host OS. The Menlo
ASIC is a Cisco-designed multiplexor and FCoE protocol offload engine with a 350MHz
24K MIPS processor. There are 2 versions of this card: Menlo-E (with an Emulex

chipset) and Menlo-Q (with a Qlogic chipset), thereby supporting existing proven driver
stacks to the customer.
Oplin
The Oplin card is made by Intel. It has 2 10GE ports to the backplane which run native
Ethernet. OS-implemented FCoE may be run over the Oplin to provide “free SAN
access” to the host. The Oplin card does not provide failover functionality.
• Choice of the Palo adapter provides enhanced virtualization. The Palo adapter
supports up to 128 vNICs presented to the ESX server which can then be used as
VMNICs for virtual machines. Further, the personality (through port profiles which
define MAC forging, VLANs, and QoS settings) of a VM can migrate (i.e. Vmotion)
using the VNTag functionality of the Palo adapter and NX5K.
• Choice of the Menlo adapter provides increased scalability. The FC-to-Ethernet

protocol encapsulation and decapsulation is offloaded from the host and performed on
the Menlo card itself.
• Choice of any of these mezzanine cards simplifies management because of reduced

cabling/switching infrastructure and less firmware to upgrade

Hard Disk Drives
• 2 SFF drive bays plug

into SAS backplane
• 73GB 15KRPM
• 146 or 300GB 10KRPM
• RAID-0 or RAID-1
• Diskless configuration
supported (requires
SAN or LAN[PXE]
boot)
Hard Disk Drives
Both types of blades have two drive bays and an SAS backplane, regardless of whether
the blade was sold as “diskless”. The supported drives – a 73GB 15KRPM SAS SFF
drive , a146GB 10KRPM SAS SFF drive and a 300GB 10KRPM SAS dirive . No local
hard drive is required. If two hard drives are installed, then they must both be of the same
size and speed and are hot-swappable. The build-in LSI RAID controller supports
RAID-0 & RAID-1.
• UCS features supported at the disk drive include statelessness by scrubbing any data
on a disk drive during a service profile migration.
• Also, local disk policies (such as RAID-0 or RAID-1) may be configured that migrate
with logical service profiles between blades.

20 and 40 - Port Fabric Interconnect
1U
2U
20 and 40 Port Fabric Interconnect (UCS 6100)
20 – Port Fabric Interconnect

The internal name of the fabric interconnect is Springfield. The 20-port interconnect is
1U high. It has 20 SFP+ ports for 10GE/FCoE . There is 1 expansion module for
connecting to the LAN and SAN networks. The 20-port interconnect supports up to 560
Gbps fabric. None of these ports are enabled or configured by default.
40 – Port Fabric Interconnect

The 40-port interconnect is 2U high. It has 40 SFP+ ports for 10GE/FCoE. There are 2
expansion modules for connecting to the LAN and SAN networks. The 40-port
interconnect supports up to 1.12 Tbps fabric.

• Choice of either the 20-port or 40-port version of the interconnect provides scalability
to the customer
• UCS features supported by the fabric interconnects include scalability. You have the
choice of using a 20-port or (shown in the next slide) a 40-port interconnect.
• Another UCS feature supported by the interconnect is enhanced phyiscal (on the fex)
and logical port virtualization as implemented in the VNTag feature.
• The interconnect also natively supports FCoE to converge Fiber Channel and Ethernet
fabrics.
• Finally, the management server (discussed later in this module) runs on the
interconnect and simplifies management so that all UCS components may be managed
through a single pane of glass.

Expansion Modules
FC-only (8 ports
@ 1/2/4 Gbsec)
FC-only (6 ports
@ 1/2/4/8 Gbsec)
Combo FC +
Ethernet
Ethernet-only
Expansion Modules
The expansion modules (also referred to as GEMs) provide connectivity into your
enterprise Ethernet LAN and Fiber Channel SAN networks. There are three expansion
modules from which to choose: FC-only, combination FC and Ethernet, and Ethernet-
only.
• FC-Only (1/2/4Gb) - This fibre channel expansion module contains 8 SFP ports that
run 1, 2, and 4 Gbps FC.
• FC-Only (1/2/4/8Gb) - This fibre channel expansion module contains 6 SFP ports that
run 1, 2, 4 and 8 Gbps FC.
• Combo - This combination expansion module contains 4 SFP+ ports that run 10GE
and 4 SFP ports that run 1, 2, and 4 Gbps FC. It is internally referred to as a diamond
card.
• Ethernet-Only - This Ethernet expansion module contains 6 SFP+ ports that run
10GE. It is internally referred to as an emerald card.

The expansion modules support increased scalability by providing choices to the

consumer for LAN and/or SAN expansion.

I/O Module (Fabric Extender)
I/O Module
FEX
The fabric extender (also referred to as an IO module or IOM) logically extends the
fabric from the switch to the blade. The name of the ASIC in the FEX is internally
referred to as Redwood. Its primary function is to connect the blades to the left or right
interconnect. Secondarily, it houses a chassis management controller (CMC).
There are 4 10Gbps ports that can be used to connect the chassis to the switch. You
connect either 1, 2, or 4 links from the FEX to the fabric interconnect. The FEX support
from 2:1 to 8:1 oversubscription rates. The chassis contains 2 FEX slots for increased
redundancy and bandwidth, though only one FEX is required for minimal configuration.
If only one FEX is installed, it must be placed into the “left” bay. If both are used, they
are hot-swappable and fully redundant. The FEX provides 10Gbps per blade port (there
are 2 ports per blade). Each blade connects to both FEXs in the chassis via the
mezzanine card.

CMC
The FEX has a chassis management controller (CMC) which is a key part of management
infrastructure. The CMC collects status data from the FEX using the IPMI (Intelligent
Platform Management Interface) protocol over the I2C (inter-integrated circuit) serial bus
and then communicates that information to the management node using the Ethernet
server link. The CMC controls power supply and fan speeds. The CMC serves as a proxy
for UCS manager to the blades for certain functionality. We will also see that the CMC
plays a part in the HA protocols.
Diag port
The FEX has a diag port used for “depot-level” troubleshooting.

The FEX supports increased scalability by multiplexing an up to 8:1 oversubscribed
blades to ports ratio.

Backplane
I/O
Modules
PSU Blade
Connectors Connectors
Redundant data and management paths

Backplane
The backplane provides redundant power to the FEXs and blades that plug into it. The
backplane also provides redundant data network (Ethernet) connectivity between the FEX
and the blades. Last, the backplane has redundant management (I2C) paths and
dedicated management network (Ethernet) connectivity and supports auto-discovery of
all components plugging into it.
The backplane supports up to 1.2 Tbps on the data plane. While the back plane is largely
passive it does have 3 SEEPROMS on it used for managing the HA capabilities of the
6120.

Introduction to UCS
Management
Introduction to UCS Management
This section will introduce the UCS management model. We will briefly look at the
management architecture, high-availability and interfaces.
• Management architecture
• What is UCS Manager and where does it run?
• What does UCS Manager not do
• Introduction to UCS Manager high-availability
• UCS Manager interfaces
• Opt-in deployment models

High-Level Management Architecture
management redundant managed

interfaces management service endpoints
switch elements
UCSM
UCSM
chassis elements
server elements
multiple protocol
support
redundant management plane

UCS Manager is the name of the management service for all the UCS components. As
such, users can manage the entire UCS system through a single pane of glass, thereby
reducing management complexity. UCSM runs on the management node. In an HA
configuration, there is a redundant instance of UCSM running on the second. Managed
switch elements include GEM modules and ports. Managed chassis elements include the
CMC, fan modules, and power supplies. Managed server elements include disks, mezz
cards, BMC and BIOS.

UCS Manager and XML
GUI
XM L
configuration
CLI state
Cisco UCS
API
Manager
operational
3rd party state
tools
UCS Manager and XML
UCS Manager’s native language is XML. Both the GUI and CLI are written using UCS
Manager’s XML API. 3rd party tools can also use this same XML API to integrate with
UCS Manager. When user’s submit management configuration requests for a managed
device to UCS Manager, UCS Manager will call the appropriate device manager to
deploy the management request. A device’s operational state is communicated back to
UCS Manager and reflected in the UI.

Overview of HA in UCS
L1 to L1
L2 to L2
Overview of HA in UCS
Configuring two interconnects in your UCS can provide redundancy for both the UCS
Manager functionality as well as the switching functionality that the interconnect
provides. You are not required to run in HA mode, but if you do, you must follow
certain rules. First, you must connect the L1 port of the “left” switch to that of the “right”
switch and L2 of left to L2 of right. These two Ethernet links are used for cluster
protocol traffic only. Second, you must connect the “left” IO module to the “left” switch
(using between 1 and 4 server links) and connect the “right” IO module to the “right”
switch.
Each UCS Manager instance either runs as the ‘primary’ or ‘subordinate’ – an election
algorithm runs at switch startup to determine which is which. All management
information is replicated between the two instances over the private network so that it
persists across failovers. A “floating” management IP address is configured for UCS
Managewr and is run on the primary instance.
While a UCS Manager instance running on an interconnect is either primary or

subordinate, both interconnects may be configured to switch Ethernet and FC traffic
simultaneously. In other words, as management nodes, the switches run “active-passive”
but as switches they may run “active-active”.

UCS Manager Principal Interfaces
GUI Navigation CLI Equivalent to GUI
UCS Principal Interfaces
GUI
The GUI is the principal access mechanism to UCS Manager. The GUI is a Java
Application downloaded and launched (using Java Webstart technology) from a web
server inside UCS Manager. It runs as a standalone application (it can survive death of
the browser that launched it, for example). Since it is 100% Java, it should be nearly
identical on a variety of client OS’s and platforms.
• Downloaded from UCS Manager using Java Webstart

• Runs as standalone GUI
• Changes made in GUI are refle
CLI
The CLI is a distinct, secondary interface to UCS Manager. Almost all tasks can be
performed either in the GUI or the CLI, although the two are independent (for example,
the GUI does not invoke the CLI). Configuration and changes made and committed in

the CLI are reflected immediately in the GUI (and vice versa). Both the CLI and GUI
subscribe to the event channels of the other.
• Use ssh (default) or telnet directly to UCS Manager IP

• Independent control mechanism from GUI
• Changes made in CLI are reflected in GUI

Out-of-the-Box Protocol Support
SNMP SMASH CLP Call-home
IPMI CIM XML
Remote KVM UCS CLI and GUI
Serial Over LAN UCS XML API
UCSM Protocol Support
UCS Manager supports a variety of industry-standard protocols for monitoring,

configuration, and call-home. Some of these interfaces (e.g. Remote KVM) are cut-
through interfaces that “go around” UCS Manager while others (e.g. Call-home) use UCS
Manager for configuration and activation. Configuration and detail about these protocols
is covered in a later module in this course.
The UCS XML API (and corresponding SDK) allow you to perform custom wrappers
around UCS configuration and monitoring. There are several use cases for this:
• Creating a multi-tenant portal with your own

presentation/authorization/authentication mechanisms
• Integrating with custom or industry-standard orchestration tools
• Populating external CMDBs to include management changes in a UCS
environment into your enterprise management system
• Scripting and integrating into custom management solutions.
• Extending the base functionality provided by UCS Manager

Opt-in Models for UCS Management
 Basic
– 1:1 mapping from server identities/behaviors to servers
– Uses built-in (hardware-derived) identities
 Stateless computing
– 1:many mapping from server identities /behaviors to
servers
– Uses pools, policies, and templates to assign identities and
behaviors to servers
 Multi-tenancy
– 1:1 or 1:many mapping from resources to organizations
– Uses RBAC to control privileges
UCS Manager Opt-in Models
In order to facilitate different levels of adoption for UCS features, customers can choose
from essentially three different management paradigms: basic, stateless, and multi-
tenancy.
Basic
In this paradigm, the user can treat blades as physical servers in an analogous fashion as
one would would a rack-mounted server. Consumers who require that the operating
systems, applications, and users using a server in a UCS are confined to a particular blade
could exploit this model. While this is the easiest model to adopt, it does not make use of
many of the features UCS provides (namely mobility and organizational sharing of
servers).
Stateless computing
In this paradigm, the user does not care on which physical blade an operating system or
application runs. Instead, users can pool servers together and then have UCS Manager
automatically select a server that satisfies the physical or logical requirements of a user

(e.g. amount of memory installed, CPU capacity) as well as identity requirements (e.g.
WWN, MAC) and behavior (e.g. firmware versions, boot order, … etc).
Multi-tenancy
In this paradigm, users (or “tenants”) of the same UCS can all share its resources. UCS
Mnager uses role-based access control (or RBAC) to grant or deny particular access to a
particular resource for a particular user based on privileges assigned to a role.

Overview of Server Deployment in UCS
Configuration Service Profile Physical

Methods Blade
Identity
(MAC Address,
WWN, Etc.)
Behavior
(Firmware,
QoS, Etc.)
Other
• Manual
• Automatic (vNICs,
• Default vHBAs, Etc.)
Overview of Deployment in UCS
Pools
Identity pools such as MAC and WWN pools allow the server administrator to pull from
a pre-defined set of identity addresses and assign them to servers. Server pools allow the
server administrator to pull from a pre-defined set of blades that satisfy certain criteria
(such as RAM and CPU capacity).
Policies
Policies allow the server administrator to consume pre-defined behaviors such as boot
order and QoS settings.
Templates
Templates allow the server administrator to consume predefined profile, NIC, and HBA
configuration.

Profiles
The service profile is the heart of the UCS Manager. It represents all the identity and
behavior of a server. When you need a blade with a particular identity and set of
behaviors, you simply assign a service profile to that blade. Insodoing, UCS Manager
configures a physical blade with the appropriate behavior and identities with LAN and
SAN connectivity.

UCS C-Series Servers
UCS C-Series Servers
UCS C-Series servers are not yet part of UCS integrated management. They are
standalone rack-mounted versions with the exact same CPU/Memory horsepower as
their B-series equivalents.
There are plans to integrate C-series rack-mounted servers into UCS integration
(connected to the 6100 fabric interconnects, either directly or through a next-generation
fex). This will likely occur in late CY 2010

C-Series Overview
 Expands UCS into rack

mount market
UCS C250 M1  Multiple offerings for different
work loads
• C200 - 1RU rack-mount server
• C210 – 2RU large internal storage
• C250 – 2RU Memory Extending
UCS C210 M1  Offers Path to Unified
Computing
UCS C200 M1
C-Series Overview
Cisco expands the UCS platform to now includes a series of servers that are in a rack
mount form factor. These models are known as the Unified Compute System C-Series
Rack mount servers. Consumers that do not use or prefer the rack mount architecture
will also be able to take advantage of the many benefits of Cisco’s UCS system without
being forced to use the B-Series Blade architecture. At launch not all UCS integration
will be available for the c-series servers, but Cisco plans to complete this integration in
CY2010.
The C-Series offers 3 models:
• C200 – First model introduced. 1RU rack-mount server designed to balance simplicity,
performance, and density for production-level virtualization, web infrastructure, and
other mainstream data center workloads
• C210 – Large internal storage model. designed to balance performance, density, and
efficiency for workloads requiring economical, high-capacity, reliable, internal storage
• C250 – Large memory server. designed to increase performance and capacity for
demanding virtualization and large-data-set workloads

Because the C-Series is designed to be part of the UCS system, it offers many of the same
advantages:
• Simplifying management – Use existing tools to manage the server or UCSM (when
available in 2010)
• Unified I/O – The C-Series will use Converged Network Adapters as well as fibre
channel and Ethernet adapters
• Virtualization – The C-Series will have Virtualization adapter like the B-series. Large
memory server is designed for large virtualized work loads

C-200 Overview
Dongle
for
2USB,
VGA,
Console
DVD
Internal
Disk
UCS C200 Front View
Console and Management

Expansion Card
Power
LOM
USB and VGA
UCS C200 Rear View
C-200 Overview
The C-200 is the general purpose high density model of the c-series line. The C-200
offers the following features:
• Up to two Intel Xeon 5500 Series multicore processors

• Up to 96 GB (DDR3)
• Up to four internal SAS or SATA disk drives; up to 2 terabytes (TB) total
• RAID capable:
• Built-in RAID 0 and 1 support for up to four SATA drives
• RAID 0 and 1 support for up to four SAS or SATA drives with optional
mezzanine card
• RAID 0, 1, 5, 6, and 10 support for up to four SAS or SATA drives with optional
LSI MegaRAID card
• Two half length x8 PCIe slots
• One full height
• One low profile
• Two integrated Gigabit Ethernet ports and one 10/100 Mbps management port
• KVM access

• Front-panel interface with video, two USB, and serial port connections.
• Back-panel video, two USB, and serial port connections
• Dual redundant power

Dongle
C-210 Overview for
2USB,
VGA,
Console
DVD
Internal
Disk
UCS C210 Front View
Console and Management Expansion Card
Power
UCS C210 Rear View LOM

USB and VGA
C-210 Overview
The C-210 is the general purpose, high density, and large internal storage model of the c-
series line. The C-210 offers the following features:

• Up to 96 GB (DDR3)
• Up to 16 internal small form factor (SFF), (SAS), or (SATA) disk drives; up to 8 TB
total
• RAID capable:
• Built-in RAID 0 and 1 support for up to four SATA drives
• RAID 0 and 1 support for up to four SAS or SATA drives with optional
mezzanine card
• RAID 0, 1, 5, 6, and 10 support for up to 16 SAS or SATA drives with optional
LSI MegaRAID card
• Five full-height PCIe slots:
• two full-height, full-length x8 PCIe card slots
• three full-height, half-length x8 PCI card slots, all with x16 connectors
• Two integrated Gigabit Ethernet ports and one 10/100 Mbps management port

• KVM access

C-250 Overview
Dongle
for
2USB,
VGA,
Console Internal
Disk
DVD UCS C250 Front View
Power
USB
Expansion Card and
VGA
UCS C250 Rear View LOM
Console and Management

C-250 Overview
The C-250 server is a high-performance, memory-intensive, two-socket, 2RU rack-mount

server designed to increase performance and capacity for demanding virtualization and
large-data-set workloads. It also can reduce the cost of smaller memory footprints. The C-
250 offers the following features:

• Up to 384GB (DDR3)
• Up to eight internal small form factor (SFF), (SAS), or (SATA) drives; up to 4 TB total
• RAID capable:
• RAID 0 and 1 support for up to eight SAS or SATA drives through optional PCI
Express (PCIe) controller
• RAID 0, 1, 5, 6, and 10 support for up to 8 SAS or SATA drives with optional
LSI MegaRAID card
• Support for up to five PCIe cards in three low-profile,
• half-length x8 and two full-height,
• half-length x16 slots
• Four integrated Gigabit Ethernet ports and 2 x 10/100 Mbps management port

• KVM access

Adapter Offerings
Virtualization
127
10GbE/FCoE
Eth
PCIe x16
FC
0 1 2 3
Eth Eth
• UCS P81e
FC
• 128 vNICs
• Uses VNlink
vNICs
• NIC Teaming done by HW
CNA
10GbE/FCoE
PCIe Bus
10GbE FC
• Emulex and Qlogic

• 2 Fibre Channel
• 2 Ethernet
• NIC Teaming through bonding driver
Ethernet or HBA
• Emulex / Qlogic (HBA)
• Broadcom Ethernet
• NIC Teaming through bonding driver
Adapter Offerings
The 3 types of adapters cards available in a UCS C-Series.
Virtual Interface Card (UCS P81e)
Cisco UCS P81E Virtual Interface Card is a virtualization-optimized Fibre Channel over
Ethernet (FCoE) PCI Express (PCIe) 2.0 x8 10-Gbps adapter designed for use with Cisco
UCS C-Series Rack-Mount Servers. The virtual interface card is a dual-port 10 Gigabit
Ethernet PCIe adapter that can support up to 128 PCIe standards-compliant virtual
interfaces, which can be dynamically configured so that both their interface type
(network interface card [NIC] or host bus adapter [HBA]) and identity (MAC address and
worldwide name [WWN]) are established using just-in-time provisioning. In addition, the
Cisco UCS P81E can support network interface virtualization and Cisco VN-Link
technology.

Converged Network Adapters
The converged network adapter (CNA) offers and the ability to provide converged
network fabrics across 10 Gbe. The adapters have 2 host-side 10GE ports and 2 FC ports
to the CAN, but then will convert all fibre channel and Ethernet traffic from these ports to
be transported across 10 Gbe of the consumers network. These CNAs then would be
linked to a Nexus 5K as an example in order to use the FCoE capabilities. There are two
manufacturers of CAN, Qlogic and Emulex.
• Emulex OneConnect
• Qlogic QLE8152
Ethernet and/or HBAs
You can also opt for traditional HBA and Ethernet Connectivity. Fro Ethernet we offer
Broadcom NetXtremeII cards with either 2 or 4 port connectivity. For HBA we offer both
Emulex and Qlogic.
Ethernet:
• Broadcom NetXtremeII 4 port

• Broadcom NetXtremeII 2 port
HBA:
• Emulex LightPulse LPe11002

• Qlogic SANBlade QLE2462

C-Series Storage
RAID Controllers Disks

• 1 Built in controller (ICH10R) • 3.5 inch and 2.5 inch form
• Option LSI 1064e based factors
mezz controller • 15K SAS (High Performance)
• Option LSI 1078 based Mega • 10K SAS (Performace)
RAID controller • 7200 SAS (High Cap / Perf)
• 7200 SATA (Cost and Cap)
• 73GB, 146GB, 300GB, and
500GB
C-Series Storage
The C-Series has multiple internal drives, 4 on the C-200 , 16 on the C-210, and 8 on the
C-250. These drives can be controlled by an internal RAID controller. The C-series offers
a single built in RAID controller, ICH10R RAID, to do simple RAID 0 and 1. This
controller is not compatible for use with VMWare ESX Server software.
In addition to the built in RAID controller, you have the option of getting either:
• LSI 1064-based controller mezzanine card – Only offers 0 and 1 capabilities

• LSI 1078-based MegaRAID controller card – Offers 0,1,5,6,and 10 support
Disks come in multiple options. Depending on the model you have the following options
C-200
• 250-GB SATA; 7200 RPM

• 500-GB SATA; 7200 RPM
• 46-GB SAS; 15,000 RPM
• 300-GB SAS; 15,000 RPM

• 1-TB SAS; 7200 RPM
C-210
• 73-GB SAS; 6G, 15,000 RPM

• 146-GB SAS; 6G, 10,000 RPM
• 300-GB SAS; 6G,10,000 RPM
• 500-GB SATA; 7200 RPM
C-250
• 73-GB SAS; 6G, 15,000 RPM

• 146-GB SAS; 6G, 10,000 RPM
• 300-GB SAS; 6G,10,000 RPM
• 500-GB SATA; 7200 RPM

C-Series Power
PSU Options
• C-200 can operate at 450W
• C-200 and 210 650W PSU
• C- 250 750W PSU
C-200/210 C-250
C-Series Power
The C-Series has a number fo power options fepending on the model. For power
efficiency with the C-200 you can option a 450 W PSU as opposed to the 650W PSU.
Because of the potential for more disk drives the C-210 comes with 650 W PSU. Finally
the C-250 with its ability to house up to 384 GB RAM uses a 750W PSU. The 650 and
450 watt PSU have the same form factor.

C-Series Cooling
Cooling Info
• Front to Back
• Internal Fans in the C200
and 210
• Hot Swappable External
Units C-250
• PSU have separate Fans
C-200
C- 210 C-250
C-Series Cooling
The C-series have front to back cooling compliant with the warm row cool row design. In
the C-200 and 210 models the fans are internal to the unit as shown in the picture above.
This means it will require opening the system to replace the fans. On the C-250 the Fans
are module sin the front of the server and are removable and hot-swappable.

Management and UCS
C-Series
This section will introduce the UCS management model. We will briefly look at the
management architecture, high-availability and interfaces.
• Management architecture of C-Series

• What is CIMC does and does not do
• Deployment of C-series

management management service managed

interfaces endpoints
Administrative tasks
Communications
CIMC User Management
Network Configuration
Server Monitoring
Inventory
BMC IPMI
Logs
RAID Configuration
BIOS Boot Options
Server Options
Web GUI
Console
This is a very high level view of the management of the C-Series. While full integration
with the UCSM is not available currently, you can still effectively manage the server
using the same tools you use currently. Included with the C-Series are also some unique
management and monitoring capabilities.
CIMC – Cisco Integrated Management Controller
CIMC is a separate management module that is built onto the motherboard. The CIMC
has its own processor which runs the CIMC software. The CIMC is shipped with a
running version of the firmware. Users can update the CIMC firmware through the
firmware Update Management page, You need not worry about installing the initial
CIMC firmware. Users do not install a host OS like Windows or Linux on the CIMC. The
host OS runs on the Intel host processor. You install the host OS on the host hard drive
using the DVD drive, or over the network. You can use the CIMC to install the host OS
using the KVM console and vMedia. You can use a web-based GUI or SSH-based CLI to
access, configure, administer, and monitor the server. Almost all tasks can be performed

in either interface, and the results of tasks performed in one interface are automatically
displayed in another.
Tasks you can perform using the CIMC:
• Power on, power off, power cycle, reset and shut down the server
• Toggle the locator LED
• Configure the server boot order
• View server properties and sensors
• Manage remote presence
• Create and manage local user accounts, and enable remote user authentication through
Active Directory
• Configure network-related settings, including NIC properties, IPv4, VLANs, and
network security
• Configure communication services, including HTTP, SSH, and IPMI Over LAN
• Manage certificates
• Configure platform event filters
• Update CIMC firmware
• Monitor faults, alarms, and server status
Tasks you CANNOT do with CIMC:
• Deploy an OS, such as Windows or Linux

• Deploy patches for software, such as an OS or an application
• Install base software components, such as anti-virus software, monitoring agents, or
backup clients
• Install software applications, such as databases, application server software, or web
servers
• Perform operator actions, including restarting an Oracle database, restarting printer
queues, or handling non-CIMC user accounts
• Configure or manage external storage on the SAN or NAS storage
BMC Baseboard Management Controller
The BMC is basically the same as the one found on the B-Series blade servers. While this
device is not used directly like the CIMC to manage the server it will be critical for
managing and monitoring the server when we have full UCSM integration. Currently it
can be used with IPMI to control basic functions of the server as well as monitoring.

BIOS
There are a number of BIOS on the C-Series for configuring either the Server itself or the
various adapter cards that can be housed inside the server.

C-Series Management Connectivity
C-Series Management Connectivity
The principal management feature is through redundant out-of-band network connections

to each rack-mounted server’s BMC. A web-page is presented by the BMC which is
similar in nature to the “server (equipment)” and “server(logical)” views in UCS --- from
this page you can power on and power off the server, perform other maintenance tasks,
and launch the same KVM remote console tool that is used with B-series blades.
Of course on board standard video and USB console access is available as well.

C-Series Deployment Scenarios
C-Series Deployment Scenarios
This slide gives you an idea of the flexibility of these particular products with Cisco
networking products and how it'll be integrated from a UCS perspective. In the middle,
you have displayed the UCS C250, 210 and 200, and how we see it being deployed in
various scenarios either with third party products, with Cisco switches like the Nexus
5000, for example, or the Cisco Nexus 2000 Fabric Extender, or into a UCS environment
moving forward. If you look at how the B-Series kind of plays in this overall portfolio,
you've got the UCS, the blade server chassis and products to the left, and really the C-
Series allows you the flexibility to play in a standalone environment, connect to various
Cisco gear with other third party gear that may be already in place in the data center, or a
migration path to the UCS in the future. Whereas you see where the Cisco Unified
Computing system on the left, how that plays, it's kind of a system in its own and doesn't
provide some of the flexibility, although it has tremendous TCO advantage and other
advantages associated with that deployment model.

Lesson 2
Introduction to UCS User

Interfaces
Overview
The purpose of this module is to familiarize yourself with the GUI and the CLI.
Focus in the practical exercises will be on the equipment areas of the GUI and
CLI; you will get plenty of practice creating and managing logical objects in the
hands-on labs in the remainder of the course
Objectives
The specific objectives of this lesson are to enable you to explain the following
about UCS GUI and CLI:
• Become familiar with navigating the GUI

• Understand the functions of the 5 main tabs on the navigation pane
• Become familiar with the CLI and CLI scoping
• Use CLI commands to perform creation and deletion of managed
objects
UCS Manager
UI Overview
Objectives

• Understand the functions of the 5 main tabs on the navigation
pane
• Use CLI commands to perform creation and deletion of managed
objects
© 2008 Cisco, Inc. All rights reserved. DCTC v1.0—2
The purpose of this module is to familiarize yourself with the GUI and the CLI. Focus in
the practical exercises will be on the equipment areas of the GUI and CLI; you will get
plenty of practice creating and managing logical objects in the hands-on labs in the
remainder of the course.

• Understand the functions of the 5 main tabs on the navigation pane
• Use CLI commands to perform creation and deletion of managed objects
2-1 UCS I Boot Camp © Cisco Systems, Inc.

Primary view of Graphical interface
NAVIGATION PANE CONTENT PANE

Primary view of Graphical interface
The left or Navigation Pane consist of a fault summary bar across the top view and a
series of 5 tabs that offer differing views of the various managed components in the
Unified Computing System. The fault summary has four conditions ranging from critical,
major, minor and warning. The faults summary contains all the cumulative totals for the
entire UCS
An expandable branch or tree function allows the operator to traverse the various
components located in the five tabs.
The right or Content Pane consist of a top toolbar with a back button, new object creation
pulldown, options & questions buttons, information button and a debug pulldown menu.
The second toolbar in the content pane offers the operator a breadcrumbs trail of object
hierarchies already traversed with the ability to rapidly return to a previous location along
the trail. At the right most portion of this bar is the current location.
The largest part of the content pane offers granular details associated with the varying
objects that have been highlighted in the navigation pane. And at the vary bottom the
© Cisco Systems, Inc. Introduction to UCS User Interfaces 2-2

content pane are the function buttons associated with committing or saving as well as
discarding the changes made here.

Navigation Pane Tabs
(Equipment)
This shows
current UCS HW
components
Navigation Pane Tabs (Equipment)
This view displays the Equipment tab contents in the Navigation pane. The display is in a
hierarchical format that list chassis 1-40 and then fabric interconnect A and B
Subcomponents such as blades and interface cards are then expanded out from the chassis
objects to perform operations on individual components and their attributes. Each
hardware object that is highlighted then has its corresponding attributes displayed on the
content pane.
The content pane may have 1 or more tabs depending on the amount of attributes or
functions. At the top of the Navigation pane is a filter pulldown for your view. The filter
is set to “all” by default but can be set to restrict the object view to the topmost objects in
that tab.
In the case of the Equipment view the Chassis or fabric interconnects are the topmost
objects.

(Servers)
This tab is for

the creation of
logical servers
Navigation Pane Tabs (Servers)
This is the servers tab. As opposed to the equipment tab, this tab focuses on the logical
deployment of servers by configuring and managing service profiles and their associated
templates, policies and pools.
This tab is the “heart” of configuration of UCS servers. The idea is that the network
administrator and SAN administrator can spend “up front” time configuring global
network and SAN settings on the LAN and SAN tabs. After this, all the information
comes together on this servers tab during the configuration of service profiles.

(LAN)
This tab is for

network related
tasks like
creating VLANs
Navigation Pane Tabs (LAN)
The LAN tab contains network configuration that will then be visible and available for
the configuration of service profiles. This tab is focused mostly on global network
configuration that will affect UCS northbound connectivity, and configure a set of
choices, pools, and policies related to the network arena that will be available for
incorporation into service profiles.
Another way of saying this is that a lot of the information on the LAN tab is created by
network administrators and then consumed by server administrators as they set up
service profiles.

Navigation Pane Tabs (SAN)
This tab is for

SAN related
tasks like
creating VSANs
Navigation Pane Tabs (SAN)
The SAN tab contains SAN storage configuration that will then be visible and
available for the configuration of service profiles. It is very analogous to the LAN tab.
Alot of the information on the SAN tab (VSAN’s available, WWN pools) is created by
storage administrators and then consumed by server administrators as they set up
service profiles.

Navigation Pane Tabs (VM)
This tab is
specifically for
Virtual Card
(Palo) Pass-
through switch
(PTS) feature
Navigation Pane Tabs (VM)
This tab deals specifically with the Palo Pass-through switch (PTS) feature for
VMware. It is not at this time used for any other purpose.
The Palo PTS feature will involve making a connection between USCM and VMware
VCenter (the “VM Providers” setup). VM Profiles (port profiles) will be pushed to
VCenter for the attachment of virtual machines to the pass-through switch. Each
virtual machine adapter will be “wired straight through” the PTS to a Palo virtualized
NIC which will have the exact same visibility as physical NIC’s in the UCS system.

Navigation Pane
Tabs ( Admin)
This tab is for UCS

related management
tasks like RBAC
Navigation Pane Tabs ( Admin)
The Admin tab is used for administrative tasks such as setting up users and external
authentication schemes. IP management (for management functionality) only is also
controlled on this screen.

Content Pane Details
UCS
Manager
Bread
control
Crumbs
buttons
View for
Content
Pane
Fault
summary
for selected
object in
NAV Pane
Actions for
selected Content
object in Pane tabs
NAV Pane for
selected
object
(different
for each
object)
Content Pane Details
Content Pane is designed to show you information of an object you select in the Nav
Pane. Any changes to the object will be done in the Content Pane. There are a series of
tabs near the top of the Content Pane, and depending on which you select it will allow
you to monitor or change the characteristics of the object. As an example if you wish to
change the firmware, then the “installed firmware” tab would be the tab you select.
In addition to the Content Pane tabs, there is also a actions menu where you can
perform a given action for the item.
Included in the content pane are also the fault summary for the object, a “bread crumb”
view to indicate where in the Nav Pane you are and buttons for controlling UCS
manager settings.

Example Content – Tabs and Sub-Tabs
(Physical Server RAM)
Example Content – Tabs and Sub-Tabs (Physical Server RAM)
This is just a nice example of the cascading tabs sometimes available on the content
pane. This is the content pane associated with (highlighting) a physical blade server on
the nav pane. The Inventory tab gives a set of sub-tabs which have all the auto-
discovered hardware information about the blade, including the position, size, and
clock speed of each DIMM. None of this particular information is configured
manually, it is all discovered automatically.

UCS
Command Line
Interface
Command Line Interface

Management Command (?)
devwasha-A# ?
acknowledge Acknowledge
clear Reset functions
commit-buffer Commit transaction buffer
connect Connect to Another CLI
decommission Decommission managed objects
discard-buffer Discard transaction buffer
end Go to exec mode
exit Exit from command interpreter
recommission Recommission Server Resources
remove Remove
scope Changes the current mode
set Set property values
show Show running system information
terminal Set terminal line parameters
top Go to the top mode
up Go up one mode
where Show information about the current mode
Management Command (?)
You can always use the question mark to see commands available in a particular
context. If you are in the middle of typing a command (at a new word boundary), you
can use “?” to see the set of subcommands available in the exact context in which you
are typing.

Management Commands (show ?)
devwasha-A# show ?
chassis Chassis
cli CLI commands
clock Display current Date
cluster Cluster mode
configuration Show information about configuration sessions
eth-uplink Ethernet Uplink
event Event Manager commands
fabric-interconnect Show Fabric Interconnect
fault Fault
identity Identity
iom IO Module
org Organizations
security Security mode
sel System Event Log
server Server
service-profile Service Profile
system System-related show commands
timezone Set timezone
version System version
vif Virtual Interfaces
Management Commands (show ?)
This is an example of “?” in the “middle of command” context. Note the “scope
context” (discussed on subsequent slides) is also taken into account.

Management Commands (scope ?)
devwasha-A# scope ?
adapter Mezzanine Adapter
chassis Chassis
eth-server Ethernet Server Domain
eth-uplink Ethernet Uplink
fabric-interconnect Fabric Interconnect
fc-uplink FC Uplink
firmware Firmware
host-eth-if Host Ethernet Interface
host-fc-if Host FC Interface
monitoring Monitor the system
org Organizations
security Security mode
server Server
service-profile Service Profile
system Systems
vhba VHBA
vnic VNIC
Management Commands (scope ?)
The scope command is used to change the focus of the object to be operated upon. This
will allow the traversing of the hierarchical tree of objects and their child or parent
objects. Don’t forget that the show command can be used in conjunction with the scope
command to see immediately the commands available in the new position.
There are parts of the “scope tree” for physical objects and other parts for logically
configured objects.
Moving around the CLI tree using scoping is analogous to using “cd” to move around a
file system.

Management Commands (scope, where, up & top)
Nav Pane Navigation CLI Equivalent to Nav Pane
Management Commands (scope, where, up & top)
In this slide we see the difference between the GUI Nav Pane to focus on a given
object, and the CLI use of the scope command to focus on the same object. Notice 2
levels are traversed and then the up command is used to ascend one level. Then the top
command is used to return to the root level.
Also of note is the use of the command “where”. Since we have to scope to the level of
a component much like changing from one directory to another, the “where” command
is useful in determining your location in the CLI “scope tree”.

Command Line (show configuration)
devwasha-A# scope org /
devwasha-A /org # scope service-profile TryPXEInst
devwasha-A /org/service-profile # show config
enter service-profile TryPXEInst instance
associate server 1/8
enter boot-definition
enter lan
enter path primary
set vnic eth0
exit
set order 2
exit
enter storage
enter local
set order 1
exit
set descr ""
set enforce-vnic-name no
set reboot-on-update yes
exit
enter default-behavior vhba
set action none
exit
............
Command Line (show configuration)
“show configuration” is a special command. It goes down recursively from the scope in
which it is typed, and shows the configuration in the format of the commands that would
have been used to create or administer the current state the object at the current scope and
all sub objects.
The example starts at a service profile and shows how all the recursive configuration
underneath that service profile might have been created with CLI commands. The output
is a reverse engineering of the CLI commands because the configuration is not really
stored as a set of CLI commands (it is stored in XML).

Management Commands (create and delete)
devwasha-A /org # create org marketing
devwasha-A /org* # commit
devwasha-A /org # top
devwasha-A# show org
Organizations:
Name
----
/ (root)
/marketing

devwasha-A /org # delete org marketing
devwasha-A /org* # commit
devwasha-A /org # top
devwasha-A# show org
Organizations:
Name
----
/ (root)
Management Commands (create and delete)
When you actually modify the configuration, using “create” and “delete” and other
commands, you must commit your changes, as shown, to actually make the configuration
(the equivalent on the GUI is pressing an “confirm” button).
The opposite of “commit” is “discard” --- this will discard all your changes since the last
commit.

UCS
Software
Connect commands

Management command (connect)
devwasha-A# connect ?
adapter Mezzanine Adapter
bmc Baseboard Management Controller
clp Connect to DMTF CLP
iom IO Module
local-mgmt Connect to Local Management CLI
nxos Connect to NXOS CLI
Management command (connect)
The “connect” command really takes you into one of the other shells available in the
CLI. We will focus on the “local-mgmt” shell and the “nxos” shell, however here is a
summary of what these connect shells do:
Adapter
Allows a local connection to the mezzanine adapter on a given blade. Syntax, connect
adapter 1/1/1 (Chassis 1 / server 1 / Adpater 1). This would be used only for
troubleshooting.
BMC
Baseboard Management Controller is an ASIC on the mother board of each blade. This
allows you to connect to the BMC (which has a running Linux kernel) and perform
basic information gathering from the BMC. Syntax, connect bmc 1/1 (Chassis 1 /
Server 1).

CLP
Connects you to a shell for the SMASH (Systems Management Architecture for Server
Hardware) CLP (Command Line Protocol). If you have a customer who uses this
protocol they can interface to the UCS and collect information about the many
components and what their status is. The connect command puts you in a shell that
accepts SMASH CLP commands. At present it is limited in the scope of commands you
can run, mostly being the show command.
IOM
I/O Module also has an ASIC running a linux kernel. Through the connect command
you can connect to the IOM similar to doing a ssh session to a linux server. You will
have a limited number of commands you can run, but this can be useful for
troubleshooting problems on an IOM.
Local-mgmt
Local management allows the user to run commands that allow for maintenance
activities to be done on the 6120 including things like show tech-support and erasing
the current database.
NXOS
connects to the underlying NXOS CLI. From here users can run typical NXOS
commands and examine connectivity through the 6120.

Management commands (connect local-mgt)
devwasha-A# connect local-mgmt
Cisco UCS 6100 Series Fabric Interconnect
TAC support: http://www.cisco.com/tac
Copyright (c) 2009, Cisco Systems, Inc. All rights reserved.
devwasha-A(local-mgmt)# ?
cd Change current directory
clear Reset functions
cluster Cluster mode
connect Connect to Another CLI
copy Copy a file
cp Copy a file
.
.
ping Test network reachability
pwd View current directory
reboot Reboots Fabric Interconnect
rm Remove a file
rmdir Remove a directory
run-script Run a script
show Show running system information
ssh SSH to another system
.
Management commands (connect local-mgt)
“local-mgmt” tasks are mainly tasks associated with the particular fabric interconnect on
which you are typing, rather than on the UCS configuration as a whole. For example,
you do not reboot the entire UCS; instead you can reboot (from the local-mgmt shell) a
particular interconnect (the one on which you are typing).

Management commands (connect nx-os)
devwasha-A# connect nxos
Cisco UCS 6100 Series Fabric Interconnect
Copyright (c) 2009, Cisco Systems, Inc. All rights reserved.
devwasha-A(nxos)# sh run int e1/1/1

version 4.1(3)N2(1.1)
interface Ethernet1/1/1
switchport mode trunk
switchport trunk allowed vlan none
no pinning server sticky
fabric-interface Eth1/7
no cdp enable
no shutdown
Management commands (connect nx-os)
The “connect nxos” command allows for connection to the NXOS shell. This provides
the user with the full set of “show” command functionalities associated with the
underlying nxos on the fabric interconnect on which you are typing.
You do not have access to any nxos configuration commands. The nxos shell is “show
and tell” only. Actual management is always accomplished using the UCS manager.

Lesson 3
UCS Connectivity and

Networking
Overview
This module will focus both on physical connectivity of the data and management
planes within UCS, and the logical configuration of LAN and SAN connectivity
within UCS and northbound of UCS.
Objectives
The specific objectives of this lesson are to enable you to explain the following
about UCS management:
• Understand UCS network and management connectivity

• Understand and use management services provided by blade server BMC
• Understand LAN Configuration within UCS
• Understand SAN Configuration within UCS
UCS Connectivity
• Understand UCS network and management

connectivity
• Understand and use management services provided
by blade server BMC
© 2008 Nuova, Inc. All rights reserved. ICNX5 v1.0—2
This module will focus both on physical connectivity of the data and management
planes within UCS, and the logical configuration of LAN and SAN connectivity within
UCS and northbound of UCS.
• Understand UCS network and management connectivity

• Understand and use management services provided by blade server BMC
3-1 UCSI Boot Camp Sales © Cisco Systems, Inc.

UCS Connectivity: Fabric Interconnect
Mgmt 0
L1
Clustering
Clustering
L2
Mgmt 5 Console Port
© 2009 Cisco Systems Inc. All rights reserved. UCS Technical Training — Networking 3
UCS Connectivity: Fabric Interconnect
The fabric interconnect provides crossover ports (L1 and L2) that are used to connect to
the other fabric interconnect in a dual-fabric configuration only. These links are
management links only; they provide control for the clustering of the management
function and replication of the managed data. As of writing, the officially supported
configuration uses direct links between the fabric interconnect only.
There is a single network management port for external client management (both CLI,
via ssh, and the GUI) on each fabric interconnect. This is a 10/100/1000 Mb port.
Serial Console runs 9600/8/N. When the management system is running, the serial
console provides a CLI login. The only time that serial console access is absolutely
required is on initial setup of the system.
© Cisco Systems, Inc. Introduction to UCS Connectivity and Networking 3-2

UCS Data Connectivity (Data Plane)
UCS Connectivity (Data Plane)
Each fabric interconnect can connect to only one IOM (fex) inside the chassis, and each
IOM can be connected to only one fabric interconnect. At the time of writing, there is
no virtual port channel (in fact, no port channel at all) between the fabric interconnect
and the IOM.
You can choose 1, 2, or 4 downlinks from a fabric interconnect to an IOM. You do not
need to have the same number of downlinks to each chassis. In the dual-fabric
scenario, it is logical but not absolutely required that both IOM’s will have the same
number of links connected to their respective fabric interconnect.
The diagram shows IOM “satellite ports” (1-8) connected via dedicated 10gb FCOE
traces on the midplane to specific adapter slots. In the half-width blade, there is a
single adapter. A full-width blade will occupy two adjacent slots (1 and 2, or 3 and 4,
for example). If a full-width blade in slots 3 and 4 has two adapters, then the “blade
adapter slot 3” and the “blade adapter slot 4” in the diagram will refer to the two
adapters on a single full-width blade.

UCS – Connectivity: Blade Slot to Fex
Pinning
8 to 1 2 to 1
UCS Connectivity: Blade Slot to Fex Pinning
Blade adapter slots are currently pinned to IOM uplinks. While the Nexus 5K/2K
relationship has optional port-channeling on its uplinks, there is no port channel
between the UCS 6100 and the IOM.
The blade adapters are pinned to IOM/fabric interconnect uplinks as such:
1 uplink: all 8 blades (naturally)

2 uplinks: odd adapter slots to first uplink , even adapter slots to second
4 uplinks: pinning is in groups: (1,5) , (2, 6), (3. 7) (4. 8)
Note: the “first uplink, second uplink” above has nothing to do with the IOM uplink
port; rather it has to do with the first fabric interconnect port configured as a fabric
(server port) to the IOM, the second, etc.
Why do they not pin a 4-uplink scenario using (1,2). (3.4), etc? The reason is that if
you had full-width blades, that scheme would pin both adapters of a full-width blade to
the same uplink! But it’s much better redundancy to pin them to different uplinks, so

that if one uplink fails, you have connectivity through the same IOM at least to the
other adapter.
Note that if an uplink fails, blade adapter slots do not automatically get repinned.
Instead. some form of adapter failover (to be controlled either by the OS, or by UCS-
managed adapter failover) must be in place to fail application traffic over to the other
IOM (and hence the other fabric interconnect).

Adapter Connectivity Details: Oplin
Eth0
To IOM 1
Eth1
To IOM 2
Teaming done by O/S
Adapter Connectivity: Oplin
The Oplin adapter card supports a single 10Gb ethernet connection to each IOM (and
hence to each fabric interconnect). No failover is provided internally by UCS.
Redundancy is achieved only through some kind of OS teaming / grouping only. If you
have a full-width blade, of course you could configure OS redundancy across two
different adapters rather than on a single adapter.
If the OS teaming scheme involves some kind of OS-created virtual adapter with a
single MAC address, then the teaming must be active/passive only (UCS cannot
support the same MAC address simultaneously across both IOMs and fabric
interconnects).
If the teaming scheme still preserves a separate MAC address for each physical adapter,
with some other (typically layer-3) load-balancing scheme, then there could still be an
active/active redundancy scheme.

Adapter Connectivity Details: CNA
(Menlo)
To Eth0
IOM 1
Eth1
HBA0
HBA1
To
IOM 2
Failover done
by Menlo for
Eths only
Adapter Connectivity Details: CNA
The Menlo CNA provides two 10Gb ethernets. You can only use both if you have two
fabric interconnects (there is no redirection of traffic of both adapters across a single
IOM connection unless you start with two separate paths and specify that UCS should
provide failover).
For the ethernet functionality, you can choose to configure either or both adapters with
UCS provided failover. In this case you would need no OS teaming; UCS would direct
traffic of eth0, for example, to the left IO Module, but it would automatically fail the
traffic to the other IOM (and fabric interconnect) in the case of IOM, uplink, or
possibly even northbound network failure. Note that in this case the OS is completely
unaware of teaming or failover; an OS can use, for example., eth0 by itself, and UCS
will automatically provide path redundancy for this single adapter.
It is not required that you choose to use the redundancy. You could treat the two
ethernets in the CNA just like those in an Oplin card and have the OS do teaming. One
advantage of this in the half-width blade, is, once again, the ability to do teaming across
two different adapters.

The fiber channel adapters presented to the BIOS and OS inside the server with the
CNA adapter card are standard QLogic and Emulex adapters. The Menlo ASIC directs
fc0 two one IOM and fc1 to the other. Any redundancy is achieved through OS
multipathing. Once agai, in the full-width blade you can do multipathing across two
different adapters.

Adapter Connectivity Details: Palo
To Eth0
IOM 1
Eth ?
HBA0
To
IOM 2 HBA ?
1. Appear as HW Ethernet
and HBA to OS
Fail Over done
2. Total # of vNICS (HBA +Eth)
by Palo for Determined by # of uplinks
Eths only
Adapter Connectivity: Palo
The Palo virtual adapter creates virtualized ethernet NIC’s (“vnics”) and fibre channel
adapters (“vhba’s”) that are seen by the blade server BIOS and OS as real, physical
adapters. The virtual adapters are created “on demand” by specification of a service
profile. Each virtualized adapter can be defined to go to one IOM (fabric interconnect)
or the other, and you can specify for each one whether you want to UCS to provide
failover (in the same style as the CNA).
While the virtual card theoretically supports 128 virtual adapters, at the time of writing
there is a limit of 58 supported, per Palo card (combination of ethernet and fibre virtual
nics/hbas).

UCS Connectivity Summary
Management
Network
Fabric Interconnect 1 Fabric Interconnect 2
Eth0 Eth0
L1 and L2
NXOS UCSM NXOS UCSM
Eth5 Eth5
BMC BMC
Chassis Management Switch


MEZZ MEZZ
I/O Mux
I/O Mux
I/O Mux
I/O Mux
CARD CARD
Blade 1 Blade 1
BMC BMC
MEZZ MEZZ
CARD CARD
CMC Blade 8 CMC Blade 8
CMC CMC
Chassis 1 Chassis 2
UCS Management Connectivity
Note how the same downlinks between fabric interconnect and fex provide
management connectivity alongside the data connectivity. These links are the only
place in UCS where data and management traffic are multiplexed on the same physical
links.
Within the UCS IOM (fex), a chassis management switch assumes control of all
management traffic, and directs it to the BMC (service processor) for each physical
blade server. The management connectivity beyond the IOM is all completely
dedicated ; dedicated links connect to a dedicated BMC port on the blade. The BMC
management is completely out of band with respect to blade server actual operations.
UCS Connectivity Summary
While this is a bit much at first glance, the fabric interconnect connectivity is
established In the following way:

• Uplinks from IOMs to the chassis carry both traffic for the logical servers (LAN
and SAN) and all management traffic internal to the UCS
• Traffic (Both logical server and management) is multiplexed and de-multiplexed in
the IOM by the I/O mux
• Logical server traffic (LAN and SAN) are sent from the I/O mux to the mezz card
of a blade
• Management traffic is sent from I/O mux to the chassis management switch in the
IOM
• Management traffic can go from the chassis management switch to the CMC
• CMC have separate traces through mid-plane to SEEPROM
Details about the types of traffic are listed below.
Fabric A/B NXOS Networks

These networks provide connectivity between the NXOS instance in each IOM, the
fabric A/B CMCs, and the management entities on MEZZ_CARDs (and potentially
hosts in the future). The network traffic consists of L2 protocols such as the ones used
to bootstrap the IOM/IO_MODULE relationship and the VIC protocol used to support
Network Interface Virtualization.
Frames on this network are Tagged in IOM where they take on a VLAN ID of 4042.
Fabric A/B UCSM Adapter Networks

These networks provide connectivity between the UCS management software instance
in IOM and the management entities on MEZZ_CARDs for the purposes of allocating
resources, defining identities, and monitoring status of the adapter. L2 protocols are
run with both UCS management software for general adapter functionality, and L3 (IP)
is used for remote login to adapter OS’s. Frames on this network are explicitly tagged
with a VLAN ID of 4043 on the adapter and the tagging is carried through IOM. IP
addresses in this network are algorithmically determined by UCS management software
and passed to the adapter during bootstrapping.
Fabric A/B UCSM Infrastructure Networks

These networks provide UCS wide connectivity between the UCS management
software instance in IOM, the fabric A/B CMC, and all BMCs. IOM bootstraps the
process by passing out chassis numbers to each CMC as they are discovered and
integrated into the system. From there, each element derives it’s own address(es)
algorithmically. The interfaces in the CMC and BMC are exposed as virtual interfaces
tagged with VLAN 4044, and this VLAN ID is carried throughout each fabric. Due to
the way this network is connected, traffic between a CMC and the BMCs in the UCS
management software chassis are forwarded via IOM. This nuance exists so that the
CMC IP interface can be brought up early in the chassis association process to facilitate
CMC image download, if required. IOM switches are statically assigned to be the head
of fabric A or fabric B during UCS management software cluster configuration. This
configuration is stored persistently… it is not algorithmically or dynamically derived.
Fabric A/B UCS management software PXE Networks

These networks are isolated networks that are used to boot host operating systems from
images hosted by UCS management software. IP addresses in this set are passed out
through DHCP as part of the PXE process. Frames from the host are Tagged, with a
default VLAN assignment of 4047 from the logical interface which is carried
throughout IOM.

VNLink and VNtag
This section is intended to disambiguate VNLink and VNtag, and show how they are
used in UCS

What is VNLink
Marketing term (not a specific technology)
“Manage Virtual Networking at DC Level”
One Solution: N1Kv

Virtual supervisor for all VM across DC
Does not unify virtualized/non-virtualized net
Other solution: VnTag and Palo

What is VNLink
VNLink is the easiest terminology to discuss, because it is really just a marketing term
and doesn’t refer to any technology specifically. VNLink features are those features able
to expose virtual machine networking directly to the network administrator, across the
data center or across a large domain.
Nexus 1000v is one solution to VNLink. It prevents a virtual supervisor that looks just
like a N5000 controlling all of the virtual machines in N1Kv enabled ESX instances
across your entire datacenter. The network administrator is able to set policies and
control access, VLAN’s, acl’s on all virtual machines using a familiar, unified switching
interface.
Note that the N1KV solution does not really integrate or unify switching amount virtual
machines and non-virtual machines; it is for virtual machine networking specifically.

VNtag: Bridges and Interface Virtualizers
VNtag-capable switch
Int Virtualizer
VNtag: Bridges and Interface Virtualizers

VNtag is not specifically a solution for virtual machines. Instead it is mean to be able to
support an object with can multiplex ports out of actual switches without themselves
actually being switches (sound familiar) These objects are called “interface virtualizers”
because their downstream interfaces are virtualized as ports on the upstream, VNtag
capable switch. The “virtual” part has nothing to do with being connected to a virtual
machine, it has to do with a satellite port on an IV being virtualized upstream.

VNTag: Virtualizes Ports on Controlling
Switch
Sound familiar?
N2K Ports Virtualized on 5K (E100/1/1)
UCS IOM Ports Virtualized on 6100 (E1/1/1)
VnTag has nothing to do with Virtual Machines
VNTag: Virtualizes Ports on Controlling Switch

This concept should be familiar. N2K, doesn’t act like a switch. It’s ports are virtualized
and appear as “E100/1/x”, etc, on an N5K. It’s a physical port on the fex, but a
virtualized port on the N5K
Of course the UCS IOM and the UCS 6100 have the exact same relationship (IOM
inward facing physical ports to each blade are virtualized as E<chassis>/1/<slot> on
UCS6100

An Interface Virtualizer
Eg N2K, UCS IO Module
table of tags(directly indexed)
12: satellite port 1

NO MAC tables!!! (period!)

src tag added
dst tag removed
An Interface Virtualizer
An IV doesn’t have MAC tables (that is the whole point, they are too slow and involve
brute force lookup). Instead a tag (VNtag) represents each satellite port. The IV adds the
src tag on ingress (it could never have a dst tag at this point, since that is inserted by the
controlling switch (next slide). For outgoing packets, the IV maintains a directly indexed
table of tags mapping to outbound ports.
The IV then removes the tag on packet egress. The station attached to the remote port is
not VNtag aware.

A VNTag Aware Switch
Eg N5K, UCS 6100
VNtag-capable switch
virt.port table of tags(directly indexed)
virt.port 12: satellite port 1

virt.port NO MAC tables!!! (period!)

src tag added
virt.port •MAC switching on vports

•Ingress/Egress same phys port OK dst tag removed
•Learn tags for each virt port
•Remove src tags/ add dst tags
•Tagging ABSOLUTELY transparent
A VNTag Aware Switch

This switch is also called the “controlling bridge”, it is where the satellite ports on the IV
device are virtualized. All MAC switching is done between these virtual ports. While the
same packet might enter the physical downlink to the IV and exit the same physical port,
we are not violating the cardinal rule of switching because the phyiscal port is not the
switch port, the virtual port is!
A destination virtual port is chosen based on standard MAC tables (per virtual port).
The N5K then inserts the destination VNTag associated with the destination virtual port
to direct traffic through the IV.

VNTag Format
VNTag Format
The illustration shows the VNTag inserted into an Ethernet frame. A tagged frame is
represented by a distinct ethertype, which will only be intelligible to VNTag-capable
switches and IV’s. The tag is divided into the source and destination areas.

Daisy-chaining Interface Virtualizers
Imagine Daisy-Chaining Fexes/IOM?
YES! All interfaces virtualized on top switch
Tag represents OUTERMOST satellite part
Intermediate fex-like device:

Table of all tags
Only adds/strips its “own” tags

Daisy-chaining IV’s
IV’s can easily be daisy-chained (N5K does not support daisy-chained, 2K’s, but the
concept is eaisly supported).
All satellite ports on all levels of daisy-chained IOV are virtuallized at the top-level
VNtag-capable switch.
The tag value on an outbound packet represents FINAL DESTINATION satellite port.
There is no daisy chaining of tags.
An IV which receives an already tagged packet from another IV (inbound toward the
switch);
(a) leaves the src tag alone
(b) adds the src tag to its tables
An IV which receives a tagged packet outbound:

(a) uses the dst tag as usual
(b) recognizes the dst tag as “not its own”, and does NOT strip it.

Menlo/Palo Just Daisy-Chained IV’s!!
Palo just like an IOM with virtual satellite ports
To Eth0
IOM 1
Eth ?
HBA0
To
IOM 2 HBA ?
• Ports tagged/untagged just

like fex ports
• Appear as virtual ports on

top-level bridge (6100)
Menlo/Palo Just Daisy-Chained IOV’s!!

What does this have to do with UCS. Ah --- Menlo/Palo cards are just daisy-chained
interface virtualizers.
Menlo virtualizes its two physical interfaces as virtual ports on the 6100. it is not
connected directly, it is daisy-chained through another IV, the UCS IOM (which as an IV
knows exactly what to do with this daisy-chaining).
Palo creates “virtual interfaces” (virtualized hardware accessible to the blade on the PCIe
bus). The VNtag part has nothing do do with the fact that these interfaces happen to be
virtual. It would work the same way if we invented some card that supported 100
physical blade-facing interfaces. The VNTag part just virtualizes these interfaces on ports
through the controlling 6100. The Palo is connected to 6100 via daisy-chaining through
the intervening IV, the UCS IOM.

What does it all have to do with Virtual
Machines?
Vntag part: nothing
Now connect each Palo Port to Virt Machine
Now: VN-Link! (VM-specific ports on 6100)
Eth0
VM1
To Eth ?
IOM 1
VM2
VM3
HBA0
HBA ?
VM4
To
IOM 2
• Ports tagged/untagged just

like fex ports
• Appear as virtual ports on

top-level bridge (6100)
What does it all have to do with Virtual Machines?

How does this all relate to a VNLink feature? The VNtag part has nothing to do with it
directly. However, we know that the Palo virtual interfaces are represented as virtual
ports on the 6100, and managed by the network administrator across the entire UCS
domain exactly as physical machine network ports are managed.
The final glue is the Palo pass-thru switch in ESX, which is able to dedicate the Palo
virtualized adapters (which are virtualized as ports on the 6100) to specific VM’s. voila!
VN-link --- VM-specific networking is now controlled alongside and indistinguishable
from physical machine network across the UCS domain.

Nexus 1000v versus Cisco M81KR
Characteristics Nexus 1000v Cisco UCS M81KR Virtual
Interface Card
Purpose Integrate all VM switching into a single Integrate VM networking so network
supervisor managed by the network admin configuration for VM is indistinguishable
from that of a physical machine in
UCSM
Port Profiles Manage port configurations through port Manage port configurations through port
profiles profiles
• Created on VSM • Created in UCSM (VM Tab)
• Enforced on VEM • Enforced by UCSM
Local Switching Switched by VEM Locally Switch By UCS 6120
QoS Configured and Managed by VSM QoS Policies in UCSM applied to vNIC
in service profile
Network Features • PVLAN / VLAN • VPC-HM • QoS • Pass Through

• QoS • ERSPAN / SPAN • VLAN Switch
• ACL • Netflow • Mac-Forging (Hypervisor
Bypass
Supported VSM can be a VM on the ESX Host or can UCS only
Environments run on a x86 appliance
Implementation In software with kernel integration to the Implemented through HW

hypervisor.
Comparing Nexus 1000v with the Cisco M81KR

There is some confusion that develops when talking about both the Nexus 1000v and the
UCS using the Cisco M81KR virtualized adapter. This typically results from the fact that
both of these products fall under the umbrella of VNlink technologies, and ultimately
they share many of the same feature sets as seen in the table in the slide. Despite this they
are very different solutions and are implemented in very different ways.
The main purpose of both of these products is to be able to integrate all your VM
networking and management thereof through a single switch managed by the network
admin. With the Nexus 1000v this is implemented through software that is installed on
either an ESX host or a standalone x86 appliance and is used to managed networking for
VMs only. With the M81KR, this is implemented through hardware and the integration
with the UCS 6120/UCSM, and as such management is centralized but unlike the Nexus
1000v you also manage you physical networking as well through this management
source.
In the table above we have the characteristics of each solution listed and compared. It
should be noted that these products are not an either/or solution but a solution that can be
implemented separately or in conjunction with one another. In other words you could run
a ESX host on a blade in a UCS containing a M81KR adapter. You can then install the

Nexus 1KV on that same ESX host and use it for VM on there. Additionally on that same
host you could through the UCSM create vNIC that use the PTS capability designed into
the M81KR. This flexibility provides a lot of power and usefulness to both products.

BMC Management
Services

BMC External IP Addresses
• Blade mgmt service to external

client
• IP’s on ext mgmt network
• NAT’d by 6120
• Service to ext clients:
• KVM / Virtual media
• IPMI
• Serial Over LAN (SOL)
BMC External IP Addresses
The BMC for each blade server is assigned by the UCS manager an external IP address,
to provide blade management services to clients running externally to UCS. These are
the only IP addresses managed by UCS Manager itself (UCS Manager does not
currently manage any layer 3 services whatsoever on behalf of the OS on blade
servers).
The BMC external IP addresses are dedicated, one per blade server. The addresses are
not actually configured or known to the BMC itself; instead they are NAT’d through the
fabric interconnect management port.
(Note, in a dual-fabric scenario, both fabric interconnects will handle a portion of the
BMC external IP addresses, assuming both interconnects are alive).
At the time of writing there are several services provided by the BMC using these
externally visible IP addresses to an external client. We will discuss the KVM/virtual
media service on the next few slides, and the IPMI and Serial-over-LAN services later
in the course.

Server External Mgmt Addresses
Server External Mgmt Addresses
In order to be able to utilize special server communication services, you will need to
setup a management IP address for each blade in the UCS system. Fortunately this is
very easy to do. Under the Admin in the Nav Pane if you expand the Communication
Services Folder you can select the Management IP Pool object. Now in the Actions
Menu in the Content Pane you can select “Create a Block of IP Addresses” which will
allow you to create a management IP pool. You simply state the beginning address of
the range, the number of IP addresses you wish in the pool, the netmask, and the default
gateway. Once complete these addresses will be assigned automatically to newly
discovered blades.
The easiest way to do this is to use the same subnet as the management IP assigned to
the UCS Management Nodes as the management ports are used to provide access to the
blades for specific services. You can of course use any IP address scheme, but you will
need to make sure that on the management network the router that the nodes attach to
are aware of this.
The following services are available if you create this management pool and blades get
IP addresses assigned to then:

• Virtual KVM access over IP – On as soon as you apply an management IP address
• Serial Over LAN (SoL) – Available only if configured in a service profile
• IPMI – Available only if configured in the service profile

KVM
KVM
You can access KVM console by going to the Nav Pane, selecting a server of your
choice, and then from the Actions Menu in the Content Pane selecting “KVM Console”.
This will give you a KVM window with “No Signal” because the blade in this example is
not associated with a service profile and booted. You also would be able to access this
KVM console from the Service Profile view as well, assuming one has been associated to
the blade.
The KVM Console is a Video-over-IP representation of the video output on the blade.
The service is provided by the blade’s BMC via the external IP address only. Earlier in
the module, you verified this BMC IP address. You do not need to specify it to bring up
the KVM Console, but it needs to be up and contactable from the system on which you
are running the UCS GUI.
The KVM is also provided as as standalone application, although it still authenticates

through the UCS Manager.
There are shutdown / boot / reset buttons on the KVM GUI. This is useful for server
administrators who may be given access to the standalone KVM application but not have
any ability to actually log in to the UCS manager GUI itself.

KVM Console Virtual Media
ISO/IMG
KVM Console Virtual Media
The KVM console provides a sub-utility which allows you to use a physical CD/DVD or
an .iso file on the client (on the machine on which you are running the GUI and KVM
Console) which presents itself to the blade as Virtual CD medium.
The blade can boot off of this medium for installations (or for later emergency repairs).
Note that if you are using VMware on the blade, the UCS KVM console virtual media is
used by the blade itself ( for example, for installing ESX itself), not for the guest
machines. The guest machines have their own Virtual Console / Virtual media methods
just like they do in any ESX installation.
• Single Virtual CD/DVD on blade, mapped to:

• Client (PC running KVM) ISO file
• Client physical CD/DVD medium
• Single Virtual floppy on blade, mapped to:

• Client IMG file
• Client physical/virtual floppy medium
• Single Virtual removable HD on blade, mapped to:

• Client IMG file
• Client physical removable medium (USB stick)

Virtual Media Manager
Virtual Media Manager
The screenshot shows how you invoke the Virtual Media manager from the KVM
Console. In this example, we are going to use a .iso file on the client as virtual media for
the blade…

Blade Server and IOM Dongles
Blade Server and IOM Dongles
The blade server dongle gives direct access to the video, USB, and serial port of a
physical blade. The dongle is often just a “backup” or “crash-cart” blade access
mechanism, since serial, video, and USB access can be provided over the network
using the BMC services. There is typically no advantage of using the physical video
dongle; you will see the exact same video rendered as you would using the remote
KVM service. One possible advantage of the dongle, even when everything is healthy
and you have access to all BMC services, could be the desire to directly attach a USB
DVD player for more optimal access to an entire install medium.

UCS Networking
Introduction

VLAN Objects in UCS
Note VLAN “Name”

Abstracts numerical configuration from server.
VLAN Objects in UCS
Network configuration in the UCS environment is distinct from any other switching
environment. All configuration is done using managed objects in the UCS model, and
switch ports are automatically configured to match the model. There could be a
temptation to want to configure UCS 6100 as a traditional switch, although the theory
of treating the entire UCS system as a unified, managed environment using its own data
model has huge advantages.
At the center of ethernet configuration is the VLAN object. A VLAN object combines
a traditional VLAN number with a name. The “extra level of indirection” required by
VLAN names is intentional. Server-bound VLAN configuration (through vNIC, as
seen later in the module) is done using VLAN names rather than numbers. This allows
a great amount of flexibility --- VLAN objects can be deleted and recreated with the
same name and a different number without modifying server configurations. All
servers using the named VLAN will automatically be connected to the new number.
Creation of VLAN objects automatically allows the VLANs on northbound ports from
the UCS 6100 fabric interconnect. There is no separate port-for-port configuration
required for northbound ethernet.

The screenshot shows 3 configured VLAN objects. VLAN “default(1)” preexists in the
configuration and cannot be deleted (or renamed or renumbered) .
As other VLANs get created, they are automatically allowed northbound. You could
connect to the nxos shell on the command line and run a command such as “show
interface e1/1” (matching the number of an ethernet uplink) to demonstrate to yourself
how new VLAN numbers are automatically allowed northbound upon VLAN object
creation.
By default VLAN “default(1)” is the native VLAN northbound. You can change this to
a different VLAN. As discussed later, the configuration will be the same across all
uplinks on the same fabric interconnect and typically the same across both in a
redundant configuration.

Northbound Networking
VLANs: VLANs:
Native – 1 Native – 1
Allowed – 1, 39, 214 Allowed – 1, 39, 214
L2-Cluster
L1-Cluster
The drawing summarizes northbound VLANs in a redundant switch scenario. The

native VLAN (1) has not been changed from the default. All VLANs corresponding to
other VLAN objects are allowed across all multiple VLANs.

VLANs: VLANs:
Allowed – 1, 39, 214 Allowed – 1, 39, 214
L2-Cluster
L1-Cluster
The drawing summarizes northbound VLANs in a redundant switch scenario. The

native VLAN (1) has not been changed from the default. All VLANs corresponding to
other VLAN objects are allowed across all multiple VLANs.

Northbound Networking Details
• All uplink same VLAN s

• Changeable Native VLAN (NB)
• You can have different VLANs on
other 6120
• Per-switch VLAN
• Not recommended
L2-Cluster
L1-Cluster
Northbound Networking Details
Identical VLAN configuration across all uplinks on same switch is required and enforced
by the model (you have no mechanism of violating this model “manually”).
Typically you will have identical configuration on switches A and B in a redundant

switch configuration. Besides management redundancy, the whole point of the dual-
switch scenario is path redundancy, all the way from the servers through the different
switches northbound.
It is possible to have different VLANs on switches A and B (but not on ports of the same
switch). In order to accomplish this, you can create “switch-specific” VLAN objects
rather than the more typical “dual-switch” VLAN objects. Switch-specific VLANs will
affect the uplinks on that switch only.
Keep in mind the following :
• All uplink ports on same fabric interconnect (A or B) must have same VLAN
configuation (native/allowed)
• Could change native VLAN northbound
• Usually have same config. across A and B

• Possible to have different VLANs on uplinks on different fabric interconnects
• Per-switch VLANs rather than typical “dual” VLAN
• Less likely since whole point of A-B is high-availability of server connectivity

Creating New VLAN
Creating New VLAN
All that is required to create a VLAN is the name and number ,and whether it will be be
a typical dual-switch VLAN (“Common/Global”) or a rarer, switch-specific VLAN.

Northbound: Port Channels
 Just choose ports (same

switch only)
 Automatically uses LACP
 PC then behaves like
physical port
Northbound: Port Channels
Port Channels can be created for northbound ethernet traffic.
On UCS Manager, all you have to do is choose the ports for the port channel and give
the port channel a number. You do not have any choice about protocols or VLANs.
The UCS manager always configures the port channel to use LACP, the Link
Aggregation Control Protocol. As discussed on a subsequent page, northbound
configuration (on the next switch up) will have to match (ports and protocol). LACP is
not a port-channel discovery protocol, just a port-channel control protocol.
Once port channels are created, they are treated like individual uplink ports in all
respects:
• they will match allowed and native VLANs with all other ports and port channels
on same switch (and usually across switchs)
• they will be available for pinning server traffic (discussed later) just like an
individual port

Creating Port Channels
Creating Port Channels
The screenshots demonstrate how creating a northbound port channel is simply a matter
of selecting the ports. (the harder part is making sure the configuration matches on the
northbound switch. The actual implementation of this depends on your switch, but
whatever it is it must support LACP).

Northbound Networking with Port
Channels
VLANs: VLANs:
Allowed – 1, 39, 214 Allowed – 1, 39, 214
L2-Cluster
L1-Cluster
VLANs: VLANs:
Allowed – 1, 39, 214 Allowed – 1, 39, 214
Northbound Networking with Port Channels
Port channels are now added to the same illustration that we had before. Once again, in
this typical example, VLAN configuration is identical across both switches. The
drawing illustrates individual uplinks on each switch and two ports configured into a
port channel on each switch.

Matching Configurations on Switch
North of UCS
Regular Ports
– Match native VLAN
– Match allowed VLANs
Port Channels
– Match connected ports
– May have vPC northbound (transparent to 6100)
Matching Configurations on Switch North of UCS
The following summarizes all the matching ethernet configuration required on the
northbound ethernet switch(es):
• Matching native VLANs: this will actually work if the native VLAN northbound
from UCS has a mismatch with the native VLAN toward UCS on the northbound
switch, but having a mismatch would be strange. Since the traffic is untagged, it
would still work, but you would be “taking a mental translation” from the native
VLAN number southbound into UCS to the native northbound VLAN within UCS.
This is likely unhealthy and not a good practice from a management standpoint. It
is much easier to match the native VLANs.
• Matching allowed VLANs. The deal is the only VLANs that will work across the
uplink are the intersection of allowed VLANs on the endpoint ports. It is legal to
have more or fewer VLANs allowed on the northbound switch. Fewer VLANs
allowed northbound will cause VLANs which “should work” in UCS drop into the
bit bucket. More VLANs allowed northbound will appear not to exist in UCS, since
all configuration in UCS is done through the VLAN objects.
• Port Channels: ports much match, and LACP must be enabled on the port channel
northbound to match the automatic configuration in UCS.

Inbound Networking (to servers)
Controlled by vNIC object in Service Profile
Applied automatically as SP is associated
Different way of thinking of things
Inbound Networking (to servers)
Inbound networking to servers is all specified through the vNIC object in a service
profile. Note this means all connectivity into a server is specified as properties of the
“theoretical server” represented by a service profile. When this service profile is then
applied to a physical blade server, the fabric ports are automatically reconfigured to
direct the proper VLAN traffic to the server.
This is a different mode of thinking for many, who object that it is “easier to configure
the fabric switch ports directly”. However, if you think about it, this administrative
model more closely maps to the actual requirements. Actual VLAN requirements are
on servers, not on ports.

vNIC in Service Profile
vNIC in Service Profile
The vNIC object represents a “theoretical or virtual specification” for a single nic port
in a server.
When a service profile containing a vNIC is applied to a blade server with physical NIC
hardware (Menlo or Oplin card), the properties specified by the vNIC are applied to the
physical adpater (and connectivity through the fabric directed to that physically
adapter). The “v” in vNIC in this case should be thought of as “virtual specification for
a NIC”.
When a vNIC is applied to the virtual card, the story is slightly different. The presence
of the service profile vNIC causes the virtual card to manufacture the corresponding
virtualized hardware – and then the vNIC properties are applied to it as if it were a
physical device.
The illustration shows two vNICs in a single service profile, corresponding to
specification for two physical or virtualized nic adapters on whatever blade the service
profile happens to be associated with.

The properties of the vNIC specify its connectivity (A or B switch, with or without
failover), VLANs, and other adapter properties and Qos (really Class of Service)
parameters.

Inbound VLANs (for vNIC)
Vlans:
Native 39
Allowed 1, 39, 214
 VLANs same as northbound links

 vNICs can connect to more than Eth0
1 VLANs To IOM 1
 vNICs can have own native
VLAN
Eth1
– Not required to have native VLAN (0 or To IOM 2
1 VLAN native)
– Unrelated to native VLAN on uplinks
Vlans:
Native 1
Allowed 1, 39, 214
Inbound VLANs (for vNIC)
VLANs available for connectivity into blade servers via the vNICs in server profiles are
the same set that are allowed northbound (via creation of VLAN objects).
If you have only “dual” VLANs (same configuration on both switches) you will be less
confused. If you have single-switch VLANs, then these will also be available only to
non-failover vNICs specified with connectivity to that particular switch.
Any vNIC can be connected to any number of VLANs, zero or one of which can be
native (untagged down to the server adapter). Native VLAN into a server is completely
unrelated to the native VLAN on northbound links. The UCS fabric logic will
automatically make the tagging/untagging transitions.
Any non-native VLANs specified for a server (via its vNIC) must be handled by a
tagging-aware device driver in the OS.

Connecting Northbound and Inbound
Ethernet Traffic: End-Host Mode
Default is UCS 6100 is in end-host mode

No MAC address-tables maintained
northbound
UCS 6100 pins northbound
Can change to switching mode (Not
Recommended)
Connecting Northbound and Inbound Ethernet Traffic: End-Host

Mode
By default the UCS6100 is set up in “end host mode”. In this mode, UCS 6100 does
not act like a traditional switch on the northbound ports. From the point of view of a
switch upwards from UCS, the UCS 6100 will appear as an end-host that happens to be
forwarding a large variety of MAC addresses. There is no need to run Spanning Tree
Protocol (and UCS will not in end-host mode) and no chance that UCS can be the cause
of layer-2 loops.
Traffic is directed from UCS server to particular uplinks by a pinning process that is
described on the next few slides. This pinning process never takes account of the
destination (northbound) MAC address; there are no MAC tables maintained
northbound.
Return traffic to the UCS server is naturally just directed down the same pinned link,
since that is the only link on which the northbound switch would have learned the MAC
address for that particular server.

UCS 6100 does (always) act like a switch inbound toward the servers, for the purpose
of server-server communication in the same UCS system, or inbound packets from a
northbound source into a UCS server.
There is an option to change the mode of the entire UCS switch to a traditional
switching mode. This would allow direct connection of file servers and the like on
UCS northbound ports. In the absence of such a need, it is highly recommended to
leave UCS in end-host mode, for simplicity’s sake.

Automatic Pinning
vNIC in SP (going to switch A)
pinned to uplink port or channel

repinned on uplink failure
Automatic Pinning
Northbound traffic form servers is pinned to a specific uplink port or port channel, on a
vNIC by vNIC basis.
If you do not specify pinning, the pinning is chosen automatically by UCS. If there are
multiple uplinks from a switch, UCS will assign pinning to the uplinks in a round-robin
fashion for each vNIC using automatic pinning. Note that the choice of uplink on a
particular switch is never based on VLAN, since all uplinks on the same UCS switch
must support the same VLANs
The drawing illustrates automatic pinning of server traffic specified by a particular

service profile vNIC to a particular uplink.
In this (automatic) pinning scenario, if the uplink chosen for pinning fails, the vNIC
traffic (and that of all vNIC that UCS has chosen to pin to the uplink) will fail over to
the next uplink on the same switch.

Static (Manual Pinning)
You create Pin Group

– 1 port or port channel on one switch
– 1 port or port channel on each switch (for failover)
– Used only for static (manual) pinning
You then manually specify vNIC pinning in vNIC
Config
If you choose static or manual pinning, you must first create a Pin Group object to
represent the choice of a single uplink or port channel on one switches (or one on each
switch for a vNIC with failover).
The Pin Group object by itself doesn’t do anything until it is referenced by a particular
vNIC. This vNIC traffic will then be directed only to that one uplink port or port
channel on that switch.
However, a vNIC for which A-B (or B-A) failover is specified can refer to a pin group
with one uplink port or port channel on each switch. If there is failure on the chosen
uplink on one switch, traffic from that vNIC will automatically fail over to the single
chosen port or port channel on the other switch.
Traffic may require pinning if you had uplinks that were actually attached to different
layer 2 fabrics northbound. Your traffic from your server would not be directed
correctly in this case unless you pinned the traffic to the correct uplink.
Even when (as is recommended) all uplinks go northbound to the same layer 2 fabric;
you may want to choose static pinning in order to have more control over the traffic on

the uplinks. You may know you have one vNIC with heavy traffic that you want to pin
to a dedicated uplink, while pinning combinations of other vNICs to different uplinks.

Pin Group Creation
Pin Group Creation
The screenshot shows the specification of a Pin Group. It happens to reference a single
port channel on switch A and a single physical port on switch B, which may be unlikely
but still works. The Pin Group is now available for binding form a vNIC.

Pin Group: ZippythePinGroup
vNIC in SP (going to switch A)

pinned to port/port channel on A
can use no other uplink on A
can failover to B
The drawing illustrates the manual/static pinning scenario. The vNIC is manually
pinned to the Pin Group named ZippythePinGroup. Traffic from the vNIC is directly
only to the single port channel on switch A, but can fail over to the single port (in this
example) on switch B

vNIC screen in Service Profile Wizard
identity (MAC)
connectivity
and VLANs
Pin Group
(not set = auto
pinning)
vNIC screen in Service Profile Wizard
The screenshot shows the vNIC creation screen which is part of the Service Profile
creation wizard. This is presented here just to summarize the properties of vNIC which
are used to direct ethernet traffic to the UCS server.
The vNIC specifies a virtualized identity (MAC address). We will see more in later
modules how this could just be specified to use the physical nic identity, could be an
individual virtualized identity (your birthday or whatever), or could be a virtualized
identity taken from a pool.
Note the single VLAN can be connected to any of the VLANs supported on the switch,
as specified by the same VLAN objects used to configure the switch uplinks. Exactly
zero or one of the VLANs can be native (untagged down to the server).
The final part of the screenshot shown above demonstrates where you would specify
the choice of automatic pinning (pin group not set) or static/manual pinning (via choice
of a Pin Group).

UCS SAN Introduction

SAN Concepts
Similarities to Ethernet
– VSAN objects tie NB-traffic to server
– vHBA object is fc port in SP
– No switching northbound
 Automatic
 Manual pinning
Differences (simpler)
– Each uplink and vHBA on only one VSAN
– vHBA goes to IOM A or B
– No failover)
– HA through OS multipathing on multiple vHBA
SAN Concepts
SAN connectivity concepts in UCS are similar to LAN concepts and analogous to them
in many cases, although SAN is typically simpler.
VSAN objects represent the choices of VSAN northbound and down to the server.
A vHBA object in a service profile is used to virtualize identity and specify

connectivity for an fc port in a blade server to which the service profile is defined.
Today, each uplink can be associated with only one SAN, and each vHBA can be
associated with only one SAN. Each vHBA specifies connectivity to either switch A or
B but not both. UCS does not perform any failover for fiber channel. Instead, it is
typical that the OS provide highly available access to external LUNs on the SAN using
multipathing device drivers in the OS across pairs of vHBAs

VSAN Objects
ID is VSAN Number

FCoE VLAN used inbound
FCoE not used northbound
FCoE VLAN not exposed as VLAN available for vNIC
VSAN Objects
Like VLAN objects, VSAN objects have a name and VSAN ID. The name indirection
is intentional; vHBAs will refer to the VSAN by name rather than number. Deletion of
a VSAN and recreation with the same name but different number will automatically
affect traffic for all vHBAs referring to that VSAN name.
The FCoE VLAN id is an implementation detail of sorts. Since FC traffic runs over
FCOE within UCS only, we typically choose an FCOE VLAN number that does not
overlap with a “regular VLAN number”. Since this VLAN will be specific to FCOE, it
need not be allowed northbound or exposed as a VLAN that can be connected to a
vNIC.
It is actually allowed (not necessarily recommended for management sanity) to run

regular VLAN traffic and FCOE traffic over the same VLAN number.
The default VSAN uses VSAN id 1 and runs over FCOE VLAN Id 1.

VSAN Objects and Northbound FC Ports
Northbound FC must run in NPV mode
Today each uplink supports only one VSAN
VSAN Objects and Northbound FC Ports
Northbound fiber channel uplinks are set automatically (and irrevocably) to run in NPV
node. You cannot directly attach a storage array to the UCS uplink ports; you must
attach to a fiber switch and enable the NPV feature on that northbound switch (run
npiv-enable, for example, on a Cisco MDS switch).
You must choose the correct VSAN for each UCS fiber channel uplink. The uplink port
will log into the upstream fibre switch regardless, but if the upstream switch is VSAN-
aware, the uplink will not enable itself for use for UCS server fibre channel traffic
unless the VSAN setting matches the port configuration northbound.

Inbound FC (to servers)
Controlled Service Profile
1 vHBA = one FC Port
Physical Cards (Menlo-Q and Menlo-E only):
– vHBA applied to specific physical fc port
– 2 per service profile (if dual fabric)
– Physical fc port without vHBA is OS visible not usable
Virtual Card: (Palo)
– vNIC spec causes creation of virtualized hardware
– Many per service profile (*128)
Inbound FC (to servers)
Fiber Channel connections are properly exposed to blade servers via specification of
vHBA objects in service profiles, analogous to vNIC objects for LAN.
A vHBA represents a single fibre channel port, and is applied to a physical FC port on a
Menlo Adapter, or causes creation of a virtualized fibre channel port on Palo.
*Note: Theoretical Max

vHBA in Service Profile
WW Node Name
(one per Svc Prof)
vHBA in Service Profile
The diagram illustrates vHBA properties. Each vHBA as a separate identity – this is
the World-wide Port Name. The virtualized World-wide Node Name identity is a
property of the service profile as a whole.
Each vHBA specifies connectivity to only one switch (with no failover) and one VSAN.

Connecting Northbound and Inbound
Ethernet Traffic: NPV Mode
FC must run in NPV mode
UCS 6100 does not switch FC traffic
– Each vHBA is pinned to an uplink port or port channel
– Automatic pinning
 only uplink with matching VSAN
 Round-robin assignment
– Manual static pinning (analogous to vNIC with Pin Grp)
Connecting Northbound and Inbound Ethernet Traffic: NPV

Mode
Since the uplinks are required to run in NPV mode, UCS 6100 cannot itself act as a
traditional fibre channel switch. You cannot attach end-storage directly to UCS 6100
fibre channel uplinks.
Outbound traffic from a vHBA is directed to a particular uplink via automatic or

manual/static pinning. However, all pinning is restricted by the choice of VSAN;
vHBA traffic is always mapped to an uplink sharing the same VSAN specification.
Manual pinning can be accomplished via the creation of FC Uplink Pin Groups. This is
exactly analogous to LAN Pin Groups, except that there is no possibility of failover
from switch to switch.
Reasons for choosing static pinning are similar for SAN and LAN --- you may want to
dedicate a particular uplink on a particular VSAN to a heavily-used vHBA, for
example.

vHBA Configuration
vHBA identity (WWPN)
single VSAN and optional FC

uplink pin group
vHBA Configuration
This screenshot shows a piece of the service profile creation wizard that we will use
over the next several module. It highlights the identity portion of the vHBA (the WW
Port Name), as well as the choice of connectivity, VSAN, and pin group (if any).

Lesson 4
UCS Basic Opt In Model
Overview
This module discusses full usage (installing, booting, accessing) of the UCS
blade servers using the basic opt-in model. The intention of this paradigm is to
use each blade server as a traditional server, rather than taking advantage of the
“service profile mobility” and blade pooling models presented in the next module.
One main theme of this module is that the “Service Profile” configuration, whose
design is intended to support mobility and pooling, is still required even if you
want to boot, install, and use an individual blade as you would use a traditional
server.
Objectives
Upon completion participants should be able to:
• Identify the Relationship between blade servers and service profiles
• Review the primary purposes of service profiles
• Create a simple service profile for a single blade with appropriate vNIC
• Use the expert wizrd
• Understand association
Service Profiles and
the Basic Opt-in Model
© 2008 Cisco Systems Inc. All rights reserved. Course Title —Module Name 1
Using Blade Servers with the Basic Opt-in Model
This module discusses full usage (installing, booting, accessing) of the UCS blade
servers using the basic opt-in model..
The intention of this paradigm is to use each blade server as a traditional server, rather
than taking advantage of the “service profile mobility” and blade pooling models
presented in the next module.
One main theme of this module is that the “Service Profile” configuration, whose
design is intended to support mobility and pooling, is still required even if you want to
boot, install, and use an individual blade as you would use a traditional server.

Lesson Objectives

• Create a simple service profile for a single blade with appropriate
vNIC
In this lesson Students will:

• Create a simple service profile for a single blade with appropriate vNIC
© Cisco Systems, Inc. UCS Basic Opt In Model4-2

Service Profiles and Blade Servers
Service Profile: Physical
ls-RH53Oracle Blade: 5
Client Server:
RH53Oracle Identity
(MAC Address,
WWN, Etc.)
Associated to
=
Behavior
(Firmware,
QoS, Etc.)
Other
(vNICs,
vHBAs, Etc.)
Service Profiles and Blade Servers
Understanding the service profile, or “logical server” is key to understanding blade

management on UCS.
The service profile represents a logical view of a single blade server, without needing to
know exactly which blade you are talking about. The profile object contains the server
personality (identity and network information and the like, as discussed on later slides)
and connectivity requirements. The profile can then be associated with a single blade at a
time.
The concept of profiles was invented to support the notion of mobility --- or transferring
of identity transparently from one blade to another, as well as pooling concepts discussed
in detail in the next module.
Even if you intend to manage the blade server as a traditional individual server (not
taking advantage of mobility or pooling) you still have to create and manage a service
profile for the blade. While you could boot a blade without a service profile, it would
have no network or SAN connectivity.

Blade Mobility with Service Profiles
Blade Failure
Time A
Identity
LAN/SAN
Config
Time B
Service Profile:
MyDBServer
Blade Mobility with Service Profiles
This graphic is just intended to review the concept of logical server mobility. A logical
server is defined with identity (e.g. UUID, MAC/WWN addresses, VLAN/VSAN
requirements). The profile can be associated with only one blade at a time, but the
association can be changed if there is a problem with a particular blade, or hardware
maintenance required on a particular blade.
The mobile logical server concept works well when the blade’s OS and data reside on
external SAN storage. A “replacement blade” for the profile will inherit the
WWN/VSAN information and boot specifications from the profile just like the original
blade. From the point of view of the LAN/SAN infrastructure, the new blade is the
same server as the old blade, and it will be exposed to and boot the exact same SAN
LUN’s without any further reconfiguration.
Blade local disks in this scenario could still be used for temporary storage and/or swap
space.

Service Profiles: Multiple Servers
Blade Failure
Profile A1
Time A
Profile A2
Time A
Service Profiles: Multiple Servers
Every blade that is booted simultaneously must have its own service profile, even if in
your mind the profiles are very similar (exact same connectivity requirements, for
example).
There are two easy ways to create large numbers of very similar service profiles:
cloning and templates. These will be discussed in more detail in the next module.

Contents of a Service Profile
• UUID/WWNN
• Boot order
• Other Policies
Contents of a Service Profile
This is just a summary of the contents of a service profile. Examples of blade identity
information unrelated to specific network or disk adapters are the UUID and World Wide
Node Name. The profile also holds network adapter identity information. All of this
information is physically applied to the blade associated at a particular time with the
profile. In this manner, blades associated with a particular profile at different points in
time are indistinguishable from one another, from the point of view of any identity
information for the purpose of licensing schemes or network and external storage
configuration.

Profile Elements in Basic Opt-in Model
• UUID/WWNN
• Boot order
• Other Policies
Profile Elements in Basic Opt-in Model
In the basic model the transference of identity information is not an issue.
In a service profile you can use a special value of “derived” . This implies that the
hardware identities in a blade’s BIOS and adapters will be used “as is”. If you want to
supply actual values in the profile, you still can, and they still will be transferred to the
blade server when it is associated with the profile.
You still must create vNIC and vHBA objects in your profile to define connectivity of
the adapters in your blades on the network. This is discussed further later in the module.

Creating a Service Profile (GUI)
Look at simplified wizard later...
Creating a Service Profile (GUI)
There are two wizards for creating service profiles, the expert wizard and the simplified
wizard. It is helpful to step through the expert wizard in order to understand the nature
of the service profile.

Service Profile
Service Profile
This screenshot shows the first page of the service profile creation process. You must set
a unique name for the profile and an optional description.
The UUID can be virtualized (more about this in the next module) or just set to the
Hardware Default, which is sufficient for a basic service profile.

Service Profile: Storage
We’ll Focus on it more in next module
Service Profile: Storage
The body of the storage section focuses on SAN connectivity via vHBA. We will focus
more on that, and on the local storage and scrub policies, in the next module. In the
module, for simplicity’s sake, we will be creating service profiles without SAN
connectivity.

Service Profile: Networking
Simple sub-wizard ---1 vNIC per fabric
Single native VLAN per vNIC
Service Profile: Networking
The Networking page has single and expert “sub-modes”.
The simple “sub-network-wizard” allows creation of up to two vNIC’s, and lets you
choose a single Native VLAN for each (but no more VLAN’s). It also does not let you
choose a virtualized identity. This is sufficient for physical adapter cards (it can use the
hardware default as the identity), but will have to be changed for Palo cards which
require a virtualized nic identity.
Note that this simple sub-wizard only allows you to create one vNIC per fabric. If you
do not have a redundant fabric interconnect (as in the system on which the screenshot
was taken), you only have a single fabric.

Service Profile: Networking (expert)
Service Profile: Networking (expert)
The expert “sub-network-wizard” allows you to invoke the detailed form for each
vNIC. This is the form we saw in the previous module. You can specify a virtualized
MAC address, and trunk as many VLANs as you like to the vNIC, with at most one
being native.
You could also optionally choose a pin group.

Service Profile: vNIC Placement
Service Profile: vNIC Placement
This piece of a service profile is meaningful only when association with a blade with
two physical adapters (B250 full-width blade). It lets you specify with which physical
adapter each of your vNIC definitions should be associated. You can choose to let the
UCS Manager automatically perform placement.

Service Profile: Boot Order
Service Profile: Boot Order
Boot order lets you specify the boot priority as a “virtualized configuration” in service
profile – this boot order will be applied to the BIOS of the specific blade server as the
service profile is associated.
It is still possible to set the boot order manually by escaping into the BIOS control
screen as you can on any traditional server.

Service Profile: Server Assignment
Server assignment is optional in the wizard.
It is probably best practice, especially with complicated service profiles, to skip server
assignment in the wizard (or if you get to this page, leave it at “assign later”), and
examine and massage your service profile as a strictly logical configuration before
actually assigning to a blade.
On this screen you can also specify whether you want the physical server to be left
powered down after the actual service profile association process, or whether you want
UCS manager to automatically boot the server (using the boot priority specified by the
boot order), after the actual association process.

Service Profile: Policies
Service Profile: Policies
We will talk about many of the policies to which you can refer from a service profile
near the end of the next module. It is fine to click “finish” before you reach this page
for now, or just not interact with this page.

Simplified Wizard
Simplified Wizard
The simplified wizard for creating service profiles works well with the basic opt-in
model. The wizard will produce a service profile with the basic characteristics outlined
on the slide. If you want to customize the service profile once the wizard is finished
(add more vNIC’s, attach vNIC’s to different or multiple VLAN’s, etc.) you could still
do so.

Create SP From “Blade Server” View
Create SP From “Blade Server” View
From the blade server (equipment) point of view, you can pop up a window to
create a service profile and immediately associate it with that blade (you could still
design a service profile or pick a template not appropriate for the blade).
The most basic operation creates a “hardware based service profile” for the blade, using
only derived identities. vNICs and VHBAs will be based on the actual physical blade
adapter (ie it will only create those for the fabrics that you actually have, and would not
create a vHBA if the blade server adapter is an Oplin).
There are other options on this form to instantiate a template and associate with this
blade, or to branch to the expert service profile wizard. With these other two options,
there is no actual guarantee that the service profile you create is actually compatible
with this blade.

Profile: Created but not Assigned or
Associated
Profile: Created but not Assigned or Associated
The screenshot shows our profile. We have not yet specified any blade to associate with
the profile, so the Selected Blade and Associated Blade fields are blank.

Associating Blade
Actually two step process (but all done with

“Associate” command):
1. Assigning blade to service profile (just a UCS config)
 Could still fail if blade not compatible with profile
2. Blade Configuration through PnuOS
Associating Blade
When you give the command to associate a blade, UCSM first tries to assign the blade to
the configuration. This does nothing to modify the blade itself. However the assignment
still checks that a blade is compatible with a profile. If it is not compatible, it will fail.
Examples of incompatible blades:
• Too few physical network adapters for number of vNICs

• vNIC specified for failover but you have an Oplin in the blade
• vNIC specified for failover, but you have only one fabric interconnect
• vHBA specified, but you have an Oplin
Once the blade is successfully assigned, the actual association configuration process
begins. This involves UCSM causing a mini-OS called PnuOS (processing node utility
OS) to be booted on the blade.

Configurating Blade through PnuOS
Physical process of applying profile to blade
 UCSM automatically powers on blade if necessary
 UCSM automatically boots or reboots blade
 UCSM provides network boot of PnuOS to blade
 UCSM communicates over network to blade’s PnuOS
 Once finished, identity/boot order has been written to blade
 Association process also configures UCS fabric for proper
VLAN (and VSAN) traffic to reach adapter on blade
Configurating Blade through PnuOS
PnuOS servers as a “pre-OS configuration Agent” for the blade. UCSM automatically
powers on the blade and sets it up for a PXE (network) boot. UCSM itself provides the
DHCP response and PnuOS download on a special dedicated management VLAN (4047)
to the blade. Once PnuOS is booted UCSM communicates with it to configure the blade
according to the specifications of the profile (adapter identities and the like).
Once PnuOS is finished, the blade boots using its BIOS boot order (which may or may
not have been provided by the profile).

Associating Blade with Profile
Associating Blade with Profile
As depicted in the screenshot, you use the Change Service Profile Association to access a
form to select the blade. When committed, this will initiate the assignment and
association of the blade.
The “Pre-provision a slot” is interesting in that it allows you to choose to “pre-associate”

a service profile with an empty slot. When a blade server is actually inserted into that
slot, it will undergo automatic discovery, and then UCS Manager will automatically try to
associate the service profile with the blade server (there is no guarantee, still, that the
service profile and new blade server are compatible).

Profile Assigned and Associating
Profile Assigned and Associating
The GUI indicates that the selected blade is successfully assigned. There were no
incompatibilities between the blade and the profile. It is in the process of associating.You
could just wait for the association to complete (the state will change to associated). You
could also already access the blade console and actually watch it booting PnuOS, just to
make sure everything looks normal.

KVM: Watching PnuOS on Blade
KVM: Watching PnuOS on Blade
It is possible to use the KVM Console watch the blade server booting PnuOS as part of
the association process. While you could interact or interrupt PnuOS, doing so might
conceivably interfere with the association process. Even though you see a login prompt
from PnuOS, the intention is that you leave it alone and let UCS manager direct it over
the network to do its blade configuration.

Booting a Blade from Virtual Media
Booting a Blade from Virtual Media
The KVM, together with the virtual media sub-feature discussed in an earlier module, are
the “manual” way to install an OS on a blade server that has a service profile.
Note once we configure networking in the OS we are likely to have less need for the
KVM console since we can just access the OS over the network (Remote Desktop for
Windows, for example).

Managing Blade from Profile (Logical
Server) view
Managing Blade from Profile (Logical Server) view
Note that once a blade is associated with a profile, you can administer the server from the
“profile point of view”. In other words, you can treat the profile as if it were the server,
without having to even know or remember which blade you are talking about.
This is obviously more valuable in the “mobile logical server” and “pooling” paradigms,
and is a good segue to the next module.

Lesson 5
Server Profiles and

Stateless Computing
Overview
This module discusses the full implementation of the stateless computing model
using service profiles. The discussion is divided into two themes:
• Mobile (relocatable) service profiles: same logical server can be booted on
different blades at different times. When a blade is associated with a
service profile, it will inherit all its identity and boot information from the
profile. Booting off a SAN LUN may be used as an easy way to achieve
statelessness of the OS.
• Server farms using server pools: multiple very similar logical service
profiles can be used to create servers in a farm. Actual blade hardware and
identities are chosen from pools..
Objectives
Upon completion of this lesson participants should be able to:
• Understand service profiles with stateless model
• Configure VSANs, vHBAs and SAN boot
• Understand server pools
• Use virtual identity pools
• Use cloning and templates
• Summarize some of the advanced service profile policiesenclosure
Server Profiles and
Stateless Computing
This module discusses the full implementation of the stateless computing model using
service profiles. The discussion is divided into two themes:
Mobile (relocatable) service profiles: same logical server can be booted on different
blades at different times. When a blade is associated with a service profile, it will
inherit all its identity and boot information from the profile. Booting off a SAN LUN
may be used as an easy way to achieve statelessness of the OS.
Server farms using server pools: multiple very similar logical service profiles can be
used to create servers in a farm. Actual blade hardware and identities are chosen from
pools.

Lesson Objectives
•Understand service profiles with stateless model
•Configure VSANs, vHBAs and SAN boot
•Understand server pools
•Use virtual identity pools
•Use cloning and templates
•Summarize some of the advanced service
profile policies
In this lesson students will:
• Understand service profiles with stateless model

• Configure VSANs, vHBAs and SAN boot
• Understand server pools
• Use virtual identity pools
• Use cloning and templates
• Summarize some of the advanced service profile policies
© Cisco Systems, Inc. Server Profiles and Stateless Computing5-2

Blade Mobility with Service Profiles (Review)
Blade Failure
Time A
Identity
LAN/SAN
Config
Time B
Service Profile:
MyDBServer
Blade Mobility with Service Profiles (Review)
This graphic is just intended to review the concept of logical server mobility. A logical
server is defined with identity (UUID, MAC/WWN addresses, VLAN/VSAN
requirements). The profile can be associated with only one blade at a time, but the
association can be changed if there is a problem with a particular blade, or hardware
maintenance required on a particular blade.
SAN boot can be part of the design of a completely mobile logical server. A “replacement
blade” for the profile will inherit the WWN/VSAN information and boot specifications
from the profile just like the original blade. From the point of view of the LAN/SAN
infrastructure, the new blade is the same server as the old blade, and it will be exposed to
and boot the exact same SAN LUN’s without any further reconfiguration.
Blade local disks in this model could still be used for temporary storage and/or swap
space.

Identity Information for Mobile Profiles
• UUID/WWNN
• Boot order
• Other Policies
Identity Information for Mobile Profiles
In the rack-mounted paradigm it was typical to use the derived values for UUID and for
adapter identities. In the full logical server paradigm, you need to have identities
defined with the logical service profile that will then be applied to the blade.
UUID is a 128-bit number (32 hex digits, 16 groups of 2). It is supposed to uniquely
identify a component worldwide. There are various UUID generation algorithms. You
can also use a UUID suffix pool. UCS Manager will presumably automatically
generate a unique prefix so that you will be guaranteed a unique UUID for each logical
server. Today the prefix is all 0’s.
For MAC and WWN, you can also make up values, or you can use pools. Pools will
guarantee uniqueness at least among all the profiles that use the same pool.

SAN and SAN Boot
SAN and SAN Boot
SAN usage, and especially SAN boot, or often central parts of stateless model design.
There will be one (always virtualized) WW Node Name per service profile, and, in the
fully stateless model, an always virtualized WW Port Name per vHBA.
When a service profile moves from blade to blade the SAN host identities will move
with it. Service profile running on a new blade will be indistinguishable from an
external SAN fabric and storage array perspective from the same profile running on the
orignal blade. You will properly access the same external storage, without any
reconfiguration on the external fibre fabric (zones, for example) or remapping of
storage LUN’s.

Revisiting SAN Part of Wizard
Add VHBA’s
On the storage page of the expert service profile eizard, you can choose expert mode to
enter a single virtualized WW Node Name, and click the Add button to invoke the form
for the properties for each vHBA.

For each vHBA, you specify a virtualized WW Port Name. TYou also choose the
VSAN and Pin Group, if any.

SAN Boot in Service Profile Boot Order
3 Key Items:
vHBA Name
WW PN of Storage Array
LUN #
– seen by the server
– LUM mapping
is transparent
SAN Boot in Service Profile Boot Order
To specify a SAN boot in service profile, you specify a “triplet”
(vHBA name, target-array-WWPN, LUN #).
This is the LUN # as exposed by the storage array mapping to the server, based
typically on combination of the server’s (for us, virtualized) WWNode Name and
WWPort Name.
In a multipathing scenario, you would specify two different triplets in the boot order.
(vHBA1, target-WWPN-1, LUN #)

(vHBA2, target-WWPN-2, LUN #)
The intention is that this would be two different paths, on two different RAID
controllers, to the same LUN.

Moving a Service Profile to Different
Blade
1. Shut server down nicely (if possible)

2. Disassociate from Current Blade
3. Associate SP to New Blade (can do right
away)
4. Old blade has to disassociate before avail.
Moving a Service Profile to Different Blade
If you are disassociating a service profile to do maintenance on a physical blade, it is

likely you want to shut the server down nicely. If the server is still up when you
disassociate, UCSM will try to shut the server OS down gracefully, but if it cannot do
so within three minutes, the server will come crashing down.
Note that true logical server mobility is likely to work flawlessly only when the old
blade and new blade have the same type of mezzanine card (both Menlo, or both Palo).
While the same profile can easily associate with blades with different types of adapters,
the problem is that an existing OS for the profile will have storage and network
adapters already associated with particular drivers. Unfortunately, the drivers are
different for Menlo and Palo.

Service Profiles and “Farm Model”
Blade Failure OS/data on blade local disk

OR SAN
Profile A1
Time A (choose blade
and identites
from pool)
Profile A2
Time A
(choose blade
and identities
from pool)
OS/data on blade local disk
OR SAN
Service Profiles and “Farm Model”
Every blade that is booted simultaneously must have its own service profile. The
profile mechanism still facilitates multiple blade servers being treated like a farm in the
following manner:
• Pools of blades can be created manually or automatically.
• A profile can be set to choose its assigned blade from a pool.
• Identities (UUID, WWNN, WWPN’s, MAC’s) can be chosen from a pool (a unique
identity taken from the pool will still belong to a particular service profile, and move
from blade to blade if that service profile moves)
• Templates or clones can easily be used for generation of similar service profiles
simultaneous booting of multiple blades in the pool.

Server Pools Concepts
Manually populated or Auto-populated

Blade can be in multiple pools
“Associate” service profile with pool
– Will choose available (unassociated) blade or WAIT!
– Will match needs of service profile
Server Pools Concepts
Server pools lend themselves to the server farm model.
Pools can be manually populated or auto-populated using Server Pool Policies, as

demonstrated on an upcoming slide.
A blade can be in multiple pools at the same time. Whichever profile “claims” a
particular blade is its current “owner”, regardless of the number of pools it is in.
To actually use a server pool, you associate a service profile with the pool. UCS
Manager automatically selects an available blade from the pool. (An available blade is
one currently discovered but not associated with any profile, and not in the process of
being associated or disassociated).

Creating and Manually Populating Server
Pool
Creating and Manually Populating Server Pool
Manually populating a server pool using the GUI is simple and natural, as shown on the
screenshot.
Remember, a blade can be in as many pools as you like. There is no distinction made
between currently available (unassociated) blades and currently unavaiable (associated)
blades when you manually populate a pool.

Auto-Populating a Server Pool
1. Create Empty Server Pool

2. Create Server Pool Qualification
1. Mix and match various criteria:
 Blade slots
 CPU/RAM, etc.
3. Create a Server Pool Policy

“blades matching this qualification will automatically be
placed in this pool”
4. Auto-placement of blade in pool happens at
blade discovery time
Auto-Populating a Server Pool
The auto-population feature lets you:
• Specify qualifications that will be used for matching specific blades

• Specify server pool policies --- these will put every blade that matches a particular
qualification into a particular server pool
Note that server pool auto-population only happens as individual blades are discovered. If
you want it to apply to existing blades, you need to re-acknowledge them.

Creating a Server Pool Qualification
Creating a Server Pool Qualification
A qualification is a named object that can combine (within one named qualification)
various types of parameters (adapter card, memory, cpu, storage). In this example, we
are creating a qualification named “PlentyOMemoryl” which contains only a memory
qualification. Blade Servers that have at least 75000 MB of RAM (72GB or so) will
meet this qualification.

Creating a Server Pool Policy
Creating a Server Pool Policy
A server pool policy ties together a named blade qualification, such as PlentyOMemory,
as shown, with an existing server pool. As blades are discovered, those that meet the
qualification will automatically be put in the pool.

Identity Pools
UUID Pools
WW Node Name Pools
WW Port Name Pools
Mac Pools
Identity Pools
There are identity pool types for all the different types of virtualized identity.
A service profile or vNIC or vHBA, as appropriate, will specify that it wants to get the
identity from a pool. Note that this is still a virtualized identity that belongs to the
service profile until you delete the service profile or specify a different identity. It is
the service profile that is the consumer of the pool, not the physical blade. Virtualized
identities retrieved from pools still move from blade to blade along with the service
profile.
UUID – Allows you to set up a sequential pool of UUID suffixes, assigned as soon as
service profile creation is complete
WWNN Pool - Allows you to set up a sequential pool of WWN for assignment of the
Adapter Node of the service profile’s HBA, assigned as soon as service profile creation
is complete
WWPN Pool - Allows you to set up a sequential pool of WWN for assignment of the
Port Node of the service profile’s vHBA, assigned as soon as service profile creation is
complete
MAC Pools - Allows you to set up a sequential pool of MAC addresses to assign to the
serivice profiles vNICs, assigned as soon as service profile creation is complete

Creating Pools
Always specify low value, how many values
Creating Pools
The screenshot shows, as an example, creating a MAC address pool. You always
specify the lowest identity and the number of values.
Each value is tracked by the UCS manager, so it is not advisable to create gigantic
identity pools, “just for the fun of it”.
The GUI enforces specific ranges (Cisco OUI 00:25:B5, for example, for MAC
addresses). The CLI does not enforce this.

Using Pools
Always Choose the Pool Name:

“Change Association” (blade pool)
WWNN specifier (WWNN pool)
vHBA form (WWPN pool)
vNIC form (MAC Pool)
Using Pools
You reference pools from the appropriate places in service profile. A vNIC can choose
on the GUI to get its identity from a particular pool, and so on.
Note that service profile tries to consume an identity from a pool even before it is
associated with a blade. remember, the service profile is the consumer.
If the pool is empty or exhausted as you configure a service profile, the service profile
creation or change still succeeds. The service profile will try one last time to get a
virtualized identity from the pool at the time you associate with a blade. If at that time
the pool is still exhausted, the association fails, waiting for a blade or an identity to
become available. Once one becomes available, it will automatically be used by the SP
(with no further human interruption required)

Advantage of Pools
Challenge With Pools Without Pools
Managing Virtual • Associating done if Must manually assign
resources are reassign server
Identiities available resources
• Will wait until
resources arrive
Templates • Allows greater speed Must make sure to

and flexibility of make changes to
server creation created profiles to
• Can update many avoid resource
profiles by updating conflicts
template
Cloning Allows greater speed Must make sure to

and flexibility in server make changes to
creation created profiles to
avoid resource
conflicts
Advantage of Pools
Using pools can be a management advantage in all scenarios requiring virtualized

identities. This is an alternative for making up a birthday for each individual virtualized
identity, and hoping and praying you never have a conflict. With pools, you will be
actively prevented from virtualized identity conflicts within the same UCS system.
Pools have a particular advantage in a “server farm” model. Clones and templates can
give you ways of easily creating many similar service profiles to be used
simultaneously for different servers. This is detailed on the next few slides.

Service Profile Templates
Service Profile:ORC10gRAC-1
Service Profile
Template: • UUID: 10:02:03:04:05
OracleServer • Boot Order: VM / SAN
Target 50:00:00:34:FA
• UUID: OracleUUPool • Host Firmware:
• Boot Order: OrcBootPolicy 9.0.03.146
• Host Firmware: • WWNN: 20:00:00:00:10
OrcFWPolicy • WWPN: 20:00:00:00:20
• SOL: SOLPolicy • MAC: 20:03:CA:00:05
• WWNN: OrcWWNNPool
Adapter Policy: Eth0

VLAN: Oracle
Fabric: A
MAC: OrcMacPool
• UUID: 10:02:03:04:F3
• Boot Order: VM / SAN
Adapter Policy: fc0 Target 50:00:00:34:FA
VSAN: Oracle • Host Firmware:
Fabric: A 9.0.03.146
WWPN: OrcWWPNPool • WWNN: 20:00:00:00:50
• WWPN: 20:00:00:00:40
• MAC: 20:03:CA:00:FA
Service Profile:ORC10gRAC-4
Service Profile Templates
Templates are an alternate way of creating large numbers of similar profiles.
Templates come in two flavors. Initial templates are used to set the initial properties of
an instance of the template, but modification of the template does not modify existing
instances.
An updating template is a slight variation. Modification of the template does modify

existing instances of the template when you have an updating template.

Creating Template
Looks just like creating profile
Pick blade pool
Pick identity pool
Initial or Updating?
Creating Template
The process of creating a template is nearly identical to that of creating a blade profile.
You can associate the template to a server pool – in this case, a profile that is created as
an instance of the template will have a blade selected from the pool immediately. You
are likely to also choose MAC and WWN pools for the virtual adapters that you have in
the template. Once again, as a profile as created as an instance of the template, actual
values will immediately be selected from the pools.

Creating Service Profile(s) from
Template
Creating Service Profile(s) from Template
Once a template is created, you can create as many instances of the template as you
like. Each instance will be a separate logical service profile. The whole model really
only works well when the template specifies that its virtualized identities are retrieved
from pools. In the example, 44 instances of the service profile would be created
(SPWebFarm1, SPWebFarm2, and so on) and each retrieve separate identities from the
same pools.
If pools are exhausted for some of the instantiated instances, those instances will fail to
associate, but will wait for pool values to become available (and then immediately
associate without any further intervention)
They will be named using the specified prefix and a numerical suffix.

Unbinding and Rebinding From Template
SP from updating or initial template is “bound
Unbinding and Rebinding From Template
Service Profiles that are instantiated form templates are always “bound” to the
template. Even if it came from an initial template (so it is safe from future
modifications of the template), a service profile that came from a template cannot be
individually modified until you unbind it. This is pretty much a “configuration safety”
issue, “warning” the administrator that individual modifications to a service profile that
came from a template are somewhat of a violation of the intention of the template.

Service Profile and Template Cloning
All pool associations retained on cloning
Cloned SP gets new values/blade from pool
“Specific virtual identities” not cloned
Service Profile and Template Cloning
Note how cloning profiles lends itself well to the pool / server farm model. If a profile
is associated with pools (associated with server pool, or its identity associated with
MAC, WWN, and UUID suffix pools, then the clone will have these same pool
associations.
If the original is associated with server pool, the clone will also be, and a blade will be
selected for association immediately on creation of the cloned profile.
If the original is associated with (MAC, WWN) pools, specific values will be assigned
out of the pool immediately as the clone is created.

Lesson 6
Multi-tenancy:
Organizations and RBAC
Overview
In this lesson we will now examine how Organizations and the RBAC system
allow for multitenancy.
Objectives
Upon completion students will be should be able to :
• Understand how orgs and RBAC can work separately and together
• Define functionality of the hierarchical organization tree
• Make use of objects higher in the tree in a service profile
• Define locales for RBAC
• Understand roles and define new roles
• Define users that have roles in particular locales
Lesson Objectives
Understand how orgs and RBAC can work separately and together
Define functionality of the hierarchical organization tree
Make use of objects higher in the tree in a service profile
Define locales for RBAC
Understand roles and define new roles
Define users that have roles in particular locales
Upon completion students will:
• Understand how orgs and RBAC can work separately and together
• Define functionality of the hierarchical organization tree
• Make use of objects higher in the tree in a service profile
• Define locales for RBAC
• Understand roles and define new roles
• Define users that have roles in particular locales

Multitenancy Features
Organizations (Organize Resources)

– Pools
– Policies
– Profiles
RBAC
– Locales
– Roles
– Priviledges
Multi-tenancy Features
The organization structure (“orgs”) defines a management hierarchy within the UCS
system. The hierarchy is strictly for management of resources and logical objects
inside the system, it has no effect whatsoever on the actual operation of a blade and its
operating system.
The organization structure goes hand-in-hand with the Role-Based Access Control
(RBAC) delegated management features. The RBAC features not only allow fine-
grained privileges to be assigned to different management users , but also allow
restriction of a user’s privileges to a specific set of organizations.
© Cisco Systems, Inc. Multitenancy: Organizations and RBAC 6-2

Locale
User
Roles
Organization
Sub-Organization
It is not absolutely required that you use RBAC if you are managing a hierarchical set
of organizations, or vice versa.
Orgs have advantages outside of RBAC: The structural management hierarchy allows
you to better manage large amounts of equipment and large numbers of logical objects.
In addition, some of the pool inheritance features that we will discuss can let you
design preference logic that you could not achieve with a single flat org.
Similarly, the RBAC style of delegated administration has advantages even if you do
not have a hierarchical org structure. Using RBAC, you can still have users who have
control over the logical server structure and other users that have control over the
border (LAN and SAN) configuration, for example.
Remember:
• Orgs and RBAC could be used independently

• Orgs without RBAC
• Structural hierarchy for resources

• RBAC without Orgs – More for defining job roles globally (e.g. SAN activities
vs. LAN activities)
• Everything in root org (as we have been doing so far)

Organization Concepts
Root is the parent to all Orgs

Children can use Parent resources
Do not have to create Orgs
–Can use Root as your Org
–RBAC will be based on Global UCS role
Organization Concepts
We have actually been using orgs all along in all the discussions of configuration of
logical servers. There is always an org named “root” (/), and if you do not create any
other orgs, all logical server, pool, and policy administration is done within this single
org.

Example Organization Hierarchy
root (/)
SWDev QA
SWgrpA SWgrpB IntTest
Example Organization Hierarchy
Multiple orgs are always configured in a hierarchical structure.
Create a sub-org of a particular level by using the create org command, or equivalent
GUI right-click gesture, in the scope of the parent org which will contain the new org.
Understanding the hierarchy is important due to the pool and policy “inheritance”
features that will be discussed on the upcoming pages.

What’s in an Org?
Orgs Organize Resources:
• Parent Org has different resources
than Child
• Resources can have similar
names
• Management Hierarcy controls
what resources an Org can access
• Resource has
same name
but in different
Orgs
• Child Can use
Parents Pool
What’s in an Org?
The most important element of an org. is the service profile. Each service profile lives
in one and only one org. The profiles themselves (the entire service profile objects) are
not inherited in any way.
There is no direct way to move a logical service profile (or any other object, for that
matter) from one org to another.
The slide shows the profiles and policies that are also contained in an org. When you
view an org in the GUI, links to configure all the profiles and policies within that
particular org will automatically appear underneath the new org.
The final element contained within an org are the server pools, MAC pools, and WWN
pools. These appear in slightly different places in the GUI --- server pools are near the
bottom of the Servers panel on the navigation pane, and MAC pools and WWN pools
are on the LAN and SAN panes respectively. When you create a new org, appropriate
links for the pools within the new org will appear.

Blades and Organizational Concepts
Blades are exclusive to Org

Blades are not exclusive in pools
Manually put different blades in specific
pools
Create Qualifiers to automate
Blades and Organizational Concepts
Note that blade equipment itself is never part of an org. Any blade can appear in as
many pools in as many different orgs as you like. Any blade can be associated with a
logical server in any org.
While blade pools (server pools) are part of an org, they are in a way advisory only.
There is nothing that prevents you from associating any individual blade to any service
profile in any org (bypassing the pools), regardless of how many pools the blade may
be in.
Note how the rule is basically “first come first served”. The first profile that
successfully selects a blade for association (rather through pools, or directly without
pools) is the owner of that blade until it is made available again through disassociation.

Server Pools and Orgs
When associating Profile

with Server Pool
Second “MyPool”:
 Association if pool in local org or
Parent
– First try in local org
– If does not exist :
 Try pool with same name in next
highest org
First  Continue upward to root org
Server Pools and Orgs
The hierarchical nature of server pools (and other pools, discussed on a subsequent
slide) is one of the key features to understand about hierarchical organizations.
There are two big rules:
A logical service profile has access to names of server pools defined in its own local
org, as well as server pools defined in orgs further up the tree.
For example, imagine there is no server pool named MyPool in the same org as a
logical server. However, imagine there is such a named pool in the org that is the
grandparent. The logical server can still do its blade selection from that pool.
Imagine there is a server pool named MyPool in the same org as a logical server. If
that logical server specifies it wants to select a blade form MyPool, it will look first for
a blade in that pool in the same org. If no blade is available from that pool, it will look
for a pool with the same name (MyPool) in the parent org, and so on up to the root org
until it successfully can allocate a blade for the profile.

Server Pool Example
Can Associate a service profile in Org “QA” (sub-
org of root) with any of these pools:
Server Pool Example
This screenshot demonstrates a logical service profile in the org named CueAye. The
profile has access to a server pool in its own organization (the pool BurgessPool in this
example), as well as server pools in the parent pool (the pool YMCAPool in this
example). If there were a parent of that parent (there is not, since the parent is the root
pool), the service profile in question would have access to its pools as well.

Server Pool Example: Pool in Org and
Parent Org with same pool name
Server Pool Example: Pool in Org and Parent Org with same
pool name
This screenshot demonstrates what happens when there are pools with the same name
in the organization in question (where the service profile in question lives) and in the
parent (or grandparent, etc). When the logical service profile specifies selection from
that pool name, it will search first in the pool in its own organization. If that pool has
no blades available, then it will search in the pool with the same name in the parent
organization, and so on.
Note that this feature can be used as a strategy if you want to have a set of “preferred
blades” for a logical server and “less preferred blades” for the same logical server. You
can put the preferred blades in a pool in the same org as the logical service profile, and
put the less preferred blades in a pool with the same name in a higher level
organization.

MAC Pools, WWNN Pools, WWPN Pools
UUID Pools
Analogous to Server Pools

Same rules - local org or upward
Find an available (MAC,WWN, UUID) in first
pool with that name
MAC Pools, WWNN Pools, WWPN Pools UUID Pools
MAC Pools, WWN Pools, and presumably UUID pools work identically to server
pools.
• Pools in parent/grandparent/etc org but with no such identical pool name in the
“org in question” are still available to service profiles in the “org in question”
• If there is a pool with the same name, then the service profile will look for an
available entity starting from the same org as the service profile, and if none are
available, then go upward using a pool with the same name.

Other Server Parameters in Org (not
Pools)
Example:
Separate Policies
Between Orgs
Other Server Parameters in Org (not Pools)
A few of the other parameters for logical servers behave similarly to pools but slightly
different in that they are just policies and there is nothing to allocate:
•Names of these policies in the “org in question” and all parent/grandparent/etc orgs
are available to a service profile in the “org in question”
•If there are policies with same name in the “org in question” and a higher level org,
a service profile in the “org in question” can only use the lower one.
One way to look at this is: “a lower level policy with the same name as a higher level
policy overrides the higher level policy for all logical servers at that lower level
(or for those even lower than the lower level).
Remember:
•Similar but slightly different for following parameters of server profile:

•Boot Policy
•Local disk config policy
•Host firmware policy

•Can associate a profile with a policy in same or higher org
•If policy in same org as logical server has same name as policy in higher org,
higher one is not available for that logical server

RBAC
Locale: myloc
Role:myrole
root (/)
priv1 /SWDev
priv2 /Eng/HWEng
priv3
User is assigned certain

privs (one or more
roles) over certain
locales (one or more)
User: jim
RBAC
This slide shows a summary of role-based access control on the UCS system.
A user is created and is given one or more roles to be applied in one or more locales.
A locale is a collection of orgs – these orgs do not need to be related in the hierarchy in
any special way. The only purpose of a locale is for application of a user’s roles (ie,
they are used only with the RBAC scheme, and don’t have any other management
meaning).
A role is a set of privileges. There are several predefined roles, and you can create your
own roles with custom sets of privileges.
Note there is no way to have a user have some privileges in some orgs, and different
privileges in a different orgs. All of those privileges that are org-related in the
combination of all of a user’s roles will all be applied equally to all the orgs in all the
locales specified for that user.
There are privileges that are not org-related. If these privileges are present in the set of
roles assigned to a user, they will apply regardless of the locales assigned to the user.

Roles and Privileges
Roles Privileges
Collection of Privileges Specific, individual,
permissions.
Can create custom roles Cannot create privileges
Predefined Roles based on Special privileges: AAA,
job roles Admin
Roles and Privileges
Some special privileges:
• admin: can do everything on the whole system (including every org)

• aaa: Authentication, authorization, and accounting (administration of the RBAC
feature itself)
Note that neither of the above privileges is org-specific.
Some predefined roles:

• admin: predefined role that has the admin privilege
• aaa: predefined role that has the aaa privilege
We will look at some more predefined roles on subsequent pages. The only predefined
user is the user named “admin” who has the “admin role” and therefore the “admin
privilege” which allows complete administration of the entire system.

Predefined Roles and Privileges: Server
Equipment
Predefined Roles and Privileges: Server Equipment
The screenshot demonstrates the predefined server-equipment role, and its associated
privileges.
These privileges allow manipulation of the physical equipment (not service profiles).

Predefined Roles and Privileges: Server
Profile
Predefined Roles and Privileges: Server Profile
Another predefined role is the “server-profile” role.
You can see on the screenshot that the set of privileges for the this role are many of the
privileges that allow administration of logical service profiles themselves and all the
coordinate policies that go along with them.
The privileges for the “Server” role are org-specific. That is, a user given this role will
have these particular privileges only in the orgs that are part of the locales associated
with that user.
Remember again, it is the user (not the role) that is the glue between privileges and the
specific orgs in which those privileges apply.

Predefined Roles and Privileges:
Network
Predefined Roles and Privileges: Network
The predefined “Network” role defines a set of privileges which are not org-specific.
They have to do with configuring the global LAN parameters (which VLAN”s are
supported globally and per switch in the entire UCS, for example).

Creating Locales
Creating Locales
Recall locales are just sets of orgs, and the only reason for creating one is to assign it to
a user so that that users org-specific privileges will apply only to those orgs.
In the GUI you can drag sets of orgs into the locale creation pane using standard GUI
gestures (CTRL-select and drag).

Creating User
Creating User
The screenshot shows the user creation screen. You can choose any combination of
roles and any combination of locales.
The union of all privileges for all the roles will be applied to the union of all orgs in the
locales.
There is no way to say “let a user have privilege set A in Org X but let the same user
have privilege set B in Org Y”.

Lesson 7
UCS System Maintenance
Overview
The primary learning objectives for this module are for you to be able to
understand firmware upgrades, to manage backup and restores and to manage
high availability. We also discuss running scripts of CLI commands, and
configuring some of the external interfaces such as the call-home facility.
Objectives
Upon completion participants should be able to:
• Understand firmware upgrades
• Backup and restore configuration
• Manage high availability
• Perform other maintenance tasks.
UCS System
Maintenance
Objectives
• Perform other maintenance tasks

The primary learning objectives for this module are for you to be able to understand
firmware upgrades, to manage backup and restores and to manage high availability. We
also discuss running scripts of CLI commands, and configuring some of the external
interfaces such as the call-home facility.
• Perform other maintenance tasks

Upgrade Overview
Manage from
equipment POV
Manage Using
Service Profile *Note overlap: All firmware must first exist in UCSM catalogue
© 2008 Cisco Systems Inc. All rights reserved. UCS System Product Overview—data center and Server Trends 3
Upgrade Overview
“Equipment point of view” firmware upgrades are often associated with a clean sweep
firmware upgrade. This may be an activity you are likely to do as soon as you boot a
brand new UCS system for the first time. Though much of the firmware that comes
from the factory is likely to be up-to-date, you will still need to check all the firmware
and apply updates as necessary.
Managing firmware via service profile is a completely different way of looking at

firmware. Using service profile, you can specify “by policy” the revisions of firmware
that should be on a blade when it is associated with a particular service profile. You
need not worry what firmware is on the blade before its association with your service
profile; the association itself will check the firmware revisions, and, if they don’t match
your service profile specifications, will update the appropriate firmware.
No matter which point of view you are working with as you manage UCS firmware, all
firmware must first be loaded into the USCM catalogue, or repository, before it can
actually be applied to a component on the system.
© Cisco Systems, Inc. UCS System Maintenance 7-2

UCS Repository (Catalogue) of Firmware
 Firmware download to UCS

 “Full bundle” or individual firmware
 Can have multiple bundles or versions
 Available then for all upgrades
UCS Repository (Catalogue) of Firmware
In order for firmware to be available for use in a UCS component it must first be
downloaded into the UCSM catalogue, or firmware repository. Firmware’s presence in
this repository does not mean that the firmware may yet be actually in use on a UCS
component. Putting firmware in the catalogue is a separate operation from actually
applying it to a component, whether that firmware installation will be using the
equipment point-of-view or Service profile methods.
Downloading firmware to the manager’s catalogue is driven by an action on the

manager itself. Today, there is no built-in way of downloading firmware periodically
(no “automatic updater”).
Software retrieved from Cisco is placed on a local server accessible to the UCS
manager. This software will either be a “single file bundle”, that incorporates a revision
of firmware for all the components, or an individual firmware component.
When UCS manager downloads a “bundle” it will automatically extract the individual
components and catalogue them accordingly.

A firmware download to the repository will never harm a running system (it affects
none of the firmware actually running)

Download Bundle to Repository
Download Bundle to Repository
The firmware download action (to the UCS repository) on the GUI is accessed by
highlighting the word “Equipment” on the equipment pane and accessing the firmware
management pane.
Not the firmware download cannot currently use http, or https; this implies you must
have a server accessible to the UCS manager that uses ftp, tftp, scp, or sftp.
You do not need to tell the downloader whether you are downloading an individual
component or an entire bundle (as in the screenshot). The downloader will figure it out
based on the contents of the download (each download is in a proprietary archive
format that includes a table of contents).

Repository Contents (“package view”)
Repository Contents (“package view”)
There are two views of the existing contents of the software repository.
The “package view” categorizes the component software by the download file that it
came in. You do not manage the individual components from this view, but if you
delete components from the “Images” view (next slide) they will be marked as such in
this package view. If you delete all the components that make up a package, the entire
package will be deleted from this view.

Repository Contents (“Image view”)
Repository Contents (“Image view”)
This is the individual component view of the repository.
You can delete components from this view. You can even delete firmware which is
running on a “flashable component” (anything other than the fabric interconnect or
manager), since deleting it from the catalogue does not affect the running component.
You cannot delete running fabric interconnect or manager firmware (the manager
maintains a “startup link” for these directly into the catalogue itself; the catalogue copy
is the running copy). You can of course delete old unused (or new, never used) versions
of fabric interconnect or manager software.

“Equipment POV” Firmware Managemnt
 Done “bottom up”

 New install or “full sweep upgrade”
 Adapter, BMC, IOM have:
 Update (backup flash area)
 Activate (swap backup / active flash next reboot)
 Fabric Interconnect (kernel, system) / Manager

 Activate Only
When you upgrade firmware from the “equipment point of view” you will generally
make sure upgrades are done “bottom up” (get new firmware running on adapters
before BMC before IOM/fabric interconnect /manager).
You are way more likely to get components that cannot communicate with new
firmware at the higher (IOM, fabric interconnect) layers if you have not yet updated
the firmware on the “leaves”.
The adapter, BMC and IOM components are “special”. You always flash new firmware
into an unused flash area. This operation (“update”) is non-interruptive. All adapters,
BMC, and IOM’s in a large system could be “updated” simultaneously, since this flash
operation only affects the backup flash. After the update the new firmware is
“activated” by just setting a pointer so that when the component reboots the primary
and backup flash areas just swap roles.
The fabric interconnect components (kernel and system software) and the manager
application itself have only an “Activate” operation and no “update”. This software
basically just lives in a file system, and when you activate it, you set a pointer so that

on next reboot the component boots off the new firmware version by pointing straight
into the catalogue.

Upgrade (IOM, BMC, Adapter)
If you are doing a “clean sweep” firmware upgrade you can do the “update” portion for
IOM/BMC/Adapter globally (all at the same time for all components); there is no
danger in this since it is just flashing a backup flash area. From the “global form”
available from the equipment view, or from various views of individual components,
you could choose to also perform the update operation on a single or on chosen
components.

Activate
Activate
The “Activate” operation will set a pointer so that the component boots the new flash
version (“backup flash that was updated”, for IOM/BMC/adapter) on next boot.
The default is that the UCSM operation will not only set the pointer but also reboot the
component. It is possible to “Set Startup Version Only” and then the firmware will not
actually be brought into use into you manually reboot a component.

Typical “Full Sweep” Upgrade
 Update all Adapter, BMC, IOM

 Activate all Adapter (server reboot)
 Activate all BMC
 Activate all IOM, click button to “Set Startup”
 Activate fabric-interconnect and MGR
Typical “Full Sweep” Upgrade
If there is a time when you will be upgrading all components to a specific firmware
revision, you will typically:
• do all IOM/adapter/BMC “updates” (flash of backup area) simultaneously

• Activate (and reboot) adapters first (yes, this is a blade server reboot)
• Activate (and reboot) BMC (rebooting just BMC does not interrupt processing on the
blade server)
• Activate IOM, but tell it to just set the startup version, and not reboot
• Activate fabric interconnect components and manager. If you have redundant fabric
interconnects, they will reboot in a rolling fashion, and at the end, load your new
manager.

Firmware Management Using Service
Profile
Firmware Management Using Service Profile
Doing firmware upgrade from the service profile point of view is a completely different
way of looking at things. This is a more “firmware by policy” point of view.
A firmware package is a list of required firmware per component (a package can

specify one (or zero) version of firmware per component: BIOS, Adapter firmware,
Emulex FC firmware, LSI RAID firmware, Qlogic/Emulex option ROM firmware).
If there is no version at all listed for a component, or no firmware package at all, it just
means to keep the firmware as it is when service profile is associated.
There are separate “management packages” for BMC firmware.
A service profile or service profile template can point to one firmware package (having
versions for one or more components) and one management (BMC) package.
When the service profile is associated, it compares firmware versions on the blade
being associated with those listed in the package, and “makes it so” (updates firmware
as needed). Note for a component with the “update/activate” philosophy, the service
profile association will do both the update and activate.

Note that at this time there is no “firmware scrub” (no putting firmware back to some
baseline when a service profile is disassociated. today, firmware just stays as it is on
disassociation).

Firmware Packages (Host and Mgmt)
In SP/template
Firmware Packages (Host and Mgmt)
This is an example of a host firmware package (everything except BMC). The example
has shown the package referring to versions of firmware for BIOS, RAID Controller,
and Menlo-Q adapter.
A service profile or SP template will then refer to the package by name. Just like all
other policies, you can refer to a package name in the same org as an SP or in a higher
org.

Backup and
Restore

Backing up UCSM Configuration in the
GUI
Backing up UCSM Configuration in the GUI
This screenshot shows where GUI access to the backup and import operations lie
(highlighting the word “All” on the Admin Screen – it can be tricky. Details of the
backup operation are on the next slide.
Backup operations creates an “snapshot” of your system configuration and sends it to an

external server. The “configuration” snapshots (everything except “full state”) can be
restored “on the fly”. You can bring your configuration back to a known state (using “All
configuration”) without having to reboot.

Backup Operation
Backup Operation
Backup Task
Backing up a UCS System starts with the creation of a “backup task”. A “backup task”
is a managed object that defines all the criteria for backing up the configuration of the
UCS. A “backup task” contains the following information:
•Type
•Transmitting protocol
•Host name where the back up is to be stored
•Remote path and file name
•User name to use on remote host
•Password
Administrative state
Once you create the managed object that is the backup operation, to actually create the
backup the administrative state of the backup must be changed to “enabled”. In the
background, the state and/or configuration is collected (depending on the type of
backup you perform), tarred, gzipped (for full state, the others are XML) and
transferred to the backup server using the protocol you choose. You can follow the

workflow of the backup by examining the FSM associated with the backup. Once the
backup is complete, its administrative state will automatically change to “disabled”.
Backup types
The types of backups you can create include the following:
•Full-state: this is a raw backup of the data. It must be imported on the exact same
hardware (it restores all the hardware serial numbers and such), it is not for a hardware
disaster but a configuration database disaster (this is a .tar.gz of the raw database; all the
others are XML files)
•All configuration: this is the combination of “system configuration” and “logical

configuration”
•Logical configuration: this is all configuration that is *not* associated with AAA (e.g.
configured orgs, configured threshold policies, configured VLANs and VSANs in your
LAN and SAN clouds respectively …etc, configured server, uplink and SAN ports).
•System Configuration: this is all configuration that specifically pertains to the AAA
role (authentication, authorization, and accounting). (e.g. radius, ldap, tacacs, users,
…etc).
Protocols
Your choice of protocols include FTP, TFTP, SCP, and SFTP. Note that you must
choose a protocol that is running on the backup server and that you must provide user
credentials that are valid on the backup server. Also note that the path name to remote
file is *relative* to the directory where the protocol “puts” you upon “login” (despite
the reference to a “fully qualified path name”). For example, if you select FTP and
login anonymously, then you are not prompted for a password. The directory you “land
in” depends on how the FTP server on that backup host is configured. If you select
SCP, then you must provide valid user credentials and the directory you land in will be
the home directory of that user.
You can only create one backup configuration per backup server. Backups are identified
in UCSM using the hostname or IP of the backup server in the configuration. Note that
hostnames are resolved through the external DNS server defined as you set up the UCS
manager (this can be changed on the Admin tab)

Restore (import) Operation
 Full State: imported at initial setup only

 Other options:
Restore (import) Operation
Restore of a “full state” backup is done from the serial console only, as part of the
“initial setup” dialogue (you can get back to this dialogue if you do (be careful) “erase
configuration” from the local management shell.). The full raw database is just put
back into place. Note if you really have different hardware (different chassis or blade
serial numbers, for example), a full state restore is not what you want. Full state restore
is for complete configuration data disaster only.
For the other options, you can import the XML backup “on the fly”. If you use the
“replace” option, all existing objects not in the saved file will be deleted (and blade
servers will undergo a disassociation from any previous SP), and new data is inserted
(and blade servers do undergo an association process).
Note that often after import it looks like all your new associations are “failing”. In
reality they are often just waiting for the disassociations from previous SP to complete,
and then the new associations will succeed once blades become available.

Restoring Full State Backup at Switch
Startup
Restoring Full State Backup at Switch Startup
This is the only way to restore the full state backup.
Note that you must provide network configuration for the switch in order for it to go
over the network to the backup server to retrieve the backup file. If you have a DHCP
server, the process will try to use that as well. This IP is just for the restore itself, then
the permanent static management IP is taken from the restore file itself.

Restoring UCSM Configuration at Switch
Startup (continued)
Restoring UCSM Configuration at Switch Startup (continued)
Note that when doing the restoration at switch startup (as opposed to at runtime), you
must adhere to the following rules:
•The backup file you created is a full-state backup only.

•The remote file path is an absolute path (from the root directory) to the backup file on
the backup server (although this still could get interpreted in a “relative” fashion”, by a
tftp or ftp server, for example)

Manage High
Availability

Overview of HA in UCS System
Management Node Management Node

UCSM L1 L1 UCSM
L2 L2
Server Ports Server Ports
IOM Mid-Plane IOM
SEEPROM
Chassis
Overview of HA in UCS System
Configuring two nodes in your UCS System can provide redundancy for both the
UCSM functionality as well as the switching functionality that the management nodes
provide. You are not required to run in HA mode, but if you do, you must follow certain
rules. First, you must connect the L1 port of the “left” switch to that of the “right”
switch and L2 of left to L2 of right. Second, you must connect the “left” FEX to the
“left” switch (using between 1 and 4 server links) and connect the “right” FEX to the
“right” switch.
*Note that both fabric extenders share a serial EEPROM. This chip stores cluster state
information and is used to resolve split brains (both partitions in time and in space).
Each CMC maintains its own part of the shared chassis storage in the SEEPROM.
Chassis storage contains a combination of “static” and “dynamic” information. For
example, the static portion contains the nodeid for each node configured in the cluster.
The dynamic portion contains a monotonically increasing integer which represents the
version of the configuration as seen by that node. There is no need to replicate the
contents of the SEEPROM. Each node maintains (writes to) its own portion while both
nodes may read from both sections.

Split Brains
 Partition in space
– Total private network failure
– Resolved by SEEPROM
 Partition in time
– Node tries to start alone with old configuration
– Resolved by SEEPROM
Split Brains
Partition in space
A partition in space occurs when nodes fail to communicate with each other over the
private network (L1 and L2 links both failed). To resolve this split brain (and assuming
both switches are active at the time of the private network failure), each CMC will act
on the behalf of their fabric’s UCSM instance to reach the serial eeprom first and write
their node id in the primary field (known as a “quorum race”). The “winner” will
remain in the cluster and the loser will abort. When the links are restored, the “losing”
node can rejoin the cluster and act as subordinate.
Partition in time
A partition in time occurs when one of the nodes is down for a period of time during
which changes to the configuration are made on the active primary node. These
changes would have not been replicated to the down node (obviously). If the primary
node were to shut down after having made configuration changes to the database but
prior to being able to replicate them to the other (downed) node AND that downed node
were to try to join the cluster alone, then that condition is referred to as a partition in

time. To resolve this split brain, a version number that represents the configuration is
written to the eeprom. On solo startup, a node compares its version to that of the other
node (recall both nodes can read both parts of the EEPROM). If it is the same or
higher, it can start the cluster. Otherwise, it will not, but it can be forced with the
“cluster force” command.

Viewing and Changing Management HA
 connect local-mgmt
 show cluster extended-state
myswitch-A# show cluster extended-state
A: UP, PRIMARY
B: UP, SUBORDINATE
A: memb state UP, lead state PRIMARY, dme state: UP
B: memb state UP, lead state SUBORDINATE, dme state: UP
link state UP, heartbeat state PRIMARY_OKHA READY
 cluster lead
 cluster force
Viewing and Changing Management HA
This is an example of viewing, and changing primary node assignment in the UCS
system.
myswitch-A# connect local-mgmt

Cisco Nexus Operating System (NX-OS) Software
Copyright (c) 2002-2008, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software may be covered under the GNU Public
License or the GNU Lesser General Public License. A copy of
each such license is available at
http://www.gnu.org/licenses/gpl.html and
http://www.gnu.org/licenses/lgpl.html
myswitch-A# show cluster extended-state

A: UP, PRIMARY
B: UP, SUBORDINATE
A: memb state UP, lead state PRIMARY, dme state: UP

B: memb state UP, lead state SUBORDINATE, dme state: UP
link state UP, heartbeat state PRIMARY_OK
HA READY
Chassis, serial: CHASSIS 81, ip: 127.5.1.254, state: active
The output of the ‘show cluster extended-state’ command displays the following
information:
•Running mode of each instance
•Running mode of each DME
•Status of private network
•Status of UCSM HA
•Chassis ID
• IP address of primary IOM
Force or Lead?
To suggest that this node be primary in the cluster, run the ‘cluster lead’ command.
This command will attempt to promote this (subordinate) node to be the primary node.
If the UCSM controller determines that this node is not “fit” to lead the cluster, then
this command will abort its attempt.
To force that this node be primary in the cluster, run the ‘cluster force’ command. This
will override any information in the SEEPROM and UCSM logic that would otherwise
contradict this node being elected primary.

Performing Other
Maintenance Tasks

Setting the Time
 Switch: clock set month day year hour min sec
 CMC gets date from switch
 BMC gets date from switch
Setting the Time
When installing a switch for the first time (or if you don’t run NTP on the switch), you
may need to set the system clock. This clock is used as a reference clock for both the
CMC and BMC so that all log file entries in a UCS System may be synchronized. You
could also configure the switch as an NTP client if you wish to synchronize all the UCS
System system clocks with an external time server. If you don’t configure NTP, then
your system clocks will likely drift.
You can also set the time and time zone as follows:
ourswitch-A# scope system
ourswitch-A# scope services
ourswitch-A /services # set clock jan 5 2009 13 52 0
ourswitch-A /services # set timezone America/Los_Angeles
ourswitch-A /services* # commit buffer
You can also set the time zone from within the GUI.

Copying Scripts to UCS Manager
Copying Scripts to UCS Manager
You can write UCSM CLI scripts on another host, copy them to the /workspace
directory of the management software using SCP, FTP, or TFTP, and then run them
from the ‘connect local-mgmt’ interface of UCSM. Note that the ‘cp’ command will
detect the existence of files by the same name and will prompt you if you wish to
“clobber” it or not.
You cannot copy scripts from within the GUI.
*Note: you must be in the local-mgmt shell to perform these tasks.

Example Script Usage
 Save “show config” output for a Service Profile

# scope org /
# show config >scp://user1@mysrvr/home/user1/SP_config.out
1. Edit config file on server (fix, add scoping)

2. Copy script back to fabric interconnect
3. Delete service profile from original org
4. Use script to recreate SP in a different org
Example Script Usage
One idea for accomplishing something useful using the script interface is to effectively
“move” as service profile.
As shown, you could execute “show config” in the scope of the service profile to get
nearly the set of command used to recreate the profile. You may have edit the script on
an external server (often commands like “associate” are interspersed in “create”
commands; whereas you can’t really associate a service profile until it is created.
Note also, when you run the script it will always run from the top scope (you run it
from the local-mgmt shell). You can put explicit scope commands in the script itself to
make it “move” to the correct places)

Erasing Configuration
 erase configuration
– Removes the database and UCSM configuration
– Automatically reboots switch
– Prompts user to configure UCSM at startup
Erasing the configuration

Erasing the configuration (from the ‘local-mgmt’ interface) erases the database and
removes the UCSM configuration. This command will automatically reboot the fabric
and prompt the user to configure UCSM from scratch. Note this will not affect the other
fabric interconnect in an HA configuration. If you want, you could run “erase
configuration” on both.

Using External
Management
Protocols
In this section, you will identify each of the supported management protocols that can
be used to gather information from outside a UCS system.

SNMP
SNMP
UCS supports SNMP v3 read-only monitoring of the UCS fabric interconnect as

defined in the MIBs for the Nexus 5000 product running 4.0.(0)(N1)(1a). There are
approximately 60 different MIBs defined for the NX5K (e.g. AAA, CallHome, STP,
VLAN, and FC device aliases).
You can have UCS send SNMP traps to define trap hosts. Once again, as of the current
release, trap information is related only to the MIB’s defined for the switch component
of the fabric interconnect.

Syslog
Syslog
Syslog is a traditional UNIX logging facility that can be configured to send messages
from different programs (or “facilities”) of varying severity (e.g. info, warn, crit,
emerg) to the console, to a file, or to an external host (aka “loghost”). Syslog contains
several built-in facilities such as kern (or kernel), print (i.e. print server), and mail (i.e.
SMTP server). Users can also configure their own custom facilities for applications
that don’t fall neatly into one of the predefined facilities. These facilities are named
local0, local1, … etc.
In UCS, you can configure syslog to report to a local destination or a remote

destination.
Local destination configuration includes
•Console
•Monitor
•file
Remote destination configuration (for up to 3 log hosts) includes
•Level

•Hostname
•facility

Serial Over LAN
Serial Over LAN
Serial Over LAN is a mechanism that enables the input and output of the serial port of a
managed system to be redirected via an IPMI session over IP (i.e. using TELNET or
SSH).
In UCS, the serial ports on the blades are not connected to a traditional serial port
interface. To allow users to access applications on the servers via the serial port, I/O of
the serial port is redirected to the BMC and made available to external users on the
management network. For example a user wishing to access a blade server that is
running Linux via the serial port can ssh to the management network IP address
associated with that blade and log in. On the blade server the login will be seen as
coming through the serial port.
In UCS, this is configured through a Serial Over LAN policy, which defines
•Enable or disable the service
•Baud rate for terminal sessions

IPMI
IPMI
IPMI operates independently of the operating system and allows administrators to

manage a system remotely even in the absence of an operating system or even if the
monitored system is powered off, but connected to a power source. IPMI can also
function after the operating system has started. IPMI prescribes only the structure and
format of the interfaces as a standard, while detailed implementations may vary.
IPMI can send out alerts via a direct serial connection, a LAN or a SOL connection to a
remote client. System administrators can then use IPMI messaging to query platform
status, to review hardware logs, or to issue other requests from a remote console
through the same connections.
The IPMI consists of the BMC and other satellite controllers. The satellite controllers
within the same chassis connect to the BMC via the system interface called IPMB
(Intelligent Platform Management Bus/Bridge) — an enhanced implementation of I2C
(Inter-Integrated Circuit). The BMC connects to satellite controllers or another BMC in
another chassis via IPMC (Intelligent Platform Management Chassis) bus/bridge. It
may be managed with the Remote Management Control Protocol (RMCP), a
specialized wire protocol defined by this specification.
A FRU holds the inventory (such as vendor id, manufacturer etc.) of potentially
replaceable devices. A Sensor Data Records (SDR) repository provides the properties of

the individual sensors present on the board. For example, the board may contain sensors
for temperature, fan speed, and voltage.

SMASH-CLP
Hardware components of chassis 1

include 8 fan modules, 2 FEXs, 4 PSUs,
and 4 servers
SMASH-CLP
The Systems Management Architecture for Server Hardware (SMASH) is a suite of

specifications that deliver industry standard protocols to manage elements of a data
center. The SMASH Command Line Protocol specification provides an interface to
heterogeneous servers independent of server hardware, operating system, or network
protocol method. It is a standard method for local and remote management of server
hardware using out-of-band communication. SMASH is developed by the DMTF
Server Management Working Group (SMWG).
With SMASH CLP-enabled products, users can execute common operations (such as
system power on and off, system log display, boot order configuration and text-based
remote console) using the same commands across different vendor platforms.
In UCS, you connect to the “clp” shell from the CLI to issue SMASH commands. Use
‘cd’ to traverse the information hierarchy and use ‘show’ to display values at different
nodes in the information hierarchy.

Call-Home
Call-Home
UCS Manager 1.0 uses the CallHome functionality as defined in the Nexus 5000.
CallHome creates email messages in either XML or standard text format and sends the
messages to predefined recipients when certain events occur. Configurable profiles
define
•criticality levels (e.g. debug, warning, fatal, …)

•email format (xml or txt)
•email recipients
In UCS, you can enable CallHome policies to generate notifications on thermal,
voltage, power, identity, fru, and equipment events. You can send also full inventory
data at the push of button or schedule it to be sent on a periodic basis.
The “general” page of the call home is the setup for all your “From” information. You
can provide fake information here if you are just testing. The exception is just the
bottom part where you list the SMTP server to which you want to send information.
Note this does not accept an SMTP server that authenticates.

Call Home Profiles
Call Home Profiles
Call home profiles allow you to establish a list of email addresses to which to send
messages on events that match certain categories, or “alert groups”.
CiscoTac is a special “alert group” that matches all critical events, as well as the
sending of inventory.
The other “alert group” that matches inventory setting is the “diagnostic” group (not the
inventory group; this matches inventory events --- insertions and removals of
hardware).

Call Home Policy
Call Home Policy
The policies allow you to enable and disable call-home for certain categories of
problems (fru-problem, equipment-inoperable, and the like)

Call Home – Send Inventory
Call Home – Send Inventory
You can periodically send inventory, or click to send on a “one time basis”. A full
“smart call home” XML email is sent to email addresses in call-home profiles that
match the “CiscoTac” or “diagnostic” alert groups at “normal” severity or below.
The intention is that this email get digested by an artificial intelligence at Cisco TAC,
who will file the customer’s entire inventory in a database. When a customer calls,
Ciso TAC can pull up information and see exactly what the customer’s inventory is.
The call-home functionality itself, however, is just an email.

Master UCSI BootCamp vAPTOS 3110 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Master UCSI BootCamp vAPTOS 3110 PDF

Uploaded by

Copyright:

Available Formats

Lesson 1

Upon completing this lesson you will be able to:

• Introduce Data Server Trends

1-1 UCSI Boot Camp © Cisco Systems, Inc.

The 90’s Early 2000’s Now

Scalability: The server evolution

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-2

Redundant power and networking

Scalability: Changes in server design

• Maximum use of physical space

1-3 UCSI Boot Camp © Cisco Systems, Inc.

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-4

CPU and Memory Density

• Core is still a physical processing element embedded in the module (socket). An OS

1-5 UCSI Boot Camp © Cisco Systems, Inc.

On the Nehalem architecture on the Cisco B-series blades, a “hyperthreading” setting is

An OS will always report the number or processors at the “lowest granularity”, ie

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-6

1-7 UCSI Boot Camp © Cisco Systems, Inc.

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-8

1-9 UCSI Boot Camp © Cisco Systems, Inc.

Server Orchestrators / Manager of Manager

Chassis Mgr Chassis Mgr Chassis Mgr

Network Mgr Network Mgr Network Mgr

Server Mgr Server Mgr Server Mgr

Vendor A Vendor B Vendor C

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-10

• Describe logical and physical perspectives of UCS

1-11 UCSI Boot Camp © Cisco Systems, Inc.

Logical View of UCS

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-12

2 x power supplies redundant

Front View of UCS

At time of writing the blade options are:

• Half-width blade: B200

1-13 UCSI Boot Camp © Cisco Systems, Inc.

UCS feature support:

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-14

Rear View of UCS

1-15 UCSI Boot Camp © Cisco Systems, Inc.

Type Form Factor Number of Number of

Half-width ½ chassis width 12 1

Full-width full chassis width 48 2

Type On Board CPU Sockets Internal Drives

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-16

• Number of CPUs to populate (1-2)

• Identity elements such as MACs, WWNs, and UUIDs

1-17 UCSI Boot Camp © Cisco Systems, Inc.

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-18

Memory Configurations: B200

• DDR3 1333MHz across 3 channels

1-19 UCSI Boot Camp © Cisco Systems, Inc.

• DDR3 1066 MHz across 3 channels

• same size across whole server

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-20

Full Width Blade

1-21 UCSI Boot Camp © Cisco Systems, Inc.

• Memory ASIC controls 4 physical DIMM

Memory Configurations: Full Width

© Cisco Systems, Inc. Introduction to Cisco’s UCS Platform 1-22

VM I/O Virtualization Existing Driver Cost Effective

Eth Eth Eth

0 1 2 3 127 10GbE FC For hosts that