Deploying OpenStack with Cisco

Compute, Network and Storage

Duane DeCapite, OpenStack Product Management
Ashok Rajagopalan, UCS Product Management
November 2013
2013 Cisco and/or its affiliates. All rights reserved.

COMMUNITY PARTICIPATION
Code Contributions and blueprints across Core

services
Networking model, Compute Service and
Dashboard, HA, Scheduling,
OpenStack Foundation Board member

OpenStack
@ Cisco

CISCO OPENSTACK ENGINEERING

Automation (Puppet) and architectures (HA) for

production deployment and operational support

Quantum/Neutron/Nova Plug-ins for Cisco

product lines
UCS, Nexus, CSR1000V
Scalable networking services

FWaaS, LBaaS, VPNaaS

CUSTOMERS
Private, Public
Extend cloud model for rapid provisioning of network

services, bare-metal, intelligent workload placement

2013 Cisco and/or its affiliates. All rights reserved.

Drive innovation through real-world use cases

Innovation in Cloud Computing through OpenStacks Network

Service and Ciscos Open Network Environment (SDN)
Applications each see
their own logical DC

VM
VM

VM
VM

VM
VM

VM
VM

VM
VM

VM
VM

OpenStack
Compute

API-driven
Open Cloud Platform

OpenStack
Networking

(Nova)
(Nova)

(Neutron)
(Neutron)

l
Physica
Platform
APIs

Programmable
Infrastructure

One Platform Kit (onePK)

on ISR G2, ASR 1K

2013 Cisco and/or its affiliates. All rights reserved.

OPEN NETWORK
ENVIRONMENT
a

Controllers
and Agents

Cisco ONE Controller SW

Openflow Agents

VM
VM

OpenStack
Storage

(Swift,
(Swift, Cinder,
Cinder, CEPH)
CEPH)

Virtual
Virtual
Overlays

VXLAN Gateway, Openstack,

Service Chaining
CSR 1KV
3

Lighthouse Customers in Production with Cisco OpenStack Solutions

Comprehensive Cisco Integrated Solution for

OpenStack
Operational efficiency with UCS and

networking integrations with OpenStack

Pre-defined reference configurations and

performance optimized solutions

SaaS applications and GRID scaleout

applications
2013 Cisco and/or its affiliates. All rights reserved.

Compute and Storage

2013 Cisco and/or its affiliates. All rights reserved.

Cisco UCS Leadership

and Momentum

As of Q3FY13 UCS revenue reached a $2B annualized run rate.

In Q3FY13, Data Center revenue was 515M growing 77% Y/Y

As of May 2013, there are over 23,000 unique UCS customers which
represents 89% Y/Y growth

More than half of all Fortune 500 customers have invested in UCS

Over 500 customers have booked over 1 Million in UCS solutions

and over 1,200 have booked over $500,000

Over 3,400 Channel Partners are actively selling UCS worldwide and
over 1700 UCS specialized partners in the channel world wide

As of CY12 Q4 Cisco is one of the Top 5 Server Vendors, #2 in Blade

servers based on Worldwide Revenue Share1

73 World Record Performance Benchmarks to date

Source: 1 IDC Worldwide Quarterly Server Tracker, Q1 2013, May 2013, Revenue Share

2013 Cisco and/or its affiliates. All rights reserved.

UCS Compute Portfolio

Performance Optimized for Bare Metal, Virtualized, and Cloud Applications
Cisco
Cisco UCS:
UCS: Many
Many Server
Server Form
Form Factors,
Factors, One
One System
System

Blade

Rack

Scale Out

Industry-Leading
Industry-Leading Compute
Compute Without
Without Compromise
Compromise
Enterprise
Intensive / Mission Critical
Performance
UCS C240 M3

UCS C24 M3
Entry, Expandable Rack
Server for Storage
Intensive Workloads

Ideal Platform for Big Data,

ERP, and Database
Applications

UCS C22 M3
Entry Rack Server for
Distributed and Web
Infrastructure Applications

UCS C220 M3
Versatile, General Purpose
Enterprise Infrastructure, and
Application Server

UCS B22 M3
Entry Blade Server for IT
Infrastructure and Web
Applications

UCS B200 M3
Optimal Choice for VDI,
Private Cloud, or Dense
Virtualization/
Consolidation Workloads

2013 Cisco and/or its affiliates. All rights reserved.

UCS C420 M3
Enterprise Class, 4-Socket
Server for Large, MemoryIntensive Bare Metal, and
Virtualized Applications

UCS C460 M2
UCS C260 M2

Mission-Critical, 4-Socket
Server for Large, CPUIntensive Applications

Mission-Critical, 2-Socket
Extended Memory Server for
Large, Memory-Intensive
Applications

UCS B440 M2

UCS B420 M3

Mission-Critical, 4-Socket
Enterprise Class, 4-Socket
Blade for Large, CPUBlade for Large, MemoryIntensive Bare Metal
Intensive Bare Metal
UCS B230 M2
and Virtualized Applications
and Virtualized ApplicationsDensity-optimized CPU and
Memory-Intensive 2-Socket
Blade for Bare Metal and
Virtualized Applications

Unified Management
Blade and Rack Servers Managed a Cohesive Resource Pool
UCS Manager

UNIFIED MANAGEMENT
A SINGLE UNIFIED SYSTEM FOR
BLADE AND RACK SERVERS

ServiceService
Profile: HR_App1
UCS
Profile

C-Series Rack
Optimized Servers

A Major Market
Transformation in Unified
Server Management
2013 Cisco and/or its affiliates. All rights reserved.

VNIC1
Unified Device Management
MAC: 08:00:69:02:01:2E
HR_WEB_VLAN (ID=50)
Network Policy
VNIC2
MAC: 08:00:69:02:01:2F
HR_DB_VLAN (ID=210)
Policy
HBAStorage
1 and 2
WWN: 5080020000075740
WWN: 5080020000075741
Server
VSAN
ID: 12Policy
Boot Order: SAN
BIOS Settings:
Turbo On
HyperThreading On

Benefits of UCS Manager and

Service Profiles for Both Blade
and Rack-Optimized Servers

B-Series
Blade Servers

Add Capacity
Without Complexity
8

Scaling the Cisco Cloud Architecture

L2/L3 Switching

Single Rack 16
servers

Single Domain
Up to 10 racks, 160 servers

UCS Manager

Multiple Domains, up to 10K nodes

UCS Central
2013 Cisco and/or its affiliates. All rights reserved.

Cisco UCS OpenStack Solution Accelerator Paks

Compute-intensive

(2) UCS 96-Port 6296 Fabric

Interconnect
(2) Nexus 2232 PP
(6) UCS C220 M3 Servers w/
dual Intel Xeon E5-2665 2.4
GHz Processors, 192GB of
Memory, Mega RAID, 4 x
900GB 10K SAS HDDs

2013 Cisco and/or its affiliates. All rights reserved.

Mixed-use

(2) UCS 96-Port 6296 Fabric

Interconnect
(2) Nexus 2232 PP
(6) UCS C220 M3 Servers w/
dual Xeon E5-2665 2.7 GHz
Proc, 192GB of Memory, Mega
RAID, 4 x 600GB 10K SAS
HDDs
(2) UCS C240 M3 Servers w/
dual Intel Xeon E5-2665 2.4
GHz Proc, 192GB of Memory, 1
x Mega RAID, 24 x 900GB 10K
SAS HDDs

Storage-intensive

(2) UCS 96-Port 6296 Fabric

Interconnect
(2) Nexus 2232 PP
(8) UCS C240 M3 Servers w/
dual Intel Xeon E5-2665 2.4
GHz Proc, 192GB of Memory, 1
x Mega RAID, 24 x 900GB 10K
SAS HDDs

10

OpenStack Compute-Intensive Solutions Pak

High-Density
2 Control Nodes
C220-M3s

2 Compute Nodes
C220-M3s

nova-api
keystone-api
glance-api
cinder-api
horizon-UI

nova-api
quantum-api
cinder-api

Network Node
quantum-api
CEPH
MON/MDS/RAD
OS

KVM Hypervisor

2013 Cisco and/or its affiliates. All rights reserved.

2 Storage Nodes
C220-M3s
cinder-api
CEPH Deploy
CEPH Object
Storage Devices

11

OpenStack Mixed-Use Solutions Pak

Mixed-Workload
2 Control Nodes
C220-M3s

4 Compute Nodes
C220-M3s

nova-api
keystone-api
glance-api
cinder-api
horizon-UI

nova-api
quantum-api
cinder-api

Network Node
quantum-api
CEPH
MON/MDS/RAD
OS

KVM Hypervisor

2013 Cisco and/or its affiliates. All rights reserved.

2 Storage Nodes
C240-M3s
cinder-api
CEPH Deploy
CEPH Object
Storage Devices

12

OpenStack Storage-Intensive Solutions Pak

Storage-intensive
2 Control/Storage Nodes
C240-M3s
nova-api
keystone-api
glance-api
cinder-api
horizon-UI
Network Node
quantum-api
CEPH
MON/MDS/RAD
OS

2013 Cisco and/or its affiliates. All rights reserved.

6 Compute/Storage
Nodes
C240-M3s
nova-api
quantum-api
cinder-api

KVM Hypervisor
CEPH Deploy
CEPH Object
Storage Devices

13

Nova: how it works today

Client
API
calls

Compute
(Nova)

Infrastructure

nova-api
nova-api

Hypervisors
Hypervisors
Bare-metal
Bare-metal
Nodes
Nodes

nova-compute
nova-compute
nova-volume
nova-volume

(will
(will be
be replaced
replaced by
by
Cinder)
Cinder)

nova-network
nova-network

(will
(will be
be replaced
replaced by
by
Neutron)
Neutron)

nova-schedule
nova-schedule

2013 Cisco and/or its affiliates. All rights reserved.

14

Nova bare-metal with UCS Manager Blueprint (CDN)

Client
API
calls

Compute
(Nova)

Infrastructure

nova-api
nova-api

Hypervisors
Hypervisors
Bare-metal
Bare-metal
Nodes
Nodes

nova-compute
nova-compute
nova-volume
nova-volume

(will
(will be
be replaced
replaced by
by
Cinder)
Cinder)

nova-network
nova-network

(will
(will be
be replaced
replaced by
by
Neutron)
Neutron)

nova-schedule

UCS Edition

3
2013 Cisco and/or its affiliates. All rights reserved.

UCS Manager
Creates Server Profile
based on request
parameters
Returns identity for
storage in OS DB
15

Networking

2013 Cisco and/or its affiliates. All rights reserved.

16

Cisco Unified Fabric

Continuous Market Leadership
DC TECHNOLOGY LEADER

40,000+
11M+

Cisco NX-OS
Customers

10GE Ports
Shipped

Cisco FabricPath
Customers

Cisco FEX
Customers

1,500+
11,000+

DATA CENTER SWITCHING LEADER

Market share by revenue

in Q3 2012 for DC Ethernet
Switching at 71.7%*

Market share by revenue

in Q3 2012 for FCoE SAN
Switching at 87.3%**

*Source: Infonetics, Q3 2012 DC Network Equipment Report, December 2012 **Source: DellOro, SAN Switching, November 2012
Data
current
of AllDecember
Cisco Confidential
2011 Cisco
and/or its as
affiliates.
rights reserved. 2012. Subject to change without notice.

17

Cisco Unified Fabric Innovations

LAN

LAN/SAN

Cisco
Cisco
Cisco
Nexus 3000 Nexus 4000 Nexus 5000

Cisco Nexus 1010

Cisco
Cisco
Nexus 7000
Nexus 6000

Cisco
Nexus 2000

Cisco Nexus 1000V

CISCO NX-OS: From Hypervisor to Core CISCO DCNM: Single Pane of Management
DELIVERING TO YOUR DATA CENTER NEEDS
Workload
Resilient,
Workload
Resilient, High
High
Mobility
Performance,
Mobility Within/
Within/
Performance,
Across
Scalable
Fabric
Across DCs
DCs
Scalable Fabric
2011 Cisco and/or its affiliates. All rights reserved.

Secure
Secure
Separation/
Separation/
Multitenancy
Multitenancy

LAN+SAN
LAN+SAN
Convergence
Convergence

Operational
Operational Efficiency
Efficiency
Cisco Confidential

18

Cisco Nexus Plugin Diagram

http://docwiki.cisco.com/wiki/OpenStack:Grizzly-Nexus-Plugin

2013 Cisco and/or its affiliates. All rights reserved.

19

Cisco Nexus Plugins for Neutron Benefits

Automated VLAN Provisioning
Configure VLANs on the Nexus switch

Layer 3 Gateway

Map Nexus Switch Virtual Interface (SVI) to tenant VLAN

Scalability with Top of Rack (ToR) Nexus as default Layer 3 Gateway
Eliminates configuration and bottleneck of host-based software L3 forwarding Agent
Multi-Homed Host Deployments
Virtual Port Channel (vPC) for High Availability (HA) and link optimization to multiple Nexus switches

Hardware and Software-based Networking

Performance benefits of hardware-based ToR switch (Nexus 3000, 5000, 6000, 7000)
Flexibility of software-defined Networking with Nexus 1000V

2013 Cisco and/or its affiliates. All rights reserved.

20

Nexus Switch as Layer 3 Gateway

Management Network

nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
*-plugin-agent
nova-compute
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent

dhcp-agent
dhcp-agent

Flat Networking Traffic

VLAN Traffic across Nodes
GRE or VXLAN tunnels across Nodes

*-plugin-agent
*-plugin-agent
L2B/OVS
L2B/OVS
L2B/OVS
L2B/OVS

Network
Network Node
Node

Compute
Node
L2B/OVS
Compute
Node
Compute
Node
L2B/OVS
Compute
Node
Compute
Node
Compute
Node
Compute
Compute Node
Node

Data Network
SVI configured on Nexus for L3
forwarding and external
Gateway
Removes bottleneck of generic
server-based network node
with Linux IP tables
2013 Cisco and/or its affiliates. All rights reserved.

nova-api
nova-api
nova-scheduler
nova-scheduler
neutron-server
neutron-server
keystone
keystone
mysql,
mysql, rabbit...
rabbit...
Nexus
Nexus PI
PI

Cloud
Cloud Controller
Controller
Node
Node

API Network
External Network
Internet
Internet

API Network is
typically routable to
enable public access

21

Service Chaining with Nexus 1000V

Foundation of Virtual Services

Architecture
vPath Service Insertion/Chaining
VXLAN Overlay Networking

Management Network

dhcp-agent
dhcp-agent
dhcp-agent
dhcp-agent
dhcp-agent
dhcp-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
l3-agent
l3-agent
l3-agent
l3-agent
l3-agent
l3-agent

nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
*-plugin-agent
nova-compute
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent

VSM/N1000V
Network
Node
VSM/N1000V
Network
Node
Network
Node
Network
Node
Network
Network Node
Node

Compute
Node
N1000V
Compute
Node
Compute
Node
N1000V
Compute
Node
Compute
Node
Compute
Node
Compute
Compute Node
Node

Data Network

nova-api
nova-api
nova-scheduler
nova-scheduler
neutron-server
neutron-server
keystone
keystone
mysql,
mysql, rabbit...
rabbit...

Cloud
Cloud Controller
Controller
Node
Node

API Network
External Network
Internet
Internet

2013 Cisco and/or its affiliates. All rights reserved.

API Network is
typically routable to
enable public access

22

CSR 1000V Routing

Management Network

Network or Compute node(s) hosts CSR

nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
nova-compute
*-plugin-agent
nova-compute
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent
*-plugin-agent

dhcp-agent
dhcp-agent

CSR Provides per tenant

isolation and full IOS
capabilities including VPN,
BGP, OSFP, MPLS, etc.

*-plugin-agent
*-plugin-agent
CSR
CSR 1000V
1000V
VSM/N1000V
VSM/N1000V

Network
Network Node
Node

Compute
Node
N1000V
Compute
Node
Compute
Node
N1000V
Compute
Node
Compute
Node
Compute
Node
Compute
Compute Node
Node

Data Network

nova-api
nova-api
nova-scheduler
nova-scheduler
quantum-server
quantum-server
keystone
keystone
mysql,
mysql, rabbit...
rabbit...

Cloud
Cloud Controller
Controller
Node
Node

API Network
External Network
Internet
Internet

2013 Cisco and/or its affiliates. All rights reserved.

API Network is
typically routable to
enable public access

23

New OpenStack Services from Cisco Advanced Services

Portfolio
Strategy and
Assessment

(Available Now)

Validation

(Available Now)

Design &
Deployment

(December 2013)

Optimization
Optimization

(December
(December 2013)
2013)

Problems Solved

Key Deliverables

Is OpenStack the correct

platform for my business
What are my key requirements
for OpenStack?

Strategy Assessment high

level roadmap and
architecture
Prioritization of use cases

Pre-defined design
Rapid installation & Test
Lack of OpenStack skillsets

Pre-defined design
Test plan
Knowledge Transfer

How to create or add

production safety, availability
and scale to my openstack
deployment.
Custom application assistance
Topology and requirements
evolution

2013 Cisco and/or its affiliates. All rights reserved.

Network Scale and High

availability design
Storage Integration
Cell deployment design
Design review
Software Upgrade procedures
Day 2 Support for Customized
deployments

Key Benefits
Understand role of
OpenStack in your DC/Cloud
strategy

Experiment with OpenStack

installation in your data center
environment
Accelerate production
readiness
Optimally deployed on Cisco
hardware
Ensure deployment evolution
Targeted support expertise for
your customized solution

24

RedHat OpenStack with Cisco UCS

2013 Cisco and/or its affiliates. All rights reserved.

25

Cisco OpenStack Installer

To run the install script, copy and paste the following on your command line (as root with your proxy set if
necessary as above):
curl -s -k -B https://raw.github.com/CiscoSystems/grizzly-manifests/multi-node/install_os_puppet | /bin/bash
With a proxy, use:
https_proxy=http://proxy.example.com:80/ curl -s -k -B https://raw.github.com/CiscoSystems/grizzlymanifests/multi-node/install_os_puppet > install_os_puppet
chmod +x install_os_puppet
./install_os_puppet -p http://proxy.example.com:80/

High Availability Option

The Cisco OpenStack High-Availability Guide differs from the OpenStack High Availability Guide by
providing an active/active, highly scalable model for OpenStack deployments. The architecture
consists of the following components used to provide high-availability to OpenStack services
Galera Cluster for MySQL ,RabbitMQ Clustering, RabbitMQ Mirrored Queues, HAProxy,
Keepalived

http://docwiki.cisco.com/wiki/COE_Grizzly_Release:_High-Availability_Manual_Installation_Guide
2013 Cisco and/or its affiliates. All rights reserved.

27

Summary and Next Steps

Cisco offers a complete Compute, Networking and Storage Solution for OpenStack
Cisco provides Advanced and Technical Services to help migrate from Pilot to Production
Please let us know how we can help you with OpenStack by contacting us at Openstack-support@cisco.com
More information can be found at www.cisco.com/go/OpenStack

Back Up

Cisco and/or its affiliates. All rights reserved.

2010
2013
Cisco and/or its affiliates. All rights reserved.

29

Cisco UCS SmartPlay Configurations

Deployment Automation of OpenStack on UCS

Step 1 : Configuring Nodes using Python SDK
Pre-configure UCS

Provision UCS Servers

Hostname / IP address

Chassis/Server Discovery

Logical credentials

Service Profile Association

Resource allocation preferences

PXE boot devices deployed

Register Nodes

Cobbler database update

Only Point of User Touch

Step 2 : Cobbler/Puppet based Node Subscription

Event Listener

Host OS Install

Updates the newly added node

info in puppet

PXE boot for initial OS install

Puppet apply

RHEL 6.4 installation on bare-metal

servers

Add hosts/system in OpenStack

Sync all the plugins from Puppet

Master

OpenStack Handover
Inventory of nova nodes on
controller
VM Provisioning
OpenStack Services Deployment

Cobbler/Puppet based Node Subscription

Build Node

1. Read conf file

2. Apply policies
3. Update Puppet/Cobbler DB
4. PXE Boot

5. Puppet sync

Control
Node

(glance, scheduler,
API-deamons)
Compute Nodes

(nova-

compute, libvirtd)

2013 Cisco and/or its affiliates. All rights reserved.

32

OpenStack Compute Nodes Provisioned!

2013 Cisco and/or its affiliates. All rights reserved.

33

OpenStack Neutron Architecture

Clients

Neutron (Formerly Quantum) Service

2013 Cisco and/or its affiliates. All rights reserved.

Networks

34

Getting Started with Cisco Nexus Plugins for Neutron

OpenStack Module Structure
/neutron/plugins/cisco/ - Contains the Network Plugin Framework
/client - CLI module for core and extensions API
/common - Modules common to the entire plugin
/conf - All configuration files
/db - Persistence framework
/models - Class(es) which tie the logical abstractions to the physical topology
/nexus - Nexus-specific modules
/test/nexus - A fake Nexus driver for testing the plugin

https://wiki.openstack.org/wiki/Cisco-quantum
2013 Cisco and/or its affiliates. All rights reserved.

35

Edit ../neutron/conf/neutron.conf
core_plugin = neutron.plugins.cisco.network_plugin.PluginV2
[keystone_authtoken]
auth_host = <authorization host's IP address>
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = <keystone admin name>
admin_password = <keystone admin password>

https://wiki.openstack.org/wiki/Cisco-quantum
2013 Cisco and/or its affiliates. All rights reserved.

36

Configure Database, vSwitch & VLAN Parameters

/neutron/plugins/cisco/cisco_plugins.ini file
mysql -u<mysqlusername> -p<mysqlpassword> -e "create database neutron_l2network
vswitch_plugin=neutron.plugins.openvswitch.ovs_neutron_plugin.OVSNeutronPluginV2
/neutron/plugins/openvswitch/ovs_neutron_plugin.ini
[OVS]
bridge_mappings = physnet1:br-eth1
network_vlan_ranges = physnet1:1000:1100
Tenant_network_type = vlan

https://wiki.openstack.org/wiki/Cisco-quantum
2013 Cisco and/or its affiliates. All rights reserved.

37

Configure Nexus Switch Credentials

/neutron/plugins/cisco/cisco_plugins.ini file
[NEXUS_SWITCH:1.1.1.1]
# Hostname and port used of the node
compute-1=1/1
# Hostname and port used of the node
compute-2=1/2
# Port number where the SSH will be running at the Nexus Switch, e.g.: 22 (Default)
ssh_port=22
# Provide the Nexus credentials, if you are using Nexus switches. If not this will be ignored.
username=admin
password=mySecretPasswordForNexus

https://wiki.openstack.org/wiki/Cisco-quantum
2013 Cisco and/or its affiliates. All rights reserved.

38