You are on page 1of 67

Building Hybrid Clouds

with CSR 1000v

Steven Carter, Solutions Architect
Chris Hocker, Consulting Systems Engineer

• CSR Deployment in AWS

• On-Prem Deployment Options in VMware & OpenStack
• Building Scalable Overlay Networks
• Deploying CSR Features
CSR Deployment in AWS
CSR 1000V Architecture – Virtualized ASR 1001
Forwarding Plane (FP) Control Plane
Virtualized IOS XE
FFP Client • Generalized to work on any x86 system
/ Driver IOS
• Hardware specifics abstracted through a
Forwarding Mgr. Chassis Mgr.
virtualization layer
Forwarding Mgr.
FFP code Linux Container
• Forwarding (ESP) and Control (RP)
mapped to vCPUs
vCPU vMemory vDisk vNIC • Bootflash: NVRAM: are mapped into
memory from hard dis

Hypervisor (VMware / Citrix / KVM)

• No dedicated crypto engine – we
leverage the Intel AES-NI instruction set
to provide hardware crypto assist.
CPU Memory Disk NIC

Physical Hardware
CSR 1000V Architecture - IOSd
Forwarding Plane Control Plane • Runs as a process under the Guest Linux
FFP Client IOS
/ Driver
• IOS timing is governed by Linux Kernel
Chassis Mgr.
Chassis Mgr.
Forwarding Mgr. scheduling
Forwarding Mgr.
• Provides virtualized management ports
FFP code • Since these are managed by their respective
software processes
vCPU vMemory vDisk vNIC
• No direct hardware component access!
• Communicates with other software processes
Hypervisor (VMware / Citrix / KVM) via IPC
• Runs Control plane features
CPU Memory Disk NIC • CLI and configuration processing
Physical Hardware • SNMP handling, routing protocols, session
Q: Where can I find the CSR on AWS?
A: In the AWS marketplace!

1. Search for “Cisco”

2. Pick a flavor
CSR 1000V Licensing for AWS
Two Options…
Bring Your Own License “BYOL” AWS Marketplace Billing

• Provision “BYOL” CSR instances from AWS • Provision hourly billed CSR instances from AWS
Marketplace Marketplace
• Only pay AWS for basic instance-type fees • Pay AWS for basic instance-type usage AND fees
• Purchase desired license from Cisco or Cisco for CSR usage
• AWS pays Cisco for CSR usage fees they collect.
• Install purchased license onto “BYOL” version of You pay Cisco nothing directly.
CSR you provisioned from the AWS Marketplace
• No license file to manage or install
CSR 1000V Licensing Structure Example:
Pick one option from each column…
IP Base
Technology Package Throughput License Type 250 Mbps
(See next slide for details)
10 Mbps
IP Base Perpetual
50 Mbps

100 Mbps
SEC 250 Mbps
500 Mbps (1-year or 3-year)
AppX 1 Gbps

2.5 Gbps

5 Gbps Usage
AX (target date Q1 CY15)
10 Gbps
* CSR add-on license options not shown above
CSR 1000V Features Per Technology Package
IOS-XE Features
 Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS
 Multicast: IGMP, PIM
IPBase  High Availability: HSRP, VRRP, GLBP
(formerly Standard)  Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS
 Basic Security: ACL, AAA, RADIUS, TACACS+
 Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF
IPBase Plus…
SEC  Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN,
(formerly Advanced)

IPBase Plus…
 Advanced Networking: L2TPv3, BFD, MPLS, VRF, VXLAN
 Application Experience: WCCPv2, AppXNAV, NBAR2, AVC, IP SLA
 Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS

(formerly Premium)

 Features in Red will not work in Amazon – infrastructure issues (lack of L2

support, Multicast not supported)
What are all the different CSR 1000V types listed?
1. Cloud Services Router 1000V BYOL
• Can be any tech package and throughput level depending on license purchased from Cisco and
installed on CSR (not all throughputs supported)

2. Cloud Services Router 1000V Security Tech Package

• Includes features from the Security technology package. Performance based on AWS instance
type selected (more or less vCPU/vMemory)

3. Cloud Services Router 1000V AX Tech Package

• Includes features from the AX technology package. Performance based on AWS instance type
selected (more or less vCPU/vMemory)

4. “Maximum Performance” versions of the above three

• Enables SR-IOV enhanced networking for higher performance

5. CSR Direct Connect 1 Gig and Multi-Gig

• Instances used for securing AWS Direct Connect circuits
CSR 1000V in Microsoft Azure

 Available in Azure Marketplace

(End of June):

 Search for “Cisco”

 CSR 1000V product page will
contain pricing, support, and
deployment information
CSR with InterCloud Fabric

Secure L2 Extension VLAN A
InterCloud InterCloud
Extender Switch VM VM
VM VM Trunk


On-Prem AWS
Cisco ASAv Firewall and Management Features
Subset of ASAv features are
Cisco® ASA Feature Set not supported in AWS

 VLAN tagging
 Virtualization displaces multiple-context and clustering
 Parity with all other Cisco ASA platform features

Cisco  Traditional (Cisco ASDM and CSM) management tools

ASAv  Dynamic routing includes OSPF, EIGRP, and BGP

in AWS  IPv6 inspection support, NAT66, and NAT46/NAT64

 REST API for programmed configuration and monitoring
 Cisco TrustSec® PEP with SGT-based ACLs
 Zone-based firewall, Equal-Cost Multipath
 Policy Based Routing, VxLAN Support (VTEP)
Removed clustering and  Failover Active/Standby HA model
multiple-context mode
VPC 101
Maps to AWS
• Logically isolated network with its Elastic IP
own IP range, routes, security, etc.
• IP ranges can be overlapping
Internet IP
• Internet gateway routes outside and 54.x.x.x
between VPCs
• Public IP or NAT for egress
• VPC peering needed to route
between VPCs
• Security: • Subnet “router” routes within the
• Network ACLs at the border of VPC
• Security Groups within the VPC • Subnet “router” is really an
encap/decap device b/w hypervisors
CSR placement in the AWS network
• NAT at the Internet GW
• Will break services that do not work
Maps to AWS
over NAT, such as GET-VPN Elastic IP
• Tunnel source will be a private
address Gi2 Gi1
• Tunnel destination from the Internet IP
perspective of VPN peers will be a 54.x.x.x
public address
• Assign EC2 elastic IP address so that Gi2 Gi1
address does not change if the
CSR1K is shutdown
• Other VPCs see Elastic IP address • CSR should be the default gateway
unless using VPC peering for the application VMs
No Link Local Broadcast in the VPC
• No Link local multicast or broadcast
• Affected Services Include: NAT

• IGPs 54.x.x.x
• Proxy ARP, Gratuitous ARP > LISP-VM
• GRE as work-around for some services
• FHRP difficult b/c of AWS Routing
Multiple Ways to Insert CSR as Gateway
• Two Armed Mode
• CSR has one interface in each network
• Instances have default gateway changed to point
to CSR IP or change AWS Route Table default g1 g2
• Limitation on # of interfaces for CSR imposed by

• One Armed Mode

• CSR has single interface and a default gateway
pointed towards AWS Internet Gateway
• Other subnets have route added to their route AWS IGW g1 VPC
table, pointing to the CSR as gateway Router

• Instances in other subnets don’t need their default

gateway manually changed. Continue to use
AWS Route Table.
Management and Front Door VRF
• Management and remote access of the CSR will
happens over a public interface (i.e. Floating IP)
• No interactive console on AWS
• Cisco VPN designs recommend front-door VRF
• Simplifies routing: send a default route over the tunnel
• Improves security: isolating the LAN from the public internet

• Configuring VRF causes loss of connectivity

• EEM script used to work around.
• Internet access required for other AWS services (e.g.
• Can not use front-door VRFs in these scenarios
CSR Advantages over…
Virtual Private Gateway: VPC Peering:
• Scalability • Overlapping CIDR blocks
• Continuity of Operations • Peering between regions
• Spoke-to-spoke routing • Transitive peering relationships
• Richer routing features • Multiple peerings per VPC
• Security/Application Visibility • Unicast Reverse Path Forwarding
• Spoke-to-spoke routing
Multi-Site, Full Mesh Hybrid Cloud

Full Tunnel Mesh

West Coast Region East Coast Region

Corporate Network
Overlay Options HQ

On-Prem Anchored Overlay:
• Traditional physical enterprise with good connectivity at HQ
• Redundant DM-VPN at HQ AWS AWS
• Extends enterprise network to other sites, field offices, teleworkers, West East
and public clouds Home Branch

Cloud Anchored Overlay:

• Traditional physical enterprise with less-good connectivity or wanting HQ
geographic redundancy
• Virtual-only enterprise with Cloud-based DC Head-End Head-End
• Redundant DM-VPN in Cloud West East

• Extends enterprise to other sites, field offices, teleworkers, and public

clouds AWS AWS
West East
Home Branch
On-Prem Deployment
Options in VMware &
On-Prem Termination CSR 1000V
Hardware vs. Virtual
• Hardware: Performance, Determinism
• Virtual: Flexibility
Places in the Network 1000/ISR
• Border for Entire Organization
• Hardware: ASR/ISR Border

• Data Center for Individual Tenants: Campus

• Software: CSR CSR 1000V
Data Center
CSR in Private Cloud
• Tenant Router, Head-End, or NFV
• Supported on Multiple Hypervisors
• Managed by tenant or network team
• Manual or orchestrated deployment
• Dedicated hosts or distributed with
tenant workloads Tenant Gateway

Tenant VLANs

Hypervisor Hypervisor
CSR Images for On-Prem Deployment
Deployment in VMware

• Deploy as OVA
• Chose performance

Virtual Interfaces = Router Interfaces

g0 g1

Deployment in OpenStack
Neutron server Compute server
Hosting Device
Plugging Driver

Some server CSR1kv

service plugin Notifications

service plugin …
Driver specific
VPN-aaS communication
service plugin

Hosting devices
What is supported today – April 2015.
Openstack “I” Release “J” release “K” release

CSR as Tenant VM Supported

Routing-aaS - Merged
CSR as replacement of Neutron router

VPN-aaS CSR out of band bring up Merged

CSR for site-to-site IPsec VPN

FW-aaS plugin - - Merged

CSR as FW enabled by ACLs
Building Scalable Overlay
Enterprise VPN Termination into AWS

virtual private cloud

AWS cloud corporate office/branch

• Connect one or many physical locations into an Amazon VPC. IPSec, DMVPN,
FlexVPN, EZVPN, etc…
• Up to 1,000 concurrent VPN tunnels per CSR, and no per-tunnel charges from
• Familiar configuration, familiar troubleshooting, not a black box.
Back-End Corporate Access

Subnet 1 Subnet 1


Corporate Users

Site to Site VPN connection

(Data & management)
Internet Users
Corporate Data Center
Remote Access and Site-to-Site VPN to AWS

Subnet 1 Subnet 1


Internet Corporate Users

Internet Users
connecting via
VPN (ikev2 and Site to Site VPN connection
IPSec/L2TP) (Data & management)
Corporate Data Center
Interconnecting AWS VPCs Using the CSR 1000V

virtual private cloud virtual private cloud

US west region US east region

AWS cloud

• Easily integrate multiple AWS regions into existing VPN topology as new sites
• Can be leveraged for hierarchical designs with in regions.
• Distribute applications across the globe, and keep the network simple
DMVPN Design Model 1
Full Tunnel for AWS Application VMs

• DMVPN sites have access to DMVPN

AWS-hosted applications through Default
IPSec tunnels to CSR Route

• Uses front-door VRF for VPN

G1 G2
• AWS application VMs run in the
global routing table
• AWS application VMs do not have
local internet access or local G1 – VRF INET
access to AWS public services* G2, Tun0 - Global

• Requires EEM Script

*New feature called VPC endpoints for S3 service
Embedded Event Manager
• Provides real-time network event Create the Cisco EEM Applet:
detection and onboard automation. event manager applet fvrf
event none
• Adapt the behavior of your network action 1.0 cli command "enable”
action 1.1 cli command "conf t”
devices to network conditions action 1.2 cli command "interface gig1”
action 1.3 cli command "vrf forwarding
• More than 20 event detectors internet-vrf”
action 1.4 cli command "ip address dhcp”
• Simple applets and more complex action 2.0 cli command "end”
Run the Cisco EEM Applet:
event manager run fvrf
DMVPN Design Model 2
Direct Internet Access for AWS Application VMs

• DMVPN sites have direct access DMVPN

to AWS-hosted applications Routes
• VPN and AWS application VMs run
in global routing table AWS IGW Tun0

• Leverage NAT overload to the G1 G2

Elastic IP address Default
• AWS application VMs have local
internet access and local access to
AWS public services G1, G2, Tun0 - Global
CSR VPN High Availability
• No virtual IP as with HSRP, CSR
since AWS doesn’t allow Subnet
multicast Subnet A

• AWS Route Tables for app

subnets are re-pointed to
opposite CSR App
Subnet B
• Failure detection is automatic
• CSR itself calls AWS API to
adjust AWS Route Table
routes Before HA Failover
After HA Failover
CSR VPN HA Configuration
Create IAM ChangeRouteRole
"Version": "2012-10-17",
"Statement": [
"Effect": "Allow",
"Action": [
"Resource": "*"
CSR VPN HA Configuration
Deploy CSR and Assign IAM Role
CSR VPN HA Configuration
Configure GRE Tunnel, BFD, and EIGRP
interface Tunnel1

ip address

bfd interval 500 min_rx 500 multiplier 3

tunnel source GigabitEthernet1

tunnel destination CSR
! Subnet A

router eigrp 1

bfd interface Tunnel1 Tunnel1

passive-interface GigabitEthernet1 Subnet B
CSR VPN HA Configuration
Configure EEM

event manager environment CIDR

event manager environment ENI eni-d679128f

event manager environment RTB rtb-631bda06

event manager environment REGION us-west-2/

event manager applet replace-route2

event syslog pattern "\(Tunnel1\) is down: BFD peer down notified"

action 1.0 publish-event sub-system 55 type 55 arg1 "$RTB" arg2

"$CIDR" arg3 "$ENI" arg4 "$REGION"
Direct Connect With CSR 1000V
• Remove existing BGP configuration from customer router
• Create new BGP neighbor relationship between tunnel interface addresses (to ensure
routes are learned via tunnel)
• Advertise prefixes from campus/data center and AWS VPC
Direct Connect
Corporate HQ Sub-Interface: BGP Virtual Private Cloud Gig1.100

Customer Virtual Private CSR 1000V

Router Gateway (VGW) IP:
(Cisco EIP:

IPSec Tunnel
Interface Tunnel 1 Interface Tunnel 1
Destination: Destination:
BGP Advertisements: BGP Advertisements:
Deploying CSR Features
Firewall and Application Visibility in the AWS Cloud
• Stateful firewall between AWS
regions and physical locations
• Familiar Zone-Based Firewall
• Application Visibility and Control
virtual private cloud • Uses NBAR2 to identify over 1,000
AWS cloud corporate office/branch different applications
• Monitor and control application
Flexible NetFlow Records
• Track packet loss, latency, jitter,
and response time of your cloud.
Edge Router and Firewall

Subnet 1 Subnet 1



Internet users accessing AWS

resources using translated IPs
Internet Users
Zone Based Firewall Configuration Example (1/2)
class-map type inspect match-any tunnel-
match protocol icmp Outside Inside
match protocol http
match protocol https g1 g2
match protocol ssh
match access-group name tunnel-inside

ip access-list extended tunnel-inside
permit tcp any host eq 3389

policy-map type inspect tunnel-inside

class type inspect tunnel-inside
class class-default
drop log
Zone Based Firewall Configuration Example (2/2)
zone security outside
zone security inside
zone security tunnel
Outside Inside

g1 g2
zone-pair security tunnel-inside source
tunnel destination inside
service-policy type inspect tunnel-inside
interface Tunnel0
zone-member security tunnel
interface GigabitEthernet1
zone-member security outside
interface GigabitEthernet2
zone-member security inside
Floating IP:
interface GigabitEthernet1
g1 g2
ip nat outside
interface GigabitEthernet2
ip nat inside
ip nat inside source list nat interface GigabitEthernet1 overload
ip nat inside source static tcp 80 80 extendable
ip access-list standard nat
Needs to be the Internal Address
Enterprise-Wide Application Visibility
• Uses Netflow and IP SLA
• GUI for application visibility
• IP SLA configuration and monitoring
• Extends application visibility to your
cloud border
Enterprise-Wide Security Visibility
• Uses Netflow
• GUI for security visibility
• Extends application visibility to your cloud:
• Detecting Sophisticated and Persistent
• Identifying BotNet Command & Control FlowCollector

Activity https

• Uncovering Network Reconnaissance

• Finding Internally Spread Malware Management
• Revealing Data Loss
• Actively monitor and measure ip sla 1
icmp-echo source-ip
• Includes data about response time, ip sla 2
one-way latency, jitter, packet loss, icmp-echo source-ip
voice-quality scoring, network tag DMVPN_SLA
resource availability, application ip sla group schedule 1 1-3 schedule-
period 60 frequency 60 start-time now life
performance, and server response forever
time ip sla responder

• Performance data can be used in

routing decisions and EEM
• Detect Partner Failover
Remote Worker VPN Access into AWS

virtual private cloud

AWS cloud

• IPSec and SSLVPN access via AnyConnect for teleworkers and remote users
• AAA server options for user database
• Easily host copies of your apps in regions close to your remote users
• No similar service offered natively by AWS
SSL VPN Configuration Example (1/3)
Create a Server Certificate
crypto key generate rsa label sslvpn-key
• A self-signed certificated is modulus 2048
generated by default when the CSR
is launched.
crypto pki trustpoint sslvpn-self-signed
• Can generate a new self-signed
enrollment selfsigned
certificate or provision a certificate
from an Enterprise CA subject-name cn=csr-aws-sslvpn

revocation-check none

rsakeypair sslvpn-key

crypto pki enroll sslvpn-self-signed

virtual private
AWS cloud
SSL VPN Configuration Example (2/3)
Configure User Database and Address Pool
• User database can be on AAA aaa new-model

server or defined locally aaa authentication login sslvpn local

aaa authorization exec default local

aaa authorization network sslvpn local

username chocker privilege 15 secret 5


virtual private
cloud ip local pool pool1
AWS cloud
SSL VPN Configuration Example (3/3)
Configure Crypto
crypto ssl proposal proposal1 crypto ssl profile profile1

protection rsa-aes128-sha1 match policy policy1

! aaa authentication list sslvpn

crypto ssl authorization policy auth- aaa authorization group list sslvpn auth-
policy1 policy1

netmask authentication remote user-credentials

pool pool1 !

! crypto vpn anyconnect

crypto ssl policy policy1 3.1.05187-k9.pkg sequence 1
ssl proposal proposal1

pki trustpoint sslvpn-self-signed sign

ip interface GigabitEthernet1 port 443

 REST is Representational State Transfer
 Based on HTTP. Client-Server model. Stateless. are/restapi/restapi/RESTAPIintro.html

 Identify resources through URIs - /api/v1/global/ntp/servers

 Request & Response type: JSON (Javascript Object Notation)
 Common Methods: PUT, POST, GET, DELETE

PUT /api/v1/global/host-name 200 Ok

Content-Type: application/json Content-Type: application/json

Accept: application/json
“host-name”: “eng-router”
“host-name”: “eng-router”
} }

200 Ok

GET /license/UDI Content-Type: application/json

Accept: application/json {
“link: “/license/UDI”,
Cisco CSR 1000v Summary
• Extends enterprise network to public cloud
• Normalize operations across multiple public clouds
• Hybrid cloud designs using CSR in the public cloud and ASR1K/ISR/CSR1K
• Primary use case - secure connectivity using IPSec, DMVPN, SSL VPN, etc.
• Enterprise-class networking services including Routing, FW, and NAT
• Rich telemetry for security and performance monitoring with Netflow/AVC
• Used with AWS Direct Connect for encryption and overlay routing
• HSRP-like High Availability for AWS VPCs
CSR 1000v in AWS
Design Guide
Evaluation Licenses
• Only BYOL instances need an evaluation license, since non-BYOL instances
are pre-licensed as part of the hourly cost.
• By default BYOL instances boot with all features and 100 Kbps throughput.
• 60-day evaluation licenses are self-serve at:
• Router# show license udi
• AWS VPC Presentations



• CSR in AWS Support Forum


• CSR in AWS Test Drive


• CSR in AWS Marketplace

• Evalulation Licenses
Thank you
Participate in the “My Favorite Speaker” Contest
Promote Your Favorite Speaker and You Could be a Winner
• Promote your favorite speaker through Twitter and you could win $200 of Cisco
Press products (@CiscoPress)
• Send a tweet and include
• Your favorite speaker’s Twitter handle @ciscocloudguy
• Two hashtags: #CLUS #MyFavoriteSpeaker

• You can submit an entry for more than one of your “favorite” speakers
• Don’t forget to follow @CiscoLive and @CiscoPress
• View the official rules at
Complete Your Online Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner
will receive a $750 Amazon
gift card.
• Complete your session surveys
though the Cisco Live mobile
app or your computer on
Cisco Live Connect.
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions