Brkarc 2003

BRKARC-2003
Cisco ASR 9000 Architecture

Christian Schmutzer
Version 4.0, 29-Jan-2013

BRKARC-2003 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco ASR9000 Next-Gen Edge Routing Platform

Key Design Goals & System Benefits
ASR9912 ASR9001
Architectural Design for Longevity
Product Portfolio with significant HW and SW commonality

Highly integrated Network Processors for High Speed Scale and Feature Flexibility Cisco IOS XR based
Truly modular, full distributed OS Enhanced for the Edge (L2 and L3)
ASR9000v
ASR9010
nV (Network Virtualization) for Operational Simplicity
ASR9006
BRKARC-2003
2013 Cisco and/or its affiliates. All rights reserved.
Cisco Public
Agenda
ASR9000 Hardware Overview

System Introduction and Chassis Overview
System Components
Carrier Class, Scalable System Architecture

Fabric Architecture Linecard Architecture Cisco IOS XR Overview Control Plane and Packet Forwarding Infrastructure
Cisco nV Network Virtualization Summary

ASR 9000 Hardware Overview
BRKARC-2003
Cisco Public
System Introduction and Chassis Overview
ASR 9000 Chassis Overview

Identical hardware, software, and forwarding Across multiple physical form factors Flexibility in data- to control-plane relationship
15Tbps*
3.5 Tbps*
1.7 Tbps* 120 Gbps*
ASR 9001
Switch fabric capacity Size Max 1/10/100GE 120Gbps/system 4x10 + 2 I/O bays 2RU 40/12*
ASR 9006
440G/slot 4 I/O slots 10RU 160/144/8*
ASR 9010
440G/slot 8 I/O slots 21RU 320/288/16*
ASR 9922
770G/slot 20 I/O slot 44RU 800/720/40*
Hardware in development to double fabric capacity and port density over 18-24mo
*actual, shipping, year 2012, engineering-math, not doubled, usable densities

ASR 9010 and ASR 9006 Chassis

Integrated cable management with cover System fan trays Side-to-back airflow Front-to-back airflow
RSP (0-1) (switch fabric) Line Card (0-3, 4-7) System fan trays Air draw
Line Card (0-3) RSP (0-1) (switch fabric)
cable management
V1 power shelf: 3 Modular V1 PS V2 power shelf: 4 Modular V2 PS

2013 Cisco and/or its affiliates. All rights reserved. Cisco Public
2 power shelves 6 or 8 PS
BRKARC-2003
ASR 9001 Compact Chassis
Sub-slot 0 with MPA
Sub-slot 1 with MPA
Redundant (AC or DC) Power Supplies Field Replaceable
GPS, 1588
Console, Aux, Management
BITS
EOBC ports for nV Edge (2xSFP)
Fixed 4x10G SFP+ ports

Fan Tray Field Replaceable
BRKARC-2003
Cisco Public
ASR9922 Large Scale Chassis

8+8 ACs or 11+1 DCs
Front-to-back airflow
20x Line Cards (10 top, 10 bottom) Fan trays 4x (ASR-9922-FAN)
Height : 44 RU (AC & DC) Depth : 28.65 (800mm) Width : 17.75 (fits 19 rack) 2 x RPs
4+1 Fabric Cards initially 6+1 Fabric Cards in future
BRKARC-2003
Cisco Public
System Components
Power and Cooling
ASR-9010-FAN
ASR-9006-FAN
Fans unique to chassis Variable speed for ambient temperature variation Redundant fan-tray Low noise, NEBS and OSHA compliant
DC Supplies
A B A B
1.5 kW 1) 2.1 kW
AC Supplies
Power Supply
A B
3 kW 3 kW
Single power zone All power supplies run in active mode Power draw shared evenly 50 Amp DC Input or 16 Amp AC for Easy CO Install
1) Version 1 only
Version 1 Power vs Version 2 Power System

PEM Insertion from the Front
PEM1
PEM2 PEM3 PEM1 PEM2 PEM3
Power Switch: V1 in the back V2 in the front
PEM4
V1 Power
V2 Power
Power Feed Cabling from the Back
M3
M2
M1
M0
V2 AC power
PWR A-, M3 RTN A+, M3 PWR B-, M3 RTN B+, M3

Cisco Public
V2 DC power
13
BRKARC-2003
Route Switch Processors (RSPs) and Route Processors (RPs)

RSP used in ASR9006/ASR9010, RP used in ASR9922
RSP RSP440 9922-RP
Processors
PPC/Freescale 2 Core 1.5GHz
Intel x86 4 Core 2.27 GHz RSP440-TR: 6GB RSP440-SE: 12GB Yes, 2 x 1G/10G SFP+ 220+220G (with dual RSP)
Intel x86 4 Core 2.27 GHz -TR: 6GB -SE: 12GB
RAM
RSP-4G: 4GB RSP-8G: 8GB
nV EOBC ports Switch fabric bandwidth

BRKARC-2003
No
Yes, 2 x 1G/10G SFP+ 660+110 (7-fabric model)

Cisco Public
92G + 92G (with dual RSP)
RSP440 Faceplate and Interfaces

Same Front Panel on RP for ASR9922
2x 1G nV Edge EOBC
GPS Interface ToD, 1pps, 10Mhz
2x Mgmt Eth
BITS/DTI/J.211
PTP 1) nV Edge Sync 1)
Alarms
Console & Aux

USB
Cisco Public
Status LEDs
BRKARC-2003
1) Future SW support
15
RSP Engine Architecture

BITS/1pps/ 10Mhz/ToD
Clock MEM HDD CPU
Time FPGA
Timing Domain
CF card or USB Mgt Eth Mgt Eth
EOBC
Internal communication between RPs and Line Cards 4G disk Punt FPGA Arbiter
Console
Aux
FIA
Arbitration
I/O FPGA Alarm
Crossbar Fabric ASIC Crossbar Fabric ASIC
NVRAM
Front Panel
Boot Flash
CPU Complex
Switch fabric
BRKARC-2003
Cisco Public
ASR 9000 Ethernet Line Card Overview

-L, -B, -E
First-generation LC (Trident)
A9K-40G
A9K-4T
A9K-8T/4
A9K-2T20G
A9K-8T
A9K-16T/8
-TR, -SE
Second-gen LC (Typhoon)
A9K-MOD160 A9K-MOD80 A9K-24x10GE A9K-2x100GE
MPAs 20x1GE 2x10GE 4x10GE 1x40GE 2x40GE
A9K-36x10GE
Modular SPA Linecard

20Gbps, feature ritch, high scale, low speed Interfaces
Quality of Service 128k Queues 128k Policers H-QoS Color Policing Powerful & Flexible QFP Processor Flexible uCode Architectue for Feature Richness L2 + L3 ServicesL FR, PPP, HDLC, MLPPP, LFI L3VPN, MPLS, Netflow, 6PE/6VPE
BRKARC-2003
Scalability Distributed Control and Data Plane 20Gbits, 4 SPA Bays
L3 i/f, route, session protocol scaled for MSE needs
High Availability IC-Stateful Switch Over Capability MR-APS IOS-XR base for high scale and Reliability
SIP-700
SPA Support ChOC-3/12/48 (STM1/4/16)
POS: OC3/STM1, OC12/STM4, OC-48/STM16, OC192/STM64
SPAs
ChT1/E1, ChT3/E3, CEoPs, ATM

Cisco Public 18
ASR 9000 ISM (Integrated Service Module)

CDS Streaming: TV and internet streaming
Error repair
Feature
Applications
ASR 9000 ISM Capabilities

Ultra-Dense VoD, TV, Internet Streaming, Error Repair, CGv6 30-40 Gbps streaming capacity ~3 Gbps cache fill rate Works with all CDS appliances Up to 8,000 SD equivalent 3.2 TBytes at FCS - Modular Design MPEG2 & AVC/H.264 MPEG over UDP / RTP RTSP / SDP NEBS / ETSI compliant
CGN (carrier grade NAT): NAT44, DS-Lite NAT64
Bandwidth Compatibility Concurrent Streams Content Cache Video Formats Transport Session Protocols Environmental
CDS: Manage 8,000 streams up to 40G per second CGv6: 20M translations, 1M translations/sec., ~15Gbps throughput / ISM Cisco Public BRKARC-2003 2013 Cisco and/or its affiliates. All rights reserved.
ASR 9000 Optical Interface Support

XFP
All Linecards use Transceivers
Based on Density and Interface Type the Transceiver is different

1GE (SFP) T, SX, LX, ZX, CWDM/DWDM 10GE (XFP & SFP+): SR, LR, ZR, ER, DWDM 40GE (QSFP): SR4, LR4 100GE (CFP): SR10, LR4, DWDM 1)
SFP, SFP+
QSFP
All 10G and 40G Ports do support G.709/OTN/FEC
CFP
For latest Transceiver Support Information http://www.cisco.com/en/US/prod/collateral/routers/ps9853/data_sheet_c78-624747.html

1) Using Optical Shelf (ONS15454 M2/M6)
Carrier Class, Scalable System Architecture
BRKARC-2003
Cisco Public
Fabric Architecture
Cisco ASR9000 System Architecture

At a Glance
Line Card RSP
CPU
CPU BITS/DTI
FIA
FIC
Switch Fabric
BRKARC-2003 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
ASR9000 Switch Fabric Overview

3-Stage Fabric
Stage 1 Stage 2 Stage 3
Fabric frame format: Super-frame Fabric load balancing: Unicast is per-packet Multicast is per-flow
Ingress Linecard
fabric
Egress Linecard
fabric
8x55Gbps
fabric
FIA FIA FIA

2nd gen Line Card
Arbiter RSP0
FIA FIA FIA
2nd gen Line Card
8x55Gbps
fabric
Fabric bandwidth:
Arbiter RSP1 2nd gen Fabric (RSP440)
8x55Gbps =440Gbps/slot with dual RSP 4x55Gbps =220Gbps/slot with single RSP
1st/2nd Generation switch fabric compatibility

System With 2nd Generation Fabric
Ingress Linecard
8x23G bi-directional = 184Gbps
FIA0 FIA1
Dual-FIA 8xNPs 1st gen Linecard
2nd Generation Fabric (RSP440)
Egress Linecard
fabric
fabric
Arbiter RSP0
FIA FIA FIA

2nd gen Line Card
FIA
fabric
Single-FIA 4xNPs 1st gen Linecard
Arbiter
RSP1
1st/2nd Generation switch fabric compatibility

System with 1st Generation Fabric
8x23G bi-directional = 184Gbps 8x23G bi-directional = 184Gbps
FIA0 FIA1
Dual-FIA 8xNPs 1st gen Linecard
1nd Generation Fabric (RSP)
fabric fabric Arbiter RSP0
FIA FIA FIA

2nd gen Line Card
FIA
Single-FIA 4xNPs 1st gen Linecard
fabric
Arbiter RSP1
BRKARC-2003
Cisco Public
ASR 9922 Fabric Architecture : 5-plane System

Supported Today
5x8x55G bi-directional = 550Gbps
FIA FIA FIA

fabric 2nd gen Line Card
fabric
FIA FIA FIA

2nd gen Line Card Fabric cards
550Gbps/LC or 440Gbps/LC with fabric redundancy

ASR 9922 Fabric Architecture : 7-plane System

Not supported today !
770Gbps/LC or 660Gbps/LC with fabric redundancy
FIA FIA FIA

fabric 2nd gen Line Card
fabric
FIA FIA FIA

2nd gen Line Card

BRKARC-2003
Fabric cards
Cisco Public
Linecard Architecture
Generic Linecard Architecture Components
Pluggable physical interfaces

speeds: GE, 10GE, 40GE, 100GE form factors: SFP, SFP+, XFP, QSFP, CFP media/reach: T, SR, LR, ZR, LR4, SR10 colors: gray, CWDM, DWDM, Tunable
PHY
CPU
Distributed Control planes SW switched packets Inline Netflow Program HW forwarding tables
CPU
Network Processor
forwarding and feature engine for the LC scales bandwidth via multiple NPs up to 8 NPs/LC for performance vs. density options highly integrated silicon as opposed to multiple discrete components shorter connections, faster communication channels higher performance, density with lower power draw simplified software development model
NP
NP
Fabric Interface ASIC

BRKARC-2003
interface between forwarding processor and system switch fabric arbitration, framing, accounting in HW provides buffering and virtual output queuing for the switch fabric QoS awareness for Hi/Lo and ucast/mcast total flexibility regarding relative priority of unicast vs. multicast
Cisco Public
FIA
ASR 9000 Line Card Architecture Overview

1x10GE 1x10GE
Trident NP1 NP2 NP3

B0 B1
CPU
4x23G = 92G
PHY PHY
FIA0
Switch Fabric
Trident LC example: A9K-4T
3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP +
Typhoon NP1 NP2 NP3 NP4 NP5 NP6 NP7
CPU
FIA0 FIA1 FIA2 FIA3
Typhoon LC example: A9K-24x10G
Cisco Public
RSP0
8x55G = 440G
Switch Fabric
Switch Fabric
RSP1
9010/9006
BRKARC-2003
24port 10GE Linecard Architecture

CPU
NP NP NP NP NP NP NP NP
FIA
8x55G
FIA FIA FIA

Switch Fabric
Switch Fabric
RSP0
Switch Fabric
RSP1
BRKARC-2003
Each NP: 60Gbps bi-directional Each FIA: 60Gbps bi-directional 120Gbps uni-directional 2x45Mpps Cisco and/or its affiliates. All rights reserved. Cisco Public 2013

CPU
6x10GE PHY (SFP+) 6x10GE PHY (SFP+) 6x10GE PHY (SFP+) 6x10GE PHY (SFP+) 6x10GE PHY (SFP+) 6x10GE PHY (SFP+)
BRKARC-2003
NP
FIA
NP
NP NP NP NP
FIA
FIA FIA FIA FIA
Cisco Public
8x55G
Switch Fabric
RSP0
Switch Fabric Switch Fabric

RSP1
MUX FPGA
100G
CPU
Ingress NP
100GE MAC/PHY
100G
FIA
8x55G
Egress NP
FIA
FIA FIA
MUX FPGA
Cisco Public
Switch Fabric
RSP0
100G
Ingress NP
Switch Fabric
100GE MAC/PHY
100G
Egress NP
Switch Fabric
RSP1
BRKARC-2003
Module Cards MOD80

CPU
NP
MPA Bay 0
FIA
8x55G
Switch Fabric
RSP0
NP
MPA Bay 1
FIA
Switch Fabric
Switch Fabric
RSP1
BRKARC-2003
Cisco Public
Module Cards MOD160

CPU
NP
MPA Bay 0
FIA
8x55G
NP NP
MPA Bay 1
FIA FIA FIA

Switch Fabric
Switch Fabric
RSP0
NP
Switch Fabric
RSP1
BRKARC-2003
Cisco Public
MPA Port Mapping Examples for 10GE Ports

MOD80 MOD160
0
1 2 3 4 port 10GE MPA
NP
0 1 2 3 4 port 10GE MPA
NP NP NP
2 port 10GE MPA
0 1 2 port 10GE MPA
NP
0 1
NP
Network Processor Architecture Details

-L/-B and -E, -TR and -SE has same memory size
NP complex
-L/-B and -E, -TR and -SE has different memory size
FIB
LOOKUP MEMORY
-
MAC
Forwarding chip (multi core)
STATS MEMORY FRAME MEMORY
TCAM
TCAM: VLAN tag, QoS and ACL classification Stats memory: interface statistics, forwarding statistics etc
Frame memory: buffer, Queues

Lookup Memory: forwarding tables, FIB, MAC, ADJ -TR/-SE, -L/-B/-E
Different TCAM/frame/stats memory size for different per-LC QoS, ACL, logical interface scale Same lookup memory for same system wide scale scale mixing different variation of LCs doesnt impact system wide
BRKARC-2003
Cisco Public
ASR9001 Architecture
Identical HW Components as the Modular Systems
MPAs 2,4x10GE 20xGE 1x40GE
NP
Internal EOBC
FIA
LC CPU RP CPU
SFP+ 10GE
On-board 4x10 SFP+ ports
SFP+ 10GE SFP+ 10GE
SFP+ 10GE
MPAs 2,4x10GE 20xGE 1x40GE
NP
FIA
Switch Fabric
BRKARC-2003
Cisco Public
Cisco IOS-XR Overview
Cisco IOS-XR High Level Benefits
Modular Runtime SW upgrade/downgrade support
Distributed scalable with multi chassis support

Platform Independent POSIX compliant Management Interface Unified Data Model (XML)
High Availability Hot Standby and Process Restart

Security Control, Data and Management Plane
BRKARC-2003
Cisco Public
Cisco IOS-XR Software Modularity

Multicast
Ability to upgrade independently MPLS, Multicast, Routing protocols and Line Cards Ability to release software packages independently Ability to have composites into one manageable unit if desired Notion of optional packages if technology not desired on device (Multicast, MPLS)
MPLS
RPL
BGP Routing Composite OSPF ISIS
Manageability Security
Forwarding Host Composite Base

IOX Admin OS Line card
BRKARC-2003
Cisco Public
Reference Slide
Distributed In-Memory Database

DRP 1) Local-DRP RP-A
Reliable Multicast IPC improves scale and performance
Global
Management Applications (CLI/XML?SNMP) Local-Ra Consolidated System View Global
Distributed data management model improves performance and Scale
IP
Intf
OSPF
BGP
ISIS
IP
Intf
OSPF
BGP
ISIS
Reliable Multicast and Unicast IPC
LCa Local-LCa
Single Consolidated view of the system eases maintenance
IP
Intf
ARP
PPP
ACL
VLAN
QOS
1) DRPs are only supported in CRS

IOS-XR Distributed Forwarding Infrastructure

RP
IP Stack VLAN IM NetFlow
RP Drivers
Global Int. Mgr.
LC-CPU
IM VLAN PPP
PPP
ARP
ARP
HDLC NetFlow
FIB
Global IDB & AIB
Switching Fabric
Ingress FIB CPU CPU Ingress AIB & IDB

Switching Fabric
Egress FIB
CPU
CPU
Egress AIB & IDB
LC
LC
LC
LC
Single Stage Forwarding (other routers)

Single global Adjacency Information Base (AIB) distributed to all line cards Single global Interface Management DB distributed to all line cards Only Ingress FIB forces forwarding features to be run in RP
BRKARC-2003 2013 Cisco and/or its affiliates. All rights reserved.
Two Stage Forwarding (ASR9000)

Each line card has independent AIB only for local interfaces Each line card has independent Interface DB for local interfaces
Both Ingress and Egress FIB allows forwarding features to be independently run in LCs
Cisco Public 44
ASR9000 Fully Distributed Control Plane

LPTS (local packet transport service): control plane policing
EOBC (1G internal switch)
RP
CPU
Punt FPGA
LC2
FIA
Switch Fabric
CPU
BRKARC-2003
LC1
Switch Fabric
Typhoon LPTS NP NP NP NP FIA NP NP FIA FIA
RP CPU: Routing, MPLS, Multicast, HSRP/VRRP, etc
Switch Fabric ASIC
LC CPU: ARP, ICMP, BFD, Netflow, OAM, etc
FIA
NP
2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Reference Slide
Local Packet Transport Services (LPTS)

The Control Plane Protection
Application1 on RP Application1 on RP Local Stack on LC
LPTS enables applications to reside on any or all RPs, DRPs, or LCs

Active/Standby, Distributed Applications, Local processing
IFIB forwarding is based on matching control plane flows

Built in dynamic firewall for control plane traffic
LPTS is transparent and automatic

Transit Traffic
Received Traffic
Forwarding Information Base (FIB)
LPTS Internal FIB (IFIB)
Bad packets
IOS-XR High Availability Software Design Principles
NSR (GR) Non-Stop Forwarding Separate Control and Data Planes RP/DRP Redundancy Process Restartability: Active/Standby Failover Active State Check pointing
All subsystems: Separate Address Spaces memory faults affect only 1 process, recovery = restart process
Software Maintenance Updates (SMUs)
Allows for software package installation/removal leveraging on Modularity and Process restart Redundant processors are not mandatory (unlike ISSU) and in many cases is non service impacting and may not require reload. Mechanism for
delivery of software features (e.g. Multicast, MPLS) delivery of critical bug fixes without the need to wait for next maintenance release
BRKARC-2003
Cisco Public
48
Control Plane and Packet Forwarding Infrastructure
Layer 3 Control Plane Overview
LDP
RSVP-TE
BGP Static ISIS
OSPF EIGRP
RP CPU
RP
LSD
Over internal EOBC
RIB
Selective VRF download
ARP
LC CPU
SW FIB
FIB
Adjacency
LC NPU
AIB
LC CPU
AIB: Adjacency Information Base RIB: Routing Information Base FIB: Forwarding Information Base LSD: Label Switch Database
BRKARC-2003
Cisco Public
IOS-XR Two-Stage Forwarding Overview

Scalable and Predictable
NP NP NP NP NP
FIA
FIA
Ingress NP
100G
FIA
100GE MAC/PHY
Switch Fabric ASIC
Switch Fabric
FIA
Egress NP 2
100G
Switch Fabric ASIC
FIA
NP NP 1 Typhoon 2
FIA
Ingress NP
100G
100GE MAC/PHY
FIA
Switch Fabric
FIA
Egress NP
100G
Uniform packet flow for simplicity and predictable performance

MAC Learning and Sync

Hardware based MAC learning: ~4Mpps/NP
RP
Punt FPGA
1 NP learn MAC address in hardware (around 4M
pps)
2 NP flood MAC notification (data plane) message
to all other NPs in the system to sync up the MAC address system-wide. MAC notification and MAC sync are all done in hardware
CPU
FIA
Switch Fabric
Switch Fabric
CPU
3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP +
LC1
CPU
3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP +
LC2
NP 1 2 NP
FIA
NP NP NP NP NP FIA FIA
NP FIA NP NP NP NP NP FIA FIA
2
Switch Fabric ASIC
Switch Fabric ASIC
3x10GE SFP +
3x10GE SFP +
NP
NP
FIA
3x10GE SFP +
3x10GE SFP +
3x10GE SFP +
NP
Cisco Public
FIA
BRKARC-2003
FGID Fabric Group ID
Multicast Replication Overview

1 2
Fabric to LC Replication LC fabric to FIA Replication
MGID Multicast Group ID
MFIB Multicast Forwarding Information Base
3 4
FIA to NP Replication NP to egress port Replication
MFIB
3x10GE SFP + 3x10GE SFP +
Typhoon 4 NP 4
MGID
Efficiency: replicate if required Line rate: for fully loaded chassis Simple and clean architecture Predictable performance
FIA
FIA
FGID/ FPOE
Ingress NP 100GE MAC/PHY

MFIB
3x10GE SFP +
3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP + 3x10GE SFP +
NP
NP NP
Switch Fabric ASIC

MGID/ FPOE
FIA
1 Switch Fabric
MGID/ FPOE
Switch Fabric ASIC

2
MGID
FIA
Egress NP 4
FIA
NP NP
FIA
Ingress NP 100GE MAC/PHY
NP
FIA
Switch Fabric
FIA
Egress NP
Uniform packet flow for simplicity and predictable performance

Internal System QoS Overview
End-to-End priority (P1,P2, 2xBest-effort) propagation Unicast VOQ and back pressure Unicast and Multicast separation
Ingress side of LC Egress side of LC
CPU
2
CPU
3
NP NP
PHY PHY
PHY PHY
NP NP 1
1
FIA
Switch Fabric
FIA
Ingress Port QoS
4 VOQ per each virtual port in the entire system Up to 4K VOQs per FIA
4 Egress Queues per each virtual port, aggregated rate per NP

Cisco Public
Egress Port QoS
BRKARC-2003
ASR9000 nV Network Virtualization
ASR 9000 nV Technology Overview

SP Services/ Content Third-Party Services/ Content
Before: nV Technology Individual device to manage
After: nV Technology
Cisco Prime IP NGN Core
One virtual system to manage

nV Edge
Residentia l
Edge
Complex network protocols
Business Converged
nV
nV Satellite
No network protocols within virtual system, Remote satellite is plug-n-play, zero touch Single feature set, one release cycle Scale to 10 of 1000s physical ports
Feature inconsistency, inter-operability
Aggregation
Access
Physical port limit
BRKARC-2003
Cisco Public
nV Satellite
ASR 9000 nV Satellite Overview

Plug and Play, zero Touch Satellite Access Device
Satellite Protocol
Satellite access ports ASR9K local ports
Satellite
nv fabric links Host

One ASR 9000 nV System
Satellite and ASR 9000 Host run satellite protocol for auto-discovery, provisioning and management Satellite and Host could be co-located or in different location. There is no distance limitation between satellite and Host The connection between satellite and host is called nv fabric link, which could be L1 or over L2 virtual circuit (future)
Satellite access port have feature parity with ASR9K local ports it works/feels just as local port
BRKARC-2003
Cisco Public
Satellite Hardware ASR 9000v Overview

Power Feeds
Redundant -48vDC Power Feeds Single AC power feed Max Power 210W Nominal Power 159 W
Field Replaceable Fan Tray

Redundant Fans ToD/PSS Output
1 RU ANSI & ETSI Compliant
Bits Out
44x10/100/1000 Mbps Pluggables

Full Line Rate Packet Processing and Traffic Management
Copper and fiber SFP optics Speed/duplex auto negotiation
4x10G SFP+
Initially used as Fabric Ports ONLY (could be used as access port in the future) Copper and fiber SFP+ optics
Industrial Temp Rated

-40C to +65C Operational Temperature -40C to +70C Storage Temperature
Cisco Public
SFP Support Information http://www.cisco.com/en/US/docs/optical/cpt/pluggables/guide/b_cpt_pluggables.html #reference_A3D61C5FFC0A471BB39C3635CDC05E95

Satellite Hardware ASR901 Overview
GPS 1) (1pps, 10MHz, ToD)
Mgmt 1) Ethernet
4x GE Combo Ports (SFP or RJ45)
2x DC Feeds (-24 or -48 VDC)
4x GE (RJ45)
4x GE (SFP)
BITS 1)
Console
2)
1) Not supported/used when operating in nV Satellite Mode 2) Used for low level debugging only
Satellite Hardware ASR903 Overview

Router Switch Processor
Currently only 1x RSP supported
Six I/O Modules

1 port 10GE Module (XFP) nV fabric links only 8 port 1GE Module (SFP) access ports only 8 port 1GE Module (RJ45) access ports only
Fan Module
2x Power Modules
DC PEM, 1x -24 or -48 VDC AC PEM, 1x 115..230 VAC
Cisco Public 61
Satellite Host Control Plane

Satellite discovery and control protocol
CPU
MAC-DA MAC-SA Control VID Payload/FCS
CPU
Satellite ASR 9000v ASR 9000 Host
Discovery Phase
A CDP-like link-level protocol that discovers satellites and maintains a periodic heartbeat Heartbeat sent once every second, used to detect satellite or fabric link failures. CFM based fast failure detection plan for future release
Control Phase
Used for Inter-Process Communication between Host and Satellite Cisco proprietary protocol over TCP socket, it could get standardized in the future Get/Set style messages to provision the satellites and also to retrieve notifications from the satellite
Satellite Host Data Plane Encapsulation

MAC-DA MAC-SA VLANs (OPT) Payload MAC-DA MAC-DA MAC-SA nV-tag VLANs (OPT) Payload/FCS MAC-SA VLANs (OPT) Payload
Satellite ASR 9000v ASR 9000 Host
On the Satellite
Satellite receives Ethernet frame on its access port Special nV-tag is added Local xconnect between access and fabric port (no MAC learning !) Packet is put into fabric port egress queue and transmitted out toward host
BRKARC-2003
On the Host
Host receives the packet on its satellite fabric port Checks the nV tag, then maps the frame to the corresponding satellite virtual access port Packet Processing identical to local ports (L2/L3 features, qos, ACL, etc all done in the NPU) Packet is forwarded out of a local, or satellite fabric port to same or different satellite
Cisco Public
Initial Satellite Configuration

Satellite Protocol
Satellite access ports ASR9K local ports
Satellite 101
nv fabric links
Host
nv satellite 101 define satellite type asr9000v ipv4 address 10.0.0.101
interface TenGigE 0/2/0/2 configure satellite fabric port ipv4 point-to-point ipv4 unnumbered Loopback100 nv satellite-fabric-link satellite 101 remote-ports satellite to fabric port mapping GigabitEthernet 0/0/0-9
Satellite Port Configuration

Comparison to Local Port Configuration
Satellite Protocol
Satellite access ports
Satellite 101
nv fabric links
ASR9K local ports
Host
Satellite access port configuration examples
Local port configuration examples
interface GigabitEthernet 101/0/0/1 ipv4 address 1.2.2.2 255.255.255.0 interface TenGig 101/0/0/1.1 encapsulation dot1q 101 rewrite ingress tag pop 1 sym interface Bundle-ethernet 200 ipv4 address 1.1.1.1 255.255.255.0 interface GigabitEthernet 101/0/0/2 bundle-id 200
interface GigabitEthernet 0/0/0/1 ipv4 address 2.2.2.2 255.255.255.0 interface TenGig 0/0/0/1.1 encapsulation dot1q 101 rewrite ingress tag pop 1 sym interface Bundle-ethernet 100 ipv4 address 1.1.1.1 255.255.255.0 interface GigabitEthernet 0/0/0/2 bundle-id 100
Cisco Public
Satellite Deployment Models

ASR9000v Example
44x1GE Access ports 4x10GE Fabric ports 44x1GE Access ports 4x10GE Fabric ports
It can mix model 1 and 2 on the same satellite
Mode 1: Static pinning No fabric port redundancy

Access ports are mapped to a single Fabric Link Fabric Link failure does bring Access Port down
Mode 2: Fabric bundle Fabric port redundancy

Fabric links are forming a Link-Bundle
Access port traffic is hashed across Bundle Members Fabric link failure keeps all Access Ports up, re-hashing of Traffic
Cisco Public 66
Reference Slide
Satellite Monitoring and Troubleshooting
Normal operation, like show CLIs are done on the Host directly, for example
Satellite inventory reporting, environmental monitoring Interface counts, stats SNMP MIB NMS support, including ACT, ANA/ PRIME
Low level debug could still be done directly on the satellite device
User can telnet into satellite via out-of-band management console, or in-band from Host, and run regular show/debug CLIs
BRKARC-2003
Cisco Public
Reference Slide
Satellite Software Management

Everything Controller from the Host
RP/0/RSP0/CPU0:R1#sh install active Node 0/RSP0/CPU0 [RP] [SDR: Owner] Boot Device: disk0: Boot Image: /disk0/asr9k-os-mbi-4.2.1.22K.CSCtz10483-0.0.4.i/0x100305/mbiasr9k-rsp3.vm Active Packages: disk0:asr9k-px-4.2.1.22K.CSCtz10483-0.0.4.i disk0:asr9k-satellite-px-4.2.1.22K satellite image PIE disk0:asr9k-mini-px-4.2.1.22K disk0:asr9k-mpls-px-4.2.1.22K disk0:asr9k-mcast-px-4.2.1.22K disk0:asr9k-fpd-px-4.2.1.22K RP/0/RSP0/CPU0:R1#install nv satellite ? <100-65534> Satellite ID all All active satellites
Golden satellite image is always there in satellite flash card for image fall back
RP/0/RSP0/CPU0:R1#install nv satellite 100 ? activate Install a new image on the satellite, transferring first if necessary transfer Transfer a new image to the satellite, do not install yet RP/0/RSP0/CPU0:R1#install nv satellite 100 active
Satellite Plug and Play

Configure, Install and Ready-to-Go
Initial satellite configuration (on ASR9K Host
Rack & Plug
Go
Critical Error LED ON bad hardware, RMA Major Error LED ON Unable to connect to ASR9K host
Missing the initial satellite configuration? L1 issue, at least one of the uplink port light green? Security check (optional), is the satellite SN# correct?
Status light green ready to go, satellite is fully managed by Host
BRKARC-2003
Cisco Public
69
nV Edge
ASR9000 nV Edge Overview
Linecard chassis Ctrl Links Linecard chassis
Fabric Links
Fabric chassis
Leverage existing IOS-XR CRS multi-chassis SW infrastructure Simplified/Enhanced for ASR 9000 nV Edge
Ctrl Links
InterChassis Links
CRS Multi-Chassis
ASR 9000 nV Edge
Single control plane, single management plane, fully distributed data plane across two physical chassis one virtual nV system
nV Edge Architecture Details

One Virtual ASR 9000 nV System
Control Plane EOBC Extension
0 Active RSP 1 Standby RSP
RSP440 nV EOBC ports for control plane connection
Secondary RSP
Secondary RSP
Internal EOBC
LC
LC
LC
LC
LC
LC
LC
LC
Inter-chassis data link (L1 connection) 10G bundle (up to 16 ports)
Regular 10G data ports
Control plane connection: Active RSP and standby RSP are on the different chassis, they communicate via external EOBC links
Data plane connection: bundle regular data links into special nV fabric link to simulate switch fabric function between two physical chassis for data packet Flexible co-located or different location deployment (upto 10msec latency)
nV Edge Configuration
Configure nV Edge globally

nv edge-system serial FOX1437GC1R rack 1 serial FOX1439G63M rack 0
static mapping of chassis serial# and rack#
Configure the inter-chassis fabric(data plane) links

interface TenGigE1/2/0/0 nv edge interface interface TenGigE0/2/0/0 nv edge interface
NO need to configure the inter-chassis control plane EOBC ports. Its plug-and-play
After this configuration, rack 1 will reload and then join cluster after it boot up Now you successfully convert two standalone ASR 9000 into one ASR 9000 nV Edge As simple as this !!!
nV Edge Interface Numbering
Interfaces on 1st Chassis (Rack 0)

GigabitEthernet0/1/1/0 GigabitEthernet0/1/1/1.1 ... unassigned unassigned Up Shutdown Up Down
Interface on 2nd Chassis (Rack 1)

GigabitEthernet1/1/1/0 GigabitEthernet1/1/1/1.22 ... unassigned unassigned Up Shutdown Up Down
Interfaces on a Satellite connected to the nV Edge Virtual System

GigabitEthernet100/1/1/0 GigabitEthernet100/1/1/1.123 ... unassigned unassigned Up Up Up Up
BRKARC-2003
Cisco Public
74
nV Edge System Monitoring

RP/0/RSP0/CPU0:ASR4-Rack0(admin)#show dsc Thu Apr 12 03:01:12.225 UTC --------------------------------------------------------Node ( Seq#) Role Serial# State --------------------------------------------------------0/RSP0/CPU0 ( 0) ACTIVE FOX1545GRM1 PRIMARY-DSC 0/RSP1/CPU0 ( 31785) STANDBY FOX1545GRM1 NON-DSC 1/RSP0/CPU0 ( 31763) STANDBY FOX1325G77H NON-DSC 1/RSP1/CPU0 ( 32001) ACTIVE FOX1325G77H BACKUP-DSC RP/0/RSP0/CPU0:ASR4-Rack0#show platform Thu Apr 12 03:00:32.799 UTC Node Type State Config State ----------------------------------------------------------------------------0/RSP0/CPU0 A9K-RSP440-SE(Active) IOS XR RUN PWR,NSHUT,NMON 0/RSP1/CPU0 A9K-RSP440-SE(Standby) IOS XR RUN PWR,NSHUT,NMON 0/0/CPU0 A9K-2x100GE-TR IOS XR RUN PWR,NSHUT,MON 0/1/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,NMON 0/1/0 A9K-MPA-2X40GE DISABLED PWR,SHUT,MON 0/1/1 A9K-MPA-20X1GE OK PWR,NSHUT,MON 0/3/CPU0 A9K-SIP-700 IOS XR RUN PWR,NSHUT,MON 0/3/0 SPA-8XOC12-POS OK PWR,NSHUT,MON 0/3/1 SPA-2XCHOC12/DS0 OK PWR,NSHUT,MON 0/3/2 SPA-2XOC48POS/RPR OK PWR,NSHUT,MON 1/RSP0/CPU0 A9K-RSP440-SE(Standby) IOS XR RUN PWR,NSHUT,MON 1/RSP1/CPU0 A9K-RSP440-SE(Active) IOS XR RUN PWR,NSHUT,MON 1/3/CPU0 A9K-24x10GE-TR IOS XR RUN PWR,NSHUT,MON 1/4/CPU0 A9K-24x10GE-SE IOS XR RUN PWR,NSHUT,MON
nV Topologies
Single-homed, static pinning
Dual-homed to nV Edge, static pinning

nV Edge
Satellite
Satellite
Single-homed, fabric bundle
Dual-homed to nV Edge, fabric bundle

nV Edge
Satellite
Satellite
BRKARC-2003
Cisco Public
76
Cisco ASR9000 Next-Gen Edge Routing Platform

Key Design Goals & System Benefits
ASR9912 ASR9001
Architectural Design for Longevity
Product Portfolio with significant HW and SW commonality

Highly integrated Network Processors for High Speed Scale and Feature Flexibility Cisco IOS XR based
Truly modular, full distributed OS Enhanced for the Edge (L2 and L3)
ASR9000v
ASR9010
nV (Network Virtualization) for Operational Simplicity
ASR9006
BRKARC-2003
Cisco Public
77
Call to Action
Visit the Cisco Campus at the World of Solutions

to experience Cisco innovations in action
Get hands-on experience attending one of the Walk-in Labs

Schedule face to face meeting with one of Ciscos engineers at the Meet the Engineer center Discuss your projects challenges at the Technical Solutions Clinics
BRKARC-2003
Cisco Public
78
BRKARC-2003
Cisco Public
79

Brkarc 2003

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Brkarc 2003

Uploaded by

Copyright:

Available Formats

BRKARC-2003

Cisco ASR 9000 Architecture

Version 4.0, 29-Jan-2013

Cisco ASR9000 Next-Gen Edge Routing Platform

Architectural Design for Longevity

Product Portfolio with significant HW and SW commonality

nV (Network Virtualization) for Operational Simplicity

2013 Cisco and/or its affiliates. All rights reserved.

ASR9000 Hardware Overview

Carrier Class, Scalable System Architecture

Cisco nV Network Virtualization Summary

ASR 9000 Hardware Overview

2013 Cisco and/or its affiliates. All rights reserved.

System Introduction and Chassis Overview

ASR 9000 Chassis Overview

1.7 Tbps* 120 Gbps*

*actual, shipping, year 2012, engineering-math, not doubled, usable densities

ASR 9010 and ASR 9006 Chassis

Line Card (0-3) RSP (0-1) (switch fabric)

V1 power shelf: 3 Modular V1 PS V2 power shelf: 4 Modular V2 PS

ASR 9001 Compact Chassis

Sub-slot 0 with MPA

Sub-slot 1 with MPA

Redundant (AC or DC) Power Supplies Field Replaceable

Console, Aux, Management

EOBC ports for nV Edge (2xSFP)

Fixed 4x10G SFP+ ports

2013 Cisco and/or its affiliates. All rights reserved.

ASR9922 Large Scale Chassis

20x Line Cards (10 top, 10 bottom) Fan trays 4x (ASR-9922-FAN)

4+1 Fabric Cards initially 6+1 Fabric Cards in future

2013 Cisco and/or its affiliates. All rights reserved.

Power and Cooling

Version 1 Power vs Version 2 Power System

Power Switch: V1 in the back V2 in the front

Power Feed Cabling from the Back

PWR A-, M3 RTN A+, M3 PWR B-, M3 RTN B+, M3

2013 Cisco and/or its affiliates. All rights reserved.

Route Switch Processors (RSPs) and Route Processors (RPs)

PPC/Freescale 2 Core 1.5GHz

Intel x86 4 Core 2.27 GHz -TR: 6GB -SE: 12GB

RSP-4G: 4GB RSP-8G: 8GB

nV EOBC ports Switch fabric bandwidth

Yes, 2 x 1G/10G SFP+ 660+110 (7-fabric model)

92G + 92G (with dual RSP)

2013 Cisco and/or its affiliates. All rights reserved.

RSP440 Faceplate and Interfaces

GPS Interface ToD, 1pps, 10Mhz

PTP 1) nV Edge Sync 1)

Console & Aux

2013 Cisco and/or its affiliates. All rights reserved.

RSP Engine Architecture

Clock MEM HDD CPU

CF card or USB Mgt Eth Mgt Eth

I/O FPGA Alarm

Crossbar Fabric ASIC Crossbar Fabric ASIC

2013 Cisco and/or its affiliates. All rights reserved.

ASR 9000 Ethernet Line Card Overview

Modular SPA Linecard

Scalability Distributed Control and Data Plane 20Gbits, 4 SPA Bays

L3 i/f, route, session protocol scaled for MSE needs

SPA Support ChOC-3/12/48 (STM1/4/16)

POS: OC3/STM1, OC12/STM4, OC-48/STM16, OC192/STM64