BRKCRS-2815 (2020) - Connecting Multiple Fabrics Together

Cisco SD-Access –
Connecting Multiple Sites in

a Single Fabric Domain
Mahesh Nagireddy
Technical Marketing Engineer
CCIE R&S
BRKCRS-2815
Cisco SD-Access for
Distributed Campus
Mahesh Nagireddy
Technical Marketing Engineer
CCIE R&S
BRKCRS-2815
Cisco Webex Teams
Questions?
Use Cisco Webex Teams to chat
with the speaker after the session
How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space
BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Agenda
Cisco SD-Access Basic Concepts
Cisco SD-Access for Distributed Campus
• What is a Fabric Site ?
• What is a Fabric Domain ?
• What is a Transit/Peer Network ?
Cisco SD-Access Transit Types

• Cisco SD-Access : IP as Transit/Peer Network
• Cisco SD-Access : SD-WAN as Transit
• Cisco SD-Access : SD-Access as Transit
Cisco SD-Access Policy across Distributed Campus

Cisco SD-Access Distributed Campus Forwarding/Packet Walk
• Multi-Site Forwarding
• DC/WAN/Shared Services Forwarding
• Internet Access Forwarding
• SD-Access Transit Architecture Deep-Dive
Cisco SD-Access Distributed Campus Uses cases and Demo

TUE WED THU FRI
Keynote BRKCRS-2815 BRKCRS-2818 BRKCRS-2819

Cisco SD-Access – 08:30 Build a Software Defined Enterprise 08:30 Creating multi-domain architecture
09:00 Connecting Multiple Sites with Cisco SDWAN & SD-Access using Cisco SD-Access
in a Single Fabric 09:00
BRKCRS-2830 BRKCRS-3811
Cisco SD-Access – Lessons Cisco SD-Access –
BRKCRS-2821 learned from Design & Deployment
09:45
Policy Driven Manageability
BRKCRS-2810 Cisco SD-Access –
Cisco SD-Access –
A Look Under the Hood
11:00 Connecting to the DC,
FW, WAN and more! BRKCRS-2502 BRKCRS-2812
11:00 Best Practices for Design and Cisco SD-Access – Integrating
BRKCRS-2832 Deployment of Cisco SD-Access with your existing network
Extending Cisco
BRKCRS-2825
11:15 BRKARC-2020
SD-Access beyond Cisco SD Access - 11:30
Enterprise walls Cisco SD-Access - Scaling the
BRKCRS-1400 Fabric to 100s of Sites Troubleshooting the fabric
Recipe for transforming 14:30 BRKCRS-2824

Enterprise Networks
with IBN
BRKCRS-3810 Intuitive Zero-Trust Design,
Migration When Securing
Cisco SD-Access deep dive 14:45
the SD-Access Workplace
BRKCRS-2823
BRKCRS-2811 Cisco SD-Access – 16:45
Keynote
Cisco SD-Access – 17:00 Firewall Integration Customer 17:00
Connecting the Fabric Appreciation 18:30
to External Networks
IBN
Technology
Cisco SD-Access © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
SD-Access Basic
Concepts
Cisco Software Defined Access
The Foundation for Cisco’s Intent-Based Network
Cisco DNA Center

One Automated
Network Fabric
Policy Automation Assurance Single fabric for Wired and
Wireless with full automation
Outside
B B
Identity-Based
C
Policy and Segmentation
Policy definition decoupled
from VLAN and IP address
AI-Driven
Insights and Telemetry
SD-Access
Extension Client Mobility Analytics and visibility into
User and Application experience
Policy follows User
IoT Network Employee Network © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Cisco SD-Access
Fabric Roles & Terminology
▪ Network Automation – Simple GUI
Automation and APIs for intent-based Automation
Identity of wired and wireless fabric devices
Cisco ISE Cisco DNA Center
Services
▪ Network Assurance – Data Collectors
analyze Endpoint to Application flows
Assurance and monitor fabric network status
▪ Identity Services – NAC & ID Services
(e.g. ISE) for dynamic Endpoint to Group
Fabric Border IP Fabric Wireless mapping and Policy definition
Nodes Controllers
B B ▪ Control-Plane Nodes – Map System that

manages Endpoint to Device relationships
Control-Plane
Intermediate ▪ Fabric Border Nodes – A fabric device
C Nodes
Nodes (Underlay) (e.g. Core) that connects External L3
network(s) to the SD-Access fabric
Campus ▪ Fabric Edge Nodes – A fabric device

(e.g. Access or Distribution) that connects
Fabric Edge
Nodes Fabric Fabric Wireless
Access Points
Wired Endpoints to the SD-Access fabric
E E E E ▪ Fabric Wireless Controller – A fabric device

(WLC) that connects Fabric APs and
Wireless Endpoints to the SD-Access fabric
SD-Access Single-Site Topology
Challenges :
• One Subnet available across all buildings/Sites
• One Big Failure Domain
• Scale Limitations – IP Pools supported per site or
Border/Control plane Scale
B/C B/C
Fabric
E E E
B/C Border/Control Node
E Edge Node
F Fusion Node
SD-Access Multi-Site Topology
Challenges :
• No End to End Segmentation.
• Fusion Routers at every site.
F F F
B/C B/C B/C
Fabric E E E
B/C Border/Control Node

E Edge Node
F Fusion Node
SD-Access Multi-Site with SD-Access Transit
Cisco SD-Access Transit

F
B/C B/C B/C
E E E
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
SD-Access for Distributed Campus
Benefits
Maintain End-to-End Segmentation SD-Access Scale requirements
Smaller & Isolated Fault Domains Fusion device not at every site
Fabric Domain
OR
Fabric Site
Fabric Domain vs Fabric Site
Fabric Domain Fabric Site

C
B B
Fabric
Site
E E E
▪ Includes One or more Fabric sites with Transit ▪ Includes CP,B,FE, Fabric WLC & ISE PSN
network ▪ Benefits
▪ Managed by Single DNAC cluster. ▪ Scalability
▪ No End to End Segmentation between Fabric ▪ Resiliency
Domains ▪ Survivability.
▪ Fabric Site may cover a single physical location,
multiple locations, or just a subset of a location
Cisco DNAC System Scale Rel:1.3.1.0
Parameters DN2-HW-APL DN2-HW-APL-L DN2-HW-APL-XL

No of Devices 1000 2000 5000
(Switch/Route/WLC)
No of Access Points 4000 6000 12000
No of Endpoints 25,000 40,000 100,000

(Concurrent)
No of endpoints – wired: Any Any Wired: 40,000
wireless ratio Wireless: 60,000
No of Fabric Domains 10 20 20
No of Fabric Sites 500 1000 2000
No of Virtual Networks per 64/Site 64/site 256/site

Fabric Site
No of Fabric Devices per 500/site 600/site 1200/site
Fabric/site
No if IP Pools 100/site 300/site 600/site
Cisco SD-Access Transit Types
Transit/Peer Network Types
Transit/Peer Network type include
• IP-Based Transit - Leverages a traditional IP-based (VRF-LITE, MPLS) network, which

requires remapping of VRFs and SGTs between sites.
• Cisco SD-Access Transit - Enables a native Cisco SD-Access (VXLAN,SGT) fabric, with
a domain-wide Control Plane node for inter-site communication.
• Cisco SD-WAN Transit – Leverages the Cisco SD-WAN as transit and carries the context
in the Cisco SD-WAN encapsulation.
Cisco SD-Access Multi-Site Fabric
When to use IP Based Transit?
Cloud
Data Center • Organizations already using existing WAN
• Sites in different regions - Higher latencies
Typical use cases

o Connecting to Shared Services(DC)
LTE
IP Based o Direct Internet Access
Transit/Peer HQ o P2P IPSEC encryption
MPLS INTERNET
Network o Policy Based Routing
o WAN Accelerators
o Traffic engineering
Remote Branch 1
o Mobile Backhaul LTE
Remote Branch 2 Remote Branch 3
When to Cisco SD-Access Transit? – Distributed Campus/Metro Deployments
Cloud
Data Center o “Dark Fiber” links or DWDM links
o Sites are in same Metropolitan area

(a few hundred miles apart means Lower
Latency)
o Higher MTU support

Dark Fiber
Cisco SD-Access HQ
Typical use cases
Transit Metro
Metro o Native unified policy across the locations
and end-to-end segmentation using VNs
and SGTs
Campus 1 o Smaller and Isolated fault domains
Campus 2 Campus 3 o Resiliency and Scalability
When to use SDWAN based Transit?
Cloud
Data Center • Organizations that have moved to SD-WAN
• Sites in different regions - Higher latencies
LTE Typical use cases

SD-WAN
HQ o Consistent policy and end-to-end segmentation
Transit/Peer
MPLS INTERNET using VNs and SGTs
Network o Smaller and Isolated fault domains
o Resiliency and Scalability
Remote Branch 1 o Policy Based Routing
Remote Branch 2 Remote Branch 3
o Traffic engineering
o Mobile Backhaul LTE
Cisco SD-Access:
IP as Transit/Peer Network
IP Transit / Peer Network
Network Plane Analysis Perspectives
1. Control-Plane: How routes / prefixes are communicated

2. Data-Plane: Which encapsulation method is used to carry data
3. Policy Plane: How group and segmentation information is communicated
4. Management Plane: How Management Infrastructure is Integrated
Communicating to Peer Network – IP
Control/Data/Policy Plane
1
CONTROL-PLANE 1 LISP eBGP External Domain(BGP/IGP)
11
DATA-PLANE VXLAN VRF-LITE External Domain(IP/MPLS/VXLAN
11
POLICY-PLANE SGT in VXLAN SGT* External Domain ( IP ACL/SGT)
Tagging
C
B
B
B
External/Peer Domain
E E E
• Manual & Every hop needs to support SGT propagation

Inter-Connecting Fabrics/Sites
IP Based WAN MANAGEMENT
Cisco DNA-Center &
POLICY
SGTs in SXP
Per VRF
C C
SD-Access Transit SD-Access

B B B B
Fabric Site (WAN) Fabric Site
Border Border Border
BGP BGP
LISP MP-BGP / Other LISP CONTROL-PLANE
VRF-lite VRF-lite
1
VXLAN SGT (16 bits) 802.1Q 802.1Q VXLAN SGT (16 bits)
MPLS
DATA-PLANE
Header VNID (24 bits) VLAN ID (12 bits) Labels VPNID (20 bits) VLAN ID (12 bits) Header VNID (24 bits)
Inter-Connecting Fabrics/Sites
DMVPN
1
1
CONTROL-PLANE LISP DMVPN/GRE LISP
1
DATA/POLICY-PLANE VXLAN+SGT IP+SGT inline tagging VXLAN+SGT
C C
B B IP Network B B
DMVPN Tunnels
E E E E E E
Cisco SD-Access:
SD-WAN as Transit
Interconnecting Fabric Sites
Cisco SD-WAN Cisco DNA Center
API
MANAGEMENT
&
POLICY
vManage
B|C
SD-Access
B|C Transit B|C
B|C
Fabric Site SD-Access
Cisco SD-WAN
Fabric Site
LISP OMP LISP CONTROL-PLANE
1
VXLAN SGT (16 bits) IPSEC SGT (16 bits) VXLAN SGT (16 bits)
DATA-PLANE
Header VNID (24 bits) VPNID (20 bits) Header VNID (24 bits)
Cisco SD-Access:
SD-Access as Transit
Cisco SD-Access Multi-Site
Consistent Segmentation and Policy across sites
Cisco SD-Access Multi-Site Advantages:
Cloud
Data Center ➢ End-to-end Segmentation and policy
➢ Smaller or isolated Failure Domains
➢ Horizontally scaled networks
➢ Single view of Entire Network
Metro ➢ Local breakout at each Site for Direct
Cisco SD-Access
Metro
HQ Internet Access (DIA) and other
Metro
Transit Services
➢ Elimination of Fusion router at every
Campus 1 site*
Campus 2 Campus 3
Key Considerations
Cisco SD-Access Multi-Site Key
Cloud
Data Center Considerations:
➢ High-bandwidth connection (Ethernet

full port speed with no sub-rate
services)
Metro
➢ Low latency (less than 10ms as a
Cisco SD-Access HQ
Metro Metro general guideline),
Transit
➢ Should accommodate the MTU setting
Campus 1 used for SD-Access in the campus
Campus 2 Campus 3 network (typically 9100 bytes).
Cisco SD-Access Multi-Site – SD-Access Transit
CONTROL-PLANE
1
LISP LISP LISP
C C C C
B B B B
Border Border
Cisco SD-Access Fabric Site 1 Cisco SD-Access Fabric Site 2
Cisco DNA-Center
DATA+POLICY-PLANE
12
VXLAN+SGT VXLAN+SGT VXLAN+SGT
Cisco SD-Access Transit Control Plane for Global Scale
West site Prefixes Only East + West East site Prefixes Only
C Register west Register east

prefixes prefixes C
TC TC
West Site B B
Cisco SD-Access East Site
Transit
BR-W BR-E
• Each site only maintains state for in-site end-points.

• Off site traffic follows default to transit.
• Survivability, each site is a fully autonomous resiliency domain
• Each Site has its own unique subnets
Transit Control Plane Deployment Location
C
C C C
C C
West Site B B
Cisco SD-Access East Site
Transit
BR-W BR-E
➢ Device must be dedicated to the transit control plane node role.

➢ Doesn’t have to be physically deployed in Transit Area
➢ Ideally, device should not be in the data forwarding (transit path) between sites.
➢ Requires IP connectivity in the underlay from site borders at all fabric sites
➢ Deploy 2 Transit Control Plane nodes for redundancy and load balancing.
Cisco SD-Access Multisite
Fabric Border support Matrix
Cisco SD-Access Cisco SD-Access IP-Based Cisco SW-WAN

Border Node Transit Transit Transit
C9K YES YES NO
ASR1K/ISR4K YES YES YES
C6K NO YES NO
N7K NO YES NO
Remote Building 1 Remote Building 2 Remote Building N Key Decision Points
Site BN • Tends to be like a Metro area
Site B1 Site B2
B E C
with multiple buildings or sites
C B C B C B C B
• Requires direct Internet

access at multiple sites
ISP
Cisco SD-Access ISP
Internet Transit Internet • Requires local resiliency

MAN and smaller fault domains
T T
DNAC
5-7 NCP + ISP
• 2 Transit CP
DC NDP
Cluster
ISE
• 2-4 Site Borders

2 PAN 2 PXG
5-10 PSN
Internet
DDI
(Multiple Exits)
1 DHCP 1
DNS
1 IPAM
AB AB EB EB
Site HQ
CP CP
HQ Campus
Cisco SD-Access Policy across Distributed
Campus
Cisco DNAC Policy
Segmentation Strategy
Macro Segmentation Micro Segmentation

Network
Network
Building Management Campus Users Building Management Campus Users

VN VN VN VN
Virtual Network (VN) Scalable Group (SG)

First level Segmentation ensures zero Second level Segmentation ensures role
communication between specific groups. based access control between two groups
Ability to consolidate multiple networks into within a Virtual Network. Provides the ability to
one management plane. segment the network into either line of
businesses or functional blocks.
BRKCRS-2054 41
VN across Multiple Fabric Sites Cisco DNA-Center
API
C C C
B B PSN
B B B B PSN
PSN
Fabric Site 1 Fabric Site 2 Fabric Site N
VN-IOT VN-LAB
VN-Employee © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
ISE Deployment Model
Standalone and Distributed
1:1 redundancy
▪ Applies to both physical and virtual deployment
▪ Compatible with load balancers
▪ No changes to current Licensing Model
Lab and Small HA Deployment Small Multi-node Deployment Large Deployment

Evaluation 2 x (PAN+MNT+PSN) 2 x (PAN+MNT), <= 5 PSN 2 PAN, 2 MNT, <=50 PSN
35xx 100 Endpoints 20,000 Endpoints 500,000 Endpoints

36xx 100 Endpoints 50,000 Endpoints 2,000,000 Endpoints(3695-PAN & MNT)
ISE Performance & Scale

https://community.cisco.com/t5/security-documents/ise-performance-amp-scale/ta-p/3642148
BRKCRS-2815
ISE Distributed Deployment
Model 1 - Dedicated PSN per Site
PAN MnT DC 1 PAN MnT DC 2
PXG
PXG

C C C
B B PSN
B B PSN B B
PSN
• PSN Nodes dedicated to every site

• Maximum of 2 PSN’s per site
• PAN’s are centralized in Data Center © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
Model 2 - PSN Clusters with Load-Balancers
PAN MnT
DC 1 PAN MnT DC 2
PXG
PXG

C C C
B B PSN
B B PSN B B
PSN
• PSN’s are behind a dedicated Load Balancer

• DNAC site settings point to Load Balancer IP
Model 1 - Dedicated PSN per Site Cisco DNA-Center
API
C C C
B B PSN
B B B B PSN
PSN
Scalable groups tags (SGTs) available across multiple fabric sites
Cisco DNA-Center
API
C C C
B B PSN
B B B B PSN
PSN
Host 1 Host 2 Host 3 Host 4 Host 5

SGT=100 SGT=200 SGT=100 SGT=200 SGT=100
Scalable groups tags (SGTs) Enforcement across multiple fabric sites
Cisco DNA-Center
API
C C C
B B PSN
B B B B PSN
PSN
Host 1 Host 2 Host 3 Host 4 Host 5

SGT=100 SGT=200 SGT=100 SGT=200 SGT=100
Cisco SD-Access Distributed Campus
Forwarding/Packet Walk
Cisco SD-Access
Multi-Site
Forwarding/Packet Walk
Cisco SD-Access Multi-Site Forwarding
Host to Host communication
C
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1 Example: Host1 wants to communicate to Host2
Host to Host communication C
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1 Example: Host1 wants to communicate to Host2
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
1 FE node in fabric site1 sends a map-request to local control plane node for host 2 IP in site 2
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access Host 2
Transit
Fabric Site 1
Host 1
2 Fabric control node in fabric site 1 sends a Negative map-reply(NMR) informing the fabric edge
that it have no information about Host 2 BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
service ipv4
encapsulation vxlan B B
itr map-resolver 192.168.3.5
etr map-server 192.168.3.5 key 7 0207025F0D0357
TC TC
etr map-server 192.168.3.5 proxy-reply
etr
sgt
no map-cache away-eids send-map-request
use-petr 192.168.3.1
Fabric Site 2
proxy-itr 192.168.4.34
exit-service-ipv4
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
3 Traffic is VxLAN encapsulated from the FE in fabric site 1 to the Site-local Fabric border node.
Cisco SD-Access Multi-Site Forwarding C
Host to Host communication B
instance-id 4097
B
remote-rloc-probe on-route-change
service ipv4 TC TC
eid-table default
route-import map-cache bgp 65000 route-map permit-all-eids
itr map-resolver 192.168.3.5 prefix-list Global/San_Jose/SJC15_SJC15_LAN_Fabric_list1
itr map-resolver 192.168.3.131 Fabric Site 2
etr map-server 192.168.3.131 key 7 15175359567F73
etr map-server 192.168.3.131 proxy-reply
exit-service-ipv4
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
4 Fabric Border in Site 1 will now query the Transit CP for Destination Subnet depending on dynamic
list on Fabric Border. BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
5 Fabric Border in Site 1 receive the mapping information from Transit CP node with destination
address as Border in Fabric site 2 BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
6 Traffic is forwarded from fabric border node in fabric site 1 to fabric site 2 using VXLAN encap with
SGT tags encoded. BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Cisco SD-Access Multi-Site Forwarding C
Host to Host communication B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
7 Fabric border node in fabric site 2 after receiving the traffic from fabric site 1 fabric border node
will query its own site control plane node for the destination host© 2020
BRKCRS-2815
based on dynamic list.
Cisco and/or its affiliates. All rights reserved. Cisco Public 60
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
8 Fabric border node in fabric site 2 will receive the mapping information from the local fabric
control plane node with the destination address as an fabric edge© 2020
BRKCRS-2815
node in fabric site 2
Cisco and/or its affiliates. All rights reserved. Cisco Public 61
B B
TC TC
Fabric Site 2
C
B B
Cisco SD-Access
Host 2
Transit
Fabric Site 1
Host 1
8 Traffic is forwarded from fabric border node in fabric site 2 to the fabric edge node in fabric site 2
using VXLAN encap with SGT tags encoded BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Cisco SD-Access
Multi-Site DC/WAN/Shared
Services Forwarding
Cisco SD-Access
Border node selection
• Default UI: Internal Border
• Cannot easily change after provisioning

• If site borders will all have full connectivity to all external networks then usually:
• There is no point importing external prefixes into fabric
• Choose external only border, simplifies fabric routing tables and E-W iBGP configuration
Internal border External border External+Internal border

Unknown
Networks Known Unknown
Known
Networks Networks
Networks
B
B B
Cisco SD-Access across Multiple Fabric sites
DC/WAN/Shared Services Forwarding – Option 1
Data Center
IP Transit

C C C
B B PSN
B B PSN B B
PSN
Fabric Site 1 Fabric Site 2 Fabric Site 3
DC/WAN/Shared Services Forwarding – Option 2
Fabric Site 4
/Data Center
B B
C

C C C
B B PSN
B B PSN B B
PSN B
DC/WAN/Shared Services Access
Edge node in Site 3 sends a map-request to site local Control
Data Center 1
plane node for destination prefix located in Data Center.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Example: Host 2 needs access to DC/Shared Service Host 2

The Site control plane node sends a negative map-reply(NMR)
Data Center 2
for that destination IP as its not registered in the local site
control plane node.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
Cisco SD-Access across Multiple fabric sites
Data Center The negative reply ensures the edge node sends the traffic to
3
the Site Border.. Based on use-petr configuration on the Edge
node
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Data Center The Site Border upon receiving the traffic sends a map-request
4 to the transit control plane node for the destination IP
information
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
Data Center The Border Node in Fabric Site 2 will receive the mapping
5 information from the Transit Control Plane Node with the
destination address as Border Node in Fabric Site 1
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
The Traffic is forwarded from Border Node in Fabric Site 3 to

Data Center 6 the Border Node in Fabric Site 1 using VXLAN encapsulation
with SGT tags and VRF encoded.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
After receiving the traffic from Site 3 Border Node, the Site 1
Data Center 7 Border Node will query its own site-local Control Plane Node for
the destination prefix.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
Data Center Border Node in Fabric Site 1 will receive the mapping information
8 from the local Control Plane Node with the destination address as
an the Border Node in the local site.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
Data Center The Border de-encapsulates, and natively forwards it to the

8
destination in the Data Center.
IP Transit
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
Cisco SD-Access
Multi-Site Internet Access
Forwarding
Internet Access operation Host 2 in Site 3 sends a packet destined for a
1 prefix on the Internet. The packet is forwarded
to its Anycast Gateway which is the Fabric
Internet
Edge in Site 1.
TC
C C C
B B PSN
B B PSN B B
PSN
Example: Host 2 wants to communicate with the Internet

Host 2
Internet Access operation
2 The Site control plane node sends a negative
reply(NMR) back as that destination IP not
Internet
registered in the local site control plane node.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
The negative map-reply ensures that the Edge
3 Node sends the traffic to the Site Border.
Internet This is based on use-petr configuration on the
Edge Node.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
After receiving the traffic from the Edge Node,
Internet 4 the Site Border will query the Transit Control
Plane Node for the destination prefix on the
Internet.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
The Transit Control Plane Node sends a negative
5 map-reply (NMR).
Internet It does not have the Internet destination IP
addressed registered its host tracking database.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
6 The negative replay ensures the site 3 border
Internet node sends the traffic to the Site 2 or Site 1
Border that has connected to internet.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
After receiving the traffic from the from the
7 Border Node in Site 3, the Site Border in Site 2
Internet will query the Transit Control Plane Node for the
destination prefix on the Internet.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
8 The transit control plane node again sends a
Internet negative replay(NMR) with as it does not have
the destination IP registered in its database.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
The Border Node in Site 2 will de-encapsulate the
7 packet and forward it natively.
Internet It does not have any configuration, use-petr,
instructing it to forward to another
device upon receiving an NMR.
TC
C C C
B B PSN
B B PSN B B
PSN
Host 2
SD-Access Transit
Architecture Deep-Dive
SD-Access Transit Architecture Deep-Dive RM(in): deny_0.0.0.0
RM(in): permit-all-eids
C=655370 & 655371 C
Data Center
AS: 65540 BGP RR
RM(in): deny-all EB
RM(out):tag_transit_eids TC
Community 655371
Import DC Routes from
IGP into BGP & LISP DB Routes advertised to
RM(out): tag_local_eids
TC via eBGP Fabric Site 2
EB registers local site
Community 655370 prefixes with AS: 65001
C=655370 to TC
RM(in): deny-all-eids
IB
EB
C
BGP RR Cisco SD-Access
Fabric Site 1 Transit
AS: 65000
BGP Configs
RM(in): tag_local_eids
Community 655370 LISP Configs
Use Cases
Cisco SD-Access Transit Use Case 1
Data Center Site Public Cloud

C
B B C/B C/B
C C
C C B/C/E
B B PSN
B B
PSN
Cisco SD-Access Transit Use Case 2
Dual Stack Migration
• Migrated to Dual Stack

• Border/Control Plane node not
able handle the current v6 Scale.
DEMO
Summary
Summary
• Cisco SD-Access Multi-Site fabric
Automated Inter-Site Connectivity Automated SDA—SDWAN Connectivity
E2E Segmentation, Policy & Assurance Flexible Group to VPN Mapping
Flexible & Scalable Flexible WAN options
SD-Access Support
For more details: cs.co/sda-compatibility-matrix
Digital Platforms for your Cisco Digital Network Architecture
BETA
Switching Routing Wireless Extended
Catalyst 9600 Catalyst 9400 ASR-1000-HX Catalyst 9800 Cisco Digital Building
NEW
ASR-1000-X
NEW
Catalyst 9500 Catalyst 9300

Catalyst 9100 APs
Catalyst 9200 AIR-CT8540

ISR 4451 Catalyst 3560-CX
NEW
ISR 4430
AIR-CT3504
AIR-CT5520
ISR 4330 Cisco IE 4K/5K
NEW
Catalyst 6800 NEW

Catalyst 4500E Nexus 7700
Aironet Aironet
Catalyst 3850 & 3650 ENCS 5400 Wave 1 APs* Wave 2 APs Cisco IE 3400
Complete your
online session
survey • Please complete your session survey
after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events
Mobile App or by logging in to the Content
Catalog on ciscolive.com/emea.
Cisco Live sessions will be available for viewing on

demand after the event at ciscolive.com.
Continue your education
Demos in the
Walk-In Labs
Cisco Showcase
Meet the Engineer

Related sessions
1:1 meetings
Thank you
Cisco DNA Center
Policy Automation Assurance
SD-ACCESS
TRANSIT
B B
B B C
C
SD-Access
SD-Access Extension Client Mobility
Extension Client Mobility
Policy follows User

Policy follows User
IoT Network Employee Network

IoT Network Employee Network BRKCRS-2815 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 101

BRKCRS-2815 (2020) - Connecting Multiple Fabrics Together

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKCRS-2815 (2020) - Connecting Multiple Fabrics Together

Uploaded by

Copyright:

Available Formats

Cisco SD-Access –

Connecting Multiple Sites in

Cisco SD-Access Transit Types

Cisco SD-Access Policy across Distributed Campus

Cisco SD-Access Distributed Campus Uses cases and Demo

Keynote BRKCRS-2815 BRKCRS-2818 BRKCRS-2819

Recipe for transforming 14:30 BRKCRS-2824

Cisco DNA Center

B B ▪ Control-Plane Nodes – Map System that

Campus ▪ Fabric Edge Nodes – A fabric device

E E E E ▪ Fabric Wireless Controller – A fabric device

B/C B/C B/C

B/C Border/Control Node

Cisco SD-Access Transit

B/C B/C B/C

Maintain End-to-End Segmentation SD-Access Scale requirements

Fabric Domain Fabric Site

Parameters DN2-HW-APL DN2-HW-APL-L DN2-HW-APL-XL

No of Endpoints 25,000 40,000 100,000

No of Fabric Sites 500 1000 2000

No of Virtual Networks per 64/Site 64/site 256/site

Transit/Peer Network type include

• IP-Based Transit - Leverages a traditional IP-based (VRF-LITE, MPLS) network, which

• Sites in different regions - Higher latencies

Typical use cases

o Sites are in same Metropolitan area

o Higher MTU support

• Sites in different regions - Higher latencies

LTE Typical use cases

1. Control-Plane: How routes / prefixes are communicated

• Manual & Every hop needs to support SGT propagation

SD-Access Transit SD-Access

LISP OMP LISP CONTROL-PLANE

➢ High-bandwidth connection (Ethernet

Cisco SD-Access Fabric Site 1 Cisco SD-Access Fabric Site 2

C Register west Register east

• Each site only maintains state for in-site end-points.

➢ Device must be dedicated to the transit control plane node role.

Cisco SD-Access Cisco SD-Access IP-Based Cisco SW-WAN

C9K YES YES NO

ASR1K/ISR4K YES YES YES

• Requires direct Internet

Internet Transit Internet • Requires local resiliency

• 2-4 Site Borders

Macro Segmentation Micro Segmentation

Building Management Campus Users Building Management Campus Users

Virtual Network (VN) Scalable Group (SG)

Fabric Site 1 Fabric Site 2 Fabric Site N

Lab and Small HA Deployment Small Multi-node Deployment Large Deployment

35xx 100 Endpoints 20,000 Endpoints 500,000 Endpoints

ISE Performance & Scale

Cisco SD-Access Transit

Fabric Site 1 Fabric Site 2 Fabric Site N

• PSN Nodes dedicated to every site

Cisco SD-Access Transit

Fabric Site 1 Fabric Site 2 Fabric Site N

• PSN’s are behind a dedicated Load Balancer

Fabric Site 1 Fabric Site 2 Fabric Site N

Fabric Site 1 Fabric Site 2 Fabric Site N

Host 1 Host 2 Host 3 Host 4 Host 5

Fabric Site 1 Fabric Site 2 Fabric Site N

Host 1 Host 2 Host 3 Host 4 Host 5

Host 1 Example: Host1 wants to communicate to Host2

Host 1 Example: Host1 wants to communicate to Host2