Campus Multilayer Arch Design Guide

CAMPUS NETWORK MULTILAYER
ARCHITECTURE AND DESIGN

GUIDELINES
SESSION RST-2032
RST-2032
11208_05_2005_c2 © 2005 Cisco Systems, Inc. All rights reserved. 1
High-Availability Networking in the Campus
Real World Network Design:

Hierarchical Network Design—
Structured Modular Foundation
Reinforced Network Infrastructure:

Infrastructure Security Hardening
Device-Level and Software Resiliency
Network Operations:
Best Practices
Real-Time Network Management:

Best Practices
Best-in-Class Support:
TAC, CA, Etc.
RST-2032
What Is High Availability?
Availability DPM Downtime Per Year (24x365)
99.000% 10000 3 Days 15 Hours 36 Minutes
99.500% 5000 1 Day 19 Hours 48 Minutes
99.900% 1000 8 Hours 46 Minutes
99.950% 500 4 Hours 23 Minutes
99.990% 100 53 Minutes
99.999% 10 5 Minutes “High

Availability”
99.9999% 1 30 Seconds
DPM—Defects per Million

RST-2032
What If You Could…
Reduce Cost Through Diminished Risk of Downtime
• Costs for downtime are high Revenue/

One day cost of lost productivity = Industry Sector Revenue/Hour Employee-
$1,644 per employee Hour
100 person office = Energy $2,817,846 $ 569
$164K per day
Telecommunications $2,066,245 $ 186
• More than just a data
network outage Manufacturing $1,610,654 $ 134
• More than just revenue Financial Institution $1,495,134 $1,079
impacted
Revenue loss Insurance $1,202,444 $ 370
Productivity loss Retail $1,107,274 $ 244
Impaired financial performance
Transportation $ 668,586 $ 107
Damaged reputation
Recovery expenses Average $1,010,536 $ 205
Source: Meta Group
RST-2032
Agenda
• Multilayer Campus Design

Principals
• Foundation Services
• Campus Design Best
Practices
• Security Considerations
• Putting It All Together
• Summary
RST-2032
Multilayer Network Design
Without a Rock Solid Foundation the Rest Doesn’t Matter
Access
• Offers hierarchy – each layer has specific
role
• Modular topology - building blocks
Si
Distribution • Easy to grow, understand, and
Si
troubleshoot
• Creates small fault domains – Clear
demarcations and isolation
Core
• Promotes load balancing and redundancy Si
Si
• Promotes deterministic traffic patterns

• Incorporates balance of both Layer 2 and
Distribution Layer 3 technology, leveraging the
Si Si
strength of both
• Utilizes Layer 3 Routing for load
balancing, fast convergence, scalability,
Access and control
Data Center
RST-2032
Defining the Access Layer
To Core
Distribution
Si Si
Access
• Aggregates network end-points • Catalyst® integrated security

• Layer 2/Layer 3 feature rich features IBNS (802.1x), (CISF): port
environment; convergence, HA, security, DHCP snooping, DAI,
security, QoS, IP multicast, etc IPSG, etc.
• Intelligent network services: QoS, • Automatic phone discovery,
trust boundary, broadcast conditional trust boundary, power
suppression, IGMP snooping over Ethernet, auxiliary VLAN, etc.
• Intelligent network services: • Spanning tree toolkit: Portfast,
PVST+, Rapid PVST+, EIGRP, UplinkFast, BackboneFast,
OSPF, DTP, PAgP/LACP, UDLD, etc. LoopGuard, BPDUGuard,
RST-2032 BPDUFilter, RootGuard, etc.
Defining the Distribution Layer
Si Si Si Si Distribution
Access
• Availability, load balancing, QoS and • Spanning tree features:

provisioning are the important Only if you need them:
considerations at this layer setting STP Root, Root Guard
Rapid PVST+—Per VLAN 802.1w
• Aggregates wiring closets (access
layer) and uplinks to core • Route summarization, fast
• Use Layer 3 switching in the convergence, redundant path
distribution layer load sharing
• Protects core from high density • HSRP or GLBP to provide first hop
peering and problems in access layer redundancy
RST-2032
Defining the Core Layer
Core
Distribution
Access
• Backbone for the network—connects network building blocks

• Performance and stability vs. complexity—less is more in the core
• Aggregation point for distribution layer
• Separate core layer helps in scalability during future growth
• Keep the design technology-independent
RST-2032
Do I Need a Core Layer?
For Optimum Convergence—Yes
Distribution 2 Distribution 2
Distribution 3
Distribution 1
Distribution 1
Distribution 3
No Core Dedicated Core Switches

• Fully meshed • Easier to add a module
distribution layers • Fewer links in the core
• Aggregation point for distribution • Easier bandwidth upgrade
layer
• Routing protocol peering reduced
• Core layer is required to scale
campus networks • Equal cost Layer 3 links for best
Physical cabling requirements
convergence
Routing complexity
RST-2032
Optimal Redundancy
When Is More Less?
• Core and
distribution Access
engineered with
redundant nodes
and links to Distribution Si Si Si Si Si Si
provide maximum
redundancy and
optimal Redundant
convergence Core Nodes
Si Si
• Network bandwidth
and capacity
engineered to Distribution Si Si
Si
Si
Si
Si
withstand node
or link failure
• 120–200ms to Access
converge around WAN Data Center Internet
most events
RST-2032
Single Points of Termination
SSO/NSF Avoiding Total Network Outage
L2 = SSO
Access L3 = SSO/NSF
Distribution Si Si Si Si Si Si
Core
Si Si
• The access layer is candidate for supervisor redundancy

• L2 access layer SSO
• L3 access layer SSO and NSF
• Network outage until physical replacement or reload vs
one to three seconds
RST-2032
Agenda

Principals
Practices
• Summary
RST-2032
ESE Campus Solution Test Bed
Verified Design Recommendations
Total of 68 Access Switches,

2950, 2970, 3550, 3560, 3750,
4507 SupII+, 4507SupIV, 6500
Sup2, 6500 Sup32, 6500 Sup720
and 40 APs (1200)
Three Distribution Blocks Si Si

Si Si Si Si
6500 with Redundant Sup720
4507 with Redundant SupV
6500 with Redundant Sup720s

Si Si
Three Distribution Blocks

6500 with Redundant Sup720s Si Si
Si
Si Si
Si
7206VXR NPEG1
4500 SupII+, 6500 Sup720,

FWSM, WLSM, IDSM2, MWAM
WAN Data Center Internet
RST-2032
Foundation Services
• Layer 3 routing protocols

• Layer 2 redundancy—spanning tree Load
PVST+ - STP (802.1D-1998)
Balancing
Rapid PVST+ - RSTP (802.D-2004)
• Unidirectional link detection ing

Trunk
• Trunking protocols—(isl/.1q)
• Load balancing GLBP
Etherchannel link aggregation
CEF equal cost load balancing
HSRP
• First hop redundancy protocols
VRRP, HSRP, and GLBP
• Quality of service Spanning

Routing Tree
RST-2032
Best Practices—Layer 3 Routing Protocols
• Typically deployed in
distribution to core, and core
to core interconnections
• Used to quickly re-route
around failed node/links while Si Si Si Si
Si Si
providing load balancing over
redundant paths
• Build triangles not squares for
deterministic convergence
Layer 3 Equal Layer 3 Equal
• Only peer on links that you Cost Link’s Cost Link’s
intend to use as transit Si Si
• Insure redundant L3 paths to

avoid black holes
• Summarize distribution to core Si Si Si Si
to limit EIGRP query diameter Si Si
or OSPF LSA propagation

• Tune CEF L3/L4 load balancing
hash to achieve maximum
utilization of equal cost paths WAN Data Center Internet
(CEF polarization)
RST-2032
Best Practice—Build Triangles Not Squares
Deterministic vs. Non-Deterministic
Triangles: Link/Box Failure Does NOT Squares: Link/Box Failure Requires
Require Routing Protocol Convergence Routing Protocol Convergence
Si Si Si
Si
Si Si Si
Si
Model A Model B
• Layer 3 redundant equal cost links support fast convergence

• Hardware based—fast recovery to remaining path
• Convergence is extremely fast (dual equal-cost paths: no need for
OSPF or EIGRP to recalculate a new path)
RST-2032
Best Practice—Passive Interfaces for IGP
Limit OSPF and EIGRP Peering Through the Access Layer
Limit unnecessary peering Distribution Si Si
Without passive interface: Routing

Updates
• Four VLANs per wiring closet,
• 12 adjacencies total
• Memory and CPU requirements
increase with no real benefit
Access
• Creates overhead for IGP
OSPF Example: EIGRP Example:
Router(config)#router ospf 1 Router(config)#router eigrp 1

Router(config-router)#passive- Router(config-router)#passive-
interface Vlan 99 interface Vlan 99
Router(config)#router ospf 1 Router(config)#router eigrp 1

Router(config-router)#passive- Router(config-router)#passive-
interface default interface default
Router(config-router)#no passive- Router(config-router)#no passive-
interface Vlan 99 interface Vlan 99
RST-2032
Provide Alternate Paths
• What happens if fails?

• No route to the core Si Si Core
anymore?
Single Path
• Allow the traffic to go to Core
through the access? T
witraffi
Do you want to use your access hN c D
o rop
switches as transit nodes? CoRout ped
re e to
How do you design for Distribution
scalability if the access used Si Si
for transit traffic?
• Install a redundant link to
the core
• Best practice: install
redundant link to core and Access
utilize L3 link between
distribution Layer A B
(summarization—coming)
RST-2032
Why You Want to Summarize at the Distribution
Limit EIGRP Queries and OSPF LSA Propagation
No Summaries
Queries Go Beyond the Core
• It is important to force
summarization at the Rest of Network
distribution towards the core Core
• For return path traffic an
OSPF or EIGRP re-route is
required
Si Si
• By limiting the number of
peers an EIGRP router must
query or the number of LSA’s
Distribution
an OSPF peer must process T
Droraffi
we can optimize this re-route U p c
Co ntil I ped
nve GP
• EIGRP example: rge
s
interface Port-channel1 Si Si
description to Core#1
ip address 10.122.0.34
255.255.255.252 Access
ip hello-interval eigrp 100 1
ip hold-time eigrp 100 3
ip summary-address eigrp 100
10.1.0.0 255.255.0.0 5
RST-2032 10.1.1.b/24 10.1.1.a/24

Why You Want to Summarize at the Distribution
Reduce the Complexity of IGP Convergence
Summaries
Stop Queries at the Core
• It is important to force
summarization at the Rest of Network
distribution towards the core Core
• For return path traffic an OSPF
or EIGRP re-route is required
• By limiting the number of peers Si Si
an EIGRP router must query or
the number of LSA’s an OSPF
|peer must process we can Summary:
optimize his re-route Distribution
10.1.0.0/16 T
Droraffi
• For EIGRP if we summarize at U ppec
Co ntil IG
the distribution we stop queries nve Pd
rge
at the core boxes for an access s
layer ‘flap’ Si Si
• For OSPF when we summarize

at the distribution (area border Access
or L1/L2 border) the flooding
of LSA’s is limited to the
distribution switches; SPF now
deals with one LSA not three
RST-2032 10.1.1.b/24 10.1.1.a/24
Best Practice—Summarize at the Distribution
Gotcha—Distribution to Distribution Link Required
• Best practice—summarize
at the distribution layer to
limit EIGRP queries or Core
OSPF LSA propagation Si Si
• Gotcha:
Summary:
Upstream: HSRP on left
distribution takes over when 10.1.0.0/16
link fails
Return path: old router still Distribution
advertises summary to core
Tra
Return traffic is dropped on Si w
Si f
ith fic D
right distribution switch No rop
Ro ped
ute
• Summarizing requires
a link between the
distribution switches Access
• Alternative design:
Use the access layer for
transit 10.1.1.b/24 10.1.1.a/24
RST-2032
CEF Load Balancing
Avoid Underutilizing Redundant Layer 3 Paths
Redundant Paths Ignored
• CEF polarization: without
some tuning CEF will
select the same path
Distribution left/left or right/right
Si Si
Default L3 Hash
• Imbalance/overload
L R
could occur
Core
Si
• Redundant paths are
Default L3 Hash Si
ignored/underutilized
L • The default CEF hash

Distribution
R ‘input’ is L3
Default L3 Hash Si Si
• We can change the default
to use L3 + L4 information
as ‘input’ to the hash
derivation
RST-2032
CEF Load Balancing
Change the Input—Get Different Output
• CEF uses a multi-step process to Src IP| Dst IP | Src Port | Dst Port
make final forwarding decision
• First it determines the longest path
match for the destination address via
an hardware lookup
• Each specific index is associated with Hardware
a next hop adjacencies table Lookup
• Default: using the packet source and
destination IP address one of the
possible adjacencies is selected via a
HW hash
Select
• Tweak: using the packet source and Specific
destination IP address and port Hash
information one of the possible Adjacency
adjacencies is selected via a HW hash Based on
Hash
• New MAC address is attached and
packet is forwarded
• If you change the input to the hash
you will change the output
• Changing the default from L3 to L3/L4
causes different hashes to be derived MAC Re-Write Src IP| Dst IP | Src Port | Dst Port
RST-2032
CEF Load Balancing
All Paths Used
• Without some tuning CEF will
select the same path left/left or
right/right and imbalance/
overload could occur
Distribution
Si
(redundant paths ignored)
L3/L4 Hash Si
• Alternating L3/L4 hash and

default L3 hash will give us the
L R L R best load balancing results
Core
Default L3 Hash Si Si • The default is L3 hash—no
modification required in core
L • In the distribution switches use:
Distribution
L3/L4 Hash
R mls ip cef load-sharing full
Si Si
to achieve better distribution

and avoid underutilization of
redundant L3 paths
RST-2032
Spanning Tree
• Highly available environments require redundant paths to ensure connectivity

in the event of a node or link failure
• Spanning tree ensures a loop-free topology and provides backup links when
there are redundant paths in the network
802.1D-1998: Classic Spanning Tree 802.1s: Multiple Spanning Tree

Protocol (STP) Protocol (MST)
802.1D-2004: Rapid Spanning Tree 802.1t: 802.1d Maintenance
Protocol (RSTP—Was 802.1w) 802.1Q: VLAN Tagging (Trunking)
RST-2032
PVST+ and Rapid PVST+, MST
Spanning Tree Toolkit, 802.1d, 802.1s, 802.1w
• PVST+: an instance of STP (802.1d) per VLAN +

Portfast, Uplinkfast, BackboneFast, BPDUGuard,
BPDUFilter, RootGuard, and LoopGuard
• Rapid PVST+: an instance of RSTP (802.1w) per
VLAN + Portfast, BPDUGuard, BPDUFilter,
RootGuard, and LoopGuard
• MST (802.1s): up to 16 instances of RSTP (802.1w);
combining many VLANS with the same physical
and logical topology into a common RSTP instance;
additionally Portfast, BPDUGuard, BPDUFilter,
RootGuard, and LoopGuard are supported
with MST
RST-2032
Spanning Tree Toolkit
Root Distribution
• PortFast*: Bypass listening-learning Switches
phase for access port
• UplinkFast: Three to five seconds F F
Si Si
convergence after link failure
• BackboneFast: Cuts convergence time F F
by Max_Age for indirect failure
• LoopGuard*: Prevents alternate or root
port to become designated in absence
of BPDUs
• RootGuard*: Prevents external switches X
Wiring
from becoming root
F B Closet
• BPDUGuard*: Disable PortFast enabled Switch
port if a BPDU is received
• BPDUFilter*: Do not send or receive
BPDUs on PortFast enabled ports
* Also Supported with MST and Rapid PVST+

RST-2032
Best Practices—Spanning Configuration
Same VLAN Same VLAN Same VLAN
• ONLY when you have to!
• More common in the Layer2 Loops
data center
• Required when a VLAN spans Si Si Si Si Si Si
access layer switches

• Required to protect against
‘user side’ loops
• Use Rapid PVST+ for best Cost Link’s Cost Link’s
convergence Si Si
• Take advantage of the

Spanning Tree Toolkit
Si Si
RootGuard to keep rogue switch Si Si
from becoming root Si Si
BPDUGuard to keep rogue switch

from participating in STP topology
LoopGuard to partition the network
in the event of an STP failure (loss WAN Data Center Internet
of BPDU’s)
RST-2032
Optimizing Convergence: PVST+ or Rapid PVST+
802.1d + Extensions or 802.1s + Extensions
• Rapid-PVST+ greatly improves the restoration times for any

VLAN that requires a topology convergence due to link UP
• Rapid-PVST+ also greatly improves convergence time over
Backbone fast for any indirect link failures
35
To Access To Server Farm
Timed to Converge in
30
25
Seconds
20 30 Seconds of
Delay/Loss
15 Tuned Away
10
5
0
PVST+ Rapid PVST+
RST-2032
Best Practices—UDLD Configuration
• Typically deployed on any

fiber optic interconnection
• Use UDLD aggressive
mode for best protection Si Si Si Si Si Si
• Turn on in global Fiber Interconnections

configuration to avoid
Layer 3 Equal
operational Layer 3 Equal
Cost Link’s
Cost Link’s
error/“misses” Si Si
• Config example
Cisco IOS: udld Si Si Si Si
aggressive Si Si
CatOS: set udld enable

set udld aggressive-
mode enable <mod/port>
RST-2032
Unidirectional Link Detection
Protecting Against One Way Communication
• Highly available networks require UDLD to

protect against one way communication or
partially failed links and the effect that they
could have on protocols like STP and RSTP Si
• Primarily used on fiber optic links where
patch panel errors could cause link up/up
with miss matched transmit/receive pairs
• Each switch port configured for UDLD will
send UDLD protocol packets (at L2) Are You
containing the port's own device/port ID, ‘Echoing’
and the neighbor's device/port IDs seen by My Hellos?
UDLD on that port
• Neighboring ports should see their own
device/port ID (echo) in the packets
received from the other side
• If the port does not see its own device/port
ID in the incoming UDLD packets for a
specific duration of time, the link is Si
considered unidirectional and is shutdown
RST-2032
Best Practices—Trunk Configuration
• Typically deployed on
interconnection between
access and distribution layers 802.1q Trunks
• Use VTP transparent mode to
decrease potential for Si Si Si Si Si Si
operational error
• Hard set trunk mode to on and
encapsulation negotiate off for Layer 3 Equal Layer 3 Equal
optimal convergence Cost Link’s Cost Link’s
Si Si
• Change the native VLAN to

something unused to avoid
VLAN hopping
Si Si Si Si
• Manually prune all VLANS Si Si
except those needed

• Disable on host ports:
CatOS: set port host Internet
WAN Data Center
Cisco IOS: switchport host
RST-2032
VTP Virtual Trunk Protocol
• Centralized VLAN Set

Pass
management VLAN 50
Through
Trunk Update
• VTP server switch F
propagates VLAN A Server Transparent
database to VTP
client switches Ok, I Just
Trunk Trunk Learnt
• Runs only on trunks VLAN 50!
Ok, I Just
• Four modes: Learnt
Server: updates clients VLAN 50!
and servers Client Client B
Client: receive updates— Trunk
cannot make changes Drop
Transparent: let updates VTP
pass through Updates
Off: ignores VTP updates
Off
C
RST-2032
DTP Dynamic Trunk Protocol
• Automatic formation of trunked Si On/On Si

switch to switch interconnection
Trunk
On: always be a trunk
Desirable: ask if the other side can/will
Auto: if the other sides asks I will Si
Auto/Desirable
Si
Off: don’t become a trunk Trunk

• Negotiation of 802.1Q or ISL
encapsulation
Si Si
ISL: try to use ISL trunk encapsulation Off/Off
NO Trunk
802.1q: try to use 802.1q encapsulation
Negotiate: negotiate ISL or 802.1q
encapsulation with peer Si Si
Non-negotiate: always use

encapsulation that is hardset Off/On, Auto, Desirable
NO Trunk
RST-2032
Optimizing Convergence: Trunk Tuning
Trunk Auto/Desirable Takes Some Time
• DTP negotiation tuning improves link up convergence time

CatOS> (enable) set trunk <port> nonegotiate dot1q <vlan>
IOS(config-if)# switchport mode trunk
IOS(config-if)# switchport nonegotiate
2.5
3550 (Cisco IOS)
Time to Converge in Seconds
4006 (CatOS)
2 4507 (Cisco IOS)
6500 (CatOS)
1.5 Si
Two Seconds
1 of Delay/Loss
Tuned Away
0.5
Voice Data
0
Trunking Desirable Trunking Nonegotiate
RST-2032
Best Practices—
EtherChannel Configuration
• Typically deployed in
distribution to core, and core
to core interconnections
• Used to provide link Si Si Si Si
Si Si
redundancy—while reducing
peering complexity
• Tune L3/L4 load balancing
hash to achieve maximum Cost Link’s Cost Link’s
utilization of channel members Si Si
• Match CatOS and Cisco IOS

PAgP settings
Si Si Si Si
• 802.3ad LACP for interop if you Si Si
need it
• Disable unless needed
CatOS: set port host WAN Data Center Internet
RST-2032 Cisco IOS: switchport host

EtherChannel Load Balancing
L3 Hash
• Network did not load Link 0 load—68%
balance using default L3
load balancing hash
Common IP addressing scheme Si Si
72 access subnets addressed Link 1 load—32%

uniformly from 10.120.x.10 to
10.120.x.215
L4 Hash
• Converted to L4 load Link 0 load—52%
balancing hash and
achieved better load sharing
Si Si
Link 1 Load—48%
cr2-6500-1(config)#port-channel load-balance src-dst-port
RST-2032
PAgP Port Aggregation Protocol
• Automatic formation of Si Si
On/On
bundled redundant switch Channel
to switch interconnection
On: always be a
Si Si
channel/bundle member On/Off
No Channel
Desirable: ask if the other
side can/will
Auto: if the other sides Si
Auto/Desirable
Si
asks I will Channel

Off: don’t become a member
of a channel/bundle Si
Si
Off/On, Auto, Desirable

NO Channel
RST-2032
PAgP Tuning
PAgP Default Mismatches
Matching EtherChannel Configuration on Both Sides

Improves Link Restoration Convergence Times
set port channel <mod/port> off
7
6
Time to Converge in
5 As Much As
Seconds
4 Seven Seconds 6500 (CatOS)

of Delay/Loss 4506 (CatOS)
3
Tuned Away
2
1
0
PAgP Mismatch PAgP Off
RST-2032
Best Practices—First Hop Redundancy
• Used to provide a resilient

default gateway/first hop
address to end stations 1st Hop Redundancy
• HSRP, VRRP, and GLBP
Si
alternatives Si Si Si Si Si
• VRRP, HSRP and GLBP

provide millisecond timers
and excellent convergence Layer 3 Equal Layer 3 Equal
Cost Link’s
performance Cost Link’s
Si
Si
• VRRP if you need multi-

vendor interoperability
• GLBP facilitates uplink load Si Si Si Si
Si Si
balancing
• Preempt timers need to
be tuned to avoid black-
holed traffic WAN Data Center Internet
RST-2032
First Hop Redundancy with VRRP
IETF Standard RFC 2338 (April 1998)
R1—Master, Forwarding Traffic; R2,—Backup
• A group of routers VRRP ACTIVE VRRP BACKUP
function as one virtual IP: 10.0.0.254 IP: 10.0.0.253
router by sharing ONE MAC: 0000.0c12.3456 MAC: 0000.0C78.9abc
vIP: 10.0.0.10 vIP:
virtual IP address and vMAC: 0000.5e00.0101 vMAC:
one virtual MAC address
• One (master) router R1 R2
performs packet
Si Si
forwarding for local hosts
• The rest of the routers act Distribution-A Distribution-B
as “back up” in case the VRRP Active VRRP Backup
master router fails Access-a
• Backup routers stay idle

as far as packet
forwarding from the client
side is concerned
IP: 10.0.0.1 IP: 10.0.0.2 IP: 10.0.0.3
MAC: aaaa.aaaa.aa01 MAC: aaaa.aaaa.aa02 MAC: aaaa.aaaa.aa03
GW: 10.0.0.10 GW: 10.0.0.10 GW: 10.0.0.10
ARP: 0000.5e00.0101 ARP: 0000.5e00.0101 ARP: 0000.5e00.0101
RST-2032
First Hop Redundancy with HSRP
Cisco Informational RFC 2281 (March 1998)
R1—Active, Forwarding Traffic; R2—Hot Standby, Idle
• A group of routers VRRP ACTIVE VRRP STANDBY
function as one virtual IP: 10.0.0.254 IP: 10.0.0.253
router by sharing ONE MAC: 0000.0c12.3456 MAC: 0000.0C78.9abc
vIP: 10.0.0.10 vIP:
virtual IP address and vMAC: 0000.0c07.ac00 vMAC:
one virtual MAC address
• One (active) router R1 R2
performs packet
Si Si
forwarding for local hosts
• The rest of the routers Distribution-A Distribution-B
provide “hot standby” VRRP Active VRRP Backup
in case the active Access-a
router fails
• Standby routers stay
idle as far as packet
forwarding from the
client side is concerned
IP: 10.0.0.1 IP: 10.0.0.2 IP: 10.0.0.3
MAC: aaaa.aaaa.aa01 MAC: aaaa.aaaa.aa02 MAC: aaaa.aaaa.aa03
GW: 10.0.0.10 GW: 10.0.0.10 GW: 10.0.0.10
ARP: 0000.0c07.ac00 ARP: 0000.0c07.ac00 ARP: 0000.0c07.ac00
RST-2032
Optimizing Convergence: HSRP Timers
HSRP Millisecond Convergence
• HSRP = default gateway

redundancy; effects traffic
Layer 2 Link’s
out of the access layer
interface Vlan5
Si Si Si Si Si Si description Data VLAN for 6k-access
ip address 10.1.5.3 255.255.255.0
ip helper-address 10.5.10.20
no ip redirects
ip pim query-interval 250 msec
ip pim sparse-mode
Layer 3 Equal Layer 3 Equal logging event link-status
Cost Link’s Cost Link’s standby 1 ip 10.1.5.1
Si Si standby 1 timers msec 200 msec 750
standby 1 priority 150
standby 1 preempt
standby 1 preempt delay minimum 180
Si Si Si Si
Si Si
RST-2032
Optimizing Convergence: HSRP Preempt Delay
Preempt Delay Needs to Be Longer Than Box Boot Time
Without Increased Preempt Delay HSRP Can Go

Active Before Box Completely Ready to Forward
Traffic L1 (Boards), L2 (STP), L3 (IGP Convergence)
standby 1 preempt delay minimum 180
Test Tool Timeout—30 Seconds

30
Time to Converge in Seconds
25
20 More Than 3550 (Cisco IOS)

30 Seconds of 2950 (Cisco IOS)
15 Delay/Loss 4506 (CatOS)
4507 (Cisco IOS)
Tuned Away
6500 (CatOS)
10
6500 (Cisco IOS)
0
No Preempt Delay Prempt Delay Tuned
RST-2032
First Hop Redundancy with GLBP
Cisco Designed, Load Sharing, Patent Pending
R1- AVG; R1, R2 Both Forward Traffic
• All the benefits of GLBP AVG/AVF,SVF GLBP AVF,SVF
HSRP plus load IP: 10.0.0.254 IP: 10.0.0.253
balancing of default MAC: 0000.0c12.3456 MAC: 0000.0C78.9abc
gateway à utilizes all vIP: 10.0.0.10 vIP: 10.0.0.10
vMAC: 0007.b400.0102
vMAC: 0007.b400.0101
available bandwidth
• A group of routers R1
function as one virtual Si Si
router by sharing one
virtual IP address but Distribution-A Distribution-B
using multiple virtual GLBP AVG/AVF, SVF GLPB AVF,SVF
MAC addresses for Access-a
traffic forwarding
• Allows traffic from a
single common subnet
to go through multiple
redundant gateways IP: 10.0.0.1 IP: 10.0.0.2 IP: 10.0.0.3
using a single virtual MAC: aaaa.aaaa.aa01 MAC: aaaa.aaaa.aa02 MAC: aaaa.aaaa.aa03
IP address GW: 10.0.0.10 GW: 10.0.0.10 GW: 10.0.0.10
ARP: 0007.B400.0101 ARP: 0007.B400.0102 ARP: 0007.B400.0101
RST-2032
First Hop Redundancy with Load Balancing
Cisco Gateway Load Balancing Protocol (GLBP)
• Each member of a GLBP redundancy group owns a unique virtual MAC

address for a common IP address/default gateway
• When end stations ARP for the common IP address/default gateway they are
given a load balanced virtual MAC address
• Host A and host B send traffic to different GLBP peers but have the same
default gateway
GLBP 1 ip 10.88.1.10 vIP GLBP 1 ip 10.88.1.10

vMAC 0000.0000.0001 10.88.1.10 vMAC 0000.0000.0002
R1 R2
.1 ARP .2
Reply
10.88.1.0/24
.4 .5
A B ARPs for 10.88.1.10

ARPs for 10.88.1.10
Gets MAC 0000.0000.0001 Gets MAC 0000.0000.0002
RST-2032
Optimizing Convergence: VRRP, HSRP,
GLBP Mean, Max, and Min—Are There Differences?
• VRRP does not have sub-second timers and all flows go through a
common VRRP peer; mean, max, and min are equal
Si Si
• HSRP has sub-second timers; however all flows go through same

HSRP peer so there is no difference between mean, max, and min
• GLBP has sub-second timers and distributes the load amongst
the GLBP peers; so 50% of the clients are not effected by an
uplink failure
Distribution to Access Link Failure
Access to Server Farm
Time in Seconds to Converge
1.2 VRRP HSRP GLBP

50% of Flows
1 GLBP Is 50%
Have ZERO
Better
0.8 Loss W/ GLBP
0.6
0.4
0.2
0
Longest Shortest Average
RST-2032
If You Span VLANS Tuning Required
By Default Half the Traffic Will Take a 2 Hop L2 Path
• Both distribution switches act as default gateway

• Blocked uplink caused traffic to take less than optimal path
Core Core
Layer 3 Distribution-A Distribution-B
GLBP Virtual GLBP Virtual MAC 2
Distribution MAC 1
Layer 2/3
Si Si
2
F2 B
F 2
Access F: Forwarding
B2
Layer 2 B: Blocking
Access-a Access-b
VLAN 2 VLAN 3
RST-2032
GLBP + STP turning
Change Port Cost to Change the Blocking Interfaces
Force STP to Block the Interface Between

the Distribution Switches
Core Core
Layer 3 Distribution-A Distribution-B
GLBP Virtual GLBP Virtual MAC 2
Distribution MAC 1
Layer 2/3 B x STP Port
Cost
Si Si Increased
Access
Layer 2
Access-a Access-b
VLAN 2 VLAN 2
RST-2032
Best Practices—Quality of Service
• Must be deployed end-to-

end to be effective; all
layers play different but End to End QoS
equal roles
Si Si Si Si
Si Si
• Ensure that mission critical
applications are not
impacted by link or transmit
queue congestion Layer 3 Equal Layer 3 Equal
Cost Link’s Cost Link’s
• Aggregation and rate Si Si
transition points must

enforce QoS policies
Si Si
• Multiple queues with Si
Si Si
Si
configurable admission
criteria and scheduling
are required
RST-2032
Transmit Queue Congestion
10/100m Queued 128k Uplink
WAN
Router
100 Meg in 128 Kb/S out—Packets Serialize in Faster Than They Serialize out
Packets Queued as They Wait to Serialize out Slower Link
1 Gig Link Queued 100 Meg Link
Distribution Switch Access Switch
1 Gig In 100 Meg out—Packets Serialize in Faster Than They Serialize out
Packets Queued as They Wait to Serialize out Slower Link
RST-2032
Enabling QoS in the Campus
Scheduling in the Campus
Police and Mark Throttle
Anomalies
• Scavenger traffic is
assigned it’s own
queue/threshold Si
• Scavenger queue is
shallow with a large
burst to penalize
Scavenger
sustained loads Gold Queue
• Multiple queues are the RX Aggressive Drop
only way to “guarantee” Data
voice quality, protect RX
mission critical and
throttle abnormal
sources
• Cisco switches have
Scavenger
RX Si TX
multiple queues
Voice
RX Voice Put into
Delay/Drop
RST-2501: Campus QoS Design Sensitive Queue
RST-2032
Agenda

Principals
Practices
• Summary
RST-2032
Campus Design Best Practices
• Daisy chaining dangers Access

• A note on asymmetric
routing and unicast Si Si Distribution
flooding
• Where’s your Layer 2/3
boundary? Core
Si
Si
• What L2/3 protocols do

run where and why? Distribution
Si Si
Access
RST-2032
Daisy Chaining Access Layer Switches
Avoid Potential Black Holes
Return Path Traffic Has a 50/50 Chance of Being ‘Black Holed’
Core
Si Si
Layer 3
50% Chance That Traffic
Will Go Down Path with
No Connectivity
Layer 3 Link
Distribution
Layer 2/3 Distribution-A Distribution-B
Si Si
Dr Tra
o f
No pped fic
De Pa wit
stin th h
atio to
n
Access
Layer 2
Access-a Access-n Access-c
VLAN 2 VLAN 2 VLAN 2
RST-2032
Daisy Chaining Access Layer Switches
New Technology Addresses Old Problems
• Stackwise technology eliminates the concern

Loopback links not required
No longer forced to have L2 link in distribution
• If you use modular (chassis based) switches these problems
are not a concern
Forwarding HSRP
Si
Active
Layer 3
Forwarding HSRP
Si
Standby
3750
RST-2032

flooding
boundary? Core
Si
Si

Si Si
Access
RST-2032
Asymmetric Routing (Unicast Flooding)
• Affects redundant
topologies with
shared L2 access
• One path upstream Asymmetric
and two paths Equal Cost
downstream Return Path
• CAM table entry CAM Timer Has

ages out on Aged out on Upstream Packet
Si Si Unicast to Active
standby HSRP Standby HSRP
HSRP
• Without a CAM Downstream
entry packet is Packet
flooded to all ports Flooded
in the VLAN
VLAN 2 VLAN 2 VLAN 2 VLAN 2
RST-2032
Best Practices Prevent Unicast Flooding
• Assign one unique

data and voice
VLAN to each
access switch Asymmetric
• Traffic is now only Equal Cost
Return Path
flooded down
one trunk
• Access switch Upstream Packet
Si Si Unicast to Active
unicasts correctly; Downstream
Packet HSRP
no flooding to
all ports Flooded on
Single Port
• If you have to:
Tune ARP and CAM
aging timers; CAM
timer exceeds
ARP timer
Bias routing metrics VLAN 3 VLAN 2
VLAN 4 VLAN 5
to remove equal
cost routes
RST-2032

flooding
boundary? Core
Si
Si

Si Si
Access
RST-2032
Keep Redundancy Simple
“If Some Redundancy Is

Good, More Redundancy
Is NOT Better”
• Root placement?
• How many
blocked links?
• Convergence?
• Complex fault resolution
RST-2032
But Don’t Go Too Far…
What Happens if You Don’t Link the Distributions?
• STP’s slow convergence can cause

considerable periods of traffic loss
• STP could cause non-deterministic
traffic flows/link load engineering
• STP convergence will cause
Layer 3 convergence
• STP and Layer 3 timers are
independent
• Unexpected Layer 3 convergence
and re-convergence could occur
• Even if you do link the distribution
switches dependence on STP and
link state/connectivity can cause
HSRP irregularities and
unexpected state transitions
RST-2032
What If You Don’t?
Black Holes and Multiple ‘Transitions’…
RST-2032
What If You Don’t?
Return Path Traffic Black Holed…
RST-2032
Layer2 Distribution Interconnection
Redundant Link from Access Layer Is Blocked
HSRP Active HSRP Active
and STP Root Layer 2 and STP Root
VLAN 20,140 Si Trunk Si VLAN 40,120
Distribution
Layer 2 Layer 2
Links Links
STP Model
Access
10.1.20.0 VLAN 20 Data 10.1.40.0 VLAN 40 Data
10.1.120.0 VLAN 120 Voice 10.1.140.0 VLAN 140 Voice
• Use only if Layer 2 VLAN spanning flexibility required

• STP convergence required for uplink failure/recovery
• More complex as STP root and HSRP should match
• Distribution to distribution link required for route summarization
RST-2032
No Spanning Tree—All Links Active
HSRP Active Layer 3 HSRP Active

VLAN 20,140 Si Si
VLAN 40,120
Distribution
Layer 2 Layer 2
Links Links
HSRP Model
Access
• Recommended ‘best practice’—tried and true

• No STP convergence required for uplink failure/recovery
• Distribution to distribution link required for route summarization
• Map L2 vlan number to L3 subnet for ease of use/management
RST-2032
GLBP Gateway Load Balancing Protocol
GLBP Active Layer 3 GLBP Active

VLAN 20,120,40,140 Si
VLAN 20,120, 40, 140
Si
Distribution
Layer 2 Layer 2
Links Links
GLBP Model
Access
• Fully utilize uplinks via GLBP

• Distribution to distribution required for route summarization
• No STP convergence required for uplink failure/recovery
RST-2032
Layer3 Access to Distribution Interconnection
All Links Are Routed
Layer 3
Si Si
Distribution
Layer 3 Layer 3
Equal Cost Equal Cost
Links Links
Routed Model
Access
• Best option for FAST convergence

• Equal cost L3 load balancing on all links
• No spanning tree required for convergence
• No HSRP/GLBP configuration required
• Not widely deployed/no VLAN spanning possible
RST-2032
PVST+, Rapid PVST+, or a Routing Protocol
STP vs. Equal Cost Layer3 Links and Routing
• When comparing the

traditional design to a
routed access layer we
Si Si Si Si
found some interesting
Si Si
things to investigate:
OSPF SPF timer when failing
distribution to access link
Cost Link’s Cost Link’s OSPF default route when
Si Si restoring distribution switch
ARP resolution/scaling when
restoring distribution switch
Si Si Si Si
Si Si RST-2031: Deploying a Fully

Routed Enterprise
Campus Network
RST-2032
Server Farm to Access
10
Time in Seconds to
9 2950 (Cisco IOS) 3550 (Cisco IOS) 4507 (CatOS)
8 4507 (Cisco IOS) 6500 (CatOS) 6500 (Cisco IOS)
Converge
Si Si
7
6
5
4 All Sub-Second
3
2
1
0
Si Si
PVST+ Rapid PVST+ EIGRP OSPF

10
Time in Seconds to
8 4507 (Cisco IOS) 6500 (CatOS) 6500 (Cisco IOS)

7
Converge
6
5 Approaching
All
4 Sub-Second
SONET Speeds
3
A B 2
1
0
RST-2032 PVST+ Rapid PVST+ EIGRP OSPF
Server Farm to Access
45
Time in Seconds to
Converge
Si Si 35 4507 (Cisco IOS) 6500 (CatOS) 6500 (Cisco IOS)
30
25
Four Seconds
20
of Loss Zero Packet
15 Loss
10
5
Si Si 0
PVST+ Rapid PVST+ EIGRP OSPF
45
402950 (Cisco IOS)
Time in Seconds to
3550 (Cisco IOS) 4507 (CatOS)

354507 (Cisco IOS) 6500 (CatOS) 6500 (Cisco IOS)
Converge
30
25 As Much as
20 40 Seconds
A B 15 of Loss
10
5
RST-2032 0
11208_05_2005_c2 © 2005 Cisco Systems, Inc. All rights reserved.
PVST+ Rapid PVST+ EIGRP OSPF 72
EIGRP vs. OSPF as Your IGP
Feasible Successor vs. LSA’s and SPF’s
• Within the campus environment

EIGRP provides for faster
convergence and greater
flexibility
Si Si Si Si Si Si • EIGRP provides for multiple
levels of route summarization
and route filtering which map to
the multiple tiers of the campus
Cost Link’s
• OSPF implements throttles on
Cost Link’s
Si Si LSA generation and SPF
calculations which limit
convergence times
Si Si Si Si
• When routes are summarized
Si Si and filtered only the distribution
peers in an EIGRP network need
to calculate new routes in event
of link or node failure
RST-2032
EIGRP to the Edge Design Rules
• EIGRP in the distribution block is

similar to EIGRP in the branch but
tuned for speed
• Limit scope of queries to a single
Si Si Si Si Si Si neighbor
• Summarize to campus core at the
distribution layer
• Control route propagation to edge
Cost Link’s Cost Link’s switch via distribute lists
Si Si
• Configure all edge switches to use
EIGRP ‘stub’
• Set hello and dead timers to ‘1’
Si Si Si Si
and ‘3’
Si Si
interface GigabitEthernet1/1
ip hello-interval eigrp 100 1
ip hold-time eigrp 100 3
WAN Data Center Internet router eigrp 100

eigrp stub connected
RST-2032
OSPF to the Edge Design Rules
• OSPF in the distribution block is similar

to OSPF in the branch but tuned for
speed
• Control number of routes and routers
Si Si Si
in each area
Si Si Si
• Configure each distribution block as a

separate totally stubby OSPF area
• Do not extend area 0 to the
Layer 3 Equal Layer 3 Equal edge switch
Cost Link’s Cost Link’s
Si
• Tune OSPF millisecond hello, dead-
Si
interval, SPF, and LSA throttle timers
interface GigabitEthernet1/1
Si Si Si Si
ip ospf dead-interval minimal
Si Si
hello-multiplier 4
router ospf 100
area 120 stub no-summary
timers throttle spf 10 100 5000
WAN Data Center Internet timers throttle lsa all 10 100 5000
timers lsa arrival 80
RST-2032
Agenda

Principals
Practices
• Summary
RST-2032
Mitigating Plug and Players
Protecting Against Well-Intentioned Users
Cisco Secure
Network Instability ACS
BPDU Guard
Unauthorized
Switch
Unauthorized
Switch Incorrect Root Guard
STP Info
Enterprise
Enterprise Server
Server
Authorized Authorized
Switch Switch
PROBLEM: SOLUTION:
• Well intentioned users place • Catalyst switches support
unauthorized network devices rogue BPDU filtering: BPDU
on the network possibly Guard, Root Guard
causing instability
RST-2032
BPDU Guard
Prevent Loops via WLAN (Windows XP Bridging)
PROBLEM:
• WLAN AP’s do not
forward BPDU’s
• Multiple Windows XP STP Loop
Formed
machines can create a
BPDU Guard
loop in the wired VLAN Disables Port
via the WLAN
SOLUTION:
• BPDU Guard configured BPDU
on all end station switch Generated
ports will prevent loop
from forming
BPDU
Win XP Discarded Win XP
Bridging Bridging
Enabled Enabled
RST-2032
Problem: Prevalence of Rogue AP’s
Example: 59 APs in Seven Miles in SJ Commute
• The majority of WLAN

deployments are unauthorized
by well intended employees
(rogue APs)—many are
insecure
Insecure
• A daily drive to work taken APs
within the car at normal speeds
with an IPAQ running a
freeware application (mix of
residences and enterprises)
• Insecure enterprise rogue
AP’s are a result of:
Well intentioned staff install due
to absence of sanctioned WLAN 59 APs Found
deployment
An infrastructure that is not
“wireless ready” to protect War Chalking
against rogue AP’s
RST-2032
Basic 802.1x Access Control
Controlling When and Where AP’s Are Connected
CatOS Configuration Example
Who Are You? set dot1x system-auth-control enable
set dot1x guest-vlan 250
I Am Joe Cisco set radius server 10.1.125.1 auth-port
1812 primary
802.1x Enabled on set radius key cisco123
Authorized set port dot1x 3/1-48 port-control
User “user” Facing
Ports auto
Who Are You? Cisco IOS Configuration Example

No radius-server host 10.1.125.1
802.1x radius-server key cisco123
Here aaa new-model
Rogue AP Disabled
aaa authentication dot1x default group
radius
D on Authorized aaa authorization default group radius
WLAN AP Ports aaa authorization config-commands
dot1x system-auth-control
Cisco IOS Per-Port configuration

Authorized AP int range fa3/1 - 48
dot1x port-control auto
RST-2032
SEC-2005: Understanding Identity-Based Networking Services
Securing Layer 2 from Surveillance Attacks
Cutting off MAC-Based Attacks
00:0e:00:aa:aa:aa
00:0e:00:bb:bb:bb Only 3 MAC
Addresses
Allowed on
250,000 the Port:
Bogus MAC’s Shutdown
per Second
PROBLEM: SOLUTION:
“Script Kiddie” Hacking Tools Port Security Limits MAC Flooding
Enable Attackers Flood Switch Attack and Locks down Port and
CAM Tables with Bogus Macs; Sends an SNMP Trap
Turning the VLAN into a “Hub”
switchport port-security
and Eliminating Privacy switchport port-security maximum 3
switchport port-security violation restrict
Switch CAM Table Limit Is Finite switchport port-security aging time 2
Number of Mac Addresses switchport port-security aging type
inactivity
RST-2032
DHCP Snooping
Protection Against Rogue/Malicious DHCP Server
1
D us
Re HCP g P DHCP
qu
est Bo HC nse Server
1000s of DHCP D po
Requests to e s
R
Overrun the 2
DHCP Server
• DHCP requests (discover) and responses (offer) tracked

• Rate-limit requests on trusted interfaces; limits DOS
attacks on DHCP server
• Deny responses (offers) on non trusted interfaces; stop
malicious or errant DHCP server
RST-2032
Securing Layer 2 from Surveillance Attacks
Protection Against ARP Poisoning
• Dynamic ARP inspection

protects against ARP Gateway = 10.1.1.1 Si
poisoning (ettercap, dsnif, MAC=A
arpspoof)
• Uses the DHCP snooping
binding table
Gratuitous ARP
• Tracks MAC to IP from 10.1.1.50=MAC_B
DHCP transactions
• Rate-limits ARP requests Gratuitous ARP
from client ports; stop port 10.1.1.1=MAC_B
scanning
• Drop BOGUS gratuitous
ARP’s; stop ARP
poisoning/MIM attacks
Attacker = 10.1.1.25 Victim = 10.1.1.50
MAC=B MAC=C
RST-2032
IP Source Guard
Protection Against Spoofed IP Addresses
• IP source guard protects

against spoofed IP Gateway = 10.1.1.1 Si
addresses MAC=A
• Uses the DHCP snooping
binding table
• Tracks IP address to port
associations
• Dynamically programs port
ACL to drop traffic not Hey, I’m 10.1.1.50 !
originating from IP address
assigned via DHCP
Attacker = 10.1.1.25 Victim = 10.1.1.50
RST-2032
Catalyst Integrated Security Features
Summary Cisco IOS
IP Source Guard
ip dhcp snooping
ip dhcp snooping vlan 2-10
Dynamic ARP Inspection ip arp inspection vlan 2-10
!
DHCP Snooping
interface fa3/1
Port Security switchport port-security
switchport port-security max 3
switchport port-security violation
• Port security prevents MAC restrict
flooding attacks
switchport port-security aging time 2
• DHCP snooping prevents client switchport port-security aging type
attack on the switch and server inactivity
• Dynamic ARP Inspection adds ip arp inspection limit rate 100
security to ARP using DHCP
ip dhcp snooping limit rate 100
snooping table
!
• IP source guard adds security to IP
source address using DHCP Interface gigabit1/1
snooping table ip dhcp snooping trust
ip arp inspection trust
RST-2032
SEC-2002: Understanding and Preventing Layer 2 Attacks
Agenda

Principals
Practices
• Summary
RST-2032
Reference Design—No VLANs Span Access Layer
• Tune CEF load balancing

• Match CatOS/IOS Etherchannel
settings and tune load
balancing Core
Si Si
• Summarize routes
towards core
• Limit redundant IGP peering
• STP Root and HSRP primary Layer 3
tuning or GLBP to load balance Distribution
on uplinks Si P-t-P Link Si
• Set trunk mode on/nonegotiate

• Disable Etherchannel
unless needed
• Set Port Host on access
layer ports:
Disable Trunking Access
Disable Etherchannel
VLAN 20 Data VLAN 40 Data
Enable PortFast 10.1.20.0/24 10.1.40.0/24
• RootGuard or BPDU-Guard VLAN 120 Voice VLAN 140 Voice
10.1.120.0/24 10.1.140.0/24
• Use security features
RST-2032
Layer 2 Distribution Interconnection
Some VLANs Span Access Layer

• Match CatOS/IOS Etherchannel
settings and tune load balancing
• Summarize routes towards core Core
Si Si
• Limit redundant IGP peering
• STP Root and HSRP primary or GLBP
and STP port cost tuning to load
balance on uplinks
Layer 2 Distribution
• Set trunk mode on/nonegotiate
Si Trunk Si
• Disable Etherchannel unless needed
• RootGuard on downlinks
• LoopGuard on uplinks
• Set port host on access
Layer ports:
Disable trunking
Disable Etherchannel Access
Enable PortFast
VLAN 20 Data VLAN 40 Data
• RootGuard or 10.1.20.0/24 10.1.40.0/24
BPDU-Guard VLAN 120 Voice VLAN 140 Voice
• Use security features 10.1.120.0/24 10.1.140.0/24
VLAN 250 WLAN
RST-2032 10.1.250.0/24
Routed Access
No VLANs Span Access Layer

• Match CatOS/IOS
Etherchannel settings Core
and tune load balancing Si Si
• Summarize routes
towards core
• Filter routes towards
the access Distribution
• Disable Etherchannel Si
Layer 3
Si
unless needed
P-t-P Link
• Set port host on access
layer ports:
Disable Trunking
Disable Etherchannel
Enable PortFast
Access
• RootGuard or BPDU-Guard VLAN 20 Data VLAN 40 Data
• Use security features 10.1.20.0/24 10.1.40.0/24
VLAN 120 Voice VLAN 140 Voice
10.1.120.0/24 10.1.140.0/24
RST-2032
Agenda

Principals
Practices
• Summary
RST-2032
Access
• Offers hierarchy – each layer has specific
role
• Modular topology - building blocks
Distribution Si Si
• Easy to grow, understand, and

troubleshoot
• Creates small fault domains – Clear
demarcations and isolation
Core
• Promotes load balancing and redundancy Si
Si
• Promotes deterministic traffic patterns

• Incorporates balance of both Layer 2 and
Distribution Layer 3 technology, leveraging the
Si Si
strength of both
• Utilizes Layer 3 Routing for load
balancing, fast convergence, scalability,
Access and control
Data Center
RST-2032
Optimizing Convergence
• Limit VLAN’s to a single closet when ever
possible to limit STP boundaries.
Access • If STP required utilize rapid PVST+
• Set trunks to ON/ON no-negotiate
• Match PaGP settings CatOS/Cisco IOS
• Consider EIGRP/routing in the access
Si
Distribution Si
• Utilize equal cost redundant connections to

Core for fastest convergence and to avoid
black holes
• Link distribution to distribution to facilitate
Core summarization and L2 spanning where
required Si
Si
• Utilize GLBP/HSRP mili-second timers

• Tune GLBP/HSRP preempt delay to avoid
black holes
Distribution • Tune Etherchannel and CEF load balancing Si Si
to insure optimum utilization of
redundant/equal cost links
Access • Build triangles not squares to take

advantage of equal cost redundant paths
for best/deterministic convergence
RST-2032
Q and A
RST-2032
Access
Load
Balancing
Si
Distribution Si
ing
Trunk
Core GLBP Si
Si
Distribution HSRP
Si Si
Spanning
Routing
Access Tree
Data Center
RST-2032
Reference Materials
http://www.cisco.com/go/srnd
• High Availability Campus Design Guide
• High Availability Campus Convergence Analysis
• High Availability Campus Design Guide – Routed
Access EIGRP (soon)
• High Availability Campus Design Guide – Routed
Access OSPF (soon)
RST-2032
Recommended Reading
• Continue your Networkers

learning experience with
further reading for this
session from Cisco Press
• Check the Recommended
Reading flyer for
suggested books
Available Onsite at the Cisco Company Store
RST-2032
RST-2032

Campus Multilayer Arch Design Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Campus Multilayer Arch Design Guide

Uploaded by

Copyright:

Available Formats

CAMPUS NETWORK MULTILAYER

ARCHITECTURE AND DESIGN

Real World Network Design:

Reinforced Network Infrastructure:

Real-Time Network Management:

Availability DPM Downtime Per Year (24x365)

99.000% 10000 3 Days 15 Hours 36 Minutes

99.500% 5000 1 Day 19 Hours 48 Minutes

99.900% 1000 8 Hours 46 Minutes

99.950% 500 4 Hours 23 Minutes

99.990% 100 53 Minutes

99.999% 10 5 Minutes “High

DPM—Defects per Million

• Costs for downtime are high Revenue/

• Multilayer Campus Design

• Promotes deterministic traffic patterns

• Aggregates network end-points • Catalyst® integrated security

• Availability, load balancing, QoS and • Spanning tree features:

• Backbone for the network—connects network building blocks

No Core Dedicated Core Switches

• The access layer is candidate for supervisor redundancy

• Multilayer Campus Design

Total of 68 Access Switches,

Three Distribution Blocks Si Si

6500 with Redundant Sup720s

Three Distribution Blocks

4500 SupII+, 6500 Sup720,

• Layer 3 routing protocols

• Unidirectional link detection ing

• Quality of service Spanning

• Insure redundant L3 paths to

to limit EIGRP query diameter Si Si

or OSPF LSA propagation

• Layer 3 redundant equal cost links support fast convergence

Limit unnecessary peering Distribution Si Si

Without passive interface: Routing

OSPF Example: EIGRP Example:

Router(config)#router ospf 1 Router(config)#router eigrp 1

Router(config)#router ospf 1 Router(config)#router eigrp 1

• What happens if fails?

RST-2032 10.1.1.b/24 10.1.1.a/24

• For OSPF when we summarize

L • The default CEF hash

• Alternating L3/L4 hash and

to achieve better distribution

• Highly available environments require redundant paths to ensure connectivity

802.1D-1998: Classic Spanning Tree 802.1s: Multiple Spanning Tree

• PVST+: an instance of STP (802.1d) per VLAN +

* Also Supported with MST and Rapid PVST+

access layer switches

• Take advantage of the

from becoming root Si Si

BPDUGuard to keep rogue switch

• Rapid-PVST+ greatly improves the restoration times for any

• Typically deployed on any

• Turn on in global Fiber Interconnections

CatOS: set udld enable

• Highly available networks require UDLD to

• Change the native VLAN to

• Manually prune all VLANS Si Si

except those needed

• Centralized VLAN Set

• Automatic formation of trunked Si On/On Si