You are on page 1of 30

Multi Site Multi Homing

March, 2004

Corporate Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 526-4100
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL
STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT
WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT
SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE
OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public
domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH
ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT
LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF
DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,
WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO
OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.;
Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA,
CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo,
Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ Net
Readiness Scorecard, LightStream, MGX, MICA, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar,
ScriptShare, SlideCast, SMARTnet, StrataView Plus, Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO are registered
trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries.

All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (0304R)

Multi Site Multi Homing
Copyright © 2004 Cisco Systems, Inc. All rights reserved.
CONTENTS

Multi Site Multi Homing 1

Overview 1

Multi-Site Multi-Homing Design Principles 3
High Availability 3
Scalability 5
Intelligent Network Services 5
HSRP 5
Routing Protocol Technologies 6
Edge Routing - BGP 6
Design Caveats 8
Work Arounds 9
Multi-Site Multi-Homing Design Recommendations 9
Border Router Layer 10
Internet Data Center Core Switching Layer 10
Firewall Layer 11
Data Center Core Switching Layer 11
Implementation Details 12
Multi-Site Multi-Homing Topology 12
Internet Cloud Router Configurations 12
Internet Edge Configurations 13
Edge Switching Layer Configurations 15
Core Switching Layer Configurations 20
BGP Attribute Tuning 23
Security Considerations 24

INDEX

Multi Site Multi Homing
Version 1.0 iii
Contents

Multi Site Multi Homing
iv Version 1.0
Multi Site Multi Homing

In almost all Enterprise infrastructures today Internet connectivity is universal, although the topology
designs may be unique. This document introduces a reference topology for Internet edge network
deployments. This covers the basis of design principles as well as an introduction of common
deployment barriers related to Internet edge topologies.

Overview
This document identifies and clarifies multi-site Internet edge designs. Multi-site Internet edge design
refers to the instance of having more than one data center connected to the Internet backbone. This can
either imply that each respective data center is multi-homed or has a single connection each. This
architecture includes the core design principles associated with all network infrastructure designs while
paying special attention to the unique requirements relevant to Internet edge multi-site topologies. Like
any infrastructure design, these aformentioned designs must be highly scalable while maintaining the
key aspects of security and redundancy. The key security functions include:
• Element security
• Identity services
• IP anti-spoofing
• Demilitarized zones (DMZ)
• Basic filtering and application definition
• Intrusion detection
The key redundancy functions associated with multi-site topologies are:
• Multiple data centers that act as Internet gateways for internal users.
• Distributed data centers that provide Internet/intranet server farm resiliency.
This document also discusses multi-homing. Multi-homing provides ISP resiliency by connecting each
data center to two or more ISPs depending on the bandwidth requirements, the server farm architecture,
or other internet services. These Internet connections can be a transit point for traffic both inbound to
the architecture and outbound to the Internet backbone for both the Internet/intranet server farms as
depicted in Figure 1. This document also describes and documents some common deployment problems
when introducing distributed data centers into any network topology.
Deploying distributed data centers introduces additional complexities to network administrators who
want to fully utilize both internet gateway locations. These challenges include:
• Application distribution

Multi Site Multi Homing
Version 1.0 1
Multi Site Multi Homing
Overview

• DNS propagation
• Replication timeouts
These design issues are covered in the Data Center Networking: Distributed Data Center SRND located
at
http://www.cisco.com/en/US/netsol/ns110/ns53/ns224/ns304/networking_solutions_design_guidances
_list.html.

Figure 1 Data Center Topology

Internet
Partners
SP1 SP2
PSTN WAN
VPN
Remote
Internet Edge
Office
Internet Gateway

DMZ

Or Or

Extranet
Private Data Center
Internet
WAN
Server
Farm

Corporate
Infrastructure Campus Core
87352

Intranet Data Center

Multi Site Multi Homing
2 Version 1.0
Multi Site Multi Homing
Multi-Site Multi-Homing Design Principles

Multi-Site Multi-Homing Design Principles
As mentioned above, Internet edge solutions touch many different types of enterprise networks and
therefore may potentially have many different topologies. They can range from any remote office
connection to a major ISP peering point. Therefore, using the common design principles associated with
all network architectures allows you to carry these recommendations into almost all Internet edge
topologies; ranging from a single-site ISP connection to a multi-site multi-homing environment.

High Availability
With topologies that connect to a single ISP, which is an enterprise connected to a single ISP, the
differences in redundancy reside at the ISP peering point. If you have a single ISP connection at the edge
of your network topology, the need for redundancy is a null issue because you have only a single exit
point. If the primary edge router fails, the Internet connection itself is down. Therefore, defining
redundancy at the edge of the network has no beneficial affect unless the provider supplies two terrestrial
circuits as depicted below. Multi-homing implementations offer redundancy in these instances, as well
as in the instances where there are multiple data centers for a single enterprise. You can leverage each
respective data center for redundancy and scalability, if you partition applications across multiple data
centers for high availability and scalability. For more information on distributed data center topologies,
refer to the Data Center Networking: Distributed Data Center SRND located at
http://www.cisco.com/en/US/netsol/ns110/ns53/ns224/ns304/networking_solutions_design_guidances
_list.html.

Multi Site Multi Homing
Version 1.0 3
Multi Site Multi Homing
Multi-Site Multi-Homing Design Principles

Figure 2 Multi-Site Multi-Homing Design

Internet

SP 1 SP 2
SP 3

Corporate WAN

West Coast East Coast
Remote offices Remote offices

87353
West Coast Data Center East Coast Data Center

Multi-site internet edge topologies are also composed of multiple layers. There must be no single point
of failure within the network architecture. Therefore, complete Internet edge device redundancy in this
architecture is a necessity. The infrastructure devices, such as routers and switches, coupled with specific
Layer 2 and Layer 3 technologies, help achieve this device redundancy. To meet this redundancy
requirement, Internet edge topologies use some of the key functions of the IOS software.
The Layer 3 features, used for high availability, offer redundant default gateways for networked hosts
and provide a predictable traffic flow in both normal operating conditions and under the adverse
conditions surrounding a network link or device failure. These Layer 3 features include:
• Hot Standby Router Protocol (HSRP)
• Multigroup Hot Standby Router Protocol (MHSRP)
• Routing protocol metric tuning (EIGRP and OSPF)
These Layer 3 functions also apply to redundancy by offering multiple default gateways in the network
topologies. HSRP and Multigroup HSRP offer Layer 3 gateway redundancy, whereas the dynamic
routing protocols offer a look into network availability from a higher level.

Multi Site Multi Homing
4 Version 1.0
Multi Site Multi Homing
Multi-Site Multi-Homing Design Principles

For instance, you could deploy HSRP between the edge routers to propagate a single default gateway
instance to the internal networks. In this case, if the primary router fails, the HSRP address is still active
on the secondary router instance, therefore the defined static route is still valid.

Scalability
The network architecture must be scalable to accommodate increasing user support, as well as
unforeseen bursts in network traffic. While feature availability and the processing power of network
devices are important design considerations; physical capacity attributes, like port density, can limit
architecture scalability. The termination of circuits can become a burden on device scalability within the
border layer of this topology. This burden is the same for the firewall device provisioning layer and the
Layer 3 switching layer. Port density scalability is also important at the Layer 3 switching layer because
it provides additional connections for host devices, in this case, servers.

Intelligent Network Services
In all network topologies, the intelligent network services present in the IOS software, such as QoS
functions and high availability technologies like HSRP, are used to ensure network availability. Below,
HSRP is documented and detailed for typical deployment scenarios.

HSRP
HSRP enables a set of routers to work together, giving the appearance of a single virtual router or default
gateway to the hosts on a LAN. HSRP is particularly useful in environments where critical applications
are running and fault-tolerant networks have been designed. By sharing an IP address and a MAC
address, two or more routers acting as a single virtual router are able to seamlessly assume the routing
responsibility in a defined event or an unexpected failure. This enables hosts on a LAN to continue to
forward IP packets to a consistent IP and MAC address enabling a transparent changeover of routing
devices.
HSRP allows you to configure hot standby groups to share responsibility for an IP address. You can give
each router a priority, which enables you to weight the prioritization of routers for active router selection.
One of the routers in each group is elected to be the active forwarder and one is elected as the stand-by
router. This is done according to the router's configured priorities. The router with the highest priority
wins and, in the event of a tie in priority, the greater value of their configured IP addresses breaks the
tie. Other routers in this group monitor the active and stand-by routers' status to enable further fault
tolerance.
All HSRP routers participating in a standby group watch for hello packets from the active and the
standby routers. They learn the hello and dead timers, as wells as the shared standby IP address from the
active router in the group, if these parameters are not explicitly configured on each individual router.
Although this is a dynamic process, Cisco recommends that you define the HSRP dead timers in the
topology. If the active router becomes unavailable due to scheduled maintenance, power failure, or other
reasons; the stand-by assumes this functionality transparently within a few seconds. Failover occurs
when three successive hello packets are missed and the dead timer is reached. The standby router
promptly takes over the virtual addresses, identity, and responsibility. When the secondary interface
assumes mastership, the new master sends a gratuitous ARP, which updates the L2 switch's content
addressable memory (CAM). This then becomes the primary route for the devices accessing this
gateway. These HSRP timers can be configured on a per instance of HSRP.

Multi Site Multi Homing
Version 1.0 5
Multi Site Multi Homing
Multi-Site Multi-Homing Design Principles

Routing Protocol Technologies
Before introducing and examining the basic ways in which autonomous systems can be connected to
ISPs, some basic terminology and concepts of routing must be established.
There are three basic routing approaches:
Static
Default
Dynamic
Static routing refers to routes to destinations manually listed in the router. Network reachability, in this
case, is not dependent on the existence and state of the network itself. Whether a destination is up or
down, the static routes remain in the routing table, and traffic is still sent toward that destination.
Default routing refers to a “last resort” outlet. Traffic to destinations that are unknown to the router are
sent to the default outlet. Default routes are also manually listed in the router. Default routing is the
easiest form of routing for a domain connected to a single exit point.
Dynamic routing refers to the router learning routes via an internal or external routing protocol. Network
reachability is dependent on the existence and state of the network. If a destination is down, the route
disappears from the routing table and traffic is sent toward that destination.
These three routing approaches are possibilities for all the configurations considered in forthcoming
sections, but usually there is an optimal approach. Thus, in illustrating different autonomous systems,
this document considers whether static, dynamic, default, or some combination of these routing
approaches is optimal. This document also considers whether interior or exterior routing protocols are
appropriate. Interior gateway protocols (IGPs) can be used for the purpose of advertising the customer's
networks. An IGP can be used between the enterprise and provider for the enterprise to advertise its
routes. This has all the benefits of dynamic routing because network information and changes are
dynamically sent to the provider. Also, the IGP's distribute the network routes upstream to the BGP
function.

Edge Routing - BGP
Border gateway protocols (BGPs) perform interdomain routing in TCP/IP networks. BGP is an exterior
gateway protocol (EGP), which means that it performs routing between multiple autonomous systems or
domains and exchanges routing and reachability information with other BGP systems.
BGP devices exchange routing information upon initial data exchange and during incremental updates.
When a router first connects to the network, BGP routers exchange their entire BGP routing tables and,
when the routing table changes, those same routers send only the changed portion of their routing tables.
BGP routers do not send regularly scheduled routing updates and BGP routing updates advertise only
the optimal path to a network.
BGP uses a single routing metric to determine the best path to a given network. This metric consists of
an arbitrary unit number that specifies the degree of preference of a particular link. The BGP metric
typically is assigned to each link by the network administrator. The value assigned to a link can be based
on any number of criteria, including the number of autonomous systems through which the path passes,
stability, speed, delay, or cost.
BGP performs three types of routing: interautonomous system routing, intra-autonomous system
routing, and pass-through autonomous system routing.
• Interautonomous system routing occurs between two or more BGP routers in different autonomous
systems. Peer routers in these systems use BGP to maintain a consistent view of the internetwork
topology. BGP neighbors communicating between autonomous systems must reside on the same

Multi Site Multi Homing
6 Version 1.0
Multi Site Multi Homing
Multi-Site Multi-Homing Design Principles

physical network. The Internet serves as an example of an entity that uses this type of routing
because it is comprised of autonomous systems or administrative domains. Many of these domains
represent the various institutions, corporations, and entities that make up the Internet. BGP is
frequently used to provide path determination to provide optimal routing within the Internet.
• Intra-autonomous system routing occurs between two or more BGP routers located within the same
autonomous system. Peer routers within the same autonomous system use BGP to maintain a
consistent view of the system topology. BGP is also used to determine which router serves as the
connection point for specific external autonomous systems. Once again, the Internet provides an
example of interautonomous system routing. An organization, such as a university, could make use
of BGP to provide optimal routing within its own administrative domain or autonomous system. The
BGP protocol can provide both inter- and intra-autonomous system routing services.
• Pass-through autonomous system routing occurs between two or more BGP peer routers that
exchange traffic across an autonomous system that does not run BGP. In a pass-through autonomous
system environment, the BGP traffic did not originate within the autonomous system in question and
is not destined for a node in the autonomous system. BGP must interact with whatever
intra-autonomous system routing protocol is being used to successfully transport BGP traffic
through that autonomous system.

BGP Attributes

BGP attributes support the control of both inbound and outbound network routes. These attributes can
be adjusted to control the decision making process of BGP itself. The BGP attributes are a set of
parameters that describe the characteristics of a prefix (route). The BGP decision process uses these
attributes to select its best routes. Specific attributes associated with larger topologies like this one are
addressed later in this document. More specifically, the MED attribute and the use of route reflectors are
addressed.
Figure 3 displays a multi site multi homed topology.

Multi Site Multi Homing
Version 1.0 7
Multi Site Multi Homing
Design Caveats

Figure 3 Multi Site Internet Edge Topology

Internet

SP 1 SP 2
SP 3

Corporate WAN

West Coast East Coast
Remote offices Remote offices

87353
West Coast Data Center East Coast Data Center

Design Caveats
In certain multi-site deployments, device placement becomes a caveat to the overall design. In a specific
instance, the placement of the firewall and how it is introduced into the architecture from a routing
standpoint are of major concern. There are two main caveats to be concerned with when designing your
network:
• Inability to terminate IGP on firewall device
• Lack of upstream route health or interface uptime
In a design where the PIX firewall is placed at the edge of the network between the Internet border
routers and the internet data center core switches, the PIX can become a black hole route to the end-users
that are geographically adjacent to that data center. In detail, when deploying a PIX firewall, the most
common deployment is to have the device configured with static routes upstream to the internet border
routers and with a static route downstream to the internal Layer 3 switching platform. Since static

Multi Site Multi Homing
8 Version 1.0
Multi Site Multi Homing
Multi-Site Multi-Homing Design Recommendations

routing is the configuration of choice, you can assume that the firewall cannot participate in the IGP
routing protocol. If the external routes from the internet border routers disappear from the routing table,
the internal routing process has no idea that this is no longer a valid route to the Internet.
Since the PIX is not participating in an IGP routing protocol, the firewall has no intelligence of the
routing updates that take place above the firewall layer. Therefore, the device still accepts packets
destined for the Internet. This is usually the case because the Layer 3 switching layer below the PIX
device propagates or redistributes a static route of 0.0.0.0 into the IGP downstream.

Work Arounds
The aformentioned problem is common when deploying distributed data centers and has the following
three work arounds:
• The first work around is using the BGP routing protocol to inform the Layer 3 switching platform
of the route change by tunneling the I-BGP traffic through the firewall to the peer on the inside
interface or the Layer 3 switching platform that houses the IGP routing process. This design is
documented in “High Availability via BGP Tunneling.”
• With a future release of HSRP, the you could use HSRP tracking to track the HSRP interface of the
Internet border routers. This assumes that the border routers also implement a tracking instance of
the upstream ISP interfaces. This has not been tested or documented.
• Finally, you could use the Firewall Service Module (FWSM) in the edge Layer 3 switching platform.
This deployment allows you to process OPSF routes internal to the IGP by having the firewall device
participate in the OSPF process. This deployment that has been tested and is documented in this
document.

Multi-Site Multi-Homing Design Recommendations
As mentioned above; multi-site Internet edge topologies are different than single site topologies in
various ways. Also, the scale of these topologies may be different. But these topologies are increasingly
important to enterprise business functions. Hence, the scalability of these topologies can not be
overlooked. It is also imperative to this type of architecture to have complete redundancy. The details of
the functional layers of the Internet edge topologies and how they interact with one another are detailed
below.
When deploying a distributed data center environment, you must adhere to certain characteristics. For
example, these topologies still use a similar ISP multi-homing relationship, but the attributes are slightly
different. Also, since this architecture is distributed, it becomes a network that has multiple Internet
gateways in different data centers. This network is usually partitioned in such a way that locally adjacent
users traverse their respective local data centers. This type of design recommendation assumes that you
have configured the internal IGP to route locally adjacent end-users through their respective data centers
while still offering redundancy to the other data center in the event of failure.
This distributed data center design topology deployed of a not-so stubby area networks. This allows you
to define multiple Internet data center topologies without changing the integrity of the core
infrastructure. Each of the geographically dispersed area's and autonomous systems are represented
below in Figure 4.

Multi Site Multi Homing
Version 1.0 9
Multi Site Multi Homing
Multi-Site Multi-Homing Design Recommendations

Figure 4 Internet Edge AS/ Area Topology

East Coast West Coast
Internet ISP
BGP BGP
Cloud
ISP AS1 ISP AS2
ISP AS 1/2

BGP AS 100

OSPF OSPF
NSSA NSSA
251 252

OSPF Area
0

Corporate LAN
Connectivity

87354
Border Router Layer
Border routers, typically deployed in pairs, are the edge-facing devices of the network. The number of
border routers deployed is a decision of provisioning, based on memory requirements, and physical
circuit termination. The border router layer is where you provision ISP termination and initial security
parameters. The border router layer serves as the gateway of the network and uses an externally facing
Layer 3 routing protocol, like BGP, integrated with an internally facing OSPF to intelligently route
traffic throughout the external and internal networks, respectively. This layer starts the OSPF process
internally into the network. The Internet edge in an enterprise environment may provide Internet
connectivity to an ISP through the use of multi-homed internet border routers. This layer also injects the
gateway of last resort route into the IGP through specific BGP parameters defined below.

Internet Data Center Core Switching Layer
The Layer 3 switching layer is the layer in the multi-site internet edge topology that serves as the
gateway to the core of the network. This is also a functional layer of the internet server farm design. This
layer may act as either a core layer or an aggregation layer in some design topologies. Yet, the primary
function, from the internet edge design topology standpoint, is to advertise the IGP routing protocol
internally to the infrastructure. OSPF processes for each data center interfaces with Area 0 at this layer,
as shown in Figure 4.

Multi Site Multi Homing
10 Version 1.0
Multi Site Multi Homing
Multi-Site Multi-Homing Design Recommendations

Firewall Layer
The firewall layer is a security layer that allows stateful packet inspection into the network infrastructure
and to the services and applications offered in the server farms and database layers. In this topology, the
firewall layer is represented by the FWSM in the Catalyst 6500 series switching platform. This layer also
acts as the network address translation (NAT) device in most design topologies. NAT, at the Internet
edge, is common based on the ever depleting Ipv4 address pool associated with ISP's. This allows many
ISP's to give a limited address range, which, in turn, requires NAT pools at the egress point of the
topology.

Data Center Core Switching Layer
The data center core layer in this topology is the transport layer between data centers. This assumes that
the layers are represented in the same Area 0 in the OPSF routing process. This layer is also the
termination point for both the geographically adjacent WAN routers or the geographically adjacent
LAN's in the architecture. This layer allows you to control the administrative distances or actual costs
associated with the gigabit links to the upstream edge Layer 3 switches. Figure 5 displays how the
network topology is partitioned into two different geographic areas.

Figure 5 Physical Layer Topology

Internet ISP
Networks Cloud Networks
1.x.x.x,...5.x.x.x ISP AS 1/2 6.x.x.x,...12.x.x.x

.1 .1
Network Network
East Coast 172.16.11.x 172.16.10.x West Coast
internet edge internet edge
.254 .254
.1 Network Network .1
172.16.253.x FWSM Outside 172.16.254.x
East Coast West Coast
FWSM Intside
edge .100 .100 edge
.2 Network Network .2
.1 172.16.251.x 172.16.252.x .1

East Coast West Coast
core .1 Network .100 core
172.16.250.x

Corporate LAN
87355

Connectivity

Multi Site Multi Homing
Version 1.0 11
Multi Site Multi Homing
Implementation Details

Implementation Details
Below are the implementation details associated with defining and configuring the multi-site Internet
edge topology. Also, there are specific configurations associated with each layer that allow for the route
control and failover of the topology stated above.

Multi-Site Multi-Homing Topology
In this section, the router configurations were taken from the each of the East Coast routers as depicted
in the Figure 5. These configurations were defined solely for this testbed and are not representative of
the normal ISP recommended configurations.

Internet Cloud Router Configurations
The Internet cloud routers were configured with loopback interfaces for testing purposes. These
interfaces allow ping traffic to traverse the internal network outbound to the internet backbone. Below,
each configuration was defined with each of the respective network segments. This also made it easier
to determine the routes locally adjacent to each of the internet gateway routers.

Internet Cloud Router ISP AS1
hostname InternetCloud1

interface Loopback0
ip address 2.0.0.1 255.255.255.0 secondary
ip address 3.0.0.1 255.255.255.0 secondary
ip address 4.0.0.1 255.255.255.0 secondary
ip address 5.0.0.1 255.255.255.0 secondary
ip address 1.0.0.1 255.255.255.0

When looking into the BGP process below, you can see that only specific subnets were defined for
redistribution. Since this solution is not performance focused, the decision was made to only propagate
those routes. This allows for BGP redistribution to the lower layers to ensure that the internal OSPF
redistribution is working correctly.
router bgp 1
network 1.0.0.0
network 2.0.0.0
network 3.0.0.0
network 4.0.0.0
network 5.0.0.0
network 20.10.5.0
network 172.16.11.0
redistribute connected
neighbor 20.10.5.254 remote-as 2
neighbor 172.16.11.254 remote-as 100

Internet Cloud Router ISP AS2

The configuration of the second Internet cloud router is the same as the first except that different IP
addresses were used.
hostname InternetCloud2

interface Loopback0

Multi Site Multi Homing
12 Version 1.0
Multi Site Multi Homing
Implementation Details

ip address 7.0.0.1 255.255.255.0 secondary
ip address 8.0.0.1 255.255.255.0 secondary
ip address 9.0.0.1 255.255.255.0 secondary
ip address 11.0.0.1 255.255.255.0 secondary
ip address 12.0.0.1 255.255.255.0 secondary
ip address 6.0.0.1 255.255.255.0
no ip directed-broadcast
!

router bgp 2
network 6.0.0.0
network 7.0.0.0
network 8.0.0.0
network 9.0.0.0
network 11.0.0.0
network 12.0.0.0
network 20.10.5.0
network 172.16.10.0
redistribute connected
neighbor 20.10.5.1 remote-as 1
neighbor 172.16.10.254 remote-as 100

Internet Edge Configurations
The first layer in the topology was the Internet border router layer. At this layer, the peering relationship
via BGP to the ISP routers takes place. Also at this layer, the first instance of the OSPF process begins.
The BGP process propagates a default route into the OSPF routing instance. Below are the internet edge
router configurations.

East Coast Internet Edge Configurations
EdgeRouter1#wr t
Building configuration...

!
hostname EdgeRouter1
!

The interface configurations below represent the donwstream link to the outside interface or segment of
the FWSM link:

Note The OSPF hello and dead-interval timers must be the same across all links and interface:

!
interface FastEthernet0/0
ip address 172.16.253.254 255.255.255.0
no ip route-cache
ip ospf hello-interval 1
ip ospf dead-interval 3
no ip mroute-cache
duplex full
!

The following configuration examples are associated with the upstream links to the ISP clouds:
interface FastEthernet3/0

Multi Site Multi Homing
Version 1.0 13
Multi Site Multi Homing
Implementation Details

ip address 172.16.11.254 255.255.255.0
no ip redirects
no ip route-cache
no ip mroute-cache
duplex half

The following OSPF and BGP edge configurations allow the edge to redistribute BGP processes to the
internal network. The redistribute bgp command within the OSPF process causes this redistribution. This
assumes that the router can propagate those routes internal to the other network segments.
Injecting full BGP routes into an IGP is not recommended. Doing so adds excessive routing overhead to
any IGP. Interior routing protocols were never designed to handle more than the networks inside your
autonomous systems, plus some exterior routes from other IGPs.
This does not mean that BGP routes should never be injected into IGPs. Depending on the number of
BGP routes and how critical the need for them to be in the IGP, injecting partial BGP routes into IGP
may well be appropriate.
Below are the OSPF and BGP configurations respectively:

Note Router OSPF is defined as a not so stubby area (NSSA). This is needed to redistribute the external routes
form the upstream routing instance:

For the sake of the testbed topology and to define that routes have been updated properly, specific BGP
routes were redistributed into the architecture:
router ospf 500
log-adjacency-changes
area 251 nssa
redistribute bgp 100
network 172.16.251.0 0.0.0.255 area 251
network 172.16.253.0 0.0.0.255 area 251

Note In typical Internet edge deployments, the edge routing instance does not redistribute the BGP process
into the OSPF process, but rather uses the default-information originate command to define a default
route to edge routing instance. That default route is then redistributed via the OSPF process to the
internal network only if the edge routing instance has a default route itself:

router ospf 500
log-adjacency-changes
area 251 nssa
network 172.16.251.0 0.0.0.255 area 251
network 172.16.253.0 0.0.0.255 area 251
default-information originate route-map SEND_DEFAULT_IF

The ACL's below state that if the router has any entry in its routing table from the next hop ISP router,
then it sends the default route internal to the network. This configuration must be deployed on both edge
routing devices.
access-list 1 permit 0.0.0.0
access-list 2 permit 172.16.11.1

route-map SEND_DEFAULT_IF permit 10
match ip address 1
match ip next-hop 2

Multi Site Multi Homing
14 Version 1.0
Multi Site Multi Homing
Implementation Details

Note The route map SEND_DEFAULT_IF is associated with the default-information originate command. This
route map matches on the condition that the 0/0 default (access-list 1) has a next hop of 172.16.11.1
(access-list 2). This satisfies the condition that the 0/0 is learned via EBGP rather than I-BGP.

Below is the BGP routing instance that defines the upstream BGP neighbor that is necessary for the
above route-map to work.
router bgp 100
no synchronization
bgp log-neighbor-changes
network 172.16.11.0
redistribute connected
neighbor 172.16.11.1 remote-as 1
no auto-summary

Below, are the routes available to the East Coast edge routers:

Note Setting the route propagation via OSPF on the FWSM requires defining route-maps that only allow
specific traffic to the edge layer. Therefore, the only internal route propagated is the 172.16.251.x. This
can be controlled by supernetting the segment to allow only specific addresses.

EdgeRouter1#sho ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route

Gateway of last resort is not set

B 1.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 2.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 3.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 4.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 20.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 5.0.0.0/8 [20/0] via 172.16.11.1, 00:09:13
B 6.0.0.0/8 [20/0] via 172.16.11.1, 00:08:13
172.16.0.0/24 is subnetted, 6 subnets
C 172.16.253.0 is directly connected, FastEthernet0/0
O IA 172.16.251.0 [110/11] via 172.16.253.1, 00:44:14, FastEthernet0/0
C 172.16.11.0 is directly connected, FastEthernet3/0
B 7.0.0.0/8 [20/0] via 172.16.11.1, 00:08:14
B 8.0.0.0/8 [20/0] via 172.16.11.1, 00:08:14
B 9.0.0.0/8 [20/0] via 172.16.11.1, 00:08:14
B 11.0.0.0/8 [20/0] via 172.16.11.1, 00:08:14
B 12.0.0.0/8 [20/0] via 172.16.11.1, 00:08:14

Edge Switching Layer Configurations
The edge switching layer configurations house the FWSM as well as the NSSA instance of the OSPF
process. The instance of OSPF that the upstream Internet edge routing layer binds to is the outside
interface of the FWSM. The inside instance of the FWSM is an OSPF neighbor to the process running
locally on the MSFC on the switch itself. These switches run in a same NSSA area, while running two
different OSPF processes. This ensures the tuning of the protocol that defines which routes can be

Multi Site Multi Homing
Version 1.0 15
Multi Site Multi Homing
Implementation Details

propagated to the upstream network. This configuration recommendation is to ensure that restricted
routes are not externally advertised. With areas defined this way, you can tune the routes appropriately
by defining route maps to allow a specific network segment outbound. For instance; if you only wanted
to advertise the VIP addresses of the server farm outbound, you could create a route-map that only allows
those specific addresses outbound.

East Coast Edge Switching Layer Configurations
EASTEDGE1#wr t
Building configuration...

hostname EASTEDGE1

!
Below, is the configuration that associates specific VLAN's with the FWSM. VLAN 200 is the internal
vlan and VLAN 300 is the external vlan:
firewall module 2 vlan-group 1
firewall vlan-group 1 200,300

!
vlan dot1q tag native

Gigabit 1/1 is the downstream link to the OSPF core Layer 3 switching layer. Note it has be configured
to be in VLAN 200:
!
!
interface GigabitEthernet1/1
no ip address
switchport
switchport access vlan 200

FastEthernet 1/1 is the upstream link to the edge routing layer. Note it has be configured to be in VLAN
300:
!
interface FastEthernet3/1
no ip address
duplex full
speed 100
switchport
switchport access vlan 300
!

Note Interface VLAN 200 OSPF configurations are the same across all OSPF interfaces from the timers
perspective:

!
interface Vlan200
ip address 172.16.251.2 255.255.255.0
ip ospf hello-interval 1
ip ospf dead-interval 3

Note The OSPF routing process below is the internal OSPF neighbor to the core switching layer:

!

Multi Site Multi Homing
16 Version 1.0
Multi Site Multi Homing
Implementation Details

router ospf 500
log-adjacency-changes
area 251 nssa
network 172.16.251.0 0.0.0.255 area 251
!
ip classless
no ip http server
!
!
arp 127.0.0.12 0000.2100.0000 ARPA
!
!
line con 0
line vty 0 4
login
transport input lat pad mop telnet rlogin udptn nasi
!
end

Below are the configurations associated with the FWSM. Notice the OSPF configuration in the FWSM
itself and how it binds itself to the OSPF process.
EDGE1# sess slot 2 proc 1
The default escape character is Ctrl-^, then x.
You can also type 'exit' at the remote prompt to end the session
Trying 127.0.0.21 ... Open

FWSM passwd:

Welcome to the FWSM firewall

Type help or '?' for a list of available commands.
EASTFWSM> en
Password:
EASTFWSM# wr t
Building configuration...
: Saved
:
FWSM Version 1.1(1)
no gdb enable
nameif vlan200 inside security100
nameif vlan300 outside security0
enable password 8Ry2YjIyt7RRXU24 encrypted
passwd 2KFQnbNIdI.2KYOU encrypted
hostname EASTFWSM
fixup protocol ftp 21
fixup protocol h323 H225 1720
fixup protocol h323 ras 1718-1719
fixup protocol ils 389
fixup protocol rsh 514
fixup protocol smtp 25
fixup protocol sqlnet 1521
fixup protocol sip 5060
fixup protocol skinny 2000
fixup protocol http 80
names
access-list outside permit tcp any any
access-list outside permit udp any any
access-list outside permit icmp host 6.0.0.1 any echo-reply
access-list inside permit tcp any any
access-list inside permit udp any any
access-list inside permit icmp host 172.16.250.10 any echo

Multi Site Multi Homing
Version 1.0 17
Multi Site Multi Homing
Implementation Details

ACL 500 defines which addresses need to be matched to support the advertisement of the OSPF:
access-list 500 permit 172.16.251.0255.255.255.0
pager lines 24
icmp permit any inside
icmp permit any outside
mtu inside 1500
mtu outside 1500
ip address inside 172.16.251.100 255.255.255.0
ip address outside 172.16.253.1 255.255.255.0
no failover
failover lan unit secondary
failover timeout 0:00:00
failover poll 15
failover ip address inside 0.0.0.0
failover ip address outside 0.0.0.0
pdm history enable
arp timeout 14400
static (inside,outside) 172.16.250.10 172.16.250.10 netmask 255.255.255.255 0 0
access-group inside in interface inside
access-group outside in interface outside

The OSPF interface timer configurations below are common across the architecture:
interface inside

ospf hello-interval 1
ospf dead-interval 3
!
!
interface outside

ospf hello-interval 1
ospf dead-interval 3
!
The route-map below states that the permitted advertised traffic must match the access-list 500
addresses. This route-map is then bound to the OSPF process that is redistributing the routes, as seen
below in router OSPF 100:
route-map 500 permit 10
match ip address 500
!
The OSPF configurations below are representative of the recommended security configuration. Within
the configuration, two different OSPF routing processes were defined to control inbound and outbound
route propagation:
router ospf 500
network 172.16.251.0 255.255.255.0 area 251

area 251 nssa

log-adj-changes

redistribute ospf 100

router ospf 100
network 172.16.253.0 255.255.255.0 area 251

area 251 nssa

log-adj-changes

redistribute ospf 500 subnets route-map 500

Multi Site Multi Homing
18 Version 1.0
Multi Site Multi Homing
Implementation Details

!
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h323 0:05:00 sip 0:30:00
sip_media 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
aaa-server LOCAL protocol local
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable
no sysopt route dnat
telnet timeout 5
ssh timeout 5
terminal width 80
Cryptochecksum:03e78100e37fef97b96c15d54be90956
: end
[OK]

Showing the routes available to the FWSM ensures that the proper outside/inside routes were
propagated:

EASTFWSM# sho route
C 127.0.0.0 255.255.255.0 is directly connected, eobc
O N2 1.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 2.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 3.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 4.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 20.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 5.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:46:35, outside

O N2 6.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:35, outside

172.16.0.0 255.255.255.0 is subnetted, 5 subnets
O IA 172.16.252.0 [110/12] via 172.16.251.1, 1:16:14, inside

C 172.16.253.0 is directly connected, outside

O IA 172.16.254.0 [110/22] via 172.16.251.1, 1:10:14, inside

O IA 172.16.250.0 [110/11] via 172.16.251.1, 1:21:35, inside

C 172.16.251.0 is directly connected, inside

O N2 7.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:36, outside

O N2 8.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:36, outside

O N2 9.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:36, outside

O N2 11.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:36, outside

O N2 12.0.0.0 255.0.0.0 [110/1] via 172.16.253.254, 0:45:36, outside

EASTFWSM# exit

Multi Site Multi Homing
Version 1.0 19
Multi Site Multi Homing
Implementation Details

Logoff

[Connection to 127.0.0.21 closed by foreign host]

Below are the routes associated with the edge switching layers. Notice that the edge layer has two routes
to the Internet backbone: one primary route via the OSPF process running locally on the switch and the
redundant route running through Area 0 to the secondary switch. This is the same respectively on each
switch.
EASTEDGE1#sho ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route

Gateway of last resort is not set

O N2 1.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 2.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 3.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 4.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 20.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 5.0.0.0/8 [110/1] via 172.16.251.100, 00:45:56, Vlan200
O N2 6.0.0.0/8 [110/1] via 172.16.251.100, 00:44:56, Vlan200
172.16.0.0/24 is subnetted, 5 subnets
O IA 172.16.252.0 [110/3] via 172.16.251.1, 01:15:39, Vlan200
O 172.16.253.0 [110/11] via 172.16.251.100, 01:20:56, Vlan200
O N2 172.16.254.0 [110/13] via 172.16.251.1, 01:09:35, Vlan200
O IA 172.16.250.0 [110/2] via 172.16.251.1, 01:20:56, Vlan200
C 172.16.251.0 is directly connected, Vlan200
O N2 7.0.0.0/8 [110/1] via 172.16.251.100, 00:44:57, Vlan200
O N2 8.0.0.0/8 [110/1] via 172.16.251.100, 00:44:57, Vlan200
O N2 9.0.0.0/8 [110/1] via 172.16.251.100, 00:44:57, Vlan200
O N2 11.0.0.0/8 [110/1] via 172.16.251.100, 00:44:57, Vlan200
O N2 12.0.0.0/8 [110/1] via 172.16.251.100, 00:44:57, Vlan200

Core Switching Layer Configurations
The core switching layers are the layers that house the OPSF Area 0 process. This layer becomes the
transport for Internet destined traffic for each of the respective data centers in the event of failure. This
is also the layer where the configurations are controlled to ensure that traffic is destined to the right
geographical areas. This is accomplished using the ip ospf cost configurations on interfaces where the
OSPF neighbor areas are present.

East Coast Core Switching Layer Configurations
EASTCOASTCORE#wr t
Building configuration...

!
hostname EASTCOASTCORE
!

!
interface Port-channel1

Multi Site Multi Homing
20 Version 1.0
Multi Site Multi Homing
Implementation Details

no ip address
switchport
switchport trunk encapsulation dot1q

The interface configurations below state that Gigabit 1/1 is the OSPF interface that neighbors to the East
Coast edge layer. This is where the you can tune the OSPF cost to define that any users locally adjacent
to the East Coast core would chose this upstream link for the internet traffic. This same configuration is
also tuned to ensure any west coast traffic traverse the west coast routes:
!
interface GigabitEthernet1/1
ip address 172.16.251.1 255.255.255.0
ip ospf hello-interval 1
ip ospf dead-interval 3
ip ospf cost 5
!
interface GigabitEthernet1/2
no ip address
switchport
switchport trunk encapsulation dot1q
channel-group 1 mode on
!
interface GigabitEthernet2/1
no ip address
switchport
switchport trunk encapsulation dot1q
channel-group 1 mode on
!
interface Vlan1
no ip address
shutdown
!
interface Vlan200
ip address 172.16.250.1 255.255.255.0
ip ospf hello-interval 1
ip ospf dead-interval 3
!
router ospf 500
log-adjacency-changes
area 251 nssa
network 172.16.250.0 0.0.0.255 area 0
network 172.16.251.0 0.0.0.255 area 251
!
ip classless
no ip http server
!
!
!
line con 0
line vty 0 4
login
transport input lat pad mop telnet rlogin udptn nasi
!
end

The following displays the routes associated with East Coast core. Note the path preference is the
respective edge switching layer.
EASTCOASTCORE#sho ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP

Multi Site Multi Homing
Version 1.0 21
Multi Site Multi Homing
Implementation Details

i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route

Gateway of last resort is not set

O N2 1.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 2.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 3.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 4.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 20.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 5.0.0.0/8 [110/1] via 172.16.251.100, 00:06:46, GigabitEthernet1/1
O N2 6.0.0.0/8 [110/1] via 172.16.251.100, 00:05:53, GigabitEthernet1/1
172.16.0.0/24 is subnetted, 5 subnets
O IA 172.16.252.0 [110/2] via 172.16.250.254, 00:36:37, Vlan200
O 172.16.253.0 [110/11] via 172.16.251.100, 00:41:53, GigabitEthernet1/1
O IA 172.16.254.0 [110/12] via 172.16.250.254, 00:30:32, Vlan200
C 172.16.250.0 is directly connected, Vlan200
C 172.16.251.0 is directly connected, GigabitEthernet1/1
O N2 7.0.0.0/8 [110/1] via 172.16.251.100, 00:05:54, GigabitEthernet1/1
O N2 8.0.0.0/8 [110/1] via 172.16.251.100, 00:05:54, GigabitEthernet1/1
C 127.0.0.0/8 is directly connected, EOBC0/0
O N2 9.0.0.0/8 [110/1] via 172.16.251.100, 00:05:54, GigabitEthernet1/1
O N2 11.0.0.0/8 [110/1] via 172.16.251.100, 00:05:54, GigabitEthernet1/1
O N2 12.0.0.0/8 [110/1] via 172.16.251.100, 00:05:54, GigabitEthernet1/1

West Coast Core Switching Layer Configurations
The West Coast configurations refer to the IP routes. The West Coast core prefers the West
Coast edge as its primary routes:

WESTCOASTCORE#wr t
Building configuration...

!
hostname WESTCOASTCORE
!

!
!
!
interface Port-channel1
no ip address
switchport
switchport trunk encapsulation dot1q
!
interface GigabitEthernet1/1
ip address 172.16.252.1 255.255.255.0
ip ospf hello-interval 1
ip ospf dead-interval 3
ip ospf cost 5

!
interface GigabitEthernet1/2
no ip address
switchport
switchport trunk encapsulation dot1q
channel-group 1 mode on
!
interface GigabitEthernet2/1
no ip address
switchport
switchport trunk encapsulation dot1q
channel-group 1 mode on
!

Multi Site Multi Homing
22 Version 1.0
Multi Site Multi Homing
Implementation Details

!
interface Vlan1
no ip address
shutdown
!
interface Vlan200
ip address 172.16.250.254 255.255.255.0
ip ospf hello-interval 1
ip ospf dead-interval 3
!
router ospf 500
log-adjacency-changes
area 252 nssa
network 172.16.250.0 0.0.0.255 area 0
network 172.16.252.0 0.0.0.255 area 252
!

The following displays the routes associated with West Coast core. Note the path
preference is that of the West Coast edge switching layer.
WESTCOASTCORE#sho ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default, U - per-user static route, o - ODR
P - periodic downloaded static route

Gateway of last resort is not set

O N2 1.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 2.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 3.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 4.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 20.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 5.0.0.0/8 [110/1] via 172.16.252.100, 00:02:37, GigabitEthernet1/1
O N2 6.0.0.0/8 [110/1] via 172.16.252.100, 00:01:44, GigabitEthernet1/1
172.16.0.0/24 is subnetted, 5 subnets
C 172.16.252.0 is directly connected, GigabitEthernet1/1
O IA 172.16.253.0 [110/12] via 172.16.250.1, 00:26:19, Vlan200
O 172.16.254.0 [110/11] via 172.16.252.100, 00:26:19, GigabitEthernet1/1
C 172.16.250.0 is directly connected, Vlan200
O IA 172.16.251.0 [110/2] via 172.16.250.1, 00:26:19, Vlan200
O N2 7.0.0.0/8 [110/1] via 172.16.252.100, 00:01:45, GigabitEthernet1/1
O N2 8.0.0.0/8 [110/1] via 172.16.252.100, 00:01:45, GigabitEthernet1/1
C 127.0.0.0/8 is directly connected, EOBC0/0
O N2 9.0.0.0/8 [110/1] via 172.16.252.100, 00:01:45, GigabitEthernet1/1
O N2 11.0.0.0/8 [110/1] via 172.16.252.100, 00:01:45, GigabitEthernet1/1
O N2 12.0.0.0/8 [110/1] via 172.16.252.100, 00:01:45, GigabitEthernet1/1

BGP Attribute Tuning
In Internet edge topologies, controlling the outbound routes is first and foremost. This is how your
network topology is seen by the world. Which also suggests that, by default, this is how traffic returns
to your site. Controlling the outbound traffic of the topology allows you to manipulate the amount of
traffic that comes in from one ISP or another. In detail, if you wanted to define that all traffic leaves your

Multi Site Multi Homing
Version 1.0 23
Multi Site Multi Homing
Security Considerations

topology from the one ISP link, yet all traffic destined to the topology comes inbound on another ISP
link; you must implement autonomous system prepending. This is most commonly deployed in instances
where you do not want to leave an link idle. For more information regarding route control via BGP, refer
to “Single Site Multi Homing.”

Security Considerations
Security is a necessity in all network architectures today, regardless of your Internet connectivity. Proper
requirements must be taken to ensure that the network architecture and the network devices are securely
provisioned and managed.
Internet edge security is discussed in “Internet Edge Security Design Principles” and “Internet Edge
Security Implementation.” This section provides a brief summary from those guides of the security
functions supported within Internet edge designs. These functions include:
• Element Security - The secure configuration and management of the devices that collectively define
the Internet edge.
• Identity Services - The inspection of IP traffic across the Internet edge requires the ability to identify
the communicating endpoints. Although this can be accomplished with explicit user/host session
authentication mechanisms, usually IP identity across the Internet edge is based on header
information carried within the IP packet itself. Therefore, IP addressing schemas, address
translation mechanisms, and application definition (IP protocol/port identity) play key roles in
identity services.
• IP Anti-Spoofing - This includes support for the requirements of RFC-2827, which requires
enterprises to protect their assigned public IP address space; and RFC-1918, which allows the use
of private IP address spaces within enterprise networks.
• Demilitarized Zones (DMZ) - A basic security policy for enterprise networks is that internal network
hosts should not be directly accessible from hosts on the Internet (as opposed to replies from Internet
hosts for internally initiated session, which are statefully permitted). For those hosts, such as web
servers, mail servers, VPN devices, etc., which are required to be directly accessible from the
Internet, it is necessary to establish quasi-trusted network areas between, or adjacent to both, the
Internet and the internal enterprise network. Such DMZs allow internal hosts and Internet hosts to
communicate with DMZ hosts, but the separate security policies between each area prevent direct
communication originating from Internet hosts from reaching internal hosts.
• Basic Filtering and Application Definition - Derived from enterprise security policies, ACLs are
implemented to provide explicitly permitted and/or denied IP traffic which may traverse between
areas (Inside, Outside, DMZ, etc.) defined to exist within the Internet edge.
• Stateful Inspection - Provides the ability to establish and monitor session states of traffic permitted
to flow across the Internet edge, and deny that traffic which fails to match the expected state of an
existing or allowed session.
• Intrusion Detection - The ability to promiscuously monitor network traffic across discrete points
within the Internet edge, and alarm and/or take action upon detecting suspect behavior that may
threaten the enterprise network.

Multi Site Multi Homing
24 Version 1.0
INDEX

EIGRP 4
A
Element security 1
ACL 14 element security 24
Application distribution 1 exterior gateway protocol 6

B F
basic filtering 1 Firewall Service Module 9
basic filtering and application definition 24 FWSM 9, 11, 15, 16, 17, 19
BGP 6, 9, 10, 12, 13, 14
BGP attributes 7
BGP attribute tuning 23
H
BGP routing table 6 hot standby router protocol 4
border gateway protocols 6 HSRP 4, 5, 9
border routers 10 HSRP dead timers 5

C I
CAM 5 Identity services 1
content addressable memory 5 identity services 24
IGP 9, 10
IGPs 6
D
injecting partial BGP routes 14
default routing 6 interautonomous system routing 6
Demilitarized zones 1 interior gateway protocols 6
demilitarized zones 24 intra-autonomous system routing 7
device redundancy 4 intrusion detection 1, 24
DMZ 1, 24 IP anti-spoofing 1
DNS propagation 2 ISP peering point 3
dynamic routing 6 ISP resiliency 1

E L
EBGP 15 Layer 3 features 4
EGP 6
Multi Site Multi Homing
Version 1.0 25
Index

M S

MAC address 5 SEND_DEFAULT_IF 15
MHSRP 4 stateful inspection 24
multi 1 static routing 6
multigroup hot standby router protocol 4
multi-homing 1
multi-site internet edge topologies 4
multi-site topologies 1

N

NAT 11
network address translation 11
not so stubby area 14
not-so stubby area 9
NSSA 14

O

OPSF 9
OSPF 10, 12, 14, 16
OSPF cost 21
OSPF dead-interval timers 13
OSPF hello timers 13
OSPF interface timer configurations 18

P

P anti-spoofing 24
pass-through autonomous system routing 7
PIX firewall 8

R

Replication timeouts 2
route-map 18
routing protocol metric tuning 4

Multi Site Multi Homing
26 Version 1.0