Cisco WAAS

(Wide Area Application Services)

Technical Overview

Philip Nedev - SE pnedev@cisco.com

WAAS4.0

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco Public

1

1

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

2

The WAN Is A Barrier To Consolidation
• Applications are designed for LAN environments
High bandwidth Low latency Reliability

Round Trip Time (RTT) ~ 0mS
LAN Switch Client Server

• WAN characteristics hinder consolidation
Round Trip Time (RTT) ~ many many milliseconds

Already congested Low bandwidth Latency Packet Loss
Client LAN Switch Routed Network LAN Switch Server

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

3

Bandwidth
• Bandwidth constraints keep applications from performing well • Too much data and too small of a pipe causes congestion, packet loss, and backpressure

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

4

Packet Loss, Congestion, and Retransmission
• Packet loss and congestion cause retransmission which hinders application performance and throughput • Commonly caused by saturated device transmit queues in the network path

Packet Loss Congestion

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

5

The Impact of Packet Loss
4,510 4,010
R

R=

MSS 1 .2 RTT p 0.5
: Average Throughput

3,510
Throughput (Mbps)

MSS: Packet Size RTT: Round-Trip Time P : Packet Loss

3,010 2,510 2,010 1,510 1,010 510 10 0.00001%

0.0001%

0.001%

0.01%

0.1%

1.0%

Packet Loss Probability
Assuming 1250-Byte packet size, and 100ms RTT
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

6

Latency
• Latency impairs application performance in three ways:
Network latency – the amount of time necessary for a message to traverse the network Transport latency – the amount of time necessary for the transport mechanism (TCP) to acknowledge and retransmit data Application latency – “chattiness” of an application protocol causing messages to be exchanged across the network

Round Trip Time (RTT) ~ many many milliseconds

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

7

The Impact of Latency
R=
R

MSS 1 .2 RTT p 0.5
: Average Throughput

MSS: Packet Size RTT: Round-Trip Time

Throughput

Expected Expected
1.544Mbps

P

: Packet Loss

Actual Actual

500Kbps

80 ms Round Trip Time (RTT)
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

8

Need for Application-Specific Acceleration
• Many application protocols can not be adequately optimized through simple compression and transport optimizations alone • Application protocols are commonly developed in “utopian” environments, i.e. the client and the server are on the same LAN or very close to one another • Application-induced or protocol-induced latency and unnecessary data transfers hinder overall enduser performance

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

9

Need for Application-Specific Acceleration
• The Common Internet File System (CIFS) and Network File System (NFS) are two examples of such protocols • CIFS and NFS make a portion of a local file system network accessible, and must maintain all of the semantics of the local file system itself, including:
User (or process) authentication and authorization Information security Locating information, directory traversal File access control and locking semantics I/O operations, including open, close, read, write, seek
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

10

Need for Application-Specific Acceleration
• The result is that hundreds upon thousands of messages must traverse the network before any usable data is served or function is completed!
Protocol version selection User authentication User authorization Meta data operations Find file

File open, FID Lock segment ranges Read data

Write data

Close file

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

11

Need for Application-Specific Acceleration
• Applying compression to communications between the client and server certainly minimizes the amount of bandwidth consumed by each protocol message
But many hundreds or thousands messages must still go back and forth across the WAN in sequence!

• Applying transport optimizations to communications between the client and server improves the ability of each message to efficiently and fully utilize available network capacity
But many hundreds or thousands of messages must still go back and forth across the WAN in sequence!

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

12

Need for Application-Specific Acceleration
• In this simple example of a 1MB Word document open, over 1000 messages are exchanged • With a 40mS RTT WAN, this equates to over 52 seconds of “wait” time before the document is usable!
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

13

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

14

Cisco WAAS Overcomes the WAN
• Cisco WAAS is a solution that leverages a hardware footprint (WAE) in the remote office and in the data center to overcome application performance problems in WAN environments
Remote Office

Data Center
Optim WAN Optim iized zed Co Conne nnecti ctions ons
ns ec io nnecttio ns ized Co nn ed Co Opttim iz Op im

Remote Office

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

15

Cisco WAAS Enables Consolidation
• Cisco Wide Area Application Services (WAAS)
Transparent integration Robust optimizations Auto discovery

• Infrastructure Consolidation
Remove costly servers Centralize data protection Save WAN resources
WAN

• Application Acceleration
Application adapters Advanced compression Throughput optimizations Policy-based configuration
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Server and Server and storage storage infrastructure infrastructure

Cisco Cisco WAE WAE

CISCO Public

16

Cisco WAAS Print Services
• Centrally Managed Print Services
Print driver distribution Client driver download repository Status and health reporting

• Supports Any Printer
Full feature compatibility Job control and status monitoring Guest and disconnected printing

• Print Server Configuration
Network parameters (IP, name, etc) Queue definition and ACLs

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

17

WAAS Accelerates Broad Range of Applications
Application
File Sharing • • • Email • • Internet and Intranet Data Transfer Software Distribution • • • • • Database Applications • • Data Protection • • • UNIX (NFS) Exchange (MAPI) SMTP/POP3, IMAP Notes HTTP, HTTPS, WebDAV FTP SMS Altiris SQL Oracle Notes Backup Applications Replication Applications Any TCP-based Application • 2X-10X • 2X-10X • • • 2X-50X 2X-50X 2X-100X • 2X-50X

Protocol
Windows (CIFS) •

Typical Improvement
2X-100X

Other

2X-10X

* Performance improvement varies based on user workload, compressibility of data, and WAN characteristics and utilization. Actual numbers are case-specific and results may vary.
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

18

Cisco WAAS Performance – File Services
Operations over T1 (1.544Mbps), 80mS RTT
20 Seconds Opening 5-MB PowerPoint Saving 5-MB PowerPoint Drag and Drop of 5MB PowerPoint 40 Seconds 60 Seconds 80 Seconds

Legend Operation Over Native WAN First Operation with WAAS Future Operation with WAAS 20 Seconds 40 Seconds 60 Seconds 80 Seconds

Download of 8MB Package Microsoft SMS Operation over native WAN

Legend First operation with WAAS, no preposition First operation with WAAS, with preposition Future operation with WAAS

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

19

Cisco WAAS Optimization Architecture

L7: Application Optimization

Video Web Video Web Unified Management Unified Management

Local Enterprise File Local File Enterprise Email Email Services Services Apps Services Services Apps

Other Other Apps Apps

L4: Transport Optimization

Content Content Distribution Distribution

TCP Flow TCP Flow Optimizations Optimizations (TFO) (TFO)

Persistent Persistent Session-Based Session-Based Compression Compression

Data Data Redundancy Redundancy Elimination Elimination (DRE) (DRE)

Application Classification and Policy Engine Application Classification and Policy Engine Logical and Physical Integration Logical and Physical Integration Security Security Monitoring Monitoring Quality of Service Quality of Service

Network Infrastructure

Core Routing & Switching Services Core Routing & Switching Services

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

20

WAAS File Services Introduction
• Cisco Wide Area Application Services provides the industry’s most innovative and robust file services optimizations:
Application protocol proxy (CIFS, NFSv2) to handle protocol message workload at the edge to mitigate the impact of latency Application data and meta data cache to serve usable content at the edge to mitigate unnecessary data transfers when safe Network compression (DRE, LZ) to minimize bandwidth usage during data transfer scenarios (read or write) TCP optimizations (TFO) to improve utilization of the available network capacity

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

21

WAAS File Services Introduction
• • Intelligent local handling and optimization of Intelligent local handling and optimization of protocol mitigates latency protocol mitigates latency • • File caching to remove the need to File caching to remove the need to unnecessarily transfer files, validation unnecessarily transfer files, validation ensures stale data is never served ensures stale data is never served • • Transparent integration ensures no client or Transparent integration ensures no client or server changes to apply optimization server changes to apply optimization • • Sessions maintained end-to-end ensures Sessions maintained end-to-end ensures no security reconfiguration no security reconfiguration • • Auditing, access-control, and quotas are Auditing, access-control, and quotas are fully preserved fully preserved • • Scheduled preposition to prepopulate Scheduled preposition to prepopulate DRE and edge data cache DRE and edge data cache

WAN
Files

FILE.DOC
Cache

• • Disconnected mode of operation Disconnected mode of operation allows R/O access to fully-cached allows R/O access to fully-cached content when the server is unreachable content when the server is unreachable

• • Advanced WAN optimization layer Advanced WAN optimization layer improves throughput and efficiency improves throughput and efficiency • • Data Redundancy Elimination (DRE) Data Redundancy Elimination (DRE) eliminates redundant network data eliminates redundant network data • • TCP optimizations to improve TCP optimizations to improve protocol ability to fully utilize network protocol ability to fully utilize network
CISCO Public
22

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

WAAS File Services Introduction
Branch Office
IT
Backup

Regional Office
IT
File Backup Cache NAS DAS DAS Files

File
Cache NAS DAS DAS
Files

Data Center Remote Office
IT IT

Backup

NAS SAN Files

WAN

File Cache NAS DAS DAS
Files

Backup

Decentralized Optimized Centralized andStorage Centralized Storage
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

23

Intelligent Message Suppression
• IMS provides latency reduction
Eliminate unnecessary message transfer and minimize WAN RTTs Batch composite commands Message prediction and pre-fetch
IP Network
WAN Optimization DRE/TFO/LZ Cache and Protocol Proxy ~90% msgs 10% actual storage

• File performance optimizations
Read-ahead caching during file access to increase read cache hits Asynchronous write-behind caching when safe, synchronous writebehind to ensure file integrity

NAS

Origin Server 100% of capacity

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

24

Application-Specific Acceleration
• Application and protocol awareness
Eliminate unnecessary chatter Save WAN bandwidth Pre-populate edge cache as necessary Enable disconnected operations
Application Specific Acceleration Safe Caching Read-ahead Prediction Batching

• Intelligent protocol acceleration
Read-ahead, prediction, and batching Safe data and metadata caching Improves application response time Provide origin server offload
WAN
WAN Optimization DRE/TFO/LZ

• WAASv4 application adapters
CIFS (Windows File Services) Windows printing
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Origin Server Offloaded

CISCO Public

25

Data Caching and Integrity
• Edge file segment caching and meta data caching
Data cached on-demand as files or directories are opened Prepopulation of edge cache via CDN-like preposition

• Coherency, concurrency, and ACL
Cache validation guarantees no stale data served File locking and AAA handled synchronously with server

OPEN OPEN FILE.DOC FILE.DOC
FILE.DOC

IP Network

NAS

Files
AAA, OPEN, LOCK AAA, OPEN, LOCK APPROVED, LOCKED, VALIDATED APPROVED, LOCKED, VALIDATED

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

26

Integration with WAN Optimization
• File services adapter leverage WAN optimization capabilities provided by DRE, TFO, and LZ
DRE and LZ improves open and save operation performance through compression and data suppression TFO enables the protocol to more effectively, efficiently utilize available WAN resources

WAN
FILE.DOC

DRE CACHE

DRE CACHE

LZ

LZ

FILE.DOC

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

27

Intelligent File Preposition
• Intelligent preposition capabilities with flexible configuration to prepopulate cache with files before the first user request • Leverage Data Redundancy Elimination (DRE) and LZ compression to improve transfer performance and user save performance
IP Network

NAS

Files

Distribute Distribute FILE.DOC FILE.DOC at 3am at 3am
FILE.DOC

Fetch Fetch FILE.DOC FILE.DOC

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

28

File Blocking
• Cisco WAAS can be configured to prevent specific types of files from being stored on the data center file server or NAS device • Prevent non-desirable file types from consuming precious WAN resources, improve productivity
Save Save SONG.MP3 SONG.MP3 IP Network

NAS

X
MP3
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Files

CISCO Public

29

File Services Flexible Integration Options
Non-Transparent Using Published Names, DFS Compatible Data Center
NAS

Branch1

Windows Client

WAN

\\Pluto\Demo

Core WAE Name: Core1

Edge WAE Name: BR1Cache

\\BR1-Pluto\Demo

Transparent Using WCCPv2 or Policy-Based Routing Data Center
Router
NAS

Branch1

Windows Client

WAN

\\Pluto\Demo

Core WAE Name: Core1
© 2005 Cisco Systems, Inc. All rights reserved.

\\Pluto\Demo
Edge WAE
CISCO Public
30

CSBU-EBC

Integration Example – Software Distribution
• Transparently optimize CIFS to a remote software distribution server to provide LAN-like access to hotfixes, service-packs, and other updates • Preposition to pre-populate edge cache with large packages that users will request to improve download and installation performance
Data Center
Router
NAS

Branch Office
WAN

Download Download XP-SP2.msi XP-SP2.msi from \\pluto from \\pluto

\\Pluto\SWUpdates

Distribute Distribute XP-SP2.msi XP-SP2.msi

XP-SP2

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

31

Print Services Network Integration
• Central Manager WAEs register as an Active Directory computer and provide print driver repository and driver distribution functionality • Edge WAEs register as an Active Directory computer and provide local print services to an office
Data Center
WAN
DC

Branch Office

PRINT

PRINT

Central Manager WAE Central Print Server PhantomManager WAE Phantom Print Server
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Edge WAE Edge WAE Phantom Print Server Phantom Print Server
CISCO Public
32

Cisco WAAS Print Services
• Many organizations have difficulty consolidating file services because of the WAN burden that would be created due to print services traffic • Cisco WAAS provides Windows-compatible print services to eliminate the need for print jobs to traverse the WAN
Data Center
NAS

Branch Office
Router
WAN

JOB JOB
FILE

Driver Driver Distribution Distribution

Print Print FILE.DOC FILE.DOC

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

33

Print Driver Distribution
• Print drivers are uploaded to the Central Manager WAE and then distributed to edge print servers or groups of devices • Printer drivers are then accessible at the edge of the network for local download (PRINT$ share) to support “Click-N-Print” functionality
Data Center
WAN

Branch Office
JOB JOB
PRINT FILE

Download Download Driver Driver and and PRINT! PRINT!

DC

Upload Upload Drivers Drivers

PRINT

Distribute HP Distribute HP LaserJet LaserJet Driver Driver

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

34

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

35

Seamless, Transparent Integration
• Integration into the network fabric with high availability, loadbalancing, and failover regardless of interception mechanism
Physical inline WCCPv2 Policy-Based Routing CSM/ACE Modules
Src Mac AAA Dst Mac BBB Src IP 1.1.1.10 Src TCP 15131 Dst IP 2.2.2.10 Dst TCP 80

APP DATA

• Compliance with network valueadded features
Preservation of packet headers Classification - QoS, NBAR, Queuing, Policing, Shaping Security - Firewall policies, Access Control Lists Reporting - NetFlow, monitoring
Src Mac BBB Dst Mac AAA Src IP 1.1.1.10 Src TCP 15131 Dst IP 2.2.2.10 Dst TCP 80 optimized

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

36

Cisco WAE Physical Inline Deployment
• Physical inline interception
Physical in-path deployment between switch and router or firewall Mechanical fail-to-wire upon hardware, software, or power failure Requires no router configuration

• Scalability and high availability
Two two-port groups Serial clustering with load-sharing and fail-over Redundant network paths and asymmetric routing

• Seamless integration
Transparency and automatic discovery 802.1q support, configurable VLANs Supported on all WAE appliances
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Cisco WAE 4-port inline card

WAN WAE1
CISCO Public
37

Inline Interception Deployment Modes

In-path, single WAE, single WAN connection
MGMT WAN WAE1

In-path cluster, single WAN connection
MGMT WAN WAE1 WAE2

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

38

Inline Interception Deployment Modes
In-path, single WAE, redundant WAN links
WAN MGMT

WAN WAE1

In-path cluster, redundant WAN links
WAN MGMT

WAN WAE1 WAE2

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

39

Cisco WAE WCCPv2 Deployment
• WCCPv2 interception
Out-of-path with redirection of flows to be optimized (all flows or selective via redirect-list) Automatic load-balancing, load redistribution, fail-over, and failthrough operation
Original Original Flow Flow

• Scalability and high availability
Up to 32 WAEs within a service group and up to 32 routers Linear performance and scalability increase as devices are added
Interception Interception Redirection Redirection Service Service Group Group

• Seamless integration
Transparency and automatic discovery Supported on all WAE platforms
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Optimized Optimized Flow WAN Flow

CISCO Public

40

Cisco WAE PBR Deployment
• Policy-Based Routing (PBR)
Out-of-path with redirection of flows to be optimized (all flows or selective via access-list) WAE treated as a next-hop router
Original Original Flow Flow

• High availability
Failover capability allows a secondary WAE to be used should the primary WAE fail IP SLAs ensure availability by tracking WAE liveliness

Policy Route Policy Route WAE = Next Hop WAE = Next Hop

• Seamless integration
Transparency and automatic discovery Supported on all WAE platforms
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Optimized Optimized WAN Flow Flow

CISCO Public

41

Cisco WAE ACE Deployment
• Application Control Engine (ACE)
Industry-leading scalability and performance for the most demanding data center networks Supports up to 16Gbps throughput, 4M concurrent TCP connections, and 350K connections/sec setup
WAN

Optimized Optimized Flow Flow

• Seamless integration
Fully integrated with the Catalyst 6500 series of intelligent switches Transparency and automatic discovery Supported on all WAE appliances

Catalyst Catalyst 6509 w/ 6509 w/ ACE ACE

Original Original Flow Flow

• Industry Leading Functionality
Solution for scaling servers, appliances, and network devices Virtual partitions, flexible resource assignment, security, and control
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

42

Cisco WAAS Auto-Discovery
• Cisco WAE devices automatically discover one another and negotiate optimization capabilities
Performed per TCP connection Flexible optimization configuration using ATP Exchange of peer capabilities and limitations
A A
WCCPv2 WCCPv2 or PBR or PBR WCCPv2 WCCPv2 or PBR or PBR

B B

WAN

A:B TCP SYN A:B TCP SYN

B:A TCP SYN/ACK B:A TCP SYN/ACK

A:B TCP SYN SYN/ACK B:A TCP B:A TCP A:B TCP SYN SYN/ACK (marked) (marked) (marked) (marked) ACCELERATION II would like ACCELERATION would like CONFIRMED! to accelerate CONFIRMED! to accelerate this connection! this connection! Here are my details Here are my details

A:BB:A TCP SYN/ACK TCP SYN A:BB:A TCP SYN/ACK TCP SYN (marked) (marked)

WAE1

WAE2

II know WAE1 is Acknowledge know WAE1 is Acknowledge in the path, let’s Acceleration! in the path, let’s Acceleration! Here are my details accelerate! Here are my details accelerate!

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

43

Non-Transparent Optimization Challenges
• Complex configuration and possibility of human error
Doubles network management effort Requires management of two routing topologies Requires management of duplicate feature configuration
WAN

Branch Offices

• Compromises network features of upstream routers, switches, and firewalls
Loss of visibility at L3/L4 Firewall policies, ACLs QoS, NBAR NetFlow
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Data Center
File/Print Email Application Servers Backup

CISCO Public

44

Network Integration Overview
• With the exception of inline, Cisco WAEs attach to the LAN as an appliance • Relies on packet interception/redirection to enable application acceleration and WAN optimization
Interception in each site where deployed Interception in both directions of packet flow

• Transparent optimizations maintain compatibility with most IOS features and other platforms

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

45

Use of Tertiary IFs or Sub-IFs
• With non-inline modes, the WAE must not be attached to the same segment as the interface performing redirection • This is required to avoid routing loops, as we have no way to notify the router to bypass the interception and redirection (shown below)
PBR or WCCPv2 IP Network

Infinite Loop

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

46

Use of Tertiary IFs or Sub-IFs (Cont.)
Tertiary Interface PBR or WCCPv2 IP Network

Fa0/0 Fa1/0 Redirect Exclude

Sub-Interface

PBR or WCCPv2 Fa0/0.10 Fa0/0.20

IP Network

Redirect Exclude

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

47

One-Arm Deployment Example
11 66 22 33 44 55
IP Network

• Pros
Simplicity Single interface

• Cons
Performance constrained Higher router utilization

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

48

Two-Arm Deployment Example
11 22 33 44

IP Network

55

• Pros
Better performance Lower router utilization

• Cons
Add’l switch port consumed Add’l configuration Interface adjacency to node Usually feasible in branch only

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

49

Hierarchical Network Placement Considerations
Locality is key • High locality to the core yields a more global level of optimization
Intercept traffic going to/coming from the WAN exclusively (based on placement of interception) Closer to the WAN entry/exit prevents intrasite access from traversing the WAEs Provide optimization for all attached distribution, access layers

• High locality to the access layer yields a more focused level of optimization
Optimization restricted to a specific access layer unless significant changes to network routing are introduced May cause intrasite access to traverse the WAEs which causes unnecessary WAE resource utilization

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

50

Hierarchical Network Placement Considerations
• Core layer typically reserved for high performance forwarding • Distribution layer provides optimal deployment location for WAAS – close to core, aggregation for downstream access layers • Access layer can be used, but too contained to be used for large scale optimizations
Core

Distribution

Access

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

51

IOS Versions for WCCPv2

Per KB article #1011 http://acpluto/kb/waas/kb.asp?action=article_show&articleID=1011
© 2005 Cisco Systems, Inc. All rights reserved.

CSBU-EBC

CISCO Public

52

WCCPv2 Interception Considerations
• WAAS uses service groups 61 and 62 for traffic interception and redirection
Service group 61 – hash bucket assignment based on source IP address of the packet Service group 62 – hash bucket assignment based on destination IP address of the packet

• One service group needs to be in the path of traffic for each direction of traffic flow
Ingress interception (preferred) – analyze, intercept, and redirect as packets enter an interface – less CPU utilization Egress interception – analyze, intercept, and redirect as packets prepare to exit an interface – higher CPU utilization

• Placement of the services should not be overlooked

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

53

WCCPv2 Configuration – Routers
Recommended
62/in LAN and 61/in WAN 62/in LAN and 61/in WAN keeps flows to a particular server keeps flows to a particular server pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow yielding directions of traffic flow yielding better likelihood of compression better likelihood of compression per server per server Load-balancing based on nodes Load-balancing based on nodes outside of the location outside of the location
LAN WAN

61/in 62/in

61/in LAN and 62/in WAN 61/in LAN and 62/in WAN keeps flows from a particular client keeps flows from a particular client pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow yielding directions of traffic flow yielding better likelihood of compression per better likelihood of compression per client client Load-balancing based on nodes Load-balancing based on nodes within the location within the location

LAN

WAN

62/in
61/in 62/in

• Note: most routers only support GRE-redirect, GRE-return, and hash assignemnt, • Note: most routers only support GRE-redirect, GRE-return, and hash assignemnt, which are default WCCP service configuration parameters which are default WCCP service configuration parameters
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

54

WCCPv2 Configuration – Router Isolation
Branch: 62/in LAN and 61/in WAN Branch: 62/in LAN and 61/in WAN keeps flows to a particular server keeps flows to a particular server pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow yielding directions of traffic flow yielding better likelihood of compression better likelihood of compression per server per server Load-balancing based on nodes Load-balancing based on nodes outside of the location outside of the location
LAN WAN

DC: 62/in WAN1 and 61/out WAN1 DC: 62/in WAN1 and 61/out WAN1 keeps flows to a particular server keeps flows to a particular server pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow yielding directions of traffic flow yielding better likelihood of compression better likelihood of compression per server per server No ACLs required to not redirect No ACLs required to not redirect flows to/from unoptimized branch flows to/from unoptimized branch Load-balancing based on nodes Load-balancing based on nodes outside of the location outside of the location

WAN1 IP Network WAN2

LAN

61/in 62/in

61/out

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

55

WCCPv2 Configuration – Switches
Recommended
61/in LAN and 62/in WAN 61/in LAN and 62/in WAN keeps flows from a particular keeps flows from a particular server server pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow yielding directions of traffic flow yielding better likelihood of compression better likelihood of compression Load-balancing based on nodes Load-balancing based on nodes within the location within the location
IP Network

62/in LAN and 61/in WAN 62/in LAN and 61/in WAN keeps flows to a particular client keeps flows to a particular client pinned to the same WAE in both pinned to the same WAE in both directions of traffic flow directions of traffic flow Load-balancing based on nodes Load-balancing based on nodes outside of the location outside of the location

62/in 61/in

• Note: configuration on switches is configured on L3 interfaces or SVIs only. Configure • Note: configuration on switches is configured on L3 interfaces or SVIs only. Configure with appropriate parameters (L2-redirect, L2-return, mask assignment) with appropriate parameters (L2-redirect, L2-return, mask assignment)

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

56

WCCPv2 Availability Monitoring
• WCCPv2 keepalive (heartbeat) information is exchanged every 10 seconds between WAEs and the router(s) • Should a WAE be unresponsive for three consecutive heartbeats, it is removed from the service group • WCCPv2 heartbeat is stateful and process-based

A

B

C

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

57

WCCPv2 Failover
• Should a WAE within a service group fail, the portion of the load that it was handling is automatically distributed to other WAEs within the service group • Should no additional WAEs be available, the service group is taken offline, and packets are not redirected
Buckets 86–128 Buckets 1–85 Buckets 86–170 Buckets 129–170 Buckets 171–255

X
A B C
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

58

Which Interception Method to Use?
WCCPv2 Inline CSM/ACE PBR

Number of Active WAEs Maximum Number of WAEs Maximum Number of TCP Connections
(with WAE-7326)

32

2
(serial cluster, tested limit)

16000
(not practical but possible)

1

32

2
(serial cluster, tested limit)

16000
(not practical but possible)

8
(IOS dependent)

240K

15K

4M

7.5K

Maximum Throughput Recommended Use

Up to 32Gbps
(platform dependent)

Up to 2Gbps
(two inline pairs)

Up to 16Gbps
(platform dependent)

Up to 1Gbps

Generally Recommended

Only if WCCPv2 can not be used
(SP managed or low-end router)

Very large scale data center deployments

Last resort

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

59

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

60

Networks Without Compression

Congestion! Congestion!

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

61

Data Transfer Without Compression

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

62

Networks With Compression
No Congestion No Congestion or Less Congestion or Less Congestion

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

63

Data Transfer With Compression

WAN

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

64

Cisco WAAS Advanced Compression
• Data Redundancy Elimination (DRE): application-agnostic compression eliminates redundant data from TCP streams providing up to 100:1 compression • Persistent LZ Compression: session-based compression provides up to an additional 10:1 compression even for messages that have been optimized by DRE

LZ

LZ

DRE

DRE

Synchronized DRE Context

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

65

Advanced Compression Block Diagram
Signature Matching Add New Entries DRE FIFO Cache Synchronization Cache Synchronization DRE FIFO Signature Matching Add New Entries

Fingerprint Chunk Identification LZ LZ

Fingerprint Chunk Identification

TCP Proxy

TCP Proxy

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

66

DRE Chunk Identification
• Analyze incoming data streams using a sliding window to identify “chunks” • Each chunk assigned a 5-byte signature • Single-pass used to identify chunks at multiple levels
Basic chunks Chunk aggregation (nesting)

Fp mod
No boundary found

Fp mod
No boundary found

Fp mod
No boundary found

Fp mod
No boundary found

Fp mod
Boundary identified!

• After chunks are identified, DRE will begin pattern matching
Chunk1 5B Sig Fp mod

First look for largest chunks Look for smaller chunks if necessary

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

67

DRE Chunk Identification

Level-0 Chunk Level-0 Chunk “Basic Chunk” “Basic Chunk” ~256 bytes ~256 bytes

Level-1 Chunk Level-1 Chunk ~1024 bytes ~1024 bytes

Level-2 Chunk Level-2 Chunk ~4096 bytes ~4096 bytes

Level-3 Chunk Level-3 Chunk ~16384 bytes ~16384 bytes

Original Data

• Each chunk is assigned a 5-byte signature

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

68

DRE Pattern Matching

DRE Database

NO MATCH NO MATCH NO MATCH

Original Original Message Message

NO MATCH

Encoded Encoded Message Message

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

69

TFO Transport Flow Optimization

WAAS4.0

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco Public

70

70

Terminology – Maximum Window Size (MWS)

4

3

2

1

ACK

• The Maximum Window Size (MWS) is the maximum amount of a data a node can have outstanding in the network unacknowledged • The node can not continue transmission until previous transmissions have been acknowledged
Problematic over LFNs – Long Fat Networks “elephants” Inability to fully utilize the available network resources

• Generally, ACKnowledgements are sent when an entire TCP window has been received • Upon encountering packet loss, the node would be required to retransmit the entire window of data (guaranteed delivery)
Problematic over low-speed links Problematic with large windows

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

71

Terminology – Bandwidth Delay Product (BDP)
RTT 10 ms Bandwidth 155 Mbps (OC-3) Amount of data that can be in transit at any one point in time: 155Mbps x 10 ms = 192 KB

• The Bandwidth Delay Product (BDP) of a network defines the amount of data that can be in flight within a network at any one point in time

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

72

Challenge
• Common TCP implementations on client and server operating systems can be bottlenecks to application performance
Inability to fill-the-pipe, i.e. utilize available bandwidth Inefficient recovery from packet loss, retransmission Bandwidth starvation for short-lived connections

• Cisco WAAS Transport Flow Optimization (TFO) utilizes industry-standard TCP optimizations to remove these application performance barriers

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

73

Cisco WAAS Transport Flow Optimizations
• Cisco WAAS Transport Flow Optimizations (TFO) is designed to overcome common challenges associated with standard TCP implementations
Window Scaling – capitalize on available bandwidth Large Initial Windows – maximize transmission after connection establishment Selective Acknowledgement – efficient packet loss recovery and retransmission mechanisms Binary Increase Congestion (BIC) – quick return to maximum throughput upon encountering congestion

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

74

TCP Maximum Window Size (MWS)
• MWS (maximum window size) determines the maximum amount of data that can be in transit and unacknowledged at any given time • BDP (bandwidth delay product) defines the amount of data that can be contained within a network at any given time
If MWS > BDP, then application may not be throughput bound (i.e. application can “fill the pipe”) If BDP > MWS, then application will not be able to fully utilize the network capacity (i.e. application can not “fill the pipe”)

• Does not account for application-layer (L7) latency such as found with protocol-specific messaging
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

75

Link Utilization and MWS, BDP
BDP BDP

Unusable network capacity! Unusable network capacity! Bandwidth

MWS MWS Link Utilization Link Utilization

Latency
CISCO Public
76

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

WAAS TFO Window Scaling
• Cisco WAAS TFO window scaling (based on RFC 1323) scales the TCP window to 2MB to overcome problems with filling LFNs (Long Fat Networks) • Window Scaling applies a binary shift to the decimal value supplied in the data field

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

77

Link Utilization After Window Scaling
BDP BDP Cisco WAAS TFO Cisco WAAS TFO Able to fill the pipe! Able to fill the pipe! Bandwidth

Original MWS Original MWS

Latency
CISCO Public
78

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco WAAS Large Initial Windows
• While 80% of network traffic is typically associated with long-lived connections (elephants), approximately 80% of network connections are short-lived (mice) • Short-lived connections transmit smaller numbers of packets and are torn down before ever leaving the slow-start phase of TCP • Cisco WAAS Large Initial Windows, based on RFC3390, increases initial window size to expedite entry into congestion avoidance mode for high throughput

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

79

Cisco WAAS Large Initial Windows
Packet Loss Packet Loss

Segments per Round Trip (cwnd)

TFO TFO

Slow-Start Slow-Start (discovery) (discovery)

Congestion Congestion Avoidance Avoidance (high-throughput) (high-throughput)

TCP TCP

Round Trips

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

80

Standard TCP Acknowledgement
• Standard TCP implementations acknowledge receipt of data by acknowledging the entire window has been received • Loss of a packet causes retransmission of the entire TCP window, causing performance degradation as the window becomes larger
Transmit 1 2 3 Receive 1 2 ACK Retransmit 1 2 3 ACK 1 1 2 2 3 3
CISCO Public
81

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Cisco WAAS Selective Acknowledgement
• Cisco WAAS employs TCP extensions to improve acknowledgement of transmitted data, improve delivery of missing segments, and unnecessary minimize retransmission • Based on RFC 2018 and extensions

Transmit 1 2 3 ACK Retransmit 3 ACK

Receive 1 1 2 2

1 1

2 2

3 3

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

82

Without TCP Proxy

WAN

X TIMEOUT! RESEND ?

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

83

TCP Proxy and TFO

WAN

Window Scaling Large Initial Windows Congestion Mgmt Improved Retransmit

X

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

84

Standard TCP Congestion Avoidance
• Standard TCP implementations employ an exponential slow start to increase throughput to the slow start threshold • From the slow start threshold, the congestion window is increased linearly by one packet per round-trip until packet loss is encountered • Upon encountering packet loss, the congestion window is cut in half to return to a throughput level safe given the congested environment • The net result is “saw-tooth” throughput, and return to maximum throughput can take hours for long-lived connections and LFNs

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

85

Standard TCP Congestion Avoidance
Packet loss causes connection to enter into linear congestion avoidance (+1 cwnd per ACK) cwnd dropped by 50% on packet loss Segments per Round Trip (Congestion Window)

loss

Linear Congestion Avoidance (+1 cwnd per ACK)

loss

Exponential Slow Start (2x pkts per RTT) Low throughput during this period
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Round Trips
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

86

“Saw-tooth” TCP Throughput
Return to maximum throughput could take a very long time!

Packet loss cwnd

Packet loss

Packet loss

Packet loss

TCP

Slow start Congestion avoidance

Time (RTT)

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

87

Binary Increase Congestion (BIC)
• Cisco WAAS employs the Binary Increase Congestion (BIC) congestion avoidance system to improve throughput in lossy environments • Uses a binary search to adaptively increase the congestion window, resulting in a stable and timely return to higher levels of throughput • Decreases congestion window only by 1/8 (rather than 1/2 as compared to TCP) when packet loss is encountered, mitigating the majority of the performance penalty

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

88

WAAS Throughput and Congestion Avoidance
Adaptive Increase to cwnd Adaptive Increase to cwnd cwnd == cwnd + f(cwnd, history) cwnd cwnd + f(cwnd, history)

Cwnd decreased by 1/8 on Cwnd decreased by 1/8 on packet loss vs 1/2 with TCP packet loss vs 1/2 with TCP

Packet loss Packet loss cwnd

Packet loss Packet loss

Packet loss Packet loss

Packet loss Packet loss

Cisco Cisco WAAS TFO WAAS TFO

Slow start Congestion avoidance

Time (RTT)
Standard Standard TCP TCP

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

89

Comparing TCP and TFO

Cisco TFO provides significant throughput Cisco TFO provides significant throughput improvements over standard TCP implementations improvements over standard TCP implementations

cwnd

TFO TFO TCP TCP

Slow start Congestion avoidance

Time (RTT)

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

90

TCP Throughput and Latency Optimizations
• TCP window scaling improves link utilization and throughput • Optimized TCP stack improves recovery and congestion handling • Priority for transactional traffic
Link Utilization Link Utilization Packets per Round-Trip Packets per Round-Trip
Bandwidth scalability Cisco Cisco WAAS TFO WAAS TFO
Link Utilization

• Compatible and friendly to other TCP connections on the network • Large initial windows improves throughput for short-lived connections

Cisco Cisco WAAS TFO WAAS TFO Standard Standard TCP TCP TCPFriendliness

RTT Fairness
Packets/RTT

Standard Standard TCP TCP

Bandwidth
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

Packet Loss Probability
CISCO Public
91

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

92

WAAS Intuitive Central Management
• Comprehensive Management
Central configuration Device grouping Monitoring, statistics Alerts, reporting

• Easy-to-use Interface
Graphical U/I, Wizards IOS CLI Roles-based administration

• Proven Scalability
1000’s of nodes Redundancy and recovery
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

93

Cisco WAE Family Positioning
Enterprise Data Center ACE

Performance

WAE-7326 Regional Office or Small Data Center

WAE-612

Branch or Remote Office WAE-512

NME-WAE-502 NME-WAE-302

Scalability
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

94

Remote Office Hardware Platforms
• NM-WAE Module
Lowest CapEx and OpEx, integrated within the ISR, addresses 80% of remote branch offices Single processor system, can be clustered with WCCPv2, PBR NM-WAE
Router-Integrated Network Module for the Cisco Integrated Services Router

Supported in ISR models 2811, 2821, 2851, 3825, and 3845

• WAE-512 Appliance
Remote office appliance platform Up to 20Mbps WAN connections 1500 optimized TCP connections WAE-512
Remote Office Appliance

250GB RAID-1 disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

Performance and scalability are subjective and may vary based on a variety of Performance and scalability are subjective and may vary based on a variety of conditions. WAE WAN bandwidth is not limited by hardware or software conditions. WAE WAN bandwidth is not limited by hardware or software
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

95

Data Center Hardware Platforms
• WAE-612 Appliance
Regional hub and medium data center deployments Up to 155Mbps WAN connections WAE-612
Regional Hub and Data Center Appliance

6000 optimized TCP connections 300GB RAID-1 SAS disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

• WAE-7326 Appliance
Enterprise data center deployments Up to 310Mbps WAN connections 7500 optimized TCP connections WAE-7326
Enterprise Data Center Appliance

900GB RAID-1 SCSI disk capacity Deploy w/ inline, WCCPv2, PBR, CSM/ACE

Performance and scalability are subjective and may vary based on a variety of Performance and scalability are subjective and may vary based on a variety of conditions. WAE WAN bandwidth is not limited by hardware or software conditions. WAE WAN bandwidth is not limited by hardware or software
CSBU-EBC © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

96

Increased Scalability
• Sizing tool will do the job

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

97

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

98

Summary
• Not all application protocols can be optimized through generic WAN optimization – some require application-specific acceleration to function properly over a WAN • Cisco WAAS provides robust application-specific and network-layer optimizations to enable application delivery and file server consolidation • Cisco WAAS file services provides integration flexibility and can help enable consolidation of additional CIFS or NFSbased platforms such as software distribution servers • Cisco WAAS also provides Windows-compatible and centrally-managed print services with driver distribution and disconnected printing capabilities

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

99

Why Choose Cisco WAAS?
• Application-specific optimization for file and print services helps to enable file server and data protection consolidation while enabling offline access to fully cached files • High performance WAN optimization to reduce bandwidth consumption and maximize throughput, efficiency to significantly improve application delivery over the WAN • Network transparency preserves investment in existing network feature configurations and physical integration provides industry’s best total cost of ownership model • Robust and proven secure central management platform scales to meet the needs of the largest organizations • Integration with industry-leading application networking technologies such as the ACE module for data center integration, scalability, and performance • Cisco’s world-class 24x7x365 technical assistance center
CISCO Public
100

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

Agenda
• Enterprise Application Delivery Challenges • Introducing Cisco Wide Area Application Services • Network Integration and Deployment • In-Depth Examination of Optimizations • Management and WAE Platforms • Summary • Q&A

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

101

Questions?

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

102

102

Time for break!

CSBU-EBC

© 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

103

Presentation_ID CSBU-EBC

© 2003, Cisco Systems, Inc. All rights reserved. © 2005 Cisco Systems, Inc. All rights reserved.

CISCO Public

104

104

Sign up to vote on this title
UsefulNot useful