P. 1
BRKSAN-3707

BRKSAN-3707

|Views: 2,872|Likes:
Published by jeffgrantinct

More info:

Published by: jeffgrantinct on Mar 10, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

08/21/2011

pdf

text

original

Sections

Advanced SAN Design— Virtualization Technologies and Intelligent Applications Design Considerations

BRKSAN-3707

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

2

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

1

Agenda
Brief Review of Virtual Fabrics
Virtual Fabrics (VSANs) Port-Channels, Trunking and IVR

Virtualization Technologies
SAN Device Virtualization (SDV) N-Port ID Virtualization (NPIV) N-Port Virtualizer (NPV) FlexAttach

Intelligent Application
Data Mobility Manager (DMM) Storage Media Encryption (SME) SANTap Storage Virtualization
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

3

Virtual Fabrics

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

4

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

2

Virtual Fabric: Three Key Concepts
Virtual Fabric
Provide independent (‘virtual’) fabric services on a single physical switch

Virtual Fabric Trunking and Port-Channels
Ability to transport multiple virtual fabrics over a single ISL or common group of ISLs

Fabric Routing (IVR)
Ability to provide selected connectivity between virtual fabrics without merging them

Trunk BRCD = Port Channel Cisco Group of ISLs = Port Channel
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

5

Cisco’s Approach to Virtual Fabric: Virtual SAN (VSANs)
A VSAN provides a method to allocate ports within a physical fabric to create virtual fabrics Virtual fabrics created from larger cost-effective physical fabric Reduces wasted ports with islands Fabric events are isolated per VSAN—maintains HA Hardware-based isolation—traffic is explicitly tagged across ISLs with VSAN membership info Statistics gathered per VSAN
Cisco MDS 9000 Family with VSAN Service Physical SAN Islands Are Virtualized onto Common SAN Infrastructure

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

6

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

3

VSAN—MDS Family
Each port on the MDS family exists in a VSAN Up to 256 VSANs in a single switch (hardware can support up to 4095) Logical configuration to move a port from one fabric to another WWN-based VSANs can provide automated VSAN membership Basis for Virtual Fabric Trunking (VFT) Extended Header (ANSI T11 FC-FS-2 section 10)
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

VSAN ‘A’

VSAN VSAN ‘B’ ‘B’

VSAN ‘C’

VSAN ‘D’

7

VSAN Numbering Rules
Configured VSANs

VSAN 1 is the default VSAN
All ports are originally in VSAN1

VSAN 10 VSAN 20 VSAN 30

VSAN 2 through 4093 can be assigned to ‘user’ VSANs—VSAN 0, 4094, 4095 are reserved
Currently 256 VSANs is supported from the range of 2–4093
Enhanced ISL (EISL) Trunk Carries Tagged Traffic from Multiple VSANs

Trunking E_Port (TE_Port) VSAN 30 Is Not Propagated Across EISL Due to Nonexistence on Remote Switch Trunking E_Port (TE_Port) Port Is In VSAN 4094 (Isolated VSAN)
VSAN 10 VSAN 20

VSAN 4094 is a reserved as ‘special’ VSAN
Called the ‘isolated VSAN’ Used to isolate ports whose port-VSAN has been deleted Not propagated across switches Always present, can’t be deleted Always in suspended state
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Host Is Isolated From the Fabric

VSAN 30

Configured VSANS

8

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

4

Standard Fibre Channel Frame Fields
4B

SOF (Start of Frame) VSAN Header

ANSI T11.3 task group is the standard committee working on Virtual Fabrics T11.3 FC-FS-2 fabric services includes virtual fabrics specification
Defines “Extended-Headers” In FC-FS-2 Section 10.2 Defines frame tagging mechanism

8B

24B

FC Header

Applicable to N_Ports, F_Ports and E_Ports
Enables Inter-Switch Link to support trunking virtual interfaces Define the trunking virtual interfaces for end devices (hosts, storages)

0 -> 2112B

Payload

The ANSI T11 FS-SP group has accepted Cisco VSAN as standard (FC-FS-2 Section 10)
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

4B 4B

CRC EOF (End of Frame)
9

VSAN Header Field
R_CTL 8 Ver 2 Frame MPLS More User Type Present Header Priority 4 1 1 3

VSAN Number 12 bits

CDL TTL Present 1 8

# PAD P_VL Bytes 2 4

Rsvd 2

OAM 8

Msg Info 8

Each frame on a VSAN trunk carries an extra 8 bytes of header: User priority—3 bits—used for QoS functionality to designate priority of frame VSAN ID—12 bits—used to mark the frame as part of a particular VSAN—supports up to 4096 VSANs MPLS flag—1 bit—used to designate whether this frame is subject to Multi-Protocol Label Switching processing—future use Time-to-live (TTL)—8 bits—used to help avoid routing loops— standard part of an IP frame Other misc. fields including version, frame type, and other reserved fields
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

10

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

5

Trunking and Port-Channels

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

11

EISLs and TE-Port
1. The Trunking E_Port (TE_Port)
Negotiated between MDS switches—default Carries tagged frames from multiple VSANs Can be optionally disabled to yield E_Port Only understood by MDS switches Also has a native VSAN assignment (for E_Port) Trunk all VSANs (1-4093) by default Not to be confused with Brocade ISL aggregation (trunking)
Cisco MDS 9513 Director with VSAN Service

Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk Carries Tagged Traffic from Multiple VSANs

2. The EISL link
The resultant link created by two connected TE_Ports Superset of ISL functionality Carry individual control protocol information per VSAN (e.g. zoning updates) Can be extended over distance (DWDM, FCIP, etc.)

Trunking Cisco MDS E_Port (TE_Port) 9216 Trunking Fabric with E_Port VSAN Service (TE_Port)

Enhanced ISL (EISL) Trunk Carries Tagged Traffic Notice: Blue VSAN Doesn’t Have to Reside From Multiple VSANs on Switch for it to Traverse Switch
12

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

6

VSAN—EISL Establishment (Negotiation Protocol)
Two interconnected switch ports conduct an ELP (Exchange Link Parameters) exchange—forms two E_Ports and ISL links
(Standard-based negotiation)
ELP Exchange
E E

Two switches then conduct an ESC (Exchange Switch Capabilities) exchange—determines whether Cisco switch on other end or not capable of EISL
(Standard-based negotiation)

ESC Exchange
Two Cisco Switches E E

If yes—then proceed to negotiate EISL/ISL If Cisco switches, two switches then conduct an EPP (Exchange Peer Parameters: Cisco prop protocol) exchange—determines whether to stay as ISL, move to EISL (VSAN-enabled), or isolate in case of mismatched port VSANs These modes are negotiated based on the configuration of the switches and the parameters of the ports; isolation can occur if VSANs are mismatched
* Provided ELP Parameters Match Such as Timers and Switches in Interoperability Mode if Required
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

EPP Exchange
E E

Normal ISL Or
TE TE

EISL Formed Or Isolated
E E

Done

13

Port Channels
Port Aggregation Feature Used to Create a Single Logical ISL from 1–16 Physical ISLs
Increases bandwidth and availability Very granular load balancing per exchange/src/dst or per src/dst (policy on a per VSAN basis) Interfaces can both be added and removed in a nondisruptive manner in production environments Preserved FC guarantee of inorder delivery (IOD)
4 Link Port Channel EISL

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

14

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

7

Port Channel Protocol (PCP)
Exchange-based in-order load balancing
Mode 1: based on src/dst FC_ID/OX_ID/RX_ID Mode 2: based on src/dst FC_ID

Consistently detect misconfiguration Transition mis-configured ports to isolated state so as to be able to correct the misconfiguration Synchronize bring up of ports in a channel across peer switches Provide the ability for the system to automatically create port channels among compatible ports

Up to 160 Gbps Port Channel with HA

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

15

VSANs, EISLs, TE_ports, Port Channels—How All These Work Together
Hierarchical relationship—
Port Channels provide link aggregation to yield virtual ISL (E_Port) Single-link ISL or Port Channel ISL can be configured to become EISL—(TE_Port) VSANs can be selective grafted or pruned from EISL trunks
VSAN 10 20 METRIC 100 50 VSAN 10 20 Metric 50 100

All member links of a Port Channel must have same configuration prior to creating channel (e.g., TE_Port or E_Port, VSANs enabled, etc.) Port Channel technology provides high availability and fast recovery for VSAN trunk (EISL) Multiple Port Channels yield multiple paths for custom traffic engineering
BRKSAN-3707 14694_05_2008_X1

8 Gbps PortChannel Trunking E_Port (TE_Port)

p 20 ku N ac SA 10 B V AN VS

10 AN ly VS On

E_Port

Trunking E_Port (TE_Port) 4 Link (8 Gbps) PortChannel Configured as EISL

E_Port

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

16

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

8

VSANs and Non-Cisco Switches
The VSANs feature involves a frame tagging mechanism which is not understood by 3rd party fabrics MDS Family switches support heterogeneous switch interoperability—non-VSAN aware Cisco “Interoperability Mode” is configured per-VSAN—no loss of functionality in MDS 9000 switches MDS switches negotiate a standard E_Port with non-Cisco switches MDS 9000 E_Ports also have a port VSAN Therefore, the entire non-Cisco switch, including all its ports, will reside in the port VSAN of the connecting E_Port
EISL Trunks Carrying Numerous VSANs Simple ISL Links E_Ports Non-Cisco Fabric Switches

Each Non-Cisco Switch Belongs to Only One VSAN

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

17

Fabric Routing

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

18

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

9

Fabric Routing: Cisco Inter-VSAN Routing
We use fabric as an extension of virtual fabrics to enable cross-fabric connectivity Done without merging the routed fabrics
Without propagation of irrelevant fabric events Without concern for overlapping domain IDs Without concern for fabric interoperability differences Without fabric services interference across multiple fabrics
VSAN VSAN VSAN
Physical SAN Physical SAN

Physical SAN

Physical Islands

Virtual Fabric

Enable end devices from different virtual fabrics to access one another
VSAN

VSAN

VSAN

Routed Virtual Fabric

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

19

Fabric Routing Applications— Sharing Common Resource
Overlay data replication fabrics on common physical fabric
No need for separate pair of switches for each replication connection Use one virtual fabric per replication connection
MS

Common Physical Fabric

MS Marketing SAN

Sales SAN

MS

Being able to share common SAN Extension circuits amongst multiple virtual fabrics Fabric routing adds resiliency to the solution
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

HR SAN

TAPE SAN

Tape Media Server
MS

20

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

10

Fabric Routing Applications— SAN Extension Solutions
Minimize the impact of change in fabric services across geographically distributed sites Limit fabric control traffic such as RSCNs and Build/Reconfigure Fabric (BF/RCF) to local VSANs Augments the high availability of the solution
Filters unnecessary events, Isolates from remote faults, Enables selective visibility

Works with any transport service (FC, SONET/SDH, DWDM/CWDM, FCIP)
Inter-VSAN Connection Between Completely Isolated Fabrics
IVR Isolation Minimizes Impact if Transit VSAN Lost Replication VSAN_1 PortChannel Protects Against Loss of Member Links/Paths EISL#1 in Port Channel Replication VSAN_4

IVR

Transit VSAN_3 (IVR)

EISL#2 in Port Channel

IVR

Local VSAN_2
BRKSAN-3707 14694_05_2008_X1

IP WAN

Local VSAN_5 21

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

Virtualization Technologies

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

22

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

11

SAN Device Virtualization (SDV)
Allows provisioning with virtualized servers and storage devices Significantly reduces time to replace HBAs and Storage devices
No reconfiguration of zoning, VSANs, etc. required on MDS No need to reconfigure storage array LUN masking after replacing HBAs Eliminates re-building driver files on AIX and HP-UX after replacing storage
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Server

Storage Arrays

X
Physical to Virtual Mapping Virtual Initiator Virtual Target

Y

Presents virtual WWN to servers and storage device

23

NPIV (N_Port Identifier Virtualization)
Designed for virtual server environments — Linux on zSeries, VMware Assigning multiple port IDs to a single N_Port Multiple applications on the same port can use different IDs in the same VSAN Zoning and port security can be implemented at the application level Data Center Session on Server Virtualization: DCT-2868 3 Name Server entries 3 Name Server entries Virtual Servers 3 Virtual Devices 3 Virtual Devices All share 1 FC Port but All share 1 FC Port but Email maintain individual identity maintain individual identity Web
Print

3 Logins
LUN 1 N_Port ID=1 LUN 2 N_Port ID=2 LUN 3 N_Port ID=3 N_Port Controller
HBA

FC

F_Port

3 FCIDs MDS 9000
Cisco Public

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

24

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

12

NPIV FLOGI/FDISC Login Process
NPIV Enabled Switch

When host physical port comes up, it first does a FLOGI and PLOGI into the switch to register into the FC Name Server NPIV capable devices (typically HBAs) will continue login process using FDISC (Fabric Discovery) to register virtual PWWN into the FC Name Server using the same physical interface

F

P1

NPIV Capable HBA

vP1 vP2 vP3

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

25

Blade Switch Explosion Issues
Scalability
Each Blade Switch uses a single Domain ID Theoretical maximum number of Domain IDs is 239 per VSAN Supported number of domains is quite smaller (depends on OSM) EMC: 40 domains Cisco Tested: 75 HP: 40 domains Other OSM Do Not Post

Manageability
More switches to manage Shared management of blade switches between storage and server administrators
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

BRKSAN-3707 14694_05_2008_X1

26

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

13

Cisco MDS’ N-Port Virtualizer (NPV)
MDS NPV
NPV enables the switch to act as a NPIV host NPV mode is no longer a “switch” Changing from switching mode to NPV mode is disruptive Upgrading SAN OS code is non-disruptive NPV switch uplink is no longer an ISL (NP-port) NPV switch DOES NOT use a Domain ID No longer limited to Domain ID boundaries

Manageability
Less amount of switches to manage NPV enable switch is now managed like a NPIV enabled host Eliminates the need for server administrators to manage the SAN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

27

Differences Between NPIV and NPV
NPIV (N-Port ID Virtualization)
Functionality geared towards server’s host bus adapters (HBA) NPIV provides a means to assign multiple Server Logins to a single physical interface The use of different virtual pWWN allows access control (zoning) and port security to be implemented at the application level Usage applies to applications such as VMWare, MS Virtual Server and Linux Xen

NPV (N-Port Virtualizer)
Functionality geared towards MDS fabric switches (MDS 9124, MDS 9134, Nexus 5000 and blade switches) NPV provides the FC switch’s connections (uplink) to act as server connections – instead of acting like a standard ISL Utilizes NPIV type functionality to allow multiple server logins from other switch ports to use NP-port uplink

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

28

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

14

NPV FLOGI/FDISC Login Process
When NP port comes up on a NPV edge switch, it first FLOGI and PLOGI into the core to register into the FC Name Server End Devices connected on NPV edge switch does FLOGI but NPV switch converts FLOGI to FDISC command, creating a virtual PWWN for the end device and allowing to login using the physical NP port. All I/O of end device will always flow through same NP port
F

NPV Core Switch

F

NP P1 NP P2

NPV Edge Switch
F F

P4 = vP2

P5 = vP3

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

29

Nested NPIV FLOGI/FDISC Login Process
NPV-Core Switch

When NP port comes up on a NPV edge switch, it first FLOGI and PLOGI into the core to register into the FC Name Server End Devices connected on NPV edge switch does FLOGI but NPV switch converts FLOGI to FDISC command, creating a virtual PWWN for the end device and allowing to login using the physical NP port. NPIV capable devices connected on NPV switch will continue FDISC login process for all virtual PWWN which will go through same NP port as physical end device
F

F

F

NP P1 NP P2

NPV Edge Switch
F

P3 = vP1

P4 = vP5

vP2 vP3 vP4
BRKSAN-3707 14694_05_2008_X1

vP6 vP7 vP8

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

30

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

15

NPV Supported Switches
NPV Edge Switches
MDS 9124, MDS 9134 and NX5K IBM and HP Blade Switches

NPV Core Switches
MDS 9500 Family of Directors MDS 9216A, MDS 9216i and MDS 9222i 3rd Party Switches Needs to support NPIV Needs Testing/Qualification

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

31

MDS 9124—NPV Architecture
NPV Architecture
Total of 6 Port-Groups – every 4 ports By default, first port in each Port-Group (ports 1, 5, 9, 13, 17 and 21) is set to “NP” mode for uplink to NPV Core Switch (Can be changed) All other ports are set to “F” for device connectivity (DOES NOT SUPPORT FL-Ports)

Port-Group1: Ports 1 – 4 Port-Group2: Ports 5 – 8 Port-Group3: Ports 9 – 12 Port-Group4: Ports 13 – 16 Port-Group5: Ports 17 – 20 Port-Group6: Ports 21 - 24

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

32

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

16

MDS 9134—NPV Architecture
NPV Architecture
Total of 10 Port-Groups
Port-Group consists of 4 ports for 1/2/4Gig ports grouping Each 10Gig port is its own Port-Group

By default, first port in each Port-Group (ports 1, 5, 9, 13, 17, 21, 25 and 29) is set to “NP” mode for uplink to NPV Core Switch (Can be changed) Both 10Gig port is set to “NP” mode All other ports are set to “F” for device connectivity (DOES NOT SUPPORT FL-Ports)

Port-Group1: Ports 1 – 4 Port-Group3: Ports 9 – 12 Port-Group5: Ports 17 – 20 Port-Group7: Ports 25-28 Port-Group9: Port 1 (10G)

Port-Group2: Ports 5 – 8 Port-Group4: Ports 13 – 16 Port-Group6: Ports 21 – 24 Port-Group8: Ports 29-32 Port-Group10: Port 2 (10G)

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

33

Port Mapping for HP Blade Switches
HP Blade Switch Port-Group Mapping
External Links (All links set to NP-port only)
PG 1 PG 2 PG 3 PG 3 PG 4 PG 4 PG 5 EXT 7 PG 6 EXT 8 PG 5

PG 1 -> EXT Port 1 PG 2 -> EXT Port 2

EXT 1

EXT 2

EXT 3

EXT 4

PG 4 -> EXT Port 5 and EXT Port 6 PG 5 -> EXT Port 7 PG 6 -> EXT Port 8

Internal Links (All links set to Fport only)
PG 2 PG 6 PG 2 PG 1 PG 1 PG 3 PG 4 PG 3 PG 1 PG 6 PG 5 PG 2

EXT 5

EXT 6

PG 3 -> EXT Port 3 and EXT Port 4

PG 1 -> Bays 3,4 and 11 PG 2 -> Bays 1,2 and 12 PG 3 -> Bays 9 and 10

Bay 10

Bay 11

Bay 12

PG 6

Bay 14

PG 5 Bay 15

PG 4 -> Bays 8 and 16 PG 5 -> Bays 7, 14 and 15 PG 6 -> Bays 6, 7 and 13
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Bay 13

34

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Bay 16

Bay 2

Bay 3

Bay 1

Bay 4

Bay 6

Bay 8

Bay 7

Bay 9

Bay 5

PG 4

17

Port Mapping for IBM Blade Switches
IBM Blade Switch Port-Group Mapping
PG 1 PG 1 PG 2 PG 3 PG 4 PG 4 Port 18 PG 5 Port 19

External Links (All links set to NPport only)
PG 1 -> Port 0 and Port 15 PG 2 -> Port 16 PG 3 -> Port 17 PG 4 -> Port 18 PG 5 -> Port 19

Port 15

Port 16

Internal Links (All links set to Fport only)
PG 1 -> Bays 1 and 3 PG 2 -> Bays 2, 4 and 7 PG 3 -> Bays 5, 6 and 8 PG 4 -> Bays 9, 13 and 14 PG 5 -> Bays 10, 11 and 12

PG 1

PG 2

PG 1

PG 2

PG 3

Port 17

Port 0

PG 3

PG 2

PG 4 Bay 13

PG 3

PG 5

PG 5

Bay 10

Bay 11

Bay 12

BRKSAN-3707 14694_05_2008_X1

Bay 14
35 36

Bay 2

Bay 3

Bay 1

Bay 4

Bay 6

Bay 5

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

Number of NPIV Logins: MDS 9200/9500
Type of Logins Logins per Port Logins per Line Card Logins per Switch Logins per physical fabric Number of Logins 126 400 2,000 10,000

These are the number of logins allowed on all Gen1 and Gen2 line cards. The limits applied to on a per switch will also apply to all MDS 9200 and MDS 9500. MDS 9124/9134 and Blade switches will have different limits and will be shown later.

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Bay 7

Bay 8

Bay 9

PG 4

PG 5

18

Number of NPIV Logins: MDS 9124/9134 and Blade Switches
Switching Mode Logins per Port Logins per Port-Group Logins per MDS 9124 Logins per MDS 9134 Logins per MDS 9124e Logins per IBM Blade Switch Logins per Nexus 5000 42 168 1,008 1,680 1,008 840 2,048 NPV Mode 114 114 684 1,140 684 570 2,048

The stated numbers are verified logins and are the supported number of logins.

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

37

Intelligent Fabric Applications Data Mobility Manager

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

38

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

19

Data Migration Solutions—Host Base
Host Base Migration
Benefits
Uses existing host base volume management Non-disruptive to application server Heterogeneous array migration No added cost (other than cost of volume manager with mirroring capability)
Application Server Server Data Flow Mirrored Data Flow 9G Host Volume RAID 1

Fabric A

Fabric B

Draw Backs
CPU intensive when migrating Affects application performance
9G 9G

Existing Storage Vendor X
BRKSAN-3707 14694_05_2008_X1

New Storage Vendor Y

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

39

Data Migration Solutions—Storage Base
Host Base Migration
Benefits
Offloads application server CPU Migrates multiple servers at a time
Server Data Flow Migration Data Flow 9G Host Volume Applicatio n Server

Draw Backs
Uses Proprietary replication technology from array Requires separate port for specific replication (migration) on array Migration within same vendor’s family of storage and may have to be within same tier Very costly $$$
Existing Storage Vendor X Fabric A

Fabric B

9G

9G

New Storage Vendor X

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

40

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

20

Data Migration Solutions—Network Base
Network Base Migration
Benefits
Offloads Application CPU Lower cost tool Heterogeneous across array vendors More scalable No single point of failure
Server Data Flow Migration Data Flow 9G Host Volume Application Server

Draw Backs
Single disruption to application server during cut-over

Fabric A

Fabric B

9G

9G

Existing Storage Vendor X
BRKSAN-3707 14694_05_2008_X1

New Storage Vendor Y

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

41

Supported Hardware for DMM
32-port FC Storage Services Module
Fully distributed architecture provides huge aggregate performance Embedded ASICs for inline SCSI processing Integrated 32 Fibre Channel port

Number of SSMs Required
A minimum of 1 SSM A minimum of 2 SSMs is supported for Dual Fabric

Advanced Feature Support in SANOS 3.2(1)
FC-Redirect DMM utilizes FC-Redirect
BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

42

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

21

What Is FC-Redirect (FCR)?
Is a Target centric Transport infra structure feature on the MDS supervisor, does the FC DID/SID re-write only. Seamless integration of one or more intelligent services in a fabric for a specific Host & Disk (I_T) pair. No re-wiring or re-configuring existing Host’s & Disk’s. No Splitting of fabrics into multiple VSAN's. Operate in a heterogeneous switch environment Disk must be attached to a FC-Redirect aware MDS, Host & SSM can be located anywhere in the fabric.

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

43

Life of Packet from Host to Disk
VT < H VT < H FWD VI > T VI > T SSM

2

DPP

2
[H => VT]

1
[H =>T] FCID: H Target Switch [H => T] MAC FWD H > VT

3

[VI => T]

FC

4

FCID: T

1

H>T

Link Between Target SW & Host T

MAC VI > T

FWD

MAC H>T

3

Trunk Link Between Target SW & SSM SW

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

44

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

22

Server I/O Handling—Synchronous Mode
Server

Dealing with Server IOs
Writes to “Migrated” Area are Mirrored Writes to “Being Migrated” Area are queued temporarily (till region has been migrated) Writes to “To be Migrated” Area are written to Existing Storage only Server Reads are read from Existing Storage only

Migrated

Being Migrated To be Migrated

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

45

Server I/O Handling—Synchronous Mode
Server

Dealing with Server IOs
Writes to “Migrated” Area are Mirrored Writes to “Being Migrated” Area are queued temporarily (till region has been migrated) Writes to “To be Migrated” Area are written to Existing Storage only Server Reads are read from Existing Storage only

Migrated

Being Migrated To be Migrated

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

46

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

23

Server I/O Handling—Synchronous Mode
Server

Dealing with Server IOs
Writes to “Migrated” Area are Mirrored Writes to “Being Migrated” Area are queued temporarily (till region has been migrated) Writes to “To be Migrated” Area are written to Existing Storage only Server Reads are read from Existing Storage only

Migrated

Being Migrated To be Migrated

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

47

Server I/O Handling—Synchronous Mode
Server

Dealing with Server IOs
Writes to “Migrated” Area are Mirrored Writes to “Being Migrated” Area are queued temporarily (till region has been migrated) Writes to “To be Migrated” Area are written to Existing Storage only Server Reads are read from Existing Storage only

Migrated

Being Migrated To be Migrated

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

48

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

24

Server I/O Handling—Asynchronous Mode
Server

Mark all regions in MRL dirty
Modified Region Log [MRL]

While (MRL regions left) { Select a Region; Copy Region; Clear MRL Region }

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

49

Server I/O Handling—Asynchronous Mode
Server
Modified Region Log [MRL]

Dealing with Server IOs
• Writes are written to Existing Storage only • MRL entry is updated for each Write issued

Multiple passes of MRL done until all regions are clear For cut-over last MRL pass done with the LUN in the offline mode

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

50

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

25

Server I/O Handling—Asynchronous Mode
Server
Modified Region Log [MRL]

Dealing with Server IOs

Server Reads are read from Existing Storage only

Existing Storage LUN
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved.

New Storage LUN
Cisco Public

51

DMM for Core-Edge
Environment Configuration
Place SSM at the Core switches for both Fabric A and Fabric B Existing Storage and New Storage should be on the same switch where SSM resides Storage SHOULD NOT be connected to the SSM Storage can be connected on 16-port MPS 12-port 24-port
Edge Switches SSM Core Switches SSM Existing Storage New Storage

Storage Services Module
Install DMM license Enable DMM feature Recommended that SSM ports not to be used for any devices
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Server

52

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

26

Intelligent Fabric Applications Storage Media Encryption

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

53

Cisco SME Overview
Application Server
Name: XYZ SSN: 1234567890 Amount: $123,456 Status: Gold

Encrypts storage media (data at rest)
Strong IEEE compliant AES-256 encryption Integrated as transparent fabric service

Key Management Center
Encrypt IP

Supports heterogeneous tape devices, and VTLs Offers secure, comprehensive key management

Name: XYZ @!$%!%!%!%%^& SSN: 1234567890 *&^%$#&%$#$%*!^ Amount: $123,456 @*%$*^^^^%$@*) Status: Gold %#*@(*$%%%%#@

Compresses tape data Allows offline media recovery

Tape Library

Built upon FIPS level-3 system architecture Networkers Session BRKSAN-2893

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

54

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

27

Transparent Fabric Service
Application Servers

Integrates seamlessly with existing Cisco MDS fabrics Non-disruptive deployment (FC-R)
No appliances to insert in data path No SAN re-wiring or re-configuration

MPS-18/4

MPS-18/4

Redirects traffic flows after enabling encryption Highly saleable performance Load balances automatically Reliable, highly available service

Tape Library

Routes traffic to another MPS when one fails

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

55

Cisco SME Enabled Platforms
HIGH-PERFORMANCE INTEGRATED SOLUTION WITH MULTI-GIGABIT THROUGHPUT
18 4-Gbps ports for FC, 4 GigE for IP Services

MDS 9222i

MDS 9216A MDS 9216i

MDS 9506

MDS 9509

MDS 9513

18/4-Port Multiservice Module (MSM)

Cisco Fabric Manager w/Key Management Center
BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

56

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

28

SME Cluster
Application Servers

Consists of up to four SME enable switches (nodes) in the same physical fabric Node-to-node communication via IPFC through management interface Quorum based cluster Provides scalability, reliability, availability and automatic load balancing
Scalability is achieved by adding additional line card in the fabric Target based load balancing Re-routes traffic when failure occurs

MSM-18/4

MSM-18/4

Tape Library

Single point of management with Cisco FM Can provide services across multiple VSANs One cluster per physical fabric

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

57

Cisco Key Management Center (KMC)
Cisco Key Management Center FMS Key Catalog DB Application Servers

Essential key lifecycle management
Archives, recovers, distributes, and shreds media keys

Transports keys and management traffic securely (SSH, HTTPS) Integrates with Cisco FM server
No additional software to install Intuitive provisioning and management with Cisco FM Web client

MSM-18/4 MSM-18/4 Fabric ’A’

MSM-18/4 MSM-18/4 Fabric ’B’

Tape Library

May use the local data base or the enterprise data base for the desired level of reliability and availability. Key Catalog data base options:
PostgreSQL Oracle 10g Express Third party key manager (ex: EMC’s RSA)

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

58

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

29

Intelligent Fabric Applications SANTap

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

59

SANTap Intelligent Write Splitting
Initiators Initiator’s VSAN (SANTap) Initiator target I/O Not in primary data path Appliance Targets and Appliance VSAN

SANTap

SAN

Copy of primary I/O

Target

Appliance Partners leverages SANTap services
Part of the Cisco Storage Services Module (SSM)

Out-of-band architecture
SANTap redirects I/O and eliminates need for host splitter
BRKSAN-3707 14694_05_2008_X1

Virtual SAN configurations
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

60

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

30

SANTap Partner Solutions
Appliance Appliance

Network-Based Data Protection
Support heterogeneous storage and servers Integrated with Cisco MDS9000 SANTap Supports VMWare Virtual Machines (RDM)

CDP/CRR Recovery at Local or Remote Site
Tracks all data changes to every protected LUN Utilizes bookmarks for application-aware recovery Enables Read/Write processing of replicated LUNs

Heterogeneous Replication
Works with any supported storage True Any to Any Volume Replication

CRR Advanced WAN functionality
WAN data reduction and compression FC to TCP/IP conversion TCP Optimization

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

61

SANTap Deployment—Before/After
BEFORE
Application Server

AFTER
Application Server Front-End VSAN

0 DVTLUNs 1 2 PRODUCTION VSAN SSM 9 Virtual Initiators 0 1 2 Storage Array CVT CVTLUNs 0 1 2 Storage Array AVTLUNs SSM Back-End VSAN AVT DVT Appliances Cluster

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

62

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

31

CRR—Asynchronous Flow
Main Data Center
Application Server

Remote Data Center
Application Server

EMC RecoverPoint Asynchronous Replication
1 4 1. 2. 3. 4. Write I/O is sent to SSM module Write I/O is then forward to both local Storage Array and local Appliance Both local Storage Array and local Appliance acknowledge Write I/O back to the SSM Once SSM receives both acknowledgements, then sends acknowledgment to Application Server

3 SAN SSM 2 1

4 SSM SAN

WAN
2 3

Appliances
1. 2. 3. 4.

Appliances

2

3

I/O is sent through the WAN to remote Appliance I/O is then sent to replication LUN(s) through the SSM I/O is then acknowledged back to the Remote Appliance Remote Appliance then sends acknowledgement back to Primary Data Center Appliance through the WAN

Storage Array
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Storage Array
63

SSM Line Card

X-Bar

2 Gbps each Forwarding Engine

2 Gbps each

2 Gbps DPP2

2 Gbps DPP3

2 Gbps DPP6

2 Gbps DPP7

2 Gbps DPP1

2 Gbps DPP4

2 Gbps DPP5

2 Gbps DPP8

DVT

Ports 1– 4

Ports 5– 8

Ports 9-12

Ports 13-16

Ports 17-20

Ports 21-24

Ports 25-28

Ports 29-32

BRKSAN-3707 14694_05_2008_X1

Host1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

64

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

32

SSM Line Card

X-Bar

2 Gbps each Forwarding Engine

2 Gbps each

2 Gbps DPP2

2 Gbps DPP3

2 Gbps DPP6

2 Gbps DPP7

2 Gbps DPP1

2 Gbps DPP4

2 Gbps DPP5

2 Gbps DPP8

DVT

Ports 1– 4

Ports 5– 8

Ports 9-12

Ports 13-16

Ports 17-20

Ports 21-24

Ports 25-28

Ports 29-32

BRKSAN-3707 14694_05_2008_X1

Host1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

65

Front-End VSAN—Zoning
Only physical host initiators and DVTs reside in Front-End VSANs Normal zoning applies where “Host Initiator” is zoned with DVT NOTE: A single host initiator zoned with 2 or more separate DVTs, must make sure that all of those DVTs reside on the same DPP
Fabric-A
RecoverPoint Front-End VSAN 30

Fabric-B
RecoverPoint Front-End VSAN 40

Host1 Zone

Host1 HBA1 DVT1, DVT2

Host1 Zone

Host1 HBA2 DVT1, DVT2

Host2 Zone

Host2 HBA1 DVT3, DVT4

Host2 Zone

Host2 HBA2 DVT3, DVT4

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

66

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

33

Back-End VSAN—Zoning
Fabric-A SANTap Back-End VSAN Fabric-B SANTap Back-End VSAN

ApplianceTargets

SSM 9VIs APP1-P0 CVT APP2-P0

ApplianceTargets

SSM 9VIs APP1-P2 CVT APP2-P2

ApplianceInitiators

Storage Ports

APP1-P1 APP2-P1

ApplianceInitiators

Storage Ports

APP1-P3 APP2-P3

Appliance VTInitiators

AVT Initiators APP1-P1 APP2-P1

Appliance VTInitiators

AVT Initiators APP1-P3 APP2-P3

Appliance VTTarget

AVT Targets APP1-P0 APP2-P0

Appliance VTTarget

AVT Targets APP1-P2 APP2-P2

BE-Host Zone Host VI Storage Ports Appliance Local Storage

BE-Host Zone Host VI Storage Ports Appliance Local Storage Port Storage APP1-P3 APP2-P3

Storage Port APP1-P1 APP2-P1

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

67

SANTap Limits
Table 1: SANTap Limits

SSI Images
Max # of ITL per DPP Max # of ITL per SSM Max # of Sessions per SSM Max # of LUNs per Initiator per DVT Max # of LUNs per DVT Max # of host (initiators) per DVT Max # of DVTs per SSM Max # of DVTLUNs per SSM LUN ID Addressing size

3.0(2j)
1,024 1,024 1,024

3.1(2m)
1,024 2,048 2,048

3.1(3)
1,024 4,080 2,048

3.2(3i)
3,096 24,576 2,048

256 for all SSI images 1,024 16 16 1,024 16 1,024 16 16 2,048 16 1,024 16 32 4,096 16 3,096 64 64 16,384 32

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

68

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

34

Intelligent Fabric Applications Storage Virtualization

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

69

SAN-Based Storage Virtualization
Performance architecture
Leverages next-generation “intelligent” SAN switches

Scalable architecture
Virtual volumes Split-path architecture for high performance A “stateless” virtualization architecture does not store any information written by the application.
Meta-Data Meta-Data

High speed, high throughput data mapping Purpose-built ASICs (DPP) that handle and redirect I/O at line speed, with almost no additional latency Based on instructions provided by the MetaData Appliances

Multi-vendor arrays

Provides advanced functionality Supports heterogeneous environments

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

70

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

35

Storage Virtualization Logical Topology

Front-End VSAN

Pooled resources

Back-End VSAN

Virtual targets

Virtual initiators

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

71

Data Flows
Control Frame Data Frame

Meta-Data Appliance

IP

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

72

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

36

Network-Based Volume Management
Applications

Simplify volume presentation and management
Create, delete, change storage volumes Provides front-end LUN Masking and mapping of storage volume to hosts

Centralize management and control
Single Invista console to manage virtual volumes, clones, and mobility jobs

Virtual volumes

Reduce management complexity of a heterogeneous storage
Single management interface to allocate and reallocate storage resources
BRKSAN-3707 14694_05_2008_X1

Physical storage
© 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

73

Dynamic Volume Mobility Explained
Virtualization
Hosts see Storage Virtualization as an array Presents virtual volumes to hosts
Virtual Volumes

Virtual LUN: 10

Maps virtual volumes to physical volumes

To Move a Volume:
Data Path Controlle r Data Path Controlle r

Select source and target volumes Network synchronizes the volumes, then changes the virtualphysical mapping Array: 2 LUN: 30 No I/O disruption to host

Virtual initiators

Array: 1 LUN: 20

EMC
BRKSAN-3707 14694_05_2008_X1

HDS
Cisco Public

© 2008 Cisco Systems, Inc. All rights reserved.

74

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

37

Heterogeneous Point-in-Time Copies
Applications

Create point-in-time copies
Source and clone can be on different, heterogeneous storage arrays

Enable replication across heterogeneous storage
Leverage existing storage investments Reduce replication storage capacity and management costs
Virtual volume

SAN

Active volume

Maximize replication benefits to support service levels
Backup and recovery Testing, development, and training Parallel processing, reporting, and queries Physical storage
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

Clone

Clone Clone

Data

75

VSAN Considerations
Back-End VSAN
Multiple Back-End VSAN supported by some partners Zone all 9 VIs to storage ports Best practice to create fcalias for all 9 VIs
HR VSAN 20
FC FC FC

Storage VSAN 10
VT1

MDS 9xxx
DEV VSAN 30
VT2
FC FC FC

Front-End VSAN
Up to 32 Virtual Targets per SSM Zone server HBA to one Virtual Target

VI 1- 9

Invista
VT3
FC FC FC

Control VSAN
Communication to external CPC Zone up IP interfaces for VSAN and SSM’s CPP
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

ERP VSAN 40 ERP Admin

76

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

38

Q and A

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

77

Recommended Reading
Continue your Cisco Live learning experience with further reading from Cisco Press Check the Recommended Reading flyer for suggested books

Available Onsite at the Cisco Company Store
BRKSAN-3707 14694_05_2008_X1 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public

78

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

39

Complete Your Online Session Evaluation
Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Passport points for each session evaluation you complete. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
Don’t forget to activate your Cisco Live virtual account for access to all session material on-demand and return for our live virtual event in October 2008. Go to the Collaboration Zone in World of Solutions or visit www.cisco-live.com.

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

79

BRKSAN-3707 14694_05_2008_X1

© 2008 Cisco Systems, Inc. All rights reserved.

Cisco Public

80

© 2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

40

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->