Advanced SAN Design - Virtualization Technologies and Intelligent Applications Design Considerations

Advanced SAN Design Virtualization Technologies and Intelligent Applications Design Considerations
BRKSAN-3707
BRKSAN-3707 14694_05_2008_X1
2008 Cisco Systems, Inc. All rights reserved.
Cisco Public
2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr
Agenda
Brief Review of Virtual Fabrics
Virtual Fabrics (VSANs) Port-Channels, Trunking and IVR
Virtualization Technologies
SAN Device Virtualization (SDV) N-Port ID Virtualization (NPIV) N-Port Virtualizer (NPV) FlexAttach
Intelligent Application
Data Mobility Manager (DMM) Storage Media Encryption (SME) SANTap Storage Virtualization
BRKSAN-3707 14694_05_2008_X1 2008 Cisco Systems, Inc. All rights reserved. Cisco Public
Virtual Fabrics
BRKSAN-3707 14694_05_2008_X1
Cisco Public
Virtual Fabric: Three Key Concepts

Virtual Fabric
Provide independent (virtual) fabric services on a single physical switch
Virtual Fabric Trunking and Port-Channels

Ability to transport multiple virtual fabrics over a single ISL or common group of ISLs
Fabric Routing (IVR)

Ability to provide selected connectivity between virtual fabrics without merging them
Trunk BRCD = Port Channel Cisco Group of ISLs = Port Channel

Ciscos Approach to Virtual Fabric: Virtual SAN (VSANs)

A VSAN provides a method to allocate ports within a physical fabric to create virtual fabrics Virtual fabrics created from larger cost-effective physical fabric Reduces wasted ports with islands Fabric events are isolated per VSANmaintains HA Hardware-based isolationtraffic is explicitly tagged across ISLs with VSAN membership info Statistics gathered per VSAN
Cisco MDS 9000 Family with VSAN Service Physical SAN Islands Are Virtualized onto Common SAN Infrastructure
BRKSAN-3707 14694_05_2008_X1
Cisco Public
VSANMDS Family
Each port on the MDS family exists in a VSAN Up to 256 VSANs in a single switch (hardware can support up to 4095) Logical configuration to move a port from one fabric to another WWN-based VSANs can provide automated VSAN membership Basis for Virtual Fabric Trunking (VFT) Extended Header (ANSI T11 FC-FS-2 section 10)
VSAN A
VSAN VSAN B B
VSAN C
VSAN D
VSAN Numbering Rules

Configured VSANs
VSAN 1 is the default VSAN

All ports are originally in VSAN1
VSAN 10 VSAN 20 VSAN 30
VSAN 2 through 4093 can be assigned to user VSANsVSAN 0, 4094, 4095 are reserved
Currently 256 VSANs is supported from the range of 24093
Enhanced ISL (EISL) Trunk Carries Tagged Traffic from Multiple VSANs
Trunking E_Port (TE_Port) VSAN 30 Is Not Propagated Across EISL Due to Nonexistence on Remote Switch Trunking E_Port (TE_Port) Port Is In VSAN 4094 (Isolated VSAN)
VSAN 10 VSAN 20
VSAN 4094 is a reserved as special VSAN

Called the isolated VSAN Used to isolate ports whose port-VSAN has been deleted Not propagated across switches Always present, cant be deleted Always in suspended state
Host Is Isolated From the Fabric
VSAN 30
Configured VSANS
Standard Fibre Channel Frame Fields

4B
SOF (Start of Frame) VSAN Header
ANSI T11.3 task group is the standard committee working on Virtual Fabrics T11.3 FC-FS-2 fabric services includes virtual fabrics specification
Defines Extended-Headers In FC-FS-2 Section 10.2 Defines frame tagging mechanism
8B
24B
FC Header
Applicable to N_Ports, F_Ports and E_Ports

Enables Inter-Switch Link to support trunking virtual interfaces Define the trunking virtual interfaces for end devices (hosts, storages)
0 -> 2112B
Payload
The ANSI T11 FS-SP group has accepted Cisco VSAN as standard (FC-FS-2 Section 10)
4B 4B
CRC EOF (End of Frame)

9
VSAN Header Field

R_CTL 8 Ver 2 Frame MPLS More User Type Present Header Priority 4 1 1 3
VSAN Number 12 bits
CDL TTL Present 1 8
# PAD P_VL Bytes 2 4
Rsvd 2
OAM 8
Msg Info 8
Each frame on a VSAN trunk carries an extra 8 bytes of header: User priority3 bitsused for QoS functionality to designate priority of frame VSAN ID12 bitsused to mark the frame as part of a particular VSANsupports up to 4096 VSANs MPLS flag1 bitused to designate whether this frame is subject to Multi-Protocol Label Switching processingfuture use Time-to-live (TTL)8 bitsused to help avoid routing loops standard part of an IP frame Other misc. fields including version, frame type, and other reserved fields
10
Trunking and Port-Channels
BRKSAN-3707 14694_05_2008_X1
Cisco Public
11
EISLs and TE-Port

1. The Trunking E_Port (TE_Port)
Negotiated between MDS switchesdefault Carries tagged frames from multiple VSANs Can be optionally disabled to yield E_Port Only understood by MDS switches Also has a native VSAN assignment (for E_Port) Trunk all VSANs (1-4093) by default Not to be confused with Brocade ISL aggregation (trunking)
Cisco MDS 9513 Director with VSAN Service
Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk Carries Tagged Traffic from Multiple VSANs
2. The EISL link

The resultant link created by two connected TE_Ports Superset of ISL functionality Carry individual control protocol information per VSAN (e.g. zoning updates) Can be extended over distance (DWDM, FCIP, etc.)
Trunking Cisco MDS E_Port (TE_Port) 9216 Trunking Fabric with E_Port VSAN Service (TE_Port)
Enhanced ISL (EISL) Trunk Carries Tagged Traffic Notice: Blue VSAN Doesnt Have to Reside From Multiple VSANs on Switch for it to Traverse Switch
12
BRKSAN-3707 14694_05_2008_X1
Cisco Public
VSANEISL Establishment (Negotiation Protocol)

Two interconnected switch ports conduct an ELP (Exchange Link Parameters) exchangeforms two E_Ports and ISL links
(Standard-based negotiation)
ELP Exchange
E E
Two switches then conduct an ESC (Exchange Switch Capabilities) exchangedetermines whether Cisco switch on other end or not capable of EISL
(Standard-based negotiation)
ESC Exchange
Two Cisco Switches E E
If yesthen proceed to negotiate EISL/ISL If Cisco switches, two switches then conduct an EPP (Exchange Peer Parameters: Cisco prop protocol) exchangedetermines whether to stay as ISL, move to EISL (VSAN-enabled), or isolate in case of mismatched port VSANs These modes are negotiated based on the configuration of the switches and the parameters of the ports; isolation can occur if VSANs are mismatched
* Provided ELP Parameters Match Such as Timers and Switches in Interoperability Mode if Required
EPP Exchange
E E
Normal ISL Or
TE TE
EISL Formed Or Isolated

E E
Done
13
Port Channels
Port Aggregation Feature Used to Create a Single Logical ISL from 116 Physical ISLs
Increases bandwidth and availability Very granular load balancing per exchange/src/dst or per src/dst (policy on a per VSAN basis) Interfaces can both be added and removed in a nondisruptive manner in production environments Preserved FC guarantee of inorder delivery (IOD)
4 Link Port Channel EISL
BRKSAN-3707 14694_05_2008_X1
Cisco Public
14
Port Channel Protocol (PCP)

Exchange-based in-order load balancing
Mode 1: based on src/dst FC_ID/OX_ID/RX_ID Mode 2: based on src/dst FC_ID
Consistently detect misconfiguration Transition mis-configured ports to isolated state so as to be able to correct the misconfiguration Synchronize bring up of ports in a channel across peer switches Provide the ability for the system to automatically create port channels among compatible ports
Up to 160 Gbps Port Channel with HA
BRKSAN-3707 14694_05_2008_X1
Cisco Public
15
VSANs, EISLs, TE_ports, Port ChannelsHow All These Work Together

Hierarchical relationship
Port Channels provide link aggregation to yield virtual ISL (E_Port) Single-link ISL or Port Channel ISL can be configured to become EISL(TE_Port) VSANs can be selective grafted or pruned from EISL trunks
VSAN 10 20 METRIC 100 50 VSAN 10 20 Metric 50 100
All member links of a Port Channel must have same configuration prior to creating channel (e.g., TE_Port or E_Port, VSANs enabled, etc.) Port Channel technology provides high availability and fast recovery for VSAN trunk (EISL) Multiple Port Channels yield multiple paths for custom traffic engineering
BRKSAN-3707 14694_05_2008_X1
8 Gbps PortChannel Trunking E_Port (TE_Port)
p 20 ku N ac SA 10 B V AN VS
10 AN ly VS On
E_Port
Trunking E_Port (TE_Port) 4 Link (8 Gbps) PortChannel Configured as EISL
E_Port
Cisco Public
16
VSANs and Non-Cisco Switches

The VSANs feature involves a frame tagging mechanism which is not understood by 3rd party fabrics MDS Family switches support heterogeneous switch interoperabilitynon-VSAN aware Cisco Interoperability Mode is configured per-VSANno loss of functionality in MDS 9000 switches MDS switches negotiate a standard E_Port with non-Cisco switches MDS 9000 E_Ports also have a port VSAN Therefore, the entire non-Cisco switch, including all its ports, will reside in the port VSAN of the connecting E_Port
EISL Trunks Carrying Numerous VSANs Simple ISL Links E_Ports Non-Cisco Fabric Switches
Each Non-Cisco Switch Belongs to Only One VSAN
BRKSAN-3707 14694_05_2008_X1
Cisco Public
17
Fabric Routing
BRKSAN-3707 14694_05_2008_X1
Cisco Public
18
Fabric Routing: Cisco Inter-VSAN Routing

We use fabric as an extension of virtual fabrics to enable cross-fabric connectivity Done without merging the routed fabrics
Without propagation of irrelevant fabric events Without concern for overlapping domain IDs Without concern for fabric interoperability differences Without fabric services interference across multiple fabrics
VSAN VSAN VSAN
Physical SAN Physical SAN
Physical SAN
Physical Islands
Virtual Fabric
Enable end devices from different virtual fabrics to access one another
VSAN
VSAN
VSAN
Routed Virtual Fabric
BRKSAN-3707 14694_05_2008_X1
Cisco Public
19
Fabric Routing Applications Sharing Common Resource

Overlay data replication fabrics on common physical fabric
No need for separate pair of switches for each replication connection Use one virtual fabric per replication connection
MS
Common Physical Fabric
MS Marketing SAN
Sales SAN
MS
Being able to share common SAN Extension circuits amongst multiple virtual fabrics Fabric routing adds resiliency to the solution
HR SAN
TAPE SAN
Tape Media Server

MS
20
10
Fabric Routing Applications SAN Extension Solutions

Minimize the impact of change in fabric services across geographically distributed sites Limit fabric control traffic such as RSCNs and Build/Reconfigure Fabric (BF/RCF) to local VSANs Augments the high availability of the solution
Filters unnecessary events, Isolates from remote faults, Enables selective visibility
Works with any transport service (FC, SONET/SDH, DWDM/CWDM, FCIP)

Inter-VSAN Connection Between Completely Isolated Fabrics
IVR Isolation Minimizes Impact if Transit VSAN Lost Replication VSAN_1 PortChannel Protects Against Loss of Member Links/Paths EISL#1 in Port Channel Replication VSAN_4
IVR
Transit VSAN_3 (IVR)
EISL#2 in Port Channel
IVR
Local VSAN_2
BRKSAN-3707 14694_05_2008_X1
IP WAN
Local VSAN_5 21
Cisco Public
Virtualization Technologies
BRKSAN-3707 14694_05_2008_X1
Cisco Public
22
11
SAN Device Virtualization (SDV)

Allows provisioning with virtualized servers and storage devices Significantly reduces time to replace HBAs and Storage devices
No reconfiguration of zoning, VSANs, etc. required on MDS No need to reconfigure storage array LUN masking after replacing HBAs Eliminates re-building driver files on AIX and HP-UX after replacing storage
Server
Storage Arrays
X
Physical to Virtual Mapping Virtual Initiator Virtual Target
Presents virtual WWN to servers and storage device
23
NPIV (N_Port Identifier Virtualization)

Designed for virtual server environments Linux on zSeries, VMware Assigning multiple port IDs to a single N_Port Multiple applications on the same port can use different IDs in the same VSAN Zoning and port security can be implemented at the application level Data Center Session on Server Virtualization: DCT-2868 3 Name Server entries 3 Name Server entries Virtual Servers 3 Virtual Devices 3 Virtual Devices All share 1 FC Port but All share 1 FC Port but Email maintain individual identity maintain individual identity Web
Print
3 Logins
LUN 1 N_Port ID=1 LUN 2 N_Port ID=2 LUN 3 N_Port ID=3 N_Port Controller
HBA
FC
F_Port
3 FCIDs MDS 9000

Cisco Public
BRKSAN-3707 14694_05_2008_X1
24
12
NPIV FLOGI/FDISC Login Process

NPIV Enabled Switch
When host physical port comes up, it first does a FLOGI and PLOGI into the switch to register into the FC Name Server NPIV capable devices (typically HBAs) will continue login process using FDISC (Fabric Discovery) to register virtual PWWN into the FC Name Server using the same physical interface
P1
NPIV Capable HBA
vP1 vP2 vP3
BRKSAN-3707 14694_05_2008_X1
Cisco Public
25
Blade Switch Explosion Issues

Scalability
Each Blade Switch uses a single Domain ID Theoretical maximum number of Domain IDs is 239 per VSAN Supported number of domains is quite smaller (depends on OSM) EMC: 40 domains Cisco Tested: 75 HP: 40 domains Other OSM Do Not Post
Manageability
More switches to manage Shared management of blade switches between storage and server administrators
2008 Cisco Systems, Inc. All rights reserved. Cisco Public
BRKSAN-3707 14694_05_2008_X1
26
13
Cisco MDS N-Port Virtualizer (NPV)

MDS NPV
NPV enables the switch to act as a NPIV host NPV mode is no longer a switch Changing from switching mode to NPV mode is disruptive Upgrading SAN OS code is non-disruptive NPV switch uplink is no longer an ISL (NP-port) NPV switch DOES NOT use a Domain ID No longer limited to Domain ID boundaries
Manageability
Less amount of switches to manage NPV enable switch is now managed like a NPIV enabled host Eliminates the need for server administrators to manage the SAN
27
Differences Between NPIV and NPV

NPIV (N-Port ID Virtualization)
Functionality geared towards servers host bus adapters (HBA) NPIV provides a means to assign multiple Server Logins to a single physical interface The use of different virtual pWWN allows access control (zoning) and port security to be implemented at the application level Usage applies to applications such as VMWare, MS Virtual Server and Linux Xen
NPV (N-Port Virtualizer)

Functionality geared towards MDS fabric switches (MDS 9124, MDS 9134, Nexus 5000 and blade switches) NPV provides the FC switchs connections (uplink) to act as server connections instead of acting like a standard ISL Utilizes NPIV type functionality to allow multiple server logins from other switch ports to use NP-port uplink
BRKSAN-3707 14694_05_2008_X1
Cisco Public
28
14
NPV FLOGI/FDISC Login Process

When NP port comes up on a NPV edge switch, it first FLOGI and PLOGI into the core to register into the FC Name Server End Devices connected on NPV edge switch does FLOGI but NPV switch converts FLOGI to FDISC command, creating a virtual PWWN for the end device and allowing to login using the physical NP port. All I/O of end device will always flow through same NP port
F
NPV Core Switch
NP P1 NP P2
NPV Edge Switch

F F
P4 = vP2
P5 = vP3
BRKSAN-3707 14694_05_2008_X1
Cisco Public
29
Nested NPIV FLOGI/FDISC Login Process

NPV-Core Switch
When NP port comes up on a NPV edge switch, it first FLOGI and PLOGI into the core to register into the FC Name Server End Devices connected on NPV edge switch does FLOGI but NPV switch converts FLOGI to FDISC command, creating a virtual PWWN for the end device and allowing to login using the physical NP port. NPIV capable devices connected on NPV switch will continue FDISC login process for all virtual PWWN which will go through same NP port as physical end device
F
NP P1 NP P2
NPV Edge Switch

F
P3 = vP1
P4 = vP5
vP2 vP3 vP4

BRKSAN-3707 14694_05_2008_X1
vP6 vP7 vP8
Cisco Public
30
15
NPV Supported Switches

NPV Edge Switches
MDS 9124, MDS 9134 and NX5K IBM and HP Blade Switches
NPV Core Switches

MDS 9500 Family of Directors MDS 9216A, MDS 9216i and MDS 9222i 3rd Party Switches Needs to support NPIV Needs Testing/Qualification
BRKSAN-3707 14694_05_2008_X1
Cisco Public
31
MDS 9124NPV Architecture

NPV Architecture
Total of 6 Port-Groups every 4 ports By default, first port in each Port-Group (ports 1, 5, 9, 13, 17 and 21) is set to NP mode for uplink to NPV Core Switch (Can be changed) All other ports are set to F for device connectivity (DOES NOT SUPPORT FL-Ports)
Port-Group1: Ports 1 4 Port-Group2: Ports 5 8 Port-Group3: Ports 9 12 Port-Group4: Ports 13 16 Port-Group5: Ports 17 20 Port-Group6: Ports 21 - 24
BRKSAN-3707 14694_05_2008_X1
Cisco Public
32
16
MDS 9134NPV Architecture

NPV Architecture
Total of 10 Port-Groups
Port-Group consists of 4 ports for 1/2/4Gig ports grouping Each 10Gig port is its own Port-Group
By default, first port in each Port-Group (ports 1, 5, 9, 13, 17, 21, 25 and 29) is set to NP mode for uplink to NPV Core Switch (Can be changed) Both 10Gig port is set to NP mode All other ports are set to F for device connectivity (DOES NOT SUPPORT FL-Ports)
Port-Group1: Ports 1 4 Port-Group3: Ports 9 12 Port-Group5: Ports 17 20 Port-Group7: Ports 25-28 Port-Group9: Port 1 (10G)
Port-Group2: Ports 5 8 Port-Group4: Ports 13 16 Port-Group6: Ports 21 24 Port-Group8: Ports 29-32 Port-Group10: Port 2 (10G)
BRKSAN-3707 14694_05_2008_X1
Cisco Public
33
Port Mapping for HP Blade Switches

HP Blade Switch Port-Group Mapping
External Links (All links set to NP-port only)
PG 1 PG 2 PG 3 PG 3 PG 4 PG 4 PG 5 EXT 7 PG 6 EXT 8 PG 5
PG 1 -> EXT Port 1 PG 2 -> EXT Port 2
EXT 1
EXT 2
EXT 3
EXT 4
PG 4 -> EXT Port 5 and EXT Port 6 PG 5 -> EXT Port 7 PG 6 -> EXT Port 8
Internal Links (All links set to Fport only)

PG 2 PG 6 PG 2 PG 1 PG 1 PG 3 PG 4 PG 3 PG 1 PG 6 PG 5 PG 2
EXT 5
EXT 6
PG 3 -> EXT Port 3 and EXT Port 4
PG 1 -> Bays 3,4 and 11 PG 2 -> Bays 1,2 and 12 PG 3 -> Bays 9 and 10
Bay 10
Bay 11
Bay 12
PG 6
Bay 14
PG 5 Bay 15
PG 4 -> Bays 8 and 16 PG 5 -> Bays 7, 14 and 15 PG 6 -> Bays 6, 7 and 13

Bay 13
34
Bay 16
Bay 2
Bay 3
Bay 1
Bay 4
Bay 6
Bay 8
Bay 7
Bay 9
Bay 5
PG 4
17
Port Mapping for IBM Blade Switches

IBM Blade Switch Port-Group Mapping
PG 1 PG 1 PG 2 PG 3 PG 4 PG 4 Port 18 PG 5 Port 19
External Links (All links set to NPport only)

PG 1 -> Port 0 and Port 15 PG 2 -> Port 16 PG 3 -> Port 17 PG 4 -> Port 18 PG 5 -> Port 19
Port 15
Port 16
Internal Links (All links set to Fport only)

PG 1 -> Bays 1 and 3 PG 2 -> Bays 2, 4 and 7 PG 3 -> Bays 5, 6 and 8 PG 4 -> Bays 9, 13 and 14 PG 5 -> Bays 10, 11 and 12
PG 1
PG 2
PG 1
PG 2
PG 3
Port 17
Port 0
PG 3
PG 2
PG 4 Bay 13
PG 3
PG 5
PG 5
Bay 10
Bay 11
Bay 12
BRKSAN-3707 14694_05_2008_X1
Bay 14
35 36
Bay 2
Bay 3
Bay 1
Bay 4
Bay 6
Bay 5
Cisco Public
Number of NPIV Logins: MDS 9200/9500

Type of Logins Logins per Port Logins per Line Card Logins per Switch Logins per physical fabric Number of Logins 126 400 2,000 10,000
These are the number of logins allowed on all Gen1 and Gen2 line cards. The limits applied to on a per switch will also apply to all MDS 9200 and MDS 9500. MDS 9124/9134 and Blade switches will have different limits and will be shown later.
BRKSAN-3707 14694_05_2008_X1
Cisco Public
Bay 7
Bay 8
Bay 9
PG 4
PG 5
18
Number of NPIV Logins: MDS 9124/9134 and Blade Switches

Switching Mode Logins per Port Logins per Port-Group Logins per MDS 9124 Logins per MDS 9134 Logins per MDS 9124e Logins per IBM Blade Switch Logins per Nexus 5000 42 168 1,008 1,680 1,008 840 2,048 NPV Mode 114 114 684 1,140 684 570 2,048
The stated numbers are verified logins and are the supported number of logins.
BRKSAN-3707 14694_05_2008_X1
Cisco Public
37
Intelligent Fabric Applications Data Mobility Manager
BRKSAN-3707 14694_05_2008_X1
Cisco Public
38
19
Data Migration SolutionsHost Base

Host Base Migration
Benefits
Uses existing host base volume management Non-disruptive to application server Heterogeneous array migration No added cost (other than cost of volume manager with mirroring capability)
Application Server Server Data Flow Mirrored Data Flow 9G Host Volume RAID 1
Fabric A
Fabric B
Draw Backs
CPU intensive when migrating Affects application performance
9G 9G
Existing Storage Vendor X

BRKSAN-3707 14694_05_2008_X1
New Storage Vendor Y
Cisco Public
39
Data Migration SolutionsStorage Base

Host Base Migration
Benefits
Offloads application server CPU Migrates multiple servers at a time
Server Data Flow Migration Data Flow 9G Host Volume Applicatio n Server
Draw Backs
Uses Proprietary replication technology from array Requires separate port for specific replication (migration) on array Migration within same vendors family of storage and may have to be within same tier Very costly $$$
Existing Storage Vendor X Fabric A
Fabric B
9G
9G
New Storage Vendor X
BRKSAN-3707 14694_05_2008_X1
Cisco Public
40
20
Data Migration SolutionsNetwork Base

Network Base Migration
Benefits
Offloads Application CPU Lower cost tool Heterogeneous across array vendors More scalable No single point of failure
Server Data Flow Migration Data Flow 9G Host Volume Application Server
Draw Backs
Single disruption to application server during cut-over
Fabric A
Fabric B
9G
9G
Existing Storage Vendor X

BRKSAN-3707 14694_05_2008_X1
New Storage Vendor Y
Cisco Public
41
Supported Hardware for DMM

32-port FC Storage Services Module
Fully distributed architecture provides huge aggregate performance Embedded ASICs for inline SCSI processing Integrated 32 Fibre Channel port
Number of SSMs Required

A minimum of 1 SSM A minimum of 2 SSMs is supported for Dual Fabric
Advanced Feature Support in SANOS 3.2(1)

FC-Redirect DMM utilizes FC-Redirect
BRKSAN-3707 14694_05_2008_X1
Cisco Public
42
21
What Is FC-Redirect (FCR)?

Is a Target centric Transport infra structure feature on the MDS supervisor, does the FC DID/SID re-write only. Seamless integration of one or more intelligent services in a fabric for a specific Host & Disk (I_T) pair. No re-wiring or re-configuring existing Hosts & Disks. No Splitting of fabrics into multiple VSAN's. Operate in a heterogeneous switch environment Disk must be attached to a FC-Redirect aware MDS, Host & SSM can be located anywhere in the fabric.
BRKSAN-3707 14694_05_2008_X1
Cisco Public
43
Life of Packet from Host to Disk

VT < H VT < H FWD VI > T VI > T SSM
DPP
2
[H => VT]
1
[H =>T] FCID: H Target Switch [H => T] MAC FWD H > VT
[VI => T]
FC
FCID: T
H>T
Link Between Target SW & Host T
MAC VI > T
FWD
MAC H>T
Trunk Link Between Target SW & SSM SW
BRKSAN-3707 14694_05_2008_X1
Cisco Public
44
22
Server I/O HandlingSynchronous Mode

Server
Dealing with Server IOs

Writes to Migrated Area are Mirrored Writes to Being Migrated Area are queued temporarily (till region has been migrated) Writes to To be Migrated Area are written to Existing Storage only Server Reads are read from Existing Storage only
Migrated
Being Migrated To be Migrated
Existing Storage LUN

BRKSAN-3707 14694_05_2008_X1 2008 Cisco Systems, Inc. All rights reserved.
New Storage LUN

Cisco Public
45

Server

Migrated

New Storage LUN

Cisco Public
46
23

Server

Migrated

New Storage LUN

Cisco Public
47

Server

Migrated

New Storage LUN

Cisco Public
48
24
Server I/O HandlingAsynchronous Mode

Server
Mark all regions in MRL dirty

Modified Region Log [MRL]
While (MRL regions left) { Select a Region; Copy Region; Clear MRL Region }

New Storage LUN

Cisco Public
49

Server

Writes are written to Existing Storage only MRL entry is updated for each Write issued
Multiple passes of MRL done until all regions are clear For cut-over last MRL pass done with the LUN in the offline mode

New Storage LUN

Cisco Public
50
25

Server
Server Reads are read from Existing Storage only

New Storage LUN

Cisco Public
51
DMM for Core-Edge

Environment Configuration
Place SSM at the Core switches for both Fabric A and Fabric B Existing Storage and New Storage should be on the same switch where SSM resides Storage SHOULD NOT be connected to the SSM Storage can be connected on 16-port MPS 12-port 24-port
Edge Switches SSM Core Switches SSM Existing Storage New Storage
Storage Services Module

Install DMM license Enable DMM feature Recommended that SSM ports not to be used for any devices
Server
52
26
Intelligent Fabric Applications Storage Media Encryption
BRKSAN-3707 14694_05_2008_X1
Cisco Public
53
Cisco SME Overview

Application Server
Name: XYZ SSN: 1234567890 Amount: $123,456 Status: Gold
Encrypts storage media (data at rest)

Strong IEEE compliant AES-256 encryption Integrated as transparent fabric service
Key Management Center

Encrypt IP
Supports heterogeneous tape devices, and VTLs Offers secure, comprehensive key management
Name: XYZ @!$%!%!%!%%^& SSN: 1234567890 *&^%$#&%$#$%*!^ Amount: $123,456 @*%$*^^^^%$@*) Status: Gold %#*@(*$%%%%#@
Compresses tape data Allows offline media recovery
Tape Library
Built upon FIPS level-3 system architecture Networkers Session BRKSAN-2893
BRKSAN-3707 14694_05_2008_X1
Cisco Public
54
27
Transparent Fabric Service

Application Servers
Integrates seamlessly with existing Cisco MDS fabrics Non-disruptive deployment (FC-R)
No appliances to insert in data path No SAN re-wiring or re-configuration
MPS-18/4
MPS-18/4
Redirects traffic flows after enabling encryption Highly saleable performance Load balances automatically Reliable, highly available service
Tape Library
Routes traffic to another MPS when one fails
BRKSAN-3707 14694_05_2008_X1
Cisco Public
55
Cisco SME Enabled Platforms

HIGH-PERFORMANCE INTEGRATED SOLUTION WITH MULTI-GIGABIT THROUGHPUT
18 4-Gbps ports for FC, 4 GigE for IP Services
MDS 9222i
MDS 9216A MDS 9216i
MDS 9506
MDS 9509
MDS 9513
18/4-Port Multiservice Module (MSM)
Cisco Fabric Manager w/Key Management Center

BRKSAN-3707 14694_05_2008_X1
Cisco Public
56
28
SME Cluster
Application Servers
Consists of up to four SME enable switches (nodes) in the same physical fabric Node-to-node communication via IPFC through management interface Quorum based cluster Provides scalability, reliability, availability and automatic load balancing
Scalability is achieved by adding additional line card in the fabric Target based load balancing Re-routes traffic when failure occurs
MSM-18/4
MSM-18/4
Tape Library
Single point of management with Cisco FM Can provide services across multiple VSANs One cluster per physical fabric
BRKSAN-3707 14694_05_2008_X1
Cisco Public
57
Cisco Key Management Center (KMC)

Cisco Key Management Center FMS Key Catalog DB Application Servers
Essential key lifecycle management

Archives, recovers, distributes, and shreds media keys
Transports keys and management traffic securely (SSH, HTTPS) Integrates with Cisco FM server
No additional software to install Intuitive provisioning and management with Cisco FM Web client
MSM-18/4 MSM-18/4 Fabric A
MSM-18/4 MSM-18/4 Fabric B
Tape Library
May use the local data base or the enterprise data base for the desired level of reliability and availability. Key Catalog data base options:
PostgreSQL Oracle 10g Express Third party key manager (ex: EMCs RSA)
BRKSAN-3707 14694_05_2008_X1
Cisco Public
58
29
Intelligent Fabric Applications SANTap
BRKSAN-3707 14694_05_2008_X1
Cisco Public
59
SANTap Intelligent Write Splitting

Initiators Initiators VSAN (SANTap) Initiator target I/O Not in primary data path Appliance Targets and Appliance VSAN
SANTap
SAN
Copy of primary I/O
Target
Appliance Partners leverages SANTap services

Part of the Cisco Storage Services Module (SSM)
Out-of-band architecture
SANTap redirects I/O and eliminates need for host splitter
BRKSAN-3707 14694_05_2008_X1
Virtual SAN configurations

60
30
SANTap Partner Solutions

Appliance Appliance
Network-Based Data Protection

Support heterogeneous storage and servers Integrated with Cisco MDS9000 SANTap Supports VMWare Virtual Machines (RDM)
CDP/CRR Recovery at Local or Remote Site

Tracks all data changes to every protected LUN Utilizes bookmarks for application-aware recovery Enables Read/Write processing of replicated LUNs
Heterogeneous Replication
Works with any supported storage True Any to Any Volume Replication
CRR Advanced WAN functionality

WAN data reduction and compression FC to TCP/IP conversion TCP Optimization
BRKSAN-3707 14694_05_2008_X1
Cisco Public
61
SANTap DeploymentBefore/After
BEFORE
Application Server
AFTER
Application Server Front-End VSAN
0 DVTLUNs 1 2 PRODUCTION VSAN SSM 9 Virtual Initiators 0 1 2 Storage Array CVT CVTLUNs 0 1 2 Storage Array AVTLUNs SSM Back-End VSAN AVT DVT Appliances Cluster
BRKSAN-3707 14694_05_2008_X1
Cisco Public
62
31
CRRAsynchronous Flow
Main Data Center
Application Server
Remote Data Center

Application Server
EMC RecoverPoint Asynchronous Replication

1 4 1. 2. 3. 4. Write I/O is sent to SSM module Write I/O is then forward to both local Storage Array and local Appliance Both local Storage Array and local Appliance acknowledge Write I/O back to the SSM Once SSM receives both acknowledgements, then sends acknowledgment to Application Server
3 SAN SSM 2 1
4 SSM SAN
WAN
2 3
Appliances
1. 2. 3. 4.
Appliances
I/O is sent through the WAN to remote Appliance I/O is then sent to replication LUN(s) through the SSM I/O is then acknowledged back to the Remote Appliance Remote Appliance then sends acknowledgement back to Primary Data Center Appliance through the WAN
Storage Array
Storage Array
63
SSM Line Card
X-Bar
2 Gbps each Forwarding Engine
2 Gbps each
2 Gbps DPP2
2 Gbps DPP3
2 Gbps DPP6
2 Gbps DPP7
2 Gbps DPP1
2 Gbps DPP4
2 Gbps DPP5
2 Gbps DPP8
DVT
Ports 1 4
Ports 5 8
Ports 9-12
Ports 13-16
Ports 17-20
Ports 21-24
Ports 25-28
Ports 29-32
BRKSAN-3707 14694_05_2008_X1
Host1
Cisco Public
64
32
SSM Line Card
X-Bar
2 Gbps each Forwarding Engine
2 Gbps each
2 Gbps DPP2
2 Gbps DPP3
2 Gbps DPP6
2 Gbps DPP7
2 Gbps DPP1
2 Gbps DPP4
2 Gbps DPP5
2 Gbps DPP8
DVT
Ports 1 4
Ports 5 8
Ports 9-12
Ports 13-16
Ports 17-20
Ports 21-24
Ports 25-28
Ports 29-32
BRKSAN-3707 14694_05_2008_X1
Host1
Cisco Public
65
Front-End VSANZoning
Only physical host initiators and DVTs reside in Front-End VSANs Normal zoning applies where Host Initiator is zoned with DVT NOTE: A single host initiator zoned with 2 or more separate DVTs, must make sure that all of those DVTs reside on the same DPP
Fabric-A
RecoverPoint Front-End VSAN 30
Fabric-B
RecoverPoint Front-End VSAN 40
Host1 Zone
Host1 HBA1 DVT1, DVT2
Host1 Zone
Host2 Zone
Host2 Zone
BRKSAN-3707 14694_05_2008_X1
Cisco Public
66
33
Back-End VSANZoning
Fabric-A SANTap Back-End VSAN Fabric-B SANTap Back-End VSAN
ApplianceTargets
SSM 9VIs APP1-P0 CVT APP2-P0
ApplianceTargets
SSM 9VIs APP1-P2 CVT APP2-P2
ApplianceInitiators
Storage Ports
APP1-P1 APP2-P1
ApplianceInitiators
Storage Ports
APP1-P3 APP2-P3
Appliance VTInitiators
AVT Initiators APP1-P1 APP2-P1
Appliance VTInitiators
AVT Initiators APP1-P3 APP2-P3
Appliance VTTarget
AVT Targets APP1-P0 APP2-P0
Appliance VTTarget
AVT Targets APP1-P2 APP2-P2
BE-Host Zone Host VI Storage Ports Appliance Local Storage
BE-Host Zone Host VI Storage Ports Appliance Local Storage Port Storage APP1-P3 APP2-P3
Storage Port APP1-P1 APP2-P1
BRKSAN-3707 14694_05_2008_X1
Cisco Public
67
SANTap Limits
Table 1: SANTap Limits
SSI Images
Max # of ITL per DPP Max # of ITL per SSM Max # of Sessions per SSM Max # of LUNs per Initiator per DVT Max # of LUNs per DVT Max # of host (initiators) per DVT Max # of DVTs per SSM Max # of DVTLUNs per SSM LUN ID Addressing size
3.0(2j)
1,024 1,024 1,024
3.1(2m)
1,024 2,048 2,048
3.1(3)
1,024 4,080 2,048
3.2(3i)
3,096 24,576 2,048
256 for all SSI images 1,024 16 16 1,024 16 1,024 16 16 2,048 16 1,024 16 32 4,096 16 3,096 64 64 16,384 32
BRKSAN-3707 14694_05_2008_X1
Cisco Public
68
34
Intelligent Fabric Applications Storage Virtualization
BRKSAN-3707 14694_05_2008_X1
Cisco Public
69
SAN-Based Storage Virtualization

Performance architecture
Leverages next-generation intelligent SAN switches
Scalable architecture
Virtual volumes Split-path architecture for high performance A stateless virtualization architecture does not store any information written by the application.
Meta-Data Meta-Data
High speed, high throughput data mapping Purpose-built ASICs (DPP) that handle and redirect I/O at line speed, with almost no additional latency Based on instructions provided by the MetaData Appliances
Multi-vendor arrays
Provides advanced functionality Supports heterogeneous environments
BRKSAN-3707 14694_05_2008_X1
Cisco Public
70
35
Storage Virtualization Logical Topology
Front-End VSAN
Pooled resources
Back-End VSAN
Virtual targets
Virtual initiators
BRKSAN-3707 14694_05_2008_X1
Cisco Public
71
Data Flows
Control Frame Data Frame
Meta-Data Appliance
IP
BRKSAN-3707 14694_05_2008_X1
Cisco Public
72
36
Network-Based Volume Management

Applications
Simplify volume presentation and management

Create, delete, change storage volumes Provides front-end LUN Masking and mapping of storage volume to hosts
Centralize management and control

Single Invista console to manage virtual volumes, clones, and mobility jobs
Virtual volumes
Reduce management complexity of a heterogeneous storage

Single management interface to allocate and reallocate storage resources
BRKSAN-3707 14694_05_2008_X1
Physical storage
73
Dynamic Volume Mobility Explained

Virtualization
Hosts see Storage Virtualization as an array Presents virtual volumes to hosts
Virtual Volumes
Virtual LUN: 10
Maps virtual volumes to physical volumes
To Move a Volume:
Data Path Controlle r Data Path Controlle r
Select source and target volumes Network synchronizes the volumes, then changes the virtualphysical mapping Array: 2 LUN: 30 No I/O disruption to host
Virtual initiators
Array: 1 LUN: 20
EMC
BRKSAN-3707 14694_05_2008_X1
HDS
Cisco Public
74
37
Heterogeneous Point-in-Time Copies

Applications
Create point-in-time copies

Source and clone can be on different, heterogeneous storage arrays
Enable replication across heterogeneous storage

Leverage existing storage investments Reduce replication storage capacity and management costs
Virtual volume
SAN
Active volume
Maximize replication benefits to support service levels

Backup and recovery Testing, development, and training Parallel processing, reporting, and queries Physical storage
Clone
Clone Clone
Data
75
VSAN Considerations
Back-End VSAN
Multiple Back-End VSAN supported by some partners Zone all 9 VIs to storage ports Best practice to create fcalias for all 9 VIs
HR VSAN 20
FC FC FC
Storage VSAN 10
VT1
MDS 9xxx
DEV VSAN 30
VT2
FC FC FC
Front-End VSAN
Up to 32 Virtual Targets per SSM Zone server HBA to one Virtual Target
VI 1- 9
Invista
VT3
FC FC FC
Control VSAN
Communication to external CPC Zone up IP interfaces for VSAN and SSMs CPP
ERP VSAN 40 ERP Admin
76
38
Q and A
BRKSAN-3707 14694_05_2008_X1
Cisco Public
77
Recommended Reading
Continue your Cisco Live learning experience with further reading from Cisco Press Check the Recommended Reading flyer for suggested books
Available Onsite at the Cisco Company Store

78
39
Complete Your Online Session Evaluation

Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Passport points for each session evaluation you complete. Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
Dont forget to activate your Cisco Live virtual account for access to all session material on-demand and return for our live virtual event in October 2008. Go to the Collaboration Zone in World of Solutions or visit www.cisco-live.com.
BRKSAN-3707 14694_05_2008_X1
Cisco Public
79
BRKSAN-3707 14694_05_2008_X1
Cisco Public
80
40

Advanced SAN Design - Virtualization Technologies and Intelligent Applications Design Considerations

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Advanced SAN Design - Virtualization Technologies and Intelligent Applications Design Considerations

Uploaded by

Copyright:

Available Formats

Advanced SAN Design Virtualization Technologies and Intelligent Applications Design Considerations

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Virtual Fabric: Three Key Concepts

Virtual Fabric Trunking and Port-Channels

Fabric Routing (IVR)

Trunk BRCD = Port Channel Cisco Group of ISLs = Port Channel

Ciscos Approach to Virtual Fabric: Virtual SAN (VSANs)

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

VSAN Numbering Rules

VSAN 1 is the default VSAN

VSAN 10 VSAN 20 VSAN 30

VSAN 4094 is a reserved as special VSAN

Host Is Isolated From the Fabric

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Standard Fibre Channel Frame Fields

SOF (Start of Frame) VSAN Header

Applicable to N_Ports, F_Ports and E_Ports

CRC EOF (End of Frame)

VSAN Header Field

VSAN Number 12 bits

CDL TTL Present 1 8

# PAD P_VL Bytes 2 4

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Trunking and Port-Channels

2008 Cisco Systems, Inc. All rights reserved.

EISLs and TE-Port

2. The EISL link

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

VSANEISL Establishment (Negotiation Protocol)

EISL Formed Or Isolated

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Port Channel Protocol (PCP)

Up to 160 Gbps Port Channel with HA

2008 Cisco Systems, Inc. All rights reserved.

VSANs, EISLs, TE_ports, Port ChannelsHow All These Work Together

8 Gbps PortChannel Trunking E_Port (TE_Port)

Trunking E_Port (TE_Port) 4 Link (8 Gbps) PortChannel Configured as EISL

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

VSANs and Non-Cisco Switches

Each Non-Cisco Switch Belongs to Only One VSAN

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Fabric Routing: Cisco Inter-VSAN Routing

Routed Virtual Fabric

2008 Cisco Systems, Inc. All rights reserved.

Fabric Routing Applications Sharing Common Resource

Common Physical Fabric

Tape Media Server

2006, Cisco Systems, Inc. All rights reserved. Presentation_ID.scr

Fabric Routing Applications SAN Extension Solutions

Works with any transport service (FC, SONET/SDH, DWDM/CWDM, FCIP)

Transit VSAN_3 (IVR)

EISL#2 in Port Channel

2008 Cisco Systems, Inc. All rights reserved.

2008 Cisco Systems, Inc. All rights reserved.