Storage Area

Network
Technologies

March 2016

 Copyright 2016 Hewlett Packard Enterprise Development LP

Course objectives

After completing this course, you should be able to:

– Explain disk connectivity options and disk technologies
– Identify storage area network (SAN) host components and technologies, including the HPE Virtual
Connect FlexFabric
– Discuss advanced Fibre Channel technologies such as Fibre Channel addressing, zoning, fabric
segmentation, and quality of service (QoS)
– Explain iSCSI SAN and technologies such as Net RAID
– Discuss SAN security
– Explain data protection terms and technologies
– Discuss storage area network design

Confidential – For Training Purposes Only 2

Disk technologies

Confidential – For Training Purposes Only 3

Parallel SCSI

– A SCSI standard established by ANSI in 1986, but still evolving

– The Common Command Set (CCS) was developed in parallel with the ANSI SCSI-1, SCSI-2, SCSI-3, and
SCSI-4 standards
– The SCSI-1 standard was too permissive and allowed too many vendor-specific options
– The result was incompatibility between products from different vendors, which made for confusion on:
– Speed and feed: Fast, Ultra, Ultra2, narrow, and wide
– Command sets: Common Command Set, Enhanced Command Set
– Termination: Passive, Active, Forced Perfect Termination

– Ultra320 and Ultra640 (AKA Fast-320) are the last offerings

IMPORTANT: When referring to SCSI disks, you need to know specific details about the interface type and signaling method.

NOTICE: Ultra640 standard reached the limits of speed/cable lengths, that made it impractical for more than two devices.
Most manufacturers skipped over Ultra640 for Serial Attached SCSI instead.

Confidential – For Training Purposes Only 4

Serial ATA (SATA)

– Hot-plug and Native Command Queuing (NCQ) support

– Transfer rates up to 300 MB/s for SATA2 and 600 MB/s for SATA3, using half-duplex
– SATA3.1 introduced support for Solid State Disks (SSD) and the Zero-Power Optical Disk Drive
– SATA3.2 combines SATA commands with the PCI Express interface to achieve device speeds
up to 16 Gb/s
– Mean Time Before Failure (MTBF) is 1.2 million hours

Confidential – For Training Purposes Only 5

Serial Attached SCSI

– SAS uses the full-duplex architecture, effectively doubling the transfer speeds
– The current SAS standard provides speed of 12 Gb/s, with a maximum theoretical speed of 16 Gb/s
– The maximum number of attached devices is 128 (compared to 16 for Parallel SCSI)
– A single SAS domain can address up to 65,535 devices using a fanout expander
– The MTBF is increased to 1.6 million hours

Confidential – For Training Purposes Only 6

Near-line SAS

– Serial Attached SCSI provides backward compatibility with SATA

– The near-line SAS drive is combination of a regular SATA drive with a SAS interface
– The near-line SAS drives enable all of the enterprise features of SAS
– Because near-line SAS uses SATA drives, performance and MTBF are limited by the SATA technology

Confidential – For Training Purposes Only 7

Native Command Queuing

What is Native Command Queuing (NCQ)?

– NCQ is a technology designed to increase the performance of SATA drives.
– Disks are enabled to internally optimize the order in which read/write commands are executed.
– NCQ is reducing the amount of unnecessary HDD head movement.
– NCQ is supported on the HPE Smart Array P400, P400i, E500, and P800 disk controllers.

Confidential – For Training Purposes Only 8

NCQ performance gains

Confidential – For Training Purposes Only 9

SAS domains
Disk Drives SAS SATA Disk Drives

SAS SATA SATA

Expander
Edge
Disk Drives

Expander

Expander
Edge

Edge
Disk Drives SAS

SATA SAS

Fanout Expander SAS Card

SAS Card SATA

Expander
Expander

Edge
Edge

SAS Card Disk Drives

SAS SATA SAS

Disk Drives
SAS SATA

Disk Drives

Confidential – For Training Purposes Only 10

Solid State Drives

– Based on Flash memory technology

– Use the same communication protocols
as magnetic disk drives SSD Controller

– Based on two technologies: Processor

[ Block Mgmt. ] NAND NAND
– Single-Level Cell (SLC) [ Wear Leveling ] NAND Flash Flash
Interface
[ ECC] Memory
– Multi-Level Cell (MLC) SATA Chip
Controller NAND Memory
Interface Interface
NAND NAND
DRAM
Flash Flash
[ Pointer Array ]
[ Cache]

NOTE: Solid State Hybrid Drives (SSHDs)

combine the large capacity of HDD with the Solid State Drive functional diagram
speed of the SSD used for caching to improve
performance and keep the price low.

Confidential – For Training Purposes Only 11

Single-level cell

– As the name suggests, SLC Flash stores one bit value

per cell, which basically is a voltage level
– The bit value is interpreted as a “0” or a “1”

– Because there are only two states, it represents only

one bit value
– Each bit can have a value of “programmed” or “erased”

Value State

0 Programmed

1 Erased

Confidential – For Training Purposes Only 12

Multi-level cell

– An MLC cell can represent multiple values

– These values can be interpreted as four distinct states:
00, 01, 10, or 11

Value State

00 Fully programmed

01 Partially programmed

10 Partially erased

11 Fully erased

Confidential – For Training Purposes Only 13

Comparing SLC and MLC

Characteristic SLC MLC

Density 16 Mb 32 Mb 64 Mb
Read speed 100 ns 120 ns 150 ns
Block size 64 Kb 128 Kb
Architecture x8 X8/x16
Endurance 100,000 cycles 10,000 cycles
Operating temperature Industrial Commercial

NOTE: MLC is less desirable for use in server storage.

Confidential – For Training Purposes Only 14

SSD wear leveling

What is wear leveling?

– Technology used to increase the overall endurance of
NAND-based SSDs
– Each NAND cell supports up to 100,000 read/write
operations
– Wear leveling continuously remaps logical SCSI blocks
to different physical pages in the NAND array, ensuring
that reads and writes remain equally distributed
– Logical-to-physical mapping is maintained as a pointer
array in the high-speed DRAM on the SSD controller
– This index is then copied to a special region of NAND to
enable rebuilding of the map in the case of a sudden power
loss

Confidential – For Training Purposes Only 15

SSD over-provisioning

– On high-end SSDs, it is possible to over-provision by 25% above the stated storage capacity
– Distributes the total number of reads and writes across a larger population of NAND blocks and pages
over time
– The SSD controller gets additional buffer space for managing page writes and NAND block erases

Confidential – For Training Purposes Only 16

SmartSSD Wear Gauge

NOTE: SmartSSD Wear Gauge is part of the Array Configuration

Utility (ACU) in the HPE Intelligent Provisioning that is embedded in
HPE ProLiant Gen8 and newer servers.

Confidential – For Training Purposes Only 17

Disk enclosures

– A disk enclosure is a specialized casing designed to hold

and power disk drives while providing a mechanism to
allow them to communicate to one or more separate
computers
– In enterprise terms, “disk enclosure” refers to a larger
physical disk chassis
– Disk enclosures do not have RAID controllers
– Disk enclosures can be connected directly to the hosts

HPE D2700 6Gb Drive Enclosure

Confidential – For Training Purposes Only 18

Fault-tolerant cabling

– Fault-tolerant cabling allows any drive enclosure to fail or

be removed while maintaining access to other
enclosures
– P2000 G3 Modular Storage Array (MSA)
– Two D2700 6Gb enclosures
– The I/O module As on the drive enclosures are shaded green
– The I/O module Bs on the drive enclosures are shaded red

Confidential – For Training Purposes Only 19

Straight-through cabling

– Straight-through cabling can sometimes provide

increased performance in the array, it also increases the
risk of losing access to one or more enclosures in the
event of an enclosure failure or removal
– P2000 G3 Modular Storage Array (MSA)
– Two D2700 6Gb enclosures
– The I/O module As on the drive enclosures are shaded green
– The I/O module Bs on the drive enclosures are shaded red

Confidential – For Training Purposes Only 20

LUN masking

– Enables host visibility of LUNs within the storage array

– LUN granularity
– Independent of zoning
– Can be implemented at the host, fabric, or array level
– Used for data security
– Selective Storage Presentation on HPE 3PAR and EVA
Arrays

Confidential – For Training Purposes Only 21

Storage virtualization

HPE 3PAR Storage Virtualization Scheme

Confidential – For Training Purposes Only 22
Fat (thick) or thin provisioning

Allocated & Available

Unused Storage
Volume B
Data

Allocated &
Data Volume B
Unused
Volume A
Data Data Volume A

Traditional Thin
Allocation Provisioning

Confidential – For Training Purposes Only 23

 HPE Storage Arrays
HPE XP7
HPE XP
HPE 3PAR P9500
EVA P6000 StoreServ 10000
Consolidation and performance

Storage

HPE 3PAR
StoreServ 7000

HPE P2000 4 PB maximum

MSA Array 247 PB external
HPE 2 PB maximum
System storage maximum
StoreVirtual 255 PB external
1200TB maximum 3.2 PB maximum
4000 Storage 720 TB maximum storage maximum
384 TB maximum 8 Gb FC ports 8 Gb FC ports
8 Gb FC ports
1/8 Gb FC ports 10 Gb/sec iSCSI 10 Gb/sec iSCSI ports
1536 TB maximum 1/10 Gb/s iSCSI
6 Gb/sec SAS ports (8) 3PAR quad-core
1/10 GbE iSCSI ports
ports (4) 3PAR 7000 2.8GHz P10000
ports
1/10 GbE iSCSI 6-core 1.8 GHz controller nodes
8 Gb FC ports ports controller nodes

Business continuity and availability

Confidential – For Training Purposes Only 24

Storage area network hosts

Confidential – For Training Purposes Only 25

SAN hosts

– Multiple HBAs to connect to different SAN fabrics

– Need to be members of a zone in each fabric
– Need to have visibility to the disk array ports within the
zone to allow them to map storage presentations
– Might have additional multipath drivers or software to
enable failover and policy-based load balancing in a
redundant fabric SAN design

Confidential – For Training Purposes Only 26

Hosts and Fibre Channel

– To communicate with Fibre Channel infrastructure, the

host requires a host bus adapter (HBA)
– Each HBA port physically connects to the fabric and
becomes visible to the SAN
– Port behavior depends on the HBA driver configuration
and type and on the configuration of the fabric port

Confidential – For Training Purposes Only 27

Converged network adapter

Converged network adapter (CNA) combines:

– Traditional host bus adapters for Fibre Channel (FC-HBA) and Ethernet network interface cards (NICs)
– Fibre Channel over Ethernet (FCoE) protocol
– Converged Enhanced Ethernet (CEE)

FC-4
Fibre Channel
FC-3 Services
FC-2
FCoE
CEE
Ethernet
Physical

Confidential – For Training Purposes Only 28

N_Port ID virtualization

What is NPIV?
– N_Port ID Virtualization (NPIV) is an industry-standard Fibre Channel protocol that provides a means to
assign multiple Fibre Channel addresses on the same physical link.
– NPIV makes a single Fibre Channel port appear as multiple virtual ports, each having its own N_Port ID
and virtual WWN.
– HPE offers an NPIV-based Fibre Channel interconnect option for server blades called Virtual Connect.

Confidential – For Training Purposes Only 29

NPIV

NPIV allows a single HBA, called an “N_Port,” to register multiple World Wide Port Names (WWPNs) and
N_Port identification numbers.

Confidential – For Training Purposes Only 30

Server virtualization with NPIV

Confidential – For Training Purposes Only 31

HPE Virtual Connect Fibre Channel

Blade SAN Configuration with Virtual Connect FC

HPE Blade Enclosure

Server 1
VC-FC
SAN Switch
FC SAN
Module Fabric
Server 2

Server 3

VC-FC
SAN Switch
Module
Server 16

N-ports
N-ports F-ports F-ports
with NPIV

Confidential – For Training Purposes Only 32

HPE Virtual Connect FlexFabric

– Up to four physical functions for each server blade

adapter network port
– The physical function corresponds to the HBA
– Four physical functions share the 10 Gb link
– One of the four physical functions can be defined as the
Fibre Channel HBA, and the remaining three will act
as NICs
– Each physical function has 100% hardware-level
performance, but the bandwidth might be fine-tuned to
quickly adapt to virtual server workload demands

Confidential – For Training Purposes Only 33

Boot from SAN

What is boot from SAN?

– The process of booting a server using external storage
devices over a SAN
– Used for server and storage consolidation
– Minimizes server maintenance and reduces backup time
– Allows for rapid infrastructure changes

Confidential – For Training Purposes Only 34

Multipath concept

– Multipath I/O (MPIO) provides automatic path failover Server Server

between the server and the disk arrays
– Some multipath solutions provide load balancing over
multiple HBA paths A1 A2 A1 A2

Without multipathing software, Without multipathing software,

the host incorrectly interprets the the server incorrectly interprets
two paths as leading to two the two paths as leading to the
storage units same storage unit

Confidential – For Training Purposes Only 35

Path failover

– Failover is handled by MPIO, and it is supported via

services, drivers, and agents
– It is transparent to the applications
– The administrator has to configure the primary and
alternate paths

Confidential – For Training Purposes Only 36

Load balancing

– MPIO provides load balancing across all installed HBAs (ports) in a server
– There are various load-balancing policies, depending on the multipath software:
– Round robin
– Least I/O
– Least bandwidth
– Shortest queue (requests, bytes, service time)

Confidential – For Training Purposes Only 37

Microsoft Multi-Port IO

– Uses redundant physical paths to eliminate single points of failure between servers and storage
– Increases data reliability and availability
– Reduces bottlenecks
– Provides fault tolerance and load balancing
– Two components:
– Drivers developed by Microsoft
– Device-specific modules (DSMs) developed by storage vendors to Microsoft standards

NOTE: Starting with Windows Server 2008, Microsoft provides

native multipathing (Microsoft MPIO) software.

Confidential – For Training Purposes Only 38

Fibre Channel advanced features

Confidential – For Training Purposes Only 39

Fibre Channel addressing

– Fibre Channel switch ports use a 24-bit address scheme

– Allows for 16 million addresses

32 bit 24 bit 24 bit

Control End of
start of dest port src port Frame payload
information CRC frame
frame address address (0 – 2112 bytes)
Words 3-6 delimiter
Word 0 Word 1 Word 2

SOF 24 byte FC header SCSI data CRC EOF

Confidential – For Training Purposes Only 40

Fibre Channel name and address

– 24-bit addresses are automatically assigned by the topology to remove the overhead of manual
administration
– Unlike the WWN addresses, port addresses are not built-in
– The switch is responsible for assigning and maintaining the port addresses
– The switch maintains the correlation between the port address and the WWN address of the device
on that port
– The Name server is a component of the fabric operating system running on the switch

Confidential – For Training Purposes Only 41

Fibre Channel port address (1 of 2)

– A 24-bit port address consist of three parts:

– Domain—Bits from 23 to 16
– Area—Bits from 15 to 08
– Port or arbitrated loop physical address (AL_PA)—Bits from 07 to 00

8 bits 8 bits 8 bits

Domain Area Port

239 addresses 256 addresses 256 addresses

Confidential – For Training Purposes Only 42

Fibre Channel port address (2 of 2)

Domain Area Port

– The address of the switch itself
– 256 possible addresses, but some
bits are reserved
– Only 239 addresses are actually
available
– This means that you can have up to
239 switches in your SAN
environment
– Provides 256 addresses
– Identifies the individual FL_Ports
supporting loops
– Can be used as the identifier of a
group of F_Ports
– Provides 256 addresses
– Identifies the attached N_ports and
NL_Ports

Available addresses:

239 x 256 x 256 = 15,663,104

Confidential – For Training Purposes Only 43

Simple Name Server

– The Name server stores information about all of the devices in the fabric
– An instance of the Name server runs on every Fibre Channel switch in a SAN
– A switch service that stores names, addresses, and attributes for up to 15 minutes and provides them as
required to other devices in the fabric

Confidential – For Training Purposes Only 44

10-bit addressing mode

– The number of physical ports on the switch is limited to 256 by the number of bits in the Area part of the
Fibre Channel address.
– Director switches, such as Brocade DCX and DCX 4, support Virtual Fabric, where the number of required
ports might easily grow to more than 256.
– To support up to 1,024 ports in a Virtual Fabric, use the 10-bit addressing mode.
– The 10-bit addressing mode uses the 8-bit Area_ID and the borrowed upper 2 bits from the AL_PA portion
of the port ID.
8 bits 8 bits 2 bits 6 bits

Domain Area Port

239 addresses 1024 addresses 64 addresses

Confidential – For Training Purposes Only 45

Arbitrated loop addressing

– In an arbitrated loop, only one of the three bytes is used byte 0 byte 1 byte 2
– The least significant 8 bits
– Known as the AL_PA
23 16 15 8 7 0

N_Port identifier

FL_Port Loop identifier AL_PA

Confidential – For Training Purposes Only 46

Arbitrated loop order sets

– An ordered set is a group of four transmission characters.

– An arbitrated loop has several order sets that are used in:
– Loop arbitration
– Opening of loop circuits
– Closing of loop circuits

– Loop arbitration is a complex process of transmitting

signals (order sets).
– The two types of order sets are:
– Frame delimiters—Exists at the start or the end of the frame
– Primitive—Order sets without frames

Port Arbitration Activity Example

Confidential – For Training Purposes Only 47

Fabric flow control

– An Arbitrated Loop uses arbitration, and a switched fabric uses flow control to prevent data overruns at the
receiver side.
– Fibre Channel implements a credit-based flow-control mechanism to prevent frame dropping.
– The transmitter (Tx) can send frames in the amount of the buffer-to-buffer (B2B) credits reported by the
receiver (Rx).
– For each packet sent, the Rx port needs to send an R_Rdy (Receiver_Ready, Fibre Channel Primitive)
signal.

Confidential – For Training Purposes Only 48

Types of flow control

Fibre Channel defines two types of flow control:

– Buffer-to-buffer (port to port)
– End-to-end (source to destination)

Buffer-to-Buffer Flow Control

N_port E_port E_port E_port E_port N_port

End-to-End Flow Control

Confidential – For Training Purposes Only 49

Fibre Channel class of service

– Fibre Channel defines several classes of service (CoS), which can be used by applications to provide the
optimal type of delivery priority and flow control, depending on the type of application data.
– Each CoS uses a connection-oriented, packet-switched, or quality of service (QoS) communication
strategy.

Confidential – For Training Purposes Only 50

Fabric zoning

– A method of restricting server access to storage

resources that are not allocated to that server
– Similar to LUN masking
– Implemented on the switch
– Operates on the basis of port identification (WWPN)
– Zoning types:
– Software based—Restricts only the fabric name service to
show only an allowed subset of devices
– Hardware based—Restricts the actual communication across
a fabric
– Port based—Zoning applied to the switch port to which a
device is connected
– WWN based—Zoning that restricts access by a device WWN

Confidential – For Training Purposes Only 51

Hard and soft zoning

Hard zoning
– A member is identified by its port number
– Known as “hard” zoning
– Enforced by a switch at a hard level
– Soft zoning
– A member is identified by its port WWN
– Known as “soft” zoning
– Enforced by the Name server, which returns filtered
responses to port queries

Confidential – For Training Purposes Only 52

Software zone enforcement

– The Name server service in the fabric masks the Name server entries that a host should not access.
– When the host logs in to the fabric, it discovers only the unmasked Name server entries.
– Software-enforced zoning has no mechanism that prevents a host from accessing storage.

Confidential – For Training Purposes Only 53

Hardware zone enforcement

– Hardware enforcement
– Frame-based
– Session-based

– Performed by the Application-Specific Integrated Circuits (ASICs) in fabric switches

– A proactive security mechanism
– Every port has a filter that allows only the traffic defined by the zoning configuration to pass through

Confidential – For Training Purposes Only 54

Zoning decisions
Zoning by HBA
– Each zone has one HBA (the
initiator)
– Each of the target devices is added
to the zone
– If the HBA also accesses tape
devices, a separate zone is created
for the HBA and the associated
tape devices
– Zoning by HBA limits disruptions
and the number of fabric change
notifications
Zoning by OS
– The minimum required zoning
method
– Multiple HBAs with the same
operating system are grouped with
the accessed storage ports
– Prevents the interaction of the
HBAs with incompatible operating
systems
Zoning by application
– Combines multiple operating
systems in the same zone
– Allows for potential disruptions
– More susceptible to administrative
errors
Confidential – For Training Purposes Only 55
Traffic isolation zones

– Traffic isolation (TI) allows data paths to be specified

– TI zoning has the following benefits:
– Separates direct attached storage device (DASD) and tape
traffic
– Selects traffic for diverse Inter-Switch Link (ISL) routes
– In conjunction with long-distance channel extension
equipment, it guarantees bandwidth for certain mission-critical
data

Confidential – For Training Purposes Only 56

Brocade QoS zones

– A Quality of Service (QoS) zone adds traffic-shaping

capabilities to regular zones
– The priority of a traffic flow is set to High or Low, based
on the name of the zone
– High-priority zone name: QoSH<id>_<zonename>
– Low-priority zone name: QoSL<id>_<zonename>

Confidential – For Training Purposes Only 57

LSAN zones

– A Logical SAN provides device connectivity between

fabrics without merging the fabrics
– It consists of zones in two or more edge or backbone
fabrics that contain the same devices
– Members must be identified by their port WWN, because
port IDs are not necessarily unique across fabrics

Confidential – For Training Purposes Only 58

Fabric segmentation

– Fabric segmentation occurs when two or more switches

are joined together by ISLs but they do not communicate
with each other
– Possible causes for fabric segmentation are:
– Zone type mismatch
– Zone content mismatch
– Zone configuration mismatch
– Duplicate domain IDs

Confidential – For Training Purposes Only 59

iSCSI storage area network

Confidential – For Training Purposes Only 60

The value of an iSCSI SAN architecture

– Built on top of a dedicated or existing Gigabit Ethernet

infrastructure
– Uses the familiar TCP/IP technology
– The IP protocol is universal and it works seamlessly,
regardless of the equipment vendor
– Customers can leverage the 10Gb Ethernet
– iSCSI components can be virtualized
– Removes distance limitations

Confidential – For Training Purposes Only 61

Built for virtualization
HPE StoreVirtual Technology

– Simple management for virtualized environments

Simple – All-inclusive licensing with enterprise-class storage features
– Virtualization platform integration for increased functionality and ease of use

– Nondisruptively scale performance and capacity

Scalable
– One homogenous storage pool with iSCSI and Fibre Channel connectivity

– Proven five 9s high availability and reliability

Highly
– Multisite disaster recovery with transparent failover
available
– Online data mobility across systems, locations, and technology changes

Confidential – For Training Purposes Only 62

HPE StoreVirtual for iSCSI and Fibre Channel

For customers:
– With disjointed storage pools across Fibre Fibre Channel 10GbE IP network
Channel and iSCSI networks (SAN/iQ OS/iSCSI)
Centralized network
– Leverages a single storage architecture for Management
all applications in the enterprise Console
iSCSI
– Standardizing on Ethernet-based technologies FC
clients
– Provides easier migration options when going clients
from Fibre Channel to iSCSI

– Looking for an all-inclusive enterprise feature set

HPE StoreVirtual 4330 or 4730

Confidential – For Training Purposes Only 63

HPE StoreVirtual storage clustering

HPE StoreVirtual offers storage clustering as a way to:

– Aggregate all components for performance
– Load balance data across all nodes
– Offer nondisruptive scalability
– Create a tiered environment for different performance
requirements
– Offer online volume migration
– Simplify management through a centralized
management console

Confidential – For Training Purposes Only 64

Scale-out architecture

Start with the current needs

– Use storage nodes to build clusters
– Leverage all critical resources
Grow as needed
– Scale performance and capacity linearly
– Data remains online as you grow
Build single or multiple tiers
– SSD, SAS, and Nearline SAS clusters
– Migrate data with Peer Motion

Confidential – For Training Purposes Only 65

Seamless and nondisruptive data mobility

StoreVirtual Peer Motion

– Seamlessly move volumes between:
– Systems
– Tiers
– Locations

Peer Motion
– Form factors
– Disk types 43
30

– Different generations 43
30

43
30

– Physical and virtual platforms 43

30

– In a matter of minutes—Swap out or swap in entire

clusters and upgrade technology nondisruptively
– All data remains online and available

Confidential – For Training Purposes Only 66

Network RAID

– Creates redundant copies of blocks that reside on

different storage nodes
– The mirroring level cannot exceed the number of nodes
in the cluster
– Supports 2-, 3-, and 4-way mirroring
– Requires 2, 3, or 4 times as much storage

Confidential – For Training Purposes Only 67

Network RAID 0

– Every block of data will be written once.

– Blocks are striped across the nodes.
– The failure of one node means the loss of the whole
volume because there is no redundancy.

Confidential – For Training Purposes Only 68

Network RAID 10

– Network RAID 10 data is striped and mirrored across

two storage systems.
– Network RAID 10 is the default data protection level
assigned when creating a volume, as long as there are
two or more storage systems in the cluster.
– Data in a volume configured with Network RAID 10 is
available and preserved in the event that one storage
system becomes unavailable.

Confidential – For Training Purposes Only 69

Network RAID 10+1

– Network RAID 10+1 data is striped and mirrored across

three or more storage systems.
– Data is available and preserved in the event that any two
storage systems become unavailable.

Confidential – For Training Purposes Only 70

Network RAID 10+2

– Network RAID 10+2 data is striped and mirrored across

four or more storage systems.
– Data is available and preserved in the event that any two
storage systems become unavailable.
– Network RAID 10+2 is designed for multisite SANs, to
preserve data in the event of an entire site becoming
unavailable.

Confidential – For Training Purposes Only 71

Network RAID 5

– Data is divided into stripes.

– Each stripe is stored on three of the storage systems,
and parity is stored on the fourth system.
– Data is available and preserved in the event that any
single storage system becomes unavailable.

Confidential – For Training Purposes Only 72

Network RAID 6

– Network RAID 6 divides the data into stripes.

– Each stripe is stored on four of the storage systems in
the cluster, and parity is stored on the fifth and sixth
systems.
– Data is preserved and available in the event that any two
storage systems become unavailable.

Confidential – For Training Purposes Only 73

Storage area network security

Confidential – For Training Purposes Only 74

SAN security

– Storage security is the act of protecting the data that resides in the SAN from unauthorized access.
– Security is an Internet Protocol (IP) issue, not a Fibre Channel issue.
– To provide proper protection, all aspects of data security must be addressed.
– On average, more resources are spent on protecting web servers than on protecting SANs.

Confidential – For Training Purposes Only 75

Security model

– The security of a computer system is responsibility of a security manager.

– Three types of attacks, corresponding to the three aspects of information security:
– Data can be made unavailable for access
– Data can be deleted or modified without permission
– Data can be examined without permission

– Security can be implemented at three levels in the SAN:

– Storage array level
– Fabric level
– Host level

Confidential – For Training Purposes Only 76

Elements of storage security

Confidential – For Training Purposes Only 77

Differentiating data security and data protection

– Data protection deals with information dependability

– Reliability, availability, fault protection, performance, and so on

– Information security includes the following core principles:

– Confidentiality – Utility
– Integrity – Privacy
– Availability – Authorized use
– Possession – Nonrepudation
– Authenticity

Confidential – For Training Purposes Only 78

Transitive trust problem

– SAN security must not be treated separated from the security of other parts of IT infrastructure
such as networking.
– If there is a network security breach, SAN data becomes exposed even if the storage infrastructure
remains intact.
– Risk mitigation includes:
– Identification (authentication)
– Authorization (LUN and tape access permissions)
– Audit
– Encryption (data on disk and tape and data in transit)

Confidential – For Training Purposes Only 79

SAN security: Where and how to implement it

Where? How?
Enable authentication for: To prevent unauthorized access:
– User – Use multilevel passwords.
– Management – Use Access Control Lists (ACLs).
– Server – Use centralized access control or Domain
authentication.
– Switch

Confidential – For Training Purposes Only 80

Fabric Access Control Lists (ACLs)

– The Brocade Fabric OS provides following policies:

– Fabric configuration server (FCS) policy
– Device connection control (DCC) policies
– Switch connection control (SCC) policy

– The FCS, DCC, and SCC policy members are specified by the device port WWN, the switch WWN,
domain IDs, or switch names, depending on the policy.

Valid methods for specifying policy members

Policy name Device port WWN or Switch WWN Domain ID Switch name
Fabric port WWN

FC_POLICY No Yes Yes Yes

DCC_POLICY_nnn Yes Yes Yes Yes

SCC_POLICY No Yes Yes Yes

Confidential – For Training Purposes Only 81

Device authentication

– The authentication of devices is an effort expended by a device to ensure the identity of another device
with which it is communicating.
– Levels of authentication:
– None
– Trusting the device address
– Challenging the device to prove its identity

Confidential – For Training Purposes Only 82

Device authorization

– Authorization is used to perform the selective presentation of devices and LUNs.

– Levels of authorization:
– No authorization
– Used on DAS
– LUN masking and selective LUN presentation based on the WWN
– iSCSI
– By using ACLs at the device level or per LUN
– NAS
– Authorization using supported operating system methods

Confidential – For Training Purposes Only 83

Data encryption

– Data transferred across an untrusted connection must be secured

– Data encryption is necessary to prevent unauthorized access in the case of lost media
– Lost CD, DVD, tape, or disk

– In general, data can be encrypted:

– In flight
– Fibre Channel, Ethernet, WAN
– At rest
– On a disk or tape

Confidential – For Training Purposes Only 84

Management security

– An important aspect of security that is applicable to SANs

– Management security includes:
– The authentication of administrators
– Single sign-on technologies (Active Directory, LDAP, and so on)
– Selective administration capability
– Role-based access
– Error tracking
– A centralized management view

Confidential – For Training Purposes Only 85

Data protection

Confidential – For Training Purposes Only 86

Data protection overview

– The primary goal of data protection is to maintain the availability of data.

– RAID is designed to protect data against bit and byte errors.
– RAID is not backup!

Confidential – For Training Purposes Only 87

Challenges in data protection

– Long backup windows

– Long recovery times
– Protection gaps
– Inconsistent recovery
– Impacts on production applications
– Disaster recovery
– Compliance

Confidential – For Training Purposes Only 88

Recovery Time and Recovery Point Objectives

– The Recovery Time Objective (RTO) is the goal for how quickly you need to have your information
available after downtime has occurred.
– The Recovery Point Objective (RPO) describes the point in time to which data must be restored to
successfully resume processing.

RPO RTO

Time

Last Backup Event Data Restored

Confidential – For Training Purposes Only 89

Data protection

– Physical tapes
– Traditional destination for backup sets
– Shelf life of up to 30 years
– Requires tape library solutions to handle complex backup
environments

– Virtual Tapes
– Replication
– Local
– Remote

– Clustering

Confidential – For Training Purposes Only 90

Data protection topologies

Direct backup
– A fast but expensive solution
– Data is backed up to locally attached tape drives
– Complex administration

Centralized server backup

– Client-server architecture
– One server has a tape library attached
– Uses a LAN to transport data
– The LAN might become a bottleneck

Centralized SAN backup

– A LAN is used only to initiate and control a data backup
– Data is moved over the SAN
– Tape libraries are connected to the SAN fabric

Confidential – For Training Purposes Only 91

Tape libraries

– Dedicated SAN-based devices

– High performance, capacity, and availability
– Compatible with the latest tape technologies
– Contain sophisticated robotics to automate tape-
changing
– Provide data encryption to comply with standards

Confidential – For Training Purposes Only 92

Virtual tape libraries

– Emulate physical tapes and libraries to back up software

– Capable of supporting parallel jobs
– Reduce backup time
– Granular recovery enables fast single-file restores
– Fibre Channel and iSCSI connectivity

HPE StoreOnce Backup

Confidential – For Training Purposes Only 93

Remote Copy introduction

– Array-based remote replication solution for an

HPE 3PAR Storage Array
– Supported modes:
– Synchronous
– Asynchronous periodic
– Asynchronous long distance

– Supported transports:
– Fibre Channel
– Ethernet
– Fibre Channel over IP

Confidential – For Training Purposes Only 94

Fibre Channel-based Remote Copy

– High performance
– Used for campus-distance solutions
– Offers low latency and high bandwidth

– Flexible
– Direct or Fibre Channel SANs are supported
– Extended-distance technologies
– Longwave links
– FCIP bridging or routing

Confidential – For Training Purposes Only 95

Native IP-based Remote Copy

– Native IP eliminates the need for expensive converters

– Distance flexibility
– Cost-effective

– Designed to be transport agnostic

– Native Gigabit Ethernet TCP/IP today
– Other protocols will be quickly assimilated

Confidential – For Training Purposes Only 96

Synchronous mode operation

2 4

Data is written to Data is written to

cache on two the cache on two
Host nodes nodes
server
3
1

Write request Write request

is forwarded
Secondary or
6 Primary Storage Backup Storage
Array Array
5
Primary
acknowledges Secondary or Backup
the Host acknowledges the
Primary

Confidential – For Training Purposes Only 97

Asynchronous periodic operation
2
Only the most
Data is written to recent data is
the cache on two written to the cache
Host nodes on the nodes
server
1

Write request
Only the most
recent data is
Primary Storage copied over, Secondary or
3 “deltas” Backup Storage
Array
Array
Primary
acknowledges
the Host Scheduled or manual resynchronization

Confidential – For Training Purposes Only 98

Synchronous long distance
Fibre Channel sync mode
Bidirectional between Source

A
and Sync targets A’
– The same volume is protected on two arrays.
– One in synchronous mode
Metropolitan distance
– One in asynchronous mode B’ (Source – Sync site) B
– Customers need to replicate the delta changes from one
Source Sync Site,
of the disaster recovery sites in case of a failure
Target 1
– In the case of a failure, a full sync of a volume is not
required

A’
DR Site,
Target 2

Continental distance
(Source, Sync – DR Site)

Confidential – For Training Purposes Only 99

Deduplication

2 4

1. Data from the first backup stream are stored to disk

2. Duplicate data as well as unique data in second backup stream are identified
3. Duplicate data in the second backup streams are eliminated
4. Uniques data in the second backup stream mare stored to disk

Confidential – For Training Purposes Only 100

Deduplication in remote and branch office setups

Confidential – For Training Purposes Only 101

Storage area network design

Confidential – For Training Purposes Only 102

SAN design considerations

Distance &
Geographic
Layout

Connectivity & Scalability

Capacity

Availability Performance

Management &
Security

Confidential – For Training Purposes Only 103

Planning considerations

– Inventory of the current environment

– Growth plan
– Current storage configuration
– LAN and SAN structure
– Application uses
– Traffic loads
– Peak periods
– Current performance
– Current constraints
– Use of the existing fiber cables
– Use of the existing components

Confidential – For Training Purposes Only 104

HPE standard supported SAN topologies

HPE simplified design HPE SAN design considerations

– Three approaches to designing a SAN – Based on the scope and requirements for a given
– You can implement: business application, HPE SAN topologies depend on
– An HPE standard SAN fabric topology design
the required:
– A subset or variation of an HPE standard SAN fabric topology design – Size
– A custom SAN fabric topology design – Availability
– Performance
– Extendibility

NOTE: HPE SAN design rules are explored in the SAN Design
Guide available at: http://www.hp.com/go/sandesign .

Confidential – For Training Purposes Only 105

HPE SAN Design Reference Guide

Confidential – For Training Purposes Only 106

SAN fabric topology overview

Single-switch fabric

Cascaded fabric

Meshed fabric

Ring fabric

Core-edge fabric

Confidential – For Training Purposes Only 107

Single-switch fabric

The smallest SAN, consists of:

– A Fibre Channel switch
– A storage system
– A server

The benefits of a single-switch fabric include:

– Easy installation and configuration of servers and storage
– Maximum fabric performance because all communicating
devices connect to the same switch
– Support for local, centralized, and distributed data access
needs

Confidential – For Training Purposes Only 108

Cascaded fabric

Cascading enables you to:

– Achieve optimum I/O activity by connecting servers and
storage to the same switch in a cascaded fabric
– Easily scale the fabric over time by adding cascaded switches

The benefits of a cascaded fabric include:

– The ability to connect SANs in diverse geographic locations
– Ease of scalability for increased server and storage
connectivity
– Shared backup and management support
– Optimum local performance when communicating devices are
connected to the same switch in the cascaded fabric
– Cost efficiency resulting from the large number of available
switch ports
– Support for local data access and the occasional centralized
data access

Confidential – For Training Purposes Only 109

Meshed fabric

Built on top of a group of switches, uses multiple ISLs

for fabric resiliency
– If one ISL fails, data is automatically rerouted through an
alternate path in the fabric

The benefits of a meshed fabric include:

– The ability to meet multiple data access needs
– Multiple paths for internal fabric resiliency
– Ease of scalability
– Shared backup and management support
– Less impact on performance from intra-switch traffic

Confidential – For Training Purposes Only 110

ISL connections in a meshed fabric

Confidential – For Training Purposes Only 111

Ring fabric

A ring of interconnected switches that enables you to:

– Scale the fabric in a modular fashion
– Achieve optimum I/O performance by connecting a group of
servers and storage to one switch

The benefits of a ring fabric include:

– Modular design and ease of scalability by adding a switch and
other devices
– Multiple paths for internal fabric resiliency
– Support for a mix of local data access and the occasional
centralized data access

Confidential – For Training Purposes Only 112

Ring fabric with satellite switches

Confidential – For Training Purposes Only 113

Core-edge fabric (1 of 2)

HPE recommends using a core-edge fabric wherever

possible
– A core-edge fabric has one or more Fibre Channel switches
(called core switches) that connect to the edge switches in the
fabric

The core-edge fabric is optimal for:

– Many-to-many connectivity environments that require high
performance
– Unknown or changing I/O traffic patterns
– SAN-wide storage pooling

Core-edge fabric (typical depiction)

Confidential – For Training Purposes Only 114

Core-edge fabric (2 of 2)

Core-edge fabric topologies can be depicted

hierarchically
– The physical implementation is typically the same as in the
depiction

The benefits of a core-edge fabric include:

– Typically, a maximum of two hops between switches
– Equal, centralized access to the devices in the core
– Increased fabric and switch redundancy with two or more
switches in the core
– Full many-to-many connectivity with evenly distributed
bandwidth
– Support for centralized and distributed data access
– The ability to designate an optimally located core switch as
the primary management switch, with direct connections to all Core-edge fabric (hierarchical depiction)
switches

Confidential – For Training Purposes Only 115

Topology data access

– Local (one-to-one)
– Data access between a local server and a storage system connected to the same switch

– Centralized (many-to-one)
– Data access between multiple, dispersed servers and one centrally located storage system

– Distributed (many-to-many)
– Data access between multiple, dispersed servers and storage systems

Confidential – For Training Purposes Only 116

Data access performance by SAN fabric topology

SAN topology Data access performance

Local Centralized Distributed
Single-switch fabric Highest Highest Highest
Cascaded fabric Highest Not recommended Not recommended
Meshed fabric Medium Medium High
Ring fabric Highest Medium Not recommended
Core-edge fabric (15:1, 7:1) Medium High High
Core-edge fabric (3:1, 1:1) High Highest Highest

Confidential – For Training Purposes Only 117

Topology maximums

– The maximum number of supported switches and ports for specific fabric topologies can vary.
– The number of switches and ports depends on:
– The number of hops in the fabric topology
– The number of ISLs

– Consider the following:

– User ports are for servers and storage.
– It is assumed that you have the minimum number of ISLs.
– If you require more ISLs, this reduces the number of user ports available for server and storage connections.
– If you connect a Storage Management Appliance to the fabric, this further reduces the number of ports available for
server and storage connections.

Confidential – For Training Purposes Only 118

B-series switch and port topology maximums

SAN topology Number of switches Total number of ports Number of user ports
Single-switch fabric 1 512 512
Cascaded fabric
56
Meshed fabric 2,300
Ring fabric 15
2,560
Ring fabric with satellite 1,212
switches 56
Core-edge fabric 2,300

Confidential – For Training Purposes Only 119

C-series switch and port topology maximums

SAN topology Number of switches Total number of ports Number of user ports
Single-switch fabric 1 528 528
Cascaded fabric 3,500
(cascaded with
60 12 Director switches and
10 Fabric switches)
Meshed fabric 4,000
(maximum of 12
Ring fabric 15 Director switches)
Ring fabric with satellite 3,500
switches 60
Core-edge fabric

Confidential – For Training Purposes Only 120

H-series switch and port topology maximums

SAN topology Number of switches Total number of ports Number of user ports
Single-switch fabric 1 20 20
Cascaded fabric 460
30 600
Meshed fabric 512
Ring fabric 15 300 264
Ring fabric with satellite
switches 30 600 462
Core-edge fabric

Confidential – For Training Purposes Only 121

Data availability

– The data availability level required for your SAN environment is based on:
– The administrative requirements
– Examples: Backup schedules, operating procedures, and staffing
– The protection level for applications or data
– The hardware redundancy

– Data availability is arranged in 4 levels:

– Level 1: Single-connectivity fabric
– Level 2: Single resilient fabric
– Level 3: Single resilient fabric with multiple device paths
– Level 4: Multiple fabrics and device paths (NSPOF)

Confidential – For Training Purposes Only 122

Single-connectivity fabric

Level 1
– Maximum connectivity
– No fabric resiliency or redundancy
– Each switch has one path to other switch or fabric
– Each server or storage system has one path to the fabric

Confidential – For Training Purposes Only 123

Single resilient fabric

Level 2
– Provides fabric path redundancy by using multiple ISLs
between switches
– Each server and storage system has one path to the
fabric
– There is no interruption in I/O activity in the event of a
switch port or ISL failure

Confidential – For Training Purposes Only 124

Single resilient fabric with multiple device paths

Level 3
– Provides multiple server and storage system paths to
the fabric to increase availability
– There is no interruption of I/O in the event of a switch,
server HBA, or storage system path failure

NOTE: HPE recommends that each server HBA and each

storage system has a path to a different switch to increase
availability.

Confidential – For Training Purposes Only 125

Multiple fabrics and device paths (NSPOF)

Level 4
– Provides multiple data paths between servers and
storage systems, but the paths connect to physically
separate fabrics
– Provides the highest availability and no single point of
failure (NSPOF) protection
– Minimizes the vulnerability to fabric failures
– Using two fabrics might increase the implementation
costs, but it also increases the total number of available
ports

Confidential – For Training Purposes Only 126

Data availability level considerations

Fabric design Availability level SAN topologies

Level 1: single connectivity fabric No redundancy Single switch or multiple switches with
single ISL
Level 2: single resilent fabric Cascaded with two ISLs, meshed,
Medium
ring, and core-edge
Level 3: single resilent fabric with
High All
multiple device paths
Level 4: multiple fabrics and device Highest
All
paths (NSPOF)

Confidential – For Training Purposes Only 127

Thank you
Confidential – For Training Purposes Only 128