s012160

Live demonstration of IBM Spectrum Virtualize HyperSwap in a

VMware environment

Dominique SALOMON
IT Specialist Storage Certified
IBM Montpellier - France

© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.
 Dominique SALOMON presentations :

© Copyright IBM Corporation 2016. Technical University/Symposia materials may 2

IBM Systems Technical Events | ibm.com/training/events not be reproduced in whole or in part without the prior written permission of IBM.
 Session objectives

• Since Spectrum Virtualize code 7.5 HyperSwap has been introduced a

new multisite High Availability feature provided across the Spectrum
Virtualize portfolio. In this demonstration we show the resiliency of the
Spectrum Virtualize HyperSwap solution with a VMWare vSphere
environment. Thru an intuitive interface the following outage will be tested :
- Loss of an SVC node od Storwize controller - Loss of a backend storage
- Loss of an ESXi server - Loss of a whole site The Storwize Virtualize
HyperSwap architecture will be fully described during the session like its
step by step configuration.

© Copyright IBM Corporation 2016. Technical University/Symposia materials may 3

IBM Systems Technical Events | ibm.com/training/events not be reproduced in whole or in part without the prior written permission of IBM.
 HyperSwap Technical Overview

HyperSwap Demo environment

HyperSwap Configuration Step by Step

Failover Tests

4
HyperSwap Technical Overview

5
SVC Enhanced Stretched Cluster

 Today’s stretched cluster technology splits an SVC’s two-way cache

across two sites
 Allows hosts to keep read/write cache access to data if a site is lost
 Enhanced Stretched Cluster in version 7.2 introduced site concept to
the code for policing configurations and optimizing data flow
 Supported on IBM SAN Volume Controller – SVC only,
!! Not possible on Storwize Systems
Power domain 1 Power domain 2

Host Host

Write
Switch Switch
Read Read

Node 1 Node 2
Power domain 3

Quorum storage

Storage Storage

6
Overview of HyperSwap for SVC & Storwize

The HyperSwap® high availability function in the IBM® Spectrum Virtualize

software allows business continuity in the event of hardware failure, power
failure, connectivity failure, or disasters such as fire or flooding.
It is available on the IBM SAN Volume Controller, IBM Storwize® V7000, IBM
Storwize V7000 Unified, and IBM Storwize V5000 products.

The HyperSwap function provides highly available volumes accessible through

two sites at up to 300km apart.
A fully-independent copy of the data is maintained at each site. When data is
written by hosts at either site, both copies are synchronously updated before the
write operation is completed.
The HyperSwap function will automatically optimize itself to minimize data
transmitted between sites and to minimize host read and write latency.

7
Overview of HyperSwap (next)

If the nodes or storage at either site go offline, leaving an online and

accessible up-to-date copy, the HyperSwap function will automatically
fail over access to the online copy.
The HyperSwap function also automatically resynchronizes the two
copies when possible.
The HyperSwap function builds on two existing technologies in the
product: - Non-Disruptive Volume Move (NDVM) function introduced in
version 6.4 of the SVC software,
- Remote Copy features that includes Metro Mirror, Global Mirror, and
Global Mirror with Change Volumes.

8
Overview of HyperSwap (next)

The HyperSwap function in the SVC software works with the standard
multipathing drivers that are available on a wide variety of host types,
with no additional host support required to access the highly available
volume.
Where multipathing drivers support ALUA, the storage system will tell the
multipathing driver which nodes are closest to it, and should be used to
minimise I/O latency.
You just need to tell the storage system which site a host is connected to,
and it will configure host pathing optimally. This aspect is also enabled for
the Enhanced Stretched Cluster function.

9
High Availability with HyperSwap : The concepts

Host site 1 Host site 2 HyperSwap

HyperSwap
Volume
Volume
Hosts Cluster HS
HS Vol
Vol

SAN
Site 1 Volume Active Active Replication Site 2 Volume
(Master) (from Primary to Secondary) (Auxiliary)
Vol Vol
Site1 Site2
Change
Change Vol
Vol Site2
Site1

Storwize 1 Storwize Cluster Storwize 2

IO Group 1 IO Group 2
= 2 controllers = 2 controllers
= Node 1 + Node 2 = Node 3 + Node 4
10
HyperSwap / SVC Enhanced Stretched Cluster comparison

 Stretched Cluster requires splitting nodes in an I/O group

–Impossible with Storwize family since an I/O group is
confined to an enclosure
 After a site fails write cache is disabled in Streched Cluster
configuration
–Could affect performance
 HyperSwap keeps nodes in an I/O group together
–Copies data between two I/O groups
–Suitable for Storwize family of products
–Retains full read/write performance with only one site

11
HyperSwap / SVC Enhanced Stretched Cluster comparison
SVC Cluster :

IO Group = Cache mirrored between the 2

Node 1
Node 2 nodes of same IO group

SVC Enhanced Streched Cluster :

IO Group = Node 1 ---------------------------------------------------- >> Node 2
Cache mirrored between the 2 nodes
of same IO group

SVC & Storwize HyperSwap :

IO Group 1 IO Group 2
Cache mirrored Cache mirrored
= 2 Storwize controllers between the 2 nodes between the 2 nodes = 2 Storwize controllers
= Node 1 + Node 2 of same IO group of same IO group = Node 3 + Node 4

Replication

Storwize Cluster
12
HyperSwap / SVC Enhanced Stretched Cluster comparison

 Stretched Cluster discards old data during resynchronization

– If one site is out-of-date, and the system is automatically resynchronizing that
copy, that site’s data isn’t available for disaster recovery, giving windows where
both sites are online but loss of one site could lose data
 HyperSwap uses Global Mirror with Change Volumes technology to retain
the old data during resynchronization
– Allows a site to continually provide disaster recovery protection throughout its
lifecycle
 Stretched cluster did not know which sites hosts were in
– To minimize I/O traffic across sites more complex zoning and management of
preferred nodes for volumes was required
 Can use HyperSwap function on any Storwize family system supporting multiple
I/O groups
– Storwize V5000, Storwize V7000 Gen1/Gen2 and SVC

13
HyperSwap – Limits & restrictions
 Limits and Restrictions with code 7.5
– Max of 1024 HyperSwap volumes per cluster
• Each HyperSwap volume requires four FC mappings and max mappings is 4096
– Max capacity is 1PB per I/O group or 2PB per cluster
• Much lower limit for Gen1 Storwize V7000
• Run into limit of remote copy bitmap space
– Can’t replicate HyperSwap volumes to another cluster for DR
– Limited FlashCopy Manager support
• Can’t do revFC to HyperSwap volumes
– Max of 8 paths per HyperSwap volume same as regular volume
– No GUI support (support in next release 7.6)
 Requirements
– Remote copy license
– Size public/private SANs as we do with ESC today
• Only applicable if using ISLs between sites/IO groups
 Recommended Use Cases
– Active/Passive site configuration
– Hosts access volumes from one site only

14
IBM Spectrum Virtualize™ - Code 7.6
Announce : Oct 6, GA : November 27 2015

What’s New / Key Features Customer Benefit(s)

External Encryption
 Add encryption support for externally virtualized storage
•V7000: Externally virtualized storage  Address security needs with hardware-assist on internal
storage
•SVC: SW based model DH8 for external
virtualization / HW based model 24F

Distributed RAID (on internal storage)  Improve data protection and availability with significantly smaller
rebuild time

IP Quorum base support  Simplify high availability implementations while reducing costs

VMware vSphere Virtual Volumes integration  Bring advanced storage functions to the virtual machine level with
Storwize readiness for VMware Virtual Volumes

Simplified HyperSwap setup and  Reduce time and simplify set up of HA configurations
management via GUI
Integrated Comprestimator  Enable a quicker and easier way to see compression results
 Unique in the industry –showing value on live data

New host adapter: 4port 16Gb FC HIC  Enable additional number of ports for next-generation fibre
channel networks
New flash drive: 3.2 TB 12Gb  Increase flash capacity in the same floor space up to 50% with
new flash drive option
 
New Volume CLIs

GUI support for HyperSwap
HyperSwap 
IP Quorum

What’s new in 7.6

R7.6 - VOLUME CLI COMMANDS
What makes up a HyperSwap volume?
 New Volume CLIs

Creating a HyperSwap volume in 7.5 Creating a HyperSwap volume in 7.6

mkvdisk master_vdisk
1.
1. mkvolume my_volume
mkvdisk aux_vdisk
2.

mkvdisk master_change_volume
3.

mkvdisk aux_change_volume
4.

mkrcrelationship –activeactive
5.

chrcrelationship -masterchange
6.

chrcrelationship -auxchange
7.

addvdiskacces
8.
 New Volume CLIs

5 new CLI commands for administering Volumes:

mkvolume


mkimagevolume


addvolumecopy


rmvolumecopy


rmvolume


Also:

lsvdisk now includes “volume_id”, “volume_name” and “function”
fields to easily identify the individual vdisks that make up a
HyperSwap volume
 New Volume CLIs


mkvolume
–
Create a new empty volume using storage from existing storage pools.
–
Volume is always formatted (zeroed).
–
Can be used to create:
•
Basic volume - any topology
•
Mirrored volume - standard topology
•
Stretched volume - stretched topology
•
HyperSwap volume - hyperswap topology

–
The type of volume created is determined by the system topology and the number of
storage pools specified

mkimagevolume
–
Create a new image mode volume.
–
Can be used to import a volume, preserving existing data.
–
Implemented as a separate command to provide greater differentiation between the
action of creating a new empty volume and creating a volume by importing data on
an existing mdisk.
 New Volume CLIs


addvolumecopy
–
Add a new copy to an existing volume.
–
The new copy will always be synchronized from the existing copy.
–
For stretched and hyperswap topology systems this creates a highly available
volume.
–
Can be used to create:
•
Mirrored volume - standard topology
•
Stretched volume - stretched topology
•
HyperSwap volume - hyperswap topology

rmvolumecopy
–
Remove a copy of a volume. Leaves the volume intact.
–
Converts a Mirrored, Stretched or HyperSwap volume into a basic volume.
–
For a HyperSwap volume this includes deleting the active-active relationship and the
change volumes.
–
Allows a copy to be identified simply by its site.
–
The –force parameter from rmvdiskcopy is replaced by individual override
parameters, making it clearer to the user exactly what protection they are bypassing.
 New Volume CLIs


rmvolume
–
Remove a volume.
–
For a HyperSwap volume this includes deleting the active-active relationship and the
change volumes.
–
The –force parameter from rmvdisk is replaced by individual override parameters,
making it clearer to the user exactly what protection they are bypassing.
R7.6 - GUI SUPPORT FOR HYPERSWAP
GUI Support for HyperSwap – Configuring system topology

Add Nodes
----------------------------------------
-
Rename System
Rename Sites
Modify System Topology
Turn Off All Identify LEDs
Flip Layout
Update>
----------------------------------------
-
Power Off
----------------------------------------
-
Properties
GUI Support for HyperSwap – Configuring system topology

Topology
:

IBM Confidential
GUI Support for HyperSwap – Creating an HyperSwap volume
Create Volumes
Quick Volume Creation Advanced

Basic HyperSwap Custom

Quantity: Capacity: Capacity savings: Name:

1 24 GiB Compressed My_hs_volume +
Consistency group: None
London Hursley

Pool: Pool1 Pool: Pool2

I/O group: Auto select I/O group: Auto select

Summary
1 volume
1 copy in Hursley
1 copy in London
1 active-active relationship
2 change volumes

Create Create and Map to Host Cancel

IBM Confidential
GUI Support for HyperSwap – Viewing volumes
R7.6 - IP QUORUM FOR HYPERSWAP
& STRECHED CLUSTER
 IP Quorum : How it works

Primarily for use with Stretch Cluster and Hyperswap configurations
– Allows the user to have a quorum device at a 3rd site that does NOT have
fibre channel connectivity


A new SVC/Storwize CLI interface creates a Java application that the user can then
deploy on a server at the third site
– No further configuration required – all configuration details (e.g. IP addresses +
certificates) are put into the Java application by SVC/Storwize
– 1st release has limited support for releases of Java (although no compatibility
problems are anticipated)


Quorum application connects via secure sockets to the service IP address of each
SVC node and acts as a quorum device
– If SVC configuration is changed (e.g. add nodes, change IP addresses,
change certificate) the SVC logs an error telling user that their Java application
is out of date
– If the quorum application cannot connect to all nodes the SVC cluster logs an
error reporting that there is a connectivity issue that needs to be resolved
34 © 2015 IBM Corporation
 1st Release Limitations / 2nd Release Enhancements

No GUI support in first 7.6 release
– Will be added in next release


Limited testing with versions of Java
– May expand qualification via SCORE requests or in 16A release


Only connects to service IP address on 1st port of the node
– For security reasons users would like to have a dedicated IP address (+VLAN).
– Some users also would like to dedicate a physical Ethernet port just for this connection


1st release only designed for deployment on internal customer network
– Does not adequately address security concerns above
– Minimal testing of the round trip latency for communication to the quorum application


1st release only allows fixed lease period
– Allow a shorter lease time (15 seconds) for faster failover/failback
– Allow a longer lease time for quorum app across internet
– 2nd release will allow a preferred site to be configured to bias how tie-break is resolved

35 © 2015 IBM Corporation

HyperSwap 7.6 – Comparison with Enhanced Streched Cluster
  SVC Stretched Cluster
Products that function is available on SVC only
Complexity of configuration CLI or GUI on single system; simple object
creation
SVC and Storwize HyperSwap
SVC with 2 or more I/O groups; Storwize
V7000 and Storwize V5000
CLI or GUI on single system; simple object
creation

Sites data stored on 2 2

Distance between sites
Independent copies of data maintained
Technology for host to access multiple
copies and automatically fail over
Up to 300km Up to 300km
2 2 (4 if additionally Volume Mirroring to two
pools in each site)
Standard host multipathing driver Standard host multipathing driver

Cache retained if only one site online? No Yes

Host-to-storage-system path optimization
Synchronization and resynchronization of
copies
Stale consistent data retained during
resynchronization for disaster recovery?
Manual configuration of preferred node per Automatic configuration based on host site
volume prior to 7.5, automatic based on host (requires ALUA/TPGS support from
site as HyperSwap from 7.5 multipathing driver)
Automatic Automatic
No Yes

Scope of failure and resynchronization Single volume Single Volume

Ability to use FlashCopy together with High Yes (though no awareness of site locality of Limited: can use FlashCopy maps with
Availability solution data) HyperSwap volume as source, avoids sending
data across link between sites
Ability to use Metro Mirror, Global Mirror, or One remote copy, can maintain current copies No
Global Mirror together with High Availability on up to four sites
solution
Maximum highly available volume count 4096 1024

Licensing Included in base product Requires Remote Mirroring license for

volumes. Exact license requirements may vary
by product.
36
HyperSwap Demonstration

37
Demonstration Website : http://www.ibm.com/systems/clientcenterdemonstrations

Booking the IBM Spectrum Virtualize HyperSwap demonstration

38

Demonstration Website

© 2015 IBM Corporation

Demonstration Roadbook

© 2015 IBM Corporation

Demonstration booking process
The IBM Systems & Software Demonstration Center provides to IBMer and BPs a large catalog of remote
demonstrations based on real infrastructures. The demonstration environments are accessible from
anyplace with an internet connection through VPN (using openVPN certificates).

Live Demonstrations
3. Book the with scenarios

demonstration

.Automated
2. Search a demo
demonstration
5. Download and Install the
openVPN Certificate (only
for the first demo)

1. Connect to the 5. Access to the demo

DemoCenter portal environment through the
openVPN connection using the
credential sent by email. The
IBMer or
or BP
BP information about the demo is
provided in the demonstration
web page or in the attached
documents.

© 2015 IBM Corporation

Booking the demo http://www.ibm.com/systems/clientcenterdemonstrations

© 2015 IBM Corporation

Booking the demo : 1) Confirmation mail with & OpenVPN certificate

This mail is received quickly after demo booking (less than 1 hour)

© 2015 IBM Corporation

Booking the demo : 2) Mail with Password

This mail is received 15 mn before the beginning of the demo

© 2015 IBM Corporation

Demonstration Access

Use password sent in

the second mail

© 2015 IBM Corporation

Spectrum Virtualize HyperSwap demonstration

© 2015 IBM Corporation

Spectrum Virtualize HyperSwap demonstration

© 2015 IBM Corporation

 Spectrum Virtualize HyperSwap demonstration : Starting point

Normal Operation : Everything is green

Initially, both IP Quorum are up and running, and one of them is the active quorum,
notified by the green mark.
© 2015 IBM Corporation
1) Failure of an IP Quorum :

Just click on one of On the web interface, you will notice

the IP Quorum to that the green mark will move to the
disable it and confirm second IP Quorum after a few
the operation seconds..
© 2015 IBM Corporation
1) Failure of an IP Quorum :

On the Storwize 7000 cluster web GUI, in the Performance page, you can see that losing the quorum has
no impact on the performance

© 2015 IBM Corporation

1) Failure of an IP Quorum :

Let’s now reactivate the disabled IP Quorum.

Click on it and confirm the prompt.

On the demonstration interface, nothing will change. The active quorum won’t failback on this IP Quorum.

On the Storwize V7000 web interface, on the Performance page, you can see that there is no impact on the
performance.
© 2015 IBM Corporation
2) Failure of both IP Quorums :
When both IP Quorum are enabled, disable first the active quorum : Click on it and confirm the prompt.
After a few seconds, the green mark will move to the remaining IP Quorum.
Let’s now disable this second IP Quorum. Click on it and confirm the prompt.
You will notice that the green mark disappears: there is no active quorum anymore.

There is no quorum disk acting as tiebreaker and cluster is now prompt to split brain.

But You can check on the Storwize V7000 web interface, on the Performance page, you can see that there is
no impact on the performance.
© 2015 IBM Corporation
3) Failure of a Storwize V7000
In this scenario, we will generate an outage on a Storwize V7000 part of the cluster (so Site 1 or 2, not 3).
To make the outage more “dramatic”, we will generate an outage on the Storwize V7000 which is the Primary
system of the relationship. The primary system is the one with the mention Primary on the web interface.
In our example , we will disable the Storwize V7000 Site 1.
CLI verification :superuser>lsnodecanister

© 2015 IBM Corporation

3) Failure of a Storwize V7000
After a few seconds, the Storwize V7000 will be disabled, and will appear red on the map.
The traffic will be interrupted for a short time.
After a few more seconds, you will notice that:
- The VMware host will change its FC path to communicate with the remaining Storwize V7000
- The Storwize V7000 role changed. The primary system is now the V7000 Site 2.

The HyperSwap relationship status changed : The primary system is now the second site
The volumes are now desynchronizing. The percentage of synchronization is decreasing.
You can notice a freezing time, which correspond to the last up-to-date copy on the offline
(secondary) system. © 2015 IBM Corporation
3) Failure of a Storwize V7000

Volumes :

© 2015 IBM Corporation

3) Failure of a Storwize V7000

We will now resolve the outage by reactivating the failed Storwize V7000.
Click on the failed Storwize V7000 (the red one) and confirm the action.

- The traffic is back to normal. VMware Host on Site 1 will address his IOs to the Storwize V7000 Site
1, and the VMware Host on Site 2 will address the Storwize V7000 Site 2.
- The HyperSwap details shows that the volume are now synchronizing: the percentage will increase to
100%.
- HyperSwap is copying data from the master volume to its change volume.
If an error occurred during the resynchronization (loss of the site containing the up-to-date copy, loss
of link between the Storwizes...), the secondary copy would be inconsistent. In that case, it would be© 2015
possible
IBM Corporation
to restore, from change volumes, a previous stale data (dated from the freeze_time), in a
3) Failure of a Storwize V7000

Resynchronization on going

© 2015 IBM Corporation

3) Failure of a Storwize V7000

In the Flashcopy Mappings page, you can see that flashcopy mappings between master volumes and their
change volume have been started: the data changed during the synchronization are backed up in the
change volume in case of issue.

FlashCopy Mappings :

Volumes :

© 2015 IBM Corporation

3) Failure of a Storwize V7000 Resynchronization finished

© 2015 IBM Corporation

4) Whole site Failure
Virtual Machine has
restarted on remaining site

59

© 2015 IBM Corporation

3) Whole site Failure

60

© 2015 IBM Corporation

3) Whole site Failure : Restarted V7000 & ESX

61

© 2015 IBM Corporation

3) Whole site Failure

62

© 2015 IBM Corporation

3) Whole site Failure : Restarted V7000 & ESX

© 2015 IBM Corporation

3) Whole site Failure : Resynchro finished
Note: The virtual machine won’t migrate back to the Site 2 because the HA functionality set up for the
demonstration doesn’t include it.

© 2015 IBM Corporation

3) Whole site Failure : Resynchro finished

65

© 2015 IBM Corporation

3) Whole site Failure : Restarted V7000 & ESX

You can migrate back manually Virtual Machine 2 to Site 2

66

© 2015 IBM Corporation

3) Whole site Failure : Restarted V7000 & ESX

© 2015 IBM Corporation

QUESTIONS

68
 Session Evaluations
YOUR OPINION MATTERS!

1 2 3 4
Submit four or more session
evaluations by 5:30pm Wednesday
to be eligible for drawings!
*Winners will be notified Thursday morning. Prizes must be picked up at
registration desk, during operating hours, by the conclusion of the event.
 Continue growing your IBM
skills
ibm.com/training
provides a comprehensive
portfolio of skills and career
accelerators that are designed
to meet all your training needs.

If you can’t find the training that is right for you with our Global
Training Providers, we can help.

Contact IBM Training at dpmc@us.ibm.com

Global Skills Initiative

Thank you!

ibm.com/systems

71 © 2015 IBM Corporation