EMC® Symmetrix® Remote Data Facility (SRDF

)
PRODUCT GUIDE
P/N 300-001-165 REV A05

EMC Corporation Corporate Headquarters: Hopkinton, MA 01748-9103
1-508-435-1000 www.EMC.com

Copyright © 2001-2007 EMC Corporation. All rights reserved. Published February, 2007 EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners.

2

EMC Symmetrix Remote Data Facility (SRDF) Product Guide

Contents

Preface............................................................................................................................ 11 Chapter 1 Introducing SRDF
SRDF product overview................................................................... Base SRDF family products...................................................... SRDF family options ................................................................. SRDF/Synchronous (SRDF/S) ................................................ SRDF/Asynchronous................................................................ SRDF/Data Mobility (SRDF/DM) .......................................... SRDF/Automated Replication (SRDF/AR) .......................... SRDF/Star................................................................................... SRDF/Consistency Groups (SRDF/CG)................................ SRDF/Cluster Enabler for MSCS or VCS............................... SRDF functional overview............................................................... Basic SRDF configuration ......................................................... SRDF interfamily connectivity................................................. SRDF supported data protection options............................... Monitoring and controlling SRDF........................................... SRDF director hardware .................................................................. SRDF director functions............................................................ SRDF configurations......................................................................... SRDF configuration using switched Fibre Channel ............. SRDF fully switched fabric connectivity ................................ Switched and Concurrent SRDF.............................................. SRDF with native GigE ............................................................. SRDF and FarPoint .................................................................... 16 16 17 18 18 19 19 20 20 21 22 22 23 24 24 25 25 27 30 30 32 33 34

Chapter 2

SRDF Technical Concepts
SRDF volume types .......................................................................... 38 Primary (source) volumes ........................................................ 38
EMC Symmetrix Remote Data Facility (SRDF) Product Guide
3

......................................................................................................................................... How a consistency group works .............................................................................................................................................. Concurrent SRDF ...................................................................................................................................... Secondary modes of operation ................................ Dynamic Sparing with remotely mirrored pairs (SRDF)................................................................. Dynamic SRDF groups ............................................................................. Semi-synchronous mode ............................................................................................................................................................................................. SRDF system-level attributes.................................................................................... SRDF bidirectional link configuration................................................................................................................................................................................ 68 Write operations in an ESCON bidirectional 4 EMC Symmetrix Remote Data Facility (SRDF) Product Guide ............................................................................ Invalid tracks attribute .............. Technical considerations............................................. SRDF link configurations...................................................................... SRDF unidirectional link configuration ........................... SRDF link and volume states ................. Dynamic SRDF devices.......................................... SRDF groups............. Adaptive copy modes ...................................... SRDF volume states (Symmetrix view). Synchronous mode........................................................................................................................ SRDF link states ............... Domino modes..............Contents Secondary (target) volumes ..... Primary modes of operation ........................................................................................ SRDF dual-directional link configuration....................................................... SRDF volumes: Special considerations................................................................... PPRC command support... Configuration settings affecting device ready and link states ...... Logical volume states.............................. Open systems metavolumes ..................................................... 68 Write operations in a unidirectional or dual-directional configuration ..................... SRDF/Consistency Groups (SRDF/CG)...... Continuous processing ............ 39 39 39 41 42 43 44 44 44 45 46 47 47 47 47 48 48 48 49 50 51 53 53 54 56 56 58 58 59 59 60 62 62 64 65 Chapter 3 SRDF Operations Write operations.................................. Additional SRDF modes and attributes .... Symmetrix RAID 10 (mirrored striped mainframe volumes) ............................. Local volumes .......................................................... EMC Compatible Peer ........................................................................................................................................................ Host accessibility .........

......................................................................................................................... 86 SRDF/A history................................................................................................................................................ MSC ...................................... 77 R1/R2 swap ......................50 SRDF/A Multi-Session Consistency............................ 91 SRDF/A single session mode............................................. 75 Using TimeFinder BCVs with secondary devices ............................. 95 Not Ready (NR) state (system startup)........................................................................................... 81 Chapter 4 SRDF/Asynchronous Operations SRDF/A overview .................................................................................................................. 69 Primary volume read operations.... 72 Business continuance using SRDF....... 87 Enginuity 5670 SRDF/A single session .................................................................... 95 EMC Symmetrix Remote Data Facility (SRDF) Product Guide 5 ... 77 SRDF Multi-Hop ........................................................................................................... 80 Migrating data from R1 to a larger R2 device..... 87 Enginuity 5670................................................. 68 Read operations................................... 75 Using TimeFinder/Mirror BCVs with primary devices......................... 85 Requirements and limitations .......... 92 SRDF/A single session mode dependent-write consistency......... 87 Enginuity 5671 Multiple SRDF/A Single Session SRDF groups per Symmetrix.................................................. 75 Using a BCV as a primary (source) device ......................................... MSC............................................................. 79 R1/R2 swap procedure history..................................................................... 71 Failover to the secondary Symmetrix system ............................................................. 89 SRDF/A Reserve Capacity ..................................................................... 76 SRDF remote command support ....................................... 87 Enginuity 5671 Concurrent SRDF support............... 71 Recovery for a large number of invalid tracks ....................................................... 93 SRDF/A single session mode states.............. 88 Enginuity 5671 Tunable Cache utilization.......................... 88 Enginuity 5671 Dynamic SRDF support........................................... 84 SRDF/A benefits .................. 76 Using a BCV as a secondary (target) device................ 90 Locality of reference............................ 71 Failback to the primary Symmetrix system .. 69 Recovery operations ........................... 89 Tolerance mode.............................. 73 Business continuance using SRDF and TimeFinder....................Contents configuration .................................... 69 Secondary volume read operations....................................... 87 Enginuity 5671 SRDF/A Multi-Session Consistency........................ 79 Dynamic R1/R2 swap .........................................................................

..... 108 Temporary link loss... SRDF/Star automation for mainframe....................Contents Inactive state........................ 122 Chapter 5 SRDF/Star Operations SRDF/Star overview ............................................... 105 Pend-dropping SRDF/A single session mode ...... 112 Entering SRDF/A multi-session consistency ............................................................................... 109 Failback from secondary symmetrix devices ....................................................................... 108 Permanent link loss ......................................................................................... SRDF/Star control for mainframe....................................... 126 128 128 129 130 130 131 132 Index ...... How SRDF/Star works................. SRDF/Star automation for open systems ............................................................................................................................................... 103 Switching to SRDF/S mode from SRDF/A single session mode....................... 104 Coming out of the SRDF/A active state .................................................................. SRDF/Star control for Open Systems............................................................................. 110 SRDF/A multi-session consistency (MSC) mode .................................... 133 6 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .... 106 SRDF/A single session cleanup process ................................. 107 SRDF/A single session mode recovery scenarios.................... 102 Switching to SRDF/A mode ........ Known requirements and limitations at this release........................................................... SRDF/Star benefits.......... 97 SRDF/A single session mode state transitions ................................................. 108 Primary Symmetrix global memory full condition ......... 113 Performing a SRDF/A MSC consistent cycle switch ............................................................................. 102 Transition from synchronous to asynchronous.............. 114 SRDF/A MSC mode delta set switching.................................................................... 111 SRDF/A MSC mode dependent-write consistency. 105 Deactivating SRDF/A single session mode.................................................................................. 103 Transition from adaptive copy disk mode to SRDF/A....... 104 Dropping SRDF/A single session mode......................................................... 96 SRDF/A single session mode delta set switching ............. 116 SRDF/A MSC session cleanup process .............................................................................................................................. 95 Active state ........ 102 Transition from adaptive copy write-pending mode to asynchronous ......

.......... 34 SRDF with and without FarPoint............................................................................................... and 3 in a consistency group ................................. 76 SRDF multi-hop configuration ................................................................................................................................................................. 27 Data vaulting solution .............................................. 74 SRDF single-hop configuration (BCV functioning as a primary SRDF device) .... 63 Primaries 1............................................... 73 Primary-to-secondary resynchronization .................................................................... 23 Two production sites and one recovery site.................................................................................................... 31 Switched and Concurrent SRDF configuration example.......................................................................... 60 Primary and Secondary relationships .................................................................. 74 Secondary-to-primary resynchronization........ 40 PPRC and GDPS support ..................................... 30 Switched SRDF over Fibre Channel with Enginuity 5x67............................................................................................ 46 SRDF logical volume state....... 95 EMC Symmetrix Remote Data Facility (SRDF) Product Guide 7 ..................................................................................................................................................... 54 Semi-synchronous mode ..................................... 64 Failed link between Primary 2 and Target 1.............. 78 R1/R2 swap concept ............. 32 Switched GigE network configuration example.............................. 55 Concurrent SRDF configuration....................... 2................. 93 SRDF/A single session allowed state transitions ........................................................................... 62 Failed link between Primary 2 and Target 1........... 35 Dynamic SRDF.....................................Figures Title 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Page Basic SRDF configuration.......................................................... 28 Sites containing both primary and secondary volumes .. 64 SRDF business continuance ........................................................................................................................................................................................................................................................ 29 Switched SRDF with multiple primary and secondary devices.............................. 79 Synchronous and asynchronous block transfer comparison ..................................................... 91 SRDF/A delta sets and their relationships ... 49 Synchronous mode......

........................................................ 98 SRDF/A single session SRDF transfer is halted prior to Primary Symmetrix cycle switch ...............................................Figures 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 Single session capture delta set collects application write I/O............... 117 SRDF/A MSC Primary Symmetrix halts the SRDF transfer ................................... 116 SRDF/A MSC Primary Symmetrix transmit delta set switch is emptied ............................. 101 SRDF/A single session primary Symmetrix begins SRDF transfer.................. 101 SRDF/A single session transition path..................................... 127 SRDF/Star configuration reference...................... 102 SRDF/A MSC delta sets and their relationships..................................................................................................................... 98 SRDF/A single session new capture delta available for host I/O............... 121 SRDF/A MSC Secondary Symmetrix begins the apply delta set restore process ............................................................................................... 121 Concurrent SRDF configured for SRDF/Star support .. 131 8 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .................... 117 SRDF/A MSC Secondary apply delta set restore complete ............................. 100 SRDF/A single session secondary Symmetrix begins restore of apply delta set .... 98 SRDF/A single session Primary Symmetrix delta set switch....................................................................................................................... 119 SRDF/A MSC Secondary Symmetrix cycle switch........................... 114 SRDF/A MSC capture delta set collects application write I/O ................... 118 SRDF/A MSC Primary Symmetrix cycle switch while I/O is deferred...................................................................................... 99 SRDF/A single session secondary Symmetrix delta set switch........................................................................... 120 SRDF/A MSC Secondary new receive delta set is available . 119 SRDF/A MSC I/O is released and a new capture delta set continue to accept Host I/O .. 120 SRDF/A MSC Primary Symmetrix systems begin SRDF transfer............................................................................. 99 SRDF/A single session secondary Symmetrix wait for apply delta set to be restored... 100 SRDF/A single session secondary Symmetrix new receive delta set is available for SRDF .............................................................................................. 97 SRDF/A single session transmit delta set empties ........ 112 SRDF/A MSC allowed state transitions ............................................................................

. 52 Secondary (R2) volume accessibility ......... 52 EMC Symmetrix Remote Data Facility (SRDF) Product Guide 9 ............................................................Tables Title Page 1 2 3 4 Symmetrix model numbers ...................................................................................... 24 Primary (R1) volume accessibility ............ 17 Data Protection options for SRDF disk devices .........................................................................

Tables 10 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

EMC periodically releases new versions of both the EMC Enginuity Operating Environment and Symmetrix Remote Data Facility (SRDF). refer to: ◆ EMC Support Matrix at http://www.com Follow these links: Support. Solutions Guides EMC Symmetrix Remote Data Facility (SRDF) Product Guide 11 .com/interoperabilityindex. If an SRDF feature does not function properly or does not function as described in this guide. Detailed SRDF installation and configuration information is available in the documents listed in Related documentation on page 11. some functions described in this guide may not be supported by all versions of Enginuity currently in use.Preface As part of its effort to continuously improve and enhance the performance and capabilities of the EMC product line. please contact the EMC Customer Support Center for assistance.com Related documentation For information on configuring SRDF.EMC. Audience This guide provides an overview of SRDF and SRDF family of product offerings for users requiring a basic knowledge of SRDF concepts and operation.EMC.jsp ◆ EMC Networked Storage Topology Guide at http://Powerlink. contact your EMC Sales Representative or refer to the EMC Powerlink website at: http://powerlink. Therefore. For the most up-to-date information on product features.EMC. refer to your product release notes. For additional information on SRDF and all Symmetrix systems-related publications. Document Library.

Software.Preface ◆ Symmetrix Remote Data Facility (SRDF) Product Guide at http://Powerlink. ! CAUTION A caution contains information essential to avoid data loss or damage to the system or equipment.com Follow these links: Support. Document Library. or ask your site representative to contact the appropriate EMC partner vendor or EMC Technical Support/Engineering.com ◆ EMC SRDF Connectivity Guide at http://Powerlink. consult the appropriate product specification manuals available from each vendor.EMC. SRDF ◆ Symmetrix (Model or Series) Product Guide at http://Powerlink. but not hazard-related. Document Library.EMC. 12 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Symmetrix For detailed information regarding solutions that involve other qualified vendors' equipment.com Follow these links: Support. Conventions used in this document EMC uses the following conventions for notes and caution notices. The caution may apply to hardware or software. Note: A note presents information that is important.EMC. ! IMPORTANT An important notice contains information essential to operation of the software. The important notice applies only to software.

processes. URLs. keywords. notifications In procedures: • Names of dialog boxes. italic Used for: • Arguments used in examples of command-line syntax • Variables in examples of screen or file output • Variables in path names Courier. filenames. programs. system calls. man pages. icons. filenames. links. environment variables. environment variables (for example.Preface Typographical conventions EMC uses the following type style conventions in this guide: Normal font In running text: • Interface elements (for example. command line and text). utilities • Pathnames. attributes. menu names. buttons. italic Variables used in a command-line sample EMC Symmetrix Remote Data Facility (SRDF) Product Guide 13 . directory names. buttons. bold Courier. button names. daemons. fields • Selections from the user interface. computer names. service keys. file systems. kernels Used for: • Full publications titles referenced in text • Unique word usage in text Anything requiring extra emphasis Used for: • System output • Filenames • Complete paths • Command-line entries • URLs Used for: • User entry • Options in command-line syntax Bold Italic Bold Italic Courier Courier. menus. options. services. groups. bold. pools. utilities. notifications. applications. functions. including menu items and field entries • Key names • Window names In running text: • Command names. clauses. dialog box names) outside of procedures • Items that user selects outside of procedures • Java classes and interface names • Names of resources. DQL statements. Boolean expressions.

you must be a WebSupport customer. software updates. go to EMC WebSupport on Powerlink. and service. Product information — For documentation. or for information about EMC products. product. organization. Please send your opinion of this guide to: techpub_comments@EMC. licensing.Preface <> Angle brackets enclose parameter or variable values supplied by the user Where to get help EMC support. Information about your site configuration and the circumstances under which the problem occurred is required. go to the EMC Powerlink website (registration required) at: http://Powerlink. To open a case on EMC WebSupport. Your Comments Your suggestions will help us continue to improve the accuracy.com 14 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . release notes. and overall quality of the user publications. and licensing information can be obtained as follows.EMC.com Technical support — For technical support.

.... director hardware............................................. SRDF functional overview................... 16 22 25 27 Introducing SRDF 15 ........................................................................................................ SRDF configurations......... SRDF director hardware ......................... This chapter contains the following topics: ◆ ◆ ◆ ◆ SRDF product overview............................. and example configurations..........................Invisible Body Tag 1 Introducing SRDF This chapter introduces SRDF and explains basic concepts including SRDF volume types................

By maintaining copies of data in different physical locations. SRDF enables you to perform the following operations with minimal impact on normal business processing: ◆ ◆ ◆ ◆ ◆ ◆ Disaster restart Disaster restart testing Recovery from planned outages Remote backup Data center migration Data replication and mobility Base SRDF family products The SRDF family consists of three base solutions: ◆ ◆ ◆ SRDF/Synchronous (SRDF/S. SRDF/Data Mobility (SRDF/DM ) — Rapid transfer of data from source volumes to remote volumes anywhere in the world. formerly SRDF) — High-performance. SRDF/Asynchronous (SRDF/A ) — High-performance extended distance asynchronous replication using a Delta Set architecture for optimal bandwidth utilization and minimal host performance impact.Introducing SRDF SRDF product overview The EMC® Symmetrix® Remote Data Facility (SRDF®) family of replication software offers various levels of Symmetrix based business continuance and disaster recovery solutions. or hundreds to thousands of kilometers apart. in different buildings within the same campus. host-independent. 16 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . real-time synchronous remote replication from one Symmetrix to one or more Symmetrix systems. mirrored copies of data. The SRDF products offer the capability to maintain multiple. permitting information to be shared and content to be distributed. or information consolidated for parallel processing activities. The Symmetrix systems can be in the same room. host-independent.

SRDF/AR (multi-hop and single-hop). These options include: ◆ SRDF/Automated Replication (SRDF/AR) solutions for meeting very specific. remote replication service-level requirements SRDF/Star for advanced multisite failover with continuous protection SRDF/Consistency Groups (SRDF/CG) for data consistency SRDF/Cluster Enabler (SRDF/CE) for integration with host-based clustering products such as Microsoft Cluster Server (MSCS) and VERITAS Cluster Server (VCS). and TimeFinder/Snap. including: SRDF/A. but generally it applies to any of the variations. this document uses the term TimeFinder to represent all EMC TimeFinder related products. For simplicity. SRDF/DM. TimeFinder/Clone. SRDF product overview 17 . Most of the document focuses on SRDF/S. This product guide refers to the Symmetrix models shown in Table 1 on page 17. which includes TimeFinder/Mirror. 8830 DMX3000 DMX-3 Cabinet description 1/2 Bay 1 Bay 2 Bay 3 Bay 1 system bay and from 2 to 8 storage bays a.Introducing SRDF SRDF family options There are a number of additional options and features that can be added to the base solutions to solve specific service level requirements. The Symmetrix 3832 is a 3830 model with a split backplane. this document uses the term SRDF to represent all EMC SRDF related products. ◆ ◆ ◆ Note: For simplicity. Table 1 Symmetrix model numbers Symmetrix 3xxx/5xxx series 3330/5330 and 3630/5630 3430/5430. 3830/3832a /5830 Symmetrix 8000 series 8230 8430. . and SRFD/CG. 8530 Symmetrix DMX series DMX800 (rackmount) DMX1000 DMX2000 3700/5700 and 3930/5930 8730. Note: The SRDF family of products can also be used with the EMC TimeFinder® family of products.

It does not require additional host software for duplicating data on the participating Symmetrix units. and testing The SRDF/S operation is transparent to the host operating system and host applications. SRDF/A is another mode of remote replication that allows customers to asynchronously replicate data while maintaining a dependent write consistent copy of the data on the secondary (R2) device at all times. eliminating the redundancy of multiple same track changes being transferred over the link. The dependant write consistent point-in time copy of the data at the remote side is typically only seconds behind the primary (R1) side. potentially reducing the required bandwidth. SRDF/S offers the following major features and benefits: ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ High data availability High performance Flexible configurations Host and applications software transparency Automatic recovery from a component or link failure Significantly reduced recovery time after a disaster Reduced backup and recovery costs Reduced disaster recovery complexity. SRDF/A session data is transferred to the secondary Symmetrix system in predefined timed cycles or delta sets. SRDF/Asynchronous Beginning with Enginuity™ level 5670.Introducing SRDF SRDF/Synchronous (SRDF/S) Symmetrix Remote Data Facility/Synchronous (SRDF/S) is a business continuance solution that maintains a real-time (synchronous) copy of data at the logical volume level in Symmetrix 3 8xxx or Symmetrix DMX™ and Symmetrix DMX-3 systems in the same or separate locations. specifically: ◆ ◆ ◆ Semi-synchronous mode Adaptive copy write-pending mode Adaptive copy disk mode Note: “Primary modes of operation” on page 53 and “Secondary modes of operation” on page 56 provide more information on these SRDF/S modes of operation. SRDF/S offers greater flexibility through additional modes of operation. the Symmetrix DMX system supports the asynchronous replication product named SRDF/Asynchronous (SRDF/A). 18 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . planning.

a multi-hop SRDF/AR configuration can provide long distance disaster restart with zero data loss at a middle or "bunker" site. restartable image of their data at the secondary site. Note: “Business continuance using SRDF and TimeFinder” on page 75 provides more information on single-hop and multi-hop configurations. if greater protection is required. Use a single-hop SRDF/AR configuration that permits controlled data loss (depending on the cycle time). content to be distributed. However. a partial delta set of data can be discarded. and can be used for local or remote transfers. Note: “Adaptive copy modes” on page 56 provide more information on adaptive copy mode operation. preserving dependant write consistency on the secondary site with a data loss of no more than two SRDF/A cycles. SRDF/DM transfers data from primary volumes to secondary volumes permitting information to be shared. This level of protection is intended for customers who require minimal host application impact while maintaining a dependant write consistent. disaster restart solutions using SRDF product overview 19 . In the event of a disaster at the R1 site or if SRDF links are lost during data transfer. SRDF/Data Mobility (SRDF/DM) SRDF/Data Mobility (SRDF/DM) is an SRDF product offering that permits operation in SRDF adaptive copy mode only and is designed for data replication and/or migration between two or more Symmetrix systems. SRDF/DM supports all Symmetrix systems and all Enginuity levels that support SRDF. SRDF/Automated Replication (SRDF/AR) SRDF/Automated Replication (SRDF/AR) is an automation solution that uses both SRDF and TimeFinder to provide a periodic asynchronous replication of a restartable data image. Compared to traditional disaster recovery solutions with their long recovery time and high data loss. Note: Chapter 4. and access to be local to additional processing environments.Introducing SRDF SRDF/A provides a long-distance replication solution with minimal impact on performance.” provides more detailed information on SRDF/A. Adaptive copy mode enables applications using that volume to avoid propagation delays while data is transferred to the remote site. ”SRDF/Asynchronous Operations.

SRDF/Star provides advanced multisite business continuity protection available for mainframe and open systems environments.Introducing SRDF SRDF/AR provide remote restart with a short restart time and low data loss. enterprises can quickly resynchronize the SRDF/S and SRDF/A copies by replicating only the differences between the sessions—allowing for much faster resumption of protected services after a source site failure. and then just as quickly restore the primary site when conditions permit. With SRDF/Star. This capability takes the promise of concurrent synchronous and asynchronous operations (from the same source device) to its logical conclusion. have dependent write logic imbedded in them to ensure data integrity if a failure occurs in: ◆ ◆ ◆ The host processor The software The storage subsystem 20 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . and in particular database management systems (DBMSs). or through UNIX and Windows environments using Solutions Enabler’s symreplicate command line interface. SRDF/Star Available at Enginuity level 5x71. The SRDF/AR process can be implemented with TimeFinder/Mirror in a mainframe z/OS environment. It enables concurrent SRDF/S with consistency groups and SRDF/A with MSC operations from the same source volumes with the ability to incrementally establish an SRDF/A session between the two remote sites in the event of a primary site outage—a capability only available through SRDF/Star software. SRDF/Star allows you to quickly re-establish protection between the two remote sites in the event of a primary site failure. SRDF/Consistency Groups (SRDF/CG) EMC SRDF/Consistency Groups (SRDF/CG) is an SRDF product offering designed to ensure the dependent write consistency of the data remotely mirrored by the SRDF operations in the event of a rolling disaster. Most applications. This protection is accomplished by using geographically separated replicas with hardware and software products from EMC Corporation. SRDF/AR offers data protection with dependent write consistency across a distance.

The DBMS writes again to the log volume to indicate that the database update was made. Note: “SRDF/Consistency Groups (SRDF/CG)” on page 62 provides more information on consistency group operations. In a remote disk copy environment. in a rolling disaster where a communication loss occurs that affects only a subset of the disk controllers performing the remote copy function. SRDF/Cluster Enabler for MSCS or VCS SRDF/Cluster Enabler for MSCS or VCS provides high availability and automated failover through storage-based replication and server clustering through SRDF/S and MSCS or VCS. 2. data consistency cannot be ensured if one of these writes was remotely mirrored. for example. 3. For example. SRDF/CG prevents dependent I/O from reaching its remote mirror in the case where a predecessor I/O only gets as far as the local mirror. SRDF product overview 21 . related write I/O operation is completed. For more information consult the Microsoft Cluster Server (MSCS) and VERITAS Cluster Server (VCS) documentation. when updating a database. The DBMS writes to the disk containing the transaction log. SRDF/CG prevents a rolling disaster from affecting the integrity of the data at the remote site. a DBMS takes the following steps: 1. SRDF/CG suspends the remote mirroring for all volumes defined to the consistency group before completing the intercepted I/O and returning control to the application. In this way.Introducing SRDF A dependent write is a write is not issued by an application until a prior. This could occur. When SRDF/CG detects any write I/O to a volume that cannot communicate with its remote mirror. The DBMS writes the data to the actual database dataset. An example of dependent write is a database update. but its predecessor was not remotely mirrored.

R2) device contains valid data and is available for performing business continuance tasks through its original device address or restoring (copying) data to the primary (source. a local host connects to Symmetrix A. At the recovery site. R1) device. (Not ready means disabled for both reads and writes. ”SRDF Operations. 22 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . SRDF uses a method of data protection known as remote mirroring. Note: Additional SRDF configurations are explained in “SRDF configurations” on page 27. The Symmetrix systems communicate through SRDF links. making the R2 device fully accessible again to its host. refer to Chapter 3. the R2 device is either write disabled or not ready to its host. as one method of protecting data by maintaining data on both a production volume and a mirror volume within the same storage unit. When the main storage systems are down for a planned or unplanned event. Remote mirroring is similar to local mirroring. a second (optional) host connects to Symmetrix B with the secondary (target) or R2 volume containing the remotely mirrored data. The local SRDF device.) After the R2 device becomes synchronized with its R1 device.” Basic SRDF configuration A basic SRDF configuration consisting of a production site and a recovery site is illustrated in Figure 1 on page 23. except that the production volume resides in one storage unit while its remote mirrors (up to two remote mirrors are supported with Concurrent SRDF) reside in a different storage unit. After the split. or RAID 1. At the production site. R2) device. Note: For more information on SRDF operations. the secondary (target. SRDF enables fast switchover from the primary (source) data to the secondary (target) copy. While the R2 device is mirrored with the R1 device.Introducing SRDF SRDF functional overview Symmetrix systems use local mirroring. R1) device if there is a loss of data on that device. forming an SRDF pair. known as the primary (source. The device containing the production data to be remotely mirrored is called the primary (source) or R1 volume. is configured in a pairing relationship with a secondary (target. you can split the R2 device from the R1 device at any time.

Enginuity level 5x67 is the first Symmetrix family of Enginuity that supports connectivity with subsequent families of Enginuity (interfamily). This feature’s usefulness is realized with all Enginuity releases from level 5x68 and higher. Interfamily SRDF enablesan upgrade of a network of Symmetrix systems over a period of weeks or months while maintaining SRDF protection.Introducing SRDF Site A (Production) Site B (Recovery) Local Host Remote Host Active Host Path Recovery Path Symmetrix A SRDF Links Symmetrix B R1 R2 Local Volume Primary (Source) Volume Local Volume Secondary (Target) Volume Figure 1 Basic SRDF configuration SRDF interfamily connectivity SRDF supports connectivity between Symmetrix models running various levels of Enginuity.com/emcpubs/elab/esm/pdf/ srdf%20interfamily%20support. SRDF functional overview 23 .EMC.pdf. allowing nondisruptive Enginuity upgrades in SRDF environments. This connectivity support is described at: https://elabnavigator.

Dynamic Sparing is used as additional protection for mirrored Parity RAID and SRDF volumes. Increases data availability by copying the data on a failing volume to a spare volume until the original device is replaced. Release 5771 only. RAID 5 technology stripes data and distributes parity blocks across all the disk drives in the RAID group.data from lost volume is regenerated from remaining members A Parity RAID (3+1) group consists of three data volumes to one parity volume.EMC. For more information on SRDF related software. A Parity RAID (7+1) group consists of seven data volumes to one parity volume.com or contact your EMC Sales Representative. After SDRF is up and running. Provides: • High performance with automatic striping across hypervolumes • High availability . Replaces a faulty drive automatically from a list of available spares residing in the Symmetrix system without CE involvement on site. Data Protection options for SRDF disk devices a Description Provides the highest level of performance and availability for all mission-critical and business-critical applications by maintaining a duplicate copy of a volume on two disk devices. Provides a combination of RAID 1 and RAID 0 for open systems environments. Solutions Enabler for UNIX and Windows environments. Data is striped across mirrored pairs. Provides: • High performance dependent upon volume layout and external striping • High availability . consult your EMC Sales Representative for configuration rules.lost hypervolumes regenerated from remaining members RAID 5 is configured in (3+1) and (7+1) groups. or the SRDF Host Component for z/OS. Provides a combination of RAID 1 and RAID 0 used for mainframe environments. Not all data protection options are available on all Symmetrix systems or Enginuity levels. you can monitor and control its operation by purchasing the appropriate EMC ControlCenter® software. When configuring multiple data protection options.Introducing SRDF SRDF supported data protection options Table 2 The Symmetrix data protection options described in Table 2 on page 24 can be purchased separately and implemented into the Symmetrix SRDF operation. 24 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Data Protection Option Mirroring (RAID 1) Symmetrix RAID 1/0 Symmetrix RAID 10 Parity RAID RAID 5 Permanent Member Sparing Dynamic Sparing a. Monitoring and controlling SRDF An EMC representative installs and initially configures SRDF at your site using the Symmetrix service processor. consult the EMC Powerlink website at: http://Powerlink.

Introducing SRDF SRDF director hardware SRDF director hardware consists of a director and adapter board set. consult EMC Customer Support. For more information. SRDF uses a storage protocol based on the Gigabit Ethernet. GigE remote adapter (RE) Fibre Channel remote adapter (RF). ESCON remote adapter (RA). supported with Symmetrix DMX series. The host attachment. I/O protocol. Fibre Channel FC-4. fiber-optic protocol support. and disk data structures that each host requires are independent of the SRDF operation between Symmetrix systems. or ESCON specifications to remotely mirror data between Symmetrix systems. one processor on each board for SRDF to provide redundant links in case a communications link fails or in the unlikely event a director fails. ! CAUTION EMC recommends that the Symmetrix system has at least two director boards for SRDF protection. and configurable for SRDF over Gigabit Ethernet (GigE). SRDF director functions The SRDF director/adapter board sets described above provide the link connections. SRDF director hardware 25 . depending on the protocol: ◆ Multiprotocol Channel Director (MPCD). SRDF director hardware includes any of four types of director/adapter board sets. if one processor on the director is used for SRDF and the other processor(s) is/are used to connect to a host. This hardware provides the communications physical layer for SRDF data and information exchanges between Symmetrix systems. Note: Some restrictions apply in mixed environments with iSeries and other host types. GigE remote directors (RE) or Fibre Channel remote directors (RF) maintain a peer-to-peer relationship at the transport layer of communications. and communications control between two Symmetrix systems in an SRDF configuration. ◆ ◆ ◆ The SRDF network options listed above can be used for any host environment. Having two director boards for SRDF avoids a potential single point of failure.

With Symmetrix 8xxx models.Introducing SRDF The ESCON remote director (RA) board set that normally sends data across an SRDF link is known as an RA-1. An RA-1 functions like a host channel interface. With Symmetrix DMX1000. an optional four-processor ESCON board can be used for SRDF. there can be multiple RA pairs in an SRDF configuration. ◆ With Symmetrix 3xxx or 5xxx models. An RA-2 functions like a storage director interface. An RA-1 and its corresponding RA-2 are known as an RA pair. an optional four-processor ESCON board can be used for SRDF. and DMX3000 models. The ESCON RA board set that normally receives data sent across an SRDF link is known as an RA-2. ◆ ◆ Note: Each Symmetrix Product Guide provides detailed descriptions of the SRDF supported hardware functionality. up to a maximum of 16 pairs. DMX2000. 26 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

Site A (Production) Site C (Production) Local Host Site B (Recovery) Local Host Active Host Path Remote Host Active Host Path Symmetrix A Recovery Path Symmetrix B Symmetrix C R1 SRDF Link SRDF Link R1 R2 R2 Local Volume R1 Source Volume R2 Target Volume Figure 2 Two production sites and one recovery site SRDF configurations 27 .Introducing SRDF SRDF configurations This section provides examples of typical SRDF configurations. In Figure 2 on page 27. two production sites (A and C with Symmetrix DMX-3 system bay) send data across SRDF links to one recovery site (B Symmetrix DMX).

C. one recovery site (G) provides a data vaulting solution for six production sites (A through F). and F Site D Site E Site F Figure 3 Data vaulting solution 28 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . E.Introducing SRDF In Figure 3 on page 28. D. B. Site A Site B Site C Site G Site G acts as a Data Vaulting site for Sites A.

Introducing SRDF Figure 4 on page 29 illustrates the versatility of SRDF. Symmetrix D Symmetrix E Symmetrix C R2 R2 R1 R2 R1 R2 R1 Symmetrix F R1 R2 R1 Symmetrix B R1 R2 Symmetrix G = Primary Volumes = Secondary Volumes R1 R2 Symmetrix A Figure 4 Sites containing both primary and secondary volumes SRDF configurations 29 . some sites have either primary (R1) or secondary (R2) devices. while other sites have both primary (R1) and secondary (R2) devices.

Introducing SRDF SRDF configuration using switched Fibre Channel Figure 5 on page 30 shows a more flexible switched SRDF configuration using Fibre Channel switches connected through E_Ports. Switched SRDF also enables fewer RFs to be present on the fabric. multiple primary (source) R1 devices are remotely connected using a Storage Area Network (SAN) to multiple secondary (target) R2 devices. Switched SRDF involves non blocking switching devices that interconnect two or more nodes. Switched SRDF allows all RFs in the Symmetrix systems connected to the fabric to communicate with each other. Note that in this configuration. depending on system performance and redundancy requirements. providing greater connectivity and configuration flexibility. 30 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . R2 Switch E_Port Switch E_Port R2 Switch E_Port E_Port R1 R1 R1 = Primary (Source) Devices R2 = Secondary (Target) Devices R2 Figure 5 Switched SRDF with multiple primary and secondary devices SRDF fully switched fabric connectivity Enginuity 5x67 and later supports fully switched open SRDF (fibre) connections.

The ability to support point-to-multipoint connectivity is unrelated to primary (source) or secondary (target) device relationships. Symmetrix A is communicating with Symmetrix B and Symmetrix C simultaneously through two RF directors.Introducing SRDF In Figure 6 on page 31. When Symmetrix A is communicating with Symmetrix B. there is a single logical path from each RF director to one other RF director. 5669 R2 Volumes RF Point-to-Point FC Switch Symmetrix B 3XXX/5XXX. either of its RF directors can communicate with either RF director in Symmetrix C. 5267 RF RF R1 Volumes R2 Volumes RF R1 Volumes Symmetrix C 8000. SRDF configurations 31 . These are point-to-point Fibre Channel connections. Symmetrix A DMX. the same as was provided with earlier Enginuity levels. 5568 Point-toMultipoint RF RF R2 Volumes R1 Volumes Figure 6 Switched SRDF over Fibre Channel with Enginuity 5x67 There are four logical paths from Symmetrix A to Symmetrix C (two logical paths on each RF director to each RF director in Symmetrix C). Also note that all three Symmetrix systems have primary (source or R1) devices. When Symmetrix A is communicating with Symmetrix C.

either of its RF directors can communicate with either RF director in Symmetrix C. 5669 Symmetrix B 3XXX/5XXX. possibly using Adaptive Copy Disk mode and/or FarPoint. 32 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The ability to support point-to-multipoint connectivity is not related to primary (source) or secondary (target) device relationships. there is a single logical path from each RF director to one other RF director. Symmetrix A is communicating with Symmetrix B and Symmetrix C simultaneously through two RF directors. Symmetrix A DMX. 5267 FC Switch RF RF RF RF RA RA Symmetrix Devices 00-0F R1 Volumes R1 Volumes 10-1F R1 Volumes R1 Volumes R2 Volumes Symmetrix C 8000. When Symmetrix A is communicating with Symmetrix C.Introducing SRDF Switched and Concurrent SRDF In Figure 7 on page 32. 5x67 Figure 7 Symmetrix Devices 00-1F R2 Volumes (second remote copy of all SRDF devices in Symmetrix A) Switched and Concurrent SRDF configuration example There are four logical paths from Symmetrix A to Symmetrix C (two logical paths on each RF director to each RF director in Symmetrix C). These are point-to-point Fibre Channel connections. When Symmetrix A is communicating with Symmetrix B. the same as was provided with earlier Enginuity levels. as well as using Concurrent SRDF to mirror its primary devices (R1s) to a second set of secondary devices (R2s) in Symmetrix D. Each RF director in Symmetrix A is configured to support two SRDF groups. A separate SRDF group in Symmetrix A is configured with FarPoint™ on ESCON RAs for Concurrent SRDF to Symmetrix D. 5568 Any Supported Link RF RF R2 Volumes RA RA Symmetrix D 3XXX/5XXX or 8000.

The Symmetrix 3xxx/5xxx systems can be configured with a secondary volume that is one of two remote copies from a Symmetrix 8000 primary volume. the same local primary volume can be remotely mirrored in two different locations. the secondary mode of adaptive copy can be enabled for either one or both secondary R2s. with the exception of data compression. Unless otherwise specified. With Enginuity level 5567 and the Symmetrix 8000 systems. Symmetrix 8000 series systems provide native IP support through the GigE remote director. which is a feature of the MPCD only.Introducing SRDF For Concurrent SRDF. however. In Figure 7. Note: As of Symmetrix Enginuity level 5671. the Symmetrix 3xxx/5xxx systems cannot be configured to provide two remote copies of the same primary volume. the following information applies to both Symmetrix 8000 series and Symmetrix DMX GigE remote director configurations. Symmetrix DMX series systems provide native IP support through the Multiprotocol Channel Director (MPCD). This functionality is called Concurrent SRDF. SRDF configurations 33 . SRDF with native GigE Native IP support for any SRDF based product on Symmetrix systems is based on GigE technology that enables direct Symmetrix system-to-IP-network attachment. the RA links are extended via channel extenders over a WAN using adaptive copy mode and/or FarPoint for asynchronous replication to Symmetrix D. the primary SRDF mode is a synchronous mode of Symmetrix As primary devices (R1s). SRDF/A is supported on one of the two groups using Concurrent SRDF. Note: For more information on Concurrent SRDF. and allows a Symmetrix system to connect to existing Ethernet infrastructure and directly access high-speed data transmission conduits via IP. The MPCD and GigE remote directors provide comparable functionality. however. refer to “Concurrent SRDF” on page 60. This increases the options for Symmetrix system to Symmetrix system connectivity.

Hence. Symmetrix A GigE Switch/Router RE RE Symmetrix B GigE Switch/Router WAN RE RE IP Network Figure 8 Switched GigE network configuration example SRDF and FarPoint FarPoint is an SRDF feature used only with ESCON extended distance solutions (and certain ESCON campus solutions) to optimize the performance of the SRDF links. this configuration will have four logical TCP connections between the Symmetrix pair. Each Symmetrix GigE source port will establish a logical TCP connection to each GigE target port. The next write I/O occurs only after the remote Symmetrix system has acknowledged receipt of the previous I/O from the local Symmetrix system and an ending status is presented to the local Symmetrix system as shown in the top half of Figure 9 on page 35. over each SRDF link. The standard ESCON SRDF protocol (without SRDF FarPoint) allows only one write I/O to occupy a communication link at a time. Standard SRDF without FarPoint 34 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . This feature works by allowing each RA to transmit multiple I/Os. in series.Introducing SRDF Figure 8 on page 34 illustrates GigE connectivity into an IP network infrastructure. Note: The EMC Network Storage Topology Guide and the EMC SRDF Connectivity Guide contain further information.

With FarPoint.Introducing SRDF SRDF with FarPoint With SRDF FarPoint. uses the free link bandwidth to send another write operation from a different primary SRDF configurations 35 . FarPoint does not change the SRDF protocol. The number of write I/Os that can be placed on the link at one time is determined by the capacity of the type of SRDF link being used. multiple copy tasks can be placed on the SRDF link for a single logical volume. Without FarPoint. the Symmetrix system returns a completion status to the host only after the write operation is performed on the remote Symmetrix system. This method (referred to as pipelining) enables the SRDF communication link to be more fully utilized. while waiting for the status of the first write operation. the Symmetrix system. In synchronous mode. Symmetrix SRDF without FarPoint Symmetrix Data Response Symmetrix SRDF with FarPoint Symmetrix Data Response Data Response Data Response Figure 9 SRDF with and without FarPoint Preserving synchronization From the point of view of the host. the Symmetrix system waits for one write operation to complete before sending the next one. depending on the distribution of application I/O write activity across multiple logical volumes. a single remote adapter can serially transmit more than one write I/O over the SRDF link. With Enginuity level 5x67 or later.

so in the case of a synchronous device. and using FarPoint in this situation does not improve system performance. so the synchronous condition is fully preserved. Note: The introduction of PAV/MA does increase I/O concurrency at the volume level. write operations to a single device are still serialized by the host. An SRDF configuration using FarPoint can improve performance substantially at the Symmetrix system level across a group of SRDF devices depending on the length of the SRDF link. potentially improving throughput in FarPoint configuration operating in a mainframe environment. the write pipeline is cleared before any read operations are performed. Interaction with the host remains unchanged. are serviced using the standard SRDF protocol (without FarPoint). In this situation. This means that the maximum number of write operations per device is still low.Introducing SRDF volume. However. Servicing remote reads Remote read operations (mainly used for recovery purposes). as well as other special I/Os. Performance impact of FarPoint 36 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . the FarPoint solution may provide some benefits. and the data on the remote SRDF device is 100 percent consistent from the host's point of view. a new write operation does not start until the previous write operation to that device receives a status from the remote system. Even on a short link.

............................................ 38 42 44 47 48 53 56 58 60 62 SRDF Technical Concepts 37 ......................................................................................................................... system-level attributes........................................................................................... Secondary modes of operation... SRDF groups .................................... modes of operation...................................... SRDF link configurations................................................ Additional SRDF modes and attributes.. SRDF link and volume states ..................................Invisible Body Tag 2 SRDF Technical Concepts This chapter describes SRDF links............... Concurrent SRDF ........................ SRDF/Consistency Groups (SRDF/CG) ............................................................................... Primary modes of operation......................... SRDF volumes: Special considerations ................................ This chapter contains the following topics: ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ SRDF volume types ................. and consistency groups.....

provides a combination of RAID 1 and RAID 0 for open systems. (the primary volume is then referred to as a mirrored pair) RAID 10. supported on Enginuity level 5771 RAID 1. Primary (source) volumes Primary (source) volumes contain production data that is mirrored in a different Symmetrix system.SRDF Technical Concepts SRDF volume types SRDF refers to Symmetrix volumes as: ◆ ◆ ◆ ◆ ◆ Primary (source) volumes (R1) Secondary (target) volumes (R2) Local volumes Dynamic SRDF devices EMC Compatible Peer (PPRC mode) devices Note: In the context of this discussion. Updates to a primary volume are automatically mirrored to a secondary (target) volume in the other Symmetrix system. a primary volume can be paired with a Business Continuance Volume (BCV) to provide an additional working copy of the data at the same location. conventional mirroring. volumes are synonymous with devices. (the primary volume is a striped mirrored device) RAID 1/0. Primary volumes can be locally protected by: ◆ A dynamic spare “Dynamic Sparing with remotely mirrored pairs (SRDF)” on page 44 provides more information) Permanent member sparing. 38 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Data is striped across mirrored pairs Symmetrix Parity RAID (3+1) and Parity RAID (7+1) protection (the primary volume is a Parity RAID data volume) RAID 5 (3+1) and RAID 5 (7+1) protection (the primary volume is a RAID 5 data volume) ◆ ◆ ◆ ◆ ◆ ◆ Additionally. Primary volumes are also referred to as R1 volumes.

RAID 10. you can create SRDF device pairs from non-SRDF devices. delivered in Enginuity level 5568. and then synchronize and manage them in the same way as configured SRDF pairs. this capability is only allowed in non-FarPoint configurations. but they do not participate in SRDF activity. As with primary volumes. Parity RAID (3+1). secondary volumes can be locally protected by: ◆ A dynamic spare provides “Dynamic Sparing with remotely mirrored pairs (SRDF)” on page 44 more information) Permanent member sparing. provides a combination of RAID 1 and RAID 0 for open systems. enables you to create. while the Symmetrix system is in operation. devices can be configured to be Dynamic SRDF capable devices.SRDF Technical Concepts Secondary (target) volumes Secondary (target) volumes contain a mirrored copy of data from a primary volume. With Dynamic SRDF. supported on Enginuity level 5771 RAID 1. The Dynamic SRDF functionality. Local volumes Local volumes can reside on an SRDF enabled Symmetrix system. Local volumes are typically protected either by RAID 1. Figure 10 on page 40 shows Dynamic SRDF connections in a switched fabric environment. or a dynamic permanent member spare. Secondary volumes are also referred to as R2 volumes. (the primary volume is a striped mirrored device) RAID 1/0. and swap SRDF pairs. delete. Dynamic SRDF devices SRDF volume types 39 . RAID 5 (7+1). Since Enginuity level 5567. Parity RAID (7+1). RAID 5 (3+1). conventional mirroring (the primary volume is then referred to as a mirrored pair) RAID 10. using EMC host-based SRDF control software. The only Dynamic SRDF functionality delivered in Enginuity level 5567 is R1/R2 personality swap where primary and secondary volume pair roles can be reversed. Data is striped across mirrored pairs Symmetrix Parity RAID (3+1) and Parity RAID (7+1) protection (the primary volume is a Parity RAID data volume) RAID 5 (3+1) and RAID 5 (7+1) protection (the primary volume is a RAID 5 data volume) ◆ ◆ ◆ ◆ ◆ ◆ A secondary volume also can be paired with a BCV to provide an additional working copy of the data at the same location. Further.

Possibility #1 Switch Switch Switch Switch R2 R1 Possibility #2 Symmetrix Figure 10 Dynamic SRDF Dynamic SRDF provides the following capabilities: ◆ ◆ ◆ Personality swap between primary and secondary volumes Terminate and reestablish a relationship with a new secondary volume Create a new primary/secondary pair relationship from non-SRDF devices Note: The above functionality is not available for devices protected by Parity RAID. enable the user to have additional control over their SRDF configuration. These capabilities. 40 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . SRDF/A devices also can be configured as Dynamic SRDF capable devices. and SRDF devices. Using EMC host-based software. when combined with Dynamic SRDF groups.SRDF Technical Concepts Note: At Enginuity level 5x71. SRDF groups. Further. Dynamic SRDF is supported over the following topologies: ◆ ◆ ◆ ◆ ESCON point-to-point connection (RA) Fibre Channel point-to-point connection (RF) Switched Fibre Channel fabric connection (RF) GigE connection (RE) Symmetrix R2 Symmetrix API/ CLI/ ECC/etc. the personality swap function is unavailable for ESCON Farpoint configuration. Consult the appropriate host-based software product guide for specific instructions on how to create or remove RAs. the user can create or remove RA directors. Prior to Enginuity level 5x71 this function was not available for SRDF/A.

including failover/failback functionality. Beginning with Enginuity level 5568.redbooks. no host-based EMC SRDF control software can operate on the device.html SRDF volume types 41 . architectural level 2. Note: Advanced SRDF features such as Concurrent SRDF or SRDF/Asynchronous are not supported on EMC Compatible Peer. All other Symmetrix data protection mechanisms are supported. Enginuity level 5671 adds support for PPRC version 1. Compatible Peer implements PPRC version 1. it is truly a PPRC device and can only be controlled by PPRC commands received from the host. Note: Control of these devices via out-of-band software is unsupported. From the time that the device enters PPRC mode. For more detailed information about implementations.com/abstracts/sg246374.SRDF Technical Concepts EMC Compatible Peer Enginuity level 5568 (or higher) enables the Symmetrix system to support native IBM Peer-to-Peer Remote Copy (PPRC) commands through a Symmetrix feature called EMC Compatible Peer. Compatible Peer mode requires Dynamic SRDF capable devices.ibm. architectural levels 3 and 4 Hyperswap support. go to: http://www. A Dynamic SRDF device becomes a PPRC mode device upon receipt of a PPRC CESTPAIR command. Parity RAID data protection is not supported on EMC Compatible Peer.

Enginuity level 5669 increases the maximum number of definable SRDF groups from 16 to 64. Prior to Enginuity level 5568. static SRDF groups were required to have at least one static SRDF device assigned to the groups. Logical volumes (devices) are assigned to SRDF groups. each Symmetrix system supported up to 16 static SRDF groups. With Enginuity levels up to and including 5x68. This enhancement allows more flexibility when working with the following SRDF features: ◆ ◆ ◆ ◆ EMC Compatible Peer Switched SRDF Concurrent SRDF Dynamic SRDF 42 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .SRDF Technical Concepts SRDF groups SRDF groups define relationships between Symmetrix systems. An SRDF group configured through Symmetrix configuration is called a static SRDF group. An SRDF group is a set of SRDF director port connections configured to communicate with another set of SRDF director ports in another Symmetrix system. Further device assignments could be made via static (configurations changes) or dynamic SRDF.

when combined with dynamic SRDF devices. a user can dynamically create empty SRDF groups and dynamically associate the groups with Fibre Channel or GigE SRDF directors. Using EMC host-based software. Both of these operations are accomplished using EMC host-based SRDF control software. Removing dynamic SRDF groups is also possible. This feature. Dynamic SRDF groups created through this method are persistent through Symmetrix power on or IMPL.SRDF Technical Concepts Dynamic SRDF groups At Enginuity level 5669 or above. and SRDF devices. SRDF groups. Consult the appropriate host-based software product guide for specific instructions on how to create or remove RAs. SRDF groups 43 . the user can create or remove RA directors. Note: EMC Compatible Peer groups require static SRDF groups. Dynamic groups are not supported for PPRC. allows the user complete control over their SRDF configuration.

When the Dynamic Sparing option is invoked for a remotely mirrored SRDF pair. it will retrieve the unavailable data from the good member of the remote pair. it fails over a period of time during which it gives clues that it is failing. the purpose of a dynamic spare is to take the place of a failed or failing disk device. Open systems metavolumes When users have open systems hosts attached to Symmetrix systems. when a disk drive fails. As the name suggests. or hot spares. Metavolumes are functionally the same as logical volume sets implemented with host volume manager software. they can create a logical device that spans multiple Symmetrix logical volumes. If a disk drive begins to fail. A metavolume is a logical volume set created from individual Symmetrix logical volumes that can comprise one-to-multiple physical disks. This logical device is a metavolume. the Symmetrix system automatically invokes the dynamic spare. If the Symmetrix system cannot copy all data from the failing device to the spare.SRDF Technical Concepts SRDF volumes: Special considerations The following Symmetrix features require special considerations when used in an SRDF environment: ◆ ◆ ◆ ◆ Dynamic Sparing with remotely mirrored pairs (SRDF) Open systems metavolumes mainframe RAID 10 volumes PPRC command support Dynamic Sparing with remotely mirrored pairs (SRDF) Typically. 44 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . it does not fail suddenly. The system continues processing I/O requests with the spare functioning as one volume of a mirrored pair while the failing device and its remote mirror all operate without interruption. Some metavolumes are created to define volumes larger than the current Symmetrix maximum hypervolume size of approximately 32 GB. A Symmetrix system can be configured with physical disk devices known as dynamic spares. the Symmetrix system automatically activates an available spare in the Symmetrix system containing the failing device and copies data from the failing device to the spare. The Symmetrix system monitors the operation of all its disk drives and can detect when a disk is beginning to fail.

Note: When configuring a metavolume. Contact EMC Customer Service for support where an R2 is larger than an R1. Symmetrix metavolumes can contain up to 255 logical volumes and be up to 7. Symmetrix RAID 10 stripes data of a logical device across multiple Symmetrix logical devices. At Enginuity level 5669.65 TB in size. each metavolume device is counted as a single logical volume.SRDF Technical Concepts Physically. you must also create them for the secondary volumes. Using metavolumes will reduce the maximum number of host-visible devices. a metavolume is two or more Symmetrix hypervolumes presented to a host as a single addressable device. some number of member devices (optional). the entire set is protected. The primary and secondary meta members must be equal in number. The metavolume consists of a head device. In an SRDF configuration. The size of the non-RAID 10 volume must equal the sum of all members in the RAID 10 volume. RAID 10 uses four pairs of disks in its Symmetrix implementation. RAID 10 striped volumes can be remotely paired with non-RAID 10 volumes. in an SRDF configuration. Symmetrix RAID 10 (mirrored striped mainframe volumes) To improve mainframe volume performance. each member of the metavolume must be counted toward the maximum number of host-supported Symmetrix logical volumes. Concatinated meta volumes must have equal size members (primary and secondary). Since each member of the stripe group is mirrored. IBM 3390) and have the same mirror configuration. Note: Enginuity level 5568. Striping occurs across this group of four devices with a striping unit of one cylinder. Four Symmetrix devices (each one-fourth the size of the host facing mainframe device) appear as one mainframe device to the host. Any four Symmetrix logical devices can be chosen to define a RAID 10 group provided they are the same type (for example. Consult the specific Symmetrix model product guide for information on supported Symmetrix logical volume limits. SRDF volumes: Special considerations 45 . if you create metavolumes for the primary volumes. Metavolumes can be composed of nonsequential and nonadjacent volumes. That is. the sum of the R2 volumes can be larger than the sum of the R1 volumes. and a tail device. Extra care must be observed with striped meta volumes.

html 46 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .ibm. Enginuity level 5771 adds support for PPRC version 1.com/abstracts/sg246374.redbooks. Symmetrix systems will support these capabilities in IBM’s Geographically Dispersed Parallel Sysplex (GDPS) solution. Compatible Peer is available on Symmetrix systems with connections to ESCON and/or FICON hosts. architectural levels 3 and 4 Hyper-Swap support. As a result. Enginuity level 5568 supports PPRC version 1. PPRC is the remote copying solution available with IBM storage systems as shown in Figure 11 on page 46. Note: Contact your local EMC representative for specific details regarding your Symmetrix system's support for Compatible Peer. go to: http://www.SRDF Technical Concepts PPRC command support Enginuity level 5568 or higher enables the Symmetrix system to support native IBM Peer-to-Peer Remote Copy (PPRC) commands through a Symmetrix feature called Compatible Peer. architecture level 2 (CGROUP FREEZE/RUN functionality. Primary Site Automated site failover via Automated site failover via compiled REXX “Scripts” compiled REXX Scripts Symmetrix Fully PPRC Symmetrix Fully PPRC Command Compliant Command Compliant Recovery Site PPRC commands PPRC commands converted to SRDF converted to SRDF PPRC commands PPRC commands converted to SRDF converted to SRDF R1 R1 SRDF SRDF R2 R2 SYM-000092 Figure 11 PPRC and GDPS support Note: For more information on PPRC. including failover/failback functionality).

SRDF link configurations 47 . write operations move in one direction. while other groups send data in the opposite direction. and ATM links and/or IP network connections) that require data to move in two directions. a dual-directional link configuration may be required. For more information. E3. If an SRDF group contains both primary and secondary volumes. This is a unidirectional configuration. T1. in which data moves in the same direction over every link in the SRDF group. consult the EMC SRDF Connectivity Guide. R1) volumes reside in one Symmetrix system and all secondary (target. Note: For information about how write operations occur using an ESCON SRDF bidirectional link configuration. SRDF bidirectional link configuration SRDF dual-directional link configuration For an ESCON-based extended-distance SRDF configuration (for example using E1. from primary to secondary. write operations move data in both directions over the SRDF links for that group. some groups send data in one direction. multiple SRDF groups are used. go to “Write operations” on page 68.SRDF Technical Concepts SRDF link configurations The links between a given pair of Symmetrix systems in an SRDF configuration can use one of the following methods for transmitting data: ◆ ◆ ◆ Unidirectional Bidirectional Dual-directional SRDF unidirectional link configuration If all primary (source. T3. With a dual-directional configuration. This is called an SRDF bidirectional configuration. R2) volumes reside in another Symmetrix system.

the Symmetrix system retains the last known logical state for the affected devices. it is restored to a ready state. Prior to Enginuity level 5669. The Symmetrix systems are configured properly on both sides of the SRDF configuration. 48 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . these configuration settings are applied at the SRDF group level. ◆ ◆ The SRDF link is offline if one or more of the following occurs: ◆ ◆ Configuration settings affecting device ready and link states If all SRDF links fail. This behavior is a pure ready state. The SRDF link is online when the following occurs: ◆ The remote adapter is operational and enabled on both sides of the SRDF configuration. The remote adapter is online but the link is offline (damaged or disconnected cable or other damaged hardware) — link disabled. ”SRDF/Asynchronous Operations. At Enginuity level 5669 and later. If a device is in a not-ready state before a link failure. prevents SRDF links from coming online following a Symmetrix power cycle. both configuration settings described above were applied at the Symmetrix system level. Note: The device level pure ready state behavior does not apply to SRDF/A mode of operation. if a device is in a ready state before a link failure. The configuration setting. it is restored in a not-ready state after the link is again made ready.SRDF Technical Concepts SRDF link and volume states SRDF link states SRDF links can be in two possible states: online or offline. Consult Chapter 4. However. all devices in the group remain not ready on the SRDF link after all links fail if the prevent automatic recovery after all links fail setting is set to Yes in the Symmetrix configuration. In other words.” for information on how SRDF/A works. Force RAs offline after power up. The external link infrastructure components are operational. The remote adapter is offline — link disabled.

source R1 or secondary. look at the two states as two layers: ◆ ◆ An internal layer (SRDF view )— The SRDF state An external layer (host view) — The host interface state These two layers are configured by different sets of internal Symmetrix parameters. as shown in Figure 12 on page 49. SRDF link and volume states 49 .SRDF Technical Concepts Logical volume states This section describes the three device states that an SRDF logical volume (primary. Ensure you fully comprehend this section before attempting any SRDF operations. target R2) reflects to the host connected to the Symmetrix unit on which the volumes are located. The device states can be: ◆ ◆ ◆ Not Ready Read Only Write Enabled ! CAUTION Understanding the device states of an SRDF logical volume is fundamental to SRDF operation. The state ultimately seen by the host is determined by the combination of these two device states. Host Interface State Host SRDF State Figure 12 SRDF logical volume state In general. which can be set by either EMC Customer Service or by software. Device states are determined by a combination of two substates: the SRDF state and the device host interface state.

◆ ◆ Secondary (R2) volume states A secondary (R2) volume can have one of the three states listed for SRDF operations: ◆ Not Ready — In this state. ◆ 50 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The following sections describe these two substates and how the various device substate combinations determine the actual overall device state presented to the host. SRDF volume states (Symmetrix view) Primary (R1) volume states This section lists the substates a primary (R1) or secondary (R2) volume can have for SRDF operations. the secondary (R2) volume is available for read-only operations. Changes to the R1 volumes are marked invalid as owed to the secondary volumes R2. the primary (R1) volume is available for read/write operations. the secondary (R2) volume responds not ready to the host for all read and write operations to that volume. This is the default secondary (R2) volume state. • Remotely Not Ready — If R1 volumes are remotely not ready. For example. A primary (R1) volume can have the following states listed for SRDF operations: ◆ Ready and Read/Write — In this state. the host continues to recognize that volume as available for read/write operations as all reads and/or writes continue uninterrupted with the secondary (R2) volume in that remotely mirrored pair. only when a secondary (R2) device is write enabled for both the host interface state and the SRDF state can the host write to the device. the secondary (R2) volume responds not ready to the host for all read and write operations to that volume. Read Only — In this state.SRDF Technical Concepts Table 3 on page 52 and Table 4 on page 52 summarize the device state seen by the host as determined by the individual states or substates. This is the default primary (R1) volume state. write updates will not propagate to the secondary volumes. Not Ready — SRDF volumes can be not ready locally or not ready remotely on the SRDF link: • Locally Not Ready — If the local (R1) volume fails. Host Not Ready — In this state.

This represents the volume state as seen by the host interface. the secondary (R2) volume responds not ready to the host for all host accesses to that volume.SRDF Technical Concepts ◆ Read/Writ e — In this state. the secondary (R2) volume is available for read/write operations. Note: When a secondary volume is placed into a Read/Write state. Not Ready — In this state. the secondary (R2) volume responds device write protected to the host for all write operations to that volume. the secondary (R2) volume is available for read/write operations. Read Only — In this state. the primary (R1) volume is available for read/write operations. This is the default primary (R1) volume state. Table 4 on page 52 lists SRDF link and volume states 51 . Read Only — In this state. This state is seen by the host connected to the Symmetrix unit in which that volume resides: ◆ Write Enabled — In this state. This is the default secondary (R2) volume state. Symmetrix volume states (host view) This section lists the states a primary (R1) or secondary (R2) volume has for host operations. This state is seen by the host connected to the Symmetrix unit in which that volume resides: ◆ Write Enabled — In this state. the corresponding primary volume is placed into a Remotely Not Ready state. SRDF Not Ready — In this state. Table 3 on page 52 lists the accessibility for a primary (R1) volume. Primary (R1) volume states A primary (R1) volume can have one of the following states. ◆ ◆ Host accessibility The tables in this section describe the accessibility state of the primary (R1) and secondary (R2) volumes to the host connected to the Symmetrix system containing the primary (R1) volumes. the primary (R1) volume responds not ready to the host for all host accesses to that volume. the primary (R1) volume responds device write protected to the host for all write operations to that volume. The accessibility of the host to a particular Symmetrix volume depends on its state from both the host and SRDF view. ◆ ◆ Secondary (R2) volume states A secondary (R2) volume can have one of the following states.

Table 3 Primary (R1) volume accessibility SRDF state Read/Write Not Ready Read/Write Not Ready Read/Write Not Ready Accessibility Read/Write Depends on secondary (R2) volume availability Read Only Depends on secondary (R2) volume availability Unavailable Unavailable Host interface state Write Enabled Write Enabled Write Disabled Write Disabled Not Ready Not Ready Table 4 Secondary (R2) volume accessibility SRDF state Not Ready Read Only Read/Write Not Ready Read Only Read/Write Not Ready Read Only Read/Write Accessibility Unavailable Read Only Read/Write Unavailable Read Only Read Only Unavailable Unavailable Unavailable Host interface state Write Enabled Write Enabled Write Enabled Write Disabled Write Disabled Write Disabled Not Ready Not Ready Not Ready 52 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Consult the EMC Solutions Enabler Symmetrix SRDF CLI Product Guide and the Symmetrix SRDF Host Component for z/OS Product Guide for how this information is presented in these operating environments.SRDF Technical Concepts the accessibility for a secondary (R2) volume.

the local Symmetrix system does not accept additional write operations to the primary volume. synchronous mode maintains a realtime mirror image of data between the primary and secondary volumes. Note: In an SRDF/Asynchronous (SRDF/A) environment. 1. ensuring the highest possible data availability. Synchronous mode provides realtime mirroring of data between the local Symmetrix system and the remote Symmetrix systems. or if the host connection to the devices participating in SRDF is via a mainframe FICON interface. Synchronous mode processing is described in the following steps and illustrated in Figure 13 on page 54. Primary modes of operation 53 . Note: Semi-synchronous mode is not supported in Enginuity level 5771 or higher. the local Symmetrix system now accepts additional host write operations to the primary volume. The remote Symmetrix system sends an acknowledgement to the local Symmetrix system. Synchronous mode Available with the SRDF/S product offering. The local Symmetrix system sends an I/O complete message to the local host. SRDF synchronous mode supports multiple concurrent write I/Os at the volume level.SRDF Technical Concepts Primary modes of operation Select primary modes of operation for individual devices and manage these modes at the logical volume level. Note: In the mainframe PAV environment. 2. 3. Data must be successfully stored in both the local and remote Symmetrix systems before an acknowledgement is sent to the primary site host. Data is written to global memory of both systems before the application I/O is completed to the host. The local Symmetrix system containing the primary (source) volume receives a write operation from the host. select primary modes of operation for individual devices and manage SRDF/A at the SRDF group level. 4. The write I/O operation is propagated to the remote Symmetrix system (containing the secondary or target volume).

any number of read operations can be performed to the primary device while awaiting acknowledgement of the first write operation. Read operations continue normally. Data from the first write operation is propagated to the remote Symmetrix system containing the secondary volume. Semi-synchronous mode will not allow the next write operation to a primary device until a positive acknowledgement is received from the remote Symmetrix system that the first write operation was received in the remote Symmetrix global memory. The user application now considers the I/O to be complete. Data must be successfully stored in the Symmetrix system containing the primary volume before an acknowledgement is sent to the local host.SRDF Technical Concepts Host A 1 4 2 3 Global Memory Director Host B Global Memory Director Symmetrix containing primary (R1) volume Symmetrix containing secondary (R2) volume Figure 13 Synchronous mode Semi-synchronous mode Used mainly for the extended distance solution. semi-synchronous mode allows the primary and secondary volumes to be out of synchronization by one write I/O operation. 1. However. The local Symmetrix system sends an I/O complete message to the local host. 3. 2. Semi-synchronous mode writes data to the primary device in the local Symmetrix system. A second write I/O is not accepted. completes the I/O. The local Symmetrix system containing the primary volume receives a write operation from the user application. and then synchronizes the data with the secondary device in remote Symmetrix system as shown in the following steps and illustrated in Figure 14 on page 55. 54 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

semi-synchronous mode is not supported when the Symmetrix host attachment is through FICON connection. The remote Symmetrix system sends an acknowledgement from the first write operation to the local Symmetrix system. Note: In the mainframe environment. the semi-synchronous mode provides an added performance advantage. semi-synchronous mode is not recommended because it provides no performance benefit. Global Memory Director Global Memory Director Symmetrix containing primary (R1) volume Symmetrix containing secondary (R2) volume Figure 14 Semi-synchronous mode Primary modes of operation 55 . for this primary volume. In the mainframe PAV environment. Note: Because the I/O is completed before synchronizing data with the remote system. another write I/O from the host can be accepted. EMC recommends that SRDF users seeking the increased parallelism offered by PAVs run in SRDF in synchronous mode.SRDF Technical Concepts 4. When the acknowledgement is received.

When the maximum skew value is reached. select secondary modes of operation for individual devices and manage the following modes at the logical volume level. whichever is currently specified. secondary modes are used in conjunction with the primary operational modes described in “Primary modes of operation” on page 53. With SRDF/Data Mobility.SRDF Technical Concepts Secondary modes of operation Note: SRDF/Data Mobility is not available in synchronous or semi-synchronous modes. These modes allow the primary and secondary volumes to be more than one I/O out of synchronization. 56 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . only these secondary modes of operation are supported: ◆ ◆ Adaptive Copy Write-pending mode Adaptive Copy Disk mode Adaptive copy modes Adaptive copy modes facilitate data sharing and migration. There are two adaptive copying modes: adaptive copy write-pending (AW) mode and adaptive copy disk (AD) mode. The maximum number of I/Os that can be out of synchronization is known as the maximum skew value. As with the primary modes of operation. A background process moves. the write-pending tasks to the primary volume and its corresponding secondary volume on the other side of the SRDF link. either synchronous or semi-synchronous. Both modes allow write tasks to accumulate on the local system before being sent to the remote system. The maximum skew value for a volume can be set using the SRDF monitoring and control software. but is supported in adaptive copy mode only. Refer to “Adaptive copy modes” on page 56 for more information. With the SRDF/Synchronous product offering. write tasks accumulate in global memory. The default value is equal to the entire logical volume. The device remains in the primary mode until the number of tracks to remotely copy becomes less than the maximum skew value. or destages. the primary volume reverts to its primary mode of operation. Adaptive copy write-pending mode With adaptive copy write-pending mode.

In addition. the primary volume reverts to its primary mode of operation. more bandwidth is used because the unit of transfer is the entire track. that is. resulting in more efficient use of SRDF link bandwidth. either synchronous or semi-synchronous. additionally. Adaptive copy disk mode Adaptive copy disk mode is similar to adaptive copy write-pending mode. An additional advantage is that the unit of transfer across the SRDF link is the updated blocks rather than an entire track. Consequently. thus improving overall system performance. Parity RAID (7+1). The disadvantage is that global memory is temporarily consumed by the data until it is transferred across the link. adaptive copy write pending mode should only be used where detailed information about the host write workload is fully understood. or RAID 5 (7+1). except that write tasks accumulate on the primary volume rather than in global memory.SRDF Technical Concepts Note: Adaptive copy write-pending mode reverts to the currently specified primary mode (synchronous or semi-synchronous) if 75 percent of the write-pending limit for the Symmetrix system is reached. When the skew value is reached. RAID 5(3+1). Parity RAID (3+1). ! CAUTION Adaptive copy disk mode should not be used if the primary volumes are not protected by RAID 1. A background process destages the write tasks to the corresponding secondary volume. while less global memory is consumed it is typically slower to read data from disk than from global memory. because it is slower to read data from disk than global memory. device resynchronization time will increase. The advantages and disadvantages of this mode are opposite from those of the adaptive copy write-pending mode. regardless of whether the maximum skew value specified for each device is reached. Secondary modes of operation 57 . whichever is currently specified. RAID 10. The advantage to this mode is that it is faster to read data from global memory than from disk.

SRDF Technical Concepts

Additional SRDF modes and attributes
There are several SRDF modes and attributes that affect SRDF behavior under certain conditions. These attributes are described in the following sections.

Domino modes

Domino modes effectively stop all write operations to both primary and secondary volumes if all mirrors of a primary or secondary device fail, or if all SRDF links in a link group become unavailable. While such a shutdown temporarily halts production processing, domino modes can prevent data integrity exposure caused by rolling disasters. There are two types of domino modes:
◆ ◆

Device domino mode Link domino mode

Device domino mode

You can set device domino mode at the device level on primary volumes. If this mode is set to Yes on a primary volume, and the secondary volume becomes unavailable to its primary volume for any reason, the primary volume becomes unavailable to its host. You can set link domino mode at the SRDF group level at either side of the SRDF links. If this mode is set to Yes for an SRDF group, and the last remaining link in the SRDF group fails, all primary (source) volumes in the SRDF group become unavailable (not ready) to their host. Once the not ready condition is set, you must re-enable the volumes using EMC host-based software.

Link domino mode

!

CAUTION With either domino mode, the appropriate primary volumes are made not ready and all related applications stop. This is an extreme measure. A more moderate measure (if you are using SRDF in a mainframe environment, or an open systems environment with EMC PowerPath® software) is to implement consistency groups (go to “SRDF/Consistency Groups (SRDF/CG)” on page 62 for more information).

58

EMC Symmetrix Remote Data Facility (SRDF) Product Guide

SRDF Technical Concepts

Invalid tracks attribute

The invalid tracks attribute can only be set on secondary SRDF volumes. When the invalid tracks attribute is enabled, SRDF makes the device not ready to the host if you attempt to access data from that secondary volume when it is not synchronized with its primary volume. The purpose of the invalid tracks attribute is to inform the user that the data on the secondary devices may not be suitable for use in disaster recovery situations. It is a user decision to proceed using data from devices that were made not ready by the invalid tracks attribute. In such cases another form of data recovery may be more appropriate. If a user decides to use the existing data on the secondary volume, and the not-ready condition has been set by this attribute, the not-ready condition can be reset by host-based SRDF control software.

SRDF system-level attributes

You can set system-level attributes to do the following:

Force RAs offline after powerup — This attribute applies to

all SRDF director types and forces SRDF adapter ports offline following a power outage. This attribute can be used to prevent a data integrity exposure due to a rolling disaster following a power outage.

Prevent automatic link recovery after all links fail

— This attribute prevents SRDF device pairs from automatically resuming their SRDF relationship following a failure of all SRDF links in an SRDF group. This attribute can provide the opportunity to preserve a consistent copy of data on the secondary devices (using Business Continuance Volumes) by ensuring that device level resynchronization does not occur automatically following link restoration.
Note: Prior to Enginuity level 5568, the Prevent Automatic Recovery attribute is a system level attribute. At Enginuity level 5568 and higher, the Prevent Automatic Recovery attribute is an SRDF group-level attribute.

Additional SRDF modes and attributes

59

SRDF Technical Concepts

Concurrent SRDF
Enginuity level 5567 and later supports the ability for a single primary volume to be remotely mirrored to two secondary volumes concurrently. This feature is called Concurrent SRDF and is supported in ESCON, Fibre Channel, and Gigabit Ethernet SRDF configurations. Concurrent SRDF requires that each remote mirror operate in the same primary mode, either both synchronous or both semi-synchronous, but allows either (or both) volumes to be placed into one of the adaptive copy modes. Figure 15 on page 60 shows a concurrent SRDF configuration in which the primary volume is communicating with one secondary volume in synchronous mode. Concurrently, the same primary volume is communicating with its other secondary volume in one of the adaptive copy modes (adaptive copy write-pending mode or adaptive copy disk mode). Any combination of synchronous/semi-synchronous and adaptive copy is allowed with the exception of one volume operating in synchronous mode and the other operating in semi-synchronous mode.
Symmetrix DMX

Symmetrix DMX

Synchronous

Secondary

Primary Symmetrix 8000
Adaptive Copy

Secondary

Figure 15

Concurrent SRDF configuration

60

EMC Symmetrix Remote Data Facility (SRDF) Product Guide

ending status for an I/O is not presented until the remote Symmetrix system acknowledges receipt of the I/O to the primary Symmetrix system. ending status is presented to the host when the synchronous volume acknowledges receipt of the I/O.SRDF Technical Concepts Normal operating rules for SRDF also apply to concurrent SRDF configurations. Concurrent SRDF 61 . ending status is not presented until both volumes acknowledge receipt of the I/O. If one remote mirror is in synchronous mode and one remote mirror is in adaptive copy mode. If both secondary volumes are operating in synchronous mode. When operating in synchronous mode.

SRDF Technical Concepts SRDF/Consistency Groups (SRDF/CG) SRDF/CG ensures the consistency of the data remotely copied by SRDF/S operations in the event of a rolling disaster. How a consistency group works Assume that you have an SRDF configuration in which three Symmetrix systems contain primary (source) devices. Primary 1 Primary 2 Primary 3 Secondary 1 Secondary 2 Figure 16 Primary and Secondary relationships 62 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The units with primary devices send data to the units with secondary devices as shown in Figure 16 on page 62. An SRDF consistency group is a group of Symmetrix devices specially configured to act in unison to maintain the integrity of a database distributed across multiple Symmetrix systems controlled by a mainframe host software or open systems host software using EMC PowerPath software. Note: Another way to ensure the integrity of a remote database is to use domino mode (go to “Domino modes” on page 58). and two additional Symmetrix systems contain secondary (target) devices.

SRDF/Consistency Groups (SRDF/CG) 63 . and 3. However. the consistency group automatically stops Primaries 1 and 3 from sending data to Secondaries 1 and 2. Primary 1 Primary 2 Primary 3 Secondary 1 Secondary 2 Figure 17 Failed link between Primary 2 and Target 1 The result is that the copy of the data spread across Secondaries 1 and 2 becomes inconsistent. Without a consistency group. if Primaries 1. assume that the links between Primary 2 and Secondary 1 fail. and the link between Primary 2 and Secondary 1 fails. the dependent-write consistency of the data (spanning Secondaries 1 and 2) remains intact. Primaries 1 and 3 continue to write data to the Secondaries 1 and 2 while Primary 2 does not as shown in Figure 17 on page 63. Thus. belong to a consistency group. 2.SRDF Technical Concepts Next. as shown in Figure 19 on page 64. as shown in Figure 18 on page 64.

and data transfer from the primary devices to the secondary devices resumes. they are propagated after the affected links are again operational. Such updates are not immediately sent to the remote side. However. 64 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .SRDF Technical Concepts Consistency Group Primary 1 Primary 2 Primary 3 Secondary 1 Secondary 2 Figure 18 Primaries 1. 2. and 3 in a consistency group Consistency Group Primary 1 Primary 2 Primary 3 Secondary 1 Secondary 2 Figure 19 Failed link between Primary 2 and Target 1 Continuous processing I/O to the primary devices in the consistency group can still occur even when the devices are not ready on the SRDF links.

SRDF Technical Concepts Technical considerations For more information about consistency groups and the consistency group utility. go to: ◆ ◆ EMC Consistency Group for z/OS Product Guide EMC SRDF Family Product Guide SRDF/Consistency Groups (SRDF/CG) 65 .

SRDF Technical Concepts 66 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

............... Migrating data from R1 to a larger R2 device.......... 68 69 71 73 75 79 80 81 SRDF Operations 67 ............................... Business continuance using SRDF and TimeFinder ..................................................................................................................... R1/R2 swap ............................................................................................. Recovery operations ............................................................................................................................................ Read operations......Invisible Body Tag 3 SRDF Operations This chapter describes basic SRDF operations and covers the following topics: ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ Write operations .......................................... Business continuance using SRDF................... Dynamic R1/R2 swap ............................

Instead. data flows in two directions over the same SRDF link. In synchronous or semi-synchronous mode. This is due to the nature of ESCON protocol and the channel-to-control unit architecture of ESCON-based SRDF. Write operations in a unidirectional or dual-directional configuration This process applies to write operations over links that use a unidirectional or dual-directional protocol. normally the receiving end of the SRDF link. Write operations in an ESCON bidirectional configuration 68 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . When the RA-1 reads the data from the RA-2 volume. in which data moves in only one direction over a given link.SRDF Operations Write operations The write task is the most common SRDF/S operation. This section describes the write operations for unidirectional. When it receives the I/O. the write operation from RA-2 to RA-1 is effectively accomplished. the RA-2 must ask the RA-1 to read the data from a primary volume serviced by the RA-2. When an RA-2. and bidirectional configurations. The I/O then moves across the SRDF links to the remote Symmetrix system. normally the sending end of the SRDF link. the RA-2 cannot simply transmit the data over the SRDF link to the RA-1. must send write data to its corresponding RA-1. the local host sends a write I/O. dual-directional. In a bidirectional protocol. the remote Symmetrix system returns an acknowledgement to the local Symmetrix system.

in which the primary host reads data from a secondary volume in the remote Symmetrix system. Several events can cause such a read operation to take place: ◆ Read operations from the primary host If data is not in global memory and all of the primary (source) devices are in a not-ready state.SRDF Operations Read operations Primary volume read operations Secondary volume read operations Read operations from a primary volume behave as normal read operations and are not impacted in any way by SRDF. the local Symmetrix system requests the data from the remote Symmetrix system. This method is faster than accessing the data from the RAID rebuild. Note: If a data availability problem is caused by a CRC error. This process can eliminate the need to read the data from the remote Symmetrix system. In these cases. Read operations 69 . ◆ ◆ ◆ In a read operation. an invalid track. The Symmetrix system always performs a full-track read for count-key-data (CKD) and at least a contiguous-block read for fixed block architecture (FBA) data. Read operations from a secondary volume can be initiated in response to read I/Os issued by the host attached to the primary Symmetrix system or by the host attached to the secondary Symmetrix system. If a primary device is in a ready state but the requested track is invalid on the local device. Read operations. the local Symmetrix system requests the track data from the remote Symmetrix system. If a disk adapter has a problem accessing a primary device as in the case of a drive timeout or cyclic redundancy check (CRC) error. are performed only to recover from a local data availability problem. data can be destaged (moved from global memory to disk) within the local Symmetrix system if it is in global memory. or a RAID rebuild. If a track on a primary device is currently available only from a RAID rebuild. the remote Symmetrix system reads data from the secondary device and sends the data across the link to the local Symmetrix system.

SRDF Operations Read operations from the secondary host SRDF secondary volumes can optionally be made read-only while SRDF is in operation. A host attached to the secondary Symmetrix system may read data from the secondary devices provided the operating system has support for reading data from devices that are hardware write-protected. it does not interfere with SRDF operations. 70 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Such support varies by operating system type. SRDF ensures that the most current image of data is made available to the secondary host. Consult your server operating-system vendor. Note: If the secondary host has read-only access to the secondary device.

2. The failover process involves changing the states of the secondary volumes to read/write and transferring production processing to the secondary host. When the failure occurs. If a power on or IML is required of the primary Symmetrix system. the remote Symmetrix system can be ready for operations in minutes. The following steps are required to transfer processing from the secondary host back to the primary host: 1. Halt processing on the secondary host and change the state of the secondary devices to read-only or Host Not Ready. You can use EMC software products such as EMC ControlCenter. Failover to the secondary Symmetrix system If a remote host is to be activated to continue production processing. production processing can resume on the primary host. Processing continues on the remote host until the local host and local Symmetrix system are operational again and a failback is performed. For adequate protection against logical data corruption. EMC recommends that you regularly back up your data. you must initiate a failover process. Solutions Enabler. Recovery operations 71 . After the failover has been initiated. and the primary volumes are operating in synchronous mode.SRDF Operations Recovery operations ! CAUTION This section explains recovery in the context of hardware failure. If the local host or local Symmetrix system fails. updates made to the secondary volumes appear as invalid tracks owed to the primary volumes. Failback to the primary Symmetrix system After the primary host and the Symmetrix system containing the primary volumes are again operational. or the SRDF Host Component for z/OS to control the failover process. make sure that the SRDF links are physically disabled to prevent movement of invalid track metadata across the SRDF links. the primary and secondary volumes are synchronized within one I/O and there are no tracks owed to either the primary or secondary side.

there might be a large number of invalid tracks (for example. Solutions Enabler. 500 GB) owed to the primary volumes. effectively resynchronizing the primary and secondary volumes. Use EMC software products such as EMC ControlCenter. Solutions Enabler. If the SRDF links were physically disabled in the previous step. you can resynchronize the primary and secondary volumes while the secondary host continues production processing. 5. Bring the SRDF links online and restart the local host. or the Stored Procedure Executive (SPE) of SRDF Host Component for z/OS to automate or semi-automate this process. re-enable the links. or the SRDF Host Component for z/OS to send invalid track metadata information from the secondary volumes to the primary volumes. 4. 72 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . 50 GB). Bring the Symmetrix system on which the primary volumes reside to a ready state. you can shut down the secondary host and restart the primary host.SRDF Operations 3. The primary volumes automatically receive the appropriate data from the secondary volumes. Then. Note: You can use EMC software products such as EMC ControlCenter. In this case. when there is a relatively small number of invalid tracks on the primary volumes (for example. Note: This capability is enabled by using the Solutions Enabler or the SRDF Host Component. Recovery for a large number of invalid tracks If the recovery site (the secondary host and the Symmetrix system containing the secondary volumes) has handled production processing for a long period of time.

for example. Remote processing — For example. Decision Support System (DSS) operations.SRDF Operations Business continuance using SRDF Business continuance refers to practices that enable you to achieve nearly nonstop. This enables you. ◆ ◆ ◆ Concurrent operations You can temporarily suspend the SRDF links so that you can read and write data on both the primary and secondary volumes concurrently. SRDF enables business continuance by allowing you to suspend remote mirroring and temporarily enable the secondary volumes for read/write activity. Symmetrix 1 Symmetrix 2 Production Host R1 SRDF LInk R2 Business Continuance Host Remote Mirroring Temporarily Suspended and Target Volume (R2) Enabled for Read/Write Activities Figure 20 SRDF business continuance The following business continuance practices are possible with SRDF: ◆ Backups — Avoid taking system offline while nightly backups run. 24x7 business operations. to run backups on the secondary volumes while production processing continues on the primary volumes (a business continuance practice). This last operation propagates any updates made to the primary volumes while the links were suspended and overwrites any changed data on the secondary volumes. some systems must produce daily billing statements. effectively creating two separate host/storage systems (Figure 20 on page 73). Application testing. Business continuance using SRDF 73 . bringing both volumes into a synchronized state (Figure 21 on page 74). You can then resume the links and copy data from the primary volumes to the secondary volumes. These statements can be produced at a remote site without interrupting normal processing.

the testing was successful. the links between primary and secondary are reestablished.SRDF Operations R1 Primary Data After completion of concurrent operations. you performed application testing on the secondary volumes. production processing was halted on the primary volumes. for example. you can resynchronize data in the opposite direction. R1 Primary Data During an R2-to-R1 resynchronizing operation. Updates R2 Secondary Data Figure 21 Primary-to-secondary resynchronization Alternatively. This operation is useful if. and you want to keep the updates. data is copied from the secondary volumes to the primary volumes. Updates R2 Secondary Data Figure 22 Secondary-to-primary resynchronization 74 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . and updates are propagated from the primary volumes to the secondary volumes. from secondary to primary.

Using TimeFinder/Mirror BCVs with primary devices TimeFinder operations (establish. TimeFinder BCVs can be locally mirrored and/or remotely mirrored via SRDF. SRDF. Note: The SRDF/Automated Replication solution (introduced on page 19) combines host-based automation with TimeFinder and SRDF functionality. pending the resumption of the BCV R1 to R2 device relationship (Figure 23 on page 76). Restore operations to a primary device (R1) will result in the restored data being propagated across the SRDF link if the remote device is not in a suspended state. when used with EMC TimeFinder software. dynamically assignable mirror volumes called business continuance volumes (BCVs). data transmission can be suppressed and the BCV R1 device will mark the changed data as invalid (owed) to the secondary device. split.com. or contact your EMC representative. A BCV device may also function as an SRDF primary device. TimeFinder. The sections that follow describe how SRDF can be used with TimeFinder for business continuance operations. described below. the default behavior is to propagate the changed data to the secondary device.SRDF Operations Business continuance using SRDF and TimeFinder The Symmetrix TimeFinder product offering uses one or more software-controllable. Optionally. Note: For information about EMC TimeFinder. A BCV R1 device may optionally be locally mirrored.EMC. allows you to perform business continuance operations without temporarily suspending remote mirroring. Such a device is called a BCV R1 device. provides an automated long distance disaster restart solution. When a BCV R1 is attached to a standard device. If the remote device is in a suspended state. restored data will be marked as invalid (owed) to the secondary device. Document Library. When the BCV is split. Business continuance using SRDF and TimeFinder 75 Using a BCV as a primary (source) device . Software. then follow these links: Support. the corresponding secondary device is suspended. This functionality. Concurrent SRDF is not supported for BCV R1 devices because a BCV R1 device is not allowed to have two remote mirrors. go to the documentation (available on the EMC website: http://Powerlink. restore) are allowed between a BCV and a primary device.

the corresponding primary device is suspended. Data is mirrored to the R2 device on the Symmetrix at Site B. 76 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Please go to the SRDF Host Component or Solution Enabler documentation for information on using BCV R2 devices. By default. restore operations will not result in data propagating back to the primary device. it is again available for SRDF operations. 3. split. TimeFinder software options are available to enable this function. The BCV device is split. Figure 23 SRDF single-hop configuration (BCV functioning as a primary SRDF device) Using a BCV as a secondary (target) device A BCV device may also function as an SRDF secondary device. Such a device is called a BCV R2 device. When a BCV R2 is attached to a standard device. Note: Host software may restrict functionality of BCV R2 devices. Standard Device Host X SRDF Link 1 2 3 R2 R1 BCV Symmetrix Site A 1. Symmetrix Site B 2. restore) are allowed between a BCV and a secondary device.SRDF Operations Using TimeFinder BCVs with secondary devices TimeFinder operations (establish. If data propagation back to the primary device is desired. The R2 on the Symmetrix at Site B is resynchronized with the R1 BCV at Site A. When the BCV R2 is split. Changed data will be marked as invalid (owed) to the primary device following a restore operation. A BCV R2 device may optionally be locally mirrored.

SRDF Operations

SRDF remote command support

You can issue SRDF and BCV control commands from a host or server connected to the local Symmetrix system and have those commands transmitted across the SRDF link. A host or server does not have to be located at the secondary location, allowing you for example, to create a dependant write consistent point-in time copy of your data at the remote site without needing a host computer or server at the remote site. SRDF’s multi-hop capability creates the ability to provide disaster recovery protection at great distances without any data loss. A middle or bunker site transitions the synchronous replication solution by using TimeFinder in conjunction with SRDF. SRDF multi-hop uses a BCV R1 on the secondary side of an SRDF synchronous configuration to perform an incremental synchronization with a secondary volume on a third Symmetrix system. When split, the BCV R1 copies only changed tracks to the secondary volume located in the third Symmetrix system. This process provides two benefits. It eliminates the additional performance impact on synchronous operations across great distances. In addition, the differential resynchronization dramatically reduces bandwidth requirements.
Note: Although the local side of an SRDF multi-hop configuration requires an SRDF license with the ability to do synchronous or semi-synchronous operations, you can use SRDF - Data Mobility (in adaptive copy mode) on Symmetrix systems connected to the SRDF extended distance configuration. This requires an initial full synchronization to establish the relationship.

SRDF Multi-Hop

Figure 24 on page 78 illustrates SRDF multi-hop using ESCON connectivity however, any type of SRDF connection is supported by multi-hop.

Business continuance using SRDF and TimeFinder

77

SRDF Operations

Production Site

Recovery Site

Synchronous Campus Fiber RA-1 RA-2 RA-1

SRDF Extended Distance

Any Supported MAN or WAN

RA-2

Production Data Mirrored to Remote Symmetrix
Production R1 Data

Production R2 Data

BCV Production Data

Differential Synchronization

Copy Production Data

BCV/R1 Re-Establish/Differential Split

R2

Figure 24

SRDF multi-hop configuration

78

EMC Symmetrix Remote Data Facility (SRDF) Product Guide

SRDF Operations

R1/R2 swap
Dynamic SRDF provides the capability of swapping the R1/R2 personality of an SRDF device pair. During a personality swap, primary R1 devices become secondary R2 devices. The secondary R2 devices become primary R1 devices. Swapping personalities allows the R2 side to take over operations while retaining a remote mirror on the R1 side. Swapping is especially useful after failing over an application from the R1 side to the R2 side. Before performing an R1/R2 swap, the SRDF pair must be in a suspended state. After the swap operation is complete, the SRDF device pair remains in the suspended state. Incremental or full resynchronization capability is maintained and controlled through EMC SRDF host based control software. For specific procedures for controlling the R1/R2 swap operation, go to the SRDF Host Component for z/OS Product Guide or the Solutions Enabler SRDF CLI Product Guide.

R1/R2 swap procedure history

When introduced in Enginuity 5x65 and 5x66, the R1/R2 swap feature used an online configuration change procedure to affect the personality swap of the primary (source, R1) and secondary (target, R2) devices. This process was often lengthy, taking several minutes to complete. Beginning with Enginuity 5567, a new capability called Dynamic SRDF was introduced, which dramatically improved the performance of R1/R2 personality swaps (Figure 25 on page 79).
Initial View After R1/R2 Swap

R1 R1 SRDF Primary (Source)

R2 R2 SRDF Secondary (Target)

R2 R1 SRDF Primary (Source)

R1 R2 SRDF Secondary (Target)

Figure 25

R1/R2 swap concept

R1/R2 swap

79

Host software can then be used to create Dynamic SRDF primary and secondary device relationships. Primary (source) R1 device(s) become secondary (target) R2 device(s) and secondary (target) R2 device(s) become primary (source) R1 device(s). Dynamic swap is not supported under the following conditions: ◆ If Enginuity Versions 5567 and 5568 are mixed between your local and remote Symmetrix system In Enginuity Version 5669 or higher configurations where the R2 device is larger than the R1 device If the devices to be swapped are participating in an active SRDF/A session When using ESCON RA directors configured with FarPoint For Parity RAID (3+1) and (7+1) protected devices Note: For more detail regarding specific platform requirements and functionality for dynamic swap. Dynamic R1/R2 swap is faster than the static swap.SRDF Operations Dynamic R1/R2 swap Dynamic R1/R2 swap is available with Enginuity Version 5567 or higher. Host software can also be used to swap the SRDF personalities of the SRDF devices. ◆ ◆ ◆ ◆ 80 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . go to either the EMC Solutions Enabler Symmetrix SRDF CLI Product Guide or the Symmetrix SRDF Host Component for z/OS Product Guide. Swaps using Dynamic SRDF can be accomplished faster but must also be enabled in your Symmetrix system configuration to use this feature.

Certain SRDF operations are blocked when the R2 device is larger than the R1 device: ◆ ◆ Swap operations are not allowed.SRDF Operations Migrating data from R1 to a larger R2 device Enginuity Version 5669 provides a temporary method for migrating data from an R1 device to a larger R2 device. once the host has recognized the additional capacity cannot be restored back to the R1 device. Striped metadevices are supported with restrictions. Data migrated to a larger R2 device. ◆ ◆ ◆ Migrating data from R1 to a larger R2 device 81 . Mainframe meta devices are supported. Concatenated metadevices are not supported. Data migration to a larger R2 device is only allowed on Symmetrix DMX hardware running Enginuity 5669 or higher.

SRDF Operations 82 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

............................................................................................................. 86 SRDF/A history .......................... 85 Requirements and limitations ........... 95 SRDF/A single session mode delta set switching......................................... 91 SRDF/A single session mode...........Invisible Body Tag 4 SRDF/Asynchronous Operations This chapter provides a technical overview of SRDF/Asynchronous (SRDF/A).................... 102 SRDF/A single session cleanup process................................................................................................................... 122 SRDF/Asynchronous Operations 83 ........................................................................................... 87 Tolerance mode ........ 112 SRDF/A MSC mode delta set switching ................................. The following topics are covered: ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ ◆ SRDF/A overview ........................................... 111 SRDF/A MSC mode dependent-write consistency .. 84 SRDF/A benefits ..... 97 SRDF/A single session mode state transitions........................................ 116 SRDF/A MSC session cleanup process ................... 93 SRDF/A single session mode states........ 108 SRDF/A multi-session consistency (MSC) mode .............................................. 92 SRDF/A single session mode dependent-write consistency................................................................................................................................. 107 SRDF/A single session mode recovery scenarios ............. 90 Locality of reference....................................

Data is transferred to the secondary Symmetrix system in cycles. This solution is intended for customers requiring the ability to preserve dependent-write consistency within and across the database and application environment at an extended distance secondary site with minimal host application impact. This chapter explains how the dependent-write consistency is maintained during the SRDF/A single session mode and SRDF/A Multi Session Consistency (MSC) mode cycle switches.SRDF/Asynchronous Operations SRDF/A overview SRDF/A provides a long distance disaster restart solution with minimal impact on performance. using delta sets. 84 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

com. point-to-point and switched fabric Fibre Channel. such as switches or routers Supports hosts listed in the EMC Support Matrix for CKD and FBA data emulation types Note: Note: For more information on supported hosts. and GigE) Requires no additional hardware. or contact you EMC Sales Representative ◆ ◆ ◆ ◆ ◆ ◆ ◆ Imposes minimal impact on the back-end disk directors Provides a performance response time equivalent to writing to non-SRDF devices Allows failover and failback capability between the R1 and the R2 ◆ SRDF/A benefits 85 .SRDF/Asynchronous Operations SRDF/A benefits SRDF/A provides the following features and benefits: ◆ Supports extended data replication with database and application consistency Promotes efficient link utilization resulting in lower link bandwidth requirements Maintains a dependent-write consistent point in time image on the secondary (R2) devices Supports all current SRDF topologies (ESCON.com or http://Powerlink. FarPoint. go to the EMC Support Matrix at http://EMC.EMC.

consult the EMC Solutions Enabler Symmetrix SRDF CLI Product Guide and the Symmetrix SRDF Host Component Product Guide. special considerations must be followed.SRDF/Asynchronous Operations Requirements and limitations Known requirements and limitations for this release of SRDF/A are as follows: ◆ When using either TimeFinder/Clone™ BCVs or TimeFinder/Snap™ BCVs. ◆ ◆ Note: For more information on specific restrictions. Consult the EMC Solution Enabler TimeFinder CLI Product Guide or the SRDF Host Component for z/OS Product Guide. SRDF/A is supported for open systems through Solutions Enabler.2 and higher. 86 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . SRDF/AR (previously known as SAR) does not support SRDF/A devices. mainframe interface is supported through SRDF Host Component V5.

By doing this. Implicit in this feature is the ability to define a primary for one SRDF/A group and a secondary to another SRDF/A group in the same Symmetrix DMX. SRDF/A history 87 . Enginuity 5671 SRDF/A Multi-Session Consistency.SRDF/Asynchronous Operations SRDF/A history The following sections describe the highlights of SRDF/A development and their availability. Note: Bidirectional operation within a single SRDF/A group is not supported.50 initiated the use of SRDF/A Multi-Session Consistency. MSC Enginuity 5671 Multiple SRDF/A Single Session SRDF groups per Symmetrix Multiple SRDF/A groups (up to 64 depending on the configuration) allowed per Symmetrix array.50 and higher for mainframe SRDF/A. MSC. This configuration allows a single SRDF group to participate in asynchronous mode within a single Symmetrix. This feature includes the addition of open systems support and eases the existing restrictions implemented in Enginuity 5670. Enginuity 5670 SRDF/A single session Enginuity 5670 supported single session mode SRDF/A. This SRDF/A group can not use Dynamic SRDF or participate in Concurrent SRDF operations. This feature provides for a special mode of SRDF/A operation where cycle switching is controlled by the host application via a Symmetrix system call interface. for mainframe. you now have the ability to use SRDF/A with bidirectional operations. Enginuity 5670. All source-to-target data flow operations within a SRDF/A group still remain unidirectional. and could be used to provide dependent-mainframe across several Symmetrix systems. for mainframe and Open Systems supports multiple Symmetrix with no restrictions to the number of SRDF/A groups per Symmetrix operating in SRDF/A MSC mode.50 SRDF/A Multi-Session Consistency. The initial support is limited to a single SRDF/A group allowed per Symmetrix but multiple Symmetrix systems can participate in a SRDF/A MSC configuration. MSC. MSC SRDF/A Multi-Session Consistency. Enginuity 5670.

otherwise SRDF/A drops. To remove a Dynamic SRDF/A group. The same primary device can be replicated synchronously using SRDF/S mode through one SRDF link and asynchronously using SRDF/A mode through another link. Enginuity 5671 Concurrent SRDF support The feature provides the ability to replicate a group of devices in Synchronous mode to one secondary site and in Asynchronous mode to another secondary extended distance site. the mirror must be made not ready and all outstanding I/Os in that session for that device must be drained or discarded. This feature enables limited multisite protection capability. there would be controlled data loss. The data loss is at most two times the SRDF/A cycle time. 88 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . In the event of a regional disruption that affects the primary site and the synchronous site. EMC has developed SRDF/Star (go to the next section. as well as adding and removing devices from SRDF/A groups. the group must be empty of devices and the SRDF/A session must be inactive and drained. The data has been simultaneously replicated asynchronously to a secondary site much further away. A concurrent mirror may be added dynamically to a device that is in an SRDF/A group subject to the normal restrictions for dynamic concurrent SRDF. only one concurrent mirror per device is allowed to be in an SRDF/A group. In the event of a primary site failure. The environment is still dependent-write consistent and restartable but there may be some data loss.SRDF/Asynchronous Operations This feature may also include new functionality to facilitate interhost (mainframe and open Systems) communications in a mixed. multihost control environment. Adding or removing a device from an SRDF/A group requires the use of the SRDF/A tolerance mode feature before actually adding and/or removing the mirror. Enginuity 5671 Dynamic SRDF support This feature allows adding and removing Dynamic SRDF groups that participate in SRDF/A. there would be no data loss since it is synchronously replicated to a local/regional secondary site. establishing remote replication between the remaining sites would mean configuration changes and a full synchronization process. Performance is equivalent to a conventional Concurrent SRDF/S configuration. the addition of the SRDF/A leg does not impose additional performance restrictions. However. SRDF/Star). To remove a dynamic mirror from an SRDF/A session. To avoid the limitation. The limitation to the solution is that in the event of a loss of the primary site.

to set the cache usage limit to a percentage of the system write pending limit. SRDF/A history 89 . Previously. SRDF/A Delta Set Extension or DSE enables asynchronous replication operations to remain active in the event system cache resources are becoming in danger of reaching the system write pending or SRDF/A max cache limit. which SRDF/A sessions to drop first. or even temporary network outages. at which point you could choose to throttle the host for a given amount of time or to drop SRDF/A immediately. additional resource allocation can be applied to address temporary workload peaks. Without this feature. With SRDF/A Reserve Capacity functions enabled. SRDF/A Reserve Capacity SRDF/A Reserve Capacity enhances SRDF/A's ability to maintain operational state when encountering network resource shortfalls that would have previously suspended SRDF/A operations. The time SRDF/A remains active will depend on the system not reaching the system write pending limit or SRDF/A max cache limit. not just the SRDF/A application. into preconfigured storage pools. This functionality is achieved by offloading some or all of the active Delta Set data. This feature also adds the ability for a user to define.SRDF/Asynchronous Operations Enginuity 5671 Tunable Cache utilization This host software configurable feature allows the individual SRDF/A processes. single session or MSC. and they work together to maximize availability of continuous remote replication operations while minimizing operational overhead. that needs to be transmitted to the target site. The two functions that implement SRDF/A Reserve Capacity are Transmit Idle and Delta Set Extension (DSE). the user can effectively assign a priority to sessions keeping SRDF/A active upon for as long as cache resources allows. periods of network congestion. In so doing. SRDF/A Transmit Idle enables asynchronous replication operations to remain active in the event all links are lost temporarily due to network outages. if cache resources are being stressed. SRDF/A cycles could grow until they reach the Symmetrix system write pending limit. which is user setable. if system write pending limits are being exceeded. performance suffers across the entire Symmetrix. SRDF/A Transmit Idle and Delta Set Extension have the ability to work together to improve the overall resiliency of SRDF/A during workload and network resource imbalances.

These conditions could include making the secondary Symmetrix R2 devices Read/Write. but note that if it is set on for any SRDF/A group within the MSC configuration. When tolerance mode is set to on. SRDF/A MSC still allows tolerance mode to be turned on. For more information on enabling/disabling consistency go to the EMC Solutions Enabler Symmetrix SRDF CLI Product Guide. the result could be inconsistencies for the entire MSC group. The host software to implement tolerance mode is different for mainframe and Open Systems. 90 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .SRDF/Asynchronous Operations Tolerance mode Tolerance mode allows certain conditions to occur that would normally drop SRDF/A. The mainframe software exports the use of tolerance mode directly. Note: For more information on Tolerance mode go to the Symmetrix SRDF Host Component for z/OS Product Guide. where the Open Systems software externalizes it through consistency enabling/disabling. the dependent-write consistency is NOT guaranteed.

7 blocks Asynchronous mode has: Less bandwidth (7 blocks vs.e. and thus reduces the processing overhead per host I/O. However. (i. The main difference has to do with the fact that I/Os sent on the link are usually the same size as the host I/Os. the systems send the data across the SRDF links only once. This is a major advantage over competitive asynchronous replication solutions. Figure 26 on page 91 shows both the locality of reference and the concatenation of small blocks to one larger I/O for transmission. the gain in bandwidth efficiency from the locality of reference is mainly from rewriting to the same block and not rewriting to the same track. there are many cases where the system combines the original I/Os and sends them as one large I/O across the link. The advantage gained from the locality of reference on the SRDF link is not necessarily the same as the advantage gained in cache memory. 10 blocks Asynchronous mode: 3 I/Os. as opposed to SRDF Adaptive Copy Disk mode where the system always sends full tracks. These asynchronous solutions consume as much bandwidth as an synchronous solution. every write is sent across the link and the locality of reference is not utilized at all. Even if this operation does not necessarily decrease the bandwidth. In such a case. In competitive solutions of this type. which must (by definition) send every I/O across the links. repeated writes) in the same cycle. it does decrease the number of IO/s the RA handles. 10) Less SRDF overhead (3 I/Os vs. Even if there are multiple data updates. Applications tend to write data in proximity of time and place Track 0 Track 1 Track 2 Synchronous mode: 10 I/Os.SRDF/Asynchronous Operations Locality of reference Locality of reference in SRDF/A environments improves the efficiency of the SRDF network links. The rules for combining a number of small blocks to one larger I/O are complex. The logic is similar to SRDF Adaptive Copy Write Pending mode. and not discussed here. 10) SYM-001280 Figure 26 Synchronous and asynchronous block transfer comparison Locality of reference 91 .

between the primary Symmetrix and the secondary Symmetrix. one cycle at a time. 92 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . SRDF/A single session mode refers to the implementation of SRDF/A between one (1) primary Symmetrix using one (1) SRDF group to one (1) secondary Symmetrix using one (1) SRDF group. Symmetrix systems implement asynchronous mode host writes from the primary Symmetrix to the secondary Symmetrix using dependent-write consistent delta sets transferred in cycles. Each delta set contains groups of I/Os for processing. which are managed for dependent-write consistency by the Enginuity operating environment.SRDF/Asynchronous Operations SRDF/A single session mode Different from traditional ordered write asynchronous approaches. SRDF/A transfers these sets of data using cycles of operation.

The active cycle on the secondary Symmetrix contains the N-2 data version in the apply delta set. Figure 27 on page 93 illustrates the delta sets and their relationships. The active cycle on the primary Symmetrix contains the current host writes or N data version in the capture delta set. 2. 3. The primary inactive cycle is the transmit delta set and the secondary Symmetrix inactive cycle is the receive delta set. The inactive cycle contains the N-1 data version that is transferred via SRDF/A from the primary Symmetrix to the secondary Symmetrix. This is the guaranteed dependent-write consistent image in the event of a disaster or failure. Enginuity controls the cycle switching without any host software involvement.SRDF/Asynchronous Operations SRDF/A single session mode dependent-write consistency SRDF/A single session mode is the implementation of SRDF/A from a single SRDF group on the primary Symmetrix to a single SRDF group on the secondary Symmetrix. Dependent-write consistency is achieved through the processing of ordered SRDF/A delta sets (cycles) between the primary Symmetrix and the secondary Symmetrix. 1. Multiple instances of SRDF/A single session mode operation between Symmetrix systems are available with Enginuity 5x71 or higher. Primary Symmetrix Secondary Symmetrix R1 Capture N Transmit N-1 Apply N-2 Receive N-1 R2 R1 R2 Capture “Active” cycle Transmit “Inactive” cycle Receive “Inactive” cycle Apply “Active” cycle SYM-001276 Figure 27 SRDF/A delta sets and their relationships SRDF/A single session mode dependent-write consistency 93 .

SRDF/Asynchronous Operations Dependent-write consistency is ensured within SRDF/A by the host adapter obtaining the active cycle number from a single location in global memory and assigning it to each I/O at the beginning of the I/O and retaining that cycle number even if a cycle switch occurs during the life of that I/O. two I/Os with a dependent relationship between them can either be in the same cycle. 94 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . or the dependent I/O can be in a subsequent cycle. even though it is not physically an atomic event across a range of volumes. This results in the cycle switch process being atomic for dependent-write sequences. As a result.

This means that all the remote devices on the primary Symmetrix are not-ready on the SRDF link. or Adaptive Copy Write Pending/Disk Mode). semi-synchronous. Inactive state SRDF/A single session mode states 95 . or adaptive copy) SYM-001277 Figure 28 SRDF/A single session allowed state transitions Not Ready (NR) state (system startup) When the SRDF environment is configured. By doing so. In inactive state the devices are ready on the link. SRDF/A is inactive. the state would transition to the inactive SRDF/A state. and the SRDF links come up. semisynchronous. all SRDF volumes are in a not ready (NR) state by default.SRDF/Asynchronous Operations SRDF/A single session mode states Figure 28 on page 95 shows the three logical states SRDF/A can be in: ◆ ◆ ◆ Not Ready (NR) Inactive Active NR (All devices are Not Ready on the links) Active Host commands or Enginuity Remote consistent or inconsistent (asynchronous) Host commands Host commands Inactive (synchronous. Most of these commands maintain a dependent-write consistent copy on the secondary Symmetrix and one of them (deactivate) does not. Various commands can transition SRDF/A from an active state to an inactive state. and all devices work in their assigned modes (synchronous. These devices can be made ready on the link by issuing the commands from the host software.

The secondary Symmetrix is either consistent or inconsistent. 96 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . SRDF/A declares the secondary Symmetrix consistent once all of the previously owed tracks from the primary Symmetrix have been transferred to the secondary Symmetrix devices.SRDF/Asynchronous Operations Active state The active state is the normal running state of SRDF/A. The consistent active state always represents a dependent-write consistent image of the data. Dependent-write consistency is not maintained for these owed tracks. the last cycle containing this data is fully copied to global memory and in the N-2 cycle (apply delta set) on the secondary Symmetrix. Specifically. The inconsistent active state represents previously owed tracks that have not yet been transferred to the secondary Symmetrix.

Minimum cycle time must be reached Figure 29 on page 97 displays the application write I/Os are being collected in the capture delta set on the primary Symmetrix. how the delta set switching works for SRDF/A single session mode. Once these conditions are satisfied. the primary Symmetrix sends a commit message to the secondary Symmetrix to begin the secondary Symmetrix cycle switch in unison with the primary Symmetrix cycle switch. Figures 29 through 38 assume that SRDF/A has been activated and two cycle switches have occurred previously. meaning all of the data has been transferred to the secondary Symmetrix. this is the N-1 copy. The previous cycles transmit delta set is completing the SRDF transfer to the receive delta set. SRDF/A single session mode delta set switching 97 .SRDF/Asynchronous Operations SRDF/A single session mode delta set switching This section examines in detail. 1 R1 N Apply N-2 Receive N-1 Capture N Transmit N-1 R2 N-2 1. Transmit delta set must have completed transferring the data to the secondary Symmetrix system 2. The secondary Symmetrix apply delta set is being restored (data marked write pending to the R2 devices). which is the N-2 copy. Before a primary Symmetrix cycle switch can occur two things must be achieved: 1. Capture delta set (DS) collects application write I/O R1 N Primary Symmetrix R2 N-2 Secondary Symmetrix SYM-001265 Figure 29 Single session capture delta set collects application write I/O Figure 30 on page 98 explains the primary Symmetrix waits for the minimum cycle time to elapse and the transmit delta set to empty.

Primary waits for the minimum cycle time. 1 R1 N Apply N-2 Receive N-1 Capture N R2 N-2 2c 2b R1 N Transmit N 2 R2 N-2 Secondary Symmetrix 1. 1 R1 N Apply N-2 Receive N-1 Capture N Transmit R2 N-2 R1 N 2b 2 R2 N-2 Secondary Symmetrix Primary Symmetrix 1. Primary waits for the minimum cycle time. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS SYM-001268 Primary Symmetrix Figure 32 SRDF/A single session Primary Symmetrix delta set switch 98 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . this is happening in unison with the secondary cycle switch discussed in Figure 34 on page 99. Capture delta set (DS) collects application write I/O 2. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) SYM-001266 Primary Symmetrix Figure 30 SRDF/A single session transmit delta set empties The SRDF transfer is halted prior to the primary Symmetrix cycle switch.SRDF/Asynchronous Operations 1 R1 N Apply N-2 Receive N-1 Capture N Transmit R2 N-2 R1 N 2 R2 N-2 Secondary Symmetrix 1. Again. as seen in Figure 31 on page 98. This is done automatically through Enginuity because it is a single session (single SRDF group) SRDF/A environment. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted SYM-001267 Figure 31 SRDF/A single session SRDF transfer is halted prior to Primary Symmetrix cycle switch Figure 32 on page 98 displays the primary Symmetrix cycle switch between the capture and transmit delta set. Primary waits for the minimum cycle time. Capture delta set (DS) collects application write I/O 2. Capture delta set (DS) collects application write I/O 2.

Secondary receives commit from Primary a) Check if the data in Apply DS is restored (data marked write pending to the R2 devices) SYM-001270 Figure 34 SRDF/A single session secondary Symmetrix wait for apply delta set to be restored SRDF/A single session mode delta set switching 99 . Secondary Symmetrix received the commit message from the primary Symmetrix (step 2a).SRDF/Asynchronous Operations The new capture delta set is available to continue receiving new Host I/O as seen in Figure 33 on page 99. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O 3. Capture delta set (DS) collects application write I/O 2. 1 R1 N R2 1. 2. Primary waits for the minimum cycle time. Apply delta set (N-2 copy) must complete its restore process (marking the data write pending to the R2 devices). the secondary Symmetrix verifies the apply delta set has been restored (data has been marked write pending to the R2 devices). 1 R1 N 3a R2 2d R1 N Capture N Apply Receive N-2 2c 2b Transmit N-1 2 R2 Secondary Symmetrix 2d Primary Symmetrix 1. Primary waits for the minimum cycle time. Capture delta set (DS) collects application write I/O 2. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O SYM-001269 2d R1 N Capture N Apply Receive N-2 2c 2b Transmit N-1 2 R2 Secondary Symmetrix 2d Primary Symmetrix Figure 33 SRDF/A single session new capture delta available for host I/O Before a secondary Symmetrix cycle switch can occur two things must be achieved: 1. Once the secondary Symmetrix receives the commit message from the primary Symmetrix. This occurs while the primary Symmetrix is performing the cycle switch between the capture and transmit delta sets. Figure 34 on page 99.

Primary waits for the minimum cycle time. 1 R1 N Apply N-2 3a R2 2d R1 N Capture N 2c 2b 3b Transmit N-1 Receive N-2 2 R2 Secondary Symmetrix 2d Primary Symmetrix 1. 1 R1 N Apply N-2 3a R2 2d R1 N Capture N 2c 2b 3b Transmit N-1 Receive 2 3c R2 2d Primary Symmetrix Secondary Symmetrix 1. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O 3. Capture delta set (DS) collects application write I/O 2. Primary waits for the minimum cycle time.SRDF/Asynchronous Operations The next step is a delta set cycle switch on the secondary Symmetrix between the receive (inactive) and apply (active) delta sets as shown in Figure 35 on page 100. This preserves the dependent-write consistent copy at the secondary Symmetrix prior to receiving the next dependent-write consistent copy. Secondary receives commit from Primary a) Check if the data in Apply DS is restored (data marked write pending to the R2 devices) b) Secondary cycle switch – Receive DS becomes Apply DS c) New Receive DS available for SRDF transfer SYM-001272 Figure 36 SRDF/A single session secondary Symmetrix new receive delta set is available for SRDF The secondary Symmetrix sends an acknowledgement to the primary Symmetrix at this point. The data in the apply delta set begins the restore process as shown in Figure 37 on page 101. 100 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Secondary receives commit from Primary a) Check if the data in Apply DS is restored (data marked write pending to the R2 devices) b) Secondary cycle switch – Receive DS becomes Apply DS SYM-001271 Figure 35 SRDF/A single session secondary Symmetrix delta set switch Figure 36 on page 100 shows a new receive delta set is available for the SRDF transfer. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O 3. Capture delta set (DS) collects application write I/O 2.

Primary waits for the minimum cycle time. Primary receives acknowledgement of Secondary cycle switch a) SRDF transfer begins SYM-001274 Figure 38 SRDF/A single session primary Symmetrix begins SRDF transfer SRDF/A single session mode delta set switching 101 . 1 R1 N 3a 3e Capture N Apply N-2 R2 N-2 2d R1 N 2c 3b Receive Transmit N-1 2b. Capture delta set (DS) collects application write I/O 2. Primary waits for the minimum cycle time. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O 3. 4a N-1 2 3c 2d Primary Symmetrix R2 N-2 Secondary Symmetrix 1. Secondary receives commit from Primary a) Check if the data in Apply DS is restored (data marked write pending to the R2 devices) b) Secondary cycle switch – Receive DS becomes Apply DS c) New Receive DS available for SRDF transfer d) Secondary sends Primary acknowledgement e) Begin restore of Apply DS SYM-001273 Figure 37 SRDF/A single session secondary Symmetrix begins restore of apply delta set Figure 38 on page 101 depicts the SRDF transfer of the primary Symmetrix transmit delta set to the secondary Symmetrix receive delta set. Secondary receives commit from Primary a) Check if the data in Apply DS is restored (data marked write pending to the R2 devices) b) Secondary cycle switch – Receive DS becomes Apply DS c) New Receive DS available for SRDF transfer d) Secondary sends Primary acknowledgement e) Begin restore of Apply DS 4. and for the Transmit DS to empty a) Primary tells Secondary to commit the Receive DS (begins Secondary step 3 in unison) b) SRDF transfer halted c) Primary cycle switch occurs – Capture DS becomes the Transmit DS d) New Capture DS available for Host I/O 3. Capture delta set (DS) collects application write I/O 2.SRDF/Asynchronous Operations 1 R1 N 3a 3e Capture N Apply N-2 R2 N-2 2d R1 N 2c 2b 3b Transmit N-1 Receive 2 3c 2d Primary Symmetrix R2 N-2 Secondary Symmetrix 1.

Synchronous SRDF-A Adaptive copy disk Adaptive copy pend off and SRDF-A SYM-001278 Adaptive copy WP Figure 39 SRDF/A single session transition path Switching to SRDF/A mode The host software can be used to switch to asynchronous mode. and then moves the system to active state (both primary and secondary Symmetrix).SRDF/Asynchronous Operations SRDF/A single session mode state transitions This section provides a general overview of how SRDF/A single session mode moves between states discussed in SRDF/A Single Session Mode States. As a result. Transition from synchronous to asynchronous When switching from SRDF synchronous mode. If there were previously owed tracks to be copied. the delta sets are established on the primary and secondary Symmetrix and the SRDF/A mechanism is enabled. meaning that all devices assigned to an SRDF group that is configured to operate in asynchronous mode operate in asynchronous mode when the SRDF/A state is active. where all of the devices are in sync. the secondary Symmetrix shows a consistent state and the data is dependent-write consistent. SRDF verifies that all the devices are ready. there is no dependent-write consistency at the secondary Symmetrix until the last owed track has been sent to the secondary Symmetrix and is in the N-2 cycle (apply delta set). Note: SRDF/A is an SRDF group-level feature. This chapter assumes that tolerance mode is set to off. Use Figure 39 on page 102 as a reference for “Switching to SRDF/A mode” on page 102. This happens when the DA director places the track owed to the secondary Symmetrix in the capture delta set (s) and SRDF/A cycle switching occurs until that track is in 102 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

Host I/Os continue to be serviced in the current SRDF/A cycles (capture delta set).000 tracks owed take a minimum of three SRDF/A cycle switches to transmit the data. Once this occurs. when the DA scans a device in the pending off mode. SRDF/A produces a consistent state on the secondary Symmetrix and a dependent-write consistent copy of data after all resync operations are complete and the two additional cycle switches have occurred. There are another two cycle switches required to ensure the data is in the apply delta set. Transition into SRDF/A mode from adaptive copy disk mode is immediate. it adds the slot to the active cycle (capture delta set). which means the data is dependent-write consistent. Transition from adaptive copy write-pending mode to asynchronous When the mode is set to SRDF/A from adaptive copy write pending mode all devices are moved into the adaptive copy pending off mode. If there are not any slots left write pending to the SRDF mirror that are not in the SRDF/A cycle. These are copy I/Os scheduled by the disk adapter to be serviced by SRDF/A. Note: It is recommended to capture a dependent-write consistent copy (locally and/or remotely) on a set of BCVs or Clones prior to performing this process.000 tracks to avoid using all of the cache in the primary Symmetrix. the device can transition out of the pending off mode. For example. 90. SRDF/A indicates the state is consistent.SRDF/Asynchronous Operations the apply delta set. Transition from adaptive copy disk mode to SRDF/A SRDF/A single session mode state transitions 103 . rather than creating a separate SRDF queue record. With SRDF/A active. or the N-2 copy of data. two cycle switches are required for the secondary Symmetrix to report a consistent state and have the data be dependent-write consistent. Each cycle switch (new capture delta set) limits the copy I/Os to 30. The length of time to send the tracks owed with asynchronous mode depends on the number of outstanding tracks owed prior to switching to asynchronous mode. Once all devices transition out of pending off mode. Tracks owed to the secondary Symmetrix as a result of adaptive copy disk skew are scheduled as resynchronization operations.

DROP and PEND-DROP. Some performance degradation occurs with synchronous mode while the transition takes place. Coming out of the SRDF/A active state This section refers to the SRDF/A active state that is consistent. you are not able to create a dependent-write consistent image on the secondary Symmetrix with either method of dropping SRDF/A. The third option is simply remove SRDF/A from the active state and transition the SRDF mode to another state without preserving the dependent-write consistency at the secondary Symmetrix. This is only allowed for SRDF/A single session mode. Once a transition is requested. meaning the data is in a dependent-write consistent state. Note: It is recommended to capture the resulting dependent-write consistent data with either a set of BCVs or Clones prior to any resynchronization. The following caveats apply: ◆ The transition is not immediate. This means tolerance mode is off. Note: If you are in an active state with an inconsistent secondary Symmetrix. This is not a recommended option. Requires both Symmetrix systems to be running Enginuity 5x71 or higher ◆ ◆ Note: It is recommended to capture a dependent-write consistent copy (locally and/or remotely) on a set of BCVs or Clones prior to performing this process. meaning you are still transferring accumulated copy I/Os. it may take some time for SRDF/A mode to be dropped and replaced with synchronous. During the resynchronization activity the dependent-write consistent image at the secondary Symmetrix is compromised.SRDF/Asynchronous Operations Switching to SRDF/S mode from SRDF/A single session mode It is possible to transition to a synchronous state from SRDF/A without losing dependent-write consistency for Enginuity 5x71 and above. In order to maintain a dependent-write consistent copy two options are discussed. SRDF/A supports several methods of dropping out of the active state into the Not Ready state. 104 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

there is not a need to resolve owed tracks upon resuming SRDF/A. Enginuity may also drop an SRDF/A single session mode for a number of reasons: ◆ Enginuity initiated drop Primary Symmetrix system write pending limit reached • Bandwidth not sized properly • Global memory not sized properly • Workload allocation for the specific implementation • Secondary Symmetrix device write pending limit reached – unbalanced configuration • Any combination of the above ◆ Secondary Symmetrix devices made NR on the link • By host command • Due to excessive device or link errors ◆ All links lost • Manually • External network or network equipment issues • Excessive link errors These all result in a dependent-write consistent image being preserved on the secondary Symmetrix.SRDF/Asynchronous Operations Dropping SRDF/A single session mode This option puts the devices in a Not Ready state immediately and the current cycle does not complete. By dropping SRDF/A on the cycle boundary. Resuming SRDF would then require resolving the tracks in the normal way. SRDF/A single session mode state transitions 105 . Dropping out of asynchronous mode does not compromise the dependent-write consistency of the data at the secondary Symmetrix. Pend-dropping SRDF/A single session mode PEND-DROP puts the devices in a Not Ready state only at the end of the current in-process cycle. PEND-DROP. Write-pending tracks in the active cycle are converted to tracks owed on the primary Symmetrix only. This results in tracks being converted to tracks owed on both the primary and secondary Symmetrix systems of the SRDF relationship.

106 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . data continues to flow and the dependent-write consistency of the data at the secondary Symmetrix is compromised. Because the devices are left ready on the SRDF link. These tracks owed are not dependent-write consistent.SRDF/Asynchronous Operations Deactivating SRDF/A single session mode SRDF/A mode offers the option of moving out of the active state immediately while leaving the SRDF devices ready on the link. The capture and transmit delta sets data are marked as owed tracks to the secondary Symmetrix similar to a resync operation.

The secondary Symmetrix marks and discards the receive delta set. Once SRDF is resumed. Any resync process compromises the dependent-write consistent image. a cleanup process occurs automatically within Enginuity. SRDF/A single session cleanup process 107 . It is very important to capture a “gold” copy of the dependent-write consistent data on the secondary Symmetrix R2 devices prior to any resynchronization. The “gold” copy can be captured on a remote set of BCVs or Clones. but the data is marked as being owed to the secondary Symmetrix. The primary Symmetrix does a “cleanup” of the delta sets.SRDF/Asynchronous Operations SRDF/A single session cleanup process Once SRDF/A single session mode is dropped. The secondary Symmetrix makes sure the apply (N-2) delta set is safely applied to disk. This data is marked as tracks owed to the primary Symmetrix. The primary Symmetrix marks new incoming writes as being owed to the secondary Symmetrix. The capture and transmit delta sets are discarded. these tracks are scheduled to be sent from the primary Symmetrix if the copy direction has not changed. this is the dependent-write consistent image. capture and transmit. All of these tracks owed are sent to the secondary Symmetrix once SRDF is resumed if the copy direction desired is primary to secondary.

! Permanent link loss If SRDF/A experiences a permanent link loss. it drops all of the devices on the link to not ready state. Temporary link loss If SRDF/A suffers a temporary loss (<10 seconds by default) on all of the SRDF links. In addition. database. This may result in an elongated cycle. The track tables are compared and merged based on normal host recovery procedures used by EMC host software. but the secondary Symmetrix dependent-write consistency is not compromised and the primary and secondary Symmetrix device relationship are not suspended. any new write I/Os on the primary Symmetrix system result in tracks being marked owed to the remote mirror. These are owed to the primary Symmetrix. The amount of time SRDF waits until it declares a link loss permanent is configurable. normal SRDF recovery procedures are followed. The secondary Symmetrix inactive cycle (receive delta set) data is marked owed to the remote mirror. All of these tracks are owed to the secondary Symmetrix once the links are restored. This results in all data in the active and inactive primary Symmetrix cycles (capture and transmit delta sets) being changed from write pending for the remote mirror to owed to the remote mirror. or host failure if SRDF is restarted in Synchronous or Semi-Synchronous mode. The data is then resynchronized by sending the owed tracks as part of the SRDF/A cycles. 108 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . the SRDF/A state remains active and data continues to accumulate in global memory. The active cycle (apply delta set) data completes its commit to the secondary Symmetrix devices. CAUTION Customers switching to SRDF/S mode with the link loss amount configured for more than 10 seconds could experience an application. When the links are restored.SRDF/Asynchronous Operations SRDF/A single session mode recovery scenarios This section briefly discusses the different recovery scenarios associated with SRDF/A single session mode.

and if the condition has not resolved itself at the expiration of this time. In an effort to assist customers to better manage global memory full conditions. SRDF/A single session mode recovery scenarios 109 . global memory. SRDF/A behaves based on customer-configurable settings: ◆ The primary Symmetrix system can throttle the host at the speed of the links. or the inability to de-stage data quickly enough at the secondary Symmetrix. In this case the host performance is equivalent to synchronous mode.SRDF/Asynchronous Operations Note: The data on the secondary Symmetrix devices is always dependent-write consistent in SRDF/A active/consistent state. even when the SRDF links have failed. However. and workload allocation for specific implementations. The inactive and active cycles (capture and transmit delta sets) on the primary Symmetrix consume all the available write memory in the Symmetrix system. For this reason. it is recommended a “gold” copy of the dependent-write consistent image be saved using either a set of BCVs or Clones on the secondary Symmetrix. Primary Symmetrix global memory full condition It is possible that an imbalance may occur with SRDF/A between the incoming write I/O workload and the outgoing SRDF/A bandwidth. then SRDF/A is dropped. Enginuity 5x71 introduced Tunable Cache Utilization. Contact your EMC Customer Service Representative to initiate a study of your environment to avoid such an imbalance. Factors and variables that may cause an imbalance in the SRDF/A environment may include bandwidth. The primary Symmetrix system throttles the host for a customer-defined specified period of time. the act of starting a resynchronization activity compromises the dependent-write consistency until the resynchronization is fully complete and two cycle switches have occurred. (section Enginuity 5671). unbalanced primary and secondary Symmetrix configuration. In this situation. The default behavior is to drop SRDF/A immediately when this condition occurs ◆ Note: To avoid a memory full condition. such that the global memory in the primary Symmetrix becomes full. and keep SRDF/A running. the SRDF/A environment must be properly designed and configured.

Once the workload has been transferred back to the primary Symmetrix hosts. the process for returning to the primary Symmetrix uses exactly the same methods as are used for synchronous SRDF failback operations. 110 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Once the primary Symmetrix has been repaired. the SRDF/A configuration can be reversed using either Dynamic SRDF or a configuration change. the data on the secondary Symmetrix devices represents a dependent-write consistent image of data that can be used to restart an environment with minimal data loss. SRDF/A can be activated and normal asynchronous mode protection can be resumed. In the event of an extended failover event. SRDF/A can continue to process until a planned reversal of direction can be performed again in order to restore the original SRDF/A primary/secondary relationship.SRDF/Asynchronous Operations Failback from secondary symmetrix devices In the event that a disaster occurs on the primary Symmetrix.

50 supports SRDF/A control for multiple Symmetrix provided there is a single SRDF group per Symmetrix. Beginning with Enginuity 5x71 for mainframe and Open Systems. SRDF/A multi-session consistency (MSC) mode 111 . This is referred to as SRDF/A Multi-Session Consistency or SRDF/A MSC. SRDF/A MSC configurations can also support mixed open systems and mainframe data controlled within the same SRDF/A MSC session. Achieving this requires a single coordination point to drive the cycle switch process in all participating Symmetrix systems.SRDF/Asynchronous Operations SRDF/A multi-session consistency (MSC) mode Mainframe software and Enginuity 5670. and that the switch occur during a very brief time period when no host writes are being serviced by the Symmetrix. SRDF/A is supported in configurations where there are multiple primary Symmetrix and/or multiple primary Symmetrix SRDF groups connected to multiple secondary Symmetrix and/or multiple secondary Symmetrix SRDF groups. Achieving data consistency across multiple SRDF/A groups simply requires that the cycle switch process described earlier in this chapter be coordinated among the participating Symmetrix SRDF group and/or systems. This function is provided by the SRDF control software running on the host.

Figure 40 on page 112 illustrates the delta sets and their relationships. The inactive cycle contains the N-1 data version that is transferred via SRDF/A from the primary Symmetrix to the secondary Symmetrix. I/O is processed exactly the same way in SRDF/A MSC mode as in single session mode. This is the guaranteed dependent-write consistent image in the event of a disaster or failure. 2. The primary inactive delta set is the transmit delta set and the secondary Symmetrix inactive delta set is the receive delta set. The active cycle on the primary Symmetrix contains the current host writes or N data version in the capture delta set. 1. 3. Primary Symmetrix Secondary Symmetrix R1 Capture N Transmit N-1 Apply N-2 Receive N-1 R2 R1 R2 R1 Capture N Transmit N-1 Capture N Transmit N-1 Apply N-2 Receive N-1 Apply N-2 Receive N-1 R2 R1 R2 Capture “Active” cycle Transmit “Inactive” cycle Receive “Inactive” cycle Apply “Active” cycle SYM-001275 Figure 40 SRDF/A MSC delta sets and their relationships 112 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The active cycle on the secondary Symmetrix contains the N-2 data version on the apply delta set.SRDF/Asynchronous Operations SRDF/A MSC mode dependent-write consistency From a single Symmetrix perspective.

This cycle tag is a value that is common across all participating SRDF/A sessions and eliminates the need to synchronize the cycle numbers across them. At this point SRDF/A enters MSC mode. it merely places SRDF/A in single-session mode. and conversely exiting multi-session mode does not drop or deactivate SRDF/A. This is done via the SRDF control software running on the host. As part of the process to enter MSC mode. The cycle tag is the mechanism by which dependent-write consistency is assured. Enginuity assigns a cycle tag to each capture cycle that is retained throughout that cycle’s life. SRDF/A MSC mode dependent-write consistency 113 .SRDF/Asynchronous Operations Entering SRDF/A multi-session consistency For the host to control the cycle switch process. if SRDF/A is dropped or deactivated. However. the Symmetrix systems must be aware that they are running in multi-session consistency mode. Figure 41 on page 114 updates the SRDF/A state diagram from single session mode to incorporate multi-session mode for SRDF/A. The host software: ◆ Coordinates the cycle switching for all SRDF/A sessions comprising the SRDF/A MSC configuration Monitors for a failure to propagate data to the secondary Symmetrix devices and drops all SRDF/A sessions together to maintain dependent-write consistency Performs MSC cleanup if able ◆ ◆ Note: Simply activating SRDF/A does not place a session in multi-session mode. and with each cycle switch issued thereafter. then multi-session mode is necessarily terminated and would need to be reentered once SRDF/A was made active again.

This time period is referred to as an SRDF/A window. When the host software discovers that all the SRDF groups and Symmetrix systems are ready for a cycle switch.SRDF/Asynchronous Operations Multi-session consistency Host command or Enginuity Remote site consistent or inconsistent Host command Host command Not ready (NR) All devices are Not Ready on the links Active Host command or Enginuity Remote site consistent or inconsistent Host command Host command Inactive Synchronous or adaptive copy modes SYM-001279 Figure 41 SRDF/A MSC allowed state transitions Performing a SRDF/A MSC consistent cycle switch SRDF/A MSC mode performs a coordinated cycle switch during a very short window of time when there are no host writes being completed. In multi-session mode. any write I/Os that start are disconnected and as a result no dependent-write I/Os are issued by any host to any devices in the SRDF/A MSC group. While the window is open. The SRDF/A window remains open on each SRDF group and Symmetrix system until the last SRDF group and Symmetrix system in the multi-session group acknowledges to the host software that the 114 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The table is accessed by the host adapter to obtain the cycle number at the start of each write in single session mode. and if it is on (an open window). the host adapter also checks the SRDF/A window bit. it issues a single command to each SRDF group that performs a cycle switch and opens the SRDF/A window. it disconnects upon receiving host write IO and begins polling the bit to see if the host software has closed the window. The SRDF/A window is implemented as a bit in the SRDF/A state table in global memory where the cycle number and tag are also stored.

As part of this switch and open operation. dependent-write consistency across the SRDF/A MSC group is created. the host software assigns a cycle tag value to the active cycle (capture delta set). Enginuity closes the window itself if the host software has not closed it within 5 seconds.) This cycle tag is carried by the SRDF/A process to the secondary Symmetrix and is used by the host software at the recovery site to ensure that only data from the same host cycle is applied to the secondary Symmetrix devices in each SRDF group and Symmetrix system in the event of a disaster. SRDF/A MSC mode dependent-write consistency 115 . At this point the host software issues a close command for each SRDF/A group under MSC control. read I/Os complete normally to any devices that have not received a write. (This cycle tag value is separate from the cycle number assigned internally by SRDF/A. During this window.SRDF/Asynchronous Operations switch and open command has been processed. because the host adapter is already obtaining the cycle number from global memory as part of SRDF/As existing overhead. As a result. The SRDF/A window is an attribute of the SRDF/A group and is checked at the start of each I/O. Note: Enginuity does provide a fail-safe mechanism to ensure that the window does not remain open permanently due to a host software failure. at no additional overhead.

Figures 42 through 51 assume SRDF/A MSC has been activated and two (2) cycle switches have occurred previously. 2. The primary Symmetrix transmit delta set is continuing to send the data to the secondary Symmetrix receive delta set. This next series of figures represent three SRDF/A single sessions combined together to create a single SRDF/A MSC group. 1 R1 N Apply N-2 Receive N-1 Capture N Transmit N-1 R2 N-2 1. one with a single SRDF/A group and the other with 2 SRDF/A groups. Capture delta set (DS) collects application write I/O R1 N R2 N-2 1 R1 N Capture N Transmit N-1 Capture N Transmit N-1 Apply N-2 Receive N-1 Apply N-2 Receive N-1 R2 N-2 R1 N R2 N-2 Primary Symmetrix Secondary Symmetrix SYM-001255 Figure 42 SRDF/A MSC capture delta set collects application write I/O 116 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . The primary Symmetrix transmit delta set must be empty. or mark the data write pending to the secondary Symmetrix R2 devices. The secondary Symmetrix is the same configuration as the primary. The apply delta set is continuing to restore. The secondary apply delta set must have completed marking the R2 devices write pending for the N-2 data. Before a primary Symmetrix cycle switch can occur two things must be achieved: 1. a balanced configuration. There are 2 primary Symmetrix. Figure 42 on page 116 displays the current host I/O being collected by the capture delta set on the primary Symmetrix.SRDF/Asynchronous Operations SRDF/A MSC mode delta set switching This section describes how the delta set switching works for SRDF/A MSC mode.

which is used during cleanup (if SRDF/A drops). 1 R1 N Apply N-2 Receive N-1 Capture N Transmit R2 N-2 1. and sends an acknowledgement back to the primary Symmetrix.SRDF/Asynchronous Operations Figure 43 on page 117 shows that SRDF transfer between the primary Symmetrix transmit delta set and the secondary Symmetrix receive delta set is complete. Capture DS collects application write I/O 2. Capture DS collects application write I/O 2. Primary Transmit DS complete a) Primary sends Secondary ‘ Transmit complete ’ message R1 N 2 R2 N-2 1 R1 N Capture N Transmit Apply N-2 Receive N-1 Apply N-2 Receive N-1 R2 N-2 2 R1 N Capture N Transmit R2 N-2 2 Primary Symmetrix Secondary Symmetrix SYM-001256 Figure 43 SRDF/A MSC Primary Symmetrix transmit delta set switch is emptied The primary Symmetrix halts the SRDF transfer and sends a ‘transmit complete’ message to the secondary Symmetrix as shown in Figure 44 on page 117. 1 R1 N Apply N-2 Receive N-1 Capture N Transmit R2 N-2 1. The secondary Symmetrix stores the information. Primary Transmit DS complete a) Primary sends Secondary ‘ Transmit complete ’ message R1 N 2 2c R2 N-2 1 R1 N Capture N Transmit Apply N-2 b) Primary waits for acknowledgement from Secondary c) SRDF transfer halted 2 R1 N Capture N Transmit 2c Receive N-1 Apply N-2 Receive N-1 R2 N-2 R2 N-2 2 Primary Symmetrix 2c Secondary Symmetrix SYM-001257 Figure 44 SRDF/A MSC Primary Symmetrix halts the SRDF transfer SRDF/A MSC mode delta set switching 117 .

The SRDF/A MSC host software that is coordinating the cycle switch and the process is explained in detail in “Performing a SRDF/A MSC consistent cycle switch” on page 114. The write I/O is deferred long enough for the host software to coordinate the cycle switch across all SRDF groups and primary Symmetrix.SRDF/Asynchronous Operations The secondary Symmetrix apply delta set completes the restore process by marking the data write pending to the R2 devices as shown in Figure 45 on page 118. Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message R2 N-2 b) Primary waits for acknowledgement from Secondary c) SRDF transfer halted 3. 118 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Secondary completes Apply DS (N . Capture DS collects application write I/O 2. Again this is because the primary Symmetrix transmit delta set is empty and the secondary Symmetrix apply delta set has completed the restore process. the primary Symmetrix responds to polls from the SRDF/A MSC host software with a “ready to switch” condition.2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message Capture N Transmit Apply Receive N-1 R1 N 2 2c 1 R1 N Capture N Transmit Apply Receive N-1 Apply Receive N-1 3 R2 N-2 2 R1 N Capture N Transmit 2c R2 N-2 2 Primary Symmetrix 2c Secondary Symmetrix SYM-001258 Figure 45 SRDF/A MSC Secondary apply delta set restore complete Once the primary Symmetrix receives the restore complete message from the secondary Symmetrix. 1 R1 N 3 R2 N-2 1. The SRDF/A MSC host software initiates a primary Symmetrix cycle switch once all of the participating SRDF groups in the SRDF/A MSC configuration report a “ready to switch” state. Figure 46 on page 119 displays this primary Symmetrix cycle switch between the capture and transmit delta set. the secondary Symmetrix sends a ‘restore complete’ message to the primary Symmetrix. When finished.

New Capture DS available for Host I/O SYM-001260 Figure 47 SRDF/A MSC I/O is released and a new capture delta set continue to accept Host I/O SRDF/A MSC mode delta set switching 119 . “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS b) I/O released .2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message 4. Capture DS collects application write I/O Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary c) SRDF transfer halted 5b R1 N Capture N Apply Receive N-2 5a 2c 2 Transmit N-1 R2 5b 1 R1 N Capture N Apply 3 5a 2c Receive N-2 Apply Receive N-2 R2 3. The transmit delta set contains the N-1 copy of dependent-write consistent data. At next host poll. Capture DS collects application write I/O 2. At next host poll.2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message 4. “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS SYM-001259 Figure 46 SRDF/A MSC Primary Symmetrix cycle switch while I/O is deferred The primary Symmetrix releases the deferred I/O and a new capture delta set accepts the host I/O as shown in Figure 47 on page 119.SRDF/Asynchronous Operations 1 R1 N 3 R2 N-2 1. Secondary completes Apply DS (N . Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary c) SRDF transfer halted 3. 1 R1 N 3 R2 1. Secondary completes Apply DS (N . Primary will respond “ready to switch” (Transmit complete and Apply restore complete both true) 5b R1 N 2 Transmit N-1 Capture N 5a 2c 5b 2 Transmit N-1 R2 Primary Symmetrix Secondary Symmetrix 5. 2. Primary will respond “ready to switch” (Transmit complete and Apply restore complete both true) Capture N Apply Receive N-1 5a 2c R2 N-2 R1 N 2 Transmit N 1 R1 N Capture N Apply Receive N-1 Apply Receive N-1 3 5a 2c R2 N-2 2 R1 N Transmit N Capture N 5a 2c 2 Transmit N R2 N-2 Primary Symmetrix Secondary Symmetrix 5.

2. 6b Secondary Symmetrix Primary Symmetrix 6. “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS b) I/O released . 2. as shown in Figure 48 on page 120.New Capture DS available for Host I/O Secondary receives commit message from Primary a) Secondary cycle switch – Receive DS becomes Apply DS b) New Receive DS available for SRDF transfer SYM-001262 Figure 49 SRDF/A MSC Secondary new receive delta set is available 120 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . At next host poll. 1 R1 N Apply N-2 3 R2 1. After receiving the commit message the secondary Symmetrix systems performs a cycle switch between the receive and apply delta sets. Primary will respond “ready to switch” (Transmit complete and Apply restore complete both true) 5b R1 N 2 Transmit N-1 Capture N Receive 6b 6a Apply N-2 5a 2c 5b 2 Transmit N-1 R2 Receive 5. 1 R1 N Apply N-2 3 R2 1. Capture DS collects application write I/O Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary SRDF transfer halted 5b R1 N Capture N 5a 2c 6a 2 Transmit N-1 Receive N-2 R2 3.New Capture DS available for Host I/O 6.2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message At next host poll. “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS b) I/O released .SRDF/Asynchronous Operations The primary Symmetrix sends a commit message to the secondary Symmetrix once the primary Symmetrix systems cycle switch occurs. Secondary completes Apply DS (N .2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message “ready to switch” (Transmit complete and Apply restore complete both true) 5b R1 N 2 Transmit N-1 Capture N Receive N-2 Apply N-2 4. Capture DS collects application write I/O Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary SRDF transfer halted 5b R1 N Capture N 5a 2c 6a 2 Transmit N-1 Receive 6b R2 3. Secondary receives commit message from Primary a) Secondary cycle switch – Receive DS becomes Apply DS SYM-001261 Figure 48 SRDF/A MSC Secondary Symmetrix cycle switch The secondary Symmetrix now has a new receive delta set available as shown in Figure 49 on page 120. Primary will respond 5a 2c 6a 5b 2 Transmit N-1 Receive N-2 R2 Primary Symmetrix Secondary Symmetrix 5. c) 5b 1 R1 N Capture N 3 5a 2c 6a Apply N-2 R2 Secondary completes Apply DS (N . c) 5b 1 R1 N Capture N 3 5a 2c 6a Apply N-2 R2 4.

6b Primary Symmetrix Secondary Symmetrix 6. Primary will respond “ready to switch” (Transmit complete and Apply restore complete both true) 6b R2 N-2 5a 2c. 2. 6c 6a Apply N-2 R2 4. the SRDF transfer process beginning from the primary Symmetrix systems to the secondary Symmetrix systems. 6c 6a 2 Transmit N-1 Receive N-1 6b 5b 1 R1 N Capture N Apply N-2 R2 N-2 c) 3. “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS b) I/O released . Secondary completes Apply DS (N .2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message At next host poll. Secondary completes Apply DS (N . 6c 6a 2 Transmit N-1 Receive N-1 6b R2 3. 6d 5a 2c. 6c 6a 5b 2 Transmit N-1 Receive N-1 5. 6b Primary Symmetrix Secondary Symmetrix 6. Capture DS collects application write I/O Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary c) SRDF transfer halted 5b R1 N Capture N 5a 2c. 2. 1 R1 N Apply N-2 6d R2 N-2 1.SRDF/Asynchronous Operations In Figure 50 on page 121. Primary will respond “ready to switch” (Transmit complete and Apply restore complete both true) 5b R1 N 2 Transmit N-1 Capture N Receive N-1 Apply N-2 6b R2 5a 2c.New Capture DS available for Host I/O Secondary receives commit message from Primary a) Secondary cycle switch – Receive DS becomes Apply DS b) New Receive DS available for SRDF transfer c) SRDF transfer begins SYM-001264 d) Begin Secondary Apply DS restore Figure 51 SRDF/A MSC Secondary Symmetrix begins the apply delta set restore process SRDF/A MSC mode delta set switching 121 . 6c 6a 5b 2 Transmit N-1 Receive N-1 5. 1 R1 N Apply N-2 3 R2 1. 6c 6a 5b R1 N 2 Transmit N-1 Capture N Receive N-1 Apply N-2 R2 N-2 4. Capture DS collects application write I/O Primary Transmit DS complete a) Primary sends Secondary ‘Transmit complete’ message b) Primary waits for acknowledgement from Secondary SRDF transfer halted 5b R1 N Capture N 5a 2c.2) restore (data marked write pending to the R2 devices) a) Secondary sends Primary “restore complete” message At next host poll. 5b 1 R1 N Capture N 3 5a 2c. “Switch/Open ” receive from host a) Primary cycle switch occurs while I/O deferred – Capture DS becomes the Transmit DS b) I/O released .New Capture DS available for Host I/O Secondary receives commit message from Primary a) Secondary cycle switch – Receive DS becomes Apply DS b) New Receive DS available for SRDF transfer c) SRDF transfer begins SYM-001263 Figure 50 SRDF/A MSC Primary Symmetrix systems begin SRDF transfer The secondary Symmetrix systems also begin the apply delta set restore process and the process begins again as shown in Figure 51 on page 121.

SRDF/Asynchronous Operations SRDF/A MSC session cleanup process When SRDF/A is deactivated or dropped while in multi-session consistency mode. This is where SRDF/A MSC uses the host cycle tags. The host software does not need to perform any special recovery on the primary Symmetrix. Enginuity at the secondary Symmetrix completes the restore of its apply delta set automatically. Enginuity discards any receive delta sets that are not complete. For each SRDF group. The most current data is already on the 122 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . This is determined by the following rules: In the first case. awaiting a decision from the host software. The default behavior commits all of the receive delta sets to deliver the most current dependent-write consistent data to the secondary Symmetrix devices. In this case. each R1 Symmetrix system starts the same cleanup process as in single session mode – it discards all I/O from both transmit and capture delta sets and marks the corresponding tracks owed to the secondary Symmetrix. There are three different scenarios to be considered when SRDF/A has been terminated with respect to the receive delta sets on the secondary Symmetrix. If the receive delta set is a complete delta set for each SRDF group Enginuity marks it as “needing cleanup” in cache. In the second case. the host software may choose to either commit all of the receive delta sets or discard all of the receive delta sets. The host software must use its cycle tags during recovery of the receive delta sets on the secondary Symmetrix. all receive delta sets on all secondary Symmetrix systems and SRDF groups have the same tag and are marked as “needing cleanup”. This means Enginuity discarded an incomplete receive delta set. all receive delta sets on all Symmetrix systems have the same tag number. The host software must discard all receive delta sets for this tag number in this case. Remember “needing cleanup” is Enginuity stating the receive delta set is complete. but at least one Symmetrix systems or SRDF/A group does not have a receive delta set marked “needing cleanup”. This is the result of the secondary Symmetrix receiving and acknowledging the ‘transmit complete” message in step 2 of the SRDF/A MSC cycle switch process.

SRDF/Asynchronous Operations

secondary Symmetrix devices via the apply delta set. The data that was in the discarded receive delta sets are marked as tracks owed to the primary Symmetrix devices. The third case occurs where there are different cycle tags within the apply and receive delta sets. In this case, the secondary Symmetrix can be divided into two groups. The first group has Symmetrix systems with apply delta set cycle tags that match the receive delta set cycle tags of the Symmetrix systems from the second group. In other words, Symmetrix systems from the first group have received the commit message for a certain host cycle, while the Symmetrix systems from the second group have not. In this case, the receive cycles of the Symmetrix systems from the second group are necessarily complete and the host software must force their restore. At the same time, host software must discard the receive delta sets of the Symmetrix systems from the first group regardless of their completeness.

SRDF/A MSC session cleanup process

123

SRDF/Asynchronous Operations

124

EMC Symmetrix Remote Data Facility (SRDF) Product Guide

Invisible Body Tag

5
SRDF/Star Operations

This chapter provides a technical overview of SRDF/Star. The following topics are covered:
◆ ◆

SRDF/Star overview ....................................................................... 126 How SRDF/Star works ................................................................... 129

SRDF/Star Operations

125

It is strongly recommended that all SRDF devices be locally protected and that there is capacity allocated for one replica (BCV. SRDF/Star also provides a mechanism to determine which site (B or C) has the most current data in the event of a rolling disaster that affects site A. Rather than a full resynchronization between sites B and C. SRDF/Star provides a differential B to C synchronization. dramatically reducing the time it takes to remotely protect the new production site. Figure 52 on page 127 shows a SRDF/Star configuration. SRDF/Star provides for rapid reestablishment of cross-site protection in the event of primary site (A) failure. Snap.SRDF/Star Operations SRDF/Star overview Available at Enginuity level 5x71 for mainframe and open systems environments. SRDF/Star is a solution that operates in a concurrent SRDF configuration (A-to-B and A-to-C) where one remote mirror operates in SRDF/S mode (A-to-B) and the other remote mirror operates in SRDF/A mode (A-to-C). In all cases. or clone) at both of the remote sites in the configuration. you maintain the ability to choose both which site to operate from and which site’s data to use when recovering from a primary site failure. 126 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

SRDF/Star Operations Primary Site (A) Production Site SRDF/Synchronous Local Site (B) R1 R2 SRDF/Asynchronous Active Inactive Remote Site (C) R2 BCV Figure 52 Concurrent SRDF configured for SRDF/Star support The concurrent configuration option of SRDF/A offers the ability to restart your environment at long distances with minimal data loss. Data at C would continue to age as production was resumed at site B. SRDF/Star overview 127 . while simultaneously providing a zero data loss restart capability at a local site. Such a configuration provides protection for both a site disaster and a regional disaster while minimizing performance impact and loss of data. Resuming SRDF/A between sites B and C would require a full resynchronization to renewable disaster recovery protection. This is both a time and resource consuming process. In a concurrent SRDF/A configuration without the SRDF/Star functionality. the loss of the primary A site would normally mean that the long distance replication would stop and data would no longer propagate to the C site.

SRDF/Star Operations SRDF/Star avoids this full resynchronization by allowing a B-to-C (or C-to-B) resynchronization to be done differentially using host software commands and procedures. Refer to the appropriate host platform software documentation. 128 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . with minimal data movement. in the event of a primary site failure ◆ Known requirements and limitations at this release Known requirements and limitations for this release of SRDF/A are as follows: ◆ All SRDF/Star SRDF device pairs must be the same geometry and size. Only Fibre or GigE SRDF links are supported. SRDF/Star is defined by the host software. In addition. SRDF/Star benefits SRDF/Star provides the following features and benefits: ◆ The ability to maintain business continuance despite the loss of any site in a three-site configuration The ability to resume asynchronous protection between the two secondary sites. All SRDF Groups (even inactive groups) must be defined prior to entering SRDF/Star mode. SRDF/Star allows the SRDF personalities of sites A and B to be swapped and the SRDF/A relationship to be transferred to sites B and C during planned outages with no data movement at all. refer to the Solutions Enabler Symmetrix SRDF CLI Product Guide and the Symmetrix SRDF Host Component Product Guide. Note: For more information on specific restrictions. ◆ ◆ ◆ Note: The R2 site becomes an R1 as a function of site switchovers.

BCVs are required at site B. SRDF/Star also provides a mechanism to How SRDF/Star works 129 . Unplanned Failovers — This refers to host based automation executed at the remote site that achieves the reconfiguration. Refer to Figure 53 on page 131 for all scenarios. The solution provides a mechanism to determine whether site C or site B has the most current data. If planned switching between site A and site B is going to be performed. and manage planned and unplanned failover from site A to site B or site A to site C. provide differential resynchronization between sites B and site C. Restart may occur at either site B or site C so resynchronization procedures are presented for each case. SRDF/Star provides processes and procedures to reconfigure SRDF/S and SRDF/A. MSC manages and manipulates Symmetrix data structures and SRDF features to enable differentia B-to-C or C-to-B resynchronization in the SRDF/Star configurations. C) providing for differential resynchronization between sites B and C and a restart at either site. SRDF/Star requires an active host at site A. Planned Switchovers — This refers to host based automation executed at a remote site to effect reconfiguration of SRDF/Star in support of planned site switchovers between the synchronous SRDF sites. and site C. supported BCVs are also required at site A. The initial configuration is site A has R1 devices in a SRDF/S relationship with site B and in a concurrent SRDF/A relationship with site C. ◆ ◆ Descriptions of states and events in this chapter are from the R1 point of view unless otherwise noted. ◆ Normal operation — This refers to function provided by the multi-session consistency task running in the host. Data resynchronization is possible from either site B or site C to the other site. and is a decision that is made by the user. SRDF Consistency Group is required between sites A and B. B. A host is also required at either site B or site C for recovery processing following a site A failure. The BCVs retain any consistency images created either during a planned or unplanned event.SRDF/Star Operations How SRDF/Star works The SRDF/Star solution applies to a three site configuration (A.

This will include the required session management at the SRDF/S R2 site and when necessary at the SRDF/A R2 site. SRDF/Star control for Open Systems Solutions Enabler Software along with the SRDF daemon control operations of SRDF/Star at the R1 site with the use of a user created composite group. and documented procedures are used to accomplish resynchronization and manage the reconfigurations. The symstar command was created to control. automation utilities. support utilities. MSC session management maintains the information needed to perform differential synchronization between site B and site C. The host software and Enginuity maintain the information needed to perform the differential synchronization between the synchronous and asynchronous secondary sites.SRDF/Star Operations determine when current active R1 cycle (capture) contents reach the active R2 cycle (commit). Other host software including SRDF Host Component. Mainframe customers wishing to script additional suggested procedures and/or extend them could use any number of automation products including native REXX or EMCSPE. and automate the SRDF/star processes in an Open System environment. MSC performs the session management at the SRDF/S R2 site and when necessary at the SRDF/A site C. 130 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . Consistency Groups. manage. SRDF/Star control for mainframe Normal operation of SRDF/Star is controlled by the host based Multi-Session Consistency (MSC) task at the R1 site. Automation for some basic procedure operations is provided with SRDF/Star and is supported by EMC if unaltered.

Host 2 H3 .Host 3 P1 .Symmetrix 1 Sym2 . This automation executes in a z/OS host and is written in REXX.An Implementation and Usage Primer.Symmetrix 3 Each Synchronous R2 has 2 SDDF sessions Each Asynchronous R2 has 1 SDDF session Figure 53 P1 L1(Sync) L( 2 A s yn c) Sym2 Sym3 P2 L3(Async) Site C P3 H3 SRDF/Star configuration reference SRDF/Star automation for mainframe EMC provides host-based automation for both planned switchover actions and unplanned failover actions. How SRDF/Star works 131 . Detailed descriptions and implementation guidelines can be found in SRDF/Star for z/OS .SRDF/Star Operations Site A H1 Site B H2 Sym1 H1 .Host 1 H2 . using the EMC Stored Procedure Executive automation tool.Host to Sym path 2 P3 .Host to Sym path 1 P2 .Host to Sym path 3 L1 SRDF link 1 in synchronous mode (SRDF/S) L2 SRDF link 2 in asynchronous mode (SRDF/A) L3 SRDF link 2 in asynchronous mode (SRDF/A) Sym1 .Symmetrix 2 Sym3 .

132 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . transient fault. Other licenses are also required to maintain the dependent-write consistency. and planned switch operations in the Open Systems environments. unplanned switch. for normal. This automation is delivered in Solutions Enabler with the SRDF/Star license. Detailed descriptions and implementation guidelines can be found in EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide. via the symstar command.SRDF/Star Operations SRDF/Star automation for open systems EMC provides host-based automation.

Index A adaptive copy disk mode 18. 32 D data protection options 24 data recovery 44 F fabric connectivity 30 fully switched 30 EMC Symmetrix Remote Data Facility (SRDF) Product Guide 133 . 75 data vaulting 28 database management systems 20 DBMS 21 dependent write operations 21 device states 49 domino mode 58 device domino mode 58 link domino mode 58 dual-directional configuration 47 dual-directional link protocol 47 dynamic spare 38. 40. and ATM links 47 EMC compatible peer 42 EMC ControlCenter software 24 EMC SRDF/Consistency Groups 20 Enginuity 30. 56 ATM links 47 attributes. 39 Dynamic Sparing 24. 56 adaptive copy mode 32. 44 Dynamic Sparing with mirrored pairs 44 Dynamic Sparing with SRDF 44 Dynamic SRDF 39. 42. 60 configurations SRDF 27 to 36 connectivity point-to-multipoint 31 point-to-point 31. 39. 57 write-pending mode 18. T1. 32. 60 ESCON 34. 47. 60 ESCON remote adapter (RA) 25 ESCON remote director (RA) 26 Ethernet infrastructure 33 C channel interface states 51 Concurrent SRDF 32. T3. 48. 42. 46. 43. 80 dynamic SRDF devices 43 dynamic SRDF groups 43 E E1. 33. system-level 59 B backups without a remote host 77 BCV 39 as a primary (source) device 75. 33. 76 concurrent operations 73 performing remote backups 77 SRDF multi-hop 77 business continuance operations 73 to 77 business continuance volumes (BCVs) 38. E3. 33. 40.

36 performance impact 36 preserving synchronization 35 Fibre Channel 60 Fibre Channel directors 43 Fibre Channel point-to-point connection 40 Fibre Channel remote adapter (RF) 25 Fibre Channel remote directors (RF) 25 FICON 46 primary volume 50. 46 Permanent Member Sparing 24 PowerPath 62 PPRC mode 41. 51 H host accessibility 51. 46 primary (source) volumes 38 primary (source. remote 22 modes of operation primary ?? to 55 secondary 56 to 59 Multi-hop 77 multiple RA pairs 26 multiple Symmetrix logical volumes. 44 Multiprotocol Channel Director (MPCD) 25. 51 read/write state 50 read-only state 51 L link protocols dual-directional 47 unidirectional 47 links possible states 48 local mirroring 22 local volumes 39 logical device 44. R1) volumes 47 primary devices 29. 45 logical path 32 logical paths 31 logical volume states 49 host accessibility 51 host view 51 not ready 49 134 EMC Symmetrix Remote Data Facility (SRDF) Product Guide . 31. 51 secondary volumes 50. 52 host volume manager software 44 hypervolume 44 size 44 P Parity RAID 24.Index failover (secondary Symmetrix system takeover) 71 FarPoint 32. 40 Parity RAID (3+1) 38. 44. 30. 47 host accessibility 52 host view 51 not ready state 50. 35. 39 PAV/MA 36 Peer-to-Peer Remote Copy (PPRC) 41. 51 SRDF view 50 to 51 write enabled 49 M mirroring local vs. 33 primary volume 38 primary volumes 38. 34. 39 Parity RAID (7+1) 38. 33 G GDPS 46 Gigabit Ethernet 60 GigE 33 GigE connection 40 GigE remote adapter (RE) 25 GigE remote directors (RE) 25 GigE SRDF directors 43 N Native IP 33 not ready state primary volumes 50. 51 read only 49 secondary volumes 50.

18. performance impact 36 groups 40. 39 read operations 69 to 70 read/write state primary volumes 50 secondary volumes 51 read-only state 51 secondary volumes 50 recovery operations 71 to 72 remote mirroring 22 S SAN 30 secondary (target) volumes 47 secondary devices 29. 51 states 50. and DMX3000 systems 26 Symmetrix model numbers 17 Symmetrix Remote Data Facility Refer to SRDF Symmetrix Remote Data Facility/Synchronous 18 synchronous mode 33 system-level attributes 59 T TCP connections 34 TimeFinder operations (establish. 62 to?? SRDF/Data Mobility 16. 31 secondary Symmetrix system takeover 71 secondary volumes 39. 19 SRDF/CG 17. 33. 39 RAID 10 38. 19 SRDF/S 16. split. 30. 39 RAID 5 (7+1) 38. 51 semi-synchronous mode 18. 42 hardware 25 Host Component for z/OS 24 link states 48 monitoring and controlling 24 Multi-hop 77 over Fibre Channel 30 overview 16 Symmetrix interfamily connectivity 23 unidirectional configurations 68 volumes states 50 SRDF/A 16. 19 SRDF/DM 16. DMX2000. 45 RAID 5 24 RAID 5 (3+1) 38. 62 SRDF/Star 17 SRDF/Synchronous 16 Storage Area Network (SAN) 30 swap SRDF devices 80 switched Fibre Channel configuration 30 switched Fibre Channel fabric connection 40 Switched SRDF 30. 20 SRDF/Cluster Enabler 17 SRDF/Consistency Groups 17. 44. 19 SRDF/Asynchronous 16. 18. 41 SRDF/Automated Replication 17. 38. 39. 33 Symmetrix DMX1000. 18. 54 Solutions Enabler software 24 SRDF basic configuration 22 bidirectional configuration 47 concurrent operations 73 configurations 27 to 36 control operations swap 80 devices 40 director hardware 25 director/adapter board sets 25 dual-directional configurations 68 FarPoint 34 FarPoint. 48 SRDF/AR 17. 42 Symmetrix RAID 10 45 Symmetrix 3XXX/5XXX systems 33 Symmetrix 8000 systems 33 Symmetrix DMX systems 18. 51 write-enabled state 51 R RAID 1 22. restore) 76 EMC Symmetrix Remote Data Facility (SRDF) Product Guide 135 . 51 read/write state 51 read-only state 50.Index states 50. 47 host accessibility 52 host view 51 not ready state 50.

Index U unidirectional link protocol 47 UNIX 24 V volume types 38 to 45 W WAN 33 Windows 24 write operations 68 write-enabled state 51 Z z/OS 24 136 EMC Symmetrix Remote Data Facility (SRDF) Product Guide .

Sign up to vote on this title
UsefulNot useful