Professional Documents
Culture Documents
The objectives for this module are shown here. Please take a moment to read them.
Open Systems
Mainframe
Symmetrix Remote Data Facility (SRDF) is a Symmetrix system based business continuance, disaster recovery, restart, and data mobility solution. In the simplest terms, SRDF is a configuration of multiple Symmetrix units that maintains real time copies of logical volume data in more than one location. The Symmetrix units can be in the same room, in different buildings within the same campus, or hundreds and even thousands of miles apart.
RW
M1
Source
M2 M3 M4 M1
Target
M2 M3 M4
WD
This slide displays the representation of the mirror positions when both the Source and the Target SRDF Logical Volumes have local protection (RAID-1). In this diagram, the Target-R2 volume is also represented with 4 mirror positions and has local protection implemented. Three of the mirror positions are used. The first two mirror positions represent local mirrors and the third mirror is occupied by SRDF. If a BCV is established with the R2 volume, then it will occupy the next available mirror position. Under normal circumstances, the R1 volume presents a Read-Write (RW) status to the host which access it, and the R2 presents Write-Disabled (WD) to its host.
Remote Remote Link Link Director Director Source Source Remote Remote Link Link Director Director
Remote Remote Link Link Director Director Remote Remote Link Link Director Director Target Target
A Remote Link Director is a hardware that provides communication and data paths between local and remote Symmetrix units. The Symmetrix can be configured with the following RLDs: Fibre Channel directors (RF) ESCON directors (RA) Multiprotocol Channel Directors (MPCD) available with these channel connections: FICON iSCSI for host GigE (RE) for SRDF
Symmetrix B
Target
Symmetrix A
Bi-Directional
Source Target RA Group RA Group
Symmetrix B
Source Target
Symmetrix A
Dual Configuration
Source Source RA Group 1 RA Group 1
Symmetrix B
Target Target
Target
Target
RA Group 2
RA Group 2
Source
Source
SRDF offers three types of link configurations between source (local) and target (remote) Symmetrix systems: Uni-Directional, Bidirectional and Dual Configuration. SRDF Unidirectional Link Configuration If all primary (source or R1) volumes reside in one Symmetrix system and all secondary (target or R2) volumes reside in another Symmetrix system, write operations move in one direction, from primary to secondary. Data moves in the same direction over every link in the SRDF group. SRDF Bidirectional Link Configuration If an SRDF group contains both primary and secondary volumes, write operations move data in both directions over the SRDF links for that group. SRDF Dual-Directional Link Configuration With a dual-directional configuration, multiple SRDF groups are used; some groups send data in one direction, while other groups send data in the opposite direction.
SRDF/S (Synchronous)
Secondary SRDF Mode Adaptive Copy
- Write Pending - Disk Mode
Listed are the operational modes for SRDF operations: Synchronous mode, Adaptive Copy-Write Pending mode, Adaptive Copy-Disk Copy mode, and Asynchronous mode. These operational modes are selectable based on many requirements such as RPO, bandwidth, and performance. One of the two primary SRDF modes of operations is set at the source (R1) volume during Symmetrix configuration. All source (R1) volumes are configured for either the Synchronous or Semi-Synchronous mode. These two modes are considered to be pre-determined SRDF modes, which may be altered using SymCli. Adaptive copy is the secondary mode that facilitates data sharing and migration. Asynchronous mode continually collects and sends data to the remote Symmetrix. Asynchronous mode must be set for the entire RA group. Users can set SRDF to function in a secondary or Asynchronous mode. SRDF will revert to the pre-determined primary mode if it cannot maintain the criteria to remain in the secondary mode. Domino Mode could be classified as an SRDF attribute. Not necessarily a Mode. This attribute is set or used in conjunction with other SRDF modes except SRDF/A. It effectively stops all write operations to both source and target volumes if the target volume become unavailable, or if all SRDF links become unavailable.
Synchronous Mode
4 1
RDF link
2
Source
Target
1 2 3 4
Write I/O received from host/server at the source I/O is transmitted to the target An acknowledgment is provided by target back to the source I/O is serviced to the host
SRDF and Consistency Technology - 7
SRDF Synchronous Mode is used primarily in SRDF campus environments. In this mode of operation, Symmetrix maintains a real-time mirror image of the data of the remotely mirrored volumes. Data on the source (R1) volumes and target (R2) volumes are always fully synchronized at the completion of an I/O sequence. The sequence of operations is: A write is received from the host/server at the source. The write is transmitted to the target. An acknowledgment is provided by the target back to the source. The write is acknowledged to the Host. If step 3 never happens, the source SRDF services the I/O after a pre-determined timeout to keep the production machine running.
RDF link
4
Source
Target
1 2
Write I/O received from host/server at the source I/O accumulates in/on: - Symmetrix cache Write Pending Mode - R1 volume Disk Mode I/O is serviced to the host I/O is transmitted to the target An acknowledgment is provided by target back to the source
SRDF and Consistency Technology - 8
3 4 5
SRDF Adaptive Copy Mode is used primarily for data migrations and data center moves. This operational mode is not recommended for use when mirroring for disaster recovery/restart purposes unless used with TimeFinder. This mode is very useful for initial synchronization, especially over long distances. (Used within a SRDF/Star configuration). SRDF Adaptive Copy Mode allows the source (R1) volumes and target (R2) volumes to be a out of synchronization by a number of I/Os that users can define, a skew value. There are two types of adaptive copy: Write Pending Mode and Disk Mode. Adaptive Copy data movement is handled at the track level. The target data is only usable after a full synchronization. The sequence of operations is: An I/O write is received from the host/server at the source. I/O is accumulating. I/O is serviced. The I/O is transmitted to the target. An acknowledgment is provided by the target back to the source. In Write Pending Mode, the unit of transfer across the SRDF link is the updated blocks rather than an entire track, resulting in more efficient use of SRDF link bandwidth. Data is read from global memory than from disk, thus improving overall system performance. However, the global memory is temporarily consumed by the data until it is transferred across the link. In Disk Mode, while less global memory is consumed it is typically slower to read data from disk than from global memory, additionally, more bandwidth is used because the unit of transfer is the entire track. Additionally, because it is slower to read data from disk than global memory, device resynchronization time increases. Adaptive copy disk mode should not be used if the primary volumes are not RAID protected.
Asynchronous Mode
3 1 2
RDF link
4 5
Source
Target
1 2 3 4 5
Write I/O received from host/server at the source I/O accumulates in Source Symmetrix cache I/O is serviced to the host I/O is continually transmitted to the target I/O accumulates in Target Symmetrix cache
SRDF/A provides a long-distance replication solution with minimal impact on performance. This protection level is intended for customers requiring minimal host application impact, who need to maintain a restartable copy of data at the target site at all time. SRDF/A continually process Write I/Os in batches. The interval between batches is referred to as a cycle. The sequence of operations is: An I/O write is received from the host/server into the cache of the source. I/O is accumulating. I/O is serviced. The I/O is transmitted to the target.
3 1
RDF link
2
Source
Target
1 2 3
Write I/O received from host/server at the source I/O fails to transmit to the target Both Source and Target become unavailable
Domino Mode is used in conjunction with other SRDF modes except SRDF/A. It effectively stops all write operations to both source and target volumes if target volume become unavailable, or if all SRDF links become unavailable. User will need to manually re-enable the source volumes. While such a shutdown temporarily halts production processing, domino modes can prevent data integrity exposure that causes the inconsistent image on the target volume.
Adaptive Copy
- Source Target
Source may be up to 65535 tracks per volume ahead of Target Skew value set per logical volume
Asynchronous
- SRDF/A - Source is minutes ahead of Target - SRDF/AR - Source is hours ahead of Target
SRDF and Consistency Technology - 11
SRDF offers considerable flexibility for various levels of synchronization. To determine the level of synchronization, one must understand the required Recovery Point Objective. This is the amount of data that can be lost in the event of a site outage. There are other factors like distance, bandwidth, and response time latency that must be considered before determining a synchronization level.
SRDF Devices
R1 / R2 devices SRDF Source and Target mirror devices R1 R2
R1 R1 R2 R2
Source
Source Host
RDF Link
Target
Target Host
R1 R1
R21 R21
T S
R2 R2
Source Host
2008 EMC Corporation. All rights reserved.
Target Host
SRDF and Consistency Technology - 12
When configured for SRDF, the individual Symmetrix devices are designated as either a source mirror (R1 device), a target mirror (R2 device), or a cascaded mirror (R21). A cascaded mirrored device (R21) performs as both a source and target mirror. If the source device fails, the data on its corresponding target device can be accessed by the local host. Once the source device is replaced, it can be resynchronized. SRDF configurations have at least one source (R1) device mirrored to one target (R2) device. For concurrent RDF systems, there can be two R2 targets. Cascaded RDF environments include R1 -> R21 -> R2 devices. Source (R1) devices can only belong to an RDF1 device group, target (R2) devices can only belong to an RDF2 device group, and R21 devices can only belong to an RDF21 device group. An RDF device group is a user-defined device group comprised of RDF devices from a single Symmetrix array. At the time of creation, a device group must be defined as type REGULAR, RDF1, RDF2, or RDF21 and may contain various device lists for standard, BCV, virtual (VDEV), and remote devices. If the group type is defined as RDF1, RDF2, or RDF21, the group is considered an RDF device group. By default, a device cannot belong to more than one device group. However, you can change this default behavior to allow a device to belong to multiple groups by enabling the SYMAPI_ALLOW_DEV_IN_MULT_GRPS parameter in the Symmetrix options file.
SRDF Device Group A user defined object that is used to view and manage a number of DMX devices SRDF Composite Group A DMX device group that contains devices that span multiple DMX arrays and multiple RA Groups Consistency Group A Composite group that was created with the -rdf_consistency flag to insure dependent write consistency across the composite group
EMCs white paper titled An Overview of Groups in Symmetrix and Solutions Enable Environments (date Jan 08) provides extensive definitions for the above. RA Groups or SRDF Groups are used to define relationships between multiple DMX storage arrays RA groups fall into two categories, Static and Dynamic. Static Device Groups are configured in the DMX bin file. Where Dynamic Device Groups provides the ability for the storage administrator to create an RA Group without a bin file modification. An SRDF Device Group is a user defined object that enables the user to view and manage a number of DMX devices. A SRDF Composite Group is a DMX device group that contains devices that span multiple DMX arrays and multiple RA Groups. A Consistency Group is a Composite Group that was created with the -rdf_consistency flag to insure dependent write consistency across the composite group.
3
R1 RDF Link R2 Target Host R2 Symm 3 Symm 1
2
Production Host R1
1
RDF Group #
R1
Symm 2
1 To identify the RA Groups c:\symrdf list rdfg all 2 To identify all the RDF devices within RDF Group # 1 c:\symrdf list rdfg 1 3 To identify all the SRDF Device Groups c:\symdg show all 4 To identify all the SRDF Composite Groups c:\symcg show all 5 Note If the composite group was created with the rdf_consistency flag, than its a consistency group.
SRDF groups may be referred to as RA Groups or RDF Groups Both of these terms essentially mean the same thing. Performing a symdg show dev <Dev ID> will identify which RA group a device belongs to. A user can create a Device Group that contains some or all the devices in the RA group. If asynchronous replication will be used across the RDF link than all the devices in the RA group must be a member of the device group. More on this in the SRDF/A Operation Module (Module 4). A Composite Group follows the same rules as a Symmetrix device group except it contains devices that span multiple Symmetrix arrays and multiple RDF / RA group. A Consistency Group is a Composite Group that was created with the rdf_consistency flag to enable RDF devices in separate RDF groups to maintain dependent write consistency. Write consistency is maintained by using either EMC PowerPath or Enginuity Consistency Assist (ECA) for SRDF/S or MSC (Multi Session Consistency) for SRDF/A. A complete overview of the above is presented in an EMC White Paper, titled An Overview of Groups in Symmetrix and Solutions Enabler Environments, date Jan 2008.
Dynamic SRDF
Enables user to dynamically define relationships between R1 and R2 volumes Provides flexibility for user to tailor SRDF configuration to their changing application requirements
Sym Dev 001
STD STD R1 R1
RA RA
switch
RA RA
R1 R1 STD STD
Local
2008 EMC Corporation. All rights reserved.
Remote
SRDF and Consistency Technology - 15
Prior to Dynamic SRDF, the R1 and R2 pairings were static and defined in the configuration file (BIN File) on the Symmetrix. Any changes to SRDF device pairing required a new BIN file to be defined and loaded into the Source and Target Symmetrix. Dynamic SRDF available with 5x68 Enginuity code will provide the capability to change device pairings on the fly without requiring a BIN file configuration change to be performed by an EMC Customer Engineers.
RDF Group 1
-sid 80
-sid 40
RDF Group 1
R1 R1 R1 R1 R1 R1 R1 R1
Remote Remote Link Link Director Director Remote Remote RDF Dynamic Link Link Group Director Director dyngrp2 12a
Remote Remote Link Link Director Director Remote Remote Link Link Director Director
R2 R2 R2 R2 R2 R2 R2 R2
R1 R1 R1 R1 R1 R1 R1 R1
RDF Group 2
R2 R2 R2 R2 R2 R2 R2 R2
RDF Group 2
13a
An SRDF group, also known as RDF group or RA group, logically defines relationships between Symmetrix systems. An SRDF group is a set of SRDF director port connections configured to communicate with a another set of SRDF director ports in another Symmetrix system. Logical volumes (devices) are assigned to SRDF groups. Many SRDF groups can share a physical link between the Remote Link Directors. There are two ways to create an RDF group - static and dynamic. Both share the same features and functionality, the difference between the two types is how they are created. Static RDF groups are created during the Symmetrix configuration, and almost always by EMC personnel. Dynamic RDF groups are created and deleted by users through a set of Symmetrix command line interface (SYMCLI) commands. Solutions Enabler has added support for 128 RDF groups and 32 RDF groups per director for arrays running Enginuity 5772
Composite Group
Data Data
R1/BCV
RDF Links
R2
Logs Logs
Data Data
R2
Logs Logs
Composite groups are similar to device groups. They can be type R1, R2 or Regular, and used for every action that are available with device groups. Composite groups can span Symmetrix arrays. Thus, a host running a database application spanning two Symmetrix arrays can use a composite group to perform a consistent TimeFinder split of the application data. If the type of the Composite Group is RDF1 or RDF2, the CG (Composite Group) can span RDF groups. Composite groups are a requirement for building SRDF consistency in SRDF/S and SRDF/A. When a Composite Group is created for SRDF consistency the rdf_consistency option must be specified at the group creation time.
A write I/O is not be issued by an application until a prior related write I/O is completed
A logical dependency, not a time dependency Inherent in all Database Management Systems (DBMS)
Page (data) write is dependent write I/O based on a successful log write
Power failures create a dependent write consistent image Restart transforms dependent write consistent to transactionally consistent
Ensuring dependent write consistency is the basic principle behind all of EMCs Consistency Technology solutions
SRDF Consistency Groups TimeFinder Consistent Split Consistent SNAPs and Consistent Clones SRDF/A Open Replicator for Symmetrix
SRDF and Consistency Technology - 18
Almost all commercial applications, such as databases, are inherently consistent by design. EMCs consistency technology makes it possible for the consistency to be maintained when replicas of the production data are made. All logging database management systems use the consistency principles described on this slide to maintain integrity. This is required for the protection against local power outages, loss of local channel connectivity, or storage devices. There is a logical dependency between I/Os built into database management systems, certain applications, middleware tools such as MQ Series, and operating systems. EMC can create a dependent write consistent local image with its TimeFinder family of products, whereas SRDF Consistency Groups and SRDF/A create a consistent image on one or more remote Symmetrix arrays.
SRDF Serialization
Writes to Target volumes must happen in the same order as they are written to the Source in order to have an instance in time, consistent and recoverable copy In Synchronous, and Asynchronous modes, writes are sent to the remote Symmetrix in the order received
If the remote Symmetrix is not accessible, writes are accumulated as invalid tracks When the remote Symmetrix becomes available, invalid tracks are sent without regard to serialization
Serialization maintains the order in which writes are received at the remote (target) Symmetrix. SRDF serialization must be maintained in order to have a recoverable/restartable copy of data at a target site. Through serialization, write fidelity is guaranteed. In normal operations, SRDF maintains order writes with Synchronous, Semi-synchronous, and Asynchronous modes. But when the link becomes unavailable for any reason, writes accumulate as invalid tracks which the application continues to function on the host. When the link is restored, the Adaptive Copy mode is used to propagate changes across the link. This introduces risk, since serialization is not maintained with Adaptive Copy.
R2 R2
RDF Enginuity Consistency Assist (RDF-ECA) provides consistency protection for synchronous mode devices by performing suspend operations across all SRDF/S devices in a consistency group or a named subset of all devices in a composite group. SRDF/S with RDF-ECA is supported by an RDF daemon that performs monitoring and cache recovery operations across all SRDF/S sessions in the group. If one or more source (R1) devices in an SRDF/S consistency group cannot propagate data to their corresponding target (R2) devices, the RDF daemon suspends data propagation from all R1 devices in the consistency group, halting all data flow to the R2 targets. This ensures that a consistent R2 data copy of the database exists at the point-in-time any interruption occurs. The RDF daemon monitors data copy operations and coordinates the suspension of R1 to R2 data propagation if the consistency protection is suspended (tripped). A composite group must be created using the RDF consistency protection option (-rdf_consistency) and must be enabled using the symcg enable command before the RDF daemon begins monitoring and managing the RDF-ECA consistency group.
Concurrent SRDF
One R1 can be paired with two R2 devices, concurrently Each of the two concurrent mirrors must belong to different RA groups
RDF Group 1
M1 M2
R2
M3 M4
R1
M1 M2 M3 M4
Remote Site B
BCV BCV
Remote Site C
Concurrent SRDF allows two remote SRDF mirrors of a single R1 device, e.g. use one remote copy for disaster recovery, and another for decision support or backup. Each Remote Link Director is assigned to an RA Group. With ESCON, only one RA group per RLD is allowed, but Fibre Channel SRDF RA Groups can be defined to the same RLD. Any mixture of SRDF modes is allowed, except for Sync and Semi-sync configuration and Async and Async configuration. A write IO from the host at the primary device side cannot be returned as completed until both remote Symmetrix signal the local Symmetrix that the SRDF IO is in cache at the remote side. 1 Sync and 1 Adaptive Copy remote mirror: The SRDF IO from the secondary device operating in Synchronous mode must present ending status to the sending Symmetrix before a second host IO can be accepted. The host I/O does not wait for the secondary device operating in Adaptive Copy mode.
A BCV can only be established with one of the Target volumes, not both. In case the source is locally protected, the BCV device cannot be established with its source, because all four(4) mirror positions will be occupied 2 Synchronous remote mirrors : A write IO from the host at the primary device side cannot be returned as completed until both remote Symmetrix signal the local Symmetrix that the SRDF IO is in cache at the remote side. 1 Sync and 1 Adaptive Copy remote mirror: The SRDF / IO from the secondary device operating in Synchronous mode must present ending status to the sending Symmetrix before a second host IO can be accepted. The host I/O does not wait for the secondary device operating in Adaptive Copy mode.
1
001 The Source environment Is now the Target Site A becomes Site B 054
R1 R1
2 001
R2 R2
054
R2 R2
switch
R1 R1
1 2
The above command will swap the R1/R2 personality pairs in device group testdg, resulting in the source environment switching from Site A and becoming Site B.
SRDF and Consistency Technology - 23
An R1/R2 personality swap (or R1/R2 swap) refers to when the RDF personality of the RDF device designations of a specified device group are swapped so that source R1 device(s) become target R2 device(s) and target R2 device(s) become source R1 device(s). Dynamic RDF swaps are available with Enginuity version 5567 or later. To perform an R1/R2 swap, you must have an SRDF license with Symmetrix 5567 microcode or higher and Dynamic RDF must be enabled in your Symmetrix configuration. Sample scenarios for R1/R2 Swap - Symmetrix Load Balancing In todays rapidly changing computing environments, it is often necessary to deploy applications and storage on a different Symmetrix without having to give up disaster protection. R1/R2 swap can enable this redeployment with minimal disruption, while offering the benefit of load balancing across two Symmetrix storage arrays. - Primary Data Center Relocation Sometimes a primary data center needs to be relocated to accommodate business practices. For example, several financial institutions in New York City routinely relocate their primary data center across the Hudson River to New Jersey as part of their disaster drills. R1/R2 swaps allow these customers to run their primary applications in their New Jersey data centers. The Manhattan data centers now act as the disaster protection site. - Post-Failover Temporary Protection Measure If the hosts on the source side are down for maintenance, R1/R2 swap permits the relocation of production computing to the target site without giving up the security of remote data protection. When all problems have been solved on the local Symmetrix, you have to failover again and swap the personality of the devices to go back to the original configuration.
Module Summary
Key points covered in this module: EMC SRDF functionality and its uses Concepts of SRDF SRDF Link configurations SRDF modes of operation Concept of RDF Device groups, composite groups and a consistency group Characteristics of a Dynamic SRDF environment Characteristics of a Concurrent SRDF environment Enginuity Consistency Assist (ECA) Role of ECA in managing SRDF Consistency Groups SRDF personality swap and possible business scenarios for its use
2008 EMC Corporation. All rights reserved. SRDF and Consistency Technology - 24
These are the key points covered in this module. Please take a moment to review them.