Professional Documents
Culture Documents
SRDF Overview Latency v.5
SRDF Overview Latency v.5
Deutsche Bank
VERSION 1.2
Page 1 of 21
Introduction.............................................................................................................................................. E!ecuti"e #"er"ie$..................................................................................................................................% SRDF #"er"ie$........................................................................................................................................& What is SRDF?.......................................................................................................................................6 Why SRDF?...........................................................................................................................................6 How does SRDF work?..........................................................................................................................6 SRDF Storage Protocol used by Deutsche a!k..................................................................................." Where SRDF is used at Deutsche a!k................................................................................................." D'DM Tec(nology.................................................................................................................................) #$er$iew................................................................................................................................................% &ortel &etworks #P'era (etro.............................................................................................................% *atency....................................................................................................................................................+, SRDF i!duced delays...........................................................................................................................1) *+a,-le................................................................................................................................................11 Reco,,e!datio!s for Ha!dli!g High .cti$ity Data...........................................................................12 Types of applications t(at may not -e suited to sync(ronous replication........................................+ SRDF -est practices in use at Deutsc(e .ank.....................................................................................+% /lternati"e strategies.............................................................................................................................+0 1/ SRDF Se,i0Sy!chro!ous ,ode .....................................................................................................11 2/ SRDF .da-ti$e 2o-y ,ode ...........................................................................................................16 3/ SRDF (ulti0Ho- ,ode...................................................................................................................1" 4/ #racle%5 .uto,ated Sta!dby Database...........................................................................................16 2o!clusio!............................................................................................................................................21
Page 2 of 21
Introduction
Recent engineering work by EMC has validated the Nortel Optera DWDM (Dense Wave Division Multiple er! "or use with EMC #RD$ to a distance o" %&&'M( )his docu*ent e plains which applications can bene"it "ro* this e tended distance *irroring+ and which cannot( It also o""ers alternative solutions to support the protection o" data i" the application cannot support e tended distance *irroring(
Page 3 of 21
E ecutive Overview
EMC Engineering has recently validated the Nortel Optera DWDM "or use with EMC #RD$ up to %&&'M( DWDM technology allows "or the ,packaging up- o" *ultiple #RD$ links into a s*aller nu*ber o" physical teleco**s "ibre cable thus reducing the nu*ber o" teleco**s "ibre cables re.uired without reducing the bandwidth or e""icency( )his leads to greatly reduced costs( )he Nortel Optera DWDM is not the DWDM currently selected by Deutsche /ank ,New World-( It is envisaged that this %&&'M distance will be increased very shortly a"ter "urther validation by EMC engineering( EMC #RD$ works by "orwarding write IO "ro* a host to a #y**etri onto a second re*ote #y**etri ( )his is done transparently to the host+ which only ,sees- a slightly slower write IO( During nor*al non0/C1 operation+ only write IO is sent to the re*ote #y**etri ( 2 general rule o" thu*b is an application does 3&0345 reads and only 6&045 writes( In order to calculate the additional latency we have to add the "i ed ,overheado" writing to % #y**etri units+ as opposed to 6+ plus a variable value according to distance to be replicated( )he variable value is proportionate to the speed o" light+ and it is not envisaged that EMC engineering will be able to i*prove on this in the near "uture( )he DWDM units and Connectri switch units re.uired have negligible overhead( )his overhead is per write IO( 2pplications with a heavy write IO+ or during batch runs+ *ay e perience ,IO 7ueuing- as write IOs .ueue to be sent across the #RD$ link to the second #y**etri ( 8arious host best practises can greatly reduce the potential "or IO 7ueuing+ and these are in use by Deutsche /ank( #RD$ can operate in a nu*ber o" *odes+ and these *odes can be interactively switched on a very granular basis( )hese di""erent *odes can again greatly reduce or even eli*inate IO 7ueuing( )hirdly+ #RD$ in con9unction with EMC )i*e$inder can be architected into various ,*ulti0hop- and ,ti*e sync enabled- solutions( In su**ary+ #RD$ can be e tended across greater distances than previously possible using DWDM technology+ but this e tra distance is not without cost to the e""iciency o" the applications using it( Detailed e a*ination o" the service levels re.uired and the IO pro"ile o" the application *ust be e a*ined to see i"
Page 4 of 21
it is practical to use #RD$ over e tended distances in synchronous *ode( I" the application is not suitable "or synchronous *ode #RD$ over the distance then there are other architected solutions available which *ay provide the re.uired level o" protection(
Page 1 of 21
#RD$ Overview
What is SRDF?
#RD$ generates a *irror i*age o" the data at the logical volu*e level in one or *ore re*ote #y**etri syste*s( )hese re*ote volu*es can be *ade addressable to re*ote hosts via so"tware co**ands( #RD$ #ynchronous *ode (which is the de"ault *ode o" operation at Deutsche /ank! was "irst developed "or Disaster Recovery within the custo*er-s ca*pus( #RD$ 2daptive Copy *odes were later developed to support long distance bulk data trans"ers "or data center relocations and content replication( )echnology has evolved to support Wide 2rea Networking (W2N! and *ultiple transports+ thus increasing distance and throughput "or a wider variety o" applications o" #RD$( 2dditional custo*er uses "or #RD$ include re*ote data warehousing+ re*ote test beds+ re*ote report generation+ re*ote backup and workload sharing between hosts at the sa*e or geographically re*ote sites(
Why SRDF?
#RD$ is deployed in several key areas+ delivering real bene"its to their organi:ations allowing co*panies to *aintain access to data+ so that revenue producing or supporting applications continue to serve business "unctions( #RD$ can be used in several key areas including+ but not li*ited to; Business continuance: business applications continue running despite possible disk "ailures( Disaster recovery: data recovery at the disaster recovery site in *inutes rather than days( Data centre migrations: application outage reduced to *inutes instead o" hours( Work load migrations: si*ilar to the data centre *igrations< especially use"ul "or *ini*i:ing outages during preventative *aintenance o" hardware or so"tware+ or even data center powerdowns( Shortening or eliminating backup windows: eli*inate the backup window by utili:ing #RD$-s second data copy(
#RD$ writes are "ro* cache to cache+ hence when data is written "ro* local #y**etri cache to re*ote #y**etri cache over the #RD$ link+ the
Page 6 of 21
production #y**etri waits "or an acknowledge*ent "ro* the re*ote #y**etri be"ore data is written to local disk(
Page " of 21
DWDM )echnology
#verview
Dense Wavelength Division Multiple ing (DWDM! is a process in which *ultiple di""erent or *ultiple individual channels o" data are carried at di""erent wavelengths over one pair o" "iber links( )his contrasts to conventional "iber optic syste*s in which 9ust one channel is carried over a "iber pair( $or EMC custo*ers this *eans that *ultiple #RD$ channels and server channels can be trans"erred over one pair o" "iber links along with traditional network tra""icC )his is especially i*portant in locations where "iber links are at a pre*iu*( $or e a*ple+ a custo*er *ay be leasing "iber+ so the *ore tra""ic they can run over a single link+ the *ore cost e""ective the solution( With today-s technology+ the capacity o" a single pair o" "iber strands is virtually unli*ited( )he li*itation co*es "ro* the DWDM itsel"( Optical to electrical trans"ers "or switching and channel protection are re.uired and li*it the input tra""ic per channel( #RD$ over $ibre Channel does not currently support direct connections between R$ directors using WDM or DWDM unit port connections+ due to per"or*ance li*itations and the relatively variable latencies o" such links over long distances( DWDM units+ however+ are supported "or #RD$ tra""ic via I#B connections using $ibre Channel switches such as the Connectri "a*ily o" $ibre Channel switches(
Page % of 21
Nortel Networks O1)era Metro provides the ability to route wavelengths+ and there"ore has the sa*e survivability capabilities as current )DM rings when deployed in a ring topology( O1)era Metro provides a reliable DWDM plat"or* "or enterprises with large0scale connectivity re.uire*ents( O1)era-s transparent capabilities enable these enterprises to control the cost and
DWD&
Fiber ( to )* wavelengths
Dense wavelength division *ultiple ing 2cts as an Ioptical "unnelJ 8 Multiple protocol independent strea*s on a single "iber0 optic cable pair 8 Each wavelength represents a uni.ue strea* o" data 8 which *ay have a di""erent data rate
*anage*ent re.uire*ents o" connectivity+ ensure network integrity+ Increase network robustness+ and easily acco**odate e*erging co**unications protocols( Features and Bene'its F#upport o" #ONE)>#DA and non0#ONE)>#DA inter"aces F1rotocol and bit0rate independence F=% protected wavelengths+G@ unprotected wavelengths F1 r0wavelength "le ible protection switching F#calable "ro* 6G Mbps to %(4 Dbps per wavelength F1oint0to0point and survivable ring up to 6%&k* FIn0band+ per wavelength Optical #ervice Channel F1oint and click DHI *anage*ent syste* FOpen syste*s *anage*ent plat"or* FNE/# and E)#I co*pliant
Page 6 of 21
Batency
SRDF induced delays
#ynchronous or even se*i0synchronous *irroring o" data can cause i*pacts to custo*er workloads( )he i*pact to any given workload will vary according to; 0 )he blocksi:e o" the data being re*ote *irrored 0 )he distance over which the re*ote *irroring is being done 0 )he re*ote *irroring *ode used (e(g(( #ynchronous+ se*i0 synchronous+ adaptive copy! 0 )he type o" connection between the source and target #y**etri units 0 )he arrival rate o" the write IOs at the source #y**etri )he degree to which a custo*er workload is i*pacted by delays induced by #RD$ *irroring will not only vary according to the a*ount o" the delay+ but also due to the nature o" the workload( #o*e workloads will not be i*pacted by e tended response ti*es on workload co*ponents that are critical "or recovery( Other workloads could be severely i*pacted i" the a""ected co*ponent is on the critical path "or end user transaction response ti*e( (e(g(( 2n increase in response ti*e to the online Redo logs in an Oracle environ*ent will invariably cause end user transaction response ti*e to degrade(! In order to appro i*ate the a*ount o" delay likely to be introduced by #RD$-ing the data "or any given workload+ one should; 0 Deter*ine the type o" #RD$ i*ple*entation that is likely to be installed 0 Calculate the propagation delay induced by the link (calculated by *ultiplying the round trip link distance in kilo*etres by &(&&4 *sec>k*+ and then by = i" ca*pus E#CON is to be used+ or by 6 i" a telco link (e(g( )=+ 2)M+ etc! is to be used+ or by % "or #RD$ over $ibre Channel( )o this it will be necessary to add an allowance "or protocol ti*e within the both the source and target #y**etri + as well as allowances "or delays induced by protocol converters+ network e.uip*ent+ etc(! 0 2dd the appro i*ated #RD$ link delay ti*es to the current or anticipated non #RD$-ed IO response ti*es( 0 Deter*ine the likely i*pact on the custo*er workload+ re*e*bering that the i*pact will inevitably "ollow Bittle-s Baw 6(
Bittle-s Baw is the basis upon which a lot o" .ueuing theory is built( In general ter*s+ Bittle-s Baw relates the average .ueue length (Q! to the arrival rate o" transactions (a! and the average response ti*e (R!( #peci"ically+ Bittle-s Baw states; Q = a * R( Conse.uently+ it can be seen that any increase in IO response ti*e *ay well cause a signi"icant blowout in the .ueue length within the application+ which *ay or *ay not be supportable "ro* a custo*er business perspective( Page 1) of 21
+,ample
)his docu*ent is concentrating on #RD$ over $ibre Channel( Write IO is trans*itted using #C#I over $ibre Channel+ and so according to the #C#I protocol every IO to be trans*itted actually re.uires % round trips< the "irst is the #C#I co**and word ("or #RD$ this will be WRITE!+ the re*ote #y**etri then returns the acknowledge*ent( )he second trip is "or the actual data+ "ollowed by the acknowledge*ent "ro* the re*ote #y**etri that the data has been written to cache and con"ir*ed( )his leads to the K% propagation delay described above(
2.1(S
3.6(S
)he picture above illustrates the host response ti*e without #RD$ ( Baseline!+ and the overhead o" running #RD$ over :ero distance ( Campus! "or @' and %L' blocksi:e( Working through a @' blocksi:e e a*ple+ we have a %(&M# host response ti*e "or :ero distance( 2dd to this a 6&&'M distance M the appro i*ate distance "ro* Bondon to Milton 'eynes 0 ((6&&'M N 6&&'M N 6&&'M N 6&&'M!O&(&&4!P%(& M a total o" @M# response ti*e per write IO( Aeavy write activity on 6 volu*e *ay *ean that IOs are .ueued waiting "or the previous IO to be acknowledged "ro* the re*ote #y**etri + and so you *ay get IO elongation+ with IOs waiting on IOs on IOs (see Littles Law above!( Note; )here is no signi"icant Batency through #witches or DWDMs
Page 11 of 21
Page 12 of 21
Page 13 of 21
Page 14 of 21
2lternative strategies
)he latency overhead can also be *asked "ro* the user i" an alternative replication strategy is adopted na*ely+ #e*i #ynchronous or Multi0Aop replication( 2nother strategy would be co*bining the bene"its o" #RD$ with an Oracle auto*ated standby database( )his solution re.uires only that the online redo logs be synchronously replicated+ thus drastically reducing co**unication needs( )he "ollowing strategies could help alleviate latency overhead with #RD$ deployed over e tended distances(
1 % +
SRDF links
Source
'arget
'arget behi!d at ,ost o!e write o-eratio! -er source logical $olu,e
Page 11 of 21
Page 16 of 21
Page 1" of 21
3ocal
Hop-
Hop/
EMC
EM C
EM C
S 2M M ETRI3
S 2M M ET RI3
S 2M M ETRI3
R/ R-
B02
RB02
-
R/
is another approach to the issues introduced by distance0based latency( Aere+)i*e$inder is used to create a point0in0ti*e /C8 o" the production volu*e( #RD$ Multi0Aop would then treat the /C8 as an R6 or source device( Its R% target would be at the other end o" the link( In Multi0Aop scenarios+ the links between the "irst location and the inter*ediate location are run synchronously( )hen the )i*e$inder so"tware per"or*s the splits described above( )he links between the inter*ediate site and the distant site are usually 2daptive Copy *ode due to the issues o" latency( Multi0Aop is the best o" both worlds; "ully synchronous "or per"or*ance between sites 2 and / but 2daptive Copy to keep line costs down between / and C+ the disaster recovery site(
Page 1% of 21
"rimary DB
Failover DB
Page 16 of 21
0onclusion
*(2 *!gi!eeri!g has $alidated the &ortel #-tera DWD( for use with *(2 SRDF u- to 2))<( i! a -oi!t0to0-oi!t co!figuratio!. For Deutsche a!k to re-licate data i! a Sy!chro!ous co-y ,ode betwee! sites= careful co!sideratio! ,ust be gi$e! as to whether the !ature a!d characteristics of the a--licatio! are suited to a Sy!chro!ous co-y ,ode co!figuratio!= or whether the a--licatio! user res-o!se ti,es will be ad$ersely effected by the late!cy issues described i! this docu,e!t. 5f a! a--licatio! or its co,-o!e!ts e+hibit high 5;# writes= or high tra!sactio! rates= the! alter!ati$e SRDF re-licatio! ,odes should be co!sidered to a$oid these late!cy issues.
Page 21 of 21