Professional Documents
Culture Documents
SAN Conceptual and Design Basics
SAN Conceptual and Design Basics
VMware Infrastructure 3
VMwareESXServercanbeusedinconjunctionwithaSAN(storageareanetwork),a specializedhighspeednetworkthatconnectscomputersystemstohighperformance storagesubsystems.UsingESXServertogetherwithaSANprovidesextrastoragefor consolidation,improvesreliability,andhelpswithdisasterrecovery. TouseESXServereffectivelywithaSAN,youreexpectedtobefamiliarwiththeSAN technology.ThiswhitepaperoffersabriefintroductiontosomebasicSANconcepts, butdoesntaimtobeanexhaustivesourceofinformationonSANs.IfyouareanESX ServeradministratorplanningtosetupESXServerhoststoworkwithSANs,you shouldalsoconsultotherresourcesavailableinprintandontheInternettoachievea workingknowledgeofSANconcepts.AdditionalinformationonhowESXServer interactswithSANmaybefoundinSANConfigurationGuide. Thewhitepaperdiscussesthesetopics:
! ! ! ! !
SAN Basics
ASANisaspecializedhighspeednetworkofstoragedevicesandswitchesconnected tocomputersystems.Thiswhitepaperreferstothecomputersystemsasserversor hosts. ASANpresentssharedpoolsofstoragedevicestomultipleservers.Eachservercan accessthestorageasifitweredirectlyattachedtothatserver.ASANsupports centralizedstoragemanagement.SANsmakeitpossibletomovedatabetweenvarious storagedevices,sharedatabetweenmultipleservers,andbackupandrestoredata rapidlyandefficiently.Inaddition,aproperlyconfiguredSANfacilitatesbothdisaster recoveryandhighavailability.
VMware, Inc.
Connections:HostBusAdapters(5)andStorageProcessors(6)Hostservers andstoragesystemsareconnectedtotheSANfabricthroughportsinthefabric.
! !
AhostconnectstoafabricportthroughanHBA. Storagedevicesconnecttofabricportsthroughtheirstorageprocessors.
ESX Server 1
HBA 2 5 4 fabric A
HBA 1 4 fabric B
3 SAN switch 6 SP SP
3 SAN switch
storage array
3 4 5
VMware, Inc.
SAN Components
ThecomponentsofanFCSANcanbegroupedasfollowsandarediscussedbelow:
! ! !
Figure 2showstheSANcomponentlayers.
ESX Server ESX Server
host components
HBA
HBA
HBA
HBA
fabric A
fabric B
fabric components
SAN switch A SAN switch B
SP
SP
SP
SP
storage components
storage array
storage array
Host Components
ThehostcomponentsofaSANconsistoftheserversthemselvesandthecomponents thatenabletheserverstobephysicallyconnectedtotheSAN.
!
VMware, Inc.
Fabric Components
AllhostsconnecttothestoragedevicesontheSANthroughtheSANfabric.The networkportionoftheSANconsistsofthefollowingfabriccomponents:
!
DataRoutersDataroutersareintelligentbridgesbetweenSCSIdevicesandFC devicesintheSAN.ServersintheSANcanaccessSCSIdiskortapedevicesinthe SANthroughthedataroutersinthefabriclayer. CablesSANcablesareusuallyspecialfiberopticcablesthatareusedtoconnect allofthefabriccomponents.ThetypeofSANcableandthefiberopticsignal determinethemaximumdistancesbetweenSANcomponentsandcontributeto thetotalbandwidthratingoftheSAN. CommunicationsProtocolFabriccomponentscommunicateusingtheFC communicationsprotocol.FCisthestorageinterfaceprotocolusedformostof todaysSANs.FCwasdevelopedasaprotocolfortransferringdatabetweentwo portsonaserialI/Obuscableathighspeeds.FCsupportspointtopoint, arbitratedloop,andswitchedfabrictopologies.Switchedfabrictopologyisthe basisformostcurrentSANs.
Storage Components
ThestoragecomponentsofaSANarethestoragearrays.Storagearraysincludestorage processors(SPs).TheSPsarethefrontendofthestoragearray.SPscommunicatewith thediskarray(whichincludesallthedisksinthestoragearray)andprovidethe RAID/LUNfunctionality.
Storage Processors
SPsprovidefrontsidehostattachmentstothestoragedevicesfromtheservers,either directlyorthroughaswitch.TheserverHBAsmustconformtotheprotocolsupported bythestorageprocessor.Inmostcases,thisistheFCprotocol.
VMware, Inc.
Storage Devices
Dataisstoredondiskarraysortapedevices(orboth). DiskarraysaregroupsofmultiplediskdevicesandarethetypicalSANdiskstorage device.Theycanvarygreatlyindesign,capacity,performance,andotherfeatures. Storagearraysrarelyprovidehostsdirectaccesstoindividualdrives.Instead,the storagearrayusesRAID(RedundantArrayofIndependentDrives)technologyto groupasetofdrives.RAIDusesindependentdrivestoprovidecapacity,performance, andredundancy.Usingspecializedalgorithms,severaldrivesaregroupedtoprovide commonpooledstorage.TheseRAIDalgorithms,commonlyknownasRAIDlevels, definethecharacteristicsoftheparticulargrouping. InsimplesystemsthatprovideRAIDcapability,aRAIDgroupisequivalenttoasingle LUN.ALUNisasingleunitofstorage.Dependingonthehostsystemenvironment,a LUNisalsoknownasavolumeoralogicaldrive.FromaVIClient,aLUNlookslike anyotherstorageunitavailableforaccess. Inadvancedstoragearrays,RAIDgroupscanhaveoneormoreLUNscreatedfor accessbyoneormoreservers.TheabilitytocreatemorethanoneLUNfromasingle RAIDgroupprovidesfinegranularitytothestoragecreationprocess.Youarenot limitedtothetotalcapacityoftheentireRAIDgroupforasingleLUN. Note ASANadministratormustbefamiliarwiththedifferentRAIDlevelsandunderstand howtomanagethem.Discussionofthosetopicsisbeyondthescopeofthisdocument.
Moststoragearraysprovideadditionaldataprotectionandreplicationfeaturessuchas snapshots,internalcopies,andremotemirroring.
!
VMware, Inc.
Inthecontextofthisdocument,aportistheconnectionfromadeviceintothe SAN.EachnodeintheSANeachhost,storagedevice,andfabriccomponent (routerorswitch)hasoneormoreportsthatconnectittotheSAN.Portscanbe identifiedinanumberofways: WWPNWorldWidePortName.Agloballyuniqueidentifierforaportwhich allowscertainapplicationstoaccesstheport.TheFCswitchesdiscovertheWWPN ofadeviceorhostandassignaportaddresstothedevice. Port_ID(orportaddress)WithintheSAN,eachporthasauniqueportIDthat servesastheFCaddressfortheport.ThisenablesroutingofdatathroughtheSAN tothatport.TheFCswitchesassigntheportIDwhenthedevicelogsintothe fabric.TheportIDisvalidonlywhilethedeviceisloggedon.
IndepthinformationonSANportscanbefoundathttp://www.snia.org,theWebsite oftheStorageNetworkingIndustryAssociation.
VMware, Inc.
Anactive/activediskarrayallowsaccesstotheLUNssimultaneouslythroughall thestorageprocessorsthatareavailablewithoutsignificantperformance degradation.Allthepathsareactiveatalltimes(unlessapathfails). Inanactive/passivediskarray,oneSPisactivelyservicingagivenLUN.Theother SPactsasbackupfortheLUNandmaybeactivelyservicingotherLUNI/O.I/O canbesentonlytoanactiveprocessor.Iftheprimarystorageprocessorfails,one ofthesecondarystorageprocessorsbecomesactive,eitherautomaticallyor throughadministratorintervention. Usingactive/passivearrayswithpathpolicyFixedcanpotentiallyleadtopath thrashing.SeeESXServerSANConfigurationGuideformoreinformationonresolving paththrashing.
Note
VMware, Inc.
Zoning
ZoningprovidesaccesscontrolintheSANtopology;itdefineswhichHBAscan connecttowhichSPs.YoucanhavemultipleportstothesameSPindifferentzonesto reducethenumberofpresentedpaths. WhenaSANisconfiguredusingzoning,thedevicesoutsideazonearenotvisibleto thedevicesinsidethezone.Inaddition,SANtrafficwithineachzoneisisolatedfrom theotherzones. WithinacomplexSANenvironment,SANswitchesprovidezoning.Zoningdefines andconfiguresthenecessarysecurityandaccessrightsfortheentireSAN. Typically,zonesarecreatedforeachgroupofserversthataccessasharedgroupof storagedevicesandLUNs.Youcanusezoninginseveralways.Herearesome examples:
!
ZoningforsecurityandisolationYoucanmanagezonesdefinedfortesting independentlywithintheSANsotheydontinterferewiththeactivitygoingonin theproductionzones.Similarly,youcouldsetupdifferentzonesfordifferent departments. ZoningforsharedservicesAnotheruseofzonesistoallowcommonserver accessforbackups.SANdesignsoftenhaveabackupserverwithtapeservicesthat requireSANwideaccesstohostserversindividuallyforbackupandrecovery processes.Thesebackupserversneedtobeabletoaccesstheserverstheybackup. ASANzonemightbedefinedforthebackupservertoaccessaparticularhostto performabackuporrecoveryprocess.Thezoneisthenredefinedforaccessto anotherhostwhenthebackupserverisreadytoperformbackuporrecovery processesonthathost. MultiplestoragearraysZonesarealsousefulwhentherearemultiplestorage arrays.Throughtheuseofseparatezones,eachstoragearrayismanaged separatelyfromtheothers,withnoconcernforaccessconflictsbetweenservers.
LUN Masking
LUNmaskingiscommonlyusedforpermissionmanagement.LUNmaskingisalso referredtoasselectivestoragepresentation,accesscontrol,andpartitioning, dependingonthevendor. LUNmaskingisperformedattheSPorserverlevel;itmakesaLUNinvisiblewhena targetisscanned.Theadministratorconfiguresthediskarraysoeachserverorgroup
VMware, Inc.
ofserverscanseeonlycertainLUNs.Maskingcapabilitiesforeachdiskarrayare vendorspecific,asarethetoolsformanagingLUNmasking.
ESX Server Mask LUN 11 ESX Server Mask LUN 12
HBA
HBA
21:00:00:E0:8B:19:B2:33 LUN masking: done at SP or server level; makes a LUN invisible when a target is scanned
FC switch
50:05:01:60:10:20:AD:87 SP LUNs 0 11 storage array 12 WWN (world-wide name) unique, 64-bit address assigned to Fibre Channel node
Requirements
TointegrateallcomponentsoftheSAN,youmustmeetthevendorshardwareand softwarecompatibilityrequirements,includingthefollowing:
! ! !
VMware, Inc.
10
SAN Setup
WhenyourereadytosetuptheSAN,completethesetasks. To prepare the SAN 1 Assembleandcabletogetherallhardwarecomponentsandinstallthe correspondingsoftware. a b c 2 3 Checktheversions. SetuptheHBA. Setupthestoragearray.
DocumenttheSANinstallationandalloperationalprocedures.
VMware, Inc.
11
simplenorquick.ThefirststepindesigninganoptimalSANistodefinethestorage requirementsforeachapplicationintermsof:
! ! ! ! ! !
Caching
ThoughESXServersystemsbenefitfromwritecache,thecachecouldbesaturatedwith sufficientlyintenseI/O.Saturationreducesthecacheseffectiveness. Becausethecacheisoftenallocatedfromaglobalpool,itshouldbeallocatedonlyifit willbeeffective.
!
VMware, Inc.
12
RedundantSANComponentsRedundantSANhardwarecomponents includingHBAs,SANswitches,andstoragearrayaccessports,arerequired.In somecases,multiplestoragearraysarepartofafaulttolerantSANdesign. RedundantI/OPathsI/Opathsfromtheservertothestoragearraymustbe redundantanddynamicallyswitchableintheeventofaport,device,cable,orpath failure. I/OConfigurationThekeytoprovidingfaulttoleranceiswithinthe configurationofeachserversI/Osystem. WithmultipleHBAs,theI/OsystemcanissueI/OacrossalloftheHBAstothe assignedLUNs.Failurescanhavethefollowingresults:
!
VMware, Inc.
13
VMware, Inc. 3145 Porter Drive Palo Alto, CA 94304 www.vmware.com Copyright 1998-2006 VMware, Inc. All rights reserved. Protected by one or more of U.S. Patent Nos. 6,397,242, 6,496,847, 6,704,925, 6,711,672, 6,725,289, 6,735,601, 6,785,886, 6,789,156, 6,795,966, 6,880,022 6,961,941, 6,961,806 and 6,944,699; patents pending. VMware, the VMware boxes logo and design, Virtual SMP and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. All other marks and names mentioned herein may be trademarks of their respective companies. Revision yyyymmdd Version: x.y Item: TBD
14