Setup for Microsoft Cluster Service

Update 2 and later for ESX Server 3.5, ESX Server 3i version 3.5, VirtualCenter 2.5

Setup for Microsoft Cluster Service

Setup for Microsoft Cluster Service Revision: 20090814 Item: EN-000081-03

You can find the most up-to-date technical documentation on our Web site at: http://www.vmware.com/support/ The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: docfeedback@vmware.com

20072009 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware, the VMware boxes logo and design, Virtual SMP and VMotion are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.

VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

2 VMware, Inc.

Contents

AboutThisBook

1 GettingStarted 9
Introduction 9 ClusteringSoftware 10 ClusteringHardware 10 ClusteringConfigurations 10 ClusteringVirtualMachinesonaSingleHost(ClusterinaBox) 10 ClusteringVirtualMachinesAcrossPhysicalHosts(ClusterAcrossBoxes) ClusteringPhysicalMachineswithVirtualMachines(Standby Host) 13 PrerequisitesforClustering 13 PrerequisitesforClusterinaBox 14 PrerequisitesforClusteringAcrossBoxes 14 PrerequisitesforStandbyHostClustering 15 SharedStorageSummary 15 Caveats,Restrictions,andRecommendations 16 RecommendationsforUsingMSCSandBootfromSAN 17 SettingupaClusteredContinuousReplicationEnvironmentforMicrosoft Exchange 18

11

2 ClusteringVirtualMachinesonOnePhysicalHost 19
Task1:CreatingtheFirstNode 19 Task2:CreatingtheSecondNode 20 Task3:AddingHardDiskstoNode1 21 Task4:AddingHardDiskstoNode2 24

3 ClusteringVirtualMachinesAcrossPhysicalHosts 25
Task1:CreatingtheFirstNode 25 Task2:CreatingtheSecondNode 27 Task3:AddingHardDiskstoNode1 28 Task4:AddingHardDiskstoNode2 30

VMware, Inc.

Setup for Microsoft Cluster Service

4 ClusteringPhysicalandVirtualMachines 33
Task1:CreatingtheFirstNode 33 Task2:CreatingtheSecondNode 34 Task3:InstallingMicrosoftClusterService 36 Task4:CreatingAdditionalPhysical/VirtualPairs 36

5 UpgradingClusteredVirtualMachines 37
LegacyClusterSetupOptions 37 UpgradingClusterinaBox(CIB) 38 UpgradingCIB:SharedRDMsandBootDisksinSeparateVMFSVolumes 38 UpgradingCIB:RDMsandBootDisksinSameVMFSVolume 39 UpgradingCIB:VirtualDisks 40 UpgradingClusterAcrossBoxes 40 UsingSharedPassThroughRDMs 40 UpgradingaClusterwithFilesinSharedVMFS2Volumes 41 UpgradingClustersUsingPhysicaltoVirtualClustering 42

Appendix:SetupChecklist Index 47

43

VMware, Inc.

About This Book

Thisbook,SetupforMicrosoftClusterService,firstdiscussesthetypesofclustersyoucan implementusingvirtualmachineswithMicrosoftClusterService.Itthengives stepbystepinstructionsforeachtypeofcluster,andconcludeswithachecklistof clusteringrequirementsandrecommendations. SetupforMicrosoftClusterServicecoversbothESXServer3.5andESXServer3i version 3.5.Foreaseofdiscussion,thisbookusesthefollowingproductnaming conventions:

FortopicsspecifictoESXServer3.5,thisbookusesthetermESXServer3. FortopicsspecifictoESXServer3iversion3.5,thisbookusestheterm ESX Server 3i. Fortopicscommontobothproducts,thisbookusesthetermESXServer. Whentheidentificationofaspecificreleaseisimportanttoadiscussion,thisbook referstotheproductbyitsfull,versionedname. WhenadiscussionappliestoallversionsofESXServerforVMware Infrastructure 3,thisbookusesthetermESXServer3.x.

Intended Audience
ThisbookisforsystemadministratorswhoarefamiliarwithbothVMwaretechnology andMicrosoftClusterService. NOTEThisisnotaguidetousingMicrosoftClusterService.UseyourMicrosoft documentationforinformationoninstallationandconfigurationofMicrosoftCluster Service.

VMware, Inc.

Setup for Microsoft Cluster Service

Document Feedback
VMwarewelcomesyoursuggestionsforimprovingourdocumentation.Ifyouhave comments,sendyourfeedbackto: docfeedback@vmware.com

VMware Infrastructure Documentation

TheVMwareInfrastructuredocumentationconsistsofthecombinedVMware VirtualCenterandESXServerdocumentationset.

Abbreviations Used in Figures

ThefiguresinthisbookusetheabbreviationslistedinTable 1. Table 1. Abbreviations
Abbreviation FC SAN VM# Description FibreChannel Storageareanetworktypedatastoresharedbetweenmanagedhosts Virtualmachinesonamanagedhost

Technical Support and Education Resources

Thefollowingsectionsdescribethetechnicalsupportresourcesavailabletoyou. To accessthecurrentversionsofthisbookandotherbooks,goto: http://www.vmware.com/support/pubs.

Online and Telephone Support

Useonlinesupporttosubmittechnicalsupportrequests,viewyourproductand contractinformation,andregisteryourproducts.Goto: http://www.vmware.com/support Customerswithappropriatesupportcontractsshouldusetelephonesupportforthe fastestresponseonpriority1issues.Goto: http://www.vmware.com/support/phone_support.html

VMware, Inc.

About This Book

Support Offerings
FindouthowVMwaresupportofferingscanhelpmeetyourbusinessneeds.Goto: http://www.vmware.com/support/services

VMware Education Services

VMwarecoursesofferextensivehandsonlabs,casestudyexamples,andcourse materialsdesignedtobeusedasonthejobreferencetools.Formoreinformationabout VMwareEducationServices,goto: http://mylearn1.vmware.com/mgrreg/index.cfm

VMware, Inc.

Setup for Microsoft Cluster Service

VMware, Inc.

Getting Started

Thischapterintroducesclustering,discussesthedifferenttypesofclustersand prerequisitesforeachtype,andincludessomecaveatsandrecommendationsinthe followingsections:

Introductiononpage 9 ClusteringConfigurationsonpage 10 PrerequisitesforClusteringonpage 13 Caveats,Restrictions,andRecommendationsonpage 16 RecommendationsforUsingMSCSandBootfromSANonpage 17 SettingupaClusteredContinuousReplicationEnvironmentforMicrosoft Exchangeonpage 18

Introduction
Thisdocumentdiscussestraditionalclustering(hotstandby)usingMSCSinaVMware Infrastructureenvironment.Clusteringvirtualmachinescanreducehardwarecostsof traditionalhighavailabilityclusters. VMwarealsosupportsacoldstandbyclusteringsolutionusingVMwareHAin conjunctionwithVirtualCenterclusters.VMwareHAfunctionality,aswellasthe differencesbetweenthetwoapproaches,isdiscussedintheResourceManagementGuide.

VMware, Inc.

Setup for Microsoft Cluster Service

Anumberofdifferentapplicationsuseclustering:

Statelessapplications,suchasWebserversandVPNservers. Applicationsthathavebuiltinrecoveryfeatures,suchasdatabaseservers,mail servers,andfileservers. VirtualCenterServercanbeusedasaclusteredapplication.See http://www.vmware.com/pdf/VC_MSCS.pdf.

Clustering Software
Severalclusteringsoftwareproductscanbeusedinconjunctionwithvirtualmachines. However,VMwaretestsclusteringonlywithMSCSandsupportsonlyMSCS.

Clustering Hardware
Atypicalclusteringsetupincludes:

Disksthataresharedbetweennodes.Ashareddiskisrequiredasaquorumdisk. Inaclusteracrossboxes,theshareddiskmustbeonanFCSAN. Aprivateheartbeatnetworkbetweennodes.

Clustering Configurations
SeveralclusteringconfigurationsarepossibleinaVMwareInfrastructureenvironment andarebrieflydiscussedbelow:

ClusteringVirtualMachinesonaSingleHost(ClusterinaBox)onpage 10 ClusteringVirtualMachinesAcrossPhysicalHosts(ClusterAcrossBoxes)on page 11 ClusteringPhysicalMachineswithVirtualMachines(Standby Host)onpage 13

Clustering Virtual Machines on a Single Host (Cluster in a Box)

AclusterinaboxconsistsoftwoclusteredvirtualmachinesonthesameESXServer hostconnectedtothesamestorage(eitherlocalorremote).SeeFigure 11foran example.

10

VMware, Inc.

Chapter 1 Getting Started

Figure 1-1. Cluster in a Box

private network virtual machine Node1 cluster software physical machine public network virtual machine Node2 cluster software

storage (local or SAN)

Thisconfigurationprotectsagainstfailuresattheoperatingsystemandapplication level,butitdoesnotprotectagainsthardwarefailures. Chapter 2,ClusteringVirtualMachinesonOnePhysicalHost,discusseshowtoset upaclusterinaboxusingMSCS.

Clustering Virtual Machines Across Physical Hosts (Cluster Across Boxes)

Aclusteracrossboxesconfigurationprovidesbothhardwareandsoftwarelevel protectionbyplacingtheclusternodesonseparateESXServerhosts,asshownin Figure 12.ThisconfigurationrequiressharedstorageonanFCSANforthequorum disk. Thisconfigurationprotectsagainstsoftwarefailuresandhardwarefailuresonthe physicalmachine. Chapter 3,ClusteringVirtualMachinesAcrossPhysicalHosts,discusseshowtoset upaclusteracrossboxesusingMSCS.

VMware, Inc.

11

Setup for Microsoft Cluster Service

Figure 1-2. Cluster Across Boxes

private network virtual machine Node1 cluster software physical machine public network virtual machine Node2 cluster software physical machine

storage (SAN)

Youcanexpandtheclusteracrossboxesmodelandplacemultiplevirtualmachineson multiplephysicalmachines.Forexample,youcanconsolidatefourclustersoftwo physicalmachineseachtotwophysicalmachineswithfourvirtualmachineseach. This setupprotectsyoufrombothhardwareandsoftwarefailures.Atthesametime, thissetupresultsinsignificanthardwarecostsavings. Figure 1-3. Clustering Multiple Virtual Machines Across Hosts

VM1 VM3

VM2 VM4 VM6 VM8 physical machine

VM5 VM7 physical machine

Figure 13showshowfourtwonodeclusterscanbemovedfromeightphysical machinestotwo.

12

VMware, Inc.

Chapter 1 Getting Started

Clustering Physical Machines with Virtual Machines (Standby Host)

Forasimpleclusteringsolutionwithlowhardwarerequirements,youmightchooseto haveonestandbyhost.Setupyoursystemtohaveavirtualmachinecorrespondingto eachphysicalmachineonthestandbyhost,andthencreateclusters,oneeachforeach physicalmachineanditscorrespondingvirtualmachine.Incaseofhardwarefailurein oneofthephysicalmachines,thevirtualmachineonthestandbyhostcantakeoverfor thatphysicalhost. Figure 14showsastandbyhostusingthreevirtualmachinesonasinglephysical machine.Eachvirtualmachineisrunningclusteringsoftware. Figure 1-4. Clustering Physical and Virtual Machines

virtual machine cluster software cluster software

virtual machine cluster software cluster software

virtual machine cluster software physical machine

cluster software

Prerequisites for Clustering

UsingMSCSinanyoftheconfigurationsdiscussedrequirespreparation.Thissection liststheprerequisitesfortheESXServerhostandthevirtualmachine.Foradditional softwareprerequisiteinformation,seetheGuidetoCreatingandConfiguringaServer ClusterunderWindowsServer2003ontheMicrosoftWebsite. Appendix:SetupChecklistonpage 43summarizesprerequisitesfordifferenttypes ofclusters.

VMware, Inc.

13

Setup for Microsoft Cluster Service

Prerequisites for Cluster in a Box

Tosetupaclusterinabox,youmusthave:

ESXServerhost,oneofthefollowing:

ESXServer3AnESXServerhostwithaphysicalnetworkadapterforthe serviceconsole.Iftheclusteredvirtualmachinesneedtoconnectwithexternal hosts,thenanadditionalnetworkadapterishighlyrecommended. ESXServer3iAnESXServerhostwithaphysicalnetworkadapterforthe VMkernel.Iftheclusteredvirtualmachinesneedtoconnectwithexternal hosts,aseparatenetworkadapterisrecommended.

AlocalSCSIcontroller.IfyouplantouseaVMFSvolumethatexistsonaSAN,you needanFCHBA(QLogicorEmulex).

Youcansetupsharedstorageforaclusterinaboxeitherbyusingavirtualdiskorby usingaremoterawdevicemapping(RDM)LUNinvirtualcompatibilitymode (nonpassthroughRDM). Whenyousetupthevirtualmachine,youneedtoconfigure:

Twovirtualnetworkadapters. Aharddiskthatissharedbetweenthetwovirtualmachines(quorumdisk). Optionally,additionalharddisksfordatathataresharedbetweenthetwovirtual machinesifyoursetuprequiresit.Whenyoucreateharddisks,asdescribedinthis document,thesystemcreatestheassociatedvirtualSCSIcontrollers.

Prerequisites for Clustering Across Boxes

Theprerequisitesforclusteringacrossboxesaresimilartothoseforclusterinabox. You musthave:

ESXServerhost.VMwarerecommendsthreenetworkadaptersperhostforpublic networkconnections.Theminimumconfigurationis:

ESXServer3AnESXServerhostconfiguredwithatleasttwophysical networkadaptersdedicatedtothecluster,oneforthepublicandoneforthe privatenetwork,andonenetworkadapterdedicatedtotheserviceconsole. ESXServer3iAnESXServerhostconfiguredwithatleasttwophysical networkadaptersdedicatedtothecluster,oneforthepublicandoneforthe privatenetwork,andonenetworkadapterdedicatedtotheVMkernel.

SharedstoragemustbeonanFCSAN. YoumustuseanRDMinphysicalorvirtualcompatibilitymode(passthrough RDMornonpassthroughRDM).Youcannotusevirtualdisksforsharedstorage.

VMware, Inc.

14

Chapter 1 Getting Started

Prerequisites for Standby Host Clustering

Theprerequisitesforstandbyhostclusteringaresimilartothoseforclusteringacross boxes.Youmusthave:

ESXServerhost.VMwarerecommendsthreenetworkadaptersperhostforpublic networkconnections.Theminimumconfigurationis:

ESXServer3AnESXServerhostconfiguredwithatleasttwophysical networkadaptersdedicatedtothecluster,oneforthepublicandoneforthe privatenetwork,andonenetworkadapterdedicatedtotheserviceconsole. ESXServer3iAnESXServerhostconfiguredwithatleasttwophysical networkadaptersdedicatedtothecluster,oneforthepublicandoneforthe privatenetwork,andonenetworkadapterdedicatedtotheVMkernel.

YoumustuseRDMsinphysicalcompatibilitymode(passthroughRDM). You cannotusevirtualdiskorRDMinvirtualcompatibilitymode (nonpassthroughRDM)forsharedstorage. YoucannothavemultiplepathsfromtheESXServerhosttothestorage. Runningthirdpartymultipathingsoftwareisnotsupported.Becauseofthis limitation,VMwarestronglyrecommendsthatthereonlybeasinglephysicalpath fromthenativeWindowshosttothestoragearrayinaconfigurationof standbyhostclusteringwithanativeWindowshost.TheESXServerhost automaticallyusesnativeESXServermultipathing,whichcanresultinmultiple pathstosharedstorage. UsetheSTORportMiniportdriverfortheFCHBA(QLogicorEmulex)inthe physicalWindowsmachine.

Shared Storage Summary

Table 11illustrateswhichsharedstoragesetupissupportedforwhichclustering solution.Thesetupforeachsolutionisshowninbold. Table 1-1. Shared Storage Summary
Cluster in a Box Virtualdisks PassthroughRDM (physicalcompatibilitymode) NonpassthroughRDM (virtualcompatibilitymode) Yes Yes No Cluster Across Boxes No Standby Host Clustering No

Yes
No

Yes

Yes

VMware, Inc.

15

Setup for Microsoft Cluster Service

Caveats, Restrictions, and Recommendations

Thissectionsummarizescaveats,restrictions,andrecommendationforusingMSCSin aVMwareInfrastructureenvironment.

VMwareonlysupportsthirdpartyclustersoftwarethatisspecificallylistedas supportedinthehardwarecompatibilityguides.ForlatestupdatestoVMware supportforMicrosoftoperatingsystemversionsforMSCS,orforanyother hardwarespecificsupportinformation,seetheStorage/SANCompatibilityGuidefor ESXServer3.5andESXServer3i. EachvirtualmachinehasfivePCIslotsavailablebydefault.Aclusterusesfourof theseslots(twonetworkadaptersandtwoSCSIhostbusadapters),leavingone PCIslotforathirdnetworkadapter(orotherdevice),ifneeded. VMwarevirtualmachinescurrentlyemulateonlySCSI2reservationsanddonot supportapplicationsusingSCSI3persistentreservations. UseLSILogicvirtualSCSIadapter. UseWindowsServer2003SP2(32bitor64bit)orWindows2000ServerSP4. VMwarerecommendsWindowsServer2003. Usetwonodeclustering. ClusteringisnotsupportedoniSCSIorNFSdisks. NICteamingisnotsupportedwithclustering. (SEEUPDATE)ThebootdiskoftheESXServerhostshouldbeonlocalstorage. MixedHBAenvironments(QLogicandEmulex)onthesamehostarenot supported. MixedenvironmentsusingbothESXServer2.5andESXServer3.xarenot supported. ClusteredvirtualmachinescannotbepartofVMwareclusters(DRSorHA). YoucannotusemigrationwithVMotiononvirtualmachinesthatruncluster software. SettheI/Otimeoutto60secondsormorebymodifying HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\ TimeOutValue. ThesystemmightresetthisI/Otimeoutvalueifyourecreateacluster.Youmust resetthevalueinthatcase.

16

VMware, Inc.

Chapter 1 Getting Started

Usetheeagerzeroedthickformatwhenyoucreatedisksforclusteredvirtual machines.Bydefault,theVIClientorvmkfstoolscreatedisksinzeroedthick format.Youcanconvertadisktoeagerzeroedthickformatbyimporting, cloning,orinflatingthedisk.Disksdeployedfromatemplatearealsoin eagerzeroedthickformat. Adddisksbeforenetworking,asexplainedintheVMwareKnowledgeBasearticle athttp://kb.vmware.com/kb/1513.

Recommendations for Using MSCS and Boot from SAN

(SEEUPDATE)Thissectiongivessomerecommendationsforclusteredvirtualmachines thatusebootfromSAN.ForgeneralinformationaboutbootfromSAN,seetheFibre ChannelSANConfigurationGuide. NOTEYoucannotuseclusteredvirtualmachinesthatbootfromaniSCSISAN. BootingfromSANiscomplex.Problemsyouencounterinphysicalenvironments extendtovirtualenvironments.VMwarerecommendsthefollowingwhenyouputthe bootdiskofavirtualmachineyouwishtouseinaclustersetuponaSAN.

ConsiderthebestpracticesforbootfromSANthatMicrosoftpublishesinthe followingknowledgebasearticle: http://support.microsoft.com/kb/305547/en-us

UseStorPortlsilogicdriversinsteadofSCSIportdriverswhenrunningMicrosoft ClusterServiceforWindowsServer2003guestoperatingsystems. VMwaredoesnotrecommendmigrationwithVMotionofclusteredvirtual machines. GiventhecomplexityofbootingclusteredvirtualmachinesfromSAN,VMware recommendsyoutestclusteredconfigurationsindifferentfailoverscenarios beforeyouputthemintoproductionenvironments. Ifyourenvironmentissusceptibletoconditionsthatcauseclusternodeserversto loseallpathstothestoragearray,dothefollowing(forallclusterconfigurations):

Setbussharingforthebootdisk(scsi0)toNone. Setscsi0.returnBusyOnNoConnectStatustoFALSEforeachnode.See Tosetscsi0.returnBusyOnNoConnectStatusonpage 18. Setuptheguestoperatingsystemtorestartautomaticallyafteracrash.See Tosetupautomaticrestartfortheguestoperatingsystemonpage 18.

Whenallpathstostoragearelost,theactivenodewillcrashandattempttoreboot.

VMware, Inc.

17

Setup for Microsoft Cluster Service

To set scsi0.returnBusyOnNoConnectStatus 1 LogintoaVIClientandselectthevirtualmachinefromtheinventorypanel. Theconfigurationpageforthisvirtualmachineappears. 2 3 4 5 6 IntheSummarytab,clickEditSettings. ClickOptions>Advanced>General,andthenclickConfigurationParametersto opentheConfigurationParametersdialogbox. ClickAddRow. Typescsi0.returnBusyOnNoConnectStatusintheNamecolumnandFALSEin theValuecolumn. ClickOKtoclosetheConfigurationParametersdialogbox,andthenclickOK again to close theVirtualMachinePropertiesdialog box.

To set up automatic restart for the guest operating system 1 2 3 RightclickMyComputer. ChooseProperties,thenselecttheAdvancedtabandclickSettingsunderStartup andRecovery. ChooseAutomaticallyrestartonsystemfailure.

Setting up a Clustered Continuous Replication Environment for Microsoft Exchange

Youcansetupaclusteredcontinuousreplication(CCR)environmentforMicrosoft ExchangeinyourVMwareInfrastructureenvironment. MicrosoftdiscussessetupforExchangeServer2007ontheirWebsiteat: http://technet.microsoft.com/en-us/library/bb124558.aspx MicrosoftdiscussessetupofCCRclustersontheirWebsiteat: http://technet.microsoft.com/en-us/library/bb123996.aspx WhenworkinginaVMwareInfrastructureenvironment,youusevirtualmachines insteadofusingphysicalmachinesastheclustercomponents.Usephysical compatibilitymodeRDMs.IfthebootdisksoftheCCRvirtualmachinesareonaSAN, seeRecommendationsforUsingMSCSandBootfromSANonpage 17.

18

VMware, Inc.

Clustering Virtual Machines on One Physical Host

ThischapterguidesyouthroughcreatingatwonodeMSCSclusteronasingle ESX Servermachine.Theprocessconsistsoffourtasks,discussedinthefollowing sections:

Task1:CreatingtheFirstNodeonpage 19 Task2:CreatingtheSecondNodeonpage 20 Task3:AddingHardDiskstoNode1onpage 21 Task4:AddingHardDiskstoNode2onpage 24

NOTEMicrosoftClusterServiceisalreadyinstalledforWindowsServer2003.Seethe GuidetoCreatingandConfiguringaServerClusterunderWindowsServer2003andother documentationontheMicrosoftWebsite. ForWindows2000Server,youmustinstalltheMicrosoftClusterServicesoftware.

Task 1: Creating the First Node

Creatingthefirstnodeconsistsofthesemajorsteps,discussedindetailinthissection.

Creatingthevirtualmachineforthefirstnodewithtwovirtualnetworkadapters. Installingtheoperatingsystem. Poweringdownthefirstnode.

NOTEBeforeyoucreateavirtualmachine,createavirtualdiskineagerzeroedthick formatusingvmkfstools,andselectthatdiskduringvirtualmachinecreation.

VMware, Inc.

19

Setup for Microsoft Cluster Service

To create and configure the first nodes virtual machine 1 2 3 LaunchaVIClientandconnecttotheESXServerhostoraVirtualCenterServer. Usetheusernameandpasswordoftheuserwhowillownthevirtualmachine. Intheinventorypanel,rightclickthehostandchooseNewVirtualMachine. Makethefollowingselectionsusingthewizard. Table 2-1. New Virtual Machine Properties
Page WizardType NameandLocation ResourcePool Datastore GuestOperatingSystem CPUs Memory Network Selection Typical. Chooseaname(forexample,Node1)andlocation. Selecttheresourcepoolforthevirtualmachine,orselectthe hostiftherearenoresourcepools. Chooseadatastoreasthelocationforthevirtualmachine configurationfileandthevirtualmachinedisk(.vmdk)file. ChoosetheWindows2000ServerorWindowsServer2003 operatingsystemthatyouintendtoinstall. Usethedefaultunlessyouhavespecialrequirements. Usethedefaultunlessyouneedadditionalmemoryandyour serversupportsit. ChangeNICstoConnectto2,andselectthesecondnetwork forthesecondNIC. YouneedoneNICfortheprivatenetworkandthesecond NICforthepublicnetwork. VirtualDiskCapacity ReadytoComplete IfyouneedaprimarySCSIdisklargerthan4GB,enterthe appropriatevalueintheCapacityfield. ClickOKtocreatethevirtualmachine.

InstallaWindowsServer2000orWindowsServer2003operatingsystemonthe virtualmachine.

Task 2: Creating the Second Node

CreatingthesecondnodeinvolvescloningtheNode1virtualmachineandaddingdisks thatpointtothesharedstorage.YoucanclonethenodeusingaVIClientconnectedto aVirtualCenterServer,asdescribedbelow,orusingvmkfstools.SeetheServer ConfigurationGuideforareferencetovmkfstools.

20

VMware, Inc.

Chapter 2 Clustering Virtual Machines on One Physical Host

To clone the Node1 virtual machine 1 2 3 Shutdowntheguestoperatingsystemandpoweroffthevirtualmachine. IntheVIClientinventorypanel,selectNode1andchooseClonefromthe rightbuttonmenu. Makethefollowingselectionswiththewizard: Table 2-2. Cloned Virtual Machine Properties
Page NameandLocation ResourcePartition Datastore Customization ReadytoComplete Selection Chooseaname(forexample,Node2)andlocation. Selecttheresourcepoolforthevirtualmachine,orselectthehost iftherearenoresourcepools. Chooseadatastoreasthelocationforthevirtualmachine configurationfileandthe.vmdkfile. ChooseDonotcustomize. ClickOKtocreatethevirtualmachine.

Youhavenowcreatedyoursecondclusternode,avirtualmachinewithtwonetwork adaptersonwhichtheoperatingsystemisinstalled.

Task 3: Adding Hard Disks to Node1

Afteryouhavecreatedtwovirtualmachinesasclusternodes,youarereadytoadda sharedquorumdisk.Youcanalsoaddadditionalshareddiskstotheclusterifyouplan onclusteringadditionaldatadisks.Afteryouhaveaddeddisks,youcanconfigurethe clusterspublicandprivateIPaddresses. To prepare for adding disks Youmustzerooutthedisksyouusewithaclusterinaboxscenario.Youcanuse vmkfstoolstodoso.IfyourunonanESXServer3ihost,youusethevmkfstools RemoteCLI,whichyoumustexecutewithconnectionparameters.SeetheESXServer 3iConfigurationGuideforinformationoninstallingandusingRemoteCLIcommands. CAUTIONWhenyouzerooutadisk,youlosealldata.

VMware, Inc.

21

Setup for Microsoft Cluster Service

Tocreateandzerooutthedisk,usethefollowingcommand: Service Console Remote CLI

vmkfstools -c <size> -d eagerzeroedthick -a lsilogic /vmfs/volumes/<mydir>/<myDisk>.vmdk vmkfstools.pl --server <server_address> --username <user> --password <user_password> -c <size> -d eagerzeroedthick -a lsilogic /vmfs/volumes/<mydir>/<myDisk>.vmdk

Tozerooutanexistingdisk,useathefollowingcommand: Service Console Remote CLI

vmkfstools [-w |--writezeroes] /vmfs/volumes/<mydir>/<myDisk>.vmdk vmkfstools.pl --server <server_address> --username <user> --password <user_password> [-w |--writezeroes] /vmfs/volumes/<mydir>/<myDisk>.vmdk

Repeatthisprocessforeachvirtualdiskyouwanttouseasashareddiskinthecluster. Forexample,ifyouhaveonequorumdiskandonesharedstoragedisk,youmustrun thetoolonbothdisks. To add a quorum disk and optional shared storage disk 1 2 3 SelectthevirtualmachineyoucreatedandchooseEditSettings. ClickAdd,selectHardDisk,andclickNext. SelectChooseanexistingvirtualdiskandselectoneofthedisksyouprepared. SeeToprepareforaddingdisksonpage 21. NOTEYoucanalsouseamappedSANLUNsettovirtualcompatibilitymode. Inthatcase,youdontneedtorunthevmkfstoolscommandslistedinTo prepareforaddingdisks. 4 Chooseanewvirtualdevicenode.Forexample,chooseSCSI(1:0),andusethe defaultmode. NOTEThismustbeanewcontroller.YoucannotuseSCSI0. 5 ClickFinish. ThewizardcreatesbothanewharddiskandanewSCSIcontroller.

22

VMware, Inc.

Chapter 2 Clustering Virtual Machines on One Physical Host

SelectthenewSCSIcontrollerandclickChangeControllerType. MakesurethecontrollerissettoLsiLogic(thedefault).BusLogicisnotsupported whenyouuseMSCSwithESXServer3.0orlater.

Inthesamepanel,setSCSIBusSharingtoVirtualandclickOK.

Ifyourequireadditionalshareddatadisks,repeatStep 1throughStep 6but chooseanewtargetdevice,suchasSCSI(1:1),onthecontrollerthatwasjust created.Figure 21showsyoursetupatthispoint.

Figure 2-1. Cluster in a Box Setup for One Node

NIC1 virtual switch1 VNIC1 virtual switch2 VNIC2

virtual machine Node1 VSCSI1 SCSI1 VSCSI2 SCSI2 FC local or remote storage physical machine

remote storage

VMware, Inc.

23

Setup for Microsoft Cluster Service

Task 4: Adding Hard Disks to Node2

AfteryousetupNode1,repeattheprocesstoconfigureIPaddressesandaddoneor morediskstoNode2.

SetuptheIPaddressessotheprivateandpublicnetworksmatchthoseofNode1. PointthequorumdisktothesamelocationastheNode1quorumdisk.Pointany sharedstoragediskstothesamelocationastheNode1sharedstoragedisks.

IfyouareaddinganRDMorvirtualdisktothesecondnode,chooseUseexistingdisk. CAUTIONIfyoucloneavirtualmachinewithanRDMsetup,allRDMsareconverted tovirtualdisks.UnmapallRDMsbeforecloning,andremapthemaftercloningis complete. ThecompletedsetupisshowninFigure 22. Figure 2-2. Cluster in a Box Complete Setup
NIC1 virtual switch1 (public) virtual switch2 (private) VNIC1 VNIC2 VNIC2 VNIC1

virtual machine Node1 VSCSI1 VSCSI2

virtual machine Node2 VSCSI2 VSCSI1

physical machine

SCSI1

SCSI2 FC

local or remote storage

remote storage

24

VMware, Inc.

Clustering Virtual Machines Across Physical Hosts

ThischapterguidesyouthroughcreatinganMSCSclusterthatconsistsoftwovirtual machinesontwoESXServerhosts. Althoughthisprocessissimilartotheprocessforsettingupaclusterinabox,stepsare repeatedforeaseofuse.Thechapterconsistsofthefollowingsections:

Task1:CreatingtheFirstNodeonpage 25 Task2:CreatingtheSecondNodeonpage 27 Task3:AddingHardDiskstoNode1onpage 28 Task4:AddingHardDiskstoNode2onpage 30

NOTEMicrosoftClusterServiceisalreadyinstalledforWindowsServer2003soyou dontneedtoinstallit.SeetheGuidetoCreatingandConfiguringaServerClusterunder WindowsServer2003andotherdocumentationontheMicrosoftWebsite. ForWindows2000Server,youmustinstalltheMicrosoftClusterServicesoftware.

Task 1: Creating the First Node

Creatingthefirstnodeconsistsofthesemajorsteps,discussedinthissection:

CreatingthevirtualmachineforNode1.SeePrerequisitesforClusteringAcross Boxesonpage 14forrequirements. InstallingtheoperatingsystemonNode1.

NOTEBeforeyoucreateavirtualmachine,createavirtualdiskineagerzeroedthick formatusingvmkfstools.Thenpointtothatdiskduringvirtualmachinecreation.

VMware, Inc.

25

Setup for Microsoft Cluster Service

To create the first nodes virtual machine 1 LaunchaVIClientandconnecttotheVirtualCenterServerthatmanagesthe clustersESXServerhosts. Usetheusernameandpasswordoftheuserwhowilladministerthevirtual machine. 2 3 Intheinventorypanel,rightclicktheESXServerhostandchooseNewVirtual Machine. Makethefollowingselectionswiththewizard: Table 3-1. New Virtual Machine Properties
Page WizardType NameandLocation ResourcePool Datastore GuestOperatingSystem CPUs Memory Network VirtualDiskCapacity ReadytoComplete Selection Typical. Chooseaname(forexampleNode1)andlocation. Selecttheresourcepoolforthevirtualmachine,orthehost iftherearenoresourcepools. Chooseadatastoreasthelocationforthevirtualmachine configurationfileandthe.vmdkfile. ChoosetheWindows2000ServerorWindowsServer2003 operatingsystemyouintendtoinstall. Usethedefaultsuggestedforyouroperatingsystem. Usethedefaultunlessyouneedadditionalmemoryand yourserversupportsit. ChangeNICstoConnectto2,andselectthesecond networkforthesecondNIC. IfyouneedaprimarySCSIdisklargerthan4GB,enterthe appropriatevalueintheCapacityfield. ClickOKtocreatethevirtualmachine.

InstallaWindows2000ServerorWindowsServer2003operatingsystemonthe virtualmachine.

26

VMware, Inc.

Chapter 3 Clustering Virtual Machines Across Physical Hosts

Task 2: Creating the Second Node

CreatingthesecondnodeinvolvescloningtheNode1virtualmachineontoasecond ESXServerhost,addingdiskstothatvirtualmachine,andensuringthatthediskspoint tothestoragesharedwithNode1.YoucanclonethenodebyusingaVIClient connectedtoaVirtualCenterServer,describedinthefollowingprocedure,orbyusing vmkfstools.SeetheServerConfigurationGuideforareferencetovmkfstools. NOTEIfyoucloneavirtualmachinewithRDMs,theRDMsareconvertedtovirtual disksduringtheconversionprocess.RemoveallRDMsbeforecloning,andremapthem aftercloningiscomplete. To clone the Node1 virtual machine 1 2 Shutdowntheguestoperatingsystemandpoweroffthevirtualmachine. IntheVIClientinventorypanel,selectNode1andchooseClonefromthe rightbuttonmenu.Makethefollowingselectionswiththewizard: Table 3-2. Cloned Virtual Machine Properties
Page NameandLocation HostorCluster ResourcePartition Datastore Selection Chooseaname(forexampleNode2)andlocation. Choosethesecondhostfortheclustersetup. Selecttheresourcepoolforthevirtualmachine,orselect thehostiftherearenoresourcepools. Chooseadatastoreasthelocationforthevirtualmachine configurationfileandthe.vmdkfile. Thismustbeadiskineagerzeroedthickformat. Customization ReadytoComplete ChooseDonotcustomize. ClickOKtocreatethevirtualmachine.

Youhavenowcreatedavirtualmachinewithtwonetworkadaptersonwhichthe operatingsystemyouchoseforNode1isinstalled.

VMware, Inc.

27

Setup for Microsoft Cluster Service

Task 3: Adding Hard Disks to Node1

Afteryouhavecreatedthetwovirtualmachineswiththeoperatingsysteminstalled, youneedperformthefollowingtasks:

ConfiguringtheguestoperatingsystemsprivateandpublicIPaddresses. SeethedocumentationfortheMicrosoft2003operatingsystemforconfiguration information.

Addingavirtualharddiskthatissharedbythetwovirtualmachinesasthe quorumdisk,andoptionally,oneormoreshareddatadiskstoNode1. NOTEThesedisksmustpointtoSANLUNs.BothRDMinphysicalcompatibility mode(passthroughRDM)andRDMinvirtualcompatibilitymode (nonpassthroughRDM)aresupported. Theprocedurebelowusesphysicalcompatibilitymode.

To add a quorum disk and optional shared storage disks 1 2 3 SelectthevirtualmachineyoucreatedandchooseEditSettings. ClickAdd,selectHardDisk,andclickNext. IntheSelectaDiskpage,chooseMappedSANLUNandclickNext. YourharddiskpointstoaLUNthatusesRDM. 4 IntheLUNselectionpage,chooseanunformattedLUNandclickNext. AskyourSANadministratorwhichoftheLUNsareunformatted.Youcanalsosee allformattedLUNsinthehostsConfigurationtabanddeducewhichLUNsare unformattedbycomparingthelistofformattedLUNswiththelistintheLUN selectionpage. 5 IntheSelectDatastorepage,selectadatastoreandclickNext. ThisdatastoremustbeonaSANbecauseyouneedasinglesharedRDMfilefor eachsharedLUNontheSAN. 6 SelectPhysicalasthecompatibilitymode,andclickNext. ASCSIcontrolleriscreatedwhenthevirtualharddiskiscreated. 7 Chooseanewvirtualdevicenode,forexamplechooseSCSI(1:0),andusethe defaultmode. NOTEThismustbeanewSCSIController.YoucannotuseSCSI0.

28

VMware, Inc.

Chapter 3 Clustering Virtual Machines Across Physical Hosts

ClickFinishtocompletecreatingthedisk. ThewizardcreatesbothanewSCSIcontrollerandanewharddisk.

9 10

SelectthenewSCSIcontrollerandclickChangeControllerType. SelectLsiLogicinthedialogboxthatappears. MSCSonESXServer3.xisnotsupportedinconjunctionwithBusLogic.

11

Inthesamepanel,setSCSIBusSharingtoPhysicalandclickOK.

12

Ifyouneedadditionalshareddatadisksinyourconfiguration,repeatStep 1 throughStep 8butchooseanewVirtualDeviceNode,suchasSCSI(1:1). Figure 31showsthesetupatthispoint.

VMware, Inc.

29

Setup for Microsoft Cluster Service

Figure 3-1. Cluster Across Boxes, Node1 Setup

NIC1 virtual switch1 (public) VNIC1 NIC2 virtual switch2 (private) VNIC2

virtual machine Node1 VSCSI1 physical machine SCSI1 SCSI2 FC local or remote storage VSCSI2

remote storage

Task 4: Adding Hard Disks to Node2

AfteryouhavesetupNode1,setupNode2sotheprivateandpublicnetworksmatch. ThensharethequorumandanyshareddatadisksforNode1withNode2.Usingthe RDMthatyoucreatedwhensettingupthefirstclusternode. To reuse a SAN-based RDM 1 2 3 4 5 OnNode2,clickAdd,selectHardDisk,andclickNext. IntheSelectaDiskpage,chooseUseExistingDisk,andclickNext. SelecttheRDMcreatedontheshareddatastoreinStep 5forNode1. ContinuewithStep 6throughStep 10forthequorumdisk(seeToaddaquorum diskandoptionalsharedstoragedisksonpage 28). (Optional)ContinuewithStep 6throughStep 8foreachadditionalshareddata disk(seeToaddaquorumdiskandoptionalsharedstoragedisksonpage 28).

ThecompletedsetuplookslikeFigure 32.

30

VMware, Inc.

Chapter 3 Clustering Virtual Machines Across Physical Hosts

Figure 3-2. Cluster Across Boxes Complete Setup

NIC1 virtual switch1 (public) VNIC1

NIC2 virtual switch2 (private) VNIC2

NIC2 virtual switch2 (private) VNIC2

NIC1 virtual switch1 (public) VNIC1

virtual machine Node1 VSCSI1 physical machine SCSI1 SCSI2 FC local or remote storage VSCSI2

virtual machine Node2 VSCSI2 physical machine SCSI2 FC local or remote storage SCSI1 VSCSI1

remote storage

VMware, Inc.

31

Setup for Microsoft Cluster Service

32

VMware, Inc.

Clustering Physical and Virtual Machines

ThischapterguidesyouthroughcreatinganMSCSclusterinwhicheachphysical machinehasacorrespondingvirtualmachine. Thechapterconsistsofthefollowingsections:

Task1:CreatingtheFirstNodeonpage 33 Task2:CreatingtheSecondNodeonpage 34 Task3:InstallingMicrosoftClusterServiceonpage 36 Task4:CreatingAdditionalPhysical/VirtualPairsonpage 36

Task 1: Creating the First Node

Becausethefirstnodeisaphysicalmachine,nodetailedinstructionsforcreatingthe firstnodeareincludedinthischapter.SeetheMicrosoftClusterServicedocumentation forallprerequisitesandcaveats.Youshouldsetupyoursystemasfollows:

ChoosetheAdvancedMinimumconfigurationwithintheWindowsCluster Administratorapplication. Setupthephysicalmachinetohaveatleasttwonetworkadapters. SetupthephysicalmachinetohaveaccesstothesamestorageonaSANasthe ESX Serverhostonwhichyouwillrunthecorrespondingvirtualmachine. Installtheoperatingsystemyouwanttousethroughoutthecluster.

NOTEVMwarerecommendsthatyoudontrunmultipathingsoftwareinthephysical orvirtualmachines.

VMware, Inc.

33

Setup for Microsoft Cluster Service

Task 2: Creating the Second Node

Creatingthesecondnodeconsistsofthefollowingmajorsteps:

Creatingavirtualmachinethatissetupforclusteringacrossboxes. MakingsurethesharedstoragevisiblefromNode1(thephysicalmachine)isalso visiblefromNode2(thevirtualmachine). Installingtheoperatingsystem. NetworkadaptersetupofthenodedependsonthetypeofESXServeryouare using.VMwarerecommendsthreenetworkadaptersperhostforconnectionsto theoutside.SeePrerequisitesforStandbyHostClusteringonpage 15for informationontheminimumconfiguration.

NOTEBeforeyoucreateavirtualmachine,createavirtualdiskineagerzeroedthick formatusingvmkfstools.Thenpointtothatdiskduringvirtualmachinecreation. To create the second node 1 LaunchaVIClientandconnecttotheESXServerhost. Usetheusernameandpasswordoftheuserwhowillownthevirtualmachine. 2 3 Intheinventorypanel,rightclickthehostandchooseNewVirtualMachine. Makethefollowingselectionswiththewizard: Table 4-1. New Virtual Machine Properties
Page WizardType NameandLocation ResourcePool Datastore GuestOperating System CPUs Memory Network Selection Typical. Chooseaname(forexample,Node2)andlocation. Selecttheresourcepoolforthevirtualmachine,orthehostif therearenoresourcepools. Chooseadatastoreasthelocationforthevirtualmachine configurationfileandthe.vmdkfile. ChoosetheWindows2000ServerorWindowsServer2003 operatingsystemyouwanttoinstalllater. Usethedefault. Usethedefaultunlessyouneedadditionalmemoryandyour serversupportsit. ChangeNICstoConnectto2,andselectthesecondnetworkfor thesecondNIC.

34

VMware, Inc.

Chapter 4 Clustering Physical and Virtual Machines

Table 4-1. New Virtual Machine Properties (Continued)

Page VirtualDiskCapacity ReadytoComplete Selection IfyouneedaprimarySCSIdisklargerthan4GB,enterthe appropriatevalueintheCapacityfield. ClickOKtocreatethevirtualmachine.

YouneedasharedSCSIcontrollerandsharedSCSIdisksforsharedaccesstoclustered servicesanddata.ThenextsectionsetsupthedisksforNode2topointtothequorum diskandsharedstoragedisks,ifany,forNode1. To add a quorum disk and optional shared storage disk 1 2 3 SelectthevirtualmachineyoucreatedandchooseEditSettings. ClickAdd,selectHardDisk,andclickNext. IntheSelectaDiskpage,chooseMappedSANLUNandclickNext. YourharddiskpointstoaLUNusingRDM. 4 5 6 7 IntheLUNselectionpage,choosetheLUNthatisusedbyNode1. IntheSelectDatastorepage,selectthedatastore,whichisalsothelocationofthe bootdisk,andclickNext. SelectPhysicalcompatibilitymodeandclickNext. SelectavirtualdevicenodeonadifferentSCSIControllerthantheonethatwas createdwhenyoucreatedthevirtualmachine. ThisSCSIControlleriscreatedwhenthevirtualharddiskiscreated. 8 ClickFinishtocompletecreatingthedisk. Thewizardcreatesbothanewdevicenodeandanewharddisk. 9 10 11 SelectthenewSCSIcontroller,setSCSIBusSharingtoPhysical,andclickOK. (Optional)Foradditionalstoragedisks,repeatStep 1throughStep 6butchoosea disk.Usethesamevirtualadapter. InstallWindows2000ServerorWindowsServer2003onthevirtualmachine.

VMware, Inc.

35

Setup for Microsoft Cluster Service

Task 3: Installing Microsoft Cluster Service

ThefinaltaskistoconfigureMicrosoftClusterService. SeetheGuidetoCreatingandConfiguringaServerClusterunderWindowsServer2003and otherinformationontheMicrosoftWebsite. Insomecomplexstoragesolutions,suchasanFCswitchedfabric,aparticularstorage unitmighthaveadifferentidentity(targetIDorrawdiskID)oneachcomputerinthe cluster.Althoughthisisavalidstorageconfiguration,itcausesaproblemwhenyou wanttoaddanodetothecluster. To avoid identity target problems 1 2 WithintheMicrosoftClusterAdministratorutility,disablethestoragevalidation heuristicsbyclickingtheBackbuttontoreturntotheSelectComputerpage. ClicktheAdvancedbuttonandselecttheAdvanced(minimum)configuration option.

MicrosoftClusterServiceshouldoperatenormallyinthevirtualmachineafteritis installed.

Task 4: Creating Additional Physical/Virtual Pairs

Foreachphysicalmachine:

RepeatTask1tosetupanadditionalvirtualmachineontheESXServerhost. Clusterthephysicalmachinewiththatvirtualmachine.

36

VMware, Inc.

Upgrading Clustered Virtual Machines

ThischapterdiscusseshowtoupgradeclustersthatuseVMFS2toVMFS3.Itpresents acomprehensivediscussionofallcasesinthefollowingsections:

LegacyClusterSetupOptionsonpage 37 UpgradingClusterinaBox(CIB)onpage 38 UpgradingClusterAcrossBoxesonpage 40 UpgradingClustersUsingPhysicaltoVirtualClusteringonpage 42

NOTEUpgradingissupportedonlyfromESXServer2.5.2orhigher. YoucanupgradefromESXServer2.5.2toESXServer3.0.xorESXServer3.5,andyou canupgradefromESXServer3.0.xtoESXServer3.5. BecausetherearenoearlierversionsofESXServer3i,thischapterdoesnotapplytothat platform.

Legacy Cluster Setup Options

UsingVMFS2,youhadanumberofoptionsforsettingupyourMSCScluster:

Forvirtualmachinesclusteredonasinglephysicalhost(clusterinabox),you coulduseapublicVMFSinoneoftwoways:

UsingnonpassthroughRDMs Usingsharedvirtualdisks

VMware, Inc.

37

Setup for Microsoft Cluster Service

Forvirtualmachinesclusteredonmultiplephysicalhosts(clusteracrossboxes), youhadthreeoptions:

ShareddisksonsharedVMFS TwopassthroughRDMsbackedbythesameLUNonpublicvolume AsinglepassthroughRDMonasharedVMFSvolume

Forclustersofphysicalandvirtualmachines(standbyhostclustering),youuseda publicvolumeusingpassthroughRDM.

Thischapterstepsyouthroughtheupgradeprocessforeachoftheseoptions.

Upgrading Cluster in a Box (CIB)

WithVMFS2,aclusterinaboxsetupusesapublicVMFS.Bydefault,thegeneral upgradeprocess,discussedintheUpgradeGuide,includesinformationabout upgradingofpublicVMFS2volumestoVMFS3.IfyoudidnotupgradetheVMFSused bytheclusterduringtheupgradeprocess,youcanupgradeusingthe VIClientlater.

Upgrading CIB: Shared RDMs and Boot Disks in Separate VMFS Volumes
Thissectionstepsyouthroughupgradingaclusterinaboxthatusesshared nonpassthroughRDMsthatresideinadifferentVMFS2volumethanthebootdisks fortheclustervirtualmachines. To perform the upgrade 1 2 3 Poweroffallclusteredvirtualmachines. UpgradetheESXServerhostfromESXServer2.5.2toESXServer3.x. IfyoudidnotupgradetheVMFS2volumewhereyourcluster.vmdkfilesarekept toVMFS3duringupgradeofthehost,upgradenow: a b c d 4 SelecttheupgradedhostinaVIClientandclicktheConfigurationtab. ClickStorage. Selectthevolume. ClickUpgradetoVMFS3.

Ifnecessary,upgradethevolumewhereyoursharedRDMfilesarelocatedand upgradethosefiles,asinStep 3.

38

VMware, Inc.

Chapter 5 Upgrading Clustered Virtual Machines

5 6

RightclickeachclustervirtualmachineintheinventorypanelandclickUpgrade VirtualHardware. Poweroneachvirtualmachineandverifytheclustersetup. IfthevirtualmachinefailstopoweronwitherrormessageInvalid Argument, youhaveamisconfiguredclustersetup.ThevirtualdiskusedinESX2.xisnot allowedtopoweroninESX3.xbecauseESX3.xchecksforinvaliddisktypes.

Upgrading CIB: RDMs and Boot Disks in Same VMFS Volume

Thissectionstepsyouthroughupgradingaclusterinaboxthatusesshared nonpassthroughRDMsthatresideinthesameVMFS2volumeasthebootdisksfor theclustervirtualmachines. To perform the upgrade 1 2 3 4 UpgradetheESXServerhostfromESXServer2.5.2toESXServer3.x. IntheVIClientinventorypanel,selecttheupgradedhostinaVIClient. ClicktheConfigurationtab,andclickStorage. UpgradetheVMFS2volumewhereyourcluster.vmdkfilesandyoursharedRDM filesarelocatedtoVMFS3,asfollows: a b Selectthevolumewherethefilesarelocated. ClickUpgradetoVMFS3.

ThisactionupgradestheVMFS2volumestoVMFS3andrelocatesthe.vmxfilefor theclustervirtualmachinesintotheupgradedVMFS3volumeinadirectory structure. 5 Rightclickthesecondclusternodesvirtualmachineintheinventorypaneland clickUpgradeVirtualHardware. Anerrorlikethefollowingresults:

VMware ESX Server could not completely upgrade your virtual disk "/vmfs/volumes/2a3330116-da-11....vmdk due to the following error: The system cannot find the file specified."

TheerroristheresultofStep 3,wheretheclustersvirtualmachineswererelocated tothefirstnodesdirectory. 6 Ignoretheerror. Thesystemupdatesyourvirtualhardwareregardlessoftheerror.Youcanverify thisbyviewingthe.vmxfileentriesofthesecondclusternode.

VMware, Inc.

39

Setup for Microsoft Cluster Service

Manuallyeditthe.vmxfileofthesecondclustervirtualmachinesothattheentries ofthequorumdiskandanyothershareddiskpointtothesharedRDMfilesthat areinsidethefirstnodesdirectoryinsidetheVMFS3partition. Poweroneachvirtualmachineandverifytheclustersetup.

Upgrading CIB: Virtual Disks

Thissectionstepsyouthroughupgradingaclusterinabox(CIB)thatusesshared virtualdisksfortheclustervirtualmachines. To perform the upgrade 1 Importtheoldvirtualdisktothenewvirtualdisk,asfollows:
vmkfstools -I /vmfs/volumes/vol1/<old-virtdisk>.vmdk /vmfs/volumes/vol2/<myVMDir>/<new-virtdisk>.vmdk

old-virtdisk.vmdkthevirtualdiskfromwhichyouareimporting new-virtdisk.vmdkthenewvirtualdisktowhichyouareimporting

2 3 4

Renameold-virtdisk.vmdk. Editthe.vmxfiletopointtonew-virtdisk.vmdk. Poweronthenodeandverifythattheclusterservicestartswithoutproblems.

Upgrading Cluster Across Boxes

Therearetwotypesofclustersacrossboxes.Thissectionexplainshowtoupgrade clustersacrossboxesusingsharedpassthroughRDMsandhowtoupgradeclusters acrossboxeswithsharedfilesystems.

Using Shared Pass-Through RDMs

ThissectionexplainshowtoupgradeaclusterwithpassthroughRDMsforeachnode. To upgrade the cluster 1 2 3 UpgradetheESXServerhostfromESXServer2.5.2toESXServer3.x. IntheVIClientinventorypanel,selecttheupgradedhost. ClicktheConfigurationtab,andthenclickStorage.

40

VMware, Inc.

Chapter 5 Upgrading Clustered Virtual Machines

UpgradetheVMFS2volumewhereyoursharedpassthroughRDMfilesarekept toVMFS3: a b Selectthevolume. ClickUpgradetoVMFS3.

Selectthevolumewherethebootdiskfortheclustervirtualmachineislocatedand upgradeitasinStep 3. Thisupgradesthevolumeandrelocatesthe.vmxfilesrelatedtothevirtual machinesinsidethevolume.Thenewdirectorystructureisorganizedforeasy management.

6 7 8 9

Rightclickontheclustervirtualmachineintheinventorypanelontheleft. ChooseUpgradeVirtualHardwarefromtherightbuttonmenu. RepeatthestepsfortheNode2host. Poweronthevirtualmachinesandverifythecluster.

Upgrading a Cluster with Files in Shared VMFS2 Volumes

Thissectionexplainshowtoupgradeaclusteracrossboxesifyouusedsharedfilesin asharedVMFS2volume. To upgrade the cluster 1 BeforeupgradingtoVMFS3,changethesharedVMFS2volumefromsharedto public,asfollows:
vmkfstools -L lunreset vmhba<C:T:L>:0 vmkfstools -F public vmhba<C:T:L:P>

2 3 4 5

PerformthehostupgradesfromESXServer2.5.2toESXServer3.x. SelectthefirstupgradedhostinaVIClientinventorypanel. ClicktheConfigurationtab,andclickStorage. UpgradetheVMFS2volumewhereyourcluster.vmdkfilesarekepttoVMFS3: a b Selectthevolume. ClickUpgradetoVMFS3.

CreateLUNsforeachshareddisk(thatis,oneLUNforeachshareddisk).

VMware, Inc.

41

Setup for Microsoft Cluster Service

Foreachdisk,createaseparateRDMforeachclusternodebackedbythesame physicaldevice.CreatetheRDMandimportthevirtualdisktothisRDM.
vmkfstools -i /vmfs/volumes/vol1/<old-virtdisk>.vmdk /vmfs/volumes/vol2/<myVMDir>/<rdm-for-vm1>/<myrdm.vmdk> -d rdmp:/vmfs/devices/disks/vmhbaC:T:L:P

Where

old-virtdisk=thesourcevirtualdisk. myVMDir=thetargetvirtualmachinedirectory. rdm-for-vm1=anoptionaldirectoryinwhichtostoreRDMfilesforthat virtualmachine. myrdm.vmdk=thetargetRDMfilethatthiscommandcreates. vmhbaC:T:L:P=thedevicerepresentingtherawLUNthatyouaremapping

C=controllernumber(theFCHBA). T=thestoragearraystargetnumberthroughwhichtheLUNisaccessed. L=LUNnumber. P=partitionnumber.Inthisexampleyoumustuse0asthevalueto addressthewholeLUN

Editthe.vmxfiletopointtotheRDMinsteadofthesharedfile:
scsi<X>:<Y>.filename = "/vmfs/volumes/vol2/<myVMDir>/<rdm-for-vm1>/<myrdm.vmdk>" scsi<X>:<Y>.deviceType = "scsi-passthru-rdm"

9 10 11

RightclicktheclustervirtualmachineintheinventorypanelandselectUpgrade VirtualMachine. RepeatStep 8andStep 9forNode2. Poweronthenodesandverifythattheclusterservicestartswithoutproblems.

Upgrading Clusters Using Physical to Virtual Clustering

IfyouareusingaphysicaltovirtualclusterusingVMFS2,youuseapublicdiskthatis mappedusingRDMfromthevirtualmachine. Bydefault,theupgradeprocessconvertsyourVMFS2diskstoVMFS3.Youcanalso explicitlyconvertVMFS2volumeslaterifyoudidnotconvertthemaspartofthe defaultconversion.

42

VMware, Inc.

Appendix: Setup Checklist

AdministratorswhoaresettingupMicrosoftClusterServiceonESXServer3.xcanuse thisappendixasachecklist. Theappendixincludesinformationinthefollowingtables:

Table A1RequirementsforClusteredDisks Table A2OtherClusteringRequirementsandRecommendations

Table A1liststherequirementsforclustereddisks. Table A-1. Requirements for Clustered Disks

Component Clusteredvirtualdisk (.vmdk) Clustereddisks,virtual compatibilitymode(non passthroughRDM) Single-host Clustering SCSIbussharingmodemustbe settoVirtual. DevicetypemustbeVirtual compatibilitymode. SCSIbussharingmodemustbe settoVirtual. Asingle,sharedRDMmapping fileforeachclustereddiskis required. Multihost Clustering Notsupported. DevicetypemustbeVirtual compatibilitymodeforclusteracross boxes,butnotforstandbyhost clustering. SCSIbussharingmodemustbesetto Physical. Requiresasingle,sharedRDM mappingfileforeachclustereddisk.

VMware, Inc.

43

Setup for Microsoft Cluster Service

Table A-1. Requirements for Clustered Disks (Continued)

Component Clustereddisks,physical compatibilitymode (passthroughRDM). Single-host Clustering Notsupported. Multihost Clustering DevicetypemustbePhysical compatibilitymode.Thisissetduring harddiskcreation. SCSIbussharingmodemustbesetto Physical(thedefault). Asingle,sharedRDMmappingfilefor eachclustereddiskisrequired. Alltypes AllclusterednodesmustusethesametargetID(onthevirtualSCSI adapter)forthesameclustereddisk. Aseparatevirtualadaptermustbeusedforclustereddisks.

Table A2listsotherclusteringrequirements. Table A-2. Other Clustering Requirements and Recommendations

Component Disk Requirement Ifyouplacethebootdiskonavirtualdisk,createthatdiskusingvmkfstools, specifyingtheeagerzeroedthickoption. TheonlydisksthatyoushouldnotcreatewiththeeagerzeroedthickoptionareRDM files(bothphysicalandvirtualcompatibilitymode)andthebootdisksofnative Windowshostsinstandbyclustering. Windows UseWindowsServer2003SP2(32bit),WindowsServer2003(64bit)SP2,or Windows 2000ServerSP4.VMwarerecommendsWindowsServer2003. Onlytwoclusternodes. DiskI/Otimeoutissixtysecondsormore (HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\TimeOutValue). Note:Ifyourecreatethecluster,thisvaluemightberesettoitsdefault,soyoumust changeitagain. Clusterservicemustrestartautomaticallyonfailure(first,second,andsubsequent times). ESXServer configuration VMwarerecommendsthatyoudontovercommitmemory,thatis,setMemory Reservation (minimummemory)tothesameasMemoryLimit(maximummemory). Ifyoumustovercommitmemory,theswapfilemustbelocal,notontheSAN.

44

VMware, Inc.

Appendix: Setup Checklist

Table A-2. Other Clustering Requirements and Recommendations (Continued)

Component Information requiredby technical supportto analyze clustering relatedissues Requirement Verifythatthesetupcomplieswiththechecklist. vmsupporttarball(vmkernellog,virtualmachineconfigurationfilesandlogs,) Applicationandsystemeventlogsofallvirtualmachineswiththeproblem. Clusterlogofallvirtualmachineswiththeproblem(thatis,%ClusterLog%,whichis usuallysetto%SystemRoot%\cluster\cluster.log). DiskI/Otimeout (HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\TimeOutValue) VIClientdisplaynamesandWindowsNETBIOSnamesofthevirtualmachines experiencingtheproblem. Dateandtimeandtheproblemoccurred. SANconfigurationoftheESXServersystem(LUNs,paths,andadapters). Multipathing Runningthirdpartymultipathingsoftwareisnotsupported.

VMware, Inc.

45

Setup for Microsoft Cluster Service

46

VMware, Inc.

Index

A
across boxes, clustering 25 introduction 11 prerequisites 14

D
disks adding to nodes (across boxes) 28 adding to nodes (in-a-box) 21 quorum 22, 28, 35 shared 22, 28, 35

C
caveats 16 cloning node1 21, 27 cluster across boxes first node 25 introduction 11 prerequisites 14 second node 27 upgrading 40, 41 cluster in a box first node 19 introduction 10 prerequisites 14 second node 20 upgrading 38, 39 clustering hardware 10 clustering physical and virtual machines 13, 33 first node 33 second node 34 clustering software 10 clustering virtual machines across hosts 25 introduction 11 prerequisites 14 clustering virtual machines on one host 19 introduction 10 prerequisites 14

E
ESX Server 3i 14, 15

F
first node creating (across boxes) 25 creating (in-a-box) 19 creating (standby host) 33

H
hardware 10

I
in-a-box, clustering 19 introduction 10 prerequisites 14

M
Microsoft Cluster Service (MSCS) installing 24, 36

N
N+1, prerequisites 15 node1, cloning 21, 27

VMware, Inc.

47

Setup for Microsoft Cluster Service

P
prerequisites for clustering 13

Q
quorum disk 22, 28, 35

R
requirements 16

standby host 33 introduction 13 prerequisites 15 upgrading 42 storage quorum disk 22, 28, 35 shared 22, 28, 35

S
second node creating (across boxes) 27 creating (in-a-box) 20 creating (standby host) 34 service console 14, 15 shared storage disk 22, 28, 35 shared storage summary 15

U
upgrading 38 cluster across boxes 40, 41 cluster in a box 38, 39 standby-host cluster 42

48

VMware, Inc.

Updates for Setup for Microsoft Cluster Service

LastUpdated:August14,2009 Thisdocumentprovidesupdatestothebook,SetupforMicrosoftClusterService,written fortheUpdate 2releaseforESX Server 3.5,ESX Server 3iversion 3.5,and VirtualCenter 2.5.Updateddescriptionsareorganizedbypagenumbersothatyoucan easilylocatetheareaoftheguidethathaschanges.Ifthechangespansmultiple sequentialpages,thisdocumentprovidesthestartingpagenumberonly. ThefollowingareupdatestotheSetupforMicrosoftClusterService:

UpdatesfortheCaveats,Restrictions,andRecommendationsSectiononPage 16 UpdatestotheRecommendationsforUsingMSCSandBootfromSANSectionon Page 17

Updates for the Caveats, Restrictions, and Recommendations Section on Page 16

ThefollowingrestrictionintheCaveats,Restrictions,andRecommendationssectionis nowinvalidandshouldberemoved:

ThebootdiskoftheESXServerhostshouldbeonlocalstorage.

Updates to the Recommendations for Using MSCS and Boot from SAN Section on Page 17
IntheRecommendationsforUsingMSCSandBootfromSANsection,thedescription ofbootfromSANdoesnotindicatewhichversionoftheMicrosoftWindowsoperating systemissupported.ThedocumentshouldstatethatbootfromSANisonlysupported onWindows Server 2003,andisnotsupportedonWindows Server 2000operating systems. Thissectionshouldalsoincludethefollowingrecommendationwhenyouputtheboot diskofavirtualmachineyouwishtouseinaclustersetuponaSAN.

VMware, Inc.

Update1

Ifyourenvironmentissusceptibletoconditionsthatcauseclusternodeserverstolose allpathstothestoragearray,orifyoursystemfailsanddisplaysabluescreenwhena newLSIdriverisinstalled(forallclusterconfigurations),setthelsilogic.iobar256 parametertoTrueforeachnode. To set lsilogic.iobar256 to true 1 LogintoaVIClientandselectthevirtualmachinefromtheinventorypanel. Theconfigurationpageforthisvirtualmachineappears. 2 3 4 5 6 7 IntheSummarytab,clickEditSettings. ClickOptions>Advanced>General,andthenclickConfigurationParametersto opentheConfigurationParametersdialogbox. ClickAddRow. Enterlsilogic.iobar256intheNamecolumnandTrue intheValuecolumn. ClickOKtoclosetheConfigurationParametersdialogbox. ClickOKtoclosetheVirtualMachinePropertiesdialogbox.

Update2

VMware, Inc.