9/12/2017 HALDOM Cluster 43 Home OVMfor SPARC OVMforx86 OE WOpscenter Solaris 11. Zoner_—‘Oraclo upercluster—_Exadata HA LDOM Cluster 4.3 How to create a HA-LDOM cluster using Oracle Solaris CLuster 4.3 Setup Se 2x T6320 blades (1 x CPU, 64GB mem, 1 x dual 4Gbit FC adapter wth dual 1Gbe adapter, 2 x NEM } ‘Storage: Sun Storage 2540 with dual controllers 08: Oracle Solaris 4.3 Cluster Software: Oracle Solaris Cluster 4.3 Configuration: Below is a picture of the sotup: ca ory Re EreelIi cy ie Creee ee) Cluster4.3 The basic layout is as follows. The lwo master servers are 76320 blades (SPARC) in a SunBlade 6000 chassis. Each blade has a Express Module that contains 2 x Gbe network ports and 2 x 4Gbit fc ports. This is @ combo EM adapter at the back of the SB6000, ‘chassis. The blade chassis also has 2 x Network Express Modules at the back that contains 1Gbe ports, ‘So each blade has 4 x 1Gbe network ports and 2 x fe ports. Two ports, nett and net3 are used for public network and uses OLMP. ‘The other 2 x nets is used as the cluster interconnect. | used straight cables connected direcly to the ports on each blade at the hiipsos-chronicles.yolasite,comma-Kdom-clusterphp 199/12/2017 HALDOM Cluster 43 back. | aid this because the two blades are next to each other in the chassis so i's not needed to use a switch, Ifyour nodes are far apart then you would need to connect the nodes via a switch or use tagged vians on the public net forthe cluster interconnect The storage used is @ Sun Storage 2540 array with dul controllers, Each controler has 2 x 4 Gbit FC ports, Luns were created and are visible to both nodes. Cluster Setup This page does not discuss how | installed the OS or the cluster sotware. | wil have lis inthe future on how to do thal. | basically used the EIS checklist of April 2016 fo achieve the instal of the OS and cluster. Making a HA-LDOM configuration. Please note that some of the commands are executed on both and some only on the “master” node. In this document the two nodes are nodet (calsto) and node2 (ananke). (On Both Nodes (callisto and ananke) Justa note or two here. In the example | used DLMEP forthe virtua switch (vsw) on the primary. You could also use Link ‘Aqaregation for the vsw or physical net ports. This Is just an example and you would probably change the vsw to whatever you decide on ste Create the primary domains. This must be done on both nodes. l only show the output of the first node callisto. rootéeallisto # 1dm add-vconscon port-range=5000-$100 primary-vee0 primary rootgcallisto # sveadn enable sve: /1doms/vnted:default rootécallisto # 1dm add-vewitch net-dev-dimp0 primary-vew0 primary rootécallisto # 1dn add-vdiskserver primary-vde0 primary rootcallisto # 1dm List-bindings primary rootécallisto # pkg info Idonsmanager | grep -i version rootécallisto # svcadm enable lind rootécallisto # 1dm set-core 2 primary unit 7 was removed from domain primary 6 was semoved from domaii 3 wae removed from domain primary crypto unit 2 was removed from domain primary rootécallisto # ldm list primary active se 16 6S408e 0.68 Sim rootécallisto # 1dm start-reconf primary nitiating a delayed reconfiguration operation on the primary domain. All configuration changes for other domains are disabled until the primary Gomain reboots, at xhich time the new configuration fer the primary domain will alsa take =: rootécallisto # 1dm set-menory 166 primary cet The primary domain is in 55 of a delayed reconfiguration rootécallisto # idm add-config initial rooticallisto # 1dn list-contig rootfcallisto # shutdown -y -i6 -g0 Set the failover policy to reset on both nodes rootécallisto # 1dm set-domain failure-polic ¢ primary Setup the Idem. Only on nodet hiipsos-chronicles.yolasite,comma-Kdom-clusterphp 299/12/2017 HALDOM Cluster 43 dm add-domain fldom) dm set-core 2 fldomi dm set-men 16g f1doat dm list £ldom nactive 16166 dm list-services primary dm add-vnet vnetO primary-vew0 fldomt dm set-domain master=primary fidom1 dm set-var auto-boot?=false fidemt dn list -o domain fldomt 65 UTIL out S047e5£1-78d6-4145-a176-bhda27e6efld faiiure-policy extended-napin-space ads-policy= aute-hoot?-£a (On Both Nodes add the backend device. Must use d2s2 rootfcallisto # 1dm add-vdsdev /dev/did/rdsk/d2s2 fldomi_boot@primary-vds0 root@ananke # ldm add-visdev /dev/did/rdsk/d2s2 fldom1_bootéprimary-vds0 (n first node nodet add the vdisk, bind the domain and boot. Install the guest. rootécallisto # 1dn add-vdisk fldoni_boot fidoml_bootéprimary-vde0 #1dom1 rootfcallisto # 1dm bind fldom1 rootécallisto # 1dm start fidom rootfcallisto # telnet localhost 5000 ‘The below seting is for when you want to boot from an Automated Installer (A) server, If you have added an iso image then you ‘would install from that image. If you want to boo! fom an iso image then you need to assign the image as a virtual disk tothe Idom Ike this (ldm add-vdedev /var/tmp/solarisli_3.iso fldoml_isolprimary-vde0 primary; 1dn add-vdisk fidomi_iso fldoml_Asotprinary-vds0 fldomi). You would then boot using boot #idomi_iso. Boot from the network and install from Al ok setenv network-boot-argunents host~ip=192.168.50.121,router-ip=192.168.50.1,subnet~ mask=255.255.255.0,file=http: //192.168.50.40:5555/cgi-bin/wanboot-cgi ok boot vnet0 - install hiipsos-chronicles.yolasite,comma-Kdom-clusterphp9/12/2017 HALDOM Cluster 43 You may wish ¢ reboot ¢ automated ation finished successfully the system ooted now Please refer to the /systen/volatile/install_ log file for deta Aster reboot it will be located at /var/log/install/inatal. solaris console login: root Paseword: solaris 01:16:29 solaris login: ROOT LOGIN /dev/console cle Corporation sunos 5.1 11.3 March 2016 roottsolaris:~# reboot (On both nodes check If ha-ldom data service is installed. I not Install it Footecalliste # pkg info ha-cluster/data-service/na-idom rootdananke # pkg info ha-cluster/data-sezvice/ha-ldom Sunnary: Oracle fer HA Cor xVM x86-66/SPARC Guest Domains beseription aria Cluster HA for xVM Server x86-64/SPARC Guest version: 4 fe Solaris Cluster 4.3.0.24.0 Release Branch: 0.24.0 Packaging Date: August 26, 2015 11:32:36 OM MRI: pkg? //ha-cluster/ha-cluster/data-service/na-idowt.3,5. 2o1soazer2332362 Check on both nodes that first entries in /etcinsswitch is cluster. If not change it. rootfcallisto # svecfg -s nane-service/awitch listprop config/host aster files" config/host astring rootécalliste # svcefg -s name-service/switch Listprop config/ipnodes rootfcallisto # svecfg -s nane-service/switch listprop config/netmask To set a proporty use the folloiwng command: rootécallisto # svecfg -s name-service/switch setprop config/ipnodes = astring: ‘(eluater files ans)’ rootfcalliste # svecfg -s name-service/switch listprop config/ipnodes Create resource group node 1 only rootécalliste #| clzg create -n callisto,ananke fldemi-rg Register HAStoragePlus rootdcalliste # clrt register SUNW.HAStoragePlue Create HAStoragePlus resource for boot disk rootécallisto # clrs create -g fldonl-rg -t SUNW.HAStoragePlus -p GlobalDeviceraths=/dev/global/dsk/d22 f1dom1-has-r= hiipsos-chronicles.yolasite,comma-Kdom-clusterphp9/12/2017 HALDOM Cluster 43 Enable resource fidomt-rg rootfcallisto # clzg online -M -n callisto fldeml-rg rootécallisto # clrg status Group Name Node Nane suspended status Sidon sallisto No online Register Idom data service rooticallisto # clrt register SUNW.1don rootécalliste # crt list SUNW. Shareaadaire: SUN, HAStoragePlue:11 sun. 1dom:7 Create password file for failover Idoms (This only works if SUNW.ldom:6 or 7) rootdcallisto # clpstring create -t resource -b fldoml-re fldom1-rg_fldom1~ Create fldomt resource rootécallisto #| clra create -g fldonl-rg -t SUNW.1dom -p Domain name=fidomi -p resource dependencies=fldoni-has-rs fldoni-re Check migration type property. It must be MIGRATE. If not set it to MIGRATE rootcalliste # clr show -v fldoml-ra | grep Migration type Migration type MIGRATE (Zo change do the following) cles set -p Migration type-MIGRATE fldoni-rs To stop Idom resource disable it rootécallisto # clrs status Node Name state fidomi-rs ealliste online Online ~ fldoni is active (nozmal) ananke Stdomi-has-re cailisto ananke rootfcallisto # clrs disable fldom1-rs rootécallisto # clrs status de Name ates Kes! hiipsos-chronicles.yolasite,comma-Kdom-clusterphp9/12/2017 HALDOM Cluster 43 sidomi-zs calliste offline offline - Successfully stopped fidont ananke offline offline fdomi-has-rs cailisto online online ananke ores offline rootécallisto # ldm list NAME STATE “FLAGS CONS-«VCPU MEMORY «UTIL UPTIME primary active neve SP 16165084 Th Sim idom snactive ae 16s ‘To Start enable the resource rooticallisto # clrs enable fldoni-rs rootfcallisto # crs status Cluster Resources Resource Name Node Name State status Message calliste online Online ~ fldomi is active (noxmal) ofeline ossline ‘Sidomi-has-re catlisto online online ananke offline offline rootécallisto # dm list wae stare cons. VCPU MEMORY «UTIL UPTIME primary active se 16 2.48 th S2m siden active sooo 16 ae zis ‘Switch over the resource to node 2 (ananke) rootfcallisto # clrg switch -n ananke fidomi-rg root¢ananke # ldm list NAME stare cons VCPU MEMORY UTIL UPTIME primary acti se 66s 0.58 Ih Sem Sidon active $000 16 6s 1S hen rootananke # cirs status == clus} — Resource Name Node Name state Status Message Aidoni-es caltiste offline offline ananke online online ~ Service is online Sidowi-has-re cailisto offline offline anke online online Switch back to nedet (callisto) root@ananke # clrg switch -n calliste fidomi-rg rootcallisto # 1dm list hiipsos-chronicles.yolasite,comma-Kdom-clusterphp9/12/2017 HALDOM Cluster 43 prinary active se 166s. 2.88 2h an iden active =n 500016 165.14 10m rootfcallisto # clrs status a+ cluster Resources === Resource Nane ode Name state Eldomi-rs calliste online Online - Sezvice is online. ananke offline offline cidomi-has-re cailisve online OnLine ananke offline offline ‘To test node failure stop the server from ILOM using the -f option and check whether the Idom fails over to the other node. From nodet (callisto ILOM) > step -£ /svs Ave you sure you want to inmediately stop /SY¥S (y/a)? y Stopping /S¥S innediately Check status from node? (ananke) rootananke # clrs statue === cluster Resources Resource Nane Node Name state Status Message Sidomi-rs eallisto offline offline ananke Starting Unknown - Starting Eidomi-has-rs ealliste oftline ogeline ananke online Degraded - Device service dsk/d2 associated with path /dev/global/dsk/d2s2 is Degraded. dom show up and running from node2 (ananke) root@ananke # clrs status cluster Resources Resource Name Node Name State Status Message fidomi-rs caltiste offline offline ananke ontine Online ~ fldoml is active (normal) Eldomi-has-re. callisto offline offline ananke online Degraded - Device service dsk/d2 associated with path /dev/giobai/dsk/d292 is Degraded. root¢ananke # ldm list hiipsos-chronicles.yolasite,comma-Kdom-clusterphp 79.9/12/2017 primary sadona ‘Start nodet (callisto) again and migrate back to primary -> start /s¥s Are you sure you want te start /S¥S (y/a)? y Check nodes after nodet started rootananke # cldev status o-- cluster D devices Device Instan: /dev/di/rdsk/al ananke calliste /dev/aid/rdsk/e2 ananke callisto /ev/ald/rask/as calliste /dev/aid/rdsk/ae callisto /dev/did/rdsk/as callisto Jdev/aid/eask/a6 callisro Jdeu/ahd/edsk/d? ananke /eev/did/rask/ae ananke /dev/did/rdsk/as ananke callisto root@ananke # cirs status Cluster Resources Resource Nane Node Nane State fidomi-rs ealiiate offline ananke online Eldom-has-rs calliste offline ananke online ‘Switch the resource group back to nodot (callisto) rootécallisto # clrs status cluster Resources hiipsos-chronicles.yolasite,comma-Kdom-clusterphp 1616s 0 HALDOM Cluster 4.3 2h an, 16 16s 0.18 an ok 0K Unnonitered Unnonitozed Unnonitoree ok Uunnonitored Unnonitered ok ok Status Message offline Online ~ fldori is active (normal) offline online9/12/2017 HALDOM Cluster 43 ealtiate online ~ fldori is active (normal) ananke offline sidomi-has-re cailisto online online ananke offline DesLine Koop in mind that if you want to add additional disks tothe Idom you need to ad only the vdsdev to both nodes and add the vdlisk to the ldom. The cluster wil take care ofthe rest [NBNBNENB: Save your config on both nodes withthe add-config option to kim. rootécallisto # 1dn add-config ha-ldom-latest rootécolliste # dm List-contig root@ananke # ldm add-config ha-ldom-latest root@ananke # ldm list-config factory-defauit snitial you don't then you have to go and do this all over again! Back to Home Page Back to OVM for SPARC hiipsos-chronicles.yolasite,comma-Kdom-custerphp