(09/1212017 Configure fallover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic's Blog ORACLE Sun Cluster Solaris Support | Categories SUN CLUSTER | Tuesday, April 22, 2014 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ‘live’ migration By: Juergen Schleich | Technical Analyst This blog shows an example to configure ‘Oracle Solaris Cluster Data Service for Oracle VM Server for SPARC’ on Solaris Cluster 4,1. It also mentions some hints around such a configuration, For this setup Solaris Cluster 4.1 SRU3 or higher and Oracle VM Server 3.0 or higher is required At least this is a summary of Oracle Solaris Cluster Data Service for Oracle VM Server for SPARC Guide and hitps:Pbogs oracle, comyjschiconfigure-fllover-ldom-araclo-ym-server-fr-sparc-on-solaris-lustr-41-by-using-tve-migration sno(09/1212017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic’s Blog Oracle VM Server for SPARC 3.0 Administration Guide. Please check these guides for further restrictions and requirements. This procedure is especially for ‘live' migration of guest LDom's which means no shutdown of the OS in the LDom within the failover. In earlier OVM releases this was called ‘warm’ migration. However, the word ‘live’ is used in this example. A ‘cold’ migration means that the OS in the guest LDom will be stopped before migration. Let's start: The necessary services must be identical on all the potential control domains (primary domains) which run as Solaris Cluster 4.1 nodes. It is expected that Oracle VM Sever is already installed 1) Prepare all primary domains which should manage the failover LDom with the necessary services. al1_primariest 1dm add-vconscon port-range=5000-5100 primary-vcc@ primary all_primariest svcadn enable svc: /Idons/vntsd:default all_primariest 1dm add-vswitch net-dev-net@ public-vsw1 primary all_primaries# 1dm add-vdiskserver primary-vds® primary To verify: all_primary# 1dn List-bindings primary 2) Set failure-policy on all primary domains: all_primaries# 1dn set-donain failure-policy=reset primary To verify all_primaries# Ida list -o domain primary 3) Create failover guest domain (fgd0) on one primary domain. Simple example: prinaryA# Idm add-donain fgd@ primaryas 1d set-vepu 16 Fedo primaryas ldn set-nen 86 fgd0 4) Add public network to failover guest domain: primaryA# 1d add-vnet public-net@ public-vswi fgde hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 211009/12/2017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using Ive" migration | Oracle Juergen Schlic's Blog To verify: primaryat 1dm list-bindings fede For more details to setup guest LDoms refer to Oracle VM Server for SPARC 3.0 Administration Guide 5) Set necessary values on failover guest domain fgd0 primaryA# 1dn set-donain master=prinary fgdo primarya# 1dn set-var auto-boot?=false fda To verify run: primaryA# 1dn list -o domain fgd0 auto-boot?=false is a “must have” to prevent data corruption. More details available in DocID 1585422.1 Solaris Cluster and HA VM Server Agent SUNW.ldom Data Corruption may Occur in a failover Guest Domain when “auto-boot?=true" is set 6) Select boot device for failover guest domain fgd0. Possible options for the root file system of a domain with ‘live’ migration are: Solaris Cluster global filesystem (UFS/SVM), NFS, iSCSI, and SAN LUNs because all accessible at the same time from both nodes. The recommendation is to use full raw disk because it's expected to do ‘live’ migration. The full raw disk can be provided via SAN or iSCSI to all primary domains. Remember zfs as root filesystem can ONLY be used if doing ‘cold’ migration because for ‘live' migration both nodes need to access the root file system at the same time which is not possible with zfs. Using Solaris Cluster global filesystem is an alternative but the performance is not that good as root on raw disk, Details available in DocID 1366967.1 Solaris Cluster Root Filesystem Configurations for a Guest LDom Controlled by a SUNW.Idom Resource So, root on raw filesystem is selected. Add boot device to fgd0: al1_primariest 1dm add-vdsdev /dev/did/rdsk/d7s2 boot_fgd0@primary-vdso primarya# 1d add-vdisk root_fgd@ boot _fgde@prinary-vds fede hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 30(09/1212017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic’s Blog 6a) Optional: Configure MAC addresses of LDom. The LDom Manager assign MAC automatically but the following issues can occur + Duplicate MAC address if other guest LDom's are down when creating anew LDom. + MAC address can change after failover of a LDom Assign your own MAC address is possible. This example use the suggested range between 00:14:4F:FC:00:00 — 00:14:4F:FF:FF:FF as described in Assigning MAC Addresses Automatically or Manually of Oracle VM Server for SPARC 3.0 Administration Guide. Example Identify current automatically assigned MAC addresses primarya# 1dn list -1 fgdo to see the HOSTID which is similar as MAC a ‘Idm bind fldg0' is necessary, Unbind fldgO afterwards with ‘dm unbind fldgO’ MAC: 00:14:4f:fb:50:de —> change to 00:14:4f:fc:50:de HOSTID: 0x84fb50dc — change to 0x84fb50dd public-net: 00:14:4f:fa:01:49 — change to 00:14:4f:fc:01:49 primarya® 1dn set-donain mac-addr=00:14:4f:Fc:S@:de fede primarya# 1dn set-donain hostid=oxafbsedd fede primaryA# 1dn set-vnet mac-addr=00:14:4f:#c:01:49 public-neto Fedo primaryA# 1dn list-constraints fgde (this shows assigned MAC now) For more details or if necessary to change the MAC addresses on already configured failover guest LDom then refer to DocID 1559415.1 Solaris Cluster HA-LDom Agent do not Preserve hostid and MAC Address Upon Failover 7) Bind and start the fgd0 primaryA# 1dn bind fgdo primarya# 1dn start fgdo 8) Login to LDom using console: primaryA# telnet localhost 5000 hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 410(09/1212017 Configure failover LDam (Oracle VM Server for SPARC) on Sok fs Cluster 41 by using "ve’ migration | Oracle Juargan Sehieichs Blog 9) Install Solaris10 or Solaris 11 on LDom by using install server To identify MAC address of LDom do in the console of fgd0: (0) ok devalias net {0} ok c@ /virtual -devices@100/ channel -devices@200/networkee {®) ok .properties local-mac-address 08 14 4f fc 01 49 For different installation method please refer to Installing Oracle Solaris OS on a Guest Domain of Oracle VM Server for SPARC 3.0 Administration Guide 10) Install HA-LDom (HA for Oracle VM Server Package) on all primary domain nodes if not already done all_primariest pkg info ha-cluster/data-service/ha-1don all_primaries# pkg install ha-cluster/data-service/ha-ldon 11) Check that cluster is first entry in /etc/nsswitch.conf alL_primariest svccfg -s nane-service/switch listprop config/host config/host astring “Files dn: all_primariest svccfg -s nane-service/switch listprop config/ipnodes config/ipnodes astring “files dns” all_primaries# svccfg -s nane-service/switch listprop config/netnask config/netmask astring files If not add it: all_primariest svccfg -s nane-service/switch setprop config/host = astring: “(cluster files dns” all_primariest svecfe “(Cluster files dns") primaries# svccfg -s name-service/switch setprop config/netnask = astring: luster files" 'S name-service/suitch setprop config/ipnodes = astring: More Details in DocID 1554887.1 Solaris Cluster: HA LDom Migration Fails With "Failed to establish connection with Idmd(1m) on target" 12) Create resource group for failover LDom fgd0 for primiary domains primarya# clrg create -n primaryA,primary® fldon-rg 13) Register SUNW.HAStoragePlus if not already done: prinaryA# clrt register SUNA.HAStoragePlus hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 510(09/1212017 Configure fallover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic's Blog 14) Create HAStoragePlus resource for boot device: primarya# clrs create -g fldom-rg -t SUNW.HAStoragePlus -p GlobalDevicePaths=/dev/global/dsk/d7s2 fgd@-has-rs To use d7s2 is a requirement!!! 15) Enable LDom resrouce group on current node: primarya® clrg online -M -n primaryA fldom-rg 16) Register SUNW.Idom. primarya# clrt register SUNW.1dom 17) Setup password file for non-interactive ‘live’ migration on all primary nodes al1_primariest vi /var/cluster/.pwfgd@ add root password to this file all_primariest chnod 400 /var/cluster/.pwfgd® Reguirements: + The first line of the file must contain the password + The password must be plain text + The password must not exceed 256 characters in length Anewline character at the end of the password and all lines that follow the first line are ignored. These details from Performing Non-Interactive Migrations of Oracle VM Server for SPARC 3.0 Administration Guide Attention: If you are using SUNW.|dom:6 or higher this kind of password setup will fail. Also the alternative 17a) does not work with SUNW.ldom:6 or hitps:Pbogs oracle, comyjschiconfigure-fllover-ldom-araclo-ym-server-fr-sparc-on-solaris-lustr-41-by-using-tve-migration 610(09/1212017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic’s Blog higher. For details please refer to my blog New resource type version for Solaris Cluster SUNW.ldom agent 17a) Alternative: Setup encrypted password file for non-interactive ‘live’ migration on all primary nodes al1_primariest echo “encrypted” > /var/cluster/.pwfgde all_primariest dd if=/dev/urandon of=/var/cluster/Idom_key bs=16 count=1 all_primariest chmod 400 /var/cluster/Idom_key |_primaries# echo fu_bar | /usr/sfw/bin/openss1 enc -aesi28 -e -pass var/cluster/1dom_key -out /opt/SUNWscxvm/.fgd@_passud i all_primaries# chmod 400 /opt/SUNWscxvm/.gd@_passwd The root password for failover LDom is "fu_bar" which will be encrypted. All files must be secured using "chmod 400". Both /var/cluster/idom_key and Jopt/SUNWscxvm/.{DOMAIN}_passwd file can NOT be placed in a different location and can NOT have a different name. Verify if encrypted password can be decrypted all_primariest /usr/sfw/bin/openssl enc -aesi28 - -pass file: /var/cluster/1dom_key ~in /opt/SUNascxvm/.fgd@_passwd More Details in DocID 1668567.1 Solaris Cluster HA-LDom Fails Doing ‘live’ Migration with "normal failover will be performed" or "Password cannot be longer than 256 characters" due to Wrong Value in ‘Password_file’ resource property 18) Create SUNW.Idom resource primarya# cirs create -g fldom-rg -t SUNW.ldom -p Domain_name=fgd0 -p Password file=/var/cluster/.pwfgd® -p Resource dependencies offline_restart=fgd0-has-rs fgda-rs Notice’: In Solaris Cluster 4.3 the "SPARC: How to Configure HA for Oracle VM Server Guide" recommends to use Resource_dependencies_offline_restart. Therefore | changed also the recommendation from Resource_dependencies to Resource_dependencies_offline_restart. Notice2: The domain configuration is retrieved by the “Idm list-constraints - x Idom” command from Solaris Cluster and stored in the CCR. This info is used to create or destroy the domain on the node where the resource group is brought online or offline hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 70(09/1212017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic’s Blog 19) Check Migration_type property. It should be MIGRATE for ‘live’ migration primaryA# clrs show -v fgde-rs | grep Migration_type If not MIGRATE then set it: primarya# clrs set -p Migration_type=MIGRATE fgd0-rs 20) To stop/start the SUNW.Idom resource primaryA# clrs disable fgde-rs primarya# clrs enable fgd0-rs 21) Verify the setup by switching failover LDom to other node and back. primarya# cirg switch -n primary8 fldon-rg primaryA# clrg switch -n primaryA fldon-re To monitor the migration process run ‘Idm list -o status fldgO' on the primary target domain. 22) Tune your timeout values depending on your system primaryA# clrs set -p STOP_TIMEOUT=1200 fgd@-rs Details in DocID 1423937.1 Solaris Cluster: HA LDOM Migration Fails With “Migration of domain timed out, the domain state is now shut off" 23) Consider further tuning of timeout values as described in SPARC: Tuning the HA for Oracle VM Server Fault Monitor of Oracle Solaris Cluster Data Service for Oracle VM Server for SPARC Guide For less frequent probing maybe the following setting can be used. primarya # clrs set p Thorough_probe_interval=180 -p Probe_timeout=90 fgdo-rs Last but not least, it's not supported to run a 2-node or more-node Solaris Cluster within a failover LDom! BUT with SC4.1 SRU4 or higher you can run a single-node Solaris Cluster hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration ato09/12/2017 Configure failover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using Ive" migration | Oracle Juergen Schlic's Blog within failover LDom. For details please refer to Application monitoring in Oracle VM for SPARC failover guest domain within Doc ID 1597319.1 Oracle Solaris Cluster Product Update Bulletin October 2013, Join the discussion Q aa Visit the Oracle Blog Contact Us Learn more Learn more SiteMap Legal Notices Terms of Use Privacy _Préférences de cookies hitps:Plogs oracle, comyjschiconfigure-falover-ldom-aracle-vm-server-fr-sparc-on-solaris-lustr-41-by-using-ive-migration 910(09/1212017 Configure fallover LDom (Oracle VM Server for SPARC) on Solaris Cluster 4.1 by using ive’ migration | Oracle Juergen Schlsic's Blog hitps:Pblogs oracle, comyjschiconfigure-fllover-ldom-araclo-ym-server-fr-sparc-on-solaris-lustr-41-by-using-tve-migration 10110