Upgrading Clusterware to patchset 3 (10.2.0.

4)
Posted on 10:44 PM by Rishi Mahajan Recently upgraded Oracle 10g RAC (10.2.0.1) to 10.2.0.4 patchset. Noticed something interesting while applying the patchset. I have my 10g RAC running on 10.2.0.1 (base version).I planned to apply patch as a ROLLING Upgrade. The patchset document says the patch can be applied as rolling upgrade but when you invoke the runInstaller for cluster upgrade, you won't get the option to select one of the two nodes (in 2 node cluster). Both the nodes are automatically selected and grayed out (we can't make any change). Initially i thought that my cluster is not being detected by runInstaller.Cancelled it and set the ORACLE_HOME,PATH etc...properly but got the same thing again...But later realised that it works like that only. The runInstaller completes the installation and prompts to shutdown the CRS on node 1 and then run root102.sh.This scripts bring up the cluster after execution and the process is repeated on node 2...So here i go... I have my 10.2.0.1 clusterware running [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host -----------------------------------------------------------ora.orcl.db application ONLINE ONLINE rac1 ora....l1.inst application ONLINE ONLINE rac1 ora....l2.inst application ONLINE ONLINE rac2 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application ONLINE ONLINE rac2 ora....C2.lsnr application ONLINE ONLINE rac2 ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 Node-1 [root@rac1 bin]# ./crsctl query crs softwareversion CRS software version on node [rac1] is [10.2.0.1.0] [root@rac1 bin]# ./crsctl query crs activeversion CRS active version on the cluster is [10.2.0.1.0] Node-2 [root@rac2 bin]# ./crsctl query crs softwareversion CRS software version on node [rac2] is [10.2.0.1.0] [root@rac2 bin]# ./crsctl query crs activeversion CRS active version on the cluster is [10.2.0.1.0]

.gsd application OFFLINE OFFLINE ora...SM2. db instance.. They would be brought down later.SM1.vip application OFFLINE OFFLINE ora.Click Next to move ahead.rac1.Applying the patchset.ons application OFFLINE OFFLINE ora..rac1.gsd application ONLINE ONLINE rac2 ora.orcl... [root@rac1 bin]# ...l2.C2..rac2./crsctl check crs CSS appears healthy CRS appears healthy EVM appears healthy 2) Unzip the patch (p6810189_10204_Linux-x86.inst application OFFLINE OFFLINE ora..lsnr application OFFLINE OFFLINE ora.rac2..asm application ONLINE ONLINE rac2 ora.rac2...zip) and invoke runInstaller 3) In the screenshot as shown below both the nodes are grayed out and there is not option to select one node (this was my assumpiton intially that rolling should apply one by one with node selection option).inst application ONLINE ONLINE rac2 ora.. 1) Stop service..C1.. asm and nodeapps on node 1 (rac1) [oracle@rac1 ~]$ srvctl stop instance -i orcl1 -d orcl [oracle@rac1 ~]$ srvctl stop asm -n rac1 [oracle@rac1 ~]$ srvctl stop nodeapps -n rac1 [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host -----------------------------------------------------------ora.lsnr application ONLINE ONLINE rac2 ora..ons application ONLINE ONLINE rac2 ora.l1..asm application OFFLINE OFFLINE ora..db application ONLINE ONLINE rac1 ora..vip application ONLINE ONLINE rac2 Don't stop the CRS daemons.rac1..

[root@rac1 bin]# [root@rac1 bin]# /u01/app/oracle/product/10..2.l2.rac1.0/crs/bin/crsctl stop crs /u01/app/oracle/product/10.0/crs/bin/crsctl stop crs Stopping resources....2.. it will prompt to run the following two scripts...l1.ons application OFFLINE OFFLINE .asm application OFFLINE OFFLINE ora.0/crs/bin/crsctl check crs Failure 1 contacting CSS daemon Cannot communicate with CRS Cannot communicate with EVM [root@rac1 bin]# [oracle@rac2 ~]$ crs_stat -t Name Type Target State Host -----------------------------------------------------------ora..orcl. [root@rac1 bin]# /u01/app/oracle/product/10..sh The first script stops the clusterware and after that second script completesthe patching. Successfully stopped CRS resources Stopping CSSD.(Each node one at a time) /u01/app/oracle/product/10..4) Once the installer completes..db application ONLINE ONLINE rac2 ora.2.SM1.gsd application OFFLINE OFFLINE ora.rac1..inst application ONLINE ONLINE rac2 ora. Shutting down CSS daemon.C1.2.inst application OFFLINE OFFLINE ora.0/crs/install/root102. Shutdown request successfully issued.lsnr application OFFLINE OFFLINE ora...

.0/crs Relinking some shared libraries. clscfg -upgrade completed successfully [root@rac1 install]# The above scripts will automatically startup the clusterware on node 1 (and nodeapps. Successfully accumulated necessary OCR keys.vip application ONLINE ONLINE rac2 [oracle@rac2 ~]$ [root@rac1 install]# /u01/app/oracle/product/10.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root WARNING: directory '/u01' is not owned by root Preparing to recopy patched init and RC scripts. Startup will be queued to init within 30 seconds./crs_stat -t Name Type Target State Host -----------------------------------------------------------ora. 10204 patch successfully applied.l1.rac2. This may take a while on some systems.0/crs/install/root102..asm and instance if they are enabled for startup) root@rac1 bin]# . Waiting for the patched CRS daemons to start.inst application ONLINE ONLINE rac1 . clscfg: EXISTING configuration version 3 detected..db application ONLINE ONLINE rac2 ora.2. Recopying init and RC scripts. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. privgrp 'root'.ons application ONLINE ONLINE rac2 ora.2. clscfg: version 3 is 10G Release 2. Operation successful.lsnr application ONLINE ONLINE rac2 ora.vip application OFFLINE OFFLINE ora. node : node 1: rac1 rac1-priv rac1 Creating OCR keys for user 'root'. WARNING: directory '/u01/app/oracle/product/10.2.ora..SM2.C2.sh Creating pre-patch directory for saving pre-patch clusterware files Completed patching clusterware files to /u01/app/oracle/product/10.asm application ONLINE ONLINE rac2 ora.rac2... /etc/profile: line 59: ulimit: open files: cannot modify limit: Operation not permitted Starting up the CRS daemons.rac1... . Relinking of patched files is complete.rac2.orcl.gsd application ONLINE ONLINE rac2 ora...

Successfully stopped CRS resources Stopping CSSD.507 So the software version of clusterware change to 10.0.inst application ONLINE ONLINE rac2 ora.2. db instance.0/crs/bin/crsctl stop crs Stopping resources..4.l2.ons application ONLINE ONLINE rac2 ora. Before that shutdown the resources i. asm and nodeapps on node 2 (rac2) [root@rac2 bin]# /u01/app/oracle/product/10. Shutdown request successfully issued.2...vip application ONLINE ONLINE rac2 [root@rac1 bin]# This is what i have in alertrac1.rac1.rac2.0... [root@rac1 bin]# .asm application ONLINE ONLINE rac2 ora.. 2009-09-10 14:32:18.661 [crsd(29700)]CRS-1201:CRSD started on node rac1. 2009-09-10 14:32:33. 2009-09-10 14:32:31.lsnr application ONLINE ONLINE rac2 ora.2.1.vip application ONLINE ONLINE rac1 ora.C2.424 [client(30665)]CRS-1804:Node rac1 upgraded to version 10.0..gsd application ONLINE ONLINE rac1 ora. 2009-09-10 14:32:18. Stop service.SM1.rac1. 2009-09-10 14:32:20.log under $CRS_HOME/log/rac1 [cssd(30267)]CRS-1601:CSSD Reconfiguration complete.0.e. Active nodes are rac1 rac2 ..0] [root@rac1 bin]# 5) Now is the time to repeat the scripts on node 2 (rac2).4..rac1.2./crsctl query crs activeversion CRS active version on the cluster is [10. [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ .0..ora.2...gsd application ONLINE ONLINE rac2 ora..880 [evmd(29572)]CRS-1401:EVMD started on node rac1.asm application ONLINE ONLINE rac1 ora.rac2.4 but it would be active only after upgrade of second node./crsctl query crs softwareversion CRS software version on node [rac1] is [10.lsnr application ONLINE ONLINE rac1 ora.927 [crsd(29700)]CRS-1012:The OCR service started on node rac1.SM2.C1..ons application ONLINE ONLINE rac1 ora.rac2.0] [root@rac1 bin]# .. Shutting down CSS daemon.

SM2.orcl.db application ONLINE OFFLINE ora. /etc/profile: line 59: ulimit: open files: cannot modify limit: Operation not pe rmitted Starting up the CRS daemons.. node : node 2: rac2 rac2-priv rac2 Creating OCR keys for user 'root'.rac1.rac2. Relinking of patched files is complete. privgrp 'root'.l2.ora. clscfg: version 3 is 10G Release 2... .... 10204 patch successfully applied./crsctl query crs softwareversion .rac1.inst application ONLINE OFFLINE ora. Operation successful... This may take a while on some systems... Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.vip application ONLINE OFFLINE [root@rac2 install]# /u01/app/oracle/product/10.C2.lsnr application ONLINE OFFLINE ora..gsd application ONLINE ONLINE rac1 ora.2. Successfully accumulated necessary OCR keys..rac2.rac2.l1.asm application ONLINE OFFLINE ora. Startup will be queued to init within 30 seconds.0' is not owned by root WARNING: directory '/u01/app/oracle/product' is not owned by root WARNING: directory '/u01/app/oracle' is not owned by root WARNING: directory '/u01/app' is not owned by root WARNING: directory '/u01' is not owned by root Preparing to recopy patched init and RC scripts.vip application ONLINE ONLINE rac1 ora..sh Creating pre-patch directory for saving pre-patch clusterware files Completed patching clusterware files to /u01/app/oracle/product/10. clscfg: EXISTING configuration version 3 detected..ons application ONLINE OFFLINE ora.2..rac1.gsd application ONLINE OFFLINE ora. Waiting for the patched CRS daemons to start.0/crs/install/root102.C1. WARNING: directory '/u01/app/oracle/product/10.SM1.0/crs Relinking some shared libraries.asm application ONLINE ONLINE rac1 ora.. Recopying init and RC scripts.inst application ONLINE OFFLINE ora. clscfg -upgrade completed successfully [root@rac2 install]# [root@rac2 bin]# .2..lsnr application ONLINE ONLINE rac1 ora..ons application ONLINE ONLINE rac1 ora..

gsd application ONLINE ONLINE rac1 ora.0] [root@rac2 bin]# At the end of this script.2.C2....0] [root@rac2 bin]# ..SM2...inst application ONLINE ONLINE rac1 ora.4.CRS software version on node [rac2] is [10./crsctl query crs activeversion CRS active version on the cluster is [10.db application ONLINE ONLINE rac1 ora.2.l1.SM1.0.rac2..2.0.4.asm application ONLINE ONLINE rac2 ora.vip application ONLINE ONLINE rac2 [oracle@rac1 ~]$ This patched the CRS to 10.ons application ONLINE ONLINE rac2 ora...lsnr application ONLINE ONLINE rac2 ora..0.rac2.rac1. [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host -----------------------------------------------------------ora.lsnr application ONLINE ONLINE rac1 ora.l2.rac2..inst application ONLINE ONLINE rac2 ora..gsd application ONLINE ONLINE rac2 ora.. ..orcl.vip application ONLINE ONLINE rac1 ora..C1. everyhting would be brought up automatically..ons application ONLINE ONLINE rac1 ora.rac1...rac1...4.asm application ONLINE ONLINE rac1 ora.

Sign up to vote on this title
UsefulNot useful