By Rajani Kumar Katam, Oracle RAC DBA. Satyam Computer Services private Ltd.

Step by step installation Oracle 11g (11.1.0.6.0) RAC on Red hat Enterprise LINUX AS 4 with screenshots.

The following are the sequence of steps that are to be executed on the nodes.
Install the Linux Operating System Install Required Linux Packages for Oracle RAC (refer oracle doc for the required packages Packages varies depending on the version of the operating system). Network Configuration Using the Network Configuration application, you need to configure both NIC devices as well as the /etc/hosts file. Both of these tasks can be completed using the Network Configuration GUI. Notice that the /etc/hosts settings are the same for both nodes.

For example we need to specify the entries in the /etc/hosts file as below in both the nodes.

etc/hosts
127.0.0.1 localhost.localdomain localhost # Public Network - (eth0) 192.168.1.100 linux1 192.168.1.101 linux2 # Private Interconnect - (eth1) 192.168.2.100 linux1-priv 192.168.2.101 linux2-priv # Public Virtual IP (VIP) addresses - (eth0) 192.168.1.200 linux1-vip 192.168.1.201 linux2-vip

Create "oracle" User and Directories
groupadd -g 501 oinstall groupadd -g 502 dba groupadd -g 504 asm

useradd -m -u 501 -g oinstall -G dba, asm -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle # mkdir -p /u01/app/oracle # chown -R oracle:oinstall /u01/app # chmod -R 775 /u01/app Creating directory for oracle clusterware. # mkdir -p /u01/app/crs # chown -R oracle:oinstall /u01/app/crs # chmod -R 775 /u01/app/crs

Create Mount Point for OCFS2 / Clusterware Let's now create the mount point for the Oracle Cluster File System, Release 2 (OCFS2) that will be used to store the two Oracle Clusterware shared files (OCR file and voting disk file)
# mkdir -p /u02/oradata/orcl # chown -R oracle:oinstall /u02/oradata/orcl # chmod -R 775 /u02/oradata/orcl

‡

Configure the Linux Servers for Oracle

Edit the .bash_profile file and set the required environment variables in both the nodes.

PATH=$PATH:$HOME/bin export ORACLE_SID=hrms1 export ORACLE_HOME=/u02/app/oracle/db_home export ORA_CRS_HOME=/u02/app/oracle/crs_home export PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/lib unset USERNAME

Swap Space Considerations Installing Oracle Database 11g Release 1 requires a minimum of 1GB of memory

‡ ‡

# cat /proc/meminfo | grep MemTotal # cat /proc/meminfo | grep SwapTotal

Configuring Kernel Parameters and Shell Limits In both the nodes
sysctl -p net.ipv4.ip_forward = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.default.accept_source_route = 0 kernel.sysrq = 0 kernel.core_uses_pid = 1 net.ipv4.tcp_syncookies = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 net.core.rmem_default = 4194304 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 262144 kernel.shmmax = 1073741823 kernel.sem = 250 32000 100 128 fs.file-max = 65536 net.ipv4.ip_local_port_range = 1024 65000

Setting Shell Limits for the oracle User
cat >> oracle oracle oracle oracle EOF ‡ /etc/security/limits.conf <<EOF soft nproc 2047 hard nproc 16384 soft nofile 1024 hard nofile 65536

Configure RAC Nodes for Remote Access using SSH

(Refer any UNIX doc for configuring SSH or oracle RAC installation doc)

‡

Enabling SSH User Equivalency for the Current Shell Session
$ exec /usr/bin/ssh-agent $SHELL $ /usr/bin/ssh-add Enter passphrase for /home/oracle/.ssh/id_rsa: xxxxx

Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)

‡ ‡

Reboot both nodes after configuring kernel level Install & Configure Oracle Cluster File System (OCFS2) parameters # rpm -Uvh ocfs2-tools-1.2.6-1.el5.i386.rpm # rpm -Uvh ocfs2-2.6.18-8.el5-1.2.6-1.el5.i686.rpm # rpm -Uvh ocfs2console-1.2.6-1.el5.i386.rpm

Configure OCFS2
$ su # ocfs2console &

Select [Cluster] -> [Configure Nodes...]. This will start the OCFS2 Cluster

‡

On the "Node Configuration" dialog, click the [Add] button. ‡ This will bring up the "Add Node" dialog. In the "Add Node" dialog, enter the Host name and IP address for the first node in the cluster. Leave the IP Port set to its default value of 7777. In my example, I added both nodes using linux1 / 192.168.1.100 for the first node and linux2 / 192.168.1.101 for the second node ‡ ‡ Click [Apply] on the "Node Configuration" dialog - All nodes should now be "Active". After verifying all values are correct, exit the application using [File] -> [Quit].

‡

This needs to be performed on both Oracle RAC nodes in the cluster.

Configure O2CB to Start on Boot
# /etc/init.d/o2cb offline ocfs2 # /etc/init.d/o2cb unload # /etc/init.d/o2cb configure

Format the OCFS2 Filesystem Create a partition on the SAN or shared storage for storing ocrfile and voting disk files that are created at the time of cluster ware installation. (use fdisk command as root user and create a partition) NOTE: Always recommended to create 4 partitions so that we can maintain redundant copies of voting disk file and OCR file
$ su -

# /etc/init.0.rpm Configuring and Loading the ASMLib Packages $ su # /etc/init.Pre-Installation Tasks for Oracle Clusterware 11g Verifying the Hardware and Operating System Setup with CVU $ .nointr ±L ³ocfs2´ /u02/oradata/orcl of the file system where u want to mount tat partition).ocfs2 -b 4K -C 32K -N 4 ±L ocfs2 /dev/sde2 Mount the OCFS2 File system $ su # mount -t ocfs2 -o datavolume.d/oracleasm createdisk VOL2 NOTE: Create the number of disks depending on your requirement.i686.el5-2./runcluvfy.sh stage -post hwos -n hcslinux1. (the name Configure OCFS2 to Mount Automatically at Startup We can do that by adding the following line to the /etc/fstab file on both Oracle RAC nodes in the cluster: LABEL=ocfs2 /u02/oradata/orcl ocfs2 _netdev.0.d/oracleasm configure Create ASM Disks for Oracle $ su # /etc/init.i386.d/oracleasm scandisks # /etc/init.datavolume. hcslinux2 ±verbose 11.el5.Installing Oracle clusterware software Note: Before installing clusterware please verify remote host access and user equivalence using ssh.0.4-1.6.i386.nointr 0 0 10.# mkfs. $ sh runInstaller .el5.0) # rpm -Uvh oracleasm-support-2.18-8.Install & Configure Automatic Storage Management libraries (ASMLib 2.rpm # rpm -Uvh oracleasmlib-2.4-1.rpm # rpm -Uvh oracleasm-2.3-1.d/oracleasm listdisks VOL1 VOL2 VOL3 VOL4 /dev/sde2 10 .el5.d/oracleasm createdisk vol1 /dev/sde1 # /etc/init.

.

.

.

.

.

.

.

.

.

.

.

[root@hcslnx01 crs_home]# sh root.sh Checking to see if Oracle CRS stack is already configured Setting the permissions on OCR backup directory Setting up Network socket directories Oracle Cluster Registry configuration upgraded successfully Successfully accumulated necessary OCR keys. Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: hcslnx01 hcslnx01-priv hcslnx01 node 2: hcslnx02 hcslnx02-priv hcslnx02 .

Successfully accumulated necessary OCR keys. clscfg: version 4 is 11 Release 1.sh WARNING: directory '/' is not owned by root Checking to see if Oracle CRS stack is already configured Setting the permissions on OCR backup directory Setting up Network socket directories Oracle Cluster Registry configuration upgraded successfully The directory '/' is not owned by root. Now formatting voting device: /ocfs2/voting_file Format of 1 voting devices complete. Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds.Creating OCR keys for user 'root'. Startup will be queued to init within 30 seconds. hcslnx01 Cluster Synchronization Services is inactive on these nodes.. privgrp 'root'. Cluster Synchronization Services is active on these nodes. Changing owner to root clscfg: EXISTING configuration version 4 detected. . Run root. hcslnx02 Local node checking complete. Operation successful.sh on remaining nodes to start CRS daemons. [root@hcslnx01 crs_home]# [root@hcslnx02 crs_home]# sh root.

Creating GSD application resource on (2) nodes.. Adding daemons to inittab Expecting the CRS daemons to be up within 600 seconds.. . Starting VIP application resource on (2) nodes. Cluster Synchronization Services is active on these nodes.. Oracle Cluster Registry for cluster has already been initialized Startup will be queued to init within 30 seconds...Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.. Supply -force parameter to override.. NO KEYS WERE WRITTEN.. node <nodenumber>: <nodename> <private interconnect name> <hostname> node 1: hcslnx01 hcslnx01-priv hcslnx01 node 2: hcslnx02 hcslnx02-priv hcslnx02 clscfg: Arguments check out successfully. Creating ONS application resource on (2) nodes. hcslnx01 hcslnx02 Cluster Synchronization Services is active on all the nodes. Waiting for the Oracle CRSD and EVMD to start Oracle CRS stack installed and running under init(1M) Running vipca(silent) for configuring nodeapps Creating VIP application resource on (2) nodes. -force is destructive and will destroy any previous cluster configuration.

. Done. Starting ONS application resource on (2) nodes.. You have new mail in /var/spool/mail/root [root@hcslnx02 crs_home]# ...Starting GSD application resource on (2) nodes.

gsd application 0/5 0/0 ONLINE ONLINE linux2 ora.ons application 0/3 0/0 ONLINE ONLINE linux2 ora.gsd application 0/5 0/0 ONLINE ONLINE linux1 ora.12.vip application 0/0 0/0 ONLINE ONLINE linux2 Check CRS Status $ $ORA_CRS_HOME/bin/crsctl check crs Cluster Synchronization Services appears healthy Cluster Ready Services appears healthy .linux1.linux1.linux2.linux2.ons application 0/3 0/0 ONLINE ONLINE linux1 ora.linux1.linux2. Verify Oracle Clusterware Installation Check Cluster Nodes $ $ORA_CRS_HOME/bin/olsnodes -n linux1 1 linux2 2 Confirm Oracle Clusterware Function $ $ORA_CRS_HOME/bin/crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------ora.vip application 0/0 0/0 ONLINE ONLINE linux1 ora.

Event Manager appears healthy Check Oracle Clusterware Auto-Start Scripts $ ls -l /etc/init.crsd /etc/init.d/init.cssd /etc/init.d/init.* -rwxr-xr-x 1 root root 2236 -rwxr-xr-x 1 root root 5290 -rwxr-xr-x 1 root root 49416 -rwxr-xr-x 1 root root 3859 Oct Oct Oct Oct 12 12 12 12 22:08 22:08 22:08 22:08 /etc/init. Install Oracle Database 11g Software $ sh runInstaller Installing Oracle 11g RAC software .crs /etc/init.d/init.evmd 13.d/init.d/init.

.

.

.

.

.

.

.

.

.

.

NOTE: The above command has to be executed manullay on the failed node by connecting as Oracle user. . Creating 11g RAC database by invoking DBCA utility.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

NOTE: There are five node-level tasks defined for SRVCTL: Adding and deleting node-level applications Setting and unsetting the environment for node-level applications Administering node applications Administering ASM instances Starting and stopping a group of programs that includes virtual IP addresses. and Oracle Enterprise Manager agents (for maintenance purposes). Status of all instances and services $ srvctl status database -d orcl Instance orcl1 is running on node linux1 Instance orcl2 is running on node linux2 Status of a single instance $ srvctl status instance -d orcl -i orcl2 Instance orcl2 is running on node linux2 Status of node applications on a particular node $ srvctl status nodeapps -n linux1 . listeners. Oracle Notification Services.

GSD.1.0/db_1 linux2 orcl2 /u01/app/oracle/product/11.VIP is running on node: linux1 GSD is running on node: linux1 Listener is running on node: linux1 ONS daemon is running on node: linux1 Status of an ASM instance $ srvctl status asm -n linux1 ASM instance +ASM1 is running on node linux1.255. Listener exists.255.0/eth0 GSD exists. ONS.1. Listener) $ srvctl config nodeapps -n linux1 -a -g -s -l VIP exists. Display the configuration for the ASM instance(s) $ srvctl config asm -n linux1 +ASM1 /u01/app/oracle/product/11. ONS daemon exists.: /linux1-vip/192.0/db_1 BY Rajani Kumar Katam Satyam Computer Services private Ltd.168. .200/255.0/db_1 Display the configuration for node applications .(VIP.1.1. List all configured databases $ srvctl config database orcl Display configuration for our RAC database $ srvctl config database -d orcl linux1 orcl1 /u01/app/oracle/product/11.

Sign up to vote on this title
UsefulNot useful