Professional Documents
Culture Documents
by Vincent Chan
Learn how to convert a single-instance database to Oracle Real Application Clusters (RAC) 10g
on Red Hat Enterprise Linux 3 (RHEL3), step by step.
Contents
Overview
Step 1: Preliminary Installation
Step 2: Migrate Your Database to ASM
Step 3: Install Cluster Ready Services (CRS) Software
Step 4: Install Oracle RAC Software
Step 5: Post Installation
Step 6: Test Transparent Application Failover (TAF)
Conclusion
For those with limited hands-on experience, implementing Oracle RAC 10g can be an
intimidating prospect. But it doesn't have to be that way.
In this guide, I'll provide the precise procedures for converting a single-instance Oracle 10g
database to Oracle RAC on RHEL3. We'll use Oracle Automatic Storage Management (ASM),
an integrated file system and a volume management solution built into the Oracle database
kernel, for RAC database storage because it simplifies database storage management tasks while
offering storage reliability and performance.
To summarize on a very high level, there are several ways to convert your database to RAC. If
the database is small, you can consider installing a new RAC database on ASM and
export/import your data from your current database to the RAC database. For a larger database,
the preferred method would be to use RMAN to migrate the database files over to ASM. The
method demonstrated here is a two-phase approach: first, migrating the data to ASM, and
second, converting the single-instance database to RAC. If you are new to ASM, I would
recommend taking this migration path to get familiar with ASM before leaping into ASM and
RAC at the same time. (For more information about Oracle ASM, visit the Oracle ASM Product
Center or refer to the documentation.)
This guide requires a basic understanding of RAC architecture and some familiarity with
managing and administering Oracle Database and Red Hat Enterprise Linux. Refer to the
documentation for details.
Overview
The RAC cluster comprises two Intel x86 servers running on RHEL3 (Kernel 2.4.21-27). Each
node has access to a shared storage and connectivity to the public and private network.
1. Preliminary Installation
2. Migrating Your Database to ASM
3. Installing Oracle Cluster Ready Services (CRS) Software
4. Installing Oracle RAC Software
5. Post Installation
6. Testing Transparent Application Failover (TAF)
Unless otherwise specified, you should execute all steps on both nodes.
Here's an overview of our single-instance database environment before converting to RAC:
Database File
Host Name Instance Name Database Name $ORACLE_BASE
Storage
salmon1 prod1 prod1 /u01/app/oracle ext3
You'll install the Oracle Home on each node for redundancy.The ASM and RAC instances share
the same Oracle Home on each node.
Step 1: Preliminary Installation
1a. Verify software package versions.
Install the required packages. Additional information can be obtained from the documentation.
Verify the following kernel parameters. Additional information can be obtained from the
documentation.
Using the information below on node 1, create the oracle user and the oinstall and dba groups on
the second node.
[oracle@salmon1]$ hostname
salmon1.dbsconsult.com
[oracle@salmon1]$
[oracle@salmon1]$ id
uid=500(oracle) gid=500(dba) groups=500(dba),501(oinstall)
[oracle@salmon1]$
[oracle@salmon1]$ echo $ORACLE_BASE
/u01/app/oracle
1d. Edit the oracle user environment file.
[oracle@salmon1]$ more .bash_profile
# .bash_profile
export PATH=$PATH:$HOME/bin
export ORACLE_SID=prod1
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.1.0
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
umask 022
1e. Configure the oracle user shell limits.
[root@salmon1]# more /etc/security/limits.conf
* soft nproc 2047
* hard nproc 16384
* soft nofile 1024
* hard nofile 65536
[root@salmon1]# grep pam_limits /etc/pam.d/login
session required /lib/security/pam_limits.so
1f. Configure public and private network.
Using the information below, make the necessary changes to network interface devices eth0
(public) and eth1 (private).
[root@salmon1]# redhat-config-network
Host Name IP Address Type
salmon1.dbsconsult.com 192.168.0.184 Public (eth0)
salmon2.dbsconsult.com 192.168.0.185 Public (eth0)
salmon1.dbsconsult.com 10.10.10.84 Private (eth1)
salmon2.dbsconsult.com 10.10.10.85 Private (eth1)
salmon1-
192.168.0.186 Virtual
vip.dbsconsult.com
salmon2-
192.168.0.187 Virtual
vip.dbsconsult.com
During the Cluster Ready Services (CRS) and RAC installation, the Oracle Universal Installer
(OUI) has to be able to copy the software as oracle to all RAC nodes without being prompted for
a password. In Oracle 10g, this can be accomplished using ssh instead of rsh.
To establish user equivalence, generate the user's public and private keys as the oracle user on
both nodes.
The hangcheck timer kernel module monitors the system's health and restarts a failing RAC
node. It uses two parameters, hangcheck_tick (defines the system checks frequency) and
hangcheck_margin (defines the maximum hang delay before a RAC node is reset), to determine
if a node is failing.
Add the following line in /etc/rc.d/rc.local to load the hangcheck module automatically.
Make sure that the entries below are sized appropriately in the control file before converting to
RAC. If required, recreate the database control file with the right settings.
MAXLOGFILES
MAXLOGMEMBERS
MAXDATAFILES
MAXINSTANCES
MAXLOGHISTORY
1k. Resize the database buffer cache.
When transitioning from a single instance database to RAC, additional memory is required for
the database buffer cache. In RAC, space is allocated for the Global Cache Service (GCS) in
every block buffer cache. The amount of additional memory requires depends on how the
application accesses the data—that is, if the same block is cached in more than one instance.
The Oracle Cluster Synchronization Services (CSS) daemon is required for synchronization
between the ASM instance and the database instances. The CSS daemon must be up before the
ASM instance can be started. When you installed or upgraded to Oracle Database 10g, the CSS
should have already been configured for the single-node version and it should start automatically
when the system reboots.
oracleasm-support-2.0.0-1.i386.rpm
oracleasm-2.4.21-27.EL-1.0.4-2.i686.rpm (driver for UP kernel) or oracleasm-2.4.21-
27.ELsmp-1.0.4-1.i686.rpm (driver for SMP kernel)
oracleasmlib-2.0.0-1.i386.rpm
Create the ASM disks on any one node as the root user.
Using the orapwd utility, create an orapw+ASM1A file in $ORACLE_HOME/dbs on the first
node.
[oracle@salmon1]$ cd $ORACLE_HOME/dbs
[oracle@salmon1]$ orapwd file=orapw+ASM1A password=sys entries=5
2g. Create the first ASM instance.
Create the first ASM instance on the first node. The second ASM instance will be created in Post
Installation after the CRS software is installed on the second node.
SQL> startup
ORACLE instance started.
Create a spfile immediately after the ASM instance starts. With spfile, any newly created disk
groups are automatically added to the spfile.
File created.
2i. Create disk groups.
Create three disk groups: DG1, DG2, and RECOVERYDEST. DG1 and DG2 will be used to
store Oracle data files and redo logs. RECOVERYDEST will be used as the flash recovery area.
Diskgroup created.
Diskgroup created.
Diskgroup created.
NAME TOTAL_MB
------------------------- -------------------
DG1 36864
DG2 36864
RECOVERYDEST 73728
3 rows selected.
10 rows selected.
2j. Configure flash recovery area.
Database altered.
System altered.
System altered.
2k. Migrate data files to ASM.
You must use RMAN to migrate the data files to ASM disk groups. All data files will be
migrated to the newly created disk group, DG1. The redo logs and control files are created in
DG1 and DG2. In a production environment, you should store redo logs on different set of disks
and disk controllers from the rest of the Oracle data files.
System altered.
System altered.
database mounted
released channel: ORA_DISK_1
database opened
RMAN> exit
TABLESPACE FILE_NAME
--------------------- -----------------------------------------
USERS +DG1/prod1/datafile/users.260.1
SYSAUX +DG1/prod1/datafile/sysaux.258.1
UNDOTBS1 +DG1/prod1/datafile/undotbs1.259.1
SYSTEM +DG1/prod1/datafile/system.257.1
2l. Migrate temp tablespace to ASM.
SQL> alter tablespace temp add tempfile size 100M;
Tablespace altered.
FILE_NAME
-------------------------------------
+DG1/prod1/tempfile/temp.264.3
2m. Migrate redo logs to ASM.
Drop existing redo logs and recreate them in ASM disk groups, DG1 and DG2.
System altered.
System altered.
GROUP# MEMBER
--------------- ----------------------------------
1 /u03/oradata/prod1/redo01.log
2 /u03/oradata/prod1/redo02.log
Database altered.
System altered.
Database altered.
Database altered.
Database altered.
System altered.
Database altered.
GROUP# MEMBER
--------------- ----------------------------------------
1 +DG1/prod1/onlinelog/group_1.265.3
1 +DG2/prod1/onlinelog/group_1.257.1
2 +DG1/prod1/onlinelog/group_2.266.3
2 +DG2/prod1/onlinelog/group_2.258.1
2n. Create pfile from spfile.
Create and retain a copy of the database pfile. You'll add more RAC specific parameters to the
pfile later, in the Post Installation.
File created.
2o. Add additional control file.
If an additional control file is required for redundancy, you can create it in ASM as you would on
any other filesystem.
Database altered.
System altered.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
NAME
---------------------------------------
+DG1/cf1.dbf
+DG2/cf2.dbf
After successfully migrating all the data files over to ASM, the old data files are no longer
needed and can be removed. Your single-instance database is now running on ASM!
CRS requires two files—the Oracle Cluster Registry (OCR) and the Voting Disk—on shared raw
devices or Oracle Cluster File System (OCFS). These files must be accessible to all nodes in the
cluster. Raw devices are used here to house both files.
The storage size for the OCR should be at least 100MB and the storage size for the voting disk
should be at least 20MB.
Before installing the CRS software, shut down the listener, database, and ASM instance. Mount
the CRS CD or download the software from OTN. The OUI should be launched on only the first
node. During installation, the installer automatically copies the software to the second node.
[oracle@salmon1]$ /u01/app/oracle/product/10.1.0/crs_1/bin/olsnodes -n
salmon1 1
salmon2 2
[oracle@salmon1]$ ps -ef | egrep "css|crs|evm"
export PATH=$PATH:$HOME/bin
export ORACLE_SID=prod1a
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/10.1.0/db_1
export ORA_CRS_HOME=$ORACLE_BASE/product/10.1.0/crs_1
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
umask 022
4b. Install RAC software.
Mount the Oracle Database 10g Enterprise Edition CD or download the software from OTN.
Launch the OUI on only the first node. During installation, the installer automatically copies the
software to the second node.
[oracle@salmon1]$ . ~/.bash_profile
[oracle@salmon1]$ /mnt/cdrom/runInstaller
The Network Configuration Assistant (NETCA) should only be launched and configured on one
node. At the end of the configuration process, the NETCA starts up the Oracle listener on both
nodes.
[oracle@salmon1]$ netca
On node 1:
On node 1:
LISTENER_SALMON1 =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon1-vip)(PORT = 1521))
)
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.184)(PORT = 1521))
)
)
)
SID_LIST_LISTENER_SALMON1 =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u01/app/oracle/product/10.1.0/db_1)
(PROGRAM = extproc)
)
)
On node 2:
[oracle@salmon2]$ more $ORACLE_HOME/network/admin/listener.ora
LISTENER_SALMON2 =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon2-vip)(PORT = 1521))
)
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.185)(PORT = 1521))
)
)
)
SID_LIST_LISTENER_SALMON2 =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u01/app/oracle/product/10.1.0/db_1)
(PROGRAM = extproc)
)
)
4f. tnsnames.ora file
On both nodes:
PROD1 =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon1-vip)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon2-vip)(PORT = 1521))
(LOAD_BALANCE = yes)
(CONNECT_DATA =
(SERVICE_NAME = PROD1)
(FAILOVER_MODE =
(TYPE = SELECT)
(METHOD = BASIC)
(RETRIES = 200)
(DELAY = 5)
)
)
)
PROD1A =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon1-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = PROD1)
(INSTANCE_NAME = PROD1A)
)
)
PROD1B =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = salmon2-vip)(PORT = 1521))
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = PROD1)
(INSTANCE_NAME = PROD1B)
)
)
Copy the ASM instance password file and spfile+ASM1A.ora to the new Oracle Home.
[oracle@salmon1]$ cp /u01/app/oracle/product/10.1.0/dbs/orapw+ASM1A
/u01/app/oracle/product/10.1.0/db_1/dbs
[oracle@salmon1]$ cp /u01/app/oracle/product/10.1.0/dbs/spfile+ASM1A.ora
/u01/app/oracle/product/10.1.0/db_1/dbs
5b. Create init.ora for the second ASM instance.
Using the orapwd utility, create an orapw+ASM1B file in $ORACLE_HOME/dbs on the second
node.
SQL> startup
ORACLE instance started.
File created.
For higher availability, register the ASM instances under the CRS framework. When registered,
the CRS should detect any failed instances and automatically attempt to start up the instances.
The CRS should also automatically start up the instances when the servers are rebooted.
On node 1:
Using the orapwd utility, create an orapwprod1a file in $ORACLE_HOME/dbs on the first node
and an orapwprod1b file in $ORACLE_HOME/dbs on the second node.
Modify the pfile saved in Step 2n. Add and modify the following parameters:
*.cluster_database_instances=2
*.cluster_database=true
*.remote_listener='LISTENERS_PROD1’
prod1a.thread=1
prod1a.instance_number=1
prod1a.undo_tablespace='UNDOTBS1'
prod1b.thread=2
prod1b.instance_number=2
prod1b.undo_tablespace='UNDOTBS2'
5h. Modify init.ora of both RAC instances.
On node 1:
File created.
Create the RAC data dictionary views on the first RAC instance.
On node 1:
GROUP# MEMBER
--------------- ----------------------------------------
1 +DG1/prod1/onlinelog/group_1.265.3
1 +DG2/prod1/onlinelog/group_1.257.1
2 +DG1/prod1/onlinelog/group_2.266.3
2 +DG2/prod1/onlinelog/group_2.258.1
3 +DG1/prod1/onlinelog/group_3.268.1
3 +DG2/prod1/onlinelog/group_3.259.1
4 +DG1/prod1/onlinelog/group_4.269.1
4 +DG2/prod1/onlinelog/group_4.260.1
8 rows selected.
Database altered.
5m. Create undo tablespace for the second RAC instance.
SQL> create undo tablespace UNDOTBS2 datafile size 200M;
TABLESPACE FILE_NAME
--------------------- --------------------------------------
UNDOTBS2 +DG1/prod1/datafile/undotbs2.270.1
5n. Start up the second RAC instance.
[oracle@salmon1]$ srvctl start instance -d prod1 -i prod1b
[oracle@salmon1]$ crs_stat -t
Name Type Target State Host
-----------------------------------------------------------------------
ora....1a.inst application ONLINE ONLINE salmon1
ora....1b.inst application ONLINE ONLINE salmon2
ora.prod1.db application ONLINE ONLINE salmon1
ora....M1A.asm application ONLINE ONLINE salmon1
ora....M1B.asm application ONLINE ONLINE salmon2
ora....N1.lsnr application ONLINE ONLINE salmon1
ora....on1.gsd application ONLINE ONLINE salmon1
ora....on1.ons application ONLINE ONLINE salmon1
ora....on1.vip application ONLINE ONLINE salmon1
ora....N2.lsnr application ONLINE ONLINE salmon2
ora....on2.gsd application ONLINE ONLINE salmon2
ora....on2.ons application ONLINE ONLINE salmon2
ora....on2.vip application ONLINE ONLINE salmon2
[oracle@salmon1]$ srvctl status database -d prod1
Instance prod1a is running on node salmon1
Instance prod1b is running on node salmon2
[oracle@salmon1]$ srvctl stop database -d prod1
[oracle@salmon1]$ srvctl start database -d prod1
[oracle@salmon1]$ sqlplus system/system@prod1
Connected to:
Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 - Production
With the Partitioning, Real Application Clusters, OLAP and Data Mining options
The failover mechanism in Oracle TAF enables any failed database connections to reconnect to
another node within the cluster. The failover is transparent to the user. Oracle re-executes the
query on the failed over instance and continues to display the remaining results to the user.
To demonstrate the failover mechanism of the TAF option, connect to two different database
sessions and execute these steps:
If the returned output of failover_type and failover_mode is 'NONE', verify that the PROD1
service is configured correctly in tnsnames.ora.
Connect as the sys user on prod1a instance and shut down the instance.
From the same session in Step 6a, execute the queries below to verify that the session has failed
over to another instance.
Conclusion
With proper planning and understanding of the RAC architecture, the transition from a single-
instance database to a RAC configuration is not necessarily complex. ASM and RAC
compliment each other to provide higher levels of availability, scalability, and business
continuity. Hopefully, this guide has provided a clear and concise method of performing the
conversion.