Sign In/Register for Account Help Select Country/Region

Communities

I am a...

I want to...

Search

PRODUCTS AND SERVICES Oracle Technology Network

SOLUTIONS Articles

DOWNLOADS

STORE

SUPPORT

TRAINING

PARTNERS

ABOUT

Oracle Technology Network

Page 1 Page 2 Page 3 Build Your Own Oracle RAC Cluster on Oracle Enterprise Linux and iSCSI (Continued) The information in this guide is not validated by Oracle, is not supported by Oracle, and should only be used at your own risk; it is for educational purposes only. 21. Install Oracle Grid Infrastructure for a Cluster Perform the following installation procedures from only one of the Oracle RAC nodes in the cluster (racnode1). The Oracle grid infrastructure software (Oracle Clusterware and Automatic Storage Management) will be installed to both of the Oracle RAC nodes in the cluster by the Oracle Universal Installer. You are now ready to install the "grid" part of the environment Oracle Clusterware and Automatic Storage Management. Complete the following steps to install Oracle grid infrastructure on your cluster. At any time during installation, if you have a question about what you are being asked to do, click the Help button on the OUI page. Typical and Advanced Installation Starting with 11g release 2, Oracle now provides two options for installing the Oracle grid infrastructure software:

Typical Installation The typical installation option is a simplified installation with a minimal number of manual configuration choices. This new option provides streamlined cluster installations, especially for those customers who are new to clustering. Typical installation defaults as many options as possible to those recommended as best practices.

Advanced Installation The advanced installation option is an advanced procedure that requires a higher degree of system knowledge. It enables you to select particular configuration choices, including additional storage and network choices, use of operating system group authentication for role-based administrative privileges, integration with IPMI, or more granularity in specifying Automatic Storage Management roles.

Given the fact that this article makes use of role-based administrative privileges and high granularity in specifying Automatic Storage Management roles, we will be using the "Advanced Installation" option. Configuring SCAN without DNS For the purpose of this article, although I indicated I will be manually assigning IP addresses using the DNS method for name resolution (as opposed to GNS), I will not actually be defining the SCAN in any DNS server (or GNS for that matter). Instead, I will only be defining the SCAN host name and IP address in the hosts file ( /etc/hosts) on each Oracle RAC node and any clients attempting to connect to the database cluster. Although Oracle strongly discourages this practice and highly recommends the use of GNS or DNS resolution, I felt it beyond the scope of this article to configure DNS. This section includes a workaround (Ok, a total hack) to the nslookup binary that allows the Cluster Verification Utility to finish successfully during the Oracle grid infrastructure install. Please note that the workaround documented in this section is only for the sake of brevity and should not be considered for a production implementation. Defining the SCAN in only the hosts file and not in either Grid Naming Service (GNS) or DNS is an invalid configuration and will cause the Cluster Verification Utility to fail during the Oracle grid infrastructure installation:

Figure 17: Oracle Grid Infrastructure / CVU Error - (Configuring SCAN without DNS)

INFO: INFO: INFO: INFO: INFO: INFO: INFO: INFO: INFO:

Checking Single Client Access Name (SCAN)... Checking name resolution setup for "racnode-cluster-scan"... ERROR: PRVF-4657 : Name resolution setup check for "racnode-cluster-scan" (IP address: 216.24.138.153) failed ERROR: PRVF-4657 : Name resolution setup check for "racnode-cluster-scan" (IP address: 192.168.1.187) failed ERROR: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "racnode-cluster-scan" Verification of SCAN VIP and Listener setup failed

Provided this is the only error reported by the CVU, it would be safe to ignore this check and continue by clicking the [Next] button in OUI and move forward with the Oracle grid infrastructure installation. This is documented in Doc ID: 887471.1 on the My Oracle Support web site. If on the other hand you want the CVU to complete successfully while still only defining the SCAN in the hosts file, simply modify the nslookup utility as root on both Oracle RAC nodes as follows. First, rename the original nslookup binary to nslookup.original on both Oracle RAC nodes: [root@racnode1 ~]#

mv /usr/bin/nslookup /usr/bin/nslookup.original

Next, create a new shell script named /usr/bin/nslookup as shown below while replacing 24.154.1.34 with your primary DNS, racnode-cluster-scan with your SCAN host name, and 192.168.1.187 with your SCAN IP address: #!/bin/bash HOSTNAME=${1} if [[ $HOSTNAME = " racnode-cluster-scan" ]]; then echo "Server: 24.154.1.34" echo "Address: 24.154.1.34#53" echo "Non-authoritative answer:" echo "Name: racnode-cluster-scan" echo "Address: 192.168.1.187" else /usr/bin/nslookup.original $HOSTNAME fi Finally, change the new nslookup shell script to executable: [root@racnode1 ~]#

chmod 755 /usr/bin/nslookup
Remember to perform these actions on both Oracle RAC nodes. The new nslookup shell script simply echo's back your SCAN IP address whenever the CVU calls nslookup with your SCAN host name; otherwise, it calls the original nslookup binary. The CVU will now pass during the Oracle grid infrastructure installation when it attempts to verify your SCAN:

[grid@racnode1 ~]$

cluvfy comp scan -verbose
Verifying scan Checking Single Client Access Name (SCAN)... SCAN VIP name Node Running? ListenerName Port Running? ---------------- ------------ ------------ ------------ ------------ -----------racnode-cluster-scan racnode1 true LISTENER 1521 true Checking name resolution setup for "racnode-cluster-scan"... SCAN Name IP Address Status Comment ------------ ------------------------ ------------------------ ---------racnode-cluster-scan 192.168.1.187

passed
Verification of SCAN VIP and Listener setup passed Verification of scan was successful. =============================================================================== [grid@racnode2 ~]$

cluvfy comp scan -verbose
Verifying scan Checking Single Client Access Name (SCAN)... SCAN VIP name Node Running? ListenerName Port Running?

converted by Web2PDFConvert.com

. verify your X11 display server settings which were described in the section.sh script can take several minutes to run. Configuring Intelligent Platform Management Interface (IPMI) is beyond the scope of this article.0 eth1 192.-----------------------. Open a new console window on both Oracle RAC nodes in the cluster. Run the root. Finish At the end of the installation. If the configuration assistants and CVU run successfully.sh script on both nodes in the RAC cluster: [root@racnode1 ~]# Specify Installation Location Create Inventory Prerequisite Checks Summary Setup Execute Configuration scripts /u01/app/oraInventory/orainstRoot.-----------. called fixup scripts. then Oracle Clusterware could encounter intermittent hangs. and completes other operating system configuration tasks. it would be safe to ignore this check and continue by clicking [Next] and then the [Close] button to exit the OUI. Enter the "OS Password" for the grid user and click the [Setup] button.168.-----------. the OUI will attempt to validate the SCAN information: SCAN Name racnode-cluster-scan SCAN Port 1521 Screen Shot Cluster Node Information Use this screen to add the node racnode2 to the cluster and to configure SSH connectivity.1. Provided this is the only error reported by the CVU. and Oracle Private Interconnect (VIPCA). do not click the [Next] button in OUI to bypass the error. Make any changes necessary to match the values in the table below: Interface Name Subnet eth0 192. When running root. Create an ASM Disk Group that will be used to store the Oracle Clusterware files according to the values in the table below: Interface Type Public Private Specify Network Interface Usage Storage Option Information Create ASM Disk Group Specify ASM Password Failure Isolation Support Privileged Operating System Groups Disk Group Name Redundancy Disk Path CRS External ORCL:CRSVOL1 For the purpose of this article.-----------------------. Next. configuration.loc The inventory is located at /u01/app/oraInventory 'UpdateNodeList' was successful. Instead. (starting with the node you are performing the install from). Identify the network interface to be used for the "Public" and "Private" network.sh script on both nodes in the RAC cluster one at a time starting with the node you are performing the install from: [root@racnode1 ~]# /u01/app/11. if you configured SCAN "only" in your hosts file ( /etc/hosts) and not in either Grid Naming Service (GNS) or manually using DNS. (starting with the node you are performing the install from). return to the OUI and click the [Retry] button. you can exit OUI by clicking [Next] and then [Close].168. The final step performed by OUI is to run the Cluster Verification Utility (CVU). click the [Close] button to exit the OUI.-----------. The installer will run configuration assistants for Oracle Net Services (NETCA). Make any changes necessary to match the values in the table below: OSDBA for ASM OSOPER for ASM OSASM asmdba asmoper asmadmin Set the "Oracle Base" ( $ORACLE_BASE) and "Software Location" ( $ORACLE_HOME) for the Oracle grid infrastructure installation: Oracle Base: /u01/app/grid Software Location: /u01/app/11. Instructions on how to configure Grid Naming Service (GNS) is beyond the scope of this article.1.0/grid/root.2. converted by Web2PDFConvert. Finish off this screen by clicking the [Test] button to verify passwordless SSH connectivity. If on the other hand you want the CVU to complete successfully while still only defining the SCAN in the hosts file. Run the orainstRoot.oracle or /var/tmp/.1201(asmdba). Starting with Oracle Clusterware 11g release 2 (11. log in to racnode1 as the owner of the Oracle grid infrastructure software which for this article is grid./runInstaller Screen Name Select Installation Option Select Installation Type Select Product Languages Grid Plug and Play Information Response Select " Install and Configure Grid Infrastructure for a Cluster" Select " Advanced Installation" Make the appropriate selection(s) for your environment.sh on the last node. Click [Next] and then [Close] to exit the OUI.-----------racnode-cluster-scan racnode1 true LISTENER 1521 true Checking name resolution setup for "racnode-cluster-scan". Configure Oracle Grid Infrastructure for a Cluster Go back to OUI and acknowledge the "Execute Configuration scripts" dialog window.187 passed Verification of SCAN VIP and Listener setup passed Verification of scan was successful.2). SCAN Name IP Address Status Comment -----------. Cluster Name racnode-cluster After clicking [Next]. The installer performs the Oracle grid infrastructure setup process on both Oracle RAC nodes.1202(asmoper) [grid@racnode1 ~]$ DISPLAY=<your local workstation>:0.0/grid Since this is the first install on the host. Use the default values provided by the OUI: Inventory Directory: /u01/app/oraInventory oraInventory Group Name: oinstall The installer will run through a series of checks to determine if both Oracle RAC nodes meet the minimum requirements for installing and configuring the Oracle Clusterware and Automatic Storage Management software.2. to resolve many incomplete system configuration requirements. Click the "Add" button to add " racnode2" and its virtual IP address " racnode2-vip" according to the table below: Public Node Name racnode1 racnode2 Virtual Host Name racnode1-vip racnode2-vip Next. Un-check the option to "Configure GNS".0 Select " Automatic Storage Management (ASM)". You will be prompted to run the script as root in a separate terminal session. When you run the script. After completing the steps document in that section. stay logged in as the root user account.2. as the root user account.0 [grid@racnode1 ~]$ export DISPLAY [grid@racnode1 ~]$ cd /home/grid/software/oracle/grid [grid@racnode1 grid]$ . Logging In to a Remote System Using X Terminal. follow the instructions in section Configuring SCAN without DNS to modify the nslookup utility.sh Within the same new console window on both Oracle RAC nodes in the cluster. Verify Terminal Shell Environment Before starting the Oracle Universal Installer. click the [SSH Connectivity] button. and you will encounter error CRS-0184: Cannot communicate with the CRS daemon.sh scripts. if you are using a remote client to connect to the Oracle RAC node performing the installation (SSH or Telnet to racnode1 from a workstation configured with an X Server). if necessary. If all prerequisite checks pass (as was the case for my install). Caution: After installation is complete. you will need to create the Oracle Inventory.---------------.0/grid/root. you will receive output similar to the following which signifies a successful install: . Select " Do not use Intelligent Platform Management Interface (IPMI)".-----------.sh and /u01/app/11.. The fixup script is generated during installation. If you remove these files.168. Install Oracle Grid Infrastructure Perform the following tasks as the grid user to install Oracle grid infrastructure: [grid@racnode1 ~]$ id uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall).1200(asmadmin). If OUI detects an incomplete task that is marked "fixable". the OUI continues to the Summary screen.2. you will be prompted to run the /u01/app/oraInventory/orainstRoot.oracle or its files while Oracle Clusterware is up. if any checks fail.. do not remove manually or run cron jobs that remove /tmp/. then you can easily fix the issue by generating the fixup script by clicking the [Fix & Check Again] button. acknowledge the dialog box. As described earlier in this section. This will start the "SSH Connectivity" configuration process: After the SSH configuration process successfully completes. this is considered an invalid configuration and will cause the Cluster Verification Utility to fail. The inventory pointer is located at /etc/oraInst. This is documented in Doc ID: 887471. This article makes use of role-based administrative privileges and high granularity in specifying Automatic Storage Management roles using a Job Role Separation. Click [Finish] to start the installation.com .1 on the My Oracle Support web site.0/grid/root. it raises kernel values to required minimums. I choose to " Use same passwords for these accounts".2. After the installation completes.sh The root.---------racnode-cluster-scan 192. Automatic Storage Management (ASMCA).sh [root@racnode2 ~]# /u01/app/11. the installer (OUI) will create shell script programs..sh [root@racnode2 ~]# /u01/app/oraInventory/orainstRoot. The CVU should now finish with no errors.

.up.0/grid [root@racnode2 grid]# cp root.2)..6.AFTER_INSTALL_NOV-20-2009 [root@racnode2 ~]# cd /u01/app/11.de2.t1. This article was written using Oracle Enterprise Linux 5 update 4 which uses the 2.de2.--------1.E1.type 0/5 0/ ONLINE ONLINE racnode1 ora..ry.er.. ONLINE 4cbbd0de4c694f50bfd3857ebd8ad8c4 (ORCL:CRSVOL1) [CRS] Located 1 voting disk(s). then you would use OS Watcher and RACDDT which is available through the My Oracle Support website (formerly Metalink).. If you require information contained in the original root.type 0/0 0/0 ONLINE ONLINE racnode1 Check Cluster Nodes [grid@racnode1 ~]$ olsnodes -n racnode1 racnode2 1 2 Check Oracle TNS Listener Process on Both Nodes [grid@racnode1 ~]$ ps -ef | grep lsnr | grep -v 'grep' | grep -v 'ocfs' | awk '{print $9}' LISTENER_SCAN1 LISTENER [grid@racnode2 ~]$ ps -ef | grep lsnr | grep -v 'grep' | grep -v 'ocfs' | awk '{print $9}' LISTENER Confirming Oracle ASM Function for Oracle Clusterware Files If you installed the OCR and voting disk files on Oracle ASM.de1. such as node evictions..dg ora.type 0/0 0/0 ONLINE ONLINE racnode2 ora.de1..gsd application 0/5 0/0 OFFLINE OFFLINE ora.. Check Oracle Cluster Registry (OCR) [grid@racnode1 ~]$ ocrcheck Status of Oracle Cluster Registry Version Total space (kbytes) Used space (kbytes) Available space (kbytes) ID Device/File Name is as follows : : 3 : 262120 : 2404 : 259716 : 1259866904 : +CRS Device/File integrity check succeeded Device/File not configured Device/File not configured Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged user Check Voting Disk [grid@racnode1 ~]$ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -.oc4j. The IPD/OS tool is designed to detect and analyze operating system and cluster resource-related degradation and failures.. If you install other products in the same Oracle home directory..type 0/5 0/ ONLINE ONLINE racnode1 ora..sh Script Oracle recommends that you back up the root.acfs ora..SM2.ons. you cannot use the srvctl binary in the database home to manage Oracle ASM or Oracle Net which reside in the Oracle grid infrastructure home..gsd.0/grid [root@racnode1 grid]# cp root.sh script after you complete an installation. then you can recover it from the root. and device level continuously.. Oracle ASM and Oracle RAC are running.type 0/5 0/ ONLINE ONLINE racnode1 ora.lsnr ora.9.E2.fs. For root cause analysis.6.sh root.er..lsnr ora.network ora.eons ora. and Oracle Local Registry (OLR).2) or later installations.6.. then the installer updates the contents of the existing root.de1..sh root...de2. Instructions for installing and configuring the IPD/OS tool is beyond the scope of this article and will not be discussed.ons ora.2 and later....racnode1.type 0/3 0/ ONLINE ONLINE racnode1 ora.sh.gsd ora..... It tracks the operating system resource consumption at each node.. Check CRS Status [grid@racnode1 ~]$ crsctl check crs CRS-4638: CRS-4537: CRS-4529: CRS-4533: Oracle High Availability Services is online Cluster Ready Services is online Cluster Synchronization Services is online Event Manager is online Check Clusterware Resources Note: The crs_stat command is deprecated in Oracle Clusterware 11g release 2 (11.SM1.. With Oracle Clusterware release 11.el5 #1 SMP Thu Sep 3 04:15:13 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux If you are using a Linux kernel earlier than 2.2.type 0/5 0/ ONLINE ONLINE racnode1 ora.lsnr application 0/5 0/0 ONLINE ONLINE racnode2 ora.18 kernel: [root@racnode1 ~]# uname -a Linux racnode1 2..sh script during the installation..2) .AFTER_INSTALL_NOV-20-2009 Install Cluster Health Management Software .vip ora. an alert is shown to the operator.type 0/5 0/ ONLINE ONLINE racnode1 ora.type 0/5 0/0 OFFLINE OFFLINE ora.. Back up the root.oc4j ora.com . You can download the IPD/OS tool along with a detailed installation and configuration guide at the following URL: converted by Web2PDFConvert.N1.sh. Note: To manage Oracle ASM or Oracle Net 11g release 2 (11. The voting disk data is automatically backed up in OCR as part of any configuration change and is automatically restored to any voting disk added..lsnr application 0/5 0/0 ONLINE ONLINE racnode1 ora.type 0/5 0/0 ONLINE ONLINE racnode1 ora... [grid@racnode1 ~]$ crs_stat -t -v Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------ora.asm application 0/5 0/0 ONLINE ONLINE racnode1 ora... use the srvctl binary in the Oracle grid infrastructure home for a cluster (Grid home).----------------------------.asm ora..type 0/3 0/ ONLINE ONLINE racnode1 ora.2.asm. when thresholds are reached. process.22.ons application 0/3 0/0 ONLINE ONLINE racnode1 ora.ip. Oracle Cluster Registry (OCR). When we install Oracle Real Application Clusters (the Oracle database software).sh file on both Oracle RAC nodes as root: [root@racnode1 ~]# cd /u01/app/11.eons.t1.racnode2.6.asm application 0/5 0/0 ONLINE ONLINE racnode2 ora. Oracle recommends that you install Instantaneous Problem Detection OS Tool (IPD/OS) if you are using Linux kernel 2... In real time mode. Run the following commands on both nodes in the RAC cluster as the grid user..scan1.racnode2 ASM is enabled.vip ora.vip ora. Voting Disk Management In prior releases.......... historical data can be replayed to understand what was happening at the time of failure.gsd application 0/5 0/0 OFFLINE OFFLINE ora.. it was highly recommended to back up the voting disk using the dd command after installing the Oracle Clusterware software. Backing up the voting disks in Oracle Clusterware 11g release 2 is no longer required. It collects and analyzes cluster-wide data. you should run through several tests to verify the install was successful...CRS. Postinstallation Tasks for Oracle Grid Infrastructure for a Cluster Perform the following postinstallation procedures on both Oracle RAC nodes in the cluster.. please refer to the Oracle Clusterware Administration and Deployment Guide 11 g Release 2 (11... Verify Oracle Clusterware Installation After the installation of Oracle grid infrastructure.....sh file copy. Back Up the root.sh script.9 or higher.type 0/5 0/ OFFLINE OFFLINE ora..rk... To learn more about managing the voting disks.type 0/0 0/0 ONLINE ONLINE racnode1 ora.(Optional) To address troubleshooting issues.. backing up and restoring a voting disk using the dd is not supported and may result in the loss of the voting disk. The tool can provide better explanations for many issues that occur in clusters where Oracle Clusterware. then use the following command syntax as the Grid Infrastructure installation owner to confirm that your Oracle ASM installation is running: [grid@racnode1 ~]$ srvctl status asm -a ASM is running on racnode1.18-164....ER.ons application 0/3 0/0 ONLINE ONLINE racnode2 ora.

Verify Terminal Shell Environment Before starting the ASM Configuration Assistant. If OUI detects an incomplete task that is marked "fixable". Select " Enterprise Edition". log in to racnode1 as the owner of the Oracle Database software which for this article is oracle. the Examples software is only installed from one node in your cluster ( racnode1) as the oracle user. it raises kernel values to required minimums. Now that the grid infrastructure software is functional. This will start the "SSH Connectivity" configuration process: Screen Shot After the SSH configuration process successfully completes.1300(dba). verify your X11 display server settings which were described in the section. Starting with 11g release 2 (11. Open a new console window on both Oracle RAC nodes in the cluster. When creating the "Data" ASM disk group. (starting with the node you are performing the install from). if you are using a remote client to connect to the Oracle RAC node performing the installation (SSH or Telnet to racnode1 from a workstation configured with an X Server).0/dbhome_1/root. Next. The fixup script is generated during installation. if you are using a remote client to connect to the Oracle RAC node performing the installation (SSH or Telnet to racnode1 from a workstation configured with an X Server).0/dbhome_1 Select the OS groups to be used for the SYSDBAand SYSOPER privileges: Database Administrator (OSDBA) Group: dba Database Operator (OSOPER) Group: oper The installer will run through a series of checks to determine if both Oracle RAC nodes meet the minimum requirements for installing and configuring the Oracle Database software. After the installation completes.0/dbhome_1/root. Click [Finish] to start the installation. Create Additional ASM Disk Groups using ASMCA Perform the following tasks as the grid user to create two additional ASM disk groups: [grid@racnode1 ~]$ asmca & Screen Name Disk Groups Create Disk Group Response From the "Disk Groups" tab. and completes other operating system configuration tasks. verify your X11 display server settings which were described in the section.com/technology/products/database/clustering/ipd_download_homepage. the OUI continues to the Summary screen. called fixup scripts.http://www. to resolve many incomplete system configuration requirements. The installer performs the Oracle Database software installation process on both Oracle RAC nodes.sh script on all nodes in the RAC cluster: [root@racnode1 ~]# Product Languages Database Edition Installation Location Operating System Groups Prerequisite Checks Summary Install Product Execute Configuration scripts /u01/app/oracle/product/11. click the " Create" button. we will forgo the "Create Database" option when installing the Oracle Database software.2. you have the option to install the Oracle Database 11g Examples. if you are using a remote client to connect to the Oracle RAC node performing the installation (SSH or Telnet to racnode1 from a workstation configured with an X Server). In the "Redundancy" section. Make the appropriate selection(s) for your environment. Install Oracle Database 11g Release 2 Software Perform the following tasks as the oracle user to install the Oracle Database software: [oracle@racnode1 ~]$ id uid=1101(oracle) gid=1000(oinstall) groups=1000(oinstall). If the ASMLib volumes we created earlier in this article do not show up in the "Select Member Disks" window as eligible ( ORCL:DATAVOL1 and ORCL:FRAVOL1) then click on the "Change Disk Discovery Path" button and input " ORCL:*". OUI copies the binary files from this node to all the other node in the cluster during the installation process. Next. At the end of the installation. as the root user account.0 [oracle@racnode1 ~]$ export DISPLAY [oracle@racnode1 ~]$ cd /home/oracle/software/oracle/database [oracle@racnode1 database]$ . Select the " Real Application Clusters database installation" radio button (default) and verify that both Oracle RAC nodes are checked in the "Node Name" window. These new ASM disk groups will be used later in this guide when creating the clustered database. Finish off this screen by clicking the [Test] button to verify passwordless SSH connectivity. acknowledge the dialog box.1301(oper) [oracle@racnode1 ~]$ DISPLAY=<your local workstation>:0. Verify Terminal Shell Environment Before starting the Oracle Universal Installer (OUI). choose " External (none)". Specify the Oracle base and Software location (Oracle_home) as follows: Oracle Base: /u01/app/oracle Software Location: /u01/app/oracle/product/11. Logging In to a Remote System Using X Terminal. Install Oracle Database 11g Release 2 Examples Perform the following tasks as the oracle user to install the Oracle Database Examples: [oracle@racnode1 ~]$ cd /home/oracle/software/oracle/examples [oracle@racnode1 examples]$ . Logging In to a Remote System Using X Terminal. Finally. Next. Logging In to a Remote System Using X Terminal. click the " [OK]" button. then you can easily fix the issue by generating the fixup script by clicking the [Fix & Check Again] button. For the purpose of this guide. click the [Close] button to exit the OUI.2.html 23. When creating the "Fast RecoveryArea" disk group. log in to racnode1 as the owner of the Oracle Database software which for this article is oracle. click the " [OK]" button. Install Oracle Database 11g with Oracle Real Application Clusters Perform the Oracle Database software installation from only one of the Oracle RAC nodes in the cluster (racnode1)! The Oracle Database software will be installed to both of Oracle RAC nodes in the cluster by the Oracle Universal Installer using SSH. Now that the Oracle Database 11g software is installed.2. In the "Redundancy" section. The clustered database will be created later in this guide using the Database Configuration Assistant (DBCA) after all installs have been completed. if any checks fail.sh script on both Oracle RAC nodes. if necessary. online redo logs. control files. we configured one ASM disk group named +CRS which was used to store the Oracle clusterware files (OCR and voting disk). The "Create Disk Group" dialog should now show the final remaining ASMLib volume. Enter the "OS Password" for the oracle user and click the [Setup] button. OUI copies the binary files from this node to all the other node in the cluster during the installation process. In this section. Asecond ASM disk group will be created for the Fast RecoveryArea named +FRA. you can install the Oracle Database software on the one node in your cluster ( racnode1) as the oracle user. verify your X11 display server settings which were described in the section. use " RACDB_DATA" for the "Disk Group Name".sh Finish Go back to OUI and acknowledge the "Execute Configuration scripts" dialog window. The first ASM disk group will be named +RACDB_DATA and will be used to store all Oracle physical database files (data. Screen Shot Disk Groups Create Disk Group Disk Groups 24. Install Oracle Database 11g Examples (formerly Companion) Perform the Oracle Database 11g Examples software installation from only one of the Oracle RAC nodes in the cluster (racnode1)! The Oracle Database Examples software will be installed to both of Oracle RAC nodes in the cluster by the Oracle Universal Installer using SSH. click the [SSH Connectivity] button. log in to racnode1 as the owner of the Oracle grid infrastructure software which for this article is grid. Exit the ASM Configuration Assistant by clicking the [Exit] button./runInstaller Screen Name Configure Security Updates Installation Option Grid Options Response For the purpose of this article. Create ASM Disk Groups for Data and Fast RecoveryArea Run the ASM Configuration Assistant (asmca) as the grid user from only one node in the cluster (racnode1) to create the additional ASM disk groups which will be used to create the clustered database. Next.2).com . Finally. Run the root. the installer (OUI) will create shell script programs. You will be prompted to run the script as root in a separate terminal session. 25.2. Acknowledge the warning dialog indicating you have not provided an email address by clicking the [Yes] button. The "Create Disk Group" dialog should show two of the ASMLib volumes we created earlier in this guide.oracle. we will create two additional ASM disk groups using the ASM Configuration Assistant ( asmca). choose " External (none)". check the ASMLib volume " ORCL:DATAVOL1" in the "Select Member Disks" section. Click the " Create" button again to create the second ASM disk group. If all prerequisite checks pass (as was the case for my install).1201(asmdba)./runInstaller Screen Name Installation Location Prerequisite Checks Response Specify the Oracle base and Software location (Oracle_home) as follows: Oracle Base: /u01/app/oracle Software Location: /u01/app/oracle/product/11. archived redo logs). Like the Oracle Database software install. Select " Install database software only". you will be returned to the initial dialog. Verify Terminal Shell Environment Before starting the Oracle Universal Installer (OUI). After verifying all values in this dialog are correct.sh [root@racnode2 ~]# /u01/app/oracle/product/11. you will be prompted to run the /u01/app/oracle/product/11.2. check the ASMLib volume " ORCL:FRAVOL1" in the "Select Member Disks" section. use " FRA" for the "Disk Group Name". Screen Shot converted by Web2PDFConvert. After creating the first ASM disk group.0/dbhome_1/root. When you run the script. After verifying all values in this dialog are correct. un-check the security updates checkbox and click the [Next] button to continue. During the installation of Oracle grid infrastructure.0/dbhome_1 The installer will run through a series of checks to determine if both Oracle RAC nodes meet the minimum requirements for installing and configuring the Oracle Database Examples software.

Starting with 11g release 2 (11.dg ora.type 0/5 0/ ONLINE ONLINE racnode1 ora. if any checks fail...asm ONLINE ONLINE racnode1 Started ONLINE ONLINE racnode2 Started ora..type 0/0 0/0 ONLINE ONLINE racnode1 ora.. Oracle Clusterware processes...de1. and completes other operating system configuration tasks. Screen Shot Management Options Database Credentials Database File Locations Specify ASMSNMP Password Recovery Configuration Leave the default options here. click the [Close] button to exit the OUI.SM2..vip 1 ONLINE ONLINE ora.. The fixup script is generated during installation..asm.2.. Change any parameters for your environment.. Database Content Initialization Parameters Database Storage Creation Options End of Database Creation When the DBCAhas completed. Setting environment variables in the login script for the oracle user account was covered in Section 13. Specify storage type and locations for database files.type 0/5 0/0 ONLINE ONLINE racnode1 ora.net1. Logging In to a Remote System Using X Terminal.. I left all of the Database Components (and destination tablespaces) set to their default value although it is perfectly OK to select the Sample Schemas.racnode1. Keep the default option Create Database selected.CRS.asm application 0/5 0/0 ONLINE ONLINE racnode2 ora. If OUI detects an incomplete task that is marked "fixable".racdb... Use the Oracle Database Configuration Assistant (DBCA) to create the clustered database.fs. You will be prompted to run the script as root in a separate terminal session... make certain that the $ORACLE_HOME and $PATH are set appropriately for the $ORACLE_BASE/product/11.vip ora.info SID Prefix: racdb Note: I used idevelopment. click the [Browse] button and select the disk group name +FRA.....type 0/5 0/ ONLINE ONLINE racnode1 ora. At the end of the database creation..dg ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.ons ora. Configuration Type: Admin-Managed Database naming.. Click the [Select All] button to select all servers: racnode1 and racnode2.vip 1 ONLINE ONLINE ora. Next..de1.racnode2.lsnr 1 ONLINE ONLINE racnode1 ora. exit from the DBCA.lsnr application 0/5 0/0 ONLINE ONLINE racnode2 ora.vip 1 ONLINE ONLINE Oracle Enterprise Manager racnode1 racnode2 racnode1 1 ONLINE ONLINE racnode1 Open 2 ONLINE ONLINE racnode2 Open If you configured Oracle Enterprise Manager (Database Control).ons application 0/3 0/0 ONLINE ONLINE racnode2 ora.0/dbhome_1 environment.ons ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora. Cluster database configuration.vip ora.CRS.FRA.db ora. The installer performs the Oracle Database Examples software installation process on both Oracle RAC nodes. I used a Fast Recovery Area Size of 30 GB ( 30413 MB)..2)..up. etc. I left them all at their default settings.grid -c "crs_stat -t -v" Password: ********* Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------ora... You may use any database domain....dg ora.info for the database domain.gsd application 0/5 0/0 OFFLINE OFFLINE ora.LISTENER_SCAN1.network ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.. When defining the Fast RecoveryArea size. Before executing the DBCA.type 0/5 0/ ONLINE ONLINE racnode1 ora..oc4j ora.. Global Database Name: racdb.acfs ora. verify your X11 display server settings which were described in the section..DATA. This option is available since we installed the Oracle Database 11g Examples.t1.. When you run the script.lsnr ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora. You should also verify that all services we have installed up to this point (Oracle TNS listener.er. Summary Install Product Finish 26.type 0/5 0/ ONLINE ONLINE racnode1 ora. the installer (OUI) will create shell script programs.de1..... My disk group has a size of about 33GB. you will have a fully functional Oracle RAC cluster running! Verify Clustered Database is Open [oracle@racnode1 ~]$ su .lsnr ora. Select Create a Database. the database creation will start. then you can easily fix the issue by generating the fixup script by clicking the [Fix & Check Again] button.ER..ip.. Node Selection. Keep in mind that this domain does not have to be a valid DNS domain.up..type 0/5 0/0 OFFLINE OFFLINE ora.er..de2. Check the option for Specify Fast Recovery Area. I selected to Use the Same Administrative Password for All Accounts.de2. called fixup scripts. If all prerequisite checks pass (as was the case for my install).network ora.gsd OFFLINE OFFLINE racnode1 OFFLINE OFFLINE racnode2 ora.type 0/5 0/ ONLINE ONLINE racnode1 ora..rk.gsd application 0/5 0/0 OFFLINE OFFLINE ora...lsnr ora.type 0/0 0/0 ONLINE ONLINE racnode2 ora.asm application 0/5 0/0 ONLINE ONLINE racnode1 ora.type 0/0 0/0 ONLINE ONLINE racnode1 Verify Terminal Shell Environment Before starting the Database Configuration Assistant (DBCA).vip ora.eons ora....com .ry.SM1...LISTENER.ons application 0/3 0/0 ONLINE ONLINE racnode1 ora.. if you are using a remote client to connect to the Oracle RAC node performing the installation (SSH or Telnet to racnode1 from a workstation configured with an X Server)...asm ora.acfs ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 -------------------------------------------------------------------------------Cluster Resources -------------------------------------------------------------------------------ora.idevelopment.. At the end of the installation.gsd ora. run the following as the oracle user: [oracle@racnode1 ~]$ dbca & Screen Name Welcome Screen Operations Database Templates Database Identification Response Select Oracle Real Application Clusters database.FRA.E1.t1.eons ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora....registry. it can be used to view the database configuration and current status of the database. Click Finish to start the database creation process..dg ora. it raises kernel values to required minimums.grid -c "crsctl status resource -w \"TYPE co 'ora'\" -t" Password: ********* -------------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS -------------------------------------------------------------------------------Local Resources -------------------------------------------------------------------------------ora. Enter the password (twice) and make sure the password does not start with a digit number. to resolve many incomplete system configuration requirements.E2.dg ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora.oc4j 1 OFFLINE OFFLINE ora..type 0/5 0/ ONLINE ONLINE racnode1 ora.oc4j. Create the Clustered Database To start the database creation process. Storage Type: Automatic Storage Management (ASM) Storage Locations: Use Oracle-Managed Files Database Area: +RACDB_DATA Specify the ASMSNMP password for the ASM instance. log in to racnode1 as the owner of the Oracle Database software which for this article is oracle.de2. Create the Oracle Cluster Database The database creation process should only be performed from one of the Oracle RAC nodes in the cluster (racnode1).. use the entire volume minus 10% for overhead (33-10%=30 GB)..type 0/3 0/ ONLINE ONLINE racnode1 ora.scan1... I left them all at their default settings..scan1.... I also always select to Generate Database Creation Scripts.type 0/5 0/ OFFLINE OFFLINE ora.type 0/5 0/ ONLINE ONLINE racnode1 ora.RACDB_DATA.up. if necessary...ons. the OUI continues to the Summary screen. Change any parameters for your environment.) are running before attempting to start the clustered database creation process: [oracle@racnode1 ~]$ su . Click OK on the "Summary" screen.lsnr application 0/5 0/0 ONLINE ONLINE racnode1 ora. which is to Configure Enterprise Manager / Configure Database Control for local management..type 0/3 0/ ONLINE ONLINE racnode1 ora. Click [Finish] to start the installation. Select Custom Database.gsd.eons.. For the Fast RecoveryArea.N1.... The URL for this example is: https://racnode1:1158/em [oracle@racnode1 ~]$ emctl status dbconsole converted by Web2PDFConvert.dg ONLINE ONLINE racnode1 ONLINE ONLINE racnode2 ora. After acknowledging the database creation report and script generation dialog.

0. The Database Configuration Assistant (DBCA) allows users to configure a new database to be in archive log mode. Shutdown the local instance: SQL> shutdown immediate ORA-01109: database not open Database dismounted. each instance in the RAC configuration can automatically archive redologs! converted by Web2PDFConvert. Total System Global Area 1653518336 bytes Fixed Size 2213896 bytes Variable Size 1073743864 bytes Database Buffers 570425344 bytes Redo Buffers 7135232 bytes 6. 2009 Oracle Corporation. The size of an online redolog file is completely independent of another instance's' redolog size. 15. Oracle.2. Oracle will make a copy of the online redo log before it gets reused.sql Enabling Archive Logs in a RAC Environment Whether a single instance or clustered database.com .2. Post Database Creation Tasks .(Optional) This section offers several optional tasks that can be performed on your new Oracle 11g in order to enhance availability as well as database management. Oracle will switch to the next one. 9. Although in most configurations the size is the same. OLAP. Oracle. Database altered.0 . 7. 2009. 12. Shutdown all instances accessing the clustered database as the oracle user: [oracle@racnode1 ~]$ srvctl stop database -d racdb 4.(Database Console) 27. MOUNT the database: [oracle@racnode1 ~]$ sqlplus / as sysdba SQL*Plus: Release 11. Oracle tracks and logs all changes to database blocks in online redolog files. Data Mining and Real Application Testing options SQL> archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination Oldest online log sequence Next log sequence to archive Current log sequence USE_DB_RECOVERY_FILE_DEST 69 70 70 After enabling Archive Log Mode. racnode1) as oracle and disable the cluster instance parameter by setting cluster_database to FALSE from the current instance: [oracle@racnode1 ~]$ sqlplus / as sysdba SQL> alter system set cluster_database=false scope=spfile sid='racdb1'. Enable archiving: SQL> alter database archivelog. each instance will have its own set of online redolog files known as a thread.0. All rights reserved. This step is optional but recommended.64bit Production With the Partitioning. Connected to: Oracle Database 11g Enterprise Edition Release 11. 8. 10.1. It is therefore a requirement that online redo logs be located on a shared storage device (just like the database files). All rights reserved.sql script to recompile all invalid PL/SQL packages now instead of when the packages are accessed for the first time. each instance can read another instance's current online redolog file to perform instance recovery if that instance was terminated abnormally. -----------------------------------------------------------------Logs are generated in directory /u01/app/oracle/product/11.Oracle Enterprise Manager 11g Database Control Release 11. Re-compile Invalid Objects Run the utlrp.1. In an Oracle RAC environment. Oracle moves to the next one. SQL> startup mount ORACLE instance started. As already mentioned.1. Once an online redolog fills. If the database is in "Archive Log Mode". Connected to an idle instance. Using the local instance. however most DBA's opt to bypass this option during initial database creation. Oracle writes to its online redolog files in a circular manner. it is a simple task to put the database into archive log mode.2. Athread must contain at least two online redologs (or online redolog groups). In cases like this where the database is in no archive log mode. When the current online redolog fills. ORACLE instance shut down. Real Application Clusters. https://racnode1:1158/em/console/aboutApplication Oracle Enterprise Manager 11g is running.e. Each Oracle instance will use its group of online redologs in a circular manner. This is a process known as archiving.0.0 Copyright (c) 1996.1. Log in to one of the nodes (i. 5.2. Oracle allows the DBAto put the database into "Archive Log Mode" which makes a copy of the online redolog after it fills (and before it gets reused). Note however that this will require a short database outage. 2. It is also worth mentioning that each instance has exclusive write access to its own online redolog files. 3. System altered. use the following tasks to put a RAC enabled database into archive log mode. From one of the nodes in the Oracle RAC configuration. All rights reserved.0 Production on Sat Nov 21 19:26:47 2009 Copyright (c) 1982. Automatic Storage Management. The single instance must contain at least two online redologs (or online redolog groups). 11.0/dbhome_1/racnode1_racdb/sysman/log Figure 18: Oracle Enterprise Manager . For the purpose of this article. it may be different depending on the workload and backup / recovery considerations for each node. The same holds true for a single instance configuration.0. Bring all instance back up as the oracle account using srvctl: [oracle@racnode1 ~]$ srvctl start database -d racdb 14. 2009. In a correctly configured RAC environment. System altered. Re-enable support for clustering by modifying the instance parameter cluster_database to TRUE from the current instance: SQL> alter system set cluster_database=true scope=spfile sid='racdb1'. 13. however. To facilitate media recovery. I will use the node racnode1 which runs the racdb1 instance: 1. [oracle@racnode1 ~]$ sqlplus / as sysdba SQL> @?/rdbms/admin/utlrp. Login to the local instance and verifyArchive Log Mode is enabled: [oracle@racnode1 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.0 Production on Sat Nov 21 19:33:38 2009 Copyright (c) 1982.2.

The following query can be used to determine the file names for your environment: SQL> select tablespace_name. backups.---------------.497. Although these views provide a simple and easy mechanism to query critical information regarding the database.--------AUTO 629.263. Used --------.sql ======================================== Workspace Manager ======================================== wm_create_workspace.061.sql asm_diskgroups.967.264.sql script: SQL> Tablespace Name ----------------SYSAUX UNDOTBS1 USERS SYSTEM EXAMPLE UNDOTBS2 TEMP TS Type -----------PERMANENT UNDO PERMANENT PERMANENT PERMANENT UNDO TEMPORARY Ext.:$ORACLE_HOME/rdbms/admin export ORACLE_PATH Note: The ORACLE_PATH environment variable should already be set in the . Next.com . SQL> alter database datafile '+RACDB_DATA/racdb/datafile/system. unzip the archive file to the $ORACLE_BASE directory.sql wm_goto_workspace. file_name 2 from dba_data_files 3 union 4 select tablespace_name. resource. run the help.286.sql wm_workspaces. Tablespace Size Used (in bytes) Pct.776 90 AUTO 5.200 20.703530411' resize 1024m.262.328 @help. security.sql asm_disks_perf.703530429 +RACDB_DATA/racdb/datafile/undotbs1.sql asm_clients. As the oracle user account.sql < --.880 1. to query tablespace information while logged into the Oracle database as a DBAuser: SQL> @dba_tablespaces Status ------ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE avg sum 7 rows selected. For example. In this section you will download and install a collection of Oracle DBAscripts that can be used to manage many aspects of your database including space management. Please keep in mind that the database file names (OMF files) used in this example may differ from what the Oracle Database Configuration Assistant (DBCA) creates for your environment. verify the following environment variable is set and included in your login shell script: ORACLE_PATH= $ORACLE_BASE/common/oracle/sql:.200 703. SQL> grant dba. ---------LOCAL LOCAL LOCAL LOCAL LOCAL LOCAL LOCAL Seg. SQL> alter database datafile '+RACDB_DATA/racdb/datafile/undotbs1. SQL> alter database datafile '+RACDB_DATA/racdb/datafile/undotbs2.472 66. you should now be able to run any of the SQL scripts in your $ORACLE_BASE/common/oracle/sql while logged into SQL*Plus. it helps to have a collection of accurate and readily available SQL scripts to query these views. Mgt.265.info/data/Oracle/DBA_scripts/dba_scripts_archive_Oracle. SQL> alter database datafile '+RACDB_DATA/racdb/datafile/users.003. User created.869.703530423' resize 1024m. For UNIX.336. [oracle@racnode1 ~]$ sqlplus "/ as sysdba" SQL> create user scott identified by tiger default tablespace users. SQL> alter database tempfile '+RACDB_DATA/racdb/tempfile/temp.703530411 +RACDB_DATA/racdb/datafile/system.---------------. Tablespace created.zip.260. For example. Database altered.448 10 MANUAL 75.SNIP --.sql wm_remove_workspace.zip /u01/app/oracle [oracle@racnode1 ~]$ cd /u01/app/oracle [oracle@racnode1 ~]$ unzip common.600 511.048. TABLESPACE_NAME --------------EXAMPLE SYSAUX SYSTEM TEMP UNDOTBS1 UNDOTBS2 USERS FILE_NAME -------------------------------------------------+RACDB_DATA/racdb/datafile/example.703530441 +RACDB_DATA/racdb/datafile/users. When working through this section. The Oracle DBAscripts archive can be downloaded using the following link http://www.744 96 AUTO 157. download the common. the common.059.sql wm_disable_versioning.512 2.sql asm_files. converted by Web2PDFConvert.259.Download and Install Custom Oracle Database Scripts DBA's rely on Oracle's data dictionary views and dynamic performance views in order to support and better manage their databases.703530397 +RACDB_DATA/racdb/tempfile/temp. connect to scott.288 88 ---------------.576 20 MANUAL 734. SQL> alter database datafile '+RACDB_DATA/racdb/datafile/sysaux. SQL> create tablespace indx datafile '+RACDB_DATA' size 1024m 2 autoextend on next 100m maxsize unlimited 3 extent management local autoallocate 4 segment space management auto. SQL> alter tablespace users add datafile '+RACDB_DATA' size 1024m autoextend off.840.264.227.703530447 7 rows selected.145. Database altered.131. Database altered.715.060.264 54 MANUAL 209. Database altered.232 81 MANUAL 1. substitute the data file names that were created in your environment where appropriate.sql wm_freeze_workspace.703530429' resize 1024m. you may want to make a sizable testing database.259.265.sql wm_get_workspace.sql wm_refresh_workspace. and session management.043.sql ======================================== Automatic Storage Management ======================================== asm_alias.zip archive to the $ORACLE_BASE directory of each node in the cluster. To obtain a list of all available Oracle DBAscripts while logged into SQL*Plus.703530441' resize 1024m.242.idevelopment.952. file_name 5 from dba_temp_files.> perf_top_sql_by_buffer_gets.--------63 2. performance. Mgt.400 85. Tablespace altered.sql ======================================== Automatic Shared Memory Management ======================================== asmm_components. If you are using a large drive for the shared storage. Below are several optional SQL commands for modifying and creating all tablespaces for the test database.760 948.sql wm_merge_workspace.703530435 +RACDB_DATA/racdb/datafile/sysaux.zip archive will be copied to /u01/app/oracle.260.sql wm_unfreeze_workspace. Create / Alter Tablespaces When creating the clustered database.bash_profile login script that was created in the section Create Login Script for the oracle User Account.sql asm_templates. perform the following on both nodes in the Oracle RAC cluster as the oracle user account: [oracle@racnode1 ~]$ mv common.sql asm_drop_files.sql 28.703530447' resize 1024m. For the purpose of this example.261.135.262.sql asm_disks. Grant succeeded.zip The final step is to verify (or set) the appropriate environment variable for the current UNIX shell to ensure the Oracle SQL scripts can be run from within SQL*Plus while in any directory.703530397' resize 1024m. Database altered.---------------.sql wm_enable_versioning.sql asm_files2. we left all tablespaces set to their default size.261.sql perf_top_sql_by_disk_reads. Now that the Oracle DBAscripts have been unzipped and the UNIX environment variable ( $ORACLE_PATH) has been set to the appropriate directory.703530423 +RACDB_DATA/racdb/datafile/undotbs2.

and stop operations on the cluster.073. Oracle also provides the Oracle Clusterware Control (CRSCTL) utility.088 29.168. Most of the checks described in this section use the Server Control Utility (SRVCTL) and can be run as either the oracle or grid OS user. Tablespace Name ----------------SYSAUX UNDOTBS1 USERS SYSTEM EXAMPLE INDX UNDOTBS2 TEMP TS Type -----------PERMANENT UNDO PERMANENT PERMANENT PERMANENT PERMANENT UNDO TEMPORARY Ext.131.--------AUTO 1.286.483. [grid@racnode1 ~]$ crsctl check cluster CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online All Oracle Instances .824 512.---------------.:racnode2 VIP exists.097.FRA Services: Database is enabled Database is administrator managed converted by Web2PDFConvert. Verify Oracle Grid Infrastructure and Database Configuration The following Oracle Clusterware and Oracle RAC verification checks can be performed on any of the Oracle RAC nodes in the cluster. we will only make use of the "Checking the health of the cluster" operation which uses the Clusterized (Cluster Aware) Command: crsctl check cluster Many subprograms and commands were deprecated in Oracle Clusterware 11g release 2 (11. Used --------.073.648 2.1. listening port 2016 List all Configured Databases [oracle@racnode1 ~]$ srvctl config database racdb Database .info Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: racdb Database instances: racdb1.992 2.252/255. Mgt.747.448 2 MANUAL 1.0/eth0 VIP exists.098.201. such as: Starting and stopping Oracle Clusterware resources Enabling and disabling Oracle Clusterware daemons Checking the health of the cluster Managing resources that represent third-party applications Integrating Intelligent Platform Management Interface (IPMI) with Oracle Clusterware to provide failure isolation support and to ensure cluster integrity Debugging Oracle Clusterware components For the purpose of this article (and this section).2): crs_stat crs_register crs_unregister crs_start crs_stop crs_getperm crs_profile crs_relocate crs_setperm crsctl check crsd crsctl check cssd crsctl check evmd crsctl debug log crsctl set css votedisk crsctl start resources crsctl stop resources Check the Health of the Cluster .168.400 85. ---------LOCAL LOCAL LOCAL LOCAL LOCAL LOCAL LOCAL LOCAL Seg.152 0 MANUAL 1. You can run these commands from any node in the cluster on another node in the cluster. multicast IP address 234.2. ONS daemon exists. Oracle Notification Services.220.194.(Status) [oracle@racnode1 ~]$ srvctl status nodeapps VIP racnode1-vip is enabled VIP racnode1-vip is running on node: racnode1 VIP racnode2-vip is enabled VIP racnode2-vip is running on node: racnode2 Network is enabled Network is running on node: racnode1 Network is running on node: racnode2 GSD is disabled GSD is not running on node: racnode1 GSD is not running on node: racnode2 ONS is enabled ONS daemon is running on node: racnode1 ONS daemon is running on node: racnode2 eONS is enabled eONS daemon is running on node: racnode1 eONS daemon is running on node: racnode2 Node Applications .147.(Configuration) [oracle@racnode1 ~]$ srvctl config nodeapps VIP exists.741. You can use CRSCTL commands to perform several operations on Oracle Clusterware. Oracle Clusterware 11g release 2 (11.(Clusterized Command) Run as the grid user.0/eth0 GSD exists.com .824 66. I will only be performing checks from racnode1 as the oracle OS user. Multicast port 24057.(Database Status) [oracle@racnode1 ~]$ srvctl status database -d racdb Instance racdb1 is running on node racnode1 Instance racdb2 is running on node racnode2 Single Oracle Instance .741.741.racdb2 Disk Groups: RACDB_DATA.741.304 48 MANUAL 1.280 65 AUTO 157.741.048. For the purpose of this article.264 54 AUTO 1.168.576 0 MANUAL 1.255.073.ora Domain: idevelopment.255.060.073. and Oracle Enterprise Manager agents (for maintenance purposes).0/dbhome_1 Oracle user: oracle Spfile: +RACDB_DATA/racdb/spfileracdb.: /racnode1-vip/192.840.2) introduces cluster-aware commands with which you can perform check.824 20.255. There are five node-level tasks defined for SRVCTL: Adding and deleting node-level applications Setting and un-setting the environment for node-level applications Administering node applications Administering ASM instances Starting and stopping a group of programs that includes virtual IP addresses. Mgt. Local port 6100.43.---------------. Here is a snapshot of the tablespaces I have defined for my test database environment: Status ------ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE ONLINE avg sum 8 rows selected.(Status of Specific Instance) [oracle@racnode1 ~]$ srvctl status instance -d racdb -i racdb1 Instance racdb1 is running on node racnode1 Node Applications .: /racnode2-vip/192. depending on the operation.824 1.:racnode1 VIP exists.288 6 ---------------.Database altered. parsing and calling Oracle Clusterware APIs for Oracle Clusterware objects.824 703.338.073.043.---------------.(Configuration) [oracle@racnode1 ~]$ srvctl config database -d racdb -a Database unique name: racdb Database name: racdb Oracle home: /u01/app/oracle/product/11.1.073.776 88 AUTO 2.251/255. remote port 6200 eONS daemon exists. or on all nodes in the cluster.255. CRSCTL is an interface between you and Oracle Clusterware.--------33 8.521.741.824 948. listeners. Tablespace Size Used (in bytes) Pct. start.

Owner: grid Home: <crs> /u01/app/11.261.255.259.255.1.255. NAME ------------------------------------------+FRA/racdb/controlfile/current.0/grid on node(s) racnode2.0/eth0 VIP exists.2.260.racnode2 TNS listener .168.703530391 +FRA/racdb/onlinelog/group_2.255.168. GSD.:racnode1 VIP exists.266.1. All running instances in the cluster . active_state state .703533497 +RACDB_DATA/racdb/onlinelog/group_4.-----------------------. instance_number inst_no HOST ------racnode1 racnode2 .257.264. Result: Query of CTSS for time offset passed Check CTSS state started. host_name host FROM gv$instance ORDER BY inst_id.racnode1 End points: TCP:1521 SCAN . Check: CTSS state Node Name State -----------------------------------.(Configuration) [oracle@racnode1 ~]$ srvctl config listener -a Name: LISTENER Network: 1.703530441 +RACDB_DATA/racdb/datafile/users.1.(SQL) SELECT path FROM PATH ---------------------------------- v$asm_disk.267.1.------1 1 racdb1 YES OPEN 2 2 racdb2 YES OPEN inst_id .255.258.---------.:racnode2 VIP exists.260.1.168.251/255. status . ONS daemon exists.257.703530389 +RACDB_DATA/racdb/datafile/example.703542943 +RACDB_DATA/racdb/onlinelog/group_1.-----------------------racnode1 passed Result: CTSS resource check passed Querying CTSS for time offset on all nodes..256.703533499 +RACDB_DATA/racdb/controlfile/current.0/grid on node(s) racnode2.187 VIP ..(Status) [oracle@racnode1 ~]$ srvctl status listener Listener LISTENER is enabled Listener LISTENER is running on node(s): racnode1. Reference Time Offset Limit: 1000. ASM Disk Volumes .racnode2 ASM .(SQL) select name from v$datafile union select member from v$logfile union select name from v$controlfile union select name from v$tempfile.703530429 19 rows selected. Owner: grid Home: <crs> /u01/app/11...-----------------------racnode1 0.168.703530391 +RACDB_DATA/racdb/onlinelog/group_2.703530393 +RACDB_DATA/racdb/onlinelog/group_3. VIP exists.0/eth0 SCAN VIP name: scan1.:racnode1 VIP exists.--.: /racnode2-vip/192.. TNS listener .255.(Configuration) $ srvctl config asm -a ASM home: /u01/app/11.0 passed Time offset is within the specified limits on the following set of nodes: "[racnode1]" Result: Check of clock time offsets passed Oracle Cluster Time Synchronization Services check passed Verification of Clock Synchronization across the cluster nodes was successful...703530397 +RACDB_DATA/racdb/datafile/undotbs1.(SQL) SELECT INST_ID INST_NO INST_NAME PAR STATUS -------.0/eth0 [oracle@racnode1 ~]$ srvctl config vip -n racnode2 VIP exists. instance_name inst_name . DB_STATUS -----------ACTIVE ACTIVE STATE --------NORMAL NORMAL All database files and the ASM disk group they reside in . Network: 1/192.703542993 +RACDB_DATA/racdb/datafile/sysaux.: /racnode1-vip/192.255.0/eth0 Configuration for Node Applications .com .0/grid ASM listener: LISTENER ASM is enabled.-------.270.0/255.168.259.(Status) [oracle@racnode1 ~]$ srvctl status scan SCAN VIP scan1 is enabled SCAN VIP scan1 is running on node racnode1 SCAN . remote port 6200 Name: LISTENER Network: 1. database_status db_status .2.. Listener) [oracle@racnode1 ~]$ srvctl config nodeapps -a -g -s -l -l option has been deprecated and will be ignored.703530447 +RACDB_DATA/racdb/datafile/users.703530389 +FRA/racdb/onlinelog/group_1.:racnode2 VIP exists.703530411 +RACDB_DATA/racdb/datafile/system. converted by Web2PDFConvert.: /racnode1-vip/192.0 msecs Check: Reference Time Offset Node Name Time Offset Status -----------.703530393 +FRA/racdb/onlinelog/group_3.256.703533499 +RACDB_DATA/racdb/tempfile/temp.(Configuration) [oracle@racnode1 ~]$ srvctl config scan SCAN name: racnode-cluster-scan. Local port 6100.703533497 +FRA/racdb/onlinelog/group_4. IP: /racnode-cluster-scan/192.251/255.(VIP.252/255.-----------------------racnode1 Active CTSS is in Active state.0/eth0 GSD exists.255..262. Check: CTSS Resource running on all nodes Node Name Status -----------------------------------.(Configuration of Specific Node) [oracle@racnode1 ~]$ srvctl config vip -n racnode1 VIP exists.ASM . ONS.168.703530435 +RACDB_DATA/racdb/datafile/indx. Proceeding with check of clock time offsets on all nodes.2. Check of Clusterware install passed Checking if CTSS Resource is running on all nodes.258.255.: /racnode2-vip/192. parallel .(Status) [oracle@racnode1 ~]$ srvctl status asm ASM is running on racnode1.263.racnode1 End points: TCP:1521 Verifying Clock Synchronization across the Cluster Nodes [oracle@racnode1 ~]$ cluvfy comp clocksync -verbose Verifying Clock Synchronization across the cluster nodes Checking if Clusterware is installed on all nodes.1..252/255.269.(Status of Specific Node) [oracle@racnode1 ~]$ srvctl status vip -n racnode1 VIP racnode1-vip is enabled VIP racnode1-vip is running on node: racnode1 [oracle@racnode1 ~]$ srvctl status vip -n racnode2 VIP racnode2-vip is enabled VIP racnode2-vip is running on node: racnode2 VIP .265.255.703530423 +RACDB_DATA/racdb/datafile/undotbs2.

(also available through dmesg): iSCSI Enterprise Target Software . however.acfs' on 'racnode1' Attempting to stop 'ora.net1.diskmon' on 'racnode1' CRS-2677: Stop of 'ora.FRA.4.crsd' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora.0/grid/bin/crsctl stop cluster CRS-2673: CRS-2790: CRS-2673: CRS-2673: CRS-2673: CRS-2673: CRS-2673: CRS-2677: CRS-2673: CRS-2677: CRS-2673: CRS-2677: CRS-2672: CRS-2677: CRS-2672: CRS-2677: CRS-2676: Attempting to stop 'ora.ctssd' on 'racnode1' CRS-2673: Attempting to stop 'ora.2.lsnr' on 'racnode1' succeeded Attempting to stop 'ora. If you have followed the instructions in this guide.scan1. everything has been installed and configured for Oracle RAC 11g release 2.1 racnode1 localhost.vip' on 'racnode2' Stop of 'ora. Or you may find that Enterprise Manager is not running and need to start it.lsnr' on 'racnode2' CRS-2676: Start of 'ora.CRS.scan1.racnode1. you may ask.ORCL:CRSVOL1 ORCL:DATAVOL1 ORCL:FRAVOL1 30.cssd' on 'racnode1' CRS-2677: Stop of 'ora.ctssd' on 'racnode1' Start of 'ora. ASM .lsnr' on 'racnode1' succeeded Attempting to stop 'ora.dg' on 'racnode1' Attempting to stop 'ora.0/grid/bin/crsctl start cluster -all You can also start the Oracle Clusterware stack on one or more named servers in the cluster by listing the servers separated by a space: [root@racnode1 ~]# /u01/app/11.registry.LISTENER.dg' on 'racnode1' succeeded CRS-2677: Stop of 'ora.racdb.LISTENER_SCAN1.vip' on 'racnode2' succeeded <-.LISTENER_SCAN1. and so on — should start automatically on each reboot of the Linux nodes. VIP.Logical Volumes Not Active on Boot One issue that I have run into several times occurs when using a USB drive connected to the Openfiler server.com . Stopping the Oracle Clusterware Stack on the Local Server Use the " crsctl stop cluster" command on racnode1 to stop the Oracle Clusterware stack: [root@racnode1 ~]# /u01/app/11. SCAN.evmd' on 'racnode1' succeeded CRS-2677: Stop of 'ora.Notice LISTENER_SCAN1 moved to racnode2 CRS-2677: Stop of 'ora. when you might want to take down the Oracle services on a node for maintenance purposes and restart the Oracle Clusterware stack at a later time.cssd' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora.racnode1.asm' on 'racnode1' succeeded Attempting to start 'ora.crsd' on 'racnode1' Start of 'ora.eons' on 'racnode1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'racnode1' has completed CRS-2677: Stop of 'ora.ctssd' on 'racnode1' succeeded Attempting to start 'ora.cssdmonitor' on 'racnode1' Start of 'ora. all services — including Oracle Clusterware.0.CRS.0/grid/bin/crsctl start cluster -n racnode1 racnode2 Start/Stop All Instances with SRVCTL Finally.RACDB_DATA.vip' on 'racnode1' succeeded Attempting to start 'ora. After all of that hard work.LISTENER_SCAN1.cssd' on 'racnode1' succeeded Attempting to start 'ora.asm' on 'racnode1' CRS-2677: Stop of 'ora.racdb.vip' on 'racnode2' Stop of 'ora.dg' on 'racnode1' succeeded CRS-2677: Stop of 'ora.Notice SCAN moved to racnode2 CRS-2672: Attempting to start 'ora.racnode1.LISTENER. Starting / Stopping the Cluster At this point.vip' on 'racnode1' Stop of 'ora.0/grid/bin/crsctl start cluster CRS-2672: CRS-2676: CRS-2672: CRS-2672: CRS-2676: CRS-2676: CRS-2672: CRS-2676: CRS-2672: CRS-2672: CRS-2676: CRS-2676: CRS-2672: CRS-2676: Attempting to start 'ora.lsnr' on 'racnode1' Stop of 'ora.vip' on 'racnode1' Stop of 'ora.asm' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora.dg' on 'racnode1' CRS-2677: Stop of 'ora.0.dg' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora.evmd' on 'racnode1' CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.diskmon' on 'racnode1' Start of 'ora. Also note that you can stop the Oracle Clusterware stack on all servers in the cluster by specifying -all. the Oracle Database.acfs' on 'racnode1' succeeded Start of 'ora. The following will bring down the Oracle Clusterware stack on both racnode1 and racnode2: [root@racnode1 ~]# /u01/app/11.ctssd' on 'racnode1' succeeded CRS-2677: Stop of 'ora.diskmon' on 'racnode1' succeeded Start of 'ora.0/grid/bin/crsctl stop cluster -all Starting the Oracle Clusterware Stack on the Local Server Use the " crsctl start cluster" command on racnode1 to start the Oracle Clusterware stack: [root@racnode1 ~]# /u01/app/11.diskmon' on 'racnode1' succeeded Note: If any resources that Oracle Clusterware manages are still running after you run the " crsctl stop cluster" command.db' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora.1 localhost. This section provides the commands necessary to stop and start the Oracle Clusterware stack on a local server ( racnode1). Oracle grid infrastructure was installed by the grid user while the Oracle RAC software was installed by oracle.0. Troubleshooting Confirm the RAC Node Name is Not Listed in Loopback Address Ensure that the node names ( racnode1 or racnode2) are not included for the loopback address in the /etc/hosts file.network' on 'racnode1' CRS-2677: Stop of 'ora. There are times.ons' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora. "OK.net1.14 iotype_init(91) register fileio iotype_init(91) register blockio iotype_init(91) register nullio open_path(120) Can't open /dev/rac1/crs -2 fileio_attach(268) -2 open_path(120) Can't open /dev/rac1/asm1 -2 fileio_attach(268) -2 converted by Web2PDFConvert. The following stop/start actions need to be performed as root.ons' on 'racnode1' CRS-2673: Attempting to stop 'ora. you can start/stop all instances and associated services using the following: [oracle@racnode1 ~]$ srvctl stop database -d racdb [oracle@racnode1 ~]$ srvctl start database -d racdb 31.asm' on 'racnode1' succeeded CRS-2673: Attempting to stop 'ora. If the machine name is listed in the in the loopback address entry as below: 127.registry.vip' on 'racnode2' succeeded <-.eons' on 'racnode1' CRS-2677: Stop of 'ora.lsnr' on 'racnode2' succeeded <-.evmd' on 'racnode1' succeeded Start of 'ora.cssdmonitor' on 'racnode1' CRS-2673: Attempting to stop 'ora.version 0.scan1.cssdmonitor' on 'racnode1' succeeded Attempting to start 'ora.scan1.evmd' on 'racnode1' Attempting to start 'ora.2.network' on 'racnode1' succeeded CRS-2677: Stop of 'ora.racnode1. it is not able to load the logical volumes and writes the following message to /var/log/messages .2.asm' on 'racnode1' Start of 'ora.FRA.crsd' on 'racnode1' Starting shutdown of Cluster Ready Services-managed resources on 'racnode1' Attempting to stop 'ora.localdomain localhost it will need to be removed as shown below: 127. the system is able to recognize the USB drive however.0. then the entire command fails.vip' on 'racnode1' succeeded Attempting to start 'ora.2.2. you will receive the following error during the RAC installation: ORA-00603: ORACLE server session terminated by fatal error or ORA-29702: error occurred in Cluster Group Service operation Openfiler .RACDB_DATA. network. We also have a fully functional clustered database running named racdb.lsnr' on 'racnode1' Attempting to stop 'ora. Use the -f option to unconditionally stop all resources and stop the Oracle Clusterware stack.cssdmonitor' on 'racnode1' succeeded CRS-2677: Stop of 'ora.Notice racnode1 VIP moved to racnode2 CRS-2676: Start of 'ora.cssd' on 'racnode1' Attempting to start 'ora. so how do I start and stop services?".dg' on 'racnode1' CRS-2673: Attempting to stop 'ora.localdomain localhost If the RAC node name is listed for the loopback address.db' on 'racnode1' Attempting to stop 'ora.asm' on 'racnode1' CRS-2677: Stop of 'ora.crsd' on 'racnode1' succeeded Note: You can choose to start the Oracle Clusterware stack on all servers in the cluster by specifying -all: [root@racnode1 ~]# /u01/app/11. When the Openfiler server is rebooted.

then /sbin/dmsetup ls --target multipath --exec "/sbin/kpartx -a" fi fi if [ -x /sbin/dmraid ]..94 GB] inherit '/dev/rac1/asm4' [115.static vgchange -a y --ignorelockingfailure fi fi fi # Clean up SELinux labels if [ -n "$SELINUX" ].. Jeff currently works as a Senior Database Administrator for The DBAZone..... then for file in /etc/mtab /etc/ld.so.open_path(120) Can't open /dev/rac1/asm2 -2 fileio_attach(268) -2 open_path(120) Can't open /dev/rac1/asm3 -2 fileio_attach(268) -2 open_path(120) Can't open /dev/rac1/asm4 -2 fileio_attach(268) -2 Please note that I am not suggesting that this only occurs with USB drives connected to the Openfiler server..static ].. Database Administrator and Software Engineer for over 16 years and maintains his own website site at: http://www. located in Pittsburgh... however I have only seen it with USB drives! If you do receive this error. capacity planning.. Jeff's other interests include mathematical encryption theory..info] is an Oracle Certified Professional..(the status for each logical volume on a working system would be set to ACTIVE).. Jeffrey M. and physical / logical database design in a UNIX.com .. and an Oracle ACE.." #----#----.... from the Openfiler server...Customisation Start #----# Check if /dev/sda is ready while [ ! -e /dev/sda ] do echo "Device /dev/sda for first USB Drive is not yet ready.static -v 0 if [ -x /sbin/kpartx ].000.( racnode1 and racnode2)... Avocent Corporation..94 GB] inherit '/dev/rac1/asm3' [115.. then modprobe dm-mirror > /dev/null 2>&1 /sbin/dmraid -i -a y fi #----#----....... database security. Jeff graduated from Stanislaus State University in Turlock. manually set each of the logical volumes to ACTIVE for each consecutive reboot: # '/dev/rac1/crs' [2. Java and PL/SQL programming...... Method 1 One of the first steps is to shutdown both of the Oracle RAC nodes in the cluster . Jeff has been a Sr.sysinit script (described below).static ]..sysinit script on the Openfiler server with the changes (highlighted in blue) proposed by Martin: .. restart each of the Oracle RAC nodes in the cluster . use the lvscan command again to verify the status: # lvscan ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE '/dev/rac1/crs' [2. Conclusion Oracle11g RAC allows the DBAto configure a database solution with superior fault tolerance and load balancing..info......... then modprobe dm-multipath >/dev/null 2>&1 /sbin/multipath. then action $"Setting up Logical Volume Management:" /sbin/lvm. Java Development Certified Professional... take 2 if [ -c /dev/mapper/control ].94 GB] inherit As a final test.. Finally. This article has hopefully given you an economical solution to setting up and configuring an inexpensive Oracle 11g release 2 RAC Cluster using Oracle Enterprise Linux and iSCSI technology.94 GB] inherit '/dev/rac1/asm2' [115. both of which are described below. writing web-based database administration tools... programming language processors (compilers and interpreters) in Java and C. California. This may take a while. Bane not only introduced me to Openfiler..... there are several other individuals that deserve credit in making this article a success. much of the content regarding the history of Oracle RAC can be found in his very popular book Oracle Database 10g Real Application Clusters Handbook ... and of course Linux...... that want to become more familiar with the features and benefits of Oracle11 g RAC will find the costs of configuring even a small RAC cluster costing in the range of US$15. then if /sbin/lvm.. I would like to thank Bane Radulovic from the Server BDE Team at Oracle. It may occur with other types of drives. Seagate.94 GB] inherit lvchange -a y /dev/rac1/crs # lvchange -a y /dev/rac1/asm1 # lvchange -a y /dev/rac1/asm2 # lvchange -a y /dev/rac1/asm3 # lvchange -a y /dev/rac1/asm4 Another method to set the status to active for all logical volumes is to use the Volume Group change command as follows: # vgscan Reading all physical volumes.700 and will provide the DBAwith a fully functional Oracle 11 g release 2 RAC cluster. Aspecial thanks to K Gopalakrishnan for his assistance in delivering the Oracle RAC 11g Overview section of this article. 33. Found volume group "rac1" using metadata type lvm2 # vgchange -ay 5 logical volume(s) in volume group "rac1" now active After setting each of the logical volumes to active... reboot the Openfiler server to ensure each of the logical volumes will be set to ACTIVE after the boot process. then if [ -x /sbin/multipath... This book comes highly recommended for both DBA's and Developers wanting to successfully implement Oracle RAC and fully understand how many of the advanced services like Cache Fusion and Global Resource Directory operate.MJONES . you should first check the status of all logical volumes using the lvscan command from the Openfiler server: # lvscan inactive inactive inactive inactive inactive Notice that the status for each of the logical volumes is set to inactive .static vgscan > /dev/null 2>&1 ....idevelopment. For those DBA's.Device /dev/sda for first USB Drive is ready.. I would like to express my appreciation to the following vendors for generously supplying the hardware for this article. but shared with me his experience and knowledge of the product and how to best utilize it for Oracle RAC.. While the hardware used for this article should be stable enough for educational purposes... it should never be considered for a production environment.. with a Bachelor's degree in Computer Science.Customisation END #----if [ -x /sbin/lvm.. I currently know of two methods to get Openfiler to automatically load the logical volumes on reboot.. # LVM2 initialization... Pennsylvania. Hunter [ www.. do [ -r $file ] && restorecon $file >/dev/null 2>&1 done fi ...... The following is a small portion of the /etc/rc.sysinit script to basically wait for the USB disk ( /dev/sda in my example) to be detected...MJONES .... The RAC solution presented in this article can be put together for around US$2.. Method 2 This method was kindly provided by Martin Jones.94 GB] inherit '/dev/rac1/asm4' [115. His work includes advanced performance tuning... restart each of the Oracle RAC nodes in the cluster .000 to US$20......iDevelopment.......00 GB] inherit '/dev/rac1/asm1' [115.. First. 32. His workaround includes amending the /etc/rc.. verify the external drives are powered on and then reboot the Openfiler server. Inc. After making the changes to the /etc/rc. Lastly. Finally.. Acknowledgements An article of this magnitude and complexity is generally not the work of one person alone. Author. Bane was also involved with hardware recommendations and testing. however.. Page 1 Page 2 Page 3 converted by Web2PDFConvert. Although I was able to author and successfully demonstrate the validity of the components that make up this configuration.cache . and Windows server environment... Linux...static vgscan --mknodes --ignorelockingfailure && /sbin/lvm. In this section. check that the iSCSI target service is running: # service iscsi-target status ietd (pid 2668) is running.( racnode1 and racnode2). His research and hard work made the task of configuring Openfiler seamless.... LDAP." echo "Waiting. and Intel...94 GB] inherit '/dev/rac1/asm2' [115..( racnode1 and racnode2)." sleep 5 done echo "INFO ... Then. After you have verified that each of the logical volumes will be active on boot.......94 GB] inherit '/dev/rac1/asm3' [115.00 GB] inherit '/dev/rac1/asm1' [115..

Sign up to vote on this title
UsefulNot useful