27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

eMarcel.com
ORACLE DATABASE, FUSION MIDDLEWARE, LINUX
HOME

ABOUT

ENTER THE LINUX

ORACLE

AFTER HOURS

Installing Oracle RAC 11gR2 on Linux 64bit

November28,2011

eMarcel

Oracle,

1Comment

Yet another way of installing Oracle RAC 11gR2

ThistutorialshowshowtoinstallOracleRAC11gR2onLinux64bitwithNASconnected
viaNativeMultipathing(DeviceMapperMultipath).

1. Introduction
OracleRealApplicationClusters(OracleRAC)enablesanOracledatabasetorunacross
a cluster of servers, providing fault tolerance, performance, and scalability with no
applicationchangesnecessary.OracleRACprovideshighavailabilityforapplicationsby
removingthesinglepointoffailurewithasingleserver.
This guide describes how to install, configure, and manage Oracle RAC, Oracle
Clusterware, and Oracle Automatic Storage Management (Oracle ASM). It provides
examplesforhowyoucoulddeployOracleRAConatwonodecluster,usingtheOracle
Linuxoperatingsystem.Thisinstallationguideassumesseveralpoints,seebelow.

Follow eMarcel on Twitter

Tweets
Steven Feuerstein
@sfonplsql

Follow
18 Oct

8 Most InDemand Programming

Languages of 2015: #SQL is NUMBER
ONE! Amazing...
codingdojo.com/blog/8mostin
pic.twitter.com/ZPZxmVIwsa
Retweeted by eMarcel.com

Tweet to @eMarcelCom

Recent Posts
Aroundtheworldflight12
NaplestoPalermo
OracleDatabase12cInMemory
HowToGetStarted

Table of Contents
1.Introduction
1.1.Assumptionstothisguide
2.DownloadOracleSoftware

http://emarcel.com/installingoraclerac11gr2/

UpgradeOracleDatabase12c
withASM12.1.0.1to12.1.0.2
InstallingOracleSOASuite12c
(12.1.3.0)
HowtoeasycheckOracle
WebLogicServerversion
1/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

3.OracleDatabaseRAC11gR2Overview
3.1.ArchitectureDiagram
3.2.OracleRACConfigurationreference
4.PreparationforOracleDatabaseRAC11gR2Installation
4.1.InstallingRequiredLinuxPackagesforOracleRAC11gR2
4.2.NetworkConfiguration
4.3.ManuallyAssigningStaticIPAddress(TheDNSMethod)
4.4.ConfiguringPublicandPrivateNetwork

Popular Posts
InstallingOracleDatabase11gR2
onLinuxwithASM
RecoverDecryptWebLogicServer
AdminPassword
CotojestBASHKursBASH
DataMaskingwithOracle
EnterpriseManager12cCloud
Control
InstallingOracleDatabase12cR1
onLinux6withASM

4.5.InstallingandconfiguringDNSserverforOracleRAC

Tags

4.6.ConfiguringSCANwithoutDNS(optional)

11g12.1.0.212ca320neoAirbusASM

4.7.Stopthefirewallonbothnodes
4.8.ClusterTimeSynchronizationServiceNTPservice
5.CreateJobRoleSeparationOperatingSystemPrivilegesGroups,Users,and

database

BilbaoControlcreateDAAG

delete drop farogrid howtoinstallation

linuxLinux64bit

LEBBLEPALFMLLFPO

LIRNLSGSLXGBmultitenantNaples

Directories

oracle

5.1.CreateGroupsandUserforGridInfrastructure

scriptServer SOASUITEsusudotruncate

5.2.CreateLoginScriptforthegridUserAccount

PantherapasswordPipistrelR2

WebLogic

tutorialunixupgradeweblogic

serverxplane

5.3.CreateGroupsandUserforOracleDatabaseSoftware
5.4.CreateLoginScriptfortheoracleUserAccount
5.5.VerifyThattheUsernobodyExists
5.6.CreatetheOracleBaseDirectoryPath
5.7.SetResourceLimitsfortheOracleSoftwareInstallationUsers
6.ConfiguretheLinuxServersforOracle,kernelparameters
6.1.ConfiguringSElinuxoraclenote:[ID454196.1]
7.InstallingandConfiguringASMLib2.0
7.1.InstallASMLib2.0PackagesonERHL5
7.2.ConfigureASMLib
8.CreateASMDisksforOracle
9.InstallOracleGridInfrastructureforaCluster
9.1.PostinstallationTasksforOracleGridInfrastructureforaCluster
9.2.CreateASMDiskGroupsforDataandFastRecoveryArea(FRA)
10.InstallOracleDatabase11gwithOracleRealApplicationClusters
10.1.InstallOracleDatabase11gRelease2Software

http://emarcel.com/installingoraclerac11gr2/

2/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

10.2.CreatetheOracleClusterDatabase
10.3.OracleEnterpriseManagerDebaseControl
11.VerifyOracleGridInfrastructureandDatabaseConfiguration

1.1. Assumptions of this guide

RedHatEnterpriseLinux5.6(64bit)OperatingSystemhasbeeninstalledontwo
nodes
MemoryandSwapspacerequirementsforOracleRAC11gR2hasbeenmet
NASstoragehasbeenconnectedastheDeviceMapperMultipathonRedHat
EnterpriseLinux5.6
DNSserverwiththreepublicIPsresolvingtoonedatabaseSCANaddress/domain
hasbeenconfigured
AworkaroundhasbeendescribedlaterinthisguidetoinstallClusterwarewithoutDNS
server.
Howeveritisstronglynotrecommendedlearnaproperway
ClusterTimeSynchronizationService,hereNTPisupandrunningandavailableto
thecluster
ThefollowinglistcontainsotherusefulOracleRACguidesorrelateddocumentation:
OracleRealApplicationClustersAdministrationandDeploymentGuide
OracleClusterwareAdministrationandDeploymentGuide
OracleRealApplicationClustersInstallationGuideforLinuxandUNIX
OracleGridInfrastructureInstallationGuideforLinux
OracleMaximumAvailabilityArchitectureMAA
SpecialthanksforfolkswhowrotethefollowingarticlesBigThanksGuys!:
BuildYourOwnOracleRAC11gClusteronOracleEnterpriseLinuxandiSCSI
(verywellexplainedandbroadarticle!)
Oracle11gRelease2SCANexplained
11gR2Modify/AddScanIP&Listeners

2. Download Oracle Software

DownloadandextracttherequiredOraclesoftwarepackagesfromtheOracleTechnology
Network(OTN):
NOTE:IfyoudonotcurrentlyhaveanaccountwithOracleOTN,justcreateone.This
accountisfree.
OracleDatabase11gRelease2(11.2.0.1.0)StandardEdition,StandardEditionOne,and
EnterpriseEdition
(11.2.0.1.0)Linuxx8664(Makesuretodownloadandunzipbothfilestothesame
directory,instructionsinthisguide)

http://emarcel.com/installingoraclerac11gr2/

3/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

OracleDatabase11gRelease2GridInfrastructure(11.2.0.1.0)forLinuxx8664
linux.x64_11gR2_grid.zip(1,052,897,657bytes)
Contains the Grid Infrastructure Software including Oracle Clusterware, Automated
StorageManagement(ASM),andASMClusterFileSystem.Downloadandinstallpriorto
installingOracleRealApplicationClusters,OracleRealApplicationClustersOneNode,or
otherOraclesoftwareinaGridEnvironment
OracleDatabase11gRelease2(11.2.0.1.0)forLinuxx8664
linux.x64_11gR2_database_1of2.zip(1,239,269,270bytes)
linux.x64_11gR2_database_2of2.zip(1,111,416,131bytes)

3.OracleDatabaseRAC11gR2Overview

3.1. Architecture Diagram

Highleveloverviewofwhatwargoingtobuild:

3.2. Oracle RAC Configuration reference

http://emarcel.com/installingoraclerac11gr2/

4/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

4. Preparation for Oracle Database RAC 11gR2

Installation
4.1. Installing Required Linux Packages for Oracle RAC 11gR2
RedHatEnterpriseLinux64bit(x86_64)
binutils2.17.50.0.6
compatlibstdc++333.2.3
compatlibstdc++333.2.3(32bit)
elfutilslibelf0.125
elfutilslibelfdevel0.125
elfutilslibelfdevelstatic0.125
gcc4.1.2
gccc++4.1.2
glibc2.524
glibc2.524(32bit)
glibccommon2.5
glibcdevel2.5
glibcdevel2.5(32bit)
glibcheaders2.5
http://emarcel.com/installingoraclerac11gr2/

5/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

ksh20060214
libaio0.3.106
libaio0.3.106(32bit)
libaiodevel0.3.106
libaiodevel0.3.106(32bit)
libgcc4.1.2
libgcc4.1.2(32bit)
libstdc++4.1.2
libstdc++4.1.2(32bit)
libstdc++devel4.1.2
make3.81
sysstat7.0.2
unixODBC2.2.11
unixODBC2.2.11(32bit)
unixODBCdevel2.2.11
unixODBCdevel2.2.11(32bit)
NOTE:Inabovelistpackagesversionsmayvary.Pleaseinstallthelatestversionsofthe
rpmsavailableinyourrepositoryorDVDLinuxdisk.
EachofthepackageslistedabovecanbefoundonDVDEnterpriseLinux5.6(x86_64).
Whileitispossibletoqueryeachindividualpackagetodeterminewhichonesaremissing
and need to be installed, an easier method is to run the rpm Uvh PackageName
commandfromtheDVDasfollows.
Note:Forpackagesthatalreadyexistandareuptodate,theRPMcommandwillsimply
ignoretheinstallandprintawarningmessagetotheconsolethatthepackageisalready
installed.

InstallingLinuxpackagesfromEnterpriseLinux5.6(x86_64)[DVD]
MountDVDimageto/media/cromastherootuserandrunthefollowingcommands:
mkdirp/media/cdrom
mountr/dev/cdrom/media/cdrom
cd/media/cdrom/Server
rpmUvhbinutils2.*
rpmUvhelfutilslibelf0.*
rpmUvhglibc2.*
rpmUvhglibccommon2.*
rpmUvhksh2*
rpmUvhlibaio0.*
rpmUvhlibgcc4.*
rpmUvhlibstdc++4.*
rpmUvhmake3.*
rpmUvhelfutilslibelfdevel*
rpmUvhgcc4.*
rpmUvhgccc++4.*
rpmUvhglibcdevel2.*
rpmUvhglibcheaders2.*
rpmUvhlibstdc++devel4.*
rpmUvhunixODBC2.*
rpmUvhcompatlibstdc++33*
rpmUvhlibaiodevel0.*

http://emarcel.com/installingoraclerac11gr2/

6/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

rpmUvhunixODBCdevel2.*
rpmUvhsysstat7.*

4.2. Network Configuration

Webondseparateinterfacestoacommoninterfacetoprovideredundancy,incaseofa
NICfailure.
bond0
formaintenance

eth0
eth1

bond1

eth2
eth3

bond2

eth4
eth5

bond3

eth6
eth7

Thepublicinterfacenamesassociatedwiththenetworkadaptersforeachnetworkmust
be the same on all nodes, and the private interface names associated with the network
adaptors should be the same on all nodes. For example, with our twonode cluster, you
cannot configure network adapters on node1 with bond1 as the public interface, but on
node2havebond2asthepublicinterface.Publicinterfacenamesmustbethesame,so
you must configure bond1 as public on both nodes. You should configure the private
interfaces on the same network adapters as well. If bond2 and bond3 are the private
interfacesfornode1,thenbond2andbond3mustbetheprivateinterfacesfornode2.
For the private network, the endpoints of all designated interconnect interfaces must be
completelyreachableonthenetwork.Thereshouldbenonodethatisnotconnectedto
every private network interface. You can test if an interconnect interface is reachable
usingpingand/ortraceroute.
DuringinstallationofOraclegridinfrastructure,youareaskedtoidentifytheplanneduse
foreachnetworkinterfacethatOUIdetectsonyourclusternode.Youmustidentifyeach
interfaceasapublicinterface,aprivateinterface,ornotusedandyoumustusethesame
privateinterfacesforbothOracleClusterwareandOracleRAC.
NOTE:HowtobondNICinterfacescanbefoundhere.

4.3. Manually Assigning Static IP Address (The DNS Method)

WedonotplantoextendthisRACarchitecturewithnewnodesinthefuture.Thatswhy
wechosetonotuseGNS.ManuallydefiningstaticIPaddressesisstillavailablewith
OracleClusterware11gRelease2andwillbethemethodusedinthisdocumenttoassign
allrequiredOracleClusterwarenetworkingcomponents(publicIPaddressforthenode,
RACinterconnect,virtualIPaddress,andSCAN).
Oracle recommends that static IP addresses be manually configured in a domain name
server (DNS) before starting the Oracle grid infrastructure installation. This installation
guideassumesthatDNSserverhasbeenconfiguredandresolvesthreeIPaddressesto
the same database scan name (i.e. myracdbscan) using Round Robin algoritm.
Installation and configuration of the DNS server has been described in section Installing
andconfiguringDNSserverforOracleRACofthisguide.

http://emarcel.com/installingoraclerac11gr2/

7/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

NOTE: If you do not have access to a DNS server, this guide provides an easy
workaround in the section Configuring SCAN without DNS. The workaround involves
modifying the nslookup utility and should be performed before installing Oracle grid
infrastructure.ThisistemporarysolutionuntillDNSserverisavailableanditsstronglynot
recommendedtoavoidinstallationerrorsinfurtherstage.

4.4. Configuring Public and Private Network

UsingtheNetworkConfigurationapplication,youneedtoconfigurebothNICdevicesas
well as the /etc/hosts file. Both of these tasks can be completed using the Network
Configuration GUI or manually by creating and editing appropriate files in
/etc/sysconfig/networkscripts/directory.
Noticethatthe/etc/hostssettingsarethesameforbothnodesandthatweremovedany
entrythathastodowithIPv6.
Forexample(removeorcommentthisline):
#::1localhost6.localdomain6localhost6
OurOracleRACconfigurationhasthefollowingnetworksettings.
OracleRACNode1(racnode1)
OracleRACNode1(racnode1)
Device

IPAddress

Subnet

Gateway

Purpose

bond1

10.75.40.143

255.255.255.0

10.75.40.129

Connectsracnode1tothepublic

bond2

172.18.0.11

255.255.255.192

network
Connectsracnode1(interconnect)torac
node2(racnode2priv)

bond3

172.18.0.27

255.255.255.192

bond1:1

10.75.40.251

255.255.255.0

VirtualIP(VIP)isforfast
connectionestablishmentin
failoverdictation

ConfigureDNSServers:/etc/resolv.conf
nameserver10.75.40.139
nameserver10.75.40.xxx
searchcompany.local

/etc/hoststhesameonbothnodes(makesurethatloopback(127)doesnthavehostname)
AlsoUPDATEDNSservertoresolvetobelowentries
#Donotremovethefollowingline,orvariousprograms
#thatrequirenetworkfunctionalitywillfail.

127.0.0.1localhost.localdomainlocalhost

#Publicnetwork(bond1)
10.75.40.143racnode1racnode1
10.75.40.144racnode2racnode2

#PrivateNetwork(bond2)
172.18.0.11racnode11privracnode11priv
172.18.0.12racnode21privracnode21priv

#(bond3)
172.18.0.27racnode12privracnode12priv
172.18.0.28racnode22privracnode22priv

http://emarcel.com/installingoraclerac11gr2/

8/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#VirtualNetwork(bond1:1)
10.75.40.251racnode1vipracnode1vip
10.75.40.252racnode2vipracnode2vip

#SCAN(optionalonlyifDNSserverhasnotbeenconfigured)
#10.75.40.200myracdbscan.company.localmyracdbscan

OracleRACNode2(racnode2)
Device
bond1

IPAddress
10.75.40.144

bond2

172.18.0.12

bond3
bond1:1

172.18.0.28
10.75.40.252

OracleRACNode2(racnode2)
Subnet
Gateway
Purpose
255.255.255.0
10.75.40.129
Connectsracnode2tothepublic
network
255.255.255.192

Connectsracnode2(interconnect)
toracnode1(racnode1priv)
255.255.255.192

255.255.255.0

VirtualIP(VIP)isforfast
connectionestablishmentin
failoverdictation

ConfigureDNSServers:/etc/resolv.conf
nameserver10.75.40.139
nameserver10.75.40.xxx
searchcompany.local

/etc/hoststhesameonbothnodes(makesurethatloopback(127)doesntcontainhostname)
AlsoUPDATEDNSservertoresolvetobelowentries

#Donotremovethefollowingline,orvariousprograms
#thatrequirenetworkfunctionalitywillfail.

127.0.0.1localhost.localdomainlocalhost

#Publicnetwork(bond1)
10.75.40.143racnode1racnode1
10.75.40.144racnode2racnode2

#PrivateNetwork(bond2)
172.18.0.11racnode11privracnode11priv
172.18.0.12racnode21privracnode21priv

#bond3
172.18.0.27racnode12privracnode12priv
172.18.0.28racnode22privracnode22priv

#VirtualNetwork(bond1:1)
10.75.40.251racnode1vipracnode1vip
10.75.40.252racnode2vipracnode2vip

#SCAN(optional,onlyifDNSserverhasnotbeenconfiguredyet)
#10.75.40.200myracdbscan.company.localmyracdbscan

ConfirmtheRACNodeNameisNotListedinLoopbackAddress
Ensurethatthenodenames(racnode1orracnode22)arenotincludedfortheloopback
addressinthe/etc/hostsfile.Ifthemachinenameislistedintheintheloopbackaddress
entryasbelow:
anexampleofincorrectentry:
127.0.0.1racnode1localhost.localdomainlocalhost
Abovehostnamestringracnode1mustberemovedasshownbelow:
127.0.0.1localhost.localdomainlocalhost

4.5. Installing and configuring DNS server for Oracle RAC

NOTE:IfyoudontwanttogothroughmanualDNSinstallationyoucaninstallWebmin
http://emarcel.com/installingoraclerac11gr2/

9/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

webbasedguiadmintoolandconfigureBINDDNSserver.
OracleRedHatEnterpriseLinuxdeliversrpmpackagesforthenameserverbind.Bind
isdevelopedbytheISC(Internetsystemsconsortium)andisthemostuseddnsserver.
Moreinformationcanbefoundhere.
Installingrequiredpackages
ForRedHatEL5weneedatleastthefollowingpackagesinstalled:
bindlibs9.*
bind9.*
bindutils9.*
ToinstallabovepackagesusethesameapproachasdescribedinsectionInstalling
RequiredLinuxPackagesforOracleRAC11gR2ofthisguide.
Configuringthemasterdnsserver(secondaryDNSserverisoutofthescope)
CreatingthemasterDNSserverwhichwillresideonlinuxhost.Inourcasethemaster
dnsserverresidesontemporaryhostDNSHOSTwithipadress10.75.40.139
NOTE:ForaquickanddirtyconfigurationoneconfiguredDNSserverisenoughto
completetheclusterverificationtests.Makesurethatyouspecifiedallnodenames
(private,publicandvipnamed)in/etc/hostsfileonbothnodes[racnode1/2]!
Creatingtherequiredfiles
create/etc/named.conffileandpastethefollowingcode
[root@dnshostnamed]#mv/etc/named.conf/etc/named.conf.org
[root@dnshostnamed]#vi/etc/named.conf

Pastethefollowingcode:
options{
#Thedirectorystatementdefinesthenameserver'sworkingdirectory
directory"/var/named";
#Writedumpandstatisticsfiletothelogsubdirectory.The
#pathenamesarerelativetothechrootjail.
dumpfile"/var/log/named_dump.db";
statisticsfile"/var/log/named.stats";
#Theforwardersrecordcontainsalistofserverstowhichqueries
#shouldbeforwarded.EnablethislineandmodifytheIPaddressto
#yourprovider'snameserver.Uptothreeserversmaybelisted.
#forwarders{192.0.2.1;192.0.2.2;};
#Enablethenextentrytopreferusageofthenameserverdeclaredin
#theforwarderssection.
#forwardfirst;
#Thelistenonrecordcontainsalistoflocalnetworkinterfacesto
#listenon.Optionallytheportcanbespecified.Defaultisto
#listenonallinterfacesfoundonyoursystem.Thedefaultportis
#53.
#listenonport53{127.0.0.1;};
#Thelistenonv6recordenablesordisableslisteningonIPv6
#interfaces.Allowedvaluesare'any'and'none'oralistof
#addresses.
listenonv6{none;};
#Thenextthreestatementsmaybeneededifafirewallstandsbetween

http://emarcel.com/installingoraclerac11gr2/

10/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#thelocalserverandtheinternet.
#querysourceaddress*port53;
#transfersource*port53;
#notifysource*port53;
#TheallowqueryrecordcontainsalistofnetworksorIPaddresses
#toacceptanddenyqueriesfrom.Thedefaultistoallowqueries
#fromallhosts.
#allowquery{127.0.0.1;};
#Ifnotifynotify">issettoyes(default),notifymessagesaresenttoother
#nameserverswhenthethezonedataischanged.Insteadofsetting
#aglobal'notify'statementinthe'options'section,aseparate
#'notify'canbeaddedtoeachzonedefinition.
notifyno;
};
logging{
#Logqueriestoafilelimitedtoasizeof100MB.
channelquery_logging{
file"/var/named/named_querylog"
versions3size100M;
printtimeyes;//timestamplogentries
};
categoryqueries{
query_logging;
};
#Orlogthiskindalternativelytosyslog.
channelsyslog_queries{
sysloguser;
severityinfo;
};
categoryqueries{syslog_queries;};
#Loggeneralnameservererrorstosyslog.
channelsyslog_errors{
sysloguser;
severityerror;
};
categorydefault{syslog_errors;};
#Don'tloglameservermessages.
categorylameservers{null;};
};
aclacl_transf{10.75.40.139;};
#Thefollowingzonedefinitionsdon'tneedanymodification.Thefirstone
#isthedefinitionoftherootnameservers.Thesecondonedefines
#localhostwhilethethirddefinesthereverselookupforlocalhost.
#zone"."in{
#typehint;
#file"root.hint";
#};
zone"localhost"in{
typemaster;
file"localhost.zone";
};
zone"0.0.127.inaddr.arpa"in{
typemaster;
file"127.0.0.zone";
};
zone"company.local"in{
typemaster;
file"db";
allowtransfer{acl_transf;};
};

Modifythefollowingfile:
/var/named/127.0.0.zone
[root@dnshostnamed]$mv/var/named/127.0.0.zone/var/named/127.0.0.zone.org
[root@dnshostnamed]$vi/var/named/127.0.0.zone

http://emarcel.com/installingoraclerac11gr2/

11/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

Pastethefollowingcodeinto/var/named/127.0.0.zone
$TTL1W
@INSOAlocalhost.root.localhost.(
42;serial(d.adams)
2D;refresh
4H;retry
6W;expiry
1W);minimum
INNSlocalhost.
1INPTRlocalhost.

Pastebelowcodeintothe/var/named/localhost.zone
$TTL1W
@INSOA@root(
42;serial(d.adams)
2D;refresh
4H;retry
6W;expiry
1W);minimum
INNS@
INA127.0.0.1

Modifythefollowingfile:/var/named/db
[root@dnshostnamed]#mv/var/named/db/var/named/db.org
[root@dnshostnamed]#vi/var/named/db

Pastethefollowingcodeinto/var/named/db
$ORIGIN.
company.localSOAdnshost.(
22;serial
900;refresh(15minutes)
600;retry(10minutes)
86400;expire(1day)
3600;minimum(1hour)
)
NSdnshost.
$ORIGINcompany.local.
racnode1A10.75.40.143
racnode2A10.75.40.144
racnode1vipA10.75.40.251
racnode2vipA10.75.40.252
racnode11privA172.18.0.11
racnode21privA172.18.0.12
racnode12privA172.18.0.27
racnode22privA172.18.0.28
myracdbscanA10.75.40.200
myracdbscanA10.75.40.201
myracdbscanA10.75.40.202

http://emarcel.com/installingoraclerac11gr2/

12/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

Changefilepermissions:
NOTE:Userandgroupnamedmustbecreatedinthesystem.Ifotherwisecreateit.
[root@dnshostnamed]#touch/var/named/named_querylog
[root@dnshostnamed]#chownRnamed:named/var/named/
[root@dnshostnamed]#chownnamed:named/etc/named.conf
#Afterthatwecanstartnameddaemonforthefirsttime:
[root@dnshostnamed]#/etc/init.d/namedstart

ConfigureDNSserversonRAChosts
InordertopointserverstotherightDNSservereditthefile/etc/resolv.confinRACnodes
andupdatefilewiththefollowingentries:
nameserver10.75.40.139searchcompany.local

NOTE:InproductionenvironmentyouwouldhavetwoDNSnameservers
TestDNSserverresponse:
[user@racnode1~]$nslookupmyracdbscan
nslookupmyracdbscan
Server:10.75.40.139
Address:10.75.40.139#53
Nonauthoritativeanswer:
Name:myracdbscan
Address:10.75.40.200
Name:myracdbscan
Address:10.75.40.201
Name:myracdbscan
Address:10.75.40.202

TroubleshootingDNSServer
Ifyouexperienceproblemscheckoutthefollowinglogfilesformoredetailedinformation:
/var/log/messages
/var/named/named_querylog

4.6. Configuring SCAN without DNS (optional)

ModifythenslookuputilityasrootonbothOracleRACnodesasfollows.
First,renametheoriginalnslookupbinarytonslookup.originalonbothOracleRACnodes:
[root@racnode1~]#mv/usr/bin/nslookup/usr/bin/nslookup.original

Next,createanewshellscriptnamed/usr/bin/nslookupasshownbelowwhilereplacing
xx.xxx.x.xxwithyourprimaryDNS(canbeafake),racnodeclusterscanwithyourSCAN
hostname,andyyy.yyy.y.yyywithyourSCANIPaddress.
FROMatempalte

TObelike(examplefromracnode1)

#!/bin/bash

#!/bin/bash

http://emarcel.com/installingoraclerac11gr2/

13/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

HOSTNAME=${1}

HOSTNAME=${1}

if[[$HOSTNAME=racnodeclusterscan

if[[$HOSTNAME=myracdbscan]]

]]then

then

echoServer:xx.xxx.x.xx

echoServer:10.75.40.139

echoAddress:xx.xxx.x.xx#53

echoAddress:10.75.40.139#53

echoNonauthoritativeanswer:

echo

echoName:racnodeclusterscan

echoNonauthoritativeanswer:

echoAddress:yyy.yyy.y.yyy

echoName:myracdbscan

else

echoAddress:10.75.40.200

/usr/bin/nslookup.original$HOSTNAME

echoName:myracdbscan

fi

echoAddress:10.75.40.201
echoName:myracdbscan
echoAddress:10.75.40.202
else
/usr/bin/nslookup.original$HOSTNAME
fi

Finally,changethenewnslookupshellscripttobeexecutable:
[root@racnode1~]#chmod755/usr/bin/nslookup

RemembertoperformtheseactionsonbothOracleRACnodes.Thenewnslookupshell
script simply echos back your SCAN IP address whenever the oracle installation test
scriptcallsnslookupwithyourSCANhostnameotherwise,itcallstheoriginalnslookup
binary.

4.7. Stop the firewall on both nodes

Astherootuserperformthefollowingonbothnodes:
/etc/rc.d/init.d/iptablesstop
chkconfigiptablesoff
/etc/rc.d/init.d/iptablesstatus

4.8. Cluster Time Synchronization Service NTP service

Oracle Clusterware 11g Release 2 and later requires time synchronization across all
nodeswithinaclusterwhereOracleRACisdeployed.Oracleprovidetwooptionsfortime
synchronization:anoperatingsystemconfigurednetworktimeprotocol(NTP),orthenew
Oracle Cluster Time Synchronization Service (CTSS). Oracle Cluster Time
Synchronization Service (ctssd) is designed for organizations whose Oracle RAC
databasesareunabletoaccessNTPservices.
OnbothnodesmodifytheNTPinitializationfiletosetthexflag,whichpreventstime
frombeingadjustedbackward.Restartthenetworktimeprotocoldaemonafteryou
completethistask.
TodothisonOracleEnterpriseLinux,editthe/etc/sysconfig/ntpdfiletoaddthexflag,
asinthefollowingexample:
#Droproottoid'ntp:ntp'bydefault.

http://emarcel.com/installingoraclerac11gr2/

14/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

OPTIONS="xuntp:ntpp/var/run/ntpd.pid"
#Setto'yes'tosynchwclockaftersuccessfulntpdate
SYNC_HWCLOCK=no
#Additionaloptionsforntpdate
NTPDATE_OPTIONS=""

Restartntpserviceastherootuser:
/sbin/servicentpdrestart

5. Create Job Role Separation Operating System

Privileges Groups, Users, and Directories
ThefollowingO/Sgroupswillbecreated:
Description
OracleInventoryandSoftwareOwner
OracleAutomaticStorageManagement
(ASM)Group
ASMDatabaseAdministratorGroup

OSGroup

OSUsersAssigned

Name

tothisGroup

oinstall

grid,oracle

asmadmin

grid

SYSASM

OSASM

grid,

SYSDBA

OSDBA

oracle

forASM

forASM

SYSOPER

OSOPER

forASM

forASM

asmdba

OraclePrivilege

OracleGroup
Name

ASMOperatorGroup

asmoper

grid

DatabaseAdministrator

dba

oracle

SYSDBA

OSDBA

DatabaseOperator

oper

oracle

SYSOPER

OSOPER

5.1. Create Groups and User for Grid Infrastructure

CreatingtherecommendedOSgroupsanduserforGridInfrastructureonbothOracle
RACnodes.
[root@racnode1~]#
groupaddg1000oinstall
groupaddg1200asmadmin
groupaddg1201asmdba
groupaddg1202asmoper
useraddmu1100goinstallGasmadmin,asmdba,asmoperd/home/grids/bin/bashgri
d
[root@racnode1~]#idgrid
uid=1100(grid)gid=1000(oinstall)groups=1000(oinstall),1200(asmadmin),1201(asmdba),120
2(asmoper)
#Setthepasswordforthegridaccount:
[root@racnode1~]#passwdgrid
#now,performabovetasksonthesecondnode.

5.2.CreateLoginScriptforthegridUserAccount
http://emarcel.com/installingoraclerac11gr2/

15/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

LogintobothOracleRACnodesasthegriduseraccountandcreatethefollowinglogin
script(.bash_profile):
Note:WhensettingtheOracleenvironmentvariablesforeachOracleRACnode,make
certaintoassigneachRACnodeauniqueOracleSID.
Forourenvironment,wehavethefollowingentriespernode:
racnode1:ORACLE_SID=+ASM1(.bash_profilefileonnode1)
racnode2:ORACLE_SID=+ASM2(.bash_profilefileonnode2)
MAKESUREYOUHAVE+ASM2ENTRYONNODE2BUTNOT+ASM1
Switchtogriduser,cleanandedit.bash_profilefile:
[root@racnode1~]#sugrid
[grid@racnode1~]$echo>.bash_profile
[grid@racnode1~]$vi.bash_profile

Pastethefollowingconfigurationandsavethefile.bash_profile
#
#.bash_profile
#
#OSUser:grid
#Application:OracleGridInfrastructure
#Version:Oracle11gRelease2
#
#Getthealiasesandfunctions
if[f~/.bashrc];then
.~/.bashrc
fi
aliasls="lsFA"
#
#ORACLE_SID
#
#SpecifiestheOraclesystemidentifier(SID)
#fortheAutomaticStorageManagement(ASM)instance
#runningonthisnode.
#
#EachRACnodemusthaveauniqueORACLE_SID.
#
#(i.e.+ASM1,+ASM2,...)
#
ORACLE_SID=+ASM1;exportORACLE_SID
#
#JAVA_HOME
#
#SpecifiesthedirectoryoftheJavaSDKandRuntime
#Environment.
#
JAVA_HOME=/usr/local/java;exportJAVA_HOME
#
#ORACLE_BASE
#
#SpecifiesthebaseoftheOracledirectorystructure
#forOptimalFlexibleArchitecture(OFA)compliant
#installations.TheOraclebasedirectoryforthe
#gridinstallationowneristhelocationwhere
#diagnosticandadministrativelogs,andotherlogs
#associatedwithOracleASMandOracleClusterware
#arestored.
#
ORACLE_BASE=/u01/app/grid;exportORACLE_BASE
#
#ORACLE_HOME

http://emarcel.com/installingoraclerac11gr2/

16/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#
#SpecifiesthedirectorycontainingtheOracle
#GridInfrastructuresoftware.Forgrid
#infrastructureforaclusterinstallations,theGrid
#homemustnotbeplacedunderoneoftheOraclebase
#directories,orunderOraclehomedirectoriesof
#OracleDatabaseinstallationowners,orinthehome
#directoryofaninstallationowner.During
#installation,ownershipofthepathtotheGrid
#homeischangedtoroot.Thischangecauses
#permissionerrorsforotherinstallations.
#
ORACLE_HOME=/u01/app/11.2.0/grid;exportORACLE_HOME
#
#ORACLE_PATH
#
#SpecifiesthesearchpathforfilesusedbyOracle
#applicationssuchasSQL*Plus.Ifthefullpathto
#thefileisnotspecified,orifthefileisnot
#inthecurrentdirectory,theOracleapplication
#usesORACLE_PATHtolocatethefile.
#ThisvariableisusedbySQL*Plus,FormsandMenu.
#
ORACLE_PATH=/u01/app/oracle/common/oracle/sql;
exportORACLE_PATH
#
#SQLPATH
#
#Specifiesthedirectoryorlistofdirectoriesthat
#SQL*Plussearchesforalogin.sqlfile.
#
#SQLPATH=/u01/app/common/oracle/sql;exportSQLPATH
#
#ORACLE_TERM
#
#Definesaterminaldefinition.Ifnotset,it
#defaultstothevalueofyourTERMenvironment
#variable.Usedbyallcharactermodeproducts.
#
ORACLE_TERM=xterm;exportORACLE_TERM
#
#NLS_DATE_FORMAT
#
#Specifiesthedefaultdateformattousewiththe
#TO_CHARandTO_DATEfunctions.Thedefaultvalueof
#thisparameterisdeterminedbyNLS_TERRITORY.The
#valueofthisparametercanbeanyvaliddate
#formatmask,andthevaluemustbesurroundedby
#doublequotationmarks.Forexample:
#
#NLS_DATE_FORMAT="MM/DD/YYYY"
#
#
NLS_DATE_FORMAT="DDMONYYYYHH24:MI:SS";exportNLS_DATE_FORMAT
#
#TNS_ADMIN
#
#SpecifiesthedirectorycontainingtheOracleNet
#Servicesconfigurationfileslikelistener.ora,
#tnsnames.ora,andsqlnet.ora.
#
TNS_ADMIN=$ORACLE_HOME/network/admin;exportTNS_ADMIN
#
#ORA_NLS11
#
#Specifiesthedirectorywherethelanguage,
#territory,characterset,andlinguisticdefinition
#filesarestored.
#
ORA_NLS11=$ORACLE_HOME/nls/data;exportORA_NLS11
#

http://emarcel.com/installingoraclerac11gr2/

17/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#PATH
#
#Usedbytheshelltolocateexecutableprograms;
#mustincludethe$ORACLE_HOME/bindirectory.
#
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
exportPATH
#
#LD_LIBRARY_PATH
#
#Specifiesthelistofdirectoriesthattheshared
#libraryloadersearchestolocatesharedobject
#librariesatruntime.
#
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
exportLD_LIBRARY_PATH
#
#CLASSPATH
#
#Specifiesthedirectoryorlistofdirectoriesthat
#containcompiledJavaclasses.
#
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
exportCLASSPATH
#
#THREADS_FLAG
#
#AllthetoolsintheJDKusegreenthreadsasa
#default.Tospecifythatnativethreadsshouldbe
#used,settheTHREADS_FLAGenvironmentvariableto
#"native".Youcanreverttotheuseofgreen
#threadsbysettingTHREADS_FLAGtothevalue
#"green".
#
THREADS_FLAG=native;exportTHREADS_FLAG
#
#TEMP,TMP,andTMPDIR
#
#Specifythedefaultdirectoriesfortemporary
#files;ifset,toolsthatcreatetemporaryfiles
#createtheminoneofthesedirectories.
#
exportTEMP=/tmp
exportTMPDIR=/tmp
#
#UMASK
#
#Setthedefaultfilemodecreationmask
#(umask)to022toensurethattheuserperforming
#theOraclesoftwareinstallationcreatesfiles
#with644permissions.
#
umask022

5.3. Create Groups and User for Oracle Database Software

CreatingtherecommendedOSgroupsanduserfortheOracledatabasesoftwareonboth
OracleRACnodes
http://emarcel.com/installingoraclerac11gr2/

18/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

[root@racnode1~]#
groupaddg1300dba
groupaddg1301oper
useraddmu1101goinstallGdba,oper,asmdbad/home/oracles/bin/bashoracle
[root@racnode1~]#idoracle
uid=1101(oracle)gid=1000(oinstall)groups=1000(oinstall),1201(asmdba),1300(dba),1301(op
er)
#Setthepasswordfortheoracleaccount:
[root@racnode1~]#passwdoracle
#now,performabovetasksonthesecondnode.

5.4. Create Login Script for the oracle User Account

LogintobothOracleRACnodesastheoracleuseraccountandcreatethefollowinglogin
script
(.bash_profile):
NOTE:WhensettingtheOracleenvironmentvariablesforeachOracleRACnode,make
certaintoassigneachRACnodeauniqueOracleSID.Forthisexample,weused:
inracnode1:ORACLE_SID=myracdb1
inracnode2:ORACLE_SID=myracdb2
Switchtooracleuser,cleanandedit.bash_profilefile:
[root@racnode1~]$suoracle
[oracle@racnode1~]$echo>/home/oracle/.bash_profile
[oracle@racnode1~]$vi/home/oracle/.bash_profile

Pastethefollowingconfigurationandsavethefile
Setup:ORACLE_SIDandORACLE_UNQNAMEaccordingtoyournameconvention
#
#.bash_profile
#
#OSUser:oracle
#Application:OracleDatabaseSoftwareOwner
#Version:Oracle11gRelease2
#
#Getthealiasesandfunctions
if[f~/.bashrc];then
.~/.bashrc
fi
aliasls="lsFA"
#
#ORACLE_SID
#
#SpecifiestheOraclesystemidentifier(SID)for
#theOracleinstancerunningonthisnode.
#
#EachRACnodemusthaveauniqueORACLE_SID.
#(i.e.racdb1,racdb2,...)
#
ORACLE_SID=racdb1;exportORACLE_SID
#
#ORACLE_UNQNAME

http://emarcel.com/installingoraclerac11gr2/

19/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#
#InpreviousreleasesofOracleDatabase,youwere
#requiredtosetenvironmentvariablesfor
#ORACLE_HOMEandORACLE_SIDtostart,stop,and
#checkthestatusofEnterpriseManager.With
#OracleDatabase11grelease2(11.2)andlater,you
#needtosettheenvironmentvariablesORACLE_HOME
#andORACLE_UNQNAMEtouseEnterpriseManager.
#SetORACLE_UNQNAMEequaltothedatabaseunique
#name.
#
ORACLE_UNQNAME=racdb;exportORACLE_UNQNAME
#
#JAVA_HOME
#
#SpecifiesthedirectoryoftheJavaSDKandRuntime
#Environment.
#
JAVA_HOME=/usr/local/java;exportJAVA_HOME
#
#ORACLE_BASE
#
#SpecifiesthebaseoftheOracledirectorystructure
#forOptimalFlexibleArchitecture(OFA)compliant
#databasesoftwareinstallations.
#
ORACLE_BASE=/u01/app/oracle;exportORACLE_BASE
#
#ORACLE_HOME
#
#SpecifiesthedirectorycontainingtheOracle
#Databasesoftware.
#
ORACLE_HOME=$ORACLE_BASE/product/11.2.0/dbhome_1
exportORACLE_HOME
#
#ORACLE_PATH
#
#SpecifiesthesearchpathforfilesusedbyOracle
#applicationssuchasSQL*Plus.Ifthefullpathto
#thefileisnotspecified,orifthefileisnot
#inthecurrentdirectory,theOracleapplication
#usesORACLE_PATHtolocatethefile.
#ThisvariableisusedbySQL*Plus,FormsandMenu.
#
ORACLE_PATH=/u01/app/common/oracle/sql
exportORACLE_PATH
#
#SQLPATH
#
#Specifiesthedirectoryorlistofdirectoriesthat
#SQL*Plussearchesforalogin.sqlfile.
#
#SQLPATH=/u01/app/common/oracle/sql;exportSQLPATH
#
#ORACLE_TERM
#
#Definesaterminaldefinition.Ifnotset,it
#defaultstothevalueofyourTERMenvironment
#variable.Usedbyallcharactermodeproducts.
#
ORACLE_TERM=xterm;exportORACLE_TERM
#
#NLS_DATE_FORMAT
#
#Specifiesthedefaultdateformattousewiththe
#TO_CHARandTO_DATEfunctions.Thedefaultvalueof
#thisparameterisdeterminedbyNLS_TERRITORY.The
#valueofthisparametercanbeanyvaliddate
#formatmask,andthevaluemustbesurroundedby
#doublequotationmarks.Forexample:

http://emarcel.com/installingoraclerac11gr2/

20/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#
#NLS_DATE_FORMAT="MM/DD/YYYY"
#
#
NLS_DATE_FORMAT="DDMONYYYYHH24:MI:SS"
exportNLS_DATE_FORMAT
#
#TNS_ADMIN
#
#SpecifiesthedirectorycontainingtheOracleNet
#Servicesconfigurationfileslikelistener.ora,
#tnsnames.ora,andsqlnet.ora.
#
TNS_ADMIN=$ORACLE_HOME/network/admin;exportTNS_ADMIN
#
#ORA_NLS11
#
#Specifiesthedirectorywherethelanguage,
#territory,characterset,andlinguisticdefinition
#filesarestored.
#
ORA_NLS11=$ORACLE_HOME/nls/data;exportORA_NLS11
#
#PATH
#
#Usedbytheshelltolocateexecutableprograms;
#mustincludethe$ORACLE_HOME/bindirectory.
#
PATH=.:${JAVA_HOME}/bin:${PATH}:$HOME/bin:$ORACLE_HOME/bin
PATH=${PATH}:/usr/bin:/bin:/usr/bin/X11:/usr/local/bin
PATH=${PATH}:/u01/app/common/oracle/bin
exportPATH
#
#LD_LIBRARY_PATH
#
#Specifiesthelistofdirectoriesthattheshared
#libraryloadersearchestolocatesharedobject
#librariesatruntime.
#
LD_LIBRARY_PATH=$ORACLE_HOME/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$ORACLE_HOME/oracm/lib
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/lib:/usr/lib:/usr/local/lib
exportLD_LIBRARY_PATH
#
#CLASSPATH
#
#Specifiesthedirectoryorlistofdirectoriesthat
#containcompiledJavaclasses.
#
CLASSPATH=$ORACLE_HOME/JRE
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/rdbms/jlib
CLASSPATH=${CLASSPATH}:$ORACLE_HOME/network/jlib
exportCLASSPATH
#
#THREADS_FLAG
#
#AllthetoolsintheJDKusegreenthreadsasa
#default.Tospecifythatnativethreadsshouldbe
#used,settheTHREADS_FLAGenvironmentvariableto
#"native".Youcanreverttotheuseofgreen
#threadsbysettingTHREADS_FLAGtothevalue
#"green".
#
THREADS_FLAG=native;exportTHREADS_FLAG
#
#TEMP,TMP,andTMPDIR
#
#Specifythedefaultdirectoriesfortemporary
#files;ifset,toolsthatcreatetemporaryfiles
#createtheminoneofthesedirectories.

http://emarcel.com/installingoraclerac11gr2/

21/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#
exportTEMP=/tmp
exportTMPDIR=/tmp
#
#UMASK
#
#Setthedefaultfilemodecreationmask
#(umask)to022toensurethattheuserperforming
#theOraclesoftwareinstallationcreatesfiles
#with644permissions.
#
umask022

5.5. Verify That the User nobody Exists

Beforeinstallingthesoftware,completethefollowingproceduretoverifythattheuser
nobodyexistsonbothOracleRACnodes:
Todetermineiftheuserexists,issuethefollowingcommand:
#idnobodyuid=99(nobody)gid=99(nobody)groups=99(nobody)

Ifthiscommanddisplaysinformationaboutthenobodyuser,thenyoudonothaveto
createthatuser.
Iftheusernobodydoesnotexist,thenenterthefollowingcommandastherootuserto
createit:
#/usr/sbin/useraddnobody

RepeatthisprocedureonalltheotherOracleRACnodesinthecluster.

5.6. Create the Oracle Base Directory Path

FollowinghastobeperformedonbothOracleRACnodesintheclusterasroot.
[root@racnode1~]#
mkdirp/u01/app/grid
mkdirp/u01/app/11.2.0/grid
chownRgrid:oinstall/u01
mkdirp/u01/app/oracle
chownoracle:oinstall/u01/app/oracle
chmodR775/u01

5.7. Set Resource Limits for the Oracle Software Installation Users
To improve the performance of the software on Linux systems, you must increase the
followingresourcelimitsfortheOraclesoftwareownerusers(grid,oracle):
ShellLimit
Maximumnumberofopenfiledescriptors
Maximumnumberofprocessesavailabletoasingleuser
Maximumsizeofthestacksegmentoftheprocess

Iteminlimits.conf
nofile
nproc
stack

HardLimit
65536
16384
10240

Tomakethesechanges,runthefollowingasroot:
On each Oracle RAC node, add the following lines to the /etc/security/limits.conf file
http://emarcel.com/installingoraclerac11gr2/

22/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

(thefollowingexampleshowsthesoftwareaccountownersoracleandgrid):
[root@racnode1~]#cat>>/etc/security/limits.conf<<EOF
gridsoftnproc2047
gridhardnproc16384
gridsoftnofile1024
gridhardnofile65536
oraclesoftnproc2047
oraclehardnproc16384
oraclesoftnofile1024
oraclehardnofile65536
EOF

OneachOracleRACnode,addoreditthefollowinglineinthe/etc/pam.d/loginfile,ifit
doesnotexist:
[root@racnode1~]$cat>>/etc/pam.d/login<<EOF
sessionrequiredpam_limits.so
EOF

Depending on your shell environment, make the following changes to the default shell
startup file, to change ulimit setting for all Oracle installation owners (note that these
examplesshowtheusersoracleandgrid):
FortheBourne,Bash,orKornshell,addthefollowinglinestothe/etc/profilefileby
runningthefollowingcommand:
[root@racnode1~]#cat>>/etc/profile<<EOF
if[\$USER="oracle"]||[\$USER="grid"];then
if[\$SHELL="/bin/ksh"];then
ulimitp16384
ulimitn65536
else
ulimitu16384n65536
fi
umask022
fi
EOF

FortheCshell(cshortcsh),addthefollowinglinestothe/etc/csh.loginfilebyrunningthe
followingcommand:
[root@racnode1~]#cat>>/etc/csh.login<<EOF
if(\$USER=="oracle"||\$USER=="grid")
then
limitmaxproc16384
limitdescriptors65536
endif
EOF

6. Configure the Linux Servers for Oracle, kernel

parameters
NOTE:Oneoftheoptionstoapplybelowsettingsistocopysysctl.conffilefromracnode1
hostonwhichkernelparametershavealreadybeenconfigured.
The kernel parameters described in this section will need to be defined on both Oracle
http://emarcel.com/installingoraclerac11gr2/

23/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

RAC nodes. This section provides information about setting those kernel parameters
requiredforOracle.Instructionsforplacingtheminastartupscript(/etc/sysctl.conf).
OnbothOracleRACnodes,verifythatthekernelparametersdescribedinthissectionare
set to values greater than or equal to the recommended values. Also note that when
settingthefoursemaphorevaluesthatallfourvaluesneedtobeenteredononeline.
OracleDatabase11grelease2onRHEL5requiresthekernelparametersettingsshown
below.Thevaluesgivenareminimums,soifyoursystemusesalargervalue,donot
changeit.
kernel.shmmax=4294967295
kernel.shmall=2097152
kernel.shmmni=4096
kernel.sem=25032000100128
fs.filemax=6815744
net.ipv4.ip_local_port_range=900065500
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_default=262144
net.core.wmem_max=1048576
fs.aiomaxnr=1048576

RHEL5alreadycomesconfiguredwithdefaultvaluesdefinedforthefollowingkernel
parameters:
kernel.shmall
kernel.shmmax
Usethedefaultvaluesiftheyarethesameorlargerthantherequiredvalues.
ThisguideassumesafreshnewinstallofOracleEnterpriseLinux5andassuch,manyof
therequiredkernelparametersarealreadyset(seeabove).Thisbeingthecase,youcan
simplycopy/pastethefollowingtobothOracleRACnodeswhileloggedinasroot:
[root@racnode1~]$cat>>/etc/sysctl.conf<<EOF
#Controlsthemaximumnumberofsharedmemorysegmentssystemwide
kernel.shmmni=4096
#Setsthefollowingsemaphorevalues:
#SEMMSL_valueSEMMNS_valueSEMOPM_valueSEMMNI_value
kernel.sem=25032000100128
#SetsthemaximumnumberoffilehandlesthattheLinuxkernelwillallocate
fs.filemax=6815744
#DefinesthelocalportrangethatisusedbyTCPandUDP
#traffictochoosethelocalport
net.ipv4.ip_local_port_range=900065500
#Defaultsettinginbytesofthesocket"receive"bufferwhich
#maybesetbyusingtheSO_RCVBUFsocketoption
net.core.rmem_default=262144
#Maximumsettinginbytesofthesocket"receive"bufferwhich

http://emarcel.com/installingoraclerac11gr2/

24/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

#maybesetbyusingtheSO_RCVBUFsocketoption
net.core.rmem_max=4194304
#Defaultsettinginbytesofthesocket"send"bufferwhich
#maybesetbyusingtheSO_SNDBUFsocketoption
net.core.wmem_default=262144
#Maximumsettinginbytesofthesocket"send"bufferwhich
#maybesetbyusingtheSO_SNDBUFsocketoption
net.core.wmem_max=1048576
#MaxnumberofallowableconcurrentasynchronousI/Orequests
fs.aiomaxnr=1048576
EOF

Toactivatethenewkernelparametervaluesforthecurrentlyrunningsystem,runthe
followingasrootonbothOracleRACnodesinthecluster:
[root@racnode1~]#sysctlp

6.1. Configuring SElinux oracle note: [ID 454196.1]

SwitchSELinuxfromthedefaultEnforcingmodethatitisrunningin,tothedisable
mode.
Edit/etc/sysconfig/selinuxfileastherootuser:
[root@racnode1~]$vi/etc/sysconfig/selinux

andsetaSELINUXvariabletodisabled
SELINUX=disabled
RestartOperatingsystemorissuethefollowingcommandastherootusertoswitchto
Permissivemode:
Commands,asroot:
[root@racnode1~]$getenforce(returns"Enforcing")
[root@racnode1~]$setenforce0
[root@racnode1~]$getenforce(returns"Permissive")

7. Installing and Configuring ASMLib 2.0

Theinstallationandconfigurationproceduresinthissectionshouldbeperformedonboth
oftheOracleRACnodesinthecluster.CreatingtheAutomaticStorageManagement
(ASM)disks,however,willonlyneedtobeperformedonasinglenodewithinthecluster
(racnode1).
ASMwillbeusedasthesharedfilesystemandvolumemanagerforOracleClusterware
files(OCRandvotingdisk),OracleDatabasefiles(data,onlineredologs,controlfiles,
archivedredologs),andtheFastRecoveryArea.
http://emarcel.com/installingoraclerac11gr2/

25/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

7.1. Install ASMLib 2.0 Packages on ERHL 5

TheASMLib2.0softwarestackincludesthefollowingpackages:
64bit(x86_64)Installations
ASMLibKernelDriver:oracleasmx.x.xx.el5x.x.xx.el5.x86_64.rpm(fordefault
kernel)
UserspaceLibrary:oracleasmlibx.x.xx.el5.x86_64.rpm
DriverSupportFiles:oracleasmsupportx.x.xx.el5.x86_64.rpm
OracleASMLibDownloadsforRedHatEnterpriseLinuxServer5
Downloadandinstallthefollowingdriversforkernel2.6.18238.el5
oracleasm2.6.18238.el52.0.51.el5.x86_64.rpm
oracleasmlib2.0.41.el5.x86_64.rpm
oracleasmsupport2.1.71.el5.x86_64.rpm
AfterdownloadingtherpmstobothOracleRACnodesinthecluster,installitusing
thefollowingcommandsastherootuser:
[root@racnode1~]#rpmUvhoracleasmsupport2.1.31.el5.x86_64.rpm
[root@racnode1~]#rpmUvhoracleasm2.6.18164.el52.0.51.el5.x86_64.rpm
[root@racnode1~]#rpmUvhoracleasmlib2.0.41.el5.x86_64.rpm

7.2. Configure ASMLib

ConfiguringandloadtheASMkernelmodule.ThistaskneedstoberunonbothOracle
RACnodesastherootuser.
[root@racnode1~]#/usr/sbin/oracleasmconfigurei
ConfiguringtheOracleASMlibrarydriver.
ThiswillconfiguretheonbootpropertiesoftheOracleASMlibrary
driver.Thefollowingquestionswilldeterminewhetherthedriveris
loadedonbootandwhatpermissionsitwillhave.Thecurrentvalues
willbeshowninbrackets('[]').Hitting<ENTER>withouttypingan
answerwillkeepthatcurrentvalue.CtrlCwillabort.
Defaultusertoownthedriverinterface[]:grid
Defaultgrouptoownthedriverinterface[]:asmadmin
StartOracleASMlibrarydriveronboot(y/n)[n]:y
ScanforOracleASMdisksonboot(y/n)[y]:y
WritingOracleASMlibrarydriverconfiguration:done

Thescriptcompletesthefollowingtasks:
Createsthe/etc/sysconfig/oracleasmconfigurationfile
Createsthe/dev/oracleasmmountpoint
MountstheASMLibdriverfilesystem
Enterthefollowingcommandtoloadtheoracleasmkernelmodule:
[root@racnode1~]$/usr/sbin/oracleasminit

http://emarcel.com/installingoraclerac11gr2/

26/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

Repeatthisprocedureonallnodesinthecluster(racnode1andracnode2).
NOTE:ToavoidASMissuewithMultipathdevicesonLinuxORA15186
InallRACnodesmodifythe/etc/sysconfig/oracleasmwith:
ORACLEASM_SCANORDER="dm"
ORACLEASM_SCANEXCLUDE="sd"

Inallnodesrestarttheasmlibastheroot
#/etc/init.d/oracleasmstop
#/etc/init.d/oracleasmstart

8. Create ASM Disks for Oracle

CreatingtheASMdisksonlyneedstobeperformedfromonenodeintheRACclusteras
therootuser.
OneoftheprerequisiteswastopreparethestorageastheDeviceMapperMultipath.
Usersshouldbeabletodisplayinformationaboutmultipatheddevices.
Todisplayinformationaboutavailabledevicesissuethefollowingcommandastheroot
user:
Exaple:
[root@racnode1~]#multipathll
mpath1(3600a0b800011a1ee0000040646828cc5)dm1IBM,1815FAStT
[size=1024M][features=1queue_if_no_path][hwhandler=1rdac]
\_roundrobin0[prio=6][active]
\_29:0:0:1sdf8:80[active][ready]
\_28:0:1:1sdl8:176[active][ready]
\_roundrobin0[prio=0][enabled]
\_28:0:0:1sdb8:16[active][ghost]
\_29:0:1:1sdq65:0[active][ghost]

Recordthempathnumbersandthecapacityofthedevices.
InourexamplewearegoingtocreatethefollowingASMDISKS:
Storage
Component
OCR/VotingDisk

FileSystem

DISKName

Numberofdisks

MultipathedDevice

ASM

5x(1GB)

DatabaseFiles

ASM

CRSVOL1
CRSVOL2
CRSVOL3
CRSVOL4
CRSVOL5
DATAVOL1
DATAVOL2

RecoveryArea

ASM

FRAVOL1
FRAVOL2

2x(250GB)

/dev/mapper/mpath*
i.e.
/dev/mapper/mpath0
/dev/mapper/mpath1

/dev/mapper/mpath*
i.e.
/dev/mapper/mpath5
/dev/mapper/mpath6
/dev/mapper/mpath*
i.e.
/dev/mapper/mpath7

http://emarcel.com/installingoraclerac11gr2/

2x(250GB)

/dev/mapper/mpath8

27/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com
/dev/mapper/mpath8

NOTE:DeterminempathXdevicenumberaccordingtoyourOSconfiguration!
Inthisexamplewehavethefollowingconfiguration:
1GBmpath0
1GBmpath1
1GBmpath2
1GBmpath3
1GBmpath4
250GBmpath5
250GBmpath6
250GBmpath7
250GBmpath8

TocreatetheASMdisksusingtheMultipathedDevicetargetnamestolocaldevicename
mappings,issuethefollowingcommandsastheroot:
CRSVOL:
[root@racnode1~]#/usr/sbin/oracleasmcreatediskCRSVOL1/dev/mapper/mpath0
[root@racnode1~]#/usr/sbin/oracleasmcreatediskCRSVOL2/dev/mapper/mpath1
[root@racnode1~]#/usr/sbin/oracleasmcreatediskCRSVOL3/dev/mapper/mpath2
[root@racnode1~]#/usr/sbin/oracleasmcreatediskCRSVOL4/dev/mapper/mpath3
[root@racnode1~]#/usr/sbin/oracleasmcreatediskCRSVOL5/dev/mapper/mpath4
Writingdiskheader:done
Instantiatingdisk:done

DATAVOL:
[root@racnode1~]#/usr/sbin/oracleasmcreatediskDATAVOL1/dev/mapper/mpath5
[root@racnode1~]#/usr/sbin/oracleasmcreatediskDATAVOL2/dev/mapper/mpath6
Writingdiskheader:done
Instantiatingdisk:done

FRAVOL:
[root@racnode1~]#/usr/sbin/oracleasmcreatediskFRAVOL1/dev/mapper/mpath7
[root@racnode1~]#/usr/sbin/oracleasmcreatediskFRAVOL2/dev/mapper/mpath8
Writingdiskheader:done
Instantiatingdisk:done

Tomakethediskavailableontheothernodesinthecluster(racnode2),enterthe
followingcommandastherootoneachnode:
[root@racnode2~]#/usr/sbin/oracleasmscandisks
Reloadingdiskpartitions:done
CleaninganystaleASMdisks...
ScanningsystemforASMdisks...
Instantiatingdisk"DATAVOL1"
Instantiatingdisk"DATAVOL2"
Instantiatingdisk"CRSVOL1"

http://emarcel.com/installingoraclerac11gr2/

28/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

Instantiatingdisk"CRSVOL2"
Instantiatingdisk"CRSVOL3"
Instantiatingdisk"CRSVOL4"
Instantiatingdisk"CRSVOL5"
Instantiatingdisk"FRAVOL1"
Instantiatingdisk"FRAVOL2"

Now you can test that the ASM disks were successfully created by using the following
command on both nodes in the RAC cluster as the root user account. This command
identifies shared disks attached to the node that are marked as Automatic Storage
Managementdisks:
[root@racnode1~]$/usr/sbin/oracleasmlistdisks
CRSVOL1
CRSVOL2
CRSVOL3
CRSVOL4
CRSVOL5
DATAVOL1
DATAVOL2
FRAVOL1
FRAVOL2

9. Install Oracle Grid Infrastructure for a Cluster

Perform the following installation procedures from only one of the Oracle RAC nodes in
the cluster (racnode1). The Oracle grid infrastructure software (Oracle Clusterware and
AutomaticStorageManagement)willbeinstalledtobothoftheOracleRACnodesinthe
clusterbytheOracleUniversalInstaller.
NOTE: At any time during installation, if you have a question about what you are being
askedtodo,clicktheHelpbuttonontheOUIpage.
CompletethefollowingstepstoinstallOraclegridinfrastructureonyourcluster.
Uploadinstallationpackagesontheserver(racnode1)intohomeusersdirectory.
OracleDatabase11gRelease2GridInfrastructure(11.2.0.1.0)forLinux
olinux.x64_11gR2_grid.zip/home/grid
OracleDatabase11gRelease2(11.2.0.1.0)forLinux
olinux.x64_11gR2_database_1of2.zip/home/oracle
olinux.x64_11gR2_database_2of2.zip/home/oracle
ExtracttheOracleGridInfrastructuresoftwareasthegriduserintoa/home/grid/stage
directory:
[grid@racnode1~]$cd;unziplinux.x64_11gR2_grid.zipd/home/grid/stage

SwitchtooracleuserandextractOracleDatabasesoftwareintoa/home/oracle/stage
directory
[oracle@racnode1~]$cd;foriin*.zip;dounzip$id/home/oracle/stage;done

http://emarcel.com/installingoraclerac11gr2/

29/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

PerformthefollowingtasksasthegridusertoinstallOracleGridInfrastructure:
FromthispointmakesureyouhaveXServerrunningonyourlocalhosttobeableto
displaygraphicalinterfacesforexampleXmingXServerforWindows
[grid@racnode1~]$cd/home/grid/stage/grid[grid@racnode1grid]$./runInstaller

OracleInstallerwillstart:

http://emarcel.com/installingoraclerac11gr2/

30/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

9.1. Postinstallation Tasks for Oracle Grid Infrastructure for a

Cluster
PerformthefollowingpostinstallationproceduresonbothOracleRACnodesinthecluster
toverifyOracleClusterwareInstallation.
RunthefollowingcommandsonbothnodesintheRACclusterasthegriduser:
CheckCRSStatus
[grid@racnode1~]$crsctlcheckcrs
CRS4638:OracleHighAvailabilityServicesisonline
CRS4537:ClusterReadyServicesisonline
CRS4529:ClusterSynchronizationServicesisonline
CRS4533:EventManagerisonline

CheckClusterNodes
[grid@racnode1~]$lsnodesn
racnode11racnode22

ConfirmingOracleASMFunctionforOracleClusterwareFiles
[grid@racnode1~]$srvctlstatusasma
ASMisrunningonracnode1,racnode2
ASMisenabled.

CheckOracleClusterRegistry(OCR)andvotingdisks
[grid@racnode1~]$ocrcheck
[grid@racnode1~]$crsctlquerycssvotedisk

9.2. Create ASM Disk Groups for Data and Fast Recovery Area
(FRA)
RuntheASMConfigurationAssistant(asmca)asthegriduserfromonlyonenodeinthe
cluster(racnode1)tocreatetheadditionalASMdiskgroupwhichwillbeusedlaterinthis
guidetocreatetheclustereddatabase.
TheASMdiskgroupwillbenamed+DATAandwillbeusedtostoreallOraclephysical
databasefiles(data,onlineredologs,controlfiles).
PerformthefollowingtasksasthegridusertocreateadditionalASMdiskgroup:
[grid@racnode1~]$asmca&

ScreenName
DiskGroups
CreateDisk
Group

Response
FromtheDiskGroupstab,clickthe[Create]button.
DATA

http://emarcel.com/installingoraclerac11gr2/

31/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com
TheCreateDiskGroupshowstwooftheASMLibvolumesDATAVOL1andDATAVOL2andtwo
forFRA.
WhencreatingtheDataASMdiskgroup,specifyDATAfortheDiskGroupName
IntheRedundancysection,chooseExternal(none)
CheckboxORCL:DATAVOL1andORCL:DATAVOL2intheSelectMemberDisks
section
Afterverifyingallvaluesinthisdialogarecorrect,clickthe[OK]button.
NEXTFRA
WhencreatingtheFRAASMdiskgroup,specifyFRAfortheDiskGroupName
IntheRedundancysection,chooseExternal(none)
CheckboxORCL:FRAVOL1andORCL:FRAVOL2intheSelectMemberDiskssection
Afterverifyingallvaluesinthisdialogarecorrect,clickthe[OK]button.

DiskGroups

AftercreatingtheASMdiskgroups,youwillbereturnedtotheinitialdialog.Clickthe[Exit]
button.

10. Install Oracle Database 11g with Oracle Real

Application Clusters
PerformtheOracleDatabasesoftwareinstallationfromonlyoneoftheOracleRACnodes
inthecluster(racnode1).TheOracleDatabasesoftwarewillbeinstalledtobothofOracle
RACnodesintheclusterbytheOracleUniversalInstallerusingSSHprotocol.
WewillskiptheCreateDatabaseoptionwheninstallingtheOracleDatabasesoftware.
The clustered database will be created later in this guide using the Database
ConfigurationAssistant(DBCA)afterallinstallshavebeencompleted.

10.1. Install Oracle Database 11g Release 2 Software

Makesurethatoinstallgrouphaswritepermissionto/u01/appdirectoryifnotrun
thefollowingcommands
#NODE1andNODE2asroot
[root@racnode1~]#chmod775/u01/app
[root@racnode2~]#chmod775/u01/app

PerformthefollowingtasksastheoracleuseronthefirstnodetoinstalltheOracle
Databasesoftware:
NOTE:XServerwillneedtoberunningonlocalhost.
[oracle@racnode1~]$cd/home/oracle/stage/database
[oracle@racnode1database]$./runInstaller

ScreenName
InstallationLocation

Response
SpecifytheOraclebaseandSoftwarelocation(Oracle_home)asfollows:
OracleBase:/u01/app
SoftwareLocation:/u01/app/oracle/product/11.2.0/dbhome_1
PrerequisiteChecks
TheinstallerwillrunthroughaseriesofcheckstodetermineifbothOracleRACnodes
meet the minimum requirements for installing and configuring the Oracle Database
Examplessoftware.
Startingwith11g release 2 (11.2), if any checks fail, the installer (OUI) will create shell
script programs, called fixup scripts, to resolve many incomplete system configuration
requirements.IfOUIdetectsanincompletetaskthatismarkedfixable,thenyoucan
easily fix the issue by generating the fixup script by clicking the [Fix & Check
Again]button.
http://emarcel.com/installingoraclerac11gr2/

32/36

27/10/2015

Summary
InstallProduct
Finish

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com
Again]button.
Thefixupscriptisgeneratedduringinstallation.Youwillbepromptedtorunthescript
asrootinaseparateterminalsession.Whenyourunthescript,itraiseskernelvaluesto
required minimums, if necessary, and completes other operating system configuration
tasks.
Ifallprerequisitecheckspass(aswasthecaseformyinstall),theOUIcontinuestothe
Summaryscreen.
Click[Finish]tostarttheinstallation.
TheinstallerperformstheOracleDatabasesoftwareinstallationprocessonbothOracle
RACnodes.
Attheendoftheinstallation,clickthe[Close]buttontoexittheOUI.

10.2. Create the Oracle Cluster Database

The database creation process should only be performed from one of the Oracle RAC
nodes in the cluster (racnode1). Use the Oracle Database Configuration Assistant
(DBCA)tocreatetheclustereddatabase.
BeforeexecutingtheDBCA,makesurethatthe$ORACLE_HOMEand$PATHareset
appropriatelyforthe
$ORACLE_BASE/product/11.2.0/dbhome_1environment.Settingenvironmentvariables
intheloginscriptfortheoracleuseraccountwascoveredinsectionCreateLoginScript
fortheoracleUserAccount
Tostartthedatabasecreationprocess,runthefollowingastheoracleuser:
[oracle@racnode1~]$dbca&

ScreenName
WelcomeScreen
Operations
DatabaseTemplates
DatabaseIdentification

Response
SelectOracleRealApplicationClustersdatabase.
SelectCreateaDatabase.
SelectGeneralPurposeDatabase.
Clusterdatabaseconfiguration:

ConfigurationType:
Databasenaming

GlobalDatabaseName:
SIDPrefix:

ManagementOptions
DatabaseCredentials
DatabaseFileLocations

NodeSelection.
Clickthe[SelectAll]buttontoselectallservers:racnode1andracnode2
Leavethedefaultoptionshere,whichistoConfigureEnterprise
Manager/ConfigureDatabaseControlforlocalmanagement.
SelecttoUsetheSameAdministrativePasswordforAllAccounts.Enterthe
password(twice)andmakesurethepassworddoesnotstartwithadigitnumber.
Specifystoragetypeandlocationsfordatabasefiles.

StorageType:
StorageLocations:
DatabaseArea:

NOTE:IfinstallerdoesntseeASMdiskgroup+DATAevenifyoupress[Browse]
button,thenpress[Back]buttonanddothefollowingwithoutexitingtheinstaller:
Loginasarootusertoracnode1andissuethefollowingcommand:
[root@racnode1~]#usermodgoinstallGdba,asmdbaoracle
or
[root@racnode1~]#chmod6751$GRID_HOME/bin/oracle

SpecifyASMSNMP

Gobacktoinstallerandpress[Next].NowinstallershouldseeASMdiskgroup
+DATA
SpecifytheASMSNMPpasswordfortheASMinstance.

Password
http://emarcel.com/installingoraclerac11gr2/

33/36

27/10/2015
Password
RecoveryConfiguration

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com
ChecktheoptionforSpecifyFastRecoveryArea.
FortheFastRecoveryArea,clickthe[Browse]buttonandselectthediskgroup
name+FRA

FastRecoveryAreaSize:512000MB
DatabaseContent
InitializationParameters

LeavealloftheDatabaseComponents(anddestinationtablespaces)settotheir
defaultvaluesanddonotselecttheSampleSchemas.
Changeparametersforyourenvironment
Memory:

Typical
MemorySize(SGAandPGA):8192MB
Checkbox:UseAutomaticMemoryManagement
DatabaseStorage
CreationOptions

EndofDatabaseCreation

Leaveallattheirdefaultsettings
KeepthedefaultoptionCreateDatabaseselected.
SelecttoGenerateDatabaseCreationScripts.
Click[Finish]tostartthedatabasecreationprocess.
Afteracknowledgingthedatabasecreationreportandscriptgenerationdialog,the
databasecreationwillstart.
Click[OK]ontheSummaryscreen.
Attheendofthedatabasecreation,exitfromtheDBCA.

When the DBCA has completed, we will have a fully functional Oracle RAC cluster
running.
Astheoracleuserverifywhetherclustereddatabaseisopen:
[oracle@racnode1~]$sugridc"crsctlstatusresourcew"TYPEco'ora'"t"Pass
word:*****

10.3. Oracle Enterprise Manager Debase Control

IfyouconfiguredOracleEnterpriseManager(DatabaseControl),itcanbeusedtoview
thedatabaseconfigurationandcurrentstatusofthedatabase.
TheURLforthisexampleis:https://racnode1:1158/em
[oracle@racnode1~]$emctlstatusdbconsole
OracleEnterpriseManager11gDatabaseControlRelease11.2.0.1.0
Copyright(c)1996,2009OracleCorporation.Allrightsreserved.
https://racnode1:1158/em/console/aboutApplication
OracleEnterpriseManager11gisrunning.

Logsaregeneratedindirectory/u01/app/oracle/product/11.2.0/dbhome_1/racnode1_myracd
b/sysman/log

11. Verify Oracle Grid Infrastructure and Database

Configuration
ToStart/Stop/Statusoracledatabase:

http://emarcel.com/installingoraclerac11gr2/

34/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

AllOracleInstances
[oracle@racnode1~]$srvctlstopdatabasedmyracdb
[oracle@racnode1~]$srvctlstartdatabasedmyracdb
[oracle@racnode1~]$srvctlstatusdatabasedmyracdb
Instancemyracdb1isrunningonnoderacnode1
Instancemyracdb2isrunningonnoderacnode2

SingleOracleInstance(StatusofSpecificInstance)
[oracle@racnode1~]$srvctlstopinstancedmyracdbimyracdb1
[oracle@racnode1~]$srvctlstartinstancedmyracdbimyracdb1
[oracle@racnode1~]$srvctlstatusinstancedmyracdbimyracdb1
Instancemyracdb1isrunningonnoderacnode1

CheckCRSStatus
[grid@racnode1~]$crsctlcheckcrs

CheckClusterNodes
[grid@racnode1~]$olsnodesn
racnode11racnode22

CheckOracleTNSListenerProcessonBothNodes
[grid@racnode1~]$psef|greplsnr|grepv'grep'|grepv'ocfs'|awk'{print
$9}'
[grid@racnode2~]$psef|greplsnr|grepv'grep'|grepv'ocfs'|awk'{print
$9}'

ConfirmingOracleASMFunctionforOracleClusterwareFiles
[grid@racnode1~]$srvctlstatusasma
ASMisrunningonracnode1,racnode2
ASMisenabled.

CheckOracleClusterRegistry(OCR)
[grid@racnode1~]$ocrcheck

CheckVotingDisk
[grid@racnode1~]$crsctlquerycssvotedisk

Formoredetaildinformationusethefollowingtoll:
$srvctl
Usage:srvctl<command><object>[<options>]
commands:enable|disable|start|stop|relocate|status|add|remove|modify|getenv|setenv|unse
tenv|config
objects:database|instance|service|nodeapps|vip|asm|diskgroup|listener|srvpool|server|sc
an|scan_listener|
oc4j|home|filesystem|gns

http://emarcel.com/installingoraclerac11gr2/

35/36

27/10/2015

InstallingOracleRAC11gR2onLinux64bit|eMarcel.com

Fordetailedhelponeachcommandandobjectanditsoptionsuse:
srvctl<command>hor
srvctl<command><object>h
Forexample:
[oracle@racnode1~]$srvctlstatusnodeapps

ThatsallFolks!Hopeyouenjoyedreading.

Linux64bit,oracle,RAC11gR2

Moving SPFILE from file system to ASM (ORACLE RAC11g)

How To Setup SSH equivalence , password-less SSH authentication

1Comment
Recommend

OracleonLinux

Share

Login

SortbyBest

Jointhediscussion
RanjeetBadhe 8monthsago

Excellentexplanation,stepbystep,verylucid.IamgoingtotryRAC.

Subscribe

Reply Share

AddDisqustoyoursite

Privacy

HOME ABOUT ENTER THE LINUX ORACLE AFTER HOURS

20072015eMarcel.comMasteringOracleonLinux
AllRightsReserved.
Templateby:Author

http://emarcel.com/installingoraclerac11gr2/

36/36