COOKBOOK Oracle 11gRAC R1 ASM AIX5L SAN Storage Installation Guide

DRAFT
COOKBOOK
Installation Guide
Oracle 11g RAC Release 1
with Oracle Automatic
Storage Management ASM!
on IBM

S"stem #
and i Runni ng AI$ %&
wi th SA' Storage
(ersion 1)*
A#ril +**,
ORAC&- . IBM
/oint Solut ions Center
IBM Thierry Plumeau
Oracle/IBM Joint Solutions Center
Oracle - Frederic Michiara
Oracle/IBM Joint Solutions Center
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1 o# $%$
This document is &ased on our e'(eriences)
This is not an o##icial *Oracle or IBM+ documentation)
This document ,ill &e constantly u(dated
and ,e-re o(en to any add-on
or #eed&ac. #rom your o,n e'(eriences/
on same or di##erent storage solution 000
1ocument history 2
3ersion 1ate 4(date 5ho 3alidated &y
1)6 January 7668
- Creation Frederic Michiara
Thierry Plumeau
Alain Roy
Paul Bramy
!(ril 7668
- Review Dider Wo9ciecho,s.i
Paul Bramy
Contri&utors 2
o Paul Bramy 2 Oracle Cor() *O!C:;/IBM Joint Solutions Center+
o !lain oy 2 IBM France *O!C:;/IBM Joint Solutions Center+
o Fa&ienne :e(etit 2 Oracle Cor() *O!C:;/IBM Joint Solutions Center+
S(ecial than.s to the (artici(ants o# the 11g!C ,or.sho( in S,it<erland *=ene>a+ ,ho used our last
dra#t coo&oo. release/ and hel(ed us to disco>er some ?ty(o-s@/ and >alidate the content o# the coo.&&o.)
Contact 2
o ORACLE !B" #oint $olution% Center oracli&mA#r)i&m)com
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7 o# $%$
1 The aim of this document ............................................... ......................... 6
2 About Oracle clusterware ................................................. ....................... 8
3 About Oracle Automatic storage management ........................................ 15
4 About Real Alication !luster ............................................................. .. 1"
5 About 11# RA! $ A%& on A'( .......................................................... ........ 1)
6 *hat+s new with 11g RA! imlementation on A'( ................................... 2,
" infrastructure re-uirements .................................... .............................. 2,
7.1 General Requirements ..................................................................................................... ............... 22
7.1.1 About Servers and processors ................................................................................. ................ 22
7.1.2 About RAC on IBM System p ............................................................................ ........................ 2
7.1. About !et"or# ........................................................................................................... .............. 2$
7.1.$ About SA! Stora%e ................................................................................................. ................. 2&
7.1.' (roposed in)rastructure "it* 2 servers ................................................................... ................. 2+
7.1.& ,*at do "e protect - ....................................................................................................... ........ 2.
7.1.7 About IBM Advanced (o"er /irtuali0ation and RAC .................................................... ............. 1
7.2 Coo#boo# in)rastructure ...................................................................................... ........................... '
7.2.1 IBM Sytem p servers .................................................................................................... ............ &
7.2.2 2peratin% System ............................................................................................................ ........ $$
7.2. Multi3pat*in% and ASM ............................................................................ ................................ $'
7.2.$ IBM stora%e and multi3pat*in% ........................................................................ ........................ $&
7.2.' 4MC stora%e and multi3pat*in% ........................................................................... .................... '
7.2.& 5I6AC5I stora%e and multi3pat*in% ........................................................... .............................. ''
7.2.7 2t*ers7 Stora%e6e#7 5( 4/A stora%e and multi3pat*in% ....................................................... .... '7
8 %ecific cOnsiderations for RA!$A%& setu with .A!&/ installed ........... 5)
) 'nstallation stes ........................................................ .......................... 62
1, /rearing the s0stem ............................................... ........................... 64
11.1 !et"or# con)i%uration ........................................................................................ .......................... &'
11.1.1 8e)ine !et"or#s layout7 (ublic7 /irtual and (rivate 5ostnames ............................................ &'
11.1.2 Identi)y !et"or# Inter)aces cards ...................................................................... .................... &+
11.1. 9pdate *osts )ile ................................................................................................................... . 71
11.1.$ 8e)inin% 8e)ault %ate"ay on public net"or# inter)ace ...................................................... .... 72
11.1.' Con)i%ure !et"or# 6unin% (arameters ................................................... ............................... 7$
11.2 AI: 2peratin% system level7 required A(AR;s and )ilsets ........................................ ...................... 7&
11.2.1 <ilesets Requirements )or 11% RAC R1 = ASM >!2 5ACM(? .................................................... 77
11.2.2 A(AR;s Requirements )or 11% RAC R1 = ASM >!2 5ACM(? ..................................................... 7+
11. System Requirements >S"ap7 temp7 memory7 internal dis#s? ...................................................... 7.
11.$ 9sers And Groups ....................................................................................................... .................. +1
11.' @ernel and S*ell Aimits ......................................................................................... ........................ +'
11.'.1 Con)i%ure S*ell Aimits .................................................................................................... ........ +'
11.'.2 Set crs7 asm and rdbms users capabilities ................................................ ............................ +7
11.'. Set !CARGS parameter ................................................................................... ...................... +7
11.'.$ Con)i%ure System Con)i%uration (arameters .................................................................. ....... ++
11.'.' AruB)ileBrepa%e settin% .................................................................................. ........................ +.
11.'.& Async*ronous I=2 settin% ................................................................................................ ....... .1
11.& 9ser equivalences .............................................................................................. .......................... .2
11.&.1 RS5 implementation .......................................................................................................... .... .
11.&.2 SS5 implementation ...................................................................................................... ........ .$
11.7 6ime Server Sync*roni0ation ............................................................................................ ............ .+
11.+ crs environment setup ......................................................................................................... ......... ..
11.. 2racle So)t"are Requirements ......................................................................... .......................... 111
11 /rearing %torage ...................................................... ....................... 1,1
11.1 Required local dis#s >2racle Cluster"are7 ASM and RAC so)t"are? .................................. ........... 11
11.2 2racle Cluster"are 8is#s >2CR and /otin% 8is#s? ........................................... ........................... 11'
11.2.1 Required A9!;s ......................................................................................... ........................... 11&
11.2.2 5o" to Identi)y i) a A9! is used or not - ........................................................... ................... 117
11.2. Re%ister A9!;s at AI: level .............................................................................................. ..... 111
11.2.$ Identi)y A9!;s and correspondin% *dis# on eac* node ...................................... .................. 111
11.2.' Removin% reserve loc# policy on *dis#s )rom eac* node ................................ ................... 11$
11.2.& Identi)y MaCor and Minor number o) *dis# on eac* node .................................................... . 117
11.2.7 Create 9nique /irtual 8evice to access same A9! )rom eac* node .................................... 11.
11.2.+ Set 2"ners*ip = (ermissions on /irtual 8evices ........................................................ .......... 121
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$ o# $%$
11.2.. <ormatin% t*e virtual devices >0eroin%? .................................................................... ........... 122
11. ASM dis#s .................................................................................................... ............................... 12
11..1 Required A9!;s ......................................................................................... ........................... 12'
11..2 5o" to Identi)y i) a A9! is used or not - ........................................................... ................... 12'
11.. Re%ister A9!;s at AI: level .............................................................................................. ..... 127
11..$ Identi)y A9!;s and correspondin% *dis# on eac* node ...................................... .................. 12.
11..' Removin% reserve loc# policy on *dis#s )rom eac* node ................................. ................... 12
11..& Identi)y MaCor and Minor number o) *dis# on eac* node .................................................... . 1$
11..7 Create 9nique /irtual 8evice to access same A9! )rom eac* node .................................... 1&
11..+ Set 2"ners*ip = (ermissions on /irtual 8evices ........................................................ .......... 1+
11... <ormatin% t*e virtual devices >0eroin%? .................................................................... ........... 1$1
11..11 Removin% assi%ned (/I8 on *dis# ................................................................................. .... 1$2
11.$ C*ec#in% S*ared 8evices ..................................................................................... ...................... 1$
11.' Recommandations7 *ints and tips ................................................................ .............................. 1$'
11.'.1 2CR = /otin% dis#s .................................................................................. ............................. 1$'
11.'.2 ASM dis#s ................................................................................................ ............................ 1$+
12 Oracle !lusterware 1!R%2 'nstallation ................................................ 151
12.1 Cluster /eri<ication utility ............................................................................................ ............... 1'2
12.1.1 9nderstandin% and 9sin% Cluster /eri)ication 9tility ................................................... ........ 1'2
12.1.2 9sin% C/9 to 8etermine i) Installation (rerequisites are Complete ..................................... 1'2
12.2 Installation ....................................................................................................................... ........... 1&
12. (ost Installation operations ............................................................................. ............................... $
12..1 9pdate t*e Cluster"are uniD user .pro)ile ........................................................ ....................... $
12..2 /eri)y parameter CSS misscount .................................................................. ........................... '
12.. Cluster Ready Services 5ealt* C*ec# ....................................................................... ............... &
12..$ Addin% en*anced crsstat script ....................................................................... ....................... .
12..' Interconnect !et"or# con)i%uration C*ec#up ................................................................ ........ 11
12..& 2racle CAuster Re%istry content C*ec# and Bac#up .............................................. ................ 12
12.$ Some use)ull commands ................................................................................. ............................. 1.
12.' Accessin% CRS lo%s ............................................................................................................. .......... 21
12.& Cluster"are Basic 6estin% ........................................................................................... .................. 22
12.7 ,*at 5as Been 8one - ............................................................................................. .................... 2
12.+ /I( and CRS 6roubleS*outin% ......................................................................... .............................. 2
12.. 5o" to clean a )ailed CRS installation .................................................................. ........................ 2$
13 'nstall Automated %torage &anagement software ................................. 25
1.1 Installation ........................................................................................................................ ............ 2&
1.2 9pdate t*e ASM uniD user .pro)ile ........................................................................ ........................ &
1. C*ec#in% 2racle Cluster"are ................................................................................................ ........ 7
1.$ Con)i%ure de)ault node listeners .................................................................. ................................ +
1.' Create ASM Instances ................................................................................................. .................. $+
1.'.1 6*ru 8BCA ...................................................................................................... ....................... $.
1.'.2 Manual Steps ........................................................................................................... .............. &.
1.& Con)i%ure ASM local and remote listeners .................................................................... ................ 7$
1.7 C*ec# 2racle Cluster"are7 and mana%e ASM ressources ............................................. ................ +1
1.+ About ASM instances 6unin% ..................................................................................................... .... +
1.. About ASM CM8 >Command 9tility? .............................................................................. ................ +'
1.11 9se)ull Metalin# notes ................................................................................. ............................. .1
1.11 ,*at *as been done - ................................................................................................ ................ .1
14 'nstalling Oracle 11g R1 software .................................................... ..... )2
1$.1 11% R1 R8BMS Installation ..................................................................................... ...................... .$
1$.2 Symbolic lin#s creation )or listener.ora7 tnsnames.ora and sqlnet.ora ....................................... 112
1$. 9pdate t*e R8BMS uniD user .pro)ile ...................................................................................... .... 11
15 3atabase !reation On A%& ............................................................... 1,4
1'.1 6*ru 2racle 8BCA ...................................................................................................... ................. 11'
1'.2 Manual 8atabase Creation .......................................................................... ............................... 117
1'. (ost 2perations ............................................................................................. ............................. 12'
1'..1 9pdate t*e R8BMS uniD user .pro)ile .................................................................................. . 12'
1'..2 Administer and C*ec# ........................................................................................................ .. 12'
1'.$ Settin% 8atabase Aocal and remote listeners .......................................................... ................... 11
1'.' Creatin% 2racle Services ................................................................................ ............................ 1
1'.'.1 Creation t*ru srvctl command ............................................................................ ................. 1
1'.& 6ransaction Application <ailover .......................................................................... ....................... 1'.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B o# $%$
1'.7 About 8BC2!S2A4 ........................................................................................... .......................... 1&7
1'.7.1 C*ec#in% 8B C2!S2A4 ........................................................................................... ............. 1&7
1'.7.2 Movin% )rom dbconsole to Grid Control .......................................................... ..................... 171
16 asm ad4anced tools ............................................................. .............. 1"2
1&.1 )tp and *ttp access .................................................................................................. ................... 172
1" %ome usefull commands ............................. ....................................... 1"3
17.1 2racle CAuster Re%istry content C*ec# and Bac#up ................................................. .................. 17$
18 Aendi5 A 6 Oracle $ '7& technical documents ................................... 1"5
1) Aendi5 7 6 Oracle technical notes .................................................. .. 1"6
1..1 CRS and 11% Real Application Clusters ................................................................................... .... 17&
1..2 About RAC ... .................................................................................................................... ........... 1+&
1.. About CRS ... ....................................................................................................................... ........ 1+&
1..$ About /I( ... .......................................................................................................... ...................... 1+7
1..' About manual database cration ... ............................................................................................. . 1+7
1..& About Grid Control ... ............................................................................................... ................... 1+7
1..7 About 6A< ... ................................................................................................. .............................. 1+7
1..+ About Addin%=Removin% !ode ... .................................................................................. .............. 1+7
1... About ASM ... ....................................................................................................... ....................... 1+7
1..11 Metalin# note to use in case o) problem "it* CRS ... ................................................. .............. 1++
2, Aendi5 ! 6 8sefull commands ...................................................... .... 18)
21 Aendi5 3 6 9mt0 tables to use for installatio: ................................ 1))
21.1 !et"or# document to ease your installation ......................................................... ..................... 1..
21.2 Steps C*ec#out .................................................................................................................. ......... 211
21. 8is#s document to ease dis#s preparation in your implementation .............................. ............. 212
22 3ocuments; 7OO<% to loo= at .. ........................................................ .. 2,3
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C o# $%$
1 TD; !IM OF TDIS 1OC4M;ET
This document is ,ritten to (ro>ide hel( installing
Oracle11g eal !((lication Clusters *11)1+ release 1 ,ith
Oracle !utomated Storage Management on IBM System (
and i ser>ers ,ith !I")
Wit&in t&e Oracle!B" #oint $olution% Center' we are receivin( a lot of re)ue%t about RAC and A$"' t&i% i%
t&e rea%on w&y we decided to deliver t&e fir%t **(RAC A!+ coo,boo, wit& Oracle A$" -Automated
$tora(e "ana(ement.. A!+/ i% not covered a% **(RAC i% not yet %u00orted wit& A!+/' but certification
%&ould be available %oon. C&ec, on metalin, for u0date' t&e coo,boo, will be u0dated wit& A!+/ w&en
official certification will be relea%ed.
We will de%cribe %te0 by %te0 t&e arc&itecture Oracle CR$ -Clu%ter Ready $ervice. on raw di%,% and databa%e on A$"
-Automatic $tora(e "ana(ement..
1or t&e arc&itecture u%in( Oracle **(RAC wit& !B" 2P1$ a% clu%ter file% %y%tem' coo,boo, named 3Installation
Guide Oracle 11g !C elease 1 and IBM =PFS on IBM eSer>er System ( and i running !I" ,ith S!E Storage3
will be relea%ed %oon.
"etalin, -&tt04metalin,.oracle.commetalin,0l%)lml56(ui.%tartu0.
Titre Origine F#Frence
Oracle7 Databa%e Oracle Clu%terware and Oracle Real
A00lication Clu%ter% !n%tallation 2uide
**g Relea%e * -**.*. for A!+
&tt04www.oracle.com0l%db***0orta
l.0ortal6db8%elected9**:frame9;ai<6i
n%tallation6(uide%
B2+2'2
5 Day = Real A00lication Clu%ter% 2uide - **g Relea%e *
-**.*.
&tt04www.oracle.com0l%db***&om
e0a(e8remar,9ta&iti B2+2'2
Oracle Databa%e Relea%e >ote% ? **g Relea%e * -**.*. for
A!+ @L Ba%ed $y%tem% -/Abit%.
&tt04www.oracle.com0l%db***0orta
l.0ortal6db8%elected9**:frame9;ai<6i
n%tallation6(uide%
B217'
Oracle Bniver%al !n%taller Conce0t% 2uide Relea%e 5.5
&tt04download-
ea%t.oracle.comdoc%cdB*C@C*6C*
em.D5CaD//DE0reface.&tm
AD//DE-C*
Oracle7 Databa%e on A!+7'FP-B+7'Linu<7'"ac O$7
+'$olari%7'Gru/A Bni<7 O0eratin( $y%tem% !n%tallation
and Confi(uration Re)uirement% Huic, Reference -I.C.@ to
**.*.
Oracle "etalin,
&tt04metalin,.oracle.com
*/DEC/.*
A$" 1AH -1re)uently A%,ed Hue%tion%.
Oracle "etalin,
&tt04metalin,.oracle.com
JECD*@.*
Oracle ASM and Multi-Pathing Technologies
Oracle "etalin,
&tt04metalin,.oracle.com 294869.1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G o# $%$
The in#ormation contained in this (a(er resulted #rom 2
-Oracle and !B" documentation%
-Wor,%&o0 e<0erience% done in t&e Oracle!B" #oint $olution% Center
-Benc&mar,% and POC im0lementation% for cu%tomer% 0erformed by P$$C "ont0ellier
-G&i% documentation i% a Koint effort from Oracle and !B" %0eciali%t%.
Please also re#er to Oracle H IBM online documentation #or more in#ormation 2
&tt04doc%.oracle.com
o Oracle Databa%e **g Relea%e * -**.*. Documentation
&tt04www.oracle.comtec&nolo(ydocumentationdataba%e.&tml
&tt04ta&iti.oracle.com
Oracle RAC &ome 0a(e 4 &tt04www.oracle.comdataba%erac6&ome.&tml
1or more information !B" $y%tem 0 4 &tt04www-CJ.ibm.com%y%tem%0
Iour comments are im(ortant #or us/ and ,e than.s the ones ,ho send us their #eed&ac. a&out (re>ious
release/ and a&out ho, this document did hel( them in their im(lementation) We want our tec&nical 0a0er% to
be a% &el0ful a% 0o%%ible.
Please send us your comments a&out this document to the Oracle/IBM Joint Solutions Center)
4se our email address 2
oracli&mA#r)i&m)com
Or our (hone num&er 2
J$$ *6+B GK $B GK B%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K o# $%$
7 !BO4T O!C:; C:4ST;5!;
;'tract #rom 2 OracleL 1ata&ase 7 1ay J eal !((lication Clusters =uide 11g elease 1 *11)1+
Part Eum&er B787C7-67
&tt04www.oracle.com0l%db***&ome0a(e8remar,9ta&iti
Oracle Real A00lication Clu%ter% -Oracle RAC. u%e% Oracle Clu%terware a% t&e infra%tructure t&at bind% to(et&er
multi0le node% t&at t&en o0erate a% a %in(le %erver. Oracle Clu%terware i% a 0ortable clu%ter mana(ement %olution t&at
i% inte(rated wit& Oracle Databa%e. !n an Oracle RAC environment' Oracle Clu%terware monitor% all Oracle
com0onent% -%uc& a% in%tance% and li%tener%.. !f a failure occur%' Oracle Clu%terware automatically attem0t% to re%tart
t&e failed com0onent and al%o redirect% o0eration% to a %urvivin( com0onent.
Oracle Clu%terware include% a &i(& availability framewor, for mana(in( any a00lication t&at run% on your clu%ter.
Oracle Clu%terware mana(e% a00lication% to en%ure t&ey %tart w&en t&e %y%tem %tart%. Oracle Clu%terware al%o
monitor% t&e a00lication% to ma,e %ure t&at t&ey are alway% available. 1or e<am0le' if an a00lication 0roce%% fail%' t&en
Oracle Clu%terware attem0t% to re%tart t&e 0roce%% ba%ed on %cri0t% t&at you cu%tomiLe. !f a node in t&e clu%ter fail%'
t&en youcan 0ro(ram a00lication 0roce%%e% t&at ty0ically run on t&e failed node to re%tart on anot&er node in t&e
clu%ter.
Oracle Clu%terware include% two im0ortant com0onent%4 t&e votin( di%, and t&e OCR. G&e votin( di%, i% a file t&at
mana(e% information about node member%&i0' and t&e OCR i% a file t&at mana(e% clu%ter and Oracle RAC databa%e
confi(uration information.
G&e Oracle Clu%terware in%tallation 0roce%% create% t&e votin( di%, and t&e OCR on %&ared %tora(e. !f you %elect t&e
o0tion for normal redundant co0ie% durin( t&e in%tallation 0roce%%' t&en Oracle Clu%terware automatically maintain%
redundant co0ie% of t&e%e file% to 0revent t&e file% from becomin( %in(le 0oint% of failure. G&e normal redundancy
feature al%o eliminate% t&e need for t&ird-0arty %tora(e redundancy %olution%. W&en you u%e normal redundancy'
Oracle Clu%terware automatically maintain% two co0ie% of t&e OCR file and t&ree co0ie% of t&e votin( di%, file.
Mou will learn more about t&e o0eration of t&e Oracle clu%terware' &ow to build t&e clu%ter' and t&e %tructure of an
Oracle RAC databa%e in ot&er %ection% of t&i% (uide.
See !lso4
Oracle Clusterware Administration and Deployment Guide
!nd 2
1B! ;ssentials At &tt04www.oracle.com0l%db***&ome0a(e8remar,9ta&iti
"ana(e all a%0ect% of your Oracle databa%e% wit& t&e Enter0ri%e "ana(er 2B!.
5 Day DBA FG"L PD1
5 Day = Performance Gunin( 2uide FG"L PD1
5 Day = Real A00lication Clu%ter% 2uide FG"L PD1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8 o# $%$
Oracle Cluster,are/ =rou( Mem&ershi( and Deart&eats M
Cluster needs to .no, ,ho is a mem&er at all times
Oracle Cluster,are has 7 heart&eats
Eet,or. heart&eat
I# a node does not send a heart&eat #or
MissCount *time in seconds+/ then node is
e>icted #rom cluster)
1is. heart&eat
I# dis. heart&eat is not u(dated in I/O
timeout/ then node is e>icted #rom cluster
Oracle Cluster,are/ OC and 3oting dis.s M
Oracle Cluster egistry contains all
in#ormation on the cluster)
Since 16g!C 7 OC can &e
mirrored
3oting is the cluster heart&eat)
Since 16g!C 7 >oting can ha>e
multi(le co(ies in odd num&er #or
ma9ority)
11g!C/!SM/!I" oraclibm@fr.ibm.com % o# $%$
Oracle Cluster,are and its >irtual IP-s M
- 1 -
;ach node (artici(ating to the Oracle Cluster has 2
1 Static IP address *16)$)7C)81+
1 3irtual IP address/ each >irtual address
*16)$)7C)181+ has its home node
- 7 -
I# one node #ails 2
$tatic !P -*C.J.5@.I*. from failed node wil not
be anymore reac&able
Nirtual !P -*C.J.5@.*I*. from failed node will
%witc& to one of t&e remainin( node' and will
remain reac&able
- $ -
5hen #ailed node return to normal o(eration 2
$tatic !P -*C.J.5@.I*. will be bac, and
reac&able
Nirtual !P -*C.J.5@.*I*. from failed node'
&o%ted on one of t&e remainin( node% will
%witc& bac, to it% &ome node
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16 o# $%$
Protecting Single Instance 1ata&ase and Thrid a((lication tier thru Oracle Cluster,are M
- 1 -
4sing Oacle cluster,are/ ,e can ha>e Cluster 1ata&ases/ &ut
also non cluster data&ase) In our e'em(le/ ,e ha>e one single
instance data&ase on one node)
;ach node has its o,n node 3IP/ and ,e can create an
a((lication 3IP *!PPS 3IP+ on ,ich di##erent resources are
de(endents 2
!SM Single instance/
D Single 1ata&ase Instance/
:istener
D a((lication tier
/a((s and /oracle local #iles systems
- 7 -
I# #irst node ,here !PPS 3IP is hosted #ailed TD;E 2
>ode N!P -*C.J.5@.I*. will %witc& to an ot&er node
APP$ N!P -*C.J.5@.5CC. will %witc& to a 0reffered node -if confi(ured' inK our ca%e
on t&e t&ird node. !f t&ird node wa% not available' APP$ N!P will %witc& to one avail
able node..
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11 o# $%$
- $ -
APP$ N!P$ &a% %witc&ed to 0referred J
rd
node. GFE>' All re%ource%
de0endant of APP$ N!P will be re%tarted on t&ird node.
- B -
W&en node * come bac, to normal o0eration' node N!P -*C.J.5@.I*. will %witc& bac,
automatically to it% &ome node' meanin( fir%t node. BBG APP$ N!P$ will not %witc&
bac, O
- C -
To s,itch &ac. !PPS 3IP and its de(endant resources to #irst node/ administrator ,ill ha>e to relocate the !PPS 3IP to #irst node thru cluster,are commands)
Sending command on cluster e>ents thru Oracle Cluster,are M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17 o# $%$
- 1 -
5ith Oracle cluster,are/ all cluster e>ents
*node/ data&ase/ instance/ etc M+
in#ormation *u(/ do,n/ etc M+ can &e
o&tained and used to associate an action)
In our e'am(le case/ ,e ha>e 2
$ nodes/ ,ith $ :P!-s each *!C/ !PPS/
T;ST/1;3+
IBM DMC to administrate the IBM PO5;
System ( and :P!-s *cores/ memory+
In normal o(eration/ all :P!-s are using

their de#ined cores and memory/ &ut
could also use micro-(artitionning
&et,een :P!-s)
- 7 -
!n our e<am0le ca%e' if fir%t node fail%' GFE> 4
>ode N!P -*C.J.5@.I*. from fir%t node will %witc&
on one available node.
Oracle clu%terware will &ave a 1A> -1a%t
A00lication >otification. Event 4 node* down
A uni< %&ell %cri0t -Call out %cri0t. will analyLe t&e
information and %end a re)ue%t to t&e !B" F"C.
o Re)ue%t to remove re%ource% from
GE$GDEN LPARP% from %econd and t&ird
node.
o Re)ue%t to a%%i(n new re%ource% to APP$
and RAC LPARP%.
- $ -
GFE>' t&e F"C will e<ecute t&e re)ue%t%' and will
re%iLe tle LPARP% a% re)ue%ted on eac& available
node%.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$ o# $%$
For more in#ormation 2
On &tt04otn.oracle.com
&tt04www.oracle.comtec&nolo(y0roduct%databa%eclu%terwareinde<.&tml
Oracle Cluster,are
Oracle Clu%terware i% a 0ortable clu%ter %oftware t&at allow% clu%terin( of %in(le %erver% %o t&at t&ey coo0erate a% a
%in(le %y%tem. Oracle Clu%terware al%o 0rovide% t&e re)uired infra%tructure for Oracle Real A00lication Clu%ter% -RAC..
!n addition Oracle Clu%terware enable% t&e 0rotection of any Oracle a00lication or any ot&er ,ind of a00lication wit&in a
clu%ter. !n any ca%e Oracle Clu%terware i% t&e intelli(ence in t&o%e %y%tem% t&at en%ure% t&e re)uired coo0eration
between t&e clu%ter node%.
Oracle Clu%terware
Oracle Clu%terware W&ite0a0er -PD1.
Oracle Clu%terware Gec&nical Article%
B%in( %tandard >1$ to %u00ort a t&ird votin( di%, on an E<tended Di%tance
clu%ter confi(uration on Linu<' A!+' FP-B+' or $olari% -PD1. 1ebruary 5CCI
Oracle Fome% in an Oracle Real A00lication Clu%ter% Environment -PD1. 1ebruary 5CCI
B%in( Oracle Clu%terware to 0rotect any ,ind of a00lication
B%in( Oracle Clu%terware to Protect Jrd Party A00lication% -PD1. 1ebruary 5CCI
B%in( Oracle Clu%terware to Protect Jrd Party A00lication% -PD1. 1ebruary 5CCI
B%in( Oracle Clu%terware to Protect Oracle A00lication $erver -PD1. >ovember 5CC@
B%in( Oracle Clu%terware to Protect an Oracle Databa%e *C( wit&
Oracle Enter0ri%e "ana(er 2rid Control !nte(ration -PD1. 1ebruary 5CCI
B%in( Oracle Clu%terware to Protect A $in(le !n%tance Oracle Databa%e **( -PD1. 1ebruary 5CCI
Oracle a00lication% 0rotected by Oracle Clu%terware
$iebel CR" A00lication% 0rotected by Oracle Clu%terware #anuary 5CCI
Pre-confi(ured a(ent% for Oracle Clu%terware
Providin( Fi(& Availability for $AP Re%ource% -PD1. "arc& 5CCE
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B o# $%$
$ !BO4T O!C:; !4TOM!TIC STO!=; M!E!=;M;ET
Wit& Oracle RAC' eac& in%tance mu%t &ave acce%% to t&e datafile% and recovery file% for t&e Oracle RAC databa%e.
B%in( Automatic $tora(e "ana(ement -A$". i% an ea%y way to %ati%fy t&i% re)uirement.
A$" i% an inte(rated' &i(&-0erformance databa%e file %y%tem and di%, mana(er. A$" i% ba%ed on t&e 0rinci0le t&at t&e
databa%e %&ould mana(e %tora(e in%tead of re)uirin( an admini%trator to do it. A$" eliminate% t&e need for you to
directly mana(e 0otentially t&ou%and% of Oracle databa%e file%.
A$" (rou0% t&e di%,% in your %tora(e %y%tem into one or more di%, (rou0%. Mou mana(e a %mall %et of di%, (rou0% and
A$" automate% t&e 0lacement of t&e databa%e file% wit&in t&o%e di%, (rou0%.
A$" 0rovide% t&e followin( benefit%4
Stri(ingQA$" %0read% data evenly acro%% all di%,% in a di%, (rou0 to o0timiLe 0erformance and utiliLation.
G&i% even di%tribution of databa%e file% eliminate% t&e need for re(ular monitorin( and !O 0erformance tunin(.
MirroringQA$" can increa%e data availability by o0tionally mirrorin( any file. A$" mirror% at t&e file level'
unli,e o0eratin( %y%tem mirrorin(' w&ic& mirror% at t&e di%, level. "irrorin( mean% ,ee0in( redundant co0ie%'
or mirrored co0ie%' of eac& e<tent of t&e file' to &el0 avoid data lo%% cau%ed by di%, failure%. G&e mirrored co0y
of eac& file e<tent i% alway% ,e0t on a different di%, from t&e ori(inal co0y. !f a di%, fail%' A$" can continue to
acce%% affected file% by acce%%in( mirrored co0ie% on t&e %urvivin( di%,% in t&e di%, (rou0.
Online storage recon#iguration and dynamic re&alancingQA$" 0ermit% you to add or remove di%,% from
your di%, %tora(e %y%tem w&ile t&e databa%e i% o0eratin(. W&en you add a di%, to a di%, (rou0' A$"
automatically redi%tribute% t&e data %o t&at it i% evenly %0read acro%% all di%,% in t&e di%, (rou0' includin( t&e
new di%,. G&e 0roce%% of redi%tributin( data %o t&at it i% al%o %0read acro%% t&e newly added di%,% i% ,nown a%
rebalancin(. !t i% done in t&e bac,(round and wit& minimal im0act to databa%e 0erformance.
Managed #ile creation and deletionQA$" furt&er reduce% admini%tration ta%,% by enablin( file% %tored in
A$" di%, (rou0% to be mana(ed by Oracle Databa%e. A$" automatically a%%i(n% file name% w&en file% are
created' and automatically delete% file% w&en t&ey are no lon(er needed by t&e databa%e.
A$" i% im0lemented a% a %0ecial ,ind of Oracle in%tance' wit& it% own $y%tem 2lobal Area and bac,(round 0roce%%e%.
G&e A$" in%tance i% ti(&tly inte(rated wit& t&e databa%e in%tance. Every %erver runnin( one or more databa%e
in%tance% t&at u%e A$" for %tora(e &a% an A$" in%tance. !n an Oracle RAC environment' t&ere i% one A$" in%tance
for eac& node' and t&e A$" in%tance% communicate wit& eac& ot&er on a 0eer-to-0eer ba%i%. Only one A$" in%tance i%
re)uired for eac& node re(ardle%% of t&e number of databa%e in%tance% on t&e node.
Oracle recommend% t&at you u%e A$" for your databa%e file %tora(e' in%tead of raw device% or t&e o0eratin( %y%tem
file %y%tem. Fowever' databa%e% can &ave a mi<ture of A$" file% and non-A$" file%.
See !lso4
Oracle Database 2 Day DBA
Oracle Database Storage Administrator's Guide
!nd 2
5 Day DBA FG"L PD1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C o# $%$
&tt04www.oracle.comtec&nolo(y0roduct%databa%ea%minde<.&tml
!utomatic Storage Management
Automatic $tora(e "ana(ement -A$". i% a feature in Oracle Databa%e *C(**( t&at 0rovide% t&e databa%e
admini%trator wit& a %im0le %tora(e mana(ement interface t&at i% con%i%tent acro%% all %erver and %tora(e 0latform%. A%
a vertically inte(rated file %y%tem and volume mana(er' 0ur0o%e-built for Oracle databa%e file%' A$" 0rovide% t&e
0erformance of a%ync !O wit& t&e ea%y mana(ement of a file %y%tem. A$" 0rovide% ca0ability t&at %ave% t&e DBA%
time and 0rovide% fle<ibility to mana(e a dynamic databa%e environment wit& increa%ed efficiency.
W&atR% >ew for A$" in Oracle Databa%e **(
11g !SM Ee, Features Technical 5hite Pa(er
2racle 8atabase 11% ASM ne" )eatures tec*nical "*ite paper
A$" Overview and Gec&nical Pa0er%
A$" and "ulti0at&in( Be%t Practice% and !nformation "atri< -PD1.
"ulti0at&in( %oftware and A$" information matri< and be%t 0ractice% (uide
Oracle Databa%e *C g Relea%e 5 A$" - >ew 1eature% W&ite Pa0er -PD1.
G&i% 0a0er di%cu%%e% t&e en&ancement% to A$" w&ic& are new in Relea%e 5 of Oracle Databa%e *Cg.
A$" Overview and Gec&nical Be%t Practice% W&ite Pa0er -PD1.
G&i% 0a0er di%cu%%e% t&e ba%ic% of A$" %uc& a% t&e %te0% to add di%,%' create a di%,(rou0' and create
a databa%e wit&in A$"' em0&a%iLin( be%t 0ractice%.
Ga,e t&e 2ue%%wor, Out of Databa%e !O Gunin( W&ite Pa0er -PD1.
Oracle Databa%e layout and %tora(e confi(uration% do not &ave to be com0licated any more. Learn
more about be%t 0ractice% wit& A$" t&at reduce com0le<ity and %im0lify %tora(e mana(ement.
Databa%e $tora(e Con%olidation wit& A$" W&ite Pa0er -PD1.
Databa%e %tora(e con%olidation em0owered by A$" for RAC and %in(le in%tance databa%e
environment%
Automatic $tora(e "ana(ement Overview -PD1.
G&i% %&ort 0a0er i% a %ummary of current %tora(e c&allen(e% and &ow A$" addre%%e% t&em in Oracle
Databa%e *Cg.
Automatic $tora(e "ana(ement W&ite Pa0er -PD1.
G&i% tec&nical overview de%cribe% t&e A$" arc&itecture and it% benefit%.
"i(ration to A$" wit& "inimal Down Gime' Gec&nical W&ite Pa0er -PD1.
"i(rate your databa%e to A$" u%in( Oracle Data 2uard and R"A>
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G o# $%$
B !BO4T ;!: !PP:IC!TIOE C:4ST;
Oracle RAC e<tend% Oracle Databa%e %o t&at you can %tore' u0date' and efficiently retrieve data u%in( multi0le
databa%e in%tance% on different %erver% at t&e %ame time. Oracle RAC 0rovide% t&e %oftware t&at facilitate% %erver%
wor,in( to(et&er in w&at i% called a clu%ter. G&e data file% t&at ma,e u0 t&e databa%e mu%t re%ide on %&ared %tora(e
t&at i% acce%%ible from all %erver% t&at are 0art of t&e clu%ter. Eac& %erver in t&e clu%ter run% t&e Oracle RAC %oftware.
An Oracle Databa%e databa%e &a% a one-to-one relation%&i0 between datafile% and t&e in%tance. An Oracle RAC
databa%e' &owever' &a% a one-to-many relation%&i0 between datafile% and in%tance%. !n an Oracle RAC databa%e'
multi0le in%tance% acce%% a %in(le %et of databa%e file%. G&e in%tance% can be on different %erver%' referred to a% &o%t%
or node%. G&e combined 0roce%%in( 0ower of t&e multi0le %erver% 0rovide% (reater availability' t&rou(&0ut' and
%calability t&an i% available from a %in(le %erver.
Eac& databa%e in%tance in an Oracle RAC databa%e u%e% it% own memory %tructure% and bac,(round 0roce%%e%.
Oracle RAC u%e% Cac&e 1u%ion to %ync&roniLe t&e data %tored in t&e buffer cac&e of eac& databa%e in%tance. Cac&e
1u%ion move% current data bloc,% -w&ic& re%ide in memory. between databa%e in%tance%' rat&er t&an &avin( one
databa%e in%tance write t&e data bloc,% to di%, and re)uirin( anot&er databa%e in%tance to reread t&e data bloc,% from
di%,. W&en a data bloc, located in t&e buffer cac&e of one in%tance i% re)uired by anot&er in%tance' Cac&e 1u%ion
tran%fer% t&e data bloc, directly between t&e in%tance% u%in( t&e interconnect' enablin( t&e Oracle RAC databa%e to
acce%% and modify data a% if t&e data re%ided in a %in(le buffer cac&e.
Oracle RAC i% al%o a ,ey com0onent for im0lementin( t&e Oracle enter0ri%e (rid com0utin( arc&itecture. Favin( ulti0le
databa%e in%tance% acce%%in( a %in(le %et of datafile% 0revent% t&e %erver from bein( a %in(le 0oint of failure. Any
0ac,a(ed or cu%tom a00lication t&at ran well on a Oracle Databa%e will 0erform well on Oracle RAC wit&out re)uirin(
code c&an(e%.
Mou will learn more about t&e o0eration of t&e Oracle RAC databa%e in a clu%ter' &ow to build t&e clu%ter' and t&e
%tructure of an Oracle RAC databa%e in ot&er %ection% of t&i% (uide.
See !lso4
Oracle Real Application Clusters Administration and Deployment Guide
!nd 2
5 Day DBA FG"L PD1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K o# $%$
&tt04www.oracle.comtec&nolo(y0roduct%databa%eclu%terin(inde<.&tml
Oracle eal !((lication Clusters
Oracle Real A00lication Clu%ter% -RAC. i% an o0tion to t&e award-winnin( Oracle Databa%e Enter0ri%e Edition. Oracle
RAC i% a clu%ter databa%e wit& a %&ared cac&e arc&itecture t&at overcome% t&e limitation% of traditional %&ared-not&in(
and %&ared-di%, a00roac&e% to 0rovide &i(&ly %calable and available databa%e %olution% for all your bu%ine%%
a00lication%.
Oracle Databa%e $tandard Edition include% Real A00lication Clu%ter% %u00ort for &i(&er level% of %y%tem u0time.
Real A00lication Clu%ter% **g
Oracle Real A00lication Clu%ter% Data%&eet -PD1. #uly 5CCE
Oracle Real A00lication Clu%ter% ** g Gec&nical Overview-PD1. #uly 5CCE
Wor,load "ana(ement wit& Oracle Real A00lication Clu%ter% -1A>' 1C1' Load Balancin(.-PD1. #uly
5CCE
E;5 Oracle Fome% in an Oracle Real A00lication Clu%ter% Environment-PD1. #anuary 5CCI
4P1!T;1 B%in( %tandard >1$ to %u00ort a t&ird votin( di%, on an E<tended Di%tance clu%ter
confi(uration on Linu<'A!+' FP' or $olari% -PD1. 1ebruary 5CCI
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18 o# $%$
C !BO4T 11= !C / !SM OE !I"
11g !C ,ith OC *Oracle Cluster egistry+ dis.*s+/ and 3oting *Deart&eat+ dis.*s+ on ra, dis.s/ and data&ase
on Oracle !SM *!utomated Storage Management+)
Oracle !utomated Storage Management *!SM+ Solution 2
O!C:; C:4ST;5!; M!E1!TOI 000
>o need for FAC"P
>o need for 2P1$
Oracle Clu%terware file% -OCR and Notin(. are 0laced on raw di%,%.
Only Oracle databa%e% file% -datafile%' redo lo(%' arc&ive lo(%' fla%& recovery area' S. are %tored on t&e di%,%
mana(ed by Oracle A$". >o binarie%.
A$" 0rovided wit& oracle %oftware
IBM VIOS Virtual SCSI for ASM data storage and associated raw hdisk based Voting and
OCR (NE A!ril "##$%&
IBM VIOS Virtual 'AN for all !ublic network and !ri(ate network interconnect for su!!orted
data storage o!tions (NE A!ril "##$%&
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1% o# $%$
G 5D!T-S E;5 5ITD 11= !C IMP:;M;ET!TIOE OE !I"
With Oracle 1 1g RAC on IBM POWER System p i running AI! "
D!CMP is not any more necessary as cluster,are so#t,are since
16g!C elease 1)
Oracle (ro>ide ,ith 11 g !C its o,n cluster,are / named as Oracle
Cluster,are/ or CS *Oracle Cluster eady Ser>ice+)
Oracle Cluster,are can coha&itate ,ith D!CMP under some
conditions)
5ith 11g!C/ Oracle Cluster,are is mandatory as cluster,are/ e>en
so other >endors cluster,are so#t,are are installed on the same
systems) Iou M4ST chec. that third (arty cluster,are can coha&itate
,ith Oracle cluster,are)
Subject:
Doc ID:
5ith Oracle Cluster,are 11g elease 1 2
o Oracle 3IP *3irtual IP+ are no, con#igured at the end o#
the Cluster,are installation *since 16g!C elease 7+)
5ith Oracle eal !((lication Cluster 11g elease 1 2
o Oracle Real Application Clusters 11 g Technical Overview(PDF) July !!"
5ith !SM 11g elease 1 2
o !SM so#t,are can &e/ and should &e installed in its o,n
O!C:;NDOM; *!SMNDOM;+ directory ,hich ,ill &e
di##erent than the data&ase so#t,are O!C:;NDOM;)
Oracle #o$es in an Oracle Real Application Clusters %nviron$ent(PDF) January !!&
o Ee, #eatures as ?Fast Mirror esync@ and ?Pre#erred
Mirror ead@/ ?Fast e&alance@)
o Ee, !SMCM1 commands as ?lsds.@/ ?mdN&ac.u(/
?mdNrestore@@/ ?rema(@ and ?c(@
o ;tc M Chec. ))g ASM New *eatures +echnical hite ,a!er
(Oracle Data'ase 11( AS) new *eatures technical white paper)
K IEF!ST4CT4; ;O4I;M;ETS
11g!C/!SM/!I"
oraclibm@fr.ibm.com
76 o# $%$
Oracle Real A00lication Clu%ter intend to 0rovide 4
Fi(& Availability of u%er% %ervice% to maintain continuity of t&e bu%ine%%
$cale-in wit&in eac& node' addin( re%ource% w&en 0o%%ible' a% your bu%ine%% (row
$cale out w&en %cale in i% not anymore 0o%%ible' re%0ondin( to your bu%ine%% (row
Wor,load balancin( and affinity
Etc S
Oracle eal !((lication (rotect high a>aila&ility o# your data&ases/ but all &ardware com0onent% of a clu%ter a% 4
$A> %tora(e attac&ement% from node% to $A>' includin( $A> %witc& and FBAP%
Clu%ter networ, interconnect for Oracle clu%terware networ, &eartbeat' and Oracle RAC cac&e fu%ion
mec&ani%m -Private >etwor,.' includin( networ, interface%
$tora(e
Etc S
mu%t be 0rotected OOO
11g!C/!SM/!I"
oraclibm@fr.ibm.com
71 o# $%$
K)1 =eneral ePuirements
G&i% c&a0ter will li%t t&e (eneral re)uirement% to loo, at for an Oracle RAC im0lementation on !B" %y%tem 0.
Go0ic% covered are 4
$erver% and 0roce%%or%
>etwor,
$tora(e
$tora(e attac&ement%
N!O $erver
7.1.1 About Servers and processors
11g!C/!SM/!I"
oraclibm@fr.ibm.com
77 o# $%$
7.1.2 About RAC on IBM System p
Oracle Real A00lication clu%ter could be im0lemented on LPARP%DLPARP% from $e0arated 0&y%ical !B" $y%tem 0
%erver% S
For (roduction or
testing to achie>e 2
Fi(& Availability
-0rotectin( lo%% of
a 0&y%ical %erver.
$calability -addin(
core% and memory'
or addin( new
%erver.
Databa%e
Wor,load affinity
acro%% %erver%
OR im0lemented between LPARP%' DLPARP% from a %ame 0&y%ical !B" $y%tem 0 %erver S
For (roduction i# high a>aila&ility is not the
#ocus/ &ut mostly a&out se(arating the
data&ase ,or.load)
Or #or testing/ or de>elo(ment (ur(ose)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7$ o# $%$
7.1.3 About Network
Protect Private networ, wit& Et&erc&anel !m0lementation' t&e followin( dia(ram %&ow t&e im0lementation wit& 5
%erver% 4
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7B o# $%$
1or PR!NAGE >etwor, -Clu%terRAC !nterconnect. OOO
=iga&it s,itch is mandatory #or (roduction
im(lementation/ e>en #or only 7 nodes architecture)
-Cro%%-over cable can be u%ed only for te%t 0ur0o%e' and itP% not %u00orted by Oracle $u00ort'
0lea%e read RAC 1AH on &tt04metalin,.oracle.com..
! second giga&it ethernet interconnect/ wit& a different networ, ma%,' can be %etu0 for %ecurity
0ur0o%e% or 0erformance i%%ue%.
Eet,or. cards #or (u&lic net,or. must ha>e same name on each (artici(ating node in the !C cluster
*For e'am(le en6 on all nodes+)
Eet,or. cards #or Interconnect Eet,or. *Pri>ate+ must ha>e same Eame on each (artici(ating Eode in the
!C cluster *For e'am(le en1 on all nodes+)
1 >irtual IP (er node must &e reser>ed/ and not used on the net,or. (rior to Oracle cluster,are
installation) 1on-t set IP allias at !I" le>el/ Oracle cluster,are ,ill ta.e charge o# it)
Wit& A!+ Et&erC&annel im0lemented wit& 5 >etwor, %witc&e%' wePll cover t&e lo%% of interconnect networ, card% -for
e<am0le en5 and enJ. and corre%0ondin( interconnect networ, %witc&.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7C o# $%$
7.1.4 About SAN Storage
5hen im(lementing !C/ you must &e carre#ull on the S!E storage to use) The S!E Storage must &e
ca(a&le thru it-s dri>ers o# read/,rite concurrency at same time #rom any mem&er o# the !C cluster/ ,hich
means that ?reser>eN(olicy@ attri&ute #rom dis.s *hdis./ hdis.(o,er/ dlm#dr>/ etc M+ disco>ered must &e a&le
to &e set to ?noNreser>e@ or ?noNloc.@ >alues)
For other storages a% E"C' FD$' FP and %o on' youPll &ave to c&ec, t&at t&e %tora(e to be %elected i% com0atible
and %u00orted wit& RAC. Oracle i% not %u00ortin( or certifyin( %tora(e' $tora(e Con%tructor% will %ay if t&ey %u00ort or
not' and wit& w&ic& "ulti-0at&in( tool -!B" $DDPC"' E"C PowerPat&' FDL"' etc..
Some documents to loo. at #or Oracle on IBM storage 2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7G o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7K o# $%$
7.1.5 Proposed infrastructure with 2 servers
E<am0le of infra%tructure a% it %&ould be wit& 5 %erver%' for networ, and %tora(e.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
78 o# $%$
7.1.6 What do we protect ?
Oracle RAC 0rotect a(ain%t node failure%' t&e infra%tructure %&ould cover a ma<imum of failure ca%e a% 4
1or t&e %tora(e
lo%% of * FBA on one node
lo%% of * $A> %witc&
1or t&e >etwor,
lo%% of * interconnect >etwor, card on one node
lo%% of * !nterconnect >etwor, %witc&

All com0onent% of t&e infra%tructure are 0rotected for &i(& availability a0art for t&e %tora(e' if t&e full %tora(e i% lo%t.
G&e only way to e<tend t&e &i(& availability to t&e %tora(e level i% to introduce a %econd %tora(e and im0lement RAC
a% a %tretc&ed or e<tended clu%ter.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7% o# $%$
7.1.7 About IBM Advanced Power Virtualization and RAC
E<tract from 3Certi#y 2 !C #or 4ni' On IBM !I" &ased Systems *!C only+T -5EC55CCI' c&ec, for la%t u0date.
&tt0%4metalin,.oracle.commetalin,0l%)lf809*AC4*4*5C5CJEAE/E*@JD55EIC44444
Oracle 0roduct% are certified for A!+@L on all %erver% t&at !B" %u00ort% wit& A!+@L. G&i% include% !B" $y%tem i
and $y%tem 0 model% t&at u%e POWER@ and POWER/ 0roce%%or%. Plea%e vi%it !B"R% web%ite at t&i% BRL for
more information on A!+@L %u00ort for $y%tem i detail%.
!B" 0ower ba%ed %y%tem% t&at %u00ort A!+@L include% mac&ine% branded a% R$/CCC' 0$erie%' i$erie%' $y%tem
0 and $y%tem i.
G&e minimum A!+ level% for POWER / ba%ed model% are A!+@L @.5 GL*C and A!+@L @.J GLC/
A!+@L certification% include A!+@L ver%ion% @.5 and @.J. @.* wa% de%u00orted on */-#B>-5CC/.
o Cu%tomer% %&ould review "etaLin, >ote 5I5CJ/.*
/A-bit &ardware i% re)uired to run Real A00lication Clu%ter%.
A!+ /A-bit ,ernel i% re)uired for *C(R5 RAC. A!+ J5-bit ,ernel and A!+ /A-bit ,ernel are %u00orted wit& D.5 and
*C(.
E<tract from 3RA! Technologies &atri5 for 8:'( /latforms> -**CA5CCI' c&ec, for la%t u0date.
&tt04www.oracle.comtec&nolo(y0roduct%databa%eclu%terin(certifytec&6(eneric6uni<6new.&tml
Plat#orm
Technology
Category
Technology Eotes
!B" A!+ Ser>er/Processor
!rchitecture
IBM System (/ i/
BladeCenter JS76/
JS71
!d>anced Po,er 3irtuali<ation
hard,are :P!Qs are
su((orted including 1ynamic
or shared (ool ,ith micro
(artitions
VIOS Virtual SCSI for ASM data
storage and associated raw hdisk
based Voting and OCR (NE
A!ril "##$%&
VIOS Virtual 'AN for all !ublic
network and !ri(ate network
interconnect for su!!orted data
storage o!tions (NE A!ril "##$%&

11g!C/!SM/!I"
oraclibm@fr.ibm.com
$6 o# $%$
7.1.7.1 Network and VIO Server
!m0lementin( Et&erc&annel lin,% t&ru N!O %erver% for 0ublic and 0rivate networ, between node% 4
E<am0le between 5 LPARP%' * for RAC node' and one for APP$ node.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$1 o# $%$
!m0lementin( Et&erc&annel and NLA> -Nirtual LA>. lin,% t&ru N!O %erver% for 0ublic and 0rivate networ,
between 5 RAC node% 4
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$7 o# $%$
7.1.7.2 Storage and VIO Server
1ollowin( im0lementation i% now %u00orted OOO
Certi#ication #or this architecture is done/ and it-s no, su((orted)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$$ o# $%$
Follo,ing im(lementation also su((orted 2
The hdis.s hosting the #ollo,ing com(onents are accessed thru 3IO ser>ers
A!+ o0eratin( %y%tem
Oracle clu%terware -UCR$6FO"E.
A$" %oftware -UA$"6FO"E.
RAC $oftware -UORACLE6FO"E.
Etc S
!nd OC/ 3oting and !SM dis.s are accessed thru direct attached DB!-s to the !C :P!-s)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$B o# $%$
K)7 Coo.&oo. in#rastructure
1or our infra%tructure' we u%ed a clu%ter w&ic& i% com0o%ed of t&ree 0artition% -!B" LPAR. on an !B" %y%tem
0 @EC u%in( A!+ @L.
BBG in t&e real world' to ac&ieve true &i(& availability itP% nece%%ary to &ave at lea%t two !B" $y%tem% 0 i %erver% a%
%&own bellow 4
;ach com(onent o# the in#rastructure must &e (rotected 2
Di%, acce%% 0at& -5 FBAP% and multi-0at&in( %oftware.
!nterconnect networ,
Public networ,
S
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$C o# $%$
7.2.1 IBM Sytem p servers
This is the IBM Sytem
(Ser>er ,e used #or
our installation M
&tt04www-CJ.ibm.com%erver%e%erver0%erie%&ardware&i(&end
&tt04www-CJ.ibm.com%y%tem%0
RRRRRR
GFE> youPll need * A!+@L LPAR on eac& %erver for real RAC im0lementation' wit& nece%%ary memory and Power /
CPB a%%i(ned to eac& LPAR.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$G o# $%$
Commands to (rint the con#ig #or IBM System ( on !I"C: *node1+ 2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$K o# $%$
Vnode14rootW ; (rtcon#
System Model2 IBM/%11K-MM!
"ac&ine $erial >umber4 /@*A5/C
Processor Ty(e2 Po,erPCNPO5;G
>umber Of Proce%%or%4 *
Proce%%or Cloc, $0eed4 J@CA "FL
CPB Gy0e4 /A-bit
Xernel Gy0e4 /A-bit
LPAR !nfo4 * #$C-**(-node*
"emory $iLe4 JCE5 "B
2ood "emory $iLe4 JCE5 "B
Platform 1irmware level4 E"J*C6CAI
1irmware Ner%ion4 !B"'E"J*C6CAI
Con%ole Lo(in4 enable
Auto Re%tart4 true
1ull Core4 fal%e
Eet,or. In#ormation
Fo%t >ame4 node*
!P Addre%%4 *C.J.5@.I*
$ub >etma%,4 5@@.5@@.5@@.C
2ateway4 *C.J.5@.5@A
>ame $erver4
Domain >ame4
Paging S(ace In#ormation
Gotal Pa(in( $0ace4 @*5"B
Percent B%ed4 EY
3olume =rou(s In#ormation
999999999999999999999999999999999999999999999999999999999999999999999999999999
rootv(4
PN6>A"E PN $GAGE GOGAL PP% 1REE PP% 1REE D!$GR!BBG!O>
&di%,C active 5EJ *@5 AA..CC..CC..@J..@@
999999999999999999999999999999999999999999999999999999999999999999999999999999
IEST!::;1 ;SO4C; :IST
G&e followin( re%ource% are in%talled on t&e mac&ine.
=- 9 Added or deleted from Re%ource Li%t.
Z 9 Dia(no%tic %u00ort not available.
"odel Arc&itecture4 c&r0
"odel !m0lementation4 "ulti0le Proce%%or' PC! bu%
= %y%C $y%tem ObKect
= %y%0lanarC $y%tem Planar
Z vioC Nirtual !O Bu%
Z v%c%iC BD**E.@EC./@DEDJE-N*-C@-G* Nirtual $C$! Client Ada0ter
Z &di%,C BD**E.@EC./@DEDJE-N*-C@-G*-LI*CCCCCCCCCC Nirtual $C$! Di%, Drive
Z v%aC BD**E.@EC./@DEDJE-N*-CC LPAR Nirtual $erial Ada0ter
Z vtyC BD**E.@EC./@DEDJE-N*-CC-LC A%ync&ronou% Germinal
Z ent5 BD**E.@EC./@DEDJE-N*-CA-G* Nirtual !O Et&ernet Ada0ter -l-lan.
Z ent* BD**E.@EC./@DEDJE-N*-CJ-G* Nirtual !O Et&ernet Ada0ter -l-lan.
Z entC BD**E.@EC./@DEDJE-N*-C5-G* Nirtual !O Et&ernet Ada0ter -l-lan.
Z 0ci* BEIED.CC*.DHD*E2+-P* PC! Bu%
Z 0ci/ BEIED.CC*.DHD*E2+-P* PC! Bu%
= fc%C BEIED.CC*.DHD*E2+-P*-C5-G* 1C Ada0ter
Z fcnetC BEIED.CC*.DHD*E2+-P*-C5-G* 1ibre C&annel >etwor, Protocol Device
Z f%c%iC BEIED.CC*.DHD*E2+-P*-C5-G* 1C $C$! !O Controller Protocol Device
Z dacC BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCICCACBI*5ABJ* *E55-/CC -/CC. Di%, Array Controller
Z dac* BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCDCCACBI*5ABJ* *E55-/CC -/CC. Di%, Array Controller
Z dac5 BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCECCACBIC1DA51 *E55-/CC -/CC. Di%, Array Controller
Z dacJ BEIED.CC*.DHD*E2+-P*-C5-G*-W5CC/CCACBIC1DA51 *E55-/CC -/CC. Di%, Array Controller
= L5cac&eC L5 Cac&e
= memC "emory
= 0rocC Proce%%or
= &di%,* BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCICCACBI*5ABJ*-LC *E55-/CC -/CC. Di%, Array Device
= &di%,5 BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCICCACBI*5ABJ*-L*CCCCCCCCCCCC *E55-/CC -/CC. Di%, Array Device
= &di%,J BEIED.CC*.DHD*E2+-P*-C5-G*-W5CCICCACBI*5ABJ*-L5CCCCCCCCCCCC *E55-/CC -/CC. Di%, Array Device
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$8 o# $%$
Commands to (rint the con#ig #or IBM System ( on !I"C: *node7+ 2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$% o# $%$
Vnode74rootW ; (rtcon#
System Model2 IBM/%11K-MM!
"ac&ine $erial >umber4 /@*A5/C
Processor Ty(e2 Po,erPCNPO5;G
>umber Of Proce%%or%4 *
Proce%%or Cloc, $0eed4 J@CA "FL
CPB Gy0e4 /A-bit
Xernel Gy0e4 /A-bit
LPAR !nfo4 5 #$C-**(-node5
"emory $iLe4 JCE5 "B
2ood "emory $iLe4 JCE5 "B
Platform 1irmware level4 E"J*C6CAI
1irmware Ner%ion4 !B"'E"J*C6CAI
Con%ole Lo(in4 enable
Auto Re%tart4 true
1ull Core4 fal%e
Eet,or. In#ormation
Fo%t >ame4 node5
!P Addre%%4 *C.J.5@.I5
$ub >etma%,4 5@@.5@@.5@@.C
2ateway4 *C.J.5@.5@A
>ame $erver4
Domain >ame4
Paging S(ace In#ormation
Gotal Pa(in( $0ace4 @*5"B
Percent B%ed4 5Y
3olume =rou(s In#ormation
999999999999999999999999999999999999999999999999999999999999999999999999999999
rootv(4
PN6>A"E PN $GAGE GOGAL PP% 1REE PP% 1REE D!$GR!BBG!O>
&di%,* active 5EJ *AD @A..5C..CC..5C..@@
999999999999999999999999999999999999999999999999999999999999999999999999999999
IEST!::;1 ;SO4C; :IST
G&e followin( re%ource% are in%talled on t&e mac&ine.
=- 9 Added or deleted from Re%ource Li%t.
Z 9 Dia(no%tic %u00ort not available.
"odel Arc&itecture4 c&r0
"odel !m0lementation4 "ulti0le Proce%%or' PC! bu%
= %y%C $y%tem ObKect
= %y%0lanarC $y%tem Planar
Z 0ciD BEIED.CC*.DHD*E2+-P* PC! Bu%
Z 0ci*C BEIED.CC*.DHD*E2+-P* PC! Bu%
= fc%C BEIED.CC*.DHD*E2+-P*-C@-G* 1C Ada0ter
Z fcnetC BEIED.CC*.DHD*E2+-P*-C@-G* 1ibre C&annel >etwor, Protocol Device
Z f%c%iC BEIED.CC*.DHD*E2+-P*-C@-G* 1C $C$! !O Controller Protocol Device
Z dacC BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCICCACBI*5ABJ* *E55-/CC -/CC. Di%, Array Controller
Z dac* BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCDCCACBI*5ABJ* *E55-/CC -/CC. Di%, Array Controller
Z dac5 BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCECCACBIC1DA51 *E55-/CC -/CC. Di%, Array Controller
Z dacJ BEIED.CC*.DHD*E2+-P*-C@-G*-W5CC/CCACBIC1DA51 *E55-/CC -/CC. Di%, Array Controller
Z vioC Nirtual !O Bu%
Z v%c%iC BD**E.@EC./@DEDJE-N5-C@-G* Nirtual $C$! Client Ada0ter
Z &di%,* BD**E.@EC./@DEDJE-N5-C@-G*-LI*CCCCCCCCCC Nirtual $C$! Di%, Drive
Z v%aC BD**E.@EC./@DEDJE-N5-CC LPAR Nirtual $erial Ada0ter
Z vtyC BD**E.@EC./@DEDJE-N5-CC-LC A%ync&ronou% Germinal
Z ent5 BD**E.@EC./@DEDJE-N5-CA-G* Nirtual !O Et&ernet Ada0ter -l-lan.
Z ent* BD**E.@EC./@DEDJE-N5-CJ-G* Nirtual !O Et&ernet Ada0ter -l-lan.
Z entC BD**E.@EC./@DEDJE-N5-C5-G* Nirtual !O Et&ernet Ada0ter -l-lan.
= L5cac&eC L5 Cac&e
= memC "emory
= 0rocC Proce%%or
= &di%,C BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCICCACBI*5ABJ*-LC *E55-/CC -/CC. Di%, Array Device
= &di%,5 BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCICCACBI*5ABJ*-L*CCCCCCCCCCCC *E55-/CC -/CC. Di%, Array Device
= &di%,J BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCICCACBI*5ABJ*-L5CCCCCCCCCCCC *E55-/CC -/CC. Di%, Array Device
= &di%,A BEIED.CC*.DHD*E2+-P*-C@-G*-W5CCICCACBI*5ABJ*-LJCCCCCCCCCCCC *E55-/CC -/CC. Di%, Array Device
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B6 o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B1 o# $%$
Command to get in#ormation on the :P! 2
{node1:root}/ # lparstat -i
Node Name : node1
Partition Name : JSC-11g-node1
Partition Number : 1
Type : Shared-SMT
Mode : Unapped
!ntit"ed Capaity : #$%#
Partition &roup-'( : )*+,-
Shared Poo" '( : #
.n"ine /irtua" CPU0 : 1
Ma1imum /irtua" CPU0 : 1
Minimum /irtua" CPU0 : 1
.n"ine Memory : )#+* M2
Ma1imum Memory : ,133 M2
Minimum Memory : 1#*3 M2
/ariab"e Capaity 4eight : 1*5
Minimum Capaity : #$1#
Ma1imum Capaity : 1$##
Capaity 'nrement : #$#1
Ma1imum Phy0ia" CPU0 in 0y0tem : 1,
6ti7e Phy0ia" CPU0 in 0y0tem : 1,
6ti7e CPU0 in Poo" : 1,
Shared Phy0ia" CPU0 in 0y0tem : -
Ma1imum Capaity o8 Poo" : -
!ntit"ed Capaity o8 Poo" : -
Una""oated Capaity : #$##
Phy0ia" CPU Perentage : %#$##9
Una""oated 4eight : #
{node1:root}/ #
{node2:root}/home # lparstat -i
Node Name : node*
Partition Name : JSC-11g-node*
Partition Number : *
Type : Shared-SMT
Mode : Unapped
!ntit"ed Capaity : #$%#
Partition &roup-'( : )*++#
Shared Poo" '( : #
.n"ine /irtua" CPU0 : 1
Ma1imum /irtua" CPU0 : 1
Minimum /irtua" CPU0 : 1
.n"ine Memory : )#+* M2
Ma1imum Memory : ,133 M2
Minimum Memory : 1#*3 M2
/ariab"e Capaity 4eight : 1*5
Minimum Capaity : #$1#
Ma1imum Capaity : 1$##
Capaity 'nrement : #$#1
Ma1imum Phy0ia" CPU0 in 0y0tem : 1,
6ti7e Phy0ia" CPU0 in 0y0tem : 1,
6ti7e CPU0 in Poo" : 1,
Shared Phy0ia" CPU0 in 0y0tem : -
Ma1imum Capaity o8 Poo" : -
!ntit"ed Capaity o8 Poo" : -
Una""oated Capaity : #$##
Phy0ia" CPU Perentage : %#$##9
Una""oated 4eight : #
{node*:root}/home #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B7 o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B$ o# $%$
7.2.2 Operating System
O(erating system must &e installed the same ,ay on each :P!' wit& t&e %ame maintenance level' %ame APAR
and 1!LE$EG$ level.
Chec. ?P;P!IE= TD; SIST;M@ cha(ter #or O(erating System rePuirements on !I"C:
A!+@.* i% not %u00orted
!I"C)7 / C)$ are su((orted and certi#ied
The IBM !I" clustering layer/ D!CMP #ilesets/ M4ST EOT &e installed i#
you->e chosen an im(lementation ,ithout D!CMP) !f t&i% layer i% im0lemented for
ot&er 0ur0o%e' di%,% re%%ource% nece%%ary to in%tall and run CR$ data will &ave to be 0art of an
FAC"P volume (rou0 re%ource.
I# you ha>e (re>iously installed D!CMP/ you must remo>e 2
FAC"P file%et% -clu%ter.e%.Z.
r%ct.&acm0.rte
r%ct.com0at.ba%ic.&acm0.rte
r%ct.com0at.client%.&acm0.rte
I# you did run a #irst installation o# the Oracle Cluster,are *CS+ ,ith D!CMP
installed/
!hec= if $ot$OR!?cluster director0 does e5ist and if so; remo4e it on
all nodes.
T.9: R97OOT A?? :O39% ...
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BB o# $%$
7.2.3 Multi-pathing and ASM
Please chec. Metalin. note ?Oracle !SM and Multi-Pathing Technologies@ Doc !D4 Eote27%B8G%)1
>ote' t&at Oracle Cor0oration doe% not certify A$" a(ain%t multi0at&in( utilitie%. G&e "P utilitie% li%ted below are one%
t&at ,nown wor,in( %olution%. A% we do more te%tin(' additional "P utilitie% will be li%ted &ere' t&u%' t&i% document i% an
active document.
"ulti-0at&in( allow $A> acce%% failover' and load balancin( accro% $A> 1iber C&annel attac&ement%.
O%
/latfor
m
&ulti@athing
tool
A%& 3e4ice
8sage
:otes
A!+ ;MC Po,erPath B%e raw 0artition%
t&ru t&e 0%eudo
device
devr&di%,0ower<
IBM S11 *3(ath+
EOT
S4PPOT;1 #or
!C/!SM on
!I" 000
devv0at&< A% of t&i% writin(' $DD-A!+ i% ,nown to cau%e di%covery and
device &andlin( 0roblem% for A$"' and t&u% i% not viable
%olution.
A$" need% to acce%% di%,%v0at& t&ru non root u%er' w&ic& i% not
allowed by $DD a% for today 5E "arc& 5CCE.
$ee $DDPC" %ection below for an alternative %olution to $DD
for A!+
IBM S11PCM B%e devr&di%,<
device
Mou mu%t in%tall $DDPC" file%et% and enable $DDPC"..
$DDPC" cannot co-e<i%t w $DD.
$DDPC" only wor,% wit& t&e followin( !B" %tora(e com0onent%4
D$ICCC'D$/CCC'Enter0ri%e $tora(e $erver -E$$.
$DDPC" wor,% al%o on to0 of !B" $NC -$A> Nolume
Controler.' and on to0 of ot&er %u00orted %tora(e% li,e FD$'
E"C' etc S
IBM 1!C
*edundant 1is.
!rray Controller+
B%e devr&di%,<
device
RDAC i% in%talled by default and mu%t be u%ed wit& !B" %tora(e
D$ACCC' and former 1a%Gt %erie%.
Ditachi 1ynamic
:in. Manager -
D1:M
B%e
devrd%,c<tydL
t&atP% (enerated by
FDL"
Or devdlmfdrv<
FDL" (enerate% a %c%i -c<tydL<. addre%% w&ere t&e controller i%
&i(&e%t unu%ed controller number.
FDL" no lon(er re)uire% FAC"P.
devdlmfdrv< can be u%ed out of FDL" Nolume 2rou0
Or if u%in( FDL" Nolume 2rou0' lo(ical volume% mu%t be
created wit& 3m,lvT command u%in( 3-G OT o0tion%.
Fu9itsu ;T;E4S
= Multi(ath
1ri>er
B%e devr&di%,
device
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BC o# $%$
7.2.4 IBM storage and multi-pathing
Wit& !B"' 0lea%e refer to !B" to confirm w&ic& !B" %tora(e i% %u00orted wit& RAC' if not %0ecified in our document.
IBM TotalStorage (roducts #or IBM Sytem (
IBM 1SB666/ 1SG666 and 1S8666 series are su((orted ,ith 16g!C)
!B" $tora(e D$JCC and D$ACC are not' and will not be %u00orted wit& *C(RAC.
A% for today "arc& 5E' 5CCE !B" $tora(e D$J5CC and D$JACC are not yet
%u00orted wit& *C(RAC.
IBM System Storage and TotalStorage (roducts
&tt04www-CJ.ibm.com%erver%%tora(e0roduct0roduct%60%erie%.&tml
RRRRRR
G&ere are 5 ca%e% w&en u%in( !B" %tora(e 4
!B" "P!O -"ulti-Pat& !O..
"P!O driver i% %u00orted wit& !B" Gotal $tora(e E$$' D$/CCC and D$ICCC %erie% only
And wit& !B" $NC -$A> Nolume Controler..
!B" RDAC -Redundant Di%, Array Controller. for !B" Gotal $tora(e D$ACCC.
RDAC driver i% %u00orted wit& !B" Gotal $tora(e D$ACCC %erie% only' and former 1a%Gt.
Iou M4ST use one or the other/ de(ending on the storage used)
case 12 :un-s (ro>ided &y the IBM storage ,ith IBM MPIO installed as multi-(athing dri>er)
Di%,% -LB>P%. will be %een a% &di%, at A!+ level u%in( l%0v command.
On node 1 M
{node1:root}/ # lspv
hdi0:# ##ed**8+-#-588 root7g ati7e
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
;
case 72 :un-s (ro>ided &y the IBM 1SB666 storage ,ith IBM 1!C installed as multi-(athing
dri>er)
On node 1 M
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
;
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BG o# $%$
7.2.4.1 IBM MPIO (Multi Path I/O) Setup Procedure
!I" Pac.ages needed to install
on all nodes 2
de7ie0$0ddpm$%)$*$1$#$+$b88
de7ie0$0ddpm$%)$rte
de7ie0$8p$di0:$ibm$mpio$rte
de>ices)#c()dis.)i&m)m(io)rte do,nload (age 2
&tt04www-
*.ibm.com%u00ortdocview.w%%8r%9@AC:conte<t9$G@52E:dc9DA*C:)*9&o%t=%cri0t%:uid9%%(*$ACCC5CJ:loc9en6B$:c%9utf-
I:lan(9en
MPIO #or !I" C)$ do,nload (age 2
&tt04www-*.ibm.com%u00ortdocview.w%%8uid9%%(*$ACCC5C*
On node 1
and node # $
Installing
the #ilesets 2
smitty install
'n0ta"" and Update So8t<are
'n0ta"" So8t<are
= 'NPUT de7ie / diretory 8or 0o8t<are
>/mydir?<ith?my?8i"e0et0@
S.AT46B! to in0ta"" >@
Pre00 A3
Se"et de7ie0$8p$di0:$ibm$mpio
'n0ta"" So8t<are
Type or 0e"et 7a"ue0 in entry 8ie"d0$
Pre00 !nter 6AT!B ma:ing a"" de0ired hange0$
>!ntry Aie"d0@
= 'NPUT de7ie / diretory 8or 0o8t<are $
= S.AT46B! to in0ta""
>devices.fcp.disk.ibm$C D
PB!/'!4 on"yE Fin0ta"" operation <i"" N.T ourG no
D
C.MM'T 0o8t<are update0E ye0
D
S6/! rep"aed 8i"e0E no
D
6UT.M6T'C6HHI in0ta"" reJui0ite 0o8t<areE ye0
D
!KT!N( 8i"e 0y0tem0 i8 0pae neededE ye0
D
./!B4B'T! 0ame or ne<er 7er0ion0E no
D
/!B'AI in0ta"" and he: 8i"e 0iLe0E no
D
'n"ude orre0ponding H6N&U6&! 8i"e0et0E ye0
D
(!T6'H!( outputE no
D
Proe00 mu"tip"e 7o"ume0E ye0
D
6CC!PT ne< "ien0e agreement0E no
D
Pre7ie< ne< H'C!NS! agreement0E no
D
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BK o# $%$
Chec. the insatllation
succed and the
installation summary
message 2
Installation Summary
--------------------
Name He7e" Part !7ent Be0u"t
-------------------------------------------------------------------------------
de7ie0$8p$di0:$ibm$mpio$r 1$#$#$# USB 6PPHI SUCC!SS
Install devices.sddpcm.53
Se"et :
M de7ie0$0ddpm$%)
6HH M
M D *$1$#$# '2M S(( PCM 8or 6'K /%)
M
M D *$1$#$+ '2M S(( PCM 8or 6'K /%)
M
Install Software
Type or 0e"et 7a"ue0 in entry 8ie"d0$
Pre00 !nter 6AT!B ma:ing a"" de0ired hange0$
= 'NPUT de7ie / diretory 8or 0o8t<are $
= S.AT46B! to in0ta"" >de7ie0$0ddpm$%)
C D
PB!/'!4 on"yE Fin0ta"" operation <i"" N.T ourG no
D
C.MM'T 0o8t<are update0E ye0
D
S6/! rep"aed 8i"e0E no
D
6UT.M6T'C6HHI in0ta"" reJui0ite 0o8t<areE ye0
D
!KT!N( 8i"e 0y0tem0 i8 0pae neededE ye0
D
./!B4B'T! 0ame or ne<er 7er0ion0E no
D
/!B'AI in0ta"" and he: 8i"e 0iLe0E no
D
'n"ude orre0ponding H6N&U6&! 8i"e0et0E ye0
D
(!T6'H!( outputE no
D
Proe00 mu"tip"e 7o"ume0E ye0
D
6CC!PT ne< "ien0e agreement0E no
D
Pre7ie< ne< H'C!NS! agreement0E no
D
Installation Summary
--------------------
Name He7e" Part !7ent Be0u"t
-------------------------------------------------------------------------------
de7ie0$0ddpm$%)$rte *$1$#$# USB 6PPHI SUCC!SS
de7ie0$0ddpm$%)$rte *$1$#$# B..T 6PPHI SUCC!SS
de7ie0$0ddpm$%)$rte *$1$#$+ USB 6PPHI SUCC!SS
de7ie0$0ddpm$%)$rte *$1$#$+ B..T 6PPHI SUCC!SS
de7ie0$0ddpm$%)$rte *$1$#$+ USB C.MM'T SUCC!SS
de7ie0$0ddpm$%)$rte *$1$#$+ B..T C.MM'T SUCC!SS
Eo, you need to re&oot all !I" nodes 000
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B8 o# $%$
Commands
to know AI !!"#
{node1:root}/ # pcmpat$ %uery wwpn
6dapter Name Port44N
800i# 1#######C-)%6+!+
800i1 1#######C-)612A)
Commands
to chec. dis.s 2
{node1:root}/ # lsdev -Cc disk -t 21&'
hdi0:* 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:) 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:3 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:% 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:, 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:+ 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:5 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:- 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:1# 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
hdi0:11 67ai"ab"e #6-#5-#* '2M MP'. AC *1#+
Commands
to chec. dis.s 2
{node1:root}/ # pcmpat$ %uery device
(!/#: * (!/'C! N6M!: hdi0:* TIP!: *1#+-## 6H&.B'TNM: Hoad 2a"ane
S!B'6H: +%*+151*###
OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
Path# 6dapter/Path Name State Mode Se"et !rror0
# 800i#/path# CH.S! N.BM6H # #
1 800i1/path1 CH.S! N.BM6H # #
(!/#: ) (!/'C! N6M!: hdi0:) TIP!: *1#+-## 6H&.B'TNM: Hoad 2a"ane
S!B'6H: +%*+151*##1
(!/#: 3 (!/'C! N6M!: hdi0:3 TIP!: *1#+-## 6H&.B'TNM: Hoad 2a"ane
S!B'6H: +%*+151*##*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B% o# $%$
7.2.4.2 IBM AIX RDAC (FCP.ARRAY filesets) Setup Procedure
This OE:I a((ly to use o# o# 1SB666 storage series) EOT to 1SG666/ 1S8666 and ;S866)
1!C is installed &y de#ault on !I"C:)
Eac& node mu%t &ave 5 FBA card%' for multi-0at&in(. Wit& O>LM * FBA 0er node' it will wor,% but 0at& to $A> will not
be 0rotected. GFE> in 0roduction' 5 FBA 0er node mu%t be u%ed.
!ll !I" hosts in your storage su&system must ha>e the 1!C multi(ath dri>er installed)
!n a %in(le %erver environment' A!+ allow% load %&arin( -al%o called load balancing). Mou can %et t&e load balancin(
0arameter to ye%. !n ca%e of &eavy wor,load on one 0at& t&e driver will move ot&er LB>% to t&e controller wit& le%%
wor,load and' if t&e wor,load reduce% bac, to t&e 0referred controller. Problem t&at can occur i% di%, t&ra%&in(. G&at
mean% t&at t&e driver move% t&e LB> bac, and fort& from one controller to t&e ot&er. A% a re%ult t&e controller i% more
occu0ied by movin( di%,% around t&an %ervicin( !O. G&e recommendation i% to >OG load balance on an A!+ %y%tem.
G&e 0erformance increa%e i% minimal -or 0erformance could actually (et wor%e..
1!C *#c()array #ilesets+ #or !I" su((ort round-ro&in load-&alancing
Setting the attri&utes o# the 1!C dri>er #or !I"
G&e A!+ RDAC driver file% are not included on t&e D$ACCC in%tallation CD.
Eit&er in%tall t&em from t&e A!+ O0eratin( $y%tem% CD' if t&e correct ver%ion i% included' or download t&em from t&e
followin( Web %ite4 &tt04tec&%u00ort.%ervice%.ibm.com%erverfi<e%
or &tt04www-JCA.ibm.comKctC*CCAc%y%tem%%u00ort
Commands
to chec. that
necessary #ilesets
are (resent #or
1!C M
{node1:root}/ # lslpp -( devices.fcp.disk.array.rte
Ai"e0et He7e" State Type (e0ription FUnin0ta""erG
----------------------------------------------------------------------------
de7ie0$8p$di0:$array$rte
%$)$#$%* 6 A AC SCS' B6'(iant 6rray (e7ie
Support So8t<are
State ode0:
6 -- 6pp"ied$
2 -- 2ro:en$
C -- Committed$
! -- !A'K Ho:ed$
. -- .b0o"ete$ Fpartia""y migrated to ne<er 7er0ionG
E -- 'non0i0tent State$$$Bun "pph: -7$
Type ode0:
A -- 'n0ta""p Ai"e0et
P -- Produt
C -- Component
T -- Aeature
B -- BPM Pa:age
{node1:root}/ #
{node1:root}/ # lslpp -( devices.common.I)*.fc.rte
Ai"e0et He7e" State Type (e0ription FUnin0ta""erG
----------------------------------------------------------------------------
de7ie0$ommon$'2M$8$rte
%$)$#$%# C A Common '2M AC So8t<are
State ode0:
6 -- 6pp"ied$
2 -- 2ro:en$
C -- Committed$
! -- !A'K Ho:ed$
. -- .b0o"ete$ Fpartia""y migrated to ne<er 7er0ionG
E -- 'non0i0tent State$$$Bun "pph: -7$
Type ode0:
A -- 'n0ta""p Ai"e0et
P -- Produt
C -- Component
T -- Aeature
B -- BPM Pa:age
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C6 o# $%$
Commands
to chec% the R&AC con'iguration and (BA path to hdis% $
On node1 $
{node1:root}/ # f+et,confi+ -v -A
---dar#---
U0er array name O P(S3###?JSCP
da# 6CT'/! da) 6CT'/!
(i0: (6C HUN Hogia" (ri7e
hdi0:1 da# # &5?0p8i"e
hdi0:* da) 1 &5?.CB1
hdi0:) da# * &5?.CB*
hdi0:3 da) ) &5?/ote1
hdi0:% da# 3 &5?/ote*
hdi0:, da) % &5?/ote)
hdi0:+ da# , &5?(ata1
hdi0:5 da) + &5?(ata*
hdi0:- da# 5 &5?(ata)
hdi0:1# da) - &5?(ata3
hdi0:11 da# 1# &5?(ata%
hdi0:1* da) 11 &5?(ata,
hdi0:1) da# 1* &5?tie
{node1:root}/ #
On node# $
{node2:root}/ # f+et,confi+ -v -A
---dar#---
U0er array name O P(S3###?JSCP
da# 6CT'/! da) 6CT'/!
(i0: (6C HUN Hogia" (ri7e
hdi0:# da# # &5?0p8i"e
hdi0:1 da) 1 &5?.CB1
hdi0:* da# * &5?.CB*
hdi0:) da) ) &5?/ote1
hdi0:3 da# 3 &5?/ote*
hdi0:% da) % &5?/ote)
hdi0:, da# , &5?(ata1
hdi0:+ da) + &5?(ata*
hdi0:5 da# 5 &5?(ata)
hdi0:- da) - &5?(ata3
hdi0:1# da# 1# &5?(ata%
hdi0:1* da) 11 &5?(ata,
hdi0:1) da# 1* &5?tie
{node*:root}/ #
Commands
to chec% the R&AC con'iguration and (BA path to hdis% 'or one speci'ic )dar* $
{node1:root}/ # f+et,confi+ -l dar&
da# 6CT'/! da) 6CT'/!
hdi0:1 da#
hdi0:* da)
hdi0:) da#
hdi0:3 da)
hdi0:% da#
hdi0:, da)
hdi0:+ da#
hdi0:5 da)
hdi0:- da#
hdi0:1# da)
hdi0:11 da#
hdi0:1* da)
hdi0:1) da#
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C1 o# $%$
Fcs6 is one o# the DB!)
Commands
To see the DB! #i&er
channel statistics 2
{node1:root}/ # fcstat fcs&
A'2B! CN6NN!H ST6T'ST'CS B!P.BT: 80#
(e7ie Type: AC 6dapter Fd81#5#8-G
Seria" Number: 1A31+#--*)
.ption B.M /er0ion: #*!#15+1
Airm<are /er0ion: N1(1$51K1
4or"d 4ide Node Name: #1*#######C-)A5!*-
4or"d 4ide Port Name: #11#######C-)A5!*-
AC-3 TIP!S:
Supported:
#1#####1*#########################################################
6ti7e:
#1#####1##########################################################
C"a00 o8 Ser7ie: )
Port Speed F0upportedG: * &2'T
Port Speed FrunningG: * &2'T
Port AC '(: #1,%#2##
Port Type: Aabri
Seond0 Sine Ha0t Be0et: *+-%
Tran0mit Stati0ti0 Beei7e Stati0ti0
------------------- ------------------
Arame0: 31,1% -,*#+
4ord0: 1%)+#*3 1*3-+3#5
H'P Count: #
N.S Count: #
!rror Arame0: #
(umped Arame0: #
Hin: Aai"ure Count: *,-
Ho00 o8 Syn Count: 3,-
Ho00 o8 Signa": 3,,
Primiti7e SeJ Protoo" !rror Count: #
'n7a"id T1 4ord Count: %1
'n7a"id CBC Count: #
'P o7er AC 6dapter (ri7er 'n8ormation
No (M6 Be0oure Count: #
No 6dapter !"ement0 Count: #
AC SCS' 6dapter (ri7er 'n8ormation
No (M6 Be0oure Count: #
No 6dapter !"ement0 Count: #
No Command Be0oure Count: #
'P o7er AC Tra88i Stati0ti0
'nput BeJue0t0: #
.utput BeJue0t0: #
Contro" BeJue0t0: #
'nput 2yte0: #
.utput 2yte0: #
AC SCS' Tra88i Stati0ti0
'nput BeJue0t0: *3+*1
.utput BeJue0t0: 5*#3
Contro" BeJue0t0: *%*
'nput 2yte0: 3,5133),
.utput 2yte0: 3*#+,1,
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C7 o# $%$
7.2.5 EMC storage and multi-pathing
Wit& E"C' 0lea%e refer to Fitac&i to %ee w&ic& E"C %tora(e i% %u00orted wit& RAC.
G&ere are 5 ca%e% w&en u%in( E"C %tora(e 4
case 12 :un-s (ro>ided &y the ;MC storage ,ith IBM MPIO installed as multi-(athing dri>er)
On node 1 M
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
;
G&en for di%,% to be u%ed for A$"' and on all node% 4
*. !n%tall "P!O on all node%' attac& t&e LB> to eac& node' dicover LB>P% wit& 3cf(m(r3.
5. !dentify &di%, name% on eac& node%' for a (iven LB> !D.
J. remove PN!D from &di%, and c&an(e t&e re%erve 0olicy to no re%erve u%in( 4
c&dev -l &di%,S -a 0v9clear
c&dev -l &di%,S -a re%erve60olicy9no6re%erve
A. %et ownerc&i0 to oracle4dba to t&e devr&di%,S
@. %et readwrite 0ermi%%ion% to //C to t&e devr&di%,S
/. acce%% t&e di%, t&ru devr&di%,S for A$" di%,(rou0 confi(uration
case 7 2 :un-s (ro>ided &y the ;MC storage ,ith ;MC Po,erPath installed as multi-(athing
dri>er)
Di%,% -LB>P%. will be %een a% &di%,0ower at A!+ level u%in( l%0v command.
On node 1 M
hdi0:po<er# ##ed**8+-#-588 root7g ati7e
hdi0:po<er1 none None
hdi0:po<er* none None
hdi0:po<er) none None
hdi0:po<er3 none None
;
*. !n%tall PowerPat& on all node%' attac& t&e LB> to eac& node' dicover LB>P% wit& 3cf(m(r3.
5. !dentify &di%,0ower name% on eac& node%' for a (iven LB> !D.
J. remove PN!D from &di%,0ower and c&an(e t&e re%erve 0olicy to no re%erve u%in( 4
c&dev -l &di%,0owerS -a 0v9clear
c&dev -l &di%,0owerS -a re%erve6loc,9no
A. %et ownerc&i0 to oracle4dba to t&e devr&di%,0owerS
@. %et readwrite 0ermi%%ion% to //C to t&e devr&di%,0owerS
/. acce%% t&e di%, t&ru devr&di%,0owerS for A$" di%,(rou0 confi(uration
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C$ o# $%$
7.2.5.1 EMC PowerPath Setup Procedure
$ee PowerPat& for A!+ ver%ion A.J !n%tallation : Admini%tration 2uide' P> JCC-CC*-/IJ for detail%
On node 1 and node # $
1) Install ;MC O1M dri>ers and necessary #ilesets M
@.5.C.* from ft04ft0.emc.com0ubelabai<OD"6DE1!>!G!O>$E"C.A!+.@.5.C.*.tar.[
in%tall u%in( %mit in%tall
5. remove any e<i%tin( device% attac&ed to t&e E"C
Vnode*4rootW ; rmdev ?dl &di%,+
J. run u%rl00E"C$ymmetri<binemc6cf(m(r to detect device%
B) Install Po,erPath >ersion B)$)6 minimum using smit install
@. re(i%ter PowerPat&
Vnode*4rootW ; emc0re( ?in%tall
/. initialiLe PowerPat& device%
Vnode*4rootW ; 0owermt confi(
K) >eri#y that all Po,erPath de>ices are named consistently across all cluster nodes
Vnode*4rootW ; u%rl00E"C$ymmetri<binin).ai</A \ (re0 &di%,0ower
com0are re%ult%. Con%i%tent namin( i% not re)uired for A$" device%' but LB>% u%ed
for t&e OCR and NOGE function% mu%t &ave t&e %ame device name% on all rac %y%tem%.
!dentify two %mall lun% to be u%ed for OCR and votin(
if t&e &di%,0ower+ name% for t&e OCR and NOGE device% are different' create a new
device for eac& of t&e%e function% a% follow%4
Vnode*4rootW ; m,nod devocr c ]maKor ; of OCR LB>^ ]minor ; of OCR LB>^
Vnode*4rootW ; m,nod devvote c ]maKor ; of NOGE LB>^ ]minor ; of NOGE LB>^
"aKor and minor number% can be %een u%in( t&e command _l% ?al dev&di%,0owerZP
I. On all &di%,0ower device% to be u%ed by Oracle for A$"' votin(' or t&e OCR' t&e re%erve6loc,
attribute mu%t be %et to `no`
Vnode*4rootW ; c&dev -l &di%,0ower+ -a re%erve6loc,9no
D. Nerify t&e attribute i% %et
Vnode*4rootW ; l%attr ?El &di%,0ower+
*C. $et 0ermi%%ion% on all &di%,0ower drive% to be u%ed for A$"' votin(' or t&e OCR a% follow% 4
Vnode*4rootW ; c&own oracle4dba devr&di%,0ower+
Vnode*4rootW ; c&mod //C devr&di%,0ower+
G&e Oracle !n%taller will c&an(e t&e%e 0ermi%%ion% and owner%&i0 a% nece%%ary durin( t&e CR$ in%tall
0roce%%.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CB o# $%$
7.2.6 HITACHI storage and multi-pathing
Wit& Fitac&i' 0lea%e refer to Fitac&i to %ee w&ic& FD$ %tora(e i% %u00orted wit& RAC.
G&ere are J ca%e% w&en u%in( Fitac&i FD$ %tora(e 4
case 12 :un-s (ro>ided &y the D1S storage ,ith IBM MPIO installed as multi-(athing dri>er)
SSS EOT S4PPOT;1 ,ith all D1S storage/ chec. ,ith Ditachi SSS
On node 1 M
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
;
*. !n%tall "P!O on all node%' attac& t&e LB> to eac& node' dicover LB>P% wit& 3cf(m(r3.
5. !dentify &di%, name% on eac& node%' for a (iven LB> !D.
J. remove PN!D from &di%, and c&an(e t&e re%erve 0olicy to no re%erve u%in( 4
c&dev -l &di%,S -a 0v9clear
c&dev -l &di%,S -a re%erve60olicy9no6re%erve
A. %et ownerc&i0 to oracle4dba to t&e devr&di%,
@. %et readwrite 0ermi%%ion% to //C to t&e devr&di%,
/. acce%% t&e di%, t&ru devr&di%, for A$" di%,(rou0 confi(uration
case 7 2 :un-s (ro>ided &y the D1S storage ,ith D1:M installed as multi-(athing dri>er)
SSS EOT S4PPOT;1 SSS
Release Notes 10g Release 2 (10.2) for AIX 5L Based Systems (64-Bit)
71),"4@,4 $ April 2006
Ditachi D1:M #or Storage
I# you use Ditachi D1:M *dml# de>ices+ #or storage/ then automatic storage
instances do not automatically identi#y the (hysical dis.) Instead/ the instances
identi#y only the logical >olume manager *:3M+) This is &ecause the (hysical
dis.s can only &e o(ened &y (rograms running as root)
Physical dis.s ha>e (ath names similar to the #ollo,ing2
/de>/rdlm#dr>8
/de>/rdlm#dr>%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CC o# $%$
Case $ 2 :un-s (ro>ided &y the D1S storage ,ith D1:M as multi-(athing dri>er)
Di%,% will be %een a% dlmfdrv at A!+ level u%in( l%0v command' and 0art of a FDL" N2
-volume (rou0%..
On node 1 M
d"m8dr7# ##ed**8+-#-588 root7g ati7e
d"m8dr71 none 7g?a0m ati7e
d"m8dr7* none 7g?a0m ati7e
d"m8dr7) none 7g?a0m ati7e
d"m8dr73 none 7g?a0m ati7e
d"m8dr7% none 7g?a0m ati7e
;
or
d"m8dr7# ##ed**8+-#-588 root7g ati7e
d"m8dr71 none 7g?or?di0:1 ati7e
d"m8dr7* none 7g?7oting?di0:1 ati7e
d"m8dr7) none 7g?a0m?di0:1 ati7e
d"m8dr73 none 7g?a0m?di0:* ati7e
d"m8dr7% none 7g?a0m?di0:) ati7e
;
*. !n%tall FDL" on all node%' attac& t&e LB> to eac& node' dicover LB>P% wit& 3cf(m(r3.
+ Gurn off re%erve loc,in( on all node% dln,m(r %et ?r%v on C ?%
J. Create N2% -Nolume 2rou0%. and LN% -Lo(ical Nolume%.
5 o0tion% 4
1+ Create a ,G -ith all dlm'dr. and create /,0s out o' it
On node 1 M
1G reate the 7o"ume group0
d"mm:7g -y 7g?a0m -2 -0 *%, -/ 1#1 d"m8dr71 d"m8dr7*
d"m8dr7) d"m8dr73 d"m8dr7%
*G enab"e the 7o"ume group0
d"m7aryon7g 7g?a0m
)G Create the "ogia" 7o"ume0
m:"7 -y "7?or?di0:1 -T . -< n -0 n -r n 7g?a0m *
m:"7 -y "7?7oting?di0:1 -T . -< n -0 n -r n 7g?a0m *
m:"7 -y "7?a0m?di0:1 -T . -< n -0 n -r n 7g?a0m *
m:"7 -y "7?a0m?di0:* -T . -< n -0 n -r n 7g?a0m *
m:"7 -y "7?a0m?di0:) -T . -< n -0 n -r n 7g?a0m *
;
3G Change permi00ion0Q o<nerQ group o8 the ra< de7ie0
21#511-#% .ra"e (2 'n0ta""ation &uide p*-+#
ho<n ora"e:dba /de7/r"7?or?di0:1
ho<n ora"e:dba /de7/r"7?7oting?di0:1
ho<n ora"e:dba /de7/r"7?a0m?di0:1
ho<n ora"e:dba /de7/r"7?a0m?di0:*
ho<n ora"e:dba /de7/r"7?a0m?di0:)
;
hmod ,,# /de7/r"7?or?di0:1
hmod ,,# /de7/r"7?7oting?di0:1
hmod ,,# /de7/r"7?a0m?di0:1
hmod ,,# /de7/r"7?a0m?di0:*
hmod ,,# /de7/r"7?a0m?di0:)
;
%G di0ab"e the 7o"ume group0
#+ Create on ,G 'or one dlm'dr. and one /, 'or one ,G1
2Recommended 'or ease o' administration and a.oid
do-ntime13
On node 1 M
1G reate the 7o"ume group0
d"mm:7g -y 7g?or1 -2 -0 *%, -/ 1#1 d"m8dr71
d"mm:7g -y 7g?7ot1 -2 -0 *%, -/ 1#1 d"m8dr7*
d"mm:7g -y 7g?a0m1 -2 -0 *%, -/ 1#1 d"m8dr7)
d"mm:7g -y 7g?a0m* -2 -0 *%, -/ 1#1 d"m8dr73
d"mm:7g -y 7g?a0m) -2 -0 *%, -/ 1#1 d"m8dr7%
;
*G enab"e the 7o"ume group0
d"m7aryon7g 7g?or1
d"m7aryon7g 7g?7ot1
d"m7aryon7g 7g?a0m1
d"m7aryon7g 7g?a0m*
d"m7aryon7g 7g?a0m)
;
)G Create the "ogia" 7o"ume0
m:"7 -y "7?or?di0:1 -T . -< n -0 n -r n 7g?or1 *
m:"7 -y "7?7oting?di0:1 -T . -< n -0 n -r n 7g?7ot1 *
m:"7 -y "7?a0m?di0:1 -T . -< n -0 n -r n 7g?a0m1 *
m:"7 -y "7?a0m?di0:* -T . -< n -0 n -r n 7g?a0m* *
m:"7 -y "7?a0m?di0:) -T . -< n -0 n -r n 7g?a0m) *
;
3G Change permi00ion0Q o<nerQ group o8 the ra< de7ie0
21#511-#% .ra"e (2 'n0ta""ation &uide p*-+#
ho<n ora"e:dba /de7/r"7?or?di0:1
ho<n ora"e:dba /de7/r"7?7oting?di0:1
ho<n ora"e:dba /de7/r"7?a0m?di0:1
ho<n ora"e:dba /de7/r"7?a0m?di0:*
ho<n ora"e:dba /de7/r"7?a0m?di0:)
;
hmod ,,# /de7/r"7?or?di0:1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CG o# $%$
d"m7aryo887g 7g?a0m
On node # $
6) 'denti8y <hih d"m8dr7 orre0pond to d"m8dr71 8rom
node1$
d"m8dr71 on node1 d"m8dr7# on node*
+G import the 7o"ume group0 on node*
Thi0 <i"" opy the 7g/"7 on8iguration that <a0 made on
node1
d"mimport7g -/ 1#1 -y 7g?a0m d"m8dr7#
+G enab"e the 7o"ume group0 on node*
d"m7aryon7g 7g?a0m
5G thi0 <i"" en0ure the 7g <i"" not get 7aryonPd at
boot
d"mh7g -a n 7g?a0m
On node 1 $
-G enab"e the 7o"ume group0 on node1
d"m7aryon7g 7g?a0m
C&ec, for document 4
5itac*i 8ynamic Ain# Mana%er >58AM? )or IBM AI:
Systems 9ser;s Guide 4
*ttpE==sys3
admin.net=uploads=SA!=*ldmBadminB%uide.pd)
hmod ,,# /de7/r"7?7oting?di0:1
hmod ,,# /de7/r"7?a0m?di0:1
hmod ,,# /de7/r"7?a0m?di0:*
hmod ,,# /de7/r"7?a0m?di0:)
;
%G di0ab"e the 7o"ume group0
d"m7aryo887g 7g?or1
d"m7aryo887g 7g?7ot1
d"m7aryo887g 7g?a0m1
d"m7aryo887g 7g?a0m*
d"m7aryo887g 7g?a0m)
;
On node # $
6) 'denti8y <hih d"m8dr7 orre0pond to d"m8dr71 8rom
node1Q and 0o on <ith other d"m8dr7
d"m8dr71 on node1 d"m8dr7# on node*
d"m8dr7* on node1 d"m8dr71 on node*
d"m8dr7) on node1 d"m8dr7* on node*
d"m8dr73 on node1 d"m8dr7) on node*
d"m8dr7% on node1 d"m8dr73 on node*

+G import the 7o"ume group0 on node*
Thi0 <i"" opy the 7g/"7 on8iguration that <a0 made on
node1
d"mimport7g -/ 1#1 -y 7g?or1 d"m8dr7#
d"mimport7g -/ 1#1 -y 7g?7ot1 d"m8dr71
d"mimport7g -/ 1#1 -y 7g?a0m1 d"m8dr7*
d"mimport7g -/ 1#1 -y 7g?a0m* d"m8dr7)
d"mimport7g -/ 1#1 -y 7g?a0m) d"m8dr73
;
+G enab"e the 7o"ume group0 on node*
d"m7aryon7g 7g?or1
d"m7aryon7g 7g?7ot1
d"m7aryon7g 7g?a0m1
d"m7aryon7g 7g?a0m*
d"m7aryon7g 7g?a0m)
;
5G thi0 <i"" en0ure the 7g <i"" not get 7aryonPd at
boot
d"mh7g -a n 7g?or1
d"mh7g -a n 7g?7ot1
d"mh7g -a n 7g?a0m1
d"mh7g -a n 7g?a0m*
d"mh7g -a n 7g?a0m)
;
On node 1 $
-G enab"e the 7o"ume group0 on node1
d"m7aryon7g 7g?or1
d"m7aryon7g 7g?7ot1
d"m7aryon7g 7g?a0m1
d"m7aryon7g 7g?a0m*
d"m7aryon7g 7g?a0m)
;
A. remove any PN!D from dlmfdrvS
@. acce%% t&e di%, t&ru devrlv6a%m6di%,S for A$" di%,(rou0 confi(uration
7.2.7 Others, StorageTek, HP EVA storage and multi-pathing
1or mo%t of t&e %tora(e %olution%' 0lea%e contact t&e 0rovidin( com0any for %u00orted confi(uration' a% readwrite
concurrent acce% from all RAC node% mu%t be 0o%%ible to im0lement a RAC %olution. G&at mean% 0o%%ibility to %etu0
t&e di%, re%erve60olicy to no6re%erve or e)uivalent.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CK o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C8 o# $%$
8 SP;CIFIC COESI1;!TIOES FO !C/!SM S;T4P 5ITD D!CMP
IEST!::;1
Oracle Cluster,are does not rePuires D!CMP to ,or./ &ut some customers may need to ha>e D!CMP
installed on the !C node cluster to (rotect third (arty (roducts/ or ressources) Oracle clu%terware could
re0lace FAC"P for mo%t o0eration% a% cold failover for **(' *C( and Di %in(le databa%e' a00lication% %erver% or any
a00lication%.
Please chec. #ollo,ing documents #or details on
&tt04www.oracle.comtec&nolo(y0roduct%databa%eclu%terin(inde<.&tml 4
Com0arin( Oracle Real A00licaton Clu%ter% to 1ailover Clu%ter% for Oracle Databa%e-PD1. December 5CC/
Wor,load "ana(ement wit& Oracle Real A00lication Clu%ter% -1A>' 1C1' Load Balancin(. -PD1. "ay 5CC@
B%in( Oracle Clu%terware to Protect Jrd Party A00lication% -PD1. "ay 5CC@
B%in( Oracle Clu%terware to Protect a $in(le !n%tance Oracle Databa%e -PD1. >ovember 5CC/
B%in( Oracle Clu%terware to 0rotect Oracle A00lication $erver -PD1. >ovember 5CC@
Fow to Build End to End Recovery and Wor,load Balancin( for Mour A00lication% *C( Relea%e *-PD1. Dec
5CCA
Oracle Databa%e *C g $ervice% -PD1. >ov 5CCJ
Oracle Databa%e *C g Relea%e 5 Be%t Practice%4 O0timiLin( Availability Durin( Bn0lanned Outa(e% B%in(
Oracle Clu%terware and RAC 4e-5
DO5;3;/ i# customer still need to ha>e D!CMP/ Oracle Cluster,are can coha&itate ,ith D!CMP)
Please chec. #or last status o# certi#ication on Oracle Metalin. *certi#y+
;'tract #rom metalin. in date o# March $
rd
/ 7668)
11g 1 GB-&it Certi#ication Summary
OS Product Certi#ied 5ith 3ersion Status
A!+ -@J. **(R* /A-bit Oracle Clu%terware **( Certified
A!+ -@J. **(R* /A-bit FAC"P @.A.* Certified
A!+ -@J. **(R* /A-bit FAC"P @.J Certified
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C% o# $%$
1etailed in#ormation #or D!CMP C)$ and 11g!C 2
Certi#y - !dditional In#o !C #or 4ni' 3ersion 11g1 GB-&it On IBM !I" &ased Systems *!C only+
O0eratin( $y%tem4 IBM !I" &ased Systems *!C only+ Ner%ion C)$
!C #or 4ni' Ner%ion 11g1 GB-&it
D!CMP Ner%ion C)$
$tatu%4 Certi#ied
Product 3ersion Eote2
>one available for t&i% 0roduct.
Certi#ication Eote2
B%e of FAC"P @.J re)uire% minimum %ervice level% of t&e followin(4
o A!+@L @.J GL / or later' %0ecifically bo%.rte.lvm mu%t be at lea%t @.J.C./C
o FAC"P N@.J wit& PG1@' clu%ter.e%.clvm in%talled and ifi< for APAR ![C*ICD
o R$CG -r%ct.ba%ic.rte. ver%ion 5.A.E.J and ifi< for APAR ![C*IJI
1etailed in#ormation #or D!CMP C)B)1 and 11g!C 2
Certi#y - !dditional In#o !C #or 4ni' 3ersion 11g1 GB-&it On IBM !I" &ased Systems *!C only+
O0eratin( $y%tem4 IBM !I" &ased Systems *!C only+ Ner%ion C)$
!C #or 4ni' Ner%ion 11g1 GB-&it
D!CMP Ner%ion C)B)1
$tatu%4 Certi#ied
Product 3ersion Eote2
>one available for t&i% 0roduct.
Certi#ication Eote2
B%e of FAC"P @.A re)uire% minimum %ervice level% of t&e followin(4
o A!+@L @.J GL / or later' %0ecifically bo%.rte.lvm mu%t be at lea%t @.J.C./C
o FAC"P N@.A.* -Available in media or APAR ![C5/5C.
o R$CG -r%ct.ba%ic.rte. ver%ion 5.A.E.J and ifi< for APAR ![C*IJI G&i% APAR i% inte(rated into N5.A.I.*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G6 o# $%$
Follo,ing rules ha>e to &e a((lied 2
*. FAC"P mu%t not ta,e-overfailover t&e Oracle Clu%terware re%%ource% -N!P' databa%e' etc S.
5. FAC"P N!P mu%t not be confi(ured on !P from Public node name u%ed by RAC -&o%tname.' or Oracle
Clu%terware N!P
. Wit& **(' itP% not nece%%ary to declare t&e RAC interconnect in FAC"P
A. !tP% not mandatory to declare &di%,% u%ed for A$" in FAC"P a% lo(ical volume% -LN. from Nolume 2rou0%
-N2.. !n t&i% ca%e follow t&e coo,boo, to 0re0are t&e di%,% for OCR' Notin( and A$" di%,%.
'. !f t&e c&oice i% to declare &di%,% u%ed by A$" in FAC"P Nolume 2rou0%' GFE> youPll &ave to 0re0are t&e
di%,% for OCR' Notin(' A$" %0file and A$" di%,% a% de%cribe in official Oracle document available on
&tt04ta&iti.oracle.com
Please chec. 2
OracleL eal !((lication Clusters Installation =uide
11g elease 1 *11)1+ #or :inu' and 4EI"
Part >umber B5I5/A-CJ
&tt04download.oracle.comdoc%cdB5IJ@D6C*in%tall.***b5I5/Atoc.&tm
And "etalin, note ACAAEA.*
$tatu% of Certification of Oracle Clu%terware wit& FAC"P @.J : @.A
$GAGB$ of !B" D!CMP @.J' @.A Certification% wit& Oracle RAC *C(
@.A Certification% wit& Oracle RAC *C( W&at do you need to do8 Confi(urin( D!CMP @.J or D!CMP @.A.*'
and *C(R5 wit& "ulti->ode Di%, Feartbeat -">DFB.
Even %o t&i% note i% written for *C(RAC R5' itP% al%o a00licable to **( RAC R*.
Chec. #or the #ollo,ing cha(ters 2
$)$ Con#iguring Storage #or Oracle Cluster,are Files on a, 1e>ices
The #ollo,ing su&sections descri&e ho, to con#igure Oracle Cluster,are #iles on ra, (artitions)
Confi(urin( Raw Lo(ical Nolume% for Oracle Clu%terware
Creatin( a Nolume 2rou0 for Oracle Clu%terware
Confi(urin( Raw Lo(ical Nolume% in t&e >ew Oracle Clu%terware Nolume 2rou0
!m0ortin( t&e Nolume 2rou0 on t&e Ot&er Clu%ter >ode%
Activatin( t&e Nolume 2rou0 in Concurrent "ode on All Clu%ter >ode%
For OC/>oting dis.s/ do create a >olume grou( *3=+/ and create logical >olumes *l>+ ,ith names as
/de>/OcrN1is.1/ /de>/OcrN1is.7/ /de>/3otingN1is.1/ etc
!nd don-t #orget to remo>e the reser>e (olicy on all hdis.s
$)G Con#iguring 1ata&ase File Storage on a, 1e>ices
The #ollo,ing su&sections descri&e ho, to con#igure ra, (artitions #or data&ase #iles)
Confi(urin( Raw Lo(ical Nolume% for Databa%e 1ile $tora(e
Creatin( a Nolume 2rou0 for Databa%e 1ile%
Creatin( Databa%e 1ile Raw Lo(ical Nolume% in t&e >ew Nolume 2rou0
!m0ortin( t&e Databa%e 1ile Nolume 2rou0 on t&e Ot&er Clu%ter >ode%
Activatin( t&e Databa%e 1ile Nolume 2rou0 in Concurrent "ode on All Clu%ter >ode%
For !SM dis.s/ do create a >olume grou( *3=+/ and create logical >olumes *l>+ ,ith names as
/de>/!SMN1is.1/ /de>/!SMN1is.7/ /de>/!SMN1is.$/ etc
!nd don-t #orget to remo>e the reser>e (olicy on all hdis.s
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G1 o# $%$
% IEST!::!TIOE ST;PS
Prioir to install and use Oracle 11g eal !((lication Cluster/ you must 2
1) Pre(are the in#rastructure
a. Fardware
b. $tora(e
c. >etwor,
d. $an and >etwor, connectivity
e. O0eratin( %y%tem
7) Pre(are the systems
a. Definin( >etwor, Layout' Public' Nirtual and
Private &o%tname%
b. A!+ O0eratin( %y%tem level' re)uired APARP%
and fil%et%
c. $y%tem Re)uirement% -$wa0' tem0' memory'
internal di%,%.
d. B%er% And 2rou0%
e. Xernel and $&ell Limit%
f. B%er e)uivalence%
(. Gime $erver $ync&roniLation
$) Pre(are the storage
a. Create LB>P% for OCR and Notin( di%,%' and
0e0are t&em at A!+ level
b. Create LB> for A$" in%tance% %0file' and
0e0are it at A!+ level
c. Create LB>P% for A$" di%,%' and 0e0are t&em
at A!+ level
B) Chec. that all (re- installation rePuirements are
#ull#iled
a. B%in( Oracle clu%ter verification utility -CNB.
b. B%in( in-&ou%e %&ell %cri0t%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G7 o# $%$
5hen all that is done 000/ you can (rocess the installation in the #ollo,ing order 2
1) Install Oracle 11g Cluster,are
a. A00ly nece%%ary 0atc&%et and 0atc&e%
b. C&ec, t&at Oracle clu%terware i% wor,in(
0ro0erly
7) Install Oracle 11g !utomated Storage Management
b. Create default node% li%tener%
c. Create A$" in%tance% on eac& node
d. Confi(ure A$" in%tance% Local and remote
li%tener% if re)uired
e. C&an(e nece%%ary A$" in%tance% 0arameter%
-0roce%%' etc S.
f. Create A$" di%,(rou0-%.
. Inst all Oracle 11g eal !((lication Cluster or/and
Oracle 16g eal !((lication cluster
B) Create data&ase*s+
a. Confi(ure Databa%e% in%tance% Local and
remote li%tener% if re)uired
b. C&an(e nece%%ary Databa%e% in%tance%
0arameter% -0roce%%' etc S.
c. Create databa%e-%. a%%ociated clu%ter %ervice%'
and confi(ure GA1 -Gran%action% A00lication%
1ailover. a% re)uired for your need%
d.
C) Test your cluster ,ith crash scenarios

11g!C/!SM/!I"
oraclibm@fr.ibm.com
G$ o# $%$
16 P;P!IE= TD; SIST;M
Pre0arin( t&e %y%tem i% to be done on all %erver% w&ic& are 0lanned to 0artici0ate in t&e Oracle Clu%ter.
All %erver% "B$G be a% clone%' wit& O>LM different FO$G>A"E and !P addre%%e% OOO
IMPOT!ET 000
For !:: ser>ers/ you M4ST a((ly the Oracle (re-rePuisites/ those (rerePuisites are not
o(tional/ B4T M!E1!TOI 000
For some (arameters as tuning settings/ >alues s(eci#ied are the minimum rePuired/ and
migh &e increase de(ending o# your needs 000
P:;!S; chec. Oracle documentation #or last u(date/ and MOST:I 2
Oracle M;T!:IET Eote
Pre(aring the system is a&out 2
Definin( >etwor, Layout' Public' Nirtual and Private &o%tname%
o Fo%tname and RAC Public >ode name
o >etwor, interface identification
o Default 2ateway
o >etwor, Gunin( Parameter%
A!+ O0eratin( %y%tem level' re)uired APARP% and fil%et%
$y%tem Re)uirement% -$wa0' tem0' memory' internal di%,%.
B%er% And 2rou0%
Xernel and $&ell Limit%
B%er e)uivalence%
Gime $erver $ync&roniLation
Etc S
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GB o# $%$
16)1 Eet,or. con#iguration
10.1.1 Define Networks layout, Public, Virtual and Private Hostnames
5e need 7 di##erents net,or.s' wit& a%%ociated networ, interface% on eac& node 0artici0atin( to t&e RAC clu%ter 4
Public >etwor, to be u%ed a% B%er% networ, or re%erved networ, for A00lication and Databa%e% %erver%.
Private >etwor, to be u%ed a% a Re%erved networ, for Oracle clu%terware' and RAC.
M!E1!TOI 2 Eet,or. inter#aces must ha>e same name/ same su&net and same usage 000
For each node/ 5e ha>e to de#ine 2
Pu&lic hostname and corre%0ondin( !P addre%% on t&e 3Public >etwor,T
3irtual hostname and corre%0ondin( !P addre%%' al%o on t&e 3Public >etwor,T.
!P addre%% mu%t be of %ame ran(e t&en Public &o%tname' but not u%ed on t&e networ, 0rior to Oracle
clu%terware in%tallation.
Pri>ate hostname and corre%0ondin( !P addre%% on t&e 3Private >etwor,T.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GC o# $%$
:et identi#y hostnames/ and net,or. inter#aces M
Please ma.e a ta&le as #ollo, to ha>e a clear >ie, o# your !C net,or. architecture 2
P4B:IC EO1; E!M; M4ST B; E!M; ;T4E;1 BI ?hostname@ !I" COMM!E1
Pu&lic Dostname 3IP Dostname
*3irtual IP+
Pri>ate Dostname
*!C Interconnect+
Eot 4sed
en R *Pu&lic Eet,or.+ en R *Pri>ate Eet,or.+ en R
Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP Eode Eame IP
Issue the !I" command ?hostname@ on each node to identi#y de#ault node name 2
-or first server ...
{node1:root}/ # $ostname
node1
{node1:root}/ # pin+ node1
P'N& node1: F1&.3.25..1G: %, data byte0
,3 byte0 8rom 1#$)$*%$51: imp?0eJO# tt"O*%% timeO# m0
,3 byte0 8rom 1#$)$*%$51: imp?0eJO1 tt"O*%% timeO# m0
,3 byte0 8rom 1#$)$*%$51: imp?0eJO* tt"O*%% timeO# m0
RC
----node1 P'N& Stati0ti0----
) pa:et0 tran0mittedQ ) pa:et0 reei7edQ #9 pa:et "o00
round-trip min/a7g/ma1 O #/#/# m0
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GG o# $%$
-or second server ...
{node2:root}/ # $ostname
node2
{node*:root}/ # pin+ node2
P'N& node*: F1&.3.25..2G: %, data byte0
,3 byte0 8rom 1#$)$*%$5*: imp?0eJO# tt"O*%% timeO# m0
,3 byte0 8rom 1#$)$*%$5*: imp?0eJO1 tt"O*%% timeO# m0
,3 byte0 8rom 1#$)$*%$5*: imp?0eJO* tt"O*%% timeO# m0
RC
----node* P'N& Stati0ti0----
) pa:et0 tran0mittedQ ) pa:et0 reei7edQ #9 pa:et "o00
round-trip min/a7g/ma1 O #/#/# m0
{node*:root}/ #
Other method to chec. the de#ault hostname 2
{node1:root}/r0/11$1$#/bin # lsattr -/l inet&
authm ,%%), 6uthentiation Method0 True
bootup?option no U0e 2S(-0ty"e Net<or: Con8iguration True
gate<ay &ate<ay True
ho0tname node1 No0t Name True
rout, 'P7, Boute True
route netQ-hopountQ#QQ#Q1#$)$*%$*%3 Boute True
{node1:root}/r0/11$1$#/bin #
To change the de#ault hostname 2
{node1:root}/r0/11$1$#/bin # c$dev -l inet& -a $ostname0node1
inet# hanged
Eo,/ ,e ha>e Pu&lic hostnames and corres(onding IP-s in our ta&le 2
*3irtual IP+
Pri>ate Dostname
*!C Interconnect+
Eot 4sed
Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP Eode Eame IP
node1 16)$)7C)81
node7 16) $)7C)87
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GK o# $%$
Oracle cluster,are 3IP-s IP adress and corres(onding nodes names must not &e used on the net,or. (rior to
Oracle Cluster,are installation) DonPt ma,e any A!+ allia% on t&e 0ublic networ, interface' t&e clu%terware in%tallation
will do it. #u%t re%erve * N!P and itP% &o%tname 0er RAC node.
*3irtual IP+
Pri>ate Dostname
*!C Interconnect+
Eot 4sed
Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP Eode Eame IP
node* *C.J.5@.I* node1->i( 16)$)7C)181
node5 *C. J.5@.I5 node7->i( 16) $)7C)187
Oracle Clu%terware N!PP% !P and corre%0ondin( node% name% can be declared in t&e D>$' or at minimum in t&e local
&o%t% file.
10.1.2 Identify Network Interfaces cards
!s root/ Issue the !I" command ?i#con#ig l@ to list net,or. card on each node 2
esult #rom node1 2
{node1:root}/ # ifconfi+ 1l
en& en1 en2 lo&
{node1:root}/ #
esult #rom node7 2
{node2:root}/ # ifconfi+ -l
en& en1 en2 lo&
{node*:root}/ #
{node1:root}/r0/11$1$#/bin # lsdev 2+rep en
en& Available Standard /t$ernet #etwork Interface
en1 Available Standard /t$ernet #etwork Interface
en2 Available Standard /t$ernet #etwork Interface
ent# 67ai"ab"e /irtua" '/. !thernet 6dapter F"-"anG
ent1 67ai"ab"e /irtua" '/. !thernet 6dapter F"-"anG
ent* 67ai"ab"e /irtua" '/. !thernet 6dapter F"-"anG
inet# 67ai"ab"e 'nternet Net<or: !1ten0ion
rm# (e8ined Bendering Conte1t Manager Sub0y0tem
700i# 67ai"ab"e /irtua" SCS' C"ient 6dapter
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G8 o# $%$
!s root/ Issue the #ollo,ing shell to get necessary in#ormation #rom net,or. inter#aces on each node 2
esult #rom node1 2
{node1:root}/ # for i in en& en1 en2
do
ec$o 3i
for attribut in netaddr netmask broadcast
state
do
lsattr -/l 3i -a 3attribut
done
done
en&
netaddr 1#$)$*%$51 'nternet 6ddre00 True
netma0: *%%$*%%$*%%$# Subnet Ma0: True
broada0t 2roada0t 6ddre00 True
0tate up Current 'nter8ae Statu0 True
en1
netaddr 1#$1#$*%$51 'nternet 6ddre00 True
en2
netaddr *#$*#$*%$51 'nternet 6ddre00 True
{node1:root}/ #
esult #rom node7 2
{node2:root}/ # for i in en& en1 en2
do
ec$o 3i
for attribut in netaddr netmask broadcast
state
do
lsattr -/l 3i -a 3attribut
done
done
en&
netaddr 1#$)$*%$5* 'nternet 6ddre00 True
en1
netaddr 1#$1#$*%$5* 'nternet 6ddre00 True
en2
netaddr *#$*#$*%$5* 'nternet 6ddre00 True
{node*:root}/ #
!s root/ Issue the !I" command ?i#con#ig a@ to list net,or. card on each node 2
esult e'am(le #rom node1 2
{node1:root}/ #i8on8ig Sa
en&4
8"ag0O1e#5#5,)Q5#TUPQ2B.6(C6STQN.TB6'H!BSQBUNN'N&QS'MPH!KQMUHT'C6STQ&B.UPBTQ,32'TQCN!CUSUM?.AA
H.6(QCN6'NC
inet 1&.3.25..1 netma0: #1888888## broada0t 1#$)$*%$*%%
tp?0end0pae 1)1#+* tp?re70pae ,%%),
en14
H.6(QCN6'NC
inet 1&.1&.25..1 netma0: #1888888## broada0t 1#$1#$*%$*%%
en24
H.6(QCN6'NC
inet 2&.2&.25..1 netma0: #1888888## broada0t *#$*#$*%$*%%
"o#: 8"ag0Oe#5#53bTUPQ2B.6(C6STQH..P26CUQBUNN'N&QS'MPH!KQMUHT'C6STQ&B.UPBTQ,32'TC
inet 1*+$#$#$1 netma0: #188###### broada0t 1*+$*%%$*%%$*%%
inet, ::1/#
tp?0end0pae 1)1#+* tp?re70pae 1)1#+* r81)*) 1
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G% o# $%$
Eo,/ ,e ha>e identi#ied the #ollo,ing net,or. inter#aces 2
enC i% %et to be t&e 3Public >etwor, !nterfaceT on all node%.
enC i% %et to be t&e 3N!P >etwor, !nterfaceT on all node%.
en* i% %et to be t&e 3Private >etwor, !nterfaceT' al%o named a% 3RAC !nterconnectT on all node%.
en5 i% %et a% not u%ed
TD;E/ the ta&le loo.s as #ollo, 2
*3irtual IP+
Pri>ate Dostname
*!C Interconnect+
Eot 4sed
en6 *Pu&lic Eet,or.+ en1 *Pri>ate Eet,or.+ en7
Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP Eode Eame IP
node* *C.J.5@.I* node*-vi0 *C.J.5@.*I*
node5 *C. J.5@.I5 node5-vi0 *C. J.5@.*I5
To see
details on
the net,or.
inter#ace 2
{node1:root}/ora"e -C lsattr -/l en&
a"ia03 'P73 6"ia0 in"uding Subnet Ma0: True
a"ia0, 'P7, 6"ia0 in"uding Pre8i1 Hength True
arp on 6ddre00 Be0o"ution Protoo" F6BPG True
authority 6uthoriLed U0er0 True
mtu 1%## Ma1imum 'P Pa:et SiLe 8or Thi0 (e7ie True
netaddr 1&.3.25..1 'nternet 6ddre00 True
netaddr, 'P7, 'nternet 6ddre00 True
netmask 255.255.255.& Subnet Ma0: True
pre8i1"en Pre8i1 Hength 8or 'P7, 'nternet 6ddre00 True
remmtu %+, Ma1imum 'P Pa:et SiLe 8or B!M.T! Net<or:0 True
r81)*) !nab"e/(i0ab"e TCP BAC 1)*) 4indo< Sa"ing True
0eurity none Seurity He7e" True
tp?m00d8"t Set TCP Ma1imum Segment SiLe True
tp?node"ay !nab"e/(i0ab"e TCP?N.(!H6I .ption True
tp?re70pae Set So:et 2u88er Spae 8or Beei7ing True
tp?0end0pae Set So:et 2u88er Spae 8or Sending True
{node1:root}/ora"e -C
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K6 o# $%$
TD;E/ ,e ,ill get the #ollo,ing ta&le ,ith our system 2
Pu&lic 3IP !C Interconnect *Pri>ate
Eet,or.+
Eot 4sed
en6 en1 en7
Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP Eode
Eame
IP
node* *C.J.5@.I* node*-vi0 *C.J.5@.*I* node1-rac 16)16)7C)81
node5 *C. J.5@.I5 node5-vi0 *C. J.5@.*I5 node1-rac 16)16)7C)87
5ithin our in#rastructure #or the coo.&oo./ ,e ha>e the #ollo,ing layout 2
10.1.3 Update hosts file
Iou should ha>e the #ollo,ing entries on each node for 3etc&o%t%T
4(date/chec. entries in hosts #ile on each node
{node1:root}/ # cat /etc/hosts
U Oracle Cluster,are Pu&lic nodes list
*C.J.5@.I* node*
*C.J.5@.I5 node5
U Oracle Cluster,are 3irtual IP nodes list
*C.J.5@.*I* node*-vi0
*C.J.5@.*I5 node5-vi0
U Oracle Cluster,are and !C Interconnect
*C.*C.5@.I* node*-rac
*C.*C.5@.I5 node5-rac
4O6A "
Oracle Cluster,are 3irtual IP-s are 9ust #ree
IP-s a>aila&le/ and OE:I declared in the hosts
#ile #orm each node (rior to Oracle
cluster,are installation)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K1 o# $%$
10.1.4 Defining Default gateway on public network interface
>ow' if you want to %et the de#ault gate,ay on (u&lic net,or. inter#ace/ &e carre#ull on the im(act it
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K7 o# $%$
may ha>e i# you ha>e already a de#ault gate,ay set on a di##erent net,or. inter#ace/ and multi(le net,or.
inter#aces not used #or Oracle cluster,are and !C (ur(ose)
First chec. i# de#ault gate,ay is set 2
4se ?netstat -r@ On &oth nodes)
{node1:root}/ # netstat -r
Bouting tab"e0
(e0tination &ate<ay A"ag0 Be80 U0e '8 !1p &roup0
Boute Tree 8or Protoo" Aami"y * F'nternetG:
default 1&.3.25.255 67 1 '335. en& - - 08
de8au"t -$*1*$1)1$*%3 U& # 3)% en# - -
1#$)$*%$# node1 UNSb # # en# - - OC
1#$)$*%/*3 node1 U 11 ,11)%-# en# - -
node1 "oopba: U&NS )+ 1#)33#1 "o# - -
node1-7ip "oopba: U&NS 5 5#5)1 "o# - -
1#$)$*%$*%% node1 UNSb # 3 en# - -
1#$1#$*%$# node1-ra UNSb # # en1 - - OC
1#$1#$*%/*3 node1-ra U *% )%#%%+ en1 - -
node1-ra "oopba: U&NS 1, 351 "o# - -
1#$1#$*%$*%% node1-ra UNSb # 3 en1 - -
*#$*#$*%$# node1-ra-b UNSb # # en* - - OC
*#$*#$*%/*3 node1-ra-b U 1, 1+,)+- en* - -
node1-ra-b "oopba: U&NS % )-* "o# - -
*#$*#$*%$*%% node1-ra-b UNSb # 3 en* - -
1*+/5 "oopba: U 3- 15+1#% "o# - -
Boute Tree 8or Protoo" Aami"y *3 F'nternet 7,G:
::1 ::1 UN # )* "o# - -
{node1:root}/ #
!f not %et 4
B%in( 3route addT' do set the de#ault gate,ay on (u&lic net,or. inter#ace' enC in our ca%e' and on all node% 4
M4ST B; 1OE; on each node 000
Go e%tabli%& a default (ateway -*C.J.5@.5@A in our ca%e.' ty0e4
{node1:root}/ # route add 6 16)$)7C)7CB
{node1:root}/r0/11$1$#/bin # netstat -r 2 +rep 1&.3.25.255
de8au"t 1#$)$*%$*%3 U& * 1#%# en# - -
{node2:root}/ # route add 6 16)$)7C)7CB
{node*:root}/ # netstat -r 2 +rep 1&.3.25.255
de8au"t 1#$)$*%$*%3 U& 1 +)*3 en# - -
{node*:root}/ #
G&e value C or t&e default ,eyword for t&e De%tination 0arameter mean% t&at any 0ac,et% %ent to de%tination% not
0reviou%ly defined and not on a directly connected networ, (o t&rou(& t&e default (ateway. G&e *C.J.5@.5@A addre%%
i% t&at of t&e (ateway c&o%en to be t&e default.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K$ o# $%$
10.1.5 Configure Network Tuning Parameters
Nerify t&at t&e networ, tunin( 0arameter% %&own in t&e followin( table are %et to t&e value% %&own or &i(&er value%.
G&e 0rocedure followin( t&e table de%cribe% &ow to verify and %et t&e value%.
Eet,or. Tuning
Parameter ecommended minimum 3alue on all nodes
i(Pma'len @*5
r#c1$7$ *
s&Nma' *J*CE5C
tc(Nrec>s(ace /@@J/
tc(Nsends(ace /@@J/
ud(Nrec>s(ace /@@J/C
Eote2 G&e recommended value of t&i% 0arameter i% *C time% t&e value of t&e ud06%end%0ace
0arameter. G&e value mu%t be le%% t&an t&e value of t&e %b6ma< 0arameter.
ud(Nsends(ace /@@J/
Eote2 G&i% value i% %uitable for a default databa%e in%tallation. 1or 0roduction databa%e%' t&e
minimum value for t&i% 0arameter i% A XB 0lu% t&e value of t&e databa%e DB6BLOCX6$![E
initialiLation 0arameter multi0lied by t&e value of t&e DB6"BLG!BLOCX6READ6COB>G
initialiLation 0arameter4
-DB6BLOCX6$![E Z DB6"BLG!BLOCX6READ6COB>G. = A XB
To chec. >alues M
{node1:root}/ # for i in ip%ma9len rfc1323 sb,ma9 tcp,recvspace tcp,sendspace
udp,recvspace udp,sendspace
do
no -a 2+rep 3i
done
ip%ma9len 0 512
rfc1323 0 1
sb,ma9 0 131&'2&
tcp,recvspace 0 :5535
tcp,sendspace 0 :5535
udp,recvspace 0 :5535&
udp,sendspace 0 :5535
{node1:root}/ #
{node2:root}/ # for i in ip%ma9len rfc1323 sb,ma9 tcp,recvspace tcp,sendspace
udp,recvspace udp,sendspace
do
no -a 2+rep 3i
done
ip%ma9len 0 512
rfc1323 0 1
sb,ma9 0 131&'2&
tcp,recvspace 0 :5535
tcp,sendspace 0 :5535
udp,recvspace 0 :5535&
udp,sendspace 0 :5535
{node*:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KB o# $%$
To change the
current >alues to
rePuired ones/ i#
necessary/
#ollo, these ste(s 2
1) I# you must change the >alue o# any (arameter/ enter the #ollo,ing command to
determine ,hether the system is running in com(ati&ility mode2
U /usr/s&in/lsattr -; -l sys6 -a (reC76tune
I# the system is running in com(ati&ility mode/ the out(ut is similar to the
#ollo,ing/ sho,ing that the >alue o# the (reC76tune attri&ute is ena&le2
(reC76tune ena&le Pre-C76 tuning com(ati&ility mode True
By de#ault/ ,ith !I"C:/ com(ati&ility mode is set to #alse 000
Change it to true OE:I i# necessary 000
SS i# you ,ant to ena&le the com(ati&ility mode/ issue the #ollo,ing command 2
U c$dev -l sys& -a pre52&tune0enable
2. I# the system is running in com(ati&ility mode/
TD;E
#ollo, these ste(s to change the (arameter
>alues2
;nter commands similar to the #ollo,ing to
change the >alue o# each (arameter2
# /u0r/0bin/no So parameter_nameOvalue
Aor e1amp"e:
# /u0r/0bin/no So udp?re70paeO,%%),#
!dd entries similar to the #ollo,ing to the
/etc/rc)net #ile #or each (arameter that you
changed in the (re>ious ste(2
i8 > -8 /u0r/0bin/no @ V then
/u0r/0bin/no -o udp?0end0paeO,%%),
/u0r/0bin/no -o udp?re70paeO,%%),#
/u0r/0bin/no -o tp?0end0paeO,%%),
/u0r/0bin/no -o tp?re70paeO,%%),
/u0r/0bin/no -o r81)*)O1
/u0r/0bin/no -o 0b?ma1O1)1#+*#
/u0r/0bin/no -o ipJma1"enO%1*
8i
By addin( t&e%e line% to t&e etcrc.net file' t&e value%
0er%i%t w&en t&e %y%tem re%tart%.
TD;E IO4 E;;1 TO Modi#y and ;BOOT all nodes 0
;:S;
U /usr/s&in/lsattr -; -l sys6 -a (reC76tune
5ill gi>e the #ollo,ing 2
pre52&tune disable "re-52& tunin+
compatibility mode ;rue
The system is not running in com(ati&ility mode/ enter
commands similar to the #ollo,ing to change the
(arameter >alues2
;nter commands similar to the #ollo,ing to change
the >alue o# (arameter i(Pma'len2
ipJma1"en parameter:
/u0r/0bin/no -r -o ipJma1"enO%1*
;nter commands similar to the #ollo,ing to change
the >alue o# each others (arameters2
/u0r/0bin/no -p So udp?0end0paeO,%%),
/u0r/0bin/no -p So udp?re70paeO,%%),#
/u0r/0bin/no -p So tp?0end0paeO,%%),
/u0r/0bin/no -p So tp?re70paeO,%%),
/u0r/0bin/no -p So r81)*)O1
/u0r/0bin/no -p So 0b?ma1O1)1#+*#
Eote2 I# you modi#y the i(Pma'len (arameter/ you must restart the system)
G&e%e command% modify t&e etctunable%ne<tboot file' cau%in( t&e attribute value% to 0er%i%t w&en t&e
%y%tem re%tart%.

11g!C/!SM/!I"
oraclibm@fr.ibm.com
KC o# $%$
16)7 !I" O(erating system le>el/ rePuired !P!-s and #ilsets
To ha>e the latest in#ormation (lease re#er to Metalin. Eote 7876$G)1) on
htt(2//metalin.)oracle)com/ this document include last u(date) Chec. also certi#ication
status on metalin.)oracle)com/ or otn)oracle)com
!I" release su((orted ,ith
Oracle 11g !C 1 as #or
Fe&ruary $rd/ 7668)
AI 5( version 5.3< *aintenance (evel 5 or
later
To determine ,hich >ersion o#
!I" is installed/ enter the
#ollo,ing command 2
=node14root>? @ oslevel -s
53&&-&'-&1-&'5. +ives level of ;( and sub-
level of service pack< w$ic$ means AI5( ;(' S"1
!f t&e o0eratin( %y%tem ver%ion i% lower t&an t&e minimum re)uired' u0(rade your o0eratin( %y%tem to t&i% level. A!+ @L
maintenance 0ac,a(e% are available from t&e followin( Web %ite 4
&tt04www-D*5.ibm.come%erver%u00ortfi<e%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KG o# $%$
10.2.1 Filesets Requirements for 11g RAC R1 / ASM (NO HACMP)
!I" #ilesets rePuired on !:: nodes #or 11g !C elease 1 im(lementation ,ith
!SM 000
Chec. that the
rePuired #ilsets are
installed on the
system)
->ote4 !f t&e PG1 i%
not downloadable'
cu%tomer% %&ould
re)ue%t an efi<
t&rou(& A!+
cu%tomer %u00ort..
!I" C: 4ersion 5.3;
&aintenance ?e4el 6 or later
!I"G 4ersion ...
&aintenance ?e4el ... or later
Filesets
bo%.adt.ba%e
bo%.adt.lib
bo%.adt.libm
bo%.0erf.lib0erf%tat
bo%.0erf.0erf%tat
bo%.0erf.0roctool%
r%ct.ba%ic.rte
r%ct.com0at.client%.rte
<lC.ai<@C.rte I.C.C.@
<lC.rte I.C.C.@
S(eci#ic Filesets
For ;MC Symmetri' 2
E"C.$ymmetri<.ai<.rte.@.5.C.* or &i(&er
For ;MC C:!iiOE 2
E"C.CLAR**O>.fc0.rte.@.5.C.* or
&i(&er
EOT I;T
S4PPOT;1
Chec. Metalin. and certi#y #or last u(date on
certi#ication status)
De0endin( on t&e A!+ Level t&at you intend to in%tall' verify t&at t&e re)uired file%et% are in%talled on t&e %y%tem. G&e
followin( 0rocedure de%cribe% &ow to c&ec, t&e%e re)uirement%.
To ensure that the
system meets
these
rePuirements/
#ollo, these ste(s 2
!nd Chec. that
rePuired #ilesets
are all installed)
!nd that 2
'lC)ai'C6)rte and
'lC)rteare at
minimum release
o# 8)6)6)B and
8)6)6)6
{node1:root}/ # lslpp -l bos.adt.base bos.adt.lib bos.adt.libm bos.perf.libperfstat
bos.perf.perfstat bos.perf.proctools rsct.basic.rte rsct.compat.clients.rte
9lC.ai95&.rte 9lC.rte
Ai"e0et He7e" State (e0ription
----------------------------------------------------------------------------
Path: /u0r/"ib/obWrepo0
bo0$adt$ba0e %$)$+$# C.MM'TT!( 2a0e 6pp"iation (e7e"opment
Too":it
bo0$adt$"ib %$)$#$,# C.MM'TT!( 2a0e 6pp"iation (e7e"opment
Hibrarie0
bo0$adt$"ibm %$)$+$# C.MM'TT!( 2a0e 6pp"iation (e7e"opment
Math Hibrary
bo0$per8$"ibper80tat %$)$+$# C.MM'TT!( Per8ormane Stati0ti0 Hibrary
'nter8ae
bo0$per8$per80tat %$)$+$# C.MM'TT!( Per8ormane Stati0ti0
'nter8ae
bo0$per8$protoo"0 %$)$+$# C.MM'TT!( Pro Ai"e0y0tem Too"0
r0t$ba0i$rte *$3$5$# C.MM'TT!( BSCT 2a0i Auntion
r0t$ompat$"ient0$rte *$3$5$# C.MM'TT!( BSCT !7ent Management C"ient
Auntion
1"C$ai1%#$rte -$#$#$1 C.MM'TT!( KH C/CDD Buntime 8or 6'K %$*
1"C$rte -$#$#$1 C.MM'TT!( KH C/CDD Buntime
Path: /et/obWrepo0
bo0$per8$"ibper80tat %$)$#$# C.MM'TT!( Per8ormane Stati0ti0 Hibrary
'nter8ae
bo0$per8$per80tat %$)$+$# C.MM'TT!( Per8ormane Stati0ti0
'nter8ae
r0t$ba0i$rte *$3$5$# C.MM'TT!( BSCT 2a0i Auntion
{node1:root}/ #
!f a file%et i% not in%talled and committed' t&en in%tall it. Refer to your o0eratin( %y%tem or %oftware documentation for
information about in%tallin( file%et%. !f file%et i% only 3APPL!EDT' itP% not mandatory to commit it.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KK o# $%$
10.2.2 APARs Requirements for 11g RAC R1 / ASM (NO HACMP)
Chec. that the rePuired !P!-s are installed on the system)
!I" Patches *!P!+ rePuired on !:: nodes #or 11g !C 1 im(lementation ,ith
!SM 000
->ote4 !f t&e PG1 i% not downloadable' cu%tomer% %&ould re)ue%t an efi< t&rou(& A!+ cu%tomer %u00ort..
!I" C: 4ersion 5.3;
&aintenance ?e4el 6 or later
!I"G 4ersion ...
&aintenance ?e4el ... or later
!P!-s
!MIDCIC
!MD5CJE
!MDAJAJ
![C*C/C or efi< for ![C*C/C
![CJ5/C or efi< for ![CJ5/C
J1T 2 *Eot mandatory #or the installation+/ thes !P! are
rePuired only i# you are using the associated J1T >ersion
MM
IIC8$C6 Patc& for $DX *.J.*.*/ -J5-bit.
IIG$C$$ Patc& for $DX *.A.5.* -/A-bit.
!M/@JC@ Patch #or S1T 1)B)7)7 *$7-&it+
!B" #DX *.@.C.C/ -!A/A - mi<ed mode. i% in%talled
EOT I;T
S4PPOT;1
Chec. Metalin. and certi#y #or last u(date on certi#ication
status)
Go en%ure t&at t&e %y%tem meet% t&e%e re)uirement%' follow t&e%e %te0%4
To determine
,hether an
!P! is
installed/ enter a
command %imilar
to t&e followin(4
;$is is an e9ample for AI5( 5.3< wit$ ;(&' 4
{node1:root}/ # ?usr?sbin?instfi9 -i -k AIB.C&.& IBC2&3' IBC5353
ID&1&:& ID&32:&A
6"" 8i"e0et0 8or 'I5-#5# <ere 8ound$
6"" 8i"e0et0 8or 'I-*#)+ <ere 8ound$
6"" 8i"e0et0 8or 'I-3)3) <ere 8ound$
6"" 8i"e0et0 8or 'X#1#,# <ere 8ound$
6"" 8i"e0et0 8or 'X#)*,# <ere 8ound$
{node1:root}/ #
!f an APAR i% not in%talled' download it from t&e followin( Web %ite and in%tall it4
&tt04www-D*5.ibm.come%erver%u00ortfi<e%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K8 o# $%$
16)$ System ePuirements *S,a(/ tem(/ memory/ internal dis.s+
Re7uirements to meet on A// nodes 555
!M VW C17 MB minimum
. Command to c&ec, t&e 0&y%ical memory 4 lsattr 8El sys9 8a realmem
{node1:root}/ # lsattr -/l sys& -a realmem
rea"mem )13%+*5 6mount o8 u0ab"e phy0ia" memory in Ubyte0 Aa"0e
{node1:root}/ #
{node2:root}/home # lsattr -/l sys& -a realmem
rea"mem )13%+*5 6mount o8 u0ab"e phy0ia" memory in Ubyte0 Aa"0e
{node*:root}/home #
Internal dis. VW 17 =B #or the oracle code *CSNDOM;/ !SMNDOM;/ O!C:;NDOM;+
This (art ,ill &e detailed in cha(ter ?ePuired local dis.s *Oracle Cluster,are/ !SM and !C so#t,are+@ 000
{node1:root}/ # df -k
Ai"e0y0tem 1#*3-b"o:0 Aree 9U0ed 'u0ed 9'u0ed Mounted on
/de7/hd3 *,*133 *#+51, *19 1)%-1 *)9 /
/de7/hd* 3+15%-* *,-+%*# 3)9 3,*#1 +9 /u0r
/de7/hd-7ar *,*133 *))+,5 119 %,% *9 /7ar
/de7/hd) 1)1#+*# 1*35%+, %9 *%% 19 /tmp
/de7/hd1 *,*133 *,1+,# 19 % 19 /home
/pro - - - - - /pro
/de7/hd1#opt %*3*55 *5)3## 3,9 %,,) -9 /opt
/de7/ora"e"7 1%+*5,3# 1%%#,%*3 *9 +) 19 /ora"e
/de7/r0"7 %*3*55# %1*3,-* )9 31 19 /r0
8a0-,#*:/7o"//o"(i0tribJSC *+,5*3#,3 +1#,5)53 +%9 133*+) *9 /di0trib
{node1:root}/ #
{node2:root}/home # df -k
/de7/hd3 *,*133 *#5%)* *19 1)%,3 *)9 /
/de7/hd* 3+15%-* *,-+),# 3)9 3,*#1 +9 /u0r
/de7/hd-7ar *,*133 *),#,# 1#9 35% 19 /7ar
/de7/hd) *#-+1%* *-,), --9 351* )%9 /tmp
/de7/hd1 *,*133 *,1+,# 19 % 19 /home
/pro - - - - - /pro
/de7/hd1#opt %*3*55 *5)3), 3,9 %,,) -9 /opt
/de7/ora"e"7 1%+*5,3# 1%%3##1* *9 ,3 19 /ora"e
/de7/r0"7 %*3*55# %11-3,# )9 3) 19 /r0
8a0-,#*:/7o"//o"(i0tribJSC *+,5*3#,3 +1#,5)-* +%9 133*+) *9 /di0trib
{node*:root}/home #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K% o# $%$
Paging s(ace W 7 ' !M/ wit& a minimum of ACC "B and a ma<imum of 5 2B.
Go c&ec, t&e 0a(in( %0ace confi(ured 4 lsps 8a
{node1:root}/ # lsps -a
Page Spae Phy0ia" /o"ume /o"ume &roup SiLe 9U0ed 6ti7e 6uto Type
hd, hdi0:# root7g %1*M2 + ye0 ye0 "7
{node1:root}/ #
{node2:root}/home # lsps -a
Page Spae Phy0ia" /o"ume /o"ume &roup SiLe 9U0ed 6ti7e 6uto Type
hd, hdi0:1 root7g %1*M2 * ye0 ye0 "7
{node*:root}/home #
Tem(orary 1is. S(ace 2 G&e Oracle Bniver%al !n%taller re)uire% u0 to ACC "B of free %0ace in t&e tm0 directory.
Go c&ec, t&e free tem0orary %0ace available4 d' 8% +tmp
{node1:root}/ # df -k ?tmp
/de7/hd) 1)1#+*# 1*35%+, %9 *%% 19 /tmp
{node1:root}/ #
{node2:root}/home # df -k ?tmp
/de7/hd) *#-+1%* *-,), --9 351* )%9 /tmp
{node*:root}/home #
Mou can u%e an ot&er file%y%tem in%tead of tm0.
$et t&e GE"P environment variable -u%ed by cr%' a%m' rdbm% u%er% andor oracle u%er.
and t&e G"PD!R environment variable to t&e new location.
For eample !
eport "#$%&'new(tmp
eport "$%D)R&'new(tmp
eport "$%&'new(tmp
11g!C/!SM/!I"
oraclibm@fr.ibm.com
86 o# $%$
16)B 4sers !nd =rou(s
7 o(tions (ossi&les
OPTIOE 1 OPTIOE 7
1 user #or each installation' for e<am0le 4
uni< cr% u%er for CR$6FO"E in%tallation
uni< a%m u%er for A$"6FO"E in%tallation
uni< rdbm% u%er for ORACLE6FO"E
in%tallation
1 user #or all installations/
for e<am0le oracle uni< u%er for 4
CR$6FO"E in%tallation
A$"6FO"E in%tallation
ORACLE6FO"E in%tallation
For the coo.&oo. (ur(ose/ and ease o# administration/ ,e-ll im(lement the #irst o(tion ,ith crs/ asm and
rd&ms users)
WePll al%o create an oracle u%er to own t&e oracle -UORACLE6BA$E. wic& will be %&ared by a%m -oraclea%m. and
rdbm% -oraclerdbm%..
G&i% oracle u%er -B!D9@CC. will be 0art of t&e followin( (rou0% 4 oin%tall' dba' o0er' a%m' a%mdba.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
81 o# $%$
This setu( has to &e done on all the nodes o# the cluster)
1e#ault user home #or crs/ asm and rd&ms users M4ST &e in /home/ other,ise you may ha>e trou&le
,ith ssh setu( *user ePui>alence+)
To create the #ollo,ing grou(s/ and users 2
Be sure that all the grou(s and user num&ers are identical thru the nodes)
On node1 M
m:group -P6P idOP%##P adm0OProotP oin0ta""
m:group -P6P idOP%#1P adm0OProotP r0
m:group -P6P idOP%#*P adm0OProotP dba
m:group -P6P idOP%#)P adm0OProotP oper
m:group -P6P idOP%#3P adm0OProotP a0m
m:group -P6P idOP%#%P adm0OProotP a0mdba
m:u0er idOP%##P pgrpOPoin0ta""P group0OPr0QdbaQoperQa0mQa0mdbaQ0ta88P
homeOP/home/ora"eP ora"e
m:u0er idOP%#1P pgrpOPoin0ta""P group0OPr0QdbaQoperQ0ta88P homeOP/home/r0P r0
m:u0er idOP%#*P pgrpOPoin0ta""P group0OPdbaQoperQa0mQa0mdbaQ0ta88P homeOP/home/a0mP
a0m
m:u0er idOP%#)P pgrpOPoin0ta""P group0OPdbaQoperQ0ta88P homeOP/home/rdbm0P rdbm0
TD;E On node7 M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
87 o# $%$
G&e crs asm and rd:ms u%er% mu%t &ave oinstall a% 0rimary (rou0' d:a a% %econdary (rou0%.
3eri#ication2 c&ec, if t&e file etc(rou0 contain% line% %uc& a% 4
*t+e numbers could be di,,erent)
{node1:root}/ # cat ?etc?+roup 2+rep crs
0ta88:Y:1:ip0eQ00hdQora"eQr0Qa0mQrdbm0
oin0ta"":Y:%##:ora"eQr0Qa0mQrdbm0
r0:Y:%#1:ora"eQr0
dba:Y:%#*:ora"eQr0Qa0mQrdbm0
oper:Y:%#):ora"eQr0Qa0mQrdbm0
a0m:Y:%#3:ora"eQa0m
a0mdba:Y:%#%:ora"eQa0m
{node1:root}/ #
{node1:root}/ # 0u - r0
{node1:r0}/r0 # id
uid05&1EcrsF +id05&&EoinstallF +roups01EstaffF<5&1EcrsF<5&2EdbaF<5&3EoperF
{node1:r0}/r0 #
{node1:root}/ # 0u - a0m
{node1:a0m}/ora"e/a0m # id
uid05&2EasmF +id05&&EoinstallF +roups01EstaffF<5&2EdbaF<5&3EoperF<5&5EasmF<5&5EasmdbaF
{node1:a0m}/ora"e/a0m #
{node1:root}/ # 0u - rdbm0
{node1:rdbm0}/ora"e/rdbm0 # id
uid05&3ErdbmsF +id05&&EoinstallF +roups01EstaffF<5&2EdbaF<5&3EoperF
{node1:rdbm0}/ora"e/rdbm0 #
{node1:root}/ # 0u - ora"e
{node1:ora"e}/ora"e # id
uid05&&EoracleF +id05&&EoinstallF
+roups01EstaffF<5&1EcrsF<5&2EdbaF<5&3EoperF<5&5EasmF<5&5EasmdbaF
{node1:ora"e}/ora"e #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8$ o# $%$
Set a (ass,ord to
crs user
asm user
rd&ms user
Set the same (ass,ord
#or all the nodes o# the
cluster/ ,ith the
#ollo,ing command 2
{node1:root}/ # pa00<d r0
Changing pa00<ord 8or Zr0Z
r0P0 Ne< pa00<ord:
!nter the ne< pa00<ord again:
{node1:root}/ #
{node1:root}/ # pa00<d a0m
Changing pa00<ord 8or Za0mZ
a0mP0 Ne< pa00<ord:
{node1:root}/ #
{node1:root}/ # pa00<d rdbm0
Changing pa00<ord 8or Zrdbm0Z
rdbm0P0 Ne< pa00<ord:
{node1:root}/ #
{node*:root}/ # pa00<d r0
Changing pa00<ord 8or Zr0Z
r0P0 Ne< pa00<ord:
{node*:root}/ #
{node*:root}/ # pa00<d a0m
Changing pa00<ord 8or Za0mZ
a0mP0 Ne< pa00<ord:
{node*:root}/ #
{node*:root}/ # pa00<d rdbm0
Changing pa00<ord 8or Zrdbm0Z
rdbm0P0 Ne< pa00<ord:
{node*:root}/ #
!nd connect at least
once to each node ,ith
each user *crs/ asm/
rd&ms+/ to >alidate the
(ass,ord/ as #irst
conne'ion ,ill as. to
change the (ass,ord 2
{node1:root}/ # te"net node1
;;$
U0er : r0
Pa00<ord : =====
;;
{node*:root}/ # te"net node*
;;$
U0er : r0
Pa00<ord : =====
;;
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8B o# $%$
16)C Ternel and Shell :imits
Con#iguring Shell :imits/ and System Con#iguration *;'tract #rom Oracle 1ocumentation+
Eote2
G&e 0arameter and %&ell limit value% %&own in t&i% %ection are minimum recommended
>alues only. 1or 0roduction databa%e %y%tem%' Oracle recommend% t&at you tune t&e%e value%
to o0timiLe t&e 0erformance of t&e %y%tem. Refer to your o0eratin( %y%tem documentation for
more information about tunin( ,ernel 0arameter%.
Oracle recommends that you 2
set shell limits #or root and oracle users/
set system con#iguration (arameters as descri&ed in this section on all cluster nodes #or root users)
10.5.1 Configure Shell Limits
3eri#y that the shell limits sho,n in t&e followin( table are %et to t&e value% %&own. G&e 0rocedure followin( t&e table
de%cribe% &ow to verify and %et t&e value%.
Shell :imit
*!s Sho,n in smit+
ecommended 3alue #or oracle user
In our case 2
crs user
asm user
rd&ms user
ecommended 3alue
#or root user
So#t FI:; si<e -* -Bnlimited. -* -Bnlimited.
So#t CP4 time -* -Bnlimited.
Eote2 G&i% i% t&e default value.
-* -Bnlimited.
Eote2 G&i% i% t&e default value.
So#t 1!T! segment -* -Bnlimited. -* -Bnlimited.
So#t ST!CT si<e -* -Bnlimited. -* -Bnlimited.
To >ie, the current >alue
s(eci#ied #or these shell
limits/ and to change
them i# necessary/
Follo, these ste(s2
1) ;nter the #ollo,ing command2
U smit chuser
2. In the B%er >A"E #ield/ enter the user name o# the Oracle so#t,are
o,ner/ #or e'am(le crs/ asm and rd&ms)
$) Scroll do,n the list and >eri#y that the >alue sho,n #or the so#t limits
listed in the (re>ious ta&le is -1)
!f nece%%ary' edit t&e e<i%tin( value.
B) 5hen you ha>e #inished ma.ing changes/ (ress F16 to e'it)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8C o# $%$
O #or root and oracle user on each node/ to chec. thru ulimit command 2
{node1:root}/ # ulimit -a
timeEsecondsF unlimited
fileEblocksF unlimited
dataEkbytesF unlimited
stackEkbytesF 51C53&5
memoryEkbytesF unlimited
coredumpEblocksF unlimited
nofilesEdescriptorsF unlimited
{node1:root}/
{node1:root}/ # su - crs
{node1:r0}/r0/11$1$# # ulimit -a
{node1:r0}/r0/11$1$# #
{node1:root}/ # su - asm
{node1:a0m}/ora"e/a0m/11$1$# # ulimit -a
{node1:a0m}/ora"e/a0m/11$1$# #
{node1:root}/ # su - rdbms
{node1:rdbm0}/ora"e/rdbm0/11$1$# # ulimit -a
{node1:rdbm0}/ora"e/rdbm0/11$1$# #
Add t&e followin( line% to t&e etc%ecuritylimit% file on eac& node 4 Do %et unlimited wit& 3-*T
default4
f%iLe 9 -*
core 9 -*
c0u 9 -*
data 9 -*
r%% 9 -*
%tac, 9 -*
nofile% 9 -*
Or %et at minimum Oracle %0ecified value% a% follow 4
default4
f%iLe 9 -*
core 9 -*
c0u 9 -*
data 9 @*5CCC
r%% 9 @*5CCC
%tac, 9 @*5CCC
nofile% 9 5CCC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8G o# $%$
10.5.2 Set crs, asm and rdbms users capabilities
!dd ca(a&ilities C!PNE4M!N!TT!C/ C!PNBIP!SSN!CN3MM/ C!PNPOP!=!T; to users crs/ asm/ rd&ms
and oracle)
Go %et cr%' a%m and rdbm% u%er 3ca0abilitie%T on eac& node 4
{node1:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ crs
{node1:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ asm
{node1:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ rdbms
{node1:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ oracle
{node2:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ crs
{node*:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ asm
{node*:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ rdbms
{node1:root}/ # c$user capabilities0CA",#6*A,A;;ACG<CA",)B"ASS,HAC,I**<CA","HJ"A7A;/ oracle
Go c&ec, u%er 3ca0abilitie%T on eac& node' wit& cr% u%er for e<am0le 4
{node1:root}/r0/11$1$#/bin # lsuser -f crs 2 +rep capabilities
apabi"itie0OC6P?NUM6?6TT6CNQC6P?2IP6SS?B6C?/MMQC6P?PB.P6&6T!
{node2:root}/r0/11$1$#/bin # lsuser -f crs 2 +rep capabilities
apabi"itie0OC6P?NUM6?6TT6CNQC6P?2IP6SS?B6C?/MMQC6P?PB.P6&6T!
{node*:root}/r0/11$1$#/bin #
10.5.3 Set NCARGS parameter
Change (arameters ?ncargs@to 178
Go %et 3ncar(%T attribute on eac& node 4
{node1:root}/ # c$dev -l sys& -a ncar+s012.
0y0# hanged
{node1:root}/
{node2:root}/ # c$dev -l sys& -a ncar+s012.
0y0# hanged
{node*:root}/
Go c&ec, 3ncar(%T attribute on eac& node 4
{node1:root}/r0/11$1$#/bin # lsattr -/l sys& -a ncar+s
narg0 1*5 6B&/!N/ "i0t 0iLe in 3U byte b"o:0 True
{node2:root}/r0/11$1$#/bin # lsattr -/l sys& -a ncar+s
narg0 1*5 6B&/!N/ "i0t 0iLe in 3U byte b"o:0 True
{node*:root}/r0/11$1$#/bin #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8K o# $%$
10.5.4 Configure System Configuration Parameters
3eri#y that the ma'imum num&er o# (rocesses allo,ed (er user is set to 1G$8B or greater 2
Eote2
1or 0roduction %y%tem%' t&i% value %&ould be at lea%t *5I 0lu% t&e %um of t&e PROCE$$E$ and
PARALLEL6"A+6$ERNER$ initialiLation 0arameter% for eac& databa%e runnin( on t&e %y%tem.
To chec.
the >alue
M
{node1:root}/ # lsattr -/l sys& -a ma9uproc
ma9uproc 1&25 Ma1imum number o8 PB.C!SS!S a""o<ed per u0er True
{node1:root}/ #
To edit
and
modi#y the
>alue M
Settin+ value to 1:3.5
{node1:root}/ # c$dev -l sys& -a ma9uproc0K1:3.5K
{node1:root}/ #
{node1:root}/ # lsattr -/l sys& -a ma9uproc
ma9uproc 1:3.5 Ma1imum number o8 PB.C!SS!S a""o<ed per u0er True
{node1:root}/ #
JH
1) ;nter the #ollo,ing command2
U smit chgsys
2. 3eri#y that the >alue sho,n #or Ma'imum num&er o# POC;SS;S allo,ed (er user
is greater than or ePual to 1G$8B)
!f nece%%ary' edit t&e e<i%tin( value.
$) 5hen you ha>e #inished ma.ing changes/ (ress F16 to e'it)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
88 o# $%$
10.5.5 Lru_file_repage setting
3eri#y that the lruN#ileNre(age (arameter is set to 6 2
G&e default value i% `*`' &ut it is recommended to set this to X6X. G&i% %ettin( &int% to t&e N"" to only %teal file
0a(e% -from t&e A!+ file buffer cac&e. and leave t&e com0utational 0a(e% -from t&e $2A. alone.
By %ettin( `lru6file6re0a(e9C`' A!+ only free% file cac&e memory. G&i% (uarantee% wor,in( %tora(e %tay% in memory'
and allow% file cac&e to (row.
$o in t&e 0a%t you mi(&t &ave %et ma<client at 5CY for databa%e %erver%. Goday you could %et ma<client at DCY and
`lru6file6re0a(e9C`. G&e e<act %ettin( will vary ba%ed on your a00lication and amount of memory. Contact !B" $u00ort
if you need &el0 determinin( t&e o0timum %ettin(.
This ne, lruN#ileNre(age (arameter is only a>aila&le on !I" C)7 M:6BJ and !I" C)$ M:61J
To chec. the >alue on each node M
{node1:root}/ # vmo -( lru,file,repa+e
N6M! CUB (!A 2..T M'N M6K UN'T TIP!
(!P!N(!NC'!S
--------------------------------------------------------------------------------
"ru?8i"e?repage 1 1 1 # 1 boo"ean (
--------------------------------------------------------------------------------
{node1:root}/ #
Change the >alue to 6 on each node 2
{node1:root}/ # vmo -p -o lru,file,repa+e0&
Settin+ lru,file,repa+e to & in ne9tboot file
Settin+ lru,file,repa+e to &
{node1:root}/ #
{node1:root}/ # vmo -( lru,file,repa+e
N6M! CUB (!A 2..T M'N M6K UN'T TIP!
(!P!N(!NC'!S
--------------------------------------------------------------------------------
"ru?8i"e?repage & 1 & # 1 boo"ean (
--------------------------------------------------------------------------------
{node1:root}/ #
Ty(ical >mo settings #or Oracle 2
? lru6file6re0a(e9C -default9*. -A!+ @.5 "LCA or later.
a 1orce% file 0a(e% to be re0a(ed before com0utational 0a(e%
? min0ermY9@ -default 5C.
a Gar(et for minimum Y of 0&y%ical memory to be u%ed for file %y%tem cac&e
- ma<0ermY9DC - default IC.
- strictBmaDperm F 1
- strictBmaDclient F 1

11g!C/!SM/!I"
oraclibm@fr.ibm.com
8% o# $%$
Content o#
etctunable%ne<tboot
For node1 2
=node14root>? @ cat ?etc?tunables?ne9tboot
Z/et/tunab"e0/ne1tbootZ 3+ "ine0Q 1##1 harater0
# '2M?PB.H.&?2!&'N?T6&
# Thi0 i0 an automatia""y generated pro"og$
#
# bo0%)# 0r/bo0/u0r/0bin/per8/tune/ne1tboot 1$1
#
# Hien0ed Materia"0 - Property o8 '2M
#
# FCG C.PIB'&NT 'nternationa" 2u0ine00 Mahine0 Corp$ *##*
# 6"" Bight0 Be0er7ed
#
# US &o7ernment U0er0 Be0trited Bight0 - U0eQ dup"iation or
# di0"o0ure re0trited by &S6 6(P Shedu"e Contrat <ith '2M Corp$
#
# '2M?PB.H.&?!N(?T6&
7mo:
0trit?ma1perm O Z#Z
minperm9 O Z%Z
ma1perm9 O Z-#Z
ma1"ient9 O Z-#Z
"ru?8i"e?repage O Z#Z
0trit?ma1"ient O Z1Z
min8ree O Z)###Z
ma18ree O Z3###Z
n80o:
n80?r81)*) O Z1Z
no:
ipJma1"en O Z%1*Z
udp?0end0pae O Z,%%),Z
udp?re70pae O Z,%%),#Z
tp?0end0pae O Z*,*133Z
tp?re70pae O Z*,*133Z
0b?ma1 O Z1)1#+*#Z
r81)*) O Z1Z
ioo:
p7?min?pbu8 O Z%1*Z
num80bu80 O Z*#35Z
ma1rand<rt O Z)*Z
ma1pgahead O Z1,Z
"7m?bu8nt O Z,3Z
W*?n2u88erPerPager(e7ie O Z1#*3Z
W*?ma1Bandom4rite O Z)*Z
W*?ma1PageBead6head O Z1*5Z
W*?dynami2u88erPrea""oation O Z*%,Z
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%6 o# $%$
10.5.6 Asynchronous I/O setting
Setting !synchronous I/O 2
To chec.
the >alue
on each
node M
{node1:root}/ # smitty aio
Select c$an+e ? S$ow C$aracteristics of Async$ronous I?J
Set S;A;/ to LavailableM
JH
{node1:root}/ # lsattr -/l aio&
autoon8ig available S;A;/ to be confi+ured at system restart True
8a0tpath enab"e State o8 8a0t path True
:proprio )- Ser7er PB'.B'TI True
ma1reJ0 1,)53 Ma1imum number o8 B![U!STS True
ma10er7er0 )## M6K'MUM number o8 0er7er0 per pu True
min0er7er0 1%# M'N'MUM number o8 0er7er0 True
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%1 o# $%$
16)G 4ser ePui>alences
Be#ore installing Oracle cluster,are/ !SM so#t,ate and eal !((lication clusters so#t,are' you mu%t confi(ure
u%er e)uivalence for t&e crs/ asm and rd&ms u%er% on all clu%ter node%.
Iou ha>e t,o ty(e o# 4ser ;Pui>alence im(lementation 2
R$F -Remote %&ell.
$$F -$ecured $&ell.
5hen SSD is not a>aila&le/ the Installer uses the rsh and rc( commands
instead o# ssh and sc() .
Iou ha>e to choose one or the other/ &ut don-t im(lement &oth at the same time)
4sually/ customers ,ill im(lement SSD) !E1 i# SSD is started and used/ do con#igure SSD
On the (u&lic node name returned &y the !I" command ?hostname@ 2
node1 must ssh or rsh to node1/ as crs/ asm and rd&ms user)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%7 o# $%$
10.6.1 RSH implementation
Set u( user ePui>alence #or the oracle and root account/ to ena&le rsh/ rc(/ rlogin commands.
Iou should ha>e the entries on each node for 4 etc&o%t%' etc&o%t%.e)uiv and on rootoracle &ome directory
UFO"E.r&o%t%.
/etc/hosts)ePui>
4(date/chec. entries in hosts)ePui>
#ile
on each node
{node1:root}/ # pg /et/ho0t0$eJui7
$$$$
node1 root
node* root
node1 r0
node* r0
$$$$
{node1:root}/ #
YDOM;/)rhosts
4(date/chec. entries in )rhosts #ile on
each node #or root user 2
4(date/chec. entries in )rhosts #ile on
each node #or crs/ asm and rd&ms user 2
{node1:root}/ # su 1 root
{node1:root}/ # d
{node1:root}/ # p+3GJ*/?.r$osts
node1 root
node2 root
{node1:root}/ #
{node1:root}/ #
{node1:r0}/r0 # pg $rho0t0
node1 root
node* root
node1 r0
node* r0
{node1:r0}/r0 #
{node1:a0m}/ora"e/a0m # pg $rho0t0
node1 root
node* root
node1 a0m
node* a0m
{node1:root}/ # 0u - rdbm0
{node1:rdbm0}/ora"e/rdbm0 # pg $rho0t0
node1 root
node* root
node1 rdbm0
node* rdbm0
Eote 2 ,t is possi'le- 'ut not a.vise. 'ecause o* security reasons- to put a /01 in hosts+e2uiv an. +rhosts *iles+
Test i# the user ePui>alence is
correctly set u( *node7 is the
secondary cluster machine+)
Iou are logged on node1 as root 2
Test #or crs/ asm and rd&ms users 000
{node1:root}/ # rsh node7 -9^ no 0a%%word.
{node*:root}/ # rc( /tm(/toto node12/tm(/toto
{node*:root}/ # su - crs
{node*:r0}/r0/11$1$# # r%& node* date
Mon !(r 7$ 1K27G27K 1FT 766K
{node*:r0}/r0/11$1$# #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%$ o# $%$
10.6.2 SSH implementation
Before you in%tall and u%e Oracle Real A00lication clu%ter%' you mu%t confi(ure %ecure %&ell -$$F. for t&e ocrs/ asm
and rd&ms u%er% on all clu%ter node%. Oracle Bniver%al !n%taller u%e% t&e ssh and sc( command% durin( in%tallation to
run remote command% on and co0y file% to t&e ot&er clu%ter node%. Mou mu%t confi(ure $$F %o t&at t&e%e command%
do not 0rom0t for a 0a%%word.
Eote2
G&i% %ection de%cribe% &ow to confi(ure O0en$$F ver%ion J. I# SSD is not a>aila&le/ then
Oracle 4ni>ersal Installer attem(ts to use rsh and rc( instead.
Go determine if $$F i% runnin(' enter t&e followin( command4
Y (s -e# Z gre( sshd
!f $$F i% runnin(' t&en t&e re%0on%e to t&i% command i% 0roce%% !D number%. Go find out more
about $$F' enter t&e followin( command4
Y man ssh
For each user/ crs/ asm and rd&ms re(eat the ne't ste(s 2
Con#iguring SSD on Cluster Mem&er Eode s
Go confi(ure $$F' you mu%t fir%t create R$A and D$A ,ey% on eac& clu%ter node' and t&en co0y t&e ,ey% from all
clu%ter node member% into an aut&oriLed ,ey% file on eac& node. Go do t&i% ta%,' com0lete t&e followin( %te0%4
Create RSA and &SA
%eys on each
node" Com(lete the
#ollo,ing ste(s on
each node2
1. Lo( in a% t&e oracle u%er.
2. !f nece%%ary' create t&e )ssh directory in t&e oracle u%erR% &ome directory and %et
t&e correct 0ermi%%ion% on it4
Y m.dir [/)ssh
Y chmod K66 [/)ssh
J. Enter t&e followin( command% to (enerate an R$A ,ey for ver%ion 5 of t&e $$F
0rotocol4
Y /usr/&in/ssh-.eygen -t rsa
A. At t&e 0rom0t%4
Acce0t t&e default location for t&e ,ey file.
Enter and confirm a 0a%% 0&ra%e t&at i% different from t&e oracle u%erR% 0a%%word.
This command ,rites the (u&lic .ey to the [/)ssh/idNrsa)(u& #ile and the
(ri>ate .ey to the [/)ssh/idNrsa #ile) Ee>er distri&ute the (ri>ate .ey to
anyone)
Enter t&e followin( command% to (enerate a D$A ,ey for ver%ion 5 of t&e $$F
0rotocol4
Y /usr/&in/ssh-.eygen -t dsa
@. At t&e 0rom0t%4
Acce0t t&e default location for t&e ,ey file
Enter and confirm a 0a%% 0&ra%e t&at i% different from t&e oracle u%erR% 0a%%word
6his command -rites the pu:lic %ey to the ;+1ssh+id<dsa1pu: 'ile and the pri.ate
%ey to the ;+1ssh+id<dsa 'ile1 4e.er distri:ute the pri.ate %ey to anyone1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%B o# $%$
!dd .eys to an
authori<ed .ey
#ile2 Com(lete the
#ollo,ing ste(s2
Note-
Reeat this
rocess for each
node in the cluster
AAA
1. On t&e local node' determine if you &ave an aut&oriLed ,ey file
-[/)ssh/authori<edN.eys.. !f t&e aut&oriLed ,ey file already e<i%t%' t&en 0roceed
to %te0 5. Ot&erwi%e' enter t&e followin( command%4
Y touch [/)ssh/authori<edN.eys
Y cd [/)ssh
Y ls
Mou %&ould %ee t&e idNdsa)(u& and idNrsa)(u& ,ey% t&at you &ave created.
2. B%in( $$F' co0y t&e content% of t&e [/)ssh/idNrsa)(u& and [/)ssh/idNdsa)(u&
file% to t&e file [/)ssh/authori<edN.eys' and 0rovide t&e Oracle u%er 0a%%word
a% 0rom0ted. G&i% 0roce%% i% illu%trated in t&e followin( %ynta< e<am0le wit& a
two-node clu%ter' wit& node% node* and node5' w&ere t&e Oracle u%er 0at& i%
/home/oracle2
\oracleAnode1 )ssh]Y ssh node1 cat /home/oracle/)ssh/idNrsa)(u& VV authori<edN.eys
oracleAnode1Qs (ass,ord2
\oracleAnode1 )ssh]Y ssh node1 cat /home/oracle/)ssh/idNdsa)(u& VV authori<edN.eys
\oracleAnode1 )sshY ssh node7 cat /home/oracle/)ssh/idNrsa)(u& VV authori<edN.eys
\oracleAnode1 )sshY ssh node7 cat /home/oracle/)ssh/idNdsa)(u& VVauthori<edN.eys
. B%e $CP -$ecure Co0y. or $1GP -$ecure 1GP. to co0y t&e authori<edN.eys file
to t&e Oracle u%er .%%& directory on a remote node. G&e followin( e<am0le i% wit&
$CP' on a node called node5' w&ere t&e Oracle u%er 0at& i% /home/oracle2
\oracleAnode1 )ssh]sc( authori<edN.eys node72/home/oracle/)ssh/
A. Re0eat %te0 5 and J for eac& clu%ter node member. W&en you &ave added ,ey%
from eac& clu%ter node member to t&e aut&oriLed6,ey% file on t&e la%t node you
want to &ave a% a clu%ter node member' t&en u%e $CP to co0y t&e com0lete
aut&oriLed6,ey% file bac, to eac& clu%ter node member
Eote2
G&e Oracle u%erR% /)ssh/authori<edN.eys file on every node mu%t
contain t&e content% from all of t&e /)ssh/idNrsa)(u& and
/)ssh/idNdsa)(u& file% t&at you (enerated on all clu%ter node%.
'. C&an(e t&e 0ermi%%ion% on t&e Oracle u%erR% /)ssh/authori<edN.eys file on all
clu%ter node%4
B chmod 6,, C$.ssh$authoriDedE=e0s
!t this (oint/ i# you use ssh to log in to or run a command on another node/ you are (rom(ted #or the (ass
(hrase that you s(eci#ied ,hen you created the 1S! .ey)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%C o# $%$
;na&ling SSD 4ser ;Pui>alency on Cluster Mem&er Eodes
To ena&le Oracle
4ni>ersal Installer
to use the ssh and
sc( commands
,ithout &eing
(rom(ted #or a
(ass (hrase'
#ollo, these ste(s2
1. On the system ,here you ,ant to run Oracle 4ni>ersal Installer/ log in as the
oracle user)
7) ;nter the #ollo,ing commands2
Y e'ec /usr/&in/ssh-agent YSD;::
Y /usr/&in/ssh-add
$) !t the (rom(ts/ enter the (ass (hrase #or each .ey that you generated)
!f you &ave confi(ured $$F correctly' t&en you can now u%e t&e ssh or sc(
command% wit&out bein( 0rom0ted for a 0a%%word or a 0a%% 0&ra%e.
$. I# you are on a remote terminal' and t&e local node &a% only one vi%ual -w&ic& i%
ty0ical.' t&en u%e t&e followin( %ynta< to %et t&e D!$PLAM environment variable4
Bourne/ Torn/ and Bash shells
Y e'(ort 1ISP:!IWhostname26
C shell2
Y seten> 1ISP:!I 6
1or e<am0le' if you are u%in( t&e Ba%& %&ell' and if your &o%tname i% node*' t&en enter
t&e followin( command4
Y e'(ort 1ISP:!IWnode126
'. To test the SSD con#iguration' enter t&e followin( command% from t&e %ame
terminal %e%%ion' te%tin( t&e confi(uration of eac& clu%ter node' w&ere nodename*'
nodename5' and %o on' are t&e name% of node% in t&e clu%ter4
Y ssh nodename1 date
Y ssh nodename7 date
These commands should dis(lay the date set on each node)
!f any node 0rom0t% for a 0a%%word or 0a%% 0&ra%e' t&en verify t&at t&e
[/)ssh/authori<edN.eys file on t&at node contain% t&e correct 0ublic ,ey%.
!f you are u%in( a remote client to connect to t&e local node' and you %ee a me%%a(e
%imilar to `Warnin(4 >o <aut& datab u%in( fa,e aut&entication data for +** forwardin('`
t&en t&i% mean% t&at your aut&oriLed ,ey% file i% confi(ured correctly' but your %%&
confi(uration &a% +** forwardin( enabled. Go correct t&i%' 0roceed to %te0 /.
Eote2
G&e fir%t time you u%e $$F to connect to a node from a 0articular %y%tem'
you may %ee a me%%a(e %imilar to t&e followin(4
G&e aut&enticity of &o%t Rnode* -*AC.IE.*@5.*@J.R canRt be e%tabli%&ed.
R$A ,ey fin(er0rint i% EL4eL4eE4f/4fA4f54Af4If4DL4ED4I@4/545C4DC4D54LD.
Are you %ure you want to continue connectin( -ye%no.8
Enter yes at t&e 0rom0t to continue. Mou %&ould not %ee t&i% me%%a(e a(ain
w&en you connect from t&i% %y%tem to t&at node.
!f you %ee any ot&er me%%a(e% or te<t' a0art from t&e date' t&en t&e
in%tallation can fail. "a,e any c&an(e% re)uired to en%ure t&at only t&e date
i% di%0layed w&en you enter t&e%e command%.
Mou %&ould en%ure t&at any 0art% of lo(in %cri0t% t&at (enerate any out0ut' or
a%, any )ue%tion%' are modified %o t&at t&ey act only w&en t&e %&ell i% an
interactive %&ell.
G) To ensure that "11 #or,arding ,ill not cause the installation to #ail/ create a
user-le>el SSD client con#iguration #ile #or the Oracle so#t,are o,ner user/ as
#ollo,s2
a. B%in( any te<t editor' edit or create t&e [oracle/)ssh/con#ig file.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%G o# $%$
b. "a,e %ure t&at t&e 1orward+** attribute i% %et to no. 1or e<am0le4
Fo%t Z
1orward+** no
K) Iou must run Oracle 4ni>ersal Installer #rom this session or remem&er to
re(eat ste(s 7 and $ &e#ore you start Oracle 4ni>ersal Installer #rom a
di##erent terminal session)
Pre>enting Oracle Cluster,are Installation ;rrors Caused &y stty Commands
Durin( an Oracle Clu%terware in%tallation' Oracle Bniver%al !n%taller u%e% $$F -if available. to run command% and co0y
file% to t&e ot&er node%. Durin( t&e in%tallation' &idden file% on t&e %y%tem -for e<am0le' .ba%&rc or .c%&rc. will cau%e
in%tallation error% if t&ey contain %tty command%.
To a>oid this (ro&lem/
you must modi#y
these #iles to
su((ress all out(ut
on ST1;/ as in the
#ollo,ing
e'am(les2
Bourne/ Bash/ or Torn shell2
if c -t C db t&en
%tty intr eC
fi
C shell2
te%t -t C
if -U%tatu% 99 C. t&en
%tty intr eC
endif
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%K o# $%$
16)K Time Ser>er Synchroni<ation
Time Synchronisation
M!E1!TOI
To ensure that !C o(erates
e##iciently/ you must synchroni<e
the system time on all cluster
nodes)
Oracle recommend% t&at you u%e
<nt0d for t&i% 0ur0o%e. <nt0d i% a
com0lete im0lementation of t&e
>etwor, Gime Protocol ->GP. ver%ion
J %tandard and i% more accurate t&an
timed.
To con#igure 'nt(d/ #ollo, these ste(s on each cluster node 2
* Enter t&e followin( command to
create re)uired file%' if ece%%ary4
U touch /etc/nt()dri#t /etc/nt()trace /etc/nt()con#
5 B%in( any te<t editor' edit t&e
etcnt0.conf file4
U >i /etc/nt()con#
J Add entrie% %imilar to t&e
followin( to t&e file4
0er7er ip<address1
0er7er ip<address#
0er7er ip<address=
; $am0le >GP Confi(uration file
; $0ecify t&e !P Addre%%e% of t&ree cloc, %erver %y%tem%.
timeser>er1 16)$)7C)161
timeser>er7 16)$)7C)167
timeser>er$ 16)$)7C)16$
; "o%t of t&e router% are broadca%tin( >GP time information. !f your
; router i% broadca%tin(' t&en t&e followin( line enable% <nt0d
; to li%ten for broadca%t%.
broadca%tclient
; Write cloc, drift 0arameter% to a file. G&i% enable% t&e %y%tem
; cloc, to )uic,ly %yc&roniLe to t&e true time on re%tart.
driftfile etcnt0.drift
tracefile etcnt0.trace
A Go %tart <nt0d' follow t&e%e
%te0%4
A - Enter t&e followin( command4 ; /usr/&in/smitty 'nt(d
B - C&oo%e Start 4sing the 'nt(d Su&system' t&en c&oo%e BOTD.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%8 o# $%$
16)8 crs en>ironment setu(
Oracle environment 4 YDOM;/)(ro#ile file in OracleP% &ome directory
To &e done on
each node)
e1port .B6CH!?26S!O/ora"e
e1port 6'KTNB!6(?SC.P!OS (S for system-wide thread scope)
e1port T!MPO/tmp
e1port TMPO/tmp
e1port TMP('BO/tmp
uma0: #**
i8 > -t # @V then
0tty intr RC
8i
Eotes2
On A!+' w&en u%in( multit&readed a00lication% or LA>-free' e%0ecially w&en runnin( on mac&ine% wit& multi0le
CPB%' we %tron(ly recommend %ettin( A!+GFREAD$COPE9$ in t&e environment before %tartin( t&e a00lication'
for better 0erformance and more %olid %c&edulin(.
1or e<am0le4
;"POT !I"TD;!1SCOP;WS
$ettin( A!+GFREAD6$COPE9$ mean% t&at u%er t&read% created wit& default attribute% will be 0laced into
%y%tem-wide contention %co0e. !f a u%er t&read i% created wit& %y%tem-wide contention %co0e' it i% bound to a
,ernel t&read and it i% %c&eduled by t&e ,ernel. G&e underlyin( ,ernel t&read i% not %&ared wit& any ot&er u%er
t&read.
!I"TD;!1NSCOP; *!I" B)$)1 and later+
Purpose"
Control% contention %co0e. P %i(nifie% 0roce%%-ba%ed contention %co0e -"4>.. $ %i(nifie% %y%tem-ba%ed
contention %co0e -*4*..
,alues"
Po%%ible Nalue%4 P or $. G&e default i% P.
&isplay"
ec&o UA!+GFREAD6$COPE -t&i% i% turned on internally' %o t&e initial default value will not be %een wit& t&e
ec&o command.
Change"
A!+GFREAD6$COPE9VP\$We<0ort A!+GFREAD6$COPE C&an(e ta,e% effect immediately in t&i% %&ell.
C&an(e i% effective until lo((in( out of t&i% %&ell. Permanent c&an(e i% made by addin( t&e
A!+GFREAD6$COPE9VP\$W command to t&e etcenvironment file.
&iagnosis"
!f fewer t&read% are bein( di%0atc&ed t&an e<0ected' t&en %y%tem %co0e %&ould be tried.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%% o# $%$
16)% Oracle So#t,are ePuirements
Lin, to donwload code from 4 &tt04otn.oracle.com
&tt04www.oracle.comtec&nolo(y%oftware0roduct%databa%eoracle*C(&tdoc%*C5C*ai<%oft.&tml
Orale C1-s needed
#or the !C
installation
Oracle Databa%e **g Relea%e * -**.*.C./.C. Enter0ri%e$tandard Edition for A!+@L-Di%,*
&tt04download.oracle.comotnai<oracle**(ai<.00c/A6**(R*6databa%e6di%,*.Li0
-/@C'A5D'/D5 byte%. -c,%um - 5II@@EADDJ.
Oracle Databa%e **g Relea%e * -**.*.C./.C. Enter0ri%e$tandard Edition for A!+@L-Di%,5
&tt04download.oracle.comotnai<oracle**(ai<.00c/A6**(R*6databa%e6di%,5.Li0
-*'I/E'5I*'E/@ byte%. -c,%um - 5@JJJAJD/D.
Cluster,are and
1ata&ase
Patchset needed
Best is to
im(lement last
(atchset #or Oracle
Cluster,are and
!SM so#t,are
!((lication o# (atchset to the data&ase so#t,are de(ends i# your a((lication has &een
tested ,ith it/ it-s your choice/ your desision)
!f available' and A% needed for your 0roKect ...
;'tra Cluster,are
Patches needed
;'tra !SM
Patches needed
&$&+&&& Oracle 3atabase Famil0E (atc*
5IG5 R489!8A!CG 8IS@GR29(S 82 !26 62A4RA64 A2SS 2< 8IS@S I! 6,2
<AIAGR29(S
&$.$.&1 Oracle 3atabase Famil0E (atc*
ASM 8IS@GR29( M29!6 I!C9RS 2RAB&11 @<C4M@11
;'tra 1ata&ase
Patches needed
1irections to e'tract contents o# )g< #iles 2
*. BnLi0 t&e file4 gun<i( ]filename^
5. E<tract t&e file4 c(io -idcm> ^ ]filename^
J. !n%tallation (uide% and (eneral Oracle Databa%e *Cg documentation can be found &ere.
A. Review t&e certification matri< for t&i% 0roduct &ere.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
166 o# $%$
11 P;P!IE= STO!=;
$tora(e i% to be 0re0ared for 4
Local di%,% on eac& node to in%tall Oracle Clu%terware' A$" and RAC %oftware% M!E1!TOI
$&ared di%,% for Oracle clu%terware di%,% M!E1!TOI
11g!C/!SM/!I"
oraclibm@fr.ibm.com
161 o# $%$
$&ared di%, for Oracle A$" !n%tance% 0arameter% -$P1!LE. ecommended &ut not M!E1!TOI
$&ared di%,% for Oracle A$" di%,% M!E1!TOI
11g!C/!SM/!I"
oraclibm@fr.ibm.com
167 o# $%$
11)1 ePuired local dis.s *Oracle Cluster,are/ !SM and !C so#t,are+
G&e oracle code -clu%terware' A$" and RAC. can be located on an internal di%, and 0ro0a(ated on t&e ot&er mac&ine%
of t&e clu%ter. G&e Oracle Bniver%al !n%taller mana(e t&e clu%ter-wide in%tallation' t&at i% done only once. Re(ular file
%y%tem% are u%ed for Oracle code.
EOT! 2 Iou can also use >irtual I/O dis.s #or 2
A!+@L o0eratin( %y%tem
Oracle clu%terware -UCR$6FO"E.
Oracle A$" -UA$"6FO"E.
Oracle RAC $oftware -UORACLE6FO"E.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16$ o# $%$
On each node/ create a >olume grou( ?oracle>g@/ or use a>aila&le s(ace #rom _root>g- to create the
:ogical 3olumes M
6o list the internal dis%s "
>node1"root?+ @ lsde. 8Ccdis% A grep SCSI
+dis-. A/ailable 0irtual SCS) Dis- Dri/e
Create a .olume group called oracle.g "
>node1"root?+ @ m%.g B' ByCoracle.gC BS hdis%9
On each node ?/crs@ must &e created to host 2
Oracle Cluster,are So#t,are
?/crs@ must &e a&out B to G =B si<e)
?/crs@ must &e o,ned &y oracle2oinstall or &y crs2oinstall
On node 1 M
Create a DGB 'ile system +crs in the pre.ious .olume group 2large 'ile ena:led3 "
>node1"root?+ @ cr's B. E's# BgCoracle.gC Ba siFeG0DG0 BmC+crsC BACyesC BpCr-C
1node2!root3' 4 mount +crs
1node2!root3' 4 cho-n 8R crs"oinstall +crs
TD;E On node 7 M
On each node ?/oracle@ must &e created to host 2
Oracle !utomated Storage Management *!SM+ on
?/oracle/asm@
Oracle !C so#t,are on ?/oracle/rd&ms@
?/oracle@ must &e a&out 16 to 17 =B si<e)
?/oracle@ must &e o,ned &y oracle2oinstall
?/oracle/asm@ must &e o,ned &y asm2oinstall or oracle2oinstall
?/oracle/rd&ms@ must &e o,ned &y rd&ms2oinstall or oracle2oinstall
On node 1 M
Create a DGB 'ile system +cr in the pre.ious .olume group 2large 'ile ena:led3 "
>node1"root?+ @ cr's B. E's# BgCoracle.gC Ba siFeGC1#GC BmC+oracleC BACyesC BpCr-C
1node2!root3' 4 mount +oracle
1node2!root3' 4 cho-n oracle"oinstall +oracle
1node2!root3' 4 cho-n 8R asm"oinstall +oracle+asm
1node2!root3' 4 cho-n 8R rd:ms"oinstall +oracle+rd:ms
6(E4 On node # $ TD;E do the same #or each e'tra node)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16B o# $%$
11)7 Oracle Cluster,are 1is.s *OC and 3oting 1is.s+
The Oracle Cluster egistry *OC+ %tore% clu%ter and databa%e confi(uration information. Mou mu%t &ave a %&ared
raw device containin( at lea%t 5@/ "B of free %0ace t&at i% acce%%ible from all of t&e node% in t&e clu%ter.
OCR 0rotected by e<ternal mec&ani%m OR' OCR mirrored by Oracle Clu%terware
QThe Oracle Cluster,are >oting dis. contain% clu%ter member%&i0 information and arbitrate% clu%ter owner%&i0
amon( t&e node% of your clu%ter in t&e event of networ, failure%. Mou mu%t &ave a %&ared raw device containin( at
lea%t 5@/ "B of free %0ace t&at i% acce%%ible from all of t&e node% in t&e clu%ter.
Notin( di%, 0rotected by e<ternal
mec&ani%m
OR' votin( di%,% 0rotected by Oracle Clu%terware.
!tP% alway% even co0ie%' *' J' @ and %o on S J i% %ufficient in mo%t ca%e%.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16C o# $%$
11.2.1 Required LUNs
B%in( t&e %tora(e admini%tration con%ole' you &ave to create 4
Eit&er
5 LB>% for OCR di%,% -JCC"B %iLe.
J LB>% for Notin( di%,% -JCC"B %iLe.
to im0lement normal redundancy of Oracle clu%terware' to 0rotect OCR and Notin( di%,%.
or
* LB>% for OCR di%,% -JCC"B %iLe.
* LB>% for Notin( di%,% -JCC"B %iLe.
to im0lement e<ternal redundancy of Oracle clu%terware' and let t&e %tora(e mirrorin( mec&ani%m to 0rotect OCR
and Notin( di%,%.
G&e followin( %creen %&ow% t&e LB> ma00in( for node% u%ed in our clu%ter. G&e LB> for OCR di%,% and Notin( di%,%
&ave id% * to @. G&e%e !DP% will &el0 u% on to identify w&ic& &di%, will be u%ed.
1is.s :4E-s I1 Eum&er :4E-s Si<e
OCR* L1 JCC "B
OCR5 L7 JCC "B
Notin(* L$ JCC "B
Notin(5 LB JCC "B
Notin(J LC JCC "B
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16G o# $%$
11.2.2 How to Identify if a LUN is used or not ?
=et the :ist o# the hdis.s on node1/ #or e'am(le 2
:
i
s
t
o
#
a
>
a
i
l
a
&
l
e

h
d
i
s
.
s

o
n

n
o
d
e

1

2
r
o
o
t
>
g

i
s

h
d
i
s
.
6

0
0
0
On node 1 M
hdi0:# ##d+d)e,d*8a5db root7g ati7e
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
hdi0:+ none N0d
hdi0:5 none N0d1
hdi0:- none N0d*
hdi0:1# none N0d)
hdi0:11 none N0d3
hdi0:1* none None
hdi0:1) none None
hdi0:13 none None
{node1:root}/ #
EO P3I1 are assigned a(art #or the root>g hdis.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16K o# $%$
5hen hdis. is mar.ed root>g/ header o# the dis. might loo. li.e this 2
Hse the
'ollo-ing
command to
read hdis%
header "
On node 1 M #or hdis.6
{node1:root}/ # "0p7 \ grep root7g
$disk& &&cd'd3e:d2fa.db rootv+ ati7e
{node1:root}/ #
{node1:root}/ # l%uerypv -$ ?dev?r$disk&
######## C-C*(3C1 ######## ######## ######## \$$$$$$$$$$$$$$$$\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ##C(+()! ,(*A65(2 ######## ######## \$$}Cm/$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
On node 7 M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
168 o# $%$
5hen hdis. is not mar.ed root>g/ &ut Eone/ it-s im(ortant to chec. that it-s not used at all)
Hse the
'ollo-ing
command to
read hdis%
header "
I# all lines are
OE:I #ull o# ?6@
TD;E the hdis. is
#ree to &e used)
I# it-s not the case/
chec. #irst that
,hich :4E is
ma((ed to the
hdis. *ne't
(ages+/ and i# it-s
the :4E you
should use/ you
must then ?dd@
*<eroing+ the
/de>/rhdis.7 rdis.
{node1:root}/ #l%uerypv -$ ?dev?r$disk2
&&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&1& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&2& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&3& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&:& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&'& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&.& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&A& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&)& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&N& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&/& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&-& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
{node1:root}/ #
On node 7 M
B4T C!;F4:: 000/ any hdis. not mar.ed ?Eone@ as a result o# the ls(> command may sho, also a &lan.
header *=PFS used hdis. #or e'am(le+/ so ma.e sute that hdis. has ?Eone@ >alue out o# the ls(> command)
:ist o#
all
hdis.s
not
mar.ed
?root>g
@ or
?Eone@
on node
1 2
On node 1 M
$$$
hdi0:+ none N0d
hdi0:5 none N0d1
hdi0:- none N0d*
hdi0:1# none N0d)
hdi0:11 none N0d3
$$$
{node1:root}/ #
Hse the
'ollo-ing
command to
read hdis%
header "
!ll lines are OE:I
#ull o# ?6@
B4T the hdis. is
not #ree to &e
used 000 Because
it-s used &y IBM
=PFS in our
e'am(le
On node 1 M #or hdis.K
{node1:root}/ #l%uerypv -$ ?dev?r$disk'
&&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&1& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&2& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&3& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&:& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&'& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&.& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&A& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&)& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&N& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&/& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&-& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16% o# $%$
{node1:root}/ #
11.2.3 Register LUNs at AIX level
Be'ore
registration o'
/H40s "
On node 1 M
{node1:root}/ # "0p7
{node1:root}/ #
On node 7 M
{node*:root}/ # "0p7
hdi0:1 ##d+d)e+)3-e331 root7g ati7e
{node*:root}/ #
!s root on each node' u0date t&e OD" re0o%itory u%in( t&e followin( command 4 `c#gmgr`
Mou need to re(i%ter and identify LB>R% at A!+ level' and LB>R% will be ma00ed to &di%, and re(i%tered in t&e A!+ OD".
A'ter
registration o'
/H40s in O&M
thru c'gmgr
command "
On node 1 M
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node1:root}/ #
On node 7 M
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node*:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
116 o# $%$
11.2.4 Identify LUNs and corresponding hdisk on each node
5e .no, the :4E-s a>aila&le #or OC
and 3oting dis.s are :1 to :C)
1is.s :4E-s I1 Eum&er
OCR * L1
OCR 5 L7
Notin( * L$
Notin( 5 LB
Notin( J LC
Identi#y the dis.s a>aila&le #or OC and 3oting dis.s/ on each node/ .no,ing the :4E-s num&ers)
Tno,ing the :4E-s
num&er to use/ ,e
.no, need to identi#y
the corres(onding
hdis.s on each node o#
the cluster as detailed
in the #ollo,ing ta&le 2
1is.s :4E-s I1
Eum&er
Eode 1
Corres(onding
hdis.
Eode 7
Corres(onding
hdis.
Eode M)
Corres(onding
hdis.
OCR * L1
OCR 5 L7
Notin( * L$
Notin( 5 LB
Notin( J LC
There are t,o methods to identi#y the corres(onding hdis.s 2
Identi#y :4E I1 assign to hdis./ using ?lsc#g l hdis.R@ command
Identi#y hdis.s &y assigning momently a P3I1 to each hdis. not ha>ing one
5e strongly recommend not using P3I1 to identi#y hdis.)
I# some hdis.s are already used/ setting a P3I1 to a used hdis. can
corru(t the the hdis. header/ and ha>e generate issues as loosing
data stored on the hdis.)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
111 o# $%$
=et the :ist o# the hdis.s on node1 2
:ist o#
a>aila&le
hdis.s on
node 1 2
root>g is
hdis.6 000
On node 1 M
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node1:root}/ #
Hsing lsc'g command try to identi'y the hdis%s in the list generated :y lsp. on node1 "
Identi'y /H4 I& assign to hdis% using )lsc'g 8.l hdis%I* command
On node 1 M
{node1:root}/ # for i in 2 3 5 5 :
do
lscf+ -vl $disk3i
done
$disk2 U+5+-$##1$([(1+&K-P1-C*-T1-4*##5##6#251*62)1-(1############ 1+**-,## F,##G (i0: 6rray (e7ie
$disk: U+5+-$##1$([(1+&K-P1-C*-T1-4*##5##6#251*62)1-(5############ 1+**-,## F,##G (i0: 6rray (e7ie
{node1:root}/ #
TD;E/ 5e get the #ollo,ing ta&le 2
1is.s :4E-s I1
Eum&er
Eode 1
Corres(onding
hdis.
Eode 7
Corres(onding
hdis.
Eode M)
Corres(onding
hdis.
OCR * L1 hdis.7 &di%,8
OCR 5 L7 hdis.$
Notin( * L$ hdis.B &di%,8
Notin( 5 LB hdis.C
Notin( J LC hdis.G
Eo need to assign P3I1 ,hen using this method)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
117 o# $%$
=et the :ist o# the hdis.s on node7
:ist o# a>aila&le
hdis.s on node
7 2
root>g is
hdis.1/ ,hich is
not same hdis.
as on node 1/
,hich means
that it could &e
the same #or
each hdis.) On
&oth nodes/
same hdis.
names might
not &e attached
to same :4E)
On node 7 M
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node*:root}/ #
Hsing lsc'g command try to identi'y the hdis%s in the list generated :y lsp. on node# "
Be care'ull hdis%# on node1 is not necessary hdis%# on node#1
On node 1 M
{node*:root}/ # 8or i in * ) 3 % ,
do
"08g -7" hdi0:]i
done
$disk2 U+5+-$##1$([(1+&K-P1-C%-T1-4*##5##6#251*62)1-(1############ 1+**-,## F,##G (i0: 6rray (e7ie
$disk: U+5+-$##1$([(1+&K-P1-C%-T1-4*##5##6#251*62)1-(5############ 1+**-,## F,##G (i0: 6rray (e7ie
{node*:root}/ #
TD;E/ 5e get the #ollo,ing ta&le 2
1is.s :4E-s I1
Eum&er
Eode 1
Corres(onding
hdis.
Eode 7
Corres(onding
hdis.
Eode M)
Corres(onding
hdis.
OCR * L1 &di%,5 hdis.7 &di%,8
OCR 5 L7 &di%,J hdis.$ &di%,8
Notin( * L$ &di%,A hdis.B &di%,8
Notin( 5 LB &di%,@ hdis.C &di%,8
Notin( J LC &di%,/ hdis.G Fdi%,8
1o the same again #or any e'tra node M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11$ o# $%$
11.2.5 Removing reserve lock policy on hdisks from each node
5hy is it M!E1!TOI to change the reser>e (olicy on dis.s #rom each node accessing the same
:4E-s R
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11B o# $%$
Setu! reser(e.!olic/ on ocr and (oting hdisks0 on each node -
;'am(le #or one hdis. 2
Issue the command ?lsattr ; l hdis.7@ to >i<uali<e all attri&utes #or hdis.7
{node1:root}/ # lsattr -/l $disk2
PB?:ey?7a"ue none Per0i0tant Be0er7e Uey /a"ue True
ahe?method 8a0t?<rite 4rite Cahing method Aa"0e
ieee?7o"name ,##6#25###1*62)#####1+A,3+5+*+)! '!!! UniJue 7o"ume name Aa"0e
lun,id &9&&&1&&&&&&&&&&&& Hogia" Unit Number Aa"0e
ma1?tran08er #11##### Ma1imum TB6NSA!B SiLe True
pre8eth?mu"t 1 Mu"tip"e o8 b"o:0 to pre8eth on read Aa"0e
p7id none Phy0ia" 7o"ume identi8ier Aa"0e
J?type 0imp"e [ueuing Type Aa"0e
Jueue?depth 1# [ueue (epth True
raid?"e7e" % B6'( He7e" Aa"0e
rea00ign?to 1*# Bea00ign Timeout 7a"ue True
reserve,policy sin+le,pat$ Be0er7e Po"iy True
r<?timeout )# Bead/4rite Timeout 7a"ue True
00i?id #1,,#%## SCS' '( Aa"0e
siOe 3&& SiLe in Mbyte0 Aa"0e
<rite?ahe ye0 4rite Cahing enab"ed Aa"0e
{node1:root}/ #
Or only lsattr ; l hdis.7 Z gre( reser>e
{node1:root}/ # lsattr -/l $disk2 2 +rep reserve
{node1:root}/ #
o On IBM storage *;SS/ FasTt/ 1S""""+ 2 Change the ?reser>eN(olicy@ attri&ute to ?noNreser>e@
chde> -l hdis.R -a reser>eN(olicyWnoNreser>e
o On ;MC storage 2 Change the ?reser>eNloc.@ attri&ute to ?no@
chde> -l hdis.R -a reser>eNloc.Wno
o On D1S storage ,ith D1:M dri>er/ and no dis.s in 3olume =rou( 2 Change the ?dlmrs>le>el@
attri&ute to ?noNreser>e@
chde> -l dlm#dr>R -a dlmrs>le>elWnoNreser>e
Change the
?reser>eN(olicy@
attri&utes #or
each dis.s
dedicated to OC
and >oting dis.s/
on each nodes o#
the cluster 2
In our case/
,e ha>e an
IBM storage 000
On node 1 M
do
chde> -l hdis.Yi -a reser>eN(olicyWnoNreser>e
done
changed
changed
changed
changed
changed
{node1:root}/ #
On node 7 M
{node*:root}/ # for i in 2 3 5 5 :
do
done
changed
changed
changed
changed
changed
{node*:root}/ #
;'am(le #or one hdis. on node1 2
Issue the command ?lsattr ; l hdis.7 Zgre( reser>e@ to >i<uali<e modi#ied attri&utes #or hdis.7

{node1:root}/ # lsattr -/l $disk2 2 +rep reserve
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11C o# $%$
reserve,policy no,reserve Be0er7e Po"iy True
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11G o# $%$
11.2.6 Identify Major and Minor number of hdisk on each node
!s descri&ed &e#ore/ dis.s might ha>e di##erent names #rom one node to another #or e'am(le
hdis.7 on node1 might &e hdis.$ on node7/ etcM
1is.s :4E-s
I1
Eum&er
1e>ice Eame Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
OCR * L1 devocr6di%,* &di%,5 &di%,5
OCR 5 L7 devocr6di%,5 &di%,J &di%,J
Notin( * L$ devvotin(6di%,* &di%,A &di%,A
Notin( 5 LB devvotin(6di%,5 &di%,@ &di%,@
Notin( J LC devvotin(6di%,J &di%,/ &di%,/
Ee't ste(s ,ill e'(lain (rocedure to identi#y Ma9or and Minor num&er necessary to create the
>irtual de>ice (ointing to the right hdis. on each node)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11K o# $%$
Identi#y minor and
ma9or num&er #or
each hdis./ on
each node M
On node1 2
do
ls -la ?dev?P$disk3i
done
br<------- 1 root 0y0tem *1Q , Jan 11 1%:%5 /de7/hdi0:*
r<------- 1 root 0y0tem 21< : Jan 11 1%:%5 /de7/rhdi0:*
br<------- 1 root 0y0tem *1Q + Jan 11 1%:%5 /de7/hdi0:)
r<------- 1 root 0y0tem *1Q + Jan 11 1%:%5 /de7/rhdi0:)
br<------- 1 root 0y0tem *1Q 5 Jan 11 1%:%5 /de7/hdi0:3
r<------- 1 root 0y0tem *1Q 5 Jan 11 1%:%5 /de7/rhdi0:3
br<------- 1 root 0y0tem *1Q - Jan 11 1%:%5 /de7/hdi0:%
r<------- 1 root 0y0tem *1Q - Jan 11 1%:%5 /de7/rhdi0:%
br<------- 1 root 0y0tem *1Q 1# Jan 13 *1:15 /de7/hdi0:,
r<------- 1 root 0y0tem *1Q 1# Jan 13 *1:15 /de7/rhdi0:,
{node1:root}/ #
Nalue% a% 21; 6 are t&e maKor and minor number for &di%,5 on node*.
21 i% t&e maKor number' corre%0ondin( to a ty0e of di%,.
6 i% t&e minor number' corre%0ondin( to an order number.
1is.s :4E-s
I1
Eum&er
1e>ice Eame Eode 1
Corres(ondin
g hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
OCR * L1 devocr6di%,* &di%,5 71 G &di%,5
OCR 5 L7 devocr6di%,5 &di%,J 71 K &di%,J
Notin( * L$ devvotin(6di%,* &di%,A 71 8 &di%,A
Notin( 5 LB devvotin(6di%,5 &di%,@ 71 % &di%,@
Notin( J LC devvotin(6di%,J &di%,/ 71 16 &di%,/
Identi#y minor and
ma9or num&er #or
each hdis./ on each
node M
On node7 2
do
ls -la ?de8 do
done
br<------- 1 root 0y0tem *1Q , Jan 11 1,:1* /de7/hdi0:*
r<------- 1 root 0y0tem 21< : Jan 11 1,:1* /de7/rhdi0:*
br<------- 1 root 0y0tem *1Q + Jan 11 1,:1* /de7/hdi0:)
r<------- 1 root 0y0tem *1Q + Jan 11 1,:1* /de7/rhdi0:)
br<------- 1 root 0y0tem *1Q 5 Jan 11 1,:1* /de7/hdi0:3
r<------- 1 root 0y0tem *1Q 5 Jan 11 1,:1* /de7/rhdi0:3
br<------- 1 root 0y0tem *1Q - Jan 11 1,:1* /de7/hdi0:%
r<------- 1 root 0y0tem *1Q - Jan 11 1,:1* /de7/rhdi0:%
br<------- 1 root 0y0tem *1Q 1# Jan 13 **:*# /de7/hdi0:,
r<------- 1 root 0y0tem *1Q 1# Jan 13 **:*# /de7/rhdi0:,
{node*:root}/ #
1is.s :4E-s
I1
Eum&er
1e>ice Eame Eode 1
Corres(ondin
g hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
OCR * L1 devocr6di%,* &di%,5 5* / &di%,5 71 G
OCR 5 L7 devocr6di%,5 &di%,J 5* E &di%,J 71 K
Notin( * L$ devvotin(6di%,* &di%,A 5* I &di%,A 71 8
Notin( 5 LB devvotin(6di%,5 &di%,@ 5* D &di%,@ 71 %
Notin( J LC devvotin(6di%,J &di%,/ 5* *C &di%,/ 71 16
!s dis.s may ha>e di##erent names #rom one node to another #or e'am(le :1 could corres(ond to hdis.7 on
node1/ and hdis.1 on node7/ etcM
11g!C/!SM/!I"
oraclibm@fr.ibm.com
118 o# $%$
11.2.7 Create Unique Virtual Device to access same LUN from each node
TD;E #rom the ta&le/ #or node1 ,e need to create >irtual de>ices 2
1is.s :4E-s
I1
Eum&er
1e>ice Eame Eode 1
Corres(ondin
g hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
OCR * L1 /de>/ocrNdis.1 &di%,5 71 G &di%,5 71 G
OCR 5 L7 /de>/ocrNdis.7 &di%,J 71 K &di%,J 71 K
Notin( * L$ /de>/>otingNdis.1 &di%,A 71 8 &di%,A 71 8
Notin( 5 LB /de>/>otingNdis.7 &di%,@ 71 % &di%,@ 71 %
Notin( J LC /de>/>otingNdis.$ &di%,/ 71 16 &di%,/ 71 16
To create same >irtual de>ices on each node
called 2
/de>/ocrNdis.1
/de>/ocrNdis.7
/de>/>otingNdis.1
/de>/>otingNdis.7
/de>/>otingNdis.$
,e need to use ma9or and minor num&er o#
hdis.s ,hich ,ill ma.e the lin. &et,een the
>irtual de>ices and the hdis.s)
6sin+ t$e command 4
mknod Nevice,#ame c *aQ#um *in#um
-or first node< as root user
=node14root>? @ mknod ?dev?ocr,disk1 c 21 :
{node1:root}/ # mknod ?dev?ocr,disk2 c 21 '
{node1:root}/ # mknod ?dev?votin+,disk1 c 21 .
{node1:root}/ # mknod ?dev?votin+,disk2 c 21 C
{node1:root}/ # mknod ?dev?votin+,disk3 c 21 1&
From the ta&le/ #or node7 ,e need also to create >irtual de>ices 2
By chance Ma9or and minor num&er are the same on &oth nodes/ #or corres(onding hdis.s/ &ut it could &e
di##erent)
To create same >irtual de>ices on each node
called 2
/de>/ocrNdis.1
/de>/ocrNdis.7
/de>/>otingNdis.1
/de>/>otingNdis.7
/de>/>otingNdis.$
,e need to use ma9or and minor num&er o#
hdis.s ,hich ,ill ma.e the lin. &et,een the
>irtual de>ices and the hdis.s)
6sin+ t$e command 4
mknod Nevice,#ame c *aQ#um *in#um
-or second node< as root user
=node24root>? @ mknod ?dev?ocr,disk1 c 21 :
{node*:root}/ # mknod ?dev?ocr,disk2 c 21 '
{node*:root}/ # mknod ?dev?votin+,disk1 c 21 .
{node*:root}/ # mknod ?dev?votin+,disk2 c 21 C
{node*:root}/ # mknod ?dev?votin+,disk3 c 21 1&
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11% o# $%$
11.2.8 Set Ownership / Permissions on Virtual Devices
For each node in the cluster 2
Set o,nershi( o# the
created >irtual de>ices
=node14root>? @ c$own root4oinstall ?dev?ocr,disk1
{node1:root}/ # c$own root4oinstall ?dev?ocr,disk2
{node1:root}/ # c$own crs4dba ?dev?votin+,disk1
;$en on node2 R
TD;E set read/,rite
(ermissions on the
created >irtual de>ices
=node14root>? @ c$mod :5& ?dev?ocr,disk1
{node1:root}/ # c$mod :5& ?dev?ocr,disk2
{node1:root}/ # c$mod ::& ?dev?votin+,disk1
;$en on node2 R
11g!C/!SM/!I"
oraclibm@fr.ibm.com
176 o# $%$
TD;E chec. that settings are a((lied on node1 2
Chec.ing the
modi#ications
!#ter Oracle
cluster,are
installation/
!#ter Oracle
cluster,are
installation/
o,nershi(
and
(ermissions
o# >irtual
de>ices may
change)
{node1:root}/ # ls -la ?dev?P 2 +rep A21< :A
r<-r----- 1 root oin0ta"" *1Q , Aeb #, 1+:3) /de7/or?di0:1
r<------- 1 root 0y0tem *1Q , Jan 11 1%:%5 /de7/rhdi0:*
{node1:root}/ #
C$eck for eac$ disk usin+ followin+ command 4
{node1:root}/ # 8or i in , + 5 - 1#
C do
C "0 -"a /de7/= \ grep Z*1Q Z]i
C done
r<-r----- 1 root oin0ta"" *1Q + Aeb #, 1+:3) /de7/or?di0:*
r<-r<---- 1 r0 dba *1Q 5 Aeb #, 1+:3) /de7/7oting?di0:1
r<-r<---- 1 r0 dba *1Q - Aeb #, 1+:3) /de7/7oting?di0:*
r<-r<---- 1 r0 dba *1Q - Aeb #, 1+:3) /de7/7oting?di0:)
{node1:root}/ #
Then on node7 2

Chec.ing the
modi#ications
!#ter Oracle
cluster,are
installation/
!#ter Oracle
cluster,are
installation/
o,nershi(
and
(ermissions
o# >irtual
de>ices may
change)
{node*:root}/ # ls -la ?dev?P 2 +rep A21< :A
{node*:root}/ #
C$eck for eac$ disk usin+ followin+ command 4
{node*:root}/ # 8or i in , + 5 - 1#
C do
C "0 -"a /de7/= \ grep Z*1Q Z]i
C done
r<-r----- 1 root oin0ta"" *1Q + Aeb #, 1+:3) /de7/or?di0:*
r<-r<---- 1 r0 dba *1Q 5 Aeb #, 1+:3) /de7/7oting?di0:1
r<-r<---- 1 r0 dba *1Q - Aeb #, 1+:3) /de7/7oting?di0:*
r<-r<---- 1 r0 dba *1Q - Aeb #, 1+:3) /de7/7oting?di0:)
{node*:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
171 o# $%$
11.2.9 Formating the virtual devices (zeroing)
Eo,/ ,e do #ormat the >irtual de>ices 2
Format *èroing+ and
3eri#y that you can read
on the dis.s #rom each
node 2
On node 1 M
{node1:root}/ # 8or i in 1 *
C do
C dd i8O/de7/Lero o8O/de7/or?di0:]i b0O1#*3 ountO)##
C done
)##D# reord0 in$
)##D# reord0 out$
)##D# reord0 in$
)##D# reord0 out$
{node1:root}/ #
{node1:root}/ # 8or i in 1 * )
C do
C dd i8O/de7/Lero o8O/de7/7oting?di0:]i b0O1#*3 ountO)##
C done
)##D# reord0 in$
)##D# reord0 out$
)##D# reord0 in$
)##D# reord0 out$
)##D# reord0 in$
)##D# reord0 out$
{node1:root}/ #
3eri#y de>ices concurrent read/,rite access &y running at the same time dd command #rom each node 2
!t the same time 2
on node1
on node7
On node 1 M
{node1:root}/ # for i in 1 2
C do
C dd if0?dev?Oero of0?dev?ocr,disk3i bs01&25 count03&& S
C done
)##D# reord0 in$
)##D# reord0 out$
)##D# reord0 in$
)##D# reord0 out$
On node 7 M
{node2:root}/ # for i in 1 2
C do
C dd if0?dev?Oero of0?dev?ocr,disk3i bs01&25 count03&& S
C done
)##D# reord0 in$
)##D# reord0 out$
)##D# reord0 in$
)##D# reord0 out$
Same 'or .oting dis%s $
11g!C/!SM/!I"
oraclibm@fr.ibm.com
177 o# $%$
11)$ !SM dis.s
!&out !SM Instances (arameters #iles/ (#ile or s(#ile R
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17$ o# $%$
!&out !SM dis.s/ using rhdis.-s or >irtual de>ices R
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17B o# $%$
11.3.1 Required LUNs
4sing the storage administration console/ you ha>e to create 2
* LB> for t&e A$" %0file -*CC"B %iLe.
o G&i% di%, will contain t&e %a&red A$" in%tance %0file' meanin( all A$" in%tance% 0arameter% t&at youPll
be able to modify in a dynamic way -Only 0o%%ible wit& * %tora(e im0lementation' wit& 5 %tora(e% youPll
&ave to u%e local A$" in%tance 0file..
A bunc& of LB>% for di%,% to be u%ed wit& A$".
o Di%,%LB>P% for A$" can be eit&er
devr&di%, -raw di%,%.
or
devA$"6D!$X* -virtual device%. for ea%e of admini%tration.
G&e followin( %creen %&ow% t&e LB> ma00in( for node% u%ed in our clu%ter.
G&e%e !DP% will &el0 u% on to identify w&ic& &di%, will be u%ed.
1is.s :4E-s I1 Eum&er :4E-s Si<e
A$" %0file L6 *CC "B
Di%, * for A$" LG A 2B
Di%, 5 for A$" LK A 2B
Di%, J for A$" L8 A 2B
Di%, A for A$" L% A 2B
Di%, @ for A$" L! *meaning :un 16+ A 2B
Di%, / for A$" LB *meaning :un 11+ A 2B
Di%, E for A$" LC *meaning :un 17+ A 2B
Di%, I for A$" L1 *meaning :un 1$+ A 2B
11.3.2 How to Identify if a LUN is used or not ?
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17C o# $%$
5hen hdis. is mar.ed root>g/ header o# the dis. might loo. li.e this 2
Hse the
'ollo-ing
command to
read hdis%
header "
{node1:root}/ # "0p7 \ grep root7g
$disk& &&cd'd3e:d2fa.db rootv+ ati7e
{node1:root}/ #
{node1:root}/ # l%uerypv -$ ?dev?r$disk&
######## C-C*(3C1 ######## ######## ######## \$$$$$$$$$$$$$$$$\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ##C(+()! ,(*A65(2 ######## ######## \$$}Cm/$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
On node 7 M
5hen hdis. is not mar.ed root>g/ &ut Eone/ it-s imortant to chec. that it-s not used at all)
Hse the
'ollo-ing
command to
read hdis%
header "
I# all lines are
OE:I #ull o# ?6@
TD;E the hdis. is
#ree to &e used)
I# it-s not the case/
chec. #irst that
,hich :4E is
ma((ed to the
hdis. *ne't
(ages+/ and i# it-s
the :4E you
should use/ you
must then ?dd@
*<eroing+ the
/de>/rhdis.7 rdis.
{node1:root}/ #l%uerypv -$ ?dev?r$disk2
&&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&1& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&2& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&3& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&5& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&:& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&'& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&.& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&A& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&)& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&C& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&N& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&/& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
&&&&&&-& &&&&&&&& &&&&&&&& &&&&&&&& &&&&&&&& 2................2
Hnode1ErootI= J
On node 7 M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17G o# $%$
11.3.3 Register LUNs at AIX level
Be'ore
registration o'
ne- /H40s 'or
ASM dis%s "
On node 1 M
{node1:root}/ # "0p7
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node1:root}/ #
On node 7 M
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
{node*:root}/ #
!s root on each node' u0date t&e OD" re0o%itory u%in( t&e followin( command 4 `c#gmgr`
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17K o# $%$
Mou need to re(i%ter and identify LB>R% at A!+ level' and LB>R% will be ma00ed to &di%, and re(i%tered in t&e A!+ OD".
A'ter
registration o'
/H40s in O&M
thru c'gmgr
command "
On node 1 M
$disk1 none #one
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
$disk' none #one
$disk. none #one
$diskC none #one
$disk1& none #one
$disk11 none #one
$disk12 none #one
$disk13 none #one
$disk15 none #one
{node1:root}/ #
On node 7 M
$disk& none #one
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
$disk' none #one
$disk. none #one
$diskC none #one
$disk1& none #one
$disk11 none #one
$disk12 none #one
$disk13 none #one
$disk15 none #one
{node*:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
178 o# $%$
11.3.4 Identify LUNs and corresponding hdisk on each node
5e .no, the :4E-s to use
#or 2
!SM s(#ile dis.
!SM dis.s
1is.s :4E-s
Eum&er
Eode 1 Eode 7 Eode M)
A$" $0file Di%, L6
Di%, * for A$" LG
Di%, 5 for A$" LK
Di%, J for A$" L8
Di%, A for A$" L%
Di%, @ for A$" L!
Di%, / for A$" LB
Di%, E for A$" LC
Di%, I for A$" L1 S S S
Identi#y the dis.s a>aila&le #or !SM and !SM s(#ile dis.s/ on each node/ .no,ing the :4E-s
num&ers)
There are t,o methods to identi#y the corres(onding hdis.s 2
Identi#y :4E I1 assign to hdis./ using ?lsc#g l hdis.R@ command
Identi#y hdis.s &y assigning momently a P3I1 to each hdis. not ha>ing one
5e strongly recommend not using P3I1 to identi#y hdis.)
I# some hdis.s are already used/ setting a P3I1 to a used hdis. can
corru(t the the hdis. header/ and ha>e generate issues as loosing
data stored on the hdis
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17% o# $%$
=et the :ist o# the hdis.s on node1 2
:ist o#
a>aila&le
hdis.s on
node 1 2
root>g is
hdis.6 000
On node 1 M
hdi0:1 none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
hdi0:+ none None
hdi0:5 none None
hdi0:- none None
hdi0:1# none None
hdi0:11 none None
hdi0:1* none None
hdi0:1) none None
hdi0:13 none None
{node1:root}/ #
Hsing lsc'g command try to identi'y the hdis%s in the list generated :y lsp. on node1 "
On node 1 M
{node1:root}/ # for i in 1 1 2 3 5 5 : ' . C 1& 11 12 13 15
do
lscf+ -vl $disk3i
done
$disk1 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(& )%%* F%##G (i0: 6rray (e7ie
hdi0:* U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-H1############ )%%* F%##G (i0: 6rray (e7ie
hdi0:) U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-H*############ )%%* F%##G (i0: 6rray (e7ie
hdi0:3 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-H)############ )%%* F%##G (i0: 6rray (e7ie
hdi0:% U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-H3############ )%%* F%##G (i0: 6rray (e7ie
hdi0:, U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-H%############ )%%* F%##G (i0: 6rray (e7ie
$disk' U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(:############ )%%* F%##G (i0: 6rray (e7ie
$disk. U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-('############ )%%* F%##G (i0: 6rray (e7ie
$diskC U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(.############ )%%* F%##G (i0: 6rray (e7ie
$disk1& U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(C############ )%%* F%##G (i0: 6rray (e7ie
$disk11 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(A############ )%%* F%##G (i0: 6rray (e7ie
$disk12 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-()############ )%%* F%##G (i0: 6rray (e7ie
$disk13 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(C############ )%%* F%##G (i0: 6rray (e7ie
$disk15 U+5+-$##1$([(#1JU-P1-C*-T1-4*##*##6#25#C%3#3-(N############ )%%* F%##G (i0: 6rray (e7ie
{node1:root}/ #
TD;E
5e get the
#ollo,ing ta&le 2
1is.s :4E-s
Eum&er
A$" $0file Di%, L6 hdis.1 &di%,8
Di%, * for A$" LG hdis.K &di%,8
Di%, 5 for A$" LK hdis.8 S
Di%, J for A$" L8 hdis.%
Di%, A for A$" L% hdis.16
Di%, @ for A$" L! hdis.11
Di%, / for A$" LB hdis.17
Di%, E for A$" LC hdis.1$
Di%, I for A$" L1 hdis.1B
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$6 o# $%$
=et the :ist o# the hdis.s on node7
:ist o# a>aila&le
hdis.s on node
7 2
root>g is hdis.1/
,hich is not
same hdis. as
on node 1/
,hich means
that it could &e
the same #or
each hdis.) On
&oth nodes/
same hdis.
names might not
&e attached to
same :4E)
On node 7 M
hdi0:# none None
hdi0:* none None
hdi0:) none None
hdi0:3 none None
hdi0:% none None
hdi0:, none None
hdi0:+ none None
hdi0:5 none None
hdi0:- none None
hdi0:1# none None
hdi0:11 none None
hdi0:1* none None
hdi0:1) none None
hdi0:13 none None
{node*:root}/ #
Hsing lsc'g command try to identi'y the hdis%s in the list generated :y lsp. on node# "
Be care'ull hdis%J on node1 is not necessary hdis%J on node#1
On node 1 M
{node*:root}/ # for i in & 2 3 5 5 : ' . C 1& 11 12 13 15
do
lscf+ -vl $disk3i
done
$disk& U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H# )%%* F%##G (i0: 6rray (e7ie
hdi0:1 U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H1############ )%%* F%##G (i0: 6rray (e7ie
hdi0:* U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H*############ )%%* F%##G (i0: 6rray (e7ie
hdi0:) U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H)############ )%%* F%##G (i0: 6rray (e7ie
hdi0:3 U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H3############ )%%* F%##G (i0: 6rray (e7ie
hdi0:% U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H%############ )%%* F%##G (i0: 6rray (e7ie
$disk: U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H,############ )%%* F%##G (i0: 6rray (e7ie
$disk' U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H+############ )%%* F%##G (i0: 6rray (e7ie
$disk. U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H5############ )%%* F%##G (i0: 6rray (e7ie
$diskC U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H-############ )%%* F%##G (i0: 6rray (e7ie
$disk1& U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H6############ )%%* F%##G (i0: 6rray (e7ie
$disk12 U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H2############ )%%* F%##G (i0: 6rray (e7ie
$disk13 U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-HC############ )%%* F%##G (i0: 6rray (e7ie
$disk15 U+5+-$##1$([(#1JU-P1-C,-T1-4*##*##6#25#C%3#3-H(############ )%%* F%##G (i0: 6rray (e7ie
{node*:root}/ #
TD;E
5e get the
#ollo,ing ta&le 2
1is.s :4E-s
Eum&er
A$" $0file Di%, L6 &di%,* hdis.6 &di%,8
Di%, * for A$" LG &di%,E hdis.K &di%,8
Di%, 5 for A$" LK &di%,I hdis.8 S
Di%, J for A$" L8 &di%,D hdis.%
Di%, A for A$" L% &di%,*C hdis.16
Di%, @ for A$" L! &di%,** hdis.11
Di%, / for A$" LB &di%,*5 hdis.17
Di%, E for A$" LC &di%,*J hdis.1$
Di%, I for A$" L1 &di%,*A hdis.1B
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$1 o# $%$
11.3.5 Removing reserve lock policy on hdisks from each node
!s #or OC and 3oting dis.s/ S;T4P reser>eN(olicy on !SM s(#ile and !SM hdis.s/ on each
node 2
;'am(le #or one hdis. 2
Issue the command ?lsattr ; l hdis.K@ to >i<uali<e all attri&utes #or hdis.K
{node1:root}/ # lsattr -/l $disk'
ieee?7o"name ,##6#25####C%315#####*,)3%!!)CCC '!!! UniJue 7o"ume name Aa"0e
"un?id #1###,############ Hogia" Unit Number Aa"0e
pvid none Phy0ia" 7o"ume identi8ier Aa"0e
raid?"e7e" # B6'( He7e" Aa"0e
reserve,policy sin+l,pat$ Be0er7e Po"iy True
00i?id #1,-#,## SCS' '( Aa"0e
0iLe 3#-, SiLe in Mbyte0 Aa"0e
{node1:root}/ #
Or only lsattr ; l hdis.$ Z gre( reser>e
{node1:root}/ # lsattr -/l $disk' 2 +rep reserve
{node1:root}/ #
o On IBM storage *;SS/ FasTt/ 1S""""+ 2 Change the ?reser>eN(olicy@ attri&ute to ?noNreser>e@
chde> -l hdis.R -a reser>eN(olicyWnoNreser>e
o On ;MC storage 2 Change the ?reser>eNloc.@ attri&ute to ?no@
chde> -l hdis.R -a reser>eNloc.Wno
o On D1S storage ,ith D1:M dri>er/ and no dis.s in 3olume =rou( 2 Change the ?dlmrs>le>el@
attri&ute to ?noNreser>e@
chde> -l dlm#dr>R -a dlmrs>le>elWnoNreser>e
Change the
?reser>eN(
olicy@
attri&utes
#or each
dis.s
dedicated
to !SM/ on
each nodes
o# the
cluster 2
In our case/ ,e ha>e an IBM storage 000
On node 1 M
{node1:root}/ # for i in 1 ' . C 1& 11 12 13 15
do
done
changed
changed
changed
M
{node1:root}/ #
On node 7 M
{node*:root}/ # for i in & : ' . C 1& 12 13 15
do
done
changed
changed
changed
M
{node*:root}/ #
;'am(le #or one hdis. on node1 2
Issue the command ?lsattr ; l hdis.$ Zgre( reser>e@ to >i<uali<e modi#ied attri&utes #or hdis.$
{node1:root}/ # lsattr -/l $disk' 2 +rep reserve
{node1:root}/ #
!s descri&ed &e#ore/ dis.s might ha>e di##erent names #rom one node to another #or e'am(le hdis.K on node1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$7 o# $%$
might &e hdis.8 on node7/ etcM
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$$ o# $%$
11.3.6 Identify Major and Minor number of hdisk on each node
1is.s :4E-s
I1
Eum&er
1e>ice Eame
seen on node1 *1+ and
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
A$" $0file
Di%,
L6 devA$"%0f6di%, &di%,* &di%,C
Di%, * for A$" LG devr&di%,8 &di%,E &di%,E
Di%, 5 for A$" LK devr&di%,8 &di%,I &di%,I
Di%, J for A$" L8 devr&di%,8 &di%,D &di%,D
Di%, A for A$" L% devr&di%,8 &di%,*C &di%,*C
Di%, @ for A$" L! devr&di%,8 &di%,** &di%,**
Di%, / for A$" LB devr&di%,8 &di%,*5 &di%,*5
Di%, E for A$" LC devr&di%,8 &di%,*J &di%,*J
Di%, I for A$" L1 devr&di%,8 &di%,*A &di%,*A
5e need to get ma9or and minor num&er #or each hdis. o# node1 2
To o&tain minor
and ma9or
num&ers o# each
hdis./ on node1 M
5e need to issue
the command 2
ls la /de>/hdis.R
On node1 2
{node1:root}/ # for i in 1 ' . C 1& 11 12 13 15
do
done
br<------- 1 root 0y0tem 21< 5 Mar 1* 1*:15 /de7/hdi0:1
r<------- 1 root 0y0tem 21< 5 Mar 1* 1*:15 /de7/rhdi0:1
br<------- 1 root 0y0tem 21< 11 Mar #+ 1#:)1 /de7/hdi0:+
r<------- 1 root 0y0tem 21< 11 Mar #+ 1#:)1 /de7/rhdi0:+
br<------- 1 root 0y0tem 21< 12 Mar #+ 1#:)1 /de7/hdi0:5
r<------- 1 root 0y0tem 21< 12 Mar #+ 1#:)1 /de7/rhdi0:5
br<------- 1 root 0y0tem 21< 13 Mar #+ 1#:)1 /de7/hdi0:-
r<------- 1 root 0y0tem 21< 13 Mar #+ 1#:)1 /de7/rhdi0:-
br<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)1 /de7/hdi0:1#
r<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)1 /de7/rhdi0:1#
br<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)1 /de7/hdi0:11
r<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)1 /de7/rhdi0:11
br<------- 1 root 0y0tem 21< 1: Mar #+ 1#:)1 /de7/hdi0:1*
r<------- 1 root 0y0tem 21< 1: Mar #+ 1#:)1 /de7/rhdi0:1*
br<------- 1 root 0y0tem 21< 1' Mar *+ 1,:1) /de7/hdi0:1)
r<------- 1 root 0y0tem 21< 1' Mar *+ 1,:1) /de7/rhdi0:1)
br<------- 1 root 0y0tem 21< 1. Mar *+ 1,:1) /de7/hdi0:13
r<------- 1 root 0y0tem 21< 1. Mar *+ 1,:1) /de7/rhdi0:13
{node1:root}/ #
To #ill in the #ollo,ing ta&le/ ta.ing result #rom command #or hdis.1/ ,e get ?21; 5@ ,hich gi>e us 71 as ma9or
num&er/ and C as minor num&er)
1is.s :4E-s
I1
Eum&er
1e>ice Eame
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
A$" $0file
Di%,
L6 devA$"%0f6di%, &di%,* 71 C &di%,C
Di%, * for A$" LG devr&di%,8 &di%,E 71 11 &di%,/
Di%, 5 for A$" LK devr&di%,8 &di%,I 71 17 &di%,E
Di%, J for A$" L8 devr&di%,8 &di%,D 71 1$ &di%,I
Di%, A for A$" L% devr&di%,8 &di%,*C 71 1B &di%,D
Di%, @ for A$" L! devr&di%,8 &di%,** 71 1C &di%,*C
Di%, / for A$" LB devr&di%,8 &di%,*5 71 1G &di%,*5
Di%, E for A$" LC devr&di%,8 &di%,*J 71 1K &di%,*J
Di%, I for A$" L1 devr&di%,8 &di%,*A 71 18 &di%,*A
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$B o# $%$
*e need to get maGor and minor number for each hdis= of node2 6
To o&tain minor and
ma9or num&ers o#
each hdis./ on
node7 M
5e need to issue
the command 2
ls la /de>/hdis.R
On node7 2
{node2:root}/ # for i in & : ' . C 1& 12 13 15
do
done
br<------- 1 root 0y0tem 21< 5 Mar 1* 1*:1+ /de7/hdi0:#
r<------- 1 root 0y0tem 21< 5 Mar 1* 1*:1+ /de7/rhdi0:#
br<------- 1 root 0y0tem 21< 11 Mar #+ 1#:)* /de7/hdi0:,
r<------- 1 root 0y0tem 21< 11 Mar #+ 1#:)* /de7/rhdi0:,
br<------- 1 root 0y0tem 21< 12 Mar #+ 1#:)* /de7/hdi0:+
r<------- 1 root 0y0tem 21< 12 Mar #+ 1#:)* /de7/rhdi0:+
br<------- 1 root 0y0tem 21< 13 Mar #+ 1#:)* /de7/hdi0:5
r<------- 1 root 0y0tem 21< 13 Mar #+ 1#:)* /de7/rhdi0:5
br<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)* /de7/hdi0:-
r<------- 1 root 0y0tem 21< 15 Mar 1* 1):%% /de7/rhdi0:-
br<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)* /de7/hdi0:1#
r<------- 1 root 0y0tem 21< 15 Mar #+ 1#:)* /de7/rhdi0:1#
br<------- 1 root 0y0tem 21< 1: Mar #+ 1#:)* /de7/hdi0:1*
r<------- 1 root 0y0tem 21< 1: Mar #+ 1#:)* /de7/rhdi0:1*
br<------- 1 root 0y0tem 21< 1' Mar *+ 1,:1) /de7/hdi0:1)
r<------- 1 root 0y0tem 21< 1' Mar *+ 1,:1) /de7/rhdi0:1)
br<------- 1 root 0y0tem 21< 1. Mar *+ 1,:1) /de7/hdi0:13
r<------- 1 root 0y0tem 21< 1. Mar *+ 1,:1) /de7/rhdi0:13
{node*:root}/ #
To #ill in the #ollo,ing ta&le/ ta.ing result #rom command #or hdis.6/ ,e get ?2H; 5@ ,hich gi>e us 71 as ma9or
num&er/ and C as minor num&er)
1is.s :4E-s
I1
Eum&er
1e>ice Eame
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
A$" $0file
Di%,
L6 devA$"%0f6di%, &di%,* 5* @ &di%,C 71 C
Di%, * for A$" LG devr&di%,8 &di%,E 5* ** &di%,/ 71 11
Di%, 5 for A$" LK devr&di%,8 &di%,I 5* *5 &di%,E 71 17
Di%, J for A$" L8 devr&di%,8 &di%,D 5* *J &di%,I 71 1$
Di%, A for A$" L% devr&di%,8 &di%,*C 5* *A &di%,D 71 1B
Di%, @ for A$" L! devr&di%,8 &di%,** 5* *@ &di%,*C 71 1C
Di%, / for A$" LB devr&di%,8 &di%,*5 5* */ &di%,*5 71 1G
Di%, E for A$" LC devr&di%,8 &di%,*J 5* *E &di%,*J 71 1K
Di%, I for A$" L1 devr&di%,8 &di%,*A 5* *I &di%,*A 71 18
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$C o# $%$
11.3.7 Create Unique Virtual Device to access same LUN from each node
!s dis.s could ha>e di##erent names #rom one node to another #or e'am(le :6 corres(ond to hdis.1 on node1/
and could ha>e &een hdis.6 or other on node7/ etcM
1or A$" %0file di%,' it i% madatory to create a vitual device if &di%, name i% different on eac& node. Even %o &di%,
could be %ame for 5 node%' addin( an e<tra node could introduce a different &di%, name for t&e %ame LB> on t&e new
node' %o itP% be%t advice to create a virtual device for A$" %0file di%, OOO
TD;E #rom the #ollo,ing ta&le/ #or the !SM s(#ile dis. 2

1is.s :4E-s
I1
Eum&er
1e>ice Eame Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
!SM S(#ile 1is. L6 /de>/!SMs(#Ndis. hdis.1 71 C hdis.6 71 C
Di%, * for A$" LG devr&di%,8 &di%,E 5* ** &di%,/ 5* **
Di%, 5 for A$" LK devr&di%,8 &di%,I 5* *5 &di%,E 5* *5
Di%, J for A$" L8 devr&di%,8 &di%,D 5* *J &di%,I 5* *J
Di%, A for A$" L% devr&di%,8 &di%,*C 5* *A &di%,D 5* *A
Di%, @ for A$" L! devr&di%,8 &di%,** 5* *@ &di%,*C 5* *@
Di%, / for A$" LB devr&di%,8 &di%,*5 5* */ &di%,*5 5* */
Di%, E for A$" LC devr&di%,8 &di%,*J 5* *E &di%,*J 5* *E
Di%, I for A$" L1 devr&di%,8 &di%,*A 5* *I &di%,*A 5* *I
5e need to create a >irtual de>ice #or the asm s(#ile dis.)
To create same >irtual
de>ices on each node called
2
/de>/!SMs(#Ndis.
,e need to use ma9or and
minor num&er o# hdis.s
,hich ,ill ma.e the lin.
&et,een the >irtual de>ices
and the hdis.s)
6sin+ t$e command 4 mknod Nevice,#ame c *aQ#um *in#um
=node14root>? @ mknod ?dev?AS*spf,disk c 21 5
-or Second node< as root user
=node24root>? @ mknod ?dev?AS*spf,disk c 21 5
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$G o# $%$
EO5 #or the !SM dis.s/ ,e ha>e 7 o(tions (ossi&les 2
OPTIOE 1/ 4sing the /de>/rhdis./ then ,e don-t need to create >irtual de>ice) 5e-ll 9ust ha>e to set the
right user o,nershi( and uni' read/,rite (ermissions)
For this o(tion/ mo>e to ne't cha(ter and #ollo, o(tion 1)
Or
OPTIOE 7/ Creating 3irtual 1e>ices li.e /de>/!SMN1is.R/ i# ,anted #or humans/administrator
con>eniences)
5ith o(tion 7/ ,e-ll ha>e the #ollo,ing ta&le 2
1is.s :4E-s
I1
Eum&er
1e>ice Eame
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
A$" $0file Di%, L6 devA$"%0f6di%, &di%,C 5* @ &di%,C 5* @
Di%, * for A$" LG /de>/!SMN1is.1 &di%,/ 5* ** &di%,/ 5* **
Di%, 5 for A$" LK /de>/!SMN1is.7 &di%,E 5* *5 &di%,E 5* *5
Di%, J for A$" L8 /de>/!SMN1is.$ &di%,I 5* *J &di%,I 5* *J
Di%, A for A$" L% /de>/!SMN1is.B &di%,D 5* *A &di%,D 5* *A
Di%, @ for A$" L! /de>/!SMN1is.C &di%,*C 5* *@ &di%,*C 5* *@
Di%, / for A$" LB /de>/!SMN1is.G &di%,*5 5* */ &di%,*5 5* */
Di%, E for A$" LC /de>/!SMN1is.K &di%,*J 5* *E &di%,*J 5* *E
Di%, I for A$" L1 /de>/!SMN1is.8 &di%,*A 5* *I &di%,*A 5* *I
5e need to create a >irtual de>ice #or the asm dis.s)
By chance Ma9or and minor num&er are the same on &oth nodes/ #or corres(onding hdis.s/ &ut it could &e
di##erent)
To create same >irtual
de>ices on each node
called 2
/de>/!SMN1is.M
,e need to use ma9or and
minor num&er o# hdis.s
,hich ,ill ma.e the lin.
&et,een the >irtual de>ices
and the hdis.s)
6sin+ t$e command 4 mknod Nevice,#ame c *aQ#um *in#um
=node14root>? @ mknod ?dev?AS*,Nisk1 c 21 11
{node1:root}/ # mknod ?dev?AS*,Nisk2 c 21 12
.....
=node24root>? @ mknod ?dev?AS*,Nisk1 c 21 11
{node*:root}/ # mknod ?dev?AS*,Nisk2 c 21 12
.....
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$K o# $%$
11.3.8 Set Ownership / Permissions on Virtual Devices
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$8 o# $%$
For !SM s(#ile 2
TD;E set
o,nershi( o#
the created
>irtual de>ices
to oracle2d&a
=node14root>? @ c$own asm4oinstall ?dev?AS*spf,disk
=node24root>? @ c$own asm4oinstall ?dev?AS*spf,disk
TD;E set
read/,rite
(ermissions
o# the created
>irtual de>ices
to GG6
=node14root>? @ c$mod ::& ?dev?AS*spf,disk
=node24root>? @ c$mod ::& ?dev?AS*spf,disk
Chec.ing the
modi#ications
!#ter Oracle
cluster,are
installation/
{node1:root}/ # ls -la ?dev?P 2 +rep A21< 5A
crw-rw---- 1 asm dba 21< 5 *ar 12 154&: ?dev?AS*spf,disk
br<------- 1 root 0y0tem *1Q % Mar 1* 1*:15 /de7/hdi0:1
r<------- 1 root 0y0tem *1Q % Mar 1* 1*:15 /de7/rhdi0:1
{node1:root}/ #
{node2:root}/ # ls -la ?dev?P 2 +rep A2&< 5A
crw-rw---- 1 asm dba 21< 5 *ar 12 154&: ?dev?AS*spf,disk
br<------- 1 root 0y0tem *1Q % Mar 1* 1*:1+ /de7/hdi0:#
r<------- 1 root 0y0tem *1Q % Mar 1* 1*:1+ /de7/rhdi0:#
{node*:root}/ #
For !SM dis.s 2
5ith o(tion 1/ ,e-ll ha>e the #ollo,ing ta&le/ B4T 5e-ll not use Ma9or and Minor num&ers 2
1is.s :4E-s
I1
Eum&er
1e>ice Eame
seen on node1 *1+ and node7 *7+
Eode 1
Corres(onding
hdis.
Eode 7
Corres(onding
hdis.
A$" $0file Di%, L6 devA$"%0f6di%, &di%,* &di%,C
Di%, * for A$" LG /de>/rhdis.K on 1 --- /de>/rhdis.G on 7 &di%,E &di%,/
Di%, 5 for A$" LK /de>/rhdis.8 on 1 --- /de>/rhdis.K on 7 &di%,I &di%,E
Di%, J for A$" L8 /de>/rhdis.% on 1 --- /de>/rhdis.8 on 7 &di%,D &di%,I
Di%, A for A$" L% /de>/rhdis.16 on 1 --- /de>/rhdis.% on 7 &di%,*C &di%,D
Di%, @ for A$" L! /de>/rhdis.11 on 1 --- /de>/rhdis.16 on 7 &di%,** &di%,*C
Di%, / for A$" LB /de>/rhdis.17 on 1 --- /de>/rhdis.17 on 7 &di%,*5 &di%,*5
Di%, E for A$" LC /de>/rhdis.1$ on 1 --- /de>/rhdis.1$ on 7 &di%,*J &di%,*J
5e 9ust need to set oracle2d&a o,nerchi( to /de>/rhdis.R ma((ed to :4E #or !SM)
!nd to set GG6 read/,rite (ermissions to these rhdis.R 2
TD;E set o,nershi( o# the created >irtual de>ices to oracle2d&a

=node14root>? @ for i in ' . C 1& 11 12 13
Cdo
Cc$own asm4dba ?dev?r$disk3i
8done
node1:root-/C

=node24root>? @ for i in : ' . C 1& 12 13
Cdo
Cc$own asm4dba ?dev?r$disk3i
8done
node*:root-/C
TD;E set read/,rite (ermissions o# the created >irtual de>ices to GG6
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$% o# $%$

=node14root>? @ for i in ' . C 1& 11 12 13 15
Cdo
Cc$mod ::& ?dev?r$disk3i
Cdone
node1:root-/C

=node24root>? @ for i in : ' . C 1& 12 13
15
Cdo
Cc$mod ::& ?dev?r$disk3i
8done
node*:root-/C
Chec.ing the
modi#ications
Check also
on second
node ...
{node1:root}/ # ls -la ?dev?r$diskT 2 +rep oracle
crw-rw---- 1 asm dba 21< 11 *ar &' 1&431 ?dev?r$disk'
r<-r<---- 1 a0m dba *1Q 1* Mar #+ 1#:)1 /de7/rhdi0:5
r<-r<---- 1 a0m dba *1Q 1) Mar #+ 1#:)1 /de7/rhdi0:-
r<-r<---- 1 a0m dba *1Q 1) Mar #+ 1#:)1 /de7/rhdi0:1#
r<-r<---- 1 a0m dba *1Q 1) Mar #+ 1#:)1 /de7/rhdi0:11
r<-r<---- 1 a0m dba *1Q 1) Mar #+ 1#:)1 /de7/rhdi0:1*
r<-r<---- 1 a0m dba *1Q 1) Mar #+ 1#:)1 /de7/rhdi0:1)
{node1:root}/ #
5ith o(tion 7/ ,e-ll ha>e the #ollo,ing ta&le 2
1is.s :4E-s
I1
Eum&er
1e>ice Eame
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
A$" $0file Di%, L6 devA$"%0f6di%, &di%,C 5* @ &di%,C 5* @
Di%, * for A$" LG /de>/!SMN1is.1 &di%,/ 5* ** &di%,/ 5* **
Di%, 5 for A$" LK /de>/!SMN1is.7 &di%,E 5* *5 &di%,E 5* *5
Di%, J for A$" L8 /de>/!SMN1is.$ &di%,I 5* *J &di%,I 5* *J
Di%, A for A$" L% /de>/!SMN1is.B &di%,D 5* *A &di%,D 5* *A
Di%, @ for A$" L! /de>/!SMN1is.C &di%,*C 5* *@ &di%,*C 5* *@
Di%, / for A$" LB /de>/!SMN1is.G &di%,*5 5* */ &di%,*5 5* */
Di%, E for A$" LC /de>/!SMN1is.K &di%,*J 5* *E &di%,*J 5* *E
TD;E set
o,nershi( o# the
created >irtual
de>ices to
oracle2d&a
=node14root>? @ for i in 1 2 3 5 5
R
Cdo
Cc$own asm4dba ?dev?AS*,Nisk3i
8done
node1:root-/C
=node24root>? @ for i in 1 2 3 5 5 R
Cdo
Cc$own asm4dba ?dev?AS*,Nisk3i
8done
node*:root-/C
TD;E set
read/,rite
(ermissions o#
the created
>irtual de>ices to
GG6
=node14root>? @ for i in 1 2 3 5
5 R Cdo
Cc$mod ::& ?dev?AS*,Nisk3i
Cdone
node1:root-/C
=node24root>? @ for i in 1 2 3 5 5
R
Cdo
Cc$mod ::& ?dev?AS*,Nisk3i
8done
node*:root-/C
Chec.ing the
modi#ications

{node1:root}/ # ls -la ?dev?P 2 +rep A21< 11A
br<------- 1 root 0y0tem *1Q 11 Mar #+ 1#:)1 /de7/hdi0:+
r<------- 1 root 0y0tem *1Q 11 Mar #+ 1#:)1 /de7/rhdi0:+
crw-rw-r-- 1 asm dba 21< 11 Apr &3 15425 ?dev?AS*,Nisk1
C$eck also for all disks< and on second node ...
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B6 o# $%$
11.3.9 Formating the virtual devices (zeroing)
Eo,/ ,e need to #ormat the >irtual de>ices/ or rhdis.) In &oth o(tion/ <eroing rhdis. is su##icient #or 2
!SM s(#ile dis.
!SM dis.s
Format *èroing+ and 3eri#y
that you can read on the
dis.s #rom each
node 2
On node 1 M
Cdo
Cdd if0?dev?Oero of0?dev?r$disk3i bs0.1C2 count025&&& S
Cdone
25&&&U& records in.
25&&&U& records out.
25&&&U& records in.
...
node:root-/C for i in : ' . C 1& 12 13 15
3eri#y de>ices concurrent read/,rite access &y running at the same time dd command #rom each node 2
!t the same time 2
on node1
on node7
On node 1 M
Cdo
Cdone
25&&&U& records in.
25&&&U& records in.
...
On node 7 M
=node24root>? @ for i in : ' . C 1& 12 13 15
Cdo
Cdone
25&&&U& records in.
25&&&U& records in.
...
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B1 o# $%$
11.3.10 Removing assigned PVID on hdisk
P3I1 M4ST EOT B;;E S;T on hdis. used #or !SM/ and should not &e set #or OC/ >oting and !SM s(#ile
dis.s 0000
From #ollo,ing ta&le/ ma.e sure that no P3I1 are assigned to hdis.s #rom each node ma((ed to :4E-s)
1is.s :4E-s
I1
Eum&er
1e>ice Eame
node7 *7+
Eode 1
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
Eode 7
Corres(onding
hdis.
Ma9or
Eum)
Minor
Eum)
OCR * L1 devocr6di%,* hdis.7 5* / hdis.1 5* /
OCR 5 L7 devocr6di%,5 hdis.$ 5* E hdis.7 5* E
Notin( * L$ devvotin(6di%,* hdis.B 5* I hdis.$ 5* I
Notin( 5 LB devvotin(6di%,5 hdis.C 5* D hdis.B 5* D
Notin( J LC devvotin(6di%,J hdis.G 5* *C hdis.C 5* *C
A$" $0file
Di%,
L6 deva%m%0f6di%, hdis.1 5* @ hdis.6 5* @
Di%, * for A$" LG devr&di%,8 hdis.K 5* ** hdis.G 5* **
Di%, 5 for A$" LK devr&di%,8 hdis.8 5* *5 hdis.K 5* *5
Di%, J for A$" L8 devr&di%,8 hdis.% 5* *J hdis.8 5* *J
Di%, A for A$" L% devr&di%,8 hdis.16 5* *A hdis.% 5* *A
Di%, @ for A$" L! devr&di%,8 hdis.11 5* *@ hdis.16 5* *@
Di%, / for A$" LB devr&di%,8 hdis.17 5* */ hdis.17 5* */
Di%, E for A$" LC devr&di%,8 hdis.1$ 5* *E hdis.1$ 5* *E
Di%, I for A$" L1 devr&di%,8 hdis.1B 5* *I hdis.1B 5* *I
To remo>e P3I1
#rom hdis./ ,e ,ill
use the chde>
command 2
P3I1 must &e
remo>ed #rom hdis.
on each node)
IMPOT!ET 00000
1on-t remo>e P3I1
to hdis. ,hich are
not yours 0000
6sin+ t$e command 4 c$dev 1l $diskT 1a pv0clear
=node14root>? @ for i in 1 2 3 5 5 : 1 ' . C 1& 11 12 13 15
8do
8c$dev 1l $disk3i 1a pv0clear
8done
$disk2 c$an+ed
$disk3 c$an+ed
$disk5 c$an+ed
$disk5 c$an+ed
...
=node24root>? @ for i in & 2 3 5 5 : & ' . C 1& 12 13 15
Cdo
Cc$dev 1l $disk3i 1a pv0clear
Cdone
$disk2 c$an+ed
$disk3 c$an+ed
$disk5 c$an+ed
$disk5 c$an+ed
...
Chec. ,hith ls(> command as root on each node/ i# P3I1 are still assigned or not 000
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B7 o# $%$
11)B Chec.ing Shared 1e>ices
C&ec,in( for re%erve60olicy' and PN!D %ettin(%.
Chec.ing that no P3I1 are assigned to hdis.s/ and no singleN(ath reser>e (olicy are set 2
As root user< usin+ t$e command 4 lsattr -/l $disk1
{node1:root}/ # lsattr -/l $disk1
ieee?7o"name ,##6#25####C%3#)#####**(3%A3(A%A '!!! UniJue 7o"ume name Aa"0e
lun,id &9&&&&&&&&&&&&&&&& Hogia" Unit Number Aa"0e
pvid none Phy0ia" 7o"ume identi8ier Aa"0e
raid?"e7e" # B6'( He7e" Aa"0e
00i?id #1,-#,## SCS' '( Aa"0e
0iLe 1## SiLe in Mbyte0 Aa"0e
{node1:root}/ #
To chec= all hdis=s in one shot; use following shell scrit 6
for i in & 1 2 3 5 5 : ' . C 1& 11 12 13
do
lsattr -/l $disk3i 2 +rep reserve,policy 2 awk K=print 31<32 >K2 read rp1 rp2
lsattr -/l $disk3i 2 +rep pvid 2 awk K=print 31<32 >K2 read pv1 pv2
lsattr -/l $disk3i 2 +rep lun,id 2 awk K=print 31<32 >K2 read li1 li2
if V A3li1A W0 AA X
t$en
ec$o $disk3iK -8 K3li1K 0 K3li2K ? K3rp1K 0 K3rp2K ? K3pv1K 0 K3pv2
fi
done
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B$ o# $%$
-or first node< as root user runnin+ s$ell script
{node1:root}/ # for i in & 1 2 3 5 5 : ' . C 1& 11 12 13 15
do
if V A3li1A W0 AA X
t$en
fi
done
$disk1 -8 lun,id 0 &9&&&&&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk2 -8 lun,id 0 &9&&&1&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk: -8 lun,id 0 &9&&&5&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk' -8 lun,id 0 &9&&&:&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk. -8 lun,id 0 &9&&&'&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$diskC -8 lun,id 0 &9&&&.&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk1& -8 lun,id 0 &9&&&C&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk11 -8 lun,id 0 &9&&&a&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk12 -8 lun,id 0 &9&&&b&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk13 -8 lun,id 0 &9&&&c&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
{node1:root}/ #
-or Second node< as root user< runnin+ same s$ell script 4
{node2:root}/ # for i in & 1 2 3 5 5 : ' . C 1& 11 12 13 15
do
if V A3li1A W0 AA X
t$en
fi
done
$disk& -8 lun,id 0 &9&&&&&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk: -8 lun,id 0 &9&&&:&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk' -8 lun,id 0 &9&&&'&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk. -8 lun,id 0 &9&&&.&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$diskC -8 lun,id 0 &9&&&C&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk1& -8 lun,id 0 &9&&&a&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk12 -8 lun,id 0 &9&&&b&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
$disk13 -8 lun,id 0 &9&&&c&&&&&&&&&&&& ? reserve,policy 0 no,reserve ? pvid 0 none
{node*:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BB o# $%$
11)C ecommandations/ hints and ti(s
11.5.1 OCR / Voting disks
000 IE !EI case/ 1OE-T assign P3I1 to OC / 3oting dis.s ,hen Oracle
cluster,are has &een installed/ and in test or (roduction 000
!ssigning a P3I1 ,ill erase the hdis. header 0000/ and ,ith the ris. to
loose content)
!FT; CS
Installation 2
Do, to
identi#y
hdis.s used
as OC/ and
>otings
dis.s 2
!ll hdis.s (re(ared #or OC and >oting dis.s ha>e d&a/ or oinstall grou( assigned 2
For OC 2
{node1:root}/ # ls -la ?dev?ocrPdiskP
crw-r----- 1 root dba 2&< : *ar 12 154&3 ?dev?ocr,disk1
r<-r----- 1 root dba *#Q + Mar 1* 1%:#) /de7/or?di0:*
{node1:root}/ #
Then
{node1:root}/ # ls -la ?dev?P 2+rep A2&< :A
br<------- 1 root 0y0tem *#Q , Mar #+ 1#:)1 /de7/hdi0:*
r<-r----- 1 root dba *#Q , Mar 1* 1%:#) /de7/or?di0:1
crw------- 1 root system 2&< : *ar &' 1&431 ?dev?r$disk2
{node1:root}/ #
!nd using !I" command 2
;'am(le ,ith OC and corres(onding rhdis. 7 on node1 2
{node1:r0}/r0/11$1$#/bin -8l%uerypv -$ ?dev?r$disk22+rep KO=2>K
######1# C*26#### ####1### ###1*2AA 'A')'C'N \$$$$$$$$$$D$L{\}\
{node1:root}/ #
{node1:r0}/r0/11$1$#/bin -8l%uerypv -$ ?dev?r$disk22+rep K&&.2&&&&
--C&&&&& &&&&&&&& &&&&&&&&K
######## &&.2&&&& --C&&&&& ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
O
{node1:root}/ # l%uerypv -$ ?dev?r$disk2
######## &&.2&&&& --C&&&&& ######## ######## \$$$$$$$$$$$$$$$$\
######1# C*26#### ####1### ###1*2AA 'A')'C'N \$$$$$$$$$$D$L{\}\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BC o# $%$
!FT; CS
Installation 2
Do, to
identi#y
hdis.s used
as OC/ and
>otings
dis.s 2
For 3oting 2
{node1:root}/ # ls -la ?dev?votP,diskP
crw-r--r-- 1 crs dba 21< . Apr &5 15412 ?dev?votin+,disk1
r<-r--r-- 1 r0 dba *1Q - 6pr #% 13:1* /de7/7oting?di0:*
r<-r--r-- 1 r0 dba *1Q 1# 6pr #% 13:1* /de7/7oting?di0:)
{node1:root}/ #
Then
{node1:root}/ # ls -la ?dev?P 2+rep A21< .A
br<------- 1 root 0y0tem *1Q 5 Mar #+ 1#:)1 /de7/hdi0:3
crw------- 1 root system 21< . *ar &' 1&431 ?dev?r$disk5
r<-r--r-- 1 r0 dba *1Q 5 6pr #% 13:1) /de7/7oting?di0:1
{node1:root}/ #
;'am(le ,ith OC and corres(onding rhdis. 7 on node1 2
{node1:r0}/r0/11$1$#/bin -8l%uerypv -$ ?dev?r$disk52+rep KO=2>K
######1# 631*#### #####*## ###-%AAA 'A')'C'N \$$$$$$$$$$?$L{\}\
{node1:root}/ #
{node1:r0}/r0/11$1$#/bin -8l%uerypv -$ ?dev?r$disk52+rep K&&22&&&&
--C&&&&& &&&&&&&& &&&&&&&&K
######## &&22&&&& --C&&&&& ######## ######## \$Z$$$$$$$$$$$$$$\
{node1:root}/ #
JH
######## &&22&&&& --C&&&&& ######## ######## \$Z$$$$$$$$$$$$$$\
######1# 631*#### #####*## ###-%AAA 'A')'C'N \$$$$$$$$$$?$L{\}\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BG o# $%$
Do, to #ree
hdis. at !I"
le>el/ ,hen
hdis. are not
anymore used
#or OC or
>oting dis./ or
need to &e
reset #or a
#ailed CS
installation R
Iou must reset the hdis. header ha>ing the OC or 3oting dis. stam( 2
{node1:root}/ #dd if0?dev?Oero of0?dev?r$disk2 bs0.1C2 count025&&& S
25&&&U& records in.
TD;E Puery on the hdis. header ,ill return nothing more than all lines #ull o# _6-2
######## ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BK o# $%$
11.5.2 ASM disks
000 IE !EI case/ 1OE-T assign P3I1 to OC / 3oting dis.s ,hen Oracle
cluster,are has &een installed/ and in test or (roduction 000
!ssigning a P3I1 ,ill erase the hdis. header 0000/ and ,ith the ris. to
loose content)
!FT; !SM
Installation/
and !SM
dis.grou(
creation 2
Do, to
identi#y
hdis.s used
&y !SM)
!ll hdis.s (re(ared #or !SM are o,ned &y oracle user/ and grou( d&a 2
{node1:root}/ # ls -la ?dev?r$diskP 2 oracle
br<-r<---- 1 asm dba 21< 11 Mar #+ 1#:)1 /de7/hdi0:+
r<-r<---- 1 asm dba 21< 11 Mar #+ 1#:)1 /de7/rhdi0:+
br<-r<---- 1 asm dba 21< 12 Mar #+ 1#:)1 /de7/hdi0:5
r<-r<---- 1 asm dba 21< 12 Mar #+ 1#:)1 /de7/rhdi0:5
br<-r<---- 1 asm dba 21< 13 Mar #+ 1#:)1 /de7/hdi0:-
r<-r<---- 1 asm dba 21< 13 Mar #+ 1#:)1 /de7/rhdi0:-
br<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/hdi0:1#
r<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/rhdi0:1#
br<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/hdi0:11
r<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/rhdi0:11
br<-r<---- 1 asm dba 21< 1: Mar #+ 1#:)1 /de7/hdi0:1*
r<-r<---- 1 asm dba 21< 1: Mar #+ 1#:)1 /de7/rhdi0:1*
br<-r<---- 1 asm dba 21< 1' Mar *+ 1,:1) /de7/hdi0:1)
r<-r<---- 1 asm dba 21< 1' Mar *+ 1,:1) /de7/rhdi0:1)
br<-r<---- 1 asm dba 21< 1. Mar *+ 1,:1) /de7/hdi0:13
r<-r<---- 1 asm dba 21< 1. Mar *+ 1,:1) /de7/rhdi0:13
{node1:root}/ #
Or i# using >irtual de>ices 2
{node1:root}/ # ls -la ?dev?AS*PNiskP 2 +rep oracle
r<-r<---- 1 asm dba 21< 11 Mar #+ 1#:)1 /de7/6SM?(i0:1
r<-r<---- 1 asm dba 21< 12 Mar #+ 1#:)1 /de7/6SM?(i0:*
r<-r<---- 1 asm dba 21< 13 Mar #+ 1#:)1 /de7/6SM?(i0:)
r<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/6SM?(i0:3
r<-r<---- 1 asm dba 21< 15 Mar #+ 1#:)1 /de7/6SM?(i0:%
r<-r<---- 1 asm dba 21< 1: Mar #+ 1#:)1 /de7/6SM?(i0:,
r<-r<---- 1 asm dba 21< 1' Mar *+ 1,:1) /de7/6SM?(i0:+
r<-r<---- 1 asm dba 21< 1. Mar *+ 1,:1) /de7/6SM?(i0:5
{node1:root}/ #
TD;E/ #or e'am(le ,ith /de>/!SMN1is.1/ ,e use ma9or/minor num&er to identi#y the ePui>alent
rhdis.
Hnode1:root}/ # "0 -"a /de7/= \ ^*1Q 11_
br<-r<---- 1 asm dba 21< 11 Mar #+ 1#:)1 /de7/hdi0:+
r<-r<---- 1 asm dba 21< 11 Mar #+ 1#:)1 /de7/rhdi0:+
r<-r<---- 1 oracle dba 21< 11 Mar #+ 1#:)1 /de7/6SM?(i0:1
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B8 o# $%$
Do, to identi#y
hdis.s used &y
!SM)
OC:1IST
standing #or
oracle !SM dis.
!SM1BN=O4
P standing #or
!SM 1is.
=rou( used #or
the !SM1B
1ata&ase ,e
ha>e created in
our e'am(le
*the one you
,ill create later
M+
;'am(le ,ith !SMN1is.1 and corres(onding rhdis.K on node1 2
{node1:r0}/r0/11$1$#/bin -8l%uerypv -$ ?dev?r$disk22+rep OC:1IST
&&&&&&2& 5-52535C 555C535) &&&&&&&& &&&&&&&& 2JHC(NISY........2
{node1:root}/ #
O
######## ##5*#1#1 ######## 5######1 (1*6)(%2 \$$$$$$$$$$$$$`\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
&&&&&&2& 5-52535C 555C535) &&&&&&&& &&&&&&&& 2JHC(NISY........2
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# #61##### ###1#*#) 31%)3(33 3*%A3+%* \$$$$$$$$AS*N),7H\
######%# 3A%%%#%A )#)#)#)1 ######## ######## \J6",&&&1$$$$$$$$\
######,# ######## ######## 31%)3(33 3*%A3+%* \$$$$$$$$AS*N),7H\
######+# 3A%%%### ######## ######## ######## \J6"$$$$$$$$$$$$$\
######5# ######## ######## 31%)3(33 3*%A3+%* \$$$$$$$$AS*N),7H\
######-# 3A%%%#%A )#)#)#)1 ######## ######## \J6",&&&1$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## #1A%5+32 !(,C!### \$$$$$$$$$$$U$"$$\
######(# #1A%55C6 1%#265## #*##1### ##1##### \$$$$$$$$$$$$$$$$\
######!# ###12C5# ####13## #######* #######1 \$$$$$$$$$$$$$$$$\
######A# #######* #######* ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
EOE used !SM dis.s ,ith Puery on the hdis. header ,ill return nothing more than all lines #ull o#
_6-2
######## ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B% o# $%$
OC:1IST
standing #or
oracle !SM dis.
!SM1BNF:!SD
;CO3;I
standing #or
!SM 1is. =rou(
used #or the
!SM1B
1ata&ase Flash
eco>ery !rea
,e ha>e created
in our e'am(le
*the one you ,ill
create later M+
;'am(le o# out(ut ,ith an other rhdis. 2
TD;E Puery on the hdis. header ,ill return 2
{node1:root}/ #l%uerypv -$ ?dev?r$disk.
######## ##5*#1#1 ######## 5####### (!C52-3# \$$$$$$$$$$$$$$$`\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# 3A%*3)3C 333-%)32 ######## ######## \JHC(NISY$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# #61##### #####1#) 31%)3(33 3*%A3,3C \$$$$$$$$AS*N),-(\
######%# 31%)35%* 3%3)3A%, 3%%*%-%A )#)#)#)# \ASGH/CJI/HB,&&&&\
######,# ######## ######## 31%)3(33 3*%A3,3C \$$$$$$$$AS*N),-(\
######+# 31%)35%* 3%3)3A%, 3%%*%-## ######## \ASGH/CJI/HB$$$$$\
######5# ######## ######## 31%)3(33 3*%A3,3C \$$$$$$$$AS*N),-(\
######-# 31%)35%* 3%3)3A%, 3%%*%-%A )#)#)#)# \ASGH/CJI/HB,&&&&\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## #1A%5+3! 3!3+A5## \$$$$$$$$$$$NN&$$\
######(# #1A%55C6 13A65C## #*##1### ##1##### \$$$$$$$$$$$$$$$$\
######!# ###12C5# ####13## #######* #######1 \$$$$$$$$$$$$$$$$\
######A# #######* #######* ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
Do, to #ree
hdis. at !I"
le>el/ ,hen
hdis. are not
anymore used
#or OC or
>oting dis./ or
need to &e
reset #or a
#ailed CS
installation R
Iou must reset the hdis. header ha>ing !SM dis. stam( 2
{node1:root}/ #dd if0?dev?Oero of0?dev?r$disk2 bs0.1C2 count025&&& S
25&&&U& records in.
TD;E Puery on the hdis. header ,ill return nothing more than all lines #ull o# _6-2
######## ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######1# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######*# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######)# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######3# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######%# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######,# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######+# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######5# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######-# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######6# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######2# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######C# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######(# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######!# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
######A# ######## ######## ######## ######## \$$$$$$$$$$$$$$$$\
{node1:root}/ #
!ssigning P3I1 on the hdis. used &y !SM ,ill gi>e same result/ on the dis. header 0000
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C6 o# $%$
17 O!C:; C:4ST;5!; *CS+ IEST!::!TIOE
Oracle Cluster eady Ser>ices installation is necessary and mandatory)
Oracle Clu%terware will be in%talled in cr%**.*.C -UCR$6FO"E' UORA6CR$6FO"E. on eac& node.
On eac& node 4
Oracle Clu%terware -CR$. will be %tarted'
wit&
o Public' Private and Nirtual &o%tname defined
o Public' and Private networ, defined
o OCR and votin( di%,% confi(ured.
o Clu%ter !nterconnect confi(ured -Private >etwor,..
o Nirtual !P -N!P. will be confi(ured and %tarted.
o Oracle >otification $erver -O>$. will be %tarted.
o 2oblal $ervice Deamon -2$D. will be %tarted.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C1 o# $%$
17)1 Cluster 3eriFication utility
12.1.1 Understanding and Using Cluster Verification Utility
E;5 since 16g!C 7 0000
Cluster 3eri#ication 4tility *C34+ is a tool that (er#orms system chec.s) CNB command% a%%i%t you wit&
confirmin( t&at your %y%tem i% 0ro0erly confi(ured for 4
Oracle Clu%terware
Oracle Real A00lication Clu%ter% in%tallation.
!ntroduction to !n%tallin( and Confi(urin( Oracle Clu%terware and Oracle Real A00lication Clu%ter%
&tt04download-u,.oracle.comdoc%cdB*DJC/6C*in%tall.*C5b*A5C*intro.&tm;i*C5/*DI
Oracle Clu%terware and Oracle Real A00lication Clu%ter% Pre-!n%tallation Procedure%
&tt04download-u,.oracle.comdoc%cdB*DJC/6C*in%tall.*C5b*A5C*0art5.&tm
12.1.2 Using CVU to Determine if Installation Prerequisites are Complete
On Both nodes/ using oracle cluster,are 1is.1 as root user/ 1O run
M/cluster,are/1is.1/root(re/root(re)sh
C34 is using li&raries installed &y root(re)sh scri(t to run (ro(erly)
On node1
Vnode*4rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ;
.root0re.%&
.root0re.%& out0ut will be lo((ed in tm0root0re.out6CI-C5-CE.*C4*A4JD
Xernel e<ten%ion etc0w-%y%call./Abit6,ernel i% loaded.
Bnloadin( t&e e<i%tin( e<ten%ion4 etc0w-%y%call./Abit6,ernel....
Oracle Xernel E<ten%ion Loader for A!+
Co0yri(&t -c. *DDI'*DDD Oracle Cor0oration
Bnconfi(ured t&e ,ernel e<ten%ion %ucce%%fully
Bnloaded t&e ,ernel e<ten%ion %ucce%%fully
$avin( t&e ori(inal file% in etcora6%ave6CI-C5-CE.*C4*A4JD....
Co0yin( new ,ernel e<ten%ion to etc....
Loadin( t&e ,ernel e<ten%ion from etc
$ucce%%fully loaded etc0w-%y%call./Abit6,ernel wit& ,mid4 C<A5A@/CC
$ucce%%fully confi(ured etc0w-%y%call./Abit6,ernel wit& ,mid4
C<A5A@/CC
G&e ,ernel e<ten%ion wa% %ucce%%fuly loaded.
Confi(urin( A%ync&ronou% !O....
A%ync&ronou% !O i% already defined
Confi(urin( PO$!+ A%ync&ronou% !O....
Po%i< A%ync&ronou% !O i% already defined
C&ec,in( if (rou0 %ervice% %&ould be confi(ured....
>ot&in( to confi(ure.
Vnode*4rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ;
On node7
Vnode54rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ;
.root0re.%&
.root0re.%& out0ut will be lo((ed in tm0root0re.out6CI-C5-CE.*C4*@455
Xernel e<ten%ion etc0w-%y%call./Abit6,ernel i% loaded.
Bnloadin( t&e e<i%tin( e<ten%ion4 etc0w-%y%call./Abit6,ernel....
Bnconfi(ured t&e ,ernel e<ten%ion %ucce%%fully
Bnloaded t&e ,ernel e<ten%ion %ucce%%fully
$avin( t&e ori(inal file% in etcora6%ave6CI-C5-CE.*C4*@455....
Co0yin( new ,ernel e<ten%ion to etc....
Loadin( t&e ,ernel e<ten%ion from etc
$ucce%%fully loaded etc0w-%y%call./Abit6,ernel wit& ,mid4 C<A*DfICC
$ucce%%fully confi(ured etc0w-%y%call./Abit6,ernel wit& ,mid4 C<A*DfICC
G&e ,ernel e<ten%ion wa% %ucce%%fuly loaded.
Confi(urin( A%ync&ronou% !O....
A%ync&ronou% !O i% already defined
Confi(urin( PO$!+ A%ync&ronou% !O....
Po%i< A%ync&ronou% !O i% already defined
C&ec,in( if (rou0 %ervice% %&ould be confi(ured....
>ot&in( to confi(ure.
Vnode54rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ;
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C7 o# $%$
IM,OR+AN+ E1tract fro2 6
OracleL 1ata&ase elease Eotes
16g elease 7 *16)7+ #or !I" C: Based Systems *GB-Bit+
Part >umber B*DCEA-CJ
This 16g release note a((lies also to C34 #rom 11g 000
&tt04download-u,.oracle.comdoc%cdB*DJC/6C*relnote%.*C5b*DCEAtoc.&tm
Third Party Cluster,are
!f your de0loyment environment doe% not u%e FAC"P' i(nore t&e FAC"P ver%ion and 0atc&e%
error% re0orted by Clu%ter Nerification Btility -CNB.. On A!+ @L ver%ion @.5' t&e e<0ected 0atc& for
FAC"P v@.5 i% !M/CE@D. On A!+ @L ver%ion @.J' t&e e<0ected 0atc&e% for FAC"P v@.5 are
!M/CE@D' !M/*CJA' !M/*EEC' and !M/5*D*.
!f your de0loyment environment doe% not u%e 2P1$' i(nore t&e 2P1$ ver%ion and 0atc&e% error%
re0orted by Clu%ter Nerification Btility -CNB.. On A!+ @L ver%ion @.5 and ver%ion @.J' t&e e<0ected
0atc&e% for 2P1$ 5.J.C.J are !M/JD/D' !M/DD**' and !MEC5E/.
Chec. Ternel Parameter Settings
C34 does not chec. .ernel (arameter settings)
G&i% i%%ue i% trac,ed wit& Oracle bu( A@/@CA/.
Missing Patch ;rror Message
W&en CNB find% a mi%%in( 0atc&' it re0ort% a <<<< 0atc& i% un,nown error. G&i% %&ould be read a% <<<<
0atc& i% mi%%in(.
G&i% i%%ue i% trac,ed wit& Oracle bu( A@//AJE.
3eri#y =PFS is Installed
B%e t&e followin( command% to c&ec, for 2P1$ 4
clu>#y stage -(re c#s -n nodeNlist -s storageI1Nlist \->er&ose]
clu>#y stage -(ost c#s -n nodeNlist -# #ileNsystem \->er&ose]
G&i% i%%ue i% trac,ed wit& Oracle bu( A@/CJD.
Oracle Cluster 3eri#ication 4tility
Clu%ter Nerification Btility -CNB. i% a utility t&at i% di%tributed wit& Oracle Clu%terware *Cg. !t wa% develo0ed to a%%i%t in
t&e in%tallation and confi(uration of Oracle Clu%terware a% well a% Oracle Real A00lication Clu%ter% *Cg -RAC.. G&e
wide domain of de0loyment of CNB ran(e% from initial &ardware %etu0 t&rou(& fully o0erational clu%ter for RAC
de0loyment and cover% all t&e intermediate %ta(e% of in%tallation and confi(uration of variou% com0onent%.
Clu%ter Nerification Btility -CNB. download for all %u00orted RAC *Cg 0latform% will be 0o%ted a% t&ey become
available.
Cluster 3eri#ication 4tility FrePuently !s.ed Ouestions *P1F+
htt(2//,,,)oracle)com/technology/(roducts/data&ase/clustering/c>u/#aP/c>uN#aP)(d#
1o,nload C34 #or Oracle !C 16g "
htt(2//,,,)oracle)com/technology/(roducts/data&ase/clustering/c>u/c>uNdo,nloadNhome(age)html
1o,nload C34 readme #ile #or Oracle !C 16g on !I"C: 2
htt(2//,,,)oracle)com/technology/(roducts/data&ase/clustering/c>u/readme/!I"Nreadme)(d#
$ubKect4 Shared dis. chec. ,ith the Cluster 3eri#ication 4tility Doc !D4 Eote2$K7$C8)1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C$ o# $%$
M!T; S4; Iou ha>e ?un<i(@ tool or sym&olic lin. to un<i( in /usr/&in on &oth node
On node1
;'ecute the
#ollo,ing scri(t
!s oracle user 2
Setu( and e'(ort your TMP/ T;MP and TMP1I >aria&les
e'(ort TMPW/tm( Wit& tm0 or ot&er de%tination &avin( enou(& free %0ace
e'(ort T;MPW/tm(
e'(ort TMP1IW/tm(
Vnode*4rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terwarecluvfy ; l%
cvu0ac,.Li0 Kre0ac,.Li0
Vnode*4rootWdi%trib$oftwareOraclerdbm%**(r*ai<clu%terwarecluvfy ;
)/runclu>#y)sh stage -(re crsinst -n node1/node7 >er&ose
I# you ,ant the result in a te't #ile do the #ollo,ing 2
)/runclu>#y)sh stage -(re crsinst -n node1/node7 >er&ose V /tm(/clu>#yNoracle)t't
!nd analy<e the results 2
At this stage; node connecti4it0 is chec=ed AAA
Per#orming (re-chec.s #or cluster ser>ices setu(
Chec.ing node reacha&ility)))
C&ec,4 >ode reac&ability from node `node*`
De%tination >ode Reac&able8
------------------------------------ ------------------------
node* ye%
node5 ye%
Re%ult4 >ode reac&ability c&ec, 0a%%ed from node `node*`.
C&ec,in( u%er e)uivalence...
C&ec,4 B%er e)uivalence for u%er òracle`
>ode >ame Comment
------------------------------------ ------------------------
node5 0a%%ed
node* 0a%%ed
Re%ult4 B%er e)uivalence c&ec, 0a%%ed for u%er òracle`.
Chec.ing administrati>e (ri>ileges)))
C&ec,4 E<i%tence of u%er òracle`
>ode >ame B%er E<i%t% Comment
------------ ------------------------ ------------------------
node* ye% 0a%%ed
node5 ye% 0a%%ed
Re%ult4 B%er e<i%tence c&ec, 0a%%ed for òracle`.
C&ec,4 E<i%tence of (rou0 òin%tall`
>ode >ame $tatu% 2rou0 !D
------------ ------------------------ ------------------------
node* e<i%t% @C*
node5 e<i%t% @C*
Re%ult4 2rou0 e<i%tence c&ec, 0a%%ed for òin%tall`.
C&ec,4 "ember%&i0 of u%er òracle` in (rou0 òin%tall` ca% Primaryd
>ode >ame B%er E<i%t% 2rou0 E<i%t% B%er in 2rou0 Primary Comment
---------------- ------------ ------------ ------------ ------------ ------------
tba%**b ye% ye% ye% no failed
tba%**a ye% ye% ye% no failed
Re%ult4 "ember%&i0 c&ec, for u%er òracle` in (rou0 òin%tall` ca% Primaryd failed.
!dministrati>e (ri>ileges chec. #ailed
t&i% i% e)uivalent to &o%tname 0in( S
t&i% i% %%& or r%& te%t% for oracle u%er S
1rom node* to node5
1rom node* to node*
1rom node5 to node*
1rom node5 to node5
t&i% i% u%er and (rou0 e<i%tence te%t% on
node*
The #ollo,ing message is not a &ig issue 2
esult2 =rou( e'istence chec. #ailed #or
XoinstallX)
Iust create oinstall grou at A'(
le4el; using smitt0 as root
!t this stage/ node connecti>ity is chec.ed 000
Chec.ing node connecti>ity)))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CB o# $%$
Inter#ace in#ormation #or node Xnode1X
!nterface >ame !P Addre%% $ubnet
------------------------------ ------------------------------ ----------------
enC *C.J.5@.I* *C.J.5@.C
en* *C.*C.5@.I* *C.*C.5@.C
en5 5C.5C.5@.I* 5C.5C.5@.C
Inter#ace in#ormation #or node Xnode7X
!nterface >ame !P Addre%% $ubnet
------------------------------ ------------------------------ ----------------
enC *C.J.5@.I5 *C.J.5@.C
en* *C.*C.5@.I5 *C.*C.5@.C
en5 5C.5C.5@.I5 5C.5C.5@.C
Chec.2 Eode connecti>ity o# su&net X16)$)7C)6X
$ource De%tination Connected8
------------------------------ ------------------------------ ----------------
node*4enC node54enC ye%
Re%ult4 >ode connectivity c&ec, 0a%%ed for %ubnet `*C.J.5@.C` wit& node-%. node*'node5.
Chec.2 Eode connecti>ity o# su&net X16)16)7C)6X
------------------------------ ------------------------------ ----------------
node*4en* node54en* ye%
Re%ult4 >ode connectivity c&ec, 0a%%ed for %ubnet `*C.*C.5@.C` wit& node-%. node*'node5.
Chec.2 Eode connecti>ity o# su&net X76)76)7C)6X
------------------------------ ------------------------------ ----------------
node*4en5 node54en5 ye%
Re%ult4 >ode connectivity c&ec, 0a%%ed for %ubnet `5C.5C.5@.C` wit& node-%. node*'node5.
Suita&le inter#aces #or 3IP on su&net X76)76)7C)6X2
node* en545C.5C.5@.I*
node5 en545C.5C.5@.I5
Suita&le inter#aces #or the (ri>ate interconnect on su&net X16)$)7C)6X2
node* enC4*C.J.5@.I*
node5 enC4*C.J.5@.I5
Suita&le inter#aces #or the (ri>ate interconnect on su&net X16)16)7C)6X2
node* en*4*C.*C.5@.I*
node5 en*4*C.*C.5@.I5
esult2 Eode connecti>ity chec. (assed)
t&i% i% t&e networ, interface te%t% on
node* and node5 S
>ode connectivity of %ubnet on node*
and node5' for eac& networ, interface S
Ge%tin( %uitable interface% for N!P
-0ublic networ,.' and 0rivate networ, S
1on-t ,orry i# you ha>e messase as
Just carry on *it ,ill not a##ect the
installation+)
This is due to a C34 issue as e'(lain &ello, 2
Metalin. Eode I1 338)24.1
CA9/<G <ails ,it* 4rrorE Could not )ind a suitable set o) inter)aces )or /I(s
(er B9GE$$77277 cluv)y ma#es an incorrect assumption based on R<C 1.1+ t*at any I( address t*at
be%ins "it* any o) t*e )ollo"in% octets is non3routable and *ence may not be )it )or bein% used as a
J'/E 172.1&.D.D1.2.1&+.D.D11.D.D.D 5o"ever
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CC o# $%$
At this
stage; node
s0stem
re-uirement
s for crs is
chec=ed AAA
Chec.ing system rePuirements #or QcrsQ)))
C&ec,4 Xernel ver%ion
>ode >ame Available Re)uired Comment
------------ ------------------------ ------------------------ ----------
node* A!+ @.J A!+ @.5 0a%%ed
node5 A!+ @.J A!+ @.5 0a%%ed
Re%ult4 Xernel ver%ion c&ec, 0a%%ed.
C&ec,4 $y%tem arc&itecture
------------ ------------------------ ------------------------ ----------
node* 0ower0c 0ower0c 0a%%ed
node5 0ower0c 0ower0c 0a%%ed
Re%ult4 $y%tem arc&itecture c&ec, 0a%%ed.
C&ec,4 Gotal memory
------------ ------------------------ ------------------------ ----------
node* 52B -5CDE*@5XB. @*5"B -@5A5IIXB. 0a%%ed
node5 52B -5CDE*@5XB. @*5"B -@5A5IIXB. 0a%%ed
Re%ult4 Gotal memory c&ec, 0a%%ed.
C&ec,4 $wa0 %0ace
------------ ------------------------ ------------------------ ----------
node* *C5A2B -*CEJEA*I5AXB. *2B -*CAI@E/XB. 0a%%ed
node5 *C5A2B -*CEJEA*I5AXB. *2B -*CAI@E/XB. 0a%%ed
Re%ult4 $wa0 %0ace c&ec, 0a%%ed.
C&ec,4 1ree di%, %0ace in `tm0` dir
------------ ------------------------ ------------------------ ----------
node* ACC.J@"B -ACDD/CXB. ACC"B -ACD/CCXB. 0a%%ed
node5 ACC.J@"B -ACDD/CXB. ACC"B -ACD/CCXB. 0a%%ed
Re%ult4 1ree di%, %0ace c&ec, 0a%%ed.
C&ec,4 1ree di%, %0ace in òracle` dir
------------ ------------------------ ------------------------ ----------
node* A./J2B -AI/CCAAXB. A2B -A*DAJCAXB. 0a%%ed
node5 A./J2B -AI/CCAAXB. A2B -A*DAJCAXB. 0a%%ed
Re%ult4 1ree di%, %0ace c&ec, 0a%%ed.
Xernel ver%ion te%t S
Detectin( A!+ relea%e
$y%tem arc&itecture te%t S
Detectin( Power 0roce%%or
C&ec,in( memory
re)uirement% S
C&ec,in( %wa0 %0ace
re)uirement% S
C&ec,in( tm0 free %0ace
re)uirement% S
C&ec,in( oracle u%er &ome
directory free %0ace
re)uirement% S
At this
stage; node
s0stem
re-uiremen
ts for crs is
chec=ed AAA
Chec.ing system rePuirements #or QcrsQ))) *Continued M+
C&ec,4 Pac,a(e e<i%tence for `vac00.cm0.core4E.C.C.5`
>ode >ame $tatu% Comment
------------------------------ ------------------------------ ----------------
node* vac00.cm0.core4/.C.C.C failed
node5 vac00.cm0.core4/.C.C.C failed
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `vac00.cm0.core4E.C.C.5`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/@J/* `
>ode >ame A00lied Re)uired Comment
------------ ------------------------ ------------------------ ----------
node* un,nown !M/@J/* failed
node5 un,nown !M/@J/* failed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, failed for `!M/@J/* `.
C&ec,4 Pac,a(e e<i%tence for `vac.C4E.C.C.5`
------------------------------ ------------------------------ ----------------
node* vac.C4E.C.C.5 0a%%ed
node5 vac.C4E.C.C.5 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `vac.C4E.C.C.5`.
C&ec,4 Pac,a(e e<i%tence for `<lC.ai<@C.rte4E.C.C.A`
Chec. OE:I #or
necessary
rePuirements #or !SM
as e'(lained in the
coo.&oo. *cha(ter K+)
C34 is testing
e'istence o# all
(rerePuirements
needed #or all
im(lementations as 2
RAC im0lementation
A$" im0lementation
2P1$ im0lementation
FAC"P im0lementation
-Concurrent raw device%
or co&abitation between
A$" or 2P1$ wit&
FAC"P.
'lCMM M4ST &e at
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CG o# $%$
------------------------------ ------------------------------ ----------------
node* <lC.ai<@C.rte4E.C.C.A 0a%%ed
node5 <lC.ai<@C.rte4E.C.C.A 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `<lC.ai<@C.rte4E.C.C.A`.
C&ec,4 Pac,a(e e<i%tence for `<lC.rte4E.C.C.*`
------------------------------ ------------------------------ ----------------
node* <lC.rte4E.C.C.* 0a%%ed
node5 <lC.rte4E.C.C.* 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `<lC.rte4E.C.C.*`.
minimum release K)6)6)1
5ith elease GM) The
Oracle Cluster,are ,ill not
start M
At this
stage; node
s0stem
re-uiremen
ts for crs is
chec=ed AAA
C&ec,4 Pac,a(e e<i%tence for `(0f%.ba%e45.J.C.J`
------------------------------ ------------------------------ ----------------
node* (0f%.ba%e45.J.C.@ 0a%%ed
node5 (0f%.ba%e45.J.C.@ 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `(0f%.ba%e45.J.C.J`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/JD/D`
------------ ------------------------ ------------------------ ----------
node* !M/JD/D4(0f%.ba%e!M/JD/D4(0f%.doc%.data!M/JD/D4(0f%.m%(.en6B$ !M/JD/D 0a%%ed
node5 !M/JD/D4(0f%.ba%e!M/JD/D4(0f%.doc%.data!M/JD/D4(0f%.m%(.en6B$ !M/JD/D 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/JD/D`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/DD**`
------------ ------------------------ ------------------------ ----------
node* !M/DD**4(0f%.ba%e !M/DD** 0a%%ed
node5 !M/DD**4(0f%.ba%e !M/DD** 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/DD**`.
C&ec,4 O0eratin( %y%tem 0atc& for `!MEC5E/`
------------ ------------------------ ------------------------ ----------
node* !MEC5E/4(0f%.ba%e !MEC5E/ 0a%%ed
node5 !MEC5E/4(0f%.ba%e !MEC5E/ 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!MEC5E/`.
C&ec,4 Pac,a(e e<i%tence for `clu%ter.licen%e4@.5.C.C`
------------------------------ ------------------------------ ----------------
node* mi%%in( failed
node5 mi%%in( failed
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `clu%ter.licen%e4@.5.C.C`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/CE@D`
------------ ------------------------ ------------------------ ----------
node* un,nown !M/CE@D failed
node5 un,nown !M/CE@D failed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, failed for `!M/CE@D`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/*CJA`
------------ ------------------------ ------------------------ ----------
node* !M/*CJA4bo%.m0!M/*CJA4bo%.m0/A !M/*CJA 0a%%ed
node5 !M/*CJA4bo%.m0!M/*CJA4bo%.m0/A !M/*CJA 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/*CJA`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/*EEC`
------------ ------------------------ ------------------------ ----------
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CK o# $%$
node* !M/*EEC4r%ct.ba%ic.rte!M/*EEC4r%ct.core.errm!M/*EEC4r%ct.core.&o%trm
!M/*EEC4r%ct.core.rmc!M/*EEC4r%ct.core.%ec
!M/*EEC4r%ct.core.%en%orrm!M/*EEC4r%ct.core.util% !M/*EEC 0a%%ed
>ode5 !M/*EEC4r%ct.ba%ic.rte!M/*EEC4r%ct.core.errm!M/*EEC4r%ct.core.&o%trm
!M/*EEC4r%ct.core.rmc!M/*EEC4r%ct.core.%ec
!M/*EEC4r%ct.core.%en%orrm!M/*EEC4r%ct.core.util% !M/*EEC 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/*EEC`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/5*D*`
------------ ------------------------ ------------------------ ----------
node* !M/5*D*4bo%.adt.0rof!M/5*D*4bo%.rte.lib0t&read% !M/5*D* 0a%%ed
node5 !M/5*D*4bo%.adt.0rof!M/5*D*4bo%.rte.lib0t&read% !M/5*D* 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/5*D*`.
C&ec,4 Pac,a(e e<i%tence for Èlectric1ence-5.5.5-*45.5.5`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for Èlectric1ence-5.5.5-*45.5.5`.
C&ec,4 Pac,a(e e<i%tence for `<lfrte4D.*`
------------------------------ ------------------------------ ----------------
node* <lfrte4I.*.*.A failed
node5 <lfrte4I.*.*.A failed
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `<lfrte4D.*`.
C&ec,4 Pac,a(e e<i%tence for `(db-/.C-*4/.C`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `(db-/.C-*4/.C`.
C&ec,4 Pac,a(e e<i%tence for `ma,e-J.IC-*4J.IC`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `ma,e-J.IC-*4J.IC`.
C&ec,4 Pac,a(e e<i%tence for `freeware.(nu.tar.rte4*.*J.C.C`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `freeware.(nu.tar.rte4*.*J.C.C`.
C&ec,4 Pac,a(e e<i%tence for `#ava*A6/A.%d,4*.A.5.*`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `#ava*A6/A.%d,4*.A.5.*`.
C&ec,4 Pac,a(e e<i%tence for `#ava*J*.rte.bin4*.J.*.*/`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `#ava*J*.rte.bin4*.J.*.*/`.
C&ec,4 Pac,a(e e<i%tence for `#ava*A.%d,4*.A.5.5`
------------------------------ ------------------------------ ----------------
node* #ava*A.%d,4*.A.5.*C failed
node5 #ava*A.%d,4*.A.5.*C failed
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `#ava*A.%d,4*.A.5.5`.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C8 o# $%$
C&ec,4 O0eratin( %y%tem 0atc& for `!M/@JC@`
------------ ------------------------ ------------------------ ----------
node* !M/@JC@4#ava*A.%d, !M/@JC@ 0a%%ed
node5 !M/@JC@4#ava*A.%d, !M/@JC@ 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/@JC@`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M@IJ@C`
------------ ------------------------ ------------------------ ----------
node* un,nown !M@IJ@C failed
node5 un,nown !M@IJ@C failed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, failed for `!M@IJ@C`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/J@JJ`
------------ ------------------------ ------------------------ ----------
node* un,nown !M/J@JJ failed
node5 un,nown !M/J@JJ failed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, failed for `!M/J@JJ`.
C&ec,4 Pac,a(e e<i%tence for `m)m.%erver.rte4@.J`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `m)m.%erver.rte4@.J`.
C&ec,4 Pac,a(e e<i%tence for `m)m.client.rte4@.J`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `m)m.client.rte4@.J`.
C&ec,4 Pac,a(e e<i%tence for `%na.rte4/.*.C.A`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `%na.rte4/.*.C.A`.
C&ec,4 Pac,a(e e<i%tence for `bo%.net.tc0.%erver`
------------------------------ ------------------------------ ----------------
node* bo%.net.tc0.%erver4@.J.C.JC 0a%%ed
node5 bo%.net.tc0.%erver4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.net.tc0.%erver`.
C&ec,4 O0eratin( %y%tem 0atc& for `!MAA@DD`
------------ ------------------------ ------------------------ ----------
node* un,nown !MAA@DD failed
node5 un,nown !MAA@DD failed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, failed for `!MAA@DD`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M/CDJC`
------------ ------------------------ ------------------------ ----------
node* !M/CDJC4bo%.m0!M/CDJC4bo%.m0/A !M/CDJC 0a%%ed
node5 !M/CDJC4bo%.m0!M/CDJC4bo%.m0/A !M/CDJC 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M/CDJC`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M@I*AJ`
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C% o# $%$
------------ ------------------------ ------------------------ ----------
node* !M@I*AJ4+**.Dt.lib!M@I*AJ4+**.ba%e.rte!M@I*AJ4bo%.acct!M@I*AJ4bo%.adt.include!
M@I*AJ4bo%.adt.libm!M@I*AJ4bo%.adt.0rof!M@I*AJ4bo%.alt6di%,6in%tall.rte!M@I*AJ4bo%.dia(.com
!M@I*AJ4bo%.m0/A!M@I*AJ4bo%.m0!M@I*AJ4bo%.net.ewlm.rte!M@I*AJ4bo%.net.i0%ec.,eym(t
!M@I*AJ4bo%.net.i0%ec.rte!M@I*AJ4bo%.net.nf%.client!M@I*AJ4bo%.net.nf%.%erver!M@I*AJ4bo%.net.tc0.client
!M@I*AJ4bo%.net.tc0.%erver!M@I*AJ4bo%.net.tc0.%mit!M@I*AJ4bo%.0erf.lib0erf%tat!M@I*AJ4bo%.0erf.0erf%tat
!M@I*AJ4bo%.0erf.tool%!M@I*AJ4bo%.rte.arc&ive!M@I*AJ4bo%.rte.bind6cmd%!M@I*AJ4bo%.rte.boot
!M@I*AJ4bo%.rte.control!M@I*AJ4bo%.rte.file%y%tem!M@I*AJ4bo%.rte.in%tall!M@I*AJ4bo%.rte.libc
!M@I*AJ4bo%.rte.lvm!M@I*AJ4bo%.rte.man!M@I*AJ4bo%.rte.met&od%!M@I*AJ4bo%.rte.%ecurity
!M@I*AJ4bo%.rte.%erv6aid!M@I*AJ4bo%.%y%m(t.nim.client!M@I*AJ4bo%.%y%m(t.)uota!M@I*AJ4bo%.%y%m(t.%erv6aid
!M@I*AJ4bo%.%y%m(t.%y%br!M@I*AJ4device%.c&r0.ba%e.rte!M@I*AJ4device%.c&r0.0ci.rte!M@I*AJ4device%.c&r0.vdevice.rte
!M@I*AJ4device%.common.!B".atm.rte!M@I*AJ4device%.common.!B".et&ernet.rte!M@I*AJ4device%.common.!B".fc.rte
!M@I*AJ4device%.common.!B".fda.dia(!M@I*AJ4device%.common.!B".m0io.rte!M@I*AJ4device%.fc0.di%,.rte
!M@I*AJ4device%.0ci.CC*CCfCC.rte!M@I*AJ4device%.0ci.*A*CCAC*.dia(!M@I*AJ4device%.0ci.*A*CJJC5.rte
!M@I*AJ4device%.0ci.*A*C//C5.rte!M@I*AJ4device%.0ci.*A*C/DC5.rte!M@I*AJ4device%.0ci.*A*CEIC5.rte
!M@I*AJ4device%.0ci.*A*CffC*.rte!M@I*AJ4device%.0ci.55*C/AEA.rte!M@I*AJ4device%.0ci.JJ*CJ@CC.rte
!M@I*AJ4device%.0ci.Af****CC.com!M@I*AJ4device%.0ci.EE*C*55J.com!M@I*AJ4device%.0ci.DD*E5ECA.rte
!M@I*AJ4device%.0ci.c***CJ@I.rte!M@I*AJ4device%.0ci.df*CCCfE.com!M@I*AJ4device%.0ci.df*CCCfE.dia(
!M@I*AJ4device%.0ci.df*CCCfa.rte!M@I*AJ4device%.0ci.eA*AaI*/.rte!M@I*AJ4device%.%c%i.di%,.rte
!M@I*AJ4device%.vdevice.!B".l-lan.rte!M@I*AJ4device%.vdevice.!B".v%c%i.rte!M@I*AJ4device%.vdevice.&vterm*.rte
!M@I*AJ4device%.vtdev.%c%i.rte!M@I*AJ4%y%m(t.web%m.a00%!M@I*AJ4%y%m(t.web%m.framewor,
!M@I*AJ4%y%m(t.web%m.rte!M@I*AJ4%y%m(t.web%m.webacce%% !M@I*AJ 0a%%ed
>ode5 !M@I*AJ4+**.Dt.lib!M@I*AJ4+**.ba%e.rte!M@I*AJ4bo%.acct!M@I*AJ4bo%.adt.include!
M@I*AJ4bo%.adt.libm!M@I*AJ4bo%.adt.0rof!M@I*AJ4bo%.alt6di%,6in%tall.rte!M@I*AJ4bo%.dia(.com
!M@I*AJ4bo%.m0/A!M@I*AJ4bo%.m0!M@I*AJ4bo%.net.ewlm.rte!M@I*AJ4bo%.net.i0%ec.,eym(t
!M@I*AJ4bo%.net.i0%ec.rte!M@I*AJ4bo%.net.nf%.client!M@I*AJ4bo%.net.nf%.%erver!M@I*AJ4bo%.net.tc0.client
!M@I*AJ4bo%.net.tc0.%erver!M@I*AJ4bo%.net.tc0.%mit!M@I*AJ4bo%.0erf.lib0erf%tat!M@I*AJ4bo%.0erf.0erf%tat
!M@I*AJ4bo%.0erf.tool%!M@I*AJ4bo%.rte.arc&ive!M@I*AJ4bo%.rte.bind6cmd%!M@I*AJ4bo%.rte.boot
!M@I*AJ4bo%.rte.control!M@I*AJ4bo%.rte.file%y%tem!M@I*AJ4bo%.rte.in%tall!M@I*AJ4bo%.rte.libc
!M@I*AJ4bo%.rte.lvm!M@I*AJ4bo%.rte.man!M@I*AJ4bo%.rte.met&od%!M@I*AJ4bo%.rte.%ecurity
!M@I*AJ4bo%.rte.%erv6aid!M@I*AJ4bo%.%y%m(t.nim.client!M@I*AJ4bo%.%y%m(t.)uota!M@I*AJ4bo%.%y%m(t.%erv6aid
!M@I*AJ4bo%.%y%m(t.%y%br!M@I*AJ4device%.c&r0.ba%e.rte!M@I*AJ4device%.c&r0.0ci.rte!M@I*AJ4device%.c&r0.vdevice.rte
!M@I*AJ4device%.common.!B".atm.rte!M@I*AJ4device%.common.!B".et&ernet.rte!M@I*AJ4device%.common.!B".fc.rte
!M@I*AJ4device%.common.!B".fda.dia(!M@I*AJ4device%.common.!B".m0io.rte!M@I*AJ4device%.fc0.di%,.rte
!M@I*AJ4device%.0ci.CC*CCfCC.rte!M@I*AJ4device%.0ci.*A*CCAC*.dia(!M@I*AJ4device%.0ci.*A*CJJC5.rte
!M@I*AJ4device%.0ci.*A*C//C5.rte!M@I*AJ4device%.0ci.*A*C/DC5.rte!M@I*AJ4device%.0ci.*A*CEIC5.rte
!M@I*AJ4device%.0ci.*A*CffC*.rte!M@I*AJ4device%.0ci.55*C/AEA.rte!M@I*AJ4device%.0ci.JJ*CJ@CC.rte
!M@I*AJ4device%.0ci.Af****CC.com!M@I*AJ4device%.0ci.EE*C*55J.com!M@I*AJ4device%.0ci.DD*E5ECA.rte
!M@I*AJ4device%.0ci.c***CJ@I.rte!M@I*AJ4device%.0ci.df*CCCfE.com!M@I*AJ4device%.0ci.df*CCCfE.dia(
!M@I*AJ4device%.0ci.df*CCCfa.rte!M@I*AJ4device%.0ci.eA*AaI*/.rte!M@I*AJ4device%.%c%i.di%,.rte
!M@I*AJ4device%.vdevice.!B".l-lan.rte!M@I*AJ4device%.vdevice.!B".v%c%i.rte!M@I*AJ4device%.vdevice.&vterm*.rte
!M@I*AJ4device%.vtdev.%c%i.rte!M@I*AJ4%y%m(t.web%m.a00%!M@I*AJ4%y%m(t.web%m.framewor,
!M@I*AJ4%y%m(t.web%m.rte!M@I*AJ4%y%m(t.web%m.webacce%% !M@I*AJ 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M@I*AJ`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M//@*J`
------------ ------------------------ ------------------------ ----------
node* !M//@*J4bo%.m0!M//@*J4bo%.m0/A !M//@*J 0a%%ed
node5 !M//@*J4bo%.m0!M//@*J4bo%.m0/A !M//@*J 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M//@*J`.
C&ec,4 O0eratin( %y%tem 0atc& for `!MEC*@D`
------------ ------------------------ ------------------------ ----------
node* !MEC*@D4bo%.m0!MEC*@D4bo%.m0/A !MEC*@D 0a%%ed
node5 !MEC*@D4bo%.m0!MEC*@D4bo%.m0/A !MEC*@D 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!MEC*@D`.
C&ec,4 O0eratin( %y%tem 0atc& for `!M@DJI/`
------------ ------------------------ ------------------------ ----------
node* !M@DJI/4bo%.rte.bind6cmd% !M@DJI/ 0a%%ed
node5 !M@DJI/4bo%.rte.bind6cmd% !M@DJI/ 0a%%ed
Re%ult4 O0eratin( %y%tem 0atc& c&ec, 0a%%ed for `!M@DJI/`.
C&ec,4 Pac,a(e e<i%tence for `bo%.adt.ba%e`
------------------------------ ------------------------------ ----------------
node* bo%.adt.ba%e4@.J.C.JC 0a%%ed
node5 bo%.adt.ba%e4@.J.C.JC 0a%%ed
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G6 o# $%$
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.adt.ba%e`.
C&ec,4 Pac,a(e e<i%tence for `bo%.adt.lib`
------------------------------ ------------------------------ ----------------
node* bo%.adt.lib4@.J.C.JC 0a%%ed
node5 bo%.adt.lib4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.adt.lib`.
C&ec,4 Pac,a(e e<i%tence for `bo%.adt.libm`
------------------------------ ------------------------------ ----------------
node* bo%.adt.libm4@.J.C.JC 0a%%ed
node5 bo%.adt.libm4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.adt.libm`.
C&ec,4 Pac,a(e e<i%tence for `bo%.0erf.lib0erf%tat`
------------------------------ ------------------------------ ----------------
node* bo%.0erf.lib0erf%tat4@.J.C.JC 0a%%ed
node5 bo%.0erf.lib0erf%tat4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.0erf.lib0erf%tat`.
C&ec,4 Pac,a(e e<i%tence for `bo%.0erf.0erf%tat`
------------------------------ ------------------------------ ----------------
node* bo%.0erf.0erf%tat4@.J.C.JC 0a%%ed
node5 bo%.0erf.0erf%tat4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.0erf.0erf%tat`.
C&ec,4 Pac,a(e e<i%tence for `bo%.0erf.0roctool%`
------------------------------ ------------------------------ ----------------
node* bo%.0erf.0roctool%4@.J.C.JC 0a%%ed
node5 bo%.0erf.0roctool%4@.J.C.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `bo%.0erf.0roctool%`.
C&ec,4 Pac,a(e e<i%tence for `r%ct.ba%ic.rte`
------------------------------ ------------------------------ ----------------
node* r%ct.ba%ic.rte45.A.J.C 0a%%ed
node5 r%ct.ba%ic.rte45.A.J.C 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `r%ct.ba%ic.rte`.
C&ec,4 Pac,a(e e<i%tence for `0erl.rte4@.CCC@`
------------------------------ ------------------------------ ----------------
node* 0erl.rte4@.I.5.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `0erl.rte4@.CCC@`.
C&ec,4 Pac,a(e e<i%tence for `0erl.rte4@./`
------------------------------ ------------------------------ ----------------
node5 0erl.rte4@.I.5.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `0erl.rte4@./`.
C&ec,4 Pac,a(e e<i%tence for `0erl.rte4@.I`
------------------------------ ------------------------------ ----------------
node5 0erl.rte4@.I.5.JC 0a%%ed
Re%ult4 Pac,a(e e<i%tence c&ec, 0a%%ed for `0erl.rte4@.I`.
C&ec,4 Pac,a(e e<i%tence for `0yt&on-5.5-A45.5`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `0yt&on-5.5-A45.5`.
C&ec,4 Pac,a(e e<i%tence for `freeware.Li0.rte45.J`
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G1 o# $%$
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `freeware.Li0.rte45.J`.
C&ec,4 Pac,a(e e<i%tence for `freeware.(cc.rte4J.J.5.C`
------------------------------ ------------------------------ ----------------
Re%ult4 Pac,a(e e<i%tence c&ec, failed for `freeware.(cc.rte4J.J.5.C`.
C&ec,4 2rou0 e<i%tence for `dba`
------------ ------------------------ ------------------------
node* e<i%t% 0a%%ed
node5 e<i%t% 0a%%ed
Re%ult4 2rou0 e<i%tence c&ec, 0a%%ed for `dba`.
C&ec,4 B%er e<i%tence for `nobody`
------------ ------------------------ ------------------------
node* e<i%t% 0a%%ed
node5 e<i%t% 0a%%ed
Re%ult4 B%er e<i%tence c&ec, 0a%%ed for `nobody`.
$y%tem re)uirement failed for Rcr%R
Pre-chec. #or cluster ser>ices setu( ,as unsuccess#ul on all the nodes)
1on-t ,orry a&out the ?Pre-chec. #or cluster ser>ices setu( ,as unsuccess#ul on all the nodes)@
this is a normal message as ,e don-t ,ant all !P! and FI:;S;TS to &e installed)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G7 o# $%$
17)7 Installation
G&i% in%tallation Ku%t &ave to be done only %tartin( from one node. Once t&e fir%t node i% in%talled' Oracle OB!
automatically %tart% t&e co0y of t&e mandatory file% on t&e ot&er% node%' u%in( rc( command. G&i% %te0 %&ould not la%t
lon(. But in any case/ don-t thin. the O4I is stalled/ and loo. at the net,or. tra##ic &e#ore canceling the
installation 0
!s root user on each node/ 1O Create a sym&olic lin. #rom /usr/s&in/lsattr to /etc/lsattr
ln -s /usr/s&in/lsattr /etc/lsattr
?/etc/lsattr@ is used in >i( chec. action
On each node 2
un the !I" command X/usr/s&in/sli&cleanX as XrootX to clean all unre#erenced
li&raries #rom memory 000
{node1:root}/ # ?usr?sbin?slibclean
{node1:root}/ #
Then on node*
From #irst node !s
root user/ e'ecute 2
4nder 3EC Client session/ or other gra(hical inter#ace/ e'ecute 2
{node1:root}/ # 9$ost U
access control disabled< clients can connect from any $osts
{node1:root}/ #
On each node/ set right
o,nershi( and (ermissions
to #ollo,ing directories 2
{node1:root}/ #
{node1:root}/ # ho<n r0:oin0ta"" /r0
{node1:root}/ # hmod ,,% /r0
{node1:root}/ #
{node1:root}/ # ho<n ora"e:oin0ta"" /ora"e
{node1:root}/ # hmod ,,% /ora"e
:ogin as crs or oracle user *crs in our case+ and #ollo, the (rocedure hereunderM
Setu( and e'(ort your
1ISP:!I/ TMP and T;MP
>aria&les
5ith /tm( or other destination ha>ing enough #ree s(ace/ a&out
C66M& on each node)
{node14crs}/ # e9port NIS"(AB0node141
{node1:r0}/ # e9port ;*"0?tmp
{node1:r0}/ # e9port ;/*"0?tmp
{node1:r0}/ # e9port ;*"NIH0?tmp
IF !I"C: release C)$ is
used/ do modi#y the #ile
ora(aram)ini/ and cluster)ini
in 1is.1/installer
u(date entries !I"C766 to !I"C$66 on &oth #iles/ and e'ecute 2
Y/^cdromNmountN(ointV/runInstaller
Or e'ecute 2 )/runInstaller -ignoreSysPrerePs
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G$ o# $%$
Ma.e sure to
e'ecute root(re)sh on
each node &e#ore you
clic. to the ne't ste( *I#
not done yet ,ith C34+)
ma.e an EFS mount
o# the CS 1is.1 on
other nodes/ or remote
co(y #iles to other
nodes/ TD;E run
root(re)sh on each
node 000
Vnode*4cr%Wdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ; l%
cluvfy in%tall root0re r0m runcluvfy.%& u0(rade
doc re%0on%e root0re.%& run!n%taller %ta(e welcome.&tml
Vnode*4cr%Wdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ;
Vnode*4cr%Wdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ; )/runInstaller
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
Mour 0latform re)uire% t&e root u%er to 0erform certain 0re-in%tallation
O$ 0re0aration. G&e root u%er %&ould run t&e %&ell %cri0t Rroot0re.%&R before
you 0roceed wit& Oracle in%tallation. root0re.%& can be found at t&e to0 level
of t&e CD or t&e %ta(e area.
An%wer RyR if root &a% run Rroot0re.%&R %o you can 0roceed wit& Oracle
in%tallation.
An%wer RnR to abort in%tallation and t&en a%, root to run Rroot0re.%&R.
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
Das Qroot(re)shQ &een run &y rootR \y/n] *n+
$tartin( Oracle Bniver%al !n%taller...
C&ec,in( Gem0 %0ace4 mu%t be (reater t&an *DC "B. Actual *5*D "B Pa%%ed
C&ec,in( %wa0 %0ace4 mu%t be (reater t&an *@C "B. Actual @*5 "B Pa%%ed
C&ec,in( monitor4 mu%t be confi(ured to di%0lay at lea%t 5@/ color%. Actual */EEE5*/ Pa%%ed
Pre0arin( to launc& Oracle Bniver%al !n%taller from tm0Ora!n%tall5CCI-C5-CE6**-CA-J@A". Plea%e wait
...Vnode*4cr%Wdi%trib$oftwareOraclerdbm%**(r*ai<clu%terware ; Oracle Bniver%al !n%taller' Ner%ion **.*.C./.C
Production
Co0yri(&t -C. *DDD' 5CCE' Oracle. All ri(&t% re%erved.
!t the O4I 5elcome %creen
Mou can c&ec, if any oracle 0roduct i%
already in%talled' clic, on 3!n%talled
Product%T 4
Just clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GB o# $%$
By default' Oracle Bniver%al in%taller
may try to create ora!nventory
directory' but OB! &a% no ri(&t% to do
%o.
GFE> you may &ave t&i% me%%a(e.
#u%t clic, 3OXT to modify !nventory
location de%tination.
S(eci#y In>entory directory and credentials 2
,here you ,ant to create the
in>entory directory 2
/oracle/oraIn>entoty
O0eratin( %y%tem (rou0 name %&ould
be %et a% oin%tall

Then clic. Ee't ))) to Continue )))
S(eci#y Dome 1etails
$0ecify an ORACLE6FO"E name and
de%tination directory for t&e CR$ in%tallation.
G&e de%tination directory %&ould be out of t&e
UORACLE6BA$E
!dd Product :anguages i# necessay)
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GC o# $%$
This ste( ,ill 2
C&ec, o0eratin( %y%tem re)uirement% ...
C&ec, o0eratin( %y%tem 0ac,a(e re)uirement% ...
C&ec, recommended o0eratin( %y%tem 0atc&e% S
C&ec, 0&y%ical memory re)uirement% ...
C&ec, for Oracle Fome incom0atibilitie% ....
C&ec, Oracle Fome 0at& for %0ace%...
C&ec, Clu%ter file%...
C&ec, ,ernel...
C&ec, uid(id...
C&ec, ma<mimum command line len(t& ar(ument' ncar(...
C&ec, local Clu%ter $ync&roniLation $ervice% -C$$. %tatu% ...
C&ec, w&et&er Oracle D.5 RAC i% available on all %elected node%
C&ec, Oracle Di OCR 0artition %iLe ...
Product-S(eci#ic
PrerePuisite Chec.s 2
All c&ec,% %&ould be in
%tatu% 3$ucceededT.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GG o# $%$
1etails o# the
(rerePuisite
chec.s done
&y
runInstaller
Chec.ing o(erating system rePuirements )))
E<0ected re%ult4 One of @JCC.C@'/*CC.CC
Actual Re%ult4 @JCC.CE
C&ec, com0lete. G&e overall re%ult of t&i% c&ec, i%4 Pa%%ed
99999999999999999999999999999999999999999999999999999999
Chec.ing o(erating system (ac.age rePuirements )))
C&ec,in( for bo%.adt.ba%e-C.C.b found bo%.adt.ba%e-@.J.E.C.. Pa%%ed
C&ec,in( for bo%.adt.lib-C.C.b found bo%.adt.lib-@.J.C./C.. Pa%%ed
C&ec,in( for bo%.adt.libm-C.C.b found bo%.adt.libm-@.J.E.C.. Pa%%ed
C&ec,in( for bo%.0erf.lib0erf%tat-C.C.b found bo%.0erf.lib0erf%tat-@.J.E.C.. Pa%%ed
C&ec,in( for bo%.0erf.0erf%tat-C.C.b found bo%.0erf.0erf%tat-@.J.E.C.. Pa%%ed
C&ec,in( for bo%.0erf.0roctool%-C.C.b found bo%.0erf.0roctool%-@.J.E.C.. Pa%%ed
C&ec,in( for r%ct.ba%ic.rte-C.C.b found r%ct.ba%ic.rte-5.A.I.C.. Pa%%ed
C&ec,in( for r%ct.com0at.client%.rte-C.C.b found r%ct.com0at.client%.rte-5.A.I.C.. Pa%%ed
C&ec,in( for bo%.m0/A-@.J.C.@/.b found bo%.m0/A-@.J.E.*.. Pa%%ed
C&ec,in( for bo%.rte.libc-@.J.C.@@.b found bo%.rte.libc-@.J.E.*.. Pa%%ed
C&ec,in( for <lC.ai<@C.rte-I.C.C.E.b found <lC.ai<@C.rte-D.C.C.*.. Pa%%ed
C&ec,in( for <lC.rte-I.C.C.E.b found <lC.rte-D.C.C.*.. Pa%%ed
99999999999999999999999999999999999999999999999999999999
Chec.ing recommended o(erating system (atches
C&ec,in( for !MIDCIC-bo%.rte.aio'@.J.C.@*.b found -bo%.rte.aio'@.J.E.C.. Pa%%ed
C&ec,in( for !MD5CJE-bo%.rte.aio'@.J.C.@5.b found -bo%.rte.aio'@.J.E.C.. Pa%%ed
C&ec,in( for !MDAJAJ-bo%.rte.lvm'@.J.C.@@.b found -bo%.rte.lvm'@.J.E.C.. Pa%%ed
99999999999999999999999999999999999999999999999999999999
Chec.ing (hysical memory rePuirements )))
E<0ected re%ult4 D55"B
Actual Re%ult4 5CAI"B
99999999999999999999999999999999999999999999999999999999
Chec.ing #or Oracle Dome incom(ati&ilities ))))
Actual Re%ult4 >EW6FO"E
99999999999999999999999999999999999999999999999999999999
Chec.ing Oracle Dome (ath #or s(aces)))
99999999999999999999999999999999999999999999999999999999
Chec.ing Cluster #iles)))
99999999999999999999999999999999999999999999999999999999
Chec.ing .ernel)))
99999999999999999999999999999999999999999999999999999999
Chec.ing uid/gid)))
99999999999999999999999999999999999999999999999999999999
Chec.ing ma'mimum command line length argument/ ncarg)))
99999999999999999999999999999999999999999999999999999999
Chec.ing local Cluster Synchroni<ation Ser>ices *CSS+ status )))
99999999999999999999999999999999999999999999999999999999
Chec.ing ,hether Oracle %)7 !C is a>aila&le on all selected nodes
99999999999999999999999999999999999999999999999999999999
Chec.ing Oracle %i OC (artition si<e )))
99999999999999999999999999999999999999999999999999999999

11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GK o# $%$
Just to remem&er 000
Pu&lic/ Pri>ate/ and 3irtual Dost Eame
layout
Eet,or.
Inter#ace name
Dost Ty(e 1e#ined Eame !ssigned IP O&ser>ation
en6
Pu&lic
Dostname
node1 16)$)7C)81 !C Pu&lic node name
*Pu&lic Eet,or.+
3irtual
Dostname
node1->i( 16)$)7C)181 !C 3IP node name
*Pri>ate Eet,or.+
en1 Pri>ate
Dostame
node1-rac 16)16)7C)81 !C Interconnect node name
*Pri>ate Eet,or.+
en6
Pu&lic
Dostname
node7 16)$)7C)87 !C Pu&lic node name
*Pu&lic Eet,or.+
3irtual
Dostname
node7->i( 16)$)7C)187 !C 3IP node name
*Pri>ate Eet,or.+
en1 Pri>ate
Dostame
node7-rac 16)16)7C)87 !C Interconnect node name
*Pri>ate Eet,or.+
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G8 o# $%$
S(eci#y Cluster con#iguration
2
Mou mu%t 4
2ive a uni)ue name to t&e
clu%ter
cr%6clu%ter
Edit t&e default node to
validate entrie%
Add t&e entrie% for any new
node' node5 in our ca%e
For each cluster node/ you must 2
s(eci#y one &y one
Pu&lic Eode Eame/
Public corre%0ond to !P addre%%
lin,ed to t&e 0ublic networ,'
u%ually !P lin,ed to &o%tname.
Pu&lic Eode Eame W
node1
Pu&lic Eode Eame W
node7
Pri>ate Eode Eame
Private corre%0ond to !P addre%%
lin,ed to t&e RAC interconnect.
Pri>ate Eode Eame W
node1-rac
Pri>ate Eode Eame W
node7-rac
3irtual Dost Eame
Nirtual Fo%t >ame corre%0ond to
a free re%erved !P addre%% on
t&e 0ublic networ, to be u%e for
Oracle clu%terware N!P.
3irtual Dost Eame W
node1->i(
3irtual Dost Eame W
node7->i(
5hen done 000 clic. Ee't )))
I# you ha>e (ro&lems at this stage ,hen clic.ing on Ee't *error
messages+ Chec. your net,or. con#iguration/ and 4ser ePui>alence
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G% o# $%$
Just to remem&er 000
Pu&lic/ Pri>ate/ and 3irtual Dost Eame
layout
Eet,or.
Inter#ace
name
Dost Ty(e 1e#ined
Eame
!ssigned IP S4BE;T O&ser>ation
en6
Pu&lic
Dostname
node1 16)$)7C)81 16)$)7C)6 !C Pu&lic node name
*Pu&lic Eet,or.+
3irtual
Dostname
node1->i( 16)$)7C)181 16)$)7C)6 !C 3IP node name
*Pri>ate Eet,or.+
en1 Pri>ate
Dostame
node1-rac 16)16)7C)81 16)16)7C)6 !C Interconnect node name
*Pri>ate Eet,or.+
en6
Pu&lic
Dostname
node7 16)$)7C)87 16)$)7C)6 !C Pu&lic node name
*Pu&lic Eet,or.+
3irtual
Dostname
node7->i( 16)$)7C)187 16)$)7C)6 !C 3IP node name
*Pri>ate Eet,or.+
en1 Pri>ate
Dostame
node7-rac 16)16)7C)87 16)16)7C)6 !C Interconnect node name
*Pri>ate Eet,or.+
S(eci#y Eet,or. Inter#ace 4sage
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K6 o# $%$
For each entry *en6/en1/en7+/ Clic. ?;dit@ to S(eci#y ?Inter#ace Ty(e@ for eac& networ, card corre%0ond to t&e
0ublic networ,' and w&ic& one corre%0ond to t&e 0rivate networ, -RAC !nterconnect..

In our e'am(le/ ,ith or ,ithout !C
Interconnect &ac.u( im(lementation 2
en6 *16)$)7C)6+ mu%t e<i%t a%
3PublicT on eac& node.
en1 *16)16)7C)6+ mu%t e<i%t a%
3PrivateT on eac& node.
en7 *76)76)7C)6+ ?Do >ot B%eT.
!n other enR M Do >ot B%e
Chec. and Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K1 o# $%$
!t this stage/ you mu%t &ave already confi(ured t&e %&ared %tora(e. At lea%t for 4
* Oracle Clu%ter Re(i%try Di%,
* Notin( Di%,
OCR and votin( di%,% 0rotected by
e<ternal mec&ani%m.
Eo Oracle Cluster,are redundancy
mechanism

devocr6di%,*
devvotin(6di%,*
O
5 Oracle Clu%ter Re(i%try Di%,%
-OCR "irrorin(.
J Notin( Di%,% -Notin(
Redundancy.
Oracle Cluster,are redundancy
mechanism #or OC and 3oting dis.s
devocr6di%,*
devocr6di%,5
devvotin(6di%,*
devvotin(6di%,5
devvotin(6di%,J
In our e'am(le/ 5e ,ill im(lement 7 Oracle Cluster egistry 1is.s/ and $ 3oting 1is.s)
!:: >irtual de>ices must be reac&able by all node% 0artici0atin( to t&e RAC clu%ter in concurrency mode.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K7 o# $%$
devocr6di%,*
devocr6di%,5
In our case' we will %0ecify ?EOM!: edundancy@ a% de%cribed
bellow 4
S(eci#y OC Con#iguration' by %electin( 4
;'ternal edundancy ->o OCR mirrorin(
by Oracle Clu%terware' %&ould be 0rovided
by ot&er% o0tion%' di%,% mana(ement' etc
S.
I#
?;'ternal
edunda
ncy@
selected'
%0ecify
raw di%,
location a%
follow 4
devocr6di%,*
O
Eormal edundancy -OCR mirrorin( by
Oracle Clu%terware.
I#
?EOM!
:
edunda
ncy@
selected'
%0ecify
raw di%,
location a%
follow 4
devocr6di%,*
devocr6di%,5
%ecif0 the O!R location E t*is must be
a s*ared location on t*e s*ared stora%e
reac*able )rom all nodes.
!nd you must ha>e the read/,right
(ermissions on this shared location #rom all
nodes)
Then clic. Ee't )))
I# (ro&lems ha((ens at this stage/ do >eri#iy
that location s(eci#ied does e'ist/ and is
reacha&le #rom each !I" node/ ,ith right
read/,rite access/ and user/grou( o,ner)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K$ o# $%$
devvotin(6di%,*
devvotin(6di%,5
devvotin(6di%,J
In our case' we will %0ecify ?Eormal edundancy@ a% de%cribed
bellow 4
S(eci#y 3oting 1is. Con#iguration' by
%electin( 4
;'ternal edundancy ->o Notin( co0ie%
mana(ed by Oracle Clu%terware' %&ould
be 0rovided by ot&er% o0tion%' di%,%
mana(ement' etc S.
I#
?;'ternal
edunda
ncy@
selected'
%0ecify
raw di%,
location a%
follow 4
devvotin(6di%,*
O
Eormal edundancy -Notin( co0ie%
mana(ed by Oracle Clu%terware.
I#
?EOM!
:
edunda
ncy@
selected'
%0ecify
raw di%,
location a%
follow 4
devvotin(6di%,*
devvotin(6di%,5
devvotin(6di%,J
%ecif0 the Joting 3is= location7 t*is
must be a s*ared location on t*e s*ared
stora%e reac*able )rom all nodes.
!nd you must ha>e the read/,right
(ermissions on this shared location #rom all
nodes)
Then clic. Ee't )))
I# (ro&lems ha((ens at this stage/ do >eri#iy
that location s(eci#ied does e'ist/ and is
reacha&le #rom each !I" node/ ,ith right
read/,rite access/ and user/grou( o,ner)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KB o# $%$
Summary 2
C&ec, Clu%ter >ode% and Remote >ode%
li%t%.
G&e OB! will in%tall t&e Oracle CR$
%oftware on to t&e local node' and t&en
co0y t&i% information to t&e ot&er %elected
node%.
C&ec, t&at all node% are li%ted in 3Clu%ter
>ode%T 4
Then clic. Install )))
Install 2
The Oracle 4ni>ersal Installer ,ill
(roceed the installation on the #irst
node' t&en will co0y automatically t&e
code on t&e 5 ot&er% %elected node%.
Just ,ait #or the ne't screen )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KC o# $%$
;'ecute Con#iguration
Scri(ts 2
T;;P TDIS 5IE1O5S
OP;E
!E1 1o e'ecute scri(ts
in the #ollo,ing order/
,aiting #or each to
succeed &e#ore running
the ne't one 000
!S root 2
* On node*' E<ecute
orain%tRoot.%&
5 On node5' E<ecute
orain%tRoot.%&
J On node*' E<ecute
root.%&
A On node5' E<ecute
root.%&
orainstRoot.sh 6
;'ecute the orainstoot)sh on all nodes)
G&e file i% located in UORACLE6BA$Eora!nventory -Ora!nventory &ome. on eac& node%
=oracle=oraInventory in our case.
On node1 as root/ e'ecute
oracleora!nventoryorain%tRoot.%
&
{node1:root}/ora"e/ora'n7entory # $/orainstHoot.s$
C$an+in+ permissions of ?oracle?oraInventory to ''&.
C$an+in+ +roupname of ?oracle?oraInventory to oinstall.
;$e e9ecution of t$e script is complete
{node1:root}/ora"e/ora'n7entory #
TD;E On node7 as root/ e'ecute
oracleora!nventoryorain%tRoot.%&
{node*:root}/ora"e/ora'n7entory # .?orainstHoot.s$
C$an+in+ permissions of ?oracle?oraInventory to ''&.
C$an+in+ +roupname of ?oracle?oraInventory to oinstall.
;$e e9ecution of t$e script is complete
{node*:root}/ora"e/ora'n7entory #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KG o# $%$
Be#ore running root)sh scri(t on each node/ (lease chec. the #ollo,ing 2
C&ec, if your 0ublic networ, card -enC in our ca%e. i% a %tandard networ, ada0ter' or a virtual networ, et&ernet.
!%%ue t&e followin( command a% root 4
Iou should get the
#ollo,ing i# en6 is a
normal net,or.
inter#ace on each
node *node1/ node7+ 2
Hnode1ErootI = 3K entstat d en6
-------------------------------------------------------------
!TN!BN!T ST6T'ST'CS Fen&G :
(e7ie Type: *-Port &igabit !thernet-SK PC'-K 6dapter F131#55#*G
Nard<are 6ddre00: ##:#-:,b:ee:,1:8
!"ap0ed Time: # day0 *# hour0 ) minute0 3- 0eond0
Tran0mit Stati0ti0: Beei7e Stati0ti0:
-------------------- -------------------
Pa:et0: # Pa:et0: #
2yte0: # 2yte0: #
'nterrupt0: # 'nterrupt0: #
Tran0mit !rror0: # Beei7e !rror0: #
Pa:et0 (ropped: # Pa:et0 (ropped: #
2ad Pa:et0: #
Ma1 Pa:et0 on S/4 Tran0mit [ueue: 1
S/4 Tran0mit [ueue .7er8"o<: #
Current S/4DN/4 Tran0mit [ueue Hength: 1
2roada0t Pa:et0: # 2roada0t Pa:et0: #
Mu"tia0t Pa:et0: # Mu"tia0t Pa:et0: #
No Carrier Sen0e: # CBC !rror0: #
(M6 Underrun: # (M6 .7errun: #
Ho0t CTS !rror0: # 6"ignment !rror0: #
Ma1 Co""i0ion !rror0: # No Be0oure !rror0: #
Hate Co""i0ion !rror0: # Beei7e Co""i0ion !rror0: #
(e8erred: # Pa:et Too Short !rror0: #
S[! Te0t: # Pa:et Too Hong !rror0: #
Timeout !rror0: # Pa:et0 (i0arded by 6dapter: #
Sing"e Co""i0ion Count: # Beei7er Start Count: #
Mu"tip"e Co""i0ion Count: #
Current N4 Tran0mit [ueue Hength: 1
&enera" Stati0ti0:
-------------------
No mbu8 !rror0: #
6dapter Be0et Count: #
6dapter (ata Bate: *###
(ri7er A"ag0: Up 2roada0t Simp"e1
Himbo ,32itSupport Che:0um.88"oad
Pri7ateSegment HargeSend (ataBateSet
*-Port &igabit !thernet-SK PC'-K 6dapter F131#55#*G Spei8i Stati0ti0:
--------------------------------------------------------------------
(ink Status 4 6p
Media Speed Se"eted: 6uto negotiation
Media Speed Bunning: Un:no<n
PC' Mode: PC'-K F1##-1))G
PC' 2u0 4idth: ,3-bit
Hateny Timer: 133
Cahe Hine SiLe: 1*5
Jumbo Arame0: (i0ab"ed
TCP Segmentation .88"oad: !nab"ed
TCP Segmentation .88"oad Pa:et0 Tran0mitted: #
TCP Segmentation .88"oad Pa:et !rror0: #
Tran0mit and Beei7e A"o< Contro" Statu0: (i0ab"ed
Tran0mit and Beei7e A"o< Contro" Thre0ho"d FNighG: 3%#%,
Tran0mit and Beei7e A"o< Contro" Thre0ho"d FHo<G: *3%+,
Tran0mit and Beei7e Storage 6""oation FTK/BKG: 1,/35
Mou %&ould (et t&e followin( if en6 -0ublic networ,
interface. i% a normal networ, interface 4
!1 %o' (o to t&e ne<t %te0' e<ecutin( t&e root.%& %cri0t
Hnode1ErootI= J
entstat -d en6 Z gre( -i; X)Slin.)Sstatus)S2)Su()SX
?in= %tatus 6 8
Hnode1ErootI= J
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KK o# $%$
O
Iou should get the
#ollo,ing i# en6 is a
normal net,or.
inter#ace 2
{node1:root} / -C entstat 1d en&
!TN!BN!T ST6T'ST'CS Fen#G :
Nevice ;ype4 Iirtual I?J /t$ernet Adapter El-lanF
Nard<are 6ddre00: ee:%1:,#:##:1#:#*
!"ap0ed Time: # day0 *# hour0 ) minute0 *3 0eond0
Tran0mit Stati0ti0: Beei7e Stati0ti0:
-------------------- -------------------
Pa:et0: 1),1%, Pa:et0: )1-3-*
2yte0: 1-%#%%,1 2yte0: 13*#,-))-
'nterrupt0: # 'nterrupt0: *5%***
Tran0mit !rror0: # Beei7e !rror0: #
Pa:et0 (ropped: # Pa:et0 (ropped: #
2ad Pa:et0: #
Ma1 Pa:et0 on S/4 Tran0mit [ueue: #
S/4 Tran0mit [ueue .7er8"o<: #
Current S/4DN/4 Tran0mit [ueue Hength: #
2roada0t Pa:et0: *#5 2roada0t Pa:et0: *315)1
Mu"tia0t Pa:et0: * Mu"tia0t Pa:et0: #
No Carrier Sen0e: # CBC !rror0: #
(M6 Underrun: # (M6 .7errun: #
Ho0t CTS !rror0: # 6"ignment !rror0: #
Ma1 Co""i0ion !rror0: # No Be0oure !rror0: #
Hate Co""i0ion !rror0: # Beei7e Co""i0ion !rror0: #
(e8erred: # Pa:et Too Short !rror0: #
S[! Te0t: # Pa:et Too Hong !rror0: #
Timeout !rror0: # Pa:et0 (i0arded by 6dapter: #
Sing"e Co""i0ion Count: # Beei7er Start Count: #
Mu"tip"e Co""i0ion Count: #
Current N4 Tran0mit [ueue Hength: #
&enera" Stati0ti0:
-------------------
No mbu8 !rror0: #
6dapter Be0et Count: #
6dapter (ata Bate: *####
(ri7er A"ag0: Up 2roada0t Bunning
Simp"e1 ,32itSupport (ataBateSet
Iirtual I?J /t$ernet Adapter El-lanF Specific Statistics4
---------------------------------------------------------
B[ Hength: 3351
No Copy 2u88er0: #
Trun: 6dapter: Aa"0e
Ai"ter MCa0t Mode: Aa"0e
Ai"ter0: *%% !nab"ed: 1 [ueued: # .7er8"o<: #
(A# State4 Jperational
2u88er0 Beg 6""o Min Ma1 Ma16 Ho<Beg
tiny %1* %1* %1* *#35 %1* %#-
0ma"" %1* %1* %1* *#35 %%) %#*
medium 1*5 1*5 1*5 *%, 1*5 1*5
"arge *3 *3 *3 ,3 *3 *3
huge *3 *3 *3 ,3 *3 *3
O Mou %&ould (et t&e followin( if
enC i% a virtual networ, interface 4
!1 %o' donPt worry' **( ra(vi0 %cri0t
&a% been u0dated to ta,e care of
t&i% ca%e..
entstat -d en& 2 +rep -i/ A.Plan state4.Poperational.PA
{node1:root}/ # ent0tat Sd en# \ grep -i! Z$="an
State:$=operationa"$=Z
H6N State: .perationa"
{node1:root}/ #
in t&e UCR$6FO"Ebinrac(vi0 %cri0t' you will &ave to modify t&e followin( 4
Y;ETST!T -d YNIF Z Y=;P -i;P X)Slin.)Sstatus)S2)Su()SX
Go be re0laced by
Y;ETST!T -d YNIF Z Y=;P -i;P X)Slan state2)So(erational)SX
Wit& *C(RAC it wa% ,now a% 3Bug BB$KBG%2 !C=3IP EOT 5OTIE= OE S;3; 5ITD ;TD;E;T
3IT4!:IS!TIOE@ *This ,as only rePuired i# using !I" 3irtual Inter#aces #or the Oracle 1ata&ase 16)7)6)1 !C
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K8 o# $%$
(u&lic net,or.+
!t this stage/ you should e'ecute ?root)sh@ scri(t 2
Start ,ith node 1 and ,ait #or the result before e<ecutin( on node 5.
G&i% file i% located in UCR$6FO"E directory on eac& node -3cr%**.*.Cin%tallT in our ca%e..
OE:I For in#ormation/ The root)sh scri(t is e'ecuting t,o su& scri(ts/ and one is the rootcon#ig)sh scri(t
,hich has interresting in#ormation to ha>e a loo. at 2
1O not modi#y the #ile unless it-s necessary &e#ore running the root)sh scri(t 000
I# you ha>e to ma.e a modi#ication/ do it on all nodes 000
5hat does the rootcon#ig)sh scri(t e'ecuted &y root)sh 2
# rooton8ig$0h 8or .ra"e CBS home0
#
# Thi0 i0 run one per node during the .ra"e CBS in0ta""$
# Thi0 0ript doe0 the 8o""o<ing:
# 1G Stop i8 any &S(0 are running 8rom -$1 ora"e home0
# *G 'nitia"iLe ne< .CB de7ie or upgrade the e1i0ting .CB de7ie
# )G Setup .CB 8or running CBS 0ta:
# 3G Copy the CBS init 0ript to init$d 8or init proe00 to 0tart
# %G Start the CBS 0ta:
# ,G Con8igure Node6pp0 i8 CBS i0 up and running on a"" node0
3aria&les used &y root)sh scri(t/
the >alues are the result o# your in(uts in the Oracle Cluster,are 4ni>ersal Installer)
Iou can chec. the >alues to see i# there are OT)
S'H!NTO8a"0e
.B6?CBS?N.M!O/r0/11$1$#
CBS?.B6CH!?.4N!BOr0
CBS?(26?&B.UPOdba
CBS?/N(B?CHUST!BO8a"0e
CBS?.CB?H.C6T'.NSO/de7/or?di0:1Q/de7/or?di0:*
CBS?CHUST!B?N6M!Or0
CBS?N.ST?N6M!?H'STOnode1Q1Qnode*Q*
CBS?N.(!?N6M!?H'STOnode1Q1Qnode*Q*
CBS?PB'/6T!?N6M!?H'STOnode1-raQ1Qnode*-raQ*
CBS?H6N&U6&!?'(OP6M!B'C6N?6M!B'C6$4!5'S.55%-P1P
CBS?/.T'N&?('SUSO/de7/7oting?di0:1Q/de7/7oting?di0:*Q/de7/7oting?di0:)
CBS?N.(!H'STOnode1Qnode*
CBS?N.(!/'PSOPnode1/node1-7ip/*%%$*%%$*%%$#/en#Qnode*/node*-7ip/*%%$*%%$*%%$#/en#P
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K% o# $%$
FIST On node1
!s root/ E<ecute cr%**.*.Croot.%&
W&en fini%&ed' C$$ deamon %&ould be active on node *.
Chec. #or line ?CSS is acti>e on these nodes)
node1
{node1:root}/r0/11$1$# # .?root.s$
!AH#I#74 directory K?crsK is not owned by root
Che:ing to 0ee i8 .ra"e CBS 0ta: i0 a"ready on8igured
/et/ora"e doe0 not e1i0t$ Creating it no<$
Setting the permi00ion0 on .CB ba:up diretory
Setting up Net<or: 0o:et diretorie0
.ra"e C"u0ter Begi0try on8iguration upgraded 0ue008u""y
The diretory P/r0P i0 not o<ned by root$ Changing o<ner to root
Sue008u""y aumu"ated nee00ary .CB :ey0$
U0ing port0: CSSO3-5-% CBSO3-5-, !/MCO3-5-5 and !/MBO3-5-+$
node TnodenumberC: TnodenameC Tpri7ate interonnet nameC Tho0tnameC
node 1: node1 node1-ra node1
node *: node* node*-ra node*
Creating .CB :ey0 8or u0er ProotPQ pri7grp P0y0temP$$
.peration 0ue008u"$
No< 8ormatting 7oting de7ie: /de7/7oting?di0:1
No< 8ormatting 7oting de7ie: /de7/7oting?di0:*
No< 8ormatting 7oting de7ie: /de7/7oting?di0:)
Aormat o8 ) 7oting de7ie0 omp"ete$
Startup <i"" be Jueued to init <ithin )# 0eond0$
6dding daemon0 to inittab
!1peting the CBS daemon0 to be up <ithin ,## 0eond0$
Cluster Sync$roniOation Services is active on t$ese nodes.
node1
Cluster Sync$roniOation Services is inactive on t$ese nodes.
node2
Hoa" node he:ing omp"ete$ Bun root$0h on remaining node0 to 0tart CBS daemon0$
{node1:root}/r0/11$1$# #
DonPt worry about 3WAR>!>24 directory Rcr%R i% not owned by rootT
G&i% i% Ku%t a me%%a(e to for(et
IF CSS is not acti>e at the end o# the root)sh scri(t 2
C&ec, your networ, ' %&ared di%,% confi(uration' and owner and acce%% 0ermi%%ion% -readwrite. on OCR and Notin(
di%,% from eac& 0artici0atin( node. And e<ecute a(ain t&e root.%& %cri0t on node &avin( t&e 0roblem.
!f CC$ %tart on one node' but not on t&e ot&er%' C&ec, %&ared Di%,% -OCRNotin(. for concurrent readwrite acce%%
from all node%' u%in( uni< dd command.
!f A$" or 2P1$ i% im0lemented wit& FAC"P in%talled and confi(ured for ot&er 0ur0o%e% t&en &avin( databa%e on
concurrent raw device%' Mou mu%t declare di%,% re%%ource% in FAC"P to be able to %tart t&e CR$ -C$$..
!f A$" or 2P1$ i% im0lemented' and FAC"P i% in%talled but not u%ed at all' GFE> remove FAC"P or declare di%,%
re%%ource% in FAC"P to be able to %tart t&e CR$ -C$$..
11g!C/!SM/!I"
oraclibm@fr.ibm.com
186 o# $%$
The IBM !I" clustering layer/ D!CMP #ilesets/ M4ST EOT &e installed i# you->e chosen an
im(lementation ,ithout D!CMP) !f t&i% layer i% im0lemented for ot&er 0ur0o%e' di%,% re%%ource% nece%%ary to
in%tall and run CR$ data will &ave to be 0art of an FAC"P volume (rou0 re%ource.
I# you ha>e (re>iously installed D!CMP/ you must remo>e 2
FAC"P file%et% -clu%ter.e%.Z.
r%ct.&acm0.rte
r%ct.com0at.ba%ic.&acm0.rte
r%ct.com0at.client%.&acm0.rte
I# you did run a #irst installation o# the Oracle Cluster,are *CS+ ,ith D!CMP installed/
Chec. i# /o(t/OC:cluster directory does e'ist and i# so/ remo>e it on all nodes)
TO B; !B:; TO 4E !=!IE the root)sh scri(t on the node/ you must 2
Eit&er
Clean t&e failed CR$ in%tallation' and %tart a(ain t&e CR$ in%tallation 0rocedure.
"etalin, >ote 5JDDDI.* - *C( RAC4 Fow to Clean B0 After a 1ailed CR$ !n%tall -%ame 0rocedure for *C( and
**(.
Only $u00orted met&od by Oracle.
OR
1o the #ollo,ing 9ust to #ind out and sol>e the (ro&lem wit&out in%tallin( a(ain at eac& try' !nd ,hen
sol>ed/ #ollo, again the Metalin. Eote 7$%%%8)1 - *C( RAC4 Fow to Clean B0 After a 1ailed CR$ !n%tall to
clean 0ro0erly t&e %y%tem' and %tart a(ain t&e in%tallation a% %u00orted by oracle.
!s root user on each node 2
1o e'ecute {node1:root}/ # ]CBS?N.M!/bin/r0t" 0top r0
to clean any remainin+ crs deamons
{node1:root}/ # rmitab h1
t$is will remove oracle CHS entry in t$e ?etc?inittab
{node1:root}/ # rmitab h*
{node1:root}/ # rmitab h)
{node1:root}/ # rm SB8 /opt/.BCH=
kill remainin+ process from output 4 ps-ef2+rep crs and ps-ef2+rep d.bin
{node1:root}/ # rm SB /et/ora"e/=
O e'ecute Vnode*4rootW ; UCR$6FO"Ein%tallrootdelete.%& on eac& node
TD;E
For all OC
and 3oting
1is.s 2
C&an(e owner' (rou0 -oracle4dba. and 0ermi%%ion -//C. for devocr6di%,* and devvotin(6di%,*
on eac& node of t&e clu%ter on node* and node5 S
Era%e OCR and Notin( di,% content 1ormat -[eroin(. on t&e di%,% from one node 4
{node1:root}/ # dd i8O/de7/Lero o8O/de7/or?di0:1 b0O1#*3 ountO)## a
{node1:root}/ # dd i8O/de7/Lero o8O/de7/or?di0:* b0O1#*3 ountO)## a
{node1:root}/ # dd i8O/de7/Lero o8O/de7/7oting?di0:1 b0O1#*3 ountO)## a
{node1:root}/ # dd i8O/de7/Lero o8O/de7/7oting?di0:) b0O1#*3 ountO)## a
{node1:root}/ # dd i8O/de7/Lero o8O/de7/7oting?di0:) b0O1#)3 ountO)## a
11g!C/!SM/!I"
oraclibm@fr.ibm.com
181 o# $%$
TD;E On node7
!s root' E<ecute
cr%**.*.Croot.%&
5hen #inished/ CSS deamon should &e acti>e on node 1/ 7)
Iou should ha>e the #ollo,ing #inal result 2
CSS is acti>e on these nodes)
node1
node7
I# CSS is not acti>e on all nodes/ or on one o# the nodes' t&i% mean% t&at you
could &ave a 0roblem wit& t&e networ, confi(uration' or t&e %&ared di%,% confi(uration for
acce%%in( OCR and Notin( Di%,%.
C&ec, your networ, ' %&ared di%,% confi(uration' and owner and acce%% 0ermi%%ion%
-readwrite. on OCR and Notin( di%,% from eac& 0artici0atin( node. And e<ecute a(ain t&e
root.%& %cri0t on node &avin( t&e 0roblem.
Chec. also as oracle user the #ollo,ing command #rom each node 2
{node1:r0}/r0/11$1$# # /r0/11$1$#/bin/olsnodes
node1
node2
{node1:r0}/r0/11$1$# # r0h node*
{node*:r0}/r0/11$1$# # olsnodes
node1
node2
{node*:r0}/r0/11$1$# #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
187 o# $%$
TD;E On node7
!s root/ E<ecute cr%**.*.Croot.%&
W&en fini%&ed' C$$ deamon %&ould be active on node * and 5.
Chec. #or line ?CSS is acti>e on these nodes)
node1
node7
{node*:root}/r0/11$1$# # .?root.s$
!AH#I#74 directory K?crsK is not owned by root
Che:ing to 0ee i8 .ra"e CBS 0ta: i0 a"ready on8igured
/et/ora"e doe0 not e1i0t$ Creating it no<$
Setting the permi00ion0 on .CB ba:up diretory
Setting up Net<or: 0o:et diretorie0
.ra"e C"u0ter Begi0try on8iguration upgraded 0ue008u""y
The diretory P/r0P i0 not o<ned by root$ Changing o<ner to root
"08g: !K'ST'N& on8iguration 7er0ion 3 deteted$
"08g: 7er0ion 3 i0 11 Be"ea0e 1$
Sue008u""y aumu"ated nee00ary .CB :ey0$
U0ing port0: CSSO3-5-% CBSO3-5-, !/MCO3-5-5 and !/MBO3-5-+$
node TnodenumberC: TnodenameC Tpri7ate interonnet nameC Tho0tnameC
node 1: node1 node1-ra node1
node *: node* node*-ra node*
"08g: 6rgument0 he: out 0ue008u""y$
N. U!IS 4!B! 4B'TT!N$ Supp"y -8ore parameter to o7erride$
-8ore i0 de0truti7e and <i"" de0troy any pre7iou0 "u0ter
on8iguration$
.ra"e C"u0ter Begi0try 8or "u0ter ha0 a"ready been initia"iLed
Startup <i"" be Jueued to init <ithin )# 0eond0$
!1peting the CBS daemon0 to be up <ithin ,## 0eond0$
Cluster Sync$roniOation Services is active on t$ese nodes.
node1
node2
Cluster Sync$roniOation Services is active on all t$e nodes.
4aiting 8or the .ra"e CBS( and !/M( to 0tart
.ra"e CBS 0ta: in0ta""ed and running under initF1MG
Hunnin+ vipcaEsilentF for confi+urin+ nodeapps
Creatin+ II" application resource on E2F nodes...
Creatin+ 7SN application resource on E2F nodes...
Creatin+ J#S application resource on E2F nodes...
Startin+ II" application resource on E2F nodes...
Startin+ 7SN application resource on E2F nodes...
Startin+ J#S application resource on E2F nodes...
None.
{node*:root}/r0/11$1$# #
DonPt worry about 3WAR>!>24 directory Rcr%R i% not owned by rootT
G&i% i% Ku%t a me%%a(e to for(et
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18$ o# $%$
On the second node/ at he end o# the root)sh scri(t 2
Iou should ha>e the #ollo,ing lines 2
...
Creatin+ II" application resource on E2F nodes...
Creatin+ 7SN application resource on E2F nodes...
Creatin+ J#S application resource on E2F nodes...
Startin+ II" application resource on E2F nodes...
Startin+ 7SN application resource on E2F nodes...
Startin+ J#S application resource on E2F nodes...
None.
{node*:root}/r0/11$1$# #
IF EOT/ !hec= for the line 6
...
Zen& is not public. "ublic interfaces s$ould be used to confi+ure virtual I"sM<
{node*:root}/r0/11$1$# #
Zen& is not public. "ublic interfaces s$ould be used to confi+ure virtual I"sM
TD;E
ead ne't (ages to understand and sol>e it 000
4E 3IPC! manualy as e'(lained on ne't (age 000
OTD;5IS;/ s.i( the ne't 3IPC! (ages M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18B o# $%$
!t this stage/ i# you get "
3IP should ha>e &een
con#igured in silent mode
,ith the root)sh scri(ts
e'ecuted on node7
TDIS IS EOT TD;
C!S;
Eote $1GC8$)1 3IPC! F!I:S COMP:!IEIE= TD!T IET;F!C; IS EOT P4B:IC
Sym(toms
Durin( CR$ in%tall w&ile runnin( root.%&' G&e followin( me%%a(e% are di%0layed
Oracle CR$ %tac, in%talled and runnin( under init-*".
Runnin( vi0ca-%ilent. for confi(urin( nodea00%
G&e (iven interface-%.' èn6X is not (u&lic) Pu&lic inter#aces should &e used to
con#igure >irtual IPs)
Cause
W&en verifyin( t&e !P addre%%e%' N!P u%e% call% to determine if a !P addre%% i% valid or
not. !n t&i% ca%e' N!P find% t&at t&e !P% are non routable -1or e<am0le !P addre%%e% li,e
*D5.*/I.Z and *C.*C.Z..
Oracle i% aware t&at t&e !PR% can be made 0ublic but %ince mo%tly %uc& !PR% are u%ed for
Private' it di%0lay t&i% error me%%a(e.
Solution
The ,or.around is to re-run >i(ca manually as root
;.vi0ca
or add t&e N!P u%in( %rvctl add nodea00%
IO4 M4ST COEFI=4; 3IP &y running >i(ca scri(t as root user)
lo(on a% root on %econd node
{node1:root}/ # e9port NIS"(AB
{node1:root}/ # cd 3CHS,GJ*/?bin
{node1:root}/r0/11$1$#/bin # .?vipca
On #irst or second node as root user/ you
must setu( the 1ISP:!I
before runnin( t&e vi0ca %cri0t located in
cr%**.*.Cbin
The 3IP ?5elcome@ gra(hical screen
will a00ear at t&e end of t&e root.%& %cri0t
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18C o# $%$
Just to remem&er 000 Pu&lic/ Pri>ate/ and 3irtual Dost Eame layout
Pu&lic 3IP !C
Interconnect
!C Interconnect
Bac%up
Eet,or. card on each node en6 en6 en* en5
1 o# 7 2 Select one and only one
net,or. inter#ace)
Select the net,or. inter#ace
corres(onding to the Pu&lic Eet,or.
Remember t&at eac& 0ublic networ, card
on eac& node mu%t &ave t&e %ame name'
3en6T for e<am0le in our ca%e.
en1 i% t&e RAC !nterconnect' or 0rivate
networ,.
Please chec. wit& 3ifconfi( ?aT on eac&
node a% root.
Select ?en6@ in our case
Then clic. Ee't )))
Just to remem&er 000 Pu&lic/ Pri>ate/ and 3irtual Dost Eame layout
Pu&lic 3IP !C Interconnect *Pri>ate Eet,or.+
en6 en1
Eode Eame IP Eode Eame IP Eode Eame IP
node* *C.J.5@.I* node1->i( 16)$)7C)181 node*-rac *C.*C.5@.I*
node5 *C. J.5@.I5 node7->i( 16) $)7C)187 node*-rac *C.*C.5@.I5
7 o# 7 2
In the 3irtual IPs #or cluster nodes
screen/ you must (ro>ide the 3IP node
name for node* and %tro,e t&e GAB ,ey to
automatically feel t&e re%t.
Chec. >alidity of t&e entrie% before
0roceedin(.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18G o# $%$
The Summary screen ,ill a((ear/
0lea%e validate t&e entrie%' or (o bac, to
modify.
Then clic. Finish )))
The 3IP con#iguration !ssistant ,ill
(roceed wit& creation' confi(uration and
%tartu0 of all a00lication re%ource% on all
%elected node%.
N!P' 2$D and O>$ will be t&e a00lication
re%ource% to be created.
5ait ,hile (rogressing )))
!f you donPt (et any error%' youPll be
0rom0ted to clic, OX a% t&e confi(uration i%
*CCY com0leted.
Then clic. OT )))
Chec. the Con#iguration results.
Then clic. ;'it )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18K o# $%$
4sing ?i#con#ig a@ on each node/ chec. that each net,or. card con#igured #or Pu&lic net,or. is ma((ing a >irtual IP)
On node 1 2
{node1:root}/ # ifconfi+ -a
en#:
8"ag0O1e#5#5,)Q5#TUPQ2B.6(C6STQN.TB6'H!BSQBUNN'N&QS'MPH!KQMUHT'C6STQ&B.UPBTQ,32'TQCN!CUSUM?.AAH.6(QCN6'
NC
inet 1#$)$*%$51 netma0: #1888888## broada0t 1#$)$*%$*%%
inet 1#$)$*%$151 netma0: #1888888## broada0t 1#$)$*%$*%%
en1:
NC
inet 1#$1#$*%$51 netma0: #1888888## broada0t 1#$1#$*%$*%%
en*:
NC
inet *#$*#$*%$51 netma0: #1888888## broada0t *#$*#$*%$*%%
inet, ::1/#
{node1:root}/ #
Oracle clu%terware N!P for node* i%
a%%i(nedma00ed to networ, interface
enC corre%0ondin( to t&e 0ublic
networ,.
N!P from node* i% *C.J.5@.*I*
11g!C/!SM/!I" oraclibm@fr.ibm.com 188 o# $%$
On node 7 2
{node*:root}/ # ifconfi+ -a
en#:
NC
inet 1#$)$*%$5* netma0: #1888888## broada0t 1#$)$*%$*%%
en1:
NC
inet 1#$1#$*%$5* netma0: #1888888## broada0t 1#$1#$*%$*%%
en*:
NC
inet *#$*#$*%$5* netma0: #1888888## broada0t *#$*#$*%$*%%
inet, ::1/#
{node*:root}/ #
Oracle clu%terware N!P for node5 i%
networ,'
N!P from node5 i% *C.J.5@.*I5
4sing ?i#con#ig a@ on each node/ chec. that each net,or. card con#igured #or Pu&lic net,or. is ma((ing a >irtual IP)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18% o# $%$
I# node 1 is re&ooted 2
TD;E On node 7 2 5e-ll see 3IP #rom node1/ 3IP #rom node1 is then still reacha&le)
{node*:root}/ # ifconfi+ -a
en#:
NC
inet 1&.3.25.1.1 netmask &9ffffff&& broadcast 1&.3.25.255
en1:
NC
inet 1#$1#$*%$5* netma0: #1888888## broada0t 1#$1#$*%$*%%
en*:
NC
inet *#$*#$*%$5* netma0: #1888888## broada0t *#$*#$*%$*%%
inet, ::1/#
{node*:root}/ #
On node1 #ailure or re&oot/
Oracle clu%terware N!P for node* will
be %witc&ed to node5' and %till
networ,'
N!P from node* i% *C.J.5@.*I*
N!P from node5 i% *C.J.5@.*I5
W&en node* will come bac, to normal
o0eration%' GFE> vi0 from node*
&o%ted tem0orary on node5' will %witc&
bac, to it% &ome node' on node*.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%6 o# $%$
Coming &ac. to this
(re>ious screen)
Just clic. OT to
continue )))
$ con#iguration assistants ,ill &e automatically e'ecuted)
3Oracle notification $erver Confi(uration A%%i%tantT
3Oracle Private !nterconnect A%%i%tantT
TOracle Clu%ter Nerification BtilityT
!f %ucce%%ful' >e<t %creen 3End of !n%tallationT will a00ear automatically OOO
!f not C&ec, for t&e re%ult to be %ucce%%ful.
Con#iguration
!ssistants 2
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1 o# $%$
Out(ut generated #rom con#iguration assistants M
Jutput +enerated from confi+uration assistant AJracle #otification Server Confi+uration AssistantA4
Command 0 ?crs?11.1.&?install?onsconfi+ add,confi+ node14:251 node24:251
The .NS on8iguration i0 reated 0ue008u""y
Stopping .NS re0oure Pora$node1$on0P
6ttempting to 0top bora$node1$on0b on member bnode1b
Stop o8 bora$node1$on0b on member bnode1b 0ueeded$
The re0oure ora$node1$on0 0topped 0ue008u""y 8or re0tart
6ttempting to 0tart bora$node1$on0b on member bnode1b
Start o8 bora$node1$on0b on member bnode1b 0ueeded$
The re0oure ora$node1$on0 re0tarted 0ue008u""y
Stopping .NS re0oure Pora$node*$on0P
6ttempting to 0top bora$node*$on0b on member bnode*b
Stop o8 bora$node*$on0b on member bnode*b 0ueeded$
The re0oure ora$node*$on0 0topped 0ue008u""y 8or re0tart
6ttempting to 0tart bora$node*$on0b on member bnode*b
Start o8 bora$node*$on0b on member bnode*b 0ueeded$
The re0oure ora$node*$on0 re0tarted 0ue008u""y
Confi+uration assistant AJracle #otification Server Confi+uration AssistantA succeeded
-----------------------------------------------------------------------------
Jutput +enerated from confi+uration assistant AJracle "rivate Interconnect Confi+uration AssistantA4
Command 0 ?crs?11.1.&?bin?oifcf+ setif -+lobal en&?1&.3.25.&4public en1?1&.1&.25.&4cluster,interconnect
Confi+uration assistant AJracle "rivate Interconnect Confi+uration AssistantA succeeded
-----------------------------------------------------------------------------
Jutput +enerated from confi+uration assistant AJracle Cluster Ierification 6tilityA4
Command 0 ?crs?11.1.&?bin?cluvfy sta+e -post crsinst -n node1<node2
Per8orming po0t-he:0 8or "u0ter 0er7ie0 0etup
Che:ing node reahabi"ity$$$
Node reahabi"ity he: pa00ed 8rom node Znode1Z$
Che:ing u0er eJui7a"ene$$$
U0er eJui7a"ene he: pa00ed 8or u0er Zr0Z$
Che:ing C"u0ter manager integrity$$$
Che:ing CSS daemon$$$
(aemon 0tatu0 he: pa00ed 8or ZCSS daemonZ$
C"u0ter manager integrity he: pa00ed$
Che:ing "u0ter integrity$$$
C"u0ter integrity he: pa00ed
Che:ing .CB integrity$$$
Che:ing the ab0ene o8 a non-"u0tered on8iguration$$$
6"" node0 8ree o8 non-"u0teredQ "oa"-on"y on8iguration0$
UniJuene00 he: 8or .CB de7ie pa00ed$
Che:ing the 7er0ion o8 .CB$$$
.CB o8 orret /er0ion Z*Z e1i0t0$
Che:ing data integrity o8 .CB$$$
(ata integrity he: 8or .CB pa00ed$
.CB integrity he: pa00ed$
Che:ing CBS integrity$$$
Che:ing daemon "i7ene00$$$
Hi7ene00 he: pa00ed 8or ZCBS daemonZ$
Hi7ene00 he: pa00ed 8or ZCSS daemonZ$
Hi7ene00 he: pa00ed 8or Z!/M daemonZ$
Che:ing CBS hea"th$$$
CBS hea"th he: pa00ed$
CBS integrity he: pa00ed$
Che:ing node app"iation e1i0tene$$$
Che:ing e1i0tene o8 /'P node app"iation FreJuiredG
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7 o# $%$
Che: pa00ed$
Che:ing e1i0tene o8 .NS node app"iation Foptiona"G
Che: pa00ed$
Che:ing e1i0tene o8 &S( node app"iation Foptiona"G
Che: pa00ed$
"ost-c$eck for cluster services setup was successful.
Confi+uration assistant AJracle Cluster Ierification 6tilityA succeeded
-----------------------------------------------------------------------------
The Z/r0/11$1$#/8gtoo""og0/on8igToo"6""Command0Z 0ript ontain0 a"" ommand0 to be e1euted by the
on8iguration a00i0tant0$ Thi0 8i"e may be u0ed to run the on8iguration a00i0tant0 out0ide o8 .U'$ Note
that you may ha7e to update thi0 0ript <ith pa00<ord0 Fi8 anyG be8ore e1euting the 0ame$
-----------------------------------------------------------------------------
I# all or (arts o# assistants are #ailed or not e'ecuted/ c&ec, for 0roblem% in lo(
oracleora!nventorylo(%in%tallAction%88888.lo( -a% %&own on t&e run!n%taller window. and %olve t&em.
C&ec, al%o cr%**.*.Ccf(toollo(%confi(Gool1ailedCommand%
{node1:r0}/r0/11$1$#/8gtoo""og0 # cat confi+;oolAllCommands
@ Copyri+$t EcF 1CCC< 2&&5< Jracle. All ri+$ts reserved.
?crs?11.1.&?bin?rac+ons add,confi+ node14:2&& node24:2&&
?crs?11.1.&?bin?oifcf+ setif -+lobal en&?1&.3.25.&4public en1?1&.1&.25.&4cluster,interconnect
?crs?11.1.&?bin?cluvfy sta+e -post crsinst -n node1<node2
{node1:r0}/r0/11$1$#/8gtoo""og0 #
!f you mi%% t&o%e %te0% or clo%ed t&e 0reviou% %creen of t&e run!n%taller' you will &ave to run t&em manually before
movin( to t&e ne<t %te0. #u%t ada0t t&e line% wit& your own %ettin( -node name%' 0ublic0rivate networ,..
On one node as crs user 6
1or 3Oracle notification $erver Confi(uration A%%i%tantT
cr%**.*.Cbinracgons addNcon#ig node12G766 node72G766
1or 3cr% Private !nterconnect A%%i%tantT
cr%**.*.Cbinoi#c#g seti# -glo&al en6/16)$)7C)62(u&lic en1/16)7C)7C)62clusterNinterconnect
1or Tcr% Clu%ter Nerification BtilityT
cr%**.*.Cbinclu>#y stage -(ost crsinst -n node1/node7
;nd o# Installation
Then clic. ;'it )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$ o# $%$
17)$ Post Installation o(erations
12.3.1 Update the Clusterware unix user .profile
To &e done on each node #or crs *in our case+ or oracle uni' user)
vi YDOM;/)(ro#ile file in cr%P% &ome directory. !dd the entries in &old &lue color
P6TNO/u0r/bin:/et:/u0r/0bin:/u0r/ub:]N.M!/bin:/u0r/bin/K11:/0bin:$
e1port P6TN
i8 > -0 Z]M6'HZ @ # Thi0 i0 at She"" 0tartup$ 'n norma"
then eho Z]M6'HMS&Z # operationQ the She"" he:0
8i # periodia""y$
!N/O]N.M!/$:0hr
e1port !N/
#The 8o""o<ing "ine i0 added by Hien0e U0e Management in0ta""ation
e1port P6TNO]P6TN:/u0r/opt/i8or/"0/o0/ai1/bin
e1port P6TNO]P6TN:/u0r/Wa7a13/bin
e1port M6NP6TNO]M6NP6TN:/u0r/"oa"/man
e1port 6'KTNB!6(?SC.P!OS
e1port T!MPO/tmp
e1port TMPO/tmp
e1port TMP('BO/tmp
uma0: #**
e9port CHS,GJ*/0?crs?11.1.&
e9port JHAC(/,CHS,GJ*/03CHS,GJ*/
e9port JHAC(/,GJ*/03JHA,CHS,GJ*/
e9port (N,(I)HAHB,"A;G03CHS,GJ*/?lib43CHS,GJ*/?lib32
e9port (I)"A;G03(N,(I)HAHB,"A;G
e1port P6TNO3CHS,GJ*/?bin:]P6TN
i8 > -t # @V then
0tty intr RC
8i
1o disconnect #rom crs user/ and reconnect to load modi#ied YDOM;/)(ro#ile
Content o#
).shrc #ile
{node1:r0}/home/r0 # cat .ks$rc
e1port /'SU6HO7i
e1port PS1OP{P]Fho0tnameGP:P]H.&'NP}]P4( # P
{node1:r0}/home/r0 #

11g!C/!SM/!I"
oraclibm@fr.ibm.com
B o# $%$
12.3.2 Verify parameter CSS misscount
MISSCO4ET 1;FIEITIOE !E1 1;F!4:T 3!:4;S
G&e C$$ mi%%count 0arameter re0re%ent% t&e ma<imum time' in %econd%' t&at a &eartbeat can be mi%%ed before
enterin( into a clu%ter reconfi(uration to evict t&e node. G&e followin( are t&e default value% for t&e mi%%count
0arameter and t&eir re%0ective ver%ion% w&en u%in( Oracle Clu%terwareZ4
16g1 H 16g72
Linu< /C $econd%
Bni< JC $econd%
N"$ JC $econd%
Window% JC $econd%
ZC$$ mi%%count default value w&en u%in( vendor -non-Oracle. clu%terware i% /CC
%econd%. G&i% i% to allow t&e vendor clu%terware am0le time to re%olve any 0o%%ible
%0lit brain %cenario%.
Subject:
Doc ID:
Chec. css
misscount
{node1:root}/r0/11$1$# # crsctl +et css misscount
Confi+uration parameter misscount is not defined.
we s$ould $ave a defined value
Chec. css
dis.timeout
{node1:root}/r0/11$1$# # crsctl +et css disktimeout
2&&
Chec. css
re&oottime
{node1:root}/r0/11$1$# # crsctl +et css reboottime
3
To coa am(ute the right >alues/ do read metalin. note 7%BB$6)1/ and use #ollo,ing note to change the >alue 2
Subject: Doc ID:
Set css misscount
!nd chec. M
Yeep only one node up and runnin+< stop t$e ot$ers
)ackup t$e content of your JCH WWW
No modify t$e CSS parameters wit$ t$e crsctl command as root user
{node1:root}/r0/11$1$# # crsctl set css misscount 3&
Confi+uration parameter misscount is now set to 3&.
{node1:root}/r0/11$1$# # crsctl +et css misscount
)#
Hestart all ot$er nodes WWW
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C o# $%$
12.3.3 Cluster Ready Services Health Check
Chec. CS (rocesses on each nodes 2
{node1:r0}/r0 # ps -ef2+rep crs
r0 -#**3 )-))3, 1 Mar #+ - *%:1+ /r0/11$1$#/bin/o00d$bin
r0 155,33 *)+,#3 # Mar #+ - #:#% /r0/11$1$#/bin/e7m"ogger$bin -o
/r0/11$1$#/e7m/"og/e7m"ogger$in8o -" /r0/11$1$#/e7m/"og/e7m"ogger$"og
r0 **%3#5 *3,#13 # Mar #+ - 1:%+ /r0/11$1$#/bin/o"0omon$bin
r0 *)+,#3 )*++-, # Mar #+ - 1:** /r0/11$1$#/bin/e7md$bin
r0 *3,#13 *,*1-# # Mar #+ - #:## /bin/0h - d
/r0/11$1$#/"og/node1/00d/o"0omonV u"imit - un"imitedV /r0/11$1$#/bin/o"0omon \\ e1it ]E
root *5*533 *-#--, % Mar #+ - *%1:## /r0/11$1$#/bin/r0d$bin reboot
root *-#--, 1 # Mar #+ - #:## /bin/0h /et/init$r0d run
root )1%)-* ))1-5# # Mar #+ - #:*1 /r0/11$1$#/bin/oprod run -t 1### -m %##
-8
r0 )-))3, )1-,*5 # Mar #+ - #:## /bin/0h - u"imit - un"imitedV d
/r0/11$1$#/"og/node1/00dV /r0/11$1$#/bin/o00d \\ e1it ]E
r0 )-+%#5 1 # #:## Tde8untC
a0m %33++5 1 # Mar 1# - #:## /r0/11$1$#/bin/o"0:d$bin
r0 ,**,+* 1 # Mar 1% - #:## /r0/11$1$#/opmn/bin/on0 -d
r0 ,*,+,3 ,**,+* # Mar 1% - #:#1 /r0/11$1$#/opmn/bin/on0 -d
rdbm0 3,+)++# 1 # Mar 1% - #:## /r0/11$1$#/bin/o"0:d$bin
r0 3,51-35 3+,+51, # 1,:)3:%- pt0/# #:## grep r0
r0 3+,+51, ,+%-3* # 1,:#,:#, pt0/# #:## -:0h
r0 3+-,%*3 3+,+51, + 1,:)3:%- pt0/# #:## p0 -e8
{node1:r0}/r0 # ps -ef2+rep d.bin
r0 -#**3 )-))3, 1 Mar #+ - *%:1+ /r0/11$1$#/bin/o00d$bin
r0 *)+,#3 )*++-, # Mar #+ - 1:** /r0/11$1$#/bin/e7md$bin
root *5*533 *-#--, + Mar #+ - *%1:## /r0/11$1$#/bin/r0d$bin reboot
a0m %33++5 1 # Mar 1# - #:## /r0/11$1$#/bin/o"0:d$bin
rdbm0 3,+)++# 1 # Mar 1% - #:## /r0/11$1$#/bin/o"0:d$bin
r0 3+-,%*5 3+,+51, # 1,:)%:11 pt0/# #:## grep d$bin
{node1:r0}/r0 #
Iou ha>e com(leted the CS install) Eo, you ,ant to >eri#y i# the install is >alid)
To ;nsure that the CS install on all the nodes is >alid/ the #ollo,ing should &e chec.ed on all the nodes)
1. ;nsure that you ha>e success#ully com(leted running root)sh on all nodes during the install) -Plea%e
do not re-run root.%&' t&i% i% very dan(erou% and mi(&t corru0t your in%tallation' G&e obKect of t&i% %te0 i% to only
confirm if t&e root.%& wa% run %ucce%%fully after t&e in%tall
2. un the command YO!NCSNDOM;/&in/crsNstat) Plea%e en%ure t&at t&i% command doe% not error out but
dum0% t&e information for eac& re%ource. !t doe% not matter w&at CR$ %tat return% for eac& re%ource. !f t&e
cr%6%tat e<it% after 0rintin( information about eac& re%ource t&en it mean% t&at t&e CR$D daemon i% u0 and
t&e client cr%6%tat utility can communicate wit& it.
G&i% will al%o indicate t&at t&e CR$D can read t&e OCR.
!f t&e cr%6%tat error% out wit& CR$-C5C54 >o re%ource% are re(i%tered' G&en t&i% mean% t&at t&ere are
no re%ource% re(i%tered' and at t&i% %ta(e you mi%%ed t&e N!P confi(uration. G&i% i% not an error but i%
mo%tly becau%e at t&i% %ta(e you mi%%ed t&e N!P confi(uration.
!f t&e cr%6%tat error% out wit& CR$-C*IA4 Cannot communicate wit& t&e CR$ daemon' G&en t&i% mean%
t&e CR$ deamon% are not %tarted.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G o# $%$
;'ecute
crsNstat t
on one node
as oracle user 2
{node2:r0}/r0/11$1$#/bin # crs,stat -t
#ame ;ype ;ar+et State Gost
------------------------------------------------------------
ora.node1.+sd application J#(I#/ J#(I#/ node1
ora.node1.ons application J#(I#/ J#(I#/ node1
ora.node1.vip application J#(I#/ J#(I#/ node1
ora.node2.+sd application J#(I#/ J#(I#/ node2
ora.node2.ons application J#(I#/ J#(I#/ node2
{node*:r0}/r0/11$1$#/bin #
;'ecute
crsNstat ls
on one node
as oracle user 2
{node2:r0}/r0/11$1$#/bin # crs,stat -ls
#ame Jwner "rimary "riv7rp "ermission
-----------------------------------------------------------------
ora.node1.+sd crs oinstall rw9r-9r--
ora.node1.ons crs oinstall rw9r-9r--
ora.node1.vip root oinstall rw9r-9r--
ora.node2.+sd crs oinstall rw9r-9r--
ora.node2.ons crs oinstall rw9r-9r--
ora.node2.vip root oinstall rw9r-9r--
{node*:r0}/r0/11$1$#/bin #
{node1:r0}/r0 # r0?0tat -he"p
U0age: r0?0tat >re0oure?name >$$$@@ >-7@ >-"@ >-J@ >- "u0ter?member@
r0?0tat >re0oure?name >$$$@@ -t >-7@ >-J@ >- "u0ter?member@
r0?0tat -p >re0oure?name >$$$@@ >-J@
r0?0tat >-a@ app"iation -g
r0?0tat >-a@ app"iation -r >- "u0ter?member@
r0?0tat -8 >re0oure?name >$$$@@ >-J@ >- "u0ter?member@
r0?0tat -"0 >re0oure?name >$$$@@ >-J@
{node1:r0}/r0 #
. un the command YO!NCSNDOM;/&in/olsnodes. G&i% %&ould return all t&e node% of t&e clu%ter.
$ucce%%ful run of t&i% command would mean t&at t&e c%% i% u0 and runnin(. Al%o t&e C$$ from eac& node can
tal, to t&e C$$ of ot&er node%.
;'ecute
olsnodes
on &oth node
as oracle user 2
{node1:r0}/r0 # olsnodes
node1
node2
{node1:r0 # r0h node*
{node2:r0}/r0 # olsnodes
node1
node2
{node*:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K o# $%$
{node1:r0}/r0 # olsnodes -$elp
U0age: o"0node0 >-n@ >-p@ >-i@ >TnodeC \ -"@ >-g@ >-7@
<here
-n print node number <ith the node name
-p print pri7ate interonnet name <ith the node name
-i print 7irtua" 'P name <ith the node name
TnodeC print in8ormation 8or the 0pei8ied node
-" print in8ormation 8or the "oa" node
-g turn on "ogging
-7 run in 7erbo0e mode
{node1:r0}/r0 #
{node1:r0}/r0 # olsnodes -n -p -i
node1 1 node1-ra node1-7ip
node* * node*-ra node*-7ip
{node1:r0}/r0 #
B) Out(ut o# crsctl chec. crs / cssd / crsd / e>md returns XM) daemon a((ears healthyX
CS health
chec.
{node1:r0}/r0 # crsctl c$eck crs
Cluster Sync$roniOation Services appears $ealt$y
Cluster Heady Services appears $ealt$y
/vent *ana+er appears $ealt$y
=node14crs>?crs @
{node2:r0}/r0 # crsctl c$eck crs
/vent *ana+er appears $ealt$y
=node14crs>?crs @
cssd/ crsd/
e>md
health
chec.
{node1:r0}/r0 # crsctl c$eck cssd
{node1:r0}/r0 #
{node1:r0}/r0 # crsctl c$eck crsd
{node1:r0}/r0 #
{node1:r0}/r0 # crsctl c$eck evmd
!7ent Manager appear0 hea"thy
{node1:r0}/r0 #
{node2:r0}/r0 # crsctl c$eck cssd
{node*:r0}/r0 #
{node*:r0}/r0 # crsctl c$eck crsd
{node*:r0}/r0 #
{node*:r0}/r0 # crsctl c$eck evmd
!7ent Manager appear0 hea"thy
{node*:r0}/r0 #
CS
so#t,are
>ersion
Puery
{node1:root}/r0 # r0t" Juery r0 ati7e7er0ion
.ra"e C"u0ter<are ati7e 7er0ion on the "u0ter i0 >11$1$#$,$#@
{node1:root}/r0 #
{node2:r0}/r0 # crsctl %uery crs activeversion
.ra"e C"u0ter<are ati7e 7er0ion on the "u0ter i0 >11$1$#$,$#@
{node*:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8 o# $%$
12.3.4 Adding enhanced crsstat script
!dd the #ollo,ing shell scri(t in YO!NCSNDOM;/&in directory
A% cr% u%er 4
Vnode*4cr%Wcr% ; cd UORA6CR$6FO"Ebin
Vnode*4cr%Wcr%**.*.Cbin ;
Vnode*4cr%Wcr%**.*.Cbin ; vi cr%tat
Add t&e followin( line% 4
#--------------------------- 2egin She"" Sript ----------------------------
#Y/u0r/bin/:0h
#
# Samp"e 1#g CBS re0oure 0tatu0 Juery 0ript
#
# (e0ription:
# - Beturn0 8ormatted 7er0ion o8 r0?0tat -tQ in tabu"ar
# 8ormatQ <ith the omp"ete r0 name0 and 8i"tering :ey<ord0
# - The argumentQ ]BSC?U!IQ i0 optiona" and i8 pa00ed to the 0riptQ <i""
# "imit the output to N6 re0oure0 <ho0e name0 math ]BSC?U!I$
# BeJuirement0:
# - ].B6?CBS?N.M! 0hou"d be 0et in your en7ironment
BSC?U!IO]1
[ST6TO-u
64UO/u0r/bin/a<: # i8 not a7ai"ab"e u0e /u0r/bin/a<:
# Tab"e header:eho ZZ
]64U c
P2!&'N {print8 Z9-3%0 9-1#0 9-150cnZQ ZN6 Be0oureZQ ZTargetZQ ZStateZV
print8 Z9-3%0 9-1#0 9-150cnZQ Z-----------ZQ Z------ZQ Z-----ZV}P
# Tab"e body:
].B6?CBS?N.M!/bin/r0?0tat ][ST6T \ ]64U c
P2!&'N { ASOZOZV 0tate O #V }
]1d/N6M!/ aa ]*d/P]BSC?U!IP/ {appname O ]*V 0tateO1}V
0tate OO # {ne1tV}
]1d/T6B&!T/ aa 0tate OO 1 {apptarget O ]*V 0tateO*V}
]1d/ST6T!/ aa 0tate OO * {app0tate O ]*V 0tateO)V}
0tate OO ) {print8 Z9-3%0 9-1#0 9-150cnZQ appnameQ apptargetQ app0tateV 0tateO#V}P
#--------------------------- !nd She"" Sript ------------------------------
C&ec, t&at 3AWX9u%rbinaw,T i% t&e ri(&t 0at& on your %y%tem OOO
C&ec, t&at ORA6CR$6FO"E i% %et in your cr% .0rofile file on eac& node OOO
$ave t&e file and e<it
Vnode*4cr%Wcr%**.*.Cbin ; c&mod =< cr%%tat
Vnode*4cr%Wcr%**.*.Cbin ;
Remote co0y t&e file on al lnode% 4
Vnode*4cr%Wcr%**.*.Cbin ; rc0 cr%%tat node54cr%**.*.Cbin
{node1:r0}/r0/11$1$#/bin # crsstat
N6 Be0oure Target State
----------- ------ -----
ora$node1$g0d .NH'N! .NH'N! on node1
ora$node1$on0 .NH'N! .NH'N! on node1
ora$node1$7ip .NH'N! .NH'N! on node1
ora$node*$g0d .NH'N! .NH'N! on node*
ora$node*$on0 .NH'N! .NH'N! on node*
ora$node*$7ip .NH'N! .NH'N! on node*
{node1:r0}/r0/11$1$#/bin #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
% o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16 o# $%$
12.3.5 Interconnect Network configuration Checkup
After CR$ in%tallation i% com0leted' verify t&at t&e 0ublic and clu%ter interconnect &ave been %et to t&e de%ired
value% by enterin( t&e followin( command% a% root4
4ote " oifcf( i% found in t&e ]CR$ FO"E^binoifcf(
{node1:r0}/r0 # oi88g
Name:
oi88g - .ra"e 'nter8ae Con8iguration Too"$
U0age: oi88g i8"i0t >-p >-n@@
oi88g 0eti8 {-node TnodenameC \ -g"oba"} {Ti8?nameC/T0ubnetC:Ti8?typeC}$$$
oi88g geti8 >-node TnodenameC \ -g"oba"@ > -i8 Ti8?nameC>/T0ubnetC@ >-type Ti8?typeC@
@
oi88g de"i8 >-node TnodenameC \ -g"oba"@ >Ti8?nameC>/T0ubnetC@@
oi88g >-he"p@
TnodenameC - name o8 the ho0tQ a0 :no<n to a ommuniation0 net<or:
Ti8?nameC - name by <hih the inter8ae i0 on8igured in the 0y0tem
T0ubnetC - 0ubnet addre00 o8 the inter8ae
Ti8?typeC - type o8 the inter8ae { "u0ter?interonnet \ pub"i \ 0torage }
{node1:r0}/r0 #
oifcf+ +etif
Subject: Doc ID:
G&i% command %&ould return value% for (lobal 30ublicT and (lobal 3clu%ter6interconnectTb for e<am0le4
{node1:root}/r0/11$1$#/bin # oifcf+ +etif
en# 1#$)$*%$# g"oba" pub"i
en1 1#$1#$*%$# g"oba" "u0ter?interonnet
!f t&e command doe% not return a value for (lobal clu%ter6interconnect' enter t&e followin(
command%4
{node1:r0}/r0/11$1$#/bin # oi88g de"i8 -g"oba"
# oi88g 0eti8 -g"oba" Tinter8ae nameC/T0ubnetC:pub"i
# oi88g 0eti8 Sg"oba" Tinter8ae nameC/T0ubnetC:"u0ter?interonnet
For e'am(le2
{node1:r0}/r0/11$1$#/bin # oifcf+ delif -+lobal
{node1:r0}/r0/11$1$#/bin # oifcf+ setif -+lobal en&?1&.3.25.&.&4public
{node1:r0}/r0/11$1$#/bin # oifcf+ setif -+lobal en1?1&.1&.25.&.&4cluster,interconnect
I# necessary and only #or trou&leshooting (ur(ose' di%able t&e automatic reboot of A!+ node% w&en node
fail to communicate wit& CR$ daemon%' or fail to acce%% OCR and Notin( di%,.
$ubKect4 16g !C2 Sto((ing e&oot :oo(s 5hen CS Pro&lems Occur Doc !D4 Eote27$%%8%)1
$ubKect4 16g !C2 Trou&leshooting CS e&oots Doc !D4 Eote27GCKG%)1
I# one node crashed a#ter running d&ca/ netca tools/ ,ith CS codedum( and !uthentication OS1 error/
chec. ?crsd)log@ #ile #or missing YCSNDOM;/crs/crs/auth directory)
GFE> you need to re-create manually t&e mi%%in( directory' create t&e aut& directory wit& correct owner'
(rou0' and 0ermi%%ion u%in( followin( metalin, note 4
$ubKect4 Crs Crashed 5ith !uthentication Osd ;rror Doc !D4 Eote2$C8B66)1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11 o# $%$
12.3.6 Oracle CLuster Registry content Check and Backup
Chec. Oracle
Cluster
egistry
Integrity
!s oracle
user/
;'ecute
ocrchec.
{node1:r0}/r0 # ocrc$eck
Statu0 o8 .ra"e C"u0ter Begi0try i0 a0 8o""o<0 :
/er0ion : *
Tota" 0pae F:byte0G : )#,-+*
U0ed 0pae F:byte0G : %+1,
67ai"ab"e 0pae F:byte0G : )#1*%,
'( : 1-*5)1,1*#
(e7ie/Ai"e Name : /de7/or?di0:1
(e7ie/Ai"e integrity he: 0ueeded
(e7ie/Ai"e Name : /de7/or?di0:*
C"u0ter regi0try integrity he: 0ueeded
{node1:r0}/r0 #
Chec. OC
dis.s
locations 2
{node1:r0}/r0 # cat ?etc?oracle?ocr.loc
oron8ig?"oO/de7/or?di0:1
ormirroron8ig?"oO/de7/or?di0:*
"oa"?on"yOA6HS!
{node1:r0}/r0 #
Chec. 3oting
1is.s
!s oracle
user/
;'ecute 2
{node1:r0}/r0 # crsctl %uery css votedisk
#$ # /de7/7oting?di0:1
1$ # /de7/7oting?di0:*
*$ # /de7/7oting?di0:)
Hoated ) 7oting di0:F0G$
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17 o# $%$
4sing ?ocrcon#ig@ crs tool to e'(ort OC content2
11g!C/!SM/!I" oraclibm@fr.ibm.com 1$ o# $%$
{node1:r0}/r0 # ocrconfi+
Name:
oron8ig - Con8iguration too" 8or .ra"e C"u0ter Begi0try$
Synop0i0:
oron8ig >option@
option:
-e1port T8i"enameC >-0 on"ine@ - !1port "u0ter regi0ter ontent0 to a 8i"e
-import T8i"enameC - 'mport "u0ter regi0try ontent0 8rom a 8i"e
-upgrade >Tu0erC >TgroupC@@ - Upgrade "u0ter regi0try 8rom pre7iou0 7er0ion
-do<ngrade >-7er0ion T7er0ion 0tringC@
- (o<ngrade "u0ter regi0try to the 0pei8ied 7er0ion
-ba:up"o TdirnameC - Con8igure periodi ba:up "oation
-0ho<ba:up >auto\manua"@ - Sho< ba:up in8ormation
-manualbackup - "erform JCH backup
-re0tore T8i"enameC - Be0tore 8rom phy0ia" ba:up
-rep"ae or\ormirror >T8i"enameC@ - 6dd/rep"ae/remo7e a .CB de7ie/8i"e
-o7er<rite - .7er<rite .CB on8iguration on di0:
-repair or\ormirror T8i"enameC - Bepair "oa" .CB on8iguration
-he"p - Print out thi0 he"p in8ormation
Note:
6 "og 8i"e <i"" be reated in
].B6CH!?N.M!/"og/Tho0tnameC/"ient/oron8ig?TpidC$"og$ P"ea0e en0ure
you ha7e 8i"e reation pri7i"ege0 in the abo7e diretory be8ore
running thi0 too"$
{node1:r0}/r0 #
;'(ort content o# OC 2
{node1:r0}/r0/11$1$#/bin # 0u
rootP0 Pa00<ord:
{node1:root}/r0/11$1$#/bin # ocrconfi+ -e9port ?oracle?ocr,e9port.dmp1 -s online
{node1:root}/r0/11$1$#/bin # ls -la ?oracle?P.dmp
-r<-r--r-- 1 root 0y0tem 1#,3*# Jan )# 15:)# ?oracle?ocr,e9port.dmp
you must not edit/modi#y this e'(orted #ile
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B o# $%$
4sing ?ocrcon#ig@ crs tool to &ac.u( OC content 2
{node1:r0}/r0 # ocrconfi+ -s$owbackup
{node1:r0}/r0 #
{node1:r0}/r0 # ocrconfi+ -manualbackup
PB.T-*#: 'n0u88iient permi00ion to proeed$ BeJuire pri7i"eged u0er
{node1:r0}/r0 # su
rootP0 Pa00<ord:
{node1:root}/r0 # ocrconfi+ -manualbackup
node1 *##5/#*/*3 #5:#5:35 /r0/11$1$#/data/r0?"u0ter/ba:up?*##5#**3?#5#535$or
{node1:root}/r0 #
!#ter #e, hours/ ,ee.s/ months/ ?ocrcon#ig sho,&ac.u(@ could dis(lay the #ollo,ing 2
3ie, OC automatic (eriodic &ac.u( managed &y Oracle Cluster,are
{node1:r0}/r0 # ocrconfi+ -s$owbackup
node* *##5/#)/1, 13:3%:35 /r0/11$1$#/data/r0?"u0ter/ba:up##$or
node* *##5/#)/1, 1#:3%:3+ /r0/11$1$#/data/r0?"u0ter/ba:up#1$or
node* *##5/#)/1, #,:3%:3+ /r0/11$1$#/data/r0?"u0ter/ba:up#*$or
node* *##5/#)/1% #,:3%:3, /r0/11$1$#/data/r0?"u0ter/day$or
node* *##5/#)/#+ #*:%*:3, /r0/11$1$#/data/r0?"u0ter/<ee:$or
node1 *##5/#*/*3 #5:#-:*1 /r0/11$1$#/data/r0?"u0ter/ba:up?*##5#**3?#5#-*1$or
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C o# $%$
:ist o# o(tions #or sr>ctl command 2
{node1:root}/r0/11$1$#/bin # 0r7t" -h
U0age: 0r7t" >-/@
U0age: 0r7t" add databa0e -d TnameC -o Tora"e?homeC >-m Tdomain?nameC@ >-p T0p8i"eC@ >-6 Tname\ipC/netma0:@ >-r {PB'M6BI
\ PNIS'C6H?ST6N(2I \ H.&'C6H?ST6N(2I \ SN6PSN.T?ST6N(2I}@ >-0 T0tart?option0C@ >-n Tdb?nameC@ >-y {6UT.M6T'C \ M6NU6H}@
U0age: 0r7t" add in0tane -d TnameC -i Tin0t?nameC -n Tnode?nameC
U0age: 0r7t" add 0er7ie -d TnameC -0 T0er7ie?nameC -r ZTpre8erred?"i0tCZ >-a ZTa7ai"ab"e?"i0tCZ@ >-P TT6A?po"iyC@
U0age: 0r7t" add 0er7ie -d TnameC -0 T0er7ie?nameC -u {-r ZTne<?pre8?in0tCZ \ -a ZTne<?a7ai"?in0tCZ}
U0age: 0r7t" add nodeapp0 -n Tnode?nameC -6 Tname\ipC/netma0:>/i81>\i8*\$$$@@
U0age: 0r7t" add a0m -n Tnode?nameC -i Ta0m?in0t?nameC -o Tora"e?homeC >-p T0p8i"eC@
U0age: 0r7t" add "i0tener -n Tnode?nameC -o Tora"e?homeC >-" T"i0tener?nameC@
U0age: 0r7t" on8ig databa0e
U0age: 0r7t" on8ig databa0e -d TnameC >-a@ >-t@
U0age: 0r7t" on8ig 0er7ie -d TnameC >-0 T0er7ie?nameC@ >-a@ >-S T"e7e"C@
U0age: 0r7t" on8ig nodeapp0 -n Tnode?nameC >-a@ >-g@ >-0@ >-"@ >-h@
U0age: 0r7t" on8ig a0m -n Tnode?nameC
U0age: 0r7t" on8ig "i0tener -n Tnode?nameC
U0age: 0r7t" di0ab"e databa0e -d TnameC
U0age: 0r7t" di0ab"e in0tane -d TnameC -i ZTin0t?name?"i0tCZ
U0age: 0r7t" di0ab"e 0er7ie -d TnameC -0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@
U0age: 0r7t" di0ab"e a0m -n Tnode?nameC >-i Tin0t?nameC@
U0age: 0r7t" enab"e databa0e -d TnameC
U0age: 0r7t" enab"e in0tane -d TnameC -i ZTin0t?name?"i0tCZ
U0age: 0r7t" enab"e 0er7ie -d TnameC -0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@
U0age: 0r7t" enab"e a0m -n Tnode?nameC >-i Tin0t?nameC@
U0age: 0r7t" geten7 databa0e -d TnameC >-t ZTname?"i0tCZ@
U0age: 0r7t" geten7 in0tane -d TnameC -i Tin0t?nameC >-t ZTname?"i0tCZ@
U0age: 0r7t" geten7 0er7ie -d TnameC -0 T0er7ie?nameC >-t ZTname?"i0tCZ@
U0age: 0r7t" geten7 nodeapp0 -n Tnode?nameC >-t ZTname?"i0tCZ@
U0age: 0r7t" modi8y databa0e -d TnameC >-n Tdb?name@ >-o TohomeC@ >-m TdomainC@ >-p T0p8i"eC@ >-r {PB'M6BI \
PNIS'C6H?ST6N(2I \ H.&'C6H?ST6N(2I \ SN6PSN.T?ST6N(2I}@ >-0 T0tart?option0C@ >-y {6UT.M6T'C \ M6NU6H}@
U0age: 0r7t" modi8y in0tane -d TnameC -i Tin0t?nameC -n Tnode?nameC
U0age: 0r7t" modi8y in0tane -d TnameC -i Tin0t?nameC {-0 Ta0m?in0t?nameC \ -r}
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -i To"d?in0t?nameC -t Tne<?in0t?nameC >-8@
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -i Ta7ai"?in0t?nameC -r >-8@
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G o# $%$
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -n -i Tpre8erred?"i0tC >-a Ta7ai"ab"e?"i0tC@ >-8@
U0age: 0r7t" modi8y a0m -n Tnode?nameC -i Ta0m?in0t?nameC >-o Tora"e?homeC@ >-p T0p8i"eC@
U0age: 0r7t" re"oate 0er7ie -d TnameC -0 T0er7ie?nameC -i To"d?in0t?nameC -t Tne<?in0t?nameC >-8@
U0age: 0r7t" remo7e databa0e -d TnameC >-8@
U0age: 0r7t" remo7e in0tane -d TnameC -i Tin0t?nameC >-8@
U0age: 0r7t" remo7e 0er7ie -d TnameC -0 T0er7ie?nameC >-i Tin0t?nameC@ >-8@
U0age: 0r7t" remo7e nodeapp0 -n ZTnode?name?"i0tCZ >-8@
U0age: 0r7t" remo7e a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-8@
U0age: 0r7t" remo7e "i0tener -n Tnode?nameC >-" T"i0tener?nameC@
U0age: 0r7t" 0eten7 databa0e -d TnameC {-t TnameCOT7a"C>QTnameCOT7a"CQ$$$@ \ -T TnameCOT7a"C}
U0age: 0r7t" 0eten7 in0tane -d TnameC >-i Tin0t?nameC@ {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0eten7 0er7ie -d TnameC >-0 T0er7ie?nameC@ {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0eten7 nodeapp0 -n Tnode?nameC {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0tart databa0e -d TnameC >-o T0tart?option0C@
U0age: 0r7t" 0tart in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-o T0tart?option0C@
U0age: 0r7t" 0tart 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@@ >-o T0tart?option0C@
U0age: 0r7t" 0tart nodeapp0 -n Tnode?nameC
U0age: 0r7t" 0tart a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0tart?option0C@
U0age: 0r7t" 0tart "i0tener -n Tnode?nameC >-" T"0nr?name?"i0tC@
U0age: 0r7t" 0tatu0 databa0e -d TnameC >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ@ >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 nodeapp0 -n Tnode?nameC
U0age: 0r7t" 0tatu0 a0m -n Tnode?nameC
U0age: 0r7t" 0top databa0e -d TnameC >-o T0top?option0C@
U0age: 0r7t" 0top in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-o T0top?option0C@
U0age: 0r7t" 0top 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@@ >-8@
U0age: 0r7t" 0top nodeapp0 -n Tnode?nameC >-r@
U0age: 0r7t" 0top a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0top?option0C@
U0age: 0r7t" 0top "i0tener -n Tnode?nameC >-" T"0nr?name?"i0tC@
U0age: 0r7t" un0eten7 databa0e -d TnameC -t ZTname?"i0tCZ
U0age: 0r7t" un0eten7 in0tane -d TnameC >-i Tin0t?nameC@ -t ZTname?"i0tCZ
U0age: 0r7t" un0eten7 0er7ie -d TnameC >-0 T0er7ie?nameC@ -t ZTname?"i0tCZ
U0age: 0r7t" un0eten7 nodeapp0 -n Tnode?nameC -t ZTname?"i0tCZ
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18 o# $%$
17)B Some use#ull commands
!S root user
Command to start/sto(
the CS deamons 2
To start the CS 2
{node1:root}/r0/11$1$#/bin # crsctl start crs
Attemptin+ to start CHS stack
;$e CHS stack will be started s$ortly
To sto( the CS 2
{node1:r0}/r0/11$1$#/bin # su
rootP0 Pa00<ord:
{node1:root}/ # cd /r0/11$1$#/bin
{node1:root}/r0/11$1$#/bin # crsctl stop crs
Stoppin+ resources. ;$is could take several minutes.
Successfully stopped CHS resources.
Stoppin+ CSSN.
S$uttin+ down CSS daemon.
S$utdown re%uest successfully issued.

!ll crsctl command a>aila&le 2
{node1:root}/r0/11$1$#/bin # crsctl
U0age: r0t" he: r0 - he:0 the 7iabi"ity o8 the CBS 0ta:
r0t" he: 00d - he:0 the 7iabi"ity o8 CSS
r0t" he: r0d - he:0 the 7iabi"ity o8 CBS
r0t" he: e7md - he:0 the 7iabi"ity o8 !/M
r0t" 0et 00 TparameterC T7a"ueC - 0et0 a parameter o7erride
r0t" get 00 TparameterC - get0 the 7a"ue o8 a CSS parameter
r0t" un0et 00 TparameterC - 0et0 CSS parameter to it0 de8au"t
r0t" Juery 00 7otedi0: - "i0t0 the 7oting di0:0 u0ed by CSS
r0t" add 00 7otedi0: TpathC - add0 a ne< 7oting di0:
r0t" de"ete 00 7otedi0: TpathC - remo7e0 a 7oting di0:
r0t" enab"e r0 - enab"e0 0tartup 8or a"" CBS daemon0
r0t" di0ab"e r0 - di0ab"e0 0tartup 8or a"" CBS daemon0
r0t" 0tart r0 - 0tart0 a"" CBS daemon0$
r0t" 0top r0 - 0top0 a"" CBS daemon0$ Stop0 CBS re0oure0 in a0e o8 "u0ter$
r0t" 0tart re0oure0 - 0tart0 CBS re0oure0$
r0t" 0top re0oure0 - 0top0 CBS re0oure0$
r0t" debug 0tatedump e7m - dump0 0tate in8o 8or e7m obWet0
r0t" debug 0tatedump r0 - dump0 0tate in8o 8or r0 obWet0
r0t" debug 0tatedump 00 - dump0 0tate in8o 8or 00 obWet0
r0t" debug "og 00 >modu"e:"e7e"@{Qmodu"e:"e7e"} $$$
- Turn0 on debugging 8or CSS
r0t" debug trae 00 - dump0 CSS in-memory traing ahe
r0t" debug "og r0 >modu"e:"e7e"@{Qmodu"e:"e7e"} $$$
- Turn0 on debugging 8or CBS
r0t" debug trae r0 - dump0 CBS in-memory traing ahe
r0t" debug "og e7m >modu"e:"e7e"@{Qmodu"e:"e7e"} $$$
- Turn0 on debugging 8or !/M
r0t" debug trae e7m - dump0 !/M in-memory traing ahe
r0t" debug "og re0 Tre0name:"e7e"C turn0 on debugging 8or re0oure0
r0t" Juery r0 0o8t<are7er0ion >TnodenameC@ - "i0t0 the 7er0ion o8 CBS 0o8t<are
in0ta""ed
r0t" Juery r0 ati7e7er0ion - "i0t0 the CBS 0o8t<are operating 7er0ion
r0t" "0modu"e0 00 - "i0t0 the CSS modu"e0 that an be u0ed 8or debugging
r0t" "0modu"e0 r0 - "i0t0 the CBS modu"e0 that an be u0ed 8or debugging
r0t" "0modu"e0 e7m - "i0t0 the !/M modu"e0 that an be u0ed 8or debugging
'8 nee0ary any o8 the0e ommand0 an be run <ith additiona" traing by
adding a ZtraeZ argument at the 7ery 8ront$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1% o# $%$
!1amp"e: r0t" trae he: 00

11g!C/!SM/!I"
oraclibm@fr.ibm.com
76 o# $%$
17)C !ccessing CS logs
To >ie, CS
logs
cd /crs/11)1)6/log/nodename/M)
In our case nodename ,ill &e node1 #or CS logs on node1
cd /crs/11)1)6/log/node1
!nd nodename ,ill &e node7 #or CS logs on node7
cd /crs/11)1)6/log/node7
Contents e'am(le o# ))/crs/log/node1 ,ith node1 2
{node1:root}/r0/11$1$#/"og/node1 # ls -la
tota" *%,
dr<1r-1r-t 5 root dba *%, Mar 1* 1,:*1 $
dr<1r-1r-1 3 ora"e dba *%, Mar 1* 1,:*1 $$
dr<1r-1--- * ora"e dba *%, Mar 1* 1,:*1 admin
-r<-r<-r-- 1 root dba *3331 6pr 1+ 1*:*+ a"ertnode1$"og
dr<1r-1--- * ora"e dba -5)#3 6pr 1+ 1*:*+ "ient
dr<1r-1--- * root dba *%, 6pr 1, 1):%, r0d
dr<1r-1--- 3 ora"e dba *%, Mar ** *1:%, 00d
dr<1r-1--- * ora"e dba *%, Mar 1* 1,:** e7md
dr<1r<1r-t % ora"e dba 3#-, 6pr 1, 1*:35 rag
{node1:root}/r0/11$1$#/"og/node1 #
:oo. at Metalin. Eote 2
$ubKect4 Oracle Cluster,are consolidated logging in 16g7 Doc !D4 Eote2$$11G8)1
;'tract #rom the note 2
Oracle Cluster,are
consolidated logging in
16g7
000 Same #or 11g 000
CBS "og0 are in ].B6?CBS?N.M!/"og/Tho0tnameC/r0d/
CSS "og0 are in ].B6?CBS?N.M!/"og/Tho0tnameC/00d/
!/M "og0 are in ].B6?CBS?N.M!/"og/Tho0tnameC/e7md
].B6?CBS?N.M!/e7m/"og/
Be0oure 0pei8i "og0 are in
].B6?CBS?N.M!/"og/Tho0tnameC/rag and the
].B6CH!?N.M!/"og/Tho0tnameC/rag
B/M "og0 are in ].B6?CBS?N.M!/"og/Tho0tnameC/"ient and
the ].B6CH!?N.M!/"og/Tho0tnameC/"ient
C"u0ter Net<or: Communiation "og0 are in the
].B6?CBS?N.M!/"og diretory
.PMN "og0 are in the ].B6?CBS?N.M!/opmn/"og0
Ne< in 1#g Be"ea0e * i0 a a"ertTnodenameC$"og pre0ent in
the ].B6?CBS?N.M!/"og/Tho0tnameC
:oo. at #ollo,ing Metalin. Eote to get all logs necessary #or needed su((ort diagnostics 2
$ubKect4 CS 16g 7 1iagnostic Collection =uide 1oc I14 Eote2$$6$C8)1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
71 o# $%$
17)G Cluster,are Basic Testing
Prior to (o furt&er' you %&ould te%t &ow t&e clu%terware i% be&avin( at %im0le te%t% to validate your oracle
clu%terware in%tallation.
!ction to &e done 2
5hat should ha((en 0000
* 1 + Eode 1 and Eode 7 are 4P and
unning
e&oot node1
Before reboot of node* 4
Name Type Target State No0t
------------------------------------------------------------
ora$node1$g0d app"iation .NH'N! .NH'N! node1
ora$node1$on0 app"iation .NH'N! .NH'N! node1
ora$node*$g0d app"iation .NH'N! .NH'N! node*
ora$node*$on0 app"iation .NH'N! .NH'N! node*
{node*:r0}/r0/11$1$#/bin #
W&ile node* i% rebootin( 4
C&ec, on node5' and N!P from node* %&ould a00ear w&ile node* i% out of order.
------------------------------------------------------------
ora$node1$g0d app"iation .NH'N! .AAH'N!
ora$node1$on0 app"iation .NH'N! .AAH'N!
{node*:r0}/r0/11$1$#/bin #
W&en node* i% bac, wit& CR$ u0 and runnin(' t&e N!P will come bac, to it% ori(inal
0o%ition on node*.
After reboot of node* 4
------------------------------------------------------------
{node*:r0}/r0/11$1$#/bin #
unning
e&oot node7
C&ec, on node*' and N!P from node5 %&ould a00ear w&ile node5 i% out of order.
W&en node5 i% bac, wit& CR$ u0 and runnin(' t&e N!P will come bac, to it% ori(inal
0o%ition on node5.
unning
e&oot node1 and node7
After reboot' bot& node% will come bac, wit& CR$ u0 and runnin(' wit& N!P from
bot& node% on t&eir re%0ective 0o%ition%' N!P -node*. on node* and N!P -node5. on
node5.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
77 o# $%$
17)K 5hat Das Been 1one R
!t this stage 2
The Oracle Cluster egistry and 3oting 1is. are created and con#igured
The Oracle Cluster eady Ser>ices is installed/ and started on all nodes)
The 3IP *3irtual IP+/ =S1 and OES a((lication resources are con#igured on all
nodes)
17)8 3IP and CS Trou&leShouting
!f 0roblem% occur% wit& N!P confi(uration a%%i%tant' 0lea%e u%e t&e
metalin, note% %0ecified in t&i% c&a0ter.
"etalin, >ote 5D/I@/.*- Confi(urin( t&e !B" A!+ @L O0eratin( $y%tem for t&e Oracle *C( N!P
"etalin, >ote 5DAJJ/.*- C&an(in( t&e c&ec, interval for t&e Oracle *C( N!P
"etalin, >ote 5E/AJA.*- "odifyin( t&e N!P of a Clu%ter >ode
srvctl modify nodeapps -n <node_name> [-o <oracle_home>] [-A <new_vip_address>]

Options Description:
-n <node_name> Node name.
-o <oracle_home> Oracle home for the cluster database.
-A <new_vip_address> The node level VIP address (<name|ip>/netmask[/if1[|if2|...]]).
An example of the 'modify nodeapps' command is as follows:
$ srvctl stop nodeapps n node1
$ srvctl modify nodeapps -n node1 -A 10.3.25.181/255.255.255.0/en0
$ srvctl start nodeapps n node1
Note: This command should be run as root.
"etalin, >ote 5DIID@.*- "odifyin( t&e default (ateway addre%% u%ed by t&e Oracle *C( N!P
"etalin, >ote 5/AIAE.*- Fow to Confi(ure Nirtual !P% for *C( RAC
Do, to delete 3IP IP
allias on (u&lic net,or.
card/ i# they are
(ersistents e>en a#ter
the CS shutdo,n 2
E<am0le for our ca%e 4
On node* a% root I#con#ig en6 delete 16)$)7C)181
On node5 a% root I#con#ig en6 delete 16)$)7C)187
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7$ o# $%$
17)% Do, to clean a #ailed CS installation
"etalin, >ote 5JDDDI.* - *C( RAC4 Fow to Clean B0 After a 1ailed CR$ !n%tall
On both :odes 6
!s root user on each node 2
YO!NCSNDOM;/&in/crsctl sto( crs -to clean any remainin( cr% deamon%.
rmita& h1 t&i% will remove oracle CR$ entry in t&e etcinittab
rmita& h7
rmita& h$
rm # $ot$OR!?K
,ill remainin( 0roce%% from out0ut 4 0%-ef\(re0 cr% and 0%-ef\(re0 d.bin
rm /etc/oracle/S
Iou should no, remo>e the CS installation/ 7 o(tions 2
* Iou ,ant to .ee( the oraIn>entory a% it% u%ed for ot&er oracle 0roduct% w&ic& are in%talled and
u%ed' GFE> run run!n%taller a% oracle u%er to unin%tall t&e CR$ in%tallation. W&en done remove t&e
content of t&e CR$ directory on bot& node% 4 rm # /crs/11)1)6/S
O
5 Iou don-t care a&out the oraIn>entory/ and there-s no other oracle (roducts installed on
t&e node%' GFE> remove t&e full content of t&e UORACLE6BA$E includin( t&e ora!nventory
directory 4 rm # /oracle/S
Change o,ner/ grou( and (ermission #or /de>/ocrNdis. and /de>/>oteNdis. on each node o# the cluster 2
On node1 $ =node14root>? @ c$own root.oinstall ?dev?ocr,diskP
{node1:root}/ # c$own crs.oinstall ?dev?votin+,diskP
{node1:root}/ # c$mod ::& ?dev?ocr,diskP
{node1:root}/ # hmod ,,# /de7/7oting?di0:=
On node# $ {node1:root}/ # rs$ node2
=node24root>? @ c$own root.oinstall ?dev?ocr,disk1P
{node*:root}/ # c$own crs.oinstall ?dev?votin+,diskP
{node*:root}/ # c$mod ::& ?dev?ocr,diskP
{node*:root}/ # c$mod ::& ?dev?votin+,diskP
To erase !:: OC and 3oting di.s content Format *èroing+ on the dis.s #rom one node 2
for i in 1 2
do
dd if0?dev?Oero of0?dev?ocr,disk3i bs01&25 count03&& S
done
)##D# reord0 in$
)##D# reord0 out$
;;;;;;$
for i in 1 2 3
do
dd if0?dev?Oero of0?dev?votin+,disk3i bs01&25 count03&& S
done
)##D# reord0 in$
)##D# reord0 out$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7B o# $%$
1$ IEST!:: !4TOM!T;1 STO!=; M!E!=;M;ET SOFT5!;
At t&i% %ta(e' Oracle Clu%terware i% in%talled and "B$G be %tarted on all node%.
A$" i% an Oracle clu%ter file% %y%tem -Oracle Automated $tora(e "ana(ement.' and a% a C1$ it can be
inde0endent from t&e databa%e %oftware' and u0dated to u00er relea%e inde0endently from databa%e %oftware.
G&at i% t&e rea%on w&y we will in%tall A$" %oftware in it% own ORACLE6FO"E directory t&at wePll define a%
A$"6FO"E - oraclea%m**.*.C ..
Eo,/ in order to ma.e a>aila&le !SM cluster #iles systems to Oracle !C data&ase/ ,e need to 2
!n%tall A$" %oftware in it% own ORACLE6FO"E
Create and confi(ure a Li%tener on eac& node
Eit&er G&ru >etca -Oracle >etwor, Confi(uration A%%i%tant.
Or "anually
Create and confi(ure A$" in%tance on eac& node
Eit&er G&ru DBCA -Oracle Databa%e Confi(uration A%%i%tant.
Or "anually
Pre0are LB>P% Di%,% for A$" -Done in 0reviou% $tora(e c&a0ter.
Create and confi(ure an A$" Di%,(rou0
Eit&er G&ru DBCA -Oracle Databa%e Confi(uration A%%i%tant.
Or $HL command%' u%in( $HLPlu%.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7C o# $%$
1$)1 Installation
Oracle A$" in%tallation Ku%t &ave to be done only %tartin( from one node. Once t&e fir%t node i% in%talled' Oracle
OB! automatically %tart% t&e co0y of t&e mandatory file% on t&e %econd node' u%in( rc( or sc( command. G&i% %te0
could la%t lon(' de0endin( on t&e networ, %0eed -one &ourS.' wit&out any me%%a(e. $o' donPt t&in, t&e OB! i%
%talled' and loo, at t&e networ, traffic before cancelin( t&e in%tallation O
Mou can al%o create a %ta(in( area. G&e name of t&e %ubdirectorie% i% in t&e format 3Di%,*T to 3Di%,JT.
On each node 2
From #irst node !s
{node1:root}/ #
{node1:root}/ # ho<n r0:oin0ta"" /ora"e/a0m
{node1:root}/ # hmod ,,% /ora"e/a0m
{node1:root}/ #
:ogin as asm *in our case+/ or oracle user and #ollo, the (rocedure hereunderM
Setu( and
e'(ort your
1ISP:!I/ TMP and
T;MP >aria&les
5ith /tm( or other destination ha>ing enough #ree s(ace/ a&out C66M& on
each node)
{node14asm}/ # e9port NIS"(AB0node14&.&
If not set in asm .profile< do 4
{node1:a0m}/ # e9port ;*"0?tmp
{node1:a0m}/ # e9port ;/*"0?tmp
{node1:a0m}/ # e9port ;*"NIH0?tmp
Chec. that Oracle Cluster,are *including 3IP/ OES and =S1+ is started on each node 000
!s asm user #rom
node1 2
{node1:asm}/ora"e/a0m # crs,stat -t
------------------------------------------------------------
ora$$$$!1$"0nr app"iation .NH'N! .NH'N! node1
ora$node1$7ip app"iation .NH'N! .NH'N! node1
ora$$$$!*$"0nr app"iation .NH'N! .NH'N! node*
ora$node*$7ip app"iation .NH'N! .NH'N! node*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7G o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7K o# $%$
Vnode14asmWdi%trib$oftwareOraclerdbm%**(r*ai<databa%e ; l%
doc in%tall re%0on%e run!n%taller %ta(e welcome.&tml
Vnode*4a%mWdi%trib$oftwareOraclerdbm%**(r*ai<databa%e ; )/runInstaller
$tartin( Oracle Bniver%al !n%tallerS
C&ec,in( Gem0 %0ace4 mu%t be (reater t&an *DC "B. Actual *D@C "B Pa%%ed
C&ec,in( %wa0 %0ace4 mu%t be (reater t&an *@C "B. Actual J@IA "B Pa%%ed
Pre0arin( to launc& Oracle Bniver%al !n%taller from tm0Ora!n%tall5CCI-C5-*J6CJ-AE-JAP". Plea%e wait S
Vnode*4a%mWdi%trib$oftwareOraclerdbm%**(r*ai<databa%e ; Oracle Bniver%al !n%taller' Ner%ion **.*.C./.C
Production
Iou can chec. the installed Products 2
Just clic. Ee't M
Select the installation ty(e 2
Mou &ave t&e o0tion to c&oo%e Enter0ri%e'
$tandard Edition' or Cu%tom to 0roceed.
Choose the ?Custom@ o0tion to avoid creatin(
a databa%e by default.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
78 o# $%$
S(eci#y File :ocations 2
Do not c&an(e t&e $ource field
$0ecify a different ORACLE6FO"E >ame wit&
it% own directory for t&e Oracle %oftware
in%tallation.
This O!C:;NDOM; must &e di##erent
then the CS O!C:;NDOM;)
OraA%m**(6&ome*
oraclea%m**.*.C
Then clic. Ee't )))
I# you don-t see the #ollo,ing screen ,ith
Eode selection/ it might &e that your CS is
do,n on one or all nodes) Please chec. i#
CS is u( and running on all nodes)
S(eci#y Dard,are Cluster Installation Mode 2
$elect Cluster Installation
A>D t&e ot&er node% on to w&ic& t&e Oracle
RDB"$ %oftware will be in%talled. !t i% not
nece%%ary to %elect t&e node on w&ic& t&e OB! i%
currently runnin(. Clic, >e<t.
Then clic. Ee't )))
The installer ,ill chec. some (roduct-
s(eci#ic PrerePuisite)
DonPt ta,e care of t&e line% wit& c&ec,in( at
%tatu% 3>ot e<ecutedT' G&e%e are Ku%t warnin(%
becau%e A!+ maintenance level mi(&t be &i(&er
t&en @JCC' w&ic& i% t&e ca%e in our e<am0le
-"LCJ..
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
7% o# $%$
1etails o# the
(rerePuisite
chec.s done
&y
runInstaller
Chec.ing o(erating system rePuirements )))
E<0ected re%ult4 One of @5CC.CCA'@JCC.CC5
Actual Re%ult4 @JCC.CC5
99999999999999999999999999999999999999999999999999999999999999999999999
Chec.ing o(erating system (ac.age rePuirements )))
C&ec,in( for bo%.adt.ba%e-C.C.b found bo%.adt.ba%e-@.J.C.@*.. Pa%%ed
C&ec,in( for bo%.adt.lib-C.C.b found bo%.adt.lib-@.J.C.@C.. Pa%%ed
C&ec,in( for bo%.adt.libm-C.C.b found bo%.adt.libm-@.J.C.AC.. Pa%%ed
C&ec,in( for bo%.0erf.lib0erf%tat-C.C.b found bo%.0erf.lib0erf%tat-@.J.C.@C.. Pa%%ed
C&ec,in( for bo%.0erf.0erf%tat-C.C.b found bo%.0erf.0erf%tat-@.J.C.@C.. Pa%%ed
C&ec,in( for bo%.0erf.0roctool%-C.C.b found bo%.0erf.0roctool%-@.J.C.@C.. Pa%%ed
99999999999999999999999999999999999999999999999999999999999999999999999
Chec.ing recommended o(erating system (atches
C&ec,in( for !M@DJI/-bo%.rte.bind6cmd%'@.J.C.*.b found -bo%.rte.bind6cmd%'@.J.C.@*.. Pa%%ed
C&ec,in( for !M/CDJC-bo%.m0'@.J.C.*.b found -bo%.m0'@.J.C.@A.. Pa%%ed
C&ec,in( for !M/CDJC-bo%.m0/A'@.J.C.*.b found -bo%.m0/A'@.J.C.@A.. Pa%%ed
C&ec,in( for !M//@*J-bo%.m0/A'@.J.C.5C.b found -bo%.m0/A'@.J.C.@A.. Pa%%ed
C&ec,in( for !M//@*J-bo%.m0'@.J.C.5C.b found -bo%.m0'@.J.C.@A.. Pa%%ed
C&ec,in( for !MEC*@D-bo%.m0'@.J.C.55.b found -bo%.m0'@.J.C.@A.. Pa%%ed
C&ec,in( for !MEC*@D-bo%.m0/A'@.J.C.55.b found -bo%.m0/A'@.J.C.@A.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.m0/A'@.J.C.*.b found -bo%.m0/A'@.J.C.@A.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.acct'@.J.C.*.b found -bo%.acct'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.adt.include'@.J.C.*.b found -bo%.adt.include'@.J.C.@J.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.adt.libm'@.J.C.*.b found -bo%.adt.libm'@.J.C.AC.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.adt.0rof'@.J.C.*.b found -bo%.adt.0rof'@.J.C.@J.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.alt6di%,6in%tall.rte'@.J.C.*.b found -bo%.alt6di%,6in%tall.rte'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.cif%6f%.rte'@.J.C.*.b found -bo%.cif%6f%.rte'@.J.C.@C.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.dia(.com'@.J.C.*.b found -bo%.dia(.com'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.0erf.lib0erf%tat'@.J.C.*.b found -bo%.0erf.lib0erf%tat'@.J.C.@C.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.0erf.0erf%tat'@.J.C.*.b found -bo%.0erf.0erf%tat'@.J.C.@C.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.0erf.tool%'@.J.C.*.b found -bo%.0erf.tool%'@.J.C.@5.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.boot'@.J.C.*.b found -bo%.rte.boot'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.arc&ive'@.J.C.*.b found -bo%.rte.arc&ive'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.bind6cmd%'@.J.C.*.b found -bo%.rte.bind6cmd%'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.control'@.J.C.*.b found -bo%.rte.control'@.J.C.@C.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.file%y%tem'@.J.C.*.b found -bo%.rte.file%y%tem'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.in%tall'@.J.C.*.b found -bo%.rte.in%tall'@.J.C.@A.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.libc'@.J.C.*.b found -bo%.rte.libc'@.J.C.@J.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.lvm'@.J.C.*.b found -bo%.rte.lvm'@.J.C.@J.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.man'@.J.C.*.b found -bo%.rte.man'@.J.C.@C.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.met&od%'@.J.C.*.b found -bo%.rte.met&od%'@.J.C.@*.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.%ecurity'@.J.C.*.b found -bo%.rte.%ecurity'@.J.C.@J.. Pa%%ed
C&ec,in( for !M@I*AJ-bo%.rte.%erv6aid'@.J.C.*.b found -bo%.rte.%erv6aid'@.J.C.@5.. Pa%%ed
99999999999999999999999999999999999999999999999999999999999999999999999
3alidating O!C:;NB!S; location *i# set+ )))
99999999999999999999999999999999999999999999999999999999999999999999999
Chec.ing #or (ro(er system clean-u())))
99999999999999999999999999999999999999999999999999999999999999999999999
Chec.ing #or Oracle Dome incom(ati&ilities ))))
Actual Re%ult4 >EW6FO"E
99999999999999999999999999999999999999999999999999999999999999999999999
Chec.ing Oracle Cluster,are >ersion )))
99999999999999999999999999999999999999999999999999999999999999999999999
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$6 o# $%$
!>aila&le Product Com(onents 2
$elect t&e 0roduct com0onent% for Oracle **(
A$" %oftware t&at you want to in%tall.
Then clic. Ee't )))
Pri>ileged O(erating Systems =rou(s 2
Nerify t&e B>!+ 0rimary (rou0 name of t&e u%er
w&ic& control% t&e in%tallation of t&e Oracle**(
A$" %oftware. -B%e uni< command id to find out.
And %0ecify t&e Privile(ed O0eratin( $y%tem
2rou0% to t&e value found.
!n our e<am0le' t&i% mu%t be 4
3dbaT for Databa%e Admini%trator -O$DBA.
(rou0
3o0erT for Databa%e O0erator -O$OPER.
(rou0
3a%mT for admini%trator -$M$A$". (rou0
-Primary (rou0 of uni< a%m u%er. to be %et for
bot& entrie%.
Then clic. Ee't )))
Create 1ata&ase 2
Choose ?Install data&ase So#t,are Only@
Choose ?Con#igure !utomatic Storage
Management *!SM+@/ we want to in%tall t&e
A$" %oftware in it% own ORACLE6FO"E at t&i%
%ta(e.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$1 o# $%$
Summary 2
The Summary screen ,ill &e (resented)
C&ec, Clu%ter >ode% and Remote >ode% li%t%.
G&e OB! will in%tall t&e Oracle **( A$"
%oftware on to t&e local node' and t&en co0y t&i%
information to t&e ot&er %elected node%.
Install 2
The Oracle 4ni>ersal Installer ,ill (roceed
the installation on the #irst node' t&en will
co0y automatically t&e code on t&e ot&er%
%elected node%.
At @CY' it may ta,e time to 0a%% over t&e @CY'
donPt be affrai%' in%tallation i% 0roce%%in('
runnin( a te%t %cri0t on remote node%
At t&i% %ta(e' you could &it me%%a(e about 31ailed Attac&ed FomeT OOO
i# similar screen message a((ears/ 9ust run the s(eci#ied command on the s(eci#ied node as asm user)
1o e'ecute the scri(t as &ello, *chec. and ada(t the scri(t u(on your message+) 5hen done/ clic. on ?OT@
2
From node2 6
{node*:root}/ora"e/a0m/11$1$# # 0u S a0m
{node2:asm}/ora"e/a0m -C ?oracle?asm?11.1.&?oui?bin?runInstaller 1attac$Gome 1
noCluster/nabled JHAC(/,GJ*/0?oracle?asm?11.1.& JHAC(/,GJ*/,#A*/0OraA%m**(6&ome*
C(6S;/H,#JN/S0node1<node2 LI#I/#;JHB,(JCA;IJ#0?oracle?oraInventoryM (JCA(,#JN/0node2
Startin+ Jracle 6niversal InstallerR
#o pre-re%uisite c$ecks found in oraparam.ini< no system pre-re%uisite c$ecks will be
e9ecuted.
;$e inventory pointer is located at ?etc?oraInst.loc
;$e inventory is located at ?oracle?oraInventory
ZAttac$Gome[ was successful.
{node*:a0m}/ora"e/a0m/11$1$#/bin #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$7 o# $%$
;'ecute Con#iguration
Scri(ts 2
oraclea%m**.*.Croot.%& to
e<ecute on eac& node a% root
u%er' on node*' t&en on node5.
5hen 1one/ come &ac. on
this screen and clic. on
?OT@))) 000
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$$ o# $%$
On node1 6
{node1:root}/ora"e/a0m/11$1$# # .?root.s$
Bunning .ra"e 11g root$0h 0ript$$$
The 8o""o<ing en7ironment 7ariab"e0 are 0et a0:
.B6CH!?.4N!BO a0m
.B6CH!?N.M!O /ora"e/a0m/11$1$#
!nter the 8u"" pathname o8 the "oa" bin diretory: >/u0r/"oa"/bin@:
The 8i"e ZdbhomeZ a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@: y
Copying dbhome to /u0r/"oa"/bin $$$
The 8i"e Zoraen7Z a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@: y
Copying oraen7 to /u0r/"oa"/bin $$$
Creating /et/oratab 8i"e$$$
!ntrie0 <i"" be added to the /et/oratab 8i"e a0 needed by
(ataba0e Con8iguration 600i0tant <hen a databa0e i0 reated
Aini0hed running generi part o8 root$0h 0ript$
No< produt-0pei8i root ation0 <i"" be per8ormed$
Aini0hed produt-0pei8i root ation0$
{node1:root}/ora"e/a0m/11$1$# #
On node2 6
{node2:root}/ora"e/a0m/11$1$# # .?root.s$
.B6CH!?.4N!BO a0m
.B6CH!?N.M!O /ora"e/a0m/11$1$#
The 8i"e ZdbhomeZ a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@: y
Copying dbhome to /u0r/"oa"/bin $$$
Creating /et/oratab 8i"e$$$
{node*:root}/ora"e/a0m/11$1$# #
;nd o# Installation 2
Clic, on 3!n%talled Product%T to c&e, wit& t&e
ora!nventory t&at A$" i% in%talled on eac&
node.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$B o# $%$
Just clic. on ?Close@/ then ?;'it@ )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$C o# $%$
1$)7 4(date the !SM uni' user )(ro#ile
To &e done on each node #or asm *in our case+ or oracle uni' user)
vi YDOM;/)(ro#ile file in a%mP% &ome directory. !dd the entries in &old &lue color
e1port P6TN
8i # periodia""y$
!N/O]N.M!/$:0hr
e1port !N/
e1port T!MPO/tmp
e1port TMPO/tmp
e1port TMP('BO/tmp
uma0: #**
e1port CBS?N.M!O/r0/11$1$#
e1port .B6CH!?CBS?N.M!O]CBS?N.M!
e9port JHAC(/,GJ*/03JHAC(/,)AS/?asm?11.1.&
e1port
H(?H'2B6BI?P6TN03JHAC(/,GJ*/?lib:]CBS?N.M!/"ib43JHAC(/,GJ*/?lib32:]CBS?N.M!/"ib)*
e1port H'2P6TNO]H(?H'2B6BI?P6TN
e1port P6TNO3JHAC(/,GJ*/?bin:]CBS?N.M!/bin:]P6TN
e9port ;#S,AN*I#03JHAC(/,GJ*/?network?admin
e9port JHAC(/,SIN0UAS*1
i8 > -t # @V then
0tty intr RC
8i
On node5' O!C:;NSI1 will be %et a% bellow 4
$$$
e9port JHAC(/,SIN0UAS*2
;
1o disconnect #rom asm user/ and reconnect to load modi#ied YDOM;/)(ro#ile
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$G o# $%$
1$)$ Chec.ing Oracle Cluster,are
!t this stage/ Oracle !SM so#t,are is installed in its o,n O!C:;NDOM; directory t&at wePll define a%
A$"6FO"E - oraclea%m**.*.C ..
Eo,/ ,e need to 2
Create and confi(ure a Li%tener on eac& node
Create and confi(ure A$" in%tance on eac& node
Create and confi(ure an A$" Di%,(rou0
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$K o# $%$
1$)B Con#igure de#ault node listeners
In order to con#igure !SM instances' and t&en create A$" Di%, 2rou0%' we need to confi(ure and %tart a default
li%tener on eac& node.
:istener on each node must 2
Confi(ured wit& ORACLE6FO"E %et to oraclea%m**.*.C
Li%ten to $tatic !P and N!P
B%in( GCP and !PC 0rotocol%
Starting the installation 000
{node1:asm}/ora"e/a0m/11$1$#/bin # id
uidO%#*Fa0mG gidO%##Foin0ta""G
group0O1F0ta88GQ%#1Fr0GQ%#*FdbaGQ%#)FoperGQ%#3Fa0mGQ%#%Fa0mdbaG
{node1:a0m}/ora"e/a0m/11$1$#/bin # p<d
/ora"e/a0m/11$1$#/bin
{node1:a0m}/ora"e/a0m/11$1$#/bin # .?netca
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$8 o# $%$
Oracle Eet Con#iguration !ssistant/
Con#iguration 2
$elect 3Clu%ter Confi(urationT a%
ty0e of Oracle >et $ervice%.
Clic. ?Ee't@ M
eal !((lication Clusters/ !cti>e Eodes
2
3$elect All node%T to confi(ure.
Clic. ?Ee't@ M
5elcome 2
$elect 3Li%tener Confi(urationT
Clic. ?Ee't@ M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
$% o# $%$
:istener Con#iguration/ :istener 2
$elect 3AddT
Clic. ?Ee't@ M
:istener Eame 2
Xee0 by default name a%
3L!$GE>ERT.
Clic. ?Ee't@ M
:istener Con#iguration/ Select
Protocols 2
$elect Protocol% 3GCPT' and 3!PCT
Clic. ?Ee't@ M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B6 o# $%$
:istener Con#iguration/ TCP / IP
Protocol 2
B%e t&e %tandard 0ort number of
3*@5*T' or ot&er 0ort for your
0roKect.
Port *@5* mean% automatic re(i%terin(
of A$" or databa%e% in%tance% wit&in
t&e li%tenerP%.
Clic. ?Ee't@ M
4sing other (ort ,ill mean that con#iguration o# ?:OC!:@ and ;MOT;@ :istener-s ,ill &e M!E1!TOI)
:ater/ I# you need to con#igure more than a listener/ for e<am0le one for eac& databa%e in t&e clu%ter'
you %&ould 4
Fave a li%tener for A$" on eac& node 4
Xee0 a default li%tener in t&e UA$"6FO"E -oraclea%m**.*.C..
B%in( 0ort different t&an *@5* to avoid automatic re(i%terin( of all t&e re%ource% on t&i% li%tener.
Confi(ure local and remote li%tener% for A$" in%tance%.
Fave li%tener-%. for RDB"$ on eac& node 4
$et your e<tra li%tener% out of t&e UA$"6FO"E' in t&eir own ORACLE6FO"E
B%in( 0ort different t&an *@5* and different from t&e one from A$" li%tener%.
Confi(ure local and remote li%tener% for eac& Databa%e in%tance%.
!n ca%e of e<tra li%tener%' t&ey %&ould be confi(ured from t&eir own UORACLE6FO"E directory' and not from t&e
UA$"6FO"E di%rectory. $o creation of e<tra li%tener% will &ave to be done w&en RDB"$ will be in%talled ...
:istener Con#iguration/
IPC Protocol 2
Enter !PC Xey value' 3E+GPROCT
for e<am0le.
Clic. ?Ee't@ M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B1 o# $%$
:istener Con#iguration/ More
:isteners R 2
$elect Protocol% 3GCPT' and 3!PCT
Clic. ?Ee't@ M
TD;E
On the telnet session/ ,e can see 2
Oracle >et $ervice%
Confi(uration4
Confi(urin( Li%tener4L!$GE>ER
node*...
node5...
Li%tener confi(uration com0lete.
:istener Con#iguration/ :istener
Con#iguration 1one 2
Li%tener Confi(uration com0lete O
Clic. ?Ee't@ M
:istener Con#iguration/ 5elcome 2
Clic. on ?Finish@ to e'it 000
1ollowin( me%%a(e will a00ear on
e<it 4
Oracle >et $ervice% confi(uration
%ucce%%ful. G&e e<it code i% C
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B7 o# $%$
Chec.ing i# :isteners are registered in Oracle Cluster,are 2
* li%tener on eac& node mu%t be re(i%tered OOO
To sto( the de#ault node
listener on s(eci#ied node 2
Eode1 #or e'am(le M
{node1:a0m}/ora"e/a0m/11$1$#/bin # srvctl stop listener -n node1
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat -t
------------------------------------------------------------
ora$$$$!1$"0nr app"iation .AAH'N! .AAH'N!
{node1:a0m}/ora"e/a0m/11$1$#/bin #
To start the de#ault node
listener on s(eci#ied node 2
Eode1 #or e'am(le M
{node1:a0m}/ora"e/a0m/11$1$#/bin # srvctl start listener -n node1
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat -t
------------------------------------------------------------
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B$ o# $%$
To get
in#ormation
a&out the
listener
resource on
one node/ #rom
the Oracle
Cluster,are
egistry 2
Eode1 #or
e'am(le M
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat 2+rep lsnr
N6M!Oora$node1$H'ST!N!B?N.(!1$"0nr
N6M!Oora$node*$H'ST!N!B?N.(!*$"0nr
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat -p ora.node1.(IS;/#/H,#JN/1.lsnr
TIP!Oapp"iation
6CT'.N?SCB'PTO/ora"e/a0m/11$1$#/bin/rag<rap
6CT'/!?PH6C!M!NTO#
6UT.?ST6BTO1
CN!CU?'NT!B/6HO,##
(!SCB'PT'.NOCBS app"iation 8or "i0tener on node
A6'H./!B?(!H6IO#
A6'HUB!?'NT!B/6HO#
A6'HUB!?TNB!SN.H(O#
N.ST'N&?M!M2!BSOnode1
.PT'.N6H?B!S.UBC!SO
PH6C!M!NTOre0trited
B![U'B!(?B!S.UBC!SOora$node1$7ip
B!ST6BT?6TT!MPTSO%
SCB'PT?T'M!.UTO,##
ST6BT?T'M!.UTO,##
ST.P?T'M!.UTO,##
UPT'M!?TNB!SN.H(O+d
USB?.B6?6H!BT?N6M!O
USB?.B6?CN!CU?T'M!.UTO#
USB?.B6?C.NN!CT?STBO/ a0 0y0dba
USB?.B6?(!2U&O#
USB?.B6?('SC.NN!CTO8a"0e
USB?.B6?AH6&SO
USB?.B6?'AO
USB?.B6?'NST?N.T?SNUT(.4NO
USB?.B6?H6N&O
USB?.B6?N!TM6SUO
USB?.B6?.P!N?M.(!O
USB?.B6?.P'O8a"0e
USB?.B6?PA'H!O
USB?.B6?PB!C.NN!CTOnone
USB?.B6?SB/O
USB?.B6?ST6BT?T'M!.UTO#
USB?.B6?ST.P?M.(!Oimmediate
USB?.B6?ST.P?T'M!.UTO#
USB?.B6?/'PO
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat ora.node1.(IS;/#/H,#JN/1.lsnr
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N! on node1
{node1:a0m}/ora"e/a0m/11$1$#/bin # r0,stat -t ora.node1.(IS;/#/H,#JN/1.lsnr
------------------------------------------------------------
{node1:a0m}/ora"e/a0m/11$1$#/bin # crs,stat -ls ora.node1.(IS;/#/H,#JN/1.lsnr
Name .<ner Primary Pri7&rp Permi00ion
-----------------------------------------------------------------
ora$$$$!1$"0nr a0m oin0ta"" r<1r<1r--
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BB o# $%$
{node1:a0m}/ora"e/a0m/11$1$#/bin # srvctl confi+ listener -n node1
node1 H'ST!N!B?N.(!1
C&ec, t&at
entry i% OX'
%&owin(
L!$GE>ER6>ODE
*' and wit& vi0 from
node*' and %tatic !P
from node*.
{node1:a0m}/ora"e/a0m/11$1$#/net<or:/admin # at listener.ora
# "i0tener$ora$node1 Net<or: Con8iguration Ai"e:
/ora"e/a0m/11$1$#/net<or:/admin/"i0tener$ora$node1
# &enerated by .ra"e on8iguration too"0$
H'ST!N!B?N.(!1 O
F(!SCB'PT'.N?H'ST O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O 'PCGFU!I O !KTPB.CGG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node1-7ipGFP.BT O 1%*1GF'P O A'BSTGG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O 1#$)$*%$51GFP.BT O 1%*1GF'P O
A'BSTGG
G
G
{node1:a0m}/ora"e/a0m/11$1$#/net<or:/admin #
C&ec, t&at entry i% OX' %&owin( L!$GE>ER6>ODE5' and wit& vi0 from node5' and %tatic !P from node5.
!t may &a00en% t&at
on remote node%'
t&e li%tener.ora file
(et not 0ro0erly
confi(ured' and
may %&ow %ame
content a% t&e one
from node*.
C&ec& on e<tra
node%' if any S
{node2:a0m}/ora"e/a0m/11$1$#/net<or:/admin # cat listener.ora
# "i0tener$ora$node* Net<or: Con8iguration Ai"e:
/ora"e/a0m/11$1$#/net<or:/admin/"i0tener$ora$node*
H'ST!N!B?N.(!* O
F(!SCB'PT'.N?H'ST O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node*-7ipGFP.BT O 1%*1GF'P O A'BSTGG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O 1#$)$*%$5*GFP.BT O 1%*1GF'P O
A'BSTGG
G
G
{node*:a0m}/ora"e/a0m/11$1$#/net<or:/admin #
Go (et a %tatu% from t&e li%tener 4
e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
{node1:a0m}/ora"e/a0m/11$1$#/bin # .?lsnrctl status listener
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on 1)-A!2-*##5 *1:%*:*,
Copyright FG 1--1Q *##+Q .ra"e$ 6"" right0 re0er7ed$
Conneting to F6((B!SSOFPB.T.C.HOtpGFN.STOGFP.BTO1%*1GG
ST6TUS o8 the H'ST!N!B
------------------------
6"ia0 H'ST!N!B?N.(!1
/er0ion TNSHSNB 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# -
Prodution
Start (ate 1)-A!2-*##5 *1:)-:#5
Uptime # day0 # hr$ 1) min$ *# 0e
Trae He7e" o88
Seurity .N: Hoa" .S 6uthentiation
SNMP .N
Hi0tener Parameter Ai"e /ora"e/a0m/11$1$#/net<or:/admin/"i0tener$ora
Hi0tener Hog Ai"e /ora"e/diag/tn0"0nr/node1/"i0tener?node1/a"ert/"og$1m"
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BC o# $%$
Hi0tening !ndpoint0 Summary$$$
F(!SCB'PT'.NOF6((B!SSOFPB.T.C.HOipGFU!IO!KTPB.CGGG
F(!SCB'PT'.NOF6((B!SSOFPB.T.C.HOtpGFN.STO1#$)$*%$151GFP.BTO1%*1GGG
The "i0tener 0upport0 no 0er7ie0
The ommand omp"eted 0ue008u""y
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BG o# $%$
Go (et a %tatu% of t&e
%ervice% mana(ed by
t&e li%tener 4
{node1:a0m}/ora"e/a0m/11$1$#/bin # $/"0nrt" 0er7ie0 "i0tener
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# -
Prodution on 1)-A!2-*##5 *1:%):3-
The "i0tener 0upport0 no 0er7ie0
Go (et &el0 on
li%tenerP% command% 4
{node1:a0m}/ora"e/a0m/11$1$#/bin # .?lsnrctl $elp
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on
1)-A!2-*##5 *1:%%:#%
The 8o""o<ing operation0 are a7ai"ab"e
6n a0teri0: F=G denote0 a modi8ier or e1tended ommand:
0tart 0top 0tatu0
0er7ie0 7er0ion re"oad
0a7e?on8ig trae 0pa<n
hange?pa00<ord Juit e1it
0et= 0ho<=
Go (et t&e ver%ion relea%e of t&e li%tener %oftware 4
{node1:a0m}/ora"e/a0m/11$1$#/bin # .?lsnrctl version
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on 1)-A!2-*##5
*1:%5:1#
TNSHSNB 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution
TNS 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution
Uni1 (omain So:et 'PC NT Protoo" 6daptor 8or '2M/6'K B'SC Sy0tem/,###:
/er0ion 11$1$#$,$# - Prodution
TCP/'P NT Protoo" 6dapter 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# -
Prodution
.ra"e 2eJueath NT Protoo" 6dapter 8or '2M/6'K B'SC Sy0tem/,###: /er0ion
11$1$#$,$# - ProdutionQQ
11g!C/!SM/!I"
oraclibm@fr.ibm.com
BK o# $%$
1$)C Create !SM Instances
Create/ con#igure and start the !SM instances M
!nd create the !SM dis.grou(s)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B8 o# $%$
13.5.1 Thru DBCA
!s asm uni' user/ #rom one node 2
{node1:a0m}/ora"e/a0m # e1port ('SPH6IOnode1 :#$#
{node1:a0m}/ora"e/a0m # e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
{node1:a0m}/ora"e/a0m # d ].B6CH!?N.M!/bin
{node1:a0m}/ora"e/a0m/11$1$#/bin # dba a
1ata&ase Con#iguration
!ssistant 2 5elcome
$elect 3Real A00lication
Clu%ter% Databa%e T
TD;E clic. ?Ee't@
#or ne't screen M
!ssistant 2 O(erations
$elect 3Confi(ure
Automatic $tora(e
"ana(ement T
TD;E clic. ?Ee't@
#or ne't screen M
!ssistant 2 Eode Selection
Clic, on 3$elect ALLT'
>ode* and node5 in our
ca%e.
TD;E clic. ?Ee't@
#or ne't screen M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
B% o# $%$
!ssistant 2 Create !SM Instance
On this screen/ you can 2
"odify$et A$" in%tance%
0arameter%.
Enter A$" in%tance
0a%%word for 3$M$T u%er.
C&oo%e t&e ty0e of A$"
in%tance% 0arameter file%'
a'd %0ecify file name or
di%, location.
Clic. on ? @ 2
"ore detail% on ne<t 0a(e S
Set the (ass,ord #or ?SIS? user
administering !SM instances 2
Choose the ty(e o# (arameter
#ile #or !SM instances 2
"ore detail% on ne<t 0a(e S
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C6 o# $%$
!&out !SM Parameters 2
!SMN1IST=O4PS S
De%cri0tion4 G&i% value i% t&e li%t of t&e Di%, 2rou0 name% to be mounted by t&e A$" at %tartu0 or w&en ALGER D!$X2ROBP ALL "OB>G
command i% u%ed.
!SMN1ISTSTIE= S
De%cri0tion4 A comma %e0arated li%t of 0at&% u%ed by t&e A$" to limit t&e %et of di%,% con%idered for di%covery w&en a new di%, i% added to a
Di%, 2rou0. G&e di%, %trin( %&ould matc& t&e 0at& of t&e di%,' not t&e directory containin( t&e di%,. 1or e<am0le4 devrd%,Z.
!SMNPO5;N:IMIT S
De%cri0tion4 G&i% value i% t&e ma<imum 0ower on t&e A$" in%tance for di%, rebalancin(.
Ran(e of Nalue%4 * to **
Default Nalue4 *
C:4ST;N1!T!B!S;
De%cri0tion 4 $et CLB$GER6DAGABA$E to GRBE to enable Real A00lication Clu%ter% o0tion.
Ran(e of Nalue%4 GRBE \ 1AL$E
Default Nalue 4 1AL$E
1BN4EIO4;NE!M;
1I!=EOSTICN1;ST
Re0lace B$ER6DB"P6DE$G' CORE6DB"P6DE$G' BACX2ROB>D6DB"P6DE$G
$0ecifie% t&e 0at&name for a directory w&ere t&e %erver will write debu((in( trace file% on be&alf of a u%er 0roce%%.
$0ecifie% t&e 0at&name -directory or di%c. w&ere trace file% are written for t&e bac, (round 0roce%%e% -L2WR' DBW n' and %o on. durin(
Oracle o0eration%. !t al%o define% t&e location of t&e databa%e alert file w&ic& lo(% %i(nificant event% and me%%a(e%.
G&e directory name %0ecifyin( t&e core dum0 location -for B>!+..
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C1 o# $%$
IEST!EC;NTIP;
:!=;NPOO:NSI`;
De%cri0tion 4 $0ecifie% t&e %iLe of t&e lar(e 0ool allocation &ea0' w&ic& i% u%ed by $&ared $erver for %e%%ion memory' 0arallel e<ecution for
me%%a(e buffer%' and R"A> bac,u0 and recovery for di%, !O buffer%.
Ran(e of Nalue%4 /CCX -minimum.b ^9 5CCCC" -ma<imum i% o0eratin( %y%tem %0ecific..
Default Nalue 4 C' unle%% 0arallel e<ecution or DBWR6!O6$LANE$ are confi(ured
:OC!:N:IST;E;
De%cri0tion 4 A Oracle >et addre%% li%t w&ic& identifie% databa%e in%tance% on t&e %ame mac&ine a% t&e Oracle >et li%tener%. Eac& in%tance
and di%0atc&er re(i%ter% wit& t&e li%tener to enable client connection%. G&i% 0arameter override% "G$6L!$GE>ER6ADDRE$$ and
"G$6"BLG!PLE6L!$GE>ER$ 0arameter% t&at were ob%olete a% I.*.
Ran(e of Nalue%4 A valid Oracle >et addre%% li%t.
Default Nalue 4 -ADDRE$$6L!$G9-Addre%%9-Protocol9GCP.-Fo%t9local&o%t.-Port9*@5*.. -Addre%%9-Protocol9!PC.-Xey9DBname...
SD!;1NPOO:NSI`;
De%cri0tion 4 $0ecifie% t&e %iLe of t&e %&ared 0ool in byte%. G&e %&ared 0ool contain% obKect% %uc& a% %&ared cur%or%' %tored 0rocedure%'
control %tructure%' and Parallel E<ecution me%%a(e buffer%. Lar(er value% can im0rove 0erformance in multi-u%er %y%tem%.
Ran(e of Nalue%4 JCC Xbyte% - o0eratin( %y%tem de0endent.
Default Nalue 4 !f /A bit' /A"B' el%e */"B
SPFI:;
De%cri0tion4 $0ecifie% t&e name of t&e current %erver 0arameter file in u%e.
Ran(e of Nalue%4 %tatic 0arameter
Default Nalue4 G&e $P1!LE 0arameter can be defined in a client %ide P1!LE to indicate t&e name of t&e %erver 0arameter file to u%e. W&en t&e
default %erver 0arameter file i% u%ed by t&e %erver' t&e value of $P1!LE will be internally %et by t&e %erver.
IEST!EC;NE4MB;
De%cri0tion 4 A Clu%ter Databa%e 0arameter t&at a%%i(n% a uni)ue number for ma00in( t&e in%tance to one free li%t (rou0 owned by a
databa%e obKect created wit& %tora(e 0arameter 1REEL!$G 2ROBP$. B%e t&i% value in t&e !>$GA>CE clau%e of t&e ALGER GABLE ...
ALLOCAGE E+GE>G %tatement to dynamically allocate e<tent% to t&i% in%tance.
Ran(e of Nalue%4 * to "A+6!>$GA>CE$ -%0ecified at databa%e creation..
Default Nalue 4 Lowe%t available number -de0end% on in%tance %tartu0 order and on t&e !>$GA>CE6>B"BER value% a%%i(ned to ot&er
in%tance%.
:OC!:N:IST;E;
De%cri0tion 4 A Oracle >et addre%% li%t w&ic& identifie% databa%e in%tance% on t&e %ame mac&ine a% t&e Oracle >et li%tener%. Eac& in%tance
and di%0atc&er re(i%ter% wit& t&e li%tener to enable client connection%. G&i% 0arameter override% "G$6L!$GE>ER6ADDRE$$ and
"G$6"BLG!PLE6L!$GE>ER$ 0arameter% t&at were ob%olete a% I.*.
Ran(e of Nalue%4 A valid Oracle >et addre%% li%t.
Default Nalue 4 -ADDRE$$6L!$G9-Addre%%9-Protocol9GCP.-Fo%t9local&o%t.-Port9*@5*.. -Addre%%9-Protocol9!PC.-Xey9DBname...
Subject: Doc ID:
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C7 o# $%$
Choose the ty(e o# (arameter #ile t&at you would li,e to u%e for t&e new A$" in%tance%
OPTIOE 1
3Create initialiLation
0arameter file -!1!LE.T
And %0ecify t&e filename
..
OPTIOE 7
$elect 3Create %erver
0arameter file -$P1!LE.T'
Re0lace
VORACLE6FO"EWdb%%
0file=A$".ora by
devA$"%0f6di%,
TD;E Clic. on ?Ee't@ M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C$ o# $%$
!ssistant 2
Clic. on ?OT@' DBCA will create
an A$" !n%tance on eac& node S
!ssistant 2
DBCA i% creatin( t&e A$"
in%tance%.
W&en done' c&ec, t&e content of
t&e init=A$"Sora file on eac&
node.
Mou %&ould al%o find a 3=A$"T
directory in t&e UORACLE6BA$E.
{node1:a0m}/ora"e/a0m/11$1$#/db0 # at initD6SM1$ora
SPA'H!OP/de7/6SM0p8?di0:P
{node1:a0m}/ora"e/a0m/11$1$#/db0 #
{node2:a0m}/ora"e/a0m/11$1$#/db0 # at initD6SM*$ora
SPA'H!OP/de7/6SM0p8?di0:P
{node*:a0m}/ora"e/a0m/11$1$#/db0 #
!ssistant 2
A$" in%tance% are created'
Clic. on b Create Ee, c to create
an A$" Di%, 2rou0.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CB o# $%$
'f 0ou don+t see an0
candidate dis=s;
3o clic= on L!hange 3is=
3isco4er0 /ath> M
S and change to the right ath.
'f 0ou still don+t see an0
candidates dis=s; do chec=
the dis=s rearation.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CC o# $%$
!&out !SM ;"T;E!: edundancy M
Redundancy will be done by e<ternal
mec&ani%m' and not by A$".
Thru SO:Plus on node1/
it ,ill &e the #ollo,ing synta' 2
0u - a0m
e1port .B6CH!?S'(OD6SM1
e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
0J"p"u0 /no"og
onnet /a0 0y0dba
Then e1eute 8o""o<ing S[H :
CB!6T! ('SU&B.UP (6T6?('SU&B.UP
N.BM6H B!(UN(6NCI
A6'H&B.UP &B.UP1 ('SU
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CG o# $%$
!&out !SM EOM!: edundancy
,ithout Failure grou(s M
EOM!: edundancy %tand for 5 co0ie% of
t&e bloc, -* ori(inal' and * co0y..
Redundancy will be done by A$".
Bloc, will be mirrored between t&e di%,%'
en%urin( t&at ori(inal and co0y are not on t&e
%ame di%,.
!f one co0y i% lo%t' A$" will en%ure to re-
create t&e co0y' ma,in( %ure t&at alway% 5
co0ie% are available on different di%,%.
N.BM6H B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
CK o# $%$
!&out !SM DI=D edundancy ,ithout
Failure grou(s M
DI=D edundancy %tand for J co0ie% of t&e
bloc, -* ori(inal' and 5 co0ie%..
en%urin( t&at ori(inal and co0ie% are not on
t&e %ame di%,.
create t&e co0y' ma,in( %ure t&at alway% J
N'&N B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C8 o# $%$
!&out !SM EOM!: edundancy ,ith
7 Failure grou(s M
EOM!: edundancy %tand for 5 co0ie%
of t&e bloc, -* ori(inal' and * co0y..
en%urin( t&at ori(inal and co0ie% are not
on t&e %ame di%,.
1ailure (rou0% will en%ure t&at ori(inal and
co0ie are in different (rou0 wit&in t&e
di%,(rou0' meanin( t&at if t&e 5 failure
(rou0% are made of di%,% from 5 different%
%tora(e' ori(inal and co0ie will be on
different %tora(e.
N.BM6H B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PQ
A6'H&B.UP &B.UP* ('SU
P/de7/6SM?di0:3PQ
P/de7/6SM?di0:%PQ
P/de7/6SM?di0:,PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
C% o# $%$
!&out !SM DI=D edundancy ,ith $
Failure grou(s M
EOM!: edundancy %tand for J co0ie%
of t&e bloc, -* ori(inal' and 5 co0ie%..
en%urin( t&at ori(inal and co0ie% are not
on t&e %ame di%,.
1ailure (rou0% will en%ure t&at ori(inal and
co0ie are in different (rou0 wit&in t&e
di%,(rou0' meanin( t&at if t&e 5 failure
(rou0% are made of di%,% from 5 different%
%tora(e' ori(inal and co0ie will be on
different %tora(e.
N'&N B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PQ
P/de7/6SM?di0:3PQ
A6'H&B.UP &B.UP) ('SU
P/de7/6SM?di0:%PQ
P/de7/6SM?di0:,PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G6 o# $%$
!&out Setting or EOT !SM Eame to dis.s M
Synta' ,ith 1IST naming M
!n t&i% ca%e' we force t&e name of
eac& A$" di%,% to ea%e
admini%tration.
!f one di%, i% dro00ed' and %ame di%,
to be re-added to %ame or new Di%,
2rou0' A$" Di%, name &a% to be
diferrent from t&e 0reviou% OOO
Thru SO:Plus/ it ,ill &e the #ollo,ing synta' 2
N.BM6H B!(UN(6NCI
('SU
P/de7/6SM?di0:1P N6M! 6SM?di0:1Q
P/de7/6SM?di0:*P N6M! 6SM?di0:*Q
P/de7/6SM?di0:)P N6M! 6SM?di0:)V
Synta' ,ith automatic 1IST
naming M
A$" in%tance will (enerate
automatically an A$" di%, name to
eac& di%,' A$" di%, name will be
uni)ue. And if a di%, i% dro00ed and
added a(ain in %ame or new
di%,(rou0' a new A$" di%, name will
be (enerated OOOO
N.BM6H B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:)PV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G1 o# $%$
For coo.&oo. (ur(ose/ ,e ,ill create an !SM 1IST=O4P called ?1!T!N1=1@
Wit& 5 1ailure% 2rou0% called 4
32ROBP*T wit& 5 di%,%
o devA$"6di%,*
o devA$"6di%,5
and
32ROBP5T
o devA$"6di%,A
o devA$"6di%,@
Clic, on 3 T t&en A$" di%, 2rou0 creation will
%tart S
CB!6T! ('SU&B.UP (6T6?(&1
N.BM6H B!(UN(6NCI
P/de7/6SM?di0:1PQ
P/de7/6SM?di0:*PQ
P/de7/6SM?di0:3PQ
P/de7/6SM?di0:%PV
!SM alert log history ,ith command e'ecuted &y 1BC! 2
G&u 1eb *A **4@J4A5 5CCI
$HL^ CREAGE D!$X2ROBP DAGA6D2* >ormal REDB>DA>CM 1A!L2ROBP 2ROBP5 D!$X RdevA$"6di%,AR $![E ACD/" '
RdevA$"6di%,@R $![E ACD/" 1A!L2ROBP 2ROBP* D!$X RdevA$"6di%,*R $![E ACD/" '
RdevA$"6di%,5R $![E ACD/"
>OGE4 A%%i(nin( number -*'C. to di%, -devA$"6di%,A.
>OGE4 A%%i(nin( number -*'*. to di%, -devA$"6di%,@.
>OGE4 A%%i(nin( number -*'5. to di%, -devA$"6di%,*.
>OGE4 A%%i(nin( number -*'J. to di%, -devA$"6di%,5.
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCCC
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCC*
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCC5
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCCJ
>OGE4 initiatin( P$G u0date4 (r0 9 *
,fd06u0date-.4 *
,fd06u0dateB(-.4 *
>OGE4 (rou0 DAGA6D2*4 initial P$G location4 di%, CCCC -P$G co0y C.
>OGE4 (rou0 DAGA6D2*4 initial P$G location4 di%, CCC5 -P$G co0y *.
>OGE4 P$G u0date (r0 9 * com0leted %ucce%%fully
>OGE4 cac&e re(i%tered (rou0 DAGA6D2* number9* incarn9C<D//bDace
>OGE4 cac&e o0enin( di%, C of (r0 *4 DAGA6D2*6CCCC 0at&4devA$"6di%,A
>OGE4 cac&e o0enin( di%, * of (r0 *4 DAGA6D2*6CCC* 0at&4devA$"6di%,@
>OGE4 cac&e o0enin( di%, 5 of (r0 *4 DAGA6D2*6CCC5 0at&4devA$"6di%,*
>OGE4 cac&e o0enin( di%, J of (r0 *4 DAGA6D2*6CCCJ 0at&4devA$"6di%,5
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G7 o# $%$
Z allocate domain *' invalid 9 GRBE
,Kbdomatt %end to node *
>OGE4 attac&ed to recovery domain *
>OGE4 cac&e creatin( (rou0 *C<D//BDACE -DAGA6D2*.
>OGE4 cac&e mountin( (rou0 *C<D//BDACE -DAGA6D2*. %ucceeded
>OGE4 allocatin( 1*+C on (r0 * di%, DAGA6D2*6CCCC
>OGE4 allocatin( 1*+C on (r0 * di%, DAGA6D2*6CCC5
>OGE4 di%,(rou0 mu%t now be re-mounted 0rior to fir%t u%e
>OGE4 cac&e di%mountin( (rou0 *C<D//BDACE -DAGA6D2*.
>OGE4 l(wr not bein( m%(Rd to di%mount
,Kbdomdet %end to node *
detac& from dom *' %endin( detac& me%%a(e to node *
freein( rdom *
>OGE4 detac&ed from domain *
>OGE4 cac&e di%mounted (rou0 *C<D//BDACE -DAGA6D2*.
,fd06di%mount-.4 5
,fd06di%mountB(-.4 5
,fd06di%mount-.4 J
,fd06di%mountB(-.4 J
>OGE4 De-a%%i(nin( number -*'C. from di%, -devA$"6di%,A.
>OGE4 De-a%%i(nin( number -*'*. from di%, -devA$"6di%,@.
>OGE4 De-a%%i(nin( number -*'5. from di%, -devA$"6di%,*.
>OGE4 De-a%%i(nin( number -*'J. from di%, -devA$"6di%,5.
$BCCE$$4 di%,(rou0 DAGA6D2* wa% created
>OGE4 cac&e re(i%tered (rou0 DAGA6D2* number9* incarn9C<ba5bDadC
>OGE4 A%%i(nin( number -*'J. to di%, -devA$"6di%,5.
>OGE4 A%%i(nin( number -*'C. to di%, -devA$"6di%,A.
>OGE4 A%%i(nin( number -*'*. to di%, -devA$"6di%,@.
>OGE4 A%%i(nin( number -*'5. to di%, -devA$"6di%,*.
>OGE4 %tart &eartbeatin( -(r0 *.
G&u 1eb *A **4@J4@5 5CCI
,fd06)uery-.4 /
G&u 1eb *A **4@J4@5 5CCI
,fd06)ueryB(-.4 /
>OGE4 cac&e o0enin( di%, C of (r0 *4 DAGA6D2*6CCCC 0at&4devA$"6di%,A
>OGE4 1*+C found on di%, C fcn C.C
>OGE4 cac&e o0enin( di%, * of (r0 *4 DAGA6D2*6CCC* 0at&4devA$"6di%,@
>OGE4 cac&e o0enin( di%, 5 of (r0 *4 DAGA6D2*6CCC5 0at&4devA$"6di%,*
>OGE4 1*+C found on di%, 5 fcn C.C
>OGE4 cac&e o0enin( di%, J of (r0 *4 DAGA6D2*6CCCJ 0at&4devA$"6di%,5
>OGE4 cac&e mountin( -fir%t. (rou0 *C<BA5BDADC -DAGA6D2*.
Z allocate domain *' invalid 9 GRBE
,Kbdomatt %end to node *
>OGE4 attac&ed to recovery domain *
>OGE4 cac&e recovered (rou0 * to fcn C.C
G&u 1eb *A **4@J4@J 5CCI
>OGE4 o0enin( c&un, * at fcn C.C ABA
>OGE4 %e)95 bl,9C
>OGE4 cac&e mountin( (rou0 *C<BA5BDADC -DAGA6D2*. %ucceeded
G&u 1eb *A **4@J4@J 5CCI
,fd06)uery-.4 E
,fd06)ueryB(-.4 E
>OGE4 !n%tance u0dated com0atible.a%m to *C.*.C.C.C for (r0 *
$BCCE$$4 di%,(rou0 DAGA6D2* wa% mounted
$BCCE$$4 CREAGE D!$X2ROBP DAGA6D2* >ormal REDB>DA>CM 1A!L2ROBP 2ROBP5 D!$X RdevA$"6di%,AR $![E ACD/" '
RdevA$"6di%,@R $![E ACD/" 1A!L2ROBP 2ROBP* D!$X RdevA$"6di%,*R $![E ACD/" '
RdevA$"6di%,5R $![E ACD/"
ALGER $M$GE" $EG a%m6di%,(rou0%9RDAGA6D2*R $COPE9BOGF $!D9R=A$"*Rb
ALGER $M$GE" $EG a%m6di%,(rou0%9RDAGA6D2*R $COPE9BOGF $!D9R=A$"5Rb
>OGE4 enlar(in( ACD for (rou0 *C<ba5bDadC -DAGA6D2*.
G&u 1eb *A **4@A4C@ 5CCI
$BCCE$$4 ACD enlar(ed for (rou0 *C<ba5bDadC -DAGA6D2*.
G&u 1eb *A *545@4JD 5CCI
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G$ o# $%$
*e now ha4e an A%& dis=
grou called L3ATAE3#1>
,it* 2 )ailures %roups E
GR29(1 "it*
o =dev=ASMBdis#1
o =dev=ASMBdis#2
GR29(2 "it*
o =dev=ASMBdis#$
o =dev=ASMBdis#'
!ssistant 2 !SM 1is. =rou(s
!SM dis. grou( is then
dis(layed ,ith in#ormation as 2
Di%, 2rou0 >ame
$iLe -"B.' total ca0acity
1ree -"B.' Available %0ace
Redundancy ty0e
"ount %tatu%
From the menu/ it-s (ossi&le to 2
Create a new A$" di%, (rou0
Add di%, -%. to an e<i%tin( A$" di%, (rou0
"ana(e A$" tem0late
"ount a %elected A$" di%, (rou0
"ount all A$" di%, (rou0
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GB o# $%$
From
!f you want to create a new A$" di%,(rou0' clic, on 3 T' and O>LM candidate di%,% will be di%0layed 4
devA$"6di%,*' devA$"6di%,5' devA$"6di%,A and devA$"6di%,@ are not anymore 3CandidateT OOO
!f you %elect 3$&ow AllT' All di%,% will be %&own 4
o devA$"6di%,* and devA$"6di%,5 a% 3"emberT
-member of an A$" Di%,2rou0 wit&in 1ailure 2rou0 2ROBP*.
A$" Di%, name 3DAGA6D2*6CC5T a% been a%%i(ned by A$" in%tance to device devA$"6di%,*
A$" Di%, name 3DAGA6D2*6CCJT a% been a%%i(ned by A$" in%tance to device devA$"6di%,5
o devA$"6di%,A and devA$"6di%,@ a% 3"emberT
-member of an A$" Di%,2rou0 wit&in 1ailure 2rou0 2ROBP5.
A$" Di%, name 3DAGA6D2*6CCCT a% been a%%i(ned by A$" in%tance to device devA$"6di%,A
A$" Di%, name 3DAGA6D2*6CC*T a% been a%%i(ned by A$" in%tance to device devA$"6di%,@
o devA$"6di%,J' devA$"6di%,/ and devA$"6di%,E a% 3CandidateT
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GC o# $%$
From
!f you want to add di%,% in an A$" di%,2rou0' $elect t&e Di%,2rou0 3DAGA6D2*T' and clic, on 3 T.
O>LM 3CandidateT di%,% will
be di%0layed'
and A$" REDB>DA>CM
ty0e can not be modified OOO
add in 32ROBP*T di%,
o devA$"6di%,J
and
add in 32ROBP5T di%,
o devA$"6di%,/
6HT!B ('SU&B.UP (6T6?(&1
6((
A6'H&B.UP &B.UP1 ('SU P/de7/6SM?di0:)P
A6'H&B.UP &B.UP* ('SU P/de7/6SM?di0:*PV
!SM alert log history ,ith command e'ecuted &y 1BC! 2
$HL^ ALGER D!$X2ROBP ALGER D!$X2ROBP DAGA6D2* ADD 1A!L2ROBP 2ROBP* D!$X RdevA$"6di%,JR $![E ACD/" 1A!L2ROBP
2ROBP5 D!$X RdevA$"6di%,/R $![E ACD/"b
>OGE4 A%%i(nin( number -*'A. to di%, -devA$"6di%,/.
>OGE4 A%%i(nin( number -*'@. to di%, -devA$"6di%,J.
>OGE4 re)ue%tin( all-in%tance member%&i0 refre%& for (rou09*
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCCA
>OGE4 initialiLin( &eader on (r0 * di%, DAGA6D2*6CCC@
>OGE4 cac&e o0enin( di%, A of (r0 *4 DAGA6D2*6CCCA 0at&4devA$"6di%,/
>OGE4 cac&e o0enin( di%, @ of (r0 *4 DAGA6D2*6CCC@ 0at&4devA$"6di%,J
>OGE4 re)ue%tin( all-in%tance di%, validation for (rou09*
G&u 1eb *A *545@4A5 5CCI
>OGE4 di%, validation 0endin( for (rou0 *C<ba5bDadC -DAGA6D2*.
$BCCE$$4 validated di%,% for *C<ba5bDadC -DAGA6D2*.
,fd06u0date-.4 I
G&u 1eb *A *545@4A@ 5CCI
,fd06u0dateB(-.4 I
>OGE4 member%&i0 refre%& 0endin( for (rou0 *C<ba5bDadC -DAGA6D2*.
,fd06)uery-.4 D
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GG o# $%$
,fd06)ueryB(-.4 D
,fd06)uery-.4 *C
,fd06)ueryB(-.4 *C
$BCCE$$4 refre%&ed member%&i0 for *C<ba5bDadC -DAGA6D2*.
$BCCE$$4 ALGER D!$X2ROBP ALGER D!$X2ROBP DAGA6D2* ADD 1A!L2ROBP 2ROBP* D!$X RdevA$"6di%,JR $![E ACD/" 1A!L2ROBP
2ROBP5 D!$X RdevA$"6di%,/R $![E ACD/"b
>OGE4 %tartin( rebalance of (rou0 *C<ba5bDadC -DAGA6D2*. at 0ower *
$tartin( bac,(round 0roce%% ARBC
G&u 1eb *A *545@4AD 5CCI
ARBC %tarted wit& 0id95C' O$ id9/C/JIC
>OGE4 a%%i(nin( ARBC to (rou0 *C<ba5bDadC -DAGA6D2*.
>OGE4 1*+C co0y J relocatin( from /@@JA4A5DAD/E5DA to A45
G&u 1eb *A *545/4*J 5CCI
>OGE4 %to00in( 0roce%% ARBC
$BCCE$$4 rebalance com0leted for (rou0 *C<ba5bDadC -DAGA6D2*.
G&u 1eb *A *545/4*/ 5CCI
>OGE4 re)ue%tin( all-in%tance member%&i0 refre%& for (rou09*
,fd06u0date-.4 **
G&u 1eb *A *545/4*D 5CCI
,fd06u0dateB(-.4 **
,fd06u0date-.4 *5
,fd06u0dateB(-.4 *5
>OGE4 member%&i0 refre%& 0endin( for (rou0 *C<ba5bDadC -DAGA6D2*.
,fd06)uery-.4 *J
,fd06)ueryB(-.4 *J
$BCCE$$4 refre%&ed member%&i0 for *C<ba5bDadC -DAGA6D2*.
G&u 1eb *A *54AA4J* 5CCI
Addin( new di%,% S
*e now ha4e the A%& dis=
grou L3ATAE3#1>
,it* 2 )ailures %roups E
GR29(1 "it*
o =dev=ASMBdis#1
o =dev=ASMBdis#2
o =dev=ASMBdis#
GR29(2 "it*
o =dev=ASMBdis#$
o =dev=ASMBdis#'
o =dev=ASMBdis#&
11g!C/!SM/!I"
oraclibm@fr.ibm.com
GK o# $%$
From
Clic, on 3 T to 4
definemana(e t&e ty0e of tri00in( and t&e
level of redundancy for eac& ty0e of obKect
to be %tored on A$" di%, (rou0%.
add new ty0e of obKect wit& it% own
attribute%.
From
Clic, on 3 T to mount %elected A$" di%,% (rou0'
or clic, on 3 T to mount all A$" di%,% (rou0
!ssistant 2
Clic. on ?Finish@/ then
?Ies@ to e'it M
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G8 o# $%$
13.5.2 Manual Steps
!s asm uni' user/ #rom one node 2
{node1:a0m}/ora"e/a0m # e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
{node1:a0m}/ora"e/a0m # d ].B6CH!?N.M!/db0
{node1:a0m}/ora"e/a0m/11$1$#/db0 # 7i initD6SM1$ora
!dd the line and sa>e 2
SPFI:;WQ/de>/!SMs(#Ndis.Q
{node1:a0m}/ora"e/a0m/11$1$#/db0 # rp initD6SM1$ora node*:
/ora"e/a0m/11$1$#/db0/initD6SM*$ora
On node1/ do create a
(#ile init!SM)ora in
/oracle/asm/11)1)6/d&s
5ith the s(eci#ied
>alues 2
D6SM1$??ora"e?ba0eOP/ora"eP#.B6CH!?26S! 0et 8rom en7ironment
D6SM*$??ora"e?ba0eOP/ora"eP#.B6CH!?26S! 0et 8rom en7ironment
D6SM1$a0m?di0:group0OPP
D6SM*$a0m?di0:group0OPP
D6SM1$a0m?di0:0tringOP/de7/6SM?di0:=P
D6SM*$a0m?di0:0tringOP/de7/6SM?di0:=P
=$"u0ter?databa0eOtrue
=$diagno0ti?de0tOP/ora"eP
D6SM*$in0tane?numberO*
D6SM1$in0tane?numberO1
=$in0tane?typeOPa0mP
=$"arge?poo"?0iLeO1*M
=$proe00e0O1##
On each node/
node1/ then node7
!S asm user/ 1o
create the #ollo,ing
directories
On node1
{node1:a0m}/ora"e/a0m # cd ..
{node1:a0m}/ora"e # mkdir admin
{node1:a0m}/ora"e # mkdir admin?UAS*
{node1:a0m}/ora"e # mkdir admin?UAS*?$dump
{node1:a0m}/ora"e # mkdir admin?UAS*?pfile
On node7
{node2:a0m}/ora"e/a0m # cd ..
{node*:a0m}/ora"e # mkdir admin
{node*:a0m}/ora"e # mkdir admin?UAS*
{node*:a0m}/ora"e # mkdir admin?UAS*?$dump
{node*:a0m}/ora"e # mkdir admin?UAS*?pfile
11g!C/!SM/!I"
oraclibm@fr.ibm.com
G% o# $%$
On node1/ let create the !SM instance
On node1
{node1:a0m}/ora"e/a0m # cd 11.1.&?dbs
{node1:a0m}/ora"e/a0m/11$1$#/db0 # e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node1:a0m}/ora"e/a0m/11$1$#/db0 # e9port JHAC(/,SIN0UAS*1
{node1:a0m}/ora"e/a0m/11$1$#/db0 # s%lplus ?nolo+
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Mon Aeb 15 **:)1:#- *##5
Copyright FG 1-5*Q *##+Q .ra"e$ 6"" right0 re0er7ed$
S[HC connect ?as sysasm
Conneted to an id"e in0tane$
S[HC
S[HC startup nomunt pfile0?oracle?asm?11.1.&?init!SM)ora d
6SM in0tane 0tarted
Tota" Sy0tem &"oba" 6rea 1*%5*-1*# byte0
Ai1ed SiLe 1)#13%, byte0
/ariab"e SiLe 1*3%*+,,3 byte0
(ataba0e 2u88er0 # byte0
Bedo 2u88er0 # byte0
S[HC
S[HC create spfile0[?dev?AS*spf,disk[ from pfile0[ 0?oracle?asm?11.1.&?init!SM)ora[ \
Ai"e reated$
S[HC
S[HC s$utdown
6SM in0tane 0hutdo<n
S[HC
S[HC startup nomount\
6SM in0tane 0tarted
S[HC
S[HC set linesiOe 1&&&
S[HC select I#S;A#C/,#6*)/H< I#S;A#C/,#A*/< GJS;,#A*/< I/HSIJ#< S;A;6S from
v3instance\
'NST6NC!?NUM2!B 'NST6NC!?N6M! N.ST?N6M! /!BS'.N ST6TUS
--------------- ---------------- -------------------- ----------------- ------------
1 D6SM1 node1 11$1$#$,$# ST6BT!(
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K6 o# $%$
T;DE On node7/ let start the !SM instance
On node7
{node2:a0m}/ora"e/a0m # cd 11.1.&?dbs
{node*:a0m}/ora"e/a0m/11$1$#/db0 # e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node*:a0m}/ora"e/a0m/11$1$#/db0 # e9port JHAC(/,SIN0UAS*2
{node*:a0m}/ora"e/a0m/11$1$#/db0 # s%lplus ?nolo+
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Mon Aeb 15 **:)1:#- *##5
Conneted to an id"e in0tane$
S[HC
S[HC startup nomunt
6SM in0tane 0tarted
S[HC
S[HC set linesiOe 1&&&
S[HC select I#S;A#C/,#6*)/H< I#S;A#C/,#A*/< GJS;,#A*/< I/HSIJ#< S;A;6S from v3instance\
--------------- ---------------- -------------------- ----------------- ------------
1 D6SM* node1 11$1$#$,$# ST6BT!(
S[HC
1o Puery the glo&al >ie, or cluster >ie,/ .no, as g>Yinstance
S[HC 0e"et 'NST6NC!?NUM2!BQ 'NST6NC!?N6M!Q N.ST?N6M!Q /!BS'.NQ ST6TUS 8rom
g7]in0taneV
--------------- ---------------- -------------------- ----------------- ------------
* D6SM* node* 11$1$#$,$# ST6BT!(
1 D6SM1 node1 11$1$#$,$# ST6BT!(
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K1 o# $%$
On node1/ then node7
1o create an oracle (ass,ord (#ile in /oracle/asm/11)1)6/d&s
{node1:a0m}/ora"e/a0m/11$1$#/db0 # orapwd
U0age: orap<d 8i"eOT8nameC pa00<ordOTpa00<ordC entrie0OTu0er0C 8oreOTy/nC
ignorea0eOTy/nC no0y0dbaOTy/nC
<here
8i"e - name o8 pa00<ord 8i"e FreJuiredGQ
pa00<ord - pa00<ord 8or SIS Foptiona"GQ
entrie0 - ma1imum number o8 di0tint (26 FreJuiredGQ
8ore - <hether to o7er<rite e1i0ting 8i"e Foptiona"GQ
ignorea0e - pa00<ord0 are a0e-in0en0iti7e Foptiona"GQ
no0y0dba - <hether to 0hut out the SIS(26 "ogon Foptiona" (ataba0e /au"t on"yG$
There mu0t be no 0pae0 around the eJua"-to FOG harater$
On node1
{node1:a0m}/ora"e/a0m/11$1$#/db0 # orap<d 8i"eO/ora"e/a0m/11$1$#/db0/orap<D6SM1
pa00<ordOora"e entrie0O*#
{node1:a0m}/ora"e/a0m/11$1$#/db0 # "0 -"a orap<=
-r<-r----- 1 a0m oin0ta"" 1%), Aeb 13 1#:%) orap<D6SM1
On node7
{node*:a0m}/ora"e/a0m/11$1$#/db0 # orap<d 8i"eO/ora"e/a0m/11$1$#/db0/orap<D6SM1
pa00<ordOora"e entrie0O*#
{node*:a0m}/ora"e/a0m/11$1$#/db0 # "0 -"a orap<=
-r<-r----- 1 a0m oin0ta"" 1%), Aeb 13 1#:%) orap<D6SM*
Chec.ing Oracle
Cluster,are
{node1:a0m}/ora"e/a0m # crs,stat -t
------------------------------------------------------------
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K7 o# $%$
5e need to register the !SM instances in Oracle Cluster,are egistry *OC+)
Re(i%terin(
A$" in%tance%
wit&in Oracle
Clu%terware 4
-or node1
{node1:a0m}/ora"e/a0m # srvctl add asm 1n node1 1o ?oracle?asm?11.1.&
-or node2
{node1:a0m}/ora"e/a0m # srvctl add asm 1n node2 1o ?oracle?asm?11.1.&
Chec.ing
Oracle
Cluster,are
a#ter !SM
resources
registration)
------------------------------------------------------------
ora$$$$SM1$a0m app"iation .NH'N! .NH'N! node1
ora$$$$SM*$a0m app"iation .NH'N! .NH'N! node*
$tart an% %to0
A$" in%tance
re%%ource 4
Stop AS* on node1< and node2
{node1:a0m}/ora"e/a0m # srvctl stop asm 1n node1
Start AS* on node1< and node2
{node1:a0m}/ora"e/a0m # srvctl start asm 1n node1
Add entry in
etcoratab from
eac& node 4
Add UAS*14?oracle?asm?11.1.&4#
Jn node1 4
{node1:a0m}/ora"e/a0m # cat ?etc?oratab
UAS*14?oracle?asm?11.1.&4#
Jn node2 4
{node2:a0m}/ora"e/a0m # cat ?etc?oratab
UAS*24?oracle?asm?11.1.&4#
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K$ o# $%$
1$)G Con#igure !SM local and remote listeners
Create and !dd entry #or local and remote listeners in Y!SMNDOM;/net,or./admin/tnsnames)ora
*/oracle/asm/11)1)6/net,or./admin/tnsnames)ora+
On node* {node1:a0m}/ora"e/a0m?11.1.&?network?admin # cat tnsnames.ora
# tn0name0$ora Net<or: Con8iguration Ai"e:
/ora"e/produt0/a0m/net<or:/admin/tn0name0$ora
(IS;/#/HS,UAS* 0
EANNH/SS,(IS; 0
EANNH/SS 0 E"HJ;JCJ( 0 ;C"FEGJS; 0 node1-vipFE"JH; 0 1521FF
F
(IS;/#/H,UAS*1 0
EANNH/SS,(IS; 0
F
(IS;/#/H,UAS*2 0
EANNH/SS,(IS; 0
F
On node5 {node2:a0m}/ora"e/a0m?11.1.&?network?admin # cat tnsnames.ora
(IS;/#/HS,UAS* 0
EANNH/SS,(IS; 0
F
(IS;/#/H,UAS*1 0
EANNH/SS,(IS; 0
F
(IS;/#/H,UAS*2 0
EANNH/SS,(IS; 0
F
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KB o# $%$
Set ?:OC!:N:IST;E;@/ ?;MOT;N:ISTE;@ and ?S;3IC;S@ at !SM instance le>el
For node1/ and
node7/
#rom node1 2
{node1:a0m}/ora"e # e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node1:a0m}/ora"e # e9port JHAC(/,SIN0UAS*1
{node1:a0m}/ora"e # s%lplus ?nolo+
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Mon Aeb 15 *):%#:*3 *##5
Conneted$
S[HC s$ow parameter service
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring D6SM
S[HC s$ow parameter listener
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
"oa"?"i0tener 0tring
remote?"i0tener 0tring
S[HC
:et set localNlistener and remoteNlistener
remote6li%tener from node*' and node5 "B$G BE GFE $A"E' and E>GR!E$ "B$G BE
PRE$E>G in t&e tn%name%.ora from eac& node.
local6li%tener from node*' and node5 are different%' and E>GR!E$ "B$G BE PRE$E>G
in t&e tn%name%.ora from eac& node.
local6li%tener from node*' and node5 are not t&e one% defined in t&e li%tener.ora file% from
eac& node.
S[HC A(;/H SBS;/* S/; remote,listener0K(IS;/#/HS,UAS*K SCJ"/0)J;G SIN0KPK\
Sy0tem a"tered$
S[HC A(;/H SBS;/* S/; local,listener0K(IS;/#/H,UAS*1K SCJ"/0)J;G SIN0KUAS*1K\
Sy0tem a"tered$
S[HC A(;/H SBS;/* S/; local,listener0K(IS;/#/H,UAS*2K SCJ"/0)J;G SIN0KUAS*2K\
Sy0tem a"tered$
S[HC s$ow parameters listener
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
"oa"?"i0tener 0tring H'ST!N!B?D6SM1
remote?"i0tener 0tring H'ST!N!BS?D6SM
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KC o# $%$
Chec. #or
node7/
#rom node7 2
{node*:a0m}/ora"e # e9port JHAC(/,SIN0UAS*1
{node*:a0m}/ora"e # s%lplus ?nolo+
Conneted$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring D6SM
S[HC s$ow parameters listener
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
"oa"?"i0tener 0tring H'ST!N!B?D6SM*
remote?"i0tener 0tring H'ST!N!BS?D6SM
S[HC
Chec.ing the listener status on each node/ ,e ,ill see 7 instances registered/ J!SM1 and J!SM7
{node1:a0m}/ora"e/a0m/11$1$#/net<or:/admin # lsnrctl status
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on 1--A!2-*##5
##:#1:%+
------------------------
Prodution
Start (ate 1)-A!2-*##5 **:#3:%%
Uptime % day0 1 hr$ %+ min$ * 0e
Trae He7e" o88
SNMP .N
Ser7ie0 Summary$$$
Ser7ie ZD6SMZ ha0 * in0taneF0G$
'n0tane ZD6SM1ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZD6SM*ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G 8or thi0 0er7ie$$$
Ser7ie ZD6SM?KPTZ ha0 * in0taneF0G$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KG o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
KK o# $%$
Chec.ing the listener ser>ices on each node/ ,e ,ill see 7 instances registered/ J!SM1 and J!SM7
{node1:a0m}/ora"e/a0m/11$1$#/net<or:/admin # lsnrctl services
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on 1--A!2-*##5
##:#*:3+
Ser7ie0 Summary$$$
Nand"erF0G:
Z(!('C6T!(Z e0tab"i0hed:# re8u0ed:# 0tate:ready
H.C6H S!B/!B
B!M.T! S!B/!B
F6((B!SSOFPB.T.C.HOTCPGFN.STOnode1-7ipGFP.BTO1%*1GG
Nand"erF0G:
B!M.T! S!B/!B
F6((B!SSOFPB.T.C.HOTCPGFN.STOnode*-7ipGFP.BTO1%*1GG
Nand"erF0G:
H.C6H S!B/!B
B!M.T! S!B/!B
F6((B!SSOFPB.T.C.HOTCPGFN.STOnode1-7ipGFP.BTO1%*1GG
Nand"erF0G:
B!M.T! S!B/!B
F6((B!SSOFPB.T.C.HOTCPGFN.STOnode*-7ipGFP.BTO1%*1GG
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K8 o# $%$
For remote connection to !SM instances 2
!dd entry in Y!SMNDOM;/net,or./admin/tnsnames)ora
*/oracle/asm/11)1)6/net,or./admin/tnsnames)ora+
On node*' and
node5 S
{node1:a0m}/ora"e/a0m?11.1.&?network?admin # cat tnsnames.ora
(IS;/#/HS,UAS* 0
EANNH/SS,(IS; 0
F
(IS;/#/H,UAS*1 0
EANNH/SS,(IS; 0
F
(IS;/#/H,UAS*2 0
EANNH/SS,(IS; 0
F
6SM O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node1-7ipGFP.BT O 1%*1GG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node*-7ipGFP.BT O 1%*1GG
FH.6(?26H6NC! O ye0G
FC.NN!CT?(6T6 O
FS!B/!B O (!('C6T!(G
FS!B/'C!?N6M! O D6SMG
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
11g!C/!SM/!I"
oraclibm@fr.ibm.com
K% o# $%$
1$)K Chec. Oracle Cluster,are/ and manage !SM ressources
Chec.ing i# !SM instances are registered in Oracle Cluster,are 2
* A$" in%tance on eac& node mu%t be re(i%tered OOO
To sto( the !SM instance
on s(eci#ied node 2
Eode1 #or e'am(le M
{node1:a0m}/ora"e/a0m # srvctl stop asm -n node1
------------------------------------------------------------
ora$$$$SM1$a0m app"iation .AAH'N! .AAH'N!
To start the !SM instance
on s(eci#ied node 2
Eode1 #or e'am(le M
{node1:a0m}/ora"e/a0m # srvctl start asm -n node1
------------------------------------------------------------
11g!C/!SM/!I"
oraclibm@fr.ibm.com
86 o# $%$
To chec. status and con#ig
o# !SM cluster resources on
node1 and node7 in the
cluster 2
{node1:a0m}/ora"e/a0m # 0r7t" 0tatu0 a0m -n node1
6SM in0tane D6SM1 i0 running on node node1$
{node1:a0m}/ora"e/a0m # 0r7t" 0tatu0 a0m -n node*
6SM in0tane D6SM* i0 running on node node*$
{node1:a0m}/
{node1:a0m}/ora"e/a0m # 0r7t" on8ig a0m -n node1
D6SM1 /ora"e/a0m/11$1$#
{node1:a0m}/ora"e/a0m # 0r7t" on8ig a0m -n node*
D6SM* /ora"e/a0m/11$1$#
To list all attri&utes #rom the
resource !SM on node1 2
{node1:a0m}/ora"e/a0m/11$1$#/net<or:/admin # crs,stat -p
ora.node1.AS*1.asm
N6M!Oora$node1$6SM1$a0m
TIP!Oapp"iation
6CT'.N?SCB'PTO/ora"e/a0m/11$1$#/bin/rag<rap
6CT'/!?PH6C!M!NTO#
6UT.?ST6BTO1
CN!CU?'NT!B/6HO)##
(!SCB'PT'.NOCBS app"iation 8or 6SM in0tane
A6'H./!B?(!H6IO#
A6'HUB!?'NT!B/6HO#
A6'HUB!?TNB!SN.H(O#
.PT'.N6H?B!S.UBC!SO
PH6C!M!NTOre0trited
B![U'B!(?B!S.UBC!SO
B!ST6BT?6TT!MPTSO%
SCB'PT?T'M!.UTO,##
ST6BT?T'M!.UTO-##
ST.P?T'M!.UTO15#
UPT'M!?TNB!SN.H(O+d
USB?.B6?6H!BT?N6M!O
USB?.B6?C.NN!CT?STBO/ a0 0y0a0m
USB?.B6?(!2U&O#
USB?.B6?AH6&SO
USB?.B6?'AO
USB?.B6?H6N&O
USB?.B6?N!TM6SUO
USB?.B6?.P!N?M.(!Omount
USB?.B6?.P'O8a"0e
USB?.B6?PA'H!O
USB?.B6?SB/O
USB?.B6?/'PO
11g!C/!SM/!I"
oraclibm@fr.ibm.com
81 o# $%$
To see resources
read/,rite
(ermissions and
o,nershi( ,ithin
the cluster,are 2
{node1:a0m}/ora"e/a0m/11$1$# # crs,stat -ls
Name .<ner Primary Pri7&rp Permi00ion
-----------------------------------------------------------------
ora$$$$SM1$a0m a0m oin0ta"" r<1r<1r--
ora$$$$!1$"0nr a0m oin0ta"" r<1r<1r--
ora$node1$g0d r0 oin0ta"" r<1r-1r--
ora$node1$on0 r0 oin0ta"" r<1r-1r--
ora$node1$7ip root oin0ta"" r<1r-1r--
ora$$$$SM*$a0m a0m oin0ta"" r<1r<1r--
ora$$$$!*$"0nr a0m oin0ta"" r<1r<1r--
ora$node*$g0d r0 oin0ta"" r<1r-1r--
ora$node*$on0 r0 oin0ta"" r<1r-1r--
ora$node*$7ip root oin0ta"" r<1r-1r--
Chec. !SM
deamons
1or e<am0le on
node* 4
{node1:root}/ # su - asm
{node1:a0m}/ora"e/a0m # ps -ef2+rep AS*
a0m 1,)5+3 1 # Aeb 13 - 1#:#+ a0m?"mon?D6SM1
a0m 15#)3# 1 # Aeb 13 - #:15 a0m?gmon?D6SM1
a0m **1*1, 1 # Aeb 13 - *:1- a0m?rba"?D6SM1
a0m **-%%# 1 # Aeb 13 - #:1* a0m?"g<r?D6SM1
a0m *)+,53 1 # Aeb 13 - 11:13 a0m?"md#?D6SM1
a0m ),3,+# 1 # Aeb 13 - #:#5 a0m?mman?D6SM1
a0m 31+-5* 1 # Aeb 13 - 1+:1% a0m?dia#?D6SM1
a0m 3)3*,# 1 # Aeb 13 - #:),
/ora"e/a0m/11$1$#/bin/ragimon daemon ora$node1$6SM1$a0m
a0m 3%55,* 1 # Aeb 13 - #:1% a0m?:pt?D6SM1
a0m 3,)#13 1 # Aeb 13 - #:), a0m?":#?D6SM1
a0m %3#5+, 1 # Aeb 13 - #:#+ a0m?p0p#?D6SM1
a0m %33-#, 1 # Aeb 13 - 1:*3 a0m?pmon?D6SM1
a0m %++,*, 1 # Aeb 13 - #:*, a0m?ping?D6SM1
a0m %-513, 1 # Aeb 13 - *:1% a0m?diag?D6SM1
a0m ,#,)15 1 # Aeb 13 - #:1# a0m?db<#?D6SM1
a0m ,)#5-, 1 # Aeb 13 - 1*:)# a0m?"m0#?D6SM1
a0m ,)%#,# 1 # Aeb 13 - #:#+ a0m?0mon?D6SM1
a0m ,)-#+* 1 # Aeb 13 - *:*+ a0m?7:tm?D6SM1
a0m -1),%* -%3,#5 # #-:%*:#* pt0/% #:## grep 6SM
11g!C/!SM/!I"
oraclibm@fr.ibm.com
87 o# $%$
1$)8 !&out !SM instances Tuning
Iou should increase the !SM Instance (rocess (arameter on each node M
"ovin( from t&e default *CC value to 5CC' or ACC de0endin( of your !O activitie%.
Goo low 0roce%% value may (enerate readwrite !O acce%% error from t&e databa%e in%tance% to t&e A$" di%,
2rou0%.
{node1:a0m}/ora"e # e9port JHAC(/,SIN0UAS*1
{node1:a0m}/ora"e # s%lplus ?nolo+
Connected
S[HC s$ow parameter processes
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
proe00e0 integer 1##
S[HC
^proe00e0_ parameter an not be hanged on the 8"yQ in dynami$
Iou mu0t modi8y the parameter on both 6SM in0tane0Q and permanent"y in the 6SM
0p8i"eQ then 0top and re0tart 6SM in0tane to ati7ate the hange :
S[HC alter system set processes02&& scope0spfile SIN0KPK\
Sy0tem a"tered$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
proe00e0 integer 1##
S[HC e9it
(et stop and restart AS* instances t$ru clusterware commands 4
{node1:a0m}/ora"e/a0m # crs,stat -t 2 +rep asm
ora$$$$SM1$a0m app"iation .AAH'N! .AAH'N!
ora$$$$SM*$a0m app"iation .AAH'N! .AAH'N!
{node1:a0m}/ora"e/a0m # crs,stat -t 2 +rep asm
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8$ o# $%$
Then under S[HP"u0 :
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
proe00e0 integer *##
S[HC
Subject: Doc ID:
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8B o# $%$
1$)% !&out !SM CM1 *Command 4tility+
!ccess to document 2
bCommanding !SM c on &tt04www.oracle.comorama(
&tt04www.oracle.comtec&nolo(yorama(oracleC/-maro5/a%m.&tml
Acce%%' tran%fer' and admini%ter A$" file% wit&out $HL command%.
By Aru0 >anda
b!dd Storage/ Eot Pro9ects c on &tt04www.oracle.comorama(
&tt04www.oracle.comtec&nolo(yorama(oracleCA-mayoJAtec&6mana(ement.&tml
Automatic $tora(e "ana(ement let% DBA% do t&eir Kob% and ,ee0 t&eir wee,end%.
By #onat&an 2ennic,
To access the
content o# the
!SM 1is.s
=rou(s 2
From node1 2
{node1:a0m}/ora"e/a0m # e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node1:a0m}/ora"e/a0m # e9port JHAC(/,SIN0UAS*1
{node1:a0m}/ora"e/a0m # asmcmd -a sysasm -p
6SMCM( >D@ C he"p
a0mmd >-7@ >-a T0y0a0m\0y0dbaC@ >-p@ >ommand@
The en7ironment 7ariab"e0 .B6CH!?N.M! and .B6CH!?S'( determine the
in0tane to <hih the program onnet0Q and 6SMCM( e0tab"i0he0 a
beJueath onnetion to itQ in the 0ame manner a0 a S[HPHUS / 6S
SIS(26$ The u0er mu0t be a member o8 the SIS(26 group$
Spei8ying the -7 option print0 the a0mmd 7er0ion number and
e1it0 immediate"y$
Spei8y the -a option to hoo0e the type o8 onnetion$ There are
on"y t<o po00ibi"itie0: onneting a0 Z0y0a0mZ or a0 Z0y0dbaZ$
The de8au"t 7a"ue i8 thi0 option i0 un0pei8ied i0 Z0y0a0mZ$
Spei8ying the -p option a""o<0 the urrent diretory to be di0p"ayed
in the ommand promptQ "i:e 0o:
6SMCM( >D(6T6A'H!/.BCH/C.NTB.HA'H!@ C
>ommand@ 0pei8ie0 one o8 the 8o""o<ing ommand0Q a"ong <ith it0
parameter0$
Type Zhe"p >ommand@Z to get he"p on a 0pei8i 6SMCM( ommand$
ommand0:
--------
he"p
d
p
du
8ind
"0
"0t
"0dg
m:a"ia0
m:dir
p<d
rm
rma"ia0
md,backup E#ew wit$ 117F
md,restore E#ew wit$ 117F
lsdsk E#ew wit$ 117F
remap E#ew wit$ 117F
6SMCM( >D@ C
Subject: Doc ID:
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8C o# $%$
4sing !SMC1 ?ls@ command 2
6SMCM( >D@ C he"p "0
"0 >-"0drtHagN@ >name@
Hi0t >name@ or it0 ontent0 a"phabetia""y i8 >name@ re8er0 to a
diretory$ >name@ an ontain the <i"dard Z=Z and i0 the urrent
diretory i8 un0pei8ied$ (iretory name0 in the di0p"ay "i0t
ha7e the Z/Z 0u88i1 to "ari8y their identity$ The 8ir0t t<o optiona"
8"ag0 0pei8y ho< muh in8ormation i0 di0p"ayed 8or eah 8i"eQ in the
8o""o<ing manner:
Fno 8"agG /]6SM?6H'6S$N6M!
-" /]6SM?6H'6S$N6M!Q /]6SM?6H'6S$SIST!M?CB!6T!(V
/]6SM?A'H!$TIP!Q /]6SM?A'H!$B!(UN(6NCIQ
/]6SM?A'H!$STB'P!(Q /]6SM?A'H!$M.('A'C6T'.N?(6T!
-0 /]6SM?6H'6S$N6M!V
/]6SM?A'H!$2H.CU?S'X!Q /]6SM?A'H!$2H.CUSQ
/]6SM?A'H!$2IT!SQ /]6SM?A'H!$SP6C!
'8 the u0er 0pei8ie0 both 8"ag0Q then the ommand 0ho<0 an union o8
their re0peti7e o"umn0Q <ith dup"iate0 remo7ed$
'8 an entry in the "i0t i0 an u0er-de8ined a"ia0 or a diretoryQ
then -" di0p"ay0 on"y the /]6SM?6H'6S o"umn0Q and -0 0ho<0 on"y
the a"ia0 name and it0 0iLeQ <hih i0 Lero beau0e it i0 neg"igib"e$
Moreo7erQ the di0p"ayed name ontain0 a 0u88i1 that i0 in the 8orm o8
an arro< pointing to the ab0o"ute path o8 the 0y0tem-reated 8i"ename
it re8erene0:
t?db1$8 OC Ddi0:groupName/(2Name/(6T6A'H!/SIST!M$*%,$1
See the -H option be"o< 8or an e1eption to thi0 ru"e$
Aor di0: group in8ormationQ thi0 ommand Juerie0 /]6SM?('SU&B.UP?ST6T
by de8au"tQ <hih an be modi8ied by the - and -g 8"ag0$
The remaining 8"ag0 ha7e the 8o""o<ing meaning0:
-d '8 an argument i0 a diretoryQ "i0t on"y it0 name
Fnot it0 ontent0G$
-r Be7er0e the 0orting order$
-t Sort by time 0tamp F"ate0t 8ir0tG in0tead o8 by name$
-H '8 an argument i0 an u0er a"ia0Q di0p"ay in8ormation on
the 8i"e it re8erene0$
-a '8 an argument i0 a 0y0tem-reated 8i"enameQ 0ho< the
"oation o8 it0 u0er-de8ined a"ia0Q i8 any$
- Aor di0: group in8ormationQ 0e"et 8rom /]6SM?('SU&B.UPQ
or &/]6SM?('SU&B.UPQ i8 the -g 8"ag i0 a"0o ati7e$
'8 the 6SM 0o8t<are 7er0ion i0 1#gB1Q then the e88et0
o8 thi0 8"ag i0 a"<ay0 ati7eQ <hether or not it i0 0et$
-g Aor di0: group in8ormationQ 0e"et 8rom
&/]6SM?('SU&B.UP?ST6TQ or &/]6SM?('SU&B.UPQ i8 the -
8"ag i0 a"0o ati7eV &/]6SM?('SU&B.UP$'NST?'( i0
in"uded in the output$
-N Suppre00 the o"umn header in8ormationQ 0o that
0ripting i0 ea0ier$
Note that Z"0 DZ <ou"d return in8ormation on a"" di0:group0Q in"uding
<hether they are mounted$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8G o# $%$
Not a"" po00ib"e 8i"e o"umn0 or di0: group o"umn0 are in"uded$
To 7ie< the omp"ete 0et o8 o"umn0 8or a 8i"e or a di0: groupQ
Juery the /]6SM?A'H! and /]6SM?('SU&B.UP 7ie<0$
6SMCM( >D@ C
4sing !SMC1 ?lsct@ command 2
6SMCM( >D@ C he"p "0t
"0t >-gN@ >group@
Hi0t a"" "ient0 and their in8ormation 8rom /]6SM?CH'!NT$ '8 group i0
0pei8iedQ then return on"y in8ormation on that group$
-g Se"et 8rom &/]6SM?CH'!NT$
-N Suppre00 the o"umn header0 8rom the output$
6SMCM( >D@ C
4sing !SMC1 ?du@ command 2
6SMCM( >D@ C he"p du
du >-N@ >dir@
(i0p"ay tota" 0pae u0ed 8or 8i"e0 "oated reur0i7e"y under >dir@Q
0imi"ar to ^du S0_ under UN'KV de8au"t i0 the urrent diretory$ T<o
7a"ue0 are returnedQ both in unit0 o8 megabyte0$ The 8ir0t 7a"ue doe0
not ta:e into aount mirroring o8 the di0:group <hi"e the 0eond doe0$
Aor in0taneQ i8 a 8i"e oupie0 1## M2 o8 0paeQ then it atua""y
ta:e0 up *## M2 o8 0pae on a norma" redundany di0:group and )## M2
o8 0pae on a high redundany di0:group$
>dir@ an a"0o ontain <i"dard0$
The SN 8"ag 0uppre00e0 the o"umn header0 8rom the output$
6SMCM( >D@ C
4sing !SMC1 ?lsdg@ command 2
6SMCM( >D@ C $elp lsd+
"0dg >-gN@ >group@
Hi0t a"" di0:group0 and their in8ormation 8rom /]6SM?('SU&B.UP$ '8
>group@ i0 0pei8iedQ then return on"y in8ormation on that group$ The
ommand a"0o in8orm0 the u0er i8 a reba"ane i0 urrent"y under <ay
8or a di0:group$
Thi0 ommand Juerie0 /]6SM?('SU&B.UP?ST6T by de8au"tQ <hih an be
modi8ied by the - and -g 8"ag0$
- Se"et 8rom /]6SM?('SU&B.UPQ or &/]6SM?('SU&B.UPQ
i8 the -g 8"ag i0 a"0o ati7e$ '8 the 6SM 0o8t<are
7er0ion i0 1#gB1Q then the e88et0 o8 thi0 8"ag i0
a"<ay0 ati7eQ <hether or not it i0 0et$
-g Se"et 8rom &/]6SM?('SU&B.UP?ST6TQ or &/]6SM?('SU&B.UPQ
i8 the - 8"ag i0 a"0o ati7eV
&/]6SM?('SU&B.UP$'NST?'( i0 in"uded in the output$
-N Suppre00 the o"umn header0 8rom the output$
Not a"" po00ib"e di0: group o"umn0 are in"uded$ To 7ie< the
omp"ete 0et o8 o"umn0 8or a di0: groupQ Juery the /]6SM?('SU&B.UP
11g!C/!SM/!I"
oraclibm@fr.ibm.com
8K o# $%$
7ie<$
6SMCM( >D@ C
11g!C/!SM/!I"
oraclibm@fr.ibm.com
88 o# $%$
unning !SMC1 command *lsdg/ lsct/ ls/ du/ lsct+ 2
From node1 2
{node1:ora"e}/ora"e -C e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node1:ora"e}/ora"e -C e9port JHAC(/,SIN0UAS*1
{node1:ora"e}/ora"e -C asmcmd -a sysasm -p
6SMCM( >D@ C
6SMCM( >D@ C ls+d
AS*C*N VUX 8 lsd+
State ;ype Hebal Sector )lock A6 ;otal,*) -ree,*) He%,mir,free,*) 6sable,file,*) Jffline,disks #ame
*J6#;/N #JH*A( # 512 5&C: 1&5.5': 255': 253.2 5&C: 1&153 & NA;A,N71?
6SMCM( >D@ C
6SMCM( >D@ C lsd+ -+
'n0t?'( State Type Beba" Setor 2"o: 6U Tota"?M2 Aree?M2 BeJ?mir?8ree?M2 U0ab"e?8i"e?M2 .88"ine?di0:0 Name
1 M.UNT!( N.BM6H N %1* 3#-, 1#35%+, *3%+, *3)5* 3#-, 1#13) # (6T6?(&1/
* M.UNT!( N.BM6H N %1* 3#-, 1#35%+, *3%+, *3)5* 3#-, 1#13) # (6T6?(&1/
6SMCM( >D@ C
6SMCM( >D@ C du
U0ed?M2 Mirror?u0ed?M2
*3,+ 3-%*
6SMCM( >D@ C
11g!C/!SM/!I" oraclibm@fr.ibm.com 8% o# $%$
unning !SMC1 command *lsdg/ lsct/ ls/ du/ lsct+ 2
From node1
{node1:ora"e}/ora"e -C e9port JHAC(/,GJ*/0?oracle?asm?11.1.&
{node1:ora"e}/ora"e -C e9port JHAC(/,SIN0UAS*1
{node1:ora"e}/ora"e -C asmcmd -a sysasm -p
6SMCM( >D@ C
6SMCM( >D@ C ls
(6T6?(&1/
6SMCM( >D@ C
6SMCM( >D@ C ls -la
State ;ype Hebal #ame
*J6#;/N #JH*A( # NA;A,N71?
AS*C*N VUX 8
6SMCM( >D@ C ls -s
Setor 2"o: 6U Tota"?M2 Aree?M2 BeJ?mir?8ree?M2 U0ab"e?8i"e?M2 .88"ine?di0:0 Name
%1* 3#-, 1#35%+, *3%+, *3)5* 3#-, 1#13) # (6T6?(&1/
6SMCM( >D@ C
6SMCM( >D@ C ls -+
'n0t?'( Name
1 (6T6?(&1/
* (6T6?(&1/
6SMCM( >D@ C
6SMCM( >D@ C du
L#o database create at t$is sta+e< so no resultM
6SMCM( >D@ C
6SMCM( >D@ C lsct
L#o database create at t$is sta+e< so no resultM
6SMCM( >D@ C
6SMCM( >D@ C "0t
(2?Name Statu0 So8t<are?/er0ion Compatib"e?7er0ion 'n0tane?Name (i0:?&roup
JSC1(2 C.NN!CT!( 11$1$#$,$# 11$1$#$#$# JSC1(21 (6T6?(&1
6SMCM( >D@ C "0t -g
'n0tane?'( (2?Name Statu0 So8t<are?/er0ion Compatib"e?7er0ion 'n0tane?Name (i0:?&roup
1 JSC1(2 C.NN!CT!( 11$1$#$,$# 11$1$#$#$# JSC1(21 (6T6?(&1
* JSC1(2 C.NN!CT!( 11$1$#$,$# 11$1$#$#$# JSC1(2* (6T6?(&1
6SMCM( >D@ C
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%6 o# $%$
1$)16 4se#ull Metalin. notes
Subject: Doc ID:
Subject: Doc ID:
Subject: Doc ID:
Subject:
Doc ID:
Subject: Doc ID:
Subject: Doc ID:
Subject: Doc ID:
1$)11 5hat has &een done R
!t this stage 2
The Oracle Cluster egistry and 3oting 1is. are created and con#igured
The Oracle Cluster eady Ser>ices is installed/ and started on all nodes)
The 3IP *3irtual IP+/ =S1 and OES a((lication resources are con#igured on all
nodes)
!SM Dome is installed
1e#ault node listener #or !SM are created
!SM instances are created and started
!SM 1is.grou( is created
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%1 o# $%$
1B IEST!::IE= O!C:; 11G 1 SOFT5!;
G&i% coo,boo, i% dealin( wit& **( RAC databa%e-%.' but if you &ave al%o *C( databa%e to run in clu%ter mode' you
can u%e **( Oracle Clu%terware and **( A$"' but ,ee0in( *C( databa%e-%..
Eac& RAC databa%e %oftware will &ave it% own ORACLE6FO"E OOO
1or all new RAC 0roKect%' wePll advice to u%e Oracle **( clu%terware and Oracle **( A$".
1or **( RAC databa%e-%.' all new feature% from **( A$" will be available for u%e.
1or *C( RAC databa%e-%.' not all **( A$" new feature% are available for u%e' you &ave to u%e different A$"
di%,(rou0% for *C( and **( RAC databa%e-%.' and %et 0ro0er A$" di%(rou0 3com0atibleTattribute.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%7 o# $%$
!t this stage 2
Oracle clu%terware i% in%talled in UORA6CR$6FO"E -cr%**.*.C.
Oracle A$" %oftware i% in%talled in UA$"6FO"E -oraclea%m**.*.C.
Default Li%tener for A$" in%tance% are confi(ured wit&in t&e UA$"6FO"E
A$" in%tance% are u0 and runnin(
At lea%t on A$" Di%, 2rou0 i% created and mounted on eac& node.
!n our ca%e' we &ave a Di%,(rou0 called
3DAGA6D2*T w&ere to create an Oracle
clu%ter databa%e.
B4T #or no,/ ,e need to install the Oracle 11g 1ata&ase So#t,are in its o,n YO!C:;NDOM;
*/oracle/rd&ms/11)1)6+ using the rd&ms uni' user)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%$ o# $%$
1B)1 11g 1 1BMS Installation
Oracle RAC o0tion in%tallation Ku%t &ave to be done only %tartin( from one node. Once t&e fir%t node i% in%talled'
Oracle OB! automatically %tart% t&e co0y of t&e mandatory file% on t&e %econd node' u%in( rc( command. G&i% %te0
could la%t lon(' de0endin( on t&e networ, %0eed -one &ourS.' wit&out any me%%a(e. $o' donPt t&in, t&e OB! i%
%talled' and loo, at t&e networ, traffic before cancelin( t&e in%tallation O
Mou can al%o create a %ta(in( area. G&e name of t&e %ubdirectorie% i% in t&e format 3Di%,*T to 3Di%,JT
On each node 2
From #irst node !s
{node1:root}/ #
{node1:root}/ # ho<n rdbm0:oin0ta"" /ora"e/rdbm0
{node1:root}/ # hmod ,,% /ora"e/rdbm0
{node1:root}/ #
:ogin as rd&ms *in our case+/ or oracle user and #ollo, the (rocedure hereunderM
Setu( and
e'(ort your
1ISP:!I/ TMP and
T;MP >aria&les
5ith /tm( or other destination ha>ing enough #ree s(ace/ a&out C66M& on
each node)
{node14rdbms}/ # e9port NIS"(AB0node14&.&
If not set in asm .profile< do 4
{node1:rdbm0}/ # e9port ;*"0?tmp
{node1:rdbm0}/ # e9port ;/*"0?tmp
{node1:rdbm0}/ # e9port ;*"NIH0?tmp
Chec. that Oracle Cluster,are *including 3IP/ OES/ =S1/ :istener-s and !SM ressources+ is started on
each node 000) !SM instances and listener-s are not mandatory #or the 1BMS installation)
!s asm user #rom
node1 2
{node1:rdbms}/ora"e/rdbm0 # crs,stat -t
------------------------------------------------------------
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%B o# $%$
Vnode14rd&msWdi%trib$oftwareOraclerdbm%**(r*ai<databa%e ; )/runInstaller
$tartin( Oracle Bniver%al !n%taller...
C&ec,in( Gem0 %0ace4 mu%t be (reater t&an *DC "B. Actual *DJJ "B Pa%%ed
C&ec,in( %wa0 %0ace4 mu%t be (reater t&an *@C "B. Actual J@IA "B Pa%%ed
Pre0arin( to launc& Oracle Bniver%al !n%taller from tm0Ora!n%tall5CCI-C5-*A6C5-A*-ADP". Plea%e wait
...Vnode*4rdbm%Wdi%trib$oftwareOraclerdbm%**(r*ai<databa%e ; Oracle Bniver%al !n%taller' Ner%ion **.*.C./.C
Production
Just clic. Ee't )))
Select the installation ty(e 2
Mou &ave t&e o0tion to c&oo%e Enter0ri%e'
$tandard Edition' or Cu%tom to 0roceed.
Choose the ?Custom@ o0tion to avoid
creatin( a databa%e by default.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%C o# $%$
S(eci#y File :ocations 2
Do not c&an(e t&e $ource field
$0ecify a different ORACLE6FO"E >ame
wit& it% own directory for t&e Oracle
%oftware in%tallation.
This O!C:;NDOM; must &e
di##erent then the CS and !SM
O!C:;NDOM;)
OraDb**(6&ome*
oraclerdbm%**.*.C
Then clic. Ee't )))
I# you don-t see the #ollo,ing screen
,ith Eode selection/ it might &e that
your CS is do,n on one or all nodes)
Please chec. i# CS is u( and
running on all nodes)
S(eci#y Dard,are Cluster Installation
Mode 2
$elect Cluster Installation
A>D t&e ot&er node% on to w&ic& t&e
Oracle RDB"$ %oftware will be in%talled.
!t i% not nece%%ary to %elect t&e node on
w&ic& t&e OB! i% currently runnin(. Clic,
>e<t.
Then clic. Ee't )))
The installer ,ill chec. some (roduct-
s(eci#ic PrerePuisite)
DonPt ta,e care of t&e line% wit& c&ec,in(
at %tatu% 3>ot e<ecutedT' G&e%e are Ku%t
warnin(% becau%e A!+ maintenance level
mi(&t be &i(&er t&en @JCC' w&ic& i% t&e
ca%e in our e<am0le -"LCJ..
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%G o# $%$
1etails o# the
(rerePuisite
chec.s done
&y
runInstaller
$eckin+ operatin+ system re%uirements ...
/9pected result4 Jne of 53&&.&5<:1&&.&&
Actual Hesult4 53&&.&'
C$eck complete. ;$e overall result of t$is c$eck is4 "assed
00000000000000000000000000000000000000000000000000000000
C$eckin+ operatin+ system packa+e re%uirements ...
C$eckin+ for bos.adt.baseE&.&F\ found bos.adt.baseE5.3.'.&F. "assed
C$eckin+ for bos.adt.libE&.&F\ found bos.adt.libE5.3.&.:&F. "assed
C$eckin+ for bos.adt.libmE&.&F\ found bos.adt.libmE5.3.'.&F. "assed
C$eckin+ for bos.perf.libperfstatE&.&F\ found bos.perf.libperfstatE5.3.'.&F. "assed
C$eckin+ for bos.perf.perfstatE&.&F\ found bos.perf.perfstatE5.3.'.&F. "assed
C$eckin+ for bos.perf.proctoolsE&.&F\ found bos.perf.proctoolsE5.3.'.&F. "assed
C$eckin+ for rsct.basic.rteE&.&F\ found rsct.basic.rteE2.5...&F. "assed
C$eckin+ for rsct.compat.clients.rteE&.&F\ found rsct.compat.clients.rteE2.5...&F. "assed
C$eckin+ for bos.mp:5E5.3.&.5:F\ found bos.mp:5E5.3.'.1F."assed
C$eckin+ for bos.rte.libcE5.3.&.55F\ found bos.rte.libcE5.3.'.1F. "assed
C$eckin+ for 9lC.ai95&.rteE..&.&.'F\ found 9lC.ai95&.rteEC.&.&.1F. "assed
C$eckin+ for 9lC.rteE..&.&.'F\ found 9lC.rteEC.&.&.1F. "assed
00000000000000000000000000000000000000000000000000000000
C$eckin+ recommended operatin+ system patc$es
C$eckin+ for IB.C&.&Ebos.rte.aio<5.3.&.51F\ found Ebos.rte.aio<5.3.'.&F. "assed
C$eckin+ for IBC2&3'Ebos.rte.aio<5.3.&.52F\ found Ebos.rte.aio<5.3.'.&F. "assed
C$eckin+ for IBC5353Ebos.rte.lvm<5.3.&.55F\ found Ebos.rte.lvm<5.3.'.&F. "assed
00000000000000000000000000000000000000000000000000000000
C$eckin+ kernel parameters
C$eck complete. ;$e overall result of t$is c$eck is4 #ot e9ecuted ]]]]
J6I-1.&&14 ;$e operatin+ system KAI Iersion 53&&.&'K is not supported.
Hecommendation4 "erform operatin+ system specific instructions to update t$e kernel parameters.
00000000000000000000000000000000000000000000000000000000
C$eckin+ p$ysical memory re%uirements ...
/9pected result4 C22*)
Actual Hesult4 2&5.*)
00000000000000000000000000000000000000000000000000000000
C$eckin+ available swap space re%uirements ...
/9pected result4 3&'2*)
Actual Hesult4 355.*)
00000000000000000000000000000000000000000000000000000000
Ialidatin+ JHAC(/,)AS/ location Eif setF ...
00000000000000000000000000000000000000000000000000000000
C$eckin+ ma9mimum command line len+t$ ar+ument< ncar+...
00000000000000000000000000000000000000000000000000000000
C$eckin+ for proper system clean-up....
00000000000000000000000000000000000000000000000000000000
C$eckin+ Jracle Clusterware version ...
00000000000000000000000000000000000000000000000000000000
C$eckin+ uid?+id...
Chec. com(lete) The o>erall result o# this chec. is2 Passed
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%K o# $%$
!>aila&le Product Com(onents 2
$elect t&e 0roduct com0onent% for Oracle
Databa%e **( t&at you want to in%tall.
IEFO 2 Com0ared to *C(RAC R* in%tallation'
t&ere i% no 3Real A00lication Clu%terT o0tion to
%elect.
Then clic. Ee't )))
Pri>ileged O(erating Systems =rou(s 2
Nerify t&e B>!+ 0rimary (rou0 name of t&e u%er
w&ic& control% t&e in%tallation of t&e Oracle**(
databa%e %oftware. -B%e uni< command id to find
out.
And %0ecify t&e Privile(ed O0eratin( $y%tem
2rou0% to t&e value found.
!n our e<am0le' t&i% mu%t be 4
3dbaT for Databa%e Admini%trator -O$DBA.
(rou0
3o0erT for Databa%e O0erator -O$OPER.
(rou0
3a%mT for admini%trator -$M$A$". (rou0
-Primary (rou0 of uni< a%m u%er. to be %et for
bot& entrie%.
Then clic. Ee't )))
Create 1ata&ase 2
Choose ?Install data&ase So#t,are only@/ we
donPt want to create a databa%e at t&i% %ta(e.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%8 o# $%$
Summary 2
The Summary screen ,ill &e (resented)
Con#irm that the !C data&ase so#t,are and
other selected o(tions ,ill &e installed)
C&ec, Clu%ter >ode% and Remote >ode% li%t%.
G&e OB! will in%tall t&e Oracle *C( %oftware on to
t&e local node' and t&en co0y t&i% information to
t&e ot&er %elected node%.
Install 2
The Oracle 4ni>ersal Installer ,ill (roceed the
installation on the #irst node' t&en will co0y
automatically t&e code on t&e ot&er% %elected
node%.
At @CY' it may ta,e time to 0a%% over t&e @CY'
donPt be affrai%' in%tallation i% 0roce%%in(' runnin(
a te%t %cri0t on remote node%
At t&i% %ta(e' you could &it me%%a(e about 31ailed Attac&ed FomeT OOO
i# similar screen message a((ears/ 9ust run the s(eci#ied command on the s(eci#ied node as asm user)
1o e'ecute the scri(t as &ello, *chec. and ada(t the scri(t u(on your message+) 5hen done/ clic. on ?OT@ 2
From node2 6
{node2:root}/ora"e/rdbm0/11$1$# # su 1 rdbms
{node2:rdbms}/ora"e/rdbm0/11$1$# -C ?oracle?rdbms?11.1.&?oui?bin?runInstaller 1attac$Gome 1
noCluster/nabled JHAC(/,GJ*/0?oracle?rdbms?11.1.& JHAC(/,GJ*/,#A*/0OraDb**(6&ome*
C(6S;/H,#JN/S0node1<node2 LI#I/#;JHB,(JCA;IJ#0?oracle?oraInventoryM (JCA(,#JN/0node2
Startin+ Jracle 6niversal InstallerR
#o pre-re%uisite c$ecks found in oraparam.ini< no system pre-re%uisite c$ecks will be e9ecuted.
;$e inventory pointer is located at ?etc?oraInst.loc
;$e inventory is located at ?oracle?oraInventory
ZAttac$Gome[ was successful.
{node*:rdbm0}/ora"e/rdbm0/11$1$#/bin #
;'ecute Con#iguration Scri(ts
11g!C/!SM/!I"
oraclibm@fr.ibm.com
%% o# $%$
,ill (o(-u( 2
!S root/ e'ecute root)sh on each
node)
For our case/ this scri(t is
located in the
/oracle/(roducts/asm
Just clic. OT )))
{node1:root}/ # id
uidO#FrootG gidO#F0y0temG group0O*FbinGQ)F0y0GQ+F0eurityGQ5FronGQ1#FauditGQ11F"pG
{node1:root}/ # ?oracle?rdbms?11.1.&?root.s$
.B6CH!?.4N!BO rdbm0
.B6CH!?N.M!O /ora"e/rdbm0/11$1$#
The 8i"e ZdbhomeZ a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@:
The 8i"e Zoraen7Z a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@:
{node1:root}/ #
---------------------------------------------------------------
{node2:root}/ # id
uidO#FrootG gidO#F0y0temG group0O*FbinGQ)F0y0GQ+F0eurityGQ5FronGQ1#FauditGQ11F"pG
{node2:root}/ # /ora"e/rdbm0/11$1$#/root$0h
.B6CH!?.4N!BO rdbm0
.B6CH!?N.M!O /ora"e/rdbm0/11$1$#
The 8i"e ZdbhomeZ a"ready e1i0t0 in /u0r/"oa"/bin$ .7er<rite itE Fy/nG >n@:
11g!C/!SM/!I"
oraclibm@fr.ibm.com
166 o# $%$
{node*:root}/ #
Coming &ac. to
this (re>ious
screen/
Just clic. OT
;nd o# Installation 2
G&i% %creen will automatically a00ear.
C&ec, t&at it i% %ucce%%ful and write down t&e
BRL li%t of t&e #5EE a00lication% t&at &ave
been de0loyed -i%)l0lu%' S..
Then clic. ;'it )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
161 o# $%$
1B)7 Sym&olic lin.s creation #or listener)ora/ tnsnames)ora and sPlnet)ora
In order to create the sym&olic lin.s/ ma.e sure that the #iles ?listener)ora@/ ?tnsnames)ora@ and
?sPlnet)ora@ does e'ist in /oracle/asm/11)1)6/net,or./admin/listener)ora on &oth nodes)
!s rd&ms user on each node 2
{node1:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?listener.ora
?oracle?rdbms?11.1.&?network?admin?listener.ora
{node1:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?tnsnames.ora
?oracle?rdbms?11.1.&?network?admin?tnsnames.ora
{node1:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?s%lnet.ora
?oracle?rdbms?11.1.&?network?admin?s%lnet.ora
{node1:rdbm0}/ # "0 S"a /ora"e/rdbm0/11$1$#/net<or:/admin/=$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- "i0tener$ora -C /ora"e/
a0m/11$1$#/net<or:/admin/"i0tener$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- tn0name0$ora -C
/ora"e/a0m/11$1$#/net<or:/admin/tn0name0$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- 0J"net$ora -C
/ora"e/a0m/11$1$#/net<or:/admin/0J"net$ora
{node1:rdbm0}/ #
Doin( t&i% will avoid dealin( wit& many li%tener.ora' tn%name%.ora and %)lnet.ora' a% %ettin( G>$6AD"!>
variable will not be enou(& a% %ome oracle a%%i%tant% tool% will not ta,e con%ideration of t&i% variable.
!s rd&ms user on node7 2
{node2:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?listener.ora
?oracle?rdbms?11.1.&?network?admin?listener.ora
{node2:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?tnsnames.ora
?oracle?rdbms?11.1.&?network?admin?tnsnames.ora
{node2:rdbm0}/ # ln -s ?oracle?asm?11.1.&?network?admin?s%lnet.ora
?oracle?rdbms?11.1.&?network?admin?s%lnet.ora
{node2:rdbm0}/ # "0 S"a /ora"e/rdbm0/11$1$#/net<or:/admin/=$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- "i0tener$ora -C /ora"e/
a0m/11$1$#/net<or:/admin/"i0tener$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- tn0name0$ora -C
/ora"e/a0m/11$1$#/net<or:/admin/tn0name0$ora
"r<1r<1r<1 1 ora"e dba 3+ 6pr *) 1#:1- 0J"net$ora -C
/ora"e/a0m/11$1$#/net<or:/admin/0J"net$ora
{node*:rdbm0}/ #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
167 o# $%$
1B)$ 4(date the 1BMS uni' user )(ro#ile
To &e done on each node #or rd&ms *in our case+ or oracle uni' user)
vi YDOM;/)(ro#ile file in rdbm%P% &ome directory. !dd the entries in &old &lue color
e1port P6TN
8i # periodia""y$
!N/O]N.M!/$:0hr
e1port !N/
e1port T!MPO/tmp
e1port TMPO/tmp
e1port TMP('BO/tmp
uma0: #**
e1port CBS?N.M!O/r0/11$1$#
e1port .B6?CBS?N.M!O]CBS?N.M!
e1port 6SM?N.M!O].B6CH!?26S!/a0m/11$1$#
e1port .B6?6SM?N.M!O]6SM?N.M!
e9port JHAC(/,GJ*/03JHAC(/,)AS/?rdbms?11.1.&
e1port
H(?H'2B6BI?P6TN03JHAC(/,GJ*/?lib:]CBS?N.M!/"ib43JHAC(/,GJ*/?lib32:]CBS?N.M!/"ib)*
e1port H'2P6TNO]H(?H'2B6BI?P6TN
e1port P6TNO3JHAC(/,GJ*/?bin:]CBS?N.M!/bin:]P6TN
e9port ;#S,AN*I#03AS*,GJ*/?network?admin
e9port JHAC(/,SIN0
i8 > -t # @V then
0tty intr RC
8i
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16$ o# $%$
1C 1!T!B!S; C;!TIOE OE !SM
!n our ca%e' we &ave a Di%,(rou0 called
3DAGA6D2*T w&ere to create an Oracle
clu%ter databa%e.
M!E1!TOI
Change (ermission
to allo, rd&ms user to
,rite on directories
on,ned &y asm user)
{node2:root}/ # r0h node1 hmod -B gD< /ora"e/a0m/11$1$#/net<or:
{node*:root}/ # r0h node* hmod -B gD< /ora"e/a0m/11$1$#/net<or:
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/8gtoo""og0
{node*:root}/ # r0h node* hmod -B gD< /ora"e/8gtoo""og0
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/admin
{node*:root}/ # r0h node* hmod -B gD< /ora"e/admin
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/diag
{node*:root}/ # r0h node* hmod -B gD< /ora"e/diag
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16B o# $%$
1C)1 Thru Oracle 1BC!
Connect as rd&ms uni' user
from fir%t node'
and setu( your 1ISP:!I
E<ecute d&ca H to lanc& t&e
databa%e confi(uration a%%i%tant
From node1 2
{node1:rdbm0}/ # e1port .B6CH!?N.M!O/ora"e/rdbm0/11$1$#
{node1:rdbm0}/ # e1port .B6CH!?S'(O
{node1:rdbm0}/ # d ].B6CH!?N.M!/bin
{node1:rdbm0}/ora"e/rdbm0/11$1$#/bin #$/dba
Chec. that all resources from
Oracle Clu%terware are %tarted
on t&eir &ome node.
From node1 or node7 2
{node2:rdbm0}/ora"e/rdbm0 # crs,stat -t
------------------------------------------------------------
{node*:rdbm0}/ora"e/rdbm0 #
1BC! 5elcome Screen 2
Select the ?Oracle eal !((lication Cluster
1ata&ase@ o(tion)
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16C o# $%$
O(erations 2
Select the ?Create a 1ata&ase@ o(tion)
Then clic. Ee't )))
Eode Selection 2
Ma.e sure to select all !C nodes)
Then clic. Ee't )))
1ata&ase Tem(lates 2
Select ?=eneral Pur(ose@
Or 3Cu%tom Databa%eT if you want to (enerate
t&e creation %cri0t%.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16G o# $%$
1ata&ase Identi#ication 2
S(eci#y the ?=lo&al 1ata&ase Eame@
G&e 3$!D Prefi<T will be automatically
u0dated. -by default it i% t&e 2lobal Databa%e
>ame.
1or our e<am0le 4 #$C*DB
Then clic. Ee't )))
Management O(tions 2
Chec. ?Con#igure the data&ase ,ith
;nter(rise Manager@ i# you ,ant to use the
1ata&ase Control -local admini%tration..
Or DonPt c&ec, if you 0lan to admini%trate t&e
databa%e u%in( t&e 2rid Control -(lobal
networ, admini%tration.
Iou can also set !lert and &ac.u( 2
Then clic. Ee't )))
1ata&ase Credentials 2
S(eci#y same (ass,ord #or all
administrator users'
or %0ecify individual 0a%%word for eac& u%er.
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16K o# $%$
Storage O(tions 2
Choose !utomatic Storage
Management *!SM+
Then clic. Ee't )))
Iou-ll &e (rom(ted to enter !SM sys
(ass,ord 2
Then Clic. ?OT@
Create 1is. =rou(

Eo, the !SM 1is.s =rou( is created
Select 1is.=rou( to &e used 000
DAGA6D2* in our e<am0le

Then clic. O. )))
1ata&ase File :ocations 2

Select 4se Oracle-Managed Files
!E1 Select 1is.=rou( to use #or the
1ata&ase Files)
Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
168 o# $%$
eco>ery Con#iguration 2

Select the 1is.=rou( to &e used #or the
Flash eco>ery !rea)
In our case 2
J1!T!N1=1
and si<e o# B6%G
;na&le !rchi>ing/ and clic. on ?;dit
!rchi>e Mode arameters@ to s(eci#y an
other !SM 1is. =rou( as di##erent
!rchi>e :og 1estinations)

Iou can also chec. the File :ocation
3aria&les M
Clic. ?O.@ then ?Ee't@ )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
16% o# $%$
1ata&ase Content 2

Select the o(tions needed

Then clic. Ee't )))

Initiali<ation Parameters 2

Select the (arameters needed
Iou can clic. on ?!ll Initiali<ation
Parameters@ to >ie, or modi#y M

Iou can also select the #olders
?Si<ing@
?Character Sets@
?Connection Mode@
to >alidate/modi#y/s(eci#y settings at your
con>enience)
Then clic. Ee't )))
!&out Oracle Cluster data&ase instances (arameters/ ,e-ll ha>e to loo. on these (arameters 2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
116 o# $%$
!n%tance >ame Nalue
Z a%m60referred6read6failure6(rou0% O>LM u%able if you did create and A$" di%,(rou0 in normal or
&i(& redoundancy wit& a minimum of 5 failure% (rou0%.
Naluer will be %et a% t&e name of t&e 0referred failure (rou0.
I# you ,ant to set a di##erent location than the de#ault choosen !SM dis. grou( #or the control #iles/ you
should modi#y the >alue o# (arameter *controlN#iles+
Z control6file% -3=DAGA6D2*VDB6>A"EWcontrolC*.ctlT'
3=DAGA6D2*VDB6>A"EWcontrolC5.ctlT'
3=DAGA6D2*VDB6>A"EWcontrolCJ.ctlT.
Z db6create6file6de%t =DAGA6D2*
-By default' all file% from t&e databa%e will be created on t&e
%0ecified A$" Di%,(rou0.
Z db6recovery6file6de%t =DAGA6D2*
-Default location for bac,u0%' and arc&ive% lo(%.
Z remote6li%tener L!$GE>ER$6#$C*DB
#$C*DB* undo6table%0ace B>DOGB$*
#$C*DB5 undo6table%0ace B>DOGB$5
#$C*DB* local6li%tener L!$GE>ER6#$C*DB*
#$C*DB5 Local6li%tener L!$GE>ER6#$C*DB5
Z lo(6arc&ive6de%t6* RLOCAG!O>9=DAGA6D2*R
#$C*DB* !n%tance6number *
#$C*DB5 !n%tance6number 5
#$C*DB* t&read *
#$C*DB5 t&read 5
Z db6name #$C*DB
Z db6recovery6file6de%t6%iLe A5DAD/E5D/
Z dia(no%tic6de%t -ORACLE6BA$E.
-Default location for t&e databa%e trace% and lo(% file%' it
re0lace t&e B$ER' BACX2ROB>D and DB"P de%tination%.
In
YO!C:;NDOM;/net,or./admin
/ do edit tn%name%.ora file on eac&
node' and add t&e followin( line% 4
:IST;E;SNJSC11B could &e
automatically added &y 1BC! in
the tnsnames)ora
:IST;E;SNJSC11B 9
-ADDRE$$6L!$G 9
-ADDRE$$ 9 -PROGOCOL 9 GCP.-FO$G 9 node*-vi0.-PORG 9 *@5*..
-ADDRE$$ 9 -PROGOCOL 9 GCP.-FO$G 9 node5-vi0.-PORG 9 *@5*..
.
:IST;E;NJSC11B1 9
-ADDRE$$6L!$G 9
-ADDRE$$ 9 -PROGOCOL 9 GCP.-FO$G 9 node*-vi0.-PORG 9 *@5*..
.
:IST;E;NJSC11B7 9
-ADDRE$$6L!$G 9
-ADDRE$$ 9 -PROGOCOL 9 GCP.-FO$G 9 node5-vi0.-PORG 9 *@5*..
.
11g!C/!SM/!I"
oraclibm@fr.ibm.com
111 o# $%$
Eit&er -By default.
Or
Security Settings 2

Select the o(tions
needed
By de#ault
?Tee( the enhanced 11g
de#ault security
settings)@

Then clic. Ee't )))
!utomatic Maintenance Tas.s 2

By de#ault/ ?;na&le !utomatic
Maintenance Tas.s@)

Then clic. Ee't )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
117 o# $%$
1ata&ase Storage 2
Chec. the data#iles organi<ation
!dd one e'tra log mem&er on each thread
no, ,ith the assistant/
or later using OS: commands as 2
ALGER DAGABA$E
ADD LO21!LE GFREAD * 2ROBP @ $![E
@*5CCX
ALGER DAGABA$E
ADD LO21!LE GFREAD 5 2ROBP / $![E
@*5CCX
Then clic. Ee't )))
;MIE1;
Change (ermission
to allo, rd&ms user to
,rite on directories
on,ned &y asm user)
{node2:root}/ # r0h node1 hmod -B gD< /ora"e/a0m/11$1$#/net<or:
{node*:root}/ # r0h node* hmod -B gD< /ora"e/a0m/11$1$#/net<or:
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/8gtoo""og0
{node*:root}/ # r0h node* hmod -B gD< /ora"e/8gtoo""og0
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/admin
{node*:root}/ # r0h node* hmod -B gD< /ora"e/admin
{node*:root}/ # r0h node1 hmod -B gD< /ora"e/diag
{node*:root}/ # r0h node* hmod -B gD< /ora"e/diag
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11$ o# $%$
Creation O(tions 2
Select the o(tions needed
L Create Databa%e
L 2enerate Databa%e Creation $cri0t%

Then clic. Finish )))
Summary 2

Chec. the descri(tion
$ave t&e FG"L %ummary file if needed

Then clic. O. )))
1ata&ase Creation scri(t generation M
5hen #inished 2
Clic. ?O.@ )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11B o# $%$
1ata&ase creation on (rogress 2
Just ,ait ,hile (rocessing )))
Chec. /oracle/c#gtoollogs/d&ca/JSC11B
#or the logs in case o# #ailure to create the
data&ase)
Pass,ords Management
;nter in (ass,ord management/ i# you
need to change (as,ord' and unloc,
%ome u%er account% t&at are loc,ed by
default -for %ecurity 0ur0o%e..
Then clic. ;'it )))
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11C o# $%$
;'ecute
crsNstat t
on one node
as rd&ms
user 2
{node1:rdbm0}/ora"e/rdbm0 # r00tat
----------- ------ -----
ora.^SC1N).^SC1N)1.inst J--(I#/ J--(I#/
ora.^SC1N).^SC1N)2.inst J--(I#/ J--(I#/
ora.^SC1N).db J--(I#/ J--(I#/
ora$node1$6SM1$a0m .NH'N! .NH'N! on node1
ora$node1$H'ST!N!B?N.(!1$"0nr .NH'N! .NH'N! on node1
ora$node*$6SM*$a0m .NH'N! .NH'N! on node*
ora$node*$H'ST!N!B?N.(!*$"0nr .NH'N! .NH'N! on node*
Starting cluster
data&ase
;'ecute
crsNstat t
on one node
as rd&ms
user 2
{node1:rdbm0}/ora"e/rdbm0 # r00tat
----------- ------ -----
ora$JSC1(2$JSC1(21$in0t .NH'N! .NH'N! on node1
ora$JSC1(2$JSC1(2*$in0t .NH'N! .NH'N! on node*
ora$JSC1(2$db .NH'N! .NH'N! on node1
{node1:rdbm0}/ora"e/rdbm0 # r00tat JSC1(2
----------- ------ -----
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11G o# $%$
1C)7 Manual 1ata&ase Creation
Fere are t&e %te0% to be followed to create a Real A00lication Clu%ter% databa%e 4
LetP% ima(ine we want to create an **( clu%ter databa%e called #$C5DB on our 5 RAC node% clu%ter' wit& in%tance%
#$C5DB* on node* and #$C5DB5 on node5.
1) Ma.e the necessary su& directories on each node 2
On node1 2
{node1:root}/ # su 1 rdbms
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/0ript0
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/adump
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/hdump
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/dpdump
{node1:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/p8i"e
{node1:rdbm0}/home/rdbm0 # m:dir /ora"e/diag/rdbm0/W0*db
{node1:rdbm0}/home/rdbm0 # m:dir /ora"e/diag/rdbm0/W0*db/JSC*(21
On node7 2
{node*:root}/ # su 1 rdbms
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/0ript0
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/adump
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/hdump
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/dpdump
{node*:rdbm0}/home/rdbm0 # m:dir ].B6CH!?26S!/admin/JSC*(2/p8i"e
{node*:rdbm0}/home/rdbm0 # m:dir /ora"e/diag/rdbm0/W0*db
{node*:rdbm0}/home/rdbm0 # m:dir /ora"e/diag/rdbm0/W0*db/JSC*(2*
7) Create any !SM dis.grou(*s+ i# needed 2
From node1 2
{node1:root}/ # su 1 asm
{node1:a0m}/home/a0m #
{node1:a0m}/home/a0m # e1port .B6CH!?S'(OD6SM1
{node1:a0m}/home/a0m # e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
{node1:a0m}/ora"e/a0m # 0J"p"u0 /no"og
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Mar *5 1#:*5:%3 *##5
S[HC onnet /a0 0y0a0m
Conneted$
S[HC 0et "ine0iLe 1###
S[HC 0e"et nameQ 0tate 8rom 7]a0m?di0:groupV
N6M! ST6T!
------------------------------ -----------
(6T6?(&1 M.UNT!(
S[HC
S[HC 0e"et N!6(!B?ST6TUSQP6TN 8rom 7]a0m?di0: order by P6TNV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11K o# $%$
N!6(!B?ST6TU P6TN
------------
--------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
M!M2!B /de7/6SM?di0:1
M!M2!B /de7/6SM?di0:*
M!M2!B /de7/6SM?di0:)
M!M2!B /de7/6SM?di0:%
M!M2!B /de7/6SM?di0:,
C6N('(6T! /de7/6SM?di0:+
C6N('(6T! /de7/6SM?di0:5
C6N('(6T! /de7/6SM?di0:-
C6N('(6T! /de7/6SM?di0:1#
C6N('(6T! /de7/6SM?di0:11
C6N('(6T! /de7/6SM?di0:1*
1* ro<0 0e"eted$
S[HC
Iou can use the dis.s that are in dis. header status ?C!E1I1!T;@ or ?FOM;@
S[HC CB!6T! ('SU&B.UP (6T6?(&* N.BM6H B!(UN(6NCI
P/de7/6SM?di0:+PQ
P/de7/6SM?di0:5PQ
P/de7/6SM?di0:-PQ
P/de7/6SM?di0:1#PV
(i0:group reated$
S[HC 0e"et N!6(!B?ST6TUSQP6TN 8rom 7]a0m?di0: order by P6TNV
N!6(!B?ST6TU P6TN
------------
--------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
M!M2!B /de7/6SM?di0:*
M!M2!B /de7/6SM?di0:)
M!M2!B /de7/6SM?di0:%
M!M2!B /de7/6SM?di0:,
M!M2!B /de7/6SM?di0:+
!M2!B /de7/6SM?di0:-
M!M2!B /de7/6SM?di0:1#
C6N('(6T! /de7/6SM?di0:1*
1* ro<0 0e"eted$
S[HC
S[HC 0e"et 'NST?'(Q N6M!Q ST6T!Q .AAH'N!?('SUS 8rom g7]a0m?di0:groupV
'NST?'( N6M! ST6T! .AAH'N!?('SUS
---------- ------------------------------ ----------- -------------
* (6T6?(&1 M.UNT!( #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
118 o# $%$
* (6T6?(&* ('SM.UNT!( #
1 (6T6?(&1 M.UNT!( #
1 (6T6?(&* M.UNT!( #
, ro<0 0e"eted$
S[HC
5e need to mount 1!T!N1=7 on the second node 000
On node5 4
{node*:root}/ora"e/diag # 0u - a0m
{node*:a0m}/ora"e/a0m # 0J"p"u0 /no"og
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Mar *5 1):31:*) *##5
S[HC onnet /a0 0y0a0m
Conneted$
S[HC
S[HC a"ter di0:group (6T6?(&* mountV
(i0:group a"tered$
S[HC 0e"et 'NST?'(Q N6M!Q ST6T!Q .AAH'N!?('SUS 8rom g7]a0m?di0:groupV
'NST?'( N6M! ST6T! .AAH'N!?('SUS
---------- ------------------------------ ----------- -------------
1 (6T6?(&1 M.UNT!( #
1 (6T6?(&* M.UNT!( #
* (6T6?(&1 M.UNT!( #
* (6T6?(&* M.UNT!( #
, ro<0 0e"eted$
S[HC
1!T!N1=7 is no, mounyed on all nodes 000
Then ,ith asmcmd command/ ,e ,ill see 2
{node1:a0m}/ora"e/a0m # a0mmd -p
6SMCM( >D@ C "0
(6T6?(&1/
(6T6?(&*/
6SMCM( >D@ C
$) Create necessary su& directories #or the JSC71B data&ase ,ithin the choosen !SM dis.grou( 2
From node1 2
{node1:root}/ # su 1 asm
{node1:a0m}/home/a0m #
{node1:a0m}/home/a0m # e1port .B6CH!?S'(OD6SM1
{node1:a0m}/home/a0m # e1port .B6CH!?N.M!O/ora"e/a0m/11$1$#
{node1:a0m}/home/a0m # a0mmd Sp
{node1:a0m}/ora"e/a0m # a0mmd -p
6SMCM( >D@ C "0
(6T6?(&1/
6SMCM( >D@ C m:dir ;
11g!C/!SM/!I"
oraclibm@fr.ibm.com
11% o# $%$
B) Ma.e a init^SI1V)ora in your YO!C:;NDOM;/d&s directory)
Clu%ter-
Wide
Parameter
% for
Databa%e `
#$C5DB `4
##############################################################################
# Copyright FG 1--1Q *##1Q *##* by .ra"e Corporation
##############################################################################
###########################################
# 6rhi7e
###########################################
"og?arhi7e?de0t?1OPH.C6T'.NOD(6T6?(&*/P
"og?arhi7e?8ormatO9t?90?9r$db8
###########################################
# Cahe and '/.
###########################################
db?b"o:?0iLeO51-*
###########################################
# C"u0ter (ataba0e
###########################################
"u0ter?databa0e?in0tane0O*
#remote?"i0tenerOH'ST!N!BS?JSC*(2
###########################################
# Cur0or0 and Hibrary Cahe
###########################################
open?ur0or0O)##
###########################################
# (ataba0e 'denti8iation
###########################################
db?domainOZZ
db?nameOJSC*(2
###########################################
# Ai"e Con8iguration
###########################################
db?reate?8i"e?de0tOD(6T6?(&*
db?reo7ery?8i"e?de0tOD(6T6?(&*
db?reo7ery?8i"e?de0t?0iLeO3*-3-,+*-,
###########################################
# Mi0e""aneou0
###########################################
a0m?pre8erred?read?8ai"ure?group0OZZ
ompatib"eO11$1$#$#$#
diagno0ti?de0tO/ora"e
memory?targetO*,*133###
###########################################
# Proe00e0 and Se00ion0
###########################################
proe00e0O1%#
###########################################
# Seurity and 6uditing
###########################################
audit?8i"e?de0tO/ora"e/admin/JSC*(2/adump
audit?trai"Odb
remote?"ogin?pa00<ord8i"eOe1"u0i7e
11g!C/!SM/!I"
oraclibm@fr.ibm.com
176 o# $%$
###########################################
# Shared Ser7er
###########################################
di0pather0OZFPB.T.C.HOTCPG FS!B/'C!OJSC*(2K(2GZ
###########################################
# C"u0ter (ataba0e
###########################################
db?uniJue?nameOJSC*(2
ontro"?8i"e0O/ora"e/admin/JSC1(2/0ript0/tempContro"$t"
JSC*(21$in0tane?numberO1
JSC*(2*$in0tane?numberO*
JSC*(2*$threadO*
JSC*(21$threadO1
JSC*(21$undo?tab"e0paeOUN(.T2S1
JSC*(2*$undo?tab"e0paeOUN(.T2S*

C) un the #ollo,ing sPl(lus command to connect to the data&ase2

From node1 2
{node1:root}/ # su 1 rdbms
{node1:rdbm0}/home/rdbm0 #
{node1:rdbm0}/home/rdbm0 # e1port .B6CH!?S'(OJSC*(21
{node1:rdbm0}/home/rdbm0 # e1port .B6CH!?N.M!O/ora"e/rdbm0/11$1$#
{node1:rdbm0}/home/rdbm0 # 0J"p"u0 /no"og
connect ? as sysdba
G) Startu( u( the data&ase in EOMO4ET mode2
From node1 2
S[HC 0tartup nomount p8i"eOZ/ora"e/admin/JSC*(2/0ript0/init$oraZV
K) Create the 1ata&ase *!SM dis.grou(*s+ must &e created and mounted on all nodes+ 2
From node1 2
S[HC CB!6T! (6T626S! ZJSC*(2Z
M6K'NST6NC!S )*
M6KH.&N'ST.BI 1
M6KH.&A'H!S 1-*
M6KH.&M!M2!BS )
M6K(6T6A'H!S 1#*3
(6T6A'H! S'X! )##M 6UT.!KT!N( .N N!KT 1#*3#U M6KS'X! UNH'M'T!(
!KT!NT M6N6&!M!NT H.C6H
SIS6UK (6T6A'H! S'X! 1*#M 6UT.!KT!N( .N N!KT 1#*3#U M6KS'X! UNH'M'T!(
SM6HHA'H! (!A6UHT T!MP.B6BI T62H!SP6C! T!MP T!MPA'H! S'X! *#M 6UT.!KT!N( .N N!KT ,3#U
M6KS'X! UNH'M'T!(
SM6HHA'H! UN(. T62H!SP6C! ZUN(.T2S1Z (6T6A'H! S'X! *##M 6UT.!KT!N( .N N!KT %1*#U
M6KS'X! UNH'M'T!(
CN6B6CT!B S!T 4!5'S.55%-P1%
N6T'.N6H CN6B6CT!B S!T 6H1,UTA1,
H.&A'H! &B.UP 1 S'X! %1*##UQ &B.UP * S'X! %1*##UV
11g!C/!SM/!I"
oraclibm@fr.ibm.com
171 o# $%$
8) Create a Tem(orary Ta&les(ace2
From node1 2
$HL^ CREAGE GE"PORARM GABLE$PACE `GE"P` GE"P1!LE
R=DAGA6D25P $![E AC" REB$E
%) Create a 7nd 4ndo Ta&les(ace2
From node1 2
$HL^ CREAGE B>DO GABLE$PACE `B>DOGB$5` DAGA1!LE
R=DAGA6D25P $![E 5CC" REB$E
>E+G @*5CX "A+$![E B>L!"!GEDb
16) un the necessary scri(ts to &uild >ie,s/ synonyms/ etc)2

From node1 2
G&e 0rimary %cri0t% t&at you mu%t run are4
@/oracle/rd&ms/11)1)6/rd&ms/admin//catalog)sPl
-- create% t&e view% of data dictionary table% and t&e dynamic 0erformance view%
@/oracle/rd&ms/11)1)6/rd&ms/admin/cat(roc)sPl
-- e%tabli%&e% t&e u%a(e of PL$HL functionality and create% many of t&e PL$HL Oracle
%u00lied 0ac,a(e%
@/oracle/rd&ms/11)1)6/rd&ms/admin/catclust)sPl
-- create t&e clu%ter view%
un any other necessary catalog that you rePuire M
11) ;dit init^SI1V)ora and set a((ro(riate >alues #or the 7nd instance on the 7nd Eode2
in%tance6name9#$C5DB5
in%tance6number95
local6li%tener9L!$GE>ER6#$C5DB5
t&read95
undo6table%0ace9B>DOGB$5

17) From the #irst instance/ run the #ollo,ing command2

From node1 2
S[HC onnet SIS/ora"e a0 SIS(26
S[HC 0hutdo<n immediateV
S[HC 0tartup mount p8i"eOZ/ora"e/admin/JSC1(2/0ript0/init$oraZV
S[HC a"ter databa0e arhi7e"ogV
S[HC a"ter databa0e openV
S[HC 0e"et group# 8rom 7]"og <here group# O)V
S[HC 0e"et group# 8rom 7]"og <here group# O3V
S[HC 0e"et group# 8rom 7]"og <here group# O,V
S[HC 6HT!B (6T626S! 6(( H.&A'H! TNB!6( * &B.UP ) S'X! %1*##UQ
&B.UP 3 S'X! %1*##UQ
&B.UP , S'X! %1*##UV
S[HC 6HT!B (6T626S! !N62H! PU2H'C TNB!6( *V
S[HC reate 0p8i"eOPD(6T6?(&*/JSC*(2/0p8i"eJSC*(2$oraP AB.M
p8i"eOP/ora"e/admin/JSC*(2/0ript0/init$oraPV
S[HC 0hutdo<n immediateV
From node1 2
11g!C/!SM/!I"
oraclibm@fr.ibm.com
177 o# $%$
{node1:rdbm0}/home/rdbm0 # eho SPA'H!OPD(6T6?(&*/JSC*(2/0p8i"eJSC*(2$oraP C
/ora"e/rdbm0/11$1$#/db0/initJSC*(21$ora
From node7 2
{node*:rdbm0}/home/rdbm0 # eho SPA'H!OPD(6T6?(&*/JSC*(2/0p8i"eJSC*(2$oraP C
/ora"e/rdbm0/11$1$#/db0/initJSC*(2*$ora
1$) Create the oracle instance (ass,ord #iles
From node1 2
{node1:rdbm0}/ora"e/rdbm0/11$1$#/bin # orap<d
8i"eO/ora"e/rdbm0/11$1$#/db0/orap<JSC*(21 pa00<ordOora"e 8oreOy
From node7 2
{node*:rdbm0}/ora"e/rdbm0/11$1$#/bin # orap<d
8i"eO/ora"e/rdbm0/11$1$#/db0/orap<JSC*(2* pa00<ordOora"e 8oreOy
1B) Start the second Instance) *!ssuming that your cluster con#iguration is u( and running+
1C) Con#igure listener)ora / sPlnet)ora / tnsnames)ora/
B%e netca andor netmgr to c&ec, t&e confi(uration of t&e li%tener and confi(ure Oracle >et %ervice% -by
default t&e >et %ervice may be e)ual to t&e (lobal databa%e name -%ee in%tance 0arameter ser>iceNnames
..
1G) Con#igure Oracle ;nter(rise Manager *d&console or grid control agent+
G&en %tart t&e 2rid a(ent 4
Uemctl %tart a(ent
1K) Chec. /etc/orata&
G&e file %&ould contain a reference to t&e databa%e name' not to t&e in%tance name.
G&e la%t field %&ould alway% be 3>T on a RAC environment to avoid 5 in%tance% of t&e %ame name to be
%tarted.
{node1:rdbm0}/ora"e/rdbm0 # at /et/oratab
;;
D6SM1:/ora"e/a0m/11$1$#:N
JSC1(2:/ora"e/rdbm0/11$1$#:N
JSC*(2:/ora"e/rdbm0/11$1$#:N
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17$ o# $%$
18) egister the data&ase resources ,ith cluster sr>ctl command 2
From node1 2
{node1:rdbm0}/home/rdbm0 # 0r7t" add databa0e Sd JSC*(2 So /ora"e/rdbm0/11$1$#
{node1:rdbm0}/home/rdbm0 # 0r7t" add in0tane Sd JSC*(2 Si JSC*(21 Sn node1
{node1:rdbm0}/home/rdbm0 # 0r7t" add in0tane Sd JSC*(2 Si JSC*(2* Sn node*
{node1:rdbm0}/home/rdbm0 #
{node1:rdbm0}/ora"e/rdbm0 # crsstat ^SC2N)
----------- ------ -----
ora$JSC*(2$JSC*(21$in0t .NH'N! .NH'N! on node1
ora$JSC*(2$JSC*(2*$in0t .NH'N! .NH'N! on node*
ora$JSC*(2$db .NH'N! .NH'N! on node1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17B o# $%$
1C)$ Post O(erations
15.3.1 Update the RDBMS unix user .profile
To &e done on each node #or rd&ms *in our case+ or oracle uni' user)
vi YDOM;/)(ro#ile file in rdbm%P% &ome directory. !dd the entries in &old &lue color
PAGF9u%rbin4etc4u%r%bin4u%rucb4UFO"Ebin4u%rbin+**4%bin4.
e<0ort PAGF
if c -% Ù"A!L` d ; G&i% i% at $&ell %tartu0. !n normal
t&en ec&o Ù"A!L"$2` ; o0eration' t&e $&ell c&ec,%
fi ; 0eriodically.
E>N9UFO"E.,%&rc
e<0ort E>N
;G&e followin( line i% added by Licen%e B%e "ana(ement in%tallation
e<0ort PAGF9UPAGF4u%ro0tiforl%o%ai<bin
e<0ort PAGF9UPAGF4u%rKava*Abin
e<0ort "A>PAGF9U"A>PAGF4u%rlocalman
e<0ort ORACLE6BA$E9oracle
e<0ort A!+GFREAD6$COPE9$
e<0ort GE"P9tm0
e<0ort G"P9tm0
e<0ort G"PD!R9tm0
uma%, C55
e<0ort CR$6FO"E9cr%**.*.C
e<0ort ORA6CR$6FO"E9UCR$6FO"E
e<0ort A$"6FO"E9UORACLE6BA$Ea%m**.*.C
e<0ort ORA6A$"6FO"E9UA$"6FO"E
e<0ort ORACLE6FO"E9UORACLE6BA$Erdbm%**.*.C
e<0ort LD6L!BRARM6PAGF9UORACLE6FO"Elib4UCR$6FO"Elib4UORACLE6FO"ElibJ54UCR$6FO"ElibJ5
e<0ort L!BPAGF9ULD6L!BRARM6PAGF
e<0ort PAGF9UORACLE6FO"Ebin4UCR$6FO"Ebin4UPAGF
e<0ort G>$6AD"!>9UA$"6FO"Enetwor,admin
e'(ort O!C:;NSI1WJSC11B1
if c -t C db t&en
%tty intr eC
fi
TD;E on node7 M
MM))
e'(ort O!C:;NSI1WJSC11B7
MM))
15.3.2 Administer and Check
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17C o# $%$
To start / sto( the data&ase/ sto((ing all instances 2
{node2:rdbm
0}/ora"e/rdbm0 # 0r7t" 0top databa0e Sd J(2C1(2
{node1:rdbm0}/ora"e/rdbm0 # r0?0tat -t \grep $db
ora$JSC1(2$db app"iation .AAH'N! .AAH'N!
{node1:rdbm0}/ora"e/rdbm0 # r0?0tat -t \grep $in0t
ora$$$$21$in0t app"iation .AAH'N! .AAH'N!
ora$$$$2*$in0t app"iation .AAH'N! .AAH'N!
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" 0tart databa0e Sd J(2C1(2
ora$JSC1(2$db app"iation .NH'N! .NH'N! node*
ora$$$$21$in0t app"iation .NH'N! .NH'N! node1
ora$$$$2*$in0t app"iation .NH'N! .NH'N! node*
To start / sto( selected data&ase instance 2
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" 0top in0tane Sd J(2C1(2 Si J(2C1(21
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" 0top in0tane Sd J(2C1(2 Si J(2C1(2*
{node1:rdbm0}/ora"e/rdbm0
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" 0tart in0tane Sd J(2C1(2 Si J(2C1(21
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" 0tart in0tane Sd J(2C1(2 Si J(2C1(2*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17G o# $%$
=etting (ro#ile #rom the ora)JSC11B)JSC11B1)inst
resource ,ithin the cluster,are
=etting (ro#ile #rom the
ora)JSC11B)JSC11B1)d& resource ,ithin the
cluster,are
{node1:rdbm0}/ora"e/rdbm0 # crs,stat -p
ora.^SC1N).^SC1N)1.inst
N6M!Oora$JSC1(2$JSC1(21$in0t
TIP!Oapp"iation
6CT'.N?SCB'PTO/ora"e/rdbm0/11$1$#/bin/rag<rap
6CT'/!?PH6C!M!NTO#
6UT.?ST6BTO1
CN!CU?'NT!B/6HO)##
(!SCB'PT'.NOCBS app"iation 8or 'n0tane
A6'H./!B?(!H6IO#
A6'HUB!?'NT!B/6HO#
A6'HUB!?TNB!SN.H(O#
.PT'.N6H?B!S.UBC!SO
PH6C!M!NTOre0trited
B![U'B!(?B!S.UBC!SOora$node1$6SM1$a0m
B!ST6BT?6TT!MPTSO%
SCB'PT?T'M!.UTO,##
ST6BT?T'M!.UTO-##
ST.P?T'M!.UTO)##
UPT'M!?TNB!SN.H(O+d
USB?.B6?6H!BT?N6M!O
USB?.B6?(!2U&O#
USB?.B6?AH6&SO
USB?.B6?'AO
USB?.B6?H6N&O
USB?.B6?N!TM6SUO
USB?.B6?.P!N?M.(!O
USB?.B6?.P'O8a"0e
USB?.B6?PA'H!O
USB?.B6?SB/O
USB?.B6?/'PO
{node1:rdbm0}/ora"e/rdbm0 # crs,stat -p
ora.^SC1N).db
N6M!Oora$JSC1(2$db
TIP!Oapp"iation
6CT'.N?SCB'PTO/r0/11$1$#/bin/rag<rap
6CT'/!?PH6C!M!NTO#
6UT.?ST6BTO1
CN!CU?'NT!B/6HO,##
(!SCB'PT'.NOCBS app"iation 8or the
(ataba0e
A6'H./!B?(!H6IO#
A6'HUB!?'NT!B/6HO,#
A6'HUB!?TNB!SN.H(O1
N.ST'N&?M!M2!BSO
.PT'.N6H?B!S.UBC!SO
PH6C!M!NTOba"aned
B![U'B!(?B!S.UBC!SO
B!ST6BT?6TT!MPTSO#
SCB'PT?T'M!.UTO,##
ST6BT?T'M!.UTO,##
ST.P?T'M!.UTO,##
UPT'M!?TNB!SN.H(O+d
USB?.B6?6H!BT?N6M!O
USB?.B6?(!2U&O#
USB?.B6?AH6&SO
USB?.B6?'AO
USB?.B6?H6N&O
USB?.B6?N!TM6SUO
USB?.B6?.P!N?M.(!O
USB?.B6?.P'O8a"0e
USB?.B6?PA'H!O
USB?.B6?SB/O
USB?.B6?/'PO
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17K o# $%$
Chec. JSC11B s(#ile or init#ile content/ 1or e<am0le from node* 4
{node1:rdbm0}/ora"e/rdbm0 # cat 11.1.&?dbs?init^SC1N)1.ora
SPA'H!OPD(6T6?(&1/JSC1(2/0p8i"eJSC1(2$oraP
{node1:rdbm0}/ora"e/rdbm0 # e9port JHAC(/,SIN0^SC1N)1
{node1:rdbm0}/ora"e/rdbm0 # s%lplus ?nolo+
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on 4ed Aeb *# 1-:)):3+ *##5
S[HC connect ?as sysdba
Conneted$
S[HC create pfile0K?oracle?rdbms?^SC1N),pfile.oraK from
spfile0KUNA;A,N71?^SC1N)?spfile^SC1N).oraK\
Ai"e reated$
S[HC e9it
(i0onneted 8rom .ra"e (ataba0e 11g !nterpri0e !dition Be"ea0e 11$1$#$,$# - ,3bit
Prodution
4ith the PartitioningQ Bea" 6pp"iation C"u0ter0 and Bea" 6pp"iation Te0ting option0
{node1:rdbm0}/ora"e/rdbm0 # ls
11$1$# JSC1(2 JSC1(2?p8i"e$ora
{node1:rdbm0}/ora"e/rdbm0 # cat ^SC1N),pfile.ora
JSC1(21$??db?ahe?0iLeO5)55,#5
JSC1(2*$??db?ahe?0iLeO1,+++*1,
JSC1(21$??Wa7a?poo"?0iLeO1*%5*-1*
JSC1(2*$??Wa7a?poo"?0iLeO31-3)#3
JSC1(21$??"arge?poo"?0iLeO31-3)#3
JSC1(2*$??"arge?poo"?0iLeO31-3)#3
JSC1(21$??ora"e?ba0eOP/ora"eP#.B6CH!?26S! 0et 8rom en7ironment
JSC1(2*$??ora"e?ba0eOP/ora"eP#.B6CH!?26S! 0et 8rom en7ironment
JSC1(21$??pga?aggregate?targetO5)55,#5#
JSC1(2*$??pga?aggregate?targetO5)55,#5#
JSC1(21$??0ga?targetO15#)%%#+*
JSC1(2*$??0ga?targetO15#)%%#+*
JSC1(21$??0hared?io?poo"?0iLeO#
JSC1(2*$??0hared?io?poo"?0iLeO#
JSC1(21$??0hared?poo"?0iLeO13,5##,3#
JSC1(2*$??0hared?poo"?0iLeO13,5##,3#
JSC1(21$??0tream0?poo"?0iLeO#
JSC1(2*$??0tream0?poo"?0iLeO#
=$audit?8i"e?de0tOP/ora"e/admin/JSC1(2/adumpP
=$audit?trai"OPdbP
=$"u0ter?databa0e?in0tane0O*
=$"u0ter?databa0eOtrue
=$ompatib"eOP11$1$#$#$#P
=$ontro"?8i"e0OPD(6T6?(&1/W01db/ontro"8i"e/urrent$*,1$,3+1-)+-%PQPD(6T6?(&1/W01db/ontro
"8i"e/urrent$*,#$,3+1
-)+-%P
=$db?b"o:?0iLeO51-*
=$db?reate?8i"e?de0tOPD(6T6?(&1P
=$db?domainOPP
=$db?nameOPJSC1(2P
=$db?reo7ery?8i"e?de0tOPD(6T6?(&1P
=$db?reo7ery?8i"e?de0t?0iLeO3*-3-,+*-,
=$diagno0ti?de0tOP/ora"eP
=$di0pather0OPFPB.T.C.HOTCPG FS!B/'C!OJSC1(2K(2GP
JSC1(21$in0tane?numberO1
JSC1(2*$in0tane?numberO*
=$"og?arhi7e?de0t?1OPH.C6T'.NOD(6T6?(&1/P
=$"og?arhi7e?8ormatOP9t?90?9r$db8P
=$memory?targetO*,*133###
=$open?ur0or0O)##
=$proe00e0O1%#
=$remote?"i0tenerOPH'ST!N!BS?JSC1(2P
=$remote?"ogin?pa00<ord8i"eOPe1"u0i7eP
JSC1(2*$threadO*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
178 o# $%$
JSC1(21$threadO1
JSC1(21$undo?tab"e0paeOPUN(.T2S1P
JSC1(2*$undo?tab"e0paeOPUN(.T2S*P
11g!C/!SM/!I"
oraclibm@fr.ibm.com
17% o# $%$

Chec.ing listener status on each node 2
{node1:rdbm0}/ora"e/rdbm0 # lsnrctl status
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on *#-A!2-*##5 *1:1%:#1
------------------------
Prodution
Start (ate 1--A!2-*##5 1):%-:*,
Uptime 1 day0 + hr$ 1% min$ ), 0e
Trae He7e" o88
SNMP .N
Ser7ie0 Summary$$$
Ser7ie ZJSC1(2Z ha0 * in0taneF0G$
'n0tane ZJSC1(21ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZJSC1(2*ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G 8or thi0 0er7ie$$$
Ser7ie ZJSC1(2K(2Z ha0 * in0taneF0G$
'n0tane ZJSC1(21ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G 8or thi0 0er7ie$$$
Ser7ie ZJSC1(2?KPTZ ha0 * in0taneF0G$
Chec. also 2
{node1:rdbm0}/ora"e/rdbm0 # lsnrctl service

11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$6 o# $%$
1C)B Setting 1ata&ase :ocal and remote listeners
!&out Oracle Cluster data&ase instances (arameters/ ,e-ll ha>e to loo. on these (arameters 2
!n%tance >ame Nalue
Z remote6li%tener L!$GE>ER$6#$C*DB
#$C*DB* local6li%tener L!$GE>ER6#$C*DB*
#$C*DB5 Local6li%tener L!$GE>ER6#$C*DB5
remote6li%tener from node*' and node5 "B$G BE GFE $A"E' and E>GR!E$ "B$G BE PRE$E>G in t&e
tn%name%.ora from eac& node.
local6li%tener from node*' and node5 are different%' and E>GR!E$ "B$G BE PRE$E>G in t&e
tn%name%.ora from eac& node.
local6li%tener from node*' and node5 are not t&e one% defined in t&e li%tener.ora file% from eac& node.
!t data&ase le>el/ you should see
From node1 2
{node1:rdbm0}/ora"e/rdbm0 # e9port JHAC(/,GJ*/0?oracle?rdbms?11.1.&
{node1:rdbm0}/ora"e/rdbm0 # 0J"p"u0 /no"og
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** 1%:1-:#3 *##5
S[HC onnet /a0 0y0dba
Conneted$
S[HC 0ho< parameter "i0tener
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
remote?"i0tener 0tring H'ST!N!BS?JSC1(2
S[HC
From node7 2
Conneted$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
S[HC
OE:I ?remoteNlistener@ (arameter is set/ ?localNlistener@ (arameter #or each instance is not mandatory as ,e-re using
(ort 1C71 *automatic registration+ #or the de#ault node listeners) But i# not using 1C71 (ort/ it is mandatory to set them
to ha>e a (ro(er load &alancing and #ailo>er o# user connections)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$1 o# $%$
3JHAC(/,GJ*/?network?admin?tnsnames.ora
!$ic$ will be in ?oracle?asm?11.1.&?network?admin
M!E1!TOI 000
On all nodes/ #or
JSC11B
1ata&ase/ you
should ha>e or
add the #ollo,ing
lines 2
(IS;/#/HS,^SC1N) 0
EANNH/SS,(IS; 0
F
(IS;/#/H,^SC1N)1 0
EANNH/SS,(IS; 0
F
(IS;/#/H,A^SC1N)2 0
EANNH/SS,(IS; 0
F
I# remoteNlistener/ and localNlistener are not set/ em(ty or not ,ell set/ you should issue the #ollo,ing
commands 2
From node1 2
Conneted$
S[HC A(;/H SBS;/* S/; local,listener0K(IS;/#/H,^SC1N)1K SCJ"/0)J;G SIN0K^SC1N)1K\
System Altered.
S[HC A(;/H SBS;/* S/; local,listener0K(IS;/#/H,^SC1N)2K SCJ"/0)J;G SIN0K^SC1N)2K\
System Altered.
S[HC
I# needed to set _remoteNlistener-/ command ,ill &e as #ollo, 2
S[HC A(;/H SBS;/* S/; remote,listener0K(IS;/#/HS,^SC1N)K SCJ"/0)J;G SIN0KPK\
System Altered.
From node1 2
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
"oa"?"i0tener 0tring H'ST!N!B?JSC1(21
S[HC
From node7 2
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
"oa"?"i0tener 0tring H'ST!N!B?JSC1(2*
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$7 o# $%$
1C)C Creating Oracle Ser>ices
Wit& **(' itP% not anymore 0o%%ible to create %ervice% t&ru DBCA tool' o0tion &a% been removed.
G&e 5 o0tion% are 4
t&ru %rvctl tool from Oracle Clu%terware
t&ru Oracle 2rid Control' or DB Con%ole
15.5.1 Creation thru srvctl command
E;3; Create a ser>ice ,ith same name as data&ase 000
Subject:
Doc ID:
1etails o# command to add oracle ser>ices thru sr>ctl 2
sr>ctl add ser>ice -d ^nameV -s ^ser>iceNnameV -r ^(re#erredNlistV \-a â>aila&leNlistV] \-P ^T!FN(olicyV]
\-u]
-d Databa%e name
-s $ervice name
-a for %ervice%' li%t of available in%tance%' t&i% li%t cannot include 0referred in%tance%
-P for %ervice%' GA1 0reconnect 0olicy ? >O>E' BA$!C' PRECO>>ECG
-r for %ervice%' li%t of 0referred in%tance%' t&i% li%t cannot include available in%tance%.
-u u0date% t&e 0referred or available li%t for t&e %ervice to %u00ort t&e %0ecified in%tance.
Only one in%tance may be %0ecified wit& t&e
-u %witc&. !n%tance% t&at already %u00ort t&e %ervice %&ould not be included.
;'am(les for a A RAC node% Clu%ter.
Wit& a clu%ter databa%e named ORA.
A in%tance% named ORA*' ORA5' ORAJ and ORAA.
!dd a ST1NB!TCD ser>ice to an e<i%tin( databa%e wit& 0referred in%tance% --r. and available in%tance% --a..
B%e ba%ic failover to t&e available in%tance%.
sr>ctl add ser>ice -d !C -s ST1NB!TCD -r O!1/O!7 -a O!$/O!B
!dd a ST1NB!TCD ser>ice to an e<i%tin( databa%e wit& 0referred in%tance% in li%t one and
available in%tance% in li%t two. B%e 0reconnect at t&e available in%tance%.
sr>ctl add ser>ice -d O!C:; -s ST1NB!TCD -r O!1/O!7 -a O!$/O!B -P P;COEE;CT
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$$ o# $%$
In our case/ ,e ,ant to 2
!dd an O:TP ser>ice to an e<i%tin( #$C*DB databa%e
wit& #$C*DB* and #$C*DB5 a% 0referred in%tance% --r.
B%in( ba%ic failover to t&e available in%tance%.
!s rd&ms user/ #rom one node 2
Hnode1ErdbmsI=oracle=rdbms J sr>ctl add ser>ice -d JSC11B -s O:TP -r JSC11B1/JSC11B7
!dd a B!TCD ser>ice to an e<i%tin( #$C*DB databa%e
wit& #$C*DB5 a% 0referred in%tance% --r.
and #$C*DB* a% available in%tance% --a..
Hnode1ErdbmsI=oracle=rdbms J sr>ctl add ser>ice -d JSC11B -s B!TCD -r JSC11B7 -a JSC11B1
!dd a 1ISCO ser>ice to an e<i%tin( #$C*DB databa%e
wit& #$C*DB5 a% available in%tance% --a.
and #$C*DB* a% 0referred in%tance% --r..
Hnode1ErdbmsI=oracle=rdbms J sr>ctl add ser>ice -d JSC11B -s 1ISCO -r JSC11B1 -a JSC11B7
Ouery the cluster
&e#ore creating any
ser>ice
------------------------------------------------------------
ora$JSC1(2$db app"iation .NH'N! .NH'N! node1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$B o# $%$
Create and manage O:TP ser>ice 2
:et see in details/ !s rd&ms user/ #rom one node M
To create the ser>ice O:TP
wit& #$C*DB* and #$C*DB5 a% 0referred in%tance% --r.
{node1:rdbm0}/ora"e/rdbm0 # srvctl add service -d ^SC1N) -s J(;" -r ^SC1N)1<^SC1N)2
{node1:rdbm0}/ora"e/rdbm0 # srvctl confi+ service -d ^SC1N) -s J(;"
.HTP PB!A: JSC1(21 JSC1(2* 6/6'H:
To Puery a#ter
O:TP ser>ice
creation
------------------------------------------------------------
ora$$$$(21$0r7 app"iation .AAH'N! .AAH'N!
ora$$$$(2*$0r7 app"iation .AAH'N! .AAH'N!
ora$$$$.HTP$0 app"iation .AAH'N! .AAH'N!
To Puery and
get #ull name
o# O:TP
ser>ice
resource
;ntries added
&y the creation
o# Oracle
ser>ices at
Oracle
Cluster,are
le>el
{node1:rdbm0}/ora"e/rdbm0 # crs,stat 2+rep J(;"
N6M!Oora$JSC1(2$.HTP$JSC1(21$0r7
N6M!Oora$JSC1(2$.HTP$JSC1(2*$0r7
N6M!Oora$JSC1(2$.HTP$0
{node1:rdbm0}/ora"e/rdbm0 # crsstat J(;"
----------- ------ -----
ora$JSC1(2$.HTP$JSC1(21$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$.HTP$JSC1(2*$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$.HTP$0 .AAH'N! .AAH'N!
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$C o# $%$
To start the ser>ice O:TP
{node1:rdbm0}/ora"e/rdbm0 # srvctl start service -d ^SC1N) -s J(;"
{node1:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s J(;"
Ser7ie .HTP i0 running on in0taneF0G JSC1(21Q JSC1(2*
To Puery a#ter O:TP
ser>ice startu(
------------------------------------------------------------
ora$$$$(21$0r7 app"iation .NH'N! .NH'N! node1
ora$$$$(2*$0r7 app"iation .NH'N! .NH'N! node*
ora$$$$.HTP$0 app"iation .NH'N! .NH'N! node1
;o stop J(;" service 4
{node1:rdbm0}/ora"e/rdbm0 # srvctl stop service -d ^SC1N) -s J(;"
To Puery a#ter
sto((ing O:TP
ser>ice
------------------------------------------------------------
ora$$$$(21$0r7 app"iation .AAH'N! .AAH'N!
ora$$$$.HTP$0 app"iation .AAH'N! .AAH'N!
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$G o# $%$
TD;E IO4 M4ST add the #ollo,ing lines in the tnsnames)ora #ile on all nodes
Sr>ctl tool does not
add it ,hen creating
the ser>ice)
:oo.ing at
descri(tion/ ,e can
see 2
!ll 3IP declared
:oad Balancing
con#igured
Failo>er
Con#igured #or
?S;SSIOE@ and
?S;:;CT@
.HTP O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O .HTPG
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
=ood method is to modi#y one tnsnames)ora on one node/ and remote co(y the
modi#ied tnsnames)ora #ile to all other nodes/ ma.ing sure that all node ha>e the
same content)
Test conne'ion using O:PT conne'ion string #rom node 7 M
{node2:rdbm0}/ora"e/rdbm0 # s%lplus Ksys?password_oltp as sysdbaK
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #5:35:)5 *##5
Conneted to:
.ra"e (ataba0e 11g !nterpri0e !dition Be"ea0e 11$1$#$,$# - ,3bit Prodution
S[HC select instance,name from v3instance\
'NST6NC!?N6M!
----------------
JSC1(21
S[HC
Test conne'ion using O:PT conne'ion string #rom node 1 M
;
S[HC select instance,name from v3instanceV
'NST6NC!?N6M!
----------------
JSC1(2*
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$K o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$8 o# $%$
:et sto( the O:TP ser>ice to see i# ,e can still connect thru SO:Plus M
-rom node2 4
{node2:rdbm0}/ora"e/rdbm0 # 0r7t" stop service -d ^SC1N) -s J(;"
{node*:rdbm0}/ora"e/rdbm0 # crsstat 2+rep J(;"
ora$JSC1(2$.HTP$JSC1(21$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$.HTP$JSC1(2*$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$.HTP$0 .AAH'N! .AAH'N!
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #-:1#:15 *##5
!BB.B:
.B6-1*%13: TNS:"i0tener doe0 not urrent"y :no< o8 0er7ie reJue0ted in onnet
de0riptor
!nter u0er-name:
!BB.B:
.B6-#1#1+: in7a"id u0ername/pa00<ordV "ogon denied
!nter u0er-name:
!BB.B:
SP*-#1%+: unab"e to C.NN!CT to .B6CH! a8ter ) attempt0Q e1iting S[H=P"u0
-rom node1 4
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #-:#-:)% *##5
!BB.B:
de0riptor
!nter u0er-name:
!BB.B:
!nter u0er-name:
!BB.B:
Conclusion 2 i# the O:TP ser>ice is sto((ed/ no ne, connections ,ill &e allo,ed on any data&ase
instances e>en so nodes listeners are still u( and running 000
estarting O:TP ser>ice ,ill ena&le the connections M
{node*:rdbm0}/ora"e/rdbm0 # 0r7t" 0tart 0er7ie -d JSC1(2 -0 .HTP
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1$% o# $%$
Create and manage B!TCD ser>ice 2
To create the ser>ice B!TCD
wit& #$C*DB5 a% 0referred in%tance% --r.
and #$C*DB* a% available in%tance% --a..
{node1:rdbm0}/ora"e/rdbm0 # srvctl add service -d ^SC1N) -s )A;CG -r ^SC1N)2 1a ^SC1N)1
{node1:rdbm0}/ora"e/rdbm0 # srvctl confi+ service -d ^SC1N) -s )A;CG
26TCN PB!A: JSC1(2* 6/6'H: JSC1(21
To Puery
a#ter B!TCD
ser>ice
creation
------------------------------------------------------------
ora$$$$6TCN$0 app"iation .AAH'N! .AAH'N!
ora$$$$(21$0r7 app"iation .NH'N! .NH'N! node1
ora$$$$(2*$0r7 app"iation .NH'N! .NH'N! node*
ora$$$$.HTP$0 app"iation .NH'N! .NH'N! node1
To Puery and get #ull name o# B!TCD ser>ice resource
;ntries
added &y the
creation o#
Oracle
ser>ices at
Oracle
Cluster,are
le>el
{node1:rdbm0}/ora"e/rdbm0 # crs,stat 2+rep )A;CG
N6M!Oora$JSC1(2$26TCN$JSC1(2*$0r7
N6M!Oora$JSC1(2$26TCN$0
{node1:rdbm0}/ora"e/rdbm0 # crsstat )A;CG
----------- ------ -----
ora$JSC1(2$26TCN$JSC1(2*$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$26TCN$0 .AAH'N! .AAH'N!
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B6 o# $%$
To start the ser>ice B!TCD
{node1:rdbm0}/ora"e/rdbm0 # srvctl start service -d ^SC1N) -s )A;CG
To Puery a#ter B!TCD ser>ice startu(
----------- ------ -----
ora$JSC1(2$26TCN$JSC1(2*$0r7 .NH'N! .NH'N! on node*
ora$JSC1(2$26TCN$0 .NH'N! .NH'N! on node*
{node1:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s )A;CG
Ser7ie 26TCN i0 running on in0taneF0G JSC1(2*
To sto( B!TCD ser>ice 2
{node1:rdbm0}/ora"e/rdbm0 # srvctl stop service -d ^SC1N) -s )A;CG
To Puery a#ter sto((ing B!TCD ser>ice
----------- ------ -----
the ser>ice)
:oo.ing at
descri(tion/ ,e can
see 2
!ll 3IP declared
:oad Balancing
con#igured
Failo>er
Con#igured #or
?S;SSIOE@ and
?S;:;CT@
)A;CG O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O 26TCNG
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
same content)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B1 o# $%$
B!TCD Ser>ice Testing M
{node1:rdbm0}/ora"e/rdbm0 # tnspin+ )A;CG
TNS Ping Uti"ity 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on *)-
A!2-*##5 *1:1%:13
Copyright FG 1--+Q *##+Q .ra"e$ 6"" right0 re0er7ed$
U0ed parameter 8i"e0:
U0ed TNSN6M!S adapter to re0o"7e the a"ia0
6ttempting to ontat F(!SCB'PT'.N O F6((B!SS O FPB.T.C.H O TCPGFN.ST O node1-
7ipGFP.BT O 1%*1GG F6((B!SS O FPB.T.C.H O TCPGFN.ST O node*-7ipGFP.BT O 1%*1GG
FH.6(?26H6NC! O ye0G FC.NN!CT?(6T6 O FS!B/!B O (!('C6T!(G FS!B/'C!?N6M! O 26TCNG
FA6'H./!B?M.(! O FTIP! O S!H!CTG FM!TN.( O 26S'CG FB!TB'!S O 15#G F(!H6I O %GGGG
.U F1# m0eG
Test conne'ion using B!TCD conne'ion string #rom node 7 M
B!TCD ser>ice M4ST B; started 000
{node2:rdbm0}/ora"e/rdbm0 # s%lplus Ksys?password_batc$ as sysdbaK
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #5:35:)5 *##5
Conneted to:
'NST6NC!?N6M!
----------------
JSC1(2* !e[re connected on instance ^SC1N)2.
S[HC
Test conne'ion using B!TCD conne'ion string #rom node 1 M
;
S[HC select instance,name from v3instanceV
'NST6NC!?N6M!
----------------
S[HC
!s you can see/ connections ,ill only goes to instance JSC11B7 *node7+/ as this is the (re#erred instance
#or the B!TCD ser>ice/ no connections ,ill goes to instance JSC11B1 *node1+ unless node7 #ails/ then
B!TCD ,ill use the a>aila&le instance as set in the con#iguration)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B7 o# $%$
:et sto( the B!TCD ser>ice to see i# ,e can still connect thru SO:Plus M
-rom node2 4
{node2:rdbm0}/ora"e/rdbm0 # srvctl stop service -d ^SC1N) -s )A;CG
{node*:rdbm0}/ora"e/rdbm0 # crsstat )A;CG
----------- ------ -----
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #-:1#:15 *##5
!BB.B:
de0riptor
!nter u0er-name:
!BB.B:
!nter u0er-name:
!BB.B:
-rom node1 4
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** #-:#-:)% *##5
!BB.B:
de0riptor
!nter u0er-name:
!BB.B:
!nter u0er-name:
!BB.B:
Conclusion 2 i# the B!TCD ser>ice is sto((ed/ no ne, connections ,ill &e allo,ed on any data&ase
instances e>en so nodes listeners are still u( and running 000
e-starting B!TCD ser>ice ,ill ena&le the connections M
{node*:rdbm0}/ora"e/rdbm0 # srvctl start service -d ^SC1N) -s )A;CG
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B$ o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BB o# $%$
:et crash node7 ,hich host the B!TCD ser>ice (re#erred instance called JSC11B7 to see ,hat ,ill
ha((ened to e'isting connection M
B!TCD ser>ice M4ST B; started 000
-rom node2 4
----------- ------ -----
:et connect to othe ser>ice #rom node1 M
{node1:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin # s%lplus Ksys?password_batc$ as
sysdbaK
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Ari Aeb ** 11:#3:3+ *##5
Conneted to:
'NST6NC!?N6M!
----------------
S[HC
-rom node2 4
{node*:root}/ # reboot
Bebooting $ $ $
-rom node1 Esame uni9 and S`("lus session as beforeF 4
S[HC /
'NST6NC!?N6M!
----------------
JSC1(21 !e[re still connected< but on instance ^SC1N)1
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BC o# $%$
-rom node1 Enew uni9 sessionF 4
{node1:rdbm0}/ora"e/rdbm0 # crsstat
----------- ------ -----
ora$JSC1(2$26TCN$JSC1(2*$0r7 .NH'N! .NH'N! on node1
ora$JSC1(2$26TCN$0 .NH'N! .NH'N! on node1
ora$JSC1(2$JSC1(2*$in0t .NH'N! .AAH'N!
ora$JSC1(2$.HTP$JSC1(21$0r7 .NH'N! .NH'N! on node1
ora$JSC1(2$.HTP$JSC1(2*$0r7 .NH'N! .AAH'N!
ora$JSC1(2$.HTP$0 .NH'N! .NH'N! on node1
ora$node*$6SM*$a0m .NH'N! .AAH'N!
ora$node*$H'ST!N!B?N.(!*$"0nr .NH'N! .AAH'N!
ora$node*$g0d .NH'N! .AAH'N!
ora$node*$on0 .NH'N! .AAH'N!
ora$node*$7ip .NH'N! .NH'N! on node1
{node1:rdbm0}/ora"e/rdbm0 # crsstat vip
----------- ------ -----
----------- ------ -----
{node1:rdbm0}/ora"e/rdbm0 # srvctl confi+ service -d ^SC1N) -s )A;CG
26TCN PB!A: JSC1(2* 6/6'H: JSC1(21
{node1:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s )A;CG
Ser7ie 26TCN i0 running on in0taneF0G JSC1(21
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BG o# $%$
5hen node7 is &ac. OE:IE;/ Oracle Cluster,are ,ill start/ 3IP #rom node7 #ailed on node1 ,ill get &ac.
on its home node *node7+/ !SM instance7/ 1ata&ase instance ,ill restart as others resources lin.ed to
node7)
{node*:rdbm0}/ora"e/rdbm0 # crsstat
----------- ------ -----
ora$JSC1(2$.HTP$JSC1(2*$0r7 .NH'N! .NH'N! on node*
B4T ser>ice status ,ill not change unless ,e decide to s,itch &ac. to JSC11B7 as (re#erred instance #or
B!TCD ser>ice
To relocate the ser>ice on its (re#erred instance JSC11B7 M
{node*:rdbm0}/ora"e/rdbm0 # srvctl relocate service -d ^SC1N) -s )A;CG -i ^SC1N)1 -t
^SC1N)2 -f
{node*:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s )A;CG
----------- ------ -----
{node*:rdbm0}/ora"e/rdbm0 # srvctl confi+ service -d ^SC1N) -s )A;CG
26TCN PB!A: JSC1(2* 6/6'H: JSC1(21
-rom node1 Esame uni9 and S`("lus session as beforeF 4
S[HC /
'NST6NC!?N6M!
----------------
JSC1(2*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1BK o# $%$
S[HC
5e s,itched &ac. to (re#erred instance
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B8 o# $%$
Create and manage 1ISCO ser>ice 2
To create the ser>ice 1ISCO
!dd an 1ISCO ser>ice to an e<i%tin( #$C*DB databa%e
wit& #$C*DB* a% 0referred in%tance% --r.
and #$C*DB5 a% available in%tance% --a..
{node1:rdbm0}/ora"e/rdbm0 # srvctl add service -d ^SC1N) -s NISCJ -r ^SC1N)1 1a ^SC1N)2
To Puery
a#ter
1ISCO
ser>ice
creation
----------- ------ -----
ora$JSC1(2$('SC.$JSC1(21$0r7 .AAH'N! .AAH'N!
ora$JSC1(2$('SC.$0 .AAH'N! .AAH'N!
{node1:rdbm0}/ora"e/rdbm0 # crsstat NISCJ
----------- ------ -----
{node1:rdbm0}/ora"e/rdbm0 # srvctl confi+ service -d ^SC1N) -s NISCJ
('SC. PB!A: JSC1(21 6/6'H: JSC1(2*
{node1:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s NISCJ
Ser7ie ('SC. i0 not running$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1B% o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C6 o# $%$
To start the ser>ice 1ISCO
{node1:rdbm0}/ora"e/rdbm0 # srvctl start service -d ^SC1N) -s NISCJ
To Puery a#ter 1ISCO ser>ice startu(
----------- ------ -----
ora$JSC1(2$('SC.$JSC1(21$0r7 .NH'N! .NH'N! on node1
ora$JSC1(2$('SC.$0 .NH'N! .NH'N! on node1
{node1:rdbm0}/ora"e/rdbm0 # srvctl status service -d ^SC1N) -s NISCJ
Ser7ie ('SC. i0 running on in0taneF0G JSC1(21
To sto( 1ISCO ser>ice 2
{node1:rdbm0}/ora"e/rdbm0 # srvctl stop service -d ^SC1N) -s NISCJ
To Puery a#ter sto((ing 1ISCO ser>ice
----------- ------ -----
the ser>ice)
:oo.ing at
descri(tion/ ,e can
see 2
!ll 3IP declared
:oad Balancing
con#igured
Failo>er
Con#igured #or
?S;SSIOE@ and
?S;:;CT@
NSICJ O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O ('SC.G
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
same content)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C1 o# $%$
5e no, ha>e $ ser>ices #or the JSC11B cluster data&ase 2
O:TP running on (re#erred data&ase instance JSC11B1 on node1 and JSC11B7 on node7)
B!TCD running on (re#erred data&ase instance JSC11B7 on node7)
1ISCO running on (re#erred data&ase instance JSC11B1 on node1)
----------- ------ -----
ora$JSC1(2$('SC.$0 .NH'N! .NH'N! on node1
{node1:rdbm0}/ora"e/rdbm0 # crsstat srv
----------- ------ -----
{node1:rdbm0}/ora"e/rdbm0 # for service in J(;" )A;CG NISCJ
8 do
8 srvctl confi+ service -d ^SC1N) -s 3service
8 srvctl status service -d ^SC1N) -s 3service
8 ec$o KK
8 done
.HTP PB!A: JSC1(21 JSC1(2* 6/6'H:
Ser7ie .HTP i0 running on in0taneF0G JSC1(21Q JSC1(2*
26TCN PB!A: JSC1(2* 6/6'H: JSC1(21
('SC. PB!A: JSC1(21 6/6'H: JSC1(2*
Ser7ie ('SC. i0 running on in0taneF0G JSC1(21
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C7 o# $%$
!t data&ase le>el/ you should chec. data&ase (arameter ?ser>iceNnames@/ ensuring that ser>ices created
are listed 2
From node1 2
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Sat Aeb *) *1:15:1* *##5
Conneted$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring .HTPQ JSC1(2Q ('SC.
S[HC
On node1/ ,e-ll see O:TP and 1ISCO as they are (re#ered on JSC11B1)
B4T not B!TCD as it-s only a>aila&le on JSC11B1)
JSC11B M4ST &e also declared in the ser>iceNnames list)
From node7 2
{node*:rdbm0}/ora"e/rdbm0 # s%lplus ?nolo+
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Sat Aeb *) *1:*#:3# *##5
Conneted$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring .HTPQ JSC1(2Q 26TCN
S[HC
On node7/ ,e-ll see O:TP and B!TCD as they are (re#ered on JSC11B7)
B4T not 1ISCO as it-s only a>aila&le on JSC11B7)
JSC11B M4ST &e also declared in the ser>iceNnames list)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C$ o# $%$
I# ser>iceNnames not set/ em(ty or not ,ell set/ you should issue the #ollo,ing commands 2
From node1 2
{node1:ora"e}/ora"e -C e9port JHAC(/,GJ*/0?oracle?rdbms?11.1.&
{node1:ora"e}/ora"e -C e9port JHAC(/,SIN0^SC1N)1
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Sat Aeb *) *1:15:1* *##5
Conneted$
S[HC A(;/H SBS;/* S/; service,names0K^SCN)K<KJ(;"K<KNISCJK SCJ"/0)J;G SIN0K^SC1N)1K\
System Altered.
S[HC A(;/H SBS;/* S/; service,names0K^SCN)K<KJ(;"K<K)A;CGK SCJ"/0)J;G SIN0K^SC1N)2K\
System Altered.
S[HC
Chec. that ser>ices are registered ,ith command 2 lsnrctl status M) On node1 ,ith listenerNnode1
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on *)-A!2-*##5 *1:3%:3)
------------------------
6"ia0 (IS;/#/H,#JN/1
/er0ion TNSHSNB 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution
Start (ate **-A!2-*##5 #+:%+:*)
Uptime 1 day0 1) hr$ 35 min$ *1 0e
Trae He7e" o88
SNMP .N
Services Summary...
Ser7ie Z26TCNZ ha0 1 in0taneF0G$
Ser7ie Z('SC.Z ha0 1 in0taneF0G$
Ser7ie Z.HTPZ ha0 * in0taneF0G$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CB o# $%$
'n0tane ZJSC1(2*ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G
8or thi0 0er7ie$$$
This is normal as B!TCD ser>ice is
Available on #$C*DB* in%tance' node*
Prefered on #$C*DB5 in%tance' node5
'n0tane ZJSC1(21ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G
8or thi0 0er7ie$$$
This is normal as 1ISCO ser>ice is
Prefered on #$C*DB* in%tance' node*
Available on #$C*DB5 in%tance' node5
'n0tane ZJSC1(21ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G
8or thi0 0er7ie$$$
'n0tane ZJSC1(2*ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G
8or thi0 0er7ie$$$
This is normal as O:TP ser>ice is
Prefered on #$C*DB* in%tance' node*
Prefered on #$C*DB5 in%tance' node5
Chec. that ser>ices are registered ,ith command 2 lsnrctl status M) On node7 ,ith listenerNnode7
HSNBCTH 8or '2M/6'K B'SC Sy0tem/,###: /er0ion 11$1$#$,$# - Prodution on *)-A!2-*##5 *1:%%:35
------------------------
6"ia0 (IS;/#/H,#JN/2
Prodution
Start (ate **-A!2-*##5 11:11:)*
Uptime 1 day0 1# hr$ 33 min$ 1+ 0e
Trae He7e" o88
SNMP .N
Hi0tener Hog Ai"e /ora"e/diag/tn0"0nr/node*/"i0tener?node*/a"ert/"og$1m"
F(!SCB'PT'.NOF6((B!SSOFPB.T.C.HOtpGFN.STO1#$)$*%$15*GFP.BTO1%*1GGG
F(!SCB'PT'.NOF6((B!SSOFPB.T.C.HOtpGFN.STO1#$)$*%$5*GFP.BTO1%*1GGG
Services Summary...
'n0tane ZD6SM1ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZD6SM*ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZD6SM1ZQ 0tatu0 B!6(IQ ha0 1 hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZD6SM*ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G 8or thi0 0er7ie$$$
'n0tane ZJSC1(2*ZQ 0tatu0 B!6(IQ ha0 * hand"erF0G 8or thi0 0er7ie$$$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CC o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CG o# $%$
:et chec. ,hat ,ill ha((end as data&ase le>el ,hen #ailling node7
Be#ore #ailure o# one node/ crsNstat t ,ill sho, 2
----------- ------ -----
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Sat Aeb *) **:13:%# *##5
Conneted$
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring .HTPQ JSC1(2Q ('SC.
S[HC
I# node7 #ails/ or re&oot #or any reason 2 5hat should ,e see on node1 R
!#ter #ailure 2 On node1/ !S B!TCD ,as (re#ered on JSC11B7/ and a>aila&le on JSC11B1/
TD;E a#ter node7 #ailure/ B!TCD ser>ice ,ill &e s,ithed to a>aila&le instance JSC11B1)
S[H=P"u0: Be"ea0e 11$1$#$,$# - Prodution on Sat Aeb *) **:13:%# *##5
Conneted$
S[HC 0ho< parameter 0er7ie
N6M! TIP! /6HU!
------------------------------------ ----------- ------------------------------
0er7ie?name0 0tring .HTPQ JSC1(2Q ('SC.Q )A;CG
S[HC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1CK o# $%$
!#ter #ailure o# node7/ ?crsNstat t@ ,ill sho, 2
3IP #rom node7 s,ith to node1)
OES/ =S1/ :istener/ !SM7 instance/ and JSC11B7 instance are s,itch to OFF:IE; state)
O:TP ser>ice is s,itch to OFF:IE; state on node7) But still OE:IE; state on node1
B!TCD ser>ice is s,itch to OE:IE; state #rom node7 to node1)
TD;E B!TCD ser>ice is still a>aila&le thru node1)
----------- ------ -----
ora$JSC1(2$JSC1(2*$in0t .NH'N! .AAH'N!
ora$JSC1(2$.HTP$JSC1(2*$0r7 .NH'N! .AAH'N!
ora$node*$6SM*$a0m .NH'N! .AAH'N!
ora$node*$H'ST!N!B?N.(!*$"0nr .NH'N! .AAH'N!
ora$node*$g0d .NH'N! .AAH'N!
ora$node*$on0 .NH'N! .AAH'N!
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C8 o# $%$
1C)G Transaction !((lication Failo>er
3JHAC(/,GJ*/?network?admin?listener.ora
:istener #rom node1 is listening on node1->i( *16)$)7C)181+ and on node1 *16)$)7C)81+)
:istener #rom node7 is listening on node7->i( *16)$)7C)187+ and on node7 *16)$)7C)87+)
4sers ,ill connect thru node1->i( or node7->i(
I# :oad Balancing is con#iguring on the client connection string/ users ,ill &e dis(atch on &oth
nodes de(ending o# the ,or.load)
1B!-s can connect thru node1 or node7 #or administration (ur(ose)
IPC necessary)
Chec.ing content o# listener)ora on node1 2
{node1:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin # cat listener.ora
# "i0tener$ora$node1 Net<or: Con8iguration Ai"e:
/ora"e/rdbm0/11$1$#/net<or:/admin/"i0tener$ora$node1
H'ST!N!B?N.(!1 O
F(!SCB'PT'.N?H'ST O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node1-vipGFP.BT O 1%*1GF'P O A'BSTGG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O 1#$)$*%$51GFP.BT O 1%*1GF'P O A'BSTGG
G
G
{node1:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin #
Chec.ing content o# listener)ora on node7 2
{node*:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin # cat listener.ora
# "i0tener$ora$node* Net<or: Con8iguration Ai"e:
/ora"e/rdbm0/11$1$#/net<or:/admin/"i0tener$ora$node*
H'ST!N!B?N.(!* O
F(!SCB'PT'.N?H'ST O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node2-vipGFP.BT O 1%*1GF'P O A'BSTGG
F6((B!SS O FPB.T.C.H O TCPGFN.ST O 1#$)$*%$5*GFP.BT O 1%*1GF'P O A'BSTGG
G
G
{node*:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1C% o# $%$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G6 o# $%$
Do, to Puery ?:IST;E;@ resources
{node*:rdbm0}/ora"e/rdbm0 # crsstat (IS;/#/H
----------- ------ -----
Do, to o&tain ?:IST;E;@ ressources con#ig
{node*:rdbm0}/ora"e/rdbm0 # srvctl confi+ listener -n node1
node1 H'ST!N!B?N.(!1
{node*:rdbm0}/ora"e/rdbm0 # srvctl confi+ listener -n node2
node* H'ST!N!B?N.(!*
'8 "i0tener i0 not the de8au"t oneQ or de8au"t one but not <ith de8au"t naming a0
^H'ST!N!B?N.(!N6M!_ TN!N ; 'magine "i0tener i0 named ^H'ST!N!B?JSC_
{node*:rdbm0}/ora"e/rdbm0 # srvctl confi+ listener -n node2 1l (IS;/#/H,^SC
node* H'ST!N!B?JSC
Do, to STOP ?:IST;E;@ ressources con#ig
{node*:rdbm0}/ora"e/rdbm0 # srvctl stop listener -n node1
{node*:rdbm0}/ora"e/rdbm0 # srvctl start listener -n node1 -l listener,^SC
Do, to ST!T ?:IST;E;@ ressources con#ig
{node*:rdbm0}/ora"e/rdbm0 # srvctl start listener -n node1
{node*:rdbm0}/ora"e/rdbm0 # srvctl start listener -n node1 -l listener,^SC
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G1 o# $%$
:et-s ha>e a loo. on the tnsnames)ora #ile #rom on node M
!:: clients connections M4ST use 3IP-s 000
3JHAC(/,GJ*/?network?admin?tnsnames.ora
?;MOT;N:IST;E;
@ #or !SM instances 2
=A$"*
=A$"5
-Added by netca.
{node1:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin # at tn0name0$ora
/ora"e/rdbm0/11$1$#/net<or:/admin/tn0name0$ora
(IS;/#/HS,UAS* O
F6((B!SS?H'ST O
G
?:OC!:N:IST;E;@
#or !SM instance 2
=A$"5
-Added manualy.
(IS;/#/H,UAS*2 O
F6((B!SS?H'ST O
G
?:OC!:N:IST;E;@
#or !SM instance 2
=A$"*
-Added manualy.
(IS;/#/H,UAS*1 O
F6((B!SS?H'ST O
G
Connection String #or
!SM instances ,ith 2
Load Balancin(
1ailover -$e%%ion
and $elect.
G&i% entry could be
u%ed for by t&e DBA to
connect remotely to
A$" in%tance%
-Added manualy.
AS* O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O D6SMG
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G7 o# $%$
?;MOT;N:IST;E;
@ #or 1ata&ase
instances 2
#$C*DB*
#$C*DB5
-Added by netcadbca.
(IS;/#/HS,^SC1N) O
F6((B!SS?H'ST O
G
?:OC!:N:IST;E;@
#or !SM instance 2
#$C*DB*
?:OC!:N:IST;E;@
#or !SM instance 2
#$C*DB5
-Added manualy.
(IS;/#/H,^SC1N)2 O
F6((B!SS?H'ST O
G
(IS;/#/H,^SC1N)1 O
F6((B!SS?H'ST O
G
This connection
string ,ill only allo,
to connect to
data&ase instance
JSC11B7
>O 1ailover
>O Load Balancin(
^SC1N)2 O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O JSC1(2G
F'NST6NC!?N6M! O JSC1(2*G
G
G
This connection
string ,ill only allo,
to connect to
data&ase instance
JSC11B1
>O 1ailover
>O Load Balancin(
^SC1N)1 O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
F'NST6NC!?N6M! O JSC1(21G
G
G
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G$ o# $%$
This connection
string ,ill allo, to
connect to
data&ase instances
JSC11B1 and
JSC11B7
>O 1ailover
>O Load
Balancin(
-Added by netca.
^SC1N) O
F(!SCB'PT'.N O
F6((B!SS O FPB.T.C.H O TCPGFN.ST O node*-7ipGFPT.BT O 1%*1GG
FC.NN!CT?(6T6 O
G
G
Wit&out line% a% 4
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
Load Balancin( will be ac&ieved' BBG >O 1ailover will be en%ured OOO
$o Entry for #$C*DB %&ould me modified a% below 4
This connection
string ,ill allo, to
connect to
data&ase instances
JSC11B1 and
JSC11B7
1ailover
-$e%%ion and
$elect.
Load Balancin(
-Added by netca.
^SC1N) O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GB o# $%$
Connection String
#or O:TP 1ata&ase
cluster Ser>ice
instances ,ith 2
Load Balancin(
1ailover -$e%%ion
and $elect.
3PreferredT'
3AvailableT and 3>ot
B%edT will be
mana(ed t&ru Oracle
Clu%terware
-Added manualy.
J(;" O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
Connection String
#or B!TCD 1ata&ase
cluster Ser>ice
instances ,ith 2
Load Balancin(
1ailover -$e%%ion
and $elect.
3PreferredT'
B%edT will be
mana(ed t&ru Oracle
Clu%terware
-Added manualy.
)A;CG O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O 26TCNG
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
Connection String
#or 1ISCO 1ata&ase
cluster Ser>ice
instances ,ith 2
Load Balancin(
1ailover -$e%%ion
and $elect.
3PreferredT'
B%edT will be
mana(ed t&ru Oracle
Clu%terware
-Added manualy.
NISCJ O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FS!B/'C!?N6M! O ('SC.G
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
{node1:rdbm0}/ora"e/rdbm0/11$1$#/net<or:/admin #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GC o# $%$
IMPOT!ET/ W&at %&ould be on t&e client %ide wit& t&e local tn%name%.ora file 8
For :oad Balancing and #ailo>er on all nodes ,ithout going thru Oracle cluster,are ser>ices' ! will &ave to
add t&e #$C*DB entrie% in my local client tn%name%.ora.
G&e name of t&e
connection %trin(
#$C*DB could be
re0laced by an ot&er
name' a% lon( a% t&e
de%cri0tion includin(
$ERN!CE6>A"E i%
not c&an(ed OOO
^SC1N) O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
For :oad Balancing and #ailo>er on all nodes going thru Oracle cluster,are ser>ices' ! will &ave to c&oice
add t&e OLGP entrie% in my local client tn%name%.ora.
G&e name of t&e
connection %trin(
OLGP could be
re0laced by an ot&er
name' a% lon( a% t&e
de%cri0tion includin(
$ERN!CE6>A"E i%
not c&an(ed OOO
J(;" O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
Aor e1amp"eQ hanging the name o8 the onnetion 0tring .HTP to T!ST <i"" "oo: "i:e
;/S; O
F(!SCB'PT'.N O
FC.NN!CT?(6T6 O
FA6'H./!B?M.(! O
FTIP! O S!H!CTG
FM!TN.( O 26S'CG
FB!TB'!S O 15#G
F(!H6I O %G
G
G
G
For instance 7 as (re#erred instance and #ailo>er to a>aila&le instance 1 thru Oracle cluster,are ser>ices' !
will &ave to c&oice add t&e BAGCF entrie% in my local client tn%name%.ora.
For instance 1 as (re#erred instance and #ailo>er to a>aila&le instance 7 thru Oracle cluster,are ser>ices' !
will &ave to c&oice add t&e D!$CO entrie% in my local client tn%name%.ora.
IMPOT!ET/ On t&e client %ide' ma,e %ure !P&o%tname re%iolution for N!PP% are %olved eit&er by u%in( D>$ or
&o%t% file' even %o you c&an(ed t&e N!P &otname by real !P. Ot&erwi%e you may end-u0 wit& intermittent connection
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GG o# $%$
0roblem%.
1C)K !&out 1BCOESO:;
15.7.1 Checking DB CONSOLE
Since 16g!C 7/ and ,ith 11g!C also/ d&console is only con#igured on #irst nore at data&ase creation)
5hen using 1BCOESO:;/ each data&ase ,ill ha>e it-s o,n d&console 000
:oo. at Metalin. Eote 2
$ubKect4 Do, to manage 1B Control 16)7 #or !C 1ata&ase ,ith emca Doc !D4 Eote2$%C1G7)1
$ubKect4 Trou&leshooting 1ata&ase Control Startu( Issues Doc !D4 Eote2CB%6K%)1
Do, to
start/sto( the
d&console
agent 2
W&en u%in( DBCO>$OLE' t&e oracle a(ent %tart and %to0 automatically wit& t&e dbcon%ole.
!f needed' itP% %till 0o%%ible to %tart%to0 t&e a(ent u%in( t&e emctl tool a% follow 4
{node1:rdbm0}/ora"e/rdbm0 # e1port .B6CH!?N.M!O/ora"e/rdbm0/11$1$#
{node1:rdbm0}/ora"e/rdbm0 # e1port .B6CH!?S'(OJSC1(21
{node1:rdbm0}/ora"e/rdbm0 # emt" 0tart agent
.B
{node1:rdbm0}/ora"e/rdbm0 # emt" 0top agent
Chec. status o#
d&console agent
#or node1
{node1:rdbm0}/ora"e/rdbm0 # emt" 0tatu0 agent
.ra"e !nterpri0e Manager 1#g (ataba0e Contro" Be"ea0e 1#$*$#$)$#
Copyright FG 1--,Q *##, .ra"e Corporation$ 6"" right0 re0er7ed$
---------------------------------------------------------------
6gent /er0ion : 1#$1$#$%$1
.MS /er0ion : 1#$1$#$%$#
Protoo" /er0ion : 1#$1$#$*$#
6gent Nome : /ora"e/rdbm0/11$1$#/node1?JSC1(21
6gent binarie0 : /ora"e/rdbm0/11$1$#
6gent Proe00 '( : 1%#+%,*
Parent Proe00 '( : 13)++5,
6gent UBH : http://node1:)-)5/emd/main
Started at : *##+-#3-*) 1#:33:)5
Started by u0er : ora"e
Ha0t Be"oad : *##+-#3-*) 1#:33:)5
Ha0t 0ue008u" up"oad : *##+-#3-*) 1+:)*:*)
Tota" Megabyte0 o8 KMH 8i"e0 up"oaded 0o 8ar : +$,3
Number o8 KMH 8i"e0 pending up"oad : #
SiLe o8 KMH 8i"e0 pending up"oadFM2G : #$##
67ai"ab"e di0: 0pae on up"oad 8i"e0y0tem : )+$+59
---------------------------------------------------------------
6gent i0 Bunning and Beady
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1GK o# $%$
Chec. status o# ;M agent
#or node1/ and node7 2
Commands to
start
1BCOESO:; 2
{node1:rdbm0}/ora"e/rdbm0 # emt" 0tart dbon0o"e
http://node1:11%5/em/on0o"e/about6pp"iation
Starting .ra"e !nterpri0e Manager 1#g (ataba0e Contro"
$$$$$$$$$$$$$$$$$ 0tarted$
------------------------------------------------------------------
Hog0 are generated in diretory
/ora"e/rdbm0/11$1$#/node1?JSC1(21/0y0man/"og
Commands to
sto(
1BCOESO:; 2
{node1:rdbm0}/ora"e/rdbm0 # emt" 0top dbon0o"e
Stopping .ra"e !nterpri0e Manager 1#g (ataba0e Contro" $$$
$$$ Stopped$
Chec. status o#
d&console
{node1:rdbm0}/ora"e/rdbm0 # emt" 0tatu0 dbon0o"e
.ra"e !nterpri0e Manager 1#g i0 running$
------------------------------------------------------------------
Hog0 are generated in diretory
/ora"e/rdbm0/11$1$#/node1?JSC1(21/0y0man/"og
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G8 o# $%$
4(date your local
host #ile ,ith entries
#or node1 and node7)
!E1
!ccess 1BCOESO:;
thru
htt(2//node1211C8/em
Connect ,ith oracle
data&ase ?sys@ user
as ?SIS1B!@
Just clic. on ?I agree@ to
acce(t the Oracle
1ata&ase 16g :icensing
In#ormation)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1G% o# $%$
?oo= at &etalin= :ote to disco4er dbconsole; or get hel to sol4e an0 issues 6
$ubKect4 Do, to con#igure d&console to dis(lay in#ormation #rom the !SM 1IST=O4PS Doc !D4 Eote2$7%C81)1
$ubKect4 ;m 1&console 16)1)6)" 1oes Eot 1isco>er ac 1ata&ase !s ! Cluster Doc !D4 Eote2$$BCBG)1
$ubKect4 1BConsole Sho,s ;>erything 1o,n Doc !D4 Eote2$$78GC)1
$ubKect4 Do, To Con#ig 1&console *16)1)' or 16)7+ ;MC! 5ith !nother Dostname Doc !D4 Eote2$$G61K)1
$ubKect4 Do, To Change The 1B Console :anguage To ;nglish Doc !D4 Eote2$K61K8)1
$ubKect4 1&console Fails !#ter ! Physical Mo>e O# Machine To Ee, 1omain/Dostnam Doc !D4 Eote2B61%B$)1
$ubKect4 Do, to Trou&leshoot Failed :ogin !ttem(ts to 1B Control Doc !D4 Eote2B6B876)1
$ubKect4 Do, to manage 1B Control 16)7 #or !C 1ata&ase ,ith emca Doc !D4 Eote2$%C1G7)1
$ubKect4 Do, To 1ro(/ Create !nd ecreate 1B Control In ! 16g 1ata&ase Doc !D4 Eote27K8166)1
$ubKect4 O>er>ie, O# The ;MC! Commands !>aila&le #or 1B Control 16)7 Installations Doc !D4 Eote2$$61$6)1
Subject: Doc ID:
Subject: Doc ID:
Subject: Doc ID:
Testing i#
d&console is
,or.ing or
not 000
For each !C node 2
!s oracle user on node 1 2
;'(ort 1ISP:!IWRRRRRR
;'(ort O!C:;NSI1WJSC11B1
Chec. i# 1BConsole is running 2 emctl status d&console
IF d&console not running
TD;E 1/ e'ecute 2 emctl start d&console
7/ access d&console using Internet &ro,ser 2
using sys as user/ connected as sysd&a/ ,ith its (ass,ord
IF d&console started and reacha&le ,ith htt(2//node1211C8/em
TD;E d&console is OT on Eode 1
;:S; See metalin. note #or 1BConsole trou&leshooting
000
;:S; 1/ access d&console using Internet &ro,ser 2
using sys as user/ connected as sysd&a/ ,ith its (ass,ord
IF d&console is reacha&le ,ith htt(2//node1211C8/em
TD;E d&console is OT on Eode 1
;:S; See metalin. note #or 1BConsole trou&leshooting
000
;E1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K6 o# $%$
Trou&leshooting
Ti(s
The ?d&console@ may not start a#ter doing an install through the Oracle ;nter(rise Manager *O;M+
and selecting a data&ase)

G&e %olution i% to4
Edit the file: ${ORACLE_HOME}/<hostname>_${ORACLE_SID}/sysman/config/emd.properties
Locate the entry where the EMD_URL is set.
This entry should have the format:
EMD_URL=http://<hostname>:%EM_SERVLET_PORT%/emd/main
If you see the string: %EM_SERVLET_PORT% in the entry, then replace the complete string with an
unused port number that is not defined in the "/etc/services" file. If this string is missing and no port
number is in its place, then insert an unused port number that is not defined in the "/etc/services" file in
between the "http://<hostname>:" and the "/emd/main" strings.
Use the command emctl start dbconsole to start the dbconsole after making this change.
For example:
EMD_URL=http://myhostname.us.oracle.com:5505/emd/main
15.7.2 Moving from dbconsole to Grid Control
Iou must either use 1BCOESO:;/ or =I1 COETO:/ not &oth at the same time 000
To mo>e #rom Locally "ana(ed -DBCon%ole. to Centrally "ana(ed -2rid Control. 4
Mou mu%t &ave an e<i%tin( 2R!D Control' or in%tall a new 2rid Control on a%e0arate LPAR' or %erver. 2rid
Control i% available on A!+@L' but could be in%talled on any %u00orted o0eratin( %y%tem.
Mou mu%t in%tall t&e Oracle 2rid A(ent on eac& RAC node A!+ LPAR' wit& %ame uni< u%er a% Oracle
Clu%terware owner' and in t&e %ame ora!nventory.
TD;E you must #ollo, this metalin. note 2
$ubKect4 Do, to change a 16)7)6)' 1ata&ase #rom :ocally Managed to Centrally Managed Doc !D4
Eote2B66BKG)1
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K1 o# $%$
1G !SM !13!EC;1 TOO:S
1G)1 #t( and htt( access
Subject: Doc ID:
Subject: Doc ID:
Subject: Doc ID:
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K7 o# $%$
1K SOM; 4S;F4:: COMM!E1S
Command to start/sto( the 1ata&ase/ and 1ata&ases Instances 2
From any node 2
For data&ase
Hnode1ErdbmsI=oracle=rdbms J sr>ctl start data&ase d !SM1B to %tart t&e Databa%e in%tance
Hnode1ErdbmsI=oracle=rdbms J sr>ctl sto( data&ase d !SM1B to %to0 t&e Databa%e in%tance
For instance 1
Hnode1ErdbmsI=oracle=rdbms J sr>ctl start instance d !SM1B i !SM1B1to %tart t&e DB.. in%tance
Hnode1ErdbmsI=oracle=rdbms J sr>ctl sto( instance d !SM1B i !SM1B1to %to0 t&e DB.. in%tance
For instance 7
Hnode2ErdbmsI=oracle=rdbms J sr>ctl start instance d !SM1B i !SM1B7to %tart t&e DB.. in%tance
Hnode2ErdbmsI=oracle=rdbms J sr>ctl sto( instance d !SM1B i !SM1B7to %to0 t&e DB.. in%tance
To access an !SM instance ,ith sPl(lus
From node1 2
Hnode1ErdbmsI=oracle=rdbms J set O!C:;NSI1WJ!SM1
Hnode1ErdbmsI=oracle=rdbms J sPl(lus /nolog
connect /as sysd&a
sho, sga
MMM
From node7 2
Hnode2ErdbmsI=oracle=rdbms J set O!C:;NSI1WJ!SM7
connect /as sysd&a
sho, sga
MMM
To acces a 1ata&ase instance stored in !SM ,ith sPl(lus
From node1 2
Hnode1ErdbmsI=oracle=rdbms J e'(ort O!C:;NSI1W!SM1B1
connect /as sysd&a
sho, sga
MM)
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K$ o# $%$
1K)1 Oracle C:uster egistry content Chec. and Bac.u(
Chec.
Oracle
Cluster
egistry
Integrity
!s oracle
user/
;'ecute
ocrchec.
{node1:r0}/r0 # ocrc$eck
Statu0 o8 .ra"e C"u0ter Begi0try i0 a0 8o""o<0 :
/er0ion : *
Tota" 0pae F:byte0G : )#,-+*
U0ed 0pae F:byte0G : ,%3#
67ai"ab"e 0pae F:byte0G : )##3)*
'( : 1-*5)1,1*#
(e7ie/Ai"e Name : /de7/or?di0:1
(e7ie/Ai"e Name : /de7/or?di0:*
C"u0ter regi0try integrity he: 0ueeded
{node1:r0}/r0 #
!S root
user 2
;'(ort
Oracle
Cluster
egistry
content
{node1:r0}/r0/11$1$#/bin # 0u
rootP0 Pa00<ord:
{node1:root}/r0/11$1$#/bin # ocrconfi+ -e9port ?oracle?ocr,e9port5.dmp -s
online
{node1:root}/r0/11$1$#/bin # ls -la ?oracle?P.dmp
-rw-r--r-- 1 root system 1&:52& ân 3& 1.43&
?oracle?ocr,e9port.dmp
{node1:r0}/r0/11$1$#/bin #
you must not edit/modi#y this e'(orted #ile
3ie, OC automatic (eriodic &ac.u( managed &y Oracle Cluster,are
{node1:r0}/r0/11$1$# # ocrconfi+ -s$owbackup
node1 *##5/#)/*% #-:*3:1- /r0/11$1$#/data/r0?"u0ter/ba:up##$or
node1 *##5/#)/*% #%:*3:15 /r0/11$1$#/data/r0?"u0ter/ba:up#1$or
node1 *##5/#)/*% #1:*3:15 /r0/11$1$#/data/r0?"u0ter/ba:up#*$or
node1 *##5/#)/*) 1):*3:1% /r0/11$1$#/data/r0?"u0ter/day$or
node* *##5/#)/13 #,:3%:33 /r0/11$1$#/data/r0?"u0ter/<ee:$or
node* *##5/#)/1, 1+:1):)- /r0/11$1$#/data/r0?"u0ter/ba:up?*##5#)1,?1+1))-$or
node1 *##5/#*/*3 #5:#-:*1 /r0/11$1$#/data/r0?"u0ter/ba:up?*##5#**3?#5#-*1$or
{node1:r0}/r0/11$1$# #

11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KB o# $%$
18 !PP;E1I" ! 2 O!C:; / IBM T;CDEIC!: 1OC4M;ETS
a Gec&nical Document% on Oracle Real A00lication Clu%ter 4
htt(2//,,,)oracle)com/technology/(roducts/data&ase/clustering/inde')html
aGec&nical Document% on Oracle Real A00lication Clu%ter 4
a>ote45I5CJ/.* ? "inimum $oftware Ner%ion% and Patc&e% Re)uired to $u00ort Oracle Product% on !B" $y%tem 0
a>ote4JC5IC/.* ? !B" 2eneral Parallel 1ile $y%tem -2P1$. and Oracle RAC on A!+ @L and !B" e$erver $y%tem 0
a>ote4JA*@CE.* ? Oracle Product% on Linu< on !B" POWER
a>ote4ACAAEA* ? $tatu% of Certification of Oracle Clu%terware wit& FAC"P @.J : @.A
a#$C Coo,boo,% -available at &tt04www.oraclerac%i(.or(. 4
aDi RAC Relea%e 5 A!+ FAC"P 2P1$
a*C( RAC Relea%e * A!+ 2P1$ A$" Concurrent Raw Device% on !B" $A> $tora(e
a*C( RAC Relea%e 5 A!+ 2P1$ A$" on !B" $A> $tora(e
a!B" Gec& document% -&tt04www.ibm.com. 4
a!m0lementin( *C(RAC wit& A$" on A!+@L
aOracle *C( RAC on A!+ wit& Nerita% $tora(e 1oundation for Oracle RAC
aOracle Di RAC on A!+ wit& NER!GA$ $1RAC
aOracleR% licen%in( in a multi-core environment
aOracle Arc&itecture and Gunin( on A!+
aDia(no%in( Oracle7 Databa%e Performance on A!+7 B%in( !B"7 >"O> and Oracle $tat%0ac, Re0ort%
aOb%ervation% B%in( $"G on an Oracle Di OLGP Wor,load
aOracle Di : *C( on !B" A!+@L4 Gi0% : Con%ideration%
a!m0act of Advanced Power NirtualiLation 1eature% on Performance C&aracteri%tic% of a #ava7 C== Oracle7
ba%ed A00lication
aPerformance !m0act of B0(radin( a Data Ware&ou%e A00lication from A!+ @.5 to A!+ @.J wit& $"G
a$imultaneou% "ulti-t&readin( -$"G. on !B" POWER@4 Performance im0rovement on commercial OLGP wor,load
u%in( Oracle databa%e ver%ion *C(
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KC o# $%$
1% !PP;E1I" B 2 O!C:; T;CDEIC!: EOT;S
G&i% a00endi< 0rovide% %ome u%eful note% comin( from Oracle %u00ort. G&e%e note% can be found in "etalin,.
1%)1 CS and 16g eal !((lication Clusters
3oc
'36
!oteE2'.11.1
%ubGec
t6
CRS and 11% Real Application
Clusters
T0e6 B9AA46I!
%tatus
6
(9BAIS548
!ontent T0e6 64:6=:3
56MA
!reation 3ate6 1'384C3
211
?ast Re4ision
3ate6
1'3MAR3
211'
"6H"JS/
-------
Thi0 doument i0 to pro7ide additiona" in8ormation on CBS FC"u0ter Beady Ser7ie0G
in 1#g Bea" 6pp"iation C"u0ter0$

SCJ"/ S A""(ICA;IJ#
-------------------
Thi0 doument i0 intended 8or B6C (ataba0e 6dmini0trator0 and .ra"e 0upport
enginneer0$
CHS and 1&+ H/A( A""(ICA;IJ# C(6S;/HS
-------------------------------------

CBS FC"u0ter Beady Ser7ie0G i0 a ne< 8eature 8or 1#g Bea" 6pp"iation C"u0ter0
that pro7ide0 a 0tandard "u0ter inter8ae on a"" p"at8orm0 and per8orm0
ne< high a7ai"abi"ity operation0 not a7ai"ab"e in pre7iou0 7er0ion0$
CHS Y/B -AC;S
-------------
Prior to in0ta""ing CBS and 1#g B6CQ there are 0ome :ey point0 to remember about
CBS and 1#g B6C:
- CBS i0 B![U'B!( to be in0ta""ed and running prior to in0ta""ing 1#g B6C$
- CBS an either run on top o8 the 7endor "u0ter<are F0uh a0 Sun C"u0terQ
NP Ser7ieguardQ '2M N6CMPQ TruC"u0terQ /erita0 C"u0terQ AuWit0u Prime"u0terQ
et$$$G or an run <ithout the 7endor "u0ter<are$ The 7endor "u0ter<are
<a0 reJuired in -i B6C but i0 optiona" in 1#g B6C$
- The CBS N.M! and .B6CH!?N.M! mu0t be in0ta""ed in ('AA!B!NT "oation0$
- Shared HoationF0G or de7ie0 8or the /oting Ai"e and .CB F.ra"e
Con8iguration Bepo0itoryG 8i"e mu0t be a7ai"ab"e PB'.B to in0ta""ing CBS$ The
7oting 8i"e 0hou"d be at "ea0t *#M2 and the .CB 8i"e 0hou"d be at "ea0t 1##M2$
- CBS and B6C reJuire that the 8o""o<ing net<or: inter8ae0 be on8igured prior
to in0ta""ing CBS or B6C:
- Pub"i 'nter8ae
- Pri7ate 'nter8ae
- /irtua" FPub"iG 'nter8ae
Aor more in8ormation on thi0Q 0ee Note *,353+$1$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KG o# $%$
- The root$0h 0ript at the end o8 the CBS in0ta""ation 0tart0 the CBS 0ta:$
'8 your CBS 0ta: doe0 not 0tartQ 0ee Note *3###1$1$
- .n"y one 0et o8 CBS daemon0 an be running per B6C node$
- .n Uni1Q the CBS 0ta: i0 run 8rom entrie0 in /et/inittab <ith Zre0pa<nZ$
- '8 there i0 a net<or: 0p"it Fnode0 "oo0e ommuniation <ith eah otherG$ .ne
or more node0 may reboot automatia""y to pre7ent data orruption$
- The 0upported method to 0tart CBS i0 booting the mahine$ M6NU6H ST6BTUP .A
TN! CBS ST6CU 'S N.T SUPP.BT!( UNT'H 1#$1$#$3 .B N'&N!B$
- The 0upported method to 0top i0 0hutdo<n the mahine or u0e Zinit$r0 0topZ$
- Ui""ing CBS daemon0 i0 not 0upported un"e00 you are remo7ing the CBS
in0ta""ation 7ia Note *)---5$1 beau0e 8"ag 8i"e0 an beome mi0mathed$
- Aor maintenaneQ go to 0ing"e u0er mode at the .S$
.ne the 0ta: i0 0tartedQ you 0hou"d be ab"e to 0ee a"" o8 the daemon proe00e0
<ith a p0 -e8 ommand:
>ra1@/u#1/home/betaC p0 -e8 \ grep r0
ora"e 1),) --- # 11:*):*1 E #:## /u#1/r0?home/bin/e7m"ogger$bin -o /u#1
ora"e --- 1 # 11:*1:)- E #:#1 /u#1/r0?home/bin/e7md$bin
root 1##) 1 # 11:*1:)- E #:#1 /u#1/r0?home/bin/r0d$bin
ora"e 1##* 1 # 11:*1:)- E #:#1 /u#1/r0?home/bin/o00d$bin
CHS NA/*J# -6#C;IJ#A(I;B
------------------------
Nere i0 a 0hort de0ription o8 eah o8 the CBS daemon proe00e0:
CHSN4
- !ngine 8or N6 operation
- Manage0 Papp"iation re0oure0P
- Start0Q 0top0Q and 8ai"0 Papp"iation re0oure0P o7er
- Spa<n0 0eparate Pation0P to 0tart/0top/he: app"iation re0oure0
- Maintain0 on8iguration pro8i"e0 in the .CB F.ra"e Con8iguration Bepo0itoryG
- Store0 urrent :no<n 0tate in the .CB$
- Bun0 a0 root
- '0 re0tarted automatia""y on 8ai"ure
JCSSN4
- .CSS( i0 part o8 B6C and Sing"e 'n0tane <ith 6SM
- Pro7ide0 ae00 to node member0hip
- Pro7ide0 group 0er7ie0
- Pro7ide0 ba0i "u0ter "o:ing
- 'ntegrate0 <ith e1i0ting 7endor "u0te<areQ <hen pre0ent
- Can a"0o run0 <ithout integration to 7endor "u0t<are
- Bun0 a0 .ra"e$
- Aai"ure e1it au0e0 mahine reboot$
--- Thi0 i0 a 8eature to pre7ent data orruption in e7ent o8 a 0p"it brain$
/I*N4
- &enerate0 e7ent0 <hen thing0 happen
- Spa<n0 a permanent hi"d e7m"ogger
- !7m"oggerQ on demandQ 0pa<n0 hi"dren
- San0 a""out diretory and in7o:e0 a""out0$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1KK o# $%$
- Bun0 a0 .ra"e$
- Be0tarted automatia""y on 8ai"ure
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K8 o# $%$
CHS (J7 NIH/C;JHI/S
-------------------
4hen troub"e0hooting CBS prob"em0Q it i0 important to re7ie< the diretorie0
under the CBS Nome$
].B6?CBS?N.M!/r0/"og - Thi0 diretory in"ude0 trae0 8or CBS re0oure0 that are
WoiningQ "ea7ingQ re0tartingQ and re"oating a0 identi8ied by CBS$
].B6?CBS?N.M!/r0/init - 6ny ore dump0 8or the r0d$bin daemon 0hou"d be <ritten
here$ Note 151*$1 ou"d be u0ed to debug the0e$
].B6?CBS?N.M!/00/"og - The 00 "og0 indiate a"" ation0 0uh a0
reon8iguration0Q mi00ed he:in0 Q onnet0Q and di0onnet0 8rom the "ient
CSS "i0tener $ 'n 0ome a0e0 the "ogger "og0 me00age0 <ith the ategory o8
Fauth$ritG 8or the reboot0 done by ora"e$ Thi0 ou"d be u0ed 8or he:ing the
e1at time <hen the reboot oured$
].B6?CBS?N.M!/00/init - Core dump0 8rom the o00d primari"y and the pid 8or the
00 daemon <ho0e death i0 treated a0 8ata" are "oated here$ '8 there are
abnorma" re0tart0 8or 00 then the ore 8i"e0 <i"" ha7e the 8ormat0 o8
ore$TpidC$ Note 151*$1 ou"d be u0ed to debug the0e$
].B6?CBS?N.M!/e7m/"og - Hog 8i"e0 8or the e7m and e7m"ogger daemon0$ Not u0ed
a0 o8ten 8or debugging a0 the CBS and CSS diretorie0$
].B6?CBS?N.M!/e7m/init - Pid and "o: 8i"e0 8or !/M$ Core 8i"e0 8or !/M 0hou"d
a"0o be <ritten here$ Note 151*$1 ou"d be u0ed to debug the0e$
].B6?CBS?N.M!/0r7m/"og - Hog 8i"e0 8or .CB$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1K% o# $%$
S;A;6S -JH CHS H/SJ6HC/S
------------------------
68ter in0ta""ing B6C and running the /'PC6 F/irtua" 'P Con8iguration 600i0tantG
"aunhed <ith the B6C root$0hQ you 0hou"d be ab"e to 0ee a"" o8 your CBS
re0oure0 <ith r0?0tat$ !1amp"e:
d ].B6?CBS?N.M!/bin
$/r0?0tat
N6M!Oora$ra1$g0d
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!

N6M!Oora$ra1$oem
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!
N6M!Oora$ra1$on0
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!
N6M!Oora$ra1$7ip
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!
N6M!Oora$ra*$g0d
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!

N6M!Oora$ra*$oem
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!

N6M!Oora$ra*$on0
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!
N6M!Oora$ra*$7ip
TIP!Oapp"iation
T6B&!TO.NH'N!
ST6T!O.NH'N!
11g!C/!SM/!I"
oraclibm@fr.ibm.com
186 o# $%$
;$ere is also a script available to view CHS resources in a format t$at is
easier to read. ûst create a s$ell script wit$4
--------------------------- )e+in S$ell Script ----------------------------
#Y/u0r/bin/:0h
#
# Samp"e 1#g CBS re0oure 0tatu0 Juery 0ript
#
# (e0ription:
# - Beturn0 8ormatted 7er0ion o8 r0?0tat -tQ in tabu"ar
# 8ormatQ <ith the omp"ete r0 name0 and 8i"tering :ey<ord0
# - The argumentQ ]BSC?U!IQ i0 optiona" and i8 pa00ed to the 0riptQ <i""
# "imit the output to N6 re0oure0 <ho0e name0 math ]BSC?U!I$
# BeJuirement0:
# - ].B6?CBS?N.M! 0hou"d be 0et in your en7ironment
BSC?U!IO]1
[ST6TO-u
64UO/u0r/1pg3/bin/a<: # i8 not a7ai"ab"e u0e /u0r/bin/a<:
# Tab"e header:eho ZZ
]64U c
P2!&'N {print8 Z9-3%0 9-1#0 9-150cnZQ ZN6 Be0oureZQ ZTargetZQ ZStateZV
print8 Z9-3%0 9-1#0 9-150cnZQ Z-----------ZQ Z------ZQ Z-----ZV}P
# Tab"e body:
].B6?CBS?N.M!/bin/r0?0tat ][ST6T \ ]64U c
P2!&'N { ASOZOZV 0tate O #V }
]1d/N6M!/ aa ]*d/P]BSC?U!IP/ {appname O ]*V 0tateO1}V
0tate OO # {ne1tV}
]1d/T6B&!T/ aa 0tate OO 1 {apptarget O ]*V 0tateO*V}
]1d/ST6T!/ aa 0tate OO * {app0tate O ]*V 0tateO)V}
0tate OO ) {print8 Z9-3%0 9-1#0 9-150cnZQ appnameQ apptargetQ app0tateV 0tateO#V}P
--------------------------- /nd S$ell Script ------------------------------
/9ample output4
>opb0o"1@/u#1/home/u0upportC $/r00tat
----------- ------ -----
ora$/1#SN$/1#SN1$in0t .NH'N! .NH'N! on opb0o"1
ora$/1#SN$/1#SN*$in0t .NH'N! .NH'N! on opb0o"*
ora$/1#SN$db .NH'N! .NH'N! on opb0o"*
ora$opb0o"1$6SM1$a0m .NH'N! .NH'N! on opb0o"1
ora$opb0o"1$H'ST!N!B?.PC2S.H1$"0nr .NH'N! .NH'N! on opb0o"1
ora$opb0o"1$g0d .NH'N! .NH'N! on opb0o"1
ora$opb0o"1$on0 .NH'N! .NH'N! on opb0o"1
ora$opb0o"1$7ip .NH'N! .NH'N! on opb0o"1
ora$opb0o"*$6SM*$a0m .NH'N! .NH'N! on opb0o"*
ora$opb0o"*$H'ST!N!B?.PC2S.H*$"0nr .NH'N! .NH'N! on opb0o"*
ora$opb0o"*$g0d .NH'N! .NH'N! on opb0o"*
ora$opb0o"*$on0 .NH'N! .NH'N! on opb0o"*
ora$opb0o"*$7ip .NH'N! .NH'N! on opb0o"*
11g!C/!SM/!I"
oraclibm@fr.ibm.com
181 o# $%$
CHS H/SJ6HC/ AN*I#IS;HA;IJ#
---------------------------
Iou an u0e 0r7t" to manage the0e re0oure0$ 2e"o< are 0ynta1 and e1amp"e0$
---------------------------------------------------------------------------
CHS H/SJ6HC/ S;A;6S
0r7t" 0tatu0 databa0e -d Tdataba0e-nameC >-8@ >-7@ >-S T"e7e"C@
0r7t" 0tatu0 in0tane -d Tdataba0e-nameC -i Tin0tane-nameC C>QTin0tane-name-"i0tC@
>-8@ >-7@ >-S T"e7e"C@
0r7t" 0tatu0 0er7ie -d Tdataba0e-nameC -0 T0er7ie-nameC>QT0er7ie-name-"i0tC@
>-8@ >-7@ >-S T"e7e"C@
0r7t" 0tatu0 nodeapp0 >-n Tnode-nameC@
0r7t" 0tatu0 a0m -n Tnode?nameC
/A*"(/S4
Statu0 o8 the databa0eQ a"" in0tane0 and a"" 0er7ie0$
0r7t" 0tatu0 databa0e -d .B6CH! -7
Statu0 o8 named in0tane0 <ith their urrent 0er7ie0$
0r7t" 0tatu0 in0tane -d .B6CH! -i B6C#1Q B6C#* -7
Statu0 o8 a named 0er7ie0$
0r7t" 0tatu0 0er7ie -d .B6CH! -0 !BP -7
Statu0 o8 a"" node0 0upporting databa0e app"iation0$
0r7t" 0tatu0 node
S;AH; CHS H/SJ6HC/S
0r7t" 0tart databa0e -d Tdataba0e-nameC >-o T 0tart-option0C@
>- Tonnet-0tringC \ -J@
0r7t" 0tart in0tane -d Tdataba0e-nameC -i Tin0tane-nameC
>QTin0tane-name-"i0tC@ >-o T0tart-option0C@ >- Tonnet-0tringC \ -J@
0r7t" 0tart 0er7ie -d Tdataba0e-nameC >-0 T0er7ie-nameC>QT0er7ie-name-"i0tC@@
>-i Tin0tane-nameC@ >-o T0tart-option0C@ >- Tonnet-0tringC \ -J@
0r7t" 0tart nodeapp0 -n Tnode-nameC
0r7t" 0tart a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0tart?option0C@
/A*"(/S4
Start the databa0e <ith a"" enab"ed in0tane0$
0r7t" 0tart databa0e -d .B6CH!
Start named in0tane0$
0r7t" 0tart in0tane -d .B6CH! -i B6C#)Q B6C#3
Start named 0er7ie0$ (ependent in0tane0 are 0tarted a0 needed$
0r7t" 0tart 0er7ie -d .B6CH! -0 CBM
Start a 0er7ie at the named in0tane$
0r7t" 0tart 0er7ie -d .B6CH! -0 CBM -i B6C#3
Start node app"iation0$
0r7t" 0tart nodeapp0 -n my"u0t-3
11g!C/!SM/!I"
oraclibm@fr.ibm.com
187 o# $%$
S;J" CHS H/SJ6HC/S
0r7t" 0top databa0e -d Tdataba0e-nameC >-o T0top-option0C@
>- Tonnet-0tringC \ -J@
0r7t" 0top in0tane -d Tdataba0e-nameC -i Tin0tane-nameC >QTin0tane-name-"i0tC@
>-o T0top-option0C@>- Tonnet-0tringC \ -J@
0r7t" 0top 0er7ie -d Tdataba0e-nameC >-0 T0er7ie-nameC>QT0er7ie-name-"i0tC@@
>-i Tin0tane-nameC@>- Tonnet-0tringC \ -J@ >-8@
0r7t" 0top nodeapp0 -n Tnode-nameC
0r7t" 0top a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0tart?option0C@
/A*"(/S4
Stop the databa0eQ a"" in0tane0 and a"" 0er7ie0$
0r7t" 0top databa0e -d .B6CH!
Stop named in0tane0Q 8ir0t re"oating a"" e1i0ting 0er7ie0$
0r7t" 0top in0tane -d .B6CH! -i B6C#)QB6C#3
Stop the 0er7ie$
0r7t" 0top 0er7ie -d .B6CH! -0 CBM
Stop the 0er7ie at the named in0tane0$
0r7t" 0top 0er7ie -d .B6CH! -0 CBM -i B6C#3
Stop node app"iation0$ Note that in0tane0 and 0er7ie0 a"0o 0top$
0r7t" 0top nodeapp0 -n my"u0t-3
ANN CHS H/SJ6HC/S
0r7t" add databa0e -d TnameC -o Tora"e?homeC >-m Tdomain?nameC@ >-p T0p8i"eC@
>-6 Tname\ipC/netma0:@ >-r {PB'M6BI \ PNIS'C6H?ST6N(2I \ H.&'C6H?ST6N(2I}@
>-0 T0tart?option0C@ >-n Tdb?nameC@
0r7t" add in0tane -d TnameC -i Tin0t?nameC -n Tnode?nameC
0r7t" add 0er7ie -d TnameC -0 T0er7ie?nameC -r Tpre8erred?"i0tC
>-a Ta7ai"ab"e?"i0tC@ >-P TT6A?po"iyC@ >-u@
0r7t" add nodeapp0 -n Tnode?nameC -o Tora"e?homeC
>-6 Tname\ipC/netma0:>/i81>\i8*\$$$@@@
0r7t" add a0m -n Tnode?nameC -i Ta0m?in0t?nameC -o Tora"e?homeC
J";IJ#S4
-6 7ip rangeQ nodeQ and databa0eQ addre00 0pei8iation$ The 8ormat o8
addre00 0tring i0:
>T"ogia" ho0t nameC@/T/'P addre00C/Tnet ma0:C>/Tho0t inter8ae1> \
ho0t inter8ae* \$$@C@ >Q@ >T"ogia" ho0t nameC@/T/'P addre00C/Tnet ma0:C
>/Tho0t inter8ae1> \ ho0t inter8ae* \$$@C@
-a 8or 0er7ie0Q "i0t o8 a7ai"ab"e in0tane0Q thi0 "i0t annot in"ude
pre8erred in0tane0
-m domain name <ith the 8ormat û0$mydomain$om_
-n node name that <i"" 0upport one or more in0tane0
-o ].B6CH!?N.M! to "oate .ra"e binarie0
-P 8or 0er7ie0Q T6A preonnet po"iy - N.N!Q PB!C.NN!CT
-r 8or 0er7ie0Q "i0t o8 pre8erred in0tane0Q thi0 "i0t annot in"ude
a7ai"ab"e in0tane0$
-0 0p8i"e name
-u update0 the pre8erred or a7ai"ab"e "i0t 8or the 0er7ie to 0upport the
0pei8ied in0tane$ .n"y one in0tane may be 0pei8ied <ith the -u
0<ith$ 'n0tane0 that a"ready 0upport the 0er7ie 0hou"d not be
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18$ o# $%$
in"uded$
/A*"(/S4
6dd a ne< node:
0r7t" add nodeapp0 -n my"u0t-1 -o ].B6CH!?N.M! S6
1)-$153$*#1$1/*%%$*%%$*%%$#/hme#
6dd a ne< databa0e$
0r7t" add databa0e -d .B6CH! -o ].B6CH!?N.M!
6dd named in0tane0 to an e1i0ting databa0e$
0r7t" add in0tane -d .B6CH! -i B6C#1 -n my"u0t-1
0r7t" add in0tane -d .B6CH! -i B6C#* -n my"u0t-*
0r7t" add in0tane -d .B6CH! -i B6C#) -n my"u0t-)
6dd a 0er7ie to an e1i0ting databa0e <ith pre8erred in0tane0 F-rG and
a7ai"ab"e in0tane0 F-aG$ U0e ba0i 8ai"o7er to the a7ai"ab"e in0tane0$
0r7t" add 0er7ie -d .B6CH! -0 ST(?26TCN -r B6C#1QB6C#* -a B6C#)QB6C#3
6dd a 0er7ie to an e1i0ting databa0e <ith pre8erred in0tane0 in "i0t one and
a7ai"ab"e in0tane0 in "i0t t<o$ U0e preonnet at the a7ai"ab"e in0tane0$
0r7t" add 0er7ie -d .B6CH! -0 ST(?26TCN -r B6C#1QB6C#* -a B6C#)QB6C#3 -P
PB!C.NN!CT
H/*JI/ CHS H/SJ6HC/S
0r7t" remo7e databa0e -d Tdataba0e-nameC
0r7t" remo7e in0tane -d Tdataba0e-nameC >-i Tin0tane-nameC@
0r7t" remo7e 0er7ie -d Tdataba0e-nameC -0 T0er7ie-nameC >-i Tin0tane-nameC@
0r7t" remo7e nodeapp0 -n Tnode-nameC
/A*"(/S4
Bemo7e the app"iation0 8or a databa0e$
0r7t" remo7e databa0e -d .B6CH!
Bemo7e the app"iation0 8or named in0tane0 o8 an e1i0ting databa0e$
0r7t" remo7e in0tane -d .B6CH! -i B6C#)
0r7t" remo7e in0tane -d .B6CH! -i B6C#3
Bemo7e the 0er7ie$
0r7t" remo7e 0er7ie -d .B6CH! -0 ST(?26TCN
Bemo7e the 0er7ie 8rom the in0tane0$
0r7t" remo7e 0er7ie -d .B6CH! -0 ST(?26TCN -i B6C#)QB6C#3
Bemo7e a"" node app"iation0 8rom a node$
0r7t" remo7e nodeapp0 -n my"u0t-3
*JNI-B CHS H/SJ6HC/S
0r7t" modi8y databa0e -d TnameC >-n Tdb?name@ >-o TohomeC@ >-m TdomainC@
>-p T0p8i"eC@ >-r {PB'M6BI \ PNIS'C6H?ST6N(2I \ H.&'C6H?ST6N(2I}@
>-0 T0tart?option0C@
0r7t" modi8y in0tane -d Tdataba0e-nameC -i Tin0tane-nameC -n Tnode-nameC
0r7t" modi8y in0tane -d TnameC -i Tin0t?nameC {-0 Ta0m?in0t?nameC \ -r}
0r7t" modi8y 0er7ie -d Tdataba0e-nameC -0 T0er7ie?nameC -i Tin0tane-nameC
-t Tin0tane-nameC >-8@
0r7t" modi8y 0er7ie -d Tdataba0e-nameC -0 T0er7ie?nameC -i Tin0tane-nameC
-r >-8@
0r7t" modi8y nodeapp0 -n Tnode-nameC >-6 Taddre00-de0riptionC @ >-1@
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18B o# $%$
J";IJ#S4
-i Tin0tane-nameC -t Tin0tane-nameC the in0tane name F-iG i0 rep"aed by the
in0tane name F-tG
-i Tin0tane-nameC -r the named in0tane i0 modi8ied to be a pre8erred in0tane
-6 addre00-"i0t 8or /'P app"iationQ at node "e7e"
-0 Ta0m?in0t?nameC add or remo7e 6SM dependeny
/A*"(/S4
Modi8y an in0tane to e1eute on another node$
0r7t" modi8y in0tane -d .B6CH! -n my"u0t-3
Modi8y a 0er7ie to e1eute on another node$
0r7t" modi8y 0er7ie -d .B6CH! -0 N.T?26TCN -i B6C#1 -t B6C#*
Modi8y an in0tane to be a pre8erred in0tane 8or a 0er7ie$
0r7t" modi8y 0er7ie -d .B6CH! -0 N.T?26TCN -i B6C#* Sr
H/(JCA;/ S/HIIC/S
0r7t" re"oate 0er7ie -d Tdataba0e-nameC -0 T0er7ie-nameC >-i Tin0tane-name C@-
tTin0tane-name C >-8@
/A*"(/S4
Be"oate a 0er7ie 8rom one in0tane to another
0r7t" re"oate 0er7ie -d .B6CH! -0 CBM -i B6C#3 -t B6C#1
/#A)(/ CHS H/SJ6HC/S E;$e resource may be up or down to use t$is functionF
0r7t" enab"e databa0e -d Tdataba0e-nameC
0r7t" enab"e in0tane -d Tdataba0e-nameC -i Tin0tane-nameC >QTin0tane-name-"i0tC@
0r7t" enab"e 0er7ie -d Tdataba0e-nameC -0 T0er7ie-nameC@ >Q T0er7ie-name-"i0tC@
>-i Tin0tane-nameC@
/A*"(/S4
!nab"e the databa0e$
0r7t" enab"e databa0e -d .B6CH!
!nab"e the named in0tane0$
0r7t" enab"e in0tane -d .B6CH! -i B6C#1Q B6C#*
!nab"e the 0er7ie$
0r7t" enab"e 0er7ie -d .B6CH! -0 !BPQCBM
!nab"e the 0er7ie at the named in0tane$
0r7t" enab"e 0er7ie -d .B6CH! -0 CBM -i B6C#)
NISA)(/ CHS H/SJ6HC/S E;$e resource must be down to use t$is functionG
0r7t" di0ab"e databa0e -d Tdataba0e-nameC
0r7t" di0ab"e in0tane -d Tdataba0e-nameC -i Tin0tane-nameC >QTin0tane-name-"i0tC@
0r7t" di0ab"e 0er7ie -d Tdataba0e-nameC -0 T0er7ie-nameC@ >QT0er7ie-name-"i0tC@
>-i Tin0tane-nameC@
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18C o# $%$
/A*"(/S4
(i0ab"e the databa0e g"oba""y$
0r7t" di0ab"e databa0e -d .B6CH!
(i0ab"e the named in0tane0$
0r7t" di0ab"e in0tane -d .B6CH! -i B6C#1Q B6C#*
(i0ab"e the 0er7ie g"oba""y$
0r7t" di0ab"e 0er7ie -d .B6CH! -0 !BPQCBM
(i0ab"e the 0er7ie at the named in0tane$
0r7t" di0ab"e 0er7ie -d .B6CH! -0 CBM -i B6C#)QB6C#3
-or more information on t$is see t$e Jracle1&+ Heal Application Clusters
Administrator[s 7uide - Appendi9 )
H/(A;/N NJC6*/#;S
.ra"e1#g Bea" 6pp"iation C"u0ter0 'n0ta""ation and Con8iguration
.ra"e1#g Bea" 6pp"iation C"u0ter0 6dmini0tratore0 &uide
1%)7 !&out !C )))
&!34+1 5 )ini$u$ so*tware versions an. patches re2uire. to Support Oracle Pro.ucts on +++
&3"63+1 5 Pre5,nstall chec7s *or 1!( RD8)S on A,9
!:"!+1 5 RAC; Fre2uently As7e. <uestions
1&36!&+1 5 Raw Devices an. Cluster Filesyste$s =ith Real Application Clusters
:3">!+1 5 1!( ,nstallation on Ai? >+3- Faile. with Chec7in( operatin( syste$ version $u+++
1%)$ !&out CS )))
43&:"+1 5 1!@; #ow to Stop the Cluster Rea.y Services (CRS)
:>&"1+1 5 #ow to veri*y i* CRS install is Aali.
4>"4:+1 5 1!( RAC; Trou'leshootin( CRS Re'oots
>:3!1+1 5 CRS an. 1!( Real Application Clusters
4&:3"+1 5 Repairin( or Restorin( an ,nconsistent OCR in RAC
:3&1:+1 5 Place$ent o* votin( an. OCR .is7 *iles in 1!(RAC
3:::&+1 5 1!( RAC; #ow to Clean Bp A*ter a Faile. CRS ,nstall
"33+1 5 CRS 1!( Dia(nostic Collection @ui.e
":":3+1 5 #ow to Restore a Cost Aotin( Dis7 in 1!(
3::&:+1 5 1!( RAC; Stoppin( Re'oot Coops =hen CRS Pro'le$s Occur
:&!"3+1 5 #O= TO R%)OA% CRS ABTO START ADD R%START FOR A RAC ,DSTADC%
:&!4:+1 5 #O= TO R%)OA% CRS ABTO START ADD R%START FOR A RAC ,DSTADC%
&6:6:+1 5 CRS #o$e ,s Only Partially Copie. to Re$ote Do.e
&>!64+1 5 #ow to recreate ODS-@SD-A,P .elete. *ro$ ocr 'y crsEunre(ister
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18G o# $%$
1%)B !&out 3IP )))
:4&>4+1 5 Con*i(urin( the ,8) A,9 >C Operatin( Syste$ *or the Oracle 1!( A,P
:6334+1 5 Chan(in( the chec7 interval *or the Oracle 1!( A,P
"4636+1 5 )o.i*yin( the A,P o* a Cluster Do.e
:&&:>+1 5 )o.i*yin( the .e*ault (ateway a..ress use. 'y the Oracle 1!( A,P
46&6"+1 5 #ow to Con*i(ure Airtual ,Ps *or 1!( RAC
1%)C !&out manual data&ase cration )))
6!!>+1 5 1!( )anual Data'ase Creation in Oracle (Sin(le ,nstance an. RAC)
1%)G !&out =rid Control )))
&6"!"+1 5 %nterprise )ana(er @ri. Control 1!+1+!+3+! Release Dotes
""6!+1 5 %) 1!@ @ri. Control Preinstall Steps *or A,9 >+
1%)K !&out T!F )))
"1:"+1 5 Trou'leshootin( TAF ,ssues in 1!( RAC
1%)8 !&out !dding/emo>ing Eode )))
4:3!+1 5 Re$ovin( a Do.e *ro$ a 1!( RAC Cluster
"!>1+1 5 A..in( a Do.e to a 1!( RAC Cluster
1%)% !&out !SM )))
636>+1 5 1!@ Dew Stora(e Features an. %nhance$ents
4&6&1+1 5 Re5creatin( AS) ,nstances an. Dis7(roups
&"""+1 5 S@A siFin( *or AS) instances an. .ata'ases that use AS)
"6"3&+1 5 Creatin( an AS)5ena'le. Data'ase
6:::+1 5 Dew Feature on AS) (Auto$atic Stora(e )ana(er)+
>1:+1 5 Steps To )i(rate Data'ase Fro$ Don5AS) to AS) An. Aice5Aersa
:336+1 5 #ow To )ove Archive Files *ro$ AS)
"!!44+1 5 )ana(e AS) instance5creatin( .is7(roup-a..in(G.roppin(GresiFin( .is7s+
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18K o# $%$
3!!6"+1 5 #ow To Delete Archive Co( Files Out O* 0As$H
4>433+1 5 AS) Technical 8est Practices http;GG$etalin7+oracle+co$G$etalin7Gpls2lG.ocsGAS)+p.*
For full article, download Automatic Storage Management (154K/pdf)
:6&4:+1 5 Oracle AS) an. )ulti5Pathin( Technolo(ies
1%)16 Metalin. note to use in case o# (ro&lem ,ith CS )))
5/JIDE.* - *C24 Fow to $to0 t&e Clu%ter Ready $ervice% -CR$.
5D@IE*.* - Fow to verify if CR$ in%tall i% Nalid
5/@E/D.* - *C( RAC4 Grouble%&ootin( CR$ Reboot%
5@DJC*.* - CR$ and *C( Real A00lication Clu%ter%
5/IDJE.* - Re0airin( or Re%torin( an !ncon%i%tent OCR in RAC
5DJI*D.* - Placement of votin( and OCR di%, file% in *C(RAC
5JDDDI.* - *C( RAC4 Fow to Clean B0 After a 1ailed CR$ !n%tall
5E5JJ5.* - CR$ *C( Dia(no%tic Collection 2uide
5JDDID.* - *C( RAC4 $to00in( Reboot Loo0% W&en CR$ Problem% Occur
5DICEJ.* - FOW GO RE"ONE CR$ ABGO $GARG A>D RE$GARG 1OR A RAC !>$GA>CE
5DIC/D.* - FOW GO RE"ONE CR$ ABGO $GARG A>D RE$GARG 1OR A RAC !>$GA>CE
5IADAD.* - CR$ Fome !% Only Partially Co0ied to Remote >ode
11g!C/!SM/!I"
oraclibm@fr.ibm.com
188 o# $%$
76 !PP;E1I" C 2 4S;F4:: COMM!E1S
Cr%ctl 4 Go admini%trate t&e clu%terware
Ocrconfi( 4 Go bac,u0' e<0ort' im0ort re0air ... OCR -Oracle Clu%ter Re(i%try. content%.
Ocrdum0 4 Go dum0 Oracle clu%ter re(i%try content
$rvctl 4 Go admini%trate Oracle clu%terware re%ource%
Cr%6%tat ?&el0 4 Go )uery %tate of Oracle clu%terware re%%ource%
Clufvy 4 Go 0re0o%t c&ec, in%tallation %ta(e%
11g!C/!SM/!I"
oraclibm@fr.ibm.com
18% o# $%$
crsctl 4 ;o administrate t$e clusterware.
{node1:r0}/r0 # r0t"
U0age: r0t" he: r0 - he:0 the 7iabi"ity o8 the .ra"e C"u0ter<are
r0t" he: 00d - he:0 the 7iabi"ity o8 C"u0ter SynhroniLation Ser7ie0
r0t" he: r0d - he:0 the 7iabi"ity o8 C"u0ter Beady Ser7ie0
r0t" he: e7md - he:0 the 7iabi"ity o8 !7ent Manager
r0t" he: "u0ter >-node TnodenameC@ - he:0 the 7iabi"ity o8 CSS aro00 node0
r0t" 0et 00 TparameterC T7a"ueC - 0et0 a parameter o7erride
r0t" get 00 TparameterC - 0et0 the 7a"ue o8 a C"u0ter SynhroniLation Ser7ie0 parameter
r0t" un0et 00 TparameterC - 0et0 the C"u0ter SynhroniLation Ser7ie0 parameter to it0 de8au"t
r0t" Juery 00 7otedi0: - "i0t0 the 7oting di0:0 u0ed by C"u0ter SynhroniLation Ser7ie0
r0t" add 00 7otedi0: TpathC - add0 a ne< 7oting di0:
r0t" de"ete 00 7otedi0: TpathC - remo7e0 a 7oting di0:
r0t" enab"e r0 - enab"e0 0tartup 8or a"" .ra"e C"u0ter<are daemon0
r0t" di0ab"e r0 - di0ab"e0 0tartup 8or a"" .ra"e C"u0ter<are daemon0
r0t" 0tart r0 >-<ait@ - 0tart0 a"" .ra"e C"u0ter<are daemon0
r0t" 0top r0 >-<ait@ - 0top0 a"" .ra"e C"u0ter<are daemon0$ Stop0 .ra"e C"u0ter<are managed re0oure0 in a0e o8 "u0ter$
r0t" 0tart re0oure0 - 0tart0 .ra"e C"u0ter<are managed re0oure0
r0t" 0top re0oure0 - 0top0 .ra"e C"u0ter<are managed re0oure0
r0t" debug 0tatedump 00 - dump0 0tate in8o 8or C"u0ter SynhroniLation Ser7ie0 obWet0
r0t" debug 0tatedump r0 - dump0 0tate in8o 8or C"u0ter Beady Ser7ie0 obWet0
r0t" debug 0tatedump e7m - dump0 0tate in8o 8or !7ent Manager obWet0
r0t" debug "og 00 >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or C"u0ter SynhroniLation Ser7ie0
r0t" debug "og r0 >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or C"u0ter Beady Ser7ie0
r0t" debug "og e7m >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or !7ent Manager
r0t" debug "og re0 >re0name:"e7e"@ $$$ - turn0 on debugging 8or !7ent Manager
r0t" debug trae 00 >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or C"u0ter SynhroniLation Ser7ie0
r0t" debug trae r0 >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or C"u0ter Beady Ser7ie0
r0t" debug trae e7m >modu"e:"e7e"@ {Qmodu"e:"e7e"} $$$ - turn0 on debugging 8or !7ent Manager
r0t" Juery r0 0o8t<are7er0ion >TnodenameC@ - "i0t0 the 7er0ion o8 .ra"e C"u0ter<are 0o8t<are in0ta""ed
r0t" Juery r0 ati7e7er0ion - "i0t0 the .ra"e C"u0ter<are operating 7er0ion
r0t" "0modu"e0 00 - "i0t0 the C"u0ter SynhroniLation Ser7ie0 modu"e0 that an be u0ed 8or debugging
r0t" "0modu"e0 r0 - "i0t0 the C"u0ter Beady Ser7ie0 modu"e0 that an be u0ed 8or debugging
r0t" "0modu"e0 e7m - "i0t0 the !7ent Manager modu"e0 that an be u0ed 8or debugging
'8 nee00ary any o8 the0e ommand0 an be run <ith additiona" traing by adding a PtraeP
argument at the 7ery 8ront$ !1amp"e: r0t" trae he: 00
{node1:r0}/r0 #
11g!C/!SM/!I" oraclibm@fr.ibm.com 1%6 o# $%$
ocrconfi+ 4 ;o backup< e9port< import repair R JCH EJracle Clusterware
He+istryF contents.
{node1:r0}/r0 # oron8ig
Name:
oron8ig - Con8iguration too" 8or .ra"e C"u0ter Begi0try$
Synop0i0:
oron8ig >option@
option:
-e1port T8i"enameC >-0 on"ine@
- !1port "u0ter regi0ter ontent0 to a
8i"e
-import T8i"enameC - 'mport "u0ter regi0try ontent0 8rom a
8i"e
-upgrade >Tu0erC >TgroupC@@
- Upgrade "u0ter regi0try 8rom pre7iou0
7er0ion
-do<ngrade >-7er0ion T7er0ion 0tringC@
- (o<ngrade "u0ter regi0try to the
0pei8ied 7er0ion
-ba:up"o TdirnameC - Con8igure periodi ba:up "oation
-0ho<ba:up >auto\manua"@ - Sho< ba:up in8ormation
-manua"ba:up - Per8orm .CB ba:up
-re0tore T8i"enameC - Be0tore 8rom phy0ia" ba:up
-rep"ae or\ormirror >T8i"enameC@ - 6dd/rep"ae/remo7e a .CB de7ie/8i"e
-o7er<rite - .7er<rite .CB on8iguration on di0:
-repair or\ormirror T8i"enameC - Bepair "oa" .CB on8iguration
-he"p - Print out thi0 he"p in8ormation
Note:
].B6CH!?N.M!/"og/Tho0tnameC/"ient/oron8ig?TpidC$"og$ P"ea0e en0ure
running thi0 too"$
{node1:r0}/r0 #
ocrdump 4 ;o dump t$e content of t$e JCH.
{node1:r0}/r0 # ordump -he"p
Name:
ordump - (ump ontent0 o8 .ra"e C"u0ter Begi0try to a 8i"e$
Synop0i0:
ordump >T8i"enameC\-0tdout@ >-ba:up8i"e Tba:up8i"enameC@ >-:eyname T:eynameC@
>-1m"@ >-noheader@
(e0ription:
(e8au"t 8i"ename i0 .CB(UMPA'H!$ !1amp"e0 are:
promptC ordump
<rite0 "u0ter regi0try ontent0 to .CB(UMPA'H! in the urrent diretory
promptC ordump MIA'H!
<rite0 "u0ter regi0try ontent0 to MIA'H! in the urrent diretory
promptC ordump -0tdout -:eyname SIST!M
<rite0 the 0ubtree o8 SIST!M in the "u0ter regi0try to 0tdout
promptC ordump -0tdout -1m"
<rite0 "u0ter regi0try ontent0 to 0tdout in 1m" 8ormat
Note0:
The header in8ormation <i"" be retrie7ed ba0ed on be0t e88ort ba0i0$
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%1 o# $%$
].B6CH!?N.M!/"og/Tho0tnameC/"ient/ordump?TpidC$"og$ Ma:e 0ure
running thi0 too"$
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%7 o# $%$
srvctl 4 ;o administrate t$e clusterware resources.
{node1:r0}/r0 # 0r7t"
U0age: 0r7t" TommandC TobWetC >Toption0C@
ommand: enab"e\di0ab"e\0tart\0top\re"oate\0tatu0\add\remo7e\modi8y\geten7\0eten7\un0eten7\on8ig
obWet0: databa0e\in0tane\0er7ie\nodeapp0\a0m\"i0tener
Aor detai"ed he"p on eah ommand and obWet and it0 option0 u0e:
0r7t" TommandC TobWetC -h
{node1:r0}/r0 #
srvctl 1$ 4 ;o obtain all possible commands.
{node1:r0}/r0 # 0r7t" -h
U0age: 0r7t" >-/@
U0age: 0r7t" add databa0e -d TnameC -o Tora"e?homeC >-m Tdomain?nameC@ >-p T0p8i"eC@ >-6 Tname\ipC/netma0:@ >-r {PB'M6BI \
PNIS'C6H?ST6N(2I \ H.&'C6H?ST6N(2I \ SN6PSN.T?ST6N(2I}@ >-0 T0tart?option0C@ >-n Tdb?nameC@ >-y {6UT.M6T'C \ M6NU6H}@
U0age: 0r7t" add in0tane -d TnameC -i Tin0t?nameC -n Tnode?nameC
U0age: 0r7t" add 0er7ie -d TnameC -0 T0er7ie?nameC -r ZTpre8erred?"i0tCZ >-a ZTa7ai"ab"e?"i0tCZ@ >-P TT6A?po"iyC@
U0age: 0r7t" add 0er7ie -d TnameC -0 T0er7ie?nameC -u {-r ZTne<?pre8?in0tCZ \ -a ZTne<?a7ai"?in0tCZ}
U0age: 0r7t" add nodeapp0 -n Tnode?nameC -6 Tname\ipC/netma0:>/i81>\i8*\$$$@@
U0age: 0r7t" add a0m -n Tnode?nameC -i Ta0m?in0t?nameC -o Tora"e?homeC >-p T0p8i"eC@
U0age: 0r7t" add "i0tener -n Tnode?nameC -o Tora"e?homeC >-" T"i0tener?nameC@
U0age: 0r7t" on8ig databa0e
U0age: 0r7t" on8ig databa0e -d TnameC >-a@ >-t@
U0age: 0r7t" on8ig 0er7ie -d TnameC >-0 T0er7ie?nameC@ >-a@ >-S T"e7e"C@
U0age: 0r7t" on8ig nodeapp0 -n Tnode?nameC >-a@ >-g@ >-0@ >-"@ >-h@
U0age: 0r7t" on8ig a0m -n Tnode?nameC
U0age: 0r7t" on8ig "i0tener -n Tnode?nameC
U0age: 0r7t" di0ab"e databa0e -d TnameC
U0age: 0r7t" di0ab"e in0tane -d TnameC -i ZTin0t?name?"i0tCZ
U0age: 0r7t" di0ab"e 0er7ie -d TnameC -0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@
U0age: 0r7t" di0ab"e a0m -n Tnode?nameC >-i Tin0t?nameC@
U0age: 0r7t" enab"e databa0e -d TnameC
U0age: 0r7t" enab"e in0tane -d TnameC -i ZTin0t?name?"i0tCZ
U0age: 0r7t" enab"e 0er7ie -d TnameC -0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@
U0age: 0r7t" enab"e a0m -n Tnode?nameC >-i Tin0t?nameC@
U0age: 0r7t" geten7 databa0e -d TnameC >-t ZTname?"i0tCZ@
U0age: 0r7t" geten7 in0tane -d TnameC -i Tin0t?nameC >-t ZTname?"i0tCZ@
U0age: 0r7t" geten7 0er7ie -d TnameC -0 T0er7ie?nameC >-t ZTname?"i0tCZ@
11g!C/!SM/!I" oraclibm@fr.ibm.com 1%$ o# $%$
U0age: 0r7t" geten7 nodeapp0 -n Tnode?nameC >-t ZTname?"i0tCZ@
U0age: 0r7t" modi8y databa0e -d TnameC >-n Tdb?name@ >-o TohomeC@ >-m TdomainC@ >-p T0p8i"eC@ >-r {PB'M6BI \ PNIS'C6H?ST6N(2I \
H.&'C6H?ST6N(2I \ SN6PSN.T?ST6N(2I}@ >-0 T0tart?option0C@ >-y {6UT.M6T'C \ M6NU6H}@
U0age: 0r7t" modi8y in0tane -d TnameC -i Tin0t?nameC -n Tnode?nameC
U0age: 0r7t" modi8y in0tane -d TnameC -i Tin0t?nameC {-0 Ta0m?in0t?nameC \ -r}
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -i To"d?in0t?nameC -t Tne<?in0t?nameC >-8@
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -i Ta7ai"?in0t?nameC -r >-8@
U0age: 0r7t" modi8y 0er7ie -d TnameC -0 T0er7ie?nameC -n -i Tpre8erred?"i0tC >-a Ta7ai"ab"e?"i0tC@ >-8@
U0age: 0r7t" modi8y a0m -n Tnode?nameC -i Ta0m?in0t?nameC >-o Tora"e?homeC@ >-p T0p8i"eC@
U0age: 0r7t" re"oate 0er7ie -d TnameC -0 T0er7ie?nameC -i To"d?in0t?nameC -t Tne<?in0t?nameC >-8@
U0age: 0r7t" remo7e databa0e -d TnameC >-8@
U0age: 0r7t" remo7e in0tane -d TnameC -i Tin0t?nameC >-8@
U0age: 0r7t" remo7e 0er7ie -d TnameC -0 T0er7ie?nameC >-i Tin0t?nameC@ >-8@
U0age: 0r7t" remo7e nodeapp0 -n ZTnode?name?"i0tCZ >-8@
U0age: 0r7t" remo7e a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-8@
U0age: 0r7t" remo7e "i0tener -n Tnode?nameC >-" T"i0tener?nameC@
U0age: 0r7t" 0eten7 databa0e -d TnameC {-t TnameCOT7a"C>QTnameCOT7a"CQ$$$@ \ -T TnameCOT7a"C}
U0age: 0r7t" 0eten7 in0tane -d TnameC >-i Tin0t?nameC@ {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0eten7 0er7ie -d TnameC >-0 T0er7ie?nameC@ {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0eten7 nodeapp0 -n Tnode?nameC {-t ZTnameCOT7a"C>QTnameCOT7a"CQ$$$@Z \ -T ZTnameCOT7a"CZ}
U0age: 0r7t" 0tart databa0e -d TnameC >-o T0tart?option0C@
U0age: 0r7t" 0tart in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-o T0tart?option0C@
U0age: 0r7t" 0tart 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@@ >-o T0tart?option0C@
U0age: 0r7t" 0tart nodeapp0 -n Tnode?nameC
U0age: 0r7t" 0tart a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0tart?option0C@
U0age: 0r7t" 0tart "i0tener -n Tnode?nameC >-" T"0nr?name?"i0tC@
U0age: 0r7t" 0tatu0 databa0e -d TnameC >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ@ >-8@ >-7@ >-S T"e7e"C@
U0age: 0r7t" 0tatu0 nodeapp0 -n Tnode?nameC
U0age: 0r7t" 0tatu0 a0m -n Tnode?nameC
U0age: 0r7t" 0top databa0e -d TnameC >-o T0top?option0C@
U0age: 0r7t" 0top in0tane -d TnameC -i ZTin0t?name?"i0tCZ >-o T0top?option0C@
U0age: 0r7t" 0top 0er7ie -d TnameC >-0 ZT0er7ie?name?"i0tCZ >-i Tin0t?nameC@@ >-8@
U0age: 0r7t" 0top nodeapp0 -n Tnode?nameC >-r@
U0age: 0r7t" 0top a0m -n Tnode?nameC >-i Ta0m?in0t?nameC@ >-o T0top?option0C@
U0age: 0r7t" 0top "i0tener -n Tnode?nameC >-" T"0nr?name?"i0tC@
U0age: 0r7t" un0eten7 databa0e -d TnameC -t ZTname?"i0tCZ
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%B o# $%$
U0age: 0r7t" un0eten7 in0tane -d TnameC >-i Tin0t?nameC@ -t ZTname?"i0tCZ
U0age: 0r7t" un0eten7 0er7ie -d TnameC >-0 T0er7ie?nameC@ -t ZTname?"i0tCZ
U0age: 0r7t" un0eten7 nodeapp0 -n Tnode?nameC -t ZTname?"i0tCZ
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%C o# $%$
crs,stat 1$elp 4 ;o %uery state of Jracle Clusterware resources.
{node1:r0}/r0 # r0?0tat -he"p
U0age: r0?0tat >re0oure?name >$$$@@ >-7@ >-"@ >-J@ >- "u0ter?member@
r0?0tat >re0oure?name >$$$@@ -t >-7@ >-J@ >- "u0ter?member@
r0?0tat -p >re0oure?name >$$$@@ >-J@
r0?0tat >-a@ app"iation -g
r0?0tat >-a@ app"iation -r >- "u0ter?member@
r0?0tat -8 >re0oure?name >$$$@@ >-J@ >- "u0ter?member@
r0?0tat -"0 >re0oure?name >$$$@@ >-J@
{node1:r0}/r0 #
Cluvfy 4 ;o pre?post c$eck installation sta+es.
{node1:r0}/r0 # "u78y
US6&!:
"u78y > -he"p @
"u78y 0tage { -"i0t \ -he"p }
"u78y 0tage {-pre\-po0t} T0tage-nameC T0tage-0pei8i option0C >-7erbo0e@
"u78y omp { -"i0t \ -he"p }
"u78y omp Tomponent-nameC Tomponent-0pei8i option0C >-7erbo0e@
{node1:r0}/r0 #
cluvfy sta+e 1list
{node1:r0}/r0 # "u78y 0tage -"i0t
US6&!:
/a"id 0tage option0 and 0tage name0 are:
-po0t h<o0 : po0t-he: 8or hard<are and operating 0y0tem
-pre 80 : pre-he: 8or CAS 0etup
-po0t 80 : po0t-he: 8or CAS 0etup
-pre r0in0t : pre-he: 8or CBS in0ta""ation
-po0t r0in0t : po0t-he: 8or CBS in0ta""ation
-pre dbin0t : pre-he: 8or databa0e in0ta""ation
-pre db8g : pre-he: 8or databa0e on8iguration
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%G o# $%$
cluvfy sta+e -$elp
{node1:r0}/r0 # "u78y 0tage -he"p
US6&!:
SINT6K F8or Stage0G:
"u78y 0tage -po0t h<o0 -n Tnode?"i0tC > -0 T0torage'(?"i0tC @ >-7erbo0e@
"u78y 0tage -pre 80 -n Tnode?"i0tC -0 T0torage'(?"i0tC >-7erbo0e@
"u78y 0tage -po0t 80 -n Tnode?"i0tC -8 T8i"e?0y0temC >-7erbo0e@
"u78y 0tage -pre r0in0t -n Tnode?"i0tC >-r { 1#gB1 \ 1#gB* \ 11gB1 } @
> - Tor?"oationC @ > -J T7oting?di0:C @
> -o0dba To0dba?groupC @
> -orain7 Torain7entory?groupC @ >-7erbo0e@
"u78y 0tage -po0t r0in0t -n Tnode?"i0tC >-7erbo0e@
"u78y 0tage -pre dbin0t -n Tnode?"i0tC >-r { 1#gB1 \ 1#gB* \ 11gB1 } @
> -o0dba To0dba?groupC @ >-7erbo0e@
"u78y 0tage -pre db8g -n Tnode?"i0tC -d Tora"e?homeC >-7erbo0e@
{node1:r0}/r0 #
cluvfy comp -list
{node1:r0}/r0 # "u78y omp -"i0t
US6&!:
/a"id omponent0 are:
nodereah : he:0 reahabi"ity bet<een node0
nodeon : he:0 node onneti7ity
80 : he:0 CAS integrity
00a : he:0 0hared 0torage ae00ibi"ity
0pae : he:0 0pae a7ai"abi"ity
0y0 : he:0 minimum 0y0tem reJuirement0
"u : he:0 "u0ter integrity
"umgr : he:0 "u0ter manager integrity
or : he:0 .CB integrity
r0 : he:0 CBS integrity
nodeapp : he:0 node app"iation0 e1i0tene
admpr7 : he:0 admini0trati7e pri7i"ege0
peer : ompare0 propertie0 <ith peer0
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%K o# $%$
Cluvfy comp -$elp
{node1:r0}/r0 # "u78y omp -he"p
US6&!:
SINT6K F8or Component0G:
"u78y omp nodereah -n Tnode?"i0tC > -0rnode TnodeC @ >-7erbo0e@
"u78y omp nodeon -n Tnode?"i0tC > -i Tinter8ae?"i0tC @ >-7erbo0e@
"u78y omp 80 > -n Tnode?"i0tC @ -8 T8i"e?0y0temC >-7erbo0e@
"u78y omp 00a > -n Tnode?"i0tC @ > -0 T0torage'(?"i0tC @ >-7erbo0e@
"u78y omp 0pae > -n Tnode?"i0tC @ -" T0torage?"oationC
-L Tdi0:?0paeC {2\U\M\&} >-7erbo0e@
"u78y omp 0y0 > -n Tnode?"i0tC @ -p { r0 \ databa0e } >-r { 1#gB1 \ 1#gB* \ 11gB1
} @
> -o0dba To0dba?groupC @ > -orain7 Torain7entory?groupC @
>-7erbo0e@
"u78y omp "u > -n Tnode?"i0tC @ >-7erbo0e@
"u78y omp "umgr > -n Tnode?"i0tC @ >-7erbo0e@
"u78y omp or > -n Tnode?"i0tC @ >-7erbo0e@
"u78y omp r0 > -n Tnode?"i0tC @ >-7erbo0e@
"u78y omp nodeapp > -n Tnode?"i0tC @ >-7erbo0e@
"u78y omp admpr7 > -n Tnode?"i0tC @ >-7erbo0e@
-o u0er?eJui7 >-00hon"y@
-o r0?in0t >-orain7 Torain7entory?groupC @
-o db?in0t >-o0dba To0dba?groupC @
-o db?on8ig -d Tora"e?homeC
"u78y omp peer > -re8node TnodeC @ -n Tnode?"i0tC >-r { 1#gB1 \ 1#gB* \ 11gB1 } @
> -orain7 Torain7entory?groupC @ > -o0dba To0dba?groupC @ >-7erbo0e@
{node1:r0}/r0 #
11g!C/!SM/!I"
oraclibm@fr.ibm.com
1%8 o# $%$
71 !PP;E1I" 1 2 ;MPTI T!B:;S TO 4S; FO IEST!::!TIOE
71)1 Eet,or. document to ease your installation
>!et"or# layout7 5osts namin%7 etc ...?
Each node M3S+ ha(e sa2e network interfaces la/out and usage0 4o identif/ /our ,3B'IC0 and ,RIVA+E network interface0 and fill in table on ne1t !age 5
11g!C/!SM/!I" oraclibm@fr.ibm.com 1%% o# $%$
Pu&lic/ Pri>ate/ and 3irtual Dost Eame layout
1or eac& node 0artici0atin( to t&e clu%ter' fill in &o%tname and a%%ociated !P for 4
Public networ,
Nirtual networ,
Private networ,
Eode Eet,or.
Inter#ace name
Dost Ty(e 1e#ined hostname !ssigned IP O&ser>ation
1
enNNNN
Pu&lic !C Pu&lic node name *Pu&lic Eet,or.+
3irtual !C 3IP node name *Pri>ate Eet,or.+
enNNNN Pri>ate !C Interconnect node name *Pri>ate Eet,or.+
7
enNNNN
$
enNNNN
B
enNNNN
M
enNNNN
71)7 Ste(s Chec.out
Action
Done 8
>ode * >ode 5 >ode J >ode A >ode @ >ode / >ode ..
71)$ 1is.s document to ease dis.s (re(aration in your im(lementation
>2CR7 /otin%7 ASM sp)ile and ASM dis#s?
1is.s
:4E-s
Eum&er
1e>ice Eame
Eode 1 Eode 7 Eode $ Eode B
hdis.
Ma9or
Eum)
Minor
Eum)
hdis.
Ma9or
Eum)
Minor
Eum)
hdis.
Ma9or
Eum)
Minor
Eum)
hdis.
Ma9or
Eum)
Minor
Eum)
Di%, e<am0le LC deve<am0le6di%, Fdi%,* JC 5 Fdi%,C JC 5 Fdi%,* JC 5 Fdi%,* JC 5
Di%, for OCR *
Di%, for OCR 5
Di%, for Notin( *
Di%, for Notin( 5
Di%, for Notin( J
Di%, for A$" %0file
Di%, * for A$"
Di%, 5 for A$"
Di%, J for A$"
Di%, A for A$"
Di%, @ for A$"
Di%, / for A$"
Di%, E for A$"
Di%, I for A$"
Di%, D for A$"
Di%, *C for A$"
11g!C/!SM/!I"
oraclibm@fr.ibm.com
767 o# $%$
77 1OC4M;ETS/ BOOTS TO :OOT !T ))
IBML 1SB666 and OracleL 1ata&ase #or !I")
!&stract2 G&e !B"7 $y%tem $tora(e D$ACCC i% very well %uited for Oracle Databa%e%. Learn &ow to be%t u%e t&e D$ACCC in an A!+7 environment by under%tandin(
a00licable data layout tec&ni)ue% and ot&er im0ortant con%ideration%.
Document Aut&or 4 #eff "uc&er -Advanced Gec&nical $u00ort.
Document !D 4 PS$6BB
Product-%. covered 4 A!+ @Lb D$ACCCb D$AECCb D$AICCb 2P1$b Oracleb Oracle RACb Oracle Databa%e
&tt04wJ-CJ.ibm.com%u00orttec&doc%at%ma%tr.n%fWeb!nde<PR$JCAA
Oracle !utomatic Storage
Management
4nder-the-Dood H Practical
1e(loyment =uide
Mc=ra, Dill
By4 >itin Nen(urle,ar' "urali Nallat&'
Ric& Lon(
Oracle 1ata&ase 16g Digh !>aila&ility
,ith !C Flash&ac. and 1ata =uard
Mc=ra, Dill
By 4 "att&ew Fart ' $cott #e%%e
11g!C/!SM/!I"
oraclibm@fr.ibm.com
76$ o# $%$

COOKBOOK Oracle 11gRAC R1 ASM AIX5L SAN Storage Installation Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

COOKBOOK Oracle 11gRAC R1 ASM AIX5L SAN Storage Installation Guide

Uploaded by

Copyright:

Available Formats

DRAFT

In normal o(eration/ all :P!-s are using

C) Test your cluster ,ith crash scenarios

You might also like