You are on page 1of 100

Deploying a Highly Available Web

Server on Red Hat Enterprise Linux 6


Mark Heslin
Prinipal So!t"are Engineer
#ersion $%&
August '&$$
1801 Varsity Drive
Raleigh NC 27606-2072 USA
Phone !1 "1" 7#$ %700
Phone 888 7%% $281
&a' !1 "1" 7#$ %701
P( )o' 1%#88
Resear*h +riangle Par, NC 2770" USA
-in.' is a registere/ tra/e0ar, o1 -in.s +orval/s2 Re/ 3at4 Re/ 3at 5nter6rise -in.' an/ the Re/ 3at
7Sha/o80an7 logo are registere/ tra/e0ar,s o1 Re/ 3at4 9n*2 in the Unite/ States an/ other
*o.ntries2
:i*roso1t an/ ;in/o8s are U2S2 registere/ tra/e0ar,s o1 :i*roso1t Cor6oration2
UN9< is a registere/ tra/e0ar, o1 +he (6en =ro.62
9ntel4 the 9ntel logo an/ <eon are registere/ tra/e0ar,s o1 9ntel Cor6oration or its s.>si/iaries in the
Unite/ States an/ other *o.ntries2
All other tra/e0ar,s re1eren*e/ herein are the 6ro6erty o1 their res6e*tive o8ners2
? 2011 >y Re/ 3at4 9n*2 +his 0aterial 0ay >e /istri>.te/ only s.>@e*t to the ter0s an/ *on/itions set
1orth in the (6en P.>li*ation -i*ense4 V120 or later Athe latest version is 6resently availa>le at
htt6BB8882o6en*ontent2orgBo6en6.>BC2
+he in1or0ation *ontaine/ herein is s.>@e*t to *hange 8itho.t noti*e2 Re/ 3at4 9n*2 shall not >e lia>le
1or te*hni*al or e/itorial errors or o0issions *ontaine/ herein2
Distri>.tion o1 0o/i1ie/ versions o1 this /o*.0ent is 6rohi>ite/ 8itho.t the e'6li*it 6er0ission o1 Re/
3at 9n*2
Distri>.tion o1 this 8or, or /erivative o1 this 8or, in any stan/ar/ A6a6erC >oo, 1or0 1or *o00er*ial
6.r6oses is 6rohi>ite/ .nless 6rior 6er0ission is o>taine/ 1ro0 Re/ 3at 9n*2
+he =P= 1inger6rint o1 the se*.rityDre/hat2*o0 ,ey is
CA 20 86 86 2) D6 "D &C 6# &6 5C C$ 21 "1 80 CD D) $2 A6 05
Sen/ 1ee/>a*, to re1ar*h-1ee/>a*,Dre/hat2*o0
"""%redhat%o( ii re!arh)!eedbak*redhat%o(
Table of Contents
1 5'e*.tive S.00ary22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1
2 Co06onent (vervie8222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2
221 Re/ 3at 5nter6rise -in.' 622222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2
222 Re/ 3at Net8or, Satellite Server22222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %
22% 3igh Availa>ility A//-(n2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $
22%21 E.or.0222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $
22%22 Reso.r*e =ro.6 :anager2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $
22%2% &en*ing222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #
22%2%21 9P:9222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 6
22%2$ &ailover Do0ains222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 7
22%2# Conga222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 8
22%2#21 -.*i2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 8
22%2#22 Ri**i222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 8
22%26 CCS222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 "
22$ 3A--V: Vol.0e22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 10
22$21 -ogi*al Vol.0e :anager2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 10
22$22 &ile Syste02222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 10
22$2% D: :.lti6ath22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 11
22# Cl.ster ;e> Servi*e22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 12
% Re1eren*e Ar*hite*t.re Con1ig.ration222222222222222222222222222222222222222222222222222222222222 1%
%21 Cl.ster :anage0ent Server2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1$
%22 Cl.ster Server - No/e 1222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1$
%2% Cl.ster Server - No/e 2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1#
%2$ Cl.ster Server - No/e %222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1#
%2# &i>re Channel Storage Array2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 16
$ Cl.ster De6loy0ent2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 17
$21 De6loy0ent +as, &lo82222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 17
$22 De6loy :anage0ent Server 2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 18
$2221 9nstall Re/ 3at 5nter6rise -in.' 622222222222222222222222222222222222222222222222222222222222222222222222222222 18
$2222 Con1ig.re Net8or,s222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 1"
re!arh)!eedbak*redhat%o( iii """%redhat%o(
$222% Con1ig.re &ire8all222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 20
$222$ 9nstall Cl.ster :anage0ent So1t8are222222222222222222222222222222222222222222222222222222222222222222222222 21
$2% De6loy Cl.ster No/es22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2%
$2%21 9nstall Re/ 3at 5nter6rise -in.' 622222222222222222222222222222222222222222222222222222222222222222222222222222 2%
$2%22 Con1ig.re Net8or,s an/ )on/ing222222222222222222222222222222222222222222222222222222222222222222222222222222 2$
$2%2% Con1ig.re &ire8all222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 27
$2%2$ 9nstall Cl.ster No/e So1t8are222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2"
$2%2# Con1ig.re Storage222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2"
$2%2#21 Con1ig.re :.lti6athing22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 2"
$2%2#22 Create 3A--V: Vol.0e222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %1
$2%26 Con1ig.re ;e> Server22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %%
$2%27 Con1ig.re S5-in.' Se*.rity Para0eters2222222222222222222222222222222222222222222222222222222222222222222 %$
$2$ Cl.ster Creation via Conga222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %6
$2$21 Create Cl.ster222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %6
$2$22 A// &en*e Devi*es2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 %"
$2$2% A// &ailover Do0ain2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $$
$2$2$ A// Reso.r*es2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $6
$2$2# A// Servi*e =ro.62222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 $"
$2$26 Veri1y Cl.ster ;e> Servi*e2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #1
$2# Cl.ster Creation via CCS222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #2
$2#21 Create Cl.ster222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #2
$2#22 A// No/es22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #%
$2#2% A// &en*e Devi*es2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #%
$2#2$ A// &ailover Do0ain2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #$
$2#2# A// Reso.r*es2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #$
$2#26 A// Servi*e =ro.62222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 ##
$2#27 A*tivate Cl.ster2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 ##
$2#28 Veri1y Cl.ster ;e> Servi*e2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #6
# Cl.ster :anage0ent22222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #7
#21 A//ing Cl.ster No/es22222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 #7
#22 Re0oving Cl.ster No/es222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 61
#2% Relo*ating Cl.ster ;e> Servi*es222222222222222222222222222222222222222222222222222222222222222222222222222222222222 6#
#2$ &en*ing Cl.ster No/es2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 6"
#2# 906orting a Cl.ster2222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 71
6 Con*l.sion222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222 7%
A66en/i' A Re1eren*es22222222222222222222222222222222222222222222222222222222222222222222222222222222222 7$
"""%redhat%o( iv re!arh)!eedbak*redhat%o(
A66en/i' ) Re/ 3at 5nter6rise -in.' 6 F Satellite Con1ig.ration Details2222222 7#
A66en/i' C &i>re Channel Storage Provisioning22222222222222222222222222222222222222222222 8#
A66en/i' D Cl.ster Con1ig.ration &ile A*l.ster2*on1C222222222222222222222222222222222222222 8"
A66en/i' 5 3A-;e> Stat.s S*ri6t222222222222222222222222222222222222222222222222222222222222222222 "1
A66en/i' & Cl.ster Con1ig.ration :atri'222222222222222222222222222222222222222222222222222222222 "%
A66en/i' = De6loy0ent Che*,lists2222222222222222222222222222222222222222222222222222222222222222 "$
re!arh)!eedbak*redhat%o( v """%redhat%o(
1 Executive Summary
Re/ 3atGs 3igh Availa>ility A//-(n 1or Re/ 3at 5nter6rise -in.' 6 is Re/ 3atGs 6re0ier high
availa>ility *l.stering sol.tion2 +he 3igh Availa>ility A//-(n 6rovi/es relia>ility4 availa>ility
an/ s*ala>ility ARASC to *riti*al 6ro/.*tion servi*es >y eli0inating single 6oints o1 1ail.re an/
6rovi/ing a.to0ati* failover o1 those servi*es in the event o1 a *l.ster no/e 1ail.re or error
*on/ition2
+his re1eren*e ar*hite*t.re /etails the /e6loy0ent4 *on1ig.ration an/ 0anage0ent o1 a highly
availa>le 8e> servi*e .sing the 3igh Availa>ility A//-(n 1or Re/ 3at 5nter6rise -in.' 62 A
three no/e *l.ster is /e6loye/ to r.n the 8e> servi*e an/ a /e/i*ate/ 0anage0ent station is
/e6loye/ to 0anage an/ *on1ig.re the *l.ster2
+he .n/erlying storage 1or the 8e> servi*e .tiliHes a highly-availa>le -V: A3A--V:C vol.0e2
3A--V: vol.0es 6er0it *l.ster servi*e /ata to >e share/ e'*l.sively a0ong highly-availa>le
no/es 1or the least *ost an/ highest 6ossi>le 6er1or0an*e2 +he 3A--V: vol.0e 8ithin this
/o*.0ent is *reate/ on a &i>re Channel storage array >.t other share/ storage te*hnologies
Ae.g. - iSCSIC 0ay >e .se/2
A//itional re/.n/an*y an/ 6er1or0an*e in*reases are a*hieve/ thro.gh the .se o1 se6arate
6.>li* an/ 6rivate A*l.ster inter*onne*tC net8or,s2 :.lti6le net8or, a/a6ters are .se/ on
these net8or,s 8ith all inter1a*es >on/e/ together2 Si0ilarly4 /evi*e 0a66er 0.lti6athing is
.se/ to 0a'i0iHe 6er1or0an*e an/ availa>ility to the 3A--V: vol.0e2
+8o 0etho/s are availa>le 1or *reating an/ 0anaging *l.sters 8ith the 3igh Availa>ility A//-
(n 1or Re/ 3at 5nter6rise -in.' 62 +he 1irst 0etho/ is thro.gh the .se o1 Conga an/ the
luci gra6hi*al .ser inter1a*e A=U9C2 +he se*on/ 0etho/ is thro.gh the .se o1 the Cl.ster
Con1ig.ration Syste0 ACCSC *o00an/ line inter1a*e AC-9C2 )oth a66roa*hes are /etaile/
8ithin this re1eren*e ar*hite*t.re2
+he 0ost *o00on *l.ster 0anage0ent tas,s Aa//ing4 re0oving no/es4 relo*ating *l.ster
servi*es4 1en*ing no/es4 i06orting an e'isting *l.sterC are also /e0onstrate/2
+his /o*.0ent /oes not reI.ire e'tensive Re/ 3at 5nter6rise -in.' e'6erien*e >.t the
rea/er is e'6e*te/ to have a 8or,ing ,no8le/ge o1 -in.' a/0inistration an/ *l.stering
*on*e6ts2
re!arh)!eedbak*redhat%o( $ """%redhat%o(
2 Component Overview
+his se*tion 6rovi/es an overvie8 on the Re/ 3at 5nter6rise -in.' o6erating syste04 Re/
3atGs 3igh Availa>ility A//-(n an/ the other *o06onents .se/ in this re1eren*e ar*hite*t.re2
2.1 Red Hat Enterprise Linux 6
Re/ 3at 5nter6rise -in.' 6214 the latest release o1 Re/ 3atGs tr.ste/ /ata*enter 6lat1or04
/elivers a/van*es in a66li*ation 6er1or0an*e4 s*ala>ility4 an/ se*.rity2 ;ith Re/ 3at
5nter6rise -in.' 6214 6hysi*al4 virt.al an/ *lo./ *o06.ting reso.r*es *an >e /e6loye/ 8ithin
the /ata *enter2 Re/ 3at 5nter6rise -in.' 621 6rovi/es the 1ollo8ing 1eat.res an/ *a6a>ilities
Reliability+ Availability+ and Seurity ,RAS-.
:ore so*,ets4 0ore *ores4 0ore threa/s4 an/ 0ore 0e0ory
RAS har/8are->ase/ hot a// o1 CPUs an/ 0e0ory is ena>le/
:e0ory 6ages 8ith errors *an >e /e*lare/ as J6oisone/K an/ *an >e avoi/e/
/ile Syste(s.
e't$ is the /e1a.lt 1ilesyste0 an/ s*ales to 16+)
<&S is availa>le as an a//-on an/ *an s*ale to 100+)
&.se allo8s 1ile syste0s to r.n in .ser s6a*e allo8ing testing an/ /evelo60ent on
ne8er 1.se->ase/ 1ile syste0s As.*h as *lo./ 1ile syste0sC
High Availability.
5'ten/s the *.rrent *l.stering sol.tion to the virt.al environ0ent allo8ing 1or high
availa>ility o1 virt.al 0a*hines an/ a66li*ations r.nning insi/e those virt.al 0a*hines
5na>les N&Sv$ reso.r*e agent 0onitoring
9ntro/.*tion o1 CCS2 CCS is a *o00an/ line tool that allo8s 1or *o06lete C-9
a/0inistration o1 Re/ 3atGs 3igh Availa>ility A//-(n
Resoure Manage(ent.
*gro.6s organiHe syste0 tas,s so that they *an >e tra*,e/ an/ so that other syste0
servi*es *an *ontrol the reso.r*es that *gro.6 tas,s 0ay *ons.0e
*6.set a66lies CPU reso.r*e li0its to *gro.6s4 allo8ing 6ro*essing 6er1or0an*e to >e
allo*ate/ to tas,s
+here are 0any other 1eat.re enhan*e0ents to Re/ 3at 5nter6rise -in.' 62 Please see the
Re/ 3at 8e>site 1or 0ore in1or0ation2
"""%redhat%o( ' re!arh)!eedbak*redhat%o(
2.2 Red Hat Network Satellite Server
+his re1eren*e ar*hite*t.re .ses Re/ 3at Net8or, Satellite AR3NC server to 6rovision the
*l.ster no/es2 All R3N 1.n*tionality is on the net8or,4 allo8ing 0.*h greater 1le'i>ility an/
*.sto0iHation2 +he satellite server *onne*ts 8ith Re/ 3at over the 6.>li* 9nternet to
/o8nloa/ ne8 *ontent an/ .6/ates2 +his 0o/el also allo8s *.sto0ers to ta,e their Re/ 3at
Net8or, sol.tion *o06letely o11-line i1 /esire/2 A/vantages o1 .sing R3N Satellite in*l./e
Se*.rity - an en/-to-en/ se*.re *onne*tion is 0aintaine/ 1ro0 the *lient syste0s to
the R3N Satellite 8itho.t *onne*ting to the 6.>li* 9nternet2
511i*ien*y - 6a*,ages are /elivere/ signi1i*antly 1aster over a lo*al area net8or,2
Control - *lientsG Syste0 Pro1iles are store/ on the lo*al R3N Satellite4 not on the
*entral Re/ 3at Net8or, Servers2
C.sto0iHe/ .6/ates - *reate a tr.ly a.to0ate/ 6a*,age /elivery syste0 1or *.sto0
so1t8are 6a*,ages reI.ire/ >y *lient syste0s4 as 8ell as Re/ 3at 6a*,ages2 C.sto0
*hannels allo8 1ine- graine/ *ontrol o1 the /elivery o1 *.sto0 6a*,ages2
A**ess *ontrol - syste0 a/0inistrators *an >e restri*te/ to a**ess only those syste0s
8ithin their 0aintenan*e res6onsi>ilities2
)an/8i/th 0anage0ent - the >an/8i/th .se/ 1or transa*tions >et8een the *lients an/
the R3N Satellite is *ontrolle/ >y the organiHation on the lo*al area net8or,L R3N
Satellite *lients /o not have to *o06ete 8ith other *lients a**essing the *entral Re/
3at Net8or, 1ile servers2
S*ala>ility - R3N Satellite 0ay oversee an entire organiHationGs servers in *o0>ination
8ith R3N Pro'y Server2
R3N Satellite is Re/ 3atMs on-6re0ises syste0s 0anage0ent sol.tion that 6rovi/es so1t8are
.6/ates4 *on1ig.ration 0anage0ent4 6rovisioning an/ 0onitoring a*ross >oth 6hysi*al an/
virt.al Re/ 3at 5nter6rise -in.' servers2 9t o11ers *.sto0ers o66ort.nities to gain enhan*e/
6er1or0an*e4 *entraliHe/ *ontrol an/ higher s*ala>ility 1or their syste0s4 8hile /e6loye/ on a
0anage0ent server lo*ate/ insi/e the *.sto0erMs /ata *enter an/ 1ire8all2
re!arh)!eedbak*redhat%o( 0 """%redhat%o(
2.3 High Availailit! Add"#n
+he 3igh Availa>ility A//-(n 1or Re/ 3at 5nter6rise -in.' 6rovi/es high availa>ility o1
servi*es >y eli0inating single 6oints o1 1ail.re2 )y o11ering 1ailover servi*es >et8een no/es
8ithin a *l.ster4 the 3igh Availa>ility A//-(n s.66orts high availa>ility 1or .6 to 16 no/es2
AC.rrently this *a6a>ility is li0ite/ to a single -AN or /ata*enter lo*ate/ 8ithin one 6hysi*al
site2C
+he 3igh Availa>ility A//-(n also ena>les 1ailover 1or o11-the-shel1 a66li*ations s.*h as
A6a*he4 :ySE-4 an/ PostgreSE-4 any o1 8hi*h *an >e *o.6le/ 8ith reso.r*es li,e 9P
a//resses an/ single-no/e 1ile syste0s to 1or0 highly availa>le servi*es2 +he 3igh
Availa>ility A//-(n *an also >e easily e'ten/e/ to any .ser-s6e*i1ie/ a66li*ation that is
*ontrolle/ >y an init s*ri6t 6er UN9< Syste0 V ASysVC stan/ar/s2
;hen .sing the 3igh Availa>ility A//-(n4 a highly availa>le servi*e *an 1ail over 1ro0 one
no/e to another 8ith no a66arent interr.6tion to *l.ster *lients2 +he 3igh Availa>ility A//-(n
also ens.res a>sol.te /ata integrity 8hen one *l.ster no/e ta,es over *ontrol o1 a servi*e
1ro0 another *l.ster no/e2 9t a*hieves this >y 6ro06tly evi*ting no/es 1ro0 the *l.ster that are
/ee0e/ to >e 1a.lty .sing a 0etho/ *alle/ 71en*ing74 th.s 6reventing /ata *orr.6tion2 +he
3igh Availa>ility A//-(n s.66orts several ty6es o1 1en*ing4 in*l./ing >oth 6o8er an/ storage
area net8or, ASANC >ase/ 1en*ing2
+he 1ollo8ing se*tions /es*ri>e the vario.s *o06onents o1 the 3igh Availa>ility A//-(n in the
*onte't o1 this re1eren*e ar*hite*t.re2
2.3.1 Quorum
Quorum is a voting algorith0 .se/ >y the *l.ster 0anager AC:ANC2 C:AN 0anages *l.ster
I.or.0 an/ *l.ster 0e0>ershi62 C:AN r.ns as a servi*e on all the *l.ster no/es2 +o
0aintain quorum, the no/es in the *l.ster 0.st agree a>o.t their stat.s a0ong the0selves2
+he quorum /eter0ines 8hi*h no/es in the *l.ster are /o0inant2 &or e'a06le4 i1 there are
three no/es in a *l.ster an/ one no/e loses *onne*tivity4 the other t8o no/es *o00.ni*ate
8ith ea*h other an/ /eter0ine that the thir/ no/e nee/s to >e 1en*e/2 +he a*tion o1 1en*ing
ens.res that the no/e 8hi*h lost *onne*tivity /oes not *orr.6t /ata2
)y /e1a.lt ea*h no/e in the *l.ster has one quorum vote4 altho.gh this is *on1ig.ra>le2 +here
are t8o 0etho/s the no/es *an *o00.ni*ate 8ith ea*h other to /eter0ine quorum. +he 1irst
0etho/ quorum via net8or, *onsists o1 a si06le 0a@ority A#0N o1 the no/es !1 e'traC2 +he
se*on/ 0etho/ is >y a//ing a quorum /is,2 +he quorum /is, allo8s 1or .ser-s6e*i1ie/
*on/itions to e'ist 8hi*h hel6 /eter0ine 8hi*h no/eAsC sho.l/ >e /o0inant2
+his re1eren*e ar*hite*t.re .ses net8or, quorum - a /e/i*ate/ quorum /is, is not reI.ire/2
2.3.2 Resource Group Manager
+he reso.r*e gro.6 0anager ArgmanagerC 6rovi/es 1ailover *a6a>ilities 1or *olle*tions o1
*l.ster reso.r*es ,no8n as reso.r*e gro.6s or reso.r*e trees2 Rgmanager 8or,s >y allo8ing
syste0s a/0inistrators to /e1ine4 *on1ig.re4 an/ 0onitor *l.ster servi*es2 9n the event o1 a
no/e 1ail.re4 rgmanager relo*ates the *l.stere/ servi*e to another no/e to restore servi*e
"""%redhat%o( 1 re!arh)!eedbak*redhat%o(
availa>ility2 Servi*es *an >e restri*te/ to *ertain no/es4 s.*h as restri*ting httpd to one set o1
no/es 8hile mysql *an >e restri*te/ to a se6arate set o1 no/es2
+he 1ollo8ing list s.00ariHes the vario.s 6ro*esses an/ agents that *onstit.te rgmanager
&ailover Do0ains - An or/ere/ s.>set o1 0e0>ers to 8hi*h a servi*e 0ay >e >o.n/
Servi*e Poli*ies - rgmanager's servi*e start.6 an/ re*overy 6oli*ies
Reso.r*e +rees - Re6resentations o1 reso.r*es4 their attri>.tes4 6arent B *hil/ an/
si>ling relationshi6s
Reso.r*e Agents
rgmanager r.ns as a servi*e on all the no/es in a *l.ster2 91 the servi*e is not r.nning4 the
reso.r*es are not availa>le to >e >ro.ght online2 Re*overy o1 rgmanager /e6en/s on the
Distri>.te/ -o*, :anager AD-:C2 9n the event o1 a 1ail.re4 the D-: 0.st re*over 6rior to
rgmanager re*overing servi*es 1ro0 a 1aile/ host2
2.3.3 Fencing
&en*ing is the /is*onne*tion o1 a no/e 1ro0 the *l.sterGs share/ storage2 &en*ing 6revents
the a11e*te/ no/e 1ro0 iss.ing 9B( to share/ storage4 th.s ens.ring /ata integrity2 +he *l.ster
in1rastr.*t.re 6er1or0s 1en*ing thro.gh fenced, the 1en*e /ae0on.
;hen C:AN /eter0ines that a no/e has 1aile/4 it *o00.ni*ates to other *l.ster-
in1rastr.*t.re *o06onents to in1or0 the0 that the no/e has 1aile/2 +he 1aile/ no/e is 1en*e/
8hen fenced is noti1ie/2 (ther *l.ster-in1rastr.*t.re *o06onents /eter0ine 8hat a*tions to
ta,e - that is4 they 6er1or0 any re*overy that nee/s to /one2 &or e'a06le4 /istri>.te/ lo*,
0anager AD!C an/ =lo>al &ile Syste0 version 2 A"#S$%4 8hen noti1ie/ o1 a no/e 1ail.re4
s.s6en/ a*tivity .ntil they /ete*t that fenced has *o06lete/ 1en*ing the 1aile/ no/e2 U6on
*on1ir0ation that the 1aile/ no/e is 1en*e/4 D! an/ "#S$ 6er1or0 re*overy2 D! releases
lo*,s o1 the 1aile/ no/eL "#S$ re*overs the @o.rnal o1 the 1aile/ no/e2
+he 1en*ing 6rogra0 AfencedC /eter0ines 1ro0 the *l.ster *on1ig.ration 1ile 8hi*h 1en*ing
0etho/ to .se2 +8o ,ey ele0ents in the *l.ster *on1ig.ration 1ile /e1ine a 1en*ing 0etho/
1en*ing agent an/ 1en*ing /evi*e2 +he 1en*ing 6rogra0 0a,es a *all to a 1en*ing agent
s6e*i1ie/ in the *l.ster *on1ig.ration 1ile2 +he 1en*ing agent4 in t.rn4 1en*es the no/e via a
1en*ing /evi*e2 ;hen 1en*ing is *o06lete4 the 1en*ing 6rogra0 noti1ies the *l.ster 0anager2
+he 3igh Availa>ility A//-(n 6rovi/es a variety o1 1en*ing 0etho/s
Po8er 1en*ing - A 1en*ing 0etho/ that .ses a 6o8er *ontroller to 6o8er o11 an
ino6era>le no/e
Storage 1en*ing - 9n*l./es 1en*ing 0etho/s that /isa>le the &i>re Channel 6ort that
*onne*ts storage to an ino6era>le no/e2 SCS9-% 6ersistent reservations are another
*o00only .se/ storage 1en*ing 0etho/ in 8hi*h a**ess to a *o00on share/ storage
/evi*e *an >e revo,e/ to an ino6era>le no/e2
Syste0s 0anage0ent 1en*ing - &en*ing 0etho/s that /isa>le 9B( or 6o8er to an
ino6era>le no/e2 5'a06les in*l./e 9):
O
)la/eCenter4 Dell
O
DRACB:C4 3P
O
9-(4
9P:94 an/ 9): RSA 992
re!arh)!eedbak*redhat%o( 2 """%redhat%o(
'%0%0%$ 3PM3
+he 9ntelligent Plat1or0 :anage0ent 9nter1a*e A9P:9C is a stan/ar/iHe/ *o06.ter inter1a*e
that allo8s a/0inistrators to re0otely 0anage a syste02 Centere/ aro.n/ a >ase>oar/
0anage0ent *ontroller A):CC4 9P:9 s.66orts 1.n*tions to a**ess the syste0 )9(S4 /is6lay
event logs4 6o8er on4 6o8er o11 an/ 6o8er *y*le a syste02
+his re1eren*e ar*hite*t.re .ses 9P:9 to 1en*e 1a.lty *l.ster no/es a*ross the 6.>li* net8or,
thro.gh the fence_ipmilan agent2
"""%redhat%o( 6 re!arh)!eedbak*redhat%o(
2.3.4 Failover Domains
A failover domain is an or/ere/ s.>set o1 *l.ster 0e0>ers to 8hi*h a servi*e 0ay >e >o.n/2
#ailover domains4 8hile .se1.l 1or *l.ster *.sto0iHation4 are not reI.ire/ 1or o6eration2
+he 1ollo8ing is a list o1 se0anti*s governing the o6tions as to ho8 the /i11erent *on1ig.ration
o6tions a11e*t the >ehavior o1 a failover domain2
&rdering4 restriction4 an/ nofail'ac( are 1lags that *an >e *o0>ine/ in vario.s 8ays Ae2g24
ordered)restricted4 unordered)unrestricted4 et*2C2 +hese *o0>inations a11e*t >oth 8here
servi*es start a1ter initial I.or.0 1or0ation an/ 8hi*h *l.ster 0e0>ers ta,e over servi*es in
the event that the servi*e has 1aile/2
Pre1erre/ no/e or 6re1erre/ 0e0>er - +he preferred node is the 0e0>er /esignate/ to
r.n a given servi*e i1 the 0e0>er is online2 +his >ehavior *an >e e0.late/ >y
s6e*i1ying an .nor/ere/4 .nrestri*te/ failover domain o1 e'a*tly one 0e0>er2
Restri*te/ /o0ain - Servi*es >o.n/ to the /o0ain 0ay only r.n on *l.ster 0e0>ers
8hi*h are also 0e0>ers o1 the failover domain2 91 no 0e0>ers o1 the failover domain
are availa>le4 the servi*e is 6la*e/ in the stopped state2 9n a *l.ster 8ith several
0e0>ers4 .sing a restri*te/ failover domain *an ease *on1ig.ration o1 a *l.ster servi*e
As.*h as httpdC4 8hi*h reI.ires i/enti*al *on1ig.ration on all 0e0>ers that r.n the
servi*e2 9nstea/ o1 setting .6 the entire *l.ster to r.n the *l.ster servi*e4 set .6 only
the 0e0>ers in the restri*te/ failover domain that are asso*iate/ 8ith the *l.ster
servi*e2
Unrestri*te/ /o0ain - +he /e1a.lt >ehavior2 Servi*es >o.n/ to this /o0ain 0ay r.n on
all *l.ster 0e0>ers4 >.t r.n on a 0e0>er o1 the /o0ain 8henever one is availa>le2 91 a
servi*e is r.nning o.tsi/e o1 the /o0ain an/ a 0e0>er o1 the /o0ain *o0es online4
the servi*e 0igrates to that 0e0>er2
(r/ere/ /o0ain - +he or/er s6e*i1ie/ in the *on1ig.ration /i*tates the or/er o1
6re1eren*e o1 0e0>ers 8ithin the /o0ain2 +he highest-ran,ing online 0e0>er o1 the
/o0ain hosts the servi*e 2 91 0e0>er A has a higher-ran, than 0e0>er )4 the servi*e
relo*ates to A Ai1 *.rrently r.nning on )C 8hen A transitions 1ro0 o11line to online2
Unor/ere/ /o0ain - +he /e1a.lt >ehavior2 :e0>ers o1 the /o0ain have no or/er o1
6re1eren*eL any 0e0>er 0ay r.n the servi*e2 9n an .nor/ere/ /o0ain4 servi*es
al8ays 0igrate to 0e0>ers o1 their failover domain 8henever 6ossi>le2
&ail>a*, - Servi*es on 0e0>ers o1 an or/ere/ failover domain sho.l/ relo*ate >a*, to
the original no/e on*e the 1ail.re is a//resse/2 &ail>a*, *an also >e /isa>le/ to
6revent servi*e 1ro0 1ailing over >a*, an/ 1orth >et8een 1reI.ently 1ailing no/es2
re!arh)!eedbak*redhat%o( 4 """%redhat%o(
2.3.5 Conga
Conga is an agentBserver ar*hite*t.re 1or the re0ote a/0inistration o1 *l.ster no/es2 +he
agent *o06onent is *alle/ ricci an/ the server *o06onent is *alle/ luci2 (ne luci
0anage0ent server *an *o00.ni*ate 8ith ricci agents installe/ on 0.lti6le *l.ster no/es2
;hen a syste0 is a//e/ to a luci 0anage0ent server4 a.thenti*ation is only /one the 1irst
ti0e2 No a.thenti*ation is ne*essary a1ter8ar/s2 +he luci 0anage0ent inter1a*e allo8s
a/0inistrators to *on1ig.re an/ 0anage *l.ster no/es2 Co00.ni*ations >et8een luci an/
ricci is /one via <:- over SS-2
'%0%2%$ Lui
Luci 6rovi/es a 8e>->ase/ gra6hi*al .ser inter1a*e that hel6s vis.ally a/0inister the no/es
in a *l.ster4 0anage 1en*e /evi*es4 1ailover /o0ains4 reso.r*es4 servi*e gro.6s an/ other
*l.ster attri>.tes2
Luci *an *reate a *l.ster 1ro0 s*rat*h or i06ort an e'isting *l.ster2 ;hen an e'isting *l.ster
is i06orte/4 all that is reI.ire/ is net8or, a**ess to a single no/e that is a 0e0>er o1 the
*l.ster2 Luci 6arses the *etc*cluster*cluster.conf 1ile4 lo*ates the other no/es in the *l.ster
an/ then i06orts the02 Luci *an 0anage 0.lti6le *l.sters as 8ell2
So0e o1 the i06rove0ents in the ne8 version o1 luci in*l./e a re/esigne/ 0anage0ent
inter1a*e an/ enhan*e/ logging *a6a>ilities2 9n*l./e/ in the ne8 glo>al settings 1or logging
are /ae0on s6e*i1i* log settings 1or rgmanager4 qdis(d4 fenced4 corosync4 groupd2
9n the *on1ig.ration /es*ri>e/ 8ithin this re1eren*e ar*hite*t.re4 luci is r.n on a Re/ 3at
5nter6rise -in.' 6 virt.al 0a*hine2 Luci *an >e installe/ on in/ivi/.al *l.ster no/es >.t 1or
availa>ility 6.r6oses4 it is 6re1era>le to 0anage *l.ster no/es 8ith luci on a non-0e0>er
*l.ster no/e2 Luci reI.ires 6ort 808$ to >e o6en 1or *o00.ni*ations to the ricci agents on
all *l.ster 0e0>er no/es2
'%0%2%' Rii
Ricci is the *l.ster 0anage0ent an/ *on1ig.ration /ae0on that r.ns on the *l.ster no/es2
;hen ricci is installe/ it *reates a .ser a**o.nt *alle/ ricci an/ a 6ass8or/ is set 1or the
a**o.nt2 All ricci a**o.nts 0.st >e *on1ig.re/ 8ith the sa0e 6ass8or/ a*ross all *l.ster
no/es to allo8 a.thenti*ation 8ith the luci 0anage0ent server2 +he ricci /ae0on
reI.ires 6ort 11111 to >e o6en 1or >oth t*6 an/ ./6 tra11i*2
"""%redhat%o( 5 re!arh)!eedbak*redhat%o(
2.3.6 CCS
+he Cl.ster Con1ig.ration Syste0 ACCSC 8as intro/.*e/ in Re/ 3at 5nter6rise -in.' 6212
CCS 6rovi/es a 6o8er1.l 8ay o1 0anaging a Re/ 3at 5nter6rise -in.' *l.ster 1ro0 the
*o00an/ line2 CCS allo8s an a/0inistrator to *reate4 0o/i1y an/ vie8 *l.ster *on1ig.rations
1ro0 a re0ote no/e thro.gh ricci or on a lo*al 1ile syste02 CCS has a ro>.st 0an 6age
/etailing all o1 the o6tions >.t the ones 0ost *o00only .se/ are /es*ri>e/ in 6able '%0%6)$.
7o((on 77S S"ithes
S"ithes /untion
--host Re0ote no/e to r.n **s a*tion on
--*reate*l.ster Create a ne8 *l.ster *on1ig.ration
--a//no/e A// a 1en*e instan*e to the *l.ster
--a//0etho/ A// a 1en*e 0etho/ to the *l.ster
--a//1en*e/ev A// a 1en*e /evi*e to the *l.ster
--a//1en*einst A// a 1en*e instan*e to the *l.ster
--a//1ailover/o0ain A// a 1ailover /o0ain to the *l.ster
--a//1ailover/o0ainno/e A// a 1ailover /o0ain no/e to the *l.ster
--a//reso.r*e A// a reso.r*e to the *l.ster
--a//servi*e A// a servi*e to the *l.ster
--a//s.>servi*e A// a s.>servi*e to the *l.ster
--syn* --a*tivate Syn*hroniHe an/ a*tivate the *l.ster
*on1ig.ration 1ile a*ross all no/es
--*he*,*on1 Veri1y all no/es have the sa0e *l.ster2*on1
--startall Start *l.ster servi*es on all no/es
--sto6all Sto6 *l.ster servi*es on all no/es
Table 2.3.6-1: Common CCS Switches
Cl.ster *on1ig.ration /etails are store/ as <:- entries in the 1ile *etc*cluster*cluster.conf2 +o
avoi/ *on1ig.ration errors4 *hanges to this 1ile sho.l/ al8ays >e /one thro.gh the ccs .tility2
CCS a.thenti*ates 8ith ricci agents >y .sing a.to0ati*ally generate/ *erti1i*ate 1iles in
+*.ccs*. +hese 1iles allo8 CCS to *o00.ni*ate 8ith ricci se*.rely over Se*.re So*,et
-ayer ASS-C en*ry6te/ lin,s2 +o *o00.ni*ate 8ith ricci the 6ass8or/ 1or the ricci agent
on ea*h *l.ster no/e 0.st >e ,no8n >y the a/0inistrator as CCS 6ro06ts 1or the 6ass8or/
at the 1irst *onne*tion2
re!arh)!eedbak*redhat%o( 8 """%redhat%o(
2.$ HA"L%& %olu'e
+he .n/erlying storage 1or the 8e> servi*e .tiliHes a highly-availa>le -V: A3A--V:C vol.0e2
3A--V: vol.0es 6er0it *l.ster servi*e /ata to >e share/ an/ a*tivate/ e'*l.sively a0ong
highly-availa>le no/es 1or the least *ost an/ highest 6ossi>le 6er1or0an*e2 3A--V: vol.0es
are i/eal 1or a*tiveB6assive AfailoverC *l.ster *on1ig.rations 8here the no/es r.nning a *l.ster
servi*e reI.ire e'*l.sive a**ess to a storage vol.0e2 ;hen the no/e r.nning the *l.ster
servi*e is 1aile/-over4 the 3A--V: vol.0e is 0o.nte/ >y the ne't no/e allo*ate/ to res.0e
the servi*e2
+he 3A--V: vol.0e 8ithin this re1eren*e ar*hite*t.re is *reate/ on a &i>re Channel storage
array >.t other share/ storage te*hnologies Ae.g. iSCSIC 0ay >e .se/2
2.4.1 Logical Volume Manager
Vol.0e 0anagers *reate a layer o1 a>stra*tion >et8een >et8een 6hysi*al storage >y
6resenting logi*al vol.0es that *an >e 1le'i>ly 0anage/ 8ith little to no i06a*t on the
a66li*ations or servi*es a**essing the02 -ogi*al vol.0es *an >e in*rease/ in siHe4 the
.n/erlying storage relo*ate/ to another 6hysi*al /evi*e 8itho.t nee/ to .n0o.nt the 1ile
syste02
+he ar*hite*t.re o1 -V: *onsists o1 three *o06onents
Physi*al Vol.0e APVC
Vol.0e =ro.6 AV=C
-ogi*al Vol.0e A-VC
Physi*al vol.0es APVC are the .n/erlying 6hysi*al storage F i2e2 a >lo*, /evi*e s.*h as a
8hole /is, or 6artition2 A vol.0e gro.6 AV=C is the *o0>ination o1 one or 0ore 6hysi*al
vol.0es2 (n*e a vol.0e gro.6 has >een *reate/4 logi*al vol.0es A-VC *an >e *reate/ 1ro0 it
8ith ea*h logi*al vol.0e 1or0atte/ an/ 0o.nte/ si0ilar to a 6hysi*al /is,2
+he 3A--V: vol.0e 8ithin this re1eren*e ar*hite*t.re *onsists o1 one 6hysi*al vol.0e APVC
that is a 0e0>er o1 a vol.0e gro.6 AV=C 1ro0 8hi*h a single logi*al vol.0e A-VC is *reate/2
+he *l.ster 8e> servi*e /ata is store/ on the logi*al vol.0e2
2.4.2 File System
,-t. is the s.**essor to the ,-t/ 1ilesyste02 ,-t. is P(S9< *o06liant an/ 6rovi/es s.66ort
1or several syste0 *alls s.*h as read0%4 1rite0% an/ see(0%2 ,-t. is the 1o.rth generation o1 the
e'ten/e/ 1ilesyste0 1a0ily an/ is the /e1a.lt 1ilesyste0 in Re/ 3at 5nter6rise -in.' 62 ,-t.
has >een o11ere/ in test 6revie8s sin*e Re/ 3at 5nter6rise -in.' #2%4 giving *.sto0ers
*on1i/en*e in its 0at.rity2 ,-t. *an rea/ an/ 8rite to ,-t$ or ,-t/ 1ilesyste0s4 >.t ,-t$ or
,-t/ *an not rea/ an/ 8rite to an ,-t. 1ilesyste02 3o8ever4 ,-t. a//s several ne8 an/
i06rove/ 1eat.res that are *o00on 8ith 0ost 0o/ern 1ilesyste0s4 s.*h as the 1ollo8ing
5'tent->ase/ 0eta/ata
Delaye/ allo*ation
Po.rnal *he*,-s.00ing
-arge storage s.66ort
"""%redhat%o( $& re!arh)!eedbak*redhat%o(
A 0ore *o06a*t an/ e11i*ient 8ay to tra*, .tiliHe/ s6a*e in a 1ilesyste0 is the .sage o1
e'ten/->ase/ 0eta/ata an/ the /elaye/ allo*ation 1eat.re2 +hese 1eat.res i06rove 1ile
syste0 6er1or0an*e an/ re/.*e the s6a*e *ons.0e/ >y 0eta/ata2 Delaye/ allo*ation allo8s
the 1ilesyste0 to 6ost6one sele*tion o1 the 6er0anent lo*ation 1or ne8ly 8ritten .ser /ata .ntil
the /ata is 1l.she/ to /is,2 +his ena>les higher 6er1or0an*e sin*e it *an allo8 1or larger4 0ore
*ontig.o.s allo*ations4 allo8ing the 1ilesyste0 to 0a,e /e*isions 8ith 0.*h >etter
in1or0ation2
A//itionally4 1ilesyste0 *he*, an/ re6air ti0e A1s*,C in ,-t. is 0.*h 1aster than in ,-t$ an/
,-t/2 So0e 1ilesyste0 re6airs have /e0onstrate/ .6 to a si'-1ol/ in*rease in 6er1or0an*e2
C.rrently4 Re/ 3atMs 0a'i0.0 s.66orte/ siHe 1or ,-t. is 16+) in >oth Re/ 3at 5nter6rise
-in.' # an/ Re/ 3at 5nter6rise -in.' 62 A66li*ation 6er1or0an*e /e6en/s on 0any
varia>lesL in a//ition to the a*t.al 1ilesyste0 *hosen4 the a66li*ation 6er1or0an*e also
/e6en/s on the s6e*i1i* 9B( 6attern the a66li*ation generates an/ the ty6e o1 server an/
storage har/8are .se/2
+his re1eren*e ar*hite*t.re .ses the ,-t. 1ilesyste0 on the 3A--V: vol.0e2
2.4.3 DM Multipath
Devi*e 0a66er 0.lti6athing AD: :.lti6athC allo8s 0.lti6le 9B( 6aths to >e *on1ig.re/
>et8een a server an/ the *onne*tion 6aths to SAN storage array vol.0es2 +he 6aths are
aggregate/ an/ 6resente/ to the server as a single /evi*e to 0a'i0iHe 6er1or0an*e an/
6rovi/e high availa>ility2 A /ae0on AmultipathdC han/les *he*,ing 1or 6ath 1ail.res an/
stat.s *hanges2
+his re1eren*e ar*hite*t.re .ses D: :.lti6ath on the *l.ster 8e> servi*e 3A--V: vol.0e2
re!arh)!eedbak*redhat%o( $$ """%redhat%o(
2.( )luster *e Servi+e
:anage/ >y rgmanager4 the *l.ster 8e> servi*e is *on1ig.re/ thro.gh the 3igh Availa>ility
A//-(n /.ring *l.ster *reation2 +he *l.ster 8e> servi*e *onsists o1 the 1ollo8ing 1o.r
reso.r*es
9P A//ress A1021621$%21#0C
-ogi*al Vol.0e :anage0ent A3A--V:C
&ile Syste0 A5't$C
S*ri6t Ahtt6/C
Client syste0s *onne*t to the 8e> servi*e via the 9P A//ress 1ro0 a 8e> >ro8ser2 &or
/e0onstration 6.r6oses4 a s*ri6t A23-4e'-StatusC is r.n 1ro0 the *ha*ha-1e'*.cgi-'in
/ire*tory Aon the 3A--V: vol.0eC that /is6lays the *ontents o1 *etc*motd4 *.rrent /ate an/
ti0e2 Details on ho8 to *on1ig.re the 3A-;e>-Stat.s s*ri6t *an >e 1o.n/ in Appendix E. HA)
Web Status Sript2
9n the event o1 a no/e 1ailover4 rgmanager 0anages the relo*ation o1 the 8e> servi*e2 D.ring
relo*ation4 rgmanager 0o.nts the 3A--V: vol.0e an/ starts the 8e> server on the no/e
res.0ing *ontrol2 +he 6ro*ess is trans6arent to *lient syste0s 8ith only a >rie1 /elay as the
8e> servi*e is relo*ate/ to the ne't *l.ster no/e2 &ro0 a *l.ster 0anage0ent 6ers6e*tive4
the *l.ster 8e> servi*e 6l.s the 3A--V: *an >e tho.ght o1 as one logi*al entity as /e6i*te/ in
/igure '%2)$. 7luster Web Servie
"""%redhat%o( $' re!arh)!eedbak*redhat%o(
Figure 2.5-1: Cluster
Web Service
3 Reference Architecture Configuration
+his se*tion 6rovi/es an overvie8 o1 the har/8are *o06onents that 8ere .se/ in the
/e6loy0ent o1 this re1eren*e ar*hite*t.re2 +he *l.ster no/es Aha-1e'54 ha-1e'$4 ha-1e'/C
8ere *on1ig.re/ on an 3P )la/eSyste0 *7000 en*los.re .sing three 3P Pro-iant )-$60* =6
)la/e servers2 +8o 10 =>Bs ethernet net8or,s 8ere *on1ig.re/ 1or .se as the 6.>li* an/
*l.ster inter*onne*t net8or,s2 +he 3P )la/e servers share e'*l.sive a**ess Aone no/e at a
ti0eC to the 3A--V: A8e> servi*eC vol.0e lo*ate/ on an 3P Storage;or,s :SA2%2$1*
1i>re*hannel storage array2 D.ring no/e 1ailover or relo*ation o1 the 8e> servi*e4 the 3A--V:
vol.0e is 0o.nte/ >y the no/e res.0ing *ontrol4 granting e'*l.sive a**ess to the vol.0e2
+he *l.ster 0anage0ent server 8as /e6loye/ as a QV: virt.al 0a*hine2 +he hy6ervisor host
8as *on1ig.re/ 8ith a >ri/ge/ net8or, to allo8 the virt.al 0a*hine to 0anage *l.ster no/es
over the *l.ster inter*onne*t2 Client a**ess to the *l.ster 8e> servi*e is over the 6.>li*
net8or, .sing a 8e> >ro8ser2
/igure 0)$. 7luster 7on!iguration /e6i*ts an overvie8 o1 the *l.ster *on1ig.ration
re!arh)!eedbak*redhat%o( $0 """%redhat%o(
Figure 3-1: Cluster Configuration
3.1 )luster &anage'ent Server
7o(ponent Detail
3ostna0e ha-1e'-mgmt
(6erating Syste0
Re/ 3at 5nter6rise -in.' 621 A6$->itC
A2262%2-1%12021#2el62'86R6$ ,ernelC
Syste0 +y6e Virt.al :a*hine AQV:C
Pro*essor 2 Core
:e0ory $ =)
Storage %0=>
Net8or, 2
Table 3.1: Cluster Management Server Configuration
3.2 )luster Server " Node 1
7o(ponent Detail
3ostna0e ha-1e'5
(6erating Syste0
Re/ 3at 5nter6rise -in.' 621 A6$->itC
A2262%2-1%12$212el62'86R6$ ,ernelC
Syste0 +y6e 3P Pro-iant )-$60* =6
Pro*essor
E.a/ So*,et4 E.a/ Core A16 *oresC
9ntel
9
<eon
9
CPU <###0 D2267=3H
:e0ory $8 =)
Storage
$ ' 1$6 =) SA+A internal /is, /rive ARA9D 1C
2 ' Elogi* E:32#62 8=> &C 3)A
1 ' 20=)
Net8or, 8 ' )roa/*o0 Net<tre0e 99 )C:#77115 <=>
Table 3.2: Cluster Noe1 Configuration
"""%redhat%o( $1 re!arh)!eedbak*redhat%o(
3.3 )luster Server " Node 2
7o(ponent Detail
3ostna0e ha-1e'$
(6erating Syste0
Re/ 3at 5nter6rise -in.' 621 A6$->itC
A2262%2-1%12$212el62'86R6$ ,ernelC
Syste0 +y6e 3P Pro-iant )-$60* =6
Pro*essor
E.a/ So*,et4 E.a/ Core A16 *oresC
9ntel
9
<eon
9
CPU <###0 D2267=3H
:e0ory $8 =)
Storage
$ ' 1$6 =) SA+A internal /is, /rive ARA9D 1C
2 ' Elogi* E:32#62 8=> &C 3)A
1 ' 20=)
Net8or, 8 ' )roa/*o0 Net<tre0e 99 )C:#77115 <=>
Table 3.3: Cluster Noe2 Configuration
3.$ )luster Server " Node 3
7o(ponent Detail
3ostna0e ha-1e'/
(6erating Syste0
Re/ 3at 5nter6rise -in.' 621 A6$->itC
A2262%2-1%12$212el62'86R6$ ,ernelC
Syste0 +y6e 3P Pro-iant )-$60* =6
Pro*essor
E.a/ So*,et4 E.a/ Core A16 *oresC
9ntel
9
<eon
9
CPU <###0 D2267=3H
:e0ory $8 =)
Storage
$ ' 1$6 =) SA+A internal /is, /rive ARA9D 1C
2 ' Elogi* E:32#62 8=> &C 3)A
1 ' 20=)
Net8or, 8 ' )roa/*o0 Net<tre0e 99 )C:#77115 <=>
Table 3.!: Cluster Noe3 Configuration
re!arh)!eedbak*redhat%o( $2 """%redhat%o(
3.( ,ire )hannel Storage Arra!
7o(ponent Detail
3ostna0e ra-msa$6
Syste0 +y6e
3P Storage;or,s :SA2%2$1*
A1 ' 3P :SA70 e'6ansion shel1C
Controllers
CPU +y6e +.rion :+%2 1800:3H
Ca*he 1=)
2 ' 3ost Ports
&ir08are
Storage Controller Co/e Version :112R1$
:e0ory Controller &P=A Co/e Version &%00R22
Storage Controller -oa/er Co/e Version 1"200"
:anage0ent Controller Co/e Version ;$$1R%#
:anage0ent Controller -oa/er Co/e Version 12201#
5'6an/er Controller Co/e Version 1112
CP-D Co/e Version 8
3ar/8are Version #6
Physi*al Drives $8 ' 1$6=) SAS /rives A2$ en*los.re4 2$ e'6ansion shel1C
-ogi*al Drives $ ' 12% +) Virt.al Dis,s A12 /is,4 RA9D 6C
Table 3.": Cluster Storage #rra$ Configuration
"""%redhat%o( $6 re!arh)!eedbak*redhat%o(
4 Cluster Deployment
$.1 -eplo!'ent .ask ,low
/igure 1%$)$. 7luster Deploy(ent 6ask /lo" 6rovi/es an overvie8 o1 the or/er in 8hi*h the
/e6loy0ent o1 the *l.ster 0anage0ent server4 *l.ster no/es an/ *l.ster *reation tas,s are
6er1or0e/
Appendix :. Deploy(ent 7heklists 6rovi/es a /etaile/ list o1 ste6s to 1ollo8 1or /e6loying
a 3ighly Availa>le ;e> Servi*e on Re/ 3at 5nter6rise -in.' 62
re!arh)!eedbak*redhat%o( $4 """%redhat%o(
Figure 4.1-1: Cluster
Deployent !as" Flo#
$.2 -eplo! &anage'ent Server
Prior to *reating the *l.ster4 the 1ollo8ing series o1 ste6s is 6er1or0e/ to /e6loy the luci
0anage0ent server
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s
Con1ig.re &ire8all
9nstall Cl.ster :anage0ent So1t8are A2igh 3vaila'ility !anagement 3dd-&nC
+he ne't se*tions /es*ri>e ho8 to 6er1or0 the /e6loy0ent ste6s in /etail2
4.2.1 Install Red Hat Enterprise Linux 6
+he Re/ 3at 5nter6rise -in.' 6 9nstallation =.i/e
1
6rovi/es *o06lete /etails on the
installation o1 Re/ 3at 5nter6rise -in.' 6 1or 9ntel4 A:D4 an/ 9): ar*hite*t.res2 +he *l.ster
0anage0ent server *an >e /e6loye/ as either a 6hysi*al or virt.al 0a*hine2 A QV: virt.al
0a*hine 8as .se/ 1or this re1eren*e ar*hite*t.re2 Using the virt-install .tility4 the virt.al
0a*hine 8as 6rovisione/ 8ith 2 6ro*essors4 $ =) 0e0ory an/ a >ri/ge/ net8or, inter1a*e
as 1ollo8s
# virt-install --name ha-web-mgmt --ram 4096 --vcpus=2 --cpuset=auto
--os-t!pe=linu" --os-variant=rhel6 --accelerate
--networ#=bridge$vm-bridge --vnc
--dis# path=%dev%vm_storage_vg%ha-web-mgmt --p"e
A 6revio.sly *reate/ %0 =) logi*al vol.0e Aha-1e'-mgmtC 8as *reate/ to hol/ the virt.al
0a*hine itsel1 on the hy6ervisor host2 &or /etails on the 6rovisioning o1 QV: virt.al 0a*hines4
6lease *ons.lt the Re/ 3at 5nter6rise -in.' 6 Virt.aliHation =.i/e
%
2
Regar/less o1 8hether a 6hysi*al or virt.al 0a*hine is .se/4 a Re/ 3at 5nter6rise -in.' 6
installation involves the 1ollo8ing series o1 stages
12 9nstall Re/ 3at 5nter6rise -in.' 6
22 &irst)oot
%2 A66ly .6/ates
A1ter the o6erating syste0 has >een installe/ the syste0 re>oots an/ enters 8hat is re1erre/
to as #irst7oot2 D.ring #irst7oot4 a/0inistrators are g.i/e/ thro.gh the 6ro*ess o1 setting
/ate an/ ti0e4 *on1ig.ring so1t8are .6/ates4 registering 8ith Re/ 3at Net8or, AR3NC4 initial
.ser a**o.nt *reation an/ o6tions 1or Qernel A8dumpC *rash /.06s2 +he syste0 then re>oots
to a*tivate the *hanges2 A1ter login has >een *o06lete/ .n/er the ne8ly *reate/ .ser
a**o.nt4 .6/ates to the syste0 are a66lie/ to >ring the Re/ 3at 5nter6rise -in.' 6 server to
the latest versions o1 all so1t8are2
+he Re/ 3at 5nter6rise -in.' 6 9nstallation =.i/e
1
6rovi/es *o06lete instr.*tions on ea*h o1
these stages2 Please *ons.lt the g.i/e 1or 1.rther installation /etails2
"""%redhat%o( $5 re!arh)!eedbak*redhat%o(
4.2.2 Configure Networks
+he 0anage0ent server is *on1ig.re/ to 6rovi/e a**ess to all *l.ster no/es a*ross >oth the
6.>li* an/ *l.ster inter*onne*t A6rivateC net8or,s2 +he 6.>li* net8or, A56.59.5./.6C is
*on1ig.re/ on the eth0 inter1a*e an/ the *l.ster inter*onne*t A5:$.59;.5.6C is *on1ig.re/ on
the eth& inter1a*e2 Stati* 9P a//ressing is .se/ thro.gho.t the *l.ster *on1ig.ration2
12 Veri1y that Net8or,:anager is /isa>le/ on start.6 to 6revent *on1li*ts 8ith the 3igh
Availa>ility A//-(n *l.ster servi*es
# ch#config 'etwor#(anager off
# ch#config 'etwor#(anager --list
NetworkManager 0:off 1:off 2:off 3:off 4:off 5:off 6:off
22 Create the inter1a*e 1ile 1or the 6.>li* inter1a*e an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth6
DEVICE="eth0"
H!DD"="52:54:00:E2:#E:#5"
$%%&'"%&%=(tat)*
%N$%%&="+e("
NM,C%N&"%--ED="no"
I'!DD"=10.16.143.154
NE&M!/0=255.255.255.0
1!&E!2=10.16.143.254
%2 Create the inter1a*e 1ile 1or the *l.ster inter*onne*t an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth/
DEVICE="eth3"
H!DD"="52:54:00:E3:1!:33"
$%%&'"%&%=(tat)*
%N$%%&="+e("
NM,C%N&"%--ED="no"
I'!DD"=142.16#.1.154
NE&M!/0=255.255.255.0
$2 Restart the net8or, servi*es to a*tivate the *hanges
) service networ# restart
#2 U6/ate *etc*hosts
5/it the *etc*hosts 1ile to in*l./e the 9P a//resses4 hostna0eBaliases o1 all *l.ster no/e
an/ 0anage0ent server inter1a*es
re!arh)!eedbak*redhat%o( $8 """%redhat%o(
127.0.0.1 localhost localhost.localdomain
#
#----------------#
# Cluster Nodes: #
#----------------#
#
10.16.143.151 ha-web1 ha-web1.cloud.lab.en.bos.redhat.com
1!2.16".1.151 ha-web1-ci ha-web1-ci.cloud.lab.en.bos.redhat.com
10.16.143.152 ha-web2 ha-web2.cloud.lab.en.bos.redhat.com
1!2.16".1.152 ha-web2-ci ha-web2-ci.cloud.lab.en.bos.redhat.com
10.16.143.153 ha-web3 ha-web3.cloud.lab.en.bos.redhat.com
1!2.16".1.153 ha-web3-ci ha-web3-ci.cloud.lab.en.bos.redhat.com
#
#------------------#
# #anaement Node: #
#------------------#
#
10.16.143.154 ha-web-mmt ha-web-mmt.cloud.lab.en.bos.redhat.com
1!2.16".1.154 ha-web-mmt-ci ha-web-mmt-ci.cloud.lab.en.bos.redhat.com
4.2.3 Configure Firewall
)e1ore the *l.ster *an >e *reate/4 the 1ire8all 6orts 0.st >e *on1ig.re/ to allo8 a**ess to the
*l.ster net8or, /ae0ons2 +he s6e*i1i* 6orts reI.iring a**ess
Port ;u(ber Protool 7o(ponent
808$ +CP lui AConga A/0inistration 9nter1a*eC
Table !.1.3: Cluster Noe %orts
&ire8all a**ess *an >e *on1ig.re/ 8ith either the s!stem-configuration-firewall
gra6hi*al tool or the iptables *o00an/ line .tility2 Using iptables$ *on1ig.re the 1ire8all
on ea*h o1 the three *l.ster no/es F ha-1e'54 ha-1e'$4 ha-1e'/ as 1ollo8s
12 Create a >a*,.6 *o6y o1 the *.rrent iptables *on1ig.ration 1ile
# cp %etc%s!sconfig%iptables-config %etc%s!sconfig%iptables-config*orig
22 Dis6lay the *.rrent iptables *on1ig.ration
# iptables --list --line-numbers --numeric --verbose
Cha)n IN'5& 67o8)*+ !CCE'& 544 7a*ket(9 4#422 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
Cha)n ?%"!"D 67o8)*+ !CCE'& 0 7a*ket(9 0 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
Cha)n %5&'5& 67o8)*+ !CCE'& 334 7a*ket(9 3#4#4 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
"""%redhat%o( '& re!arh)!eedbak*redhat%o(
%2 Create a ne8 iptables *hain *alle/ ha-web-cluster an/ insert it into the +',-.
*hain
# iptables --new-chain ha-web-cluster
# iptables --insert +',-. --/ump ha-web-cluster
$2 A// the r.les 1or the *l.ster *o06onents to the ha-web-cluster *hain
# iptables --append ha-web-cluster --proto tcp 0destination-port 1014
--/ump 2334,.
#2 Dis6lay the ne8 iptables *on1ig.ration
# iptables --list --line-numbers --numeric --verbose
Cha)n IN'5& 67o8)*+ !CCE'& 33 7a*ket(9 2#44 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
1 340 23206 ha@we:@*8<(ter a88 @@ A A 0.0.0.0B0 0.0.0.0B0
Cha)n ?%"!"D 67o8)*+ !CCE'& 0 7a*ket(9 0 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
Cha)n %5&'5& 67o8)*+ !CCE'& 5 7a*ket(9 400 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
Cha)n ha@we:@*8<(ter 61 referen*e(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
1 0 0 !CCE'& t*7 @@ A A 0.0.0.0B0 0.0.0.0B0 t*7 >7t:#0#4
62 Save the ne8 r.les an/ veri1y iptables is a*tivate/ on syste0 >oot
# service iptables save
)7ta:8e(: /aC)ng f)rewa88 r<8e( to Bet*B(+(*onf)gB)7ta:8e(:D %0 E
# ch#config iptables on
4.2.4 Install Cluster Management Software
12 -og onto Re/ 3at Net8or, AR3NC an/ assign the 1ollo8ing *hannels to the *l.ster
0anage0ent server Aha-1e'-mgmtC
R35- Server 3igh Availa>ility Av2 6 1or 6$->it '86R6$C
R3N +ools 1or R35- Av2 6 1or 6$->it '86R6$C
22 Con1ir0 the *hannels are availa>le on the *l.ster 0anage0ent server
# !um repolist
-oa>e> 78<g)n(: 7ro><*t@)>9 rhn78<g)n9 (<:(*r)7t)on@=anager
57>at)ng "e> Hat re7o()tor)e(.
re7o )> re7o na=e
rhe8@F#6,64@(erCer@6 "e> Hat Enter7r)(e -)n<F /erCer 6C. 6 for 64@:)t
F#6,64;
rhe8@F#6,64@(erCer@ha@6 "HE- /erCer H)gh !Ca)8a:)8)t+ 6C. 6 for 64@:)t
F#6,64;
rhn@too8(@rhe8@F#6,64@(erCer@6 "HN &oo8( for "HE- 6C. 6 for 64@:)t F#6,64;
re!arh)!eedbak*redhat%o( '$ """%redhat%o(
%2 9nstall the *l.ster 0anage0ent so1t8are on ha-1e'-mgmt
# !um groupinstall 56igh 2vailabilit! (anagement7
+his *o06letes the /e6loy0ent o1 the *l.ster 0anage0ent server2 Pro*ee/ to Setion 1%0
Deploy 7luster ;odes to >egin the 6ro*ess o1 /e6loying the three *l.ster no/es2
"""%redhat%o( '' re!arh)!eedbak*redhat%o(
$.3 -eplo! )luster Nodes
Prior to *reating the *l.ster4 ea*h *l.ster no/e is /e6loye/ >y 6er1or0ing the 1ollo8ing series
o1 ste6s on ea*h *l.ster no/e
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s an/ )on/ing
Con1ig.re &ire8all
9nstall Cl.ster No/e So1t8are AJ3igh Availa>ilityK A//-(nC
Con1ig.re Storage
Con1ig.re ;e> Server So1t8are
+he ne't se*tions /es*ri>e ho8 to 6er1or0 the /e6loy0ent ste6s in /etail2
4.3.1 Install Red Hat Enterprise Linux 6
+he installation o1 Re/ 3at 5nter6rise -in.' 6 on ea*h o1 the three *l.ster no/es is 6er1or0e/
.sing a Re/ 3at Satellite server2 Details on ho8 the Satellite server 8as *on1ig.re/ *an >e
1o.n/ in Appendix <. Red Hat Enterprise Linux 6 = Satellite 7on!iguration Details2 -o*al
0e/ia *an >e .se/ in lie. o1 a Satellite server /e6loy0ent2
(n*e the Satellite server has >een *on1ig.re/4 6er1or0 the 1ollo8ing ste6s to install Re/ 3at
5nter6rise -in.' 6 on ea*h *l.ster no/e
12 )oot the no/e .sing the Pre>oot e<e*.tion 5nviron0ent AP<5C2 :any servers are
*on1ig.re/ to P<5 >oot >y /e1a.lt2 Cons.lt the server ven/or /o*.0entation 1or
*on1ig.ration /etails2
22 A1ter the Re/ 3at 5nter6rise -in.' 6 installation has *o06lete/4 veri1y the re6ositories
on the Satellite server are availa>le
# yu repolist
-oa>e> 78<g)n(: 7ro><*t@)>9 rhn78<g)n9 (<:(*r)7t)on@=anager
57>at)ng "e> Hat re7o()tor)e(.
re7o )> re7o na=e
rhe8@F#6,64@(erCer@6 "e> Hat Enter7r)(e -)n<F /erCer 6C. 6 for 64@:)t F#6,64;
rhe8@F#6,64@(erCer@ha@6 "HE- /erCer H)gh !Ca)8a:)8)t+ 6C. 6 for 64@:)t F#6,64;
rhn-tools-rhel-%"6&64-ser'er-6 ()N *ools +or (),- .'. 6 +or 64-bit %"6&64/
%2 U6/ate ea*h no/e to ta,e in the latest 6at*hes an/ se*.rity .6/ates
# !um update
5'*e6t 1or the .se o1 P<5 >oot the installation stages are si0iliar to those as /es*ri>e/ earlier
1or the 0anage0ent server in Setion 1%'%$ 3nstall Red Hat Enterprise Linux 62 &ollo8 the
ste6s a>ove an/ *ons.lt the Re/ 3at 5nter6rise -in.' 6 9nstallation =.i/e
1
1or 1.rther
installation /etails2
re!arh)!eedbak*redhat%o( '0 """%redhat%o(
4.3.2 Configure Networks and Bonding
+he *l.ster no/es are *on1ig.re/ to 6rovi/e a**ess to all *l.ster no/es a*ross >oth the 6.>li*
an/ *l.ster inter*onne*t AprivateC net8or,s2 +he 6.>li* net8or, A56.59.5./.6C is *on1ig.re/ on
the eth0 inter1a*e an/ >on/e/ to the eth8 inter1a*e 1or re/.n/an*y2 +he *l.ster inter*onne*t
A5:$.59;.5.6C is *on1ig.re/ on the eth2 inter1a*e an/ >on/e/ to the eth& inter1a*e 1or
re/.n/an*y2 Stati* 9P a//ressing is .se/ thro.gho.t the *l.ster *on1ig.ration2
12 Veri1y that Net8or,:anager is /isa>le/ on start.6 to 6revent *on1li*ts 8ith the 3igh
Availa>ility A//-(n *l.ster servi*es
# ch#config 'etwor#(anager off
# ch#config 'etwor#(anager --list
NetworkManager 0:off 1:off 2:off 3:off 4:off 5:off 6:off
22 Create >on/ *on1ig.ration 1iles 1or the 6.>li* an/ *l.ster inter*onne*t net8or,s
# echo 9alias bond0 bonding9 :: %etc%modprobe*d%bonding*conf
# echo 9alias bond8 bonding9 :: %etc%modprobe*d%bonding*conf
%2 Create the >on/ inter1a*e 1ile 1or the 6.>li* net8or, an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-'ond6
DEVICE=:on>0
I'!DD"=10.16.143.151
NE&M!/0=255.255.255.0
1!&E!2=10.16.143.254
5/E"C&-=no
$%%&'"%&%=(tat)*
%N$%%&=+e(
$%NDIN1,%'&/="=o>e=0 =))=on=100"
$2 Create the >on/ inter1a*e 1ile 1or the *l.ster inter*onne*t net8or, an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-'ond5
DEVICE=bond1
IPADDR=192.168.1.151
NETMASK=255.255.255.0
USERCTL=no
BOOTPROTO=!"!#$
ONBOOT=%&
BONDIN'(OPTS=)*od&=1)
#2 :o/i1y the inter1a*e 1ile 1or the 1irst 6.>li* inter1a*e an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth6
DEVICE=eth0
$%%&'"%&%=(tat)*
H!DD"=00:13:!4:33:24:3C
%N$%%&=+e(
M!/&E"=:on>0
/-!VE=+e(
5/E"C&-=no
"""%redhat%o( '1 re!arh)!eedbak*redhat%o(
62 Create the inter1a*e 1ile 1or the se*on/ 6.>li* inter1a*e an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth5
DEVICE=eth1
$%%&'"%&%=(tat)*
H!DD"=00:13:!4:33:24:3E
%N$%%&=+e(
M!/&E"=:on>0
/-!VE=+e(
5/E"C&-=no
72 :o/i1y the inter1a*e 1ile 1or the 1irst *l.ster inter*onne*t an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth$
DEVICE=eth2
$%%&'"%&%=(tat)*
H!DD"=00:13:!4:33:24:40
%N$%%&=+e(
M!/&E"=:on>1
/-!VE=+e(
5/E"C&-=no
82 Create the inter1a*e 1ile 1or the se*on/ *l.ster inter*onne*t an/ save the 1ile as
*etc*sysconfig*net1or(-scripts*ifcfg-eth/
DEVICE=eth3
$%%&'"%&%=(tat)*
H!DD"=00:13:!4:33:24:42
%N$%%&=+e(
M!/&E"=:on>1
/-!VE=+e(
5/E"C&-=no
"2 Restart the net8or,ing servi*e
# service networ# restart
102 Veri1y the 6.>li* >on/ is r.nning
# cat %proc%net%bonding%bond0
Ethernet Channe8 $on>)ng Dr)Cer: C3.6.0 6/e7te=:er 269 2004;
$on>)ng Mo>e: 8oa> :a8an*)ng 6ro<n>@ro:)n;
MII /tat<(: <7
MII 'o88)ng InterCa8 6=(;: 100
57 De8a+ 6=(;: 0
Down De8a+ 6=(;: 0
/8aCe Interfa*e: eth0
MII /tat<(: <7
-)nk ?a)8<re Co<nt: 0
'er=anent H a>>r: 00:13:a4:33:24:3*
/8aCe G<e<e ID: 0
/8aCe Interfa*e: eth1
re!arh)!eedbak*redhat%o( '2 """%redhat%o(
MII /tat<(: <7
-)nk ?a)8<re Co<nt: 0
'er=anent H a>>r: 00:13:a4:33:24:3e
/8aCe G<e<e ID: 0
112 Veri1y the *l.ster inter*onne*t >on/ is r.nning
# cat %proc%net%bonding%bond8
Ethernet Channe8 $on>)ng Dr)Cer: C3.6.0 6/e7te=:er 269 2004;
$on>)ng Mo>e: fa<8t@to8eran*e 6a*t)Ce@:a*k<7;
'r)=ar+ /8aCe: None
C<rrent8+ !*t)Ce /8aCe: eth2
MII /tat<(: <7
MII 'o88)ng InterCa8 6=(;: 0
57 De8a+ 6=(;: 0
Down De8a+ 6=(;: 0
/8aCe Interfa*e: eth2
MII /tat<(: <7
-)nk ?a)8<re Co<nt: 0
'er=anent H a>>r: 00:13:a4:33:24:40
/8aCe G<e<e ID: 0
/8aCe Interfa*e: eth3
MII /tat<(: <7
-)nk ?a)8<re Co<nt: 0
'er=anent H a>>r: 00:13:a4:33:24:42
/8aCe G<e<e ID: 0
122 5/it the *etc*hosts 1ile to in*l./e the 9P a//resses4 hostna0eBaliases o1 all *l.ster
no/e an/ 0anage0ent server inter1a*es
127.0.0.1 localhost localhost.localdomain
#
#----------------#
# Cluster Nodes: #
#----------------#
#
10.16.143.151 ha-web1 ha-web1.cloud.lab.en.bos.redhat.com
1!2.16".1.151 ha-web1-ci ha-web1-ci.cloud.lab.en.bos.redhat.com
10.16.143.152 ha-web2 ha-web2.cloud.lab.en.bos.redhat.com
1!2.16".1.152 ha-web2-ci ha-web2-ci.cloud.lab.en.bos.redhat.com
10.16.143.153 ha-web3 ha-web3.cloud.lab.en.bos.redhat.com
1!2.16".1.153 ha-web3-ci ha-web3-ci.cloud.lab.en.bos.redhat.com
#
#------------------#
# #anaement Node: #
#------------------#
#
10.16.143.154 ha-web-mmt ha-web-mmt.cloud.lab.en.bos.redhat.com
1!2.16".1.154 ha-web-mmt-ci ha-web-mmt-ci.cloud.lab.en.bos.redhat.com
"""%redhat%o( '6 re!arh)!eedbak*redhat%o(
1%2 Distri>.te the 1ile to the other t8o *l.ster no/es2 &or e'a06le4 i1 the 1ile 8as initially
*reate/ on *l.ster no/e ha-1e'5 then *o6y it to the other no/es as 1ollo8s
# scp -p %etc%hosts ha-web2$%etc%hosts
# scp -p %etc%hosts ha-web&$%etc%hosts
1$2 Veri1y all 6.>li* an/ *l.ster inter*onne*t inter1a*es are 6ro6erly *on1ig.re/ an/
res6on/ing
# ping ha-web8
# ping ha-web8-ci
# ping ha-web2
# ping ha-web2-ci
# ping ha-web&
# ping ha-web&-ci
# ping ha-web-mgmt
# ping ha-web-mgmt-ci
4.3.3 Configure Firewall
)e1ore the *l.ster *an >e *reate/4 the 1ire8all 6orts 0.st >e *on1ig.re/ to allo8 a**ess to the
*l.ster net8or, /ae0ons2 +he s6e*i1i* 6orts reI.iring a**ess
Port ;u(ber Protool 7o(ponent
#$0$ UDP orosyn>(an ACl.ster :anagerC
#$0# UDP orosyn>(an ACl.ster :anagerC
11111 +CP rii ACl.ster Con1ig.rationC
11111 UDP rii ACl.ster Con1ig.rationC
2106$ +CP dl( ADistri>.te/ -o*, :anagerC
168#1 +CP (odlusterd
80 +CP httpd AA6a*heC
$$% +CP httpd AA6a*heC
Table !.2.3: Cluster Noe %orts
&ire8all a**ess *an >e *on1ig.re/ 8ith either s!stem-configuration-firewall A=U9C or
the iptables .tility2 Use iptables to *on1ig.re the 1ire8all as 6er the 1ollo8ing series o1
ste6s on ea*h o1 the three *l.ster no/es Aha-1e'54 ha-1e'$4 ha-1e'/%
12 Create a >a*,.6 *o6y o1 the *.rrent iptables *on1ig.ration 1ile
# cp %etc%s!sconfig%iptables-config %etc%s!sconfig%iptables-config*orig
22 Dis6lay the *.rrent iptables *on1ig.ration
# iptables --list --line-numbers --numeric --verbose
Cha)n IN'5& 67o8)*+ !CCE'& 544 7a*ket(9 4#422 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
re!arh)!eedbak*redhat%o( '4 """%redhat%o(
Cha)n ?%"!"D 67o8)*+ !CCE'& 0 7a*ket(9 0 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
Cha)n %5&'5& 67o8)*+ !CCE'& 334 7a*ket(9 3#4#4 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
%2 Create a ne8 iptables *hain *alle/ ha-1e'-cluster an/ insert it into the +',-.
*hain
# iptables --new-chain ha-web-cluster
# iptables --insert +',-. --/ump ha-web-cluster
$2 Create a ne8 iptables *hain *alle/ ha-web-service an/ insert it into the +',-.
*hain
# iptables --new-chain ha-web-service
# iptables --insert +',-. --/ump ha-web-service
#2 A// the r.les 1or the *l.ster *o06onents to the ha-web-cluster *hain
# iptables --append ha-web-cluster --proto udp --destination-port ;404
--/ump 2334,.
# iptables --append ha-web-cluster --proto udp --destination-port ;40;
--/ump 2334,.
# iptables --append ha-web-cluster --proto tcp --destination-port 88888
--/ump 2334,.
# iptables --append ha-web-cluster --proto udp --destination-port 88888
--/ump 2334,.
# iptables --append ha-web-cluster --proto tcp --destination-port 28064
--/ump 2334,.
# iptables --append ha-web-cluster --proto udp --destination-port 28064
--/ump 2334,.
62 A// the r.le 1or 8e> servi*e *o06onents to the ha-web-service *hain
# iptables --append ha-web-service --proto tcp --destination-port 10
--/ump 2334,.
# iptables --append ha-web-service --proto tcp --destination-port 44&
--/ump 2334,.
72 Dis6lay the ne8 iptables *on1ig.ration
# iptables --list --line-numbers --numeric --verbose
Cha)n IN'5& 67o8)*+ !CCE'& 5 7a*ket(9 404 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
1 325 2#602 ha@we:@(erC)*e a88 @@ A A 0.0.0.0B0 0.0.0.0B0
2 2244 20#0 ha@we:@*8<(ter a88 @@ A A 0.0.0.0B0 0.0.0.0B0
Cha)n ?%"!"D 67o8)*+ !CCE'& 0 7a*ket(9 0 :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e
>e(t)nat)on
Cha)n %5&'5& 67o8)*+ !CCE'& 3 7a*ket(9 50# :+te(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e
>e(t)nat)on
"""%redhat%o( '5 re!arh)!eedbak*redhat%o(
Cha)n ha@we:@*8<(ter 61 referen*e(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
1 0 0 !CCE'& <>7 @@ A A 0.0.0.0B0 0.0.0.0B0 <>7 >7t:5404
2 0 0 !CCE'& <>7 @@ A A 0.0.0.0B0 0.0.0.0B0 <>7 >7t:5405
3 0 0 !CCE'& t*7 @@ A A 0.0.0.0B0 0.0.0.0B0 t*7 >7t:11111
4 0 0 !CCE'& <>7 @@ A A 0.0.0.0B0 0.0.0.0B0 <>7 >7t:11111
5 0 0 !CCE'& t*7 @@ A A 0.0.0.0B0 0.0.0.0B0 t*7 >7t:21064
6 0 0 !CCE'& <>7 @@ A A 0.0.0.0B0 0.0.0.0B0 <>7 >7t:21064
Cha)n ha@we:@(erC)*e 61 referen*e(;
n<= 7kt( :+te( target 7rot o7t )n o<t (o<r*e >e(t)nat)on
1 0 0 !CCE'& t*7 @@ A A 0.0.0.0B0 0.0.0.0B0 t*7 >7t:#0
2 0 0 !CCE'& t*7 @@ A A 0.0.0.0B0 0.0.0.0B0 t*7 >7t:443
82 Save the ne8 r.les an/ veri1y iptables is a*tivate/ on syste0 >oot
# service iptables save
# ch#config iptables on
4.3.4 Install Cluster Node Software
9nstall the 2igh 3vaila'ility 3dd-&n so1t8are on ea*h o1 the three *l.ster no/es
# !um groupinstall 56igh 2vailabilit!7
4.3.5 Configure Storage
A vol.0e is *reate/ to hol/ the *ontents o1 the *l.ster 8e> servi*e that *an >e share/
e'*l.sively a0ong the *l.ster no/es2 +he vol.0e is *on1ig.re/ so that e'*l.sive a**ess is
given to the no/e *.rrently 6rovi/ing the 8e> servi*e2 9n the event o1 a 1ailover4 a**ess to the
vol.0e is *hange/ to >e e'*l.sive to the ne't *l.ster no/e 6rovi/ing the 8e> servi*e2 Sin*e
only one *l.ster no/e at a ti0e reI.ires a**ess4 the vol.0e is *on1ig.re/ 8ith the ogical
<olume !anager 0<!% an/ is 0anage/ >y the 2igh 3vaila'ility ogical !anager 023-<!%
agent2
+he ogical =nit >um'er 0=>% 1or the vol.0e 0.st >e 6rovisione/ an/ a**essi>le to ea*h o1
the *l.ster no/es >e1ore *ontin.ing2 Appendix 7. /ibre 7hannel Storage Provisioning
/es*ri>es ho8 the -UN .se/ 1or this re1eren*e ar*hite*t.re 8as 6rovisione/2
1%0%2%$ 7on!igure Multipathing
12 9nstall the D! !ultipath Pa*,age on ea*h *l.ster no/e
# !um install device-mapper-multipath*"16_64
22 (n the 1irst *l.ster no/e Aha-1e'5C *reate a :.lti6ath *on1ig.ration 1ile
A*etc*multipath.confC 8ith .ser 1rien/ly na0es /isa>le/ an/ the /ae0on starte/
# mpathconf --enable --user_friendl!_names n --with_multipathd !
/tart)ng =<8t)7ath> >ae=on: D %0 E
re!arh)!eedbak*redhat%o( '8 """%redhat%o(
%2 (n the 1irst *l.ster no/e Aha-1e'5C vie8 the 0.lti6ath /evi*e4 6aths an/ ;orl/ ;i/e 9D
A;;9DC
# multipathd -ll
3600*0ff000>a:3f45*e#f54>01000000 >=@1 H'9M/!2324f*
()He=141 feat<re(=I1 G<e<e,)f,no,7athI hwhan>8er=I0I w7=rw
J@K@ 7o8)*+=Iro<n>@ro:)n 0I 7r)o=50 (tat<(=a*t)Ce
L@ 0:0:#:1 (>: #:16 a*t)Ce rea>+ r<nn)ng
L@ 0:0:4:1 (>* #:32 a*t)Ce rea>+ r<nn)ng
L@ 1:0:#:1 (>> #:4# a*t)Ce rea>+ r<nn)ng
J@ 1:0:4:1 (>e #:64 a*t)Ce rea>+ r<nn)ng
# ls %dev%mapper
3600*0ff000>a:3f45*e#f54>01000000 *ontro8 =+Cg@rootCo8
$2 (n the 1irst *l.ster no/e Aha-1e'5C e/it the 1ile *etc*multipath.conf an/ a// an alias 1or
the 3A--V: vol.0e A*alle/ ha-web< .sing the ;;9D 1ro0 the 6revio.s ste6
=<8t)7ath( M
=<8t)7ath M
a8)a( ha-web
ww)> "&600c0ff000dab=f4;ce1f;4d08000000"
N
N
#2 Co6y the 1ile 1ro0 the 1irst *l.ster no/e Aha-1e'5C to the other *l.ster no/es
# scp -p %etc%multipath*conf ha-web2$%etc%multipath*conf
# scp -p %etc%multipath*conf ha-web&$%etc%multipath*conf
62 Restart multipathd on ea*h *l.ster no/e
# service multipathd restart
/to77)ng =<8t)7ath> >ae=on: D %0 E
/tart)ng =<8t)7ath> >ae=on: D %0 E
72 Veri1y the *hange on ea*h *l.ster no/e
# multipathd -ll
ha@we: 63600*0ff000>a:3f45*e#f54>01000000; >=@1 H'9M/!2324f*
()He=141 feat<re(=I1 G<e<e,)f,no,7athI hwhan>8er=I0I w7=rw
J@K@ 7o8)*+=Iro<n>@ro:)n 0I 7r)o=50 (tat<(=a*t)Ce
L@ 0:0:#:1 (>: #:16 a*t)Ce rea>+ r<nn)ng
L@ 0:0:4:1 (>* #:32 a*t)Ce rea>+ r<nn)ng
L@ 1:0:#:1 (>> #:4# a*t)Ce rea>+ r<nn)ng
J@ 1:0:4:1 (>e #:64 a*t)Ce rea>+ r<nn)ng
# ls %dev%mapper
*ontro8 =+Cg@rootCo8
82 (n ea*h *l.ster no/e *on1ig.re 0.lti6ath to start on syste0 >oot
# ch#config multipathd on
"""%redhat%o( 0& re!arh)!eedbak*redhat%o(
1%0%2%' 7reate HA)L#M #olu(e
12 5ns.re the 6ara0eter locking_type is set to a val.e o1 1 Ato 6revent -V: 0eta/ata
*orr.6tionC in the glo>al se*tion o1 the 1ile *etc*lvm*lvm.conf on all no/es
# grep 9loc#ing_t!pe9 %etc%lvm%lvm*conf > grep -v 9)9
8o*k)ng,t+7e = 1
22 Con1ig.re the Physi*al Vol.0e APVC .sing the :.lti6ath /evi*e - *dev*mapper*ha-1e'
an/ /is6lay the attri>.tes2 Per1or0 this ste6 on the 1irst *l.ster no/e Aha-1e'5C only
# pvcreate %dev%mapper%ha-web
'h+()*a8 Co8<=e "B>eCB=a77erBha@we:" (<**e((f<88+ *reate>
# pvdispla! %dev%mapper%ha-web
"B>eCB=a77erBha@we:" )( a new 7h+()*a8 Co8<=e of "1#.63 1)$"
@@@ NE 'h+()*a8 Co8<=e @@@
'V Na=e B>eCB=a77erBha@we:
V1 Na=e
'V /)He 1#.63 1)$
!88o*ata:8e N%
'E /)He 0
&ota8 'E 0
?ree 'E 0
!88o*ate> 'E 0
'V 55ID 3t)Oe>@=HM2@43*P@/e0o@"M)=@G7h(@a0gOkI
%2 Create a Vol.0e =ro.6 AV=C *alle/ HA-Web-VG an/ /is6lay the attri>.tes2
Per1or0 this ste6 on the 1irst *l.ster no/e Aha-1e'5C only
# vgcreate --clustered n 62-?eb-@A %dev%mapper%ha-web
Vo8<=e gro<7 "H!@e:@V1" (<**e((f<88+ *reate>
# vgdispla! 62-?eb-@A
@@@ Vo8<=e gro<7 @@@
V1 Na=e H!@e:@V1
/+(te= ID
?or=at 8C=2
Meta>ata !rea( 1
Meta>ata /eG<en*e No 1
V1 !**e(( rea>Bwr)te
V1 /tat<( re()Ha:8e
M!Q -V 0
C<r -V 0
%7en -V 0
MaF 'V 0
C<r 'V 1
!*t 'V 1
V1 /)He 1#.62 1)$
'E /)He 4.00 M)$
&ota8 'E 436#
!88o* 'E B /)He 0 B 0
?ree 'E B /)He 436# B 1#.62 1)$
V1 55ID 5CwCeP@"#Cr@$RS3@w<H#@P8Ha@N2PM@3r8n42
re!arh)!eedbak*redhat%o( 0$ """%redhat%o(
$2 Create a -ogi*al Vol.0e A-VC *alle/ ha-web-lvol1 an/ /is6lay the attri>.tes2
Per1or0 this ste6 on the 1irst *l.ster no/e Aha-1e'5C only
# lvcreate --siBe 80AC --name ha-web-lvol8 62-?eb-@A
-og)*a8 Co8<=e "ha@we:@8Co81" *reate>
# lvdispla! 62-?eb-@A
@@@ -og)*a8 Co8<=e @@@
-V Na=e B>eCBH!@e:@V1Bha@we:@8Co81
V1 Na=e H!@e:@V1
-V 55ID w/R%2>@I#17@1GR(@&f*N@2In3@2HD<@$C<Q"0
-V r)te !**e(( rea>Bwr)te
-V /tat<( aCa)8a:8e
# o7en 0
-V /)He 10.00 1)$
C<rrent -E 2560
/eg=ent( 1
!88o*at)on )nher)t
"ea> ahea> (e*tor( a<to
@ *<rrent8+ (et to 256
$8o*k >eC)*e 253:2
#2 &or0at the vol.0e 8ith the e't$ 1ilesyste0 an/ a vol.0e la>el AHA-Web-Vol-1C2
Per1or0 this ste6 on the 1irst *l.ster no/e Aha-1e'5C only
# m#fs*e"t4 -L 62-?eb-@ol-8 %dev%62-?eb-@A%ha-web-lvol8
=ke2f( 1.41.12 613@Ma+@2010;
?)8e(+(te= 8a:e8=H!@e:@Vo8@1
%/ t+7e: -)n<F
$8o*k ()He=4046 68og=2;
?rag=ent ()He=4046 68og=2;
/tr)>e=0 :8o*k(9 /tr)7e w)>th=0 :8o*k(
655360 )no>e(9 2621440 :8o*k(
131032 :8o*k( 65.00T; re(erCe> for the (<7er <(er
?)r(t >ata :8o*k=0
MaF)=<= f)8e(+(te= :8o*k(=26#4354560
#0 :8o*k gro<7(
3236# :8o*k( 7er gro<79 3236# frag=ent( 7er gro<7
#142 )no>e( 7er gro<7
/<7er:8o*k :a*k<7( (tore> on :8o*k(:
3236#9 4#3049 163#409 2243369 2444129 #142009 ##43369 1605632
r)t)ng )no>e ta:8e(: >one
Creat)ng Ro<rna8 63236# :8o*k(;: >one
r)t)ng (<7er:8o*k( an> f)8e(+(te= a**o<nt)ng )nfor=at)on: >one
&h)( f)8e(+(te= w)88 :e a<to=at)*a88+ *he*ke> eCer+ 24 =o<nt( or
1#0 >a+(9 wh)*heCer *o=e( f)r(t. 5(e t<ne2f( @* or @) to oCerr)>e.
62 Create a 0o.nt 6oint 1or the vol.0e2 Per1or0 this ste6 on all *l.ster no/es
# m#dir -p %ha%ha-web
"""%redhat%o( 0' re!arh)!eedbak*redhat%o(
72 :o.nt the vol.0e an/ veri1y it *an >e 8ritten to2 Per1or0 this ste6 on the 1irst *l.ster
no/e Aha-1e'5C only
# mount %dev%62-?eb-@A%ha-web-lvol8 %ha%ha-web
# touch %ha%ha-web%ha-web8*test
# ls -l %ha%ha-web%ha-web8*test
@rw@r@@r@@. 1 root root 0 O<8 11 11:1# BhaBha@we:Bha@we:1.te(t
82 (n ea*h *l.ster no/e4 e/it *etc*lvm*lvm.conf an/ *hange the volume_list 1iel/ to
0at*h the >oot vol.0e AmyvgC an/ na0e o1 the no/e *l.ster inter*onne*t Aha-1e'5C2
+his restri*ts the list o1 vol.0es availa>le /.ring syste0 >oot to only the root vol.0e
an/ 6revents *l.ster no/es 1ro0 .6/ating an/ 6otentially *orr.6ting the 0eta/ata on
the 3A--V: vol.0e
Co8<=e,8)(t = D U=+CgV9 UWha@we:1V E
"2 U6/ate the initial ra0/is, i0age so the >lo*, /evi*e 0o/.les are 6re-loa/e/ /.ring
>oot 8ith the volume_list restri*tion an/ re>oot to a*tivate the *hange2 Per1or0 this
ste6 on ea*h *l.ster no/e
# dracut --hostonl! --force %boot%initramfs-DEuname -r<*img DEuname -r<
# shutdown -r now 92ctivating ramdis# L@( changes9
4.3.6 Configure Web Server
12 9nstall the 4e' Server AA6a*heC so1t8are on ea*h o1 the three *l.ster no/es
# !um groupinstall 5?eb Ferver7
22 )y /e1a.lt4 the httpd is not ena>le/ to start on syste0 >oot2 +he *l.ster 1a*ilitates the
starting an/ sto66ing o1 the 8e> server as 6art o1 the *l.ster 8e> servi*e2 Con1ir0 that
httpd is /isa>le/ /.ring syste0 >oot on ea*h *l.ster no/e
# ch#config httpd off
# ch#config --list httpd
htt7> 0:off 1:off 2:off 3:off 4:off 5:off 6:off
%2 (n ea*h *l.ster no/e4 e/it *etc*httpd*conf*httpd.conf an/ *hange Listen to in*l./e the
9P a//ress o1 the 8e> servi*e an/ ocument!oot to the 0o.nt 6oint o1 the 3A--V:
vol.0e A"ha"ha-webC
-isten 1021621$%21#080
Do*.0entRoot 7BhaBha-8e>7
re!arh)!eedbak*redhat%o( 00 """%redhat%o(
4.3.7 Configure SELinux Security Parameters
)y /e1a.lt4 S,inu- is ena>le/ /.ring the Re/ 3at 5nter6rise -in.' 6 installation 6ro*ess2 &or
0a'i0.0 se*.rity4 Re/ 3at re*o00en/s r.nning Re/ 3at 5nter6rise -in.' 6 8ith S,inu-
ena>le/2 9n this se*tion4 veri1i*ation is /one to ens.re that S,inu- is ena>le/ an/ the 1ile
*onte't set *orre*tly on the "ha"ha-web 1ilesyste0 1or .se >y A6a*he2
12 Veri1y 8hether or not S,inu- is ena>le/ .sing the getenforce .tility2 Per1or0 this
ste6 on all *l.ster no/es
# getenforce
,n+orcin
91 getenforce ret.rns JPer0issiveK then set to J5n1or*ingK an/ veri1y
# getenforce
0ermissi'e
# setenforce 1
# getenforce
,n+orcin
22 5/it the 1ile *etc*selinu-*config an/ set S,inu- to >e 6ersistent a*ross re>oots2
Per1or0 this ste6 on all *l.ster no/es
/E-IN5Q=enfor*)ng
%2 A// A-aC the 1ile *onte't Afconte-tC 1or ty6e A-tC httpd?sys?content to the /ire*tory *ha
an/ all *ontents 8ithin it2 +his 0a,es the *hanges 6er0anent2 Per1or0 this ste6 on all
*l.ster no/es
# seanage fconte$t -a -t %ttp&'sys'content't ()%a*).+,-(
;ote. 91 the semanae A*usr*s'in*semanageC .tility is not availa>le4 install the *ore
6oli*y .tilities ,it
# yu -y install policycoreutils-pyt%on
# seanage fconte$t -a -t %ttp&'sys'content't ()%a*).+,-(
$2 Vie8 the *.rrent se*.rity 6oli*y 1ile *onte't2 Per1or0 this ste6 on all *l.ster no/es
# ls -l&. )%a
drw%r-%r-%. root root uncon+ined&u:ob1ect&r:de+ault&t:s0 2ha
#2 R.n the restorecon *o00an/ to a66ly the *hanges an/ vie8 the .6/ate/ 1ile
*onte't2 (nly 6er1or0 this ste6 on the *l.ster no/e that has "ha"ha-web 0o.nte/
# restorecon -/ -v )%a
restorecon reset 2ha conte%t uncon+ined&u:ob1ect&r:de+ault&t:s0-
3s4stem&u:ob1ect&r:htt5d&s4s&content&t:s0
restorecon reset 2ha2ha-web conte%t s4stem&u:ob1ect&r:+ile&t:s0-
3s4stem&u:ob1ect&r:htt5d&s4s&content&t:s0
restorecon reset 2ha2ha-web2lost6+ound conte%t
s4stem&u:ob1ect&r:+ile&t:s0-3s4stem&u:ob1ect&r:htt5d&s4s&content&t:s0
"""%redhat%o( 01 re!arh)!eedbak*redhat%o(
restorecon reset 2ha2ha-web2ha-web1.test conte%t
uncon+ined&u:ob1ect&r:+ile&t:s0-3s4stem&u:ob1ect&r:htt5d&s4s&content&t:s0
# ls -l&. )%a
drw%r-%r-%. root root s4stem&u:ob1ect&r:htt5d&s4s&content&t:s0 2ha
+he /e6loy0ent o1 the *l.ster 0anage0ent server an/ *l.ster no/es is no8 *o06lete2
Setion 1%0 7luster 7reation via 7onga /etails the 6ro*ess o1 *reating the *l.ster an/
*on1ig.ring the 8e> servi*e .sing the luci 8e> inter1a*e2 Alternatively4 Setion 1%1 7luster
7reation via 77S /etails the 6ro*ess o1 *reating the *l.ster 1ro0 the *o00an/ line .sing the
CCS .tility2
re!arh)!eedbak*redhat%o( 02 """%redhat%o(
$.$ )luster )reation via )onga
+he luci gra6hi*al .ser inter1a*e 6ortion o1 Conga allo8s a/0inistrators to *reate4 0o/i1y
an/ vie8 a *l.ster *on1ig.ration 1ile on a re0ote no/e thro.gh the ricci servi*e2 Using luci
an a/0inistrator *an also start4 sto6 an/ relo*ate *l.ster servi*es on one or 0ore *l.ster
no/es2
9n the 6rior se*tions4 the *l.ster 0anage0ent server an/ *l.ster no/es 8ere 1.lly /e6loye/2
Do not 6ro*ee/ 8ith *reating the *l.ster .ntil these tas,s have >een 1.lly *o06lete/
Cl.ster :anage0ent Station Aha-1e'-mgmtC
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s
Con1ig.re &ire8all
9nstall Cl.ster :anage0ent So1t8are AJ3igh Availa>ility :anage0entK A//-(nC
Cl.ster No/es Aha-1e'54 ha-1e'$4 ha-1e'/C
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s an/ )on/ing
Con1ig.re &ire8all
9nstall Cl.ster No/e So1t8are AJ3igh Availa>ilityK A//-(nC
Con1ig.re Storage
Con1ig.re ;e> Server So1t8are
+he ne't se*tions /es*ri>e the ste6s involve/ in *reating the *l.ster 1ro0 the luci 8e>
inter1a*e on the *l.ster 0anage0ent server2
4.4.1 Create Cluster
Creating a *l.ster 8ith luci *onsists o1 na0ing the *l.ster4 a//ing *l.ster no/es to the
*l.ster4 entering ricci 6ass8or/s 1or ea*h no/e an/ s.>0itting the reI.est to *reate a
*l.ster2 91 the no/e in1or0ation an/ 6ass8or/s are *orre*t4 Conga a.to0ati*ally installs any
nee/e/ or 0issing *l.ster so1t8are *o06onents onto the *l.ster no/es an/ starts the *l.ster2
&ollo8 the ste6s >elo8 to start the reI.ire/ *l.ster servi*es4 *reate the *l.ster an/ a// the
no/es to it2
12 Start the ricci servi*e an/ *on1ig.re to start on syste0 >oot2 Per1or0 this ste6 on all
*l.ster no/es
# service ricci start
/tart)ng o>>Ro:>: D %0 E
generat)ng //- *ert)f)*ate(... >one
1enerat)ng N// >ata:a(e... >one
/tart)ng r)**): D %0 E
# ch#config ricci on
"""%redhat%o( 06 re!arh)!eedbak*redhat%o(
22 Con1ig.re a 6ass8or/ 1or the ricci .ser a**o.nt on all *l.ster no/es2 +he sa0e
6ass8or/ 0ay >e .se/ on all *l.ster no/es to si06li1y a/0inistration
# passwd ricci
Chang)ng 7a((wor> for <(er r)**).
New 7a((wor>:
"et+7e new 7a((wor>:
7a((w>: a88 a<thent)*at)on token( <7>ate> (<**e((f<88+.
%2 Start luci on the 0anage0ent server
# service luci start
1enerat)ng a 204# :)t "/! 7r)Cate ke+
wr)t)ng new 7r)Cate ke+ to IBCarB8):B8<*)B*ert(Bho(t.7e=I
/tart)ng (a(8a<th>: D %0 E
/tart 8<*)... D %0 E
'o)nt a we: :row(er to htt7(:BBha@we:=g=t.*8o<>.8a:.eng.:o(.re>hat.*o=:#0#4
6or eG<)Ca8ent; to a**e(( 8<*)
$2 (6en the luci 8e> inter1a*e >y 6ointing a >ro8ser to the .rl s6e*i1ie/ /.ring the
luci servi*e start
https@**ha-1e'-mgmt.cloud.la'.eng.'os.redhat.com@;6;.
an/ login .sing the root a**o.nt2
#2 A1ter login4 luci 6resents the Homebase 6age2 Sele*t Manage Clusters on the le1t si/e
o1 the Homebase as sho8n in /igure 1%1)$. Manage 7lusters2 Ne't4 sele*t Create2
re!arh)!eedbak*redhat%o( 04 """%redhat%o(
Figure 4.4-1: 0anage Clusters
62 +he Create New Cluster /ialog >o' o6ens as sho8n in /igure 1%1)'. 7reate ;e"
7luster2 Co06lete the 1ollo8ing in the /ialog >o'
5nter the Cluster Name2 Cl.ster na0es are restri*te/ to 1# *hara*ters in length2
5na>le the o6tion JUse the same password for all nodesK
5nter the Node Name o1 the 1irst *l.ster no/e >y s6e*i1ying the 6rivate cluster
interconnect2 +his ena>les *l.ster *o00.ni*ations to r.n over the 6rivate
net8or,2
5nter Password2 S6e*i1y the 6ass8or/ 1or the ricci a**o.nt2
5nter Hostname2 S6e*i1y the 6.>li* net8or, na0e2
Sele*t Add Another Node an/ *o06lete the 1iel/s 1or the other t8o *l.ster no/es as
sho8n2 ;hen /one4 ena>le the o6tion Download Packages then sele*t Create Cluster2
"""%redhat%o( 05 re!arh)!eedbak*redhat%o(
Figure 4.4-2: Create 1e# Cluster
4.4.2 Add Fence Devices
&en*ing 6rovi/es 6rote*tion against /ata *orr.6tion *a.se/ >y 0.lti6le *l.ster no/es
6er1or0ing si0.ltaneo.s 9B( to a share/ storage /evi*e2 &en*ing /is*onne*ts a *l.ster no/e
1ro0 the share/ storage /evi*e in the event o1 a *l.ster 1a.lt or no/e 1ail.re2
+he 6ri0ary 1en*ing 0etho/ .tiliHe/ >y this re1eren*e ar*hite*t.re is 9P:9 over -AN2
A//itional 1en*ing 0etho/s *an >e a66lie/ >ase/ on the reso.r*es availa>le 8ithin a
*o06.ting environ0ent2 &ollo8 the ste6s >elo8 to a// a fence device, fence device instance
an/ fence method to ea*h no/e in the the *l.ster2
12 &ro0 the Manage Clusters s*reen4 sele*t the ha-1e'-cluster lin, to o6en the *l.ster
s6e*i1i* 6age as sho8n in /igure 1%1)0. /ene Devies
Un/er the Fence Devices ta> sele*t Add2
22 +he Add Fence Device !nstance" /ialog >o' o6ens as sho8n in /igure 1%1)1. Selet
/ene Devie
9n the # $elect a fence device # /ro6-/o8n4 sele*t !PM! %an then sele*t $ubmit2
re!arh)!eedbak*redhat%o( 08 """%redhat%o(
Figure 4.4-3: Fence Devices
Figure 4.4-4: Select Fence Device
%2 +he Add Fence Device !nstance" /ialog >o' is /is6laye/ 1or the 1en*e ty6e !PM! %an2
Co06lete the 1ollo8ing in the /igure 1%1)2. Add /ene Devie 3nstane /ialog >o'
5nter a Name 1or the 9P:9 -an /evi*e instan*e2
5nter the !P address or hostname
5nter the %ogin2 91 the /evi*e has a /i11erent login than the *l.ster no/e s6e*i1y
it here2
5nter Password2 91 the /evi*e has a /i11erent 6ass8or/ than the *l.ster no/e
s6e*i1y it here2
9n the Authentication t&pe /ro6-/o8n sele*t Password
5na>le the Use %anplus o6tion
Sele*t $ubmit 8hen /one2
"""%redhat%o( 1& re!arh)!eedbak*redhat%o(
Figure 4.4-5: 2&& Fence Device 3nstance
$2 A// a 1en*e /evi*e instan*e 1or the other *l.ster no/es >y 1ollo8ing ste6s 2 an/ %
a>ove 1or ea*h no/e2 ;hen all instan*es have >een *reate/4 the Fence Devices ta>
rese0>les /igure 1%1)6. /ene Devie 3nstanes >elo8
#2 Un/er the Nodes ta> Asee /igure 1%1)4. 7luster ;odes >elo8C sele*t the lin, 1or the
1irst *l.ster no/e ha-1e'5-ci2
re!arh)!eedbak*redhat%o( 1$ """%redhat%o(
Figure 4.4-4: Fence Device 3nstances
Figure 4.4-5: Cluster 1o&es
62 S*roll /o8n to the 6ro6erties o1 the *l.ster no/e as sho8n in /igure 1%1)5. Add /ene
Method2 Un/er Fence Devices sele*t Add Fence Method2
72 +he Add Fence Method 'o Node /ialog >o' A/igure 1%1)8. Add /ene Method 6o
;ode- o6ens F enter Arimary 1or the Method Name an/ sele*t $ubmit
"""%redhat%o( 1' re!arh)!eedbak*redhat%o(
Figure 4.4-6: 2&& Fence 0et%o&
Figure 4.4-7: 2&& Fence 0et%o& !o 1o&e
82 Un/er the ne8ly *reate/ Primar& 1en*e 0etho/4 sele*t Add Fence !nstance A/igure
1%1)$&. Add /ene 3nstaneC
9n the Add Fence Device !nstance" /ialog >o' A/igure 1%1)$$. Add /ene Devie
3nstaneC4 s*roll /o8n an/ sele*t the na0e o1 the 1en*e /evi*e A6revio.sly *reate/ in
ste6 %C 1or the 1irst *l.ster no/e Aha-1e'5-ciC
Sele*t $ubmit 8hen /one2
"2 A// a Arimary 1en*e 0etho/ an/ 1en*e /evi*e instan*e 1or the other *l.ster no/es >y
1ollo8ing ste6s #4 64 an/ 7 1or ea*h no/e2
re!arh)!eedbak*redhat%o( 10 """%redhat%o(
Figure 4.4-18: 2&& Fence 3nstance
Figure 4.4-11: 2&& Fence Device 3nstance
4.4.3 Add Failover Domain
+he failover domain /eter0ines the or/er an/ *hara*teristi*s in 8hi*h *l.ster 0e0>ers
0anage the failover an/ fail'ac( o1 a *l.ster servi*e2 &ollo8 the ste6s >elo8 to a// a failover
domain to the *l.ster2
12 Un/er the Failover Domains ta> A/igure 1%1)$'. Add /ailover Do(ains 6abC sele*t
Add
22 9n the Add Failover Domain 'o Cluster /ialog >o' A/igure 1%1)$0. Add /ailover
Do(ain 6o 7lusterC4 enter the Name Aha-1e'-failoverC4 ena>le the o6tion 1or
Prioriti(ed an/ set the 0e0>er 6riorities as sho8n2 Sele*t Create 8hen /one
"""%redhat%o( 11 re!arh)!eedbak*redhat%o(
Figure 4.4-12: 2&& Failover Doains !ab
Figure 4.4-13: 2&& Failover Doain !o Cluster
;hen *o06lete4 the Failover Domains ta> rese0>les /igure 1%1)$1. Add /ailover
Do(ains ) 7o(pleted >elo8
re!arh)!eedbak*redhat%o( 12 """%redhat%o(
Figure 4.4-14: 2&& Failover Doains - Coplete&
4.4.4 Add Resources
+he highly availa>le 8e> servi*e reI.ires 1o.r reso.r*es F IA 3ddress4 23 <!4 #ilesystem
an/ Script2 &ollo8 the ste6s >elo8 to a// the reI.ire/ 8e> servi*e reso.r*es to the *l.ster2
12 &ro0 the )esources ta> A/igure 1%1)$2. Add ResouresC sele*t Add
22 9n the Add )esource 'o Cluster /ialog >o' A/igure 1%1)$6. Selet Resoure 6ypeC4
s*roll /o8n an/ sele*t !P Address
"""%redhat%o( 16 re!arh)!eedbak*redhat%o(
Figure 4.4-15: 2&& /esources
Figure 4.4-14: Select /esource !ype
9n the !P Address reso.r*e /ialog >o' A/igure 1%1)$4. Add 3P Address ResoureC4
enter the !P address A56.59.5./.5B6C to >e .se/ 1or *onne*tions to the 8e> servi*e2
5na>le the o6tion Monitor link then sele*t $ubmit:
%2 Re6eat ste6s 1 an/ 2 to a// an HA %*M reso.r*e Aha-1e'-23-<!C 8ith the o6tions
sho8n in /igure 1%1)$5. Add HA L#M Resoure2 Cli*, $ubmit 8hen /one2
re!arh)!eedbak*redhat%o( 14 """%redhat%o(
Figure 4.4-15: 2&& 39 2&&ress /esource
Figure 4.4-16: 2&& :2 ;<0 /esource
$2 Re6eat ste6s 1 an/ 2 to a// a Files&stem reso.r*e Aha-1e'-filesystemC 8ith the
o6tions sho8n in /igure 1%1)$8. Add /ilesyste( Resoure2 Cli*, $ubmit 8hen /one2
#2 Re6eat ste6s 1 an/ 2 to a// a $cript reso.r*e Aha-1e'-apache-scriptC 8ith the o6tions
sho8n in /igure 1%1)'&. Add Sript Resoure2 Cli*, $ubmit 8hen /one2
"""%redhat%o( 15 re!arh)!eedbak*redhat%o(
Figure 4.4-17: 2&& Filesyste /esource
Figure 4.4-28: 2&& Script /esource
4.4.5 Add Service Group
Service "roups are 0anage/ >y rgmanager an/ *ontain the reso.r*es nee/e/ to 6rovi/e
highly availa>le servi*es2 &ollo8 the ste6s >elo8 to a// a Service na0e/ ha-1e'-service
an/ the reso.r*es reI.ire/ >y the 8e> servi*e to the *l.ster2
12 &ro0 the $ervice +roups ta> A/igure 1%1)'$. Add Servie :roupC4 sele*t Add
22 +he Add $ervice 'o Cluster /ialog >o' o6ens A/igure 1%1)''. Add Servie 6o
7lusterC2 5nter the $ervice name Aha-1e'-serviceC an/ ena>le the o6tions sho8n2
Sele*t $ubmit 8hen /one2
re!arh)!eedbak*redhat%o( 18 """%redhat%o(
Figure 4.4-21: 2&& Service =roup
Figure 4.4-22: 2&& Service !o Cluster
%2 A// the IA 3ddress reso.r*e to the servi*e Aha-1e'-serviceC >y sele*ting Add
)esource4 sele*ting the reso.r*e ty6e 56.59.5./.5B6 1ro0 the /ro6-/o8n an/ sele*ting
$ubmit as sho8n in /igure 1%1)'0. Add Resoure 6o Servie
Re6eat the 6ro*ess 1or the re0aining three reso.r*es - 23 <!4 #ilesystem4 Script2
5a*h reso.r*e sho.l/ >e in/e6en/ent an/ have no Child )esources asso*iate/ 8ith it2
$2 +he 8e> servi*e Aha-1e'-serviceC is /is6laye/ at the to6 o1 the $ervice +roups ta>2
+he stat.s is liste/ as Unknown .ntil it is starte/2 A/igure 1%1)'1. Servie :roup
Status ?nkno"nC
#2 Start the 8e> servi*e >y ena>ling the *he*, >o' ne't to the Name Aha-1e'-serviceC
an/ sele*ting $tart 1ro0 the 0en.2 Alternatively4 the servi*e *an >e starte/ on a
s6e*i1i* no/e >y sele*ting the no/e 1ro0 the $tart on node,,, /ro6-/o8n >o' then
sele*ting the $tart i*on on the right2 (n*e a servi*e has >een starte/ the stat.s is
.6/ate/ as sho8n in /igure 1%1)'2. Servie :roup Status Running
"""%redhat%o( 2& re!arh)!eedbak*redhat%o(
Figure 4.4-23: 2&& /esource !o Service
Figure 4.4-24: Service =roup Status >n"no#n
9n the a>ove e'a06le4 ha-1e'-service is r.nning on *l.ster no/e ha-1e'$-ci2
4.4.6 Verify Cluster Web Service
Veri1y the 8e> servi*e is r.nning >y o6ening a >ro8ser 8in/o8 onto the 8e> servi*e
Ahttp@56.59.5./.5B6*cgi-'in*23-4e'-StatusC an/ r.nning the test s*ri6t A/igure 1%1)'6. HA)
Web)StatusC
re!arh)!eedbak*redhat%o( 2$ """%redhat%o(
Figure 4.4-25: Service =roup Status /unning
Figure 4.4-24: :2-Web-Status
$.( )luster )reation via ))S
+he ccs ACl.ster Con1ig.ration Syste0C *o00an/ line inter1a*e allo8s a/0inistrators to
*reate4 0o/i1y an/ vie8 a *l.ster *on1ig.ration 1ile on a re0ote no/e thro.gh the ricci
servi*e or on a lo*al 1ilesyste02 Using ccs an a/0inistrator *an also start4 sto6 an/ relo*ate
*l.ster servi*es on one or 0ore *l.ster no/es2
9n the 6rior se*tions4 the *l.ster 0anage0ent server an/ *l.ster no/es 8ere 1.lly /e6loye/2
Do not 6ro*ee/ 8ith *reating the *l.ster via CCS .ntil these tas,s have >een 1.lly *o06lete/
Cl.ster :anage0ent Station Aha-1e'-mgmtC
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s
Con1ig.re &ire8all
9nstall Cl.ster :anage0ent So1t8are AJ3igh Availa>ility :anage0entK A//-(nC
Cl.ster No/es Aha-1e'54 ha-1e'$4 ha-1e'/C
9nstall Re/ 3at 5nter6rise -in.' 6
Con1ig.re Net8or,s an/ )on/ing
Con1ig.re &ire8all
9nstall Cl.ster No/e So1t8are AJ3igh Availa>ilityK A//-(nC
Con1ig.re Storage
Con1ig.re ;e> Server So1t8are
+he ne't se*tions /es*ri>e the ste6s involve/ in *reating a *l.ster 1ro0 the ccs *o00an/
line inter1a*e2
4.5.1 Create Cluster
Cl.ster *reation is 6er1or0e/ 1ro0 the *l.ster 0anage0ent server Aha-1e'-mgmtC an/
.6/ates are /e6loye/ to the *l.ster no/es a*ross the 6.>li* net8or, inter1a*es2 +he 6ro*ess
involves *reating a 1.ll *l.ster *on1ig.ration 1ile A*etc*cluster*cluster.confC on one no/e Aha-
1e'5-ciC then /istri>.ting the *on1ig.ration an/ a*tivating the *l.ster on the re0aining no/es2
Cl.ster inter*onne*ts are s6e*i1ie/ 8ithin the *on1ig.ration 1ile 1or all no/e *o00.ni*ations2
Con1ig.re the a66ro6riate *l.ster servi*es then *reate the *l.ster2
12 Start the ricci servi*e an/ *on1ig.re to start on syste0 >oot2 Per1or0 this ste6 on all
*l.ster no/es
# service ricci start
# ch#config ricci on
22 Con1ig.re a 6ass8or/ 1or the ricci .ser a**o.nt on ea*h no/e2 +he sa0e 6ass8or/
0ay >e .se/ on all *l.ster no/es to si06li1y a/0inistration
# passwd ricci
"""%redhat%o( 2' re!arh)!eedbak*redhat%o(
%2 Create a *l.ster na0e/ ha-web-cluster 1ro0 the *l.ster 0anage0ent server
Aha-1e'-mgmt%@
# ccs --host ha-web8 --createcluster ha-web-cluster
ha@we:1 7a((wor>: AAAAAAA
4.5.2 Add Nodes
(n*e the *l.ster has >een *reate/4 s6e*i1y the 0e0>er no/es in the *l.ster *on1ig.ration2
12 A// the three *l.ster no/es Aha-1e'5-ci, ha-1e'$-ci, ha-1e'/-ciC to the *l.ster2
Per1or0 this ste6 1ro0 the *l.ster 0anage0ent server Aha-1e'-mgmtC
# ccs --host ha-web8 --addnode ha-web8-ci 0nodeid=787
No>e ha@we:1@*) a>>e>.
# ccs --host ha-web8 --addnode ha-web2-ci 0nodeid=727
No>e ha@we:2@*) a>>e>.
# ccs --host ha-web8 --addnode ha-web&-ci 0nodeid=7&7
No>e ha@we:3@*) a>>e>.
4.5.3 Add Fence Devices
A// the 1en*e 0etho/ then a// /evi*es an/ instan*es 1or ea*h *l.ster no/e to the 0etho/2
9P:9 -AN 1en*ing is .se/ in this *on1ig.ration2 (ther 1en*ing 0etho/s an/ /evi*es *an >e
.se/ /e6en/ing on the reso.r*es availa>le2 Per1or0 all ste6s 1ro0 the *l.ster 0anage0ent
server Aha-1e'-mgmt%.
12 A// a 1en*e 0etho/ 1or the Pri0ary 1en*ing /evi*es
# ccs --host ha-web8 --addmethod ,rimar! ha-web8-ci
Metho> 'r)=ar+ a>>e> to ha@we:1@*).
# ccs --host ha-web8 --addmethod ,rimar! ha-web2-ci
Metho> 'r)=ar+ a>>e> to ha@we:1@*).
# ccs --host ha-web8 --addmethod ,rimar! ha-web&-ci
Metho> 'r)=ar+ a>>e> to ha@we:1@*).
22 A// a 1en*e /evi*e 1or the 9P:9 -AN /evi*e
# ccs --host ha-web8 --addfencedev +,(+-ha-web8-ci
agent=fence_ipmilan auth=password
ipaddr=80*86*84&*2&8 lanplus=on
login=root name=+,(+-ha-web8-ci passwd=password
power_wait=; timeout=20
# ccs --host ha-web8 --addfencedev +,(+-ha-web2-ci
agent=fence_ipmilan auth=password
ipaddr=80*86*84&*2&2 lanplus=on
login=root name=+,(+-ha-web2-ci passwd=password
power_wait=; timeout=20
# ccs --host ha-web8 --addfencedev +,(+-ha-web&-ci
agent=fence_ipmilan auth=password
ipaddr=80*86*84&*2&& lanplus=on
login=root name=+,(+-ha-web&-ci passwd=password
power_wait=; timeout=20
re!arh)!eedbak*redhat%o( 20 """%redhat%o(
%2 A// a 1en*e instan*e 1or ea*h no/e to the Pri0ary 1en*e 0etho/
# ccs --host ha-web8 --addfenceinst +,(+-ha-web8-ci ha-web8-ci ,rimar!
# ccs --host ha-web8 --addfenceinst +,(+-ha-web2-ci ha-web2-ci ,rimar!
# ccs --host ha-web8 --addfenceinst +,(+-ha-web&-ci ha-web&-ci ,rimar!
4.5.4 Add Failover Domain
+he 1ailover /o0ain s6e*i1ies the 0e0>ershi6 an/ 1ailover *hara*teristi*s 1or no/es 6rovi/ing
a *l.ster servi*e2 :.lti6le 1ailover /o0ains *an >e *reate/ to 6rovi/e /i11erent 1ailover
*hara*teristi*s 1or /i11erent *l.ster servi*es2
12 A// a 1ailover /o0ain an/ a// the *l.ster no/es to it2 S6e*i1y ro.n/-ro>in 1ailover
starting 8ith no/e14 1ollo8e/ >y no/e2 then no/e%2 Per1or0 these tas,s 1ro0 the
*l.ster 0anage0ent server Aha-1e'-mgmt%@
# ccs --host ha-web8 --addfailoverdomain ha-web-failover ordered
# ccs --host ha-web8 --addfailoverdomainnode ha-web-failover ha-web8-ci 8
# ccs --host ha-web8 --addfailoverdomainnode ha-web-failover ha-web2-ci 2
# ccs --host ha-web8 --addfailoverdomainnode ha-web-failover ha-web&-ci &
4.5.5 Add Resources
De1ine ea*h o1 the reso.r*es that *o06rise the *l.ster 8e> servi*e2 +he 1ollo8ing reso.r*es
are reI.ire/ 1or *on1ig.ring a highly availa>le 8e> servi*e
9P A//ress
3A -V:
&ilesyste0
S*ri6t
Per1or0 all ste6s 1ro0 the *l.ster 0anage0ent server Aha-1e'-mgmtC2
12 A// the 3P Address reso.r*e A1#$1%$1&'$1(#C to the *l.ster *on1ig.ration2 +his
reso.r*e 6rovi/es re0ote *lient a**ess to the 8e> servi*e Aha-web-serviceC an/
*ontent
# ccs --host ha-web8 --addresource ip address=80*86*84&*8;0
monitor_lin#=on sleeptime=80
22 A// the HA L#M reso.r*e Aha-web-HA-LV)C to the *l.ster *on1ig.ration2 +his
reso.r*e is res6onsi>le 1or ena>ling an/ /isa>ling e'*l.sive a**ess to the 3A--V:
vol.0e a*ross *l.ster no/es
# ccs --host ha-web8 --addresource lvm
lv_name=ha-web-lvol8
name=ha-web-62-L@(
self_fence=on vg_name=62-?eb-@A
"""%redhat%o( 21 re!arh)!eedbak*redhat%o(
%2 A// the /ilesyste( reso.r*e Aha-web-*ilesystemC to the *l.ster *on1ig.ration2
+his reso.r*e is res6onsi>le 1or the 0o.nting an/ .n0o.nting o1 the 3A--V: vol.0e
# ccs --host ha-web8 --addresource fs
device=%dev%62-?eb-@A%ha-web-lvol8
fsid=;64&2 fst!pe=e"t4
mountpoint=%ha%ha-web
name=ha-web-files!stem self_fence=on
$2 A// the Sript reso.r*e Aha-web-apache-scriptC to the *l.ster *on1ig.ration2 +his
reso.r*e is res6onsi>le 1or the starting an/ sto66ing o1 the A6a*he AhttpdC 8e>
server
# ccs --host ha-web8 --addresource script
file=%etc%init*d%httpd
name=ha-web-apache-script
4.5.6 Add Service Group
Servi*e gro.6s are *olle*tions o1 reso.r*es that si06li1y the 0anage0ent o1 highly availa>le
*l.ster servi*es2
12 Create a servi*e gro.6 Aha-web-serviceC 1or the 8e> servi*e an/ a// the reI.ire/
reso.r*es A+,G 62-L@(G Hiles!stemG FcriptC to it
# ccs --host ha-web8 --addservice ha-web-service
domain=ha-web-failover
ma"_restarts=& name=ha-web-service
recover!=restart restart_e"pire_time=&600
# ccs --host ha-web8 --addsubservice ha-web-service
ip ref=80*86*84&*8;0
# ccs --host ha-web8 --addsubservice ha-web-service
lvm ref=ha-web-62-L@(
# ccs --host ha-web8 --addsubservice ha-web-service
fs ref=ha-web-files!stem
# ccs --host ha-web8 --addsubservice ha-web-service
script ref=ha-web-apache-script
4.5.7 Activate Cluster
(n*e the *l.ster has >een *reate/4 the *on1ig.ration nee/s to >e a*tivate/ an/ the *l.ster
starte/ on all no/es2
12 Syn*hroniHe an/ a*tivate the *l.ster *on1ig.ration a*ross all no/es
# ccs --host ha-web8 --s!nc --activate
ha@we:2@*) 7a((wor>:
# ccs --host ha-web8 --chec#conf
!88 no>e( )n (+n*.
re!arh)!eedbak*redhat%o( 22 """%redhat%o(
22 Start the *l.ster servi*es on all no/es
# ccs --host ha-web8 --startall
/tarte> ha@we:2@*)
/tarte> ha@we:3@*)
/tarte> ha@we:1@*)
4.5.8 Verify Cluster Web Service
12 Veri1y the *l.ster 8e> servi*e is availa>le 1ro0 one o1 the *l.ster no/es Aha-1e'5-ciC2
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W Mon !<g # 16:40:0# 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 -o*a89 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:1@*) (tarte>
22 Veri1y the 8e> servi*e is r.nning >y o6ening a >ro8ser 8in/o8 onto the 8e> servi*e
Ahttp@56.59.5./.5B6*cgi-'in*23-4e'-StatusC an/ r.nning the test s*ri6t A/igure 1%2)$.
HA)Web)StatusC
+his *o06letes the *l.ster *reation an/ veri1i*ation .sing CCS2 +he ne't se*tion /etails the
*o00on .se *ases in 0anaging a Re/ 3at 5nter6rise -in.' 6 highly availa>le 8e> server2
"""%redhat%o( 26 re!arh)!eedbak*redhat%o(
Figure 4.5-1: :2-Web-Status
5 Cluster Management
+he 6revio.s se*tions o1 this re1eren*e ar*hite*t.re /etaile/ the /e6loy0ent an/ *on1ig.ration
tas,s asso*iate/ 8ith /e6loying a highly availa>le 8e> servi*e .sing the 3igh Availa>ility
A//-(n 1or Re/ 3at 5nter6rise -in.'2 +he 1ollo8ing se*tions 1o*.s on the 0ost *o00on
*l.ster 0anage0ent tas,s .sing >oth the Conga A=U9C an/ CCS AC-9C inter1a*es2
+8o no/e *l.sters are a s6e*ial *ase s*enario reI.iring a *l.ster restart Aan/ >rie1
servi*e /o8nti0eC to a*tivate the *hange in 0e0>ershi6 8hen a//ing A2 -S %C or
re0oving A% -S 2C a no/e2 Conga i06li*itly han/les the restart 8hile CCS reI.ires a
0an.al restart2 )oth 0etho/s are /e0onstrate/ 8ithin the se*tions that 1ollo8
/es*ri>ing the a//ing an/ re0oval o1 *l.ster no/es2
(.1 Adding )luster Nodes
7onga A:etho/ 1C
+he 1ollo8ing ste6s /es*ri>e the 6ro*ess o1 a//ing a ne8 no/e Aha-1e'/-ciC to an e'isting
t8o no/e *l.ster .sing Conga an/ the luci 8e> inter1a*e2
12 (6en the luci 8e> inter1a*e an/ login .sing the root a**o.nt2 Un/er the Nodes ta>
sele*t Add A/igure 2%$)$. Add ;odes 6abC
re!arh)!eedbak*redhat%o( 24 """%redhat%o(
Figure 5.1-1: 2&& 1o&es !ab
22 9n the Add Nodes 'o Cluster /ialog >o'4 a// the ne8 no/e to the *l.ster as sho8n in
/igure 2%$)'. Add ;odes 6o 7luster >elo82 S6e*i1y the 6rivate cluster interconnect
na0e 1or Node Name2 Sele*t Add Nodes 8hen /one
%2 A1ter the no/e has >een a//e/ an/ the *l.ster *on1ig.ration .6/ate/4 the Nodes ta>
rese0>les /igure 2%$)0. 7luster ;odes 6ab ) ;e" ;ode Added >elo8
Con1ig.re 1en*ing ASetion 1%1%' Add /ene DeviesC an/ a// the no/e to the
failover domain ASetion 1%1%0 Add /ailover Do(ainC to *o06lete the no/e a//ition2
"""%redhat%o( 25 re!arh)!eedbak*redhat%o(
Figure 5.1-2: 2&& 1o&es !o Cluster
Figure 5.1-3: Cluster 1o&es !ab - 1e# 1o&e 2&&e&
77S A:etho/ 2C
As an alternative4 *l.ster no/es *an >e a//e/ 1ro0 the *o00an/ line .sing CCS2 +he
1ollo8ing ste6s /es*ri>e the 6ro*ess o1 a//ing a ne8 no/e Aha-1e'/-ciC to an e'isting t8o
no/e *l.ster2
12 Veri1y the *.rrent *l.ster stat.s 1ro0 any no/e
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W ?r) O<8 24 13:26:46 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 -o*a89 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tarte>
22 A// the ne8 0e0>er to the *l.ster *on1ig.ration an/ s6e*i1y the nodeid2 ;hen
e'6an/ing 1ro0 t8o no/es to three no/es Aor greaterC4 the two_node 1lag 0.st >e
/isa>le/2 +his *an >e r.n 1ro0 any *l.ster no/e or the 0anage0ent server
# ccs --host ha-web8 --setcman
# ccs --host ha-web8 --addnode ha-web&-ci --nodeid=7&7
No>e ha@we:3@*) a>>e>.
%2 A// the ne8 no/e to the 1en*e 0etho/ AArimaryC an/ an instan*e o1 the 1en*e /evi*e
AIA!I-ha-1e'-ciC to the 1en*e 0etho/2 R.n this 1ro0 any *l.ster no/e or the
0anage0ent server
# ccs --host ha-web8 --addmethod ,rimar! ha-web&-ci
Metho> 'r)=ar+ a>>e> to ha@we:3@*).
# ccs --host ha-web8 --addfencedev +,(+-ha-web&-ci agent=fence_ipmilan
auth=password ipaddr=80*86*84&*2&& lanplus=on login=root
name=+,(+-ha-web&-ci passwd=password power_wait=; timeout=20
# ccs --host ha-web8 --addfenceinst +,(+-ha-web&-ci ha-web&-ci ,rimar!
$2 A// the ne8 no/e to the 1ailover /o0ain2 +his *an >e r.n 1ro0 any *l.ster no/e or the
0anage0ent server
# ccs --host ha-web8 --addfailoverdomainnode ha-web-failover ha-web&-ci &
re!arh)!eedbak*redhat%o( 28 """%redhat%o(
#2 Pro6agate the *hange to all *l.ster 0e0>ers an/ start the *l.ster servi*es2 A >rie1
/o8nti0e is reI.ire/ to allo8 the *l.ster no/es to syn*hroniHe an/ a*tivate the
*hange2 +his *an >e r.n 1ro0 any *l.ster no/e or the 0anage0ent server
# ccs --host ha-web8 --stopall
# ccs --host ha-web8 --s!nc --activate
# ccs --host ha-web8 --chec#conf
!88 no>e( )n (+n*.
# ccs --host ha-web8 --startall
/tarte> ha@we:2@*)
/tarte> ha@we:3@*)
/tarte> ha@we:1@*)
62 Veri1y the ne8 *l.ster stat.s 1ro0 any no/e
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W ?r) O<8 24 13:34:42 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 -o*a89 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tarte>
"""%redhat%o( 6& re!arh)!eedbak*redhat%o(
(.2 Re'oving )luster Nodes
7onga A:etho/ 1C
+he 1ollo8ing ste6s /es*ri>e the 6ro*ess o1 re0oving one no/e Aha-1e'/-ciC 1ro0 a r.nning
*l.ster .sing Conga an/ the luci 8e> inter1a*e2 91 any *l.ster servi*es are r.nning on the
no/e to >e re0ove/4 1irst relo*ate the servi*e as 6er Setion 2%0 Reloating 7luster Web
Servies >e1ore 6ro*ee/ing2
12 (6en the luci 8e> inter1a*e an/ login .sing the root a**o.nt2 A no/e 0.st 1irst leave
a *l.ster >e1ore it *an >e re0ove/2 Un/er the Nodes ta> sele*t %eave Cluster A/igure
2%')$. ;ode ) Leave 7lusterC
22 Che*, the >o' ne't to the no/e to >e re0ove/ A/igure 2%')'. ;ode ) DeleteC an/
sele*t Delete
re!arh)!eedbak*redhat%o( 6$ """%redhat%o(
Figure 5.2-1: 1o&e - ;eave Cluster
%2 +he Node ta> re1le*ts the *hange in *l.ster 0e0>ershi6 A/igure 2%')0. ;ode )
Re(oval 7o(pleteC
+his *o06letes the re0oval o1 a no/e .sing Conga an/ the luci 8e> inter1a*e2
"""%redhat%o( 6' re!arh)!eedbak*redhat%o(
Figure 5.2-2: 1o&e - Delete
Figure 5.2-3: 1o&e - /eoval Coplete
77S A:etho/ 2C
As an alternative4 *l.ster no/es *an >e re0ove/ .sing CCS2 +he 1ollo8ing ste6s /es*ri>e the
6ro*ess o1 re0oving one no/e Aha-1e'/-ciC 1ro0 a r.nning *l.ster2
12 Veri1y the *.rrent *l.ster stat.s 1ro0 any no/e
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W e> O<8 2# 1#:2#:00 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 -o*a89 rg=anager
ha@we:3@*) 3 %n8)ne9 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:3@*) (tarte>
22 Relo*ate any 8e> servi*es r.nning on the no/e >eing re0ove/ to another *l.ster
0e0>er2 91 no servi*es are r.nning on the no/e >eing re0ove/ *ontin.e to the ne't
ste62 9n this e'a06le the ha-1e'-1e' servi*e is r.nning on *l.ster no/e ha-1e'/-ci so
it is relo*ate/ to no/e1 Aha-1e'5-ciC .sing the clusvcadm tool2 +his *an >e r.n 1ro0
any *l.ster no/e >.t here no/e ha-1e'$-ci is .se/
# clusvcadm -r ha-web-service -m ha-web8-ci
&r+)ng to re8o*ate (erC)*e:ha@we:@(erC)*e to ha@we:1@*).../<**e((
(erC)*e:ha@we:@(erC)*e )( now r<nn)ng on ha@we:1@*)
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W &h< O<8 2# 1#:46:26 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 -o*a89 rg=anager
ha@we:3@*) 3 %n8)ne9 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:1@*) (tarte>
re!arh)!eedbak*redhat%o( 60 """%redhat%o(
%2 Sto6 the *l.ster servi*es on the no/e ha-1e'/-ci an/ veri1y the stat.s2 +his *an >e r.n
1ro0 any *l.ster no/e
# ccs --host ha-web&-ci --stop
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W &h< O<8 2# 14:14:51 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 -o*a89 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %ff8)ne
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:1@*) (tarte>
$2 Re0ove the no/e 1ro0 the *l.ster *on1ig.ration an/ 6ro6agate the *hange to the
re0aining *l.ster 0e0>ers2 +8o-no/e *l.sters are a .niI.e *ase that reI.ire the
two_node an/ e+pected_votes 1lags to >e ena>le/2 &or all other *on1ig.rations
these 1lags /o not nee/ to >e ena>le/2 A >rie1 /o8nti0e is reI.ire/ to allo8 the *l.ster
no/es to syn*hroniHe an/ a*tivate the *hange2 +his *an >e r.n 1ro0 any *l.ster no/e
or the 0anage0ent server
# ccs --host ha-web8 --rmnode ha-web&-ci
# ccs --host ha-web8 --setcman two_node=8 e"pected_votes=8
# ccs --host ha-web8 --stopall
# ccs --host ha-web8 --s!nc --activate
# ccs --host ha-web8 --chec#conf
!88 no>e( )n (+n*.
#2 A*tivate the ne8 A2-no/eC *l.ster *on1ig.ration2 +he *l.ster servi*es 0.st >e restarte/
8hen /o8nsiHing to a t8o no/e *l.ster *on1ig.ration2 +his *an >e r.n 1ro0 any *l.ster
no/e or the 0anage0ent server
# ccs --host ha-web8 --startall
/tarte> ha@we:2@*)
/tarte> ha@we:1@*)
62 Rena0e the *l.ster *on1ig.ration 1ile on ha-8e>%-*i to 6revent it 1ro0 >eing
a**i/entally @oine/ to the *l.ster >y a**i/ent
# mv %etc%cluster%cluster*conf %etc%cluster%cluster*conf*disabled-0=-21-88
"""%redhat%o( 61 re!arh)!eedbak*redhat%o(
(.3 Relo+ating )luster *e Servi+es
7onga A:etho/ 1C
+he 1ollo8ing ste6s /es*ri>e the 6ro*ess o1 relo*ating a r.nning *l.ster 8e> servi*e Aha-1e'-
serviceC 1ro0 one *l.ster no/e to another .sing Conga an/ the luci 8e> inter1a*e2
12 (6en the luci 8e> inter1a*e an/ login .sing the root a**o.nt2 Un/er the $ervice
+roups ta> A/igure 2%0)$. Web Servie StatusC sele*t the lin, 1or the ha-1e'-service
(6ening a >ro8ser on the the *l.ster stat.s 6age also sho8s the 8e> servi*e as
r.nning on no/e ha-1e'5 A/igure 2%0)'. Web Servie <e!ore ReloateC
re!arh)!eedbak*redhat%o( 62 """%redhat%o(
Figure 5.3-1: Web Service Status
Figure 5.3-2: Web Service ?efore /elocate
22 Relo*ate the *l.ster 8e> servi*e to another *l.ster no/e >y 1irst sele*ting ha-1e'$-ci
1ro0 the no/e /ro6-/o8n list A/igure 2%0)0. Web Servie ReloateC2 A*tivate the
*hange >y sele*ting the start i*on to the right
%2 +he $ervice +roups ta> is .6/ate/ to re1le*t the *hange in servi*e lo*ation A/igure 2%0)
1. Web Servie ReloatedC
"""%redhat%o( 66 re!arh)!eedbak*redhat%o(
Figure 5.3-3: Web Service /elocate
Figure 5.3-4: Web Service /elocate&
Re1reshing the *l.ster stat.s 6age *on1ir0s the 8e> servi*e is no8 r.nning on *l.ster
no/e ha-1e'$ A/igure 2%0)2. Web Servie Status A!ter ReloateC
re!arh)!eedbak*redhat%o( 64 """%redhat%o(
Figure 5.3-5: Web Service Status 2fter /elocate
77S A:etho/ 2C
As an alternative4 *l.ster 8e> servi*es *an >e relo*ate/ 1ro0 one *l.ster no/e to another
1ro0 the *o00an/ line2 Any *l.ster no/e *an >e .se/ to relo*ate a servi*e2 +he 1ollo8ing
ste6s /e0onstrate ho8 to relo*ate the *l.ster 8e> servi*e Aha-web-serviceC 1ro0 no/e
ha-1e'$-ci to no/e ha-1e'5-ci .sing the *l.ster servi*e a/0inistration AclusvcadmC tool on
no/e ha-1e'/-ci2
12 Veri1y the *l.ster stat.s an/ *.rrent lo*ation o1 the 8e> servi*e 1ro0 no/e ha-1e'/-ci
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W &<e O<8 26 14:30:33 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tarte>
22 Relo*ate the servi*e 1ro0 no/e ha-1e'$-ci to no/e ha-1e'5-ci2 +his *an >e r.n this
1ro0 any *l.ster no/e >.t in this *ase no/e ha-1e'/-ci is .se/
# clusvcadm -r ha-web-service -m ha-web8-ci
&r+)ng to re8o*ate (erC)*e:ha@we:@(erC)*e to ha@we:1@*).../<**e((
(erC)*e:ha@we:@(erC)*e )( now r<nn)ng on ha@we:1@*)
%2 Veri1y the *l.ster stat.s an/ ne8 lo*ation o1 the 8e> servi*e 1ro0 no/e Aha-1e'/-ciC
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W &<e O<8 26 14:43:24 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:1@*) (tarte>
"""%redhat%o( 65 re!arh)!eedbak*redhat%o(
(.$ ,en+ing )luster Nodes
&en*ing a no/e *an >e /one to test the 1en*ing 0etho/ in .se2 (nly one *l.ster no/e at a
ti0e sho.l/ >e 1en*e/ to 6revent 6otential loss o1 I.or.02 9n the 1ollo8ing ste6s4 9P:9 -AN
1en*ing is teste/ >y 1en*ing no/e1 Aha-1e'5-ciC an/ 1or*ing the 8e> servi*e to relo*ate2 Sin*e
the 1en*ing >rie1ly 6o8ers /o8n no/e1 all 1en*ing an/ stat.s *o00an/s sho.l/ >e r.n 1ro0
either no/e2 Aha-1e'$-ciC or no/e% Aha-1e'/-ciC2 +he 1en*ing o1 no/es *an only >e
6er1or0e/ 1ro0 the *o00an/ line AC-9C2
12 Veri1y the *.rrent stat.s o1 the 8e> servi*e Aha-1e'-serviceC 1ro0 no/e% Aha-1e'/-ciC
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W e> O<8 23 11:1#:00 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:1@*) (tarte>
22 9ss.e the 1en*ing *o00an/ 1ro0 no/e% Aha-1e'/-ciC2 (6tionally the ver>ose 0o/e
o6tion A-vvC *an >e s6e*i1ie/
# fence_node -vv ha-web8-ci
fen*e ha@we:1@*) >eC 0.0 agent fen*e,)7=)8an re(<8t: (<**e((
agent arg(: no>ena=e=ha@we:1@*) agent=fen*e,)7=)8an a<th=7a((wor>
)7a>>r=10.16.143.231 8an78<(=on 8og)n=root 7a((w>=7a((wor> 7ower,wa)t=5
t)=eo<t=20
fen*e ha@we:1@*) (<**e((
%2 Che*, the stat.s as no/e1 Aha-1e'5-ciC is 6o8ere/ /o8n4 the 8e> servi*e Aha-1e'-
serviceC is relo*ate/ to no/e2 Aha-1e'$-ciC an/ no/e1 Aha-8e>1-*iC has 1.lly restarte/
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W e> O<8 23 11:1#:26 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %ff8)ne
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tart)ng
re!arh)!eedbak*redhat%o( 68 """%redhat%o(
+he 8e> servi*e Aha-1e'-serviceC has >een relo*ate/ to no/e2 Aha-1e'$-ciC4 has started
an/ is availa>le4 >.t no/e1 Aha-1e'5-ciC is still >ooting an/ &ffline
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W e> O<8 23 11:1#:54 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %ff8)ne
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tarte>
A1ter *l.ster no/e1 Aha-1e'5-ciC has *o06lete/ >ooting4 veri1y it is a 0e0>er o1 the *l.ster
# clustat
C8<(ter /tat<( for ha@we:@*8<(ter W e> O<8 23 11:22:20 2011
Me=:er /tat<(: P<orate
Me=:er Na=e ID /tat<(
@@@@@@ @@@@ @@@@ @@@@@@
ha@we:1@*) 1 %n8)ne9 rg=anager
ha@we:2@*) 2 %n8)ne9 rg=anager
ha@we:3@*) 3 %n8)ne9 -o*a89 rg=anager
/erC)*e Na=e %wner 6-a(t; /tate
@@@@@@@ @@@@ @@@@@ @@@@@@ @@@@@
(erC)*e:ha@we:@(erC)*e ha@we:2@*) (tarte>
"""%redhat%o( 4& re!arh)!eedbak*redhat%o(
(.( /'porting a )luster
906orting a *l.ster is .se/ 8hen a *on1ig.ration e'ists >.t the Conga AluciC 0anage0ent
inter1a*e has not i06orte/ the *on1ig.ration into the internal luci /ata>ase2 9n sit.ations
8here CCS 8as .se/ to >.il/ a *l.ster4 luci initially is not a8are o1 the *l.ster .ntil the
*on1ig.ration has >een i06orte/2 Per1or0 the 1ollo8ing ste6s to i06ort a *l.ster into the luci
inter1a*e
12 (6en the luci 8e> inter1a*e an/ login .sing the root a**o.nt2 Un/er Manage Clusters
sele*t Add A/igure 2%2)$. 7luster 3(portC
22 +he Add -.isting Cluster /ialog >o' o6ens2 Po6.late the 1iel/s as sho8n >elo8 in
/igure 2%2)'. Add Existing 7luster2 Sele*t Add Cluster 8hen *o06lete2
re!arh)!eedbak*redhat%o( 4$ """%redhat%o(
Figure 5.5-1: Cluster 3port
Figure 5.5-2: 2&& @$isting Cluster
%2 +he 3o0e>ase *on1ir0s the i06ort 8as s.**ess1.l A/igure 2%2)0. 7luster 3(port
Suess!ulC
$2 +he *l.ster is no8 i06orte/ an/ *an >e 0anage/ thro.gh the luci inter1a*e A/igure
2%2)1. 7luster ;ode StatusC
"""%redhat%o( 4' re!arh)!eedbak*redhat%o(
Figure 5.5-3: Cluster 3port Successful
Figure 5.5-4: Cluster 1o&e Status
6 Conclusion
Re/ 3atGs 3igh Availa>ility A//-(n 1or Re/ 3at 5nter6rise -in.' 6 is Re/ 3atGs 6re0ier
*l.stering sol.tion2 +he 3igh Availa>ility A//-(n 6rovi/es relia>ility4 availa>ility an/ s*ala>ility
ARASC to *riti*al 6ro/.*tion servi*es >y eli0inating single 6oints o1 1ail.re an/ 6rovi/ing
a.to0ati* failover o1 those servi*es in the event o1 a *l.ster no/e 1ail.re or error *on/ition2
+his re1eren*e ar*hite*t.re /etaile/ the /e6loy0ent4 *on1ig.ration an/ 0anage0ent o1 a
highly availa>le 8e> servi*e .sing the 3igh Availa>ility A//-(n 1or Re/ 3at 5nter6rise -in.'2
A three no/e *l.ster 8as /e6loye/ an/ *on1ig.re/ to .se a highly availa>le -V: A3A--V:C
vol.0e 1or the *l.ster 8e> servi*e *ontents2 A /e/i*ate/ 0anage0ent station 8as /e6loye/
to 0anage an/ *on1ig.re the *l.ster2 +his *on1ig.ration *an >e s*ale/ o.t >y a//ing 0ore
*l.ster no/es or *on1ig.re/ to r.n other *l.ster servi*es as reI.ire/ to >est 0eet the nee/s
o1 s6e*i1i* *o06.ting environ0ents2
Cl.ster *reation an/ 0anage0ent 8as /e0onstrate/ .sing >oth Conga an/ the luci
gra6hi*al .ser inter1a*e A=U9C4 an/ the Cl.ster Con1ig.ration Syste0 ACCSC *o00an/ line
inter1a*e AC-9C2 )oth 0etho/s *an >e .se/ inter*hangea>ly to *reate4 *on1ig.re an/ 0anage
a *l.ster2
+he 0ost *o00on *l.ster 0anage0ent tas,s Aa//ing4 re0oving no/es4 relo*ating *l.ster
servi*es4 1en*ing no/es4 i06orting an e'isting *l.sterC 8ere also /e0onstrate/2
re!arh)!eedbak*redhat%o( 40 """%redhat%o(
Appendix A: References
12 Re/ 3at 5nter6rise -in.' 6 9nstallation =.i/e
9nstalling Re/ 3at 5nter6rise -in.' 6 1or all ar*hite*t.res
5/ition 120
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6B6/1B
9nstallationR=.i/eBRe/R3atR5nter6riseR-in.'-6-9nstallationR=.i/e-en-US26/1
22 Re/ 3at 5nter6rise -in.' 6 De6loy0ent =.i/e
De6loy0ent4 Con1ig.ration an/ A/0inistration o1 Re/ 3at 5nter6rise -in.' 6
5/ition 120
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6B6/1B
De6loy0entR=.i/eBRe/R3atR5nter6riseR-in.'-6-De6loy0entR=.i/e-en-US26/1
%2 Re/ 3at 5nter6rise -in.' 6 Virt.aliHation =.i/e
=.i/e to Virt.aliHation on Re/ 3at 5nter6rise -in.' 6
5/ition %22
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6Bht0lB
Virt.aliHationBin/e'2ht0l
$2 Re/ 3at 5nter6rise -in.' 6 Cl.ster A/0inistration
Con1ig.ring an/ :anaging the 3igh Availa>ility A//-(n
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6B6/1B
Cl.sterRA/0inistrationBRe/R3atR5nter6riseR-in.'-6-Cl.sterRA/0inistration-en-US26/1
#2 Re/ 3at 5nter6rise -in.' 6 -ogi*al Vol.0e :anager A/0inistration
-V: A/0inistrator =.i/e
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6B6/1B
-ogi*alRVol.0eR:anagerRA/0inistrationBRe/R3atR5nter6riseR-in.'-6-
-ogi*alRVol.0eR:anagerRA/0inistration-en-US26/1
62 Re/ 3at 5nter6rise -in.' 6 D: :.lti6ath
D: :.lti6ath Con1ig.ration an/ A/0inistration
htt6BB/o*s2re/hat2*o0B/o*sBen-USBRe/R3atR5nter6riseR-in.'B6B6/1B
D:R:.lti6athBRe/R3atR5nter6riseR-in.'-6-D:R:.lti6ath-en-US26/1
72 J;hat is a 3ighly Availa>le -V: A3A--V:C *on1ig.ration an/ ho8 /o 9 i06le0ent itTK
Re/ 3at Qno8le/ge )ase Arti*le - Q)%068
htt6sBBa**ess2re/hat2*o0B,>B/o*sBD(C-%068
82 JUsing SCS9 Persistent Reservations 8ith Re/ 3at 5nter6rise -in.' 6
8ith the 3igh Availa>ility A//-(nK
Re/ 3at Qno8le/ge )ase Arti*le F Q)$%07"
htt6sBBa**ess2re/hat2*o0B,>B/o*sBD(C-$%07"
"""%redhat%o( 41 re!arh)!eedbak*redhat%o(
Appendix B: Red Hat Enterprise Linux 6
Satellite Configuration Details
+he /e6loy0ent o1 Re/ 3at 5nter6rise -in.' 6 on the *l.sters no/es 8as /one .sing a lo*al
Satellite server *onne*te/ to Re/ 3at Net8or, AR3NC2 +he 1ollo8ing ste6s /es*ri>e
*on1ig.ring the Satellite server in 6re6aration o1 the *l.ster no/e /e6loy0ents2
12 (6en the Satellite server 8e> inter1a*e an/ login A/igure $. Satellite @vervie"C
re!arh)!eedbak*redhat%o( 42 """%redhat%o(
Figure 1: Satellite Avervie#
22 Create an A*tivation Qey2 Un/er the $&stems ta> sele*t Activation /e&s A/igure '.
Ativation AeysC
Sele*t create new ke&2
"""%redhat%o( 46 re!arh)!eedbak*redhat%o(
Figure 2: 2ctivation Beys
%2 +he Create Activation /e& s*reen is /is6laye/ A/igure 0. Ativation Aey DetailsC2
Co06lete the 1iel/s as sho8n 1or Description4 /e&4 0ase Channels4 Add12n
-ntitlements2 -eave all other 1iel/s as /e1a.lts
Sele*t Create Activation /e& 8hen /one2
re!arh)!eedbak*redhat%o( 44 """%redhat%o(
Figure 3: 2ctivation Bey Details
$2 Sele*t Chil/ Channels2 Un/er the Child Channels ta> sele*t the *hannels reI.ire/ to
/e6loy the *l.ster no/es A/igure 1. 7hild 7hannelsC
R35- Server 3igh Availa>ility Av2 6 1or 6$->it '86R6$C
R3N +ools 1or R35- Av2 6 1or 6$->it '86R6$C
Sele*t Update /e& 8hen /one2
"""%redhat%o( 45 re!arh)!eedbak*redhat%o(
Figure 4: C%il& C%annels
#2 Create a Qi*,start 6ro1ile2 Un/er the $&stems ta> sele*t /ickstart A/igure 2. 7reate
;e" Aikstart Pro!ileC
Sele*t create new kickstart profile.
re!arh)!eedbak*redhat%o( 48 """%redhat%o(
Figure 5: Create 1e# Bic"start 9rofile
62 +he Create Qi*,start Pro1ile 6age o6ens A/igure 6. 7reate Aikstart Pro!ile ) Step $C2
5nter %abel4 0ase Channel an/ /ickstartable 'ree
Cli*, Ne.t 8hen /one2
"""%redhat%o( 5& re!arh)!eedbak*redhat%o(
Figure 4: Create Bic"start 9rofile - Step 1
72 +he Distri>.tion &ile -o*ation 6age A/igure 4. 7reate Aikstart Pro!ile ) Step 'C
o6ens2 A**e6t all /e1a.lts
Sele*t Ne.t to *ontin.e2
re!arh)!eedbak*redhat%o( 5$ """%redhat%o(
Figure 5: Create Bic"start 9rofile - Step 2
82 +he Root Pass8or/ 6age o6ens A/igure 5. 7reate Aikstart Pro!ile ) Step 0C2
5nter the root 6ass8or/ 1or the *l.ster no/es
Sele*t Finish 8hen /one2
"""%redhat%o( 5' re!arh)!eedbak*redhat%o(
Figure 6: Create Bic"start 9rofile - Step 3
"2 5na>le S5-in.'2 Un/er the /ickstart details ta> sele*t Advanced 2ptions2 Ne't sele*t
the ha-1e'-cluster-nodes 6ro1ile A/igure 8. Aikstart Pro!ileC
S*roll /o8n to the o6tion selinu.3 an/ ena>le it A/igure $&. SELinux En!oringC
Sele*t Update /ickstart 8hen /one2
re!arh)!eedbak*redhat%o( 50 """%redhat%o(
Figure 7: Bic"start 9rofile
Figure 18: S@;inu$ @nforcing
102 Asso*iate the A*tivation Qey 8ith the ,i*,start 6ro1ile2 Un/er the Activation /e&s ta>
sele*t the *he*, >o' ne't to ha-1e'-cluster-nodes-(ey A/igure $$. Assoiate
Ativation Aey to Aikstart Pro!ileC
Sele*t Update Activation /e&2 +his *o06letes the *on1ig.ration o1 the Satellite server in
6re6aration o1 the *l.ster no/e /e6loy0ents2
"""%redhat%o( 51 re!arh)!eedbak*redhat%o(
Figure 11: 2ssociate 2ctivation Bey to Bic"start 9rofile
Appendix C: Fibre Channel Storage
Provisioning
An 62-L@( vol.0e A<!$C is .se/ 1or storing the *ontent o1 the *l.ster 8e> servi*e2 +he
share/ vol.0e is *reate/ on an 2A Storage4or(s !S3$/$.fc &i>re Channel storage array2
+he array *ontains a single *ontroller AAorts 35, 3$C an/ an !S3C6 e'6ansion shel1 6rovi/ing
a total o1 $8 6hysi*al /rives2 +he ste6s >elo8 /es*ri>e ho8 to 6rovision a 20=) vol.0e
Aha-web-data-#1C 8ithin a ne8 virt.al /is, A<D5C 1ro0 the *o00an/ line2
Ste6 12 -ogin to the :SA storage array
# ssh -I -l manage ra-msa20
Ste6 22 Vie8 the availa>le virt.al /is,s Av/is,C4 6hysi*al /is,s an/ vol.0es
# show vdis#
# show dis#
# show volumes
Ste6 %2 Create a virt.al /is, to hol/ the vol.0e
# create vdis# level raid6 dis#s 8*8-82 @J8
Ste6 $2 Create a vol.0e 8ithin the virt.al /is,
# create volume vdis# @J8 siBe 20AC access no-access lun 8 ha-web-data-08
Ste6 #2 Vie8 the *.rrent host *onne*tion 0a66ings
# show hosts
Ho(t ID Na=e D)(*oCere> Ma77e> 'rof)8e
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
50060$0000C2#60# rhe8h@01,1 2e( No /tan>ar>
50060$0000C2#60! rhe8h@01,2 2e( No /tan>ar>
50060$0000C2#63E :k<7,ho(t1 2e( No /tan>ar>
50060$0000C2#63C :k<7,ho(t0 2e( No /tan>ar>
50060$0000C2#614 rheCh@01,1 2e( No /tan>ar>
50060$0000C2#644 2e( No /tan>ar>
50060$0000C2#616 rheCh@01,2 2e( No /tan>ar>
50060$0000C2#646 2e( No /tan>ar>
50060$0000C2#600 =g=t1,ho(t0 2e( No /tan>ar>
50060$0000C2#602 =g=t1,ho(t1 2e( No /tan>ar>
50060$0000C2#640 2e( No /tan>ar>
50060$0000C2#642 2e( No /tan>ar>
50060$0000C2#61! 2e( No /tan>ar>
50060$0000C2#634 2e( No /tan>ar>
50060$0000C2#636 2e( No /tan>ar>
50060$0000C2#62C 2e( No /tan>ar>
50060$0000C2#62E 2e( No /tan>ar>
50060$0000C2#62# 2e( No /tan>ar>
50060$0000C2#62! 2e( No /tan>ar>
50060$0000C2#612 2e( No /tan>ar>
re!arh)!eedbak*redhat%o( 52 """%redhat%o(
50060$0000C2#610 2e( No /tan>ar>
21?D00051E543C43 2e( No /tan>ar>
50060$0000C2#63! te(t1,ho(t1 No No /tan>ar>
50060$0000C2#63# te(t1,ho(t0 No No /tan>ar>
500143#000C3E622 =onet,ho(t0 No No /tan>ar>
500143#000C3E63E =onet,ho(t1 No No /tan>ar>
50060$0000C2#632 rheCh@02,2 No No /tan>ar>
50060$0000C2#630 rheCh@02,1 No No /tan>ar>
50060$0000C2#626 rhe8h@02,2 No No /tan>ar>
50060$0000C2#624 rhe8h@02,1 No No /tan>ar>
50060$0000C2#606 =g=t2,ho(t1 No No /tan>ar>
50060$0000C2#604 =g=t2,ho(t0 No No /tan>ar>
Ste6 62 9/enti1y 8hat 6orts ea*h host is .sing
ha-1e'5.
# cat %s!s%class%fc_host%host0%port_name
0F50060:0000*2#62#
# cat %s!s%class%fc_host%host8%port_name
0F50060:0000*2#62!
ha-1e'$.
# cat %s!s%class%fc_host%host0%port_name
0F50060:0000*2#62C
# cat %s!s%class%fc_host%host8%port_name
0F50060:0000*2#62E
ha-1e'/.
# cat %s!s%class%fc_host%host0%port_name
0F50060:0000*2#634
# cat %s!s%class%fc_host%host8%port_name
0F50060:0000*2#636
Note 91 the &i>re Channel storage array has t8o *ontrollers atta*he/ to the SAN 1a>ri* then
ea*h host has 1o.r 6ort *onne*tions instea/ o1 the t8o sho8n here2
Ste6 72 Con1ig.re the host-na0e i/Gs to the 6orts an/ veri1y
# set host-name id ;0060C0000321621 ha-web8-host0 # Contro88er ! 'ort 1
# set host-name id ;0060C0000321622 ha-web8-host8 # Contro88er ! 'ort 2
# set host-name id ;0060C0000321623 ha-web2-host0 # Contro88er ! 'ort 1
# set host-name id ;0060C0000321624 ha-web2-host8 # Contro88er ! 'ort 2
# set host-name id ;0060C00003216&4 ha-web&-host0 # Contro88er ! 'ort 1
# set host-name id ;0060C00003216&6 ha-web&-host8 # Contro88er ! 'ort 2
# show hosts
Ho(t ID Na=e D)(*oCere> Ma77e> 'rof)8e
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
50060$0000C2#60# rhe8h@01,1 2e( No /tan>ar>
50060$0000C2#60! rhe8h@01,2 2e( No /tan>ar>
50060$0000C2#63E :k<7,ho(t1 2e( No /tan>ar>
50060$0000C2#63C :k<7,ho(t0 2e( No /tan>ar>
50060$0000C2#614 rheCh@01,1 2e( No /tan>ar>
50060$0000C2#644 2e( No /tan>ar>
50060$0000C2#616 rheCh@01,2 2e( No /tan>ar>
"""%redhat%o( 56 re!arh)!eedbak*redhat%o(
50060$0000C2#646 2e( No /tan>ar>
50060$0000C2#600 =g=t1,ho(t0 2e( No /tan>ar>
50060$0000C2#602 =g=t1,ho(t1 2e( No /tan>ar>
50060$0000C2#640 2e( No /tan>ar>
50060$0000C2#642 2e( No /tan>ar>
50060$0000C2#61! 2e( No /tan>ar>
50060$0000C2#634 ha@we:3@ho(t0 2e( No /tan>ar>
50060$0000C2#636 ha@we:3@ho(t1 2e( No /tan>ar>
50060$0000C2#62C ha@we:2@ho(t0 2e( No /tan>ar>
50060$0000C2#62E ha@we:2@ho(t1 2e( No /tan>ar>
50060$0000C2#62# ha@we:1@ho(t0 2e( No /tan>ar>
50060$0000C2#62! ha@we:1@ho(t1 2e( No /tan>ar>
50060$0000C2#612 2e( No /tan>ar>
50060$0000C2#610 2e( No /tan>ar>
21?D00051E543C43 2e( No /tan>ar>
Ste6 82 Vie8 the vol.0es
# show volumes
V>)(k Na=e /)He /er)a8 N<=:er " 'o8)*+
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
VD1 ha@we:@>ata@01 14.41$ 00*0ff>a:3f400005*e#f54>01000000 wr)te@:a*k
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Ste6 "2 Restri*t vol.0e a**ess to the three *l.ster no/es
# map volume access read-write lun 8 ports 28G22
host ha-web8-host0G ha-web8-host8G
ha-web2-host0G ha-web2-host8G
ha-web&-host0Gha-web&-host8
ha-web-data-08
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:1@ho(t0 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:1@ho(t1 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:2@ho(t0 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:2@ho(t1 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:3@ho(t0 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
Info: Co==an> *o=78ete> (<**e((f<88+. @ Ma77)ng (<**ee>e>.
Ho(t ha@we:3@ho(t1 wa( =a77e> for Co8<=e ha@we:@>ata@01 w)th -5N 1.
/<**e((: Co==an> *o=78ete> (<**e((f<88+. @ &he Co8<=e6(; were =a77e>
(<**e((f<88+.
Ste6 102 Veri1y vol.0e an/ host 0a66ings
# show volume-map
Info: "etr)eC)ng >ata...
Vo8<=e V)ew D/er)a8 N<=:er 600*0ff>a:3f400005*e#f54>01000000; Na=e 6ha@we:@
>ata@01; E Ma77)ng:
'ort( -5N !**e(( Ho(t@'ort@I>ent)f)er N)*kna=e 'rof)8e
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
!19!2 1 rea>@wr)te 50060$0000C2#62# ha@we:1@ho(t0 /tan>ar>
!19!2 1 rea>@wr)te 50060$0000C2#62! ha@we:1@ho(t1 /tan>ar>
re!arh)!eedbak*redhat%o( 54 """%redhat%o(
!19!2 1 rea>@wr)te 50060$0000C2#62C ha@we:2@ho(t0 /tan>ar>
!19!2 1 rea>@wr)te 50060$0000C2#62E ha@we:2@ho(t1 /tan>ar>
!19!2 1 rea>@wr)te 50060$0000C2#634 ha@we:3@ho(t0 /tan>ar>
!19!2 1 rea>@wr)te 50060$0000C2#636 ha@we:3@ho(t1 /tan>ar>
not@=a77e> a88 other ho(t( /tan>ar>
# show host-map
Ho(t V)ew DID 650060$0000C2#634; Na=e 6ha@we:3@ho(t0; 'rof)8e 6/tan>ar>; E
Ma77)ng:
Na=e /er)a8 N<=:er -5N !**e(( 'ort(
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ha@we:@>ata@01 00*0ff>a:3f400005*e#f54>01000000 1 rea>@wr)te !19!2
Ho(t V)ew DID 650060$0000C2#636; Na=e 6ha@we:3@ho(t1; 'rof)8e 6/tan>ar>; E
Ma77)ng:
Na=e /er)a8 N<=:er -5N !**e(( 'ort(
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ha@we:@>ata@01 00*0ff>a:3f400005*e#f54>01000000 1 rea>@wr)te !19!2
Ho(t V)ew DID 650060$0000C2#62C; Na=e 6ha@we:2@ho(t0; 'rof)8e 6/tan>ar>; E
Ma77)ng:
Na=e /er)a8 N<=:er -5N !**e(( 'ort(
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ha@we:@>ata@01 00*0ff>a:3f400005*e#f54>01000000 1 rea>@wr)te !19!2
Ho(t V)ew DID 650060$0000C2#62E; Na=e 6ha@we:2@ho(t1; 'rof)8e 6/tan>ar>; E
Ma77)ng:
Na=e /er)a8 N<=:er -5N !**e(( 'ort(
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
ha@we:@>ata@01 00*0ff>a:3f400005*e#f54>01000000 1 rea>@wr)te !19!2
Ste6 112 &ro0 ea*h o1 the *l.ster no/es4 /eter0ine 8hi*h /evi*e 1iles are *on1ig.re/ 1or the
20 =) &i>re Channel /is, - *dev*sd', *dev*sdc, *dev*sdd, *dev*sde
# fdis# -l 2:%dev%null > grep 9KJis# %dev%sd9
D)(k B>eCB(>a: 146.# 1$9 14633#6#5440 :+te(
D)(k B>eCB(>:: 20.0 1$9 14444443452 :+te(
D)(k B>eCB(>*: 20.0 1$9 14444443452 :+te(
D)(k B>eCB(>>: 20.0 1$9 14444443452 :+te(
D)(k B>eCB(>e: 20.0 1$9 14444443452 :+te(
Ste6 122 Veri1y the ;orl/ ;i/e 9DGs A;;9DC 0at*h 1or ea*h /evi*e2 +he ;;9DGs 0.st >e
the sa0e a*ross ea*h *l.ster no/e
# %lib%udev%scsi_id --whitelisted --device=%dev%sdb
3600*0ff000>a:3f45*e#f54>01000000
# %lib%udev%scsi_id --whitelisted --device=%dev%sdc
3600*0ff000>a:3f45*e#f54>01000000
# %lib%udev%scsi_id --whitelisted --device=%dev%sdd
3600*0ff000>a:3f45*e#f54>01000000
# %lib%udev%scsi_id --whitelisted --device=%dev%sde
3600*0ff000>a:3f45*e#f54>01000000
"""%redhat%o( 55 re!arh)!eedbak*redhat%o(
Appendix D: Cluster Configuration File
(cluster.conf)
+,-*. /&0#on=)1.0),1
+$.2!&0 $on3#4(/&0#on=)65) n"*&=)6"78&b7$.2!&0)1
+$.2!&0nod&1
+$.2!&0nod& n"*&=)6"78&b17$#) nod&#d=)1)1
+3&n$&1
+*&!6od n"*&=)P0#*"0%)1
+d&/#$& n"*&=)IPMI76"78&b17$#)91
+9*&!6od1
+93&n$&1
+9$.2!&0nod&1
+$.2!&0nod& n"*&=)6"78&b27$#) nod&#d=)2)1
+3&n$&1
+*&!6od n"*&=)P0#*"0%)1
+d&/#$& n"*&=)IPMI76"78&b27$#)91
+9*&!6od1
+93&n$&1
+9$.2!&0nod&1
+$.2!&0nod& n"*&=)6"78&b:7$#) nod&#d=):)1
+3&n$&1
+*&!6od n"*&=)P0#*"0%)1
+d&/#$& n"*&=)IPMI76"78&b:7$#)91
+9*&!6od1
+93&n$&1
+9$.2!&0nod&1
+9$.2!&0nod&1
+3&n$&d&/#$&1
+3&n$&d&/#$& "4&n!=)3&n$&(#;*#."n) "2!6=);"8o0d)
#;"dd0=)10.16.1<:.2:1) ."n;.2=)on) .o4#n=)0oo!)
n"*&=)IPMI76"78&b17$#) ;"8d=);"8o0d)
;o8&0(8"#!=)5) !#*&o2!=)20)91
+3&n$&d&/#$& "4&n!=)3&n$&(#;*#."n) "2!6=);"8o0d)
#;"dd0=)10.16.1<:.2:2) ."n;.2=)on) .o4#n=)0oo!)
n"*&=)IPMI76"78&b27$#) ;"8d=);"8o0d)
;o8&0(8"#!=)5) !#*&o2!=)20)91
+3&n$&d&/#$& "4&n!=)3&n$&(#;*#."n) "2!6=);"8o0d)
#;"dd0=)10.16.1<:.2::) ."n;.2=)on) .o4#n=)0oo!)
n"*&=)IPMI76"78&b:7$#) ;"8d=);"8o0d)
;o8&0(8"#!=)5) !#*&o2!=)20)91
+93&n$&d&/#$&1
+0*1
+3"#.o/&0do*"#n1
+3"#.o/&0do*"#n n"*&=)6"78&b73"#.o/&0) no3"#.b"$==)0)
o0d&0&d=)1) 0&!0#$!&d=)0)1
+3"#.o/&0do*"#nnod& n"*&=)6"78&b17$#) ;0#o0#!%=)1)91
+3"#.o/&0do*"#nnod& n"*&=)6"78&b27$#) ;0#o0#!%=)2)91
+3"#.o/&0do*"#nnod& n"*&=)6"78&b:7$#) ;0#o0#!%=):)91
+93"#.o/&0do*"#n1
+93"#.o/&0do*"#n1
+0&o20$&1
re!arh)!eedbak*redhat%o( 58 """%redhat%o(
+#; "dd0&=)10.16.1<:.150) *on#!o0(.#n==)on) .&&;!#*&=)10)91
+./* ./(n"*&=)6"78&b7./o.1) n"*&=)6"78&b7>A7LVM)
&.3(3&n$&=)on) /4(n"*&=)>A7?&b7V')91
+3 d&/#$&=)9d&/9>A7?&b7V'96"78&b7./o.1) 3#d=)56<:2)
3!%;&=)&-!<) *o2n!;o#n!=)96"96"78&b)
n"*&=)6"78&b73#.&%!&*) &.3(3&n$&=)on)91
+$0#;! 3#.&=)9&!$9#n#!.d96!!;d) n"*&=)6"78&b7";"$6&7$0#;!)91
+90&o20$&1
+&0/#$& do*"#n=)6"78&b73"#.o/&0)
*"-(0&!"0!=):) n"*&=)6"78&b7&0/#$&)
0&$o/&0%=)0&!"0!) 0&!"0!(&-;#0&(!#*&=):600)1
+#; 0&3=)10.16.1<:.150)91
+./* 0&3=)6"78&b7>A7LVM)91
+3 0&3=)6"78&b73#.&%!&*)91
+$0#;! 0&3=)6"78&b7";"$6&7$0#;!)91
+9&0/#$&1
+90*1
+9$.2!&01
"""%redhat%o( 8& re!arh)!eedbak*redhat%o(
Appendix E: HA-Web Status Script
+he 3A-;e>-Stat.s s*ri6t is .se/ to veri1y that the 8e> servi*e an/ reI.ire/ reso.r*es
are 1.n*tioning *orre*tly2 +he s*ri6t is also .se1.l 1or testing the 8e> servi*e a1ter the servi*e
has >een relo*ate/ or a no/e 1ailover has o**.rre/2 Con1ig.re the s*ri6t on the 3A--V:
vol.0e as 1ollo8s
12 -ogin to the no/e that *.rrently has the 3A--V: vol.0e 0o.nte/ as *ha*ha-1e'
22 Create the .cgi-'in /ire*tory to hol/ the s*ri6t
# m#dir -p %ha%ha-web%*cgi-bin
%2 Create the 1ile *ha*ha-1e'*.cgi-'in*23-4e'-Status 8ith the 1ollo8ing *ontents
#XB:)nB:a(h
7r)ntf "Content@t+7e: teFtB78a)nYn"
B:)nB*at Bet*B=ot>
7r)ntf "AAA &he *<rrent t)=e )( JB:)nB>ateJ AAAYnYn"
$2 Set the 1ile 6er0issions on the 2*gi->in /ire*tory an/ 1ile so the s*ri6t *an >e r.n
# chmod =;; %ha%ha-web%*cgi-bin
# chmod =;; %ha%ha-web%*cgi-bin%62-?eb-Ftatus
#2 (n ea*h *l.ster no/e4 e/it *etc*httpd*conf*httpd.conf an/ *hange ,criptAlias to
the *ha*ha-1e'*.cgi-'in /ire*tory on the 3A--V: vol.0e
/*r)7t!8)a( B*g)@:)nB "BhaBha@we:B.*g)@:)nB"
Also set the 2*gi->in irectory entry
ZD)re*tor+ "BhaBha@we:B.*g)@:)nB"[
!88ow%Cerr)>e None
%7t)on( None
%r>er a88ow9>en+
!88ow fro= a88
ZBD)re*tor+[
re!arh)!eedbak*redhat%o( 8$ """%redhat%o(
62 +est the s*ri6t 1ro0 the *o00an/ line
# )%a)%a-#eb).cgi-bin):2-Web-Status
Content-t45e: te%t25lain
()N 7atellite 8ic8start on 2011-06-21
999 :elcome to ha-web1 999
ha-web1 is the +irst node o+ the );-:eb Cluster (e+erence ;rchitecture
0ro1ect
999 *he current time is *ue ;u ! 16:45:16 ,<* 2011 999
72 (n*e the *l.ster has >een *reate/ an/ the 8e> servi*e *on1ig.re/4 the s*ri6t *an >e
r.n 1ro0 the 8e> servi*e2 (6en a >ro8ser 8in/o8 onto the 8e> servi*e
Ahttp@56.59.5./.5B6*cgi-'in*23-4e'-StatusC an/ r.nning the test s*ri6t A/igure $. HA)
Web)StatusC
"""%redhat%o( 8' re!arh)!eedbak*redhat%o(
Figure 1: :2-Web-Status
Appendix F: Cluster Configuration Matrix
ha)"eb$ ha)"eb' ha)"eb0
Nodes
;ode ;a(e
3P Address
A*l.ster inter*onne*tC
ha-8e>1-*i
1"221682121#1
ha-8e>2-*i
1"221682121#2
ha-8e>%-*i
1"221682121#%
Rii Hostna(e
3P Address
A6.>li* inter1a*eC
ha-8e>1
1021621$%21#1
ha-8e>2
1021621$%21#2
ha-8e>%
1021621$%21#%
,en+ing
&en*e +y6e 9P:9 -an 9P:9 -an 9P:9 -an
&en*e Devi*e
Na0e
9P:9-ha-8e>1-*i 9P:9-ha-8e>2-*i 9P:9-ha-8e>%-*i
&en*e Devi*e
9P A//ress
1021621$%22%1 1021621$%22%2 1021621$%22%%
&en*e :etho/ Na0e Pri0ary Pri0ary Pri0ary
&en*e 9nstan*e 9P:9-ha-8e>1-*i 9P:9-ha-8e>2-*i 9P:9-ha-8e>%-*i
,ailover
/ailover Do(ain ha-8e>-1ailover ha-8e>-1ailover ha-8e>-1ailover
/ailover
6ype
PrioritiHe/ PrioritiHe/ PrioritiHe/
/ailover Priority 1 2 %
Resour+es
3P Address 1021621$%21#0 1021621$%21#0 1021621$%21#0
HA)L#M
Vol.0e =ro.6
-ogi*al Vol.0e
ha-8e>-3A--V:
3A-;e>-V=
ha-8e>-lvol1
ha-8e>-3A--V:
3A-;e>-V=
ha-8e>-lvol1
ha-8e>-3A--V:
3A-;e>-V=
ha-8e>-lvol1
/ilesyste(
+y6e
:o.nt 6oint
Devi*e
ha-8e>-1ilesyste0
e't$
BhaBha-8e>
B/evB3A-;e>-V=Bha-
8e>-lvol1
ha-8e>-1ilesyste0
e't$
BhaBha-8e>
B/evB3A-;e>-V=Bha-
8e>-lvol1
ha-8e>-1ilesyste0
e't$
BhaBha-8e>
B/evB3A-;e>-V=Bha-
8e>-lvol1
Sript
Path
ha-8e>-a6a*he-s*ri6t
Bet*Binit2/Bhtt6/
ha-8e>-a6a*he-s*ri6t
Bet*Binit2/Bhtt6/
ha-8e>-a6a*he-s*ri6t
Bet*Binit2/Bhtt6/
Servi+e 0roup
Servie ha-8e>-servi*e ha-8e>-servi*e ha-8e>-servi*e
#&&eni' (: Cluster Configuration Matri'
re!arh)!eedbak*redhat%o( 80 """%redhat%o(
Appendix G: Deployment Checklists
6ask 6ask Desription Loation Details
-eplo!'ent .asks 1 &anage'ent Server
1 9nstall Re/ 3at 5nter6rise -in.' 6 ha-1e'-mgmt Se*tion $2221
2 Con1ig.re Net8or,s ha-1e'-mgmt Se*tion $2222
% Con1ig.re &ire8all ha-1e'-mgmt Se*tion $222%
$ 9nstall Cl.ster :anage0ent So1t8are ha-1e'-mgmt Se*tion $222$
-eplo!'ent .asks 1 )luster Nodes
# 9nstall Re/ 3at 5nter6rise -in.' 6
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%21
6 Con1ig.re Net8or,s an/ )on/ing
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%22
7 Con1ig.re &ire8all
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%2%
8 9nstall Cl.ster No/e So1t8are
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%2$
" Con1ig.re Storage
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%2#
10 Con1ig.re ;e> Server
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%26
11 Con1ig.re S5-in.' Se*.rity Para0eters
ha-1e'5
ha-1e'$
ha-1e'/
Se*tion $2%27
#&&eni' ): *e&lo$ment Chec+list
"""%redhat%o( 81 re!arh)!eedbak*redhat%o(
6ask 6ask Desription Loation Details
)luster )reation .asks 1 )onga 2&ethod 13
12 Create Cl.ster ha-1e'-mgmt Se*tion $2$21
1% A// &en*e Devi*es ha-1e'-mgmt Se*tion $2$22
1$ A// &ailover Do0ain ha-1e'-mgmt Se*tion $2$2%
1# A// Reso.r*es ha-1e'-mgmt Se*tion $2$2$
16 A// Servi*e =ro.6 ha-1e'-mgmt Se*tion $2$2#
17 Veri1y Cl.ster ;e> Servi*e ha-1e'-mgmt Se*tion $2$26
)luster )reation .asks 1 ++s 2&ethod 23
12 Create Cl.ster ha-1e'-mgmt Se*tion $2#21
1% A// No/es ha-1e'-mgmt Se*tion $2#22
1$ A// &en*e Devi*es ha-1e'-mgmt Se*tion $2#2%
1# A// &ailover Do0ain ha-1e'-mgmt Se*tion $2#2$
16 A// Reso.r*es ha-1e'-mgmt Se*tion $2#2#
17 A// Servi*e =ro.6 ha-1e'-mgmt Se*tion $2#26
18 A*tivate Cl.ster ha-1e'-mgmt Se*tion $2#27
1" Veri1y Cl.ster ;e> Servi*e ha-1e'-mgmt Se*tion $2#28
#&&eni' ): *e&lo$ment Chec+list ,continue-
re!arh)!eedbak*redhat%o( 82 """%redhat%o(