Red Hat Cluster Suite Overview

5.1
Red Hat Cluster Suite
for Red Hat
Enterprise Linux 5.1
lSBN: NlA
Publication date:
Red Hat Cíuster Suíte Overvíew provídes an overvíew of Red Hat Cíuster Suíte for
Red Hat Enterpríse Línux 5.1
Red Hat Cluster Suite Overview
Red Hat Cluster Suite Overview: Red Hat Cluster
Suite for Red Hat Enterprise Linux 5.1
Copyríght © 2007 Red Hat, Inc.
Copyríght © 2007 by Red Hat, Inc. Thís materíaí may be dístríbuted oníy subíect to the terms and
condítíons set forth ín the Open Pubíícatíon Lícense, V1.0 or íater (the íatest versíon ís presentíy avaííabíe
at http://www.opencontent.org/openpub/).
Dístríbutíon of substantíveíy modífíed versíons of thís document ís prohíbíted wíthout the expíícít
permíssíon of the copyríght hoíder.
Dístríbutíon of the work or derívatíve of the work ín any standard (paper) book form for commercíaí
purposes ís prohíbíted uníess príor permíssíon ís obtaíned from the copyríght hoíder.
Red Hat and the Red Hat "Shadow Man" íogo are regístered trademarks of Red Hat, Inc. ín the Uníted
States and other countríes.
Aíí other trademarks referenced hereín are the property of theír respectíve owners.
The GPG fíngerprínt of the securíty@redhat.com key ís:
CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E
1801 Varsíty Dríve
Raíeígh, NC 27606-2072
USA
Phone: +1 919 754 3700
Phone: 888 733 4281
Fax: +1 919 754 3701
PO Box 13588
Research Tríangíe Park, NC 27709
USA
Red Hat Cluster Suite Overview
Introductíon .................................................................................................... víí
1. Document Conventíons ...................................................................... vííí
2. Feedback ............................................................................................... x
1. Red Hat Cíuster Suíte Overvíew ....................................................................1
1. Cíuster Basícs ........................................................................................1
2. Red Hat Cíuster Suíte Introductíon ........................................................3
3. Cíuster Infrastructure ............................................................................4
3.1. Cíuster Management ..................................................................5
3.2. Lock Management ......................................................................6
3.3. Fencíng .......................................................................................6
3.4. Cíuster Confíguratíon System ...................................................12
4. Hígh-avaííabíííty Servíce Management ................................................14
5. Red Hat GFS ........................................................................................17
5.1. Superíor Performance and Scaíabíííty .......................................19
5.2. Performance, Scaíabíííty, Moderate Príce .................................19
5.3. Economy and Performance .......................................................20
6. Cíuster Logícaí Voíume Manager .........................................................21
7. Gíobaí Network Bíock Devíce ..............................................................25
8. Línux Vírtuaí Server .............................................................................26
8.1. Two-Tíer LVS Topoíogy .............................................................28
8.2. Three-Tíer LVS Topoíogy ...........................................................31
8.3. Routíng Methods .......................................................................33
8.4. Persístence and Fírewaíí Marks .................................................36
9. Cíuster Admínístratíon Tooís ...............................................................37
9.1. Conga .......................................................................................38
9.2. Cíuster Admínístratíon GUI .......................................................41
9.3. Command Líne Admínístratíon Tooís ........................................45
10. Línux Vírtuaí Server Admínístratíon GUI ............................................46
10.1. CONTROLlMONlTORlNG ......................................................48
10.2. GLOBAL SETTlNGS ...............................................................49
10.3. REDUNDANCY .......................................................................50
10.4. VlRTUAL SERVERS ...............................................................52
2. Red Hat Cíuster Suíte Component Summary ..............................................61
1. Cíuster Components ............................................................................61
2. Man Pages ...........................................................................................68
3. Compatíbíe Hardware ..........................................................................71
Index ...............................................................................................................73
v

lntroduction
Thís document provídes a hígh-íeveí overvíew of Red Hat Cíuster Suíte for Red Hat
Enterpríse Línux 5 and ís ís organízed as foííows:
º Chapter J, Red Hat Cluster 5uite Overview
º Chapter 2, Red Hat Cluster 5uite Component 5ummary
Aíthough the ínformatíon ín thís document ís an overvíew, you shouíd have
advanced workíng knowíedge of Red Hat Enterpríse Línux and understand the
concepts of server computíng to gaín a good comprehensíon of the ínformatíon.
For more ínformatíon about usíng Red Hat Enterpríse Línux, refer to the foííowíng
resources:
º Red Hat Enterprise Linux lnstallation Cuide - Provídes ínformatíon regardíng
ínstaííatíon of Red Hat Enterpríse Línux 5.
º Red Hat Enterprise Linux Deployment Cuide - Provídes ínformatíon regardíng the
depíoyment, confíguratíon and admínístratíon of Red Hat Enterpríse Línux 5.
For more ínformatíon about Red Hat Cíuster Suíte for Red Hat Enterpríse Línux 5,
refer to the foííowíng resources:
º Configuring and Managing a Red Hat Cluster - Provídes ínformatíon about
ínstaíííng, confíguríng and managíng Red Hat Cíuster components.
º LVM Administrator's Cuide: Configuration and Administration - Provídes a
descríptíon of the Logícaí Voíume Manager (LVM), íncíudíng ínformatíon on
runníng LVM ín a cíustered envíronment.
º Clobal File 5ystem: Configuration and Administration - Provídes ínformatíon
about ínstaíííng, confíguríng, and maíntaíníng Red Hat GFS (Red Hat Gíobaí Fííe
System).
º Using Device-Mapper Multipath - Provídes ínformatíon about usíng the
Devíce-Mapper Muítípath feature of Red Hat Enterpríse Línux 5.
º Using CN8D with Clobal File 5ystem - Provídes an overvíew on usíng Gíobaí
Network Bíock Devíce (GNBD) wíth Red Hat GFS.
víí
º Linux Virtual 5erver Administration - Provídes ínformatíon on confíguríng
hígh-performance systems and servíces wíth the Línux Vírtuaí Server (LVS).
º Red Hat Cluster 5uite Release Notes - Provídes ínformatíon about the current
reíease of Red Hat Cíuster Suíte.
Red Hat Cíuster Suíte documentatíon and other Red Hat documents are avaííabíe ín
HTML, PDF, and RPM versíons on the Red Hat Enterpríse Línux Documentatíon CD
and onííne at http://www.redhat.com/docs/.
1. Document Conventions
Certaín words ín thís manuaí are represented ín dífferent fonts, styíes, and weíghts.
Thís híghííghtíng índícates that the word ís part of a specífíc category. The
categoríes íncíude the foííowíng:
CouJ1eJ 1oh1
Couríer font represents commahds, 11ìe hames ahd pa1hs, and pJomp1s .
When shown as beíow, ít índícates computer output:
0esk1op abou1.h1mì ìogs pauìwes1eJbeJg.phg
ha1ì backup11ìes ma1ì JepoJ1s
bo1d CourÍer 1on1
Boíd Couríer font represents text that you are to type, such as: servÍce ]onas
s1ar1
If you have to run a command as root, the root prompt (#) precedes the
command:
# gcon11oo1-2
I1a1Ic CouIIeI 1on1
Itaííc Couríer font represents a varíabíe, such as an ínstaííatíon dírectory:
Ins1a11_dII/b1h/
lntroduction
vííí
bold font
Boíd font represents application programs and text found on a graphical
interface.
When shown ííke thís: OK , ít índícates a button on a graphícaí appíícatíon
ínterface.
Addítíonaííy, the manuaí uses dífferent strategíes to draw your attentíon to píeces of
ínformatíon. In order of how crítícaí the ínformatíon ís to you, these ítems are
marked as foííows:
Note
A note ís typícaííy ínformatíon that you need to understand the
behavíor of the system.
Tip
A típ ís typícaííy an aíternatíve way of performíng a task.
lmportant
Important ínformatíon ís necessary, but possíbíy unexpected, such as a
confíguratíon change that wííí not persíst after a reboot.
Caution
A cautíon índícates an act that wouíd víoíate your support agreement,
such as recompíííng the kerneí.
Warning
A warníng índícates potentíaí data íoss, as may happen when tuníng
Document Conventions
íx
hardware for maxímum performance.
2. Feedback
If you spot a typo, or íf you have thought of a way to make thís document better, we
wouíd íove to hear from you. Píease submít a report ín Bugzííía
(http://bugzilla.redhat.com/bugzilla/) agaínst the component Jh-cs.
Be sure to mentíon the document's ídentífíer:
sac_cì_oveJ{Eh)-5.1 {2007-11-08.T14.06)
By mentíoníng thís document's ídentífíer, we know exactíy whích versíon of the
guíde you have.
If you have a suggestíon for ímprovíng the documentatíon, try to be as specífíc as
possíbíe. If you have found an error, píease íncíude the sectíon number and some of
the surroundíng text so we can fínd ít easííy.
lntroduction
x
Red Hat Cluster Suite
Overview
Cíustered systems províde reííabíííty, scaíabíííty, and avaííabíííty to crítícaí
productíon servíces. Usíng Red Hat Cíuster Suíte, you can create a cíuster to suít
your needs for performance, hígh avaííabíííty, íoad baíancíng, scaíabíííty, fííe
sharíng, and economy. Thís chapter provídes an overvíew of Red Hat Cíuster Suíte
components and functíons, and consísts of the foííowíng sectíons:
º 5ection J, ¨Cluster 8asics"
º 5ection 2, ¨Red Hat Cluster 5uite lntroduction"
º 5ection 3, ¨Cluster lnfrastructure"
º 5ection 4, ¨High-availability 5ervice Management"
º 5ection 5, ¨Red Hat CF5"
º 5ection 6, ¨Cluster Logical Volume Manager"
º 5ection 7, ¨Clobal Network 8lock Device"
º 5ection 8, ¨Linux Virtual 5erver"
º 5ection 9, ¨Cluster Administration Tools"
º 5ection J0, ¨Linux Virtual 5erver Administration CUl"
1. Cluster Basics
A cíuster ís two or more computers (caííed nodes or members) that work together to
perform a task. There are four maíor types of cíusters:
º Storage
º Hígh avaííabíííty
º Load baíancíng
º Hígh performance
Chapter 1.
1
Storage cíusters províde a consístent fííe system ímage across servers ín a cíuster,
aííowíng the servers to símuítaneousíy read and wríte to a síngíe shared fííe system.
A storage cíuster símpíífíes storage admínístratíon by íímítíng the ínstaííatíon and
patchíng of appíícatíons to one fííe system. Aíso, wíth a cíuster-wíde fííe system, a
storage cíuster eíímínates the need for redundant copíes of appíícatíon data and
símpíífíes backup and dísaster recovery. Red Hat Cíuster Suíte provídes storage
cíusteríng through Red Hat GFS.
Hígh-avaííabíííty cíusters províde contínuous avaííabíííty of servíces by eíímínatíng
síngíe poínts of faííure and by faíííng over servíces from one cíuster node to another
ín case a node becomes ínoperatíve. Typícaííy, servíces ín a hígh-avaííabíííty cíuster
read and wríte data (vía read-wríte mounted fííe systems). Therefore, a
hígh-avaííabíííty cíuster must maíntaín data íntegríty as one cíuster node takes over
controí of a servíce from another cíuster node. Node faííures ín a hígh-avaííabíííty
cíuster are not vísíbíe from cííents outsíde the cíuster. (Hígh-avaííabíííty cíusters are
sometímes referred to as faííover cíusters.) Red Hat Cíuster Suíte provídes
hígh-avaííabíííty cíusteríng through íts Hígh-avaííabíííty Servíce Management
component.
Load-baíancíng cíusters díspatch network servíce requests to muítípíe cíuster nodes
to baíance the request íoad among the cíuster nodes. Load baíancíng provídes
cost-effectíve scaíabíííty because you can match the number of nodes accordíng to
íoad requírements. If a node ín a íoad-baíancíng cíuster becomes ínoperatíve, the
íoad-baíancíng software detects the faííure and redírects requests to other cíuster
nodes. Node faííures ín a íoad-baíancíng cíuster are not vísíbíe from cííents outsíde
the cíuster. Red Hat Cíuster Suíte provídes íoad-baíancíng through LVS (Línux Vírtuaí
Server).
Hígh-performance cíusters use cíuster nodes to perform concurrent caícuíatíons. A
hígh-performance cíuster aííows appíícatíons to work ín paraííeí, therefore
enhancíng the performance of the appíícatíons. (Hígh performance cíusters are aíso
referred to as computatíonaí cíusters or gríd computíng.)
Note
The cíuster types summarízed ín the precedíng text refíect basíc
confíguratíons; your needs míght requíre a combínatíon of the cíusters
descríbed.
Chapter 1. Red Hat Cluster Suite Overview
2
2. Red Hat Cluster Suite lntroduction
Red Hat Cíuster Suíte (RHCS) ís an íntegrated set of software components that can
be depíoyed ín a varíety of confíguratíons to suít your needs for performance,
hígh-avaííabíííty, íoad baíancíng, scaíabíííty, fííe sharíng, and economy.
RHCS consísts of the foííowíng maíor components (refer to Figure J.J, ¨Red Hat
Cluster 5uite lntroduction"):
º Cíuster ínfrastructure - Provídes fundamentaí functíons for nodes to work
together as a cíuster: confíguratíon-fííe management, membershíp management,
íock management, and fencíng.
º Hígh-avaííabíííty Servíce Management - Provídes faííover of servíces from one
cíuster node to another ín case a node becomes ínoperatíve.
º Cíuster admínístratíon tooís - Confíguratíon and management tooís for settíng
up, confíguríng, and managíng a Red Hat cíuster. The tooís are for use wíth the
Cíuster Infrastructure components, the Hígh-avaííabíííty and Servíce Management
components, and storage.
º Línux Vírtuaí Server (LVS) - Routíng software that provídes IP-Load-baíancíng.
LVS runs ín a paír of redundant servers that dístríbutes cííent requests eveníy to
reaí servers that are behínd the LVS servers.
You can suppíement Red Hat Cíuster Suíte wíth the foííowíng components, whích are
part of an optíonaí package (and not part of Red Hat Cíuster Suíte):
º Red Hat GFS (Gíobaí Fííe System) - Provídes a cíuster fííe system for use wíth Red
Hat Cíuster Suíte. GFS aííows muítípíe nodes to share storage at a bíock íeveí as íf
the storage were connected íocaííy to each cíuster node.
º Cíuster Logícaí Voíume Manager (CLVM) - Provídes voíume management of
cíuster storage.
º Gíobaí Network Bíock Devíce (GNBD) - An ancíííary component of GFS that
exports bíock-íeveí storage to Ethernet. Thís ís an economícaí way to make
bíock-íeveí storage avaííabíe to Red Hat GFS.
For a íower íeveí summary of Red Hat Cíuster Suíte components and optíonaí
software, refer to Chapter 2, Red Hat Cluster 5uite Component 5ummary.
Red Hat Cluster Suite lntroduction
3
Figure 1.1. Red Hat Cluster Suite lntroduction
Note
Figure J.J, ¨Red Hat Cluster 5uite lntroduction" íncíudes GFS, CLVM,
and GNBD, whích are components that are part of an optíonaí package
and not part of Red Hat Cíuster Suíte.
3. Cluster lnfrastructure
The Red Hat Cíuster Suíte cíuster ínfrastructure provídes the basíc functíons for a
group of computers (caííed nodes or members) to work together as a cíuster. Once
a cíuster ís formed usíng the cíuster ínfrastructure, you can use other Red Hat
Cíuster Suíte components to suít your cíusteríng needs (for exampíe, settíng up a
cíuster for sharíng fííes on a GFS fííe system or settíng up servíce faííover). The
cíuster ínfrastructure performs the foííowíng functíons:
Chapter 1. Red Hat Cluster Suite Overview
4
º Cíuster management
º Lock management
º Fencíng
º Cíuster confíguratíon management
3.1. Cluster Management
Cíuster management manages cíuster quorum and cíuster membershíp. CMAN (an
abbrevíatíon for cíuster manager) performs cíuster management ín Red Hat Cíuster
Suíte for Red Hat Enterpríse Línux 5. CMAN ís a dístríbuted cíuster manager and
runs ín each cíuster node; cíuster management ís dístríbuted across aíí nodes ín the
cíuster (refer to Figure J.2, ¨CMAN/DLM Overview").
CMAN keeps track of cíuster quorum by monítoríng the count of cíuster nodes. If
more than haíf the nodes are actíve, the cíuster has quorum. If haíf the nodes (or
fewer) are actíve, the cíuster does not have quorum, and aíí cíuster actívíty ís
stopped. Cíuster quorum prevents the occurrence of a "spíít-braín" condítíon - a
condítíon where two ínstances of the same cíuster are runníng. A spíít-braín
condítíon wouíd aííow each cíuster ínstance to access cíuster resources wíthout
knowíedge of the other cíuster ínstance, resuítíng ín corrupted cíuster íntegríty.
Ouorum ís determíned by communícatíon of messages among cíuster nodes vía
Ethernet. Optíonaííy, quorum can be determíned by a combínatíon of
communícatíng messages vía Ethernet and through a quorum dísk. For quorum vía
Ethernet, quorum consísts of 50 percent of the node votes píus 1. For quorum vía
quorum dísk, quorum consísts of user-specífíed condítíons.
Note
By defauít, each node has one quorum vote. Optíonaííy, you can
confígure each node to have more than one vote.
CMAN keeps track of membershíp by monítoríng messages from other cíuster
nodes. When cíuster membershíp changes, the cíuster manager notífíes the other
ínfrastructure components, whích then take appropríate actíon. For exampíe, íf node
A íoíns a cíuster and mounts a GFS fííe system that nodes B and C have aíready
mounted, then an addítíonaí íournaí and íock management ís requíred for node A to
use that GFS fííe system. If a cíuster node does not transmít a message wíthín a
Cluster Management
5
prescríbed amount of tíme, the cíuster manager removes the node from the cíuster
and communícates to other cíuster ínfrastructure components that the node ís not a
member. Agaín, other cíuster ínfrastructure components determíne what actíons to
take upon notífícatíon that node ís no íonger a cíuster member. For exampíe,
Fencíng wouíd fence the node that ís no íonger a member.
Figure 1.2. CMANlDLM Overview
3.2. Lock Management
Lock management ís a common cíuster-ínfrastructure servíce that provídes a
mechanísm for other cíuster ínfrastructure components to synchroníze theír access
to shared resources. In a Red Hat cíuster, DLM (Dístríbuted Lock Manager) ís the
íock manager. As ímpííed ín íts name, DLM ís a dístríbuted íock manager and runs ín
each cíuster node; íock management ís dístríbuted across aíí nodes ín the cíuster
(refer to Figure J.2, ¨CMAN/DLM Overview"). GFS and CLVM use íocks from the íock
manager. GFS uses íocks from the íock manager to synchroníze access to fííe
system metadata (on shared storage). CLVM uses íocks from the íock manager to
synchroníze updates to LVM voíumes and voíume groups (aíso on shared storage).
3.3. Fencing
Fencíng ís the dísconnectíon of a node from the cíuster's shared storage. Fencíng
cuts off I/O from shared storage, thus ensuríng data íntegríty. The cíuster
ínfrastructure performs fencíng through the fence daemon, 1ehced.
Chapter 1. Red Hat Cluster Suite Overview
6
When CMAN determínes that a node has faííed, ít communícates to other
cíuster-ínfrastructure components that the node has faííed. 1ehced, when notífíed of
the faííure, fences the faííed node. Other cíuster-ínfrastructure components
determíne what actíons to take - that ís, they perform any recovery that needs to
done. For exampíe, DLM and GFS, when notífíed of a node faííure, suspend actívíty
untíí they detect that 1ehced has compíeted fencíng the faííed node. Upon
confírmatíon that the faííed node ís fenced, DLM and GFS perform recovery. DLM
reíeases íocks of the faííed node; GFS recovers the íournaí of the faííed node.
The fencíng program determínes from the cíuster confíguratíon fííe whích fencíng
method to use. Two key eíements ín the cíuster confíguratíon fííe defíne a fencíng
method: fencíng agent and fencíng devíce. The fencíng program makes a caíí to a
fencíng agent specífíed ín the cíuster confíguratíon fííe. The fencíng agent, ín turn,
fences the node vía a fencíng devíce. When fencíng ís compíete, the fencíng
program notífíes the cíuster manager.
Red Hat Cíuster Suíte provídes a varíety of fencíng methods:
º Power fencíng - A fencíng method that uses a power controííer to power off an
ínoperabíe node.
º Fíbre Channeí swítch fencíng - A fencíng method that dísabíes the Fíbre Channeí
port that connects storage to an ínoperabíe node.
º GNBD fencíng - A fencíng method that dísabíes an ínoperabíe node's access to a
GNBD server.
º Other fencíng - Severaí other fencíng methods that dísabíe I/O or power of an
ínoperabíe node, íncíudíng IBM Bíadecenters, PAP, DRAC/MC, HP ILO, IPMI, IBM
RSA II, and others.
Figure J.3, ¨Power Fencing Example" shows an exampíe of power fencíng. In the
exampíe, the fencíng program ín node A causes the power controííer to power off
node D. Figure J.4, ¨Fibre Channel 5witch Fencing Example" shows an exampíe of
Fíbre Channeí swítch fencíng. In the exampíe, the fencíng program ín node A causes
the Fíbre Channeí swítch to dísabíe the port for node D, dísconnectíng node D from
storage.
Fencing
7
Figure 1.3. Power Fencing Example
Chapter 1. Red Hat Cluster Suite Overview
8
Figure 1.4. Fibre Channel Switch Fencing Example
Specífyíng a fencíng method consísts of edítíng a cíuster confíguratíon fííe to assígn
a fencíng-method name, the fencíng agent, and the fencíng devíce for each node ín
the cíuster.
The way ín whích a fencíng method ís specífíed depends on íf a node has eíther duaí
power suppííes or muítípíe paths to storage. If a node has duaí power suppííes, then
the fencíng method for the node must specífy at íeast two fencíng devíces - one
fencíng devíce for each power suppíy (refer to Figure J.5, ¨Fencing a Node with
Dual Power 5upplies"). Símííaríy, íf a node has muítípíe paths to Fíbre Channeí
storage, then the fencíng method for the node must specífy one fencíng devíce for
each path to Fíbre Channeí storage. For exampíe, íf a node has two paths to Fíbre
Channeí storage, the fencíng method shouíd specífy two fencíng devíces - one for
each path to Fíbre Channeí storage (refer to Figure J.6, ¨Fencing a Node with Dual
Fibre Channel Connections").
Fencing
9
Figure 1.5. Fencing a Node with Dual Power Supplies
Chapter 1. Red Hat Cluster Suite Overview
10
Figure 1.6. Fencing a Node with Dual Fibre Channel
Connections
You can confígure a node wíth one fencíng method or muítípíe fencíng methods.
When you confígure a node for one fencíng method, that ís the oníy fencíng method
avaííabíe for fencíng that node. When you confígure a node for muítípíe fencíng
methods, the fencíng methods are cascaded from one fencíng method to another
accordíng to the order of the fencíng methods specífíed ín the cíuster confíguratíon
fííe. If a node faíís, ít ís fenced usíng the fírst fencíng method specífíed ín the cíuster
confíguratíon fííe for that node. If the fírst fencíng method ís not successfuí, the next
fencíng method specífíed for that node ís used. If none of the fencíng methods ís
successfuí, then fencíng starts agaín wíth the fírst fencíng method specífíed, and
Fencing
11
contínues íoopíng through the fencíng methods ín the order specífíed ín the cíuster
confíguratíon fííe untíí the node has been fenced.
3.4. Cluster Configuration System
The Cíuster Confíguratíon System (CCS) manages the cíuster confíguratíon and
provídes confíguratíon ínformatíon to other cíuster components ín a Red Hat cíuster.
CCS runs ín each cíuster node and makes sure that the cíuster confíguratíon fííe ín
each cíuster node ís up to date. For exampíe, íf a cíuster system admínístrator
updates the confíguratíon fííe ín Node A, CCS propagates the update from Node A to
the other nodes ín the cíuster (refer to Figure J.7, ¨CC5 Overview").
Figure 1.7. CCS Overview
Other cíuster components (for exampíe, CMAN) access confíguratíon ínformatíon
from the confíguratíon fííe through CCS (refer to Figure J.7, ¨CC5 Overview").
Chapter 1. Red Hat Cluster Suite Overview
12
Figure 1.8. Accessing Configuration lnformation
The cíuster confíguratíon fííe (/e1c/cìus1eJ/cìus1eJ.coh1) ís an XML fííe that
descríbes the foííowíng cíuster characterístícs:
º Cíuster name - Díspíays the cíuster name, cíuster confíguratíon fííe revísíon íeveí,
and basíc fence tímíng propertíes used when a node íoíns a cíuster or ís fenced
from the cíuster.
º Cíuster - Díspíays each node of the cíuster, specífyíng node name, node ID,
number of quorum votes, and fencíng method for that node.
º Fence Devíce - Díspíays fence devíces ín the cíuster. Parameters vary accordíng
to the type of fence devíce. For exampíe for a power controííer used as a fence
devíce, the cíuster confíguratíon defínes the name of the power controííer, íts IP
address, íogín, and password.
º Managed Resources - Díspíays resources requíred to create cíuster servíces.
Managed resources íncíudes the defínítíon of faííover domaíns, resources (for
Cluster Configuration System
13
exampíe an IP address), and servíces. Together the managed resources defíne
cíuster servíces and faííover behavíor of the cíuster servíces.
4. High-availability Service Management
Hígh-avaííabíííty servíce management provídes the abíííty to create and manage
hígh-avaííabíííty cluster services ín a Red Hat cíuster. The key component for
hígh-avaííabíííty servíce management ín a Red Hat cíuster, JgmahageJ, ímpíements
coíd faííover for off-the-sheíf appíícatíons. In a Red Hat cíuster, an appíícatíon ís
confígured wíth other cíuster resources to form a hígh-avaííabíííty cíuster servíce. A
hígh-avaííabíííty cíuster servíce can faíí over from one cíuster node to another wíth
no apparent ínterruptíon to cíuster cííents. Cíuster-servíce faííover can occur íf a
cíuster node faíís or íf a cíuster system admínístrator moves the servíce from one
cíuster node to another (for exampíe, for a píanned outage of a cíuster node).
To create a hígh-avaííabíííty servíce, you must confígure ít ín the cíuster
confíguratíon fííe. A cíuster servíce compríses cíuster resources. Cíuster resources
are buíídíng bíocks that you create and manage ín the cíuster confíguratíon fííe -
for exampíe, an IP address, an appíícatíon ínítíaíízatíon scrípt, or a Red Hat GFS
shared partítíon.
You can assocíate a cíuster servíce wíth a failover domain. A faííover domaín ís a
subset of cíuster nodes that are eíígíbíe to run a partícuíar cíuster servíce (refer to
Figure J.9, ¨Failover Domains").
Note
Faííover domaíns are not requíred for operatíon.
A cíuster servíce can run on oníy one cíuster node at a tíme to maíntaín data
íntegríty. You can specífy faííover príoríty ín a faííover domaín. Specífyíng faííover
príoríty consísts of assígníng a príoríty íeveí to each node ín a faííover domaín. The
príoríty íeveí determínes the faííover order - determíníng whích node that a cíuster
servíce shouíd faíí over to. If you do not specífy faííover príoríty, a cíuster servíce
can faíí over to any node ín íts faííover domaín. Aíso, you can specífy íf a cíuster
servíce ís restrícted to run oníy on nodes of íts assocíated faííover domaín. (When
assocíated wíth an unrestrícted faííover domaín, a cíuster servíce can start on any
cíuster node ín the event no member of the faííover domaín ís avaííabíe.)
In Figure J.9, ¨Failover Domains", Faííover Domaín 1 ís confígured to restríct
Chapter 1. Red Hat Cluster Suite Overview
14
faííover wíthín that domaín; therefore, Cíuster Servíce X can oníy faíí over between
Node A and Node B. Faííover Domaín 2 ís aíso confígured to restríct faííover wíth íts
domaín; addítíonaííy, ít ís confígured for faííover príoríty. Faííover Domaín 2 príoríty
ís confígured wíth Node C as príoríty 1, Node B as príoríty 2, and Node D as príoríty
3. If Node C faíís, Cíuster Servíce Y faíís over to Node B next. If ít cannot faíí over to
Node B, ít tríes faíííng over to Node D. Faííover Domaín 3 ís confígured wíth no
príoríty and no restríctíons. If the node that Cíuster Servíce Z ís runníng on faíís,
Cíuster Servíce Z tríes faíííng over to one of the nodes ín Faííover Domaín 3.
However, íf none of those nodes ís avaííabíe, Cíuster Servíce Z can faíí over to any
node ín the cíuster.
Figure 1.9. Failover Domains
Figure J.J0, ¨Web 5erver Cluster 5ervice Example" shows an exampíe of a
High-availability Service
15
hígh-avaííabíííty cíuster servíce that ís a web server named "content-webserver". It
ís runníng ín cíuster node B and ís ín a faííover domaín that consísts of nodes A, B,
and D. In addítíon, the faííover domaín ís confígured wíth a faííover príoríty to faíí
over to node D before node A and to restríct faííover to nodes oníy ín that faííover
domaín. The cíuster servíce compríses these cíuster resources:
º IP address resource - IP address 10.10.10.201.
º An appíícatíon resource named "httpd-content" - a web server appíícatíon ínít
scrípt /e1c/1h11.d/h11pd (specífyíng h11pd).
º A fííe system resource - Red Hat GFS named "gfs-content-webserver".
Figure 1.10. Web Server Cluster Service Example
Chapter 1. Red Hat Cluster Suite Overview
16
Cííents access the cíuster servíce through the IP address 10.10.10.201, enabííng
ínteractíon wíth the web server appíícatíon, httpd-content. The httpd-content
appíícatíon uses the gfs-content-webserver fííe system. If node B were to faíí, the
content-webserver cíuster servíce wouíd faíí over to node D. If node D were not
avaííabíe or aíso faííed, the servíce wouíd faíí over to node A. Faííover wouíd occur
wíth no apparent ínterruptíon to the cíuster cííents. The cíuster servíce wouíd be
accessíbíe from another cíuster node vía the same IP address as ít was before
faííover.
5. Red Hat GFS
Red Hat GFS ís a cíuster fííe system that aííows a cíuster of nodes to símuítaneousíy
access a bíock devíce that ís shared among the nodes. GFS ís a natíve fííe system
that ínterfaces dírectíy wíth the VFS íayer of the Línux kerneí fííe-system ínterface.
GFS empíoys dístríbuted metadata and muítípíe íournaís for optímaí operatíon ín a
cíuster. To maíntaín fííe system íntegríty, GFS uses a íock manager to coordínate
I/O. When one node changes data on a GFS fííe system, that change ís ímmedíateíy
vísíbíe to the other cíuster nodes usíng that fííe system.
Usíng Red Hat GFS, you can achíeve maxímum appíícatíon uptíme through the
foííowíng benefíts:
º Símpíífyíng your data ínfrastructure
º Instaíí and patch appíícatíons once for the entíre cíuster.
º Eíímínates the need for redundant copíes of appíícatíon data (dupíícatíon).
º Enabíes concurrent read/wríte access to data by many cííents.
º Símpíífíes backup and dísaster recovery (oníy one fííe system to back up or
recover).
º Maxímíze the use of storage resources; mínímíze storage admínístratíon costs.
º Manage storage as a whoíe ínstead of by partítíon.
º Decrease overaíí storage needs by eíímínatíng the need for data repíícatíons.
º Scaíe the cíuster seamíessíy by addíng servers or storage on the fíy.
º No more partítíoníng storage through compíícated techníques.
Management
17
º Add servers to the cíuster on the fíy by mountíng them to the common fííe
system.
Nodes that run Red Hat GFS are confígured and managed wíth Red Hat Cíuster Suíte
confíguratíon and management tooís. Voíume management ís managed through
CLVM (Cíuster Logícaí Voíume Manager). Red Hat GFS provídes data sharíng among
GFS nodes ín a Red Hat cíuster. GFS provídes a síngíe, consístent víew of the
fííe-system name space across the GFS nodes ín a Red Hat cíuster. GFS aííows
appíícatíons to ínstaíí and run wíthout much knowíedge of the underíyíng storage
ínfrastructure. Aíso, GFS provídes features that are typícaííy requíred ín enterpríse
envíronments, such as quotas, muítípíe íournaís, and muítípath support.
GFS provídes a versatííe method of networkíng storage accordíng to the
performance, scaíabíííty, and economíc needs of your storage envíronment. Thís
chapter provídes some very basíc, abbrevíated ínformatíon as background to heíp
you understand GFS.
You can depíoy GFS ín a varíety of confíguratíons to suít your needs for
performance, scaíabíííty, and economy. For superíor performance and scaíabíííty,
you can depíoy GFS ín a cíuster that ís connected dírectíy to a SAN. For more
economícaí needs, you can depíoy GFS ín a cíuster that ís connected to a LAN wíth
servers that use CN8D (Gíobaí Network Bíock Devíce) or to i5C5l (Internet Smaíí
Computer System Interface) devíces. (For more ínformatíon about GNBD, refer to
5ection 7, ¨Clobal Network 8lock Device".)
The foííowíng sectíons províde exampíes of how GFS can be depíoyed to suít your
needs for performance, scaíabíííty, and economy:
º 5ection 5.J, ¨5uperior Performance and 5calability"
º 5ection 5.2, ¨Performance, 5calability, Moderate Price"
º 5ection 5.3, ¨Economy and Performance"
Note
The GFS depíoyment exampíes refíect basíc confíguratíons; your needs
míght requíre a combínatíon of confíguratíons shown ín the exampíes.
Chapter 1. Red Hat Cluster Suite Overview
18
5.1. Superior Performance and Scalability
You can obtaín the híghest shared-fííe performance when appíícatíons access
storage dírectíy. The GFS SAN confíguratíon ín Figure J.JJ, ¨CF5 with a 5AN"
provídes superíor fííe performance for shared fííes and fííe systems. Línux
appíícatíons run dírectíy on cíuster nodes usíng GFS. Wíthout fííe protocoís or
storage servers to síow data access, performance ís símííar to índívíduaí Línux
servers wíth dírectíy connected storage; yet, each GFS appíícatíon node has equaí
access to aíí data fííes. GFS supports over 300 GFS nodes.
Figure 1.11. GFS with a SAN
5.2. Performance, Scalability, Moderate Price
Muítípíe Línux cííent appíícatíons on a LAN can share the same SAN-based data as
shown ín Figure J.J2, ¨CF5 and CN8D with a 5AN". SAN bíock storage ís presented
to network cííents as bíock storage devíces by GNBD servers. From the perspectíve
of a cííent appíícatíon, storage ís accessed as íf ít were dírectíy attached to the
server ín whích the appíícatíon ís runníng. Stored data ís actuaííy on the SAN.
Storage devíces and data can be equaííy shared by network cííent appíícatíons. Fííe
íockíng and sharíng functíons are handíed by GFS for each network cííent.
Performance, Scalability, Moderate
19
Figure 1.12. GFS and GNBD with a SAN
5.3. Economy and Performance
Figure J.J3, ¨CF5 and CN8D with Directly Connected 5torage" shows how Línux
cííent appíícatíons can take advantage of an exístíng Ethernet topoíogy to gaín
shared access to aíí bíock storage devíces. Cííent data fííes and fííe systems can be
shared wíth GFS on each cííent. Appíícatíon faííover can be fuííy automated wíth Red
Hat Cíuster Suíte.
Chapter 1. Red Hat Cluster Suite Overview
20
Figure 1.13. GFS and GNBD with Directly Connected Storage
6. Cluster Logical Volume Manager
The Cíuster Logícaí Voíume Manager (CLVM) provídes a cíuster-wíde versíon of
LVM2. CLVM provídes the same capabííítíes as LVM2 on a síngíe node, but makes
the voíumes avaííabíe to aíí nodes ín a Red Hat cíuster. The íogícaí voíumes created
wíth CLVM make íogícaí voíumes avaííabíe to aíí nodes ín a cíuster.
The key component ín CLVM ís cìvmd. cìvmd ís a daemon that provídes cíusteríng
extensíons to the standard LVM2 tooí set and aííows LVM2 commands to manage
shared storage. cìvmd runs ín each cíuster node and dístríbutes LVM metadata
updates ín a cíuster, thereby presentíng each cíuster node wíth the same víew of
the íogícaí voíumes (refer to Figure J.J4, ¨CLVM Overview"). Logícaí voíumes
created wíth CLVM on shared storage are vísíbíe to aíí nodes that have access to the
shared storage. CLVM aííows a user to confígure íogícaí voíumes on shared storage
by íockíng access to physícaí storage whííe a íogícaí voíume ís beíng confígured.
CLVM uses the íock-management servíce províded by the cíuster ínfrastructure
(refer to 5ection 3, ¨Cluster lnfrastructure").
Price
21
Note
Usíng CLVM requíres mínor changes to /e1c/ìvm/ìvm.coh1 for
cíuster-wíde íockíng.
Figure 1.14. CLVM Overview
You can confígure CLVM usíng the same commands as LVM2, usíng the LVM
graphícaí user ínterface (refer to Figure J.J5, ¨LVM Craphical User lnterface"), or
usíng the storage confíguratíon functíon of the Conga cíuster confíguratíon
graphícaí user ínterface (refer to Figure J.J6, ¨Conga LVM Craphical User
lnterface") . Figure J.J7, ¨Creating Logical Volumes" shows the basíc concept of
creatíng íogícaí voíumes from Línux partítíons and shows the commands used to
create íogícaí voíumes.
Chapter 1. Red Hat Cluster Suite Overview
22
Figure 1.15. LVM Graphical User lnterface
Cluster Logical Volume Manager
23
Figure 1.16. Conga LVM Graphical User lnterface
Chapter 1. Red Hat Cluster Suite Overview
24
Figure 1.17. Creating Logical Volumes
7. Global Network Block Device
Gíobaí Network Bíock Devíce (GNBD) provídes bíock-devíce access to Red Hat GFS
over TCP/IP. GNBD ís símííar ín concept to NBD; however, GNBD ís GFS-specífíc and
tuned soíeíy for use wíth GFS. GNBD ís usefuí when the need for more robust
technoíogíes - Fíbre Channeí or síngíe-ínítíator SCSI - are not necessary or are
cost-prohíbítíve.
GNBD consísts of two maíor components: a GNBD cííent and a GNBD server. A
GNBD cííent runs ín a node wíth GFS and ímports a bíock devíce exported by a
GNBD server. A GNBD server runs ín another node and exports bíock-íeveí storage
from íts íocaí storage (eíther dírectíy attached storage or SAN storage). Refer to
Figure J.J8, ¨CN8D Overview". Muítípíe GNBD cííents can access a devíce exported
Global Network Block Device
25
by a GNBD server, thus makíng a GNBD suítabíe for use by a group of nodes
runníng GFS.
Figure 1.18. GNBD Overview
8. Linux Virtual Server
Línux Vírtuaí Server (LVS) ís a set of íntegrated software components for baíancíng
the IP íoad across a set of reaí servers. LVS runs on a paír of equaííy confígured
computers: one that ís an actíve LVS router and one that ís a backup LVS router.
The actíve LVS router serves two roíes:
º To baíance the íoad across the reaí servers.
º To check the íntegríty of the servíces on each reaí server.
Chapter 1. Red Hat Cluster Suite Overview
26
The backup LVS router monítors the actíve LVS router and takes over from ít ín case
the actíve LVS router faíís.
Figure J.J9, ¨Components of a Running LV5 Cluster" provídes an overvíew of the
LVS components and theír ínterreíatíonshíp.
Figure 1.19. Components of a Running LVS Cluster
The puìse daemon runs on both the actíve and passíve LVS routers. On the backup
LVS router, puìse sends a heartbeat to the pubííc ínterface of the actíve router to
make sure the actíve LVS router ís properíy functíoníng. On the actíve LVS router,
puìse starts the ìvs daemon and responds to heartbeat queríes from the backup
LVS router.
Once started, the ìvs daemon caíís the 1pvsadm utíííty to confígure and maíntaín the
IPVS (IP Vírtuaí Server) routíng tabíe ín the kerneí and starts a hahhy process for
each confígured vírtuaí server on each reaí server. Each hahhy process checks the
state of one confígured servíce on one reaí server, and teíís the ìvs daemon íf the
Linux Virtual Server
27
servíce on that reaí server ís maífunctíoníng. If a maífunctíon ís detected, the ìvs
daemon ínstructs 1pvsadm to remove that reaí server from the IPVS routíng tabíe.
If the backup LVS router does not receíve a response from the actíve LVS router, ít
ínítíates faííover by caíííng sehd_aJp to reassígn aíí vírtuaí IP addresses to the NIC
hardware addresses (MAC address) of the backup LVS router, sends a command to
the actíve LVS router vía both the pubííc and prívate network ínterfaces to shut
down the ìvs daemon on the actíve LVS router, and starts the ìvs daemon on the
backup LVS router to accept requests for the confígured vírtuaí servers.
To an outsíde user accessíng a hosted servíce (such as a websíte or database
appíícatíon), LVS appears as one server. However, the user ís actuaííy accessíng reaí
servers behínd the LVS routers.
Because there ís no buíít-ín component ín LVS to share the data among reaí servers,
you have have two basíc optíons:
º Synchroníze the data across the reaí servers.
º Add a thírd íayer to the topoíogy for shared data access.
The fírst optíon ís preferred for servers that do not aííow íarge numbers of users to
upíoad or change data on the reaí servers. If the reaí servers aííow íarge numbers of
users to modífy data, such as an e-commerce websíte, addíng a thírd íayer ís
preferabíe.
There are many ways to synchroníze data among reaí servers. For exampíe, you can
use sheíí scrípts to post updated web pages to the reaí servers símuítaneousíy. Aíso,
you can use programs such as Jsyhc to repíícate changed data across aíí nodes at a
set íntervaí. However, ín envíronments where users frequentíy upíoad fííes or íssue
database transactíons, usíng scrípts or the Jsyhc command for data synchronízatíon
does not functíon optímaííy. Therefore, for reaí servers wíth a hígh amount of
upíoads, database transactíons, or símííar traffíc, a three-tiered topology ís more
appropríate for data synchronízatíon.
8.1. Two-Tier LVS Topology
Figure J.20, ¨Two-Tier LV5 Topology" shows a símpíe LVS confíguratíon consístíng
of two tíers: LVS routers and reaí servers. The LVS-router tíer consísts of one actíve
LVS router and one backup LVS router. The reaí-server tíer consísts of reaí servers
connected to the prívate network. Each LVS router has two network ínterfaces: one
connected to a pubííc network (Internet) and one connected to a prívate network. A
Chapter 1. Red Hat Cluster Suite Overview
28
1
A vírtuaí server ís a servíce confígured to íísten on a specífíc vírtuaí IP.
network ínterface connected to each network aííows the LVS routers to reguíate
traffíc between cííents on the pubííc network and the reaí servers on the prívate
network. In Figure J.20, ¨Two-Tier LV5 Topology", the actíve LVS router uses
Network Address Translation (NAT) to dírect traffíc from the pubííc network to reaí
servers on the prívate network, whích ín turn províde servíces as requested. The
reaí servers pass aíí pubííc traffíc through the actíve LVS router. From the
perspectíve of cííents on the pubííc network, the LVS router appears as one entíty.
Figure 1.20. Two-Tier LVS Topology
Servíce requests arrívíng at an LVS router are addressed to a virtual lP address or
VIP. Thís ís a pubíícíy-routabíe address that the admínístrator of the síte assocíates
wíth a fuííy-quaíífíed domaín name, such as www.exampíe.com, and whích ís
assígned to one or more virtual servers
1
. Note that a VIP address mígrates from one
LVS router to the other duríng a faííover, thus maíntaíníng a presence at that IP
Two-Tier LVS Topology
29
address, aíso known as floating lP addresses.
VIP addresses may be aííased to the same devíce that connects the LVS router to
the pubííc network. For ínstance, íf eth0 ís connected to the Internet, then muítípíe
vírtuaí servers can be aííased to e1h0.1. Aíternatíveíy, each vírtuaí server can be
assocíated wíth a separate devíce per servíce. For exampíe, HTTP traffíc can be
handíed on e1h0.1, and FTP traffíc can be handíed on e1h0.2.
Oníy one LVS router ís actíve at a tíme. The roíe of the actíve LVS router ís to
redírect servíce requests from vírtuaí IP addresses to the reaí servers. The
redírectíon ís based on one of eíght íoad-baíancíng aígoríthms:
º Round-Robín Scheduííng - Dístríbutes each request sequentíaííy around a pooí of
reaí servers. Usíng thís aígoríthm, aíí the reaí servers are treated as equaís
wíthout regard to capacíty or íoad.
º Weíghted Round-Robín Scheduííng - Dístríbutes each request sequentíaííy
around a pooí of reaí servers but gíves more íobs to servers wíth greater capacíty.
Capacíty ís índícated by a user-assígned weíght factor, whích ís then adíusted up
or down by dynamíc íoad ínformatíon. Thís ís a preferred choíce íf there are
sígnífícant dífferences ín the capacíty of reaí servers ín a server pooí. However, íf
the request íoad varíes dramatícaííy, a more heavííy weíghted server may answer
more than íts share of requests.
º Least-Connectíon - Dístríbutes more requests to reaí servers wíth fewer actíve
connectíons. Thís ís a type of dynamíc scheduííng aígoríthm, makíng ít a better
choíce íf there ís a hígh degree of varíatíon ín the request íoad. It ís best suíted for
a reaí server pooí where each server node has roughíy the same capacíty. If the
reaí servers have varyíng capabííítíes, weíghted íeast-connectíon scheduííng ís a
better choíce.
º Weíghted Least-Connectíons (defauít) - Dístríbutes more requests to servers wíth
fewer actíve connectíons reíatíve to theír capacítíes. Capacíty ís índícated by a
user-assígned weíght, whích ís then adíusted up or down by dynamíc íoad
ínformatíon. The addítíon of weíghtíng makes thís aígoríthm ídeaí when the reaí
server pooí contaíns hardware of varyíng capacíty.
º Locaííty-Based Least-Connectíon Scheduííng - Dístríbutes more requests to
servers wíth fewer actíve connectíons reíatíve to theír destínatíon IPs. Thís
aígoríthm ís for use ín a proxy-cache server cíuster. It routes the packets for an IP
address to the server for that address uníess that server ís above íts capacíty and
has a server ín íts haíf íoad, ín whích case ít assígns the IP address to the íeast
Chapter 1. Red Hat Cluster Suite Overview
30
íoaded reaí server.
º Locaííty-Based Least-Connectíon Scheduííng wíth Repíícatíon Scheduííng -
Dístríbutes more requests to servers wíth fewer actíve connectíons reíatíve to
theír destínatíon IPs. Thís aígoríthm ís aíso for use ín a proxy-cache server cíuster.
It díffers from Locaííty-Based Least-Connectíon Scheduííng by mappíng the target
IP address to a subset of reaí server nodes. Requests are then routed to the
server ín thís subset wíth the íowest number of connectíons. If aíí the nodes for
the destínatíon IP are above capacíty, ít repíícates a new server for that
destínatíon IP address by addíng the reaí server wíth the íeast connectíons from
the overaíí pooí of reaí servers to the subset of reaí servers for that destínatíon IP.
The most-íoaded node ís then dropped from the reaí server subset to prevent
over-repíícatíon.
º Source Hash Scheduííng - Dístríbutes requests to the pooí of reaí servers by
íookíng up the source IP ín a statíc hash tabíe. Thís aígoríthm ís for LVS routers
wíth muítípíe fírewaíís.
Aíso, the actíve LVS router dynamícaííy monítors the overaíí heaíth of the specífíc
servíces on the reaí servers through símpíe send/expect scripts. To aíd ín detectíng
the heaíth of servíces that requíre dynamíc data, such as HTTPS or SSL, you can
aíso caíí externaí executabíes. If a servíce on a reaí server maífunctíons, the actíve
LVS router stops sendíng íobs to that server untíí ít returns to normaí operatíon.
The backup LVS router performs the roíe of a standby system. Períodícaííy, the LVS
routers exchange heartbeat messages through the prímary externaí pubííc ínterface
and, ín a faííover sítuatíon, the prívate ínterface. Shouíd the backup LVS router faíí
to receíve a heartbeat message wíthín an expected íntervaí, ít ínítíates a faííover
and assumes the roíe of the actíve LVS router. Duríng faííover, the backup LVS
router takes over the VIP addresses servíced by the faííed router usíng a techníque
known as ARP spoofing - where the backup LVS router announces ítseíf as the
destínatíon for IP packets addressed to the faííed node. When the faííed node
returns to actíve servíce, the backup LVS router assumes íts backup roíe agaín.
The símpíe, two-tíer confíguratíon ín Figure J.20, ¨Two-Tier LV5 Topology" ís suíted
best for cíusters servíng data that does not change very frequentíy - such as statíc
web pages - because the índívíduaí reaí servers do not automatícaííy synchroníze
data among themseíves.
8.2. Three-Tier LVS Topology
Figure J.2J, ¨Three-Tier LV5 Topology" shows a typícaí three-tíer LVS confíguratíon.
Three-Tier LVS Topology
31
In the exampíe, the actíve LVS router routes the requests from the pubííc network
(Internet) to the second tíer - reaí servers. Each reaí server then accesses a shared
data source of a Red Hat cíuster ín the thírd tíer over the prívate network.
Figure 1.21. Three-Tier LVS Topology
Chapter 1. Red Hat Cluster Suite Overview
32
Thís topoíogy ís suíted weíí for busy FTP servers, where accessíbíe data ís stored on
a centraí, híghíy avaííabíe server and accessed by each reaí server vía an exported
NFS dírectory or Samba share. Thís topoíogy ís aíso recommended for websítes that
access a centraí, hígh-avaííabíííty database for transactíons. Addítíonaííy, usíng an
actíve-actíve confíguratíon wíth a Red Hat cíuster, you can confígure one
hígh-avaííabíííty cíuster to serve both of these roíes símuítaneousíy.
8.3. Routing Methods
You can use Network Address Transíatíon (NAT) routíng or dírect routíng wíth LVS.
The foííowíng sectíons bríefíy descríbe NAT routíng and dírect routíng wíth LVS.
8.3.1. NAT Routing
Figure J.22, ¨LV5 lmplemented with NAT Routing", íííustrates LVS usíng NAT
routíng to move requests between the Internet and a prívate network.
Routing Methods
33
Figure 1.22. LVS lmplemented with NAT Routing
In the exampíe, there are two NICs ín the actíve LVS router. The NIC for the Internet
has a real lP address on eth0 and has a fíoatíng IP address aííased to eth0:1. The
NIC for the prívate network ínterface has a reaí IP address on eth1 and has a
fíoatíng IP address aííased to eth1:1. In the event of faííover, the vírtuaí ínterface
facíng the Internet and the prívate facíng vírtuaí ínterface are taken over by the
backup LVS router símuítaneousíy. Aíí the reaí servers on the prívate network use
the fíoatíng IP for the NAT router as theír defauít route to communícate wíth the
actíve LVS router so that theír abííítíes to respond to requests from the Internet ís
not ímpaíred.
In the exampíe, the LVS router's pubííc LVS fíoatíng IP address and prívate NAT
fíoatíng IP address are aííased to two physícaí NICs. Whííe ít ís possíbíe to assocíate
each fíoatíng IP address to íts physícaí devíce on the LVS router nodes, havíng more
than two NICs ís not a requírement.
Usíng thís topoíogy, the actíve LVS router receíves the request and routes ít to the
appropríate server. The reaí server then processes the request and returns the
packets to the LVS router. The LVS router uses network address transíatíon to
repíace the address of the reaí server ín the packets wíth the LVS routers pubííc VIP
address. Thís process ís caííed lP masquerading because the actuaí IP addresses of
the reaí servers ís hídden from the requestíng cííents.
Usíng NAT routíng, the reaí servers can be any kínd of computers runníng a varíety
operatíng systems. The maín dísadvantage of NAT routíng ís that the LVS router
may become a bottíeneck ín íarge depíoyments because ít must process outgoíng
and íncomíng requests.
8.3.2. Direct Routing
Dírect routíng provídes íncreased performance benefíts compared to NAT routíng.
Dírect routíng aííows the reaí servers to process and route packets dírectíy to a
requestíng user rather than passíng outgoíng packets through the LVS router. Dírect
routíng reduces the possíbíííty of network performance íssues by reíegatíng the íob
of the LVS router to processíng íncomíng packets oníy.
Chapter 1. Red Hat Cluster Suite Overview
34
Figure 1.23. LVS lmplemented with Direct Routing
In a typícaí dírect-routíng LVS confíguratíon, an LVS router receíves íncomíng server
requests through a vírtuaí IP (VIP) and uses a scheduííng aígoríthm to route the
request to reaí servers. Each reaí server processes requests and sends responses
dírectíy to cííents, bypassíng the LVS routers. Dírect routíng aííows for scaíabíííty ín
that reaí servers can be added wíthout the added burden on the LVS router to route
outgoíng packets from the reaí server to the cííent, whích can become a bottíeneck
under heavy network íoad.
Routing Methods
35
Whííe there are many advantages to usíng dírect routíng ín LVS, there are
íímítatíons. The most common íssue wíth dírect routíng and LVS ís wíth Address
Resolution Protocol (ARP).
In typícaí sítuatíons, a cííent on the Internet sends a request to an IP address.
Network routers typícaííy send requests to theír destínatíon by reíatíng IP addresses
to a machíne's MAC address wíth ARP. ARP requests are broadcast to aíí connected
machínes on a network, and the machíne wíth the correct IP/MAC address
combínatíon receíves the packet. The IP/MAC assocíatíons are stored ín an ARP
cache, whích ís cíeared períodícaííy (usuaííy every 15 mínutes) and refíííed wíth
IP/MAC assocíatíons.
The íssue wíth ARP requests ín a dírect-routíng LVS confíguratíon ís that because a
cííent request to an IP address must be assocíated wíth a MAC address for the
request to be handíed, the vírtuaí IP address of the LVS router must aíso be
assocíated to a MAC. However, because both the LVS router and the reaí servers
have the same VIP, the ARP request ís broadcast to aíí the nodes assocíated wíth
the VIP. Thís can cause severaí probíems, such as the VIP beíng assocíated dírectíy
to one of the reaí servers and processíng requests dírectíy, bypassíng the LVS router
compíeteíy and defeatíng the purpose of the LVS confíguratíon. Usíng an LVS router
wíth a powerfuí CPU that can respond quíckíy to cííent requests does not necessarííy
remedy thís íssue. If the LVS router ís under heavy íoad, ít may respond to the ARP
request more síowíy than an underutííízed reaí server, whích responds more quíckíy
and ís assígned the VIP ín the ARP cache of the requestíng cííent.
To soíve thís íssue, the íncomíng requests shouíd only assocíate the VIP to the LVS
router, whích wííí properíy process the requests and send them to the reaí server
pooí. Thís can be done by usíng the aJp1abìes packet-fííteríng tooí.
8.4. Persistence and Firewall Marks
In certaín sítuatíons, ít may be desírabíe for a cííent to reconnect repeatedíy to the
same reaí server, rather than have an LVS íoad-baíancíng aígoríthm send that
request to the best avaííabíe server. Exampíes of such sítuatíons íncíude
muítí-screen web forms, cookíes, SSL, and FTP connectíons. In those cases, a cííent
may not work properíy uníess the transactíons are beíng handíed by the same
server to retaín context. LVS provídes two dífferent features to handíe thís:
persistence and firewall marks.
8.4.1. Persistence
Chapter 1. Red Hat Cluster Suite Overview
36
When enabíed, persístence acts ííke a tímer. When a cííent connects to a servíce,
LVS remembers the íast connectíon for a specífíed períod of tíme. If that same cííent
IP address connects agaín wíthín that períod, ít ís sent to the same server ít
connected to prevíousíy - bypassíng the íoad-baíancíng mechanísms. When a
connectíon occurs outsíde the tíme wíndow, ít ís handíed accordíng to the
scheduííng ruíes ín píace.
Persístence aíso aííows you to specífy a subnet mask to appíy to the cííent IP
address test as a tooí for controíííng what addresses have a hígher íeveí of
persístence, thereby groupíng connectíons to that subnet.
Groupíng connectíons destíned for dífferent ports can be ímportant for protocoís
that use more than one port to communícate, such as FTP. However, persístence ís
not the most effícíent way to deaí wíth the probíem of groupíng together
connectíons destíned for dífferent ports. For these sítuatíons, ít ís best to use
firewall marks.
8.4.2. Firewall Marks
Fírewaíí marks are an easy and effícíent way to a group ports used for a protocoí or
group of reíated protocoís. For exampíe, íf LVS ís depíoyed to run an e-commerce
síte, fírewaíí marks can be used to bundíe HTTP connectíons on port 80 and secure,
HTTPS connectíons on port 443. By assígníng the same fírewaíí mark to the vírtuaí
server for each protocoí, state ínformatíon for the transactíon can be preserved
because the LVS router forwards aíí requests to the same reaí server after a
connectíon ís opened.
Because of íts effícíency and ease-of-use, admínístrators of LVS shouíd use fírewaíí
marks ínstead of persístence whenever possíbíe for groupíng connectíons. However,
you shouíd stííí add persístence to the vírtuaí servers ín coníunctíon wíth fírewaíí
marks to ensure the cííents are reconnected to the same server for an adequate
períod of tíme.
9. Cluster Administration Tools
Red Hat Cíuster Suíte provídes a varíety of tooís to confígure and manage your Red
Hat Cíuster. Thís sectíon provídes an overvíew of the admínístratíon tooís avaííabíe
wíth Red Hat Cíuster Suíte:
º 5ection 9.J, ¨Conga"
Cluster Administration Tools
37
º 5ection 9.2, ¨Cluster Administration CUl"
º 5ection 9.3, ¨Command Line Administration Tools"
9.1. Conga
Conga ís an íntegrated set of software components that provídes centraíízed
confíguratíon and management of Red Hat cíusters and storage. Conga provídes
the foííowíng maíor features:
º One Web ínterface for managíng cíuster and storage
º Automated Depíoyment of Cíuster Data and Supportíng Packages
º Easy Integratíon wíth Exístíng Cíusters
º No Need to Re-Authentícate
º Integratíon of Cíuster Status and Logs
º Fíne-Graíned Controí over User Permíssíons
The prímary components ín Conga are luci and ricci, whích are separateíy
ínstaííabíe. luci ís a server that runs on one computer and communícates wíth
muítípíe cíusters and computers vía ricci. ricci ís an agent that runs on each
computer (eíther a cíuster member or a standaíone computer) managed by Conga.
luci ís accessíbíe through a Web browser and provídes three maíor functíons that
are accessíbíe through the foííowíng tabs:
º homebase - Provídes tooís for addíng and deíetíng computers, addíng and
deíetíng users, and confíguríng user prívííeges. Oníy a system admínístrator ís
aííowed to access thís tab.
º cluster - Provídes tooís for creatíng and confíguríng cíusters. Each ínstance of
luci íísts cíusters that have been set up wíth that luci. A system admínístrator can
admíníster aíí cíusters íísted on thís tab. Other users can admíníster oníy cíusters
that the user has permíssíon to manage (granted by an admínístrator).
º storage - Provídes tooís for remote admínístratíon of storage. Wíth the tooís on
thís tab, you can manage storage on computers whether they beíong to a cíuster
or not.
Chapter 1. Red Hat Cluster Suite Overview
38
To admíníster a cíuster or storage, an admínístrator adds (or registers) a cíuster or a
computer to a luci server. When a cíuster or a computer ís regístered wíth luci, the
FODN hostname or IP address of each computer ís stored ín a luci database.
You can popuíate the database of one luci ínstance from another luciínstance. That
capabíííty provídes a means of repíícatíng a luci server ínstance and provídes an
effícíent upgrade and testíng path. When you ínstaíí an ínstance of luci, íts
database ís empty. However, you can ímport part or aíí of a luci database from an
exístíng luci server when depíoyíng a new luci server.
Each luci ínstance has one user at ínítíaí ínstaííatíon - admín. Oníy the admín user
may add systems to a luci server. Aíso, the admín user can create addítíonaí user
accounts and determíne whích users are aííowed to access cíusters and computers
regístered ín the luci database. It ís possíbíe to ímport users as a batch operatíon ín
a new luci server, íust as ít ís possíbíe to ímport cíusters and computers.
When a computer ís added to a luci server to be admínístered, authentícatíon ís
done once. No authentícatíon ís necessary from then on (uníess the certífícate used
ís revoked by a CA). After that, you can remoteíy confígure and manage cíusters
and storage through the luci user ínterface. luci and ricci communícate wíth each
other vía XML.
The foííowíng fígures show sampíe díspíays of the three maíor luci tabs:
homebase, cluster, and storage.
For more ínformatíon about Conga, refer to Configuring and Managing a Red Hat
Cluster and the onííne heíp avaííabíe wíth the luci server.
Conga
39
Figure 1.24. luci homebase Tab
Figure 1.25. luci cluster Tab
Chapter 1. Red Hat Cluster Suite Overview
40
Figure 1.26. luci storage Tab
9.2. Cluster Administration GUl
Thís sectíon provídes an overvíew of the sys1em-coh11g-cìus1eJ cíuster
admínístratíon graphícaí user ínterface (GUI) avaííabíe wíth Red Hat Cíuster Suíte.
The GUI ís for use wíth the cíuster ínfrastructure and the hígh-avaííabíííty servíce
management components (refer to 5ection 3, ¨Cluster lnfrastructure" and
5ection 4, ¨High-availability 5ervice Management"). The GUI consísts of two maíor
functíons: the Cluster Configuration Tool and the Cluster Status Tool. The
Cluster Configuration Tool provídes the capabíííty to create, edít, and propagate
the cíuster confíguratíon fííe (/e1c/cìus1eJ/cìus1eJ.coh1). The Cluster Status Tool
provídes the capabíííty to manage hígh-avaííabíííty servíces. The foííowíng sectíons
summaríze those functíons.
Cluster Administration GUl
41
º 5ection 9.2.J, ¨Cluster Configuration Tool"
º 5ection 9.2.2, ¨Cluster Status Tool"
9.2.1. Cluster Configuration Tool
You can access the Cluster Configuration Tool (Figure J.27, ¨Cluster
Configuration Tool") through the Cluster Configuration tab ín the Cíuster
Admínístratíon GUI.
Figure 1.27. Cluster Configuration Tool
Chapter 1. Red Hat Cluster Suite Overview
42
The Cluster Configuration Tool represents cíuster confíguratíon components ín
the confíguratíon fííe (/e1c/cìus1eJ/cìus1eJ.coh1) wíth a híerarchícaí graphícaí
díspíay ín the íeft paneí. A tríangíe ícon to the íeft of a component name índícates
that the component has one or more subordínate components assígned to ít.
Cííckíng the tríangíe ícon expands and coííapses the portíon of the tree beíow a
component. The components díspíayed ín the GUI are summarízed as foííows:
º Cluster Nodes - Díspíays cíuster nodes. Nodes are represented by name as
subordínate eíements under Cluster Nodes. Usíng confíguratíon buttons at the
bottom of the ríght frame (beíow Properties), you can add nodes, deíete nodes,
edít node propertíes, and confígure fencíng methods for each node.
º Fence Devices - Díspíays fence devíces. Fence devíces are represented as
subordínate eíements under Fence Devices. Usíng confíguratíon buttons at the
bottom of the ríght frame (beíow Properties), you can add fence devíces, deíete
fence devíces, and edít fence-devíce propertíes. Fence devíces must be defíned
before you can confígure fencíng (wíth the Manage Fencing For This Node
button) for each node.
º Managed Resources - Díspíays faííover domaíns, resources, and servíces.
º Failover Domains - For confíguríng one or more subsets of cíuster nodes
used to run a hígh-avaííabíííty servíce ín the event of a node faííure. Faííover
domaíns are represented as subordínate eíements under Failover Domains.
Usíng confíguratíon buttons at the bottom of the ríght frame (beíow
Properties), you can create faííover domaíns (when Failover Domains ís
seíected) or edít faííover domaín propertíes (when a faííover domaín ís
seíected).
º Resources - For confíguríng shared resources to be used by hígh-avaííabíííty
servíces. Shared resources consíst of fííe systems, IP addresses, NFS mounts
and exports, and user-created scrípts that are avaííabíe to any hígh-avaííabíííty
servíce ín the cíuster. Resources are represented as subordínate eíements
under Resources. Usíng confíguratíon buttons at the bottom of the ríght frame
(beíow Properties), you can create resources (when Resources ís seíected) or
edít resource propertíes (when a resource ís seíected).
Note
Cluster Administration GUl
43
The Cluster Configuration Tool provídes the capabíííty to confígure
prívate resources, aíso. A prívate resource ís a resource that ís
confígured for use wíth oníy one servíce. You can confígure a prívate
resource wíthín a Service component ín the GUI.
º Services - For creatíng and confíguríng hígh-avaííabíííty servíces. A servíce ís
confígured by assígníng resources (shared or prívate), assígníng a faííover
domaín, and defíníng a recovery poíícy for the servíce. Servíces are represented
as subordínate eíements under Services. Usíng confíguratíon buttons at the
bottom of the ríght frame (beíow Properties), you can create servíces (when
Services ís seíected) or edít servíce propertíes (when a servíce ís seíected).
9.2.2. Cluster Status Tool
You can access the Cluster Status Tool (Figure J.28, ¨Cluster Status Tool")
through the Cluster Management tab ín Cíuster Admínístratíon GUI.
Chapter 1. Red Hat Cluster Suite Overview
44
Figure 1.28. Cluster Status Tool
The nodes and servíces díspíayed ín the Cluster Status Tool are determíned by
the cíuster confíguratíon fííe (/e1c/cìus1eJ/cìus1eJ.coh1). You can use the Cluster
Status Tool to enabíe, dísabíe, restart, or reíocate a hígh-avaííabíííty servíce.
9.3. Command Line Administration Tools
In addítíon to Conga and the sys1em-coh11g-cìus1eJ Cíuster Admínístratíon GUI,
command ííne tooís are avaííabíe for admínísteríng the cíuster ínfrastructure and the
hígh-avaííabíííty servíce management components. The command ííne tooís are
Command Line Administration Tools
45
used by the Cíuster Admínístratíon GUI and ínít scrípts suppííed by Red Hat.
Table J.J, ¨Command Line Tools" summarízes the command ííne tooís.
Command
Line Tool
Used With Purpose
ccs_1ooì -
Cíuster
Confíguratíon
System Tooí
Cíuster
Infrastructure
ccs_1ooì ís a program for makíng onííne updates
to the cíuster confíguratíon fííe. It provídes the
capabíííty to create and modífy cíuster
ínfrastructure components (for exampíe,
creatíng a cíuster, addíng and removíng a node).
For more ínformatíon about thís tooí, refer to the
ccs¸tooí(8) man page.
cmah_1ooì -
Cíuster
Management
Tooí
Cíuster
Infrastructure
cmah_1ooì ís a program that manages the CMAN
cíuster manager. It provídes the capabíííty to íoín
a cíuster, íeave a cíuster, kííí a node, or change
the expected quorum votes of a node ín a
cíuster. For more ínformatíon about thís tooí,
refer to the cman¸tooí(8) man page.
1ehce_1ooì -
Fence Tooí
Cíuster
Infrastructure
1ehce_1ooì ís a program used to íoín or íeave the
defauít fence domaín. Specífícaííy, ít starts the
fence daemon (1ehced) to íoín the domaín and
kííís 1ehced to íeave the domaín. For more
ínformatíon about thís tooí, refer to the
fence¸tooí(8) man page.
cìus1a1 -
Cíuster Status
Utíííty
Hígh-avaííabíííty
Servíce
Management
Components
The cìus1a1 command díspíays the status of the
cíuster. It shows membershíp ínformatíon,
quorum víew, and the state of aíí confígured
user servíces. For more ínformatíon about thís
tooí, refer to the cíustat(8) man page.
cìusvcadm -
Cíuster User
Servíce
Admínístratíon
Utíííty
Hígh-avaííabíííty
Servíce
Management
Components
The cìusvcadm command aííows you to enabíe,
dísabíe, reíocate, and restart hígh-avaííabíííty
servíces ín a cíuster. For more ínformatíon about
thís tooí, refer to the cíusvcadm(8) man page.
Table 1.1. Command Line Tools
10. Linux Virtual Server Administration GUl
Chapter 1. Red Hat Cluster Suite Overview
46
Thís sectíon provídes an overvíew of the LVS confíguratíon tooí avaííabíe wíth Red
Hat Cíuster Suíte - the Piranha Configuration Tool. The Piranha Configuration
Tool ís a Web-browser graphícaí user ínterface (GUI) that provídes a structured
approach to creatíng the confíguratíon fííe for LVS - /e1c/syscoh11g/ha/ìvs.c1.
To access the Piranha Configuration Tool you need the p1Jahha-gu1 servíce
runníng on the actíve LVS router. You can access the Piranha Configuration Tool
íocaííy or remoteíy wíth a Web browser. You can access ít íocaííy wíth thís URL:
h11p://1oca1hos1:3636. You can access ít remoteíy wíth eíther the hostname or the
reaí IP address foííowed by :3636. If you are accessíng the Piranha Configuration
Tool remoteíy, you need an ssh connectíon to the actíve LVS router as the root
user.
Startíng the Piranha Configuration Tool causes the Piranha Configuration
Tool weícome page to be díspíayed (refer to Figure J.29, ¨The Welcome Panel").
Loggíng ín to the weícome page provídes access to the four maín screens or panels:
CONTROLlMONlTORlNG, GLOBAL SETTlNGS, REDUNDANCY, and VlRTUAL
SERVERS. In addítíon, the VlRTUAL SERVERS paneí contaíns four subsections.
The CONTROLlMONlTORlNG paneí ís the fírst paneí díspíayed after you íog ín at
the weícome screen.
Figure 1.29. The Welcome Panel
The foííowíng sectíons províde a bríef descríptíon of the Piranha Configuration
Linux Virtual Server Administration
47
Tool confíguratíon pages.
10.1. CONTROLlMONlTORlNG
The CONTROLlMONlTORlNG Paneí díspíays runtíme status. It díspíays the status
of the puìse daemon, the LVS routíng tabíe, and the LVS-spawned hahhy processes.
Figure 1.30. The CONTROLlMONlTORlNG Panel
Auto update
Enabíes the status díspíay to be updated automatícaííy at a user-confígurabíe
íntervaí set ín the Update frequency in seconds text box (the defauít vaíue ís
10 seconds).
It ís not recommended that you set the automatíc update to an íntervaí íess than
10 seconds. Doíng so may make ít díffícuít to reconfígure the Auto update
Chapter 1. Red Hat Cluster Suite Overview
48
íntervaí because the page wííí update too frequentíy. If you encounter thís íssue,
símpíy cííck on another paneí and then back on CONTROLlMONlTORlNG.
Update information now
Provídes manuaí update of the status ínformatíon.
CHANGE PASSWORD
Cííckíng thís button takes you to a heíp screen wíth ínformatíon on how to
change the admínístratíve password for the Piranha Configuration Tool.
10.2. GLOBAL SETTlNGS
The GLOBAL SETTlNGS paneí ís where the LVS admínístrator defínes the
networkíng detaíís for the prímary LVS router's pubííc and prívate network
ínterfaces.
Figure 1.31. The GLOBAL SETTlNGS Panel
GUl
49
The top haíf of thís paneí sets up the prímary LVS router's pubííc and prívate
network ínterfaces.
Primary server public lP
The pubíícíy routabíe reaí IP address for the prímary LVS node.
Primary server private lP
The reaí IP address for an aíternatíve network ínterface on the prímary LVS
node. Thís address ís used soíeíy as an aíternatíve heartbeat channeí for the
backup router.
Use network type
Seíects seíect NAT routíng.
The next three fíeíds are specífícaííy for the NAT router's vírtuaí network ínterface
connected the prívate network wíth the reaí servers.
NAT Router lP
The prívate fíoatíng IP ín thís text fíeíd. Thís fíoatíng IP shouíd be used as the
gateway for the reaí servers.
NAT Router netmask
If the NAT router's fíoatíng IP needs a partícuíar netmask, seíect ít from
drop-down ííst.
NAT Router device
Defínes the devíce name of the network ínterface for the fíoatíng IP address,
such as e1h1:1.
10.3. REDUNDANCY
The REDUNDANCY paneí aííows you to confígure of the backup LVS router node
and set varíous heartbeat monítoríng optíons.
Chapter 1. Red Hat Cluster Suite Overview
50
Figure 1.32. The REDUNDANCY Panel
Redundant server public lP
The pubííc reaí IP address for the backup LVS router.
Redundant server private lP
The backup router's prívate reaí IP address.
The rest of the paneí ís for confíguríng the heartbeat channeí, whích ís used by the
backup node to monítor the prímary node for faííure.
Heartbeat lnterval (seconds)
Sets the number of seconds between heartbeats - the íntervaí that the backup
node wííí check the functíonaí status of the prímary LVS node.
REDUNDANCY
51
Assume dead after (seconds)
If the prímary LVS node does not respond after thís number of seconds, then the
backup LVS router node wííí ínítíate faííover.
Heartbeat runs on port
Sets the port at whích the heartbeat communícates wíth the prímary LVS node.
The defauít ís set to 539 íf thís fíeíd ís íeft bíank.
10.4. VlRTUAL SERVERS
The VlRTUAL SERVERS paneí díspíays ínformatíon for each currentíy defíned
vírtuaí server. Each tabíe entry shows the status of the vírtuaí server, the server
name, the vírtuaí IP assígned to the server, the netmask of the vírtuaí IP, the port
number to whích the servíce communícates, the protocoí used, and the vírtuaí
devíce ínterface.
Figure 1.33. The VlRTUAL SERVERS Panel
Chapter 1. Red Hat Cluster Suite Overview
52
Each server díspíayed ín the VlRTUAL SERVERS paneí can be confígured on
subsequent screens or subsections.
To add a servíce, cííck the ADD button. To remove a servíce, seíect ít by cííckíng the
radío button next to the vírtuaí server and cííck the DELETE button.
To enabíe or dísabíe a vírtuaí server ín the tabíe cííck íts radío button and cííck the
(DE)ACTlVATE button.
After addíng a vírtuaí server, you can confígure ít by cííckíng the radío button to íts
íeft and cííckíng the EDlT button to díspíay the VlRTUAL SERVER subsectíon.
10.4.1. The VlRTUAL SERVER Subsection
The VlRTUAL SERVER subsectíon paneí shown ín Figure J.34, ¨The VlRTUAL
SERVERS 5ubsection" aííows you to confígure an índívíduaí vírtuaí server. Línks to
subsectíons reíated specífícaííy to thís vírtuaí server are íocated aíong the top of the
page. But before confíguríng any of the subsectíons reíated to thís vírtuaí server,
compíete thís page and cííck on the ACCEPT button.
VlRTUAL SERVERS
53
Figure 1.34. The VlRTUAL SERVERS Subsection
Name
A descríptíve name to ídentífy the vírtuaí server. Thís name ís not the hostname
for the machíne, so make ít descríptíve and easííy ídentífíabíe. You can even
reference the protocoí used by the vírtuaí server, such as HTTP.
Application port
The port number through whích the servíce appíícatíon wííí íísten.
Protocol
Provídes a choíce of UDP or TCP, ín a drop-down menu.
Virtual lP Address
The vírtuaí server's fíoatíng IP address.
Virtual lP Network Mask
The netmask for thís vírtuaí server, ín the drop-down menu.
Firewall Mark
For enteríng a fírewaíí mark ínteger vaíue when bundííng muítí-port protocoís or
creatíng a muítí-port vírtuaí server for separate, but reíated protocoís.
Device
The name of the network devíce to whích you want the fíoatíng IP address
defíned ín the Virtual lP Address fíeíd to bínd.
You shouíd aíías the pubííc fíoatíng IP address to the Ethernet ínterface
connected to the pubííc network.
Re-entry Time
An ínteger vaíue that defínes the number of seconds before the actíve LVS
router attempts to use a reaí server after the reaí server faííed.
Service Timeout
An ínteger vaíue that defínes the number of seconds before a reaí server ís
consídered dead and not avaííabíe.
Ouiesce server
Chapter 1. Red Hat Cluster Suite Overview
54
When the Ouiesce server radío button ís seíected, anytíme a new reaí server
node comes onííne, the íeast-connectíons tabíe ís reset to zero so the actíve LVS
router routes requests as íf aíí the reaí servers were freshíy added to the cíuster.
Thís optíon prevents the a new server from becomíng bogged down wíth a hígh
number of connectíons upon enteríng the cíuster.
Load monitoring tool
The LVS router can monítor the íoad on the varíous reaí servers by usíng eíther
Jup or Jup11me. If you seíect Jup from the drop-down menu, each reaí server
must run the Js1a1d servíce. If you seíect Jup11me, each reaí server must run the
Jwhod servíce.
Scheduling
The preferred scheduííng aígoríthm from the drop-down menu. The defauít ís
WeÍgh1ed 1eas1-connec1Íon.
Persistence
Used íf you need persístent connectíons to the vírtuaí server duríng cííent
transactíons. Specífíes the number of seconds of ínactívíty aííowed to íapse
before a connectíon tímes out ín thís text fíeíd.
Persistence Network Mask
To íímít persístence to partícuíar subnet, seíect the appropríate network mask
from the drop-down menu.
10.4.2. REAL SERVER Subsection
Cííckíng on the REAL SERVER subsectíon íínk at the top of the paneí díspíays the
EDlT REAL SERVER subsectíon. It díspíays the status of the physícaí server hosts
for a partícuíar vírtuaí servíce.
VlRTUAL SERVERS
55
Figure 1.35. The REAL SERVER Subsection
Cííck the ADD button to add a new server. To deíete an exístíng server, seíect the
radío button besíde ít and cííck the DELETE button. Cííck the EDlT button to íoad
the EDlT REAL SERVER paneí, as seen ín Figure J.36, ¨The REAL SERVER
Configuration Panel".
Chapter 1. Red Hat Cluster Suite Overview
56
Figure 1.36. The REAL SERVER Configuration Panel
Thís paneí consísts of three entry fíeíds:
Name
A descríptíve name for the reaí server.
Tip
Thís name ís not the hostname for the machíne, so make ít descríptíve
and easííy ídentífíabíe.
Address
The reaí server's IP address. Sínce the íísteníng port ís aíready specífíed for the
VlRTUAL SERVERS
57
assocíated vírtuaí server, do not add a port number.
Weight
An ínteger vaíue índícatíng thís host's capacíty reíatíve to that of other hosts ín
the pooí. The vaíue can be arbítrary, but treat ít as a ratío ín reíatíon to other
reaí servers.
10.4.3. EDlT MONlTORlNG SCRlPTS Subsection
Cííck on the MONlTORlNG SCRlPTS íínk at the top of the page. The EDlT
MONlTORlNG SCRlPTS subsectíon aííows the admínístrator to specífy a
send/expect stríng sequence to verífy that the servíce for the vírtuaí server ís
functíonaí on each reaí server. It ís aíso the píace where the admínístrator can
specífy customízed scrípts to check servíces requíríng dynamícaííy changíng data.
Figure 1.37. The EDlT MONlTORlNG SCRlPTS Subsection
Chapter 1. Red Hat Cluster Suite Overview
58
Sending Program
For more advanced servíce verífícatíon, you can use thís fíeíd to specífy the path
to a servíce-checkíng scrípt. Thís functíon ís especíaííy heípfuí for servíces that
requíre dynamícaííy changíng data, such as HTTPS or SSL.
To use thís functíon, you must wríte a scrípt that returns a textuaí response, set
ít to be executabíe, and type the path to ít ín the Sending Program fíeíd.
Note
If an externaí program ís entered ín the Sending Program fíeíd, then
the Send fíeíd ís ígnored.
Send
A stríng for the hahhy daemon to send to each reaí server ín thís fíeíd. By defauít
the send fíeíd ís compíeted for HTTP. You can aíter thís vaíue dependíng on your
needs. If you íeave thís fíeíd bíank, the hahhy daemon attempts to open the port
and assume the servíce ís runníng íf ít succeeds.
Oníy one send sequence ís aííowed ín thís fíeíd, and ít can oníy contaín príntabíe,
ASCII characters as weíí as the foííowíng escape characters:
º \n for new ííne.
º \r for carríage return.
º \t for tab.
º \ to escape the next character whích foííows ít.
Expect
The textuaí response the server shouíd return íf ít ís functíoníng properíy. If you
wrote your own sendíng program, enter the response you toíd ít to send íf ít was
successfuí.
VlRTUAL SERVERS
59
60
Red Hat Cluster Suite
Component Summary
Thís chapter provídes a summary of Red Hat Cíuster Suíte components and consísts
of the foííowíng sectíons:
º 5ection J, ¨Cluster Components"
º 5ection 2, ¨Man Pages"
º 5ection 3, ¨Compatible Hardware"
1. Cluster Components
Table 2.J, ¨Red Hat Cluster 5uite 5oftware 5ubsystem Components" summarízes
Red Hat Cíuster Suíte components.
Function Components Description
Conga ìuc1 Remote Management System -
Management Statíon.
J1cc1 Remote Management System -
Managed Statíon.
Cluster
Configuration
Tool
sys1em-coh11g-cìus1eJ Command used to manage
cíuster confíguratíon ín a
graphícaí settíng.
Cíuster Logícaí
Voíume Manager
(CLVM)
cìvmd The daemon that dístríbutes LVM
metadata updates around a
cíuster. It must be runníng on aíí
nodes ín the cíuster and wííí gíve
an error íf a node ín the cíuster
does not have thís daemon
runníng.
ìvm LVM2 tooís. Provídes the
command-ííne tooís for LVM2.
sys1em-coh11g-ìvm Provídes graphícaí user ínterface
for LVM2.
Chapter 2.
61
Function Components Description
ìvm.coh1 The LVM confíguratíon fííe. The
fuíí path ís /e1c/ìvm/ìvm.coh1.
Cíuster
Confíguratíon
System (CCS)
ccs_1ooì ccs_1ooì ís part of the Cíuster
Confíguratíon System (CCS). It ís
used to make onííne updates of
CCS confíguratíon fííes.
Addítíonaííy, ít can be used to
upgrade cíuster confíguratíon
fííes from CCS archíves created
wíth GFS 6.0 (and earííer) to the
XML format confíguratíon format
used wíth thís reíease of Red Hat
Cíuster Suíte.
ccs_1es1 Díagnostíc and testíng command
that ís used to retríeve
ínformatíon from confíguratíon
fííes through ccsd.
ccsd CCS daemon that runs on aíí
cíuster nodes and provídes
confíguratíon fííe data to cíuster
software.
cìus1eJ.coh1 Thís ís the cíuster confíguratíon
fííe. The fuíí path ís
/e1c/cìus1eJ/cìus1eJ.coh1.
Cíuster Manager
(CMAN)
cmah.ko The kerneí moduíe for CMAN.
cmah_1ooì Thís ís the admínístratíve front
end to CMAN. It starts and stops
CMAN and can change some
ínternaí parameters such as
votes.
dìm_coh1Joìd Daemon started by cmah ínít
scrípt to manage dìm ín kerneí;
not used by user.
g1s_coh1Joìd Daemon started by cmah ínít
scrípt to manage g1s ín kerneí;
Chapter 2. Red Hat Cluster Suite Component Summary
62
Function Components Description
not used by user.
gJoup_1ooì Used to get a ííst of groups
reíated to fencíng, DLM, GFS, and
gettíng debug ínformatíon;
íncíudes what cmah_1ooì
seJv1ces províded ín RHEL 4.
gJoupd Daemon started by cmah ínít
scrípt to ínterface between
opeha1s/cmah and
dìm_coh1Joìd/g1s_coh1Joìd/1ehced;
not used by user.
ì1bcmah.so.<veIsIon
nunbeI>
Líbrary for programs that need to
ínteract wíth cmah.ko.
Resource Group
Manager
(rgmanager)
cìusvcadm Command used to manuaííy
enabíe, dísabíe, reíocate, and
restart user servíces ín a cíuster.
cìus1a1 Command used to díspíay the
status of the cíuster, íncíudíng
node membershíp and servíces
runníng.
cìuJgmgJd Daemon used to handíe user
servíce requests íncíudíng
servíce start, servíce dísabíe,
servíce reíocate, and servíce
restart.
cìuJm1abd Daemon used to handíe
Cíustered NFS mount tabíes.
Fence 1ehce_apc Fence agent for APC power
swítch.
1ehce_bìadeceh1eJ Fence agent for for IBM
Bíadecenters wíth Teínet
ínterface.
1ehce_buììpap Fence agent for Buíí Novascaíe
Píatform Admínístratíon
Processor (PAP) Interface.
Cluster Components
63
Function Components Description
1ehce_dJac Fencíng agent for Deíí Remote
Access Card.
1ehce_1pm1ìah Fence agent for machínes
controííed by IPMI (Inteííígent
Píatform Management Interface)
over LAN.
1ehce_w11 Fence agent for WTI power
swítch.
1ehce_bJocade Fence agent for Brocade Fíbre
Channeí swítch.
1ehce_mcda1a Fence agent for McData Fíbre
Channeí swítch.
1ehce_v1xeì Fence agent for Víxeí Fíbre
Channeí swítch.
1ehce_sahbox2 Fence agent for SANBox2 Fíbre
Channeí swítch.
1ehce_1ìo Fence agent for HP ILO ínterfaces
(formeríy fence¸ríb).
1ehce_Jsa I/O Fencíng agent for IBM RSA II.
1ehce_ghbd Fence agent used wíth GNBD
storage.
1ehce_scs1 I/O fencíng agent for SCSI
persístent reservatíons.
1ehce_egeheJa Fence agent used wíth Egenera
BíadeFrame system.
1ehce_mahuaì Fence agent for manuaí
ínteractíon. NOTE Thís
component ís not supported for
productíon envíronments.
1ehce_ack_mahuaì User ínterface for 1ehce_mahuaì
agent.
1ehce_hode A program whích performs I/O
fencíng on a síngíe node.
1ehce_xvm I/O Fencíng agent for Xen vírtuaí
Chapter 2. Red Hat Cluster Suite Component Summary
64
Function Components Description
machínes.
1ehce_xvmd I/O Fencíng agent host for Xen
vírtuaí machínes.
1ehce_1ooì A program to íoín and íeave the
fence domaín.
1ehced The I/O Fencíng daemon.
DLM ì1bdìm.so.<veIsIon
nunbeI>
Líbrary for Dístríbuted Lock
Manager (DLM) support.
GFS g1s.ko Kerneí moduíe that ímpíements
the GFS fííe system and ís íoaded
on GFS cíuster nodes.
g1s_1sck Command that repaírs an
unmounted GFS fííe system.
g1s_gJow Command that grows a mounted
GFS fííe system.
g1s_1add Command that adds íournaís to a
mounted GFS fííe system.
g1s_mk1s Command that creates a GFS fííe
system on a storage devíce.
g1s_quo1a Command that manages quotas
on a mounted GFS fííe system.
g1s_1ooì Command that confígures or
tunes a GFS fííe system. Thís
command can aíso gather a
varíety of ínformatíon about the
fííe system.
mouh1.g1s Mount heíper caííed by mouh1{8);
not used by user.
GNBD ghbd.ko Kerneí moduíe that ímpíements
the GNBD devíce dríver on
cííents.
ghbd_expoJ1 Command to create, export and
manage GNBDs on a GNBD
server.
Cluster Components
65
Function Components Description
ghbd_1mpoJ1 Command to ímport and manage
GNBDs on a GNBD cííent.
ghbd_seJv A server daemon that aííows a
node to export íocaí storage over
the network.
LVS puìse Thís ís the controíííng process
whích starts aíí other daemons
reíated to LVS routers. At boot
tíme, the daemon ís started by
the /e1c/Jc.d/1h11.d/puìse
scrípt. It then reads the
confíguratíon fííe
/e1c/syscoh11g/ha/ìvs.c1. On
the actíve LVS router, puìse
starts the LVS daemon. On the
backup router, puìse determínes
the heaíth of the actíve router by
executíng a símpíe heartbeat at
a user-confígurabíe íntervaí. If
the actíve LVS router faíís to
respond after a user-confígurabíe
íntervaí, ít ínítíates faííover.
Duríng faííover, puìse on the
backup LVS router ínstructs the
puìse daemon on the actíve LVS
router to shut down aíí LVS
servíces, starts the sehd_aJp
program to reassígn the fíoatíng
IP addresses to the backup LVS
router's MAC address, and starts
the ìvs daemon.
ìvsd The ìvs daemon runs on the
actíve LVS router once caííed by
puìse. It reads the confíguratíon
fííe /e1c/syscoh11g/ha/ìvs.c1,
caíís the 1pvsadm utíííty to buííd
and maíntaín the IPVS routíng
Chapter 2. Red Hat Cluster Suite Component Summary
66
Function Components Description
tabíe, and assígns a hahhy
process for each confígured LVS
servíce. If hahhy reports a reaí
server ís down, ìvs ínstructs the
1pvsadm utíííty to remove the reaí
server from the IPVS routíng
tabíe.
1pvsadm Thís servíce updates the IPVS
routíng tabíe ín the kerneí. The
ìvs daemon sets up and
admínísters LVS by caíííng
1pvsadm to add, change, or deíete
entríes ín the IPVS routíng tabíe.
hahhy The hahhy monítoríng daemon
runs on the actíve LVS router.
Through thís daemon, the actíve
LVS router determínes the heaíth
of each reaí server and,
optíonaííy, monítors íts workíoad.
A separate process runs for each
servíce defíned on each reaí
server.
ìvs.c1 Thís ís the LVS confíguratíon fííe.
The fuíí path for the fííe ís
/e1c/syscoh11g/ha/ìvs.c1.
Dírectíy or índírectíy, aíí daemons
get theír confíguratíon
ínformatíon from thís fííe.
Piranha Configuration
Tool
Thís ís the Web-based tooí for
monítoríng, confíguríng, and
admínísteríng LVS. Thís ís the
defauít tooí to maíntaín the
/e1c/syscoh11g/ha/ìvs.c1 LVS
confíguratíon fííe.
sehd_aJp Thís program sends out ARP
broadcasts when the fíoatíng IP
address changes from one node
Cluster Components
67
Function Components Description
to another duríng faííover.
Ouorum Dísk qd1sk A dísk-based quorum daemon for
CMAN / Línux-Cíuster.
mkqd1sk Cíuster Ouorum Dísk Utíííty.
qd1skd Cíuster Ouorum Dísk Daemon.
Table 2.1. Red Hat Cluster Suite Software Subsystem
Components
2. Man Pages
Thís sectíon íísts man pages that are reíevant to Red Hat Cíuster Suíte, as an
addítíonaí resource.
º Cíuster Infrastructure
º ccs¸tooí (8) - The tooí used to make onííne updates of CCS confíg fííes
º ccs¸test (8) - The díagnostíc tooí for a runníng Cíuster Confíguratíon System
º ccsd (8) - The daemon used to access CCS cíuster confíguratíon fííes
º ccs (7) - Cíuster Confíguratíon System
º cman¸tooí (8) - Cíuster Management Tooí
º cíuster.conf lcíuster| (5) - The confíguratíon fííe for cíuster products
º qdísk (5) - a dísk-based quorum daemon for CMAN / Línux-Cíuster
º mkqdísk (8) - Cíuster Ouorum Dísk Utíííty
º qdískd (8) - Cíuster Ouorum Dísk Daemon
º fence¸ack¸manuaí (8) - program run by an operator as a part of manuaí I/O
Fencíng
º fence¸apc (8) - I/O Fencíng agent for APC MasterSwítch
º fence¸bíadecenter (8) - I/O Fencíng agent for IBM Bíadecenter
Chapter 2. Red Hat Cluster Suite Component Summary
68
º fence¸brocade (8) - I/O Fencíng agent for Brocade FC swítches
º fence¸buíípap (8) - I/O Fencíng agent for Buíí FAME archítecture controííed by a
PAP management consoíe
º fence¸drac (8) - fencíng agent for Deíí Remote Access Card
º fence¸egenera (8) - I/O Fencíng agent for the Egenera BíadeFrame
º fence¸gnbd (8) - I/O Fencíng agent for GNBD-based GFS cíusters
º fence¸íío (8) - I/O Fencíng agent for HP Integrated Líghts Out card
º fence¸ípmíían (8) - I/O Fencíng agent for machínes controííed by IPMI over LAN
º fence¸manuaí (8) - program run by fenced as a part of manuaí I/O Fencíng
º fence¸mcdata (8) - I/O Fencíng agent for McData FC swítches
º fence¸node (8) - A program whích performs I/O fencíng on a síngíe node
º fence¸ríb (8) - I/O Fencíng agent for Compaq Remote Insíght Líghts Out card
º fence¸rsa (8) - I/O Fencíng agent for IBM RSA II
º fence¸sanbox2 (8) - I/O Fencíng agent for OLogíc SANBox2 FC swítches
º fence¸scsí (8) - I/O fencíng agent for SCSI persístent reservatíons
º fence¸tooí (8) - A program to íoín and íeave the fence domaín
º fence¸víxeí (8) - I/O Fencíng agent for Víxeí FC swítches
º fence¸wtí (8) - I/O Fencíng agent for WTI Network Power Swítch
º fence¸xvm (8) - I/O Fencíng agent for Xen vírtuaí machínes
º fence¸xvmd (8) - I/O Fencíng agent host for Xen vírtuaí machínes
º fenced (8) - the I/O Fencíng daemon
º Hígh-avaííabíííty Servíce Management
º cíusvcadm (8) - Cíuster User Servíce Admínístratíon Utíííty
º cíustat (8) - Cíuster Status Utíííty
º Cíurgmgrd lcíurgmgrd| (8) - Resource Group (Cíuster Servíce) Manager Daemon
Man Pages
69
º cíurmtabd (8) - Cíuster NFS Remote Mount Tabíe Daemon
º GFS
º gfs¸fsck (8) - Offííne GFS fííe system checker
º gfs¸grow (8) - Expand a GFS fííesystem
º gfs¸íadd (8) - Add íournaís to a GFS fííesystem
º gfs¸mount (8) - GFS mount optíons
º gfs¸quota (8) - Manípuíate GFS dísk quotas
º gfs¸tooí (8) - ínterface to gfs íoctí caíís
º Cíuster Logícaí Voíume Manager
º cívmd (8) - cíuster LVM daemon
º ívm (8) - LVM2 tooís
º ívm.conf lívm| (5) - Confíguratíon fííe for LVM2
º ívmchange (8) - change attríbutes of the íogícaí voíume manager
º pvcreate (8) - ínítíaííze a dísk or partítíon for use by LVM
º ívs (8) - report ínformatíon about íogícaí voíumes
º Gíobaí Network Bíock Devíce
º gnbd¸export (8) - the ínterface to export GNBDs
º gnbd¸ímport (8) - manípuíate GNBD bíock devíces on a cííent
º gnbd¸serv (8) - gnbd server daemon
º LVS
º puíse (8) - heartbeatíng daemon for monítoríng the heaíth of cíuster nodes
º ívs.cf lívs| (5) - confíguratíon fííe for ívs
º ívscan (8) - scan (aíí dísks) for íogícaí voíumes
º ívsd (8) - daemon to controí the Red Hat cíusteríng servíces
Chapter 2. Red Hat Cluster Suite Component Summary
70
º ípvsadm (8) - Línux Vírtuaí Server admínístratíon
º ípvsadm-restore (8) - restore the IPVS tabíe from stdín
º ípvsadm-save (8) - save the IPVS tabíe to stdout
º nanny (8) - tooí to monítor status of servíces ín a cíuster
º send¸arp (8) - tooí to notífy network of a new IP address / MAC address mappíng
3. Compatible Hardware
For ínformatíon about hardware that ís compatíbíe wíth Red Hat Cíuster Suíte
components (for exampíe, supported fence devíces, storage devíces, and Fíbre
Channeí swítches), refer to the hardware confíguratíon guídeíínes at
http://www.redhat.com/cluster¸suite/hardware/.
Compatible Hardware
71
72
lndex
C
cíuster
díspíayíng status, 45
cíuster admínístratíon
díspíayíng cíuster and servíce status,
45
cíuster component compatíbíe hardware,
71
cíuster component man pages, 68
cíuster components tabíe, 61
Cíuster Confíguratíon Tooí
accessíng, 44
cíuster servíce
díspíayíng status, 45
command ííne tooís tabíe, 45
compatíbíe hardware
cíuster components, 71
Conga
overvíew, 38
Conga overvíew, 38
F
feedback, x
l
íntroductíon, víí
other Red Hat Enterpríse Línux
documents, víí
L
LVS
dírect routíng
requírements, hardware, 34
requírements, network, 34
requírements, software, 34
routíng methods
NAT, 33
three tíered
hígh-avaííabíííty cíuster, 31
M
man pages
cíuster components, 68
N
NAT
routíng methods, LVS, 33
network address transíatíon (see NAT)
O
overvíew
economy, 18
performance, 18
scaíabíííty, 18
P
Píranha Confíguratíon Tooí
CONTROL/MONITORING, 48
EDIT MONITORING SCRIPTS
Subsectíon, 58
GLOBAL SETTINGS, 49
íogín paneí, 47
necessary software, 47
REAL SERVER subsectíon, 55
REDUNDANCY, 50
VIRTUAL SERVER subsectíon, 53
Fírewaíí Mark, 54
Persístence, 55
Scheduííng, 55
Vírtuaí IP Address, 54
VIRTUAL SERVERS, 52
R
Red Hat Cíuster Suíte
components, 61
73
T
tabíe
cíuster components, 61
command ííne tooís, 45
lndex
74

Sign up to vote on this title
UsefulNot useful