Seite 1

Welcome to the speech:

Welcome to the speech:
Active/Active HA Clustering
Active/Active HA Clustering
on Shared Storage with Samba
on Shared Storage with Samba
Speaker: Speaker: Dipl.-Ing. Thomas Merz / ATIX Gm! Dipl.-Ing. Thomas Merz / ATIX Gm!
Date: Date: "#/"$/%""# & SamaX' %""# "#/"$/%""# & SamaX' %""#
eMail: eMail: merz(ati).*e merz(ati).*e
Seite %
+,tline
In-rastr,ct,re T.pes / Storage 0irt,alization
!A 1l,stering
T.pes an* 'ossiilities
Application 'rolems
2iles.stem Iss,es
Share* Storage
Sama an* Share* Storage
+,tlook
Seite 3
41lassical5 In-rastr,ct,re
DAS 6Direct Attache*
Storage7
1lassical: 'arallel S1SI
Ser8er an* Storage are
one ,nit
a* catastrophe
preca,tions possile
complete comm,nication
,sing the 8er. same 9A:
'er-ormance an* scaling
ottleneck
Server
storage
Server
storage
Server
library
LAN
Seite $
Storage :et;ork
dedicated storage network
Primary Storage Library
Server
Ser8er an* storage
connection ,sing a
*e*icate* storage
net;ork
Spacial separation o-
ser8ers an* storage
possile
Ser8ers are <,st 4sheet5
2le)ile assignation o-
ser8er =-> storage
'er-ormance an*
a8ailailit. ecome
in*epen*ent -rom the
9A:
Seite #
Storage :et;ork
dedicated storage network
Primary Storage
Server
Ser8ers access the
ph.sicall. i*entical
storage
to e ,se* -or:
!A 1l,ster
!igh 'er-ormance
1l,ster
Storage 1l,ster
Single s.stem
con-ig,ration
64share* root57
Seite ?
Storage 0irt,alization
9a.er to separate the host- -rom the storage-la.er
The ph.sical storage architect,re ecomes 4in8isile5 to
the hosts
Transparenc. o- the *ata@s ph.sical location
D.namic in-rastr,ct,re
Transparent 4on the -l.5 recon-ig,ration
Transparent mo8ement an* reorganization o- *ata 6!SM7
Man. *i--erent le8els o- storage 8irt,alization possile
So-t;are- an* har*;are components are part o- the
8irt,alization
Seite A
Storage 0irt,alization
The -iles.stem le8el is part o- the storage 8irt,alization
I- a 4cl,ster -ile s.stem5 is mentione* on the ,pcoming
sli*es G2S 6Be*!at@s Gloal 2ile S.stem7 is the one
eing ,se*
All no*es share the management o- the cl,ster -s
:o *e*icate* 2ileser8er 6S'o27 nee*e*
2S consistenc. *one ,sing a special cl,ster ;i*e
locking ser8ice
Cest ,se* in SA: ase* in-rastr,ct,res
Stan*ar* applications might nee* *esign re8ie; to
;ork properl. on a share* s.stem like this
Diskless 6p,re SA: ase*7 share* root operation
possile
Seite D
!A 1l,ster Acti8e/'assi8e
SV1
active
SV2
(passive)
1oncept:
Heartbeat
The passi8e no*e ;aits -or the acti8e one to -ail
+ne or more ser8ices are r,nning on the acti8e no*e
A so-t;are component is responsile -or monitoring the
ser8ices@ an* ser8ers@ health
Di--erent implementations possile
A single host -iles.stem has to e trans-erre* -rom
one no*e to the other in case o- -ailo8er
Seite E
!A 1l,ster Acti8e/Acti8e
SV1
active/(passive)
SV2
(passive)/active
1oncept:
Heartbeat
Di--erent ser8ices r,n on each o- the no*es
Fach no*e is acti8e an* passi8e at the same time
In case o- a -ailo8er per-ormance ;ill e lo;er on the
4*o,le acti8e5 no*e
In case o- single host -iles.stems each ser8ice nee*s its
o;n -iles.stem 6or m,ltiple ser8ices ha8e to -ailo8er
togetherG e.g. :2S / Sama7
Seite 1"
!A 1l,ster :H1
SV1
active
SV2
active
1oncept:
Heartbeat
More ser8ers than necessar. are ,se* 6:H1G :H%G ...7
1l,ster can act ;itho,t per-ormance reak*o;n in case o- a
-ailo8er
2ailo8er so-t;ares normall. ha8e a limite* n,mer o- s,pporte*
no*es
2iles.stem6s7 ha8e to e trans-erre* -rom no*e to no*e in case
o- a -ailo8er
Same r,les an* limitations appl. as in the stan*ar* acti8e/acti8e
/ acti8e/passi8e scenario
SV3
active
SV4
(passive)
Seite 11
Stateless an* State-,l Ser8ices
'rolem o- transparent -ailo8ers
Theoreticall. a ser8ice has to e restarte* on the -ailo8er
no*e in the same state as it 4le-t5 the -aile* no*e
Stateless ser8ice *o not rel. on memor. ase* states
I> Transparent -ailo8er is no 6not a h,ge7 prolem
Stateful ser8ices ha8e sa8e* states in the local no*es@
memor.. The. ha8e to e sa8e* an* trans-erre* to the other
no*e to pre8ent *ata loss.
I> Tranparent -ailo8er is a 6h,ge7 prolem
I> An open connection cannot e re-estalishe*
Data loss occ,rs
Seite 1%
2iles.stem Iss,es
SV1
active
SV2
(passive)
'rolem: F)cl,si8e or parallel *ata access
A !A cl,ster m,st grant access to the same *ata to t;o or
more no*es ;itho,t corr,pting the *ata
'ossiilities:

Data replication et;een t;o or more 8ol,mes

Stan*ar* single host -iles.stem is mo,nte* 4ping-pong5

1l,ster-iles.stem is permanentl. mo,nte* on all hosts

Heartbeat
Seite 13
2iles.stem Iss,es
SV1
active
SV2
(passive)
Stan*ar* Single !ost 2iles.stem 4ping-pong5 mo,nt

Share* Storage 6SA:G Share* S1SI C,s7 necessar.

SA: ase* sol,tion e)pensi8eG Share* S1SI-C,s limite* to t;o hosts

Fach host can act as acti8e host -or one application 6I one -iles.stem7
an* act as passi8e host -or an other application 6I other -iles.stem7

M,ltiple -iles.stems per host possile

2iles.stems m,st e trans-erre* in case o- a -ailo8er

9ocal -ile access can interr,pt a -iles.stem -ailo8er

Heartbeat
Seite 1$
2iles.stem Iss,es
SV1
active
SV2
(active)
1l,ster-iles.stem is permanentl. mo,nte* on all hosts

Share* Storage 6SA:7 necessar.

SA: ase* sol,tion is e)pensi8eG n,mer o- cl,ster no*es 4not5

limite*

Fach no*e can act as acti8e host -or a application ;ith a share*
-iles.stem among all no*es

Depen*ing on the application@s *esign 4conc,rrent5 ;rite access is

possile to the ph.sical same -ile

Most applications s,pport conc,rrent rea* access to the ph.sical

same -ile

Acti8e 9oa*alancing is possile ,sing e8er. cl,ster no*e

Heartbeat
Seite 1#
2iles.stem Iss,es
SV1
active
SV2
(active
1l,ster-iles.stem is mo,nte* permanentl. on all no*es

All hosts share the management o- the -ile s.stem among each other

:o *e*icate* -ileser8er 6eg.: :2S7 necessar.

:o Single 'oint o- 2ail,re 6S'o27 i- a re*,n*ant in-rastr,ct,r an*

re*,n*ant lock ser8ers e)ist

2iles.stem access *irectl. in the SA:G no *eto,r 8ia I' ase*

connections
Heartbeat
Seite 1?
1l,ster-iles.stem Iss,es

All hosts see the ph.sicall. i*entical *ata an* ha8e -,ll ;rite an*
rea* access to them

Access restrictions are *one only in the application la.er

Fach cl,ster no*e can take o8er the role o- an. -ello; no*e

Fach no*e can r,n an. ser8ice 6:2SG SamaG MailG WeG ...7

:,mer o- cl,ster no*es 4not5 limite*

0er. -ast -ailo8er scenarios possile 6no remo,ntsG no ser8ice stops

an* startsG ...7
Seite 1A
!A on share* storage
F)ample: Sama -ileser8ices
Di--erent !A mechanisms nee* *i--erent con-ig,rations

!' Ser8iceG,ar*
+ne sm* ser8ice r,ns per no*eG all shares are ser8e* ,sing this
ser8ice

+ne smb.conf per no*e ;ith one [global] section

All share con-ig,rations are *one ;ithin this -ile

+ne nm* ser8ice per no*eG r,nning -ailo8er ser8ice

in*epen*ent

+ne ;inin** ser8ice per no*eG r,nning -ailo8er ser8ice

in*epen*ent

A Sama -ailo8er <,st a**s the 4ne;5 share con-ig,rations to the

e)isting smb.conf an* reloa*s the ser8ice
Seite 1D
!A on share* storage
F)ample: Sama -ileser8ices
Di--erent !A mechanisms nee* *i--erent con-ig,rations

Be*!at 1l,ster Ser8ice

M,ltiple sm* ser8ices r,n per no*eG shares can e assigne* to the
appropriate one. Sama ser8ices are in*epen*ent -rom each other.

+ne smb.conf per ser8ice ;ith one [global] section

All share con-ig,rations -or this ser8ice are *one ;ithin this -ile

+ne nm* ser8ice per sm* con-ig,rationG r,nning -ailo8er ser8ice

*epen*ent

+ne ;inin** ser8ice per sm* con-ig,rationG r,nning -ailo8er

ser8ice *epen*ent

A Sama -ailo8er 4mo8es5 a complete sama con-ig,ration 6sm*G

nm*G ;inin**G sm.con--2ileJsK7 to the 4ne;5 no*e
Seite 1E
!A on share* storage
F)ample: Sama -ileser8ices
Di--erent !A mechanisms nee* *i--erent con-ig,rations

!'@s Ser8iceG,ar*

+nl. one -ailo8er ser8ice necessar. an* possile -or Sama

0ario,s script patches necessar. to get Sama r,nning

Winin** an* nm* ser8ices are -ailo8er in*epen*ent

Be*!at 1l,ster Ser8ice

More -le)ile con-ig,rations possile

In*epen*ent Sama ser8ices per no*e possile

Fas. a**ing an* remo8ing o- shares to a ser8ice

Script co*e patch necessar. -or ;inin** an* nm*

Winin** an* nm* are -ailo8er *epen*ent

Seite %"
Sama an* share* storage
F)ample: Sama -ileser8ices
Jshare1KGJshare%K Jshare3K Jshare$K Jshare#KGJshare?K
1E%.1?D.1".#"/#1 1E%.1?D.1".?" 1E%.1?D.1".A" 1E%.1?D.1".D"/D1
ser8er1 ser8er% ser8er3 ser8er$
1ommon 2iles.stem: /mnt/g-s/sama/ on SA: Storage
Seite %1
Sama an* share* storage
F)ample: Sama -ileser8ices
Str,ct,r:
Share 1: /mnt/g-s/sama/shares/share1/
Share %: /mnt/g-s/sama/shares/share%/
Share 3: /mnt/g-s/sama/shares/share3/
Share $: /mnt/g-s/sama/shares/share$/
Share #: /mnt/g-s/sama/shares/share#/
Internal: /mnt/g-s/sama/internal/
4Internal5 keeps necessar. parts o- the sama con-ig,ration ;hich ha8e to
e shareale among all cl,ster no*es 6e.g. sm.con- 8ersions7.
Seite %%
1l,ster-iles.stem Iss,es
2ragmentation o- the 4Internal5-Director.:
ls -la /mnt/g-s/sama/internal/ (selective)
lr;)r;)r;) root root /gloal -> (hostname
*r;)r-)r-) root root /ser8er1
*r;)r-)r-) root root /ser8er%
*r;)r-)r-) root root /ser8er3
*r;)r-)r-) root root /ser8er$
*r;)r-)r-) root root /incl,*e
Server1 accesses /mnt/gfs/samba/intern/global/ logicall.Gthe ph.sical location is
/mnt/gfs/samba/intern/server1/.
Server2 an* all other ser8ers ;o,l* ph.sicall. access the *irectories server2,
server3, ... This is calle* Context e!en"ant #at$ %ame (1D':7
'arts o- the sama con-ig,ration ;hich ha8e to e ,niL,e -or the *i--erent no*es
can e place* here ;itho,t the nee* to a*apt path names ;hene8er 6e.g. -ailo8er7
a con-ig,ration has to e change*.
F)ample: Sama -ileser8ices
Seite %3
Sama an* share* storage

The s,*irector. internal contains share in-ormation -or all shares

e.g.: s$are1.conf
Jshare1K
path I /mnt/g-s/sama/shares/share1/
comment I Share1
rea* onl. I no
...
e.g.: s$are2.conf
Jshare%K
path I /mnt/g-s/sama/shares/share%/
comment I Share%
rea* onl. I .es
...
F)ample: Sama -ileser8ices
Seite %$
Sama an* share* storage

The s,*irector. server1 contains the con-ig -ile smb.conf -or server1
e.g.: smb.conf 6accessile -or server1 8ia /mnt/gfs/samba/global/7
JgloalK
...
...
...
incl,*e I /mnt/g-s/sama/internal/incl,*e/share1.con-
incl,*e I /mnt/g-s/sama/internal/incl,*e/share%.con-
F)ample: Sama -ileser8ices
Seite %#
Sama an* share* storage

The s,*irector. server2 contains the con-ig -ile smb.conf -or server2
e.g.: smb.conf 6accessile -or server2 8ia /mnt/gfs/samba/global/7
JgloalK
netios name I Mh
encr.pt pass;or*s I .es
sec,rit. I a*s
...
incl,*e I /mnt/g-s/sama/internal/incl,*e/share3.con-
incl,*e I /mnt/g-s/sama/internal/incl,*e/share%.con-

s$are2.conf is no; ser8e* ,sing t;o sama ser8ers in parallel. This is onl.
possile on rea* onl. shares or i- the same ph.sical -iles are not accesse* in
parallel -or ;rite access. +ther;ise *ata corr,ption &ill happen. :o reliale
locking is possile et;een t;o in*epen*ent sama ser8ices i- the accessing
application has no appropriate locking mechanism.
F)ample: Sama -ileser8ices
Seite %?
Sama an* share* storage
F)ample: Sama -ileser8ices
9oa*alancing:

:o loa* alancing possile in Acti8e/'assi8e con-ig,rations

In an Acti8e/Acti8e en8ironment loa* alancing can e *one

man,all. . assigning the shares to *i--erent no*es

Acti8e/Acti8e con-ig,rations can e limite* to ser8e *i--erent -ile

locations per acti8e ser8ice i- single host -ile s.stems are ,se*

Acti8e/Acti8e con-ig,rations can e ,se* to conc,rrentl. access the

same -iles.stem. 'reca,tions ha8e to e taken to a8oi* conc,rrent
;rite access to the 8er. same -ile at the same time. 9oa* alancing
is possile in gi8ing m,ltiple Sama ser8ers rea* access to a share
an* ,se an other one -or ;riting. 9oa* alancing ;ithin a client
connection is not possile.
Seite %A
+ther s,itale applications

Similar sol,tions are possile ;ith ser8ices like :2S as ;ellG :2S
is it more trick. to implement *,e to *i--erent -ile han*ling

:2S- an* Sama mi)e* mo*e en8ironments are possile

+ther possile ser8ices -or -ailo8er ,se:

Apache

Sen*mail/'ost-i)

1o,rier IMA'

M.SN9

...
Seite %D
+,tlook
Share* Boot 1l,ster

F)amples mentione* e-ore ass,me* a local +S on each host

In a share* root cl,ster e8er. no*e can e oote* *irectl. -rom

the SA: storageG local har**isks are not necessar.
6e)cept -rom s;ap space7.

F8er. no*e is i*entical to its -ello; hosts

6re*,ction to p,re 1'O po;er7

1hanges/Op*ates/... can e ma*e at one central point an* are

a8ailale thro,gho,t the ;hole cl,ster imme*iatel.

2or more in-ormation see iX 11/%""$ 6page 11D7 4A,s *em 'ool
schPp-en5 or online:

http://;;;.re*hat.com/magazine/""?apr"#/-eat,res/g-s/
6english 8ersion7

http://;;;.re*hat.*e/p*-/"$1"iXMagazinBe*Q!at-Ati)-Artikel.p*-
6german 8ersion7
Seite %E
Thank .o,R
An. N,estionsS
Ati) Gm!
Finsteinstr. 1"
DA#1? OnterschleiTheim
;;;.ati).*e
in-o(ati).*e