PelicanHPCTutorial

July2008
MichaelCreel
UniversitatAutnomadeBarcelona
Youcancheckformorerecentversionsofthisdocumentathttp://pareto.uab.es/mcreel/PelicanHPC/Tutorial/
PelicanTutorial.html

Contents
1.
2.
3.
4.
5.

Introduction
Initialsetup
Examplesoftware
Savingyourwork
Usingthemake_pelicanscript

Introduction
PelicanHPCisarapid(around5minutes,whenyouknowwhatyou'redoing)meansofsettingupahigh
performancecomputing(HPC)clusterforparallelcomputingusingMPI.Thistutorialgivesabasic
descriptionofwhatPelicanHPCdoes,addresseshowtousethereleasedCDimagestosetupaHPCcluster,
andgivessomebasicexamplesofusage.

DescriptionofPelicanHPC
PelicanHPCisadistributionofGNU/Linuxthatrunsasa"liveCD"(orasavirtualizationappliance).Ifthe
ISOimagefileisburnttoaCD,theresultingCDcanbeusedtobootacomputer.Thecomputeronwhich
PelicanHPCisbootedisreferredtoasthe"frontendnode",whichisthecomputerthattheuserinteractswith.
OncePelicanHPCisrunning,ascript"pelican_setup"mayberun.Thisscriptconfiguresthefrontend
nodeasanetbootserver.Afterthishasbeendone,othercomputerscanbootcopiesofPelicanHPCoverthe
network.Theseothercomputersarereferredtoas"computenodes".PelicanHPCconfigurestheclustermade
upofthefrontendnodeandthecomputenodessothatMPIbasedparallelcomputingmaybedone.
A"liveCD"suchasPelicanHPCdoesnotusetheharddiskofanyofthenodes,soitwillnotdestroyoralter
yourinstalledoperatingsystem.WhenthePelicanHPCclusterisshutdown,allofthecomputersareintheir
originalstate,andwillbootbackintowhateveroperatingsystemisinstalled.
Features

ThefrontendnodecanbearealcomputerbootedusingaCD,oravirtualmachinethatisbooted
usingtheCDimagefile.Withthissecondoption,PelicanHPCcanbeusedatthesametimeasthe
normalworkenvironment,whichmaybeanyofthecommonoperatingsystems.

Thecomputenodesarenormallyrealcomputers,buttheycanalsobevirtual.
SupportsMPIbasedparallelcomputingusingFortran(77,90),C,C++,andGNUOctave(using
MPITB).
OfferstheOpenMPIandLAM/MPIimplementationsofMPI.
Clustercanberesizedtoaddorremovenodesusingthe"pelican_restarthpc"command.
Easilyextensibletoaddpackages.Alsoeasilymodifiable,sincethePelicanHPCCDimageiscreated
usingasinglescriptthatusestheDebianLivesystemforcreatingaliveCDimage.Forthisreason,
thedistributedversionisbasicandlightweight.
Versionsexistfor32bitCPUs(Pentium4,Core,Sempron)andfor64bitCPUs(Opteron,Turion,
Core2,etc.)
Containsexamplesoftware:LinpackHPLbenchmarkandextensiveexamplesthatuseMPITBfor
GNUOctave.

Limitationsandrequirements

Thecomputenodesmustbebootedoverthenetwork.Thisisanoptionofferedbyallmodern
networkingdevicessuppliedwithmotherboards,butitoftenmustbeenabledintheBIOSsetup.
Enableit,andgiveithigherprioritythatbootingfromharddiskorothersources.Ifyouhavea
networkcardthatwon'tdonetboot,itispossibletoworkaroundthisusingromomatic.
APelicanHPCclusteris"volatile".Itdisappearswhenthemachinesareturnedoff.Itispossible(see
below)touseharddiskspaceorotherstoragetoretainwork,butthisrequiressomeknowledgeofthe
GNU/Linuxoperatingsystem.Ifusingvirtualization,isispossibletotakesnapshotsofthenodes,
whichallowsworktoberecoveredquickly.PelicanHPCisnotmeanttobeinstalledastheprimary
operatingsystemofacomputer.
APelicanHPCclusterisdesignedtobeusedbyasingleperson.

LicensingandDisclaimer

PelicanHPCisaCDimagemadebyrunningascript(seebelow).ThescriptislicensedGPLv3.The
resultingCDimagecontainssoftwarefromtheDebiandistributionofGNU/Linux,whichissubjecttothe
licenseschosenbytheauthorsofthatsoftware.
ThisreleasedPelicanHPCCDimagesaredistributedinthehopethattheywillbeuseful,butWITHOUT
ANYWARRANTY;withouteventheimpliedwarrantyofMERCHANTABILITYorFITNESSFORA
PARTICULARPURPOSE.

Initialsetup
Thefrontendandallcomputenodesmustbenetworkedtogether.IMPORTANT:thefrontendnodewillact
asaDHCPserver,sobesuretoisolatethenetworkusedfortheclusterfromothernetworks,toavoid
conflictswithotherDHCPservers.IfyoustarthandingoutIPaddressestoyourcoworkers'computers,they
maybecomeannoyed.Ifthefrontendnodehasmultiplenetworkinterfaces,youcanuseonetoconnectto
theclusterandanothertoconnecttotheInternet.
PuttheCDinthecomputerthatwillbethefrontend,andturniton.MakesuretheBIOSsetupletsyouboot
fromCD.Whenyoubootup,you'llseesomethinglikethefollowing.Eitherexploretheoptions,orpress
<Enter>tobootup.Forexample,IcangetaSpanishkeyboardbytyping"livekeyb=es"and<Enter>.

Optionscanbecombined.

TherearesomeexamplesforGNUOctaveusingMPITB,andtheLinpackHPLbenchmarkisalsoavailable.
Ifyou'reinterestedinthese,chooseyes.Ifyoujustwantaclusterandyouwillsupplytheapplications,
chooseno.

OK,thefrontendisallsetup:

Asissaysinthelastshot,youneedtotype"pelican_setup"tostartdhcp,nfs,etc.,sothatthecomputenodes
maybebooted.Ifyoutypethis,youwillseethefollowing,supposingthatyouhavemorethan1network
device.(Note,youcanenterthegraphicalenvironmentbytyping"startx"andthensettheclusterupfrom
there,ifyouprefer).

Afteryouchoosethenetdevice,servicesarestarted.Whenyouseethefollowingscreen,youcangoturnon

thecomputenodes.Choose"yes".

Here'sashotofavirtualclustersetup,thefrontendnodeisrunninginonetab,andthecomputenodeisready
tobeturnedonintheothertab.Ifthefrontendnodeisvirtualandyouwouldliketobootrealcomputenodes,
besuretospecifythatthevirtualnetworkdevicethatconnecttotheclusteris"bridged".Tohaveinternet
accessonthevirtualfrontendnode,addasecondnetworkdevicewith"NAT"networking,toshareyourreal
Internetconnection.

Whenacomputenodestartstonetboot,you'llseethiswhizby:

Whenacomputenodeisdonebooting,you'llseethis,supposingthatithasamonitor:

Here'sashotofthevirtualcluster,withthefrontendnodeandacomputenode.Thecomputenodehasnotyet
reporteditselftobeavailable(countiszero).

Here'salargershotofthesamethingyouseeinthelastshot.Nowthecountis1,whichmeansthatthe
computenodehasbooted.Keepchoosing"no"untilallofyourcomputenodesareaccountedfor.Then
choose"yes".Don'tworry,youcanaddnodesinthefutureifyoulike.

Onceyouclickyes,you'llseesomethinglikethefollowing,dependingonhowmanynodesyouhave.Note
howItype"lamnodes"tocheckthatitreallyworked.

OK,that'sit,theclusterisreadytouse.Someothertips:

changethepasswordonallnodes.Reallydothisifyourmachinesareconnectedtoanyexternal
network.IrevokeyourlicensetousePelicanHPCifyou'renotsmartenoughtodothis.Thedefault
passwordofthereleasedISOimagesis"live".Serioususerswillprobablywanttomaketheirown
versionofPelicanHPCwithaprivatepassword,usingthemake_pelicanscript.
youcanaddsoftwaretothefrontendnodeusing"aptgetinstallwhatever",supposingthatthe
frontendhasasecondnetcardthatyouhaveconfiguredtoenableInternetaccess.
thedefaultMPIsetupisinthefile/home/user/tmp/bhosts.Thisassignsrankstohostsinaroundrobin
fashion.Ifyourhostshavedifferentspeeds,numbersofcores,etc.,youshouldmodifythisfileandre
lamboot(ifyouuseLAM).
ksysguardisavailable,andasmallamountofeffortwillturnitintoaniceclustermonitor.Seethis
postforgeneralinformationonhowtodoit.
ifyouneedotherpackages,wouldliketosetyourowndefaultpassword,useastoragedeviceto
mountas/home,etc.,thenyoucanmakeyourownversionprettyeasilyusingthemake_pelicanscript
thatisavailableonthePelicanHPChomepage.Hopefully,I'llwritesomedocumentationforthatin
thenearfuture.Butyoucanprobablyfigureitoutifyoureallyneedit:)
Youcanresizethecluster(addorremovecomputenodes)wheneveryoulike,byrunning
"pelican_restarthpc".

Returntocontents

Examplesoftware
PelicanHPChastheLinpackHPLbenchmark,andsomeextensiveexamplesfromthefieldofeconometrics
thatuseMPITBforGNUOctave.Econometricsisafieldofstudythatappliesstatisticalmethodsto
economicmodels.ThesoftwareisintheEconometricsdirectory:

Thereisadocument"econometrics.pdf"thathasalotofinformation,includingsomeaboutparallel
computing:

Openaterminal,type"octave"andthen"kernel_example"(pleasenotethatunderscorebackthere):

etviola!somenicepictures:

Otherthingstotryare"pea_example","bfgsmin_example","mle_example","gmm_example",
"mc_example1","mc_example2"andafewothersI'mforgettingabout.Tofindwherethecodeis,type"help
mc_example1",forexample,whileinOctave.Thengoedittherelevantfiletolearnmoreaboutwhatitdoes.
Returntocontents

Savingyourwork
Bydefault,PelicanHPCimagesput/home/useronaramdiskwhichdisappearswhenyoushutdown.You
needtosaveyourworkbetweensessions,ifyouwanttoreuseit.Therearemanyoptions,suchasmounting
aharddisk,usingaUSBdevice,etc.IfyouhaveanInternetconnectionconfigured,youcanemailitto
yourself,asisillustratedinthenextshot:

IfyouusePelicanHPCforseriouswork,Ihighlyrecommendmountingastoragedevicetouseas/home,so
thatyourworkwillbesavedbetweensessionswithouttakinganyspecialsteps.Anexampleofthe
commandsyoucouldusetodothiswouldbeasfollows.
sudo -s
mkdir /junk
mount /dev/YOURDEV /junk
cp -a /home/user /junk
mount --bind /junk /home
exit
exit
ThevariationsandpossibilitiesherearesonumerousthatIdon'twanttoattempttoexplainthisfurther.Don't
trytodothisuntilyouknowwhatyou'redoing.
Returntocontents

Usingthemake_pelicanscript
ThedistributedISOimagesprovideabarebonesclustersetupsystem,plussomepackagesthatIuseinmy
researchandteaching.Thereareafewexamplestakenfrommywork,whichmaybeofinteresttothose
learningthebasicsofMPI,ortopeopleinterestedineconometrics.However,manyuserswillfindthat
Pelicandoesnotcontainpackagesthattheyneed,soameansofcustomizingtheCDimageisrequired.
PelicanHPCismadebyrunningasinglescript"make_pelican",whichisavailableonthedownloadpage.If
youhavetheprerequisitesforrunningthescript,itisveryeasytomakeacustomizedversionofPelican.The
prerequisitesare:

aDebianorDebianlike(Ubuntu,etc.)installationofGNU/Linux.Thiscanbeaminimalinstallation
inachrootjailifyouprefertorunsomethingelseforyournormalwork.Youcouldevenuseavirtual
machineunderWindows,ifyouareaWindowsuser.
thelivehelperpackageshouldbeobtainedfromthissourceandinstalledonyourDebianorDebian
likesystemusing"dpkgilivehelper<tab>"
examinethemake_pelicanscript,whichcontainssomeselfexplanatorycomments.Addthepackages
youneedtothepackagelistsection.Notethatyoucanchooseadifferentpasswordhere.
youneedtorunthemake_pelicanscriptastherootuser(maybenot,butIhaven'tchecked).Afast
internetconnectionishelpful,sincealotofpackagesneedtobedownloaded.Also,ithelpstobuild
theimageonafast,hopefullymulticorecomputer.Partsofthebuildprocessareparallelizedandwill
takeadvantageofmultiplecores.Buildtimeforthedefaultconfigurationonadecentdualcorelaptop
withlotofRAMisabouthalfanhour.
whenyouaredone,therewillbeafile"binary.iso"inthe../<architecture>/frontenddirectory,
where../isthelocationofthemake_pelicanscript,and<architecture>iseitheri386oramd64,
dependingonwhichyouleftuncommentedinthescript.
ThereisamanualforDebianLive.Pleasehavealookatitbeforetryingtousemake_pelican.
Additionalinformationisathttp://debianlive.alioth.debian.org/.Thisinformationisthemain
documentation,sincemake_pelicanisjustascriptthatprovidesaspecificconfigurationtotheDebian
LivesystemofbuildingaliveCDimage.Alsorememberthat"manlivehelper","manlh_config"and
"manlh_build"willgiveyouinformation.