Unix Linux Introduction

TheUnixandGNU/Linuxcommandline
TheUnixand GNU/Linux commandline

MichaelOpdenacker ThomasPetazzoni FreeElectrons
Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Jul15,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/commandline Corrections,suggestions,contributionsandtranslationsarewelcome!
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Commandmementosheet
Itisausefulcompaniontothis presentation. Examplesforthemostuseful commandsaregiveninjustone sheet. Suggestionsforuse Stickthissheetonyourwall,useit asdesktopwallpaper,makeita mousemat,printitonclothing, sliceitintobookmarks... Caution Storeawayfrommice!
Getiton http://freeelectrons.com/docs/commandline
2
TrainingContents(1)
Shells,filesystemandfilehandling Everythingisafile GNU/Linuxfilesystemstructure Commandlineinterpreters Handlingfilesanddirectories Displaying,scanningandsortingfiles Symbolicandhardlink Fileaccessrights
3
Trainingcontents(2)
StandardI/O,redirections,pipes Standardinputandoutput,redirectingtofiles Pipes:redirectingstandardoutputtoothercommands Standarderror
4
TrainingContents(3)
Taskcontrol Fullcontrolontasks Executinginbackground,suspending,resumingandaborting Listofactivetasks Killingprocesses Environmentvariables PATHenvironmentvariables Shellaliases,.bashrcfile
5
Trainingcontents(4)
Miscellaneous
Texteditors Compressingandarchiving Printingfiles Comparingfilesanddirectories Lookingforfiles Gettinginformationaboutusers
6
Unixfilesystem
7
Everythingisafile
AlmosteverythinginUnixisafile! Regularfiles Directories Directoriesarejustfiles listingasetoffiles Symboliclinks Filesreferringtothename ofanotherfile Devicesandperipherals Readandwritefromdevices aswithregularfiles Pipes Usedtocascadeprograms cat*.log|greperror Sockets Interprocesscommunication
8
Filenames
FilenamefeaturessincethebeginningofUnix Casesensitive Noobviouslengthlimit Cancontainanycharacter(includingwhitespace,except/). Filetypesstoredinthefile(magicnumbers). Filenameextensionsnotneededandnotinterpreted.Just usedforuserconvenience. Filenameexamples: README .bashrc index.htm index.html WindowsBuglist index.html.old
9
Filepaths
Apathisasequenceofnesteddirectorieswithafileor directoryattheend,separatedbythe/character Relativepath:documents/fun/microsoft_jokes.html Relativetothecurrentdirectory Absolutepath: /home/bill/bugs/crash9402031614568 /:rootdirectory. Startofabsolutepathsforallfilesonthesystem(evenfor filesonremovabledevicesornetworkshared).
10
GNU/Linuxfilesystemstructure(1)
Notimposedbythesystem.Canvaryfromonesystemtothe other,evenbetweentwoGNU/Linuxinstallations! / /bin/ /boot/ /dev/ /etc/ /home/ /lib/ Rootdirectory Basic,essentialsystemcommands Kernelimages,initrdandconfigurationfiles Filesrepresentingdevices /dev/hda:firstIDEharddisk Systemconfigurationfiles Userdirectories Basicsystemsharedlibraries
11
/lost+found /media /mnt/ filesystems /opt/ /proc/ /root/ /sbin/ /sys/ Corruptfilesthesystemtriedtorecover Mountpointsforremovablemedia: /media/usbdisk,/media/cdrom Mountpointsfortemporarilymounted Specifictoolsinstalledbythesysadmin /usr/local/oftenusedinstead Accesstosysteminformation /proc/cpuinfo,/proc/version... rootuserhomedirectory Administratoronlycommands Systemanddevicecontrols (cpufrequency,devicepower,etc.)
12
/tmp/ /usr/ /usr/local/ /var/ Temporaryfiles Regularusertools(notessentialtothesystem) /usr/bin/,/usr/lib/,/usr/sbin... Specificsoftwareinstalledbythesysadmin (oftenpreferredto/opt/) Datausedbythesystemorsystemservers /var/log/,/var/spool/mail(incoming mail),/var/spool/lpd(printjobs)...
TheUnixfilesystemstructureisdefined bytheFilesystemHierarchyStandard(FHS): http://www.pathname.com/fhs/
13
Shellsandfilehandling
14
Commandlineinterpreters
Shells:toolstoexecuteusercommands Calledshellsbecausetheyhidethedetailsonthe underlyingoperatingsystemundertheshell'ssurface. Commandsareinputinatextterminal,eitherawindowina graphicalenvironmentoratextonlyconsole. Resultsarealsodisplayedontheterminal.Nographicsare neededatall. Shellscanbescripted:providealltheresourcestowrite complexprograms(variable,conditionals,iterations...)
15
Wellknownshells
Mostfamousandpopularshells sh:TheBourneshell(obsolete) Traditional,basicshellfoundonUnixsystems,bySteveBourne. csh:TheCshell(obsolete) OncepopularshellwithaClikesyntax tcsh:TheTCshell(stillverypopular) ACshellcompatibleimplementationwithevolvedfeatures (commandcompletion,historyeditingandmore...) bash:TheBourneAgainshell(mostpopular) Animprovedimplementationofshwithlotsofaddedfeatures too.
16
fish:agreatnewshell
TheFriendlyInteractiveSHell http://www.fishshell.org/ Standardfeatures:history,commandandfilecompletion... Brandnewfeatures:commandoptioncompletion,command completionwithshortdescription,syntaxhighlighting.. Easiertoanyopenfiles:openbuiltincommand. Muchsimplerandconsistentsyntax(notPOSIXcompliant) Makesiteasiertocreateshellscripts. Commandlinebeginnerscanlearnmuchfaster! Evenexperiencedusersshouldfindthisshellveryconvenient.
17
lscommand
Liststhefilesinthecurrentdirectory,inalphanumericorder, exceptfilesstartingwiththe.character. lsa(all) Listsallthefiles(including.* files) lsl(long) Longlisting(type,date,size, owner,permissions) lst(time) Liststhemostrecentfilesfirst lsS(size) Liststhebiggestfilesfirst lsr(reverse) Reversesthesortorder lsltr(optionscanbe combined) Longlisting,mostrecentfiles attheend
18
Filenamepatternsubstitutions
Betterintroducedbyexamples! ls*txt Theshellfirstreplaces*txtbyallthefileanddirectory namesendingbytxt(including.txt),exceptthose startingwith.,andthenexecutesthelscommandline. lsd.* Listsallthefilesanddirectoriesstartingwith. dtellslsnottodisplaythecontentsofdirectories. cat?.log Displaysallthefileswhichnamesstartby1characterand endby.log
19
Specialdirectories(1)
./ Thecurrentdirectory.Usefulforcommandstakingadirectory argument.Alsosometimesusefultoruncommandsinthe currentdirectory(seelater). So./readme.txtandreadme.txtareequivalent. ../ Theparent(enclosing)directory.Alwaysbelongstothe. directory(seelsa).Onlyreferencetotheparentdirectory. Typicalusage: cd..
20
Specialdirectories(2)
~/ Notaspecialdirectoryindeed.Shellsjustsubstituteitbythe homedirectoryofthecurrentuser. Cannotbeusedinmostprograms,asitisnotareal directory. ~sydney/ Similarly,substitutedbyshellsbythehomedirectoryofthe sydneyuser.
21
Thecdandpwdcommands
cd<dir> Changesthecurrentdirectoryto<dir>. cd Getsbacktothepreviouscurrentdirectory. pwd Displaysthecurrentdirectory("workingdirectory").
22
Thecpcommand
cp<source_file><target_file> Copiesthesourcefiletothetarget. cpfile1file2file3...dir Copiesthefilestothetargetdirectory(lastargument). cpi(interactive) Asksforuserconfirmationifthetargetfilealreadyexists cpr<source_dir><target_dir>(recursive) Copiesthewholedirectory.
23
mvandrmcommands
mv<old_name><new_name> (move) Renamesthegivenfileordirectory. mvi(interactive) Ifthenewfilealreadyexits,asksforuserconfirm rmfile1file2file3... (remove) Removesthegivenfiles. rmi(interactive) Alwaysaskforuserconfirm. rmrdir1dir2dir3(recursive) Removesthegivendirectorieswithalltheircontents.
24
Creatingandremovingdirectories
mkdirdir1dir2dir3...(makedir) Createsdirectorieswiththegivennames. rmdirdir1dir2dir3...(removedir) Removesthegivendirectories Safe:onlyworkswhendirectoriesandempty. Alternative:rmr(doesn'tneedemptydirectories).
25
Displayingfilecontents
Severalwaysofdisplayingthecontentsoffiles. catfile1file2file3...(concatenate) Concatenatesandoutputsthecontentsofthegivenfiles. morefile1file2file3... Aftereachpage,askstheusertohitakeytocontinue. Canalsojumptothefirstoccurrenceofakeyword (/command). lessfile1file2file3... Doesmorethanmorewithless. Doesn'treadthewholefilebeforestarting. Supportsbackwardmovementinthefile(?command).
26
Theheadandtailcommands
head[<n>]<file> Displaysthefirst<n>lines(or10bydefault)ofthegivenfile. Doesn'thavetoopenthewholefiletodothis! tail[<n>]<file> Displaysthelast<n>lines(or10bydefault)ofthegivenfile. NoneedtoloadthewholefileinRAM!Veryusefulforhugefiles. tailf<file>(follow) Displaysthelast10linesofthegivenfileandcontinuestodisplaynew lineswhentheyareappendedtothefile. Veryusefultofollowthechangesinalogfile,forexample. Examples headwindows_bugs.txt tailfoutlook_vulnerabilities.txt
27
Thegrepcommand
grep<pattern><files> Scansthegivenfilesanddisplaysthelineswhichmatchthegiven pattern. greperror*.log Displaysallthelinescontainingerrorinthe*.logfiles grepierror*.log Same,butcaseinsensitive greprierror. Same,butrecursivelyinallthefilesin.anditssubdirectories grepvinfo*.log Outputsallthelinesinthefilesexceptthosecontaininginfo.
28
Thesortcommand
sort<file> Sortsthelinesinthegivenfileincharacterorderand outputsthem. sortr<file> Same,butinreverseorder. sortru<file> u:unique.Same,butjustoutputsidenticallinesonce. Morepossibilitiesdescribedlater!
29
Thesedcommand
sedisaStreamEDitor Itparsestextfilesandimplementsaprogramminglanguageto applytransformationsonthetext. Oneofthemostcommonusageofsedistextreplacement,which reliesonregularexpressions
sede's/abc/def/'testfilewillreplaceeverystringabc bydefinthefiletestfileanddisplaytheresultonthestandard output. sed's/^[\t]*//'testfilewillremoveanytabulationor spaceatthebeginningofaline
sed's/^|$[^|]*$|$[^|]*$|$/\1>\2/'testfile
replacelineslike |string1|string2| by string1>string2

30
sed:regularexpressions
RegularexpressionsareusefulinmanyUnixtools,notonlysed. Theyallowtomatchtheinputtextagainstanexpression.
.matchesanycharacter []matchesanycharacterlistedinsidethebrackets [^]matchesanycharacternotlistedinsidethebrackets ^matchesthebeginningoftheline $matchestheendoftheline *matchesthepreviouselementzeroormoretimes,+matchesthe previouselementoneormoretimes,?matchestheprevious elementzerooronetime definesasubexpressionthatcanbelaterrecalledbyusing \n,wherenisthenumberofthesubexpressionintheregular expression Moreathttp://www.regularexpressions.info/
31
Symboliclinks
Asymboliclinkisaspecialfilewhichisjustareferencetothe nameofanotherone(fileordirectory): Usefultoreducediskusageandcomplexitywhen2fileshave thesamecontent. Example:
anakin_skywalker_biography>darth_vador_biography
Howtoidentifysymboliclinks: lsldisplays>andthelinkedfilename. GNUlsdisplayslinkswithadifferentcolor.
32
Creatingsymboliclinks
Tocreateasymboliclink(sameorderasincp): lnsfile_namelink_name Tocreatealinkwithtoafileinanotherdirectory,withthe samename: lns../README.txt Tocreatemultiplelinksatonceinagivendirectory: lnsfile1file2file3...dir Toremovealink: rmlink_name Ofcourse,thisdoesn'tremovethelinkedfile!
33
Hardlinks
Thedefaultbehaviorforlnistocreatehardlinks Ahardlinktoafileisaregularfilewithexactlythesame physicalcontents Whiletheystillsavespace,hardlinkscan'tbedistinguished fromtheoriginalfiles. Ifyouremovetheoriginalfile,thereisnoimpactonthehard linkcontents. Thecontentsareremovedwhentherearenomorefiles (hardlinks)tothem.
34
Filesnamesandinodes
Makeshardandsymbolic(soft)linkseasiertounderstand!
Users Filenameinterface
Softlink Inode
rm
File Inode
Hardlink
rm
Inode interface Filesystem
35
Commanddocumentation
36
Commandhelp
SomeUnixcommandsandmostGNU/Linuxcommands offeratleastonehelpargument: h (ismostlyusedtointroduce1characteroptions) help (isalwaysusedtointroducethecorresponding longoptionname,whichmakesscriptseasierto understand) Youalsooftengetashortsummaryofoptionswhenyou inputaninvalidargument.
37
Manualpages
man<keyword> Displaysoneorseveralmanualpagesfor<keyword> manman MostavailablemanualpagesareaboutUnixcommands,but somearealsoaboutCfunctions,headersordatastructures, orevenaboutsystemconfigurationfiles! manstdio.h manfstab(for/etc/fstab) Manualpagefilesarelookedforinthedirectoriesspecifiedby theMANPATHenvironmentvariable.
38
Infopages
InGNU,manpagesarebeingreplacedbyinfopages. Somemanualpageseventelltorefertoinfopages instead. info<command>
infofeatures:
Documentationstructuredinsections(nodes)andsubsections (subnodes) Possibilitytonavigateinthisstructure:top,next,prev,up InfopagesgeneratedfromthesametexinfosourceastheHTML documentationpages
39
SearchingtheInternetforresources(2)
Lookingfordocumentation Lookfor<tool>or<tool>pagetofindthetoolorproject homepageandthenfindthelatestdocumentation resources. Lookfor<tool>documentationor<tool>manualin yourfavoritesearchengine. Lookingforgenerictechnicalinformation WikiPedia:http://wikipedia.org Lotsofusefuldefinitionsincomputerscience.Areal encyclopedia!Opentoanyone'scontributions.
40
SearchingtheInternetforresources(1)
Investigatingissues Mostforumsandmailinglistarchivesarepublic,andare indexedonaveryfrequentbasisbyGoogle. Ifyouinvestigateanerrormessage,copyitverbatiminthe searchform,enclosedindoublequotes(errormessage). Lotsofchancesthatsomebodyelsealreadyfacedthe sameissue. Don'tforgettouseGoogleGroups: http://groups.google.com/ Thissiteindexesmorethan20yearsofnewsgroups messages.
41
Usersandpermissions
42
Fileaccessrights
Uselsltocheckfileaccessrights
3typesofaccessrights Readaccess(r) Writeaccess(w) Executerights(x) 3typesofaccesslevels User(u):fortheownerofthe file Group(g):eachfilealsohas agroupattribute, correspondingtoagivenlist ofusers Others(o):forallotherusers
43
Accessrightconstraints
xissufficienttoexecutebinaries Bothxandrandrequiredforshellscripts. Bothrandxpermissionsneededinpracticefordirectories: rtolistthecontents,xtoaccessthecontents. Youcan'trename,remove,copyfilesinadirectoryifyou don'thavewaccesstothisdirectory. Ifyouhavewaccesstoadirectory,youCANremoveafile evenifyoudon'thavewriteaccesstothisfile(remember thatadirectoryisjustafiledescribingalistoffiles).This evenletsyoumodify(remove+recreate)afileevenwithout waccesstoit.
44
Accessrightsexamples
rwrr Readableandwritableforfileowner,onlyreadableforothers rwr Readableandwritableforfileowner,onlyreadableforusers belongingtothefilegroup. drwx Directoryonlyaccessiblebyitsowner rx Fileexecutablebyothersbutneitherbyyourfriendsnorby yourself.Niceprotectionsforatrap...
45
chmod:changingpermissions
chmod<permissions><files> 2formatsforpermissions: Octalformat(abc): a,b,c=r*4+w*2+x(r,w,x:booleans) Example:chmod644<file> (rwforu,rforgando) Orsymbolicformat.Easytounderstandbyexamples: chmodgo+r:addreadpermissionstogroupand others. chmoduw:removewritepermissionsfromuser. chmodax:(a:all)removeexecutepermissionfromall.
46
Morechmod(1)
chmodRa+rXlinux/ Makeslinuxandeverythinginitavailableto everyone! R:applychangesrecursively X:x,butonlyfordirectoriesandfilesalready executable Veryusefultoopenrecursiveaccesstodirectories, withoutaddingexecutionrightstoallfiles.
47
Morechmod(2)
chmoda+t/tmp t:(sticky).Specialpermissionfordirectories,allowing onlythedirectoryandfileownertodeleteafileina directory. Usefulfordirectorieswithwriteaccesstoanyone, like/tmp. Displayedbylslwithatcharacter.
48
Fileownership
Particularlyusefulin(embedded)systemdevelopment whenyoucreatefilesforanothersystem. chownRsco/home/linux/src(R:recursive) Makesuserscothenewownerofallthefilesin /home/linux/src. chgrpRempire/home/askywalker Makesempirethenewgroupofeverythingin /home/askywalker. chownRborg:aliensusss_entreprise/ chowncanbeusedtochangetheownerandgroupatthesame time.
49
Bewareofthedarksideofroot
rootuserprivilegesareonlyneededforveryspecific taskswithsecurityrisks:mounting,creatingdevice files,loadingdrivers,startingnetworking, changingfileownership,packageupgrades... Evenifyouhavetherootpassword,yourregular accountshouldbesufficientfor99.9%ofyourtasks (unlessyouareasystemadministrator). Inatrainingsession,itisacceptabletouseroot. Inreallife,youmaynotevenhaveaccesstothis account,orputyoursystemsanddataatriskifyou do.
50
Usingtherootaccount
Incaseyoureallywanttouseroot... Ifyouhavetherootpassword: su(switchuser) Inmoderndistributions,thesudocommandgivesyouaccesstosome rootprivilegeswithyourownuserpassword. Example:sudomount/dev/hda4/home
51
StandardI/O,redirections,pipes
52
Standardoutput
Moreaboutcommandoutput Allthecommandsoutputtingtextonyourterminaldoitbywriting totheirstandardoutput. Standardoutputcanbewritten(redirected)toafileusingthe> symbol Standardoutputcanbeappendedtoanexistingfileusingthe>> symbol
53
Standardoutputredirectionexamples
ls~saddam/*>~gwb/weapons_mass_destruction.txt catobiwan_kenobi.txt>starwars_biographies.txt cathan_solo.txt>>starwars_biographies.txt echoREADME:Nosuchfileordirectory>README Usefulwayofcreatingafilewithoutatexteditor. NiceUnixjoketoointhiscase.
54
Standardinput
Moreaboutcommandinput Lotsofcommands,whennotgiveninputarguments,can taketheirinputfromstandardinput. sort windows linux [Ctrl][D] linux windows sorttakesitsinputfrom thestandardinput:inthiscase, whatyoutypeintheterminal (endedby[Ctrl][D])
sort<participants.txt Thestandardinputofsortistakenfromthegivenfile.
55
Pipes
Unixpipesareveryusefultoredirectthestandardoutputofa commandtothestandardinputofanotherone. Examples
cat*.log|grepierror|sort greprierror.|grepvignored|sortu\ >serious_errors.log cat/home/*/homework.txt|grepmark|more
ThisoneofthemostpowerfulfeaturesinUnixshells!
56
Theteecommand
tee[a]file Theteecommandcanbeusedtosendstandard outputtothescreenandtoafilesimultaneously. make|teebuild.log Runsthemakecommandandstoresitsoutputto build.log. makeinstall|teeabuild.log Runsthemakeinstallcommandandappendsits outputtobuild.log.
57
Standarderror
Errormessagesareusuallyoutput(iftheprogramiswell written)tostandarderrorinsteadofstandardoutput. Standarderrorcanberedirectedthrough2>or2>> Example: catf1f2nofile>newfile2>errfile Note:1isthedescriptorforstandardoutput,so1>isequivalentto >. Canredirectbothstandardoutputandstandarderrortothe samefileusing&>: catf1f2nofile&>wholefile
58
Theyescommand
Usefultofillstandardinputwithalwaysthesamestring. yes<string>|<command> Keepsfillingthestandardinputof<command>with <string>(ybydefault). Examples yes|rmrdir/ bank>yesno|credit_applicant yes""|makeoldconfig (equivalenttohitting[Enter]toacceptalldefaultsettings)
59
Specialdevices(1)
Devicefileswithaspecialbehaviororcontents /dev/null Thedatasink!Discardsalldatawrittentothisfile. Usefultogetridofunwantedoutput,typicallylog information: mplayerblack_adder_4th.avi&>/dev/null /dev/zero Readsfromthisfilealwaysreturn\0characters Usefultocreateafilefilledwithzeros: ddif=/dev/zeroof=disk.imgbs=1kcount=2048 Seemannullormanzerofordetails
60
Specialdevices(2)
/dev/random Returnsrandombyteswhenread.Mainlyusedby cryptographicprograms.Usesinterruptsfromsomedevice driversassourcesoftruerandomness(entropy). Readscanbeblockeduntilenoughentropyisgathered. /dev/urandom Forprogramsforwhichpseudorandomnumbersarefine. Alwaysgeneratesrandombytes,evenifnotenoughentropy isavailable(inwhichcaseitispossible,thoughstilldifficult, topredictfuturebytesequencesfrompastones). Seemanrandomfordetails.
61
Specialdevices(3)
/dev/full Mimicsafulldevice. Usefultocheckthatyourapplicationproperlyhandles thiskindofsituation. Seemanfullfordetails.
62
Taskcontrol
63
Fullcontrolontasks
Sincethebeginning,Unixsupportstruepreemptivemultitasking. Abilitytorunmanytasksinparallel,andabortthemevenifthey corrupttheirownstateanddata. Abilitytochoosewhichprogramsyourun. Abilitytochoosewhichinputyourprogramstakes,andwhere theiroutputgoes.
64
Processes
EverythinginUnixisafile EverythinginUnixthatisnotafileisaprocess Processes Instancesofarunningprograms Severalinstancesofthesameprogramcanrunatthesametime Dataassociatedtoprocesses: Openfiles,allocatedmemory,stack,processid,parent,priority, state...
65
Runningjobsinbackground
Sameusagethroughoutalltheshells Useful
Forcommandlinejobswhichoutputcanbeexaminedlater, especiallyfortimeconsumingones. Tostartgraphicalapplicationsfromthecommandlineandthen continuewiththemouse.
Startingatask:add&attheendofyourline: find_prince_charmingcutecleverrich&
66
Backgroundjobcontrol
jobs Returnsthelistofbackgroundjobsfromthesameshell
[1]Running~/bin/find_meaning_of_lifewithoutgod& [2]+Runningmakemistakes&
fg fg%<n> Putsthelast/nthbackgroundjobinforegroundmode Movingthecurrenttaskinbackgroundmode: [Ctrl]Z bg kill%<n> Abortsthenthjob.
67
Jobcontrolexample
>jobs [1]Running~/bin/find_meaning_of_lifewithoutgod& [2]+Runningmakemistakes& >fg makemistakes >[Ctrl]Z [2]+Stoppedmakemistakes >bg [2]+makemistakes& >kill%1 [1]+Terminated~/bin/find_meaning_of_lifewithoutgod
68
Listingallprocesses
...whatevershell,scriptorprocesstheyarestartedfrom psux Listsalltheprocessesbelongingtothecurrentuser psaux(Note:psedfonSystemVsystems) Listsalltheprocessesrunningonthesystem
psaux|grepbart|grepbash USERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMAND bart30390.00.259161380pts/2S14:350:00/bin/bash bart31340.00.253881380pts/3S14:360:00/bin/bash bart31900.00.263681360pts/4S14:370:00/bin/bash bart34160.00.000pts/2RW15:070:00[bash] PID: VSZ: RSS: TTY: STAT: Processid Virtualprocesssize(code+data+stack) Processresidentsize:numberofKBcurrentlyinRAM Terminal Status:R(Runnable),S(Sleep),W(paging),Z(Zombie)...
69
Liveprocessactivity
topDisplaysmostimportantprocesses,sortedbycpu percentage
top15:44:33up1:11,5users,loadaverage:0.98,0.61,0.59 Tasks:81total,5running,76sleeping,0stopped,0zombie Cpu(s):92.7%us,5.3%sy,0.0%ni,0.0%id,1.7%wa,0.3%hi,0.0%si Mem:515344ktotal,512384kused,2960kfree,20464kbuffers Swap:1044184ktotal,0kused,1044184kfree,277660kcached PIDUSERPRNIVIRTRESSHRS%CPU%MEMTIME+COMMAND 3809jdoe250625639321312R93.80.80:21.49bunzip2 2769root160157m80m90mR2.716.05:21.01X 3006jdoe1503092815m27mS0.33.00:22.40kdeinit 3008jdoe16056248924468S0.30.20:06.59autorun 3034jdoe1502676412m24mS0.32.50:12.68kscd 3810jdoe16028929161620R0.30.20:00.06top
Youcanchangethesortingorderbytyping M:Memoryusage,P:%CPU,T:Time. Youcankillataskbytypingkandtheprocessid.

70
Killingprocesses(1)
kill<pids> Sendsanabortsignaltothegivenprocesses.Lets processessavedataandexitbythemselves.Shouldbeused first.Example: kill3039313431903416 kill9<pids> Sendsanimmediateterminationsignal.Thesystemitself terminatestheprocesses.Usefulwhenaprocessisreally stuck(doesn'tanswertokill1). kill91 Killsalltheprocessesofthecurrentuser.1:meansall processes.
71
Killingprocesses(2)
killall[<signal>]<command> Killsallthejobsrunning<command>.Example: killallbash xkill Letsyoukillagraphicalapplicationbyclickingonit! Veryquick!Convenientwhenyoudon'tknowtheapplication commandname.
72
Recoveringfromstuckgraphics
Ifyourgraphicalsessionisstuckandyoucannolongertypeinyour terminals,don'treboot! Itisverylikelythatyoursystemisstillfine.Trytoaccessatext consolebypressingthe[Ctrl][Alt][F1]keys (or[F2],[F3]formoretextconsoles) Inthetextconsole,youcantrytokilltheguiltyapplication. Oncethisisdone,youcangobacktothegraphicsessionby pressing[Ctrl][Alt][F5]or[Ctrl][Alt][F7](dependingon yourdistribution) Ifyoucan'tidentifythestuckprogram,youcanalsokillallyour processes:kill91 Youarethenbroughtbacktotheloginscreen.
73
Sequentialcommands
Cantypethenextcommandinyourterminalevenwhen thecurrentoneisnotover. Canseparatecommandswiththe;symbol: echoIlovethee;sleep10;echonot Conditionals:use||(or)or&&(and): moreGod||echoSorry,Goddoesn'texist Runsechoonlyifthefirstcommandfails ls~sd6&&cat~sd6/*>~sydney/recipes.txt Onlycatsthedirectorycontentsifthelscommandsucceeds (meansreadaccess).
74
Quoting(1)
Double(")quotescanbeusedtopreventtheshellfrom interpretingspacesasargumentseparators,aswellasto preventfilenamepatternexpansion. >echo"HelloWorld" HelloWorld >echo"Youareloggedas$USER" Youareloggedasbgates >echo*.log find_prince_charming.logcosmetic_buys.log >echo"*.log" *.log
75
Quoting(2)
Singlequotesbringasimilarfunctionality,butwhatisbetween quotesisneversubstituted >echo'Youareloggedas$USER' Youareloggedas$USER Backquotes(`)canbeusedtocallacommandwithinanother >cd/lib/modules/ùnamer`;pwd /lib/modules/2.6.91.6_FC2 Backquotescanbeusedwithindoublequotes >echo"YouareusingLinuxùnamer`" YouareusingLinux2.6.91.6_FC2
76
Measuringelapsedtime
timefind_expensive_housingnear <...commandoutput...> real0m2.304s(actualelapsedtime) user0m0.449s(CPUtimerunningprogramcode) sys0m0.106s(CPUtimerunningsystemcalls) real=user+sys+waiting waiting=I/Owaitingtime+idletime(runningothertasks)
77
Environmentvariables
Shellslettheuserdefinevariables. Theycanbereusedinshellcommands. Convention:lowercasenames Youcanalsodefineenvironmentvariables: variablesthatarealsovisiblewithinscriptsor executablescalledfromtheshell. Convention:uppercasenames. env Listsalldefinedenvironmentvariablesandtheir value.
78
Shellvariablesexamples
Shellvariables(bash) projdir=/home/marshall/coolstuff lsla$projdir;cd$projdir Environmentvariables(bash) cd$HOME exportDEBUG=1 ./find_extraterrestrial_life (displaysdebuginformationifDEBUGisset)
79
Mainstandardenvironmentvariables
Usedbylotsofapplications!
LD_LIBRARY_PATH MANPATH
Manualpagesearchpath
PATH
Sharedlibrarysearchpath
DISPLAY
Commandsearchpath
PRINTER
ScreenidtodisplayX (graphical)applicationson.
EDITOR
Defaultprintername
SHELL
Defaulteditor(vi,emacs...)
HOME
Currentshellname
TERM
Currentuserhome directory
HOSTNAME
Currentterminaltype
USER
Nameofthelocalmachine
Currentusername
80
PATHenvironmentvariables
PATH Specifiestheshellsearchorderforcommands / home/acox/bin:/usr/local/bin:/usr/kerberos/bin: /usr/bin:/bin:/usr/X11R6/bin:/bin:/usr/bin LD_LIBRARY_PATH Specifiesthesharedlibrary(binarycodelibrariessharedby applications,liketheClibrary)searchorderforld /usr/local/lib:/usr/lib:/lib:/usr/X11R6/lib MANPATH Specifiesthesearchorderformanualpages /usr/local/man:/usr/share/man
81
PATHusagewarning
Itisstronglyrecommendednottohavethe.directoryinyour PATHenvironmentvariable,inparticularnotatthebeginning: Acrackercouldplaceamaliciouslsfileinyourdirectories.It wouldgetexecutedwhenyourunlsinthisdirectoryandcould donaughtythingstoyourdata. Ifyouhaveanexecutablefilecalledtestinadirectory,thiswill overridethedefaulttestprogramandsomescriptswillstop workingproperly. Eachtimeyoucdtoanewdirectory,theshellwillwastetime updatingitslistofavailablecommands. Callyourlocalcommandsasfollows:./test
82
Alias
Shellsletyoudefinecommandaliases:shortcutsforcommandsyou useveryfrequently. Examples
aliasls='lsla' Usefultoalwaysruncommandswithdefaultarguments. aliasrm='rmi' Usefultomakermalwaysaskforconfirmation. aliasfrd='find_rambaldi_deviceasaprisky' Usefultoreplaceverylongandfrequentcommands. aliascia='./home/sydney/env/cia.sh' Usefultosetanenvironmentinaquickway (.isashellcommandtoexecutethecontentofashellscript).
83
Thewhichcommand
Beforeyourunacommand,whichtellsyouwhereitisfound bash>whichls aliasls='lscolor=tty' /bin/ls tcsh>whichls ls:aliasedtolscolor=tty bash>whichalias /usr/bin/which:noaliasin (/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin) tcsh>whichalias alias:shellbuiltincommand.
84
~/.bashrcfile
~/.bashrc
Shellscriptreadeachtimeabashshellisstarted Youcanusethisfiletodefine Yourdefaultenvironmentvariables(PATH,EDITOR...). Youraliases. Yourprompt(seethebashmanualfordetails). Agreetingmessage.
85
Commandediting
Youcanusetheleftandrightarrowkeystomovethecursorin thecurrentcommand. Youcanuse[Ctrl][a]togotothebeginningoftheline,and [Ctrl][e]togototheend. Youcanusetheupanddownarrowstoselectearliercommands. Youcanuse[Ctrl][r]tosearchinsidethehistoryofprevious commands.
86
Commandhistory(1)
history Displaysthelatestcommandsthatyouranandtheir number.Youcancopyandpastecommandstrings. Youcanrecallthelatestcommand: !! Youcanrecallacommandbyitsnumber !1003 Youcanrecallthelatestcommandmatchingastarting string: !cat
87
Commandhistory(2)
Youcanmakesubstitutionsonthelatestcommand: ^more^less Youcanrunanothercommandwiththesamearguments: more!*
88
Miscellaneous
Texteditors
89
Texteditors
Graphicaltexteditors Fineformostneeds nedit Emacs,Xemacs Kate,Gedit Textonlytexteditors Oftenneededforsysadminsandgreatforpowerusers vi,vim nano
90
Thenedittexteditor
http://www.nedit.org/ Besttexteditorfornonvioremacsexperts Featurehighlights:
Veryeasytextselectionandmoving Syntaxhighlightingformostlanguagesandformats.Canbe tailoredforyourownlogfiles,tohighlightparticularerrorsand warnings. Easytocustomizethroughmenus
Notinstalledbydefaultbyalldistributions
91
neditscreenshot
92
Emacs/Xemacs
EmacsandXemacsarepretty similar(uptoyourpreference) Extremelypowerfultexteditor features Greatforpowerusers Lessergonomicthannedit Nonstandardshortcuts Muchmorethanatexteditor (games,email,shell,browser). Somepowercommandshavetobe learnt.
93
Kateandgedit
Kateisapowerfultexteditor dedicatedtoprogramming activities,forKDE
http://kate.kde.org
Geditisatexteditorforthe Gnomeenvironment
http://projects.gnome.org/gedit/
94
vi
TextmodetexteditoravailableinallUnixsystems.Created beforecomputerswithmiceappeared. Difficulttolearnforbeginnersusedtographicaltexteditors. Veryproductiveforpowerusers. Oftencan'tbereplacedtoeditfilesinsystemadministration orinEmbeddedSystems,whenyoujusthaveatext console.
95
vimviimproved
viimplementationnowfoundinmostGNU/ Linuxhostsystems Implementslotsoffeaturesavailableinmodern editors:syntaxhighlighting,commandhistory, help,unlimitedundoandmuchmuchmore. Coolfeatureexample:candirectlyopen compressedtextfiles. ComeswithaGTKgraphicalinterface(gvim) Unfortunately,notfreesoftware(becauseofa smallrestrictioninfreedomtomakechanges)
96
vibasiccommands
Thoughviisextremely powerful,itsmain30 commandsareeasytolearn andaresufficientfor99%of everyone'sneeds! Youcanalsotakethequick tutorialbyrunning vimtutor. Getourvimementosheetifyoudidn'tgetitwiththiscourse: http://freeelectrons.com/docs/commandline
97
GNUnano
http://www.nanoeditor.org/ Anothersmalltextonly,mousefreetexteditor. AnenhancedPicoclone(nonfreeeditorinPine) Friendlyandeasiertolearnforbeginnersthankstoon screencommandsummaries. Availableinbinarypackagesforseveralplatforms. Analternativetoviinembeddedsystems. However,notavailableasabusyboxbuiltin.
98
GNUnanoscreenshot
99
Miscellaneous
Compressingandarchiving
100
Measuringdiskusage
Caution:differentfromfilesize!
duh<file>(diskusage) h:returnssizeondiskofthegivenfile,inhuman readableformat:K(kilobytes),M(megabytes)orG (gigabytes),.Withouth,dureturnstherawnumberof diskblocksusedbythefile(hardtoread). NotethatthehoptiononlyexistsinGNUdu. dush<dir> s:returnsthesumofdiskusageofallthefilesinthe givendirectory.
101
Measuringdiskspace
dfh<dir> Returnsdiskusageandfreespaceforthefilesystem containingthegivendirectory. Similarly,thehoptiononlyexistsinGNUdf. Example:
>dfh. FilesystemSizeUsedAvailUse%Mountedon /dev/hda59.2G7.1G1.8G81%/
dfh Returnsdiskspaceinformationforallfilesystems availableinthesystem.Whenerrorshappen,usefulto lookforfullfilesystems.

102
Compressinganddecompressing
Veryusefulforshrinkinghugefilesandsavingspace g[un]zip<file> GNUzipcompressionutility.Creates.gzfiles. Ordinaryperformance(similartoZip). b[un]zip2<file> Morerecentandeffectivecompressionutility. Creates.bz2files.Usually2025%betterthangzip. [un]lzma<file> Muchbettercompressionratiothanbzip2(upto10to20%). Compatiblecommandlineoptions.
103
Archiving(1)
Usefultobackuporreleaseasetoffileswithin1file tar:originallytapearchive Creatinganarchive: tarcvf<archive><filesordirectories> c:create v:verbose.Usefultofollowarchivingprogress. f:file.Archivecreatedinfile(tapeusedotherwise). Example: tarcvf/backup/home.tar/home bzip2/backup/home.tar
104
Archiving(2)
Viewingthecontentsofanarchiveorintegritycheck: tartvf<archive> t:test Extractingallthefilesfromanarchive: tarxvf<archive> Extractingjustafewfilesfromanarchive: tarxvf<archive><filesordirectories> Filesordirectoriesaregivenwithpathsrelativetothe archiverootdirectory.
105
ExtraoptionsinGNUtar
tar=gtar=GNUtaronGNU/Linux Cancompressanduncompressarchivesonthefly. Usefultoavoidcreatinghugeintermediatefiles Muchsimplertodothanwithtarandbzip2! joption:[un]compressesontheflywithbzip2 zoption:[un]compressesontheflywithgzip lzmaoption:[un]compressesontheflywithlzma Examples(whichonewillyouremember?)
gtarjcvfbills_bugs.tar.bz2bills_bugs tarcvfbills_bugs|bzip2>bills_bugs.tar.bz2
106
Checkingfileintegrity
Verylowcostsolutiontocheckfileintegrity md5sumFC3i386disk*.iso>MD5SUM ComputesaMD5(MessageDigestAlgorithm5)128bitchecksumofthe givenfiles.Usuallyredirectedtoafile. Exampleoutput: db8c7254beeb4f6b891d1ed3f689b412FC3i386disc1.iso 2c11674cf429fe570445afd9d5ff564eFC3i386disc2.iso f88f6ab5947ca41f3cf31db04487279bFC3i386disc3.iso 6331c00aa3e8c088cc365eeb7ef230eaFC3i386disc4.iso md5sumcMD5SUM CheckstheintegrityofthefilesinMD5SUMbycomparingtheiractualMD5 checksumwiththeiroriginalone.
107
Miscellaneous
Printing
108
Unixprinting
Multiuser,multijob,multiclient,multiprinter InUnix/Linux,printingcommandsdon'treallyprint.Theysend jobstoprintingqueues,possiblyonthelocalmachine,on networkprintingserversoronnetworkprinters. Printerindependentsystem: PrintserversonlyacceptjobsinPostScript ortext.Printerdriversontheservertake careoftheconversiontoeachprintersownformat. Robustsystem: Rebootasystem,itwillcontinuetoprintpendingjobs.
109
Printingcommands
Usefulenvironmentvariable:PRINTER Setsthedefaultprinteronthesystem.Example: exportPRINTER=lp lpr[P<queue>]<files> Sendsthegivenfilestothespecifiedprintingqueue ThefilesmustbeintextorPostScriptformat.Otherwise, youonlyprintgarbage. a2ps[P<queue>]<files> AnytoPostScriptconvertsmanyformatstoPostScriptand sendtheoutputtothespecifiedqueue.Usefulfeatures: severalpages/sheet,pagenumbering,infoframe...
110
Printjobcontrol
lpq[P<queue>] Listsalltheprintjobsinthegivenordefaultqueue.
lpisnotready RankOwnerJobFile(s)TotalSize 1stasloane84nsa_windows_backdoors.ps60416bytes 2ndamoore85gw_bush_iraq_mistakes.ps65024000bytes
cancel<job#>[<queue>] Removesthegivenjobnumberfromthedefaultqueue.
111
UsingPostScriptandPDFfiles
ViewingaPostScriptfile PostScriptviewersexist,buttheirqualityisprettypoor. BetterconverttoPDFwithps2pdf: ps2pdfdecss_algorithm.ps xpdfdecss_algorithm.pdf& PrintingaPDFfile Youdon'tneedtoopenaPDFreader! BetterconverttoPostScriptwithpdf2ps: pdf2psrambaldi_artifacts_for_dummies.pdf lprrambaldi_artifacts_for_dummies.ps
112
Miscellaneous
Synchronizingfiles
113
Smartdirectorycopywithrsync
rsync(remotesync)hasbeendesignedtokeepinsync directorieson2machineswithalowbandwidthconnection.
Onlycopiesfilesthathavechanged.Fileswiththesamesizeare comparedbychecksums. Onlytransferstheblocksthatdifferwithinafile! Cancompressthetransferredblocks Preservessymboliclinksandfilepermissions:alsoveryusefulfor copiesonthesamemachine. Canworkthroughssh(secureremoteshell).Veryusefultoupdate thecontentsofawebsite,forexample.
114
rsyncexamples(1)
rsynca/home/arvin/sd6_agents//home/sydney/misc/
a:archivemode.EquivalenttorlptgoD...easywaytotellyou wantrecursionandwanttopreservealmosteverything.
rsyncPavdelete/home/steve/ideas//home/bill/my_ideas/
P:partial(keeppartiallytransferredfiles)andprogress (showprogressduringtransfer) delete:deletefilesinthetargetwhichdon'texistinthesource. Caution:directorynamesshouldendwith/.Otherwise,yougeta my_ideas/ideas/directoryatthedestination.
115
rsyncexamples(2)
Copyingtoaremotemachine rsyncPav/home/bill/legal/arguments/\ bill@www.sco.com:/home/legal/arguments/ Userbillwillbepromptedforapassword. Copyingfromaremotemachinethroughssh rsyncPavessh homer@tank.duff.com:/prod/beer/\ fridge/homer/beer/ Userhomerwillbepromptedforhissshkeypassword.
116
Miscellaneous
Comparingfilesanddirectories
117
Comparingfilesanddirectories
difffile1file2 Reportsthedifferencesbetween2files,ornothingifthe filesareidentical. diffrdir1/dir2/ Reportsallthedifferencesbetweenfileswiththesame nameinthe2directories. Thesedifferencescanbesavedinafileusingthe redirection,andthenlaterreappliedusingthepatch command. Toinvestigatedifferencesindetail,betterusegraphical tools!
118
tkdiff
http://tkdiff.sourceforge.net/ Usefultooltocomparefilesandmergedifferences
119
kompare
Anothernicetooltocomparefilesandmergedifferences Partofthekdesdkpackage(FedoraCore)
120
gvimdiff
Anothernicetooltoviewdifferencesinfiles Availableinmost distributionswithgvim Apparentlynotusing diff. Noissuewithfileswith binarysections!
121
Miscellaneous
Lookingforfiles
122
Thefindcommand
Betterexplainedbyafewexamples! find.name*.pdf Listsallthe*.pdffilesinthecurrent(.)directoryor subdirectories.Youneedthedoublequotestoprevent theshellfromexpandingthe*character. finddocsname"*.pdf"execxpdf{}';' Findsallthe*.pdffilesinthedocsdirectoryand displaysoneaftertheother. Manymorepossibilitiesavailable!However,theabove2 examplescovermostneeds.
123
Thelocatecommand
Muchfasterregularexpressionsearchalternativetofind locatekeys Listsallthefilesonyoursystemwithkeysintheirname. locate*.pdf Listsallthe*.pdffilesavailableonthewholemachine locate/home/fridge/*beer* Listsallthe*beer*filesinthegivendirectory(absolutepath) locateismuchfasterbecauseitindexesallfilesina dedicateddatabase,whichisupdatedonaregularbasis. findisbettertosearchthroughrecentlycreatedfiles.
124
Miscellaneous
Variouscommands
125
Gettinginformationaboutusers
who Listsalltheusersloggedonthesystem. whoami TellswhatuserIamloggedas. groups TellswhichgroupsIbelongto. groups<user> Tellswhichgroups<user>belongsto. finger<user> Tellsmoredetails(realname,etc)about<user> Disabledinsomesystems(securityreasons).
126
Changingusers
Youdonothavetologouttologonanotheruseraccount! suhyde (Rare)Changetothehydeaccount,butkeepingtheenvironment variablesettingsoftheoriginaluser. sujekyll (Morefrequent)Logonthejekyllaccount,withexactlythe samesettingsasthisnewuser. su Whennoargumentisgiven,itmeanstherootuser.
127
Thewgetcommand
Insteadofdownloadingfilesfromyourbrowser,justcopyand pastetheirURLanddownloadthemwithwget! wgetmainfeatures httpandftpsupport Canresumeinterrupteddownloads Candownloadentiresitesoratleastcheckforbadlinks Veryusefulinscriptsorwhennographicsareavailable (systemadministration,embeddedsystems) Proxysupport(http_proxyandftp_proxyenv.variables)
128
wgetexamples
wgetc\ http://microsoft.com/customers/dogs/winxp4dogs.zip Continuesaninterrupteddownload. wgetmhttp://lwn.net/ Mirrorsasite. wgetrnphttp://www.xml.com/ldd/chapter/book/ Recursivelydownloadsanonlinebookforofflineaccess. np:"noparent".Onlyfollowslinksinthecurrentdirectory.
129
Misccommands(1)
sleep60 Waitsfor60seconds (doesn'tconsumesystemresources). wcreport.txt(wordcount) 438211518302report.txt Countsthenumberoflines,wordsandcharacters inafileorinstandardinput.
130
Misccommands(2)
bc("basiccalculator?") bcisahandybutfullfeaturedcalculator.Evenincludes aprogramminglanguage!Usetheloptiontohave floatingpointsupport. date Returnsthecurrentdate.Usefulinscriptstorecord whencommandsstartedorcompleted.
131
Checksumcommands
Achecksumorhashsumisafixedsizedatumcomputedfroman arbitraryblockofdigitaldataforthepurposeofdetecting accidentalerrorsthatmayhavebeenintroducedduringits transmissionsorstorage. http://en.wikipedia.org/wiki/Checksum TheMD5hashalgorithmisimplementedinthemd5sum command
$md5sumpatch2.6.24.7.bz2 0c1c5d6d8cd82e18d62406d2f34d1d38patch2.6.24.7.bz2
TheSHAalgorithmisimplementedintheshaXsum(sha1sum, sha256sum,etc.) Theintegrityofseveralfilescanbeverifiedagainstafilelisting thechecksumsusingthecoption.

132
Systemadministration
Seeourpresentationaboutsystemadministrationbasics: Networksetup Creatingandmountingfilesystems Accessingadministrator(root)priviledges Packagemanagement Alsoavailableonhttp://freeelectrons.com/docs/commandline
133
Applicationdevelopment
134
Compilingsimpleapplications
ThecompilerusedforallLinuxsystemsisGCC http://gcc.gnu.org Tocompileasinglefileapplication,developedinC: gccotesttest.c
Willgenerateatestbinary,fromthetest.csourcefile
ForC++: g++otesttest.cc TheWalloptionenablesmorewarnings Tocompilesourcesfilestoobjectfilesandlinktheapplication: gccctest1.c gccctest2.c gccotesttest1.otest2.o gccautomaticallycallsthelinkerld

135
Usinglibraries(1)
OnanyLinuxsystem,aClibraryisavailableandoffersalarge setofAPIsforapplicationdevelopment.
Seehttp://www.gnu.org/software/libc/manual/
OutsideoftheClibrary,thousandsofotherlibrariesareavailable forgraphicprogramming,multimedia,networking,scientific computations,andmoroe. Mostlibrariesarealreadyavailableaspackagesinyour distribution,ingeneralintwopackages

libfooisthepackagecontainingthelibraryitself.Thispackageis requiredtoexecutealreadycompiledapplications,butnotsufficient tobuildnewapplications libfoodevisthepackagecontainingtheheadersandother configurationsfilesandtoolsneededtobuildnewapplications relyingonlibfoo.
136
Usinglibraries(2)
Inyoursourcecode,includetheproperheaderfilesofthelibrary
Usually#include<foo.h>or#include<foo/foo.h> Theseheadersarepresentin/usr/include/ Refertothedocumentationofthelibraryfordetails,availablein /usr/share/doc/<package>/,ontheWeb...orintheheader files!
Tocompileyourapplicationwiththelibrary,theeasiestsolutionis tousepkgconfig,whichissupportedbymostlibrariestoday: gccotesttest.c$(pkgconfigcflagslibs) Bydefault,theapplicationaredynamicallylinkedwiththe libraries

Thelibrariesmustbepresentin/lib/fortheapplicationtowork Usethelddcommandtoseewhichlibrariesareneededbyan application
137
MakeandMakefiles
Thecompilationprocesscanbeautomatedusingthemaketool. makereadsafilecalledMakefilefromthecurrentdirectory,and executestherulesdescribedinthisfile Everyruleishasatargetname,acolon,andalistof dependencies,andthelistofcommandstogeneratethetarget fromthedependencies target:dep1dep2 command1 command2 command3
Whensimplyrunningmake,thedefaulttargetthatisgeneratedis all.Atargetisonlyregeneratedifdependencieshavechanged.
Seehttp://www.gnu.org/software/make/manual/
138
139
SimpleMakefileexample
Variablescanbedefined andlaterexpanededwith $(VARNAME) CFLAGS=Wall all:test test:t1.ot2.o gcco$@$^$(CFLAGS) Thedefaulttargetall simplydependsonthe testtarget.
Thetesttargetdepends ont1.oandt2.o.Once thesefilesaregenerated, clean: thegcccommandis $(RM)ftest executed. $@isthetargetname install: $îsthenameofall $(CP)test/usr/bin dependencies. The.ofilesaregenerated Thesetargetsareexecutedby usingimplicit runningmakecleanandmake dependencies,knownby make. install
140
Buildsystems
Makefilesarenice,buttheydon'teasilyalloweasyadaptationto thedifferentbuildenvironmentanddifferentbuildoptions Moreelaboratedbuildsystemshavebeendeveloped
Autotools(automake,autoconf),basedonMakefilesandshell scripts.Eventhoughtheyareoldandalittlebitdifficultto understand,theyarethemostpopularbuildsystemforfree softwarepackages. CMake,anewer,cleanerbuildsystem SconcsandWaf,otherbuildsystemsbasedonPython
Thetypicalstepstocompileaautotoolsbasedpackageare ./configure make sudomakeinstall

141
Debugging
TheofficialdebuggerthatcomeswiththeGNUdevelopment toolsisgdb.
Seehttp://sourceware.org/gdb/download/onlinedocs/gdb.html
Anapplicationmustbecompiledwiththegoptiontobe properlydebugged.Thisoptionaddsdebugginginformationto theapplicationbinary gccotesttest.cg Theapplicationcanthenberuninsidethegdbdebugger: gdbtest Orthedebuggercanbeattachedtotheapplicationwhileitis running: gdbtestpPID

WherePIDistheprocessIDoftherunningapplication
142
Usinggdb
gdbisatextbaseddebugger,withacommandlineinterfacelikea shell,providingdedicatedcommands.Someoftheimportant commandsare:
break(b)tosetabreakpointinthecode.Canbeusedwitha functionnameoralocationinthesourcecode,oranabsolute memoryaddress. print(p)toprintthevalueofavariable.Usedwithavariable name,evenifit'sacomplexone(whichinvolvesdereferencing structures,forexample) ctocontinuetheexecutionuntilthenextbreakpoint. next(n)toexecuteonlythenextlineofcode(stepoverany functioncall)andstep(s)toexecuteonlythenextlineofcode (stepintoanyfunctioncall) backtrace(bt)todisplaythefunctioncallstack
143
gdbsamplesession
thomas@surf:/tmp$gccotesttest.cg
thomas@surf:/tmp$gdbtest GNUgdb6.8debian [...) (gdb)breakfoo Breakpoint1at0x80483c7:filetest.c,line5. (gdb)run Startingprogram:/tmp/test2 Breakpoint1,foo(a=2,b=3)attest.c:5 5 returna+b; (gdb)pa $1=2 (gdb)pb $2=3 (gdb)c Continuing. foo=5 Programexitednormally.
144
ddd,thegraphicalgdb
Thereareseveralgraphicalfront endstogdb.Apopularoneis ddd.
Allowstonavigateinthesource codewhiledebugging Allowstosetbreakpoints, inspectandchangevariable valuedirectlybylookingatthe sourcecode
http://www.gnu.org/software/ddd/
145
Versioncontrolsystem CVS
146
Versioncontrol(1)
Activitythatconsistsinmaintainingthefullhistoryofprojectfiles Everyversionofeveryfileisrecorded Itallowsto:
Createaknowledgedatabaseabouttheprojectcodebase Revertincaseofmistake Reviewthemodificationsintroducedbetweentwodifferentversions Workasateamonasingleproject Createparalleldevelopmentorreleasebranches Tagpreviousversionsofaproject
147
Versioncontrol(2)
Usefulfor:
Sourcecode,ofcourse Documentation,translationstrings,compilationscripts, configurationfiles,etc. Binarydatacanbeversioned,butthefunctionalitieswillbelimited. Besttousetextonlyfileformats.
Generatedfilesshouldneverbeversioned. Versioncontrolisabsolutelynecessarywhenworkingasateam onaproject

Versioncontrolsystemsareoneofthefundamentaltoolsusedfor opensourcedevelopment
Versioncontrolisstillveryusefulwhenworkingaloneona project.
148
Toolsandapproaches
Alotofversioncontrolsystemsavailable
Proprietary:Perforce,Synergy,StarTeam,BitKeeper,ClearCase Freeandopensource:CVS,Subversion,Git,Mercurial,Arch, Monotone
Inthefree/opensolutions,twomainapproaches
Theclassicalone,thecentralizedapproach,implementedbyCVS andSubversion Anewerone,thedecentralizedordistributedapproach, implementedinGit,Mercurial,ArchorMonotone.
149
Centralizedapproach
Thecentralizedapproachisbasedonthepresenceofaserver hostingarepository. Therepositorycontainsthehistoryofallfilesandactsasthe referenceforallparticipantsintheproject. Typicalclient/serverapproach.
Repository Client Client

150
Conflictmanagement(1)
Whentwopersonsareworkingatthesametimeonthesamefile, whenthechangesaresenttotheserver,aconflictcanoccurifthe modificationsarenotcompatible.
A'' A' A
Repository
A' A Linus
A'' A Richard
151
Conflictmanagement(2)
Twosolutionstosolvetheconflictproblem
Thelockingapproach:preventingtwodeveloperstoworkonthe samefile. Themergeapproach:theseconddevelopersendinghischanges totheserverisresponsibleformerginghischangeswiththeother developerschangesalreadypresentontheserver.
Inthefree/opensourceprojects,thetraditionalmodelisthe mergeone.Itavoidstheburdenofhandlingthelockingand scalesbetterwithabiggerteam. ButCVSandSubversionbothimplementadvisorylocking:a developermustvoluntarilylockthefilesheisgoingtoworkwith.
152
Conflictresolutionbyfusion
A* A'' A
A* A' Linus
A''
A* A'' A Richard
153
CVS
FreeSoftware Usedtobethemostwidelyusedfreeversioncontrolsystem. NowreplacedbySubversion. Stillusedbybigprojects,butalotoflargeprojects(KDE, Apache,Eclipse)havemovedtoSubversion. Homepage:http://www.nongnu.org/cvs/ Documentation:http://ximbiot.com/cvs/manual/
154
CVSusage
Themainclientisacommandlineone,implementedinthecvs command
cvshelpgivesthelistofavailablecommands
add,admin,annotate,checkout,commit,diff,edit, editors,export,history,import,init,log,login, logout,ls,pserver,rannotate,rdiff,release, remove,rlog,rls,rtag,server,status,tag, unedit,update,version,watch,watchers
cvshelpcommandgivesthehelpofcommand.
Graphicalinterfaces
IntegrationinVim,Emacs,Anjuta,DevC++,Eclipse,Kdevelop,etc. Graphicalclients:Cervisia(forKDE),gcvs(forGnome),CrossVC, etc. Webclients:ViewVC, cvsweb,etc.
155
Createtheworkingcopy
Thefirststeptoworkonanexistingprojectistocreateaworking copy.Aworkingcopyisacopyonthedevelopermachineofthe fulltreeoftheprojectsourcecode.
Thedeveloperwillworkonthiscopyofthesourcecode,asusual (filesaredirectlyaccessibleandeditable).
cvsdrepositoryaddresscheckoutmodule
Createinthecurrentdirectoryaworkingcopyofthemodulemodule locatedinthegivenrepository
Arepositoryaddresscanbe:
Alocalpath,like:local:/home/user/cvsrepo/ TheaddressofaCVSpserver,like:pserver:user@host :/var/cvsrepo/ UsingSSH,bysettingtheenvironmentvariableCVS_RSHtossh,and usinganaddresslike:ext:user@host:/var/cvsrepo/
156
Simpleusage
Workingcopycreationtobedoneonceforall! Modificationsoftheprojectfiles
Filesaremodifieddirectly,asusual,nothingspecialisnecessary. CVSknowsthecontentsoftheoriginalversionofthefilesandisable tocomputethelistofmodificationsmadebythedeveloper. EverydirectorycontainsaCVSdirectorythatshouldnotbemodified.
Sendthemodificationstotheserver:commit
cvscommit Runsatexteditortowritethemessagethatdescribesthecommit. Thismessagewillbepreservedforeverinthehistory.Veryimportant todescribeyourchangeindetails!
Fetchthemodificationsmadebyotherdevelopers:update
cvsupdate
157
Example
$cvsd:local:/tmp/repo/checkoutproject cvscheckout:Updatingproject Uproject/README Uproject/a.c Uproject/b.c $cdproject/ project$viREADME project$cvscommitm"Addinginfos" cvscommit:Examining. /tmp/repo/project/README,v<README newrevision:1.3;previousrevision:1.2 project$cvsupdate cvsupdate:Updating. Ua.c project$
158
Filemanagement
Whennewfilesordirectoriesarecreatedinsidetheproject,they arenotautomaticallytakenintoaccountbyCVS Theymustbeexplicitlyaddedwiththeaddcommand
cvsaddfile1.c Thenewfileisnotpropagatedtotherepositoryuntilthenext commit Samethingwithdirectories
Filesanddirectoriescanberemovedwiththeremovecommand IssuesfixedinSubversion
Filesanddirectoriescannotberenamedwithoutlosingthehistory DirectoryremovalisbadlyhandledinCVS
159
Example
project$vic.c project$cvsaddc.c cvsadd:schedulingfile`c.c'foraddition cvsadd:use`cvscommit'toaddthisfilepermanently project$cvscommitm"Newfilec.c" cvscommit:Examining. /tmp/repo/project/c.c,v<c.c initialrevision:1.1 project$rma.c project$cvsremovea.c cvsremove:schedulingà.c'forremoval cvsremove:use`cvscommit'toremovethisfilepermanently project$cvscommitm"Removea.c" cvscommit:Examining. /tmp/repo/project/a.c,v<a.c newrevision:delete;previousrevision:1.2
160
Stateoftheworkingcopy
Togetinformationsaboutthestateoftheworking,usethestatus command
Willtelltheversionofthefile,iftheyhavebeenlocallymodified, etc.
Toseethemodificationsmadetotheprojectandnotcommitted yet,usethediffcommand Repository checkout Workingcopy diff

161
update
Diffcommandexample
project$cvsdiff cvsdiff:Diffing. Index:c.c ======================================================== RCSfile:/tmp/repo/project/c.c,v retrievingrevision1.1 diffur1.1c.c c.c 28Jan200916:48:110000 1.1 +++c.c 28Jan200916:50:410000 @@1+1,2@@ Test. +Test2.
162
Usingthehistory
Seethecommitmessages
cvslog cvslogfile.c
Seethedifferencesbetweentwopastrevisions
cvsdiffr1.1r1.2file.c WithCVS,theversioningisdoneonaperfilebasis:acommitis notanentitythatcanbereferredtooncecompleted.Itmakesthe usageofthehistoryverycomplicated.ThisisfixedinSubversion.
Seewhomadeachange
cvsannotate
Agraphicalinterface,eitheragraphicalclientorawebclient,will beveryusefultonavigateandusethehistoryefficiently
163
ConflictmanagementinCVS(1)
Afterchangingafile,wetrytocommitthechangetotheserver
$cvscommitm"Modification" cvscommit:Examining. cvscommit:Uptodatecheckfailedfor`main.c' cvs[commitaborted]:correctaboveerrorsfirst!
Thefilehasbeenmodifiedontheserversinceourlastupdate. Sowemustmakemergethemodificationswehavedonewith themodificationsoftheotherdevelopers,byupdatingour workingcopy.

$cvsupdate cvsupdate:Updating. RCSfile:/tmp/repo/project/main.c,v retrievingrevision1.1 retrievingrevision1.2 Mergingdifferencesbetween1.1and1.2intomain.c rcsmerge:warning:conflictsduringmerge cvsupdate:conflictsfoundinmain.c Cmain.c
164
ConflictmanagementinCVS(2)
Insidemain.c #include<stdio.h> intmain(void){ <<<<<<<main.c printf("Bonjourmonde\n"); ======= printf("Holamundo\n"); >>>>>>>1.2 return0; } Ourmodification!
Themodification madebytheother developer
165
ConflictresolutioninCVS
Theconflictmustbemanuallyresolvedbythedeveloper,no automatedtoolcanresolvesuchconflicts Theconflictisresolveddirectlyinthefilebyremovingthe markersandmergingthemodifications(eitherselectingoneof themorcreatinganewversionbasedonbothmodifications) Onceresolved,thefilecanbecommittedasusual
project$cvscommitm"InFrench" cvscommit:Examining. /tmp/repo/project/main.c,v<main.c newrevision:1.3;previousrevision:1.2
166
Repositoryinitialization
Arepositorymustbeinitializedusingtheinitcommand:
mkdir/home/user/cvsrepo cvsd:local:/home/user/cvsrepoinit
Onceinitialized,itcanbeaccessedlocallyorremotelythrough SSH.Remoteaccessthroughpserverwillrequireadditional configuration,seetheCVSdocumentation. Ifalreadyexistingprojectshavetobeimportedintherepository, usetheimportcommand

cdproject cvsd:local:/home/user/cvsrepoimportmodulename vendortagreleasetag
vendortag,symbolicnameforthebranch releasetag,symbolicnamefortherelease
167
Tagsandbranches
Tagsallowtoidentifyagivenversionoftheprojectthrougha meaningfulsymbolicname
Usefulforexampleiftheversionhasbeendeliveredtothetest teamortoacustomer.
Tagsarecreatedusingthecvstagcommand. Branchesallowtocreateparallelflowofdevelopments
Maintenanceofapreviousrelease Developmentofexperimentalnewfeatures
Createdusingtheboptionofthecvstagcommand Branchingandmergingisrelativelycomplicated,andfallsoutside thescopeofthistraining.

168
Connectionwithothertools
Mail
Sendamailateverycommit, withthediff.Allowothersto review,andnotifiesthemof thechanges..
IRC
Sendamessageat everycommit.Notifies ofthechanges.
Versioncontrol
Ateverycommit,abuildand testmachinbuildsandtests theproject.
Linkthecommitswiththebugs andthebugswiththecommits.
Buildandtest
Bugtracking
169
WhySubversion?
BecauseCVShasmanydrawbacksandshortcoming
Notpossibletosimplyrenamefilesanddirectorieswhilepreserving thehistory Noatomiccommitsidentifyingthecommitasawhole,makingit difficulttonavigateinthehistory,revertcommits,andisthesource ofrepositoryincoherencyincaseofcrashes Poorbranchingandmergingcapabilities
Inusage,SubversionisverysimilartoCVS:thecommandsare exactlythesame.
FixesallCVSshortcomings Andprovidesinterestingbranchingandmergingfeaturessuchas mergetracking. Manylargescaleprojectsalreadymadetheswitch.
170
Distributedversioncontrolsystems
Sinceafewyears,anewgenerationofversioncontrolsystems Distributedversioncontrolinsteadofacentralizedapproach Principles
Notechnicallycentralrepository,everycopyisarepository Alldeveloperscancreatelocalbranches,sharethesebranches withotherdeveloperswithoutaskingacentralauthority. Advancedbranchingandmergingcapabilities.
Moreandmorecommonlyusedinfreesoftwareprojects(Linux kernel,X.org,etc.) Mostcommonlyusedtools:Git,Mercurial
171
Relateddocuments
Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

Howtohelp
Youcanhelpustoimproveandmaintainthisdocument... Bysendingcorrections,suggestions,contributionsand translations Byaskingyourorganizationtoorderdevelopment,consulting andtrainingservicesperformedbytheauthorsofthese documents(seehttp://freeelectrons.com/). Bysharingthisdocumentwithyourfriends,colleagues andwiththelocalFreeSoftwarecommunity. Byaddinglinksonyourwebsitetoouronlinematerials, toincreasetheirvisibilityinsearchengineresults.
Linuxkernel Linuxdevicedrivers Boardsupportcode Mainstreamingkernelcode Kerneldebugging EmbeddedLinuxTraining Allmaterialsreleasedwithafreelicense! UnixandGNU/Linuxbasics Linuxkernelanddriversdevelopment RealtimeLinux,uClinux Developmentandprofilingtools Lightweighttoolsforembeddedsystems Rootfilesystemcreation Audioandmultimedia Systemoptimization
FreeElectrons
Ourservices
CustomDevelopment Systemintegration EmbeddedLinuxdemosandprototypes Systemoptimization Applicationandinterfacedevelopment Consultingandtechnicalsupport Helpindecisionmaking Systemarchitecture Systemdesignandperformancereview Developmenttoolandapplicationsupport Investigatingissuesandfixingtoolbugs

Unix Linux Introduction

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unix Linux Introduction

Uploaded by

Copyright:

Available Formats

TheUnixandGNU/Linuxcommandline

TheUnixand GNU/Linux commandline

Copyright2009,FreeElectrons. CreativeCommonsBYSA3.0license Latestupdate:Jul15,2010, Documentsources,updatesandtranslations: http://freeelectrons.com/docs/commandline Corrections,suggestions,contributionsandtranslationsarewelcome!

Texteditors Compressingandarchiving Printingfiles Comparingfilesanddirectories Lookingforfiles Gettinginformationaboutusers

TheUnixfilesystemstructureisdefined bytheFilesystemHierarchyStandard(FHS): http://www.pathname.com/fhs/

replacelineslike |string1|string2| by string1>string2

Howtoidentifysymboliclinks: lsldisplays>andthelinkedfilename. GNUlsdisplayslinkswithadifferentcolor.

Inode interface Filesystem

fg fg%<n> Putsthelast/nthbackgroundjobinforegroundmode Movingthecurrenttaskinbackgroundmode: [Ctrl]Z bg kill%<n> Abortsthenthjob.

Youcanchangethesortingorderbytyping M:Memoryusage,P:%CPU,T:Time. Youcankillataskbytypingkandtheprocessid.

duh<file>(diskusage) h:returnssizeondiskofthegivenfile,inhuman readableformat:K(kilobytes),M(megabytes)orG (gigabytes),.Withouth,dureturnstherawnumberof diskblocksusedbythefile(hardtoread). NotethatthehoptiononlyexistsinGNUdu. dush<dir> s:returnsthesumofdiskusageofallthefilesinthe givendirectory.

dfh Returnsdiskspaceinformationforallfilesystems availableinthesystem.Whenerrorshappen,usefulto lookforfullfilesystems.

P:partial(keeppartiallytransferredfiles)andprogress (showprogressduringtransfer) delete:deletefilesinthetargetwhichdon'texistinthesource. Caution:directorynamesshouldendwith/.Otherwise,yougeta my_ideas/ideas/directoryatthedestination.

TheSHAalgorithmisimplementedintheshaXsum(sha1sum, sha256sum,etc.) Theintegrityofseveralfilescanbeverifiedagainstafilelisting thechecksumsusingthecoption.

ForC++: g++otesttest.cc TheWalloptionenablesmorewarnings Tocompilesourcesfilestoobjectfilesandlinktheapplication: gccctest1.c gccctest2.c gccotesttest1.otest2.o gccautomaticallycallsthelinkerld

OutsideoftheClibrary,thousandsofotherlibrariesareavailable forgraphicprogramming,multimedia,networking,scientific computations,andmoroe. Mostlibrariesarealreadyavailableaspackagesinyour distribution,ingeneralintwopackages

Tocompileyourapplicationwiththelibrary,theeasiestsolutionis tousepkgconfig,whichissupportedbymostlibrariestoday: gccotesttest.c$(pkgconfigcflagslibs) Bydefault,theapplicationaredynamicallylinkedwiththe libraries

Thetypicalstepstocompileaautotoolsbasedpackageare ./configure make sudomakeinstall

Anapplicationmustbecompiledwiththegoptiontobe properlydebugged.Thisoptionaddsdebugginginformationto theapplicationbinary gccotesttest.cg Theapplicationcanthenberuninsidethegdbdebugger: gdbtest Orthedebuggercanbeattachedtotheapplicationwhileitis running: gdbtestpPID

Generatedfilesshouldneverbeversioned. Versioncontrolisabsolutelynecessarywhenworkingasateam onaproject

Repository Client Client

Inthefree/opensourceprojects,thetraditionalmodelisthe mergeone.Itavoidstheburdenofhandlingthelockingand scalesbetterwithabiggerteam. ButCVSandSubversionbothimplementadvisorylocking:a developermustvoluntarilylockthefilesheisgoingtoworkwith.

FreeSoftware Usedtobethemostwidelyusedfreeversioncontrolsystem. NowreplacedbySubversion. Stillusedbybigprojects,butalotoflargeprojects(KDE, Apache,Eclipse)havemovedtoSubversion. Homepage:http://www.nongnu.org/cvs/ Documentation:http://ximbiot.com/cvs/manual/

Toseethemodificationsmadetotheprojectandnotcommitted yet,usethediffcommand Repository checkout Workingcopy diff

Thefilehasbeenmodifiedontheserversinceourlastupdate. Sowemustmakemergethemodificationswehavedonewith themodificationsoftheotherdevelopers,byupdatingour workingcopy.

Themodification madebytheother developer

Onceinitialized,itcanbeaccessedlocallyorremotelythrough SSH.Remoteaccessthroughpserverwillrequireadditional configuration,seetheCVSdocumentation. Ifalreadyexistingprojectshavetobeimportedintherepository, usetheimportcommand

Createdusingtheboptionofthecvstagcommand Branchingandmergingisrelativelycomplicated,andfallsoutside thescopeofthistraining.

Ateverycommit,abuildand testmachinbuildsandtests theproject.

Moreandmorecommonlyusedinfreesoftwareprojects(Linux kernel,X.org,etc.) Mostcommonlyusedtools:Git,Mercurial

Allourtechnicalpresentations onhttp://freeelectrons.com/docs Linuxkernel Devicedrivers Architecturespecifics EmbeddedLinuxsystemdevelopment

You might also like