The Linux

Command Line
William E. Shotts, Jr.
A LinuxCommand.org Book
Copyright 2008-2009, William E. Shotts, Jr.
This work is licese! "!er the Creati#e Commos $ttri%"tio-&ocommercial-&o
'eri#ati#e Works (.0 )ite! States *icese. To #iew a copy o+ this licese, #isit
http,--creati#ecommos.org-liceses-%y-c-!-(.0-"s- or se! a letter to Creati#e
Commos, ./. Seco! Street, S"ite (00, Sa 0racisco, Cali+oria, 91.02, )S$.
*i"3

is the registere! tra!emark o+ *i"s Tor#al!s. $ll other tra!emarks %elog to

their respecti#e owers.
This %ook is part o+ the *i"3Comma!.org pro4ect, a site +or *i"3 e!"catio a!
a!#ocacy !e#ote! to helpig "sers o+ legacy operatig systems migrate ito the +"t"re.
5o" may cotact the *i"3Comma!.org pro4ect at http,--li"3comma!.org.
6rite! copies o+ this %ook, i large, easy-to-rea!-+ormat, are a#aila%le +or
p"rchase +rom l"l".com. 7r!ers may %e place! at the +ollowig,
http,--www.l"l".com-cotet-paper%ack-%ook-the-li"3-comma!-lie-/291.81
Release History
Version Date Description
09..2 'ecem%er .1, 2009 0irst E!itio.
09... &o#em%er .9, 2009 0o"rth !ra+t with almost all re#iewer +ee!%ack
icorporate! a! e!ite! thro"gh chapter (/.
09..0 7cto%er (, 2009 Thir! !ra+t with re#ise! ta%le +ormattig,
partial applicatio o+ re#iewers +ee!%ack a!
e!ite! thro"gh chapter .8.
09.08 $"g"st .2, 2009 Seco! !ra+t icorporatig the +irst e!itig
pass.
09.0/ J"ly .8, 2009 Complete! +irst !ra+t.
Table of Contents
Part 1 Introduction.........................................................................1
1 Introduction................................................................................................2
Why Use The Command Line?.......................................................................................2
What This Book Is About.................................................................................................3
Who Shoud !ead This Book..........................................................................................3
What"s In This Book.........................................................................................................#
$o% To !ead This Book..................................................................................................&
'rere(uisites...............................................................................................................&
Why I )on"t Ca It *+,U-Linux..............................................................................&
A/kno%edgments...........................................................................................................0
1our 2eedba/k Is ,eeded3..............................................................................................4
2urther !eading...............................................................................................................4
Coo5hon.........................................................................................................................4
Part 2 Learning The Shell..............................................................
2 What Is The Shell?...................................................................................10
Termina 6muators........................................................................................................78
1our 2irst 9eystrokes....................................................................................................78
Command $istory.....................................................................................................77
Cursor :o;ement.....................................................................................................77
A 2e% Words About :i/e And 2o/us...................................................................77
Try Some Sim5e Commands........................................................................................72
6nding A Termina Session............................................................................................73
The Consoe Behind The Curtain........................................................................73
2urther !eading.............................................................................................................73
3 Navigation.................................................................................................1
Understanding The 2ie System Tree............................................................................7#
The Current Working )ire/tory......................................................................................7#
Listing The Contents <= A )ire/tory..............................................................................7&
Changing The Current Working )ire/tory.....................................................................70
Absoute 'athnames................................................................................................70
!eati;e 'athnames.................................................................................................70
Some $e5=u Short/uts............................................................................................7>
i
Im5ortant 2a/ts About 2ienames........................................................................7>
!"#loring The S$ste%.............................................................................20
:ore 2un With s...........................................................................................................28
<5tions And Arguments............................................................................................27
A Longer Look At Long 2ormat.................................................................................22
)etermining A 2ie"s Ty5e With =ie................................................................................23
?ie%ing 2ie Contents With ess....................................................................................2#
What Is *Text.?.....................................................................................................2#
Less Is :ore........................................................................................................20
A +uided Tour................................................................................................................20
Symboi/ Links...............................................................................................................2@
$ard Links.....................................................................................................................38
2urther !eading.............................................................................................................38
& 'ani#ulating (iles )nd *irectories........................................................31
Wid/ards.......................................................................................................................37
Chara/ter !anges................................................................................................33
Wid/ards Work In The +UI Too..........................................................................33
mkdir A Create )ire/tories.............................................................................................3#
/5 A Co5y 2ies And )ire/tories....................................................................................3#
Use=u <5tions And 6xam5es..................................................................................3&
m; A :o;e And !ename 2ies......................................................................................30
Use=u <5tions And 6xam5es..................................................................................30
rm A !emo;e 2ies And )ire/tories...............................................................................34
Use=u <5tions And 6xam5es..................................................................................34
Be Care=u With rm3.............................................................................................3>
n A Create Links............................................................................................................3@
$ard Links................................................................................................................3@
Symboi/ Links..........................................................................................................#8
Let"s Buid A 'ayground................................................................................................#8
Creating )ire/tories..................................................................................................#8
Co5ying 2ies............................................................................................................#7
:o;ing And !enaming 2ies.....................................................................................#2
Creating $ard Links..................................................................................................#3
Creating Symboi/ Links...........................................................................................##
!emo;ing 2ies And )ire/tories...............................................................................#&
Creating Syminks With The +UI.........................................................................#4
Summing U5.............................................................................................................#4
+ Wor,ing With Co%%ands........................................................................-
What 6xa/ty Are Commands?......................................................................................#>
Identi=ying Commands...................................................................................................#@
ty5e A )is5ay A Command"s Ty5e...........................................................................#@
%hi/h A )is5ay An 6xe/utabe"s Lo/ation...............................................................#@
+etting A Command"s )o/umentation..........................................................................&8
he5 A +et $e5 2or She Buitins............................................................................&8
BBhe5 A )is5ay Usage In=ormation..........................................................................&7
man A )is5ay A 'rogram"s :anua 'age................................................................&7
a5ro5os A )is5ay A55ro5riate Commands..............................................................&3
ii
%hatis A )is5ay A ?ery Brie= )es/ri5tion <= A Command.......................................&3
The :ost Bruta :an 'age <= Them A..............................................................&3
in=o A )is5ay A 'rogram"s In=o 6ntry.......................................................................&#
!6A):6 And <ther 'rogram )o/umentation 2ies................................................&&
Creating 1our <%n Commands With aias....................................................................&0
!e;isiting <d 2riends...................................................................................................&>
2urther !eading.............................................................................................................&>
. /edirection................................................................................................&0
Standard In5utC <ut5utC And 6rror.................................................................................&@
!edire/ting Standard <ut5ut.........................................................................................08
!edire/ting Standard 6rror............................................................................................07
!edire/ting Standard <ut5ut And Standard 6rror To <ne 2ie................................02
)is5osing <= Un%anted <ut5ut................................................................................03
-de;-nu In Unix Cuture......................................................................................03
!edire/ting Standard In5ut............................................................................................03
/at A Con/atenate 2ies............................................................................................03
'i5eines........................................................................................................................0&
2iters........................................................................................................................00
uni( B !e5ort <r <mit !e5eated Lines....................................................................00
%/ A 'rint LineC WordC And Byte Counts..................................................................04
gre5 A 'rint Lines :at/hing A 'attern......................................................................04
head - tai A 'rint 2irst - Last 'art <= 2ies................................................................0>
tee A !ead 2rom Stdin And <ut5ut To Stdout And 2ies..........................................0@
Summing U5..................................................................................................................48
Linux Is About Imagination..................................................................................48
- Seeing The World )s The Shell Sees It...................................................2
6x5ansion......................................................................................................................42
'athname 6x5ansion...............................................................................................43
'athname 6x5ansion <= $idden 2ies.................................................................4#
Tide 6x5ansion........................................................................................................4#
Arithmeti/ 6x5ansion................................................................................................4&
Bra/e 6x5ansion......................................................................................................40
'arameter 6x5ansion...............................................................................................44
Command Substitution.............................................................................................4>
Duoting..........................................................................................................................4@
)oube Duotes..........................................................................................................4@
Singe Duotes...........................................................................................................>7
6s/a5ing Chara/ters................................................................................................>2
Ba/ksash 6s/a5e Se(uen/es............................................................................>2
Summing U5..................................................................................................................>3
2urther !eading.............................................................................................................>3
0 )dvanced 1e$board Tric,s.....................................................................-
Command Line 6diting..................................................................................................>#
Cursor :o;ement.....................................................................................................>#
:odi=ying Text...........................................................................................................>&
Cutting And 'asting E9iing And 1ankingF Text........................................................>&
The :eta 9ey......................................................................................................>0
iii
Com5etion....................................................................................................................>0
'rogrammabe Com5etion..................................................................................>>
Using $istory.................................................................................................................>>
Sear/hing $istory.....................................................................................................>@
$istory 6x5ansion.....................................................................................................@7
s/ri5t....................................................................................................................@7
Summing U5..................................................................................................................@2
2urther !eading.............................................................................................................@2
10 2er%issions............................................................................................03
<%nersC +rou5 :embersC And 6;erybody 6se............................................................@#
!eadingC WritingC And 6xe/uting...................................................................................@&
/hmod A Change =ie mode.......................................................................................@4
What The $e/k Is </ta?.....................................................................................@>
Setting 2ie :ode With The +UI.............................................................................788
umask A Set )e=aut 'ermissions..........................................................................787
Some S5e/ia 'ermissions................................................................................783
Changing Identities......................................................................................................78#
su A !un A She With Substitute User And +rou5 I)s..........................................78&
sudo A 6xe/ute A Command As Another User.......................................................780
Ubuntu And sudo...............................................................................................784
/ho%n A Change 2ie <%ner And +rou5................................................................784
/hgr5 A Change +rou5 <%nershi5.........................................................................78@
6xer/ising <ur 'ri;ieges............................................................................................78@
Changing 1our 'ass%ord.............................................................................................777
2urther !eading...........................................................................................................772
11 2rocesses..............................................................................................113
$o% A 'ro/ess Works.................................................................................................773
?ie%ing 'ro/esses......................................................................................................77#
?ie%ing 'ro/esses )ynami/ay With to5...............................................................770
Controing 'ro/esses..................................................................................................77>
Interru5ting A 'ro/ess.............................................................................................77@
'utting A 'ro/ess In The Ba/kground....................................................................77@
!eturning A 'ro/ess To The 2oreground...............................................................728
Sto55ing E'ausingF A 'ro/ess................................................................................727
Signas.........................................................................................................................722
Sending Signas To 'ro/esses With ki.................................................................722
Sending Signas To :uti5e 'ro/esses With kia................................................72&
:ore 'ro/ess !eated Commands.............................................................................72&
Part ! Con"iguration #nd The $n%ironment.............................12&
12 The !nviron%ent..................................................................................12-
What Is Stored In The 6n;ironment?..........................................................................72>
6xamining The 6n;ironment..................................................................................72>
Some Interesting ?ariabes.....................................................................................738
$o% Is The 6n;ironment 6stabished?.......................................................................737
What"s In A Startu5 2ie?........................................................................................732
i#
:odi=ying The 6n;ironment.........................................................................................73#
Whi/h 2ies Shoud We :odi=y?.............................................................................73#
Text 6ditors.............................................................................................................73#
Using A Text 6ditor.................................................................................................73&
Why Comments Are Im5ortant..........................................................................73>
A/ti;ating <ur Changes..........................................................................................73@
Summing U5................................................................................................................73@
2urther !eading..........................................................................................................73@
13 ) 3entle Introduction To vi.................................................................10
Why We Shoud Learn ;i.............................................................................................7#8
A Litte Ba/kground ....................................................................................................7#7
Starting And Sto55ing ;i..............................................................................................7#7
Com5atibiity :ode............................................................................................7#2
6diting :odes..............................................................................................................7#3
6ntering Insert :ode..............................................................................................7##
Sa;ing <ur Work....................................................................................................7##
:o;ing The Cursor Around.........................................................................................7#&
Basi/ 6diting................................................................................................................7#0
A55ending Text.......................................................................................................7#0
<5ening A Line.......................................................................................................7#4
)eeting Text...........................................................................................................7#>
CuttingC Co5ying And 'asting Text.........................................................................7#@
Goining Lines...........................................................................................................7&7
Sear/h And !e5a/e....................................................................................................7&7
Sear/hing Within A Line..........................................................................................7&7
Sear/hing The 6ntire 2ie.......................................................................................7&7
+oba Sear/h And !e5a/e...................................................................................7&2
6diting :uti5e 2ies...................................................................................................7&#
S%it/hing Bet%een 2ies.........................................................................................7&&
<5ening Additiona 2ies 2or 6diting......................................................................7&&
Co5ying Content 2rom <ne 2ie Into Another........................................................7&0
Inserting An 6ntire 2ie Into Another.......................................................................7&4
Sa;ing <ur Work.........................................................................................................7&>
2urther !eading..........................................................................................................7&@
1 Custo%i4ing The 2ro%#t.....................................................................1+0
Anatomy <= A 'rom5t..................................................................................................708
Trying Some Aternate 'rom5t )esigns......................................................................702
Adding Coor................................................................................................................703
Termina Con=usion............................................................................................703
:o;ing The Cursor......................................................................................................700
Sa;ing The 'rom5t......................................................................................................704
Summing U5................................................................................................................704
2urther !eading..........................................................................................................70>
Part ' Common Tas(s #nd $ssential Tools.............................1)
1& 2ac,age 'anage%ent..........................................................................1.0
#
'a/kaging Systems.....................................................................................................748
$o% A 'a/kage System Works...................................................................................747
'a/kage 2ies.........................................................................................................747
!e5ositories............................................................................................................747
)e5enden/ies.........................................................................................................742
$igh And Lo%Be;e 'a/kage Toos........................................................................742
Common 'a/kage :anagement Tasks.......................................................................743
2inding A 'a/kage In A !e5ository........................................................................743
Instaing A 'a/kage 2rom A !e5ository.................................................................743
Instaing A 'a/kage 2rom A 'a/kage 2ie.............................................................74#
!emo;ing A 'a/kage.............................................................................................74#
U5dating 'a/kages 2rom A !e5ository..................................................................74&
U5grading A 'a/kage 2rom A 'a/kage 2ie...........................................................74&
Listing Instaed 'a/kages......................................................................................740
)etermining I= A 'a/kage Is Instaed.....................................................................740
)is5aying In=o About An Instaed 'a/kage...........................................................744
2inding Whi/h 'a/kage Instaed A 2ie.................................................................744
Summing U5................................................................................................................744
The Linux So=t%are Instaation :yth.................................................................74>
2urther !eading..........................................................................................................74@
1+ Storage 'edia.......................................................................................1-0
:ounting And Unmounting Storage )e;i/es..............................................................7>8
?ie%ing A List <= :ounted 2ie Systems................................................................7>2
Why Unmounting Is Im5ortant...........................................................................7>&
)etermining )e;i/e ,ames....................................................................................7>0
Creating ,e% 2ie Systems.........................................................................................7>@
:ani5uating 'artitions With =disk..........................................................................7>@
Creating A ,e% 2ie System With mk=s..................................................................7@2
Testing And !e5airing 2ie Systems............................................................................7@3
What The =s/k?..................................................................................................7@3
2ormatting 2o55y )isks..............................................................................................7@3
:o;ing )ata )ire/ty To-2rom )e;i/es.......................................................................7@#
Creating C)B!<: Images..........................................................................................7@&
Creating An Image Co5y <= A C)B!<:.................................................................7@&
Creating An Image 2rom A Coe/tion <= 2ies.......................................................7@&
A 'rogram By Any <ther ,ame.........................................................................7@0
Writing C)B!<: Images.............................................................................................7@0
:ounting An IS< Image )ire/ty............................................................................7@0
Banking A !eBWritabe C)B!<:...........................................................................7@4
Writing An Image....................................................................................................7@4
2urther !eading..........................................................................................................7@4
6xtra Credit..................................................................................................................7@4
1. Net5or,ing............................................................................................100
6xamining And :onitoring A ,et%ork.........................................................................288
5ing.........................................................................................................................288
tra/eroute...............................................................................................................287
netstat.....................................................................................................................282
Trans5orting 2ies <;er A ,et%ork..............................................................................283
#i
=t5............................................................................................................................28#
=t5 A A Better =t5.....................................................................................................280
%get........................................................................................................................280
Se/ure Communi/ation With !emote $osts...............................................................280
ssh..........................................................................................................................284
Tunneing With SS$..........................................................................................278
s/5 And s=t5............................................................................................................277
An SS$ Cient 2or Windo%s?............................................................................272
2urther !eading..........................................................................................................272
1- Searching (or (iles..............................................................................213
o/ate A 2ind 2ies The 6asy Way...............................................................................273
Where )oes The o/ate )atabase Come 2rom?..............................................27&
=ind A 2ind 2ies The $ard Way...................................................................................27&
Tests.......................................................................................................................270
<5erators................................................................................................................27>
'rede=ined A/tions..................................................................................................227
User )e=ined A/tions..............................................................................................223
Im5ro;ing 6==i/ien/y...............................................................................................22#
xargs.......................................................................................................................22#
)eaing With 2unny 2ienames..........................................................................22&
A !eturn To The 'ayground..................................................................................220
<5tions...................................................................................................................22>
2urther !eading..........................................................................................................22@
10 )rchiving )nd 6ac,u#.........................................................................230
Com5ressing 2ies.......................................................................................................238
gHi5.........................................................................................................................237
bHi52.......................................................................................................................233
)onIt Be Com5ressi;e Com5usi;e...................................................................23#
Ar/hi;ing 2ies.............................................................................................................23#
tar............................................................................................................................23#
Hi5...........................................................................................................................2#8
Syn/hroniHing 2ies And )ire/tories............................................................................2#2
Using rsyn/ <;er A ,et%ork...................................................................................2#&
2urther !eading..........................................................................................................2#0
20 /egular !"#ressions...........................................................................2.
What Are !eguar 6x5ressions?............................................................................2#4
gre5.............................................................................................................................2#4
:eta/hara/ters And Literas........................................................................................2#@
The Any Chara/ter......................................................................................................2&8
An/hors.......................................................................................................................2&7
A Cross%ord 'uHHe $e5er...............................................................................2&7
Bra/ket 6x5ressions And Chara/ter Casses..............................................................2&2
,egation.................................................................................................................2&2
Traditiona Chara/ter !anges.................................................................................2&3
'<SIJ Chara/ter Casses......................................................................................2&#
!e;erting To Traditiona Coation <rder............................................................2&4
'<SIJ Basi/ ?s. 6xtended !eguar 6x5ressions.......................................................2&>
#ii
'<SIJ................................................................................................................2&>
Aternation...................................................................................................................2&@
Duanti=iers...................................................................................................................208
? B :at/h An 6ement Kero <r <ne Time...............................................................208
L B :at/h An 6ement Kero <r :ore Times............................................................207
M B :at/h An 6ement <ne <r :ore Times............................................................202
N O B :at/h An 6ement A S5e/i=i/ ,umber <= Times..............................................202
'utting !eguar 6x5ressions To Work.........................................................................203
?aidating A 'hone List With gre5...........................................................................203
2inding Ugy 2ienames With =ind...........................................................................20#
Sear/hing 2or 2ies With o/ate.............................................................................20&
Sear/hing 2or Text In ess And ;im........................................................................20&
Summing U5................................................................................................................204
2urther !eading..........................................................................................................204
21 Te"t 2rocessing....................................................................................2+-
A55i/ations <= Text.....................................................................................................20>
)o/uments.............................................................................................................20@
Web 'ages.............................................................................................................20@
6mai.......................................................................................................................20@
'rinter <ut5ut.........................................................................................................20@
'rogram Sour/e Code............................................................................................20@
!e;isiting Some <d 2riends.......................................................................................20@
/at...........................................................................................................................248
:SB)<S Text ?s. Unix Text...............................................................................247
sort..........................................................................................................................247
uni(.........................................................................................................................24@
Si/ing And )i/ing........................................................................................................2>8
/ut...........................................................................................................................2>8
6x5anding Tabs.................................................................................................2>3
5aste.......................................................................................................................2>#
Poin..........................................................................................................................2>&
Com5aring Text...........................................................................................................2>4
/omm......................................................................................................................2>>
di==...........................................................................................................................2>>
5at/h.......................................................................................................................2@7
6diting <n The 2y.......................................................................................................2@2
tr..............................................................................................................................2@2
!<T73Q The ,otBSoBSe/ret )e/oder !ing........................................................2@#
sed..........................................................................................................................2@&
'eo5e Who Like sed Aso Like.........................................................................383
as5e......................................................................................................................383
Summing U5................................................................................................................384
2urther !eading..........................................................................................................384
6xtra Credit..................................................................................................................38>
22 (or%atting 7ut#ut................................................................................300
Sim5e 2ormatting Toos..............................................................................................38@
n A ,umber Lines..................................................................................................38@
=od A Wra5 6a/h Line To A S5e/i=ied Length........................................................373
#iii
=mt A A Sim5e Text 2ormatter................................................................................373
5r A 2ormat Text 2or 'rinting..................................................................................374
5rint= A 2ormat And 'rint )ata................................................................................37>
)o/ument 2ormatting Systems...................................................................................327
gro==.........................................................................................................................322
Summing U5................................................................................................................32>
2urther !eading..........................................................................................................32>
23 2rinting..................................................................................................330
A Brie= $istory <= 'rinting............................................................................................338
'rinting In The )im Times......................................................................................338
Chara/terBbased 'rinters.......................................................................................337
+ra5hi/a 'rinters...................................................................................................332
'rinting With Linux......................................................................................................333
're5aring 2ies 2or 'rinting.........................................................................................333
5r A Con;ert Text 2ies 2or 'rinting........................................................................333
Sending A 'rint Gob To A 'rinter..................................................................................33&
5r A 'rint 2ies EBerkeey StyeF.............................................................................33&
5 A 'rint 2ies ESystem ? StyeF.............................................................................330
Another <5tionQ a25s..............................................................................................334
:onitoring And Controing 'rint Gobs.........................................................................3#8
5stat A )is5ay 'rint System Status......................................................................3#8
5( A )is5ay 'rinter Dueue Status........................................................................3#7
5rm - /an/e A Can/e 'rint Gobs..........................................................................3#2
Summing U5................................................................................................................3#2
2urther !eading..........................................................................................................3#2
2 Co%#iling 2rogra%s............................................................................3
What Is Com5iing?.....................................................................................................3##
Are A 'rograms Com5ied?..................................................................................3#&
Com5iing A C 'rogram...............................................................................................3#0
<btaining The Sour/e Code...................................................................................3#0
6xamining The Sour/e Tree...................................................................................3#>
Buiding The 'rogram.............................................................................................3&8
Instaing The 'rogram...........................................................................................3&#
Summing U5................................................................................................................3&#
2urther !eading..........................................................................................................3&&
Part * +riting Shell Scripts.......................................................!*&
2& Writing 8our (irst Scri#t......................................................................3&-
What Are She S/ri5ts?...............................................................................................3&>
$o% To Write A She S/ri5t.........................................................................................3&>
S/ri5t 2ie 2ormat........................................................................................................3&@
6xe/utabe 'ermissions..............................................................................................308
S/ri5t 2ie Lo/ation......................................................................................................308
+ood Lo/ations 2or S/ri5ts....................................................................................302
:ore 2ormatting Tri/ks................................................................................................302
Long <5tion ,ames................................................................................................302
i3
Indentation And Line Continuation.........................................................................302
Con=iguring ;im 2or S/ri5t Writing.....................................................................303
Summing U5................................................................................................................30#
2urther !eading..........................................................................................................30#
2+ Starting ) 2ro9ect.................................................................................3+&
2irst StageQ :inima )o/ument...................................................................................30&
Se/ond StageQ Adding A Litte )ata............................................................................304
?ariabes And Constants.............................................................................................30>
Assigning ?aues To ?ariabes And Constants.......................................................347
$ere )o/uments.........................................................................................................342
Summing U5................................................................................................................34&
2urther !eading..........................................................................................................34&
2. To#:*o5n *esign.................................................................................3.+
She 2un/tions............................................................................................................344
Lo/a ?ariabes............................................................................................................3>8
9ee5 S/ri5ts !unning..................................................................................................3>7
She 2un/tions In 1our .bashr/ 2ie..................................................................3>#
Summing U5................................................................................................................3>#
2urther !eading..........................................................................................................3>#
2- (lo5 Control; 6ranching With if.........................................................3-&
i=...................................................................................................................................3>&
6xit Status...................................................................................................................3>0
test...............................................................................................................................3>>
2ie 6x5ressions.....................................................................................................3>>
String 6x5ressions..................................................................................................3@7
Integer 6x5ressions................................................................................................3@2
A :ore :odern ?ersion <= test...................................................................................3@3
EE FF B )esigned 2or Integers........................................................................................3@&
Combining 6x5ressions...............................................................................................3@0
'ortabiity Is The $obgobin <= Litte :inds.......................................................3@>
Contro <5eratorsQ Another Way To Bran/h................................................................3@@
Summing U5................................................................................................................3@@
2urther !eading..........................................................................................................#88
20 /eading 1e$board In#ut......................................................................01
read A !ead ?aues 2rom Standard In5ut...................................................................#82
<5tions...................................................................................................................#8#
I2S..........................................................................................................................#80
1ou CanIt 'i5e read...........................................................................................#84
?aidating In5ut............................................................................................................#8>
:enus..........................................................................................................................#8@
Summing U5................................................................................................................#77
6xtra Credit.............................................................................................................#77
2urther !eading...........................................................................................................#77
30 (lo5 Control; <oo#ing With 5hile = until...........................................12
3
Loo5ing........................................................................................................................#72
%hie........................................................................................................................#72
Breaking <ut <= A Loo5...............................................................................................#7&
unti.........................................................................................................................#70
!eading 2ies With Loo5s...........................................................................................#74
Summing U5................................................................................................................#7>
2urther !eading..........................................................................................................#7>
31 Troubleshooting...................................................................................10
Synta/ti/ 6rrors...........................................................................................................#7@
:issing Duotes.......................................................................................................#28
:issing <r Unex5e/ted Tokens..............................................................................#28
Unanti/i5ated 6x5ansions......................................................................................#27
Logi/a 6rrors .............................................................................................................#23
)e=ensi;e 'rogramming.........................................................................................#23
?eri=ying In5ut.........................................................................................................#2&
)esign Is A 2un/tion <= Time............................................................................#2&
Testing.........................................................................................................................#2&
Test Cases..............................................................................................................#20
)ebugging...................................................................................................................#24
2inding The 'robem Area......................................................................................#24
Tra/ing....................................................................................................................#24
6xamining ?aues )uring 6xe/ution......................................................................#38
Summing U5................................................................................................................#38
2urther !eading..........................................................................................................#37
32 (lo5 Control; 6ranching With case...................................................32
/ase.............................................................................................................................#32
'atterns..................................................................................................................#3#
Summing U5................................................................................................................#30
2urther !eading..........................................................................................................#30
33 2ositional 2ara%eters.........................................................................3.
A//essing The Command Line...................................................................................#34
)etermining The ,umber o= Arguments.................................................................#3>
shi=t A +etting A//ess To :any Arguments............................................................#3@
Sim5e A55i/ations................................................................................................##8
Using 'ositiona 'arameters With She 2un/tions................................................##7
$anding 'ositiona 'arameters 6n :asse.................................................................##2
A :ore Com5ete A55i/ation......................................................................................###
Summing U5................................................................................................................##4
2urther !eading..........................................................................................................#&8
3 (lo5 Control; <oo#ing With for..........................................................&1
=orQ Traditiona She 2orm...........................................................................................#&7
Why i?................................................................................................................#&3
=orQ C Language 2orm.................................................................................................#&#
Summing U5................................................................................................................#&&
2urther !eading..........................................................................................................#&0
3i
3& Strings )nd Nu%bers..........................................................................&.
'arameter 6x5ansion..................................................................................................#&4
Basi/ 'arameters...................................................................................................#&4
6x5ansions To :anage 6m5ty ?ariabes...............................................................#&>
6x5ansions That !eturn ?ariabe ,ames..............................................................#08
String <5erations....................................................................................................#08
Arithmeti/ 6;auation And 6x5ansion.........................................................................#03
,umber Bases........................................................................................................#0#
Unary <5erators.....................................................................................................#0#
Sim5e Arithmeti/....................................................................................................#0#
Assignment.............................................................................................................#00
Bit <5erations.........................................................................................................#0>
Logi/.......................................................................................................................#0@
b/ A An Arbitrary 're/ision Ca/uator Language........................................................#42
Using b/..................................................................................................................#43
An 6xam5e S/ri5t..................................................................................................#43
Summing U5................................................................................................................#4&
6xtra Credit..................................................................................................................#4&
2urther !eading..........................................................................................................#4&
3+ )rra$s.....................................................................................................+
What Are Arrays?........................................................................................................#40
Creating An Array........................................................................................................#40
Assigning ?aues To An Array......................................................................................#44
A//essing Array 6ements...........................................................................................#4>
Array <5erations.........................................................................................................#>8
<ut5utting The 6ntire Contents <= An Array..........................................................#>8
)etermining The ,umber <= Array 6ements.........................................................#>8
2inding The Subs/ri5ts Used By An Array.............................................................#>7
Adding 6ements To The 6nd <= An Array.............................................................#>7
Sorting An Array......................................................................................................#>2
)eeting An Array....................................................................................................#>2
Summing U5................................................................................................................#>3
2urther !eading..........................................................................................................#>#
3. !"otica...................................................................................................-&
+rou5 Commands And Subshes...............................................................................#>&
'ro/ess Substitution...............................................................................................#>0
Tra5s............................................................................................................................#>>
Tem5orary 2ies.................................................................................................#@7
Asyn/hronous 6xe/ution.............................................................................................#@2
%ait.........................................................................................................................#@2
,amed 'i5es...............................................................................................................#@3
Setting U5 A ,amed 'i5e.......................................................................................#@#
Using ,amed 'i5es................................................................................................#@#
Summing U5................................................................................................................#@&
2urther !eading..........................................................................................................#@&
Inde"..............................................................................................................0+
3ii
6art . 8 9tro!"ctio
'art 7 A Introdu/tion
.
. 8 9tro!"ctio
1 Introduction
9 wat to tell yo" a story.
&o, ot the story o+ how, i .99., *i"s Tor#al!s wrote the +irst #ersio o+ the *i"3
kerel. 5o" ca rea! that story i lots o+ *i"3 %ooks. &or am 9 goig to tell yo" the
story o+ how, some years earlier, :ichar! Stallma %ega the ;&) 6ro4ect to create a +ree
)i3-like operatig system. That<s a importat story too, %"t most other *i"3 %ooks
ha#e that oe, as well.
&o, 9 wat to tell yo" the story o+ how yo" ca take %ack cotrol o+ yo"r comp"ter.
Whe 9 %ega workig with comp"ters as a college st"!et i the late .9/0s, there was a
re#ol"tio goig o. The i#etio o+ the microprocessor ha! ma!e it possi%le +or
or!iary people like yo" a! me to act"ally ow a comp"ter. 9t<s har! +or may people
to!ay to imagie what the worl! was like whe oly %ig %"siess a! %ig go#ermet ra
all the comp"ters. *et<s 4"st say, yo" co"l!<t get m"ch !oe.
To!ay, the worl! is #ery !i++eret. Comp"ters are e#erywhere, +rom tiy wristwatches to
giat !ata ceters to e#erythig i %etwee. 9 a!!itio to "%i="ito"s comp"ters, we also
ha#e a "%i="ito"s etwork coectig them together. This has create! a wo!ro"s ew
age o+ persoal empowermet a! creati#e +ree!om, %"t o#er the last co"ple o+ !eca!es
somethig else has %ee happeig. $ sigle giat corporatio has %ee imposig its
cotrol o#er most o+ the worl!<s comp"ters a! !eci!ig what yo" ca a! caot !o
with them. 0ort"ately, people +rom all o#er the worl! are !oig somethig a%o"t it.
They are +ightig to maitai cotrol o+ their comp"ters %y writig their ow so+tware.
They are %"il!ig *i"3.
>ay people speak o+ ?+ree!om@ with regar! to *i"3, %"t 9 !o<t thik most people
kow what this +ree!om really meas. 0ree!om is the power to !eci!e what yo"r
comp"ter !oes, a! the oly way to ha#e this +ree!om is to kow what yo"r comp"ter is
!oig. 0ree!om is a comp"ter that is witho"t secrets, oe where e#erythig ca %e
kow i+ yo" care eo"gh to +i! o"t.
+h, -se The Command Line.
Aa#e yo" e#er otice! i the mo#ies whe the ?s"per hacker,@Byo" kow, the g"y who
ca %reak ito the "ltra-sec"re military comp"ter i "!er thirty seco!sBsits !ow at
2
Why )se The Comma! *ieC
the comp"ter, he e#er to"ches a mo"seC 9t<s %eca"se mo#ie makers realiDe that we, as
h"ma %eigs, isticti#ely kow the oly way to really get aythig !oe o a comp"ter
is %y typig o a key%oar!.
>ost comp"ter "sers to!ay are oly +amiliar with the graphical user interface E;)9F a!
ha#e %ee ta"ght %y #e!ors a! p"!its that the command line interface EC*9F is a
terri+yig thig o+ the past. This is "+ort"ate, %eca"se a goo! comma! lie iter+ace is
a mar#elo"sly e3pressi#e way o+ comm"icatig with a comp"ter i m"ch the same way
the writte wor! is +or h"ma %eigs. 9t<s %ee sai! that ?graphical "ser iter+aces make
easy tasks easy, while comma! lie iter+aces make !i++ic"lt tasks possi%le@ a! this is
still #ery tr"e to!ay.
Sice *i"3 is mo!ele! a+ter the )i3 +amily o+ operatig systems, it shares the same
rich heritage o+ comma! lie tools as )i3. )i3 came ito promiece !"rig the
early .980s Ealtho"gh it was +irst !e#elope! a !eca!e earlierF, %e+ore the wi!esprea!
a!optio o+ the graphical "ser iter+ace a!, as a res"lt, !e#elope! a e3tesi#e comma!
lie iter+ace istea!. 9 +act, oe o+ the strogest reasos early a!opters o+ *i"3 chose it
o#er, say, Wi!ows &T was the power+"l comma! lie iter+ace which ma!e the
?!i++ic"lt tasks possi%le.@
+hat This /oo( Is #0out
This %ook is a %roa! o#er#iew o+ ?li#ig@ o the *i"3 comma! lie. )like some
%ooks that cocetrate o 4"st a sigle program, s"ch as the shell program, bash, this
%ook will try to co#ey how to get alog with the comma! lie iter+ace i a larger
sese. Aow !oes it all workC What ca it !oC What<s the %est way to "se itC
This is not a book about Linux system administration. While ay serio"s !isc"ssio
o+ the comma! lie will i#aria%ly lea! to system a!miistratio topics, this %ook oly
to"ches o a +ew a!miistratio iss"es. 9t will, howe#er, prepare the rea!er +or
a!!itioal st"!y %y pro#i!ig a soli! +o"!atio i the "se o+ the comma! lie, a
essetial tool +or ay serio"s system a!miistratio task.
This book is very Linux-centric. >ay other %ooks try to %roa!e their appeal %y
icl"!ig other plat+orms s"ch as geeric )i3 a! >ac7S G. 9 !oig so, they ?water
!ow@ their cotet to +eat"re oly geeral topics. This %ook, o the other ha!, oly
co#ers cotemporary *i"3 !istri%"tios. &iety-+i#e percet o+ the cotet is "se+"l +or
"sers o+ other )i3-like systems, %"t this %ook is highly targete! at the mo!er *i"3
comma! lie "ser.
+ho Should 1ead This /oo(
This %ook is +or ew *i"3 "sers who ha#e migrate! +rom other plat+orms. >ost likely
yo" are a ?power "ser@ o+ some #ersio o+ >icroso+t Wi!ows. 6erhaps yo"r %oss has
tol! yo" to a!miister a *i"3 ser#er, or may%e yo"<re 4"st a !esktop "ser who is tire! o+
(
. 8 9tro!"ctio
all the sec"rity pro%lems a! wat to gi#e *i"3 a try. That<s +ie. $ll are welcome
here.
That %eig sai!, there is o shortc"t to *i"3 elightemet. *earig the comma! lie
is challegig a! takes real e++ort. 9t<s ot that it<s so har!, %"t rather it<s so vast. The
a#erage *i"3 system has literally thousands o+ programs yo" ca employ o the
comma! lie. Cosi!er yo"rsel+ ware!H learig the comma! lie is ot a cas"al
e!ea#or.
7 the other ha!, learig the *i"3 comma! lie is e3tremely rewar!ig. 9+ yo" thik
yo"<re a ?power "ser@ ow, 4"st wait. 5o" !o<t kow what real power isByet. $!,
"like may other comp"ter skills, kowle!ge o+ the comma! lie is log lastig. The
skills leare! to!ay will still %e "se+"l te years +rom ow. The comma! lie has
s"r#i#e! the test o+ time.
9t is also ass"me! that yo" ha#e o programmig e3periece, %"t ot to worry, we<ll start
yo" !ow that path as well.
+hat2s In This /oo(
This material is presete! i a care+"lly chose se="ece, m"ch like a t"tor sittig e3t to
yo" g"i!ig yo" alog. >ay a"thors treat this material i a ?systematic@ +ashio, which
makes sese +rom a writerIs perspecti#e, %"t ca %e #ery co+"sig to ew "sers.
$other goal is to ac="ait yo" with the )i3 way o+ thikig, which is !i++eret +rom
the Wi!ows way o+ thikig. $log the way, we<ll go o a +ew si!e trips to help yo"
"!ersta! why certai thigs work the way they !o a! how they got that way. *i"3
is ot 4"st a piece o+ so+tware, it<s also a small part o+ the larger )i3 c"lt"re, which has
its ow lag"age a! history. 9 might throw i a rat or two, as well.
This %ook is !i#i!e! ito +i#e parts, each co#erig some aspect o+ the comma! lie
e3periece. Jesi!es the +irst part, which yo" are rea!ig ow, this %ook cotais,
Part 2 Learning The Shell starts o"r e3ploratio o+ the %asic lag"age o+ the
comma! lie icl"!ig s"ch thigs as the str"ct"re o+ comma!s, +ile system
a#igatio, comma! lie e!itig, a! +i!ig help a! !oc"metatio +or
comma!s.
Part !on"iguration #nd The $nvironment co#ers e!itig co+ig"ratio
+iles that cotrol the comp"ter<s operatio +rom the comma! lie.
Part % !ommon Tasks #nd $ssential Tools e3plores may o+ the or!iary
tasks that are commoly per+orme! +rom the comma! lie. )i3-like operatig
systems, s"ch as *i"3, cotai may ?classic@ comma! lie programs that are
"se! to per+orm power+"l operatios o !ata.
Part & 'riting Shell Scri(ts itro!"ces shell programmig, a a!mitte!ly
1
What<s 9 This Jook
r"!imetary, %"t easy to lear, techi="e +or a"tomatig may commo
comp"tig tasks. Jy learig shell programmig, yo" will %ecome +amiliar with
cocepts that ca %e applie! to may other programmig lag"ages.
3o4 To 1ead This /oo(
Start at the %egiig o+ the %ook a! +ollow it to the e!. 9t isIt writte as a re+erece
work, it<s really more like a story with a %egiig, mi!!le, a! a e!.
'rere(uisites
To "se this %ook, all yo" will ee! is a workig *i"3 istallatio. 5o" ca get this i
oe o+ two ways,
.. )nstall Linux on a *not so ne+, com(uter. 9t !oes<t matter which !istri%"tio
yo" choose, tho"gh most people to!ay start o"t with either )%"t", 0e!ora, or
7peS)SE. 9+ i !o"%t, try )%"t" +irst. 9stallig a mo!er *i"3 !istri%"tio
ca %e ri!ic"lo"sly easy or ri!ic"lo"sly !i++ic"lt !epe!ig o yo"r har!ware. 9
s"ggest a !esktop comp"ter that is a co"ple o+ years ol! a! has at least 22K
mega%ytes o+ :$> a! K giga%ytes o+ +ree har! !isk space. $#oi! laptops a!
wireless etworks i+ at all possi%le, as these are o+te more !i++ic"lt to get
workig.
2. -se a .Live !/.0 7e o+ the cool thigs yo" ca !o with may *i"3
!istri%"tios is r" them !irectly +rom a C':7> witho"t istallig them at all.
J"st go ito yo"r J97S set"p a! set yo"r comp"ter to ?Joot +rom C':7>,@
isert the li#e C', a! re%oot. )sig a li#e C' is a great way to test a comp"ter
+or *i"3 compati%ility prior to istallatio. The !isa!#atage o+ "sig a li#e C'
is that it may %e #ery slow compare! to ha#ig *i"3 istalle! o yo"r har! !ri#e.
Joth )%"t" a! 0e!ora Eamog othersF ha#e li#e C' #ersios.
:egar!less o+ how yo" istall *i"3, yo" will ee! to ha#e occasioal s"per"ser Ei.e.,
a!miistrati#eF pri#ileges to carry o"t the lessos i this %ook.
$+ter yo" ha#e a workig istallatio, start rea!ig a! +ollow alog with yo"r ow
comp"ter. >ost o+ the material i this %ook is ?ha!s o,@ so sit !ow a! get typigL
+h, I Don2t Call It 567-8Linux9
9 some ="arters, it<s politically correct to call the *i"3 operatig system the
?;&)-*i"3 operatig system.@ The pro%lem with ?*i"3@ is that there is o
completely correct way to ame it %eca"se it was writte %y may !i++eret
2
. 8 9tro!"ctio
people i a #ast, !istri%"te! !e#elopmet e++ort. Techically speakig, *i"3 is
the ame o+ the operatig system<s kerel, othig more. The kerel is #ery
importat o+ co"rse, sice it makes the operatig system go, %"t it<s ot eo"gh to
+orm a complete operatig system.
Eter :ichar! Stallma, the gei"s-philosopher who +o"!e! the 0ree So+tware
mo#emet, starte! the 0ree So+tware 0o"!atio, +orme! the ;&) 6ro4ect, wrote
the +irst #ersio o+ the ;&) C Compiler EgccF, create! the ;&) ;eeral 6"%lic
*icese Ethe ;6*F, etc., etc., etc. Ae insists that yo" call it ?;&)-*i"3@ to
properly re+lect the cotri%"tios o+ the ;&) 6ro4ect. While the ;&) 6ro4ect
pre!ates the *i"3 kerel, a! the pro4ect<s cotri%"tios are e3tremely !eser#ig
o+ recogitio, placig them i the ame is "+air to e#eryoe else who ma!e
sigi+icat cotri%"tios. Jesi!es, 9 thik ?*i"3-;&)@ wo"l! %e more
techically acc"rate sice the kerel %oots +irst a! e#erythig else r"s o top o+
it.
9 pop"lar "sage, ?*i"3@ re+ers to the kerel a! all the other +ree a! ope
so"rce so+tware +o"! i the typical *i"3 !istri%"tioH that is, the etire *i"3
ecosystem, ot 4"st the ;&) compoets. The operatig system marketplace
seems to pre+er oe-wor! ames s"ch as '7S, Wi!ows, >ac7S, Solaris, 9ri3,
$9G. 9 ha#e chose to "se the pop"lar +ormat. 9+, howe#er, yo" pre+er to "se
?;&)-*i"3@ istea!, please per+orm a metal search a! replace while rea!ig
this %ook. 9 wo<t mi!.
#c(no4ledgments
9 wat to thak the +ollowig people, who helpe! make this %ook possi%le,
Jey Watso, $c="isitios E!itor at Wiley 6"%lishig who origially s"ggeste! that 9
write a shell scriptig %ook.
Joh C. '#orak, ote! col"mist a! p"!it. 9 a episo!e o+ his #i!eo po!cast, ?Craky
;eeks,@ >r. '#orak !escri%e! the process o+ writig, ?Aell. Write 200 wor!s a !ay a!
i a year, yo" ha#e a o#el.@ This a!#ice le! me to write a page a !ay "til 9 ha! a %ook.
'mitri 6opo# wrote a article i 0ree So+tware >agaDie title!, ?Creatig a %ook
template with Writer,@ which ispire! me to "se 7pe7++ice.org Writer +or composig
the te3t. $s it t"re! o"t, it worke! wo!er+"lly.
>ark 6olesky per+orme! a e3traor!iary re#iew a! test o+ the te3t.
Jesse Jecker, TomasD ChrDcDoowicD, >ichael *e#i, Spece >ier also teste! a!
re#iewe! portios o+ the te3t.
K
$ckowle!gmets
Mare >. Shotts cotri%"te! a lot o+ ho"rs, polishig my so-calle! Eglish %y e!itig the
te3t.
$! lastly, the rea!ers o+ *i"3Comma!.org, who ha#e set me so may ki! emails.
Their eco"ragemet ga#e me the i!ea that 9 was really o to somethigL
:our ;eed0ac( Is 7eeded<
This %ook is a ogoig pro4ect, like may ope so"rce so+tware pro4ects. 9+ yo" +i! a
techical error, !rop me a lie at,
%shottsN"sers.so"rce+orge.et
5o"r chages a! s"ggestios may get ito +"t"re releases.
;urther 1eading
Aere are some Wikipe!ia articles o the +amo"s people metioe! i this chapter,
http,--e.wikipe!ia.org-wiki-*i"3OTor#al!s
http,--e.wikipe!ia.org-wiki-:ichar!OStallma
The 0ree So+tware 0o"!atio a! the ;&) 6ro4ect,
http,--e.wikipe!ia.org-wiki-0reeOSo+twareO0o"!atio
http,--www.+s+.org
http,--www.g".org
:ichar! Stallma has writte e3tesi#ely o the ?;&)-*i"3@ amig iss"e,
http,--www.g".org-g"-why-g"-li"3.html
http,--www.g".org-g"-g"-li"3-+a=.htmlPtools
Colophon
This %ook was writte "sig 7pe7++ice.org Writer i *i%eratio Seri+ a! Sas +ots o
a 'ell 9spiro 2(0&, +actory co+ig"re! with )%"t" 8.01. The 6'0 #ersio o+ the te3t
was geerate! !irectly %y 7pe7++ice.org Writer. The co#er was pro!"ce! "sig
9kscape. Co#er !esig %y William E. Shotts, Jr. $"thorIs co#er photograph %y Mare
>. Shotts.
/
6art 2 8 *earig The Shell
'art 2 A Learning The She
9
2 8 What 9s The ShellC
2 What Is The Shell?
Whe we speak o+ the comma! lie, we are really re+errig to the shell. The shell is a
program that takes key%oar! comma!s a! passes them to the operatig system to carry
o"t. $lmost all *i"3 !istri%"tios s"pply a shell program +rom the ;&) 6ro4ect calle!
bash. The ame ?%ash@ is a acroym +or ?Jo"re $gai SAell@, a re+erece to the +act
bash is a ehace! replacemet +or sh, the origial )i3 shell program writte %y
Ste#e Jo"re.
Terminal $mulators
Whe "sig a graphical "ser iter+ace, we ee! aother program calle! a terminal
emulator to iteract with the shell. 9+ we look thro"gh o"r !esktop me"s, we will
pro%a%ly +i! oe. M'E "ses konsole a! ;&7>E "ses gnome-terminal, tho"gh
it<s likely calle! simply ?termial@ o o"r me". There are a "m%er o+ other termial
em"lators a#aila%le +or *i"3, %"t they all %asically !o the same thigH gi#e "s access to
the shell. 5o" will pro%a%ly !e#elop a pre+erece +or oe or aother %ase! o the "m%er
o+ %ells a! whistles it has.
:our ;irst =e,stro(es
So let<s get starte!. *a"ch the termial em"latorL 7ce it comes "p, we sho"l! see
somethig like this,
[me@linuxbox ~]$
This is calle! a shell prompt a! it will appear whee#er the shell is rea!y to accept
ip"t. While it may #ary i appearace somewhat !epe!ig o the !istri%"tio, it will
"s"ally icl"!e yo"r username@machinename, +ollowe! %y the c"rret workig
!irectory Emore a%o"t that i a little %itF a! a !ollar sig.
9+ the last character o+ the prompt is a po"! sig E?P@F rather tha a !ollar sig, the
termial sessio has superuser pri#ileges. This meas either we are logge! i as the root
"ser or we selecte! a termial em"lator that pro#i!es s"per"ser Ea!miistrati#eF
.0
5o"r 0irst Meystrokes
pri#ileges.
$ss"mig that thigs are goo! so +ar, let<s try some typig. Type some gi%%erish at the
prompt like so,
[me@linuxbox ~]$ kaekfjaeifj
Sice this comma! makes o sese, the shell will tell "s so a! gi#e "s aother chace,
bash: kaekfjaeifj: command not found
[me@linuxbox ~]$
Command $istory
9+ we press the "p-arrow key, we will see that the pre#io"s comma! ?kaek+4aei+4@
reappears a+ter the prompt. This is calle! command history. >ost *i"3 !istri%"tios
remem%er the last +i#e h"!re! comma!s %y !e+a"lt. 6ress the !ow-arrow key a! the
pre#io"s comma! !isappears.
Cursor :o;ement
:ecall the pre#io"s comma! with the "p-arrow key agai. &ow try the le+t a! right-
arrow keys. See how we ca positio the c"rsor aywhere o the comma! lieC This
makes e!itig comma!s easy.
# ;e4 +ords #0out >ice #nd ;ocus
While the shell is all a%o"t the key%oar!, yo" ca also "se a mo"se with yo"r
termial em"lator. There is a mechaism %"ilt ito the G Wi!ow System Ethe
"!erlyig egie that makes the ;)9 goF that s"pports a ="ick copy a! paste
techi="e. 9+ yo" highlight some te3t %y hol!ig !ow the le+t mo"se %"tto a!
!raggig the mo"se o#er it Eor !o"%le clickig o a wor!F, it is copie! ito a
%"++er maitaie! %y G. 6ressig the mi!!le mo"se %"tto will ca"se the te3t to
%e paste! at the c"rsor locatio. Try it.
1ote, 'o<t %e tempte! to "se Ctrl-c a! Ctrl-v to per+orm copy a! paste
isi!e a termial wi!ow. They !o<t work. These cotrol co!es ha#e !i++eret
meaigs to the shell a! were assige! may years %e+ore >icroso+t Wi!ows.
..
2 8 What 9s The ShellC
5o"r graphical !esktop e#iromet Emost likely M'E or ;&7>EF, i a e++ort
to %eha#e like Wi!ows, pro%a%ly has its focus policy set to ?click to +oc"s.@
This meas +or a wi!ow to get +oc"s E%ecome acti#eF yo" ee! to click o it.
This is cotrary to the tra!itioal G %eha#ior o+ ?+oc"s +ollows mo"se@ which
meas that a wi!ow gets +oc"s %y 4"st passig the mo"se o#er it. The wi!ow
will ot come to the +oregro"! "til yo" click o it %"t it will %e a%le to recei#e
ip"t. Settig the +oc"s policy to ?+oc"s +ollows mo"se@ will make the copy a!
paste techi="e e#e more "se+"l. ;i#e it a try. 9 thik i+ yo" gi#e it a chace
yo" will pre+er it. 5o" will +i! this settig i the co+ig"ratio program +or yo"r
wi!ow maager.
Tr, Some Simple Commands
&ow that we ha#e leare! to type, let<s try a +ew simple comma!s. The +irst oe is
date. This comma! !isplays the c"rret time a! !ate.
[me@linuxbox ~]$ date
Thu ct !" #$:"#:"% &'T !(()
$ relate! comma! is cal which, %y !e+a"lt, !isplays a cale!ar o+ the c"rret moth.
[me@linuxbox ~]$ cal
ctober !(()
*u +o Tu ,e Th -r *a
# ! $ % " .
) / 0 #( ## #! #$
#% #" #. #) #/ #0 !(
!# !! !$ !% !" !. !)
!/ !0 $( $#
To see the c"rret amo"t o+ +ree space o yo"r !isk !ri#es, type df,
[me@linuxbox ~]$ df
-iles1stem #2-blocks 3sed 4vailable 3se5 +ounted on
6dev6sda! #"##"%"! "(#!$0! 00%0)#. $%5 6
6dev6sda" "0.$#0(/ !."%"%!% $(((/%$! %)5 6home
6dev6sda# #%)).% #)$)( #!!)." #$5 6boot
tm7fs !"./". ( !"./". (5 6dev6shm
.2
Try Some Simple Comma!s
*ikewise, to !isplay the amo"t o+ +ree memory, type the free comma!.
[me@linuxbox ~]$ free
total used free shared buffers cached
+em: "#$)#! "($0). 0)$. ( "$#! #!!0#.
-68 buffers6cache: $)")%/ #$)0.%
*9a7: #("!!%/ #(%)#! 0%)"$.
$nding # Terminal Session
We ca e! a termial sessio %y either closig the termial em"lator wi!ow, or %y
eterig the exit comma! at the shell prompt,
[me@linuxbox ~]$ exit
The Console /ehind The Curtain
E#e i+ we ha#e o termial em"lator r"ig, se#eral termial sessios coti"e
to r" %ehi! the graphical !esktop. Calle! virtual terminals or virtual consoles,
these sessios ca %e accesse! o most *i"3 !istri%"tios %y pressig Ctrl-
4lt--# thro"gh Ctrl-4lt--. o most systems. Whe a sessio is accesse!,
it presets a logi prompt ito which we ca eter o"r "ser ame a! passwor!.
To switch +rom oe #irt"al cosole to aother, press 4lt a! -#--.. To ret"r to
the graphical !esktop, press 4lt--).
;urther 1eading
To lear more a%o"t Ste#e Jo"re, +ather o+ the Jo"re Shell, see this Wikipe!ia
article,
http,--e.wikipe!ia.org-wiki-Ste#eOJo"re
Aere is a article a%o"t the cocept o+ shells i comp"tig,
http,--e.wikipe!ia.org-wiki-ShellOEcomp"tigF
.(
( 8 &a#igatio
3 Navigation
The +irst thig we ee! to lear to !o E%esi!es 4"st typigF is how to a#igate the +ile
system o o"r *i"3 system. 9 this chapter we will itro!"ce the +ollowig comma!s,
79d - 6rit ame o+ c"rret workig !irectory
cd - Chage !irectory
ls - *ist !irectory cotets
-nderstanding The ;ile S,stem Tree
*ike Wi!ows, a )i3-like operatig system s"ch as *i"3 orgaiDes its +iles i what is
calle! a hierarchical directory structure. This meas that they are orgaiDe! i a tree-like
patter o+ !irectories Esometimes calle! +ol!ers i other systemsF, which may cotai
+iles a! other !irectories. The +irst !irectory i the +ile system is calle! the root
directory. The root !irectory cotais +iles a! s"%!irectories, which cotai more +iles
a! s"%!irectories a! so o a! so o.
&ote that "like Wi!ows, which has a separate +ile system tree +or each storage !e#ice,
)i3-like systems s"ch as *i"3 always ha#e a sigle +ile system tree, regar!less o+ how
may !ri#es or storage !e#ices are attache! to the comp"ter. Storage !e#ices are
attache! Eor more correctly, mountedF at #ario"s poits o the tree accor!ig to the
whims o+ the system administrator, the perso Eor persosF resposi%le +or the
maiteace o+ the system.
The Current +or(ing Director,
>ost o+ "s are pro%a%ly +amiliar with a graphical +ile maager which represets the +ile
system tree as i 0ig"re .. &otice that the tree is "s"ally show "pe!e!, that is, with the
root at the top a! the #ario"s %raches !esce!ig %elow.
Aowe#er, the comma! lie has o pict"res, so to a#igate the +ile system tree we ee!
to thik o+ it i a !i++eret way.
.1
The C"rret Workig 'irectory
9magie that the +ile system is a maDe shape! like a "psi!e-!ow tree a! we are a%le to
sta! i the mi!!le o+ it. $t ay gi#e time, we are isi!e a sigle !irectory a! we ca
see the +iles cotaie! i the !irectory a! the pathway to the !irectory a%o#e "s Ecalle!
the parent directoryF a! ay s"%!irectories %elow "s. The !irectory we are sta!ig i is
calle! the current working directory. To !isplay the c"rret workig !irectory, we "se the
79d Eprit workig !irectoryF comma!.
[me@linuxbox ~]$ pwd
6home6me
Whe we +irst log i to o"r system Eor start a termial em"lator sessioF o"r c"rret
workig !irectory is set to o"r home directory. Each "ser acco"t is gi#e its ow home
!irectory a! whe operatig as a reg"lar "ser, the home !irectory is the oly place the
"ser is allowe! to write +iles.
Listing The Contents ?" # Director,
To list the +iles a! !irectories i the c"rret workig !irectory, we "se the ls comma!.
[me@linuxbox ~]$ ls
'eskto7 'ocuments +usic :ictures :ublic Tem7lates ;ideos
.2
Figure 1: File system tree as shown by a
graphical file manager
( 8 &a#igatio
$ct"ally, we ca "se the ls comma! to list the cotets o+ ay !irectory, ot 4"st the
c"rret workig !irectory, a! there are may other +" thigs it ca !o as well. We<ll
spe! more time with ls i the e3t chapter.
Changing The Current +or(ing Director,
To chage yo"r workig !irectory Ewhere we are sta!ig i o"r tree-shape! maDeF we
"se the cd comma!. To !o this, type cd +ollowe! %y the pathname o+ the !esire!
workig !irectory. $ pathame is the ro"te we take alog the %raches o+ the tree to get to
the !irectory we wat. 6athames ca %e speci+ie! i oe o+ two !i++eret waysH as
absolute pathnames or as relative pathnames. *et<s !eal with a%sol"te pathames +irst.
Absoute 'athnames
$ a%sol"te pathame %egis with the root !irectory a! +ollows the tree %rach %y
%rach "til the path to the !esire! !irectory or +ile is complete!. 0or e3ample, there is a
!irectory o yo"r system i which most o+ yo"r system<s programs are istalle!. The
pathame o+ the !irectory is 6usr6bin. This meas +rom the root !irectory Erepresete!
%y the lea!ig slash i the pathameF there is a !irectory calle! Q"srQ which cotais a
!irectory calle! Q%iQ.
[me@linuxbox ~]$ cd /usr/bin
[me@linuxbox bin]$ pwd
6usr6bin
[me@linuxbox bin]$ ls
<<<=isting of man1> man1 files <<<
&ow we ca see that we ha#e chage! the c"rret workig !irectory to 6usr6bin a!
that it is +"ll o+ +iles. &otice how the shell prompt has chage!C $s a co#eiece, it is
"s"ally set "p to a"tomatically !isplay the ame o+ the workig !irectory.
!eati;e 'athnames
Where a a%sol"te pathame starts +rom the root !irectory a! lea!s to its !estiatio, a
relati#e pathame starts +rom the workig !irectory. To !o this, it "ses a co"ple o+ special
sym%ols to represet relati#e positios i the +ile system tree. These special sym%ols are
Q.Q E!otF a! Q..Q E!ot !otF.
The Q.Q sym%ol re+ers to the workig !irectory a! the Q..Q sym%ol re+ers to the workig
!irectory<s paret !irectory. Aere is how it works. *et<s chage the workig !irectory to 6
.K
Chagig The C"rret Workig 'irectory
usr6bin agai,
[me@linuxbox ~]$ cd /usr/bin
[me@linuxbox bin]$ pwd
6usr6bin
7kay, ow let<s say that we wate! to chage the workig !irectory to the paret o+
6usr6bin which is 6usr. We co"l! !o that two !i++eret ways. Either with a a%sol"te
pathame,
[me@linuxbox bin]$ cd /usr
[me@linuxbox usr]$ pwd
6usr
7r, with a relati#e pathame,
[me@linuxbox bin]$ cd ..
[me@linuxbox usr]$ pwd
6usr
Two !i++eret metho!s with i!etical res"lts. Which oe sho"l! we "seC The oe that
re="ires the least typigL
*ikewise, we ca chage the workig !irectory +rom 6usr to 6usr6bin i two
!i++eret ways. Either "sig a a%sol"te pathame,
[me@linuxbox usr]$ cd /usr/bin
[me@linuxbox bin]$ pwd
6usr6bin
7r, with a relati#e pathame,
[me@linuxbox usr]$ cd ./bin
[me@linuxbox bin]$ pwd
6usr6bin
&ow, there is somethig importat that 9 m"st poit o"t here. 9 almost all cases, yo" ca
./
( 8 &a#igatio
omit the Q.-Q. 9t is implie!. Typig,
[me@linuxbox usr]$ cd bin
!oes the same thig. 9 geeral, i+ yo" !o ot speci+y a pathame to somethig, the
workig !irectory will %e ass"me!.
Some $e5=u Short/uts
9 ta%le (-. we see some "se+"l ways the c"rret workig !irectory ca %e ="ickly
chage!.
Table 3- 1: cd Shortcuts
Shortcut 1esult
cd
Chages the workig !irectory to yo"r home !irectory.
cd -
Chages the workig !irectory to the pre#io"s workig !irectory.
cd ~user_name
Chages the workig !irectory to the home !irectory o+
username. 0or e3ample, cd ~bob will chage the !irectory to
the home !irectory o+ "ser ?%o%.@
Important ;acts #0out ;ilenames
.. 0ileames that %egi with a perio! character are hi!!e. This oly meas that
ls will ot list them "less yo" say ls -a. Whe yo"r acco"t was create!,
se#eral hi!!e +iles were place! i yo"r home !irectory to co+ig"re thigs
+or yo"r acco"t. *ater o we will take a closer look at some o+ these +iles to
see how yo" ca c"stomiDe yo"r e#iromet. 9 a!!itio, some applicatios
place their co+ig"ratio a! settigs +iles i yo"r home !irectory as hi!!e
+iles.
2. 0ileames a! comma!s i *i"3, like )i3, are case sesiti#e. The
+ileames ?0ile.@ a! ?+ile.@ re+er to !i++eret +iles.
(. *i"3 has o cocept o+ a ?+ile e3tesio@ like some other operatig systems.
5o" may ame +iles ay way yo" like. The cotets a!-or p"rpose o+ a +ile is
.8
Chagig The C"rret Workig 'irectory
!etermie! %y other meas. $ltho"gh )i3-like operatig system !oIt "se
+ile e3tesios to !etermie the cotets-p"rpose o+ +iles, some applicatio
programs !o.
1. Tho"gh *i"3 s"pports log +ileames which may cotai em%e!!e! spaces
a! p"ct"atio characters, limit the p"ct"atio characters i the ames o+
+iles yo" create to perio!, !ash, a! "!erscore. !ost importantly" do not
embed spaces in filenames# 9+ yo" wat to represet spaces %etwee wor!s i
a +ileame, "se "!erscore characters. 5o" will thak yo"rsel+ later.
.9
1 8 E3plorig The System
!"#loring The S$ste%
&ow that we kow how to mo#e aro"! the +ile system, it<s time +or a g"i!e! to"r o+ o"r
*i"3 system. Je+ore we start howe#er, weIre goig to lear some more comma!s that
will %e "se+"l alog the way,
ls 8 *ist !irectory cotets
file 8 'etermie +ile type
less 8 Riew +ile cotets
>ore ;un +ith ls
The ls comma! is pro%a%ly the most "se! comma!, a! +or goo! reaso. With it, we
ca see !irectory cotets a! !etermie a #ariety o+ importat +ile a! !irectory
attri%"tes. $s we ha#e see, we ca simply type ls to see a list o+ +iles a!
s"%!irectories cotaie! i the c"rret workig !irectory,
[me@linuxbox ~]$ ls
'eskto7 'ocuments +usic :ictures :ublic Tem7lates ;ideos
Jesi!es the c"rret workig !irectory, we ca speci+y the !irectory to list, like so,
me@linuxbox ~]$ ls /usr
bin games kerberos libexec sbin src
etc include lib local share tm7
7r e#e speci+y m"ltiple !irectories. 9 this e3ample we will list %oth the "ser<s home
!irectory Esym%oliDe! %y the ?S@ characterF a! the 6usr !irectory,
[me@linuxbox ~]$ ls ~ /usr
6home6me:
20
>ore 0" With ls
'eskto7 'ocuments +usic :ictures :ublic Tem7lates ;ideos
6usr:
bin games kerberos libexec sbin src
etc include lib local share tm7
We ca also chage the +ormat o+ the o"tp"t to re#eal more !etail,
[me@linuxbox ~]$ ls -l
total ".
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( 'eskto7
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( 'ocuments
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( +usic
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( :ictures
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( :ublic
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( Tem7lates
dr9xr9xr-x ! me me %(0. !(()-#(-!. #):!( ;ideos
Jy a!!ig ?-l@ to the comma!, we chage! the o"tp"t to the log +ormat.
<5tions And Arguments
This %rigs "s to a #ery importat poit a%o"t how most comma!s work. Comma!s
are o+te +ollowe! %y oe or more options that mo!i+y their %eha#ior, a! +"rther, %y oe
or more arguments, the items "po which the comma! acts. So most comma!s look
ki! o+ like this,
command -options arguments
>ost comma!s "se optios cosistig o+ a sigle character prece!e! %y a !ash, +or
e3ample, ?-l@, %"t may comma!s, icl"!ig those +rom the ;&) 6ro4ect, also s"pport
long options, cosistig o+ a wor! prece!e! %y two !ashes. $lso, may comma!s allow
m"ltiple short optios to %e str"g together. 9 this e3ample, the ls comma! is gi#e
two optios, the ?l@ optio to pro!"ce log +ormat o"tp"t, a! the ?t@ optio to sort the
res"lt %y the +ile<s mo!i+icatio time.
[me@linuxbox ~]$ ls -lt
2.
1 8 E3plorig The System
We<ll a!! the log optio ?--re#erse@ to re#erse the or!er o+ the sort,
[me@linuxbox ~]$ ls -lt --reverse
The ls comma! has a large "m%er o+ possi%le optios. The most commo are liste! i
the Ta%le 1-..
Table $- 1: %ommon ls &ptions
?ption Long ?ption Description
-a --all
*ist all +iles, e#e those with ames that %egi
with a perio!, which are ormally ot liste!
Ei.e., hi!!eF.
-d --director1
7r!iarily, i+ a !irectory is speci+ie!, ls will
list the cotets o+ the !irectory, ot the
!irectory itsel+. )se this optio i
co4"ctio with the -l optio to see !etails
a%o"t the !irectory rather tha its cotets.
-- --classif1
This optio will appe! a i!icator character
to the e! o+ each liste! ame. 0or e3ample,
a ?-@ i+ the ame is a !irectory.
-h --human-readable
9 log +ormat listigs, !isplay +ile siDes i
h"ma rea!a%le +ormat rather tha i %ytes.
-l
'isplay res"lts i log +ormat.
-r --reverse
'isplay the res"lts i re#erse or!er.
&ormally, ls !isplays its res"lts i asce!ig
alpha%etical or!er.
-*
Sort res"lts %y +ile siDe.
-t
Sort %y mo!i+icatio time.
A Longer Look At Long 2ormat
$s we saw %e+ore, the ?-l@ optio ca"ses ls to !isplay its res"lts i log +ormat. This
+ormat cotais a great !eal o+ "se+"l i+ormatio. Aere is the &xam7les !irectory
+rom a )%"t" system,
22
>ore 0" With ls
-r9-r--r-- # root root $").!0. !(()-(%-($ ##:(" &x7erience ubuntu<ogg
-r9-r--r-- # root root ##/.!#0 !(()-(%-($ ##:(" kubuntu-leaflet<7ng
-r9-r--r-- # root root %)"/% !(()-(%-($ ##:(" logo-&dubuntu<7ng
-r9-r--r-- # root root %%$"" !(()-(%-($ ##:(" logo-2ubuntu<7ng
-r9-r--r-- # root root $%$0# !(()-(%-($ ##:(" logo-3buntu<7ng
-r9-r--r-- # root root $!("0 !(()-(%-($ ##:(" oo-cd-cover<odf
-r9-r--r-- # root root #"0)%% !(()-(%-($ ##:(" oo-derivatives<doc
-r9-r--r-- # root root !)/$) !(()-(%-($ ##:(" oo-max9ell<odt
-r9-r--r-- # root root 0//#. !(()-(%-($ ##:(" oo-trig<xls
-r9-r--r-- # root root %"$).% !(()-(%-($ ##:(" oo-9elcome<odt
-r9-r--r-- # root root $"/$)% !(()-(%-($ ##:(" ubuntu *ax<ogg
*et<s look at the !i++eret +iel!s +rom oe o+ the +iles a! e3amie their meaigs,
Table $-': ls (ong (isting Fields
;ield >eaning
-r9-r--r--
$ccess rights to the +ile. The +irst character i!icates the
type o+ +ile. $mog the !i++eret types, a lea!ig !ash
meas a reg"lar +ile, while a ?!@ i!icates a !irectory.
The e3t three characters are the access rights +or the
+ile<s ower, the e3t three are +or mem%ers o+ the +ile<s
gro"p, a! the +ial three are +or e#eryoe else. The +"ll
meaig o+ this is !isc"sse! i Chapter .0 8 6ermissios.
#
0ile<s "m%er o+ har! liks. See the !isc"ssio o+ liks
later i this chapter.
root
The "ser ame o+ the +ile<s ower.
root
The ame o+ the gro"p which ows the +ile.
$!("0
SiDe o+ the +ile i %ytes.
!(()-(%-($ ##:("
'ate a! time o+ the +ile<s last mo!i+icatio.
oo-cd-cover<odf
&ame o+ the +ile.
Determining # ;ile2s T,pe +ith file
$s we e3plore the system it will %e "se+"l to kow what +iles cotai. To !o this we will
"se the file comma! to !etermie a +ile<s type. $s we !isc"sse! earlier, +ileames i
*i"3 are ot re="ire! to re+lect a +ile<s cotets. While a +ileame like ?pict"re.4pg@
wo"l! ormally %e e3pecte! to cotai a J6E; compresse! image, it is ot re="ire! to i
*i"3. We ca i#oke the file comma! this way,
2(
1 8 E3plorig The System
file filename
Whe i#oke!, the file comma! will prit a %rie+ !escriptio o+ the +ile<s cotets.
0or e3ample,
[me@linuxbox ~]$ file picture.jpg
7icture<j7g: ?:&@ image data> ?-A- standard #<(#
There are may ki!s o+ +iles. 9 +act, oe o+ the commo i!eas i )i3-like operatig
systems s"ch as *i"3 is that ?e#erythig is a +ile.@ $s we procee! with o"r lessos, we
will see 4"st how tr"e that statemet is.
While may o+ the +iles o yo"r system are +amiliar, +or e3ample >6( a! J6E;, there
are may ki!s that are a little less o%#io"s a! a +ew that are ="ite strage.
Vie4ing ;ile Contents +ith less
The less comma! is a program to #iew te3t +iles. Thro"gho"t o"r *i"3 system, there
are may +iles that cotai h"ma-rea!a%le te3t. The less program pro#i!es a
co#eiet way to e3amie them.
+hat Is 5Text9.
There are may ways to represet i+ormatio o a comp"ter. $ll metho!s
i#ol#e !e+iig a relatioship %etwee the i+ormatio a! some "m%ers that
will %e "se! to represet it. Comp"ters, a+ter all, oly "!ersta! "m%ers a! all
!ata is co#erte! to "meric represetatio.
Some o+ these represetatio systems are #ery comple3 Es"ch as compresse!
#i!eo +ilesF, while others are rather simple. 7e o+ the earliest a! simplest is
calle! )S%** te+t. $SC99 Eproo"ce! Q$s-MeyQF is short +or $merica Sta!ar!
Co!e +or 9+ormatio 9terchage. This is a simple eco!ig scheme that was +irst
"se! o Teletype machies to map key%oar! characters to "m%ers.
Te3t is a simple oe-to-oe mappig o+ characters to "m%ers. 9t is #ery compact.
0i+ty characters o+ te3t traslates to +i+ty %ytes o+ !ata. 9t is importat to
"!ersta! that te3t oly cotais a simple mappig o+ characters to "m%ers. 9t
is ot the same as a wor! processor !oc"met s"ch as oe create! %y >icroso+t
Wor! or 7pe7++ice.org Writer. Those +iles, i cotrast to simple $SC99 te3t,
21
Riewig 0ile Cotets With less
cotai may o-te3t elemets that are "se! to !escri%e its str"ct"re a!
+ormattig. 6lai $SC99 te3t +iles cotai oly the characters themsel#es a! a
+ew r"!imetary cotrol co!es like ta%s, carriage ret"rs a! lie +ee!s.
Thro"gho"t a *i"3 system, may +iles are store! i te3t +ormat a! there are
may *i"3 tools that work with te3t +iles. E#e Wi!ows recogiDes the
importace o+ this +ormat. The well-kow &7TE6$'.EGE program is a e!itor
+or plai $SC99 te3t +iles.
Why wo"l! we wat to e3amie te3t +ilesC Jeca"se may o+ the +iles that cotai system
settigs Ecalle! configuration filesF are store! i this +ormat, a! %eig a%le to rea! them
gi#es "s isight a%o"t how the system works. 9 a!!itio, may o+ the act"al programs
that the system "ses Ecalle! scriptsF are store! i this +ormat. 9 later chapters, we will
lear how to e!it te3t +iles i or!er to mo!i+y systems settigs a! write o"r ow scripts,
%"t +or ow we will 4"st look at their cotets.
The less comma! is "se! like this,
less filename
7ce starte!, the less program allows yo" to scroll +orwar! a! %ackwar! thro"gh a
te3t +ile. 0or e3ample, to e3amie the +ile that !e+ies all the system<s "ser acco"ts,
eter the +ollowig comma!,
[me@linuxbox ~]$ less /etc/passwd
7ce the less program starts, we may #iew the cotets o+ the +ile. 9+ the +ile is loger
tha oe page, we ca scroll "p a! !ow. To e3it less, press the ?=@ key.
The ta%le %elow lists the most commo key%oar! comma!s "se! %y less.
Table $-3: less %ommands
Command #ction
6age )p or b Scroll %ack oe page
6age 'ow or space Scroll +orwar! oe page
)p $rrow Scroll "p oe lie
22
1 8 E3plorig The System
'ow $rrow Scroll !ow oe lie
@
>o#e to the e! o+ the te3t +ile
#@ or g >o#e to the %egiig o+ the te3t +ile
6characters Search +orwar! to the e3t occ"rrece o+ characters
n
Search +or the e3t occ"rrece o+ the pre#io"s search
h
'isplay help scree
B
T"it less
Less Is >ore
The less program was !esige! as a impro#e! replacemet o+ a earlier )i3
program calle! more. The ame ?less@ is a play o the phrase ?less is more@Ba
motto o+ mo!erist architects a! !esigers.
less +alls ito the class o+ programs calle! ?pagers,@ programs that allow the
easy #iewig o+ log te3t !oc"mets i a page %y page maer. Whereas the
more program co"l! oly page +orwar!, the less program allows pagig %oth
+orwar! a! %ackwar! a! has may other +eat"res as well.
# 6uided Tour
The +ile system layo"t o yo"r *i"3 system is m"ch like that +o"! o other )i3-like
systems. The !esig is act"ally speci+ie! i a p"%lishe! sta!ar! calle! the (inu+
Filesystem ,ierarchy Standard. &ot all *i"3 !istri%"tios co+orm to the sta!ar!
e3actly %"t most come pretty close.
&e3t, we are goig to wa!er aro"! the +ile system o"rsel#es to see what makes o"r
*i"3 system tick. This will gi#e yo" a chace to practice yo"r a#igatio skills. 7e o+
the thigs we will !isco#er is that may o+ the iterestig +iles are i plai h"ma-
rea!a%le te3t. $s we go a%o"t o"r to"r, try the +ollowig,
.. cd ito a gi#e !irectory
2. *ist the !irectory cotets with ls -l
(. 9+ yo" see a iterestig +ile, !etermie its cotets with file
1. 9+ it looks like it might %e te3t, try #iewig it with less
2K
$ ;"i!e! To"r
Remember the co(y and (aste trick2 9+ yo" are "sig a mo"se, yo" ca !o"%le
click o a +ileame to copy it a! mi!!le click to paste it ito comma!s.
$s we wa!er aro"!, !o<t %e a+rai! to look at st"++. :eg"lar "sers are largely
prohi%ite! +rom messig thigs "p. That<s the system a!miistrators 4o%L 9+ a comma!
complais a%o"t somethig, 4"st mo#e o to somethig else. Spe! some time lookig
aro"!. The system is o"rs to e3plore. :emem%er, i *i"3, there are o secretsL
Ta%le 1-1 lists 4"st a +ew o+ the !irectories we ca e3plore. 0eel +ree to try moreL
Table $-$: -irectories Found &n (inu+ Systems
Director, Comments
6
The root !irectory. Where e#erythig %egis.
6bin
Cotais %iaries EprogramsF that m"st %e preset +or the
system to %oot a! r".
6boot
Cotais the *i"3 kerel, iitial :$> !isk image E+or
!ri#ers ee!e! at %oot timeF, a! the %oot loa!er.
9terestig +iles,
6boot6grub6grub<conf or menu<lst, which
are "se! to co+ig"re the %oot loa!er.
6boot6vmlinuC, the li"3 kerel
6dev
This is a special !irectory which cotais device nodes.
?E#erythig is a +ile@ also applies to !e#ices. Aere is where
the kerel maitais a list o+ all the !e#ices it "!ersta!s.
6etc
The 6etc !irectory cotais all o+ the system-wi!e
co+ig"ratio +iles. 9t also cotais a collectio o+ shell
scripts which start each o+ the system ser#ices at %oot time.
E#erythig i this !irectory sho"l! %e rea!a%le te3t.
9terestig +iles, While e#erythig i 6etc is iterestig,
here are some o+ my all-time +a#orites,
6etc6crontab, a +ile that !e+ies whe
a"tomate! 4o%s will r".
6etc6fstab, a ta%le o+ storage !e#ices a! their
associate! mo"t poits.
6etc67ass9d, a list o+ the "ser acco"ts.
2/
1 8 E3plorig The System
Director, Comments
6home
9 ormal co+ig"ratios, each "ser is gi#e a !irectory i
6home. 7r!iary "sers ca oly write +iles i their home
!irectories. This limitatio protects the system +rom errat
"ser acti#ity.
6lib
Cotais share! li%rary +iles "se! %y the core system
programs. These are similar to '**s i Wi!ows.
6lost8found
Each +ormatte! partitio or !e#ice "sig a *i"3 +ile system,
s"ch as e3t(, will ha#e this !irectory. 9t is "se! i the case
o+ a partial reco#ery +rom a +ile system corr"ptio e#et.
)less somethig really %a! has happee! to yo"r system,
this !irectory will remai empty.
6media
7 mo!er *i"3 systems the 6media !irectory will
cotai the mo"t poits +or remo#a%le me!ia s"ch )SJ
!ri#es, C'-:7>s, etc. that are mo"te! a"tomatically at
isertio.
6mnt
7 ol!er *i"3 systems, the 6mnt !irectory cotais mo"t
poits +or remo#a%le !e#ices that ha#e %ee mo"te!
ma"ally.
6o7t
The 6o7t !irectory is "se! to istall ?optioal@ so+tware.
This is maily "se! to hol! commercial so+tware pro!"cts
that may %e istalle! o yo"r system.
67roc
The 67roc !irectory is special. 9t<s ot a real +ile system i
the sese o+ +iles store! o yo"r har! !ri#e. :ather, it is a
#irt"al +ile system maitaie! %y the *i"3 kerel. The
?+iles@ it cotais are peepholes ito the kerel itsel+. The
+iles are rea!a%le a! will gi#e yo" a pict"re o+ how the
kerel sees yo"r comp"ter.
6root
This is the home !irectory +or the root acco"t.
6sbin
This !irectory cotais ?system@ %iaries. These are
programs that per+orm #ital system tasks that are geerally
reser#e! +or the s"per"ser.
6tm7
The 6tm7 !irectory is ite!e! +or storage o+ temporary,
trasiet +iles create! %y #ario"s programs. Some
co+ig"ratios ca"se this !irectory to %e emptie! each time
the system is re%oote!.
28
$ ;"i!e! To"r
Director, Comments
6usr
The 6usr !irectory tree is likely the largest oe o a *i"3
system. 9t cotais all the programs a! s"pport +iles "se!
%y reg"lar "sers.
6usr6bin
6usr6bin cotais the e3ec"ta%le programs istalle! %y
yo"r *i"3 !istri%"tio. 9t is ot "commo +or this
!irectory to hol! tho"sa!s o+ programs.
6usr6lib
The share! li%raries +or the programs i 6usr6bin.
6usr6local
The 6usr6local tree is where programs that are ot
icl"!e! with yo"r !istri%"tio %"t are ite!e! +or system-
wi!e "se are istalle!. 6rograms compile! +rom so"rce co!e
are ormally istalle! i 6usr6local6bin. 7 a ewly
istalle! *i"3 system, this tree e3ists, %"t it will %e empty
"til the system a!miistrator p"ts somethig i it.
6usr6sbin
Cotais more system a!miistratio programs.
6usr6share
6usr6share cotais all the share! !ata "se! %y
programs i 6usr6bin. This icl"!es thigs like !e+a"lt
co+ig"ratio +iles, icos, scree %ackgro"!s, so"! +iles,
etc.
6usr6share6doc
>ost packages istalle! o the system will icl"!e some
ki! o+ !oc"metatio. 9 6usr6share6doc, we will
+i! !oc"metatio +iles orgaiDe! %y package.
6var
With the e3ceptio o+ 6tm7 a! 6home, the !irectories we
ha#e looke! at so +ar remai relati#ely static, that is, their
cotets !o<t chage. The 6var !irectory tree is where
!ata that is likely to chage is store!. Rario"s !ata%ases,
spool +iles, "ser mail, etc. are locate! here.
6var6log
6var6log cotais log files, recor!s o+ #ario"s system
acti#ity. These are #ery importat a! sho"l! %e moitore!
+rom time to time. The most "se+"l oe is
6var6log6messages. &ote that +or sec"rity reasos o
some systems, yo" m"st %e the s"per"ser to #iew log +iles .
S,m0olic Lin(s
$s we look aro"!, we are likely to see a !irectory listig with a etry like this,
29
1 8 E3plorig The System
lr9xr9xr9x # root root ## !(()-(/-## ():$% libc<so<. -D libc-!<.<so
&otice how the +irst letter o+ the listig is ?l@ a! the etry seems to ha#e two +ileamesC
This is a special ki! o+ a +ile calle! a symbolic link Ealso kow as a soft link or
symlink.F 9 most )i3-like systems it is possi%le to ha#e a +ile re+erece! %y m"ltiple
ames. While the #al"e o+ this may ot %e o%#io"s, it is really a "se+"l +eat"re.
6ict"re this sceario, a program re="ires the "se o+ a share! reso"rce o+ some ki!
cotaie! i a +ile ame! ?+oo,@ %"t ?+oo@ has +re="et #ersio chages. 9t wo"l! %e
goo! to icl"!e the #ersio "m%er i the +ileame so the a!miistrator or other
itereste! party co"l! see what #ersio o+ ?+oo@ is istalle!. This presets a pro%lem. 9+
we chage the ame o+ the share! reso"rce, we ha#e to track !ow e#ery program that
might "se it a! chage it to look +or a ew reso"rce ame e#ery time a ew #ersio o+
the reso"rce is istalle!. That !oes<t so"! like +" at all.
Aere is where sym%olic liks sa#e the !ay. *et<s say we istall #ersio 2.K o+ ?+oo,@
which has the +ileame ?+oo-2.K@ a! the create a sym%olic lik simply calle! ?+oo@ that
poits to ?+oo-2.K.@ This meas that whe a program opes the +ile ?+oo@, it is act"ally
opeig the +ile ?+oo-2.K@. &ow e#ery%o!y is happy. The programs that rely o ?+oo@
ca +i! it a! we ca still see what act"al #ersio is istalle!. Whe it is time to
"pgra!e to ?+oo-2./,@ we 4"st a!! the +ile to o"r system, !elete the sym%olic lik ?+oo@
a! create a ew oe that poits to the ew #ersio. &ot oly !oes this sol#e the pro%lem
o+ the #ersio "pgra!e, %"t it also allows "s to keep %oth #ersios o o"r machie.
9magie that ?+oo-2./@ has a %"g E!am those !e#elopersLF a! we ee! to re#ert to the
ol! #ersio. $gai, we 4"st !elete the sym%olic lik poitig to the ew #ersio a!
create a ew sym%olic lik poitig to the ol! #ersio.
The !irectory listig a%o#e E+rom the 6lib !irectory o+ a 0e!ora systemF shows a
sym%olic lik calle! ?li%c.so.K@ that poits to a share! li%rary +ile calle! ?li%c-2.K.so.@
This meas that programs lookig +or ?li%c.so.K@ will act"ally get the +ile ?li%c-2.K.so.@
We will lear how to create sym%olic liks i the e3t chapter.
3ard Lin(s
While we are o the s"%4ect o+ liks, we ee! to metio that there is a seco! type o+
lik calle! a hard link. Aar! liks also allow +iles to ha#e m"ltiple ames, %"t they !o it
i a !i++eret way. WeIll talk more a%o"t the !i++ereces %etwee sym%olic a! har!
liks i the e3t chapter.
;urther 1eading
The +"ll #ersio o+ the (inu+ Filesystem ,ierarchy Standard ca %e +o"! here,
http,--www.pathame.com-+hs-
(0
2 8 >aip"latig 0iles $! 'irectories
& 'ani#ulating (iles )nd *irectories
$t this poit, we are rea!y +or some real workL This chapter will itro!"ce the +ollowig
comma!s,
c7 8 Copy +iles a! !irectories
mv 8 >o#e-reame +iles a! !irectories
mkdir 8 Create !irectories
rm 8 :emo#e +iles a! !irectories
ln 8 Create har! a! sym%olic liks
These +i#e comma!s are amog the most +re="etly "se! *i"3 comma!s. They are
"se! +or maip"latig %oth +iles a! !irectories.
&ow, to %e +rak, some o+ the tasks per+orme! %y these comma!s are more easily !oe
with a graphical +ile maager. With a +ile maager, we ca !rag a! !rop a +ile +rom oe
!irectory to aother, c"t a! paste +iles, !elete +iles, etc. So why "se these ol! comma!
lie programsC
The aswer is power a! +le3i%ility. While it is easy to per+orm simple +ile maip"latios
with a graphical +ile maager, complicate! tasks ca %e easier with the comma! lie
programs. 0or e3ample, how co"l! we copy all the AT>* +iles +rom oe !irectory to
aother, %"t oly copy +iles that !o ot e3ist i the !estiatio !irectory or are ewer tha
the #ersios i the !estiatio !irectoryC 6retty har! with with a +ile maager. 6retty easy
with the comma! lie,
cp -u *.html destination
+ildcards
Je+ore we %egi "sig o"r comma!s, we ee! to talk a%o"t a shell +eat"re that makes
these comma!s so power+"l. Sice the shell "ses +ileames so m"ch, it pro#i!es special
characters to help yo" rapi!ly speci+y gro"ps o+ +ileames. These special characters are
(.
2 8 >aip"latig 0iles $! 'irectories
calle! wildcards. )sig wil!car!s Ewhich is also kow as globbingF allow yo" to select
+ileames %ase! o patters o+ characters. The ta%le %elow lists the wil!car!s a! what
they select,
Table .-1: /ildcards
+ildcard >eaning
E
>atches ay characters
F
>atches ay sigle character
[characters]
>atches ay character that is a mem%er o+ the set characters
[Gcharacters]
>atches ay character that is ot a mem%er o+ the set
characters
[[:class:]]
>atches ay character that is a mem%er o+ the speci+ie!
class
Ta%le 2-2 lists the most commoly "se! character classes,
Table .-': %ommonly 0sed %haracter %lasses
Character Class >eaning
[:alnum:]
>atches ay alpha"meric character
[:al7ha:]
>atches ay alpha%etic character
[:digit:]
>atches ay "meral
[:lo9er:]
>atches ay lowercase letter
[:u77er:]
>atches ay "ppercase letter
)sig wil!car!s makes it possi%le to costr"ct #ery sophisticate! selectio criteria +or
+ileames. Aere are some e3amples o+ patters a! what they match,
Table .-3: /ildcard 1+amples
Pattern >atches
E
$ll +iles
gE
$y +ile %egiig with ?g@
bE<txt
$y +ile %egiig with ?%@ +ollowe! %y
ay characters a! e!ig with ?.t3t@
(2
Wil!car!s
'ataFFF
$y +ile %egiig with ?'ata@ +ollowe!
%y e3actly three characters
[abc]E
$y +ile %egiig with either a ?a@, a
?%@, or a ?c@
H4C23:<[(-0][(-0][(-0]
$y +ile %egiig with ?J$CM)6.@
+ollowe! %y e3actly three "merals
[[:u77er:]]E
$y +ile %egiig with a "ppercase letter
[G[:digit:]]E
$y +ile ot %egiig with a "meral
E[[:lo9er:]#!$]
$y +ile e!ig with a lowercase letter or
the "merals ?.@, ?2@, or ?(@
Wil!car!s ca %e "se! with ay comma! that accepts +ileames as arg"mets, %"t weIll
talk more a%o"t that i Chapter 8.
Character 1anges
9+ yo" are comig +rom aother )i3-like e#iromet or ha#e %ee rea!ig
some other %ooks o this s"%4ect, yo" may ha#e eco"tere! the [4-I] or the
[a-C] character rage otatios. These are tra!itioal )i3 otatios a!
worke! i ol!er #ersios o+ *i"3 as well. They ca still work, %"t yo" ha#e to
%e #ery care+"l with them %eca"se they will ot pro!"ce the e3pecte! res"lts
"less properly co+ig"re!. 0or ow, yo" sho"l! a#oi! "sig them a! "se
character classes istea!.
+ildcards +or( In The 6-I Too
Wil!car!s are especially #al"a%le ot oly %eca"se they are "se! so +re="etly o
the comma! lie, %"t are also s"pporte! %y some graphical +ile maagers.
9 1autilus Ethe +ile maager +or ;&7>EF, yo" ca select +iles "sig the
E!it-Select 6atter me" item. J"st eter a +ile selectio patter with
wil!car!s a! the +iles i the c"rretly #iewe! !irectory will %e highlighte!
+or selectio.
9 /ol(hin a! 3on4ueror Ethe +ile maagers +or M'EF, yo" ca eter
wil!car!s !irectly o the locatio %ar. 0or e3ample, i+ yo" wat to see all the
+iles startig with a lowercase ?"@ i the -"sr-%i !irectory, type ?-"sr-%i-"U@
ito the locatio %ar a! it will !isplay the res"lt.
((
2 8 >aip"latig 0iles $! 'irectories
>ay i!eas origially +o"! i the comma! lie iter+ace make their way ito
the graphical iter+ace, too. 9t is oe o+ the may thigs that make the *i"3
!esktop so power+"l.
mkdir Create Directories
The mkdir comma! is "se! to create !irectories. 9t works like this,
mkdir directory...
# note on notation5 Whe three perio!s +ollow a arg"met i the !escriptio o+ a
comma! Eas a%o#eF, it meas that the arg"met ca %e repeate!, th"s,
mkdir dir1
wo"l! create a sigle !irectory ame! ?!ir.@, while
mkdir dir1 dir2 dir
wo"l! create three !irectories ame! ?!ir.@, ?!ir2@, a! ?!ir(@.
cp Cop, ;iles #nd Directories
The c7 comma! copies +iles or !irectories. 9t ca %e "se! two !i++eret ways,
cp item1 item2
to copy the sigle +ile or !irectory ?item.@ to +ile or !irectory ?item2@ a!,
cp item... directory
to copy m"ltiple items Eeither +iles or !irectoriesF ito a !irectory.
(1
cp 8 Copy 0iles $! 'irectories
Use=u <5tions And 6xam5es
Aere are some o+ the commoly "se! optios Ethe short optio a! the e="i#alet log
optioF +or c7,
Table .-$: cp &ptions
?ption >eaning
-a> --archive
Copy the +iles a! !irectories a! all o+ their attri%"tes,
icl"!ig owerships a! permissios. &ormally,
copies take o the !e+a"lt attri%"tes o+ the "ser
per+ormig the copy.
-i> --interactive
Je+ore o#erwritig a e3istig +ile, prompt the "ser +or
co+irmatio. )" this o(tion is not s(eci"ied6 cp +ill
silently over+rite "iles.
-r> --recursive
:ec"rsi#ely copy !irectories a! their cotets. This
optio Eor the -a optioF is re="ire! whe copyig
!irectories.
-u> --u7date
Whe copyig +iles +rom oe !irectory to aother, oly
copy +iles that either !o<t e3ist, or are ewer tha the
e3istig correspo!ig +iles, i the !estiatio
!irectory.
-v> --verbose
'isplay i+ormati#e messages as the copy is
per+orme!.
Table .-.: cp 1+amples
Command 1esults
c7 file1 file2
Copy file1 to file'. )" file2 exists6 it is over+ritten
+ith the contents o" file1. 9+ file' !oes ot e3ist, it
is create!.
c7 -i file1 file2
Same as a%o#e, e3cept that i+ file' e3ists, the "ser is
prompte! %e+ore it is o#erwritte.
c7 file1 file2 dir1
Copy file1 a! file' ito !irectory dir1. dir1 m"st
alrea!y e3ist.
c7 dir#6E dir!
)sig a wil!car!, all the +iles i dir1 are copie!
ito dir'. dir' m"st alrea!y e3ist.
(2
2 8 >aip"latig 0iles $! 'irectories
c7 -r dir1 dir2
Copy the cotets o+ !irectory dir1 to !irectory
dir'. 9+ !irectory dir' !oes ot e3ist, it is create!
a!, a+ter the copy, will cotai the same cotets
as !irectory dir1.
9+ !irectory dir' !oes e3ist, the !irectory dir1 Ea!
its cotetsF will %e copie! ito dir'.
mv >o%e #nd 1ename ;iles
The mv comma! per+orms %oth +ile mo#ig a! +ile reamig, !epe!ig o how it is
"se!. 9 either case, the origial +ileame o loger e3ists a+ter the operatio. mv is "se!
i m"ch the same way as c7,
mv item1 item2
to mo#e or reame +ile or !irectory ?item.@ to ?item2@ or,
mv item... directory
to mo#e oe or more items +rom oe !irectory to aother.
Use=u <5tions And 6xam5es
mv shares may o+ the same optios as c7,
Table .-2: mv &ptions
?ption >eaning
-i> --interactive
Je+ore o#erwritig a e3istig +ile, prompt the "ser +or
co+irmatio. )" this o(tion is not s(eci"ied6 mv +ill
silently over+rite "iles.
-u> --u7date
Whe mo#ig +iles +rom oe !irectory to aother, oly
mo#e +iles that either !o<t e3ist, or are ewer tha the
e3istig correspo!ig +iles i the !estiatio
!irectory.
-v> --verbose
'isplay i+ormati#e messages as the mo#e is
(K
m# 8 >o#e $! :eame 0iles
per+orme!.
Table .-3: mv 1+amples
Command 1esults
mv file1 file2
>o#e file1 to file'. )" file2 exists6 it is over+ritten
+ith the contents o" file1. 9+ file' !oes ot e3ist, it
is create!. )n either case6 file1 ceases to exist.
mv -i file1 file2
Same as a%o#e, e3cept that i+ file' e3ists, the "ser is
prompte! %e+ore it is o#erwritte.
mv file1 file2 dir1
>o#e file1 a! file' ito !irectory dir1. dir1 m"st
alrea!y e3ist.
mv dir1 dir2
9+ !irectory dir' !oes ot e3ist, create !irectory
dir' a! mo#e the cotets o+ !irectory dir1 ito
dir' a! !elete !irectory dir1.
9+ !irectory dir' !oes e3ist, mo#e !irectory dir1
Ea! its cotetsF ito !irectory dir'.
rm 1emo%e ;iles #nd Directories
The rm comma! is "se! to remo#e E!eleteF +iles a! !irectories,
rm item...
where ?item@ is oe or more +iles or !irectories.
Use=u <5tions And 6xam5es
Aere are some o+ the commo optios +or rm,
Table .-4: rm &ptions
?ption >eaning
-i> --interactive
Je+ore !eletig a e3istig +ile, prompt the "ser +or
co+irmatio. )" this o(tion is not s(eci"ied6 rm +ill
silently delete "iles.
(/
2 8 >aip"latig 0iles $! 'irectories
-r> --recursive
:ec"rsi#ely !elete !irectories. This meas that i+ a
!irectory %eig !elete! has s"%!irectories, !elete them
too. To !elete a !irectory, this optio m"st %e
speci+ie!.
-f> --force
9gore oe3istet +iles a! !o ot prompt. This
o#erri!es the --interactive optio.
-v> --verbose
'isplay i+ormati#e messages as the !eletio is
per+orme!.
Table .-5: rm 1+amples
Command 1esults
rm file1
'elete file1 siletly.
rm -i file1
Same as a%o#e, e3cept that the "ser is prompte! +or
co+irmatio %e+ore the !eletio is per+orme!.
rm -r file1 dir1
'elete file1 a! dir1 a! its cotets.
rm -rf file1 dir1
Same as a%o#e, e3cept that i+ either file1 or dir1 !o
ot e3ist, rm will coti"e siletly.
/e Care"ul +ith rm<
)i3-like operatig systems s"ch as *i"3 !o ot ha#e a "!elete comma!.
7ce yo" !elete somethig with rm, it<s goe. *i"3 ass"mes yo"<re smart a!
yo" kow what yo"<re !oig.
Je partic"larly care+"l with wil!car!s. Cosi!er this classic e3ample. *et<s say
yo" wat to !elete 4"st the AT>* +iles i a !irectory. To !o this, yo" type,
rm E<html
which is correct, %"t i+ yo" acci!etally place a space %etwee the ?U@ a! the
?.html@ like so,
rm E <html
the rm comma! will !elete all the +iles i the !irectory a! the complai that
there is o +ile calle! ?.html@.
(8
rm 8 :emo#e 0iles $! 'irectories
Here is a use"ul ti(. Whee#er yo" "se wil!car!s with rm E%esi!es care+"lly
checkig yo"r typigLF, test the wil!car! +irst with ls. This will let yo" see the
+iles that will %e !elete!. The press the "p arrow key to recall the comma! a!
replace the ls with rm.
ln Create Lin(s
The ln comma! is "se! to create either har! or sym%olic liks. 9t is "se! i oe o+ two
ways,
ln file link
to create a har! lik, a!,
ln -s item link
to create a sym%olic lik where ?item@ is either a +ile or a !irectory.
$ard Links
Aar! liks are the origial )i3 way o+ creatig liks, compare! to sym%olic liks, which
are more mo!er. Jy !e+a"lt, e#ery +ile has a sigle har! lik that gi#es the +ile its ame.
Whe we create a har! lik, we create a a!!itioal !irectory etry +or a +ile. Aar! liks
ha#e two importat limitatios,
.. $ har! lik caot re+erece a +ile o"tsi!e its ow +ile system. This meas a lik
may ot re+erece a +ile that is ot o the same !isk partitio as the lik itsel+.
2. $ har! lik may ot re+erece a !irectory.
$ har! lik is i!istig"isha%le +rom the +ile itsel+. )like a sym%olic lik, whe yo" list
a !irectory cotaiig a har! lik yo" will see o special i!icatio o+ the lik. Whe a
har! lik is !elete!, the lik is remo#e! %"t the cotets o+ the +ile itsel+ coti"e to e3ist
Ethat is, its space is ot !eallocate!F "til all liks to the +ile are !elete!.
9t is importat to %e aware o+ har! liks %eca"se yo" might eco"ter them +rom time to
time, %"t mo!er practice pre+ers sym%olic liks, which we will co#er e3t.
(9
2 8 >aip"latig 0iles $! 'irectories
Symboi/ Links
Sym%olic liks were create! to o#ercome the limitatios o+ har! liks. Sym%olic liks
work %y creatig a special type o+ +ile that cotais a te3t poiter to the re+erece! +ile or
!irectory. 9 this regar!, they operate i m"ch the same way as a Wi!ows shortc"t
tho"gh o+ co"rse, they pre!ate the Wi!ows +eat"re %y may years H-F
$ +ile poite! to %y a sym%olic lik, a! the sym%olic lik itsel+ are largely
i!istig"isha%le +rom oe aother. 0or e3ample, i+ yo" write some somethig to the
sym%olic lik, the re+erece! +ile is also writte to. Aowe#er whe yo" !elete a sym%olic
lik, oly the lik is !elete!, ot the +ile itsel+. 9+ the +ile is !elete! %e+ore the sym%olic
lik, the lik will coti"e to e3ist, %"t will poit to othig. 9 this case, the lik is sai!
to %e broken. 9 may implemetatios, the ls comma! will !isplay %roke liks i a
!istig"ishig color, s"ch as re!, to re#eal their presece.
The cocept o+ liks ca seem #ery co+"sig, %"t hag i there. We<re goig to try all
this st"++ a! it will, hope+"lly, %ecome clear.
Let2s /uild # Pla,ground
Sice we are goig to !o some real +ile maip"latio, let<s %"il! a sa+e place to ?play@
with o"r +ile maip"latio comma!s. 0irst we ee! a !irectory to work i. We<ll create
oe i o"r home !irectory a! call it ?playgro"!.@
Creating )ire/tories
The mkdir comma! is "se! to create a !irectory. To create o"r playgro"! !irectory
we will +irst make s"re we are i o"r home !irectory a! will the create the ew
!irectory,
[me@linuxbox ~]$ cd
[me@linuxbox ~]$ mkdir pla!ground
To make o"r playgro"! a little more iterestig, let<s create a co"ple o+ !irectories isi!e
it calle! ?!ir.@ a! ?!ir2@. To !o this, we will chage o"r c"rret workig !irectory to
7la1ground a! e3ec"te aother mkdir,
[me@linuxbox ~]$ cd pla!ground
[me@linuxbox 7la1ground]$ mkdir dir1 dir2
&otice that the mkdir comma! will accept m"ltiple arg"mets allowig "s to create
10
*et<s J"il! $ 6laygro"!
%oth !irectories with a sigle comma!.
Co5ying 2ies
&e3t, let<s get some !ata ito o"r playgro"!. We<ll !o this %y copyig a +ile. )sig the
c7 comma!, we<ll copy the 7ass9d +ile +rom the 6etc !irectory to the c"rret
workig !irectory,
[me@linuxbox 7la1ground]$ cp /etc/passwd .
&otice how we "se! the shortha! +or the c"rret workig !irectory, the sigle trailig
perio!. So ow i+ we per+orm a ls, we will see o"r +ile,
[me@linuxbox 7la1ground]$ ls -l
total #!
dr9xr9xr-x ! me me %(0. !((/-(#-#( #.:%( dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#( #.:%( dir!
-r9-r--r-- # me me #."( !((/-(#-#( #.:() 7ass9d
&ow, 4"st +or +", let<s repeat the copy "sig the ?-#@ optio E#er%oseF to see what it !oes,
[me@linuxbox 7la1ground]$ cp -v /etc/passwd .
J6etc67ass9dK -D J<67ass9dK
The c7 comma! per+orme! the copy agai, %"t this time !isplaye! a cocise message
i!icatig what operatio it was per+ormig. &otice that c7 o#erwrote the +irst copy
witho"t ay warig. $gai this is a case o+ c7 ass"mig that yo" kow what yo"Ire are
!oig. To get a warig, we<ll icl"!e the ?-i@ Eiteracti#eF optio,
[me@linuxbox 7la1ground]$ cp -i /etc/passwd .
c7: over9rite J<67ass9dKF
:espo!ig to the prompt %y eterig a ?y@ will ca"se the +ile to %e o#erwritte, ay
other character E+or e3ample, ?@F will ca"se c7 to lea#e the +ile aloe.
1.
2 8 >aip"latig 0iles $! 'irectories
:o;ing And !enaming 2ies
&ow, the ame ?passw!@ !oes<t seem #ery play+"l a! this is a playgro"!, so let<s
chage it to somethig else,
[me@linuxbox 7la1ground]$ mv passwd fun
*et<s pass the +" aro"! a little %y mo#ig o"r reame! +ile to each o+ the !irectories a!
%ack agai,
[me@linuxbox 7la1ground]$ mv fun dir1
to mo#e it +irst to !irectory dir#, the,
[me@linuxbox 7la1ground]$ mv dir1/fun dir2
to mo#e it +rom dir# to dir!, the,
[me@linuxbox 7la1ground]$ mv dir2/fun .
to +ially %rigig it %ack to the c"rret workig !irectory. &e3t, let<s see the e++ect o+ mv
o !irectories. 0irst we will mo#e o"r !ata +ile ito dir# agai,
[me@linuxbox 7la1ground]$ mv fun dir1
the mo#e dir# ito dir! a! co+irm it with ls,
[me@linuxbox 7la1ground]$ mv dir1 dir2
[me@linuxbox 7la1ground]$ ls -l dir2
total %
dr9xr9xr-x ! me me %(0. !((/-(#-## (.:(. dir#
[me@linuxbox 7la1ground]$ ls -l dir2/dir1
total %
-r9-r--r-- # me me #."( !((/-(#-#( #.:$$ fun
12
*et<s J"il! $ 6laygro"!
&ote that sice dir! alrea!y e3iste!, mv mo#e! dir# ito dir!. 9+ dir! ha! ot
e3iste!, mv wo"l! ha#e reame! dir# to dir!. *astly, let<s p"t e#erythig %ack,
[me@linuxbox 7la1ground]$ mv dir2/dir1 .
[me@linuxbox 7la1ground]$ mv dir1/fun .
Creating $ard Links
&ow we<ll try some liks. 0irst the har! liks. WeIll create some liks to o"r !ata +ile
like so,
[me@linuxbox 7la1ground]$ ln fun fun-hard
[me@linuxbox 7la1ground]$ ln fun dir1/fun-hard
[me@linuxbox 7la1ground]$ ln fun dir2/fun-hard
So ow we ha#e +o"r istaces o+ the +ile ?+"@. *et<s take a look o"r playgro"!
!irectory,
[me@linuxbox 7la1ground]$ ls -l
total #.
dr9xr9xr-x ! me me %(0. !((/-(#-#% #.:#) dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#% #.:#) dir!
-r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun
-r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun-hard
7e thig yo" otice is that the seco! +iel! i the listig +or fun a! fun-hard %oth
cotai a ?1@ which is the "m%er o+ har! liks that ow e3ist +or the +ile. 5o"<ll
remem%er that a +ile will aways ha#e at least oe %eca"se the +ile<s ame is create! %y a
lik. So, how !o we kow that fun a! fun-hard are, i +act, the same +ileC 9 this
case, ls is ot #ery help+"l. While we ca see that fun a! fun-hard are %oth the
same siDe E+iel! 2F, o"r listig pro#i!es o way to %e s"re. To sol#e this pro%lem, we<re
goig to ha#e to !ig a little !eeper.
Whe thikig a%o"t har! liks, it is help+"l to imagie that +iles are ma!e "p o+ two
parts, the !ata part cotaiig the +ile<s cotets a! the ame part which hol!s the +ile<s
ame. Whe we create har! liks, we are act"ally creatig a!!itioal ame parts that all
re+er to the same !ata part. The system assigs a chai o+ !isk %locks to what is calle! a
inode, which is the associate! with the ame part. Each har! lik there+ore re+ers to a
speci+ic io!e cotaiig the +ile<s cotets.
1(
2 8 >aip"latig 0iles $! 'irectories
The ls comma! has a way to re#eal this i+ormatio. 9t is i#oke! with the ?-i@ optio,
[me@linuxbox 7la1ground]$ ls -li
total #.
#!$"$"$0 dr9xr9xr-x ! me me %(0. !((/-(#-#% #.:#) dir#
#!$"$"%( dr9xr9xr-x ! me me %(0. !((/-(#-#% #.:#) dir!
#!$"$"$/ -r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun
#!$"$"$/ -r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun-hard
9 this #ersio o+ the listig, the +irst +iel! is the io!e "m%er a!, as we ca see, %oth
fun a! fun-hard share the same io!e "m%er, which co+irms they are the same
+ile.
Creating Symboi/ Links
Sym%olic liks were create! to o#ercome the two !isa!#atages o+ har! liks, har! liks
caot spa physical !e#ices a! har! liks caot re+erece !irectories, oly +iles.
Sym%olic liks are a special type o+ +ile that cotais a te3t poiter to the target +ile or
!irectory.
Creatig sym%olic liks is similar to creatig har! liks,
[me@linuxbox 7la1ground]$ ln -s fun fun-s!m
[me@linuxbox 7la1ground]$ ln -s ../fun dir1/fun-s!m
[me@linuxbox 7la1ground]$ ln -s ../fun dir2/fun-s!m
The +irst e3ample is pretty straight+orwar!, we simply a!! the ?-s@ optio to create a
sym%olic lik rather tha a har! lik. J"t what a%o"t the e3t twoC :emem%er, whe we
create a sym%olic lik, we are creatig a te3t !escriptio o+ where the target +ile is
relati#e to the sym%olic lik. 9t<s easier to see i+ we look at the ls o"tp"t,
[me@linuxbox 7la1ground]$ ls -l dir1
total %
-r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun-hard
lr9xr9xr9x # me me . !((/-(#-#" #":#) fun-s1m -D <<6fun
The listig +or fun-s1m i dir# shows that is it a sym%olic lik %y the lea!ig ?l@ i
the +irst +iel! a! that it poits to ?..-+"@, which is correct. :elati#e to the locatio o+
fun-s1m, fun is i the !irectory a%o#e it. &otice too, that the legth o+ the sym%olic
lik +ile is K, the "m%er o+ characters i the strig ?..-+"@ rather tha the legth o+ the
11
*et<s J"il! $ 6laygro"!
+ile to which it is poitig.
Whe creatig sym%olic liks, yo" ca either "se a%sol"te pathames,
ln -s /home/me/pla!ground/fun dir1/fun-s!m
or relati#e pathames, as we !i! i o"r earlier e3ample. )sig relati#e pathames is
more !esira%le %eca"se it allows a !irectory cotaiig sym%olic liks to %e reame!
a!-or mo#e! witho"t %reakig the liks.
9 a!!itio to reg"lar +iles, sym%olic liks ca also re+erece !irectories,
[me@linuxbox 7la1ground]$ ln -s dir1 dir1-s!m
[me@linuxbox 7la1ground]$ ls -l
total #.
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir#
lr9xr9xr9x # me me % !((/-(#-#. #%:%" dir#-s1m -D dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir!
-r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun
-r9-r--r-- % me me #."( !((/-(#-#( #.:$$ fun-hard
lr9xr9xr9x # me me $ !((/-(#-#" #":#" fun-s1m -D fun
!emo;ing 2ies And )ire/tories
$s we co#ere! earlier, the rm comma! is "se! to !elete +iles a! !irectories. We are
goig to "se it to clea "p o"r playgro"! a little %it. 0irst, let<s !elete oe o+ o"r har!
liks,
[me@linuxbox 7la1ground]$ rm fun-hard
[me@linuxbox 7la1ground]$ ls -l
total #!
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir#
lr9xr9xr9x # me me % !((/-(#-#. #%:%" dir#-s1m -D dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir!
-r9-r--r-- $ me me #."( !((/-(#-#( #.:$$ fun
lr9xr9xr9x # me me $ !((/-(#-#" #":#" fun-s1m -D fun
That worke! as e3pecte!. The +ile fun-hard is goe a! the lik co"t show +or fun
is re!"ce! +rom +o"r to three, as i!icate! i the seco! +iel! o+ the !irectory listig.
&e3t, we<ll !elete the +ile fun, a! 4"st +or e4oymet, we<ll icl"!e the ?-i@ optio to
show what that !oes,
12
2 8 >aip"latig 0iles $! 'irectories
[me@linuxbox 7la1ground]$ rm -i fun
rm: remove regular file JfunKF
Eter ?y@ at the prompt a! the +ile is !elete!. J"t let<s look at the o"tp"t o+ ls ow.
&otice! what happee! to fun-s1mC Sice it<s a sym%olic lik poitig to a ow-
oe3istet +ile, the lik is broken,
[me@linuxbox 7la1ground]$ ls -l
total /
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir#
lr9xr9xr9x # me me % !((/-(#-#. #%:%" dir#-s1m -D dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir!
lr9xr9xr9x # me me $ !((/-(#-#" #":#" fun-s1m -D fun
>ost *i"3 !istri%"tios co+ig"re ls to !isplay %roke liks. 7 a 0e!ora %o3, %roke
liks are !isplaye! i %likig re! te3tL The presece o+ a %roke lik is ot, i a! o+
itsel+ !agero"s %"t it is rather messy. 9+ we try to "se a %roke lik we will see this,
[me@linuxbox 7la1ground]$ less fun-s!m
fun-s1m: Lo such file or director1
*et<s clea "p a little. We<ll !elete the sym%olic liks,
[me@linuxbox 7la1ground]$ rm fun-s!m dir1-s!m
[me@linuxbox 7la1ground]$ ls -l
total /
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir#
dr9xr9xr-x ! me me %(0. !((/-(#-#" #":#) dir!
7e thig to remem%er a%o"t sym%olic liks is that most +ile operatios are carrie! o"t
o the lik<s target, ot the lik itsel+. rm is a e3ceptio. Whe yo" !elete a lik, it is
the lik that is !elete!, ot the target.
0ially, we will remo#e o"r playgro"!. To !o this, we will ret"r to o"r home !irectory
a! "se rm with the rec"rsi#e optio E-rF to !elete playgro"! a! all o+ its cotets,
icl"!ig its s"%!irectories,
[me@linuxbox 7la1ground]$ cd
1K
*et<s J"il! $ 6laygro"!
[me@linuxbox ~]$ rm -r pla!ground
Creating S,mlin(s +ith The 6-I
The +ile maagers i %oth ;&7>E a! M'E pro#i!e a easy a! a"tomatic
metho! o+ creatig sym%olic liks. With ;&7>E, hol!ig the CtrlVShi+t keys
while !raggig a +ile will create a lik rather tha copyig Eor mo#igF the +ile.
9 M'E, a small me" appears whee#er a +ile is !roppe!, o++erig a choice o+
copyig, mo#ig, or likig the +ile.
Summing U5
We<#e co#ere! a lot o+ gro"! here a! it will take a while to +"lly sik i. 6er+orm the
playgro"! e3ercise o#er a! o#er "til it makes sese. 9t is importat to get a goo!
"!ersta!ig o+ %asic +ile maip"latio comma!s a! wil!car!s. 0eel +ree to e3pa!
o the playgro"! e3ercise %y a!!ig more +iles a! !irectories, "sig wil!car!s to
speci+y +iles +or #ario"s operatios. The cocept o+ liks is a little co+"sig at +irst, %"t
take the time to lear how they work. They ca %e a real li+esa#er.
1/
K 8 Workig With Comma!s
+ Wor,ing With Co%%ands
)p to this poit, we ha#e see a series o+ mysterio"s comma!s, each with its ow
mysterio"s optios a! arg"mets. 9 this chapter, we will attempt to remo#e some o+
that mystery a! e#e create some o+ o"r ow comma!s. The comma!s itro!"ce! i
this chapter are,
t17e 8 9!icate how a comma! ame is iterprete!
9hich 8 'isplay which e3ec"ta%le program will %e e3ec"te!
man 8 'isplay a comma!<s ma"al page
a7ro7os 8 'isplay a list o+ appropriate comma!s
info 8 'isplay a comma!<s i+o etry
9hatis 8 'isplay a #ery %rie+ !escriptio o+ a comma!
alias 8 Create a alias +or a comma!
+hat $xactl, #re Commands.
$ comma! ca %e oe o+ +o"r !i++eret thigs,
.. #n executable (rogram like all those +iles we saw i 6usr6bin. Withi this
category, programs ca %e compiled binaries s"ch as programs writte i C a!
CVV, or programs writte i scripting languages s"ch as the shell, perl, pytho,
r"%y, etc.
2. # command built into the shell itsel". bash s"pports a "m%er o+ comma!s
iterally calle! shell builtins. The cd comma!, +or e3ample, is a shell %"ilti.
(. # shell "unction. These are miiat"re shell scripts icorporate! ito the
environment. We will co#er co+ig"rig the e#iromet a! writig shell
+"ctios i later chapters, %"t +or ow, 4"st %e aware that they e3ist.
1. #n alias. Comma!s that we ca !e+ie o"rsel#es, %"ilt +rom other comma!s.
18
9!eti+yig Comma!s
Identi",ing Commands
9t is o+te "se+"l to kow e3actly which o+ the +o"r ki!s o+ comma!s is %eig "se! a!
*i"3 pro#i!es a co"ple o+ ways to +i! o"t.
t17e A )is5ay A Command"s Ty5e
The t17e comma! is a shell %"ilti that !isplays the ki! o+ comma! the shell will
e3ec"te, gi#e a partic"lar comma! ame. 9t works like this,
t!pe command
where ?comma!@ is the ame o+ the comma! yo" wat to e3amie. Aere are some
e3amples,
[me@linuxbox ~]$ t!pe t!pe
t17e is a shell builtin
[me@linuxbox ~]$ t!pe ls
ls is aliased to Jls --colorMtt1K
[me@linuxbox ~]$ t!pe cp
c7 is 6bin6c7
Aere we see the res"lts +or three !i++eret comma!s. &otice that the oe +or ls Etake
+rom a 0e!ora systemF a! how the ls comma! is act"ally a alias +or the ls comma!
with the ?-- colorWtty@ optio a!!e!. &ow we kow why the o"tp"t +rom ls is !isplaye!
i colorL
9hich A )is5ay An 6xe/utabe"s Lo/ation
Sometimes there is more tha oe #ersio o+ a e3ec"ta%le program istalle! o a
system. While this is ot #ery commo o !esktop systems, it<s ot ""s"al o large
ser#ers. To !etermie the e3act locatio o+ a gi#e e3ec"ta%le, the 9hich comma! is
"se!,
[me@linuxbox ~]$ which ls
6bin6ls
9hich oly works +or e3ec"ta%le programs, ot %"iltis or aliases that are s"%stit"tes
+or act"al e3ec"ta%le programs. Whe we try to "se 9hich o a shell %"ilti, +or
19
K 8 Workig With Comma!s
e3ample, cd, we either get o respose or a error message,
[me@linuxbox ~]$ which cd
6usr6bin69hich: no cd in
N6o7t6jre#<.<(O($6bin:6usr6lib6Bt-$<$6bin:6usr6kerberos6bin:6o7t6jre#
<.<(O($6bin:6usr6lib6ccache:6usr6local6bin:6usr6bin:6bin:6home6me6bin
P
which is a +acy way o+ sayig ?comma! ot +o"!.@
6etting # Command2s Documentation
With this kowle!ge o+ what a comma! is, we ca ow search +or the !oc"metatio
a#aila%le +or each ki! o+ comma!.
hel7 A +et $e5 2or She Buitins
bash has a %"ilt-i help +acility a#aila%le +or each o+ the shell %"iltis. To "se it, type
?help@ +ollowe! %y the ame o+ the shell %"ilti. 0or e3ample,
[me@linuxbox ~]$ help cd
cd: cd [-=Q-:] [dir]
Change the current director1 to 'AR< The variable $S+& is the
default 'AR< The variable C':4TS defines the search 7ath for the
director1 containing 'AR< 4lternative director1 names in C':4TS are
se7arated b1 a colon N:P< 4 null director1 name is the same as the
current director1> i<e< J<K< Af 'AR begins 9ith a slash N6P> then
C':4TS is not used< Af the director1 is not found> and the shell
o7tion JcdableOvarsK is set> then tr1 the 9ord as a variable name<
Af that variable has a value> then cd to the value of that variable<
The -: o7tion sa1s to use the 7h1sical director1 structure instead of
follo9ing s1mbolic linksT the -= o7tion forces s1mbolic links to be
follo9ed<
# note on notation5 Whe s="are %rackets appear i the !escriptio o+ a comma!<s
syta3, they i!icate optioal items. $ #ertical %ar character i!icates m"t"ally e3cl"si#e
items. 9 the case o+ the cd comma! a%o#e,
cd [-=Q-:] [dir]
This otatio says that the comma! cd may %e +ollowe! optioally %y either a ?-*@ or a
?-6@ a! +"rther, optioally +ollowe! %y the arg"met ?!ir@.
While the o"tp"t o+ hel7 +or the cd comma!s is cocise a! acc"rate, it is %y o
20
;ettig $ Comma!<s 'oc"metatio
meas t"torial a! as we ca see, it also seems to metio a lot o+ thigs we ha#e<t
talke! a%o"t yetL 'o<t worry. We<ll get there.
--hel7 A )is5ay Usage In=ormation
>ay e3ec"ta%le programs s"pport a ?--help@ optio that !isplays a !escriptio o+ the
comma!<s s"pporte! syta3 a! optios. 0or e3ample,
[me@linuxbox ~]$ mkdir --help
3sage: mkdir [:TAL] 'AR&CTRU<<<
Create the 'AR&CTRUNiesP> if the1 do not alread1 exist<
-I> --contextMCLT&VT N*&=inuxP set securit1 context to CLT&VT
+andator1 arguments to long o7tions are mandator1 for short o7tions
too<
-m> --modeM+'& set file mode Nas in chmodP> not aMr9x W umask
-7> --7arents no error if existing> make 7arent directories as
needed
-v> --verbose 7rint a message for each created director1
--hel7 dis7la1 this hel7 and exit
--version out7ut version information and exit
Re7ort bugs to Xbug-coreutils@gnu<orgD<
Some programs !o<t s"pport the ?--help@ optio, %"t try it ayway. 7+te it res"lts i a
error message that will re#eal the same "sage i+ormatio.
man A )is5ay A 'rogram"s :anua 'age
>ost e3ec"ta%le programs ite!e! +or comma! lie "se pro#i!e a +ormal piece o+
!oc"metatio calle! a manual or man page. $ special pagig program calle! man is
"se! to #iew them. 9t is "se! like this,
man program
where ?program@ is the ame o+ the comma! to #iew.
>a pages #ary somewhat i +ormat %"t geerally cotai a title, a syopsis o+ the
comma!<s syta3, a !escriptio o+ the comma!<s p"rpose, a! a listig a! !escriptio
o+ each o+ the comma!<s optios. >a pages, howe#er, !o ot "s"ally icl"!e
e3amples, a! are ite!e! as a re+erece, ot a t"torial. $s a e3ample, let<s try #iewig
2.
K 8 Workig With Comma!s
the ma page +or the ls comma!,
[me@linuxbox ~]$ man ls
7 most *i"3 systems, man "ses less to !isplay the ma"al page, so all o+ the +amiliar
less comma!s work while !isplayig the page.
The ?ma"al@ that man !isplays is %roke ito sectios a! ot oly co#ers "ser
comma!s %"t also system a!miistratio comma!s, programmig iter+aces, +ile
+ormats a! more. The ta%le %elow !escri%es the layo"t o+ the ma"al,
Table 2-1: !an 6age &rgani7ation
Section Contents
. )ser comma!s
2 6rogrammig iter+aces kerel system calls
( 6rogrammig iter+aces to the C li%rary
1 Special +iles s"ch as !e#ice o!es a! !ri#ers
2 0ile +ormats
K ;ames a! am"semets s"ch as scree sa#ers
/ >iscellaeo"s
8 System a!miistratio comma!s
Sometimes we ee! to look i a speci+ic sectio o+ the ma"al to +i! what we are
lookig +or. This is partic"larly tr"e i+ we are lookig +or a +ile +ormat that is also the
ame o+ a comma!. Witho"t speci+yig a sectio "m%er, we will always get the +irst
istace o+ a match, pro%a%ly i sectio .. To speci+y a sectio "m%er, we "se man like
this,
man section search_term
0or e3ample,
[me@linuxbox ~]$ man " passwd
22
;ettig $ Comma!<s 'oc"metatio
This will !isplay the ma page !escri%ig the +ile +ormat o+ the 6etc67ass9d +ile.
a7ro7os A )is5ay A55ro5riate Commands
9t is also possi%le to search the list o+ ma pages +or possi%le matches %ase! o a search
term. 9t<s #ery cr"!e %"t sometimes help+"l. Aere is a e3ample o+ a search +or ma
pages "sig the search term ?+loppy@,
[me@linuxbox ~]$ apropos flopp!
createOflo771Odevices N/P - udev callout to create all 7ossible
flo771 device based on the C+* t17e
fdformat N/P - =o9-level formats a flo771 disk
flo771 N/P - format flo771 disks
gflo771 N#P - a sim7le flo771 formatter for the @L+&
mbadblocks N#P - tests a flo771 disk> and marks the bad
blocks in the -4T
mformat N#P - add an +*'* files1stem to a lo9-level
formatted flo771 disk
The +irst +iel! i each lie o+ o"tp"t is the ame o+ the ma page, the seco! +iel! shows
the sectio. &ote that the man comma! with the ?-k@ optio per+orms the e3act same
+"ctio as a7ro7os.
9hatis A )is5ay A ?ery Brie= )es/ri5tion <= A Command
The 9hatis program !isplays the ame a! a oe lie !escriptio o+ a ma page
matchig a speci+ie! keywor!,
[me@linuxbox ~]$ whatis ls
ls N#P - list director1 contents
The >ost /rutal >an Page ?" Them #ll
$s we ha#e see, the ma"al pages s"pplie! with *i"3 a! other )i3-like
systems are ite!e! as re+erece !oc"metatio a! ot as t"torials. >ay ma
pages are har! to rea!, %"t 9 thik that the gra! priDe +or !i++ic"lty has got to go
to the ma page +or bash. $s 9 was !oig my research +or this %ook, 9 ga#e it
care+"l re#iew to es"re that 9 was co#erig most o+ its topics. Whe prite!, it<s
2(
K 8 Workig With Comma!s
o#er eighty pages log a! e3tremely !ese, a! its str"ct"re makes a%sol"tely o
sese to a ew "ser.
7 the other ha!, it is #ery acc"rate a! cocise, as well as %eig e3tremely
complete. So check it o"t i+ yo" !are a! look +orwar! to the !ay whe yo" ca
rea! it a! it all makes sese.
info A )is5ay A 'rogram"s In=o 6ntry
The ;&) 6ro4ect pro#i!es a alterati#e to ma pages +or their programs, calle! ?i+o.@
9+o pages are !isplaye! with a rea!er program ame!, appropriately eo"gh, info.
9+o pages are hyperlinked m"ch like we% pages. Aere is a sample,
-ile: coreutils<info> Lode: ls invocation> Lext: dir invocation>
37: 'irector1 listing
#(<# JlsK: =ist director1 contents
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
The JlsK 7rogram lists information about files Nof an1 t17e>
including directoriesP< 7tions and file arguments can be intermixed
arbitraril1> as usual<
-or non-o7tion command-line arguments that are directories> b1
default JlsK lists the contents of directories> not recursivel1> and
omitting files 9ith names beginning 9ith J<K< -or other non-o7tion
arguments> b1 default JlsK lists just the filename< Af no non-o7tion
argument is s7ecified> JlsK o7erates on the current director1> acting
as if it had been invoked 9ith a single argument of J<K<
H1 default> the out7ut is sorted al7habeticall1> according to the
--CC-Anfo: Ncoreutils<info<gCPls invocation> .$ lines --To7----------
The info program rea!s info files, which are tree str"ct"re! ito i!i#i!"al nodes, each
cotaiig a sigle topic. 9+o +iles cotai hyperliks that ca mo#e yo" +rom o!e to
o!e. $ hyperlik ca %e i!eti+ie! %y its lea!ig asterisk, a! is acti#ate! %y placig the
c"rsor "po it a! pressig the eter key.
21
;ettig $ Comma!<s 'oc"metatio
To i#oke info, type ?i+o@ +ollowe! optioally %y the ame o+ a program. Jelow is a
ta%le o+ comma!s "se! to cotrol the rea!er while !isplayig a i+o page,
Table 2-': info %ommands
Command #ction
F
'isplay comma! help
:g37 or Hacks7ace 'isplay pre#io"s page
:g'n or *7ace 'isplay e3t page
n
&e3t - 'isplay the e3t o!e
7
6re#io"s - 'isplay the pre#io"s o!e
u
)p - 'isplay the paret o!e o+ the c"rretly !isplaye!
o!e, "s"ally a me".
&nter
0ollow the hyperlik at the c"rsor locatio
B
T"it
>ost o+ the comma! lie programs we ha#e !isc"sse! so +ar are part o+ the ;&)
6ro4ect<s ?core"tils@ package, so typig,
[me@linuxbox ~]$ info coreutils
will !isplay a me" page with hyperliks to each program cotaie! i the core"tils
package.
!6A):6 And <ther 'rogram )o/umentation 2ies
>ay so+tware packages istalle! o yo"r system ha#e !oc"metatio +iles resi!ig i
the 6usr6share6doc !irectory. >ost o+ these are store! i plai te3t +ormat a! ca
%e #iewe! with less. Some o+ the +iles are i AT>* +ormat a! ca %e #iewe! with a
we% %rowser. We may eco"ter some +iles e!ig with a ?.gD@ e3tesio. This
i!icates that they ha#e %ee compresse! with the gCi7 compressio program. The gDip
package icl"!es a special #ersio o+ less calle! Cless that will !isplay the cotets
o+ gDip-compresse! te3t +iles.
22
K 8 Workig With Comma!s
Creating :our ?4n Commands +ith alias
&ow +or o"r #ery +irst e3periece with programmigL We will create a comma! o+ o"r
ow "sig the alias comma!. J"t %e+ore we start, we ee! to re#eal a small
comma! lie trick. 9t<s possi%le to p"t more tha oe comma! o a lie %y separatig
each comma! with a semicolo character. 9t works like this,
command1T command2T command3<<<
Aere<s the e3ample we will "se,
[me@linuxbox ~]$ cd /usr# ls# cd -
bin games kerberos lib.% local share tm7
etc include lib libexec sbin src
6home6me
[me@linuxbox ~]$
$s we ca see, we ha#e com%ie! three comma!s o oe lie. 0irst we chage
!irectory to 6usr the list the !irectory a! +ially ret"r to the origial !irectory E%y
"sig Kcd -KF so we e! "p where we starte!. &ow let<s t"r this se="ece ito a ew
comma! "sig alias. The +irst thig we ha#e to !o is !ream "p a ame +or o"r ew
comma!. *et<s try ?test@. Je+ore we !o that, it wo"l! %e a goo! i!ea to +i! o"t i+ the
ame ?test@ is alrea!y %eig "se!. To +i! o"t, we ca "se the t17e comma! agai,
[me@linuxbox ~]$ t!pe test
test is a shell builtin
7opsL The ame ?test@ is alrea!y take. *et<s try ?+oo@,
[me@linuxbox ~]$ t!pe foo
bash: t17e: foo: not found
;reatL ?+oo@ is ot take. So let<s create o"r alias,
[me@linuxbox ~]$ alias foo$Kcd /usr# ls# cd -%
2K
Creatig 5o"r 7w Comma!s With alias
&otice the str"ct"re o+ this comma!,
alias name$%string%
$+ter the comma! ?alias@ we gi#e alias a ame +ollowe! imme!iately Eo whitespace
allowe!F %y a e="als sig, +ollowe! imme!iately %y a ="ote! strig cotaiig the
meaig to %e assige! to the ame. $+ter we !e+ie o"r alias, it ca %e "se! aywhere
the shell wo"l! e3pect a comma!. *et<s try it,
[me@linuxbox ~]$ foo
bin games kerberos lib.% local share tm7
etc include lib libexec sbin src
6home6me
[me@linuxbox ~]$
We ca also "se the t17e comma! agai to see o"r alias,
[me@linuxbox ~]$ t!pe foo
foo is aliased to Jcd 6usrT ls T cd -K
To remo#e a alias, the unalias comma! is "se!, like so,
[me@linuxbox ~]$ unalias foo
[me@linuxbox ~]$ t!pe foo
bash: t17e: foo: not found
While we p"rpose+"lly a#oi!e! amig o"r alias with a e3istig comma! ame, it is
ot "commo to !o so. This is o+te !oe to apply a commoly !esire! optio to each
i#ocatio o+ a commo comma!. 0or istace, we saw earlier how the ls comma! is
o+te aliase! to a!! color s"pport,
[me@linuxbox ~]$ t!pe ls
ls is aliased to Jls --colorMtt1K
To see all the aliases !e+ie! i the e#iromet, "se the alias comma! witho"t
arg"mets. Aere are some o+ the aliases !e+ie! %y !e+a"lt o a 0e!ora system. Try a!
2/
K 8 Workig With Comma!s
+ig"re o"t what they all !o,
[me@linuxbox ~]$ alias
alias l<MKls -d <E --colorMtt1K
alias llMKls -l --colorMtt1K
alias lsMKls --colorMtt1K
There is oe tiy pro%lem with !e+iig aliases o the comma! lie. They #aish whe
yo"r shell sessio e!s. 9 a later chapter, we will see how to a!! o"r ow aliases to the
+iles that esta%lish the e#iromet each time we log o, %"t +or ow, e4oy the +act that
we ha#e take o"r +irst, al%eit tiy, step ito the worl! o+ shell programmigL
1e%isiting ?ld ;riends
&ow that we ha#e leare! how to +i! the !oc"metatio +or comma!s, go a! look "p
the !oc"metatio +or all the comma!s we ha#e eco"tere! so +ar. St"!y what
a!!itioal optios are a#aila%le a! try them o"tL
;urther 1eading
There are may olie so"rces o+ !oc"metatio +or *i"3 a! the comma! lie. Aere
are some o+ the %est,
The 8ash 9eference !anual is a re+erece g"i!e to the bash shell. 9tIs still a
re+erece work %"t cotais e3amples a! is easier to rea! tha the bash ma
page.
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.html
The 8ash F): cotais aswers to +re="etly aske! ="estios regar!ig bash.
This list is aime! at iterme!iate to a!#ace! "sers, %"t cotais a lot o+ goo!
i+ormatio.
http,--mywiki.woole!ge.org-Jash0$T
The ;&) 6ro4ect pro#i!es e3tesi#e !oc"metatio +or its programs, which +orm
the core o+ the *i"3 comma! lie e3periece. 5o" ca see a complete list here,
http,--www.g".org-ma"al-ma"al.html
Wikipe!ia has a iterestig article o ma pages,
http,--e.wikipe!ia.org-wiki->aOpage
28
/ 8 :e!irectio
. /edirection
9 this lesso we are goig to "leash what may %e the coolest +eat"re o+ the comma!
lie. 9t<s calle! *;& redirection. The ?9-7@ sta!s +or input;output a! with this +acility
yo" ca re!irect the ip"t a! o"tp"t o+ comma!s to a! +rom +iles, as well as coect
m"ltiple comma!s together ito power+"l comma! pipelines. To show o++ this +acility,
we will itro!"ce the +ollowig comma!s,
cat - Cocateate +iles
sort - Sort lies o+ te3t
uniB - :eport or omit repeate! lies
gre7 - 6rit lies matchig a patter
9c - 6rit ewlie, wor!, a! %yte co"ts +or each +ile
head - 7"tp"t the +irst part o+ a +ile
tail - 7"tp"t the last part o+ a +ile
tee - :ea! +rom sta!ar! ip"t a! write to sta!ar! o"tp"t a! +iles
Standard Input@ ?utput@ #nd $rror
>ay o+ the programs that we ha#e "se! so +ar pro!"ce o"tp"t o+ some ki!. This o"tp"t
o+te cosists o+ two types. 0irst, we ha#e the program<s res"ltsH that is, the !ata the
program is !esige! to pro!"ce, a! seco!, we ha#e stat"s a! error messages that tell
"s how the program is gettig alog. 9+ we look at a comma! like ls, we ca see that it
!isplays its res"lts a! its error messages o the scree.
Meepig with the )i3 theme o+ ?e#erythig is a +ile,@ programs s"ch as ls act"ally se!
their res"lts to a special +ile calle! standard output Eo+te e3presse! as stdoutF a! their
stat"s messages to aother +ile calle! standard error EstderrF. Jy !e+a"lt, %oth sta!ar!
o"tp"t a! sta!ar! error are like! to the scree a! ot sa#e! ito a !isk +ile.
9 a!!itio, may programs take ip"t +rom a +acility calle! standard input EstdinF which
is, %y !e+a"lt, attache! to the key%oar!.
29
/ 8 :e!irectio
9-7 re!irectio allows "s to chage where o"tp"t goes a! where ip"t comes +rom.
&ormally, o"tp"t goes to the scree a! ip"t comes +rom the key%oar!, %"t with 9-7
re!irectio, we ca chage that.
1edirecting Standard ?utput
9-7 re!irectio allows "s to re!e+ie where sta!ar! o"tp"t goes. To re!irect sta!ar!
o"tp"t to aother +ile %esi!es the scree, we "se the ?X@ re!irectio operator +ollowe! %y
the ame o+ the +ile. Why wo"l! we wat to !o thisC 9t<s o+te "se+"l to store the o"tp"t
o+ a comma! i a +ile. 0or e3ample, we co"l! tell the shell to se! the o"tp"t o+ the ls
comma! to the +ile ls-out7ut<txt istea! o+ the scree,
[me@linuxbox ~]$ ls -l /usr/bin & ls-output.txt
Aere, we create! a log listig o+ the 6usr6bin !irectory a! set the res"lts to the +ile
ls-out7ut<txt. *et<s e3amie the re!irecte! o"tp"t o+ the comma!,
[me@linuxbox ~]$ ls -l ls-output.txt
-r9-r9-r-- # me me #.)/)/ !((/-(!-(# #":() ls-out7ut<txt
;oo!H a ice, large, te3t +ile. 9+ we look at the +ile with less, we will see that the +ile
ls-out7ut<txt !oes i!ee! cotai the res"lts +rom o"r ls comma!,
[me@linuxbox ~]$ less ls-output.txt
&ow, let<s repeat o"r re!irectio test, %"t this time with a twist. We<ll chage the ame o+
the !irectory to oe that !oes ot e3ist,
[me@linuxbox ~]$ ls -l /bin/usr & ls-output.txt
ls: cannot access 6bin6usr: Lo such file or director1
We recei#e! a error message. This makes sese sice we speci+ie! the o-e3istet
!irectory 6bin6usr, %"t why was the error message !isplaye! o the scree rather tha
%eig re!irecte! to the +ile ls-out7ut<txtC The aswer is that the ls program !oes
ot se! its error messages to sta!ar! o"tp"t. 9stea!, like most well-writte )i3
programs, it se!s its error messages to sta!ar! error. Sice we oly re!irecte! sta!ar!
o"tp"t a! ot sta!ar! error, the error message was still set to the scree. We<ll see
K0
:e!irectig Sta!ar! 7"tp"t
how to re!irect sta!ar! error i 4"st a mi"te, %"t +irst, let<s look at what happee! to o"r
o"tp"t +ile,
[me@linuxbox ~]$ ls -l ls-output.txt
-r9-r9-r-- # me me ( !((/-(!-(# #":(/ ls-out7ut<txt
The +ile ow has Dero legthL This is %eca"se, whe we re!irect o"tp"t with the ?X@
re!irectio operator, the !estiatio +ile is always rewritte +rom the %egiig. Sice
o"r ls comma! geerate! o res"lts a! oly a error message, the re!irectio
operatio starte! to rewrite the +ile a! the stoppe! %eca"se o+ the error, res"ltig i its
tr"catio. 9 +act, i+ we e#er ee! to act"ally tr"cate a +ile Eor create a ew, empty +ileF
we ca "se a trick like this,
[me@linuxbox ~]$ & ls-output.txt
Simply "sig the re!irectio operator with o comma! prece!ig it will tr"cate a
e3istig +ile or create a ew, empty +ile.
So, how ca we appe! re!irecte! o"tp"t to a +ile istea! o+ o#erwritig the +ile +rom the
%egiigC 0or that, we "se the ?XX@ re!irectio operator, like so,
[me@linuxbox ~]$ ls -l /usr/bin && ls-output.txt
)sig the ?XX@ operator will res"lt i the o"tp"t %eig appe!e! to the +ile. 9+ the +ile
!oes ot alrea!y e3ist, it is create! 4"st as tho"gh the ?X@ operator ha! %ee "se!. *et<s
p"t it to the test,
[me@linuxbox ~]$ ls -l /usr/bin && ls-output.txt
[me@linuxbox ~]$ ls -l /usr/bin && ls-output.txt
[me@linuxbox ~]$ ls -l /usr/bin && ls-output.txt
[me@linuxbox ~]$ ls -l ls-output.txt
-r9-r9-r-- # me me "($.$% !((/-(!-(# #":%" ls-out7ut<txt
We repeate! the comma! three times res"ltig i a o"tp"t +ile three times as large.
1edirecting Standard $rror
:e!irectig sta!ar! error lacks the ease o+ a !e!icate! re!irectio operator. To re!irect
K.
/ 8 :e!irectio
sta!ar! error we m"st re+er to its file descriptor. $ program ca pro!"ce o"tp"t o ay
o+ se#eral "m%ere! +ile streams. While we ha#e re+erre! to the +irst three o+ these +ile
streams as sta!ar! ip"t, o"tp"t a! error, the shell re+ereces them iterally as +ile
!escriptors Dero, oe a! two, respecti#ely. The shell pro#i!es a otatio +or re!irectig
+iles "sig the +ile !escriptor "m%er. Sice sta!ar! error is the same as +ile !escriptor
"m%er two, we ca re!irect sta!ar! error with this otatio,
[me@linuxbox ~]$ ls -l /bin/usr 2& ls-error.txt
The +ile !escriptor ?2@ is place! imme!iately %e+ore the re!irectio operator to per+orm
the re!irectio o+ sta!ar! error to the +ile ls-error<txt.
!edire/ting Standard <ut5ut And Standard 6rror To <ne 2ie
There are cases i which we may wish to capt"re all o+ the o"tp"t o+ a comma! to a
sigle +ile. To !o this, we m"st re!irect %oth sta!ar! o"tp"t a! sta!ar! error at the
same time. There are two ways to !o this. 0irst, the tra!itioal way, which works with
ol! #ersios o+ the shell,
[me@linuxbox ~]$ ls -l /bin/usr & ls-output.txt 2&'1
)sig this metho!, we per+orm two re!irectios. 0irst we re!irect sta!ar! o"tp"t to the
+ile ls-out7ut<txt a! the we re!irect +ile !escriptor two Esta!ar! errorF to +ile
!escriptor oe Esta!ar! o"tp"tF "sig the otatio !DY#.
1otice that the order o" the redirections is signi"icant. The re!irectio o+
sta!ar! error m"st always occ"r after re!irectig sta!ar! o"tp"t or it !oes<t
work. 9 the e3ample a%o#e,
Dls-out7ut<txt !DY#
re!irects sta!ar! error to the +ile ls-out7ut<txt, %"t i+ the or!er is chage! to
!DY# Dls-out7ut<txt
sta!ar! error is !irecte! to the scree.
:ecet #ersios o+ bash pro#i!e a seco!, more streamlie! metho! +or per+ormig this
K2
:e!irectig Sta!ar! Error
com%ie! re!irectio,
[me@linuxbox ~]$ ls -l /bin/usr '& ls-output.txt
9 this e3ample, we "se the sigle otatio YD to re!irect %oth sta!ar! o"tp"t a!
sta!ar! error to the +ile ls-out7ut<txt.
)is5osing <= Un%anted <ut5ut
Sometimes ?silece is gol!e,@ a! we !o<t wat o"tp"t +rom a comma!, we 4"st wat
to throw it away. This applies partic"larly to error a! stat"s messages. The system
pro#i!es a way to !o this %y re!irectig o"tp"t to a special +ile calle! ?-!e#-"ll@. This
+ile is a system !e#ice calle! a bit bucket which accepts ip"t a! !oes othig with it.
To s"ppress error messages +rom a comma!, we !o this,
[me@linuxbox ~]$ ls -l /bin/usr 2& /dev/null
/dev/null In -nix Culture
The %it %"cket is a aciet )i3 cocept a! !"e to its "i#ersality, has appeare!
i may parts o+ )i3 c"lt"re. Whe someoe says he-she is se!ig yo"r
commets to 6dev6null, ow yo" kow what it meas. 0or more e3amples,
see the Wikipe!ia article o ?-!e#-"ll@.
1edirecting Standard Input
)p to ow, we ha#e<t eco"tere! ay comma!s that make "se o+ sta!ar! ip"t
Eact"ally we ha#e, %"t weIll re#eal that s"rprise a little %it laterF, so we ee! to itro!"ce
oe.
cat A Con/atenate 2ies
The cat comma! rea!s oe or more +iles a! copies them to sta!ar! o"tp"t like so,
cat (file...)
K(
/ 8 :e!irectio
9 most cases, yo" ca thik o+ cat as %eig aalogo"s to the TU:& comma! i '7S.
5o" ca "se it to !isplay +iles witho"t pagig, +or e3ample,
[me@linuxbox ~]$ cat ls-output.txt
will !isplay the cotets o+ the +ile ls-out7ut<txt. cat is o+te "se! to !isplay
short te3t +iles. Sice cat ca accept more tha oe +ile as a arg"met, it ca also %e
"se! to 4oi +iles together. Say we ha#e !owloa!e! a large +ile that has %ee split ito
m"ltiple parts Em"ltime!ia +iles are o+te split this way o )SE&ETF, a! we wat to
4oi them %ack together. 9+ the +iles were ame!,
movie<m7eg<((# movie<m7eg<((! <<< movie<m7eg<(00
we co"l! 4oi them %ack together with this comma!,
cat movie.mpeg.** & movie.mpeg
Sice wil!car!s always e3pa! i sorte! or!er, the arg"mets will %e arrage! i the
correct or!er.
This is all well a! goo!, %"t what !oes this ha#e to !o with sta!ar! ip"tC &othig yet,
%"t let<s try somethig else. What happes i+ we type ?cat@ with o arg"mets,
[me@linuxbox ~]$ cat
&othig happes, it 4"st sits there like it<s h"g. 9t may seem that way, %"t it<s really
!oig e3actly what it<s s"ppose! to.
9+ cat is ot gi#e ay arg"mets, it rea!s +rom sta!ar! ip"t a! sice sta!ar! ip"t
is, %y !e+a"lt, attache! to the key%oar!, it<s waitig +or "s to type somethigL Try this,
[me@linuxbox ~]$ cat
+he ,uick brown fox jumped over the la-! dog.
&e3t, type a Ctrl-d Ei.e., hol! !ow the Ctrl key a! press ?!@F to tell cat that it has
reache! end of file EE70F o sta!ar! ip"t,
K1
:e!irectig Sta!ar! 9p"t
[me@linuxbox ~]$ cat
+he ,uick brown fox jumped over the la-! dog.
The Buick bro9n fox jum7ed over the laC1 dog<
9 the a%sece o+ +ileame arg"mets, cat copies sta!ar! ip"t to sta!ar! o"tp"t, so
we see o"r lie o+ te3t repeate!. We ca "se this %eha#ior to create short te3t +iles. *et<s
say that we wate! to create a +ile calle! ?laDyO!og.t3t@ cotaiig the te3t i o"r
e3ample. We wo"l! !o this,
[me@linuxbox ~]$ cat & la-!.dog.txt
+he ,uick brown fox jumped over the la-! dog.
Type the comma! +ollowe! %y the te3t we wat i to place i the +ile. :emem%er to
type Ctrl-d at the e!. )sig the comma! lie, we ha#e implemete! the worl!<s
!"m%est wor! processorL To see o"r res"lts, we ca "se cat to copy the +ile to st!o"t
agai,
[me@linuxbox ~]$ cat la-!.dog.txt
The Buick bro9n fox jum7ed over the laC1 dog<
&ow that we kow how cat accepts sta!ar! ip"t, i a!!itio to +ileame arg"mets,
let<s try re!irectig sta!ar! ip"t,
[me@linuxbox ~]$ cat / la-!.dog.txt
The Buick bro9n fox jum7ed over the laC1 dog<
)sig the ?Y@ re!irectio operator, we chage the so"rce o+ sta!ar! ip"t +rom the
key%oar! to the +ile laC1Odog<txt. We see that the res"lt is the same as passig a
sigle +ileame arg"met. This is ot partic"larly "se+"l compare! to passig a +ileame
arg"met, %"t it ser#es to !emostrate "sig a +ile as a so"rce o+ sta!ar! ip"t. 7ther
comma!s make %etter "se o+ sta!ar! ip"t, as we shall soo see.
Je+ore we mo#e o, check o"t the ma page +or cat, as it has se#eral iterestig optios.
Pipelines
The a%ility o+ comma!s to rea! !ata +rom sta!ar! ip"t a! se! to sta!ar! o"tp"t is
"tiliDe! %y a shell +eat"re calle! pipelines. )sig the pipe operator ?Z@ E#ertical %arF, the
K2
/ 8 :e!irectio
sta!ar! o"tp"t o+ oe comma! ca %e piped ito the sta!ar! ip"t o+ aother,
command1 Q command2
To +"lly !emostrate this, we are goig to ee! some comma!s. :emem%er how we
sai! there was oe we alrea!y kew that accepts sta!ar! ip"tC 9t<s less. We ca "se
less to !isplay, page-%y-page, the o"tp"t o+ ay comma! that se!s its res"lts to
sta!ar! o"tp"t,
[me@linuxbox ~]$ ls -l /usr/bin 0 less
This is e3tremely ha!yL )sig this techi="e, we ca co#eietly e3amie the o"tp"t
o+ ay comma! that pro!"ces sta!ar! o"tp"t.
2iters
6ipelies are o+te "se! to per+orm comple3 operatios o !ata. 9t is possi%le to p"t
se#eral comma!s together ito a pipelie. 0re="etly, the comma!s "se! this way are
re+erre! to as filters. 0ilters take ip"t, chage it somehow a! the o"tp"t it. The +irst
oe we will try is sort. 9magie we wate! to make a com%ie! list o+ all o+ the
e3ec"ta%le programs i 6bin a! 6usr6bin, p"t them i sorte! or!er a! #iew it,
[me@linuxbox ~]$ ls /bin /usr/bin 0 sort 0 less
Sice we speci+ie! two !irectories E6bin a! 6usr6binF, the o"tp"t o+ ls wo"l! ha#e
cosiste! o+ two sorte! lists, oe +or each !irectory. Jy icl"!ig sort i o"r pipelie,
we chage! the !ata to pro!"ce a sigle, sorte! list.
uni( B !e5ort <r <mit !e5eated Lines
The uniB comma! is o+te "se! i co4"ctio with sort. uniB accepts a sorte! list
o+ !ata +rom either sta!ar! ip"t or a sigle +ileame arg"met Esee the uniB ma page
+or !etailsF a!, %y !e+a"lt, remo#es ay !"plicates +rom the list. So, to make s"re o"r
list has o !"plicates Ethat is, ay programs o+ the same ame that appear i %oth the
6bin a! 6usr6bin !irectoriesF we will a!! uniB to o"r pipelie,
KK
6ipelies
[me@linuxbox ~]$ ls /bin /usr/bin 0 sort 0 uni, 0 less
9 this e3ample, we "se uniB to remo#e ay !"plicates +rom the o"tp"t o+ the sort
comma!. 9+ we wat to see the list o+ !"plicates istea!, we a!! the ?-!@ optio to
uniB like so,
[me@linuxbox ~]$ ls /bin /usr/bin 0 sort 0 uni, -d 0 less
9c A 'rint LineC WordC And Byte Counts
The 9c Ewor! co"tF comma! is "se! to !isplay the "m%er o+ lies, wor!s, a! %ytes
cotaie! i +iles. 0or e3ample,
[me@linuxbox ~]$ wc ls-output.txt
)0(! .%".. "($.$% ls-out7ut<txt
9 this case it prits o"t three "m%ers, lies, wor!s, a! %ytes cotaie! i ls-
out7ut<txt. *ike o"r pre#io"s comma!s, i+ e3ec"te! witho"t comma! lie
arg"mets, 9c accepts sta!ar! ip"t. The ?-l@ optio limits its o"tp"t to oly report
lies. $!!ig it to a pipelie is a ha!y way to co"t thigs. To see the "m%er o+
programs we ha#e i o"r sorte! list, we ca !o this,
[me@linuxbox ~]$ ls /bin /usr/bin 0 sort 0 uni, 0 wc -l
!)!/
gre7 A 'rint Lines :at/hing A 'attern
gre7 is a power+"l program "se! to +i! te3t patters withi +iles. 9t<s "se! like this,
gre7 pattern [file...]
Whe gre7 eco"ters a ?patter@ i the +ile, it prits o"t the lies cotaiig it. The
patters that gre7 ca match ca %e #ery comple3, %"t +or ow we will cocetrate o
simple te3t matches. We<ll co#er the a!#ace! patters, calle! regular e+pressions i a
K/
/ 8 :e!irectio
later chapter.
*et<s say we wat to +i! all the +iles i o"r list o+ programs that ha! the wor! ?Dip@
em%e!!e! i the ame. S"ch a search might gi#e "s a i!ea o+ some o+ the programs o
o"r system that ha! somethig to !o with +ile compressio. We wo"l! !o this,
[me@linuxbox ~]$ ls /bin /usr/bin 0 sort 0 uni, 0 grep -ip
bunCi7!
bCi7!
gunCi7
gCi7
unCi7
Ci7
Ci7cloak
Ci7gre7
Ci7info
Ci7note
Ci7s7lit
There are a co"ple o+ ha!y optios +or gre7, ?-i@ which ca"ses gre7 to igore case
whe per+ormig the search Eormally searches are case sesiti#eF a! ?-#@ which tells
gre7 to oly prit lies that !o ot match the patter.
head - tail A 'rint 2irst - Last 'art <= 2ies
Sometimes yo" !o<t wat all o+ the o"tp"t +rom a comma!. 5o" may oly wat the
+irst +ew lies or the last +ew lies. The head comma! prits the +irst te lies o+ a +ile
a! the tail comma! prits the last te lies. Jy !e+a"lt, %oth comma!s prit te
lies o+ te3t, %"t this ca %e a!4"ste! with the ?-@ optio,
[me@linuxbox ~]$ head -n " ls-output.txt
total $%$%0.
-r9xr-xr-x # root root $#$#. !(()-#!-(" (/:"/ [
-r9xr-xr-x # root root /!%( !(()-#!-(0 #$:$0 %##to77m
-r9xr-xr-x # root root ###!). !(()-##-!. #%:!) a!7
-r9xr-xr-x # root root !"$./ !((.-#(-(. !(:#. a"!dec
[me@linuxbox ~]$ tail -n " ls-output.txt
-r9xr-xr-x # root root "!$% !(()-(.-!) #(:". Cne9
-r9xr-xr-x # root root .0# !(("-(0-#( (%:!# Conetab!7ot<71
-r9-r--r-- # root root 0$( !(()-##-(# #!:!$ Conetab!7ot<71c
-r9-r--r-- # root root 0$( !(()-##-(# #!:!$ Conetab!7ot<71o
lr9xr9xr9x # root root . !((/-(#-$# (":!! Csoelim -D soelim
These ca %e "se! i pipelies as well,
K8
6ipelies
[me@linuxbox ~]$ ls /usr/bin 0 tail -n "
Cne9
Conetab!7ot<71
Conetab!7ot<71c
Conetab!7ot<71o
Csoelim
tail has a optio which allows yo" to #iew +iles i real-time. This is "se+"l +or
watchig the progress o+ log +iles as they are %eig writte. 9 the +ollowig e3ample, we
will look at the messages +ile i 6var6log. S"per"ser pri#ileges are re="ire! to !o
this o some *i"3 !istri%"tios, sice the 6var6log6messages +ile may cotai
sec"rity i+ormatio,
[me@linuxbox ~]$ tail -f /var/log/messages
-eb / #$:%(:(" t9in% dhclient: 'SC:4C2 from #0!<#./<#<#
-eb / #$:%(:(" t9in% dhclient: bound to #0!<#./<#<% -- rene9al in
#."! seconds<
-eb / #$:"":$! t9in% mountd[$0"$]: 6var6L-*v%6musicbox ex7orted to
both #0!<#./<#<(6!% and t9in)<localdomain in
#0!<#./<#<(6!%>t9in)<localdomain
-eb / #%:():$) t9in% dhclient: 'SC:R&Z3&*T on eth( to #0!<#./<#<#
7ort .)
-eb / #%:():$) t9in% dhclient: 'SC:4C2 from #0!<#./<#<#
-eb / #%:():$) t9in% dhclient: bound to #0!<#./<#<% -- rene9al in
#))# seconds<
-eb / #%:(0:". t9in% smartd[$%./]: 'evice: 6dev6hda> *+4RT
:refailure 4ttribute: / *eekOTimeO:erformance changed from !$) to !$.
-eb / #%:#(:$) t9in% mountd[$0"$]: 6var6L-*v%6musicbox ex7orted to
both #0!<#./<#<(6!% and t9in)<localdomain in
#0!<#./<#<(6!%>t9in)<localdomain
-eb / #%:!":() t9in% sshdN7amOunixP[!0!$%]: session o7ened for user
me b1 NuidM(P
-eb / #%:!":$. t9in% suN7amOunixP[!0!)0]: session o7ened for user
root b1 meNuidM"((P
)sig the ?-+@ optio, tail coti"es to moitor the +ile a! whe ew lies are
appe!e!, they imme!iately appear o the !isplay. This coti"es "til yo" type Ctrl-
c.
tee A !ead 2rom Stdin And <ut5ut To Stdout And 2ies
9 keepig with o"r pl"m%ig metaphor, *i"3 pro#i!es a comma! calle! tee which
creates a ?tee@ +ittig o o"r pipe. The tee program rea!s sta!ar! ip"t a! copies it to
%oth sta!ar! o"tp"t Eallowig the !ata to coti"e !ow the pipelieF a! to oe or more
K9
/ 8 :e!irectio
+iles. This is "se+"l +or capt"rig a pipelie<s cotets at a iterme!iate stage o+
processig. Aere we repeat oe o+ o"r earlier e3amples, this time icl"!ig tee to
capt"re the etire !irectory listig to the +ile ls<txt %e+ore gre7 +ilters the pipelie<s
cotets,
[me@linuxbox ~]$ ls /usr/bin 0 tee ls.txt 0 grep -ip
bunCi7!
bCi7!
gunCi7
gCi7
unCi7
Ci7
Ci7cloak
Ci7gre7
Ci7info
Ci7note
Ci7s7lit
Summing -p
$s always, check o"t the !oc"metatio o+ each o+ the comma!s we ha#e co#ere! i
this chapter. We ha#e oly see their most %asic "sage. They all ha#e a "m%er o+
iterestig optios. $s we gai *i"3 e3periece, we will see that the re!irectio +eat"re
o+ the comma! lie is e3tremely "se+"l +or sol#ig specialiDe! pro%lems. There are
may comma!s that make "se o+ sta!ar! ip"t a! o"tp"t, a! almost all comma!
lie programs "se sta!ar! error to !isplay their i+ormati#e messages.
Linux Is #0out Imagination
Whe 9 am aske! to e3plai the !i++erece %etwee Wi!ows a! *i"3, 9 o+te
"se a toy aalogy.
Wi!ows is like a ;ame Joy. 5o" go to the store a! %"y oe all shiy ew i
the %o3. 5o" take it home, t"r it o a! play with it. 6retty graphics, c"te
so"!s. $+ter a while tho"gh, yo" get tire! o+ the game that came with it so yo"
go %ack to the store a! %"y aother oe. This cycle repeats o#er a! o#er.
0ially, yo" go %ack to the store a! say to the perso %ehi! the co"ter, ?9 wat
a game that !oes thisL@ oly to %e tol! that o s"ch game e3ists %eca"se there is
o ?market !ema!@ +or it. The yo" say, ?J"t 9 oly ee! to chage this oe
thigL@ The perso %ehi! the co"ter says yo" ca<t chage it. The games are
/0
S"mmig )p
all seale! "p i their cartri!ges. 5o" !isco#er that yo"r toy is limite! to the games
that others ha#e !eci!e! that yo" ee! a! o more.
*i"3, o the other ha!, is like the worl!<s largest Erector Set. 5o" ope it "p
a! it<s 4"st a h"ge collectio o+ parts. $ lot o+ steel str"ts, screws, "ts, gears,
p"lleys, motors, a! a +ew s"ggestios o what to %"il!. So yo" start to play with
it. 5o" %"il! oe o+ the s"ggestios a! the aother. $+ter a while yo" !isco#er
that yo" ha#e yo"r ow i!eas o+ what to make. 5o" !o<t e#er ha#e to go %ack to
the store, as yo" alrea!y ha#e e#erythig yo" ee!. The Erector Set takes o the
shape o+ yo"r imagiatio. 9t !oes what yo" wat.
5o"r choice o+ toys is, o+ co"rse, a persoal thig, so which toy wo"l! yo" +i!
more satis+yigC
/.
8 8 Seeig The Worl! $s The Shell Sees 9t
- Seeing The World )s The Shell Sees It
9 this chapter we are goig to look at some o+ the ?magic@ that occ"rs o the comma!
lie whe yo" press the eter key. While we will e3amie se#eral iterestig a!
comple3 +eat"res o+ the shell, we will !o it with 4"st oe ew comma!,
echo 8 'isplay a lie o+ te3t
$xpansion
Each time yo" type a comma! lie a! press the eter key, bash per+orms se#eral
processes "po the te3t %e+ore it carries o"t yo"r comma!. We ha#e see a co"ple o+
cases o+ how a simple character se="ece, +or e3ample ?U@, ca ha#e a lot o+ meaig to
the shell. The process that makes this happe is calle! e+pansion. With e3pasio, yo"
type somethig a! it is e3pa!e! ito somethig else %e+ore the shell acts "po it. To
!emostrate what we mea %y this, let<s take a look at the echo comma!. echo is a
shell %"ilti that per+orms a #ery simple task. 9t prits o"t its te3t arg"mets o sta!ar!
o"tp"t,
[me@linuxbox ~]$ echo this is a test
this is a test
That<s pretty straight+orwar!. $y arg"met passe! to echo gets !isplaye!. *et<s try
aother e3ample,
[me@linuxbox ~]$ echo *
'eskto7 'ocuments ls-out7ut<txt +usic :ictures :ublic Tem7lates
;ideos
So what 4"st happee!C Why !i!<t echo prit ?U@C $s yo" recall +rom o"r work with
wil!car!s, the ?U@ character meas match ay characters i a +ileame, %"t what we !i!<t
see i o"r origial !isc"ssio was how the shell !oes that. The simple aswer is that the
shell e3pa!s the ?U@ ito somethig else Ei this istace, the ames o+ the +iles i the
/2
E3pasio
c"rret workig !irectoryF %e+ore the echo comma! is e3ec"te!. Whe the eter key is
presse!, the shell a"tomatically e3pa!s ay ="ali+yig characters o the comma! lie
%e+ore the comma! is carrie! o"t, so the echo comma! e#er saw the ?U@, oly its
e3pa!e! res"lt. Mowig this, we ca see that echo %eha#e! as e3pecte!.
'athname 6x5ansion
The mechaism %y which wil!car!s work is calle! pathname e+pansion. 9+ we try some
o+ the techi="es that we employe! i o"r earlier chapters, we will see that they are really
e3pasios. ;i#e a home !irectory that looks like this,
[me@linuxbox ~]$ ls
'eskto7 ls-out7ut<txt :ictures Tem7lates
'ocuments +usic :ublic ;ideos
we co"l! carry o"t the +ollowig e3pasios,
[me@linuxbox ~]$ echo 1*
'eskto7 'ocuments
a!,
[me@linuxbox ~]$ echo *s
'ocuments :ictures Tem7lates ;ideos
or e#e,
[me@linuxbox ~]$ echo ((2upper2))*
'eskto7 'ocuments +usic :ictures :ublic Tem7lates ;ideos
a! lookig %eyo! o"r home !irectory,
[me@linuxbox ~]$ echo /usr/*/share
6usr6kerberos6share 6usr6local6share
/(
8 8 Seeig The Worl! $s The Shell Sees 9t
Pathname $xpansion ?" 3idden ;iles
$s we kow, +ileames that %egi with a perio! character are hi!!e. 6athame
e3pasio also respects this %eha#ior. $ e3pasio s"ch as,
echo E
!oes ot re#eal hi!!e +iles.
9t might appear at +irst glace that we co"l! icl"!e hi!!e +iles i a e3pasio
%y startig the patter with a lea!ig perio!, like this,
echo <E
9t almost works. Aowe#er, i+ we e3amie the res"lts closely, we will see that the
ames ?.@ a! ?..@ will also appear i the res"lts. Sice these ames re+er to the
c"rret workig !irectory a! its paret !irectory, "sig this patter will likely
pro!"ce a icorrect res"lt. We ca see this i+ we try the comma!,
ls -d <E Q less
To correctly per+orm pathame e3pasio i this sit"atio, we ha#e to employ a
more speci+ic patter. This will work correctly,
ls -d <[G<]FE
This patter e3pa!s ito e#ery +ileame that %egis with a perio!, !oes ot
icl"!e a seco! perio!, cotais at least oe a!!itioal character a! ca %e
+ollowe! %y ay other characters.
Tide 6x5ansion
$s yo" may recall +rom o"r itro!"ctio to the cd comma!, the til!e character E?S@F has
a special meaig. Whe "se! at the %egiig o+ a wor!, it e3pa!s ito the ame o+ the
home !irectory o+ the ame! "ser, or i+ o "ser is ame!, the home !irectory o+ the
c"rret "ser,
[me@linuxbox ~]$ echo ~
6home6me
9+ "ser ?+oo@ has a acco"t, the,
[me@linuxbox ~]$ echo ~foo
/1
E3pasio
6home6foo
Arithmeti/ 6x5ansion
The shell allows arithmetic to %e per+orme! %y e3pasio. This allow "s to "se the shell
prompt as a calc"lator,
[me@linuxbox ~]$ echo 3442 5 266
%
$rithmetic e3pasio "ses the +orm,
$NNexpressionPP
where e+pression is a arithmetic e3pressio cosistig o+ #al"es a! arithmetic
operators.
$rithmetic e3pasio oly s"pports itegers Ewhole "m%ers, o !ecimalsF, %"t ca
per+orm ="ite a "m%er o+ !i++eret operatios. Aere are a +ew o+ the s"pporte!
operators,
Table 4-1: )rithmetic &perators
?perator Description
8
$!!itio
-
S"%tractio
E
>"ltiplicatio
6
'i#isio E%"t remem%er, sice e3pasio oly s"pports iteger
arithmetic, res"lts are itegers.F
5
>o!"lo, which simply meas, ? remai!er.@
EE
E3poetiatio
Spaces are ot sigi+icat i arithmetic e3pressios a! e3pressios may %e este!. 0or
e3ample, to m"ltiply +i#e s="are! %y three,
[me@linuxbox ~]$ echo 344344"**266 * 66
)"
/2
8 8 Seeig The Worl! $s The Shell Sees 9t
Sigle paretheses may %e "se! to gro"p m"ltiple s"%e3pressios. With this techi="e,
we ca rewrite the e3ample a%o#e a! get the same res"lt "sig a sigle e3pasio
istea! o+ two,
[me@linuxbox ~]$ echo 3444"**26 * 66
)"
Aere is a e3ample "sig the !i#isio a! remai!er operators. &otice the e++ect o+
iteger !i#isio,
[me@linuxbox ~]$ echo 7ive divided b! two e,uals 344"/266
-ive divided b1 t9o eBuals !
[me@linuxbox ~]$ echo with 344"8266 left over.
9ith # left over<
$rithmetic e3pasio is co#ere! i greater !etail i Chapter (2.
Bra/e 6x5ansion
6erhaps the stragest e3pasio is calle! brace e+pansion. With it, yo" ca create
m"ltiple te3t strigs +rom a patter cotaiig %races. Aere<s a e3ample,
[me@linuxbox ~]$ echo 7ront-9:;<;=>-<ack
-ront-4-Hack -ront-H-Hack -ront-C-Hack
6atters to %e %race e3pa!e! may cotai a lea!ig portio calle! a preamble a! a
trailig portio calle! a postscript. The %race e3pressio itsel+ may cotai either a
comma-separate! list o+ strigs, or a rage o+ itegers or sigle characters. The patter
may ot cotai em%e!!e! whitespace. Aere is a e3ample "sig a rage o+ itegers,
[me@linuxbox ~]$ echo ?umber.91..">
LumberO# LumberO! LumberO$ LumberO% LumberO"
$ rage o+ letters i re#erse or!er,
[me@linuxbox ~]$ echo 9@..:>
I U V , ; 3 T * R Z : L + = 2 ? A S @ - & ' C H 4
/K
E3pasio
Jrace e3pasios may %e este!,
[me@linuxbox ~]$ echo a9:91;2>;<9;A>>b
a4#b a4!b aH$b aH%b
So what is this goo! +orC The most commo applicatio is to make lists o+ +iles or
!irectories to %e create!. 0or e3ample, i+ we were photographers a! ha! a large
collectio o+ images that we wate! to orgaiDe ito years a! moths, the +irst thig we
might !o is create a series o+ !irectories ame! i "meric ?5ear->oth@ +ormat. This
way, the !irectory ames will sort i chroological or!er. We co"l! type o"t a complete
list o+ !irectories, %"t that<s a lot o+ work a! it<s error-proe too. 9stea!, we co"l! !o
this,
[me@linuxbox ~]$ mkdir Bics
[me@linuxbox ~]$ cd Bics
[me@linuxbox :ics]$ mkdir 92**C..2**D>-*91..D> 92**C..2**D>-91*..12>
[me@linuxbox :ics]$ ls
!(()-(# !(()-() !((/-(# !((/-() !((0-(# !((0-()
!(()-(! !(()-(/ !((/-(! !((/-(/ !((0-(! !((0-(/
!(()-($ !(()-(0 !((/-($ !((/-(0 !((0-($ !((0-(0
!(()-(% !(()-#( !((/-(% !((/-#( !((0-(% !((0-#(
!(()-(" !(()-## !((/-(" !((/-## !((0-(" !((0-##
!(()-(. !(()-#! !((/-(. !((/-#! !((0-(. !((0-#!
6retty slickL
'arameter 6x5ansion
We<re oly goig to to"ch %rie+ly o parameter e3pasio i this chapter, %"t we<ll %e
co#erig it e3tesi#ely later. 9t<s a +eat"re that is more "se+"l i shell scripts tha !irectly
o the comma! lie. >ay o+ its capa%ilities ha#e to !o with the system<s a%ility to
store small ch"ks o+ !ata a! to gi#e each ch"k a ame. >ay s"ch ch"ks, more
properly calle! variables, are a#aila%le +or yo"r e3amiatio. 0or e3ample, the #aria%le
ame! ?)SE:@ cotais yo"r "ser ame. To i#oke parameter e3pasio a! re#eal the
cotets o+ 3*&R yo" wo"l! !o this,
[me@linuxbox ~]$ echo 3EFGH
me
To see a list o+ a#aila%le #aria%les, try this,
//
8 8 Seeig The Worl! $s The Shell Sees 9t
[me@linuxbox ~]$ printenv 0 less
5o" may ha#e otice! that with other types o+ e3pasio, i+ yo" mistype a patter, the
e3pasio will ot take place a! the echo comma! will simply !isplay the mistype!
patter. With parameter e3pasio, i+ yo" misspell the ame o+ a #aria%le, the e3pasio
will still take place, %"t will res"lt i a empty strig,
[me@linuxbox ~]$ echo 3FEGH
[me@linuxbox ~]$
Command Substitution
Comma! s"%stit"tio allows "s to "se the o"tp"t o+ a comma! as a e3pasio,
[me@linuxbox ~]$ echo 34ls6
'eskto7 'ocuments ls-out7ut<txt +usic :ictures :ublic Tem7lates
;ideos
7e o+ my +a#orites goes somethig like this,
[me@linuxbox ~]$ ls -l 34which cp6
-r9xr-xr-x # root root )#"#. !(()-#!-(" (/:"/ 6bin6c7
Aere we passe! the res"lts o+ 9hich c7 as a arg"met to the ls comma!, there%y
gettig the listig o+ o+ the c7 program witho"t ha#ig to kow its +"ll pathame. We are
ot limite! to 4"st simple comma!s. Etire pipelies ca %e "se! Eoly partial o"tp"t
showF,
[me@linuxbox ~]$ file 34ls /usr/bin/* 0 grep -ip6
6usr6bin6bunCi7!: s1mbolic link to JbCi7!K
6usr6bin6bCi7!: &=- $!-bit =*H executable> Antel /($/.>
version # N*U*;P> d1namicall1 linked Nuses shared libsP> for
@L36=inux !<.<0> stri77ed
6usr6bin6bCi7!recover: &=- $!-bit =*H executable> Antel /($/.>
version # N*U*;P> d1namicall1 linked Nuses shared libsP> for
@L36=inux !<.<0> stri77ed
/8
E3pasio
6usr6bin6funCi7: &=- $!-bit =*H executable> Antel /($/.>
version # N*U*;P> d1namicall1 linked Nuses shared libsP> for
@L36=inux !<.<0> stri77ed
6usr6bin6g7g-Ci7: Hourne shell scri7t text executable
6usr6bin6gunCi7: s1mbolic link to J<<6<<6bin6gunCi7K
6usr6bin6gCi7: s1mbolic link to J<<6<<6bin6gCi7K
6usr6bin6mCi7: s1mbolic link to JmtoolsK
9 this e3ample, the res"lts o+ the pipelie %ecame the arg"met list o+ the file
comma!.
There is a alterate syta3 +or comma! s"%stit"tio i ol!er shell programs which is
also s"pporte! i bash. 9t "ses back-<uotes istea! o+ the !ollar sig a! paretheses,
[me@linuxbox ~]$ ls -l Iwhich cpI
-r9xr-xr-x # root root )#"#. !(()-#!-(" (/:"/ 6bin6c7
Auoting
&ow that we<#e see how may ways the shell ca per+orm e3pasios, it<s time to lear
how we ca cotrol it. Take +or e3ample,
[me@linuxbox ~]$ echo this is a test
this is a test
or,
[me@linuxbox ~]$ echo +he total is 31**.**
The total is ((<((
9 the +irst e3ample, word-splitting %y the shell remo#e! e3tra whitespace +rom the echo
comma!<s list o+ arg"mets. 9 the seco! e3ample, parameter e3pasio s"%stit"te! a
empty strig +or the #al"e o+ ?[.@ %eca"se it was a "!e+ie! #aria%le. The shell
pro#i!es a mechaism calle! <uoting to selecti#ely s"ppress "wate! e3pasios.
)oube Duotes
The +irst type o+ ="otig we will look at is double <uotes. 9+ yo" place te3t isi!e !o"%le
="otes, all the special characters "se! %y the shell lose their special meaig a! are
/9
8 8 Seeig The Worl! $s The Shell Sees 9t
treate! as or!iary characters. The e3ceptios are ?[@, ?\@ E%ackslashF, a! ?]@ E%ack-
="oteF. This meas that wor!-splittig, pathame e3pasio, til!e e3pasio, a! %race
e3pasio are s"ppresse!, %"t parameter e3pasio, arithmetic e3pasio, a! comma!
s"%stit"tio are still carrie! o"t. )sig !o"%le ="otes, we ca cope with +ileames
cotaiig em%e!!e! spaces. Say we were the "+ort"ate #ictim o+ a +ile calle!
t9o 9ords<txt. 9+ we trie! to "se this o the comma! lie, wor!-splittig wo"l!
ca"se this to %e treate! as two separate arg"mets rather tha the !esire! sigle arg"met,
[me@linuxbox ~]$ ls -l two words.txt
ls: cannot access t9o: Lo such file or director1
ls: cannot access 9ords<txt: Lo such file or director1
Jy "sig !o"%le ="otes, we stop the wor!-splittig a! get the !esire! res"ltH +"rther, we
ca e#e repair the !amage,
[me@linuxbox ~]$ ls -l Jtwo words.txtJ
-r9-r9-r-- # me me #/ !((/-(!-!( #$:($ t9o 9ords<txt
[me@linuxbox ~]$ mv Jtwo words.txtJ two.words.txt
ThereL &ow we !o<t ha#e to keep typig those pesky !o"%le ="otes.
:emem%er, parameter e3pasio, arithmetic e3pasio, a! comma! s"%stit"tio still
take place withi !o"%le ="otes,
[me@linuxbox ~]$ echo J3EFGH 34425266 34cal6J
me % -ebruar1 !((/
*u +o Tu ,e Th -r *a
# !
$ % " . ) / 0
#( ## #! #$ #% #" #.
#) #/ #0 !( !# !! !$
!% !" !. !) !/ !0
We sho"l! take a momet to look at the e++ect o+ !o"%le ="otes o comma! s"%stit"tio.
0irst let<s look a little !eeper at how wor! splittig works. 9 o"r earlier e3ample, we saw
how wor!-splittig appears to remo#e e3tra spaces i o"r te3t,
[me@linuxbox ~]$ echo this is a test
this is a test
80
T"otig
Jy !e+a"lt, wor!-splittig looks +or the presece o+ spaces, ta%s, a! ewlies Elie+ee!
charactersF a! treats them as delimiters %etwee wor!s. This meas that "="ote!
spaces, ta%s, a! ewlies are ot cosi!ere! to %e part o+ the te3t. They oly ser#e as
separators. Sice they separate the wor!s ito !i++eret arg"mets, o"r e3ample
comma! lie cotais a comma! +ollowe! %y +o"r !istict arg"mets. 9+ we a!!
!o"%le ="otes,
[me@linuxbox ~]$ echo Jthis is a testJ
this is a test
wor!-splittig is s"ppresse! a! the em%e!!e! spaces are ot treate! as !elimiters, rather
they %ecome part o+ the arg"met. 7ce the !o"%le ="otes are a!!e!, o"r comma! lie
cotais a comma! +ollowe! %y a sigle arg"met.
The +act that ewlies are cosi!ere! !elimiters %y the wor!-splittig mechaism ca"ses
a iterestig, al%eit s"%tle, e++ect o comma! s"%stit"tio. Cosi!er the +ollowig,
[me@linuxbox ~]$ echo 34cal6
-ebruar1 !((/ *u +o Tu ,e Th -r *a # ! $ % " . ) / 0 #( ## #! #$ #%
#" #. #) #/ #0 !( !# !! !$ !% !" !. !) !/ !0
[me@linuxbox ~]$ echo J34cal6J
-ebruar1 !((/
*u +o Tu ,e Th -r *a
# !
$ % " . ) / 0
#( ## #! #$ #% #" #.
#) #/ #0 !( !# !! !$
!% !" !. !) !/ !0
9 the +irst istace, the "="ote! comma! s"%stit"tio res"lte! i a comma! lie
cotaiig thirty-eight arg"mets. 9 the seco!, a comma! lie with oe arg"met that
icl"!es the em%e!!e! spaces a! ewlies.
Singe Duotes
9+ we ee! to s"ppress all e3pasios, we "se single <uotes. Aere is a compariso o+
"="ote!, !o"%le ="otes, a! sigle ="otes,
[me@linuxbox ~]$ echo text ~/*.txt 9a;b> 34echo foo6 34425266 3EFGH
text 6home6me6ls-out7ut<txt a b foo % me
8.
8 8 Seeig The Worl! $s The Shell Sees 9t
[me@linuxbox ~]$ echo Jtext ~/*.txt 9a;b> 34echo foo6 34425266 3EFGHJ
text ~6E<txt [a>b\ foo % me
[me@linuxbox ~]$ echo %text ~/*.txt 9a;b> 34echo foo6 34425266 3EFGH%
text ~6E<txt [a>b\ $Necho fooP $NN!8!PP $3*&R
$s we ca see, with each s"ccee!ig le#el o+ ="otig, more a! more o+ the e3pasios
are s"ppresse!.
6s/a5ing Chara/ters
Sometimes we oly wat to ="ote a sigle character. To !o this, we ca prece!e a
character with a %ackslash, which i this cote3t is calle! the escape character. 7+te
this is !oe isi!e !o"%le ="otes to selecti#ely pre#et a e3pasio,
[me@linuxbox ~]$ echo J+he balance for user 3EFGH is2 K3".**J
The balance for user me is: $"<((
9t is also commo to "se escapig to elimiate the special meaig o+ a character i a
+ileame. 0or e3ample, it is possi%le to "se characters i +ileames that ormally ha#e
special meaig to the shell. These wo"l! icl"!e ?[@, ?L@, ?^@, ? ?, a! others. To
icl"!e a special character i a +ileame yo" ca to this,
[me@linuxbox ~]$ mv badK'filename good.filename
To allow a %ackslash character to appear, escape it %y typig ?\\@. &ote that withi sigle
="otes, the %ackslash loses its special meaig a! is treate! as a or!iary character.
/ac(slash $scape SeBuences
9 a!!itio to its role as the escape character, the %ackslash is also "se! as part o+
a otatio to represet certai special characters calle! control codes. The +irst
thirty-two characters i the $SC99 co!ig scheme are "se! to trasmit comma!s
to teletype-like !e#ices. Some o+ these co!es are +amiliar Eta%, %ackspace,
lie+ee!, a! carriage ret"rF, while others are ot E"ll, e!-o+-trasmissio, a!
ackowle!geF.
82
T"otig
The ta%le a%o#e lists some o+ the commo %ackslash escape se="eces. The i!ea
%ehi! this represetatio "sig the %ackslash origiate! i the C programmig
lag"age a! has %ee a!opte! %y may others, icl"!ig the shell.
$!!ig the ?-e@ optio to echo will ea%le iterpretatio o+ escape se="eces.
5o" may also place them isi!e $K K. Aere, "sig the slee7 comma!, a
simple program that 4"st waits +or the speci+ie! "m%er o+ seco!s a! the e3its,
we ca create a primiti#e co"t!ow timer,
sleep 1*# echo -e J+ime%s upKaJ
We co"l! also !o this,
sleep 1*# echo J+ime%s upJ 3%Ka%
Summing -p
$s we mo#e +orwar! with "sig the shell, we will +i! that e3pasios a! ="otig will
%e "se! with icreasig +re="ecy, so it makes sese to get a goo! "!ersta!ig o+ the
way they works. 9 +act, it co"l! %e arg"e! that they are the most importat s"%4ects to
lear a%o"t the shell. Witho"t a proper "!ersta!ig o+ e3pasio, the shell will always
%e a so"rce o+ mystery a! co+"sio, a! m"ch o+ it potetial power waste!.
;urther 1eading
The bash ma page has ma4or sectios o %oth e3pasio a! ="otig which
co#er these topics i a more +ormal maer.
The 8ash 9eference !anual also cotais chapters o e3pasio a! ="otig,
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.html
8(
$scape SeBuence >eaning
]a
Jell E?$lert@ - ca"ses the comp"ter to %eepF
]b
Jackspace
]n
&ewlie. 7 )i3-like systems, this
pro!"ces a lie+ee!.
]r
Carriage ret"r
]t
Ta%
9 8 $!#ace! Mey%oar! Tricks
0 )dvanced 1e$board Tric,s
9 o+te ki!!igly !escri%e )i3 as ?the operatig system +or people who like to type.@ 7+
co"rse, the +act that it e#e has a comma! lie is a testamet to that. J"t comma! lie
"sers !o<t like to type that m"ch. Why else wo"l! so may comma!s ha#e s"ch short
ames like c7, ls, mv, a! rmC 9 +act, oe o+ the most cherishe! goals o+ the comma!
lie is laDiessH !oig the most work with the +ewest "m%er o+ keystrokes. $other goal
is e#er ha#ig to li+t yo"r +igers +rom the key%oar!, e#er reachig +or the mo"se. 9
this chapter, we will look at bash +eat"res that make key%oar! "se +aster a! more
e++iciet.
The +ollowig comma!s will make a appearace,
clear 8 Clear the scree
histor1 8 'isplay the cotets o+ the history list
Command Line $diting
bash "ses a li%rary Ea share! collectio o+ ro"ties that !i++eret programs ca "seF
calle! 9eadline to implemet comma! lie e!itig. We ha#e alrea!y see some o+ this.
We kow, +or e3ample, that the arrow keys mo#e the c"rsor %"t there are may more
+eat"res. Thik o+ these as a!!itioal tools that we ca employ i o"r work. 9tIs ot
importat to lear all o+ them, %"t may o+ them are #ery "se+"l. 6ick a! choose as
!esire!.
1ote5 Some o+ the key se="eces %elow Epartic"larly those which "se the 4lt keyF
may %e itercepte! %y the ;)9 +or other +"ctios. $ll o+ the key se="eces sho"l!
work properly whe "sig a #irt"al cosole.
Cursor :o;ement
The +ollowig ta%le lists the keys "se! to mo#e the c"rsor,
81
Comma! *ie E!itig
Table 5-1: %ursor !ovement %ommands
=e, #ction
Ctrl-a
>o#e c"rsor to the %egiig o+ the lie.
Ctrl-e
>o#e c"rsor to the e! o+ the lie.
Ctrl-f
>o#e c"rsor +orwar! oe characterH same as the right arrow key.
Ctrl-b
>o#e c"rsor %ackwar! oe characterH same as the le+t arrow key.
4lt-f
>o#e c"rsor +orwar! oe wor!.
4lt-b
>o#e c"rsor %ackwar! oe wor!.
Ctrl-l
Clear the scree a! mo#e the c"rsor to the top le+t corer. The
clear comma! !oes the same thig.
:odi=ying Text
Ta%le 9-2 lists key%oar! comma!s that are "se! to e!it characters o the comma! lie.
Table 5-': Te+t 1diting %ommands
=e, #ction
Ctrl-d
'elete the character at the c"rsor locatio
Ctrl-t
Traspose Ee3chageF the character at the c"rsor locatio with the
oe prece!ig it.
4lt-t
Traspose the wor! at the c"rsor locatio with the oe prece!ig it.
4lt-l
Co#ert the characters +rom the c"rsor locatio to the e! o+ the
wor! to lowercase.
4lt-u
Co#ert the characters +rom the c"rsor locatio to the e! o+ the
wor! to "ppercase.
Cutting And 'asting E9iing And 1ankingF Text
The :ea!lie !oc"metatio "ses the terms killing a! yanking to re+er to what we wo"l!
commoly call c"ttig a! pastig. 9tems that are c"t are store! i a %"++er calle! the
kill-ring.
82
9 8 $!#ace! Mey%oar! Tricks
Table 5-3: %ut )nd 6aste %ommands
=e, #ction
Ctrl-k
Mill te3t +rom the c"rsor locatio to the e! o+ lie.
Ctrl-u
Mill te3t +rom the c"rsor locatio to the %egiig o+ the lie.
4lt-d
Mill te3t +rom the c"rsor locatio to the e! o+ the c"rret wor!.
4lt-
Hacks7ace
Mill te3t +rom the c"rsor locatio to the %egiig o+ the c"rret
wor!. 9+ the c"rsor is at the %egiig o+ a wor!, kill the pre#io"s
wor!.
Ctrl-1
5ak te3t +rom the kill-rig a! isert it at the c"rsor locatio.
The >eta =e,
9+ yo" #et"re ito the :ea!lie !oc"metatio, which ca %e +o"! i the
:E$'*9&E sectio o+ the bash ma page, yo" will eco"ter the term ?meta
key.@ 7 mo!er key%oar!s this maps to the 4lt key %"t it was<t always so.
Jack i the !im times E%e+ore 6Cs %"t a+ter )i3F ot e#ery%o!y ha! their ow
comp"ter. What they might ha#e ha! was a !e#ice calle! a terminal. $ termial
was a comm"icatio !e#ice that +eat"re! a te3t !isplay scree a! a key%oar!
a! 4"st eo"gh electroics isi!e to !isplay te3t characters a! mo#e the c"rsor
aro"!. 9t was attache! E"s"ally %y serial ca%leF to a larger comp"ter or the
comm"icatio etwork o+ a larger comp"ter. There were may !i++eret %ra!s
o+ termials a! they all ha! !i++eret key%oar!s a! !isplay +eat"re sets. Sice
they all te!e! to at least "!ersta! $SC99, so+tware !e#elopers watig
porta%le applicatios wrote to the lowest commo !eomiator. )i3 systems
ha#e a #ery ela%orate way o+ !ealig with termials a! their !i++eret !isplay
+eat"res. Sice the !e#elopers o+ :ea!lie co"l! ot %e s"re o+ the presece o+ a
!e!icate! e3tra cotrol key, they i#ete! oe a! calle! it ?meta.@ While the
4lt key ser#es as the meta key o mo!er key%oar!s, yo" ca also press a!
release the &sc key to get the same e++ect as hol!ig !ow the 4lt key i+ yo"<re
still "sig a termial Ewhich yo" ca still !o i *i"3LF.
Completion
$other way that the shell ca help yo" is thro"gh a mechaism calle! completion.
Completio occ"rs whe yo" press the ta% key while typig a comma!. *et<s see how
8K
Completio
this works. ;i#e a home !irectory that looks like this,
[me@linuxbox ~]$ ls
'eskto7 ls-out7ut<txt :ictures Tem7lates ;ideos
'ocuments +usic :ublic
Try typig the +ollowig %"t don7t (ress the Gnter key,
[me@linuxbox ~]$ ls l
&ow press the ta% key,
[me@linuxbox ~]$ ls ls-output.txt
See how the shell complete! the lie +or yo"C *et<s try aother oe. $gai, !o<t press
&nter,
[me@linuxbox ~]$ ls 1
6ress ta%,
[me@linuxbox ~]$ ls 1
&o completio, 4"st a %eep. This happee! %eca"se ?'@ matches more tha oe etry i
the !irectory. 0or completio to %e s"ccess+"l, the ?cl"e@ yo" gi#e it has to %e
"am%ig"o"s. 9+ we go +"rther,
[me@linuxbox ~]$ ls 1o
The press ta%,
[me@linuxbox ~]$ ls 1ocuments
8/
9 8 $!#ace! Mey%oar! Tricks
The completio is s"ccess+"l.
While this e3ample shows completio o+ pathames, which is its most commo "se,
completio will also work o #aria%les Ei+ the %egiig o+ the wor! is a ?[@F, "ser ames
Ei+ the wor! %egis with ?S@F, comma!s Ei+ the wor! is the +irst wor! o the lie.F a!
host ames Ei+ the %egiig o+ the wor! is ?N@F. Aost ame completio oly works +or
host ames liste! i 6etc6hosts.
There are a "m%er o+ cotrol a! meta key se="eces that are associate! with
completio,
Table 5-$: %ompletion %ommands
=e, #ction
4lt-F
'isplay list o+ possi%le completios. &n most systems you can also
do this by pressing the tab key a second time" which is much easier#
4lt-E
9sert all possi%le completios. This is "se+"l whe yo" wat to "se
more tha oe possi%le match.
There ="ite a +ew more that 9 +i! rather o%sc"re. 5o" ca see a list i the bash ma
page "!er ?:E$'*9&E@.
Programma0le Completion
:ecet #ersios o+ bash ha#e a +acility calle! programmable completion.
6rogramma%le completio allows yo" Eor more likely, yo"r !istri%"tio pro#i!erF
to a!! a!!itioal completio r"les. )s"ally this is !oe to a!! s"pport +or
speci+ic applicatios. 0or e3ample it is possi%le to a!! completios +or the optio
list o+ a comma! or match partic"lar +ile types that a applicatio s"pports.
)%"t" has a +airly large set !e+ie! %y !e+a"lt. 6rogramma%le completio is
implemete! %y shell +"ctios, a ki! o+ mii shell script that we will co#er i
later chapters. 9+ yo" are c"rio"s, try,
set 0 less
a! see i+ yo" ca +i! them. &ot all !istri%"tios icl"!e them %y !e+a"lt.
-sing 3istor,
$s we !isco#ere! i Chapter 2, bash maitais a history o+ comma!s that ha#e %ee
etere!. This list o+ comma!s is kept i yo"r home !irectory i a +ile calle!
88
)sig Aistory
<bashOhistor1. The history +acility is a "se+"l reso"rce +or re!"cig the amo"t o+
typig yo" ha#e to !o, especially whe com%ie! with comma! lie e!itig.
Sear/hing $istory
$t ay time, we ca #iew the cotets o+ the history list %y,
[me@linuxbox ~]$ histor! 0 less
Jy !e+a"lt, %ash stores the last +i#e h"!re! comma!s yo" ha#e etere!. We will see
how to a!4"st this #al"e i a later chapter. *et<s say we wat to +i! the comma!s we
"se! to list 6usr6bin. 7e way we co"l! !o this,
[me@linuxbox ~]$ histor! 0 grep /usr/bin
$! let<s say that amog o"r res"lts we got a lie cotaiig a iterestig comma! like
this,
// ls -l 6usr6bin D ls-out7ut<txt
The "m%er ?88@ is the lie "m%er o+ the comma! i the history list. We co"l! "se this
imme!iately "sig aother type o+ e3pasio calle! history e+pansion. To "se o"r
!isco#ere! lie we co"l! !o this,
[me@linuxbox ~]$ LMM
bash will e3pa! ?L88@ ito the cotets o+ the eighty-eighth lie i the history list.
There are other +orms o+ history e3pasio that we will co#er a little later.
bash also pro#i!es the a%ility to search the history list icremetally. This meas that
we ca tell bash to search the history list as we eter characters, with each a!!itioal
character +"rther re+iig o"r search. To start icremetal search type Ctrl-r +ollowe!
%y the te3t yo" are lookig +or. Whe yo" +i! it, yo" ca either type &nter to e3ec"te
the comma! or type Ctrl-j to copy the lie +rom the history list to the c"rret
comma! lie. To +i! the e3t occ"rrece o+ the te3t Emo#ig ?"p@ the history listF,
type Ctrl-r agai. To ="it searchig, type either Ctrl-g or Ctrl-c. Aere we see it
i actio,
89
9 8 $!#ace! Mey%oar! Tricks
[me@linuxbox ~]$
0irst type Ctrl-r,
Nreverse-i-searchPJK:
The prompt chages to i!icate that we are per+ormig a re#erse icremetal search. 9t is
?re#erse@ %eca"se we are searchig +rom ?ow@ to some time i the past. &e3t, we start
typig o"r search te3t. 9 this e3ample ?-"sr-%i@,
Nreverse-i-searchPJ/usr/binK: ls -l 6usr6bin D ls-out7ut<txt
9mme!iately, the search ret"rs o"r res"lt. With o"r res"lt, we ca e3ec"te the comma!
%y pressig &nter, or we ca copy the comma! to o"r c"rret comma! lie +or
+"rther e!itig %y typig Ctrl-j. *et<s copy it. Type Ctrl-j,
[me@linuxbox ~]$ ls -l /usr/bin & ls-output.txt
7"r shell prompt ret"rs a! o"r comma! lie is loa!e! a! rea!y +or actioL
The ta%le %elow lists some o+ the keystrokes "se! to maip"late the history list,
Table 5-.: ,istory %ommands
=e, #ction
Ctrl-7
>o#e to the pre#io"s history etry. Same actio as the "p arrow.
Ctrl-n
>o#e to the e3t history etry. Same actio as the !ow arrow.
4lt-X
>o#e to the %egiig EtopF o+ the history list.
4lt-D
>o#e to the e! E%ottomF o+ the history list, i.e., the c"rret
comma! lie.
Ctrl-r
:e#erse icremetal search. Searches icremetally +rom the
c"rret comma! lie "p the history list.
4lt-7
:e#erse search, o-icremetal. With this key, type i the search
strig a! press eter %e+ore the search is per+orme!.
90
)sig Aistory
4lt-n
0orwar! search, o-icremetal.
Ctrl-o
E3ec"te the c"rret item i the history list a! a!#ace to the e3t
oe. This is ha!y i+ yo" are tryig to re-e3ec"te a se="ece o+
comma!s i the history list.
$istory 6x5ansion
The shell o++ers a specialiDe! type o+ e3pasio +or items i the history list %y "sig the
?L@ character. We ha#e alrea!y see how the e3clamatio poit ca %e +ollowe! %y a
"m%er to isert a etry +rom the history list. There are a "m%er o+ other e3pasio
+eat"res,
Table 5-2: ,istory 1+pansion %ommands
SeBuence #ction
GG
:epeat the last comma!. 9t is pro%a%ly easier to press "p arrow
a! eter.
Gnumber
:epeat history list item number.
Gstring
:epeat last history list item startig with strig.
GFstring
:epeat last history list item cotaiig strig.
9 wo"l! ca"tio agaist "sig the ?Lstrig@ a! ?LCstrig@ +orms "less yo" are a%sol"tely
s"re o+ the cotets o+ the history list items.
There are may more elemets a#aila%le i the history e3pasio mechaism, %"t this
s"%4ect is alrea!y too arcae a! o"r hea!s may e3plo!e i+ we coti"e. The A9ST7:5
EG6$&S97& sectio o+ the bash ma page goes ito all the gory !etails. 0eel +ree to
e3ploreL
script
9 a!!itio to the comma! history +eat"re i bash, most *i"3 !istri%"tios
icl"!e a program calle! scri7t that ca %e "se! to recor! a etire shell
sessio a! store it i a +ile. The %asic syta3 o+ the comma! is,
scri7t [file]
9.
9 8 $!#ace! Mey%oar! Tricks
where file is the ame o+ the +ile "se! +or storig the recor!ig. 9+ o +ile is
speci+ie!, the +ile t17escri7t is "se!. See the scri7t ma page +or a
complete list o+ the programIs optios a! +eat"res.
Summing -p
9 this chapter we ha#e co#ere! some o+ the key%oar! tricks that the shell pro#i!es to
help har!core typists re!"ce their workloa!s. 9 s"spect that as time goes %y a! yo"
%ecome more i#ol#e! with the comma! lie, yo" will re+er %ack to this chapter to pick
"p more o+ these tricks. 0or ow, cosi!er them optioal a! potetially help+"l.
;urther 1eading
The Wikipe!ia has a goo! article o comp"ter termials,
http,--e.wikipe!ia.org-wiki-Comp"terOtermial
92
.0 8 6ermissios
10 2er%issions
7peratig systems i the )i3 tra!itio !i++er +rom those i the >S-'7S tra!itio i
that they are ot oly multitasking systems, %"t also multi-user systems, as well.
What e3actly !oes this meaC 9t meas that more tha oe perso ca %e "sig the
comp"ter at the same time. While a typical comp"ter will likely ha#e oly oe key%oar!
a! moitor, it ca still %e "se! %y more tha oe "ser. 0or e3ample, i+ a comp"ter is
attache! to a etwork or the 9teret, remote "sers ca log i #ia ssh Esec"re shellF a!
operate the comp"ter. 9 +act, remote "sers ca e3ec"te graphical applicatios a! ha#e
the graphical o"tp"t appear o a remote !isplay. The G Wi!ow System s"pports this as
part o+ its %asic !esig.
The m"lti-"ser capa%ility o+ *i"3 is ot a recet Qio#atio,Q %"t rather a +eat"re that is
!eeply em%e!!e! ito the !esig o+ the operatig system. Cosi!erig the e#iromet
i which )i3 was create!, this makes per+ect sese. 5ears ago, %e+ore comp"ters were
Qpersoal,Q they were large, e3pesi#e, a! cetraliDe!. $ typical "i#ersity comp"ter
system, +or e3ample, cosiste! o+ a large cetral comp"ter locate! i oe %"il!ig a!
termials which were locate! thro"gho"t the camp"s, each coecte! to the large cetral
comp"ter. The comp"ter wo"l! s"pport may "sers at the same time.
9 or!er to make this practical, a metho! ha! to %e !e#ise! to protect the "sers +rom each
other. $+ter all, the actios o+ oe "ser co"l! ot %e allowe! to crash the comp"ter, or
co"l! oe "ser iter+ere with the +iles %elogig to aother "ser.
9 this chapter we are goig to look at this essetial part o+ system sec"rity a! itro!"ce
the +ollowig comma!s,
id 8 'isplay "ser i!etity
chmod 8 Chage a +ile<s mo!e
umask 8 Set the !e+a"lt +ile permissios
su 8 :" a shell as aother "ser
sudo 8 E3ec"te a comma! as aother "ser
cho9n 8 Chage a +ile<s ower
9(
.0 8 6ermissios
chgr7 8 Chage a +ile<s gro"p owership
7ass9d 8 Chage a "ser<s passwor!
?4ners@ 6roup >em0ers@ #nd $%er,0od, $lse
Whe we were e3plorig the system %ack i Chapter 1, we may ha#e eco"tere! a
pro%lem whe tryig to e3amie a +ile s"ch as 6etc6shado9,
[me@linuxbox ~]$ file /etc/shadow
6etc6shado9: regular file> no read 7ermission
[me@linuxbox ~]$ less /etc/shadow
6etc6shado9: :ermission denied
The reaso +or this error message is that, as reg"lar "sers, we !o ot ha#e permissio to
rea! this +ile.
9 the )i3 sec"rity mo!el, a "ser may own +iles a! !irectories. Whe a "ser ows a +ile
or !irectory, the "ser has cotrol o#er its access. )sers ca, i t"r, %elog to a group
cosistig o+ oe or more "sers who are gi#e access to +iles a! !irectories %y their
owers. 9 a!!itio to gratig access to a gro"p, a ower may also grat some set o+
access rights to e#ery%o!y, which i )i3 terms is re+erre! to as the world. To +i! o"t
i+ormatio a%o"t yo"r i!etity, "se the id comma!,
[me@linuxbox ~]$ id
uidM"((NmeP gidM"((NmeP grou7sM"((NmeP
*et<s look at the o"tp"t. Whe "ser acco"ts are create!, "sers are assige! a "m%er
calle! a user *- or uid which is the, +or the sake o+ the h"mas, mappe! to a "ser ame.
The "ser is assige! a primary group *- or gid a! may %elog to a!!itioal gro"ps. The
a%o#e e3ample is +rom a 0e!ora system. 7 other systems, s"ch as )%"t", the o"tp"t
may look a little !i++eret,
[me@linuxbox ~]$ id
uidM#(((NmeP gidM#(((NmeP
grou7sM%NadmP>!(NdialoutP>!%NcdromP>!"Nflo771P>!0NaudioP>$(Ndi7P>%%Nv
ideoP>%.N7lugdevP>#(/Nl7adminP>##%NadminP>#(((NmeP
$s we ca see, the "i! a! gi! "m%ers are !i++eret. This is simply %eca"se 0e!ora
starts its "m%erig o+ reg"lar "ser acco"ts at 200, while )%"t" starts at .000. We ca
91
7wers, ;ro"p >em%ers, $! E#ery%o!y Else
also see that the )%"t" "ser %elogs to a lot more gro"ps. This has to !o with the way
)%"t" maages pri#ileges +or system !e#ices a! ser#ices.
So where !oes this i+ormatio come +romC *ike so may thigs i *i"3, +rom a co"ple
o+ te3t +iles. )ser acco"ts are !e+ie! i the 6etc67ass9d +ile a! gro"ps are !e+ie!
i the 6etc6grou7 +ile. Whe "ser acco"ts a! gro"ps are create!, these +iles are
mo!i+ie! alog with 6etc6shado9 which hol!s i+ormatio a%o"t the "ser<s passwor!.
0or each "ser acco"t, the 6etc67ass9d +ile !e+ies the "ser ElogiF ame, "i!, gi!,
the acco"t<s real ame, home !irectory, a! logi shell. 9+ yo" e3amie the cotets o+
6etc67ass9d a! 6etc6grou7, yo" will otice that %esi!es the reg"lar "ser
acco"ts, there are acco"ts +or the s"per"ser E"i! 0F a! #ario"s other system "sers.
9 the e3t chapter, whe we co#er processes, yo" will see that some o+ these other
?"sers@ are, i +act, ="ite %"sy.
While may )i3-like systems assig reg"lar "sers to a commo gro"p s"ch as ?"sers@,
mo!er *i"3 practice is to create a "i="e, sigle-mem%er gro"p with the same ame as
the "ser. This makes certai types o+ permissio assigmet easier.
1eading@ +riting@ #nd $xecuting
$ccess rights to +iles a! !irectories are !e+ie! i terms o+ rea! access, write access, a!
e3ec"tio access. 9+ we look at the o"tp"t o+ the ls comma!, we ca get some cl"e as
to how this is implemete!,
[me@linuxbox ~]$ & foo.txt
[me@linuxbox ~]$ ls -l foo.txt
-r9-r9-r-- # me me ( !((/-($-(. #%:"! foo<txt
The +irst te characters o+ the listig are the file attributes. The +irst o+ these characters is
the file type. Aere are the +ile types yo" are most likely to see Ethere are other, less
commo types tooF,
Table 1=-1: File Types
#ttri0ute ;ile T,pe
-
$ reg"lar +ile.
d
$ !irectory.
l
$ sym%olic lik. &otice that with sym%olic liks, the remaiig +ile
attri%"tes are always ?rw3rw3rw3@ a! are !"mmy #al"es. The
real +ile attri%"tes are those o+ the +ile the sym%olic lik poits to.
92
.0 8 6ermissios
c
$ character special file. This +ile type re+ers to a !e#ice that
ha!les !ata as a stream o+ %ytes, s"ch as a termial or mo!em.
b
$ block special file. This +ile type re+ers to a !e#ice that ha!les
!ata i %locks, s"ch as a har! !ri#e or C'-:7> !ri#e.
The remaiig ie characters o+ the +ile attri%"tes, calle! the file mode" represet the
rea!, write, a! e3ec"te permissios +or the +ile<s ower, the +ile<s gro"p ower, a!
e#ery%o!y else,
?4ner 6roup +orld
r9x r9x r9x
Whe set, the r, 9, a! x mo!e attri%"tes ha#e the +ollowig e++ect o +iles a!
!irectories,
Table 1=-': 6ermission )ttributes
#ttri0ute ;iles Directories
r
$llows a +ile to %e opee! a!
rea!.
$llows a !irectory<s cotets to
%e liste! i+ the e3ec"te attri%"te
is also set.
9
$llows a +ile to %e writte to or
tr"cate!, howe#er this attri%"te
!oes ot allow +iles to %e
reame! or !elete!. The a%ility
to !elete or reame +iles is
!etermie! %y !irectory
attri%"tes.
$llows +iles withi a !irectory
to %e create!, !elete!, a!
reame! i+ the e3ec"te attri%"te
is also set.
x
$llows a +ile to %e treate! as a
program a! e3ec"te!.
6rogram +iles writte i
scriptig lag"ages m"st also
%e set as rea!a%le to %e
e3ec"te!.
$llows a !irectory to %e
etere!, e.g., c! directory.
Aere are some e3amples o+ +ile attri%"te settigs,
9K
:ea!ig, Writig, $! E3ec"tig
Table 1=-3: 6ermission )ttribute 1+amples
;ile #ttri0utes >eaning
-r9x------
$ reg"lar +ile that is rea!a%le, writa%le, a! e3ec"ta%le %y the
+ile<s ower. &o oe else has ay access.
-r9-------
$ reg"lar +ile that is rea!a%le a! writa%le %y the +ile<s ower.
&o oe else has ay access.
-r9-r--r--
$ reg"lar +ile that is rea!a%le a! writa%le %y the +ile<s ower.
>em%ers o+ the ower gro"p may rea! the +ile. The +ile is
worl!-rea!a%le.
-r9xr-xr-x
$ reg"lar +ile that is rea!a%le, writa%le, a! e3ec"ta%le %y the
+ile<s ower. The +ile may %e rea! a! e3ec"te! %y e#ery%o!y
else.
-r9-r9----
$ reg"lar +ile that is rea!a%le a! writa%le %y the +ile<s ower
a! mem%ers o+ the +ile<s gro"p ower oly.
lr9xr9xr9x
$ sym%olic lik. $ll sym%olic liks ha#e ?!"mmy@
permissios. The real permissios are kept with the act"al +ile
poite! to %y the sym%olic lik.
dr9xr9x---
$ !irectory. The ower a! the mem%ers o+ the ower gro"p
may eter the !irectory, create, reame a! remo#e +iles withi
the !irectory.
dr9xr-x---
$ !irectory. The ower may eter the !irectory a! create,
reame a! !elete +iles withi the !irectory. >em%ers o+ the
ower gro"p may eter the !irectory %"t caot create, !elete
or reame +iles.
chmod A Change =ie mode
To chage the mo!e EpermissiosF o+ a +ile or !irectory, the chmod comma! is "se!.
Je aware that oly the +ileIs ower or the s"per"ser ca chage the mo!e o+ a +ile or
!irectory. chmod s"pports two !istict ways o+ speci+yig mo!e chages, octal "m%er
represetatio, or sym%olic represetatio. We will co#er octal "m%er represetatio
+irst.
9/
.0 8 6ermissios
+hat The 3ec( Is ?ctal.
&ctal E%ase 8F, a! its co"si, he+adecimal E%ase .KF are "m%er systems o+te
"se! to e3press "m%ers o comp"ters. We h"mas, owig to the +act that we Eor
at least most o+ "sF were %or with te +igers, co"t "sig a %ase .0 "m%er
system. Comp"ters, o the other the other ha!, were %or with oly oe +iger
a! th"s !o all all their co"tig i binary E%ase 2F. Their "m%er system oly
has two "merals, Dero a! oe. So i %iary, co"tig looks like this,
0, ., .0, .., .00, .0., ..0, ..., .000, .00., .0.0, .0.....
9 octal, co"tig is !oe with the "merals Dero thro"gh se#e, like so,
0, ., 2, (, 1, 2, K, /, .0, .., .2, .(, .1, .2, .K, ./, 20, 2....
Ae3a!ecimal co"tig "ses the "merals Dero thro"gh ie pl"s the letters ?$@
thro"gh ?0@,
0, ., 2, (, 1, 2, K, /, 8, 9, $, J, C, ', E, 0, .0, .., .2, .(...
While we ca see the sese i %iary Esice comp"ters oly ha#e oe +igerF,
what are octal a! he3a!ecimal goo! +orC The aswer has to !o with h"ma
co#eiece. >ay times, small portios o+ !ata are represete! o comp"ters as
bit patterns. Take +or e3ample a :;J color. 7 most comp"ter !isplays, each
pi3el is compose! o+ three color compoets, eight %its o+ re!, eight %its o+ gree,
a! eight %its o+ %l"e. $ lo#ely me!i"m %l"e wo"l! %e a twety-+o"r !igit
"m%er,
0.0000..0..0......00..0.
Aow wo"l! yo" like to rea! a! write those ki!s o+ "m%ers all !ayC 9 !i!<t
thik so. Aere<s where aother "m%er system wo"l! help. Each !igit i a
he3a!ecimal "m%er represets +o"r !igits i %iary. 9 octal, each !igit
represets three %iary !igits. So o"r twety-+o"r !igit me!i"m %l"e co"l! %e
co!ese! !ow to a si3 !igit he3a!ecimal "m%er,
1(K0C'
Sice the !igits i the he3a!ecimal "m%er ?lie "p@ with the %its i the %iary
"m%er we ca see that the re! compoet o+ o"r color is ?1(@, the gree ?K0@,
a! the %l"e ?C'@.
These !ays, he3a!ecimal otatio Eo+te spoke as ?he3@F is more commo tha
octal, %"t as we shall soo see, octal<s a%ility to e3press three %its o+ %iary will
%e #ery "se+"l...
With octal otatio we "se octal "m%ers to set the patter o+ !esire! permissios. Sice
98
:ea!ig, Writig, $! E3ec"tig
each !igit i a octal "m%er represets three %iary !igits, this maps icely to the
scheme "se! to store the +ile mo!e. This ta%le shows what we mea,
?ctal /inar, ;ile >ode
( ((( ---
# ((# --x
! (#( -9-
$ (## -9x
% #(( r--
" #(# r-x
. ##( r9-
) ### r9x
Jy "sig three octal !igits, we ca set the +ile mo!e +or the ower, gro"p ower, a!
worl!,
[me@linuxbox ~]$ & foo.txt
[me@linuxbox ~]$ ls -l foo.txt
-r9-r9-r-- # me me ( !((/-($-(. #%:"! foo<txt
[me@linuxbox ~]$ chmod N** foo.txt
[me@linuxbox ~]$ ls -l foo.txt
-r9------- # me me ( !((/-($-(. #%:"! foo<txt
Jy passig the arg"met ?K00@, we were a%le to set the permissios o+ the ower to rea!
a! write while remo#ig all permissios +rom the gro"p ower a! worl!. Tho"gh
remem%erig the octal to %iary mappig may seem ico#eiet, yo" will "s"ally oly
ha#e to "se a +ew commo oes, / Er9xF, K Er9-F, 2 Er-xF, 1 Er--F, a! 0 E---F.
chmod also s"pports a sym%olic otatio +or speci+yig +ile mo!es. Sym%olic otatio is
!i#i!e! ito three parts, who the chage will a++ect, which operatio will %e per+orme!,
a! what permissio will %e set. To speci+y who is a++ecte!, a com%iatio o+ the
characters ?"@, ?g@, ?o@, a! ?a@ is "se! as +ollows,
Table 1=-$: chmod Symbolic >otation
S,m0ol >eaning
u
Short +or ?"se,r@ %"t meas the +ile or !irectory ower.
g
;ro"p ower.
99
.0 8 6ermissios
o
Short +or ?others,@ %"t meas worl!.
a
Short +or ?all.@ The com%iatio o+ ?"@, ?g@, a! ?o@.
9+ o character is speci+ie!, ?all@ will %e ass"me!. The operatio may %e a ?V@ i!icatig
that a permissio is to %e a!!e!, a ?-@ i!icatig that a permissio is to %e take away, or
a ?W@ i!icatig that oly the speci+ie! permissios are to %e applie! a! that all others
are to %e remo#e!.
6ermissios are speci+ie! with the ?r@, ?w@, a! ?3@ characters. Aere are some e3amples
o+ sym%olic otatio,
Table 1=-.: chmod Symbolic >otation 1+amples
7otation >eaning
u8x
$!! e3ec"te permissio +or the ower.
u-x
:emo#e e3ec"te permissio +rom the ower.
8x
$!! e3ec"te permissio +or the ower, gro"p, a! worl!.
E="i#alet to a8x.
o-r9
:emo#e the rea! a! write permissio +rom ayoe %esi!es the
ower a! gro"p ower.
goMr9
Set the gro"p ower a! ayoe %esi!es the ower to ha#e rea! a!
write permissio. 9+ either the gro"p ower or worl! pre#io"sly ha!
e3ec"te permissios, they are remo#e!.
u8x>goMrx
$!! e3ec"te permissio +or the ower a! set the permissios +or
the gro"p a! others to rea! a! e3ec"te. >"ltiple speci+icatios
may %e separate! %y commas.
Some people pre+er to "se octal otatio, some +olks really like the sym%olic. Sym%olic
otatio !oes o++er the a!#atage o+ allowig yo" to set a sigle attri%"te witho"t
!ist"r%ig ay o+ the others.
Take a look at the chmod ma page +or more !etails a! a list o+ optios. $ wor! o+
ca"tio regar!ig the ?--rec"rsi#e@ optio, it acts o %oth +iles a! !irectories, so it<s ot
as "se+"l as oe wo"l! hope sice, we rarely wat +iles a! !irectories to ha#e the same
permissios.
Setting 2ie :ode With The +UI
&ow that we ha#e see how the permissios o +iles a! !irectories are set, we ca %etter
.00
:ea!ig, Writig, $! E3ec"tig
"!ersta! the permissio !ialogs i the ;)9. 9 %oth &a"til"s E;&7>EF a!
Mo="eror EM'EF, right-clickig a +ile or !irectory ico will e3pose a properties !ialog.
Aere is a e3ample +rom M'E (.2,
Aere we ca see the settigs +or the ower, gro"p, a! worl!. 9 M'E, clickig o the
?$!#ace! 6ermissios@ %"tto %rigs "p aother !ialog that allows yo" to set each o+
the mo!e attri%"tes i!i#i!"ally. $other #ictory +or "!ersta!ig %ro"ght to "s %y the
comma! lieL
umask A Set )e=aut 'ermissions
The umask comma! cotrols the !e+a"lt permissios gi#e to a +ile whe it is create!.
9t "ses octal otatio to e3press a mask o+ %its to %e remo#e! +rom a +ile<s mo!e
attri%"tes. *et<s take a look,
[me@linuxbox ~]$ rm -f foo.txt
[me@linuxbox ~]$ umask
(((!
[me@linuxbox ~]$ & foo.txt
.0.
Figure ': ?-1 3#. File
6roperties -ialog
.0 8 6ermissios
[me@linuxbox ~]$ ls -l foo.txt
-r9-r9-r-- # me me ( !((/-($-(. #%:"$ foo<txt
We +irst remo#e! ay ol! copy o+ foo<txt to make s"re we were startig +resh. &e3t,
we ra the umask comma! witho"t a arg"met to see the c"rret #al"e. 9t respo!e!
with the #al"e (((! Ethe #al"e ((!! is aother commo !e+a"lt #al"eF, which is the
octal represetatio o+ o"r mask. We e3t create a ew istace o+ the +ile foo<txt a!
o%ser#e its permissios.
We ca see that %oth the ower a! gro"p %oth get rea! a! write permissio, while
e#eryoe else oly gets rea! permissio. The reaso that worl! !oes ot ha#e write
permissio is %eca"se o+ the #al"e o+ the mask. *et<s repeat o"r e3ample, this time
settig the mask o"rsel#es,
[me@linuxbox ~]$ rm foo.txt
[me@linuxbox ~]$ umask ****
[me@linuxbox ~]$ & foo.txt
[me@linuxbox ~]$ ls -l foo.txt
-r9-r9-r9- # me me ( !((/-($-(. #%:"/ foo<txt
Whe we set the mask to (((( Ee++ecti#ely t"rig it o++F, we see that the +ile is ow
worl! writa%le. To "!ersta! how this works, we ha#e to look at octal "m%ers agai.
9+ we take the mask a! e3pa! it ito %iary, the compare it to the attri%"tes we ca see
what happes,
?riginal "ile mode
--- r9- r9- r9-
>as(
((( ((( ((( (#(
1esult
--- r9- r9- r--
9gore +or the momet the lea!ig Deros Ewe<ll get to those i a mi"teF a! o%ser#e that
where the . appears i o"r mask, a attri%"te was remo#e!Bi this case, the the worl!
write permissio. That<s what the mask !oes. E#erywhere a . appears i the %iary #al"e
o+ the mask, a attri%"te is "set. 9+ we look at a mask #al"e o+ ((!!, we ca see what it
!oes,
?riginal "ile mode
--- r9- r9- r9-
>as(
((( ((( (#( (#(
1esult
--- r9- r-- r--
.02
:ea!ig, Writig, $! E3ec"tig
$gai, where a . appears i the %iary #al"e, the correspo!ig attri%"te is "set. 6lay
with some #al"es Etry some se#esF to get "se! to how this works. Whe yo"<re !oe,
remem%er to clea "p,
[me@linuxbox ~]$ rm foo.txt# umask ***2
>ost o+ the time yo" wo<t ha#e to chage the maskH the !e+a"lt pro#i!e! %y yo"r
!istri%"tio will %e +ie. 9 some high-sec"rity sit"atios, howe#er, yo" will wat to
cotrol it.
Some Special Permissions
Tho"gh we "s"ally see a octal permissio mask e3presse! as a three !igit
"m%er, it is more techically correct to e3press it i +o"r !igits. WhyC Jeca"se,
i a!!itio to rea!, write, a! e3ec"te permissio, there are some other, less "se!,
permissio settigs.
The +irst o+ these is the setuid bit Eoctal 1000F. Whe applie! to a e3ec"ta%le
+ile, it sets the effective user *- +rom that o+ real "ser Ethe "ser act"ally r"ig
the programF to that o+ the program<s ower. >ost o+te this is gi#e to a +ew
programs owe! %y the s"per"ser. Whe a or!iary "ser r"s a program that is
?setuid root@ , the program r"s with the e++ecti#e pri#ileges o+ the s"per"ser.
This allows the program to access +iles a! !irectories that a or!iary "ser
wo"l! ormally %e prohi%ite! +rom accessig. Clearly, %eca"se this raises
sec"rity cocers, "m%er o+ set"i! programs m"st %e hel! to a a%sol"te
miim"m.
The seco! is the setgid bit Eoctal 2000F which, like the set"i! %it, chages the
effective group *- +rom the real group *- o+ the "ser to that o+ the +ile ower. 9+
the setgi! %it is set o a !irectory, ewly create! +iles i the !irectory will %e
gi#e the gro"p owership o+ the !irectory rather the gro"p owership o+ the +ile<s
creator. This is "se+"l i a share! !irectory whe mem%ers o+ a commo gro"p
ee! access to all the +iles i the !irectory, regar!less o+ the +ile ower<s primary
gro"p.
The thir! is calle! the sticky bit Eoctal .000F. This is a hol!o#er +rom aciet
)i3, where it was possi%le to mark a e3ec"ta%le +ile as ?ot swappa%le.@ 7
+iles, *i"3 igores the sticky %it, %"t i+ applie! to a !irectory, it pre#ets "sers
+rom !eletig or reamig +iles "less the "ser is either the ower o+ the !irectory,
.0(
.0 8 6ermissios
the ower o+ the +ile, or the s"per"ser. This is o+te "se! to cotrol access to a
share! !irectory, s"ch as 6tm7.
Aere are some e3amples o+ "sig chmod with sym%olic otatio to set these
special permissios. 0irst assigig set"i! to a program,
chmod u8s program
&e3t, assigig setgi! to a !irectory,
chmod g8s dir
0ially, assigig the sticky %it to a !irectory,
chmod 8t dir
Whe #iewig the o"tp"t +rom ls, yo" ca !etermie the special permissios.
Aere are some e3amples. 0irst, a program that is set"i!,
-r9sr-xr-x
$ !irectory that has the setgi! attri%"te,
dr9xr9sr-x
$ !irectory with the sticky %it set,
dr9xr9xr9t
Changing Identities
$t #ario"s times, we may +i! it ecessary to take o the i!etity o+ aother "ser. 7+te
we wat to gai s"per"ser pri#ileges to carry o"t some a!miistrati#e task, %"t it is also
possi%le to ?%ecome@ aother reg"lar "ser +or s"ch thigs as testig a acco"t. There
are three ways to take o a alterate i!etity,
.. *og o"t a! log %ack i as the alterate "ser.
2. )se the su comma!.
(. )se the sudo comma!.
We will skip the +irst techi="e sice we kow how to !o it a! it lacks the co#eiece
o+ the other two. 0rom withi o"r ow shell sessio, the su comma! allows yo" to
ass"me the i!etity o+ aother "ser, a! either start a ew shell sessio with that "ser<s
9's, or to iss"e a sigle comma! as that "ser. The sudo comma! allows a
a!miistrator to set "p a co+ig"ratio +ile calle! 6etc6sudoers, a! !e+ie speci+ic
comma!s that partic"lar "sers are permitte! to e3ec"te "!er a ass"me! i!etity. The
.01
Chagig 9!etities
choice o+ which comma! to "se is largely !etermie! %y which *i"3 !istri%"tio yo"
"se. 5o"r !istri%"tio pro%a%ly icl"!es %oth comma!s, %"t its co+ig"ratio will +a#or
either oe or the other. We<ll start with su.
su A !un A She With Substitute User And +rou5 I)s
The su comma! is "se! to start a shell as aother "ser. The comma! syta3 looks like
this,
su (-(l)) (user)
9+ the ?-l@ optio is icl"!e!, the res"ltig shell sessio is a login shell +or the speci+ie!
"ser. This meas that the "ser<s e#iromet is loa!e! a! the workig !irectory is
chage! to the "ser<s home !irectory. This is "s"ally what we wat. 9+ the "ser is ot
speci+ie!, the s"per"ser is ass"me!. &otice that EstragelyF the ?-l@ may %e a%%re#iate!
?-@, which is how it is most o+te "se!. To start a shell +or the s"per"ser, we wo"l! !o
this,
[me@linuxbox ~]$ su -
:ass9ord:
[root@linuxbox ~]^
$+ter eterig the comma!, we are prompte! +or the s"per"ser<s passwor!. 9+ it is
s"ccess+"lly etere!, a ew shell prompt appears i!icatig that this shell has s"per"ser
pri#ileges Ethe trailig ?P@ rather tha a ?[@F a! the c"rret workig !irectory is ow the
home !irectory +or the s"per"ser Eormally 6root.F 7ce i the ew shell, we ca carry
o"t comma!s as the s"per"ser. Whe +iishe!, type ?e3it@ to ret"r to the pre#io"s
shell,
[root@linuxbox ~]^ exit
[me@linuxbox ~]$
9t is also possi%le to e3ec"te a sigle comma! rather tha startig a ew iteracti#e
comma! %y "sig su this way,
su -c KcommandK
.02
.0 8 6ermissios
)sig this +orm, a sigle comma! lie is passe! to the ew shell +or e3ec"tio. 9t is
importat to eclose the comma! i ="otes, as we !o ot wat e3pasio to occ"r i o"r
shell, %"t rather i the ew shell,
[me@linuxbox ~]$ su -c %ls -l /root/*%
:ass9ord:
-r9------- # root root )"% !(()-(/-## ($:#0 6root6anaconda-ks<cfg
6root6+ail:
total (
[me@linuxbox ~]$
sudo A 6xe/ute A Command As Another User
The sudo comma! is like su i may ways, %"t has some importat a!!itioal
capa%ilities. The a!miistrator ca co+ig"re sudo to allow a or!iary "ser to e3ec"te
comma!s as a !i++eret "ser E"s"ally the s"per"serF i a #ery cotrolle! way. 9
partic"lar, a "ser may %e restricte! to oe or more speci+ic comma!s a! o others.
$other importat !i++erece is that the "se o+ sudo !oes ot re="ire access to the
s"per"ser<s passwor!. To a"theticate "sig sudo, the "ser "ses his-her ow passwor!.
*et<s say, +or e3ample, that sudo has %ee co+ig"re! to allow "s to r" a +ictitio"s
%ack"p program calle! ?%ack"pOscript@, which re="ires s"per"ser pri#ileges. With sudo
it wo"l! %e !oe like this,
[me@linuxbox ~]$ sudo backup.script
:ass9ord:
*1stem Hacku7 *tarting<<<
$+ter eterig the comma!, we are prompte! +or o"r passwor! Eot the s"per"ser<sF a!
oce the a"theticatio is complete, the speci+ie! comma! is carrie! o"t. 7e importat
!i++erece %etwee su a! sudo is that sudo !oes ot start a ew shell, or !oes it loa!
aother "ser<s e#iromet. This meas that comma!s !o ot ee! to %e ="ote! ay
!i++eretly tha they wo"l! %e witho"t "sig sudo. &ote that this %eha#ior ca %e
o#erri!!e %y speci+yig #ario"s optios. See the sudo ma page +or !etails.
To see what pri#ileges are grate! %y sudo, "se the ?-l@ optio to list them,
[me@linuxbox ~]$ sudo -l
3ser me ma1 run the follo9ing commands on this host:
.0K
Chagig 9!etities
N4==P 4==
-0untu #nd sudo
7e o+ the rec"rret pro%lems +or reg"lar "sers is how to per+orm certai tasks
that re="ire s"per"ser pri#ileges. These tasks icl"!e istallig a! "p!atig
so+tware, e!itig system co+ig"ratio +iles, a! accessig !e#ices. 9 the
Wi!ows worl!, this is o+te !oe %y gi#ig "sers a!miistrati#e pri#ileges. This
allows "sers to per+orm these tasks. Aowe#er, it also ea%les programs e3ec"te!
%y the "ser to ha#e the same a%ilities. This is !esira%le i most cases, %"t it also
permits malware Emalicio"s so+twareF s"ch as #ir"ses to ha#e +ree reig o+ the
comp"ter.
9 the )i3 worl!, there has always %ee a larger !i#isio %etwee reg"lar "sers
a! a!miistrators, owig to the m"lti-"ser heritage o+ )i3. The approach take
i )i3 is to grat s"per"ser pri#ileges oly whe ee!e!. To !o this, the su a!
sudo comma!s are commoly "se!.
)p "til a co"ple o+ years ago, most *i"3 !istri%"tios relie! o su +or this
p"rpose. su !i!<t re="ire the co+ig"ratio that sudo re="ire!, a! ha#ig a
root acco"t is tra!itioal i )i3. This itro!"ce! a pro%lem. )sers were
tempte! to operate as root "ecessarily. 9 +act, some "sers operate! their
systems as the root "ser e3cl"si#ely, sice it !oes away with all those aoyig
?permissio !eie!@ messages. This is how yo" re!"ce the sec"rity o+ a *i"3
system to that o+ a Wi!ows system. &ot a goo! i!ea.
Whe )%"t" was itro!"ce!, its creators took a !i++eret tack. Jy !e+a"lt,
)%"t" !isa%les logis to the root acco"t E%y +ailig to set a passwor! +or the
acco"tF, a! istea! "ses sudo to grat s"per"ser pri#ileges. The iitial "ser
acco"t is grate! +"ll access to s"per"ser pri#ileges #ia sudo a! may grat
similar powers to s"%se="et "ser acco"ts.
/ho%n A Change 2ie <%ner And +rou5
The cho9n comma! is "se! to chage the ower a! gro"p ower o+ a +ile or !irectory.
S"per"ser pri#ileges are re="ire! to "se this comma!. The syta3 o+ cho9n looks like
this,
.0/
.0 8 6ermissios
chown (owner)(2(group)) file...
cho9n ca chage the +ile ower a!-or the +ile gro"p ower !epe!ig o the +irst
arg"met o+ the comma!. Aere are some e3amples,
Table 1=-2: chown )rgument 1+amples
#rgument 1esults
bob
Chages the owership o+ the +ile +rom its c"rret ower to "ser
bob.
bob:users
Chages the owership o+ the +ile +rom its c"rret ower to "ser
bob a! chages the +ile gro"p ower to gro"p "sers.
:admins
Chages the gro"p ower to the gro"p admins. The +ile ower is
"chage!.
bob:
Chage the +ile ower +rom the c"rret ower to "ser bob a!
chages the gro"p ower to the logi gro"p o+ "ser bob.
*et<s say that we ha#e two "sersH janet, who has access to s"per"ser pri#ileges a!
ton1, who !oes ot. )ser janet wats to copy a +ile +rom her home !irectory to the
home !irectory o+ "ser ton1. Sice "ser janet wats ton1 to %e a%le to e!it the +ile,
janet chages the owership o+ the copie! +ile +rom janet to ton1,
[janet@linuxbox ~]$ sudo cp m!file.txt ~ton!
:ass9ord:
[janet@linuxbox ~]$ sudo ls -l ~ton!/m!file.txt
-r9-r--r-- # root root /($# !((/-($-!( #%:$( 6home6ton16m1file<txt
[janet@linuxbox ~]$ sudo chown ton!2 ~ton!/m!file.txt
[janet@linuxbox ~]$ sudo ls -l ~ton!/m!file.txt
-r9-r--r-- # ton1 ton1 /($# !((/-($-!( #%:$( 6home6ton16m1file<txt
Aere we see "ser janet copy the +ile +rom his !irectory to the home !irectory o+ "ser
ton1. &e3t, janet chages the owership o+ the +ile +rom root Ea res"lt o+ "sig
sudoF to ton1. )sig the trailig colo i the +irst arg"met, janet also chage! the
gro"p owership o+ the +ile to the logi gro"p o+ ton1, which happes to %e gro"p
ton1.
&otice that a+ter the +irst "se o+ sudo, janet was ot prompte! +or her passwor!C This
is %eca"se sudo, i most co+ig"ratios, ?tr"sts@ yo" +or se#eral mi"tes "til its timer
.08
Chagig 9!etities
r"s o"t.
/hgr5 A Change +rou5 <%nershi5
9 ol!er #ersios o+ )i3, the cho9n comma! oly chage! +ile owership, ot gro"p
owership. 0or that p"rpose, a separate comma!, chgr7 was "se!. 9t works m"ch the
same way as cho9n, e3cept +or %eig more limite!.
$xercising ?ur Pri%ileges
&ow that we ha#e leare! how this permissios thig works, it<s time to show it o++. We
are goig to !emostrate the sol"tio to a commo pro%lemBsettig "p a share!
!irectory. *et<s imagie that we ha#e two "sers ame! ?%ill@ a! ?kare.@ They %oth
ha#e m"sic C' collectios a! wish to set "p a share! !irectory, where they will each
store their m"sic +iles as 7gg Ror%is or >6(. )ser bill has access to s"per"ser
pri#ileges #ia sudo.
The +irst thig that ee!s to happe is creatig a gro"p that will ha#e %oth bill a!
karen as mem%ers. )sig the graphical "ser maagemet tool, bill creates a gro"p
calle! music a! a!!s "sers bill a! karen to it,
&e3t, bill creates the !irectory +or the m"sic +iles,
.09
Figure 3: %reating ) >ew @roup /ith @>&!1
.0 8 6ermissios
[bill@linuxbox ~]$ sudo mkdir /usr/local/share/Ousic
:ass9ord:
Sice bill is maip"latig +iles o"tsi!e his home !irectory, s"per"ser pri#ileges are
re="ire!. $+ter the !irectory is create!, it has the +ollowig owerships a! permissios,
[bill@linuxbox ~]$ ls -ld /usr/local/share/Ousic
dr9xr-xr-x ! root root %(0. !((/-($-!# #/:(" 6usr6local6share6+usic
$s we ca see, the !irectory is owe! %y root a! has /22 permissios. To make this
!irectory shara%le, bill ee!s to chage the gro"p owership a! the gro"p permissios
to allow writig,
[bill@linuxbox ~]$ sudo chown 2music /usr/local/share/Ousic
[bill@linuxbox ~]$ sudo chmod CC" /usr/local/share/Ousic
[bill@linuxbox ~]$ ls -ld /usr/local/share/Ousic
dr9xr9xr-x ! root music %(0. !((/-($-!# #/:(" 6usr6local6share6+usic
So what !oes this all meaC 9t meas that we ow ha#e a !irectory,
6usr6local6share6+usic that is owe! %y root a! allows rea! a! write
access to gro"p music. ;ro"p music has mem%ers bill a! karen, th"s bill a!
karen ca create +iles i !irectory 6usr6local6share6+usic. 7ther "sers ca
list the cotets o+ the !irectory %"t caot create +iles there.
J"t we still ha#e a pro%lem. With the c"rret permissios, +iles a! !irectories create!
withi the >"sic !irectory will ha#e the ormal permissios o+ the "sers bill a!
karen,
[bill@linuxbox ~]$ & /usr/local/share/Ousic/test.file
[bill@linuxbox ~]$ ls -l /usr/local/share/Ousic
-r9-r--r-- # bill bill ( !((/-($-!% !(:($ testOfile
$ct"ally there are two pro%lems. 0irst, the !e+a"lt umask o this system is 0022 which
pre#ets gro"p mem%ers +rom writig +iles %elogig to other mem%ers o+ the gro"p.
This wo"l! ot %e a pro%lem i+ the share! !irectory oly cotaie! +iles, %"t sice this
!irectory will store m"sic, a! m"sic is "s"ally orgaiDe! i a hierarchy o+ artists a!
al%"ms, mem%ers o+ the gro"p will ee! the a%ility to create +iles a! !irectories isi!e
!irectories create! %y other mem%ers. We ee! to chage the umask "se! %y bill a!
..0
E3ercisig 7"r 6ri#ileges
karen to 0002 istea!.
Seco!, each +ile a! !irectory create! %y oe mem%er will %e set to the primary gro"p o+
the "ser rather tha the gro"p music. This ca %e +i3e! %y settig the setgi! %it o the
!irectory,
[bill@linuxbox ~]$ sudo chmod g5s /usr/local/share/Ousic
[bill@linuxbox ~]$ ls -ld /usr/local/share/Ousic
dr9xr9sr-x ! root music %(0. !((/-($-!% !(:($ 6usr6local6share6+usic
&ow we test to see i+ the ew permissios +i3 the pro%lem. bill sets his umask to
0002, remo#es the pre#io"s test +ile, creates a ew test +ile a! !irectory,
[bill@linuxbox ~]$ umask ***2
[bill@linuxbox ~]$ rm /usr/local/share/Ousic/test.file
[bill@linuxbox ~]$ & /usr/local/share/Ousic/test.file
[bill@linuxbox ~]$ mkdir /usr/local/share/Ousic/test.dir
[bill@linuxbox ~]$ ls -l /usr/local/share/Ousic
dr9xr9sr-x ! bill music %(0. !((/-($-!% !(:!% testOdir
-r9-r9-r-- # bill music ( !((/-($-!% !(:!! testOfile
[bill@linuxbox ~]$
Joth +iles a! !irectories are ow create! with the correct permissios to allow all
mem%ers o+ the gro"p music to create +iles a! !irectories isi!e the +usic !irectory.
The oe remaiig iss"e is umask. The ecessary settig oly lasts "til the e! o+
sessio a! m"st %e reset. 9 the e3t part o+ the %ook, we<ll look at makig the chage to
umask permaet.
Changing :our Pass4ord
The last topic we<ll co#er i this chapter is settig passwor!s +or yo"rsel+ Ea! +or other
"sers i+ yo" ha#e access to s"per"ser pri#ileges.F To set or chage a passwor!, the
7ass9d comma! is "se!. The comma! syta3 looks like this,
passwd (user)
To chage yo"r passwor!, 4"st eter the 7ass9d comma!. 5o" will %e prompte! +or
yo"r ol! passwor! a! yo"r ew passwor!,
...
.0 8 6ermissios
[me@linuxbox ~]$ passwd
NcurrentP 3LAV 7ass9ord:
Le9 3LAV 7ass9ord:
The 7ass9d comma! will try to e+orce "se o+ ?strog@ passwor!s. This meas the it
will re+"se to accept passwor!s that are too short, too similar to pre#io"s passwor!s, are
!ictioary wor!s, or too easily g"esse!,
[me@linuxbox ~]$ passwd
NcurrentP 3LAV 7ass9ord:
Le9 3LAV 7ass9ord:
H4' :4**,R': is too similar to the old one
Le9 3LAV 7ass9ord:
H4' :4**,R': it is ,4U too short
Le9 3LAV 7ass9ord:
H4' :4**,R': it is based on a dictionar1 9ord
9+ yo" ha#e s"per"ser pri#ileges, yo" ca speci+y a "ser ame as a arg"met to the
7ass9d comma! to set the passwor! +or aother "ser. There are other optios
a#aila%le to the s"per"ser to allow acco"t lockig, passwor! e3piratio, etc. See the
7ass9d ma page +or !etails.
;urther 1eading
Wikipe!ia has a goo! article o malware,
http,--e.wikipe!ia.org-wiki->alware
There are "m%er o+ comma! lie programs "se! to create a! maitai "sers a!
gro"ps. 0or more i+ormatio, see the ma pages +or the +ollowig comma!s,
adduser
useradd
grou7add
..2
.. 8 6rocesses
11 2rocesses
>o!er operatig systems are "s"ally multitasking, meaig that they create the ill"sio
o+ !oig more tha oe thig at oce %y rapi!ly switchig +rom oe e3ec"tig program to
aother. The *i"3 kerel maages this thro"gh the "se o+ processes. 6rocesses are how
*i"3 orgaiDes the !i++eret programs waitig +or their t"r at the C6).
Sometimes a comp"ter will %ecome sl"ggish or a applicatio will stop respo!ig. 9
this chapter, we will look at some o+ the tools a#aila%le at the comma! lie that let "s
e3amie what programs are !oig, a! how to termiate processes that are mis%eha#ig.
This chapter will itro!"ce the +ollowig comma!s,
7s 8 :eport a sapshot o+ c"rret processes
to7 8 'isplay tasks
jobs 8 *ist acti#e 4o%s
bg 8 6lace a 4o% i the %ackgro"!
fg 8 6lace a 4o% i the +oregro"!
kill 8 Se! a sigal to a process
killall 8 Mill processes %y ame
shutdo9n 8 Sh"t!ow or re%oot the system
3o4 # Process +or(s
Whe a system starts "p, the kerel iitiates a +ew o+ its ow acti#ities as processes a!
la"ches a program calle! init. init, i t"r, r"s a series o+ shell scripts Elocate! i
6etcF calle! init scripts, which start all the system ser#ices. >ay o+ these ser#ices are
implemete! as daemon programs, programs that 4"st sit i the %ackgro"! a! !o their
thig witho"t ha#ig ay "ser iter+ace. So e#e i+ we are ot logge! i, the system is at
least a little %"sy per+ormig ro"tie st"++.
The +act that a program ca la"ch other programs is e3presse! i the process scheme as
a parent process pro!"cig a child process.
..(
.. 8 6rocesses
The kerel maitais i+ormatio a%o"t each process to help keep thigs orgaiDe!. 0or
e3ample, each process is assige! a "m%er calle! a process *- or 6*-. 69's are
assige! i asce!ig or!er, with init always gettig 69' .. The kerel also keeps
track o+ the memory assige! to each process, as well as the processes< rea!iess to
res"me e3ec"tio. *ike +iles, processes also ha#e owers a! "ser 9's, e++ecti#e "ser
9's, etc.
Vie4ing Processes
The most commoly "se! comma! to #iew processes Ethere are se#eralF is 7s. The 7s
program has a lot o+ optios, %"t i it simplest +orm it is "se! like this,
[me@linuxbox ~]$ ps
:A' TTU TA+& C+'
"#0/ 7ts6# ((:((:(( bash
#(#!0 7ts6# ((:((:(( 7s
The res"lt i this e3ample lists two processes, process 2.98 a! process .0.29, which are
bash a! 7s respecti#ely. $s we ca see, %y !e+a"lt, 7s !oes<t show "s #ery m"ch,
4"st the processes associate! with the c"rret termial sessio. To see more, we ee! to
a!! some optios, %"t %e+ore we !o that, let<s look at the other +iel!s pro!"ce! %y 7s.
TTU is short +or ?Teletype,@ a! re+ers to the controlling terminal +or the process. )i3
is showig its age here. The TA+& +iel! is the amo"t o+ C6) time cos"me! %y the
process. $s we ca see, either process makes the comp"ter work #ery har!.
9+ we a!! a optio, we ca get a %igger pict"re o+ what the system is !oig,
[me@linuxbox ~]$ ps x
:A' TTU *T4T TA+& C++4L'
!)00 F *sl (:(( 6usr6libexec6bonobo-activation-server Wac
!/!( F *l (:(# 6usr6libexec6evolution-data-server-#<#( --
#".%) F *s (:(( 6bin6sh 6usr6bin6startkde
#")"# F *s (:(( 6usr6bin6ssh-agent 6usr6bin6dbus-launch --
#")"% F * (:(( 6usr6bin6dbus-launch --exit-9ith-session
#")"" F *s (:(# 6bin6dbus-daemon --fork --7rint-7id % W7r
#"))% F *s (:(! 6usr6bin6g7g-agent -s Wdaemon
#")0$ F * (:(( startOkdeinit --ne9-startu7 8kcminitOstart
#")0% F *s (:(( kdeinit Running<<<
#")0) F * (:(( dco7server Wnosid
and man1 more<<<
..1
Riewig 6rocesses
$!!ig the ?3@ optio Eote that there is o lea!ig !ashF tells 7s to show all o+ o"r
processes regar!less o+ what termial Ei+ ayF they are cotrolle! %y. The presece o+ a
?C@ i the TTU col"m i!icates o cotrollig termial. )sig this optio, we see a list
o+ e#ery process that we ow.
Sice the system is r"ig a lot o+ processes, 7s pro!"ces a log list. 9t is o+te help+"l
to pipe the o"tp"t +rom 7s ito less +or easier #iewig. Some optio com%iatios also
pro!"ce log lies o+ o"tp"t, so ma3imiDig the termial em"lator wi!ow may %e a
goo! i!ea, too.
$ ew col"m title! *T4T has %ee a!!e! to the o"tp"t. *T4T is short +or ?state@ a!
re#eals the c"rret stat"s o+ the process,
Table 11-1: 6rocess States
State >eaning
R
:"ig. This meas that the process is r"ig or rea!y to r".
*
Sleepig. $ process is ot r"igH rather, it is waitig +or a e#et,
s"ch as a keystroke or etwork packet.
'
)iterr"pti%le Sleep. 6rocess is waitig +or 9-7 s"ch as a !isk
!ri#e.
T
Stoppe!. 6rocess has %ee istr"cte! to stop. >ore o this later.
I
$ !e+"ct or ?Dom%ie@ process. This is a chil! process that has
termiate!, %"t has ot %ee cleae! "p %y its paret.
X
$ high priority process. 9t<s possi%le to grat more importace to a
process, gi#ig it more time o the C6). This property o+ a process
is calle! niceness. $ process with high priority is sai! to %e less
nice %eca"se it<s takig more o+ the C6)<s time, which lea#es less
+or e#ery%o!y else.
L
$ low priority process. $ process with low priority Ea ?ice@
processF will oly get processor time a+ter other processes with
higher priority ha#e %ee ser#ice!.
The process state may %e +ollowe! %y other characters. These i!icate #ario"s e3otic
process characteristics. See the 7s ma page +or more !etail.
$other pop"lar set o+ optios is ?a"3@ Ewitho"t a lea!ig !ashF. This gi#es "s e#e
more i+ormatio,
..2
.. 8 6rocesses
[me@linuxbox ~]$ ps aux
3*&R :A' 5C:3 5+&+ ;*I R** TTU *T4T *T4RT TA+& C++4L'
root # (<( (<( !#$. .%% F *s +ar(" (:$# init
root ! (<( (<( ( ( F *X +ar(" (:(( [kt]
root $ (<( (<( ( ( F *X +ar(" (:(( [mi]
root % (<( (<( ( ( F *X +ar(" (:(( [ks]
root " (<( (<( ( ( F *X +ar(" (:(. [9a]
root . (<( (<( ( ( F *X +ar(" (:$. [ev]
root ) (<( (<( ( ( F *X +ar(" (:(( [kh]
and man1 more<<<
This set o+ optios !isplays the processes %elogig to e#ery "ser. )sig the optios
witho"t the lea!ig !ash i#okes the comma! with ?JS' style@ %eha#ior. The *i"3
#ersio o+ 7s ca em"late the %eha#ior o+ the 7s program +o"! i se#eral !i++eret
)i3 implemetatios. With these optios, we get these a!!itioal col"ms,
Table 11-': 8S- Style ps %olumn ,eaders
3eader >eaning
3*&R
)ser 9'. This is the ower o+ the process.
5C:3
C6) "sage i percet.
5+&+
>emory "sage i percet.
;*I
Rirt"al memory siDe.
R**
:esi!et Set SiDe. The amo"t o+ physical memory E:$>F the
process is "sig i kilo%ytes.
*T4RT
Time whe the process starte!. 0or #al"es o#er twety +o"r ho"rs,
a !ate is "se!.
?ie%ing 'ro/esses )ynami/ay With to7
While the 7s comma! ca re#eal a lot a%o"t what the machie is !oig, it pro#i!es oly
a sapshot o+ the machie<s state at the momet the 7s comma! is e3ec"te!. To see a
more !yamic #iew o+ the machie<s acti#ity, we "se the to7 comma!,
[me@linuxbox ~]$ top
..K
Riewig 6rocesses
The to7 program !isplays a coti"o"sly "p!atig E%y !e+a"lt, e#ery ( seco!sF !isplay
o+ the system processes liste! i or!er o+ process acti#ity. The ame ?top@ comes +rom
the +act that the to7 program is "se! to see the ?top@ processes o the system. The to7
!isplay cosists o+ two parts, a system s"mmary at the top o+ the !isplay, +ollowe! %y a
ta%le o+ processes sorte! %y C6) acti#ity,
to7 - #%:"0:!( u7 .:$(> ! users> load average: (<()> (<(!> (<((
Tasks: #(0 total> # running> #(. slee7ing> ( sto77ed> ! Combie
C7uNsP: (<)5us> #<(5s1> (<(5ni> 0/<$5id> (<(59a> (<(5hi> (<(5si
+em: $#0%0.k total> $#%/.(k used> %.$.k free> #0$0!k buff
*9a7: /)""((k total> #%0#!/k used> )!.$)!k free> ##%.).k cach
:A' 3*&R :R LA ;ART R&* *SR * 5C:3 5+&+ TA+&8 C++4L'
.!%% me $0 #0 $#)"! $#!% !#// * .<$ #<( #.:!%<%! trackerd
##()# me !( ( !$(% #(0! /%( R #<$ (<$ (:((<#% to7
.#/( me !( ( !)(( ##(( ))! * (<) (<$ (:($<.. dbus-dae
.$!# me !( ( !(0%% )!%/ .".( * (<) !<$ !:"#<$/ multiloa
%0"" root !( ( #(%m 0../ ")). * (<$ $<( !:#0<$0 Vorg
# root !( ( !0). "!/ %). * (<( (<! (:($<#% init
! root #" -" ( ( ( * (<( (<( (:((<(( kthreadd
$ root RT -" ( ( ( * (<( (<( (:((<(( migratio
% root #" -" ( ( ( * (<( (<( (:((<)! ksoftirB
" root RT -" ( ( ( * (<( (<( (:((<(% 9atchdog
. root #" -" ( ( ( * (<( (<( (:((<%! events6(
) root #" -" ( ( ( * (<( (<( (:((<(. khel7er
%# root #" -" ( ( ( * (<( (<( (:(#<(/ kblockd6
.) root #" -" ( ( ( * (<( (<( (:((<(( kseriod
##% root !( ( ( ( ( * (<( (<( (:(#<.! 7dflush
##. root #" -" ( ( ( * (<( (<( (:(!<%% ks9a7d(
The system s"mmary cotais a lot o+ goo! st"++. Aere<s a r"!ow,
Table 11-3: top *nformation Fields
1o4 ;ield >eaning
.
to7
&ame o+ the program.
#%:"0:!(
C"rret time o+ !ay.
u7 .:$(
This is calle! uptime. 9t is the amo"t o+ time
sice the machie was last %oote!. 9 this
e3ample, the system has %ee "p +or si3 a! a
hal+ ho"rs.
! users
There are two "sers logge! i.
load average:
(oad average re+ers to the "m%er o+ processes
../
.. 8 6rocesses
that are waitig to r", that is, the "m%er o+
processes that are i a r"a%le state a! are
sharig the C6). Three #al"es are show,
each +or a !i++eret perio! o+ time. The +irst is
the a#erage +or the last K0 seco!s, the e3t the
pre#io"s 2 mi"tes, a! +ially the pre#io"s .2
mi"tes. Ral"es "!er ..0 i!icate that the
machie is ot %"sy.
2
Tasks:
This s"mmariDes the "m%er o+ processes a!
their #ario"s process states.
(
C7uNsP:
This row !escri%es the character o+ the
acti#ities that the C6) is per+ormig.
(<)5us
0./_ o+ the C6) is %eig "se! +or user
processes. This meas processes o"tsi!e o+ the
kerel itsel+.
#<(5s1
..0_ o+ the C6) is %eig "se! +or system
EkerelF processes.
(<(5ni
0.0_ o+ the C6) is %eig "se! %y ?ice@ Elow
priorityF processes.
0/<$5id
98.(_ o+ the C6) is i!le.
(<(59a
0.0_ o+ the C6) is waitig +or 9-7.
1
+em:
Shows how physical :$> is %eig "se!.
2
*9a7:
Shows how swap space E#irt"al memoryF is
%eig "se!.
The top program accepts a "m%er o+ key%oar! comma!s. The two most iterestig are
h, which !isplays the program<s help scree, a! B, which ="its to7.
Joth ma4or !esktop e#iromets pro#i!e graphical applicatios that !isplay i+ormatio
similar to to7 Ei m"ch the same way that Task >aager i Wi!ows worksF, %"t 9 +i!
that to7 is %etter tha the graphical #ersios %eca"se it is +aster a! it cos"mes +ar
+ewer system reso"rces. $+ter all, o"r system moitor program sho"l!<t %e the so"rce o+
the system slow!ow that we are tryig to track.
Controlling Processes
&ow that we ca see a! moitor processes, let<s gai some cotrol o#er them. 0or o"r
..8
Cotrollig 6rocesses
e3perimets, we<re goig to "se a little program calle! xlogo as o"r g"iea pig. The
xlogo program is a sample program s"pplie! with the G Wi!ow System Ethe
"!erlyig egie that makes the graphics o o"r !isplay goF which simply !isplays a re-
siDa%le wi!ow cotaiig the G logo. 0irst, we<ll get to kow o"r test s"%4ect,
[me@linuxbox ~]$ xlogo
$+ter eterig the comma!, a small wi!ow cotaiig the logo sho"l! appear
somewhere o the scree. 7 some systems, xlogo may prit a warig message, %"t it
may %e sa+ely igore!.
Ti(5 9+ yo"r system !oes ot icl"!e the xlogo program, try "sig gedit or
k9rite istea!.
We ca #eri+y that xlogo is r"ig %y resiDig its wi!ow. 9+ the logo is re!raw i the
ew siDe, the program is r"ig.
&otice how o"r shell prompt has ot ret"re!C This is %eca"se the shell is waitig +or the
program to +iish, 4"st like all the other programs we ha#e "se! so +ar. 9+ we close the
xlogo wi!ow, the prompt ret"rs.
Interru5ting A 'ro/ess
*et<s o%ser#e what happes whe we r" xlogo agai. 0irst, eter the xlogo
comma! a! #eri+y that the program is r"ig. &e3t, ret"r to the termial wi!ow
a! type Ctrl-c.
[me@linuxbox ~]$ xlogo
[me@linuxbox ~]$
9 a termial, typig Ctrl-c, interrupts a program. This meas that we politely aske!
the program to termiate. $+ter typig Ctrl-c, the xlogo wi!ow close! a! the shell
prompt ret"re!.
>ay E%"t ot allF comma! lie programs ca %e iterr"pte! %y "sig this techi="e.
'utting A 'ro/ess In The Ba/kground
*et<s say we wate! to get the shell prompt %ack witho"t termiatig the xlogo
..9
.. 8 6rocesses
program. WeIll !o this %y placig the program i the background. Thik o+ the termial
as ha#ig a foreground Ewith st"++ #isi%le o the s"r+ace like the shell promptF a! a
%ackgro"! Ewith hi!!e st"++ %ehi! the s"r+ace.F To la"ch a program so that it is
imme!iately place! i the %ackgro"!, we +ollow the comma! with a- ?^@ character,
[me@linuxbox ~]$ xlogo '
[#] !/!$.
[me@linuxbox ~]$
$+ter eterig the comma!, the xlogo wi!ow appeare! a! the shell prompt ret"re!,
%"t some +"y "m%ers were prite! too. This message is part o+ a shell +eat"re calle!
Aob control. With this message, the shell is tellig "s that we ha#e starte! 4o% "m%er .
E?[#]@F a! that it has 69' 282(K. 9+ we r" 7s, we ca see o"r process,
[me@linuxbox ~]$ ps
:A' TTU TA+& C+'
#(.($ 7ts6# ((:((:(( bash
!/!$. 7ts6# ((:((:(( xlogo
!/!$0 7ts6# ((:((:(( 7s
The shell<s 4o% cotrol +acility also gi#es "s a way to list the 4o%s that are ha#e %ee
la"che! +rom o"r termial. )sig the jobs comma!, we ca see this list,
[me@linuxbox ~]$ jobs
[#]8 Running xlogo Y
The res"lts show that we ha#e oe 4o%, "m%ere! ?.@, that it is r"ig, a! that the
comma! was xlogo Y.
!eturning A 'ro/ess To The 2oreground
$ process i the %ackgro"! is imm"e +rom key%oar! ip"t, icl"!ig ay attempt
iterr"pt it with a Ctrl-c. To ret"r a process to the +oregro"!, "se the fg comma!,
this way,
[me@linuxbox ~]$ jobs
[#]8 Running xlogo Y
[me@linuxbox ~]$ fg 81
.20
Cotrollig 6rocesses
xlogo
The comma! fg +ollowe! %y a percet sig a! the 4o% "m%er Ecalle! a AobspecF !oes
the trick. 9+ we oly ha#e oe %ackgro"! 4o%, the 4o%spec is optioal. To termiate
xlogo, type Ctrl-c.
Sto55ing E'ausingF A 'ro/ess
Sometimes we<ll wat to stop a process witho"t termiatig it. This is o+te !oe to
allow a +oregro"! process to %e mo#e! to the %ackgro"!. To stop a +oregro"!
process, type Ctrl-C. *et<s try it. $t the comma! prompt, type xlogo, the &nter
key, the Ctrl-C,
[me@linuxbox ~]$ xlogo
[#]8 *to77ed xlogo
[me@linuxbox ~]$
$+ter stoppig xlogo, we ca #eri+y that the program has stoppe! %y attemptig to
resiDe the xlogo wi!ow. We will see that it appears ="ite !ea!. We ca either restore
the program to the +oregro"!, "sig the fg comma!, or mo#e the program to the
%ackgro"! with the bg comma!,
[me@linuxbox ~]$ bg 81
[#]8 xlogo Y
[me@linuxbox ~]$
$s with the fg comma!, the 4o%spec is optioal i+ there is oly oe 4o%.
>o#ig a process +rom the +oregro"! to the %ackgro"! is ha!y i+ we la"ch a
graphical program +rom the comma!, %"t +orget to place it i the %ackgro"! %y
appe!ig the trailig ?^@.
Why wo"l! yo" wat to la"ch a graphical program +rom the comma! lieC There are
two reasos. 0irst, the program yo" wish to r" might ot %e liste! o the wi!ow
maager<s me"s Es"ch as xlogoF. Seco!ly, %y la"chig a program +rom the
comma! lie, yo" might %e a%le to see error messages that wo"l! otherwise %e i#isi%le
i+ the program were la"che! graphically. Sometimes, a program will +ail to start "p
whe la"che! +rom the graphical me". Jy la"chig it +rom the comma! lie istea!,
we may see a error message that will re#eal the pro%lem. $lso, some graphical
programs ha#e may iterestig a! "se+"l comma! lie optios.
.2.
.. 8 6rocesses
Signals
The kill comma! is "se! to ?kill@ programs. This allows "s to termiate programs
that ee! killig. Aere<s a e3ample,
[me@linuxbox ~]$ xlogo Y
[#] !/%(#
[me@linuxbox ~]$ kill !/%(#
[#]8 Terminated xlogo
We +irst la"ch xlogo i the %ackgro"!. The shell prits the 4o%spec a! the 69' o+
the %ackgro"! process. &e3t, we "se the kill comma! a! speci+y the 69' o+ the
process we wat to termiate. We co"l! ha#e also speci+ie! the process "sig a 4o%spec
E+or e3ample, ?_.@F istea! o+ a 69'.
While this is all #ery straight+orwar!, there is more to it tha that. The kill comma!
!oes<t e3actly ?kill@ programs, rather it se!s them signals. Sigals are oe o+ se#eral
ways that the operatig system comm"icates with programs. We ha#e alrea!y see
sigals i actio with the "se o+ Ctrl-c a! Ctrl-C. Whe the termial recei#es oe
o+ these keystrokes, it se!s a sigal to the program i the +oregro"!. 9 the case o+
Ctrl-c, a sigal calle! ALT E9terr"ptF is setH with Ctrl-C, a sigal calle! T*T:
ETermial Stop.F 6rograms, i t"r, ?liste@ +or sigals a! may act "po them as they
are recei#e!. The +act that a program ca liste a! act "po sigals allows a program to
!o thigs like sa#e work i progress whe it is set a termiatio sigal.
Sending Signas To 'ro/esses With kill
The kill comma! is "se! to se! sigals to programs. 9ts most commo syta3 looks
like this,
kill (-signal) BP1...
9+ o sigal is speci+ie! o the comma! lie, the the T&R+ ETermiateF sigal is set %y
!e+a"lt. The kill comma! is most o+te "se! to se! the +ollowig sigals,
Table 11-$: %ommon Signals
7um0er 7ame >eaning
# S3:
Aag"p. This is a #estige o+ the goo! ol! !ays
whe termials were attache! to remote
.22
Sigals
comp"ters with phoe lies a! mo!ems. The
sigal is "se! to i!icate to programs that the
cotrollig termial has ?h"g "p.@ The e++ect o+
this sigal ca %e !emostrate! %y closig a
termial sessio. The +oregro"! program
r"ig o the termial will %e set the sigal a!
will termiate.
This sigal is also "se! %y may !aemo
programs to ca"se a reiitialiDatio. This meas
that whe a !aemo is set this sigal, it will
restart a! re-rea! its co+ig"ratio +ile. The
$pache we% ser#er is a e3ample o+ a !aemo
that "ses the S3: sigal i this way.
! ALT
9terr"pt. 6er+orms the same +"ctio as the
Ctrl-c key set +rom the termial. 9t will
"s"ally termiate a program.
0 2A==
Mill. This sigal is special. Whereas programs
may choose to ha!le sigals set to them i
!i++eret ways, icl"!ig igorig them all
together, the 2A== sigal is e#er act"ally set to
the target program. :ather, the kerel
imme!iately termiates the process. Whe a
process is termiate! i this maer, it is gi#e o
opport"ity to ?clea "p@ a+ter itsel+ or sa#e its
work. 0or this reaso, the 2A== sigal sho"l!
oly %e "se! as a last resort whe other
termiatio sigals +ail.
#" T&R+
Termiate. This is the !e+a"lt sigal set %y the
kill comma!. 9+ a program is still ?ali#e@
eo"gh to recei#e sigals, it will termiate.
#/ CLT
Coti"e. This will restore a process a+ter a
*T: sigal.
#0 *T:
Stop. This sigal ca"ses a process to pa"se
witho"t termiatig. *ike the 2A== sigal, it is
ot set to the target process, a! th"s it caot %e
igore!.

.2(
.. 8 6rocesses
*et<s try o"t the kill comma!,
[me@linuxbox ~]$ xlogo '
[#] #$"%.
[me@linuxbox ~]$ kill -1 1"AN
[#]8 Sangu7 xlogo
9 this e3ample, we start the xlogo program i the %ackgro"! a! the se! it a S3:
sigal with kill. The xlogo program termiates a! the shell i!icates that the
%ackgro"! process has recei#e! a hag"p sigal. 5o" may ee! to press the eter key a
co"ple o+ times %e+ore yo" see the message. &ote that sigals may %e speci+ie! either %y
"m%er or %y ame, icl"!ig the ame pre+i3e! with the letters ?S9;@,
[me@linuxbox ~]$ xlogo '
[#] #$.(#
[me@linuxbox ~]$ kill -P?+ 1N*1
[#]8 Anterru7t xlogo
[me@linuxbox ~]$ xlogo '
[#] #$.(/
[me@linuxbox ~]$ kill -FPQP?+ 1N*M
[#]8 Anterru7t xlogo
:epeat the e3ample a%o#e a! try o"t the other sigals. :emem%er, yo" ca also "se
4o%specs i place o+ 69's.
6rocesses, like +iles, ha#e owers, a! yo" m"st %e the ower o+ a process Eor the
s"per"serF i or!er to se! it sigals with kill.
9 a!!itio to the list o+ sigals a%o#e, which are most o+te "se! with kill, there are
other sigals +re="etly "se! %y the system. Aere is a list o+ other commo sigals,
Table 11-.: &ther %ommon Signals
7um0er 7ame >eaning
$ Z3AT
T"it.
## *&@;
Segmetatio Riolatio. This sigal is set i+ a
program makes illegal "se o+ memory, that is, it
trie! to write somewhere it was ot allowe! to.
!( T*T:
Termial Stop. This is the sigal set %y the
termial whe the Ctrl-C key is presse!.
)like the *T: sigal, the T*T: sigal is
.21
Sigals
recei#e! %y the process a! may %e igore!.
!/ ,ALCS
Wi!ow Chage. This is a sigal set %y the
system whe a wi!ow chages siDe. Some
programs , like to7 a! less will respo! to
this sigal %y re!rawig themsel#es to +it the ew
wi!ow !imesios.
0or the c"rio"s, a complete list o+ sigals ca %e see with the +ollowig comma!,
[me@linuxbox ~]$ kill -l
Sending Signas To :uti5e 'ro/esses With killall
9t<s also possi%le to se! sigals to m"ltiple processes matchig a speci+ie! program or
"ser ame %y "sig the killall comma!. Aere is the syta3,
killall (-u user) (-signal) name...
To !emostrate, we will start a co"ple o+ istaces o+ the xlogo program a! the
termiate them,
[me@linuxbox ~]$ xlogo '
[#] #//(#
[me@linuxbox ~]$ xlogo '
[!] #//(!
[me@linuxbox ~]$ killall xlogo
[#]- Terminated xlogo
[!]8 Terminated xlogo
:emem%er, as with kill, yo" m"st ha#e s"per"ser pri#ileges to se! sigals to
processes that !o ot %elog to yo".
>ore Process 1elated Commands
Sice moitorig processes is a importat system a!miistratio task, there are a lot o+
comma!s +or it. Aere are some to play with,
.22
.. 8 6rocesses
Table 11-2: &ther 6rocess 9elated %ommands
Command Description
7stree
7"tp"ts a process list arrage! i a tree-like patter showig the
paret-chil! relatioships %etwee processes.
vmstat
7"tp"ts a sapshot o+ system reso"rce "sage icl"!ig, memory,
swap a! !isk 9-7. To see a coti"o"s !isplay, +ollow the
comma! with a time !elay Ei seco!sF +or "p!ates. 0or e3ample,
vmstat ". Termiate the o"tp"t with Ctrl-c.
xload
$ graphical program that !raws a graph showig system loa! o#er
time.
tload
Similar to the xload program, %"t !raws the graph i the termial.
Termiate the o"tp"t with Ctrl-c.
.2K
6art ( 8 Co+ig"ratio $! The E#iromet
'art 3 A Con=iguration And The
6n;ironment
.2/
.2 8 The E#iromet
12 The !nviron%ent
$s we !isc"sse! earlier, the shell maitais a %o!y o+ i+ormatio !"rig o"r shell
sessio calle! the environment. 'ata store! i the e#iromet is "se! %y programs to
!etermie +acts a%o"t o"r co+ig"ratio. While most programs "se configuration files to
store program settigs, some programs will also look +or #al"es store! i the e#iromet
to a!4"st their %eha#ior. Mowig this, we ca "se the e#iromet to c"stomiDe o"r
shell e3periece.
9 this chapter, we will work with the +ollowig comma!s,
7rintenv 8 6rit part or all o+ the e#iromet
set 8 Set shell optios
ex7ort 8 E3port e#iromet to s"%se="etly e3ec"te! programs
alias 8 Create a alias +or a comma!
+hat Is Stored In The $n%ironment.
The shell stores two %asic types o+ !ata i the e#iromet, tho"gh, with bash> the
types are largely i!istig"isha%le. They are environment variables a! shell variables.
Shell #aria%les are %its o+ !ata place! there %y bash, a! e#iromet #aria%les are
%asically e#erythig else. 9 a!!itio to #aria%les, the shell also stores some
programmatic !ata, amely aliases a! shell functions. We co#ere! aliases i Chapter K,
a! shell +"ctios Ewhich are relate! to shell scriptigF will %e co#ere! i 6art 2.
6xamining The 6n;ironment
We ca "se either the set %"ilti i bash or the 7rintenv program to see what is
store! i the e#iromet. The set comma! will show %oth the shell a! e#iromet
#aria%les, while 7rintenv will oly !isplay the latter. Sice the list o+ e#iromet
cotets will %e +airly log, it is %est to pipe the o"tp"t o+ either comma! ito less,
[me@linuxbox ~]$ printenv 0 less
.28
What 9s Store! 9 The E#irometC
'oig so, we sho"l! get somethig that looks like this,
2'&O+3=TAS&4'Mfalse
**SO4@&LTO:A'M....
S*TL4+&Mlinuxbox
@:@O4@&LTOAL-M6tm76g7g-:dt)g6*<g7g-agent:../0:#
*S&==M6bin6bash
T&R+Mxterm
V'@O+&L3O:R&-AVMkde-
SA*T*AI&M#(((
V'@O*&**ALOC2A&M.d)b("c."/%.c$eaf$#(#b((%.bd!b((-#!(/"!#00(<00.)("
-##))(".#00
@T2!ORCO-A=&*M6etc6gtk-!<(6gtkrc:6home6me6<gtkrc-!<(:6home6me6<kde6sh
are6config6gtkrc-!<(
@T2ORCO-A=&*M6etc6gtk6gtkrc:6home6me6<gtkrc:6home6me6<kde6share6confi
g6gtkrc
@*O=AHM6home6me6<fonts
,AL',A'M!0$.(#$.
ZT'ARM6usr6lib6Bt-$<$
ZTALCM6usr6lib6Bt-$<$6include
2'&O-3==O*&**ALMtrue
3*&RMme
=*OC=R*MnoM((:fiM((:diM((T$%:lnM((T$.:7iM%(T$$:soM((T$":bdM%(T$$T(#
:cdM%(T$$T(#:orM(#T("T$)T%#:miM(#T("T$)T%#:exM((T$!:E<cmdM((T$!:E<exe
:
What we see is a list o+ e#iromet #aria%les a! their #al"es. 0or e3ample, we see a
#aria%le calle! 3*&R, which cotais the #al"e ?me@. The 7rintenv comma! ca
also list the #al"e o+ a speci+ic #aria%le,
[me@linuxbox ~]$ printenv EFGH
me
The set comma!, whe "se! witho"t optios or arg"mets, will !isplay %oth the shell
a! e#iromet #aria%les, as well as ay !e+ie! shell +"ctios. )like 7rintenv,
its o"tp"t is co"rteo"sly sorte! i alpha%etical or!er,
[me@linuxbox ~]$ set 0 less
9t is also possi%le to #iew the cotets o+ a #aria%le "sig the echo comma!, like this,
.29
.2 8 The E#iromet
[me@linuxbox ~]$ echo 3RSOG
6home6me
7e elemet o+ the e#iromet that either set or 7rintenv !isplays is aliases. To
see them, eter the alias comma! witho"t arg"mets,
[me@linuxbox ~]$ alias
alias l<MKls -d <E --colorMtt1K
alias llMKls -l --colorMtt1K
alias lsMKls --colorMtt1K
alias viMKvimK
alias 9hichMKalias Q 6usr6bin69hich --tt1-onl1 --read-alias --sho9-
dot --sho9-tildeK
Some Interesting ?ariabes
The e#iromet cotais ="ite a +ew #aria%les, a! tho"gh yo"r e#iromet may !i++er
+rom the oe presete! here, yo" will likely see the +ollowig #aria%les i yo"r
e#iromet,
Table 1'-1: 1nvironment Bariables
Varia0le Contents
'A*:=4U
The ame o+ yo"r !isplay i+ yo" are r"ig a graphical
e#iromet. )s"ally this is ?,0@, meaig the +irst !isplay
geerate! %y the G ser#er.
&'ATR
Tha ame o+ the program to %e "se! +or te3t e!itig.
*S&==
The ame o+ yo"r shell program.
S+&
The pathame o+ yo"r home !irectory.
=4L@
'e+ies the character set a! collatio or!er o+ yo"r lag"age.
='O:,'
The pre#io"s workig !irectory.
:4@&R
The ame o+ the program to %e "se! +or pagig o"tp"t. This is
o+te set to 6usr6bin6less.
:4TS
$ colo-separate! list o+ !irectories that are searche! whe yo"
eter the ame o+ a e3ec"ta%le program.
:*#
6rompt Strig .. This !e+ies the cotets o+ yo"r shell prompt. $s
we will later see, this ca %e e3tesi#ely c"stomiDe!.
.(0
What 9s Store! 9 The E#irometC
:,'
The c"rret workig !irectory.
T&R+
The ame o+ yo"r termial type. )i3-like systems s"pport may
termial protocolsH this #aria%le sets the protocol to %e "se! with
yo"r termial em"lator.
TI
Speci+ies yo"r timeDoe. >ost )i3-like systems maitai the
comp"terIs iteral clock i %oordinated 0niversal Time E)TCF
a! the !isplays the local time %y applyig a o++set speci+ie! %y
this #aria%le.
3*&R
5o"r "ser ame.
'o<t worry i+ some o+ these #al"es are missig. They #ary %y !istri%"tio.
3o4 Is The $n%ironment $sta0lished.
Whe we log o to the system, the bash program starts, a! rea!s a series o+
co+ig"ratio scripts calle! startup files, which !e+ie the !e+a"lt e#iromet share! %y
all "sers. This is +ollowe! %y more start"p +iles i o"r home !irectory that !e+ie o"r
persoal e#iromet. The e3act se="ece !epe!s o the type o+ shell sessio %eig
starte!. There are two ki!s, a logi shell sessio a! a o-logi shell sessio.
$ logi shell sessio is oe i which we are prompte! +or o"r "ser ame a! passwor!H
whe we start a #irt"al cosole sessio, +or e3ample. $ o-logi shell sessio typically
occ"rs whe we la"ch a termial sessio i the ;)9.
*ogi shells rea! oe or more start"p +iles as show i Ta%le .2-2,
Table 1'-': Startup Files For (ogin Shell Sessions
;ile Contents
6etc67rofile
$ glo%al co+ig"ratio script that applies to all "sers.
~6<bashO7rofile
$ "ser<s persoal start"p +ile. Ca %e "se! to e3te! or
o#erri!e settigs i the glo%al co+ig"ratio script.
~6<bashOlogin
9+ ~6<bashO7rofile is ot +o"!, bash attempts to
rea! this script.
~6<7rofile
9+ either ~6<bashO7rofile or ~6<bashOlogin
is +o"!, bash attempts to rea! this +ile. This is the
!e+a"lt i 'e%ia-%ase! !istri%"tios, s"ch as )%"t".
&o-logi shell sessios rea! the +ollowig start"p +iles,
.(.
.2 8 The E#iromet
Table 1'-3: Startup Files For >on-(ogin Shell Sessions
;ile Contents
6etc6bash<bashrc
$ glo%al co+ig"ratio script that applies to all "sers.
~6<bashrc
$ "ser<s persoal start"p +ile. Ca %e "se! to e3te! or
o#erri!e settigs i the glo%al co+ig"ratio script.
9 a!!itio to rea!ig the start"p +iles a%o#e, o-logi shells also iherit the
e#iromet +rom their paret process, "s"ally a logi shell.
Take a look at yo"r system a! see which o+ these start"p +iles yo" ha#e. :emem%er B
sice most o+ the +ileames liste! a%o#e start with a perio! Emeaig that they are
hi!!eF, yo" will ee! to "se the ?-a@ optio whe "sig ls.
The ~6<bashrc +ile is pro%a%ly the most importat start"p +ile +rom the or!iary "serIs
poit o+ #iew, sice it is almost always rea!. &o-logi shells rea! it %y !e+a"lt a! most
start"p +iles +or logi shells are writte i s"ch a way as to rea! the ~6<bashrc +ile as
well.
What"s In A Startu5 2ie?
9+ we take a look isi!e a typical <bashO7rofile Etake +rom a Cet7S 1 systemF, it
looks somethig like this,
^ <bashO7rofile
^ @et the aliases and functions
if [ -f ~6<bashrc ]T then
< ~6<bashrc
fi
^ 3ser s7ecific environment and startu7 7rograms
:4TSM$:4TS:$S+&6bin
ex7ort :4TS
*ies that %egi with a ?P@ are comments a! are ot rea! %y the shell. These are there
+or h"ma rea!a%ility. The +irst iterestig thig occ"rs o the +o"rth lie, with the
+ollowig co!e,
if [ -f ~6<bashrc ]T then
.(2
Aow 9s The E#iromet Esta%lishe!C
< ~6<bashrc
fi
This is calle! a if compound command, which we will co#er +"lly whe we get to shell
scriptig i 6art 2, %"t +or ow we will traslate,
Af the file _~6<bashrc_ exists> then
read the _~6<bashrc_ file<
We ca see that this %it o+ co!e is how a logi shell gets the cotets o+ <bashrc. The
e3t thig i o"r start"p +ile has to !o with the :4TS #aria%le.
E#er wo!er how the shell kows where to +i! comma!s whe we eter them o the
comma! lieC 0or e3ample, whe we eter ls, the shell !oes ot search the etire
comp"ter to +i! 6bin6ls Ethe +"ll pathame o+ the ls comma!F, rather, it searches a
list o+ !irectories that are cotaie! i the :4TS #aria%le.
The :4TS #aria%le is o+te E%"t ot always, !epe!ig o the !istri%"tioF set %y the
6etc67rofile start"p +ile a! with this co!e,
:4TSM$:4TS:$S+&6bin
:4TS is mo!i+ie! to a!! the !irectory $S+&6bin to the e! o+ the list. This is a
e3ample o+ parameter e3pasio, which we to"che! o i Chapter 8. To !emostrate
how this works, try the +ollowig,
[me@linuxbox ~]$ foo$J+his is some J
[me@linuxbox ~]$ echo 3foo
This is some
[me@linuxbox ~]$ foo$3fooJtext.J
[me@linuxbox ~]$ echo 3foo
This is some text<
)sig this techi="e, we ca appe! te3t to the e! o+ a #aria%le<s cotets.
Jy a!!ig the strig $S+&6bin to the e! o+ the :4TS #aria%le<s cotets, the
!irectory $S+&6bin is a!!e! to the list o+ !irectories searche! whe a comma! is
etere!. This meas that whe we wat to create a !irectory withi o"r home !irectory
+or storig o"r ow pri#ate programs, the shell is rea!y to accommo!ate "s. $ll we ha#e
.((
.2 8 The E#iromet
to !o is call it bin, a! weIre rea!y to go.
1ote, >ay !istri%"tios pro#i!e this :4TS settig %y !e+a"lt. Some 'e%ia
%ase! !istri%"tios, s"ch as )%"t", test +or the e3istece o+ the ~6bin !irectory at
logi, a! !yamically a!! it to the :4TS #aria%le i+ the !irectory is +o"!.
*astly, we ha#e,
ex7ort :4TS
The ex7ort comma! tells the shell to make the cotets o+ :4TS a#aila%le to chil!
processes o+ this shell.
>odi",ing The $n%ironment
Sice we kow where the start"p +iles are a! what they cotai, we ca mo!i+y them to
c"stomiDe o"r e#iromet.
Whi/h 2ies Shoud We :odi=y?
$s a geeral r"le, to a!! !irectories to yo"r :4TS, or !e+ie a!!itioal e#iromet
#aria%les, place those chages i <bashO7rofile Eor e="i#alet, accor!ig to yo"r
!istri%"tio. 0or e3ample, )%"t" "ses <7rofile.F 0or e#erythig else, place the
chages i <bashrc. )less yo" are the system a!miistrator a! ee! to chage the
!e+a"lts +or all "sers o+ the system, restrict yo"r mo!i+icatios to the +iles i yo"r home
!irectory. 9t is certaily possi%le to chage the +iles i 6etc s"ch as 7rofile, a! i
may cases it wo"l! %e sesi%le to !o so, %"t +or ow, let<s play it sa+e.
Text 6ditors
To e!it Ei.e., mo!i+yF the shell<s start"p +iles, as well as most o+ the other co+ig"ratio
+iles o the system, we "se a program calle! a te+t editor. $ te3t e!itor is a program that
is, i some ways, like a wor! processor i that it allows yo" to e!it the wor!s o the
scree with a mo#ig c"rsor. 9t !i++ers +rom a wor! processor %y oly s"pportig p"re
te3t, a! o+te cotais +eat"res !esige! +or writig programs. Te3t e!itors are the
cetral tool "se! %y so+tware !e#elopers to write co!e, a! %y system a!miistrators to
maage the co+ig"ratio +iles that cotrol the system.
There are a lot o+ !i++eret te3t e!itors a#aila%le +or *i"3H yo"r system pro%a%ly has
se#eral istalle!. Why so may !i++eret oesC 6ro%a%ly %eca"se programmers like
.(1
>o!i+yig The E#iromet
writig them, a! sice programmers "se them e3tesi#ely, they write e!itors to e3press
their ow !esires as to how they sho"l! work.
Te3t e!itors +all ito two %asic categories, graphical a! te3t %ase!. ;&7>E a! M'E
%oth icl"!e some pop"lar graphical e!itors. ;&7>E ships with a e!itor calle!
gedit, which is "s"ally calle! ?Te3t E!itor@ i the ;&7>E me". M'E "s"ally ships
with three which are Ei or!er o+ icreasig comple3ityF kedit, k9rite, a! kate.
There are may te3t-%ase! e!itors. The pop"lar oes yo" will eco"ter are nano, vi,
a! emacs. The nano e!itor is a simple, easy-to-"se e!itor !esige! as a replacemet
+or the 7ico e!itor s"pplie! with the 69&E email s"ite. The vi e!itor Eo most *i"3
systems replace! %y a program ame! vim, which is short +or ?Ri 9>pro#e!@F is the
tra!itioal e!itor +or )i3-like systems. 9t will %e the s"%4ect o+ o"r e3t chapter. The
emacs e!itor was origially writte %y :ichar! Stallma. 9t is a gigatic, all-p"rpose,
!oes-e#erythig programmig e#iromet. While rea!ily a#aila%le, it is sel!om
istalle! o most *i"3 systems %y !e+a"lt.
Using A Text 6ditor
$ll te3t e!itors ca %e i#oke! +rom the comma! lie %y typig the ame o+ the e!itor
+ollowe! %y the ame o+ the +ile yo" wat to e!it. 9+ the +ile !oes ot alrea!y e3ist, the
e!itor will ass"me that yo" wat to create a ew +ile. Aere is a e3ample "sig gedit,
[me@linuxbox ~]$ gedit some.file
This comma! will start the gedit te3t e!itor a! loa! the +ile ame! ?someO+ile@, i+ it
e3ists.
$ll graphical te3t e!itors are pretty sel+-e3plaatory, so we wo<t co#er them here.
9stea!, we will cocetrate o o"r +irst te3t-%ase! te3t e!itor, nano. *et<s +ire "p nano
a! e!it the <bashrc +ile. J"t %e+ore we !o that, let<s practice some ?sa+e comp"tig.@
Whee#er we e!it a importat co+ig"ratio +ile, it is always a goo! i!ea to create a
%ack"p copy o+ the +ile +irst. This protects "s i case we mess the +ile "p while e!itig.
To create a %ack"p o+ the <bashrc +ile, !o this,
[me@linuxbox ~]$ c7 <bashrc <bashrc<bak
9t !oes<t matter what yo" call the %ack"p +ile, 4"st pick a "!ersta!a%le ame. The
e3tesios ?.%ak@, ?.sa#@, ?.ol!@, a! ?.orig@ are all pop"lar ways o+ i!icatig a %ack"p
+ile. 7h, a! remem%er that c7 will overwrite e+isting files siletly.
.(2
.2 8 The E#iromet
&ow that we ha#e a %ack"p +ile, we<ll start the e!itor,
[me@linuxbox ~]$ nano .bashrc
7ce nano starts, weIll get a scree like this,
@L3 nano !<(<$ -ile: <bashrc
^ <bashrc
^ *ource global definitions
if [ -f 6etc6bashrc ]T then
< 6etc6bashrc
fi
^ 3ser s7ecific aliases and functions
[ Read / lines ]
`@ @et Sel7` ,riteut`R Read -il`U :rev :ag`2 Cut Text`C Cur :os
`V &xit `? ?ustif1 `, ,here As`; Lext :ag`3 3nCut Te`T To *7ell
1ote5 9+ yo"r system !oes ot ha#e nano istalle!, yo" may "se a graphical e!itor
istea!.
The scree cosists o+ a hea!er at the top, the te3t o+ the +ile %eig e!ite! i the mi!!le
a! a me" o+ comma!s at the %ottom. Sice nano was !esige! to replace the te3t
e!itor s"pplie! with a email cliet, it is rather short o e!itig +eat"res.
The +irst comma! yo" sho"l! lear i ay te3t e!itor is how to e3it the program. 9 the
case o+ nano, yo" type Ctrl-x to e3it. This is i!icate! i the me" at the %ottom o+
the scree. The otatio ?`G@ meas Ctrl-x. This is a commo otatio +or cotrol
characters "se! %y may programs.
The seco! comma! we ee! to kow is how to sa#e o"r work. With nano it<s Ctrl-
.(K
>o!i+yig The E#iromet
o. With this kowle!ge "!er o"r %elts, we<re rea!y to !o some e!itig. )sig the !ow
arrow key a!-or the :age'o9n key, mo#e the c"rsor to the e! o+ the +ile, the a!! the
+ollowig lies to the <bashrc +ile,
umask ***2
export RPF+=S?+HST$ignoredups
export RPF+FP@G$1***
alias l.$%ls -d .* --color$auto%
alias ll$%ls -l --color$auto%
1ote5 5o"r !istri%"tio may alrea!y icl"!e some o+ these, %"t !"plicates wo<t
h"rt aythig.
Aere is the meaig o+ o"r a!!itios,
Line >eaning
umask (((!
Sets the umask to sol#e the
pro%lem with share! !irectories
we !isc"sse! i Chapter .0.
ex7ort SA*TCLTR=Mignoredu7s
Ca"ses the shell<s history
recor!ig +eat"re to igore a
comma! i+ the same comma!
was 4"st recor!e!.
ex7ort SA*T*AI&M#(((
9creases the siDe o+ the comma!
history +rom the !e+a"lt o+ 200
lies to .000 lies.
alias l<MKls -d <E --colorMautoK
Creates a ew comma! calle!
?l<@ which !isplays all !irectory
etries that %egi with a !ot.
alias llMKls -l --colorMautoK
Creates a ew comma! calle!
?ll@ which !isplays a log
+ormat !irectory listig.
$s we ca see, may o+ o"r a!!itios are ot it"iti#ely o%#io"s, so it wo"l! %e a goo!
i!ea to a!! some commets to o"r <bashrc +ile to help e3plai thigs to the h"mas.
)sig the e!itor, chage o"r a!!itios to look like this,
.(/
.2 8 The E#iromet
U =hange umask to make director! sharing easier
umask (((!
U Pgnore duplicates in command histor! and increase
U histor! si-e to 1*** lines
ex7ort SA*TCLTR=Mignoredu7s
ex7ort SA*T*AI&M#(((
U :dd some helpful aliases
alias l<MKls -d <E --colorMautoK
alias llMKls -l --colorMautoK
$h, m"ch %etterL With o"r chages complete, type Ctrl-o to sa#e o"r mo!i+ie!
<bashrc +ile, a! Ctrl-x to e3it nano.
+h, Comments #re Important
Whee#er yo" mo!i+y co+ig"ratio +iles it<s a goo! i!ea to a!! some commets
to !oc"met yo"r chages. S"re, yo" will remem%er what yo" chage!
tomorrow, %"t what a%o"t si3 moths +rom owC 'o yo"rsel+ a +a#or a! a!!
some commets. While yo"<re at it, itIs ot a %a! i!ea to keep a log o+ what
chages yo" make.
Shell scripts a! bash start"p +iles "se a ?P@ sym%ol to %egi a commet. 7ther
co+ig"ratio +iles may "se other sym%ols. >ost co+ig"ratio +iles will ha#e
commets. )se them as a g"i!e.
5o" will o+te see lies i co+ig"ratio +iles that are commented out to pre#et
them +rom %eig "se! %y the a++ecte! program. This is !oe to gi#e the rea!er
s"ggestios +or possi%le co+ig"ratio choices or e3amples o+ correct
co+ig"ratio syta3. 0or e3ample, the <bashrc +ile o+ )%"t" 8.01 cotais
these lies,
^ some more ls aliases
^alias llMKls -lK
^alias laMKls -4K
^alias lMKls -C-K
The last three lies are #ali! alias !e+iitios that ha#e %ee commete! o"t. 9+
yo" remo#e the lea!ig ?P@ sym%ols +rom these three lies, a techi="e calle!
uncommenting, yo" will acti#ate the aliases. Co#ersely, i+ yo" a!! a ?P@
sym%ol to the %egiig o+ a lie, yo" ca !eacti#ate a co+ig"ratio lie while
preser#ig the i+ormatio it cotais.
.(8
>o!i+yig The E#iromet
A/ti;ating <ur Changes
The chages we ha#e ma!e to o"r <bashrc will ot take a++ect "til we close o"r
termial sessio a! start a ew oe, sice the <bashrc +ile is oly rea! at the
%egiig o+ a sessio. Aowe#er, we ca +orce bash to re-rea! the mo!i+ie! <bashrc
+ile with the +ollowig comma!,
[me@linuxbox ~]$ source .bashrc
$+ter !oig this, we sho"l! %e a%le to see the e++ect o+ o"r chages. Try o"t oe o+ the
ew aliases,
[me@linuxbox ~]$ ll
Summing -p
9 this chapter we leare! a essetial skill Be!itig co+ig"ratio +iles with a te3t
e!itor. >o#ig +orwar!, as we rea! ma pages +or comma!s, take ote o+ the
e#iromet #aria%les that comma!s s"pport. There may %e a gem or two. 9 later
chapters, we will lear a%o"t shell +"ctios, a power+"l +eat"re that yo" ca also icl"!e
i the bash start"p +iles to a!! to yo"r arseal o+ c"stom comma!s.
;urther 1eading
The 9&R7C$T97& sectio o+ the bash ma page co#ers the bash start"p +iles
i gory !etail.
.(9
.( 8 $ ;etle 9tro!"ctio To #i
13 ) 3entle Introduction To vi
There is a ol! 4oke a%o"t a #isitor to &ew 5ork City askig a passer%y +or !irectios to
the city<s +amo"s classical m"sic #e"e,
Risitor, E3c"se me, how !o 9 get to Caregie AallC
6asser%y, 6ractice, practice, practiceL
*earig the *i"3 comma! lie, like %ecomig a accomplishe! piaist, is ot
somethig that we pick "p i a a+teroo. 9t takes years o+ practice. 9 this chapter, we
will itro!"ce the vi Eproo"ce! ?#ee eye@F te3t e!itor, oe o+ the core programs i the
)i3 tra!itio. vi is somewhat otorio"s +or its !i++ic"lt "ser iter+ace, %"t whe we see
a master sit !ow at the key%oar! a! %egi to ?play,@ we will i!ee! %e witess to some
great art. We wo<t %ecome masters i this chapter, %"t whe we are !oe, we will kow
how to play ?chopsticks@ i vi.
+h, +e Should Learn %i
9 this mo!er age o+ graphical e!itors a! easy-to-"se te3t-%ase! e!itors s"ch as nano,
why sho"l! we lear viC There are three goo! reasos,
vi is always a#aila%le. This ca %e a li+esa#er i+ we ha#e a system with o
graphical iter+ace, s"ch as a remote ser#er or a local system with a %roke G
co+ig"ratio. nano, while icreasigly pop"lar is still ot "i#ersal. 67S9G, a
sta!ar! +or program compati%ility o )i3 systems, re="ires that vi %e preset.
vi is lightweight a! +ast. 0or may tasks, it<s easier to %rig "p vi tha it is to
+i! the graphical te3t e!itor i the me"s a! wait +or its m"ltiple mega%ytes to
loa!. 9 a!!itio, vi is !esige! +or typig spee!. $s we shall see, a skille! vi
"ser e#er has to li+t his or her +igers +rom the key%oar! while e!itig.
We !o<t wat other *i"3 a! )i3 "sers to thik we are sissies.
7kay, may%e two goo! reasos.
.10
$ *ittle Jackgro"!
# Little /ac(ground
The +irst #ersio o+ vi was writte i .9/K %y Jill Joy, a )i#ersity o+ Cali+oria at
Jerkley st"!et who later wet o to co-+o"! S" >icrosystems. vi !eri#es its ame
+rom the wor! ?#is"al,@ %eca"se it was ite!e! to allow e!itig o a #i!eo termial with
a mo#ig c"rsor. 6re#io"s to visual editors, there were line editors which operate! o a
sigle lie o+ te3t at a time. To speci+y a chage, we tell a lie e!itor to go to a partic"lar
lie a! !escri%e what chage to make, s"ch as a!!ig or !eletig te3t. With the a!#et
o+ #i!eo termials Erather tha priter-%ase! termials like teletypesF #is"al e!itig
%ecame possi%le. vi act"ally icorporates a power+"l lie e!itor calle! ex, a! we ca
"se lie e!itig comma!s while "sig vi.
>ost *i"3 !istri%"tios !o<t icl"!e real viH rather, they ship with a ehace!
replacemet calle! vim Ewhich is short +or ?#i impro#e!@F writte %y Jram >ooleaar.
vim is a s"%statial impro#emet o#er tra!itioal )i3 vi a! is "s"ally sym%olically
like! Eor aliase!F to the ame ?#i@ o *i"3 systems. 9 the !isc"ssios that +ollow, we
will ass"me that we ha#e a program calle! ?#i@ that is really vim.
Starting #nd Stopping %i
To start vi, we simply type the +ollowig,
[me@linuxbox ~]$ vi
$! a scree like this sho"l! appear,
~
~
~ ;A+ - ;i Am7roved
~
~ version )<#<#$/
~ b1 Hram +oolenaar et al<
~ ;im is o7en source and freel1 distributable
~
~ *7onsor ;im develo7mentG
~ t17e :hel7 s7onsorX&nterD for information
~
~ t17e :BX&nterD to exit
~ t17e :hel7X&nterD or X-#D for on-line hel7
~ t17e :hel7 version)X&nterD for version info
~
~ Running in ;i com7atible mode
~ t17e :set noc7X&nterD for ;im defaults
.1.
.( 8 $ ;etle 9tro!"ctio To #i
~ t17e :hel7 c7-defaultX&nterD for info on this
~
~
~
J"st as we !i! with nano earlier, the +irst thig to lear is how to e3it. To e3it, we eter
the +ollowig comma! Eote that the colo character is part o+ the comma!F,
2,
The shell prompt sho"l! ret"r. 9+, +or some reaso, vi will ot ="it E"s"ally %eca"se we
ma!e a chage to a +ile that has ot yet %ee sa#e!F, we ca tell vi that we really mea it
%y a!!ig a e3clamatio poit to the comma!,
2,L
Ti(5 9+ yo" get ?lost@ i vi, try pressig the &sc key twice to +i! yo"r way agai.
Compati0ilit, >ode
9 the e3ample start"p scree a%o#e Etake +rom )%"t" 8.01F, we see the te3t
?:"ig i Ri compati%le mo!e.@ This meas that vim will r" i a mo!e that
is closer to the ormal %eha#ior o+ vi rather tha the ehace! %eha#ior o+ vim.
0or p"rposes o+ this chapter, we will wat to r" vim with its ehace! %eha#ior.
To !o this, yo" ha#e a +ew optios,
Try r"ig vim istea! o+ vi.
9+ that works, cosi!er a!!ig alias viMKvimK to yo"r <bashrc +ile.
$lterately, "se this comma! to a!! a lie to yo"r vim co+ig"ratio +ile,
echo _set noc7_ DD ~6<vimrc
'i++eret *i"3 !istri%"tios package vim i !i++eret ways. Some !istri%"tios
istall a miimal #ersio o+ vim %y !e+a"lt that oly s"pports a limitig set o+
.12
Startig $! Stoppig #i
vim +eat"res. While pre+ormig the lessos that +ollow, yo" may eco"ter
missig +eat"res. 9+ this is the case, istall the +"ll #ersio o+ vim.
$diting >odes
*et<s start "p vi agai, this time passig to it the ame o+ a oe3istet +ile. This is how
we ca create a ew +ile with vi,
[me@linuxbox ~]$ rm -f foo.txt
[me@linuxbox ~]$ vi foo.txt
9+ all goes well, we sho"l! get a scree like this,
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
~
_foo<txt_ [Le9 -ile]
The lea!ig til!e characters E@S@F i!icate that o te3t e3ists o that lie. This shows that
we ha#e a empty +ile. /o not ty(e anything yet2
The seco! most importat thig to lear a%o"t vi Ea+ter learig how to e3itF is that vi
.1(
.( 8 $ ;etle 9tro!"ctio To #i
is a modal editor. Whe vi starts "p, it %egis i command mode. 9 this mo!e, almost
e#ery key is a comma!, so i+ we were to start typig, vi wo"l! %asically go craDy a!
make a %ig mess.
6ntering Insert :ode
9 or!er to a!! some te3t to o"r +ile, we m"st +irst eter insert mode. To !o this, we press
the ?i@ key. $+terwar!s, we sho"l! see the +ollowig at the %ottom o+ the scree i+ vim is
r"ig i its "s"al ehace! mo!e Ethis will ot appear i vi compati%le mo!eF,
-- P?FGH+ --
&ow we ca eter some te3t. Try this,
+he ,uick brown fox jumped over the la-! dog.
To e3it isert mo!e a! ret"r to comma! mo!e, press the &sc key.
Sa;ing <ur Work
To sa#e the chage we 4"st ma!e to o"r +ile, we m"st eter a e+ command while i
comma! mo!e. This is easily !oe %y pressig the ?,@ key. $+ter !oig this, a colo
character sho"l! appear at the %ottom o+ the scree,
:
To write o"r mo!i+ie! +ile, we +ollow the colo with a ?w@ the Eter,
:w
The +ile will %e writte to the har! !ri#e a! we sho"l! get a co+irmatio message at the
%ottom o+ the scree, like this,
_foo<txt_ [Le9] #=> %.C 9ritten
.11
E!itig >o!es
Ti(5 9+ yo" rea! the vim !oc"metatio, yo" will otice that Eco+"siglyF
comma! mo!e is calle! normal mode a! ex comma!s are calle! command
mode. Jeware.
>o%ing The Cursor #round
While i comma! mo!e, vi o++ers a large "m%er o+ mo#emet comma!s, some o+
which it shares with less. Aere is a s"%set,
Table 13-1: %ursor !ovement ?eys
=e, >o%es The Cursor
l or :ight $rrow :ight oe character.
h or *e+t $rrow *e+t oe character.
j or 'ow $rrow 'ow oe lie.
k or )p $rrow )p oe lie.
( EDeroF To the %egiig o+ the c"rret lie.
`
To the +irst o-whitespace character o the c"rret
lie.
$
To the e! o+ the c"rret lie.
9
To the %egiig o+ the e3t wor! or p"ct"atio
character.
,
To the %egiig o+ the e3t wor!, igorig
p"ct"atio characters.
b
To the %egiig o+ the pre#io"s wor! or p"ct"atio
character.
H
To the %egiig o+ the pre#io"s wor!, igorig
p"ct"atio characters.
Ctrl-f or :age 'o9n 'ow oe page.
Ctrl-b or :age 37 )p oe page.
number@
To lie number. 0or e3ample, #@ mo#es to the +irst
lie o+ the +ile.
@
To the last lie o+ the +ile.
.12
.( 8 $ ;etle 9tro!"ctio To #i
Why are the h, j, k, a! l keys "se! +or c"rsor mo#emetC Jeca"se whe vi was
origially writte, ot all #i!eo termials ha! arrow keys, a! skille! typists co"l! "se
reg"lar key%oar! keys to mo#e the c"rsor witho"t e#er ha#ig to li+t their +igers +rom
the key%oar!.
>ay comma!s i vi ca %e pre+i3e! with a "m%er, as with the ?;@ comma! liste!
a%o#e. Jy pre+i3ig a comma! with a "m%er, we may speci+y the "m%er o+ times a
comma! is to %e carrie! o"t. 0or e3ample, the comma! ?24@ ca"ses vi to mo#e the
c"rsor !ow +i#e lies.
/asic $diting
>ost e!itig cosists o+ a +ew %asic operatios s"ch as isertig te3t, !eletig te3t a!
mo#ig te3t aro"! %y c"ttig a! pastig. vi, o+ co"rse, s"pports all o+ these
operatios i its ow "i="e way. vi also pro#i!es a limite! +orm o+ "!o. 9+ we press
the ?"@ key while i comma! mo!e, vi will "!o the last chage that yo" ma!e. This
will come i ha!y as we try o"t some o+ the %asic e!itig comma!s.
A55ending Text
vi has se#eral !i++eret ways o+ eterig isert mo!e. We ha#e alrea!y "se! the i
comma! to isert te3t.
*et<s go %ack to o"r foo<txt +ile +or a momet,
The Buick bro9n fox jum7ed over the laC1 dog<
9+ we wate! to a!! some te3t to the e! o+ this setece, we wo"l! !isco#er that the i
comma! will ot !o it, sice we ca<t mo#e the c"rsor %eyo! the e! o+ the lie. vi
pro#i!es a comma! to appe! te3t, the sesi%ly ame! ?a@ comma!. 9+ we mo#e the
c"rsor to the e! o+ the lie a! type ?a@, the c"rsor will mo#e past the e! o+ the lie
a! vi will eter isert mo!e. This will allow "s to a!! some more te3t,
The Buick bro9n fox jum7ed over the laC1 dog< Pt was cool.
:emem%er to press the &sc key to e3it isert mo!e.
Sice we will almost always wat to appe! te3t to the e! o+ a lie, vi o++ers a shortc"t
to mo#e to e! o+ the c"rret lie a! start appe!ig. 9t<s the ?$@ comma!. *et<s try it
.1K
Jasic E!itig
a! a!! some more lies to o"r +ile.
0irst, we<ll mo#e the c"rsor to the %egiig o+ the lie "sig the ?0@ EDeroF comma!.
&ow we type ?$@ a! a!! the +ollowig lies o+ te3t,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
Tine 2
Tine
Tine A
Tine "
$gai, press the &sc key to e3it isert mo!e.
$s we ca see, the ?$@ comma! is more "se+"l as it mo#es the c"rsor to the e! o+ the
lie %e+ore startig isert mo!e.
<5ening A Line
$other way we ca isert te3t is %y ?opeig@ a lie. This iserts a %lak lie %etwee
two e3istig lies a! eters isert mo!e. This has two #ariats,
Table 13-': (ine &pening ?eys
Command ?pens
o
The lie %elow the c"rret lie.

The lie a%o#e the c"rret lie.

We ca !emostrate this as +ollows, place the c"rsor o ?*ie (@ the press the o key.
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
$ ew lie was opee! %elow the thir! lie a! we etere! isert mo!e. E3it isert mo!e
%y pressig the &sc key. 6ress the u key to "!o o"r chage.
6ress the key to ope the lie a%o#e the c"rsor,
.1/
.( 8 $ ;etle 9tro!"ctio To #i
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
E3it isert mo!e %y pressig the &sc key a! "!o o"r chage %y pressig u.
)eeting Text
$s we might e3pect, vi o++ers a #ariety o+ ways to !elete te3t, all o+ which cotai oe
o+ two keystrokes. 0irst, the x key will !elete a character at the c"rsor locatio. x may
%e prece!e! %y a "m%er speci+yig how may characters are to %e !elete!. The d key is
more geeral p"rpose. *ike x, it may %e prece!e! %y a "m%er speci+yig the "m%er o+
times the !eletio is to %e per+orme!. 9 a!!itio, d is always +ollowe! %y a mo#emet
comma! that cotrols the siDe o+ the !eletio. Aere are some e3amples,
Table 13-3: Te+t -eletion %ommands
Command Deletes
x
The c"rret character.
$x
The c"rret character a! the e3t two characters.
dd
The c"rret lie.
"dd
The c"rret lie a! the e3t +o"r lies.
d,
0rom the c"rret c"rsor positio to the %egiig o+
the e3t wor!.
d$
0rom the c"rret c"rsor locatio to the e! o+ the
c"rret lie.
d(
0rom the c"rret c"rsor locatio to the %egiig o+
the lie.
d`
0rom the c"rret c"rsor locatio to the +irst o-
whitespace character i the lie.
d@
0rom the c"rret lie to the e! o+ the +ile.
d!(@
0rom the c"rret lie to the twetieth lie o+ the +ile.
6lace the c"rsor o the wor! ?9t@ o the +irst lie o+ o"r te3t. 6ress the x key repeate!ly
.18
Jasic E!itig
"til the rest o+ the setece is !elete!. &e3t, press the u key repeate!ly "til the
!eletio is "!oe.
1ote5 :eal vi oly s"pports a sigle le#el o+ "!o. vim s"pports m"ltiple le#els.
*et<s try the !eletio agai, this time "sig the d comma!. $gai, mo#e the c"rsor to
the wor! ?9t@ a! press d, to !elete the wor!,
The Buick bro9n fox jum7ed over the laC1 dog< 9as cool<
=ine !
=ine $
=ine %
=ine "
6ress d$ to !elete +rom the c"rsor positio to the e! o+ the lie,
The Buick bro9n fox jum7ed over the laC1 dog<
=ine !
=ine $
=ine %
=ine "
6ress d@ to !elete +rom the c"rret lie to the e! o+ the +ile,
~
~
~
~
~
6ress u three times to "!o the !eletio.
CuttingC Co5ying And 'asting Text
The d comma! ot oly !eletes te3t, it also ?c"ts@ te3t. Each time we "se the d
comma! the !eletio is copie! ito a paste %"++er Ethik clip%oar!F that we ca later
recall with the 7 comma! to paste the cotets o+ the %"++er a+ter the c"rsor or the :
comma! to paste the cotets %e+ore the c"rsor.
.19
.( 8 $ ;etle 9tro!"ctio To #i
The 1 comma! is "se! to ?yak@ EcopyF te3t i m"ch the same way the d comma! is
"se! to c"t te3t. Aere are some e3amples com%iig the 1 comma! with #ario"s
mo#emet comma!s,
Table13- $: Canking %ommands
Command Copies
11
The c"rret lie.
"11
The c"rret lie a! the e3t +o"r lies.
1,
0rom the c"rret c"rsor positio to the %egiig o+
the e3t wor!.
1$
0rom the c"rret c"rsor locatio to the e! o+ the
c"rret lie.
1(
0rom the c"rret c"rsor locatio to the %egiig o+
the lie.
1`
0rom the c"rret c"rsor locatio to the +irst o-
whitespace character i the lie.
1@
0rom the c"rret lie to the e! o+ the +ile.
1!(@
0rom the c"rret lie to the twetieth lie o+ the +ile.
*et<s try some copy a! paste. 6lace the c"rsor o the +irst lie o+ the te3t a! type 11 to
copy the c"rret lie. &e3t, mo#e the c"rsor to the last lie E@F a! type 7 to paste the
lie %elow the c"rret lie,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
+he ,uick brown fox jumped over the la-! dog. Pt was cool.
J"st as %e+ore, the u comma! will "!o o"r chage. With the c"rsor still positioe! o
the last lie o+ the +ile, type : to paste the te3t a%o#e the c"rret lie,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
.20
Jasic E!itig
=ine $
=ine %
+he ,uick brown fox jumped over the la-! dog. Pt was cool.
=ine "
Try o"t some o+ the other 1 comma!s i the ta%le a%o#e a! get to kow the %eha#ior o+
%oth the 7 a! : comma!s. Whe yo" are !oe, ret"r the +ile to its origial state.
Goining Lines
vi is rather strict a%o"t its i!ea o+ a lie. &ormally, it is ot possi%le to mo#e the c"rsor
to the e! o+ a lie a! !elete the e!-o+-lie character to 4oi oe lie with the oe
%elow it. Jeca"se o+ this, vi pro#i!es a speci+ic comma!, ? Eot to %e co+"se! with j,
which is +or c"rsor mo#emetF to 4oi lies together.
9+ we place the c"rsor o lie ( a! type the ? comma!, here<s what happes,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
Tine Tine A
=ine "
Search #nd 1eplace
vi has the a%ility to mo#e the c"rsor to locatios %ase! o searches. 9t ca !o this o
%oth a sigle lie or o#er a etire +ile. 9t ca also per+orm te3t replacemets with or
witho"t co+irmatio +rom the "ser.
Sear/hing Within A Line
The f comma! searches a lie a! mo#es the c"rsor to the e3t istace o+ a speci+ie!
character. 0or e3ample, the comma! fa wo"l! mo#e the c"rsor to the e3t occ"rrece
o+ the character ?a@ withi the c"rret lie. $+ter per+ormig a character search withi a
lie, the search may %e repeate! %y typig a semicolo.
Sear/hing The 6ntire 2ie
To mo#e the c"rsor to the e3t occ"rrece o+ a wor! or phrase, the 6 comma! is "se!.
This works the same way as we leare! earlier i the less program. Whe yo" type the
6 comma! a ?-@ will appear at the %ottom o+ the scree. &e3t, type the wor! or phrase
.2.
.( 8 $ ;etle 9tro!"ctio To #i
to %e searche! +or, +ollowe! %y the &nter key. The c"rsor will mo#e to the e3t
locatio cotaiig the search strig. $ search may %e repeate! "sig the pre#io"s search
strig with the n comma!. Aere<s a e3ample,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
6lace the c"rsor o the +irst lie o+ the +ile. Type,
/Tine
+ollowe! %y the &nter key. The c"rsor will mo#e to lie 2. &e3t, type n a! the c"rsor
will mo#e to lie (. :epeatig the n comma! will mo#e the c"rsor !ow the +ile "til it
r"s o"t o+ matches. While we ha#e so +ar oly "se! wor!s a! phrases +or o"r search
patters, vi allows the "se o+ regular e+pressions, a power+"l metho! o+ e3pressig
comple3 te3t patters. We will co#er reg"lar e3pressios i some !etail i a later chapter.
+oba Sear/h And !e5a/e
vi "ses a e3 comma! to per+orm search a! replace operatios Ecalle! ?s"%stit"tio@
i viF o#er a rage o+ lies or the etire +ile. To chage the wor! ?*ie@ to ?lie@ +or the
etire +ile, we wo"l! eter the +ollowig comma!,
28s/Tine/line/g
*et<s %reak this comma! !ow ito separate items a! see what each oe !oes,
Item >eaning
:
The colo character starts a e3 comma!.
5
Speci+ies the rage o+ lies +or the operatio. 5 is a shortc"t
meaig +rom the +irst lie to the last lie. $lterately, the
rage co"l! ha#e %ee speci+ie! #>" Esice o"r +ile is +i#e
lies logF, or #>$ which meas ?+rom lie . to the last lie i
the +ile.@ 9+ the rage o+ lies is omitte!, the operatio is oly
.22
Search $! :eplace
per+orme! o the c"rret lie.
s
Speci+ies the operatio. 9 this case, s"%stit"tio Esearch a!
replaceF.
6=ine6line6
The search patter a! the replacemet te3t.
g
This meas ?glo%al@ i the sese that the search a! replace is
per+orme! o e#ery istace o+ the search strig i the lie. 9+
omitte!, oly the +irst istace o+ the search strig o each lie
is replace!.
$+ter e3ec"tig o"r search a! replace comma! o"r +ile looks like this,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
line !
line $
line %
line "
We ca also speci+y a s"%stit"tio comma! with "ser co+irmatio. This is !oe %y
a!!ig a ?c@ to the e! o+ the comma!. 0or e3ample,
28s/line/Tine/gc
This comma! will chage o"r +ile %ack to its pre#io"s +ormH howe#er, %e+ore each
s"%stit"tio, vi stops a! asks "s to co+irm the s"%stit"tio with this message,
re7lace 9ith =ine N16n6a6B6l6`&6`UPF
Each o+ the characters withi the paretheses is a possi%le choice as +ollows,
Table 13-.: 9eplace %onfirmation ?eys
=e, #ction
1
6er+orm the s"%stit"tio.
n
Skip this istace o+ the patter.
a
6er+orm the s"%stit"tio o this a! all s"%se="et istaces
.2(
.( 8 $ ;etle 9tro!"ctio To #i
o+ the patter.
B or &sc T"it s"%stit"tig.
l
6er+orm this s"%stit"tio a! the ="it. Short +or ?last.@
Ctrl-e> Ctrl-1
Scroll !ow a! scroll "p, respecti#ely. )se+"l +or #iewig
the cote3t o+ the propose! s"%stit"tio.

9+ yo" type 1, the s"%stit"tio will %e per+orme!, n will ca"se vi to skip this istace a!
mo#e o to the e3t oe.
$diting >ultiple ;iles
9t<s o+te "se+"l to e!it more tha oe +ile at a time. 5o" might ee! to make chages to
m"ltiple +iles or yo" may ee! to copy cotet +rom oe +ile ito aother. With vi we
ca ope m"ltiple +iles +or e!itig %y speci+yig them o the comma! lie,
vi file1 file2 file3...
*et<s e3it o"r e3istig #i sessio a! create a ew +ile +or e!itig. Type :9B to e3it vi
sa#ig o"r mo!i+ie! te3t. &e3t, we<ll create a a!!itioal +ile i o"r home !irectory that
we ca play with. We<ll create the +ile %y capt"rig some o"tp"t +rom the ls comma!,
[me@linuxbox ~]$ ls -l /usr/bin & ls-output.txt
*et<s e!it o"r ol! +ile a! o"r ew oe with vi,
[me@linuxbox ~]$ vi foo.txt ls-output.txt
vi will start "p a! we will see the +irst +ile o the scree,
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
.21
E!itig >"ltiple 0iles
S%it/hing Bet%een 2ies
To switch +rom oe +ile to the e3t, "se this e3 comma!,
2n
To mo#e %ack to the pre#io"s +ile "se,
2?
While we ca mo#e +rom oe +ile to aother, vi e+orces a policy that pre#ets "s +rom
switchig +iles i+ the c"rret +ile has "sa#e! chages. To +orce vi to switch +iles a!
a%a!o yo"r chages, a!! a e3clamatio poit ELF to the comma!.
9 a!!itio to the switchig metho! !escri%e! a%o#e, vim Ea! some #ersios o+ viF also
pro#i!e some e3 comma!s that make m"ltiple +iles easier to maage. We ca #iew a list
o+ +iles %eig e!ite! with the :buffers comma!. 'oig so will !isplay a list o+ the
+iles at the %ottom o+ the !isplay,
2buffers
# 5a _foo<txt_ line #
! _ls-out7ut<txt_ line (
:ress &LT&R or t17e command to continue
To switch to aother %"++er E+ileF, type :buffer +ollowe! %y the "m%er o+ the %"++er
yo" wish to e!it. 0or e3ample, to switch +rom %"++er . which cotais the +ile foo<txt
to %"++er two cotaiig the +ile ls-out7ut<txt we wo"l! type this,
2buffer 2
a! o"r scree ow !isplays the seco! +ile.
<5ening Additiona 2ies 2or 6diting
9t<s also possi%le to a!! +iles to o"r c"rret e!itig sessio. The e3 comma! :e Eshort
+or ?e!it@F +ollowe! %y a +ileame will ope a a!!itioal +ile. *et<s e! o"r c"rret
e!itig sessio a! ret"r to the comma! lie.
.22
.( 8 $ ;etle 9tro!"ctio To #i
Start vi agai with 4"st oe +ile,
[me@linuxbox ~]$ vi foo.txt
To a!! o"r seco! +ile, eter,
2e ls-output.txt
$! it sho"l! appear o the scree. The +irst +ile is still preset as we ca #eri+y,
2buffers
# ^ _foo<txt_ line #
! 5a _ls-out7ut<txt_ line (
:ress &LT&R or t17e command to continue
1ote5 5o" caot switch to +iles loa!e! with the :e comma! "sig either the :n
or :L comma!. To switch +iles, "se the :buffer comma! +ollowe! %y the
%"++er "m%er.
Co5ying Content 2rom <ne 2ie Into Another
7+te while e!itig m"ltiple +iles, we will wat to copy a portio o+ oe +ile ito aother
+ile that we are e!itig. This is easily !oe "sig the "s"al yak a! paste comma!s we
"se! earlier. We ca !emostrate as +ollows. 0irst, "sig o"r two +iles, switch to %"++er .
Efoo<txtP %y eterig,
2buffer 1
which sho"l! gi#e "s this,
.2K
E!itig >"ltiple 0iles
The Buick bro9n fox jum7ed over the laC1 dog< At 9as cool<
=ine !
=ine $
=ine %
=ine "
&e3t, mo#e the c"rsor to the +irst lie, a! type 11 to yak EcopyF the lie.
Switch to the seco! %"++er %y eterig,
2buffer 2
The scree will ow cotai some +ile listigs like this Eoly a portio is show hereF,
total $%$)((
-r9xr-xr-x # root root $#$#. !(()-#!-(" (/:"/ [
-r9xr-xr-x # root root /!%( !(()-#!-(0 #$:$0 %##to77m
-r9xr-xr-x # root root ###!). !((/-(#-$# #$:$. a!7
-r9xr-xr-x # root root !"$./ !((.-#(-(. !(:#. a"!dec
-r9xr-xr-x # root root ##"$! !(()-("-(% #):%$ aafire
-r9xr-xr-x # root root )!0! !(()-("-(% #):%$ aainfo
>o#e the c"rsor to the +irst lie a! paste the lie we copie! +rom the prece!ig +ile %y
typig the 7 comma!,
total $%$)((
+he ,uick brown fox jumped over the la-! dog. Pt was cool.
-r9xr-xr-x # root root $#$#. !(()-#!-(" (/:"/ [
-r9xr-xr-x # root root /!%( !(()-#!-(0 #$:$0 %##to77m
-r9xr-xr-x # root root ###!). !((/-(#-$# #$:$. a!7
-r9xr-xr-x # root root !"$./ !((.-#(-(. !(:#. a"!dec
-r9xr-xr-x # root root ##"$! !(()-("-(% #):%$ aafire
-r9xr-xr-x # root root )!0! !(()-("-(% #):%$ aainfo
Inserting An 6ntire 2ie Into Another
9t<s also possi%le to isert a etire +ile ito oe that we are e!itig. To see this i actio,
let<s e! o"r vi sessio a! start a ew oe with 4"st a sigle +ile,
.2/
.( 8 $ ;etle 9tro!"ctio To #i
[me@linuxbox ~]$ vi ls-output.txt
We will see o"r +ile listig agai,
total $%$)((
-r9xr-xr-x # root root $#$#. !(()-#!-(" (/:"/ [
-r9xr-xr-x # root root /!%( !(()-#!-(0 #$:$0 %##to77m
-r9xr-xr-x # root root ###!). !((/-(#-$# #$:$. a!7
-r9xr-xr-x # root root !"$./ !((.-#(-(. !(:#. a"!dec
-r9xr-xr-x # root root ##"$! !(()-("-(% #):%$ aafire
-r9xr-xr-x # root root )!0! !(()-("-(% #):%$ aainfo
>o#e the c"rsor to the thir! lie, the eter the +ollowig e3 comma!,
2r foo.txt
The :r comma! Eshort +or ?rea!@F iserts the speci+ie! +ile %e+ore the c"rsor positio.
7"r scree sho"l! ow look like this,
total $%$)((
-r9xr-xr-x # root root $#$#. !(()-#!-(" (/:"/ [
-r9xr-xr-x # root root /!%( !(()-#!-(0 #$:$0 %##to77m
+he ,uick brown fox jumped over the la-! dog. Pt was cool.
Tine 2
Tine
Tine A
Tine "
-r9xr-xr-x # root root ###!). !((/-(#-$# #$:$. a!7
-r9xr-xr-x # root root !"$./ !((.-#(-(. !(:#. a"!dec
-r9xr-xr-x # root root ##"$! !(()-("-(% #):%$ aafire
-r9xr-xr-x # root root )!0! !(()-("-(% #):%$ aainfo
Sa%ing ?ur +or(
*ike e#erythig else i vi, there are se#eral !i++eret ways to sa#e o"r e!ite! +iles. We
ha#e alrea!y co#ere! the e3 comma! :9, %"t there are some others we may also +i!
help+"l.
9 comma! mo!e, typig II will sa#e the c"rret +ile a! e3it vi. *ikewise, the e3
comma! :9B will com%ie the :9 a! :B comma!s ito oe that will %oth sa#e the
.28
Sa#ig 7"r Work
+ile a! e3it.
The :9 comma! may also speci+y a optioal +ileame. This acts like ?Sa#e $s...@ 0or
e3ample, i+ we were e!itig foo<txt a! wate! to sa#e a alterate #ersio calle!
foo#<txt, we wo"l! eter the +ollowig,
2w foo1.txt
1ote5 While the comma! a%o#e sa#es the +ile "!er a ew ame, it !oes ot
chage the ame o+ the +ile yo" are e!itig. $s yo" coti"e to e!it, yo" will still
%e e!itig foo<txt, ot foo#<txt.
;urther 1eading
E#e with all that we ha#e co#ere! i this chapter, we ha#e %arely scratche! the s"r+ace
o+ what vi a! vim ca !o. Aere are a co"ple o+ o-lie reso"rces yo" ca "se to
coti"e yo"r 4o"rey towar!s vi mastery,
(earning The vi 1ditor 8 $ Wiki%ook +rom Wikipe!ia that o++ers a cocise g"i!e
to vi a! se#eral o+ its work-a-likes icl"!ig vim. 9t<s a#aila%le at,
http,--e.wiki%ooks.org-wiki-Ri
The Bim 8ook - The vim pro4ect has a 2/0-page %ook that co#ers EalmostF all o+
the +eat"res i vim. 5o" ca +i! it at,
+tp,--+tp.#im.org-p"%-#im-!oc-%ook-#im%ook-76*.p!+ .
$ Wikipe!ia article o Jill Joy, the creator o+ vi.,
http,--e.wikipe!ia.org-wiki-JillOJoy
$ Wikipe!ia article o Jram >ooleaar, the a"thor o+ vim,
http,--e.wikipe!ia.org-wiki-JramO>ooleaar
.29
.1 8 C"stomiDig The 6rompt
1 Custo%i4ing The 2ro%#t
9 this chapter we will look at a seemigly tri#ial !etail B o"r shell prompt. This
e3amiatio will re#eal some o+ the ier workigs o+ the shell a! the termial em"lator
program itsel+.
*ike so may thigs i *i"3, the shell prompt is highly co+ig"ra%le, a! while we ha#e
pretty m"ch take it +or grate!, the prompt is a really "se+"l !e#ice oce we lear how
to cotrol it.
#natom, ?" # Prompt
7"r !e+a"lt prompt looks somethig like this,
[me@linuxbox ~]$
&otice that it cotais o"r "ser ame, o"r host ame a! o"r c"rret workig !irectory,
%"t how !i! it get that wayC Rery simply, it t"rs o"t. The prompt is !e+ie! %y a
e#iromet #aria%le ame! :*# Eshort +or ?prompt strig oe@F. We ca #iew the
cotets o+ :*# with the echo comma!,
[me@linuxbox ~]$ echo 3BF1
[]u@]h ],]]$
1ote5 'o<t worry i+ yo"r res"lts are ot e3actly the same as the e3ample a%o#e.
E#ery *i"3 !istri%"tio !e+ies the prompt strig a little !i++eretly, some ="ite
e3otically.
0rom the res"lts, we ca see that :*# cotais a +ew o+ the characters we see i o"r
prompt s"ch as the %rackets, the at-sig, a! the !ollar sig, %"t the rest are a mystery.
The ast"te amog "s will recogiDe these as backslash-escaped special characters like
.K0
$atomy 7+ $ 6rompt
those we saw i Chapter 8. Aere is a partial list o+ the characters that the shell treats
specially i the prompt strig,
Table 1$-1: 1scape %odes 0sed *n Shell 6rompts
SeBuence Value Displa,ed
]a
$SC99 %ell. This makes the comp"ter %eep whe it is eco"tere!.
]d
C"rret !ate i !ay, moth, !ate +ormat. 0or e3ample, ?>o >ay
2K.@
]h
Aost ame o+ the local machie mi"s the trailig !omai ame.
]S
0"ll host ame.
]j
&"m%er o+ 4o%s r"ig i the c"rret shell sessio.
]l
&ame o+ the c"rret termial !e#ice.
]n
$ ewlie character.
]r
$ carriage ret"r.
]s
&ame o+ the shell program.
]t
C"rret time i 21 ho"r ho"rs,mi"tes,seco!s +ormat.
]T
C"rret time i .2 ho"r +ormat.
]@
C"rret time i .2 ho"r $>-6> +ormat.
]4
C"rret time i 21 ho"r ho"rs,mi"tes +ormat.
]u
)ser ame o+ the c"rret "ser.
]v
Rersio "m%er o+ the shell.
];
Rersio a! release "m%ers o+ the shell.
]9
&ame o+ the c"rret workig !irectory.
],
*ast part o+ the c"rret workig !irectory ame.
]G
Aistory "m%er o+ the c"rret comma!.
]^
&"m%er o+ comma!s etere! ito this shell sessio.
]$
This !isplays a ?[@ character "less yo" ha#e s"per"ser pri#ileges.
9 that case, it !isplays a ?P@ istea!.
][
Sigals the start o+ a series o+ oe or more o-pritig characters.
This is "se! to em%e! o-pritig cotrol characters which
maip"late the termial em"lator i some way, s"ch as mo#ig the
.K.
.1 8 C"stomiDig The 6rompt
c"rsor or chagig te3t colors.
]]
Sigals the e! o+ a o-pritig character se="ece.
Tr,ing Some #lternate Prompt Designs
With this list o+ special characters, we ca chage the prompt to see the e++ect. 0irst,
we<ll %ack "p the e3istig strig so we ca restore it later. To !o this, we will copy the
e3istig strig ito aother shell #aria%le that we create o"rsel#es,
[me@linuxbox ~]$ ps1.old$J3BF1J
We create a ew #aria%le calle! 7s#Oold a! assig the #al"e o+ :*# to it. We ca
#eri+y that the strig has %ee copie! with the echo comma!,
[me@linuxbox ~]$ echo 3ps1.old
[]u@]h ],]]$
We ca restore the origial prompt at ay time !"rig o"r termial sessio %y simply
re#ersig the process,
[me@linuxbox ~]$ BF1$J3ps1.oldJ
&ow that we are rea!y to procee!, let<s see what happes i+ we ha#e a empty prompt
strig,
[me@linuxbox ~]$ BF1$
9+ we assig othig to the prompt strig, we get othig. &o prompt strig at allL The
prompt is still there, %"t !isplays othig, 4"st as we aske! it to. Sice this is ki! o+
!iscocertig to look at, we<ll replace it with a miimal prompt,
BF1$JK3 J
That<s %etter. $t least ow we ca see what we are !oig. &otice the trailig space
withi the !o"%le ="otes. This pro#i!es the space %etwee the !ollar sig a! the c"rsor
.K2
Tryig Some $lterate 6rompt 'esigs
whe the prompt is !isplaye!.
*et<s a!! a %ell to o"r prompt,
$ BF1$JKaK3 J
&ow we sho"l! hear a %eep each time the prompt is !isplaye!. This co"l! get aoyig,
%"t it might %e "se+"l i+ we ee!e! oti+icatio whe a especially log-r"ig
comma! has %ee e3ec"te!.
&e3t, let<s try to make a i+ormati#e prompt with some host ame a! time-o+-!ay
i+ormatio,
$ BF1$JK: Kh K3 J
#):$$ linuxbox $
$!!ig time-o+-!ay to o"r prompt will %e "se+"l i+ we ee! to keep track o+ whe we
per+orm certai tasks. 0ially, we<ll make a ew prompt that is similar to o"r origial,
#):$) linuxbox $ BF1$J/KuVKh KW&K3 J
Xme@linuxbox ~D$
Try o"t the other se="eces liste! i the ta%le a%o#e a! see i+ yo" ca come "p with a
%rilliat ew prompt.
#dding Color
>ost termial em"lator programs respo! to certai o-pritig character se="eces to
cotrol s"ch thigs as character attri%"tes Elike color, %ol! te3t a! the !rea!e! %likig
te3tF a! c"rsor positio. We<ll co#er c"rsor positio i a little %it, %"t +irst we<ll look at
color.
Terminal Con"usion
Jack i aciet times, whe termials were hooke! to remote comp"ters, there
were may competig %ra!s o+ termials a! they all worke! !i++eretly. They
ha! !i++eret key%oar!s a! they all ha! !i++eret ways o+ iterpretig cotrol
.K(
.1 8 C"stomiDig The 6rompt
i+ormatio. )i3 a! )i3-like systems ha#e two rather comple3 s"%systems to
!eal with the %a%el o+ termial cotrol Ecalle! termca7 a! terminfoF. 9+
yo" look i the !eepest recesses o+ yo"r termial em"lator settigs yo" may +i!
a settig +or the type o+ termial em"latio.
9 a e++ort to make termials speak some sort o+ commo lag"age, the
$merica &atioal Sta!ar!s 9stit"te E$&S9F !e#elope! a sta!ar! set o+
character se="eces to cotrol #i!eo termials. 7l! time '7S "sers will
remem%er the 4L*A<*U* +ile that was "se! to ea%le iterpretatio o+ these
co!es.
Character color is cotrolle! %y se!ig the termial em"lator a )>S* escape code
em%e!!e! i the stream o+ characters to %e !isplaye!. The cotrol co!e !oes ot ?prit
o"t@ o the !isplay, rather it is iterprete! %y the termial as a istr"ctio. $s we saw i
the ta%le a%o#e, the ][ a! ]] se="eces are "se! to ecaps"late o-pritig characters.
$ $&S9 escape co!e %egis with a octal 0(( Ethe co!e geerate! %y the escape keyF
+ollowe! %y a optioal character attri%"te +ollowe! %y a istr"ctio. 0or e3ample, the
co!e to set the te3t color to ormal Eattri%"te W 0F, %lack te3t is,
]($$[(T$(m
Aere is a ta%le o+ a#aila%le te3t colors. &otice that the colors are !i#i!e! ito two gro"ps,
!i++eretiate! %y the applicatio o+ the %ol! character attri%"te E.F which creates the
appearace o+ ?light@ colors,
Table1$- ': 1scape Se<uences 0sed To Set Te+t %olors
SeBuence Text Color SeBuence Text Color
]($$[(T$(m
Jlack
]($$[#T$(m
'ark ;ray
]($$[(T$#m
:e!
]($$[#T$#m
*ight :e!
]($$[(T$!m
;ree
]($$[#T$!m
*ight ;ree
]($$[(T$$m
Jrow
]($$[#T$$m
5ellow
]($$[(T$%m
Jl"e
]($$[#T$%m
*ight Jl"e
]($$[(T$"m
6"rple
]($$[#T$"m
*ight 6"rple
]($$[(T$.m
Cya
]($$[#T$.m
*ight Cya
]($$[(T$)m
*ight ;rey
]($$[#T$)m
White
*et<s try to make a re! prompt. We<ll isert the escape co!e at the %egiig,
.K1
$!!ig Color
Xme@linuxbox ~D$ BF1$JK(K*(*#1mK)/KuVKh KW&K3 J
Xme@linuxbox ~D$
That works, %"t otice that all the te3t that we type a+ter the prompt is also re!. To +i3
this, we will a!! aother escape co!e to the e! o+ the prompt that tells the termial
em"lator to ret"r to the pre#io"s color,
Xme@linuxbox ~D$ BF1$JK(K*(*#1mK)/KuVKh KW&K3K(K*(*mK) J
Xme@linuxbox ~D$
That<s %etterL
9t<s also possi%le to set the te3t %ackgro"! color "sig the co!es liste! %elow. The
%ackgro"! colors !o ot s"pport the %ol! attri%"te.
Table 1$-3: 1scape Se<uences 0sed To Set 8ackground %olor
SeBuence /ac(ground Color SeBuence /ac(ground Color
]($$[(T%(m
Jlack
]($$[(T%%m
Jl"e
]($$[(T%#m
:e!
]($$[(T%"m
6"rple
]($$[(T%!m
;ree
]($$[(T%.m
Cya
]($$[(T%$m
Jrow
]($$[(T%)m
*ight ;rey
We ca create a prompt with a re! %ackgro"! %y applyig a simple chage to the +irst
escape co!e,
Xme@linuxbox ~D$ BF1$JK(K*(*#A1mK)/KuVKh KW&K3K(K*(*mK) J
Xme@linuxbox ~D$
Try o"t the color co!es a! see what yo" ca createL
1ote5 Jesi!es the ormal E0F a! %ol! E.F character attri%"tes, te3t may also %e
gi#e "!erscore E1F, %likig E2F, a! i#erse E/F attri%"tes as well. 9 the
iterests o+ goo! taste, may termial em"lators re+"se to hoor the %likig
attri%"te, howe#er.
.K2
.1 8 C"stomiDig The 6rompt
>o%ing The Cursor
Escape co!es ca %e "se! to positio the c"rsor. This is commoly "se! to pro#i!e a
clock or some other ki! o+ i+ormatio at a !i++eret locatio o the scree s"ch as a
"pper corer each time the prompt is !raw. Aere is a list o+ the escape co!es that
positio the c"rsor,
Table 1$-$: %ursor !ovement 1scape Se<uences
$scape Code #ction
]($$[lTcS
>o#e the c"rsor to lie l a! col"m c.
]($$[n4
>o#e the c"rsor "p n lies.
]($$[nH
>o#e the c"rsor !ow n lies.
]($$[nC
>o#e the c"rsor +orwar! n characters.
]($$[n'
>o#e the c"rsor %ackwar! n characters.
]($$[!?
Clear the scree a! mo#e the c"rsor to the "pper le+t corer Elie
0, col"m 0F.
]($$[2
Clear +rom the c"rsor positio to the e! o+ the c"rret lie.
]($$[s
Store the c"rret c"rsor positio.
]($$[u
:ecall the store! c"rsor positio.
)sig the co!es a%o#e, we<ll costr"ct a prompt that !raws a re! %ar at the top o+ the
scree cotaiig a clock Ere!ere! i yellow te3tF each time the prompt is !isplaye!.
The co!e +or the prompt is this +ormi!a%le lookig strig,
BF1$JK(K*(sK*(*#*RK*(*#A1mK*(XK*(1#mKtK*(*mK*(uK)
/KuVKh KW&K3 J
*et<s take a look at each part o+ the strig to see what it !oes,
SeBuence #ction
][
Jegis a o-pritig character se="ece. The real p"rpose o+
this is to allow bash to correctly calc"late the siDe o+ the
#isi%le prompt. Witho"t this, comma! lie e!itig +eat"res
will improperly positio the c"rsor.
]($$[s
Store the c"rsor positio. This is ee!e! to ret"r to the prompt
.KK
>o#ig The C"rsor
locatio a+ter the %ar a! clock ha#e %ee !raw at the top o+
the scree. 8e aware that some terminal emulators do not
honor this code#
]($$[(T(S
>o#e the c"rsor to the "pper le+t corer, which is lie Dero,
col"m Dero.
]($$[(T%#m
Set the %ackgro"! color to re!.
]($$[2
Clear +rom the c"rret c"rsor locatio Ethe top le+t corerF to
the e! o+ the lie. Sice the %ackgro"! color is ow re!, the
lie is cleare! to that color creatig o"r %ar. &ote that clearig
to the e! o+ the lie !oes ot chage the c"rsor positio, which
remais at the "pper le+t corer.
]($$[#T$$m
Set the te3t color to yellow.
]t
'isplay the c"rret time. While this is a ?pritig@ elemet, we
still icl"!e it i the o-pritig portio o+ the prompt, sice
we !o<t wat bash to icl"!e the clock whe calc"latig the
tr"e siDe o+ the !isplaye! prompt.
]($$[(m
T"r o++ color. This a++ects %oth the te3t a! %ackgro"!.
]($$[u
:estore the c"rsor positio sa#e! earlier.
]]
E! o-pritig characters se="ece.
X]u@]h ],D]$
6rompt strig.
Sa%ing The Prompt
7%#io"sly, we !o<t wat to %e typig that moster all the time, so we<ll wat to store o"r
prompt someplace. We ca make the prompt permaet %y a!!ig it to o"r <bashrc
+ile. To !o so, a!! these two lies to the +ile,
BF1$JK(K*(sK*(*#*RK*(*#A1mK*(XK*(1#mKtK*(*mK*(uK)
/KuVKh KW&K3 J
export BF1
Summing -p
Jelie#e it or ot, there is m"ch more that ca %e !oe with prompts i#ol#ig shell
+"ctios a! scripts that we ha#e<t co#ere! here, %"t this is a goo! start. &ot e#eryoe
.K/
.1 8 C"stomiDig The 6rompt
will care eo"gh to chage the prompt, sice the !e+a"lt prompt is "s"ally satis+actory.
J"t +or those o+ "s who like to tiker, the shell pro#i!es the opport"ity +or may ho"rs
o+ tri#ial +".
;urther 1eading
The 8ash 6rompt ,&/T& +rom the *i"3 'oc"metatio 6ro4ect pro#i!es a
pretty complete !isc"ssio o+ what the shell prompt ca %e ma!e to !o. 9t is
a#aila%le at,
http,--tl!p.org-A7WT7-Jash-6rompt-A7WT7-
Wikipe!ia has a goo! article o the $&S9 Escape Co!es,
http,--e.wikipe!ia.org-wiki-$&S9OescapeOco!e
.K8
6art 1 8 Commo Tasks $! Essetial Tools
'art # A Common Tasks And 6ssentia
Toos
.K9
.2 8 6ackage >aagemet
1& 2ac,age 'anage%ent
9+ we spe! ay time i the *i"3 comm"ity, we hear may opiios as to which o+ the
may *i"3 !istri%"tios is ?%est.@ 7+te, these !isc"ssios get really silly, +oc"sig o
s"ch thigs as the prettiess o+ the !esktop %ackgro"! Esome people wo<t "se )%"t"
%eca"se its !e+a"lt color scheme is %rowLF a! other tri#ial matters.
The most importat !etermiat o+ !istri%"tio ="ality is the packaging system a! the
#itality o+ the !istri%"tio<s s"pport comm"ity. $s we spe! more time with *i"3, we
see that its so+tware la!scape is e3tremely !yamic. Thigs are costatly chagig.
>ost o+ the top-tier *i"3 !istri%"tios release ew #ersios e#ery si3 moths a! may
i!i#i!"al program "p!ates e#ery !ay. To keep "p with this %liDDar! o+ so+tware, we
ee! goo! tools +or package management.
6ackage maagemet is a metho! o+ istallig a! maitaiig so+tware o the system.
To!ay, most people ca satis+y all o+ their so+tware ee!s %y istallig packages +rom
their *i"3 !istri%"tor. This cotrasts with the early !ays o+ *i"3, whe oe ha! to
!owloa! a! compile source code i or!er to istall so+tware. &ot that there is aythig
wrog with compilig so"rce co!eH i +act, ha#ig access to so"rce co!e is the great
wo!er o+ *i"3. 9t gi#es "s Ea! e#ery%o!y elseF the a%ility to e3amie a! impro#e the
system. 9t<s 4"st that ha#ig a pre-compile! package is +aster a! easier to !eal with.
9 this chapter, we will look at some o+ the comma! lie tools "se! +or package
maagemet. While all o+ the ma4or !istri%"tios pro#i!e power+"l a! sophisticate!
graphical programs +or maitaiig the system, it is importat to lear a%o"t the
comma! lie programs, too. They ca per+orm may tasks that are !i++ic"lt Eor
impossi%leF to !o with their graphical co"terparts.
Pac(aging S,stems
'i++eret !istri%"tios "se !i++eret packagig systems a! as a geeral r"le, a package
ite!e! +or oe !istri%"tio is ot compati%le with aother !istri%"tio. >ost
!istri%"tios +all ito oe o+ two camps o+ packagig techologies, the 'e%ia ?.!e%@
camp a! the :e! Aat ?.rpm@ camp. There are some importat e3ceptios s"ch as
;etoo, Slackware, a! 0oresight, %"t most others "se oe o+ these two %asic systems.
./0
6ackagig Systems
Table 1.-1: !aAor 6ackaging System Families
Pac(aging S,stem Distri0utions CPartial ListingD
'e%ia Style E.!e%F 'e%ia, )%"t", Ga!ros, *ispire
:e! Aat Style E.rpmF 0e!ora, Cet7S, :e! Aat Eterprise *i"3, 7peS)SE,
>a!ri#a, 6C*i"37S
3o4 # Pac(age S,stem +or(s
The metho! o+ so+tware !istri%"tio +o"! i the proprietary so+tware i!"stry "s"ally
etails %"yig a piece o+ istallatio me!ia s"ch as a ?istall !isk@ a! the r"ig a
?istallatio wiDar!@ to istall a ew applicatio o the system.
*i"3 !oes<t work that way. Rirt"ally all so+tware +or a *i"3 system will %e +o"! o
the 9teret. >ost o+ it will %e pro#i!e! %y the !istri%"tio #e!or i the +orm o+
package files a! the rest will %e a#aila%le i so"rce co!e +orm that ca %e istalle!
ma"ally. We<ll talk a little a%o"t how to istall so+tware %y compilig so"rce co!e i a
later chapter.
'a/kage 2ies
The %asic "it o+ so+tware i a packagig system is the package file. $ package +ile is a
compresse! collectio o+ +iles that comprise the so+tware package. $ package may
cosist o+ "mero"s programs a! !ata +iles that s"pport the programs. 9 a!!itio to the
+iles to %e istalle!, the package +ile also icl"!es meta!ata a%o"t the package, s"ch as a
te3t !escriptio o+ the package a! its cotets. $!!itioally, may packages cotai
pre- a! post-istallatio scripts that per+orm co+ig"ratio tasks %e+ore a! a+ter the
package istallatio.
6ackage +iles are create! %y a perso kow as a package maintainer, o+te E%"t ot
alwaysF a employee o+ the !istri%"tio #e!or. The package maitaier gets the
so+tware i so"rce co!e +orm +rom the upstream provider Ethe a"thor o+ the programF,
compiles it, a! creates the package meta!ata a! ay ecessary istallatio scripts.
7+te, the package maitaier will apply mo!i+icatios to the origial so"rce co!e to
impro#e the program<s itegratio with the other parts o+ the *i"3 !istri%"tio.
!e5ositories
While some so+tware pro4ects choose to per+orm their ow packagig a! !istri%"tio,
most packages to!ay are create! %y the !istri%"tio #e!ors a! itereste! thir! parties.
6ackages are ma!e a#aila%le to the "sers o+ a !istri%"tio i cetral repositories that may
cotai may tho"sa!s o+ packages, each specially %"ilt a! maitaie! +or the
!istri%"tio.
./.
.2 8 6ackage >aagemet
$ !istri%"tio may maitai se#eral !i++eret repositories +or !i++eret stages o+ the
so+tware !e#elopmet li+e cycle. 0or e3ample, there will "s"ally %e a ?testig@
repository that cotais packages that ha#e 4"st %ee %"ilt a! are ite!e! +or "se %y
%ra#e so"ls who are lookig +or %"gs %e+ore they are release! +or geeral !istri%"tio. $
!istri%"tio will o+te ha#e a ?!e#elopmet@ repository where work-i-progress packages
!estie! +or icl"sio i the !istri%"tio<s e3t ma4or release are kept.
$ !istri%"tio may also ha#e relate! thir!-party repositories. These are o+te ee!e! to
s"pply so+tware that, +or legal reasos s"ch as patets or ':> ati-circ"m#etio iss"es,
caot %e icl"!e! with the !istri%"tio. 6erhaps the %est kow case is that o+
ecrypte! 'R' s"pport, which is ot legal i the )ite! States. The thir!-party
repositories operate i co"tries where so+tware patets a! ati-circ"m#etio laws !o
ot apply. These repositories are "s"ally wholly i!epe!et o+ the !istri%"tio they
s"pport a! to "se them, oe m"st kow a%o"t them a! ma"ally icl"!e them i the
co+ig"ratio +iles +or the package maagemet system.
)e5enden/ies
6rograms sel!om ?sta!aloeH@ rather they rely o the presece o+ other so+tware
compoets to get their work !oe. Commo acti#ities, s"ch as ip"t-o"tp"t +or
e3ample, are ha!le! %y ro"ties share! %y may programs. These ro"ties are store! i
what are calle! shared libraries, which pro#i!e essetial ser#ices to more tha oe
program. 9+ a package re="ires a share! reso"rce s"ch as a share! li%rary, it is sai! to
ha#e a dependency. >o!er package maagemet systems all pro#i!e some metho! o+
dependency resolution to es"re that whe a package is istalle!, all o+ its !epe!ecies
are istalle!, too.
$igh And Lo%Be;e 'a/kage Toos
6ackage maagemet systems "s"ally cosist o+ two types o+ tools, low-le#el tools which
ha!le tasks s"ch as istallig a! remo#ig package +iles, a! high-le#el tools that
per+orm meta!ata searchig a! !epe!ecy resol"tio. 9 this chapter, we will look at
the tools s"pplie! with 'e%ia-style systems Es"ch as )%"t" a! may othersF a! those
"se! %y recet :e! Aat pro!"cts. While all :e! Aat-style !istri%"tios rely o the same
low-le#el program Er7mF, they "se !i++eret high-le#el tools. 0or o"r !isc"ssio, we will
co#er the high-le#el program 1um, "se! %y 0e!ora, :e! Aat Eterprise *i"3, a!
Cet7S. 7ther :e! Aat-style !istri%"tios pro#i!e high-le#el tools with compara%le
+eat"res.
Table1.- ': 6ackaging System Tools
Distri0utions Lo4ELe%el Tools 3ighELe%el Tools
'e%ia-Style
d7kg a7t-get> a7titude
./2
Aow $ 6ackage System Works
0e!ora, :e! Aat
Eterprise *i"3, Cet7S
r7m 1um
Common Pac(age >anagement Tas(s
There are may operatios that ca %e per+orme! with the comma! lie package
maagemet tools. We will look at the most commo. Je aware that the low-le#el tools
also s"pport creatio o+ package +iles, a acti#ity o"tsi!e the scope o+ this %ook.
9 the !isc"ssio %elow, the term ?package_name@ re+ers to the act"al ame o+ a
package rather tha the term ?package_file,@ which is the ame o+ the +ile that
cotais the package.
2inding A 'a/kage In A !e5ository
)sig the high-le#el tools to search repository meta!ata, a package ca %e locate! %ase!
o its ame or !escriptio.
Table 1.-3: 6ackage Search %ommands
St,le CommandCsD
'e%ia
a7t-get u7date
a7t-cache search search_string
:e! Aat
1um search search_string
E3ample, To search a 1um repository +or the emacs te3t e!itor, this comma! co"l! %e
"se!,
1um search emacs
Instaing A 'a/kage 2rom A !e5ository
Aigh-le#el tools permit a package to %e !owloa!e! +rom a repository a! istalle! with
+"ll !epe!ecy resol"tio.
Table 1.-$: 6ackage *nstallation %ommands
St,le CommandCsD
'e%ia
a7t-get u7date
./(
.2 8 6ackage >aagemet
a7t-get install package_name
:e! Aat
1um install package_name
E3ample, To istall the emacs te3t e!itor +rom a a7t repository,
a7t-get u7dateT a7t-get install emacs
Instaing A 'a/kage 2rom A 'a/kage 2ie
9+ a package +ile has %ee !owloa!e! +rom a so"rce other tha a repository, it ca %e
istalle! !irectly Etho"gh witho"t !epe!ecy resol"tioF "sig a low-le#el tool.
Table 1.-.: (ow-(evel 6ackage *nstallation %ommands
St,le CommandCsD
'e%ia
d7kg --install package_file
:e! Aat
r7m -i package_file
E3ample, 9+ the emacs-!!<#-)<fc)-i$/.<r7m package +ile ha! %ee !owloa!e!
+rom a o-repository site, it wo"l! %e istalle! this way,
r7m -i emacs-!!<#-)<fc)-i$/.<r7m
1ote5 Sice this techi="e "ses the low-le#el r7m program to per+orm the
istallatio, o !epe!ecy resol"tio is per+orme!. 9+ r7m !isco#ers a missig
!epe!ecy, r7m will e3it with a error.
!emo;ing A 'a/kage
6ackages ca %e "istalle! "sig either the high-le#el or low-tools. The high-le#el tools
are show %elow.
./1
Commo 6ackage >aagemet Tasks
Table1.- 2: 6ackage 9emoval %ommands
St,le CommandCsD
'e%ia
a7t-get remove package_name
:e! Aat
1um erase package_name
E3ample, To "istall the emacs package +rom a 'e%ia-style system,
a7t-get remove emacs
U5dating 'a/kages 2rom A !e5ository
The most commo package maagemet task is keepig the system "p-to-!ate with the
latest packages. The high-le#el tools ca per+orm this #ital task i oe sigle step.
Table 1.-3: 6ackage 0pdate %ommands
St,le CommandCsD
'e%ia
a7t-get u7dateT a7t-get u7grade
:e! Aat
1um u7date
E3ample, To apply ay a#aila%le "p!ates to the istalle! packages o a 'e%ia-style
system,
a7t-get u7dateT a7t-get u7grade
U5grading A 'a/kage 2rom A 'a/kage 2ie
9+ a "p!ate! #ersio o+ a package has %ee !owloa!e! +rom a o-repository so"rce, it
ca %e istalle!, replacig the pre#io"s #ersio,
Table 1.-4: (ow-(evel 6ackage 0pgrade %ommands
St,le CommandCsD
'e%ia
d7kg --install package_file
./2
.2 8 6ackage >aagemet
:e! Aat
r7m -3 package_file
E3ample, )p!atig a e3istig istallatio o+ emacs to the #ersio cotaie! i the
package +ile emacs-!!<#-)<fc)-i$/.<r7m o a :e! Aat system,
r7m -3 emacs-!!<#-)<fc)-i$/.<r7m
1ote5 d7kg !oes ot ha#e a speci+ic optio +or "pgra!ig a package #ers"s
istallig oe as r7m !oes.
Listing Instaed 'a/kages
These comma!s ca %e "se! to !isplay a list o+ all the packages istalle! o the system,
Table 1.-5: 6ackage (isting %ommands
St,le CommandCsD
'e%ia
d7kg --list
:e! Aat
r7m -Ba
)etermining I= A 'a/kage Is Instaed
These low-le#el tools ca %e "se! to !isplay whether a speci+ie! package is istalle!,
Table 1.-1=: 6ackage Status %ommands
St,le CommandCsD
'e%ia
d7kg --status package_name
:e! Aat
r7m -B package_name
E3ample, To !etermie i+ the emacs package is istalle! o a 'e%ia style system,
d7kg --status emacs
./K
Commo 6ackage >aagemet Tasks
)is5aying In=o About An Instaed 'a/kage
9+ the ame o+ a istalle! package is kow, the +ollowig comma!s ca %e "se! to
!isplay a !escriptio o+ the package,
Table 1.-11: 6ackage *nformation %ommands
St,le CommandCsD
'e%ia
a7t-cache sho9 package_name
:e! Aat
1um info package_name
E3ample, To see a !escriptio o+ the emacs package o a 'e%ia-style system,
a7t-cache sho9 emacs
2inding Whi/h 'a/kage Instaed A 2ie
To !etermie what package is resposi%le +or the istallatio o+ a partic"lar +ile, the
+ollowig comma!s ca %e "se!,
Table 1.-1': 6ackage File *dentification %ommands
St,le CommandCsD
'e%ia
d7kg --search file_name
:e! Aat
r7m -Bf file_name
E3ample, To see what package istalle! the 6usr6bin6vim +ile o a :e! Aat system,
r7m -Bf 6usr6bin6vim
Summing -p
9 the chapters that +ollow, we will e3plore may !i++eret programs co#erig a wi!e
rage o+ applicatio areas. While most o+ these programs are commoly istalle! %y
!e+a"lt, we may ee! to istall a!!itioal packages i+ ecessary programs are ot alrea!y
istalle! o o"r system. With o"r ew+o"! kowle!ge Ea! appreciatioF o+ package
.//
.2 8 6ackage >aagemet
maagemet, we sho"l! ha#e o pro%lem istallig a! maagig the programs we ee!.
The Linux So"t4are Installation >,th
6eople migratig +rom other plat+orms sometimes +all #ictim to the myth that
so+tware is somehow !i++ic"lt to istall "!er *i"3 a! that the #ariety o+
packagig schemes "se! %y !i++eret !istri%"tios is a hi!race. Well, it is a
hi!race, %"t oly to proprietary so+tware #e!ors who wish to !istri%"te %iary-
oly #ersios o+ their secret so+tware.
The *i"3 so+tware ecosystem is %ase! o the i!ea o+ ope so"rce co!e. 9+ a
program !e#eloper releases so"rce co!e +or a pro!"ct, it is likely that a perso
associate! with a !istri%"tio will package the pro!"ct a! icl"!e it i their
repository. This metho! es"res that the pro!"ct is well itegrate! ito the
!istri%"tio a! the "ser is gi#e the co#eiece o+ ?oe-stop shoppig@ +or
so+tware, rather tha ha#ig to search +or each pro!"ct<s we% site.
'e#ice !ri#ers are are ha!le! i m"ch the same way, e3cept that istea! o+ %eig
separate items i a !istri%"tio<s repository, they %ecome part o+ the *i"3 kerel
itsel+. ;eerally speakig, there is o s"ch thig as a ?!ri#er !isk@ i *i"3.
Either the kerel s"pports a !e#ice or it !oes<t, a! the *i"3 kerel s"pports a
lot o+ !e#ices. >ay more, i +act, tha Wi!ows !oes. 7+ co"rse, this is o+ o
cosolatio i+ the partic"lar !e#ice yo" ee! is ot s"pporte!. Whe that
happes, yo" ee! to look at the ca"se. $ lack o+ !ri#er s"pport is "s"ally ca"se!
%y oe o+ three thigs,
8. The device is too ne+. Sice may har!ware #e!ors !o<t acti#ely s"pport
*i"3 !e#elopmet, it +alls "po a mem%er o+ the *i"3 comm"ity to write the
kerel !ri#er co!e. This takes time.
2. The device is too exotic. &ot all !istri%"tios icl"!e e#ery possi%le !e#ice
!ri#er. Each !istri%"tio %"il!s their ow kerels, a! sice kerels are #ery
co+ig"ra%le Ewhich is what makes it possi%le to r" *i"3 o e#erythig +rom
wristwatches to mai+ramesF they may ha#e o#erlooke! a partic"lar !e#ice. Jy
locatig a! !owloa!ig the so"rce co!e +or the !ri#er, it is possi%le +or yo"
Eyes, yo"F to compile a! istall the !ri#er yo"rsel+. This process is ot o#erly
!i++ic"lt, %"t it is rather i#ol#e!. We<ll talk a%o"t compilig so+tware i a later
chapter.
. The hard+are vendor is hiding something. They ha#e either release!
so"rce co!e +or a *i"3 !ri#er, or ha#e they release! the techical
!oc"metatio +or some%o!y to create oe +or them. This meas that the
har!ware #e!or is tryig to keep the programmig iter+aces to the !e#ice a
./8
S"mmig )p
secret. Sice we !o<t wat secret !e#ices i o"r comp"ters, 9 s"ggest that yo"
remo#e the o++e!ig har!ware a! pitch it ito the trash, with yo"r other "seless
items.
;urther 1eading
Spe! some time gettig to kow the package maagemet system +or yo"r !istri%"tio.
Each !istri%"tio pro#i!es !oc"metatio +or its package maagemet tools. 9 a!!itio,
here are some more geeric so"rces,
The -ebian @>0;(inu+ F): chapter o package maagemet pro#i!es a
o#er#iew o+ package maagemet o 'e%ia systems ,
http,--www.!e%ia.org-!oc-0$T-ch-pkgtools.e.html
The home page +or the :6> pro4ect,
http,--www.rpm.org
The home page +or the 5)> pro4ect at '"ke )i#ersity,
http,--li"3.!"ke.e!"-pro4ects-y"m-
0or a little %ackgro"!, the Wikipe!ia has a article o meta!ata,
http,--e.wikipe!ia.org-wiki->eta!ata
./9
.K 8 Storage >e!ia
1+ Storage 'edia
9 pre#io"s chapters weI#e looke! at maip"latig !ata at the +ile le#el. 9 this chapter,
we will cosi!er !ata at the !e#ice le#el. *i"3 has amaDig capa%ilities +or ha!lig
storage !e#ices, whether physical storage, s"ch as har! !isks, or etwork storage, or
#irt"al storage !e#ices like :$9' E:e!"!at $rray o+ 9!epe!et 'isksF a! *R>
E*ogical Rol"me >aagerF.
Aowe#er, sice this is ot a %ook a%o"t system a!miistratio, we will ot try to co#er
this etire topic i !epth. What we will try to !o is itro!"ce some o+ the cocepts a!
key comma!s that are "se! to maage storage !e#ices.
To carry o"t the e3ercises i this chapter, we will "se a )SJ +lash !ri#e, a C'-:W !isk
E+or systems e="ippe! with a C'-:7> %"rerF a! a +loppy !isk Eagai, i+ the system is
so e="ippe!.F
We will look at the +ollowig comma!s,
mount 8 >o"t a +ile system
umount 8 )mo"t a +ile system
fsck 8 Check a! repair a +ile system
fdisk 8 6artitio ta%le maip"lator
mkfs 8 Create a +ile system
fdformat 8 0ormat a +loppy !isk
dd 8 Write %lock oriete! !ata !irectly to a !e#ice
genisoimage NmkisofsP 8 Create a 9S7 9KK0 image +ile
9odim NcdrecordP 8 Write !ata to optical storage me!ia
md"sum 8 Calc"late a >'2 checks"m
>ounting #nd -nmounting Storage De%ices
:ecet a!#aces i the *i"3 !esktop ha#e ma!e storage !e#ice maagemet e3tremely
.80
>o"tig $! )mo"tig Storage 'e#ices
easy +or !esktop "sers. 0or the most part, we attach a !e#ice to o"r system a! it ?4"st
works.@ Jack i the ol! !ays Esay, 2001F, this st"++ ha! to %e !oe ma"ally. 7 o-
!esktop systems Ei.e., ser#ersF this is still a largely ma"al proce!"re sice ser#ers o+te
ha#e e3treme storage ee!s a! comple3 co+ig"ratio re="iremets.
The +irst step i maagig a storage !e#ice is attachig the !e#ice to the +ile system tree.
This process, calle! mounting, allows the !e#ice to participate with the operatig system.
$s we recall +rom Chapter (, )i3-like operatig systems, like *i"3, maitai a sigle
+ile system tree with !e#ices attache! at #ario"s poits. This cotrasts with other
operatig systems s"ch as >S-'7S a! Wi!ows that maitai separate trees +or each
!e#ice E+or e3ample C:], ':], etc.F.
There is a +ile ame! 6etc6fstab that lists the !e#ices Etypically har! !isk partitiosF
that are to %e mo"te! at %oot time. Aere is a e3ample 6etc6fstab +ile +rom a
0e!ora / system,
=4H&=M6#! 6 ext$ defaults # #
=4H&=M6home 6home ext$ defaults # !
=4H&=M6boot 6boot ext$ defaults # !
tm7fs 6dev6shm tm7fs defaults ( (
dev7ts 6dev67ts dev7ts gidM">modeM.!( ( (
s1sfs 6s1s s1sfs defaults ( (
7roc 67roc 7roc defaults ( (
=4H&=M*,4:-sda$ s9a7 s9a7 defaults ( (
>ost o+ the +ile systems liste! i this e3ample +ile are #irt"al a! are ot applica%le to o"r
!isc"ssio. 0or o"r p"rposes, the iterestig oes are the +irst three,
=4H&=M6#! 6 ext$ defaults # #
=4H&=M6home 6home ext$ defaults # !
=4H&=M6boot 6boot ext$ defaults # !
These are the har! !isk partitios. Each lie o+ the +ile cosists o+ si3 +iel!s, as +ollows,
Table 12-1: ;etc;fstab Fields
;ield Contents Description
. 'e#ice Tra!itioally, this +iel! cotais the act"al ame o+ a
!e#ice +ile associate! with the physical !e#ice, s"ch as
6dev6hda# Ethe +irst partitio o+ the master !e#ice
o the +irst 9'E chaelF. J"t with to!ay<s comp"ters,
which ha#e may !e#ices that are hot pl"gga%le Elike
.8.
.K 8 Storage >e!ia
)SJ !ri#esF, may mo!er *i"3 !istri%"tios
associate a !e#ice with a te3t la%el istea!. This la%el
Ewhich is a!!e! to the storage me!ia whe it is
+ormatte!F is rea! %y the operatig system whe the
!e#ice is attache! to the system. That way, o matter
which !e#ice +ile is assige! to the act"al physical
!e#ice, it ca still %e correctly i!eti+ie!.
2 >o"t 6oit The !irectory where the !e#ice is attache! to the +ile
system tree.
( 0ile System Type *i"3 allows may +ile system types to %e mo"te!.
>ost ati#e *i"3 +ile systems are e3t(, %"t may
others are s"pporte!, s"ch as 0$T.K EmsdosF, 0$T(2
EvfatF, &T0S EntfsF, C'-:7> Eiso0..(F, etc.
1 7ptios 0ile systems ca %e mo"te! with #ario"s optios. 9t
is possi%le, +or e3ample, to mo"t +ile systems as
rea!-oly, or pre#et ay programs +rom %eig
e3ec"te! +rom them Ea "se+"l sec"rity +eat"re +or
remo#a%le me!ia.F
2 0re="ecy $ sigle "m%er that speci+ies i+ a! whe a +ile
system is to %e %acke! "p with the dum7 comma!.
K 7r!er $ sigle "m%er that speci+ies i what or!er +ile
systems sho"l! %e checke! with the fsck comma!.
?ie%ing A List <= :ounted 2ie Systems
The mount comma! is "se! to mo"t +ile systems. Eterig the comma! witho"t
arg"mets will !isplay a list o+ the +ile systems c"rretly mo"te!,
[me@linuxbox ~]$ mount
6dev6sda! on 6 t17e ext$ Nr9P
7roc on 67roc t17e 7roc Nr9P
s1sfs on 6s1s t17e s1sfs Nr9P
dev7ts on 6dev67ts t17e dev7ts Nr9>gidM">modeM.!(P
6dev6sda" on 6home t17e ext$ Nr9P
6dev6sda# on 6boot t17e ext$ Nr9P
tm7fs on 6dev6shm t17e tm7fs Nr9P
none on 67roc6s1s6fs6binfmtOmisc t17e binfmtOmisc Nr9P
sunr7c on 6var6lib6nfs6r7cO7i7efs t17e r7cO7i7efs Nr9P
.82
>o"tig $! )mo"tig Storage 'e#ices
fusectl on 6s1s6fs6fuse6connections t17e fusectl Nr9P
6dev6sdd# on 6media6disk t17e vfat Nr9>nosuid>nodev>noatime>
uhel7erMhal>uidM"((>utf/>shortnameMlo9erP
t9in%:6musicbox on 6misc6musicbox t17e nfs% Nr9>addrM#0!<#./<#<%P
The +ormat o+ the listig is, device o mountpoint type filesystemtype EoptionsF. 0or
e3ample, the +irst lie shows that !e#ice 6dev6sda! is mo"te! as the root +ile system
a! it is o+ type e3t( a! is %oth rea!a%le a! writa%le Ethe optio ?rw@F. This listig also
has two iterestig etries at the %ottom o+ the list. The e3t to last etry shows a 2
giga%yte S' memory car! i a car! rea!er mo"te! at 6media6disk, a! the last etry
is a etwork !ri#e mo"te! at 6misc6musicbox.
0or o"r +irst e3perimet, we will work with a C'-:7>. 0irst, let<s look at a system
%e+ore a C'-:7> is iserte!,
[me@linuxbox ~]$ mount
6dev6ma77er6;ol@rou7((-=og;ol(( on 6 t17e ext$ Nr9P
7roc on 67roc t17e 7roc Nr9P
s1sfs on 6s1s t17e s1sfs Nr9P
dev7ts on 6dev67ts t17e dev7ts Nr9>gidM">modeM.!(P
6dev6hda# on 6boot t17e ext$ Nr9P
tm7fs on 6dev6shm t17e tm7fs Nr9P
none on 67roc6s1s6fs6binfmtOmisc t17e binfmtOmisc Nr9P
sunr7c on 6var6lib6nfs6r7cO7i7efs t17e r7cO7i7efs Nr9P
This listig is +rom a Cet7S 2 system, which is "sig *R> E*ogical Rol"me >aagerF
to create its root +ile system. *ike may mo!er *i"3 !istri%"tios, this system will
attempt to a"tomatically mo"t the C'-:7> a+ter isertio. $+ter we isert the !isk, we
see the +ollowig,
[me@linuxbox ~]$ mount
6dev6ma77er6;ol@rou7((-=og;ol(( on 6 t17e ext$ Nr9P
7roc on 67roc t17e 7roc Nr9P
s1sfs on 6s1s t17e s1sfs Nr9P
dev7ts on 6dev67ts t17e dev7ts Nr9>gidM">modeM.!(P
6dev6hda# on 6boot t17e ext$ Nr9P
tm7fs on 6dev6shm t17e tm7fs Nr9P
none on 67roc6s1s6fs6binfmtOmisc t17e binfmtOmisc Nr9P
sunr7c on 6var6lib6nfs6r7cO7i7efs t17e r7cO7i7efs Nr9P
6dev6hdc on 6media6live-#<(<#(-/ t17e iso0..( Nro>noexec>nosuid>
nodev>uidM"((P
$+ter we isert the !isk, we see the same listig as %e+ore with oe a!!itioal etry. $t
.8(
.K 8 Storage >e!ia
the e! o+ the listig we see that the C'-:7> Ewhich is !e#ice 6dev6hdc o this
systemF has %ee mo"te! o 6media6live-#<(<#(-/, a! is type iso9KK0 Ea C'-
:7>F. 0or p"rposes o+ o"r e3perimet, we<re itereste! i the ame o+ the !e#ice.
Whe yo" co!"ct this e3perimet yo"rsel+, the !e#ice ame will most likely %e
!i++eret.
'arning5 9 the e3amples that +ollow, it is #itally importat that yo" pay close
attetio to the act"al !e#ice ames i "se o yo"r system a! do not use the
names used in this text2
$lso ote that a"!io C's are ot the same as C'-:7>s. $"!io C's !o ot
cotai +ile systems a! th"s caot %e mo"te! i the "s"al sese.
&ow that we ha#e the !e#ice ame o+ the C'-:7> !ri#e, let<s "mo"t the !isk a!
remo"t it aother locatio i the +ile system tree. To !o this, we %ecome the s"per"ser
E"sig the comma! appropriate +or o"r systemF a! "mo"t the !isk with the umount
Eotice the spelligF comma!,
[me@linuxbox ~]$ su -
:ass9ord:
[root@linuxbox ~]^ umount /dev/hdc
The e3t step is to create a ew mount point +or the !isk. $ mo"t poit is simply a
!irectory somewhere o the +ile system tree. &othig special a%o"t it. 9t !oes<t e#e
ha#e to %e a empty !irectory, tho"gh i+ yo" mo"t a !e#ice o a o-empty !irectory,
yo" will ot %e a%le to see the !irectory<s pre#io"s cotets "til yo" "mo"t the !e#ice.
0or o"r p"rposes, we will create a ew !irectory,
[root@linuxbox ~]^ mkdir /mnt/cdrom
0ially, we mo"t the C'-:7> at the ew mo"t poit. The -t optio is "se! to
speci+y the +ile system type,
[root@linuxbox ~]^ mount -t isoDNN* /dev/hdc /mnt/cdrom
$+terwar!, we ca e3amie the cotets o+ the C'-:7> #ia the ew mo"t poit,
.81
>o"tig $! )mo"tig Storage 'e#ices
[root@linuxbox ~]^ cd /mnt/cdrom
[root@linuxbox cdrom]^ ls
&otice what happes whe we try to "mo"t the C'-:7>,
[root@linuxbox cdrom]^ umount /dev/hdc
umount: 6mnt6cdrom: device is bus1
Why is thisC The reaso is that we caot "mo"t a !e#ice i+ the !e#ice is %eig "se!
%y someoe or some process. 9 this case, we chage! o"r workig !irectory to the
mo"t poit +or the C'-:7>, which ca"ses the !e#ice to %e %"sy. We ca easily reme!y
the iss"e %y chagig the workig !irectory to somethig other tha the mo"t poit,
[root@linuxbox cdrom]^ cd
[root@linuxbox ~]^ umount /dev/hdc
&ow the !e#ice "mo"ts s"ccess+"lly.
+h, -nmounting Is Important
9+ yo" look at the o"tp"t o+ the free comma!, which !isplays statistics a%o"t
memory "sage, yo" will see a statistic calle! ?%"++ers.@ Comp"ter systems are
!esige! to go as +ast as possi%le. 7e o+ the impe!imets to system spee! is
slow !e#ices. 6riters are a goo! e3ample. E#e the +astest priter is e3tremely
slow %y comp"ter sta!ar!s. $ comp"ter wo"l! %e #ery slow i!ee! i+ it ha! to
stop a! wait +or a priter to +iish pritig a page. 9 the early !ays o+ 6Cs
E%e+ore m"lti-taskigF, this was a real pro%lem. 9+ yo" were workig o a
sprea!sheet or te3t !oc"met, the comp"ter wo"l! stop a! %ecome "a#aila%le
e#ery time yo" prite!. The comp"ter wo"l! se! the !ata to the priter as +ast as
the priter co"l! accept it, %"t it was #ery slow sice priters !o<t prit #ery +ast.
This pro%lem was sol#e! %y the a!#et o+ the printer buffer, a !e#ice cotaiig
some :$> memory that wo"l! sit %etwee the comp"ter a! the priter. With
the priter %"++er i place, the comp"ter wo"l! se! the priter o"tp"t to the
%"++er a! it wo"l! ="ickly %e store! i the +ast :$> so the comp"ter co"l! go
%ack to work witho"t waitig. >eawhile, the priter %"++er wo"l! slowly spool
the !ata to the priter +rom the %"++er<s memory at the spee! at which the priter
co"l! accept it.
.82
.K 8 Storage >e!ia
This i!ea o+ %"++erig is "se! e3tesi#ely i comp"ters to make them +aster.
'o<t let the ee! to occasioally rea! or write !ata to-+rom slow !e#ices impe!e
the spee! o+ the system. 7peratig systems store !ata rea! +rom, a! to %e
writte to storage !e#ices i memory +or as log as possi%le %e+ore act"ally
ha#ig to iteract with the slower !e#ice. 7 a *i"3 system +or e3ample, yo"
will otice that the system seems to +ill "p memory the loger it is "se!. This
!oes ot mea *i"3 is ?"sig? all the memory, it meas that *i"3 is takig
a!#atage o+ all the a#aila%le memory to !o as m"ch %"++erig as it ca.
This %"++erig allows writig to storage !e#ices to %e !oe #ery ="ickly, %eca"se
the writig to the physical !e#ice is %eig !e+erre! to a +"t"re time. 9 the
meatime, the !ata !estie! +or the !e#ice is pilig "p i memory. 0rom time to
time, the operatig system will write this !ata to the physical !e#ice.
)mo"tig a !e#ice etails writig all the remaiig !ata to the !e#ice so that it
ca %e sa+ely remo#e!. 9+ the !e#ice is remo#e! witho"t "mo"tig it +irst, the
possi%ility e3ists that ot all the !ata !estie! +or the !e#ice has %ee tras+erre!.
9 some cases, this !ata may icl"!e #ital !irectory "p!ates, which will lea! to
file system corruption, oe o+ the worst thigs that ca happe o a comp"ter.
)etermining )e;i/e ,ames
9t<s sometimes !i++ic"lt to !etermie the ame o+ a !e#ice. Jack i the ol! !ays, it was<t
#ery har!. $ !e#ice was always i the same place a! it !i!<t chage. )i3-like
systems like it that way. Jack whe )i3 was !e#elope!, ?chagig a !isk !ri#e@
i#ol#e! "sig a +orkli+t to remo#e a washig machie-siDe! !e#ice +rom the comp"ter
room. 9 recet years, the typical !esktop har!ware co+ig"ratio has %ecome ="ite
!yamic a! *i"3 has e#ol#e! to %ecome more +le3i%le tha its acestors.
9 the e3amples a%o#e we took a!#atage o+ the mo!er *i"3 !esktop<s a%ility to
?a"tomagically@ mo"t the !e#ice a! the !etermie the ame a+ter the +act. J"t what
i+ we are maagig a ser#er or some other e#iromet where this !oes ot occ"rC Aow
ca we +ig"re it o"tC
0irst, let<s look at how the system ames !e#ices. 9+ we list the cotets o+ the 6dev
!irectory Ewhere all !e#ices li#eF, we ca see that there are lots a! lots o+ !e#ices,
[me@linuxbox ~]$ ls /dev
The cotets o+ this listig re#eal some patters o+ !e#ice amig. Aere are a +ew,
.8K
>o"tig $! )mo"tig Storage 'e#ices
Table 12-': (inu+ Storage -evice >ames
Pattern De%ice
6dev6fdE
0loppy !isk !ri#es.
6dev6hdE
9'E E6$T$F !isks o ol!er systems. Typical mother%oar!s
cotai two 9'E coectors or channels, each with a ca%le with
two attachmet poits +or !ri#es. The +irst !ri#e o the ca%le is
calle! the master !e#ice a! the seco! is calle! the slave
!e#ice. The !e#ice ames are or!ere! s"ch that 6dev6hda
re+ers to the master !e#ice o the +irst chael, 6dev6hdb is the
sla#e !e#ice o the +irst chaelH 6dev6hdc, the master !e#ice
o the seco! chael, a! so o. $ trailig !igit i!icates the
partitio "m%er o the !e#ice. 0or e3ample, -dev6hda# re+ers
to the +irst partitio o the +irst har! !ri#e o the system while 6
dev6hda re+ers to the etire !ri#e.
6dev6l7E
6riters.
6dev6sdE
SCS9 !isks. 7 recet *i"3 systems, the kerel treats all !isk-
like !e#ices Eicl"!ig 6$T$-S$T$ har! !isks, +lash !ri#es, a!
)SJ mass storage !e#ices, s"ch as porta%le m"sic players a!
!igital camerasF as SCS9 !isks. The rest o+ the amig system is
similar to the ol!er 6dev6hdE amig scheme !escri%e! a%o#e.
6dev6srE
7ptical !ri#es EC'-'R' rea!ers a! %"rersF
9 a!!itio, we o+te see sym%olic liks s"ch as 6dev6cdrom, 6dev6dvd a! 6dev6
flo771, which poit to the act"al !e#ice +iles, pro#i!e! as a co#eiece.
9+ yo" are workig o a system that !oes ot a"tomatically mo"t remo#a%le !e#ices,
yo" ca "se the +ollowig techi="e to !etermie how the remo#a%le !e#ice is ame!
whe it is attache!. 0irst, start a real-time #iew o+ the 6var6log6messages +ile Eyo"
may re="ire s"per"ser pri#ileges +or thisF,
[me@linuxbox ~]$ sudo tail -f /var/log/messages
The last +ew lies o+ the +ile will %e !isplaye! a! the pa"se. &e3t, pl"g i the
remo#a%le !e#ice. 9 this e3ample, we will "se a .K >J +lash !ri#e. $lmost
imme!iately, the kerel will otice the !e#ice a! pro%e it,
.8/
.K 8 Storage >e!ia
?ul !$ #(:():"$ linuxbox kernel: usb $-!: ne9 full s7eed 3*H device
using uhciOhcd and address !
?ul !$ #(:():"$ linuxbox kernel: usb $-!: configuration ^# chosen
from # choice
?ul !$ #(:():"$ linuxbox kernel: scsi$ : *C*A emulation for 3*H +ass
*torage devices
?ul !$ #(:():"/ linuxbox kernel: scsi scan: ALZ3ARU result too short
N"P> using $.
?ul !$ #(:():"/ linuxbox kernel: scsi $:(:(:(: 'irect-4ccess &as1
'isk #<(( :Z: ( 4L*A: !
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] $#!.$ "#!-b1te
hard9are sectors N#. +HP
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] ,rite :rotect is
off
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] 4ssuming drive
cache: 9rite through
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] $#!.$ "#!-b1te
hard9are sectors N#. +HP
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] ,rite :rotect is
off
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] 4ssuming drive
cache: 9rite through
?ul !$ #(:():"0 linuxbox kernel: sdb: sdb#
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] 4ttached *C*A
removable disk
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: 4ttached scsi generic
sg$ t17e (
$+ter the !isplay pa"ses agai, type Ctrl-c to get the prompt %ack. The iterestig parts
o+ the o"tp"t are the repeate! re+ereces to ?as!%b@ which matches o"r e3pectatio o+ a
SCS9 !isk !e#ice ame. Mowig this, two lies %ecome partic"larly ill"miatig,
?ul !$ #(:():"0 linuxbox kernel: sdb: sdb#
?ul !$ #(:():"0 linuxbox kernel: sd $:(:(:(: [sdb] 4ttached *C*A
removable disk
This tells "s the !e#ice ame is 6dev6sdb +or the etire !e#ice a! 6dev6sdb# +or
the +irst partitio o the !e#ice. $s we ha#e see, workig with *i"3 is +"ll o+
iterestig !etecti#e workL
Ti(5 )sig the tail -f 6var6log6messages techi="e is a great way to
watch what the system is !oig i ear real-time.
With o"r !e#ice ame i ha!, we ca ow mo"t the +lash !ri#e,
.88
>o"tig $! )mo"tig Storage 'e#ices
[me@linuxbox ~]$ sudo mkdir /mnt/flash
[me@linuxbox ~]$ sudo mount /dev/sdb1 /mnt/flash
[me@linuxbox ~]$ df
-iles1stem #2-blocks 3sed 4vailable 3se5 +ounted on
6dev6sda! #"##"%"! "#/.0%% 0))"#.% $"5 6
6dev6sda" "0.$#0(/ $#)))$). !%)).%/( ")5 6home
6dev6sda# #%)).% #)!)) #!!/"/ #$5 6boot
tm7fs ))./(/ ( ))./(/ (5 6dev6shm
6dev6sdb# #"".( ( #"".( (5 6mnt6flash
The !e#ice ame will remai the same as log as it remais physically attache! to the
comp"ter a! the comp"ter is ot re%oote!.
Creating 7e4 ;ile S,stems
*et<s say that we wat to re+ormat the +lash !ri#e with a *i"3 ati#e +ile system, rather
tha the 0$T(2 system it has ow. This i#ol#es two steps, .. EoptioalF create a ew
partitio layo"t i+ the e3istig oe is ot to o"r likig, a! 2. create a ew, empty +ile
system o the !ri#e.
'arning2 9 the +ollowig e3ercise, we are goig to +ormat a +lash !ri#e. )se a
!ri#e that cotais othig yo" care a%o"t %eca"se it will %e erase!L $gai, make
absolutely sure you are s(eci"ying the correct device name "or your system6 not
the one sho+n in the text. 9ailure to heed this +arning could result in you
"ormatting *i.e.6 erasing, the +rong drive2
:ani5uating 'artitions With fdisk
The fdisk program allows "s to iteract !irectly with !isk-like !e#ices Es"ch as har!
!isk !ri#es a! +lash !ri#esF at a #ery low le#el. With this tool we ca e!it, !elete, a!
create partitios o the !e#ice. To work with o"r +lash !ri#e, we m"st +irst "mo"t it Ei+
ee!e!F a! the i#oke the fdisk program as +ollows,
[me@linuxbox ~]$ sudo umount /dev/sdb1
[me@linuxbox ~]$ sudo fdisk /dev/sdb
&otice that we m"st speci+y the !e#ice i terms o+ the etire !e#ice, ot %y partitio
"m%er. $+ter the program starts "p, we will see the +ollowig prompt,
.89
.K 8 Storage >e!ia
Command Nm for hel7P:
Eterig a ?m@ will !isplay the program me",
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos com7atibilit1 flag
d delete a 7artition
l list kno9n 7artition t17es
m 7rint this menu
n add a ne9 7artition
o create a ne9 em7t1 '* 7artition table
7 7rint the 7artition table
B Buit 9ithout saving changes
s create a ne9 em7t1 *un disklabel
t change a 7artitionKs s1stem id
u change dis7la16entr1 units
v verif1 the 7artition table
9 9rite table to disk and exit
x extra functionalit1 Nex7erts onl1P
Command Nm for hel7P:
The +irst thig we wat to !o is e3amie the e3istig partitio layo"t. We !o this %y
eterig ?p@ to prit the partitio ta%le +or the !e#ice,
Command Nm for hel7P: p
'isk 6dev6sdb: #. +H> #.((..". b1tes
# heads> $# sectors6track> #((/ c1linders
3nits M c1linders of $# E "#! M #"/)! b1tes
'evice Hoot *tart &nd Hlocks Ad *1stem
6dev6sdb# ! #((/ #".(/8 b ,0" -4T$!
9 this e3ample, we see a .K >J !e#ice with a sigle partitio E.F that "ses .00K o+ the
a#aila%le .008 cyli!ers o the !e#ice. The partitio is i!eti+ie! as Wi!ows 92 0$T(2
partitio. Some programs will "se this i!eti+ier to limit the ki!s o+ operatio that ca
%e !oe to the !isk, %"t most o+ the time it is ot critical to chage it. Aowe#er, i the
.90
Creatig &ew 0ile Systems
iterest o+ !emostratio, we will chage it to i!icate a *i"3 partitio. To !o this, we
m"st +irst +i! o"t what 9' is "se! to i!eti+y a *i"3 partitio. 9 the listig a%o#e, we
see that the 9' ?%@ is "se! to speci+y the e3itig partitio. To see a list o+ the a#aila%le
partitio types, we re+er %ack to the program me". There we ca see the +ollowig
choice,
l list kno9n 7artition t17es
9+ we eter ?l@ at the prompt, a large list o+ possi%le types is !isplaye!. $mog them we
see ?%@ +or o"r e3istig partitio type a! ?8(@ +or *i"3.
;oig %ack to the me", we see this choice to chage a partitio 9',
t change a 7artitionKs s1stem id
We eter ?t@ at the prompt eter the ew 9',
Command Nm for hel7P: t
*elected 7artition #
Sex code Nt17e = to list codesP: M
Changed s1stem t17e of 7artition # to /$ N=inuxP
This completes all the chages that we ee! to make. )p to this poit, the !e#ice has
%ee "to"che! Eall the chages ha#e %ee store! i memory, ot o the physical !e#iceF,
so we will write the mo!i+ie! partitio ta%le to the !e#ice a! e3it. To !o this, we eter
?w@ at the prompt,
Command Nm for hel7P: w
The 7artition table has been alteredG
Calling ioctlNP to re-read 7artition table<
,4RLAL@: Af 1ou have created or modified an1 '* .<x
7artitions> 7lease see the fdisk manual 7age for additional
information<
*1ncing disks<
[me@linuxbox ~]$
9+ we ha! !eci!e! to lea#e the !e#ice "altere!, we co"l! ha#e etere! ?=@ at the prompt,
.9.
.K 8 Storage >e!ia
which wo"l! ha#e e3ite! the program witho"t writig the chages. We ca sa+ely igore
the omio"s so"!ig warig message.
Creating A ,e% 2ie System With mkfs
With o"r partitio e!itig !oe Elightweight tho"gh it might ha#e %eeF itIs time to create
a ew +ile system o o"r +lash !ri#e. To !o this, we will "se mkfs Eshort +or ?make +ile
system@F, which ca create +ile systems i a #ariety o+ +ormats. To create a e3t( +ile
system o the !e#ice, we "se the ?-t@ optio to speci+y the ?e3t(@ system type, +ollowe!
%y the ame o+ !e#ice cotaiig the partitio we wish to +ormat,
[me@linuxbox ~]$ sudo mkfs -t ext /dev/sdb1
mke!fs #<%(<! N#!-?ul-!(()P
-iles1stem labelM
* t17e: =inux
Hlock siCeM#(!% NlogM(P
-ragment siCeM#(!% NlogM(P
$0(% inodes> #".(/ blocks
)/( blocks N"<((5P reserved for the su7er user
-irst data blockM#
+aximum files1stem blocksM#"00()/%
! block grou7s
/#0! blocks 7er grou7> /#0! fragments 7er grou7
#0"! inodes 7er grou7
*u7erblock backu7s stored on blocks:
/#0$
,riting inode tables: done
Creating journal N#(!% blocksP: done
,riting su7erblocks and files1stem accounting information: done
This files1stem 9ill be automaticall1 checked ever1 $% mounts or
#/( da1s> 9hichever comes first< 3se tune!fs -c or -i to override<
[me@linuxbox ~]$
The program will !isplay a lot o+ i+ormatio whe e3t( is the chose +ile system type.
To re-+ormat the !e#ice to its origial 0$T(2 +ile system, speci+y ?#+at@ as the +ile system
type,
[me@linuxbox ~]$ sudo mkfs -t vfat /dev/sdb1
This process o+ partitioig a! +ormattig ca %e "se! aytime a!!itioal storage
!e#ices are a!!e! to the system. While we worke! with a tiy +lash !ri#e, the same
.92
Creatig &ew 0ile Systems
process ca %e applie! to iteral har! !isks a! other remo#a%le storage !e#ices like
)SJ har! !ri#es.
Testing #nd 1epairing ;ile S,stems
9 o"r earlier !isc"ssio o+ the 6etc6fstab +ile, we saw some mysterio"s !igits at the
e! o+ each lie. Each time the system %oots, it ro"tiely checks the itegrity o+ the +ile
systems %e+ore mo"tig them. This is !oe %y the fsck program Eshort +or ?+ile system
check@F. The last "m%er i each fstab etry speci+ies the or!er the !e#ices are to %e
checke!. 9 o"r e3ample a%o#e, we see that the root +ile system is checke! +irst, +ollowe!
%y the home a! boot +ile systems. 'e#ices with a Dero as the last !igit are ot
ro"tiely checke!.
9 a!!itio to checkig the itegrity o+ +ile systems, fsck ca also repair corr"pt +ile
systems with #aryig !egrees o+ s"ccess, !epe!ig o the amo"t o+ !amage. 7 )i3-
like +ile systems, reco#ere! portios o+ +iles are place! i the lost8found !irectory,
locate! i the root o+ each +ile system.
To check o"r +lash !ri#e Ewhich sho"l! %e "mo"te! +irstF, we co"l! !o the +ollowig,
[me@linuxbox ~]$ sudo fsck /dev/sdb1
fsck #<%(</ N#$-+ar-!((/P
e!fsck #<%(</ N#$-+ar-!((/P
6dev6sdb#: clean> ##6$0(% files> #..#6#".(/ blocks
9 my e3periece, +ile system corr"ptio is ="ite rare "less there is a har!ware pro%lem,
s"ch as a +ailig !isk !ri#e. 7 most systems, +ile system corr"ptio !etecte! at %oot
time will ca"se the system to stop a! !irect yo" to r" fsck %e+ore coti"ig.
+hat The fsck.
9 )i3 c"lt"re, the wor! ?+sck@ is o+te "se! i place o+ a pop"lar wor! with
which it shares three letters. This is especially appropriate, gi#e that yo" will
pro%a%ly %e "tterig the a+oremetioe! wor! i+ yo" +i! yo"rsel+ i a sit"atio
where yo" are +orce! to r" fsck.
;ormatting ;lopp, Dis(s
0or those o+ "s still "sig comp"ters ol! eo"gh to %e e="ippe! with +loppy !iskette
.9(
.K 8 Storage >e!ia
!ri#es, we ca maage those !e#ices, too. 6reparig a %lak +loppy +or "se is a two step
process. 0irst, we per+orm a low-+ormat o the !iskette, the create a +ile system. To
accomplish the +ormattig, we "se the fdformat program speci+yig the ame o+ the
+loppy !e#ice E"s"ally 6dev6fd(F,
[me@linuxbox ~]$ sudo fdformat /dev/fd*
'ouble-sided> /( tracks> #/ sec6track< Total ca7acit1 #%%( kH<
-ormatting <<< done
;erif1ing <<< done
&e3t, we apply a 0$T +ile system to the !iskette with mkfs,
[me@linuxbox ~]$ sudo mkfs -t msdos /dev/fd*
&otice that we "se the ?ms!os@ +ile system type to get the ol!er Ea! smallerF style +ile
allocatio ta%les. $+ter a !iskette is prepare!, it may %e mo"te! like other !e#ices.
>o%ing Data Directl, To8;rom De%ices
While we "s"ally thik o+ !ata o o"r comp"ters as %eig orgaiDe! ito +iles, it is also
possi%le to thik o+ the !ata i ?raw@ +orm. 9+ we look at a !isk !ri#e, +or e3ample, we
see that it cosists o+ a large "m%er o+ ?%locks@ o+ !ata that the operatig system sees as
!irectories a! +iles. Aowe#er, i+ we co"l! treat a !isk !ri#e as simply a large collectio
o+ !ata %locks, we co"l! per+orm "se+"l tasks, s"ch as cloig !e#ices.
The dd program per+orms this task. 9t copies %locks o+ !ata +rom oe place to aother. 9t
"ses a "i="e syta3 E+or historical reasosF a! is "s"ally "se! this way,
dd if$input_file of$output_file (bs$block_size (count$blocks))
*etIs say we ha! two )SJ +lash !ri#es o+ the same siDe a! we wate! to e3actly copy
the +irst !ri#e to the seco!. 9+ we attache! %oth !ri#es to the comp"ter a! they are
assige! to !e#ices 6dev6sdb a! 6dev6sdc respecti#ely, we co"l! copy e#erythig
o the +irst !ri#e to the seco! !ri#e with the +ollowig,
dd if$/dev/sdb of$/dev/sdc
.91
>o#ig 'ata 'irectly To-0rom 'e#ices
$lterately, i+ oly the +irst !e#ice were attache! to the comp"ter, we co"l! copy its
cotets to a or!iary +ile +or later restoratio or copyig,
dd if$/dev/sdb of$flash.drive.img
'arning2 The dd comma! is #ery power+"l. Tho"gh its ame !eri#es +rom ?!ata
!e+iitio,@ it is sometimes calle! ?!estroy !isk@ %eca"se "sers o+te mistype either
the if or of speci+icatios. #l+ays double check your in(ut and out(ut
s(eci"ications be"ore (ressing enter2
Creating CDE1?> Images
Writig a recor!a%le C'-:7> Eeither a C'-: or C'-:WF cosists o+ two stepsH +irst,
costr"ctig a iso image file that is the e3act +ile system image o+ the C'-:7> a!
seco!, writig the image +ile oto the C'-:7> me!ia.
Creating An Image Co5y <= A C)B!<:
9+ we wat to make a iso image o+ a e3istig C'-:7>, we ca "se !! to rea! all the
!ata %locks o++ the C'-:7> a! copy them to a local +ile. Say we ha! a )%"t" C'
a! we wate! to make a iso +ile that we co"l! later "se to make more copies. $+ter
isertig the C' a! !etermiig its !e#ice ame EweIll ass"me 6dev6cdromF, we ca
make the iso +ile like so,
dd if$/dev/cdrom of$ubuntu.iso
This techi="e works +or !ata 'R's as well, %"t will ot work +or a"!io C's, as they !o
ot "se a +ile system +or storage. 0or a"!io C's, look at the cdrdao comma!.
Creating An Image 2rom A Coe/tion <= 2ies
To create a iso image +ile cotaiig the cotets o+ a !irectory, we "se the
genisoimage program. To !o this, we +irst create a !irectory cotaiig all the +iles
we wish to icl"!e i the image a! the e3ec"te the genisoimage comma! to create
the image +ile. 0or e3ample, i+ we ha! create! a !irectory calle! ~6cd-rom-files
a! +ille! it with +iles +or o"r C'-:7>, we co"l! create a image +ile ame! cd-
rom<iso with the +ollowig comma!,
.92
.K 8 Storage >e!ia
genisoimage -o cd-rom.iso -H -Y ~/cd-rom-files
The ?-:@ optio a!!s meta!ata +or the 9ock 9idge e+tensions, which allows the "se o+
log +ileames a! 67S9G style +ile permissios. *ikewise, the ?-J@ optio ea%les the
Doliet e+tensions, which permit log +ileames +or Wi!ows.
# Program /, #n, ?ther 7ame...
9+ yo" look at o-lie t"torials +or creatig a! %"rig optical me!ia like C'-
:7>s a! 'R's, yo" will +re="etly eco"ter two programs calle! mkisofs
a! cdrecord. These programs were part o+ a pop"lar package calle!
?c!rtools@ a"thore! %y Jorg Schillig. 9 the s"mmer o+ 200K, >r. Schillig
ma!e a licese chage to a portio o+ the c!rtools package which, i the opiio
o+ may i the *i"3 comm"ity, create! a licese icompati%ility with the ;&)
;6*. $s a res"lt, a fork o+ the c!rtools pro4ect was starte! that ow icl"!es
replacemet programs +or cdrecord a! mkisofs ame! 9odim a!
genisoimage, respecti#ely.
+riting CDE1?> Images
$+ter we ha#e a image +ile, we ca %"r it oto o"r optical me!ia. >ost o+ the
comma!s we will !isc"ss %elow ca %e applie! to %oth recor!a%le C'-:7> a! 'R'
me!ia.
:ounting An IS< Image )ire/ty
There is a trick that we ca "se to mo"t a iso image while it is still o o"r har! !isk a!
treat it as tho"gh it was alrea!y o optical me!ia. Jy a!!ig the ?-o loop@ optio to
mount Ealog with the re="ire! ?-t iso9KK0@ +ile system typeF, we ca mo"t the image
+ile as tho"gh it were a !e#ice a! attach it to the +ile system tree,
mkdir /mnt/iso.image
mount -t isoDNN* -o loop image.iso /mnt/iso.image
9 the e3ample a%o#e, we create! a mo"t poit ame! 6mnt6isoOimage a! the
mo"te! the image +ile image<iso at that mo"t poit. $+ter the image is mo"te!, it
ca %e treate! 4"st as tho"gh it were a real C'-:7> or 'R'. 9emember to unmount the
.9K
Writig C'-:7> 9mages
image when it is no longer needed#
Banking A !eBWritabe C)B!<:
:ewrita%le C'-:W me!ia ee!s to %e erase! or blanked %e+ore it ca %e re"se!. To !o
this, we ca "se 9odim, speci+yig the !e#ice ame +or the C' writer a! the type o+
%lakig to %e per+orme!. The 9odim program o++ers se#eral types. The most miimal
Ea! +astestF is the ?+ast@ type,
wodim dev$/dev/cdrw blank$fast
Writing An Image
To write a image, we agai "se 9odim, speci+yig the ame o+ the optical me!ia writer
!e#ice a! the ame o+ the image +ile,
wodim dev$/dev/cdrw image.iso
9 a!!itio to the !e#ice ame a! image +ile, 9odim s"pports a #ery large set o+
optios. Two commo oes are ?-#@ +or #er%ose o"tp"t, a! ?-!ao@ which writes the !isk
i disk-at-once mo!e. This mo!e sho"l! %e "se! i+ yo" are preparig a !isk +or
commercial repro!"ctio. The !e+a"lt mo!e +or 9odim is track-at-once, which is "se+"l
+or recor!ig m"sic tracks.
;urther 1eading
We ha#e 4"st to"che! o the may ways that the comma! lie ca %e "se! to maage
storage me!ia. Take a look at the ma pages o+ the comma!s we ha#e co#ere!. Some
o+ them s"pport h"ge "m%ers o+ optios a! operatios. $lso, look +or o-lie t"torials
+or a!!ig har! !ri#es to yo"r *i"3 system Ethere are mayF a! workig with optical
me!ia.
$xtra Credit
9tIs o+te "se+"l to #eri+y the itegrity o+ a iso image that we ha#e !owloa!e!. 9 most
cases, a !istri%"tor o+ a iso image will also s"pply a checksum file. $ checks"m is the
res"lt o+ a e3otic mathematical calc"latio res"ltig i a "m%er that represets the
cotet o+ the target +ile. 9+ the cotets o+ the +ile chage %y e#e oe %it, the res"ltig
checks"m will %e m"ch !i++eret. The most commo metho! o+ checks"m geeratio
.9/
.K 8 Storage >e!ia
"ses the md"sum program. Whe yo" "se md"sum, it pro!"ces a "i="e he3a!ecimal
"m%er,
md"sum image.iso
$%e$"%).(f0bb)fbf/"c0.f.a$f0%ece image<iso
$+ter yo" !owloa! a image, yo" sho"l! r" md"sum agaist it a! compare the res"lts
with the md"sum #al"e s"pplie! %y the p"%lisher.
9 a!!itio to checkig the itegrity o+ a !owloa!e! +ile, we ca "se m!2s"m to #eri+y
ewly writte optical me!ia. To !o this, we +irst calc"late the checks"m o+ the image +ile
a! the calc"late a checks"m +or the me!ia. The trick to #eri+yig the me!ia is to limit
the calc"latio to oly the portio o+ the optical me!ia that cotais the image. We !o
this %y !etermiig the "m%er o+ 2018 %yte %locks the image cotais Eoptical me!ia is
always writte i 2018 %yte %locksF a! rea!ig that may %locks +rom the me!ia. 7
some types o+ me!ia, this is ot re="ire!. $ C'-: writte i !isk-at-oce mo!e ca %e
checke! this way,
md"sum /dev/cdrom
$%e$"%).(f0bb)fbf/"c0.f.a$f0%ece 6dev6cdrom
>ay types o+ me!ia, s"ch as 'R's re="ire a precise calc"latio o+ the "m%er o+
%locks. 9 the e3ample %elow, we check the itegrity o+ the image +ile dvd-
image<iso a! the !isk i the 'R' rea!er 6dev6dvd. Ca yo" +ig"re o"t how this
worksC
md"sum dvd-image.iso# dd if$/dev/dvd bs$2*AM count$344 34stat -c J8sJ
dvd-image.iso6 / 2*AM 66 0 md"sum
.98
./ 8 &etworkig
1. Net5or,ing
Whe it comes to etworkig, there is pro%a%ly othig that caot %e !oe with *i"3.
*i"3 is "se! to %"il! all sorts o+ etworkig systems a! appliaces, icl"!ig +irewalls,
ro"ters, ame ser#ers, &$S E&etwork $ttache! StorageF %o3es a! o a! o.
J"st as the s"%4ect o+ etworkig is #ast, so are the "m%er o+ comma!s that ca %e "se!
to co+ig"re a! cotrol it. We will +oc"s o"r attetio o 4"st a +ew o+ the most
+re="etly "se! oes. The comma!s chose +or e3amiatio icl"!e those "se! to
moitor etworks a! those "se! to tras+er +iles. 9 a!!itio, we are goig to e3plore
the ssh program that is "se! to per+orm remote logis. This chapter will co#er,
7ing - Se! a 9C>6 ECA7O:ET)EST to etwork hosts
traceroute - 6rit the ro"te packets trace to a etwork host
netstat - 6rit etwork coectios, ro"tig ta%les, iter+ace statistics,
mas="era!e coectios, a! m"lticast mem%erships
ft7 - 9teret +ile tras+er program
9get - &o-iteracti#e etwork !owloa!er
ssh - 7peSSA SSA cliet Eremote logi programF
WeIre goig to ass"me a little %ackgro"! i etworkig. 9 this, the 9teret age,
e#eryoe "sig a comp"ter ee!s a %asic "!ersta!ig o+ etworkig cocepts. To
make +"ll "se o+ this chapter we sho"l! %e +amiliar with the +ollowig terms,
96 E9teret 6rotocolF a!!ress
Aost a! !omai ame
):9 E)i+orm :eso"rce 9!eti+ierF
6lease see the ?0"rther :ea!ig@ sectio %elow +or some "se+"l articles regar!ig these
terms.
1ote5 Some o+ the comma!s we will co#er may E!epe!ig o yo"r !istri%"tioF
re="ire the istallatio o+ a!!itioal packages +rom yo"r !istri%"tioIs repositories,
.99
./ 8 &etworkig
a! some may re="ire s"per"ser pri#ileges to e3ec"te.
$xamining #nd >onitoring # 7et4or(
E#e i+ yo"Ire ot the system a!miistrator, itIs o+te help+"l to e3amie the per+ormace
a! operatio o+ a etwork.
7ing
The most %asic etwork comma! is 7ing. The 7ing comma! se!s a special
etwork packet calle! a 9>C6 ECA7O:ET)EST to a speci+ie! host. >ost etwork
!e#ices recei#ig this packet will reply to it, allowig the etwork coectio to %e
#eri+ie!.
1ote5 9t is possi%le to co+ig"re most etwork !e#ices Eicl"!ig *i"3 hostsF to
igore these packets. This is "s"ally !oe +or sec"rity reasos, to partially o%sc"re
a host +rom a potetial attacker. 9t is also commo +or +irewalls to %e co+ig"re! to
%lock 9>C6 tra++ic.
0or e3ample, to see i+ we ca reach linuxcommand<org Eoe o+ o"r +a#orite sites H-F,
we ca "se "se 7ing like this,
[me@linuxbox ~]$ ping linuxcommand.org
7ce starte!, 7ing coti"es to se! packets at a speci+ie! iter#al E!e+a"lt is oe
seco!F "til it is iterr"pte!,
[me@linuxbox ~]$ ping linuxcommand.org
:AL@ linuxcommand<org N..<$"<!"(<!#(P ".N/%P b1tes of data<
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM#
ttlM%$ timeM#() ms
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM!
ttlM%$ timeM#(/ ms
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM$
ttlM%$ timeM#(. ms
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM%
ttlM%$ timeM#(. ms
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM"
ttlM%$ timeM#(" ms
200
E3amiig $! >oitorig $ &etwork
.% b1tes from vhost<sourceforge<net N..<$"<!"(<!#(P: icm7OseBM.
ttlM%$ timeM#() ms
--- linuxcommand<org 7ing statistics ---
. 7ackets transmitted> . received> (5 7acket loss> time .(#(ms
rtt min6avg6max6mdev M #("<.%)6#()<("!6#(/<##/6(</!% ms
$+ter it is iterr"pte! Ei this case a+ter the si3th packetF %y pressig Ctrl-c, 7ing
prits per+ormace statistics. $ properly per+ormig etwork will e3hi%it Dero percet
packet loss. $ s"ccess+"l ?pig@ will i!icate that the elemets o+ the etwork Eits
iter+ace car!s, ca%lig, ro"tig a! gatewaysF are i geerally goo! workig or!er.
traceroute
The traceroute program Esome systems "se the similar trace7ath program
istea!F !isplays a listig o+ all the ?hops@ etwork tra++ic takes to get +rom the local
system to a speci+ie! host. 0or e3ample, to see the ro"te take to reach
slashdot<org, we wo"l! !o this,
[me@linuxbox ~]$ traceroute slashdot.org
The o"tp"t looks like this,
traceroute to slashdot<org N!#.<$%<#/#<%"P> $( ho7s max> %( b1te
7ackets
# i7co7<localdomain N#0!<#./<#<#P #<(.. ms #<$.. ms #<)!( ms
! E E E
$ ge-%-#$-ur(#<rockville<md<bad<comcast<net N./</)<#$(<0P #%<.!!
ms #%<//" ms #"<#.0 ms
% 7o-$(-ur(!<rockville<md<bad<comcast<net N./</)<#!0<#"%P #)<.$%
ms #)<.!. ms #)</00 ms
" 7o-.(-ur($<rockville<md<bad<comcast<net N./</)<#!0<#"/P #"<00!
ms #"<0/$ ms #.<!". ms
. 7o-$(-ar(#<ho9ardcount1<md<bad<comcast<net N./</)<#$.<"P !!</$"
ms #%<!$$ ms #%<%(" ms
) 7o-#(-ar(!<9hitemarsh<md<bad<comcast<net N./</)<#!0<$%P #.<#"%
ms #$<.(( ms #/</.) ms
/ te-(-$-(-#-cr(#<7hiladel7hia<7a<ibone<comcast<net N./</.<0(<))P
!#<0"# ms !#<()$ ms !#<"") ms
0 7os-(-/-(-(-cr(#<ne91ork<n1<ibone<comcast<net N./</.</"<#(P
!!<0#) ms !#<//% ms !!<#!. ms
#( !(%<)(<#%%<# N!(%<)(<#%%<#P %$<##( ms !#<!%/ ms !#<!.% ms
## cr#-7os-(-)-$-#<ne91ork<savvis<net N!(%<)(<#0"<0$P !#</") ms
20.
./ 8 &etworkig
cr!-7os-(-(-$-#<ne91ork<savvis<net N!(%<)(<!(%<!$/P #0<"". ms cr#-
7os-(-)-$-#<ne91ork<savvis<net N!(%<)(<#0"<0$P #0<.$% ms
#! cr!-7os-(-)-$-(<chicago<savvis<net N!(%<)(<#0!<#(0P %#<"/. ms
%!</%$ ms cr!-tengig-(-(-!-(<chicago<savvis<net N!(%<)(<#0.<!%!P
%$<##" ms
#$ hr!-tengigabitethernet-#!-#<elkgrovech$<savvis<net
N!(%<)(<#0"<#!!P %%<!#" ms %#</$$ ms %"<."/ ms
#% csr#-ve!%#<elkgrovech$<savvis<net N!#.<.%<#0%<%!P %.</%( ms
%$<$)! ms %)<(%# ms
#" .%<!)<#.(<#0% N.%<!)<#.(<#0%P ".<#$) ms ""<//) ms "!</#( ms
#. slashdot<org N!#.<$%<#/#<%"P %!<)!) ms %!<(#. ms %#<%$) ms
9 the o"tp"t, we ca see that coectig +rom o"r test system to slashdot<org
re="ires tra#ersig si3tee ro"ters. 0or ro"ters that pro#i!e! i!eti+yig i+ormatio, we
see their host ames, 96 a!!resses a! per+ormace !ata, which icl"!es three samples o+
ro"!-trip time +rom the local system to the ro"ter. 0or ro"ters that !o ot pro#i!e
i!eti+yig i+ormatio E%eca"se o+ ro"ter co+ig"ratio, etwork cogestio, +irewalls,
etc.F, we see asterisks as i the lie +or hop "m%er two.
netstat
The netstat program is "se! to e3amie #ario"s etwork settigs a! statistics.
Thro"gh the "se o+ its may optios, we ca look at a #ariety o+ +eat"res i o"r etwork
set"p. )sig the ?-ie@ optio, we ca e3amie the etwork iter+aces i o"r system,
[me@linuxbox ~]$ netstat -ie
eth( =ink enca7:&thernet S,addr ((:#d:(0:0b:00:.)
inet addr:#0!<#./<#<! Hcast:#0!<#./<#<!"" +ask:!""<!""<!""<(
inet. addr: fe/(::!#d:0ff:fe0b:00.)6.% *co7e:=ink
3: HR4'C4*T R3LLAL@ +3=TAC4*T +T3:#"(( +etric:#
RV 7ackets:!$/%// errors:( dro77ed:( overruns:( frame:(
TV 7ackets:%($!#) errors:( dro77ed:( overruns:( carrier:(
collisions:( txBueuelen:#((
RV b1tes:#"$(0/0!# N#%.<( +HP TV b1tes:!.#($"!%. N!%/<0 +HP
+emor1:fdfc((((-fdfe((((
lo =ink enca7:=ocal =oo7back
inet addr:#!)<(<(<# +ask:!""<(<(<(
inet. addr: ::#6#!/ *co7e:Sost
3: =:H4C2 R3LLAL@ +T3:#.%$. +etric:#
RV 7ackets:!!(/ errors:( dro77ed:( overruns:( frame:(
TV 7ackets:!!(/ errors:( dro77ed:( overruns:( carrier:(
collisions:( txBueuelen:(
RV b1tes:###%0( N#(/</ 2HP TV b1tes:###%0( N#(/</ 2HP
202
E3amiig $! >oitorig $ &etwork
9 the e3ample a%o#e, we see that o"r test system has two etwork iter+aces. The +irst,
calle! eth(, is the Etheret iter+ace a! the seco!, calle! lo, is the loopback
interface, a #irt"al iter+ace that the system "ses to ?talk to itsel+.@
Whe per+ormig ca"sal etwork !iagostics, the importat thigs to look +or are the
presece o+ the wor! ?)6@ at the %egiig o+ the +o"rth lie +or each iter+ace,
i!icatig that the etwork iter+ace is ea%le!, a! the presece o+ a #ali! 96 a!!ress i
the inet addr +iel! o the seco! lie. 0or systems "sig 'AC6 E'yamic Aost
Co+ig"ratio 6rotocolF, a #ali! 96 a!!ress i this +iel! will #eri+y that the 'AC6 is
workig.
)sig the ?-r@ optio will !isplay the kerelIs etwork ro"tig ta%le. This shows how the
etwork is co+ig"re! to se! packets +rom etwork to etwork,
[me@linuxbox ~]$ netstat -r
2ernel A: routing table
'estination @ate9a1 @enmask -lags +** ,indo9 irtt Aface
#0!<#./<#<( E !""<!""<!""<( 3 ( ( ( eth(
default #0!<#./<#<# (<(<(<( 3@ ( ( ( eth(
9 this simple e3ample, we see a typical ro"tig ta%le +or a cliet machie o a *$&
E*ocal $rea &etworkF %ehi! a +irewall-ro"ter. The +irst lie o+ the listig shows the
!estiatio #0!<#./<#<(. 96 a!!resses that e! i Dero re+er to etworks rather tha
i!i#i!"al hosts, so this !estiatio meas ay host o the *$&. The e3t +iel!,
@ate9a1, is the ame or 96 a!!ress o+ the gateway Ero"terF "se! to go +rom the c"rret
host to the !estiatio etwork. $ asterisk i this +iel! i!icates that o gateway is
ee!e!.
The last lie cotais the !estiatio default. This meas ay tra++ic !estie! +or a
etwork that is ot otherwise liste! i the ta%le. 9 o"r e3ample, we see that the gateway
is !e+ie! as a ro"ter with the a!!ress o+ #0!<#./<#<#, which pres"ma%ly kows what
to !o with the !estiatio tra++ic.
The netstat program has may optios a! we ha#e oly looke! at a co"ple. Check
o"t the netstat ma page +or a complete list.
Transporting ;iles ?%er # 7et4or(
What goo! is a etwork "less we kow how to mo#e +iles across itC There are may
programs that mo#e !ata o#er etworks. We will co#er two o+ them ow a! se#eral
more i later sectios.
20(
./ 8 &etworkig
ft7
7e o+ the tr"e ?classic@ programs, ft7 gets it ame +rom the protocol it "ses, the File
Transfer 6rotocol. 0T6 is "se! wi!ely o the 9teret +or +ile !owloa!s. >ost, i+ ot
all, we% %rowsers s"pport it a! yo" o+te see ):9s startig with the protocol ft7:66.
Je+ore there were we% %rowsers, there was the ft7 program. ft7 is "se! to
comm"icate with FT6 servers, machies that cotai +iles that ca %e "ploa!e! a!
!owloa!e! o#er a etwork.
0T6 Ei its origial +ormF is ot sec"re, %eca"se it se!s acco"t ames a! passwor!s i
clearte+t. This meas that they are ot ecrypte! a! ayoe sniffing the etwork ca see
them. Jeca"se o+ this, almost all 0T6 !oe o#er the 9teret is !oe %y anonymous FT6
servers. $ aoymo"s ser#er allows ayoe to logi "sig the logi ame
?aoymo"s@ a! a meaigless passwor!.
9 the e3ample %elow, we show a typical sessio with the ft7 program !owloa!ig a
)%"t" iso image locate! i the 67ub6cdOimages63buntu-/<(% !irectory o+ the
aoymo"s 0T6 ser#er fileserver,
[me@linuxbox ~]$ ftp fileserver
Connected to fileserver<localdomain<
!!( Nvs-T:d !<(<#P
Lame Nfileserver:meP: anon!mous
$$# :lease s7ecif1 the 7ass9ord<
:ass9ord:
!$( =ogin successful<
Remote s1stem t17e is 3LAV<
3sing binar1 mode to transfer files<
ft7D cd pub/cd.images/Ebuntu-M.*A
!"( 'irector1 successfull1 changed<
ft7D ls
!(( :RT command successful< Consider using :4*;<
#"( Sere comes the director1 listing<
-r9-r9-r-- # "(( "(( )$$()0""! 47r !" ($:"$ ubuntu-/<(%-
deskto7-i$/.<iso
!!. 'irector1 send 2<
ft7D lcd 1esktop
=ocal director1 no9 6home6me6'eskto7
ft7D get ubuntu-M.*A-desktop-iMN.iso
local: ubuntu-/<(%-deskto7-i$/.<iso remote: ubuntu-/<(%-deskto7-
i$/.<iso
!(( :RT command successful< Consider using :4*;<
#"( 7ening HAL4RU mode data connection for ubuntu-/<(%-deskto7-
i$/.<iso N)$$()0""! b1tesP<
!!. -ile send 2<
)$$()0""! b1tes received in ./<". secs N#(%%#<" kH6sP
201
Trasportig 0iles 7#er $ &etwork
ft7D b!e
Aere is a e3plaatio o+ the comma!s etere! !"rig this sessio,
Command >eaning
ft7 fileserver
9#oke the ft7 program a! ha#e it
coect to the 0T6 ser#er
fileserver.
anon1mous
*ogi ame. $+ter the logi prompt, a
passwor! prompt will appear. Some
ser#ers will accept a %lak passwor!,
others will re="ire a passwor! i the
+orm o+ a email a!!ress. 9 that case,
try somethig like
?"serNe3ample.com@.
cd 7ub6cdOimages63buntu-/<(%
Chage to the !irectory o the remote
system cotaiig the !esire! +ile.
&ote that o most aoymo"s 0T6
ser#ers, the +iles +or p"%lic
!owloa!ig are +o"! somewhere
"!er the 7ub !irectory.
ls
*ist the !irectory o the remote
system.
lcd 'eskto7
Chage the !irectory o the local
system to ~6'eskto7. 9 the
e3ample, the ft7 program was
i#oke! whe the workig !irectory
was ~. This comma! chages the
workig !irectory to ~6'eskto7.
get ubuntu-/<(%-deskto7-
i$/.<iso
Tell the remote system to tras+er the
+ile ubuntu-/<(%-deskto7-
i$/.<iso to the local system. Sice
the workig !irectory o the local
system was chage! to ~6'eskto7,
the +ile will %e !owloa!e! there.
b1e
*og o++ the remote ser#er a! e! the
ft7 program sessio. The comma!s
202
./ 8 &etworkig
Buit a! exit may also %e "se!.
Typig ?help@ at the ?+tpX@ prompt will !isplay a list o+ the s"pporte! comma!s. )sig
+tp o a ser#er where s"++iciet permissios ha#e %ee grate!, it is possi%le to per+orm
may or!iary +ile maagemet tasks. 9tIs cl"msy, %"t it !oes work.
lft7 A A Better ft7
ft7 is ot the oly comma! lie 0T6 cliet. 9 +act, there are may. 7e o+ %etter Ea!
more pop"larF oes is lft7 %y $le3a!er *"kyao#. 9t works m"ch like the tra!itioal
ft7 program, %"t has may a!!itioal co#eiece +eat"res icl"!ig m"ltiple protocol
s"pport Eicl"!ig ATT6F, a"tomatic re-try o +aile! !owloa!s, %ackgro"! processes,
ta% completio o+ path ames, a! may more.
9get
$other pop"lar comma! lie program +or +ile !owloa!ig is 9get. 9t is "se+"l +or
!owloa!ig cotet +rom %oth we% a! 0T6 sites. Sigle +iles, m"ltiple +iles, a! e#e
etire sites ca %e !owloa!e!. To !owloa! the +irst page o+ linuxcommand<org
we co"l! !o this,
[me@linuxbox ~]$ wget http2//linuxcommand.org/index.php
--##:(!:"#-- htt7:66linuxcommand<org6index<7h7
MD Jindex<7h7K
Resolving linuxcommand<org<<< ..<$"<!"(<!#(
Connecting to linuxcommand<orgQ..<$"<!"(<!#(Q:/(<<< connected<
STT: reBuest sent> a9aiting res7onse<<< !(( 2
=ength: uns7ecified [text6html]
[ XMD ] $>#!( --<--26s
##:(!:"# N#.#<)" +H6sP - Jindex<7h7K saved [$#!(]
The program<s may optios allow 9get to rec"rsi#ely !owloa!, !owloa! +iles i the
%ackgro"! Eallowig yo" to log o++ %"t coti"e !owloa!igF, a! complete the
!owloa! o+ a partially !owloa!e! +ile. These +eat"res are well !oc"mete! i its
%etter-tha-a#erage ma page.
Secure Communication +ith 1emote 3osts
0or may years, )i3-like operatig systems ha#e ha! the a%ility to %e a!miistere!
20K
Sec"re Comm"icatio With :emote Aosts
remotely #ia a etwork. 9 the early !ays, %e+ore the geeral a!optio o+ the 9teret,
there were a co"ple o+ pop"lar programs "se! to log i to remote hosts. These were the
rlogin a! telnet programs. These programs, howe#er, s"++er +rom the same +atal
+law that the ft7 program !oesH they trasmit all their comm"icatios Eicl"!ig logi
ames a! passwor!sF i clearte3t. This makes them wholly iappropriate +or "se i the
9teret age.
ssh
To a!!ress this pro%lem, a ew protocol calle! SSA ESec"re ShellF was !e#elope!. SSA
sol#es the two %asic pro%lems o+ sec"re comm"icatio with a remote host. 0irst, it
a"theticates that the remote host is who it says it is Eth"s pre#etig so-calle! ?ma i
the mi!!le@ attacksF, a! seco!, it ecrypts all o+ the comm"icatios %etwee the local
a! remote hosts.
SSA cosists o+ two parts. $ SSA ser#er r"s o the remote host, listeig +or icomig
coectios o port twety-two, while a SSA cliet is "se! o the local system to
comm"icate with the remote ser#er.
>ost *i"3 !istri%"tios ship a implemetatio o+ SSA calle! 7peSSA +rom the JS'
pro4ect. Some !istri%"tios icl"!e %oth the cliet a! the ser#er packages %y !e+a"lt
E+or e3ample, :e! AatF, while others Es"ch as )%"t"F oly s"pply the cliet. To ea%le a
system to recei#e remote coectios, it m"st ha#e the 77en**S-server package
istalle!, co+ig"re! a! r"ig, a! Ei+ the system is either r"ig or is %ehi! a
+irewallF it m"st allow icomig etwork coectios o TC6 port 22.
Ti(, 9+ yo" !oIt ha#e a remote system to coect to %"t wat to try these
e3amples, make s"re the 7en**S-server package is istalle! o yo"r system
a! "se localhost as the ame o+ the remote host. That way, yo"r machie will
create etwork coectios with itsel+.
The SSA cliet program "se! to coect to remote SSA ser#ers is calle!, appropriately
eo"gh, ssh. To coect to a remote host ame! remote-s1s, we wo"l! "se the ssh
cliet program like so,
[me@linuxbox ~]$ ssh remote-s!s
The authenticit1 of host Kremote-s1s N#0!<#./<#<%PK canKt be
established<
R*4 ke1 finger7rint is
%#:ed:)a:df:!$:#0:bf:$c:a":#):bc:.#:b$:)f:d0:bb<
4re 1ou sure 1ou 9ant to continue connecting N1es6noPF
20/
./ 8 &etworkig
The +irst time the coectio is attempte!, a message is !isplaye! i!icatig that the
a"theticity o+ the remote host caot %e esta%lishe!. This is %eca"se the cliet program
has e#er see this remote host %e+ore. To accept the cre!etials o+ the remote host, eter
?yes@ whe prompte!. 7ce the coectio is esta%lishe!, the "ser is prompte! +or
his-her passwor!,
,arning: :ermanentl1 added Kremote-s1s>#0!<#./<#<%K NR*4P to the list
of kno9n hosts<
me@remote-s1sKs 7ass9ord:
$+ter the passwor! is s"ccess+"lly etere!, we recei#e the shell prompt +rom the remote
system,
=ast login: *at 4ug $( #$:((:%/ !((/
[me@remote-s1s ~]$
The remote shell sessio coti"es "til the "ser eters the exit comma! at the remote
shell prompt, there%y closig the remote coectio. $t this poit, the local shell sessio
res"mes a! the local shell prompt reappears.
9t is also possi%le to coect to remote systems "sig a !i++eret "ser ame. 0or e3ample,
i+ the local "ser ?me@ ha! a acco"t ame! ?%o%@ o a remote system, "ser me co"l! log
i to the acco"t bob o the remote system as +ollows,
[me@linuxbox ~]$ ssh bobVremote-s!s
bob@remote-s1sKs 7ass9ord:
=ast login: *at 4ug $( #$:($:!# !((/
[bob@remote-s1s ~]$
$s state! %e+ore, ssh #eri+ies the a"theticity o+ the remote host. 9+ the remote host !oes
ot s"ccess+"lly a"theticate, the +ollowig message appears,
[me@linuxbox ~]$ ssh remote-s!s
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ ,4RLAL@: R&+T& S*T A'&LTA-AC4TAL S4* CS4L@&'G @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
AT A* :**AH=& TS4T *+&L& A* 'AL@ *+&TSAL@ L4*TUG
*omeone could be eavesdro77ing on 1ou right no9 Nman-in-the-middle
attackPG
208
Sec"re Comm"icatio With :emote Aosts
At is also 7ossible that the R*4 host ke1 has just been changed<
The finger7rint for the R*4 ke1 sent b1 the remote host is
%#:ed:)a:df:!$:#0:bf:$c:a":#):bc:.#:b$:)f:d0:bb<
:lease contact 1our s1stem administrator<
4dd correct host ke1 in 6home6me6<ssh6kno9nOhosts to get rid of this
message<
ffending ke1 in 6home6me6<ssh6kno9nOhosts:#
R*4 host ke1 for remote-s1s has changed and 1ou have reBuested strict
checking<
Sost ke1 verification failed<
This message is ca"se! %y oe o+ two possi%le sit"atios. 0irst, a attacker may %e
attemptig a ?ma-i-the-mi!!le@ attack. This is rare, sice e#ery%o!y kows that ssh
alerts the "ser to this. The more likely c"lprit is that the remote system has %ee chage!
somehowH +or e3ample, its operatig system or SSA ser#er has %ee reistalle!. 9 the
iterests o+ sec"rity a! sa+ety howe#er, the +irst possi%ility sho"l! ot %e !ismisse! o"t
o+ ha!. $lways check with the a!miistrator o+ the remote system whe this message
occ"rs.
$+ter it has %ee !etermie! that the message is !"e to a %eig ca"se, it is sa+e to correct
the pro%lem o the cliet si!e. This is !oe %y "sig a te3t e!itor Evim perhapsF to
remo#e the o%solete key +rom the ~6<ssh6kno9nOhosts +ile. 9 the e3ample
message a%o#e, we see this,
ffending ke1 in 6home6me6<ssh6kno9nOhosts:#
This meas that lie oe o+ the kno9nOhosts +ile cotais the o++e!ig key. 'elete
this lie +rom the +ile, a! the ssh program will %e a%le to accept ew a"theticatio
cre!etials +rom the remote system.
Jesi!es opeig a shell sessio o a remote system, ssh also allows "s to e3ec"te a
sigle comma! o a remote system. 0or e3ample, to e3ec"te the free comma! o a
remote host ame! remote-s1s a! ha#e the res"lts !isplaye! o the local system,
[me@linuxbox ~]$ ssh remote-s!s free
me@t9in%Ks 7ass9ord:
total used free shared buffers cached
+em: ))""$. "()#/% !./$"! ( ##((./ #"%"0.
-68 buffers6cache: !%!"!( "$$(#.
*9a7: #")!/". ( #")!/".
209
./ 8 &etworkig
[me@linuxbox ~]$
9tIs possi%le to "se this techi="e i more iterestig ways, s"ch as this e3ample i which
we per+orm a ls o the remote system a! re!irect the o"tp"t to a +ile o the local
system,
[me@linuxbox ~]$ ssh remote-s!s %ls *% & dirlist.txt
me@t9in%Ks 7ass9ord:
[me@linuxbox ~]$
&otice the "se o+ the sigle ="otes i the comma! a%o#e. This is !oe %eca"se we !o
ot wat the pathame e3pasio per+orme! o the local machieH rather, we wat it to
%e per+orme! o the remote system. *ikewise, i+ we ha! wate! the o"tp"t re!irecte! to
a +ile o the remote machie, we co"l! ha#e place! the re!irectio operator a! the
+ileame withi the sigle ="otes,
[me@linuxbox ~]$ ssh remote-s!s %ls * & dirlist.txt%
Tunneling +ith SS3
6art o+ what happes whe yo" esta%lish a coectio with a remote host #ia SSA
is that a encrypted tunnel is create! %etwee the local a! remote systems.
&ormally, this t"el is "se! to allow comma!s type! at the local system to %e
trasmitte! sa+ely to the remote system, a! +or the res"lts to %e trasmitte!
sa+ely %ack. 9 a!!itio to this %asic +"ctio, the SSA protocol allows most
types o+ etwork tra++ic to %e set thro"gh the ecrypte! t"el, creatig a sort o+
B6> ERirt"al 6ri#ate &etworkF %etwee the local a! remote systems.
6erhaps the most commo "se o+ this +eat"re is to allow G Wi!ow system tra++ic
to %e trasmitte!. 7 a system r"ig a G ser#er Ethat is, a machie !isplayig
a ;)9F, it is possi%le to la"ch a! r" a G cliet program Ea graphical
applicatioF o a remote system a! ha#e its !isplay appear o the local system.
9tIs easy to !o, hereIs a e3ample, letIs say we are sittig at a *i"3 system
calle! linuxbox which is r"ig a G ser#er, a! we wat to r" the xload
program o a remote system ame! remote-s1s a! see the programIs
graphical o"tp"t o o"r local system. We co"l! !o this,
2.0
Sec"re Comm"icatio With :emote Aosts
[me@linuxbox ~]$ ssh -Z remote-s!s
me@remote-s1sKs 7ass9ord:
=ast login: +on *e7 (/ #$:!$:## !((/
[me@remote-s1s ~]$ xload
$+ter the xload comma! is e3ec"te! o the remote system, its wi!ow appears
o the local system. 7 some systems, yo" may ee! to "se the ?-5@ optio
rather tha the ?-G@ optio to !o this.
sc7 And sft7
The 7peSSA package also icl"!es two programs that ca make "se o+ a SSA
ecrypte! t"el to copy +iles across the etwork. The +irst, sc7 Esec"re copyF is "se!
m"ch like the +amiliar c7 program to copy +iles. The most ota%le !i++erece is that the
so"rce or !estiatio pathames may %e prece!e! with the ame o+ a remote host,
+ollowe! %y a colo character. 0or e3ample, i+ we wate! to copy a !oc"met ame!
document<txt +rom o"r home !irectory o the remote system, remote-s1s, to the
c"rret workig !irectory o o"r local system, we co"l! !o this,
[me@linuxbox ~]$ scp remote-s!s2document.txt .
me@remote-s1sKs 7ass9ord:
document<txt #((5 ""/# "<"2H6s ((:((
[me@linuxbox ~]$
$s with ssh, yo" may apply a "ser ame to the %egiig o+ the remote hostIs ame i+
the !esire! remote host acco"t ame !oes ot match that o+ the local system,
[me@linuxbox ~]$ scp bobVremote-s!s2document.txt .
The seco! SSA +ile copyig program is sft7 which, as its ame implies, is a sec"re
replacemet +or the ft7 program. sft7 works m"ch like the origial ft7 program that
we "se! earlierH howe#er, istea! o+ trasmittig e#erythig i clearte3t, it "ses a SSA
ecrypte! t"el. sft7 has a importat a!#atage o#er co#etioal ft7 i that it
!oes ot re="ire a 0T6 ser#er to %e r"ig o the remote host. 9t oly re="ires the SSA
ser#er. This meas that ay remote machie that ca coect with the SSA cliet ca
also %e "se! as a 0T6-like ser#er. Aere is a sample sessio,
2..
./ 8 &etworkig
[me@linuxbox ~]$ sftp remote-s!s
Connecting to remote-s1s<<<
me@remote-s1sKs 7ass9ord:
sft7D ls
ubuntu-/<(%-deskto7-i$/.<iso
sft7D lcd 1esktop
sft7D get ubuntu-M.*A-desktop-iMN.iso
-etching 6home6me6ubuntu-/<(%-deskto7-i$/.<iso to ubuntu-/<(%-
deskto7-i$/.<iso
6home6me6ubuntu-/<(%-deskto7-i$/.<iso #((5 .00+H )<%+H6s (#:$"
sft7D b!e
Ti(5 The S0T6 protocol is s"pporte! %y may o+ the graphical +ile maagers +o"!
i *i"3 !istri%"tios. )sig either &a"til"s E;&7>EF or Mo="eror EM'EF, we
ca eter a ):9 %egiig with sft7:66 ito the locatio %ar a! operate o +iles
store! o a remote system r"ig a SSA ser#er.
#n SS3 Client ;or +indo4s.
*etIs say yo" are sittig at a Wi!ows machie %"t yo" ee! to log i to yo"r
*i"3 ser#er a! get some real work !oe, what !o yo" !oC ;et a SSA cliet
program +or yo"r Wi!ows %o3, o+ co"rseL There are a "m%er o+ these. The
most pop"lar oe is pro%a%ly 6"TT5 %y Simo Tatham a! his team. The
6"TT5 program !isplays a termial wi!ow a! allow a Wi!ows "ser to ope
a SSA Eor teletF sessio o a remote host. The program also pro#i!es aalogs
+or the sc7 a! sft7 programs.
6"TT5 is a#aila%le at http,--www.chiark.greee!.org."k-Ssgtatham-p"tty-
;urther 1eading
0or a %roa! Eal%eit !ate!F look at etwork a!miistratio, the *i"3
'oc"metatio 6ro4ect pro#i!es the (inu+ >etwork )dministratorEs @uide,
http,--tl!p.org-*'6-ag2-i!e3.html
Wikipe!ia cotais may goo! etworkig articles. Aere are some o+ the %asics,
http,--e.wikipe!ia.org-wiki-9teretOprotocolOa!!ress
http,--e.wikipe!ia.org-wiki-AostOame
http,--e.wikipe!ia.org-wiki-)i+ormO:eso"rceO9!eti+ier
2.2
.8 8 Searchig 0or 0iles
1- Searching (or (iles
$s we ha#e wa!ere! aro"! o"r *i"3 system, oe thig has %ecome a%"!atly clear,
a typical *i"3 system has a lot o+ +ilesL This %egs the ="estio, ?how !o we +i!
thigsC@ We alrea!y kow that the *i"3 +ile system is well orgaiDe! accor!ig to
co#etios that ha#e %ee passe! !ow +rom oe geeratio o+ )i3-like system to the
e3t, %"t the sheer "m%er o+ +iles ca preset a !a"tig pro%lem.
9 this chapter, we will look at two tools that are "se! to +i! +iles o a system. These
tools are,
locate 8 0i! +iles %y ame
find 8 Search +or +iles i a !irectory hierarchy
We will also look at a comma! that is o+te "se! with +ile search comma!s to process
the res"ltig list o+ +iles,
xargs 8 J"il! a! e3ec"te comma! lies +rom sta!ar! ip"t
9 a!!itio, we will itro!"ce a co"ple o+ comma!s to assist "s i or e3ploratio,
touch 8 Chage +ile times
stat 8 'isplay +ile or +ile system stat"s
locate ;ind ;iles The $as, +a,
The locate program per+orms a rapi! !ata%ase search o+ pathames a! o"tp"ts e#ery
ame that matches a gi#e s"%strig. Say, +or e3ample, we wat to +i! all the programs
with ames that %egi with ?Dip.@ Sice we are lookig +or programs, we ca ass"me
that the !irectory cotaiig the programs wo"l! e! with ?%i-@. There+ore, we co"l!
try to "se locate this way to +i! o"r +iles,
[me@linuxbox ~]$ locate bin/-ip
locate will search its !ata%ase o+ pathames a! o"tp"t ay that cotai the strig
2.(
.8 8 Searchig 0or 0iles
?%i-Dip@,
6usr6bin6Ci7
6usr6bin6Ci7cloak
6usr6bin6Ci7gre7
6usr6bin6Ci7info
6usr6bin6Ci7note
6usr6bin6Ci7s7lit
9+ the search re="iremet is ot so simple, locate ca %e com%ie! with other tools
s"ch as gre7 to !esig more iterestig searches,
[me@linuxbox ~]$ locate -ip 0 grep bin
6bin6bunCi7!
6bin6bCi7!
6bin6bCi7!recover
6bin6gunCi7
6bin6gCi7
6usr6bin6funCi7
6usr6bin6g7g-Ci7
6usr6bin67reunCi7
6usr6bin67reCi7
6usr6bin67reCi7-bin
6usr6bin6unCi7
6usr6bin6unCi7sfx
6usr6bin6Ci7
6usr6bin6Ci7cloak
6usr6bin6Ci7gre7
6usr6bin6Ci7info
6usr6bin6Ci7note
6usr6bin6Ci7s7lit
The locate program has %ee aro"! +or a "m%er o+ years, a! there are se#eral
!i++eret #ariats i commo "se. The two most commo oes +o"! i mo!er *i"3
!istri%"tios are slocate a! mlocate, tho"gh they are "s"ally accesse! %y a
sym%olic lik ame! locate. The !i++eret #ersios o+ locate ha#e o#erlappig
optios sets. Some #ersios icl"!e reg"lar e3pressio matchig Ewhich weIll co#er i
a "pcomig chapterF a! wil! car! s"pport. Check the ma page +or locate to
!etermie which #ersio o+ locate is istalle!.
2.1
locate 8 0i! 0iles The Easy Way
+here Does The locate Data0ase Come ;rom.
5o" may otice that, o some !istri%"tios, locate +ails to work 4"st a+ter the
system is istalle!, %"t i+ yo" try agai the e3t !ay, it works +ie. What gi#esC
The locate !ata%ase is create! %y aother program ame! u7datedb.
)s"ally, it is r" perio!ically as a cron AobH that is, a task per+orme! at reg"lar
iter#als %y the cro !aemo. >ost systems e="ippe! with locate r"
u7datedb oce a !ay. Sice the !ata%ase is ot "p!ate! coti"o"sly, yo" will
otice that #ery recet +iles !o ot show "p whe "sig locate. To o#ercome
this, itIs possi%le to r" the u7datedb program ma"ally %y %ecomig the
s"per"ser a! r"ig u7datedb at the prompt.
find ;ind ;iles The 3ard +a,
While the locate program ca +i! a +ile %ase! solely o its ame, the find program
searches a gi#e !irectory Ea! its s"%!irectoriesF +or +iles %ase! o a #ariety o+
attri%"tes. WeIre goig to spe! a lot o+ time with find %eca"se it has a lot o+
iterestig +eat"res that we will see agai a! agai whe we start to co#er programmig
cocepts i later chapters.
9 its simplest "se, find is gi#e oe or more ames o+ !irectories to search. 0or
e3ample, to pro!"ce a list o+ o"r home !irectory,
[me@linuxbox ~]$ find ~
7 most acti#e "ser acco"ts, this will pro!"ce a large list. Sice the list is set to
sta!ar! o"tp"t, we ca pipe the list ito other programs. *etIs "se 9c to co"t the
"m%er o+ +iles,
[me@linuxbox ~]$ find ~ 0 wc -l
%)(./
Wow, weI#e %ee %"syL The %ea"ty o+ find is that it ca %e "se! to i!eti+y +iles that
meet speci+ic criteria. 9t !oes this thro"gh the Eslightly strageF applicatio o+ options,
tests, a! actions. WeIll look at the tests +irst.
2.2
.8 8 Searchig 0or 0iles
Tests
*etIs say that we wat a list o+ !irectories +rom o"r search. To !o this, we co"l! a!! the
+ollowig test,
[me@linuxbox ~]$ find ~ -t!pe d 0 wc -l
#.0"
$!!ig the test -t17e d limite! the search to !irectories. Co#ersely, we co"l! ha#e
limite! the search to reg"lar +iles with this test,
[me@linuxbox ~]$ find ~ -t!pe f 0 wc -l
$/)$)
Aere are the commo +ile type tests s"pporte! %y +i!,
Table 14-1: find File Types
;ile T,pe Description
b
Jlock special !e#ice +ile
c
Character special !e#ice +ile
d
'irectory
f
:eg"lar +ile
l
Sym%olic lik
We ca also search %y +ile siDe a! +ileame %y a!!ig some a!!itioal tests, *etIs look
+or all the reg"lar +iles that match the wil! car! patter ?U.J6;@ a! are larger tha oe
mega%yte,
[me@linuxbox ~]$ find ~ -t!pe f -name J*.YBQJ -si-e 51O 0 wc -l
/%(
9 this e3ample, we a!! the -name test +ollowe! %y the wil! car! patter. &otice how
we eclose it i ="otes to pre#et pathame e3pasio %y the shell. &e3t, we a!! the
-siCe test +ollowe! %y the strig ?V.>@. The lea!ig pl"s sig i!icates that we are
lookig +or +iles larger tha the speci+ie! "m%er. $ lea!ig mi"s sig wo"l! chage
2.K
+i! 8 0i! 0iles The Aar! Way
the meaig o+ the strig to %e smaller tha the speci+ie! "m%er. &o sig meas,
?match the #al"e e3actly.@ The trailig letter ?>@ i!icates that the "it o+ meas"remet
is mega%ytes. The +ollowig characters may %e "se! to speci+y "its,
Table 14-': find Si7e 0nits
Character -nit
b
2.2 %yte %locks. This is the !e+a"lt i+ o "it is speci+ie!.
c
Jytes
9
Two %yte wor!s
k
Milo%ytes E)its o+ .021 %ytesF
+
>ega%ytes E)its o+ .0182/K %ytesF
@
;iga%ytes E)its o+ .0/(/1.821 %ytesF
find s"pports a large "m%er o+ !i++eret tests. Jelow is a r"!ow o+ the commo
oes. &ote that i cases where a "meric arg"met is re="ire!, the same ?V@ a! ?-@
otatio !isc"sse! a%o#e ca %e applie!,
Table 14-3: find Tests
Test Description
-cmin n
>atch +iles or !irectories whose cotet or attri%"tes were
last mo!i+ie! e3actly n mi"tes ago. To speci+y less tha n
mi"tes ago, "se -n a! to speci+y more tha n mi"tes
ago, "se +n.
-cne9er file
>atch +iles or !irectories whose cotets or attri%"tes were
last mo!i+ie! more recetly tha those o+ file.
-ctime n
>atch +iles or !irectories whose cotets or attri%"tes were
last mo!i+ie! nU21 ho"rs ago.
-em7t1
>atch empty +iles a! !irectories.
-grou7 name
>atch +ile or !irectories %elogig to group. group may
%e e3presse! as either a gro"p ame or as a "meric gro"p
9'.
-iname pattern
*ike the -name test %"t case isesiti#e.
-inum n
>atch +iles with io!e "m%er n. This is help+"l +or
+i!ig all the har! liks to a partic"lar io!e.
2./
.8 8 Searchig 0or 0iles
-mmin n
>atch +iles or !irectories whose cotets were mo!i+ie! n
mi"tes ago.
-mtime n
>atch +iles or !irectories whose cotets were mo!i+ie!
nU21 ho"rs ago.
-name pattern
>atch +iles a! !irectories with the speci+ie! wil! car!
pattern.
-ne9er file
>atch +iles a! !irectories whose cotets were mo!i+ie!
more recetly tha the speci+ie! file. This is #ery "se+"l
whe writig shell scripts that per+orm +ile %ack"ps. Each
time yo" make a %ack"p, "p!ate a +ile Es"ch as a logF, the
"se find to !etermie which +iles that ha#e chage! sice
the last "p!ate.
-nouser
>atch +ile a! !irectories that !o ot %elog to a #ali! "ser.
This ca %e "se! to +i! +iles %elogig to !elete! acco"ts
or to !etect acti#ity %y attackers.
-nogrou7
>atch +iles a! !irectories that !o ot %elog to a #ali!
gro"p.
-7erm mode
>atch +iles or !irectories that ha#e permissios set to the
speci+ie! mode. mode may %e e3presse! %y either octal or
sym%olic otatio.
-samefile name
Similar to the -inum test. >atches +iles that share the
same io!e "m%er as +ile name.
-siCe n
>atch +iles o+ siDe n.
-t17e c
>atch +iles o+ type c.
-user name
>atch +iles or !irectories %elogig to "ser name. The
"ser may %e e3presse! %y a "ser ame or %y a "meric "ser
9'.
This is ot a complete list. The find ma page has all the !etails.
<5erators
E#e with all the tests that find pro#i!es, we may still ee! a %etter way to !escri%e the
logical relationships %etwee the tests. 0or e3ample, what i+ we ee!e! to !etermie i+
all the +iles a! s"%!irectories i a !irectory ha! sec"re permissiosC We wo"l! look +or
all the +iles with permissios that are ot 0K00 a! the !irectories with permissios that
2.8
+i! 8 0i! 0iles The Aar! Way
are ot 0/00. 0ort"ately, find pro#i!es a way to com%ie tests "sig logical operators
to create more comple3 logical relatioships. To e3press the a+oremetioe! test, we
co"l! !o this,
[me@linuxbox ~]$ find ~ K4 -t!pe f -not -perm *N** K6 -or K4 -t!pe d
-not -perm *C** K6
5ikesL That s"re looks weir!. What is all this st"++C $ct"ally, the operators are ot that
complicate! oce yo" get to kow them. Aere is the list,
Table 14-$: find (ogical &perators
?perator Description
-and
>atch i+ the tests o %oth si!es o+ the operator are tr"e.
>ay %e shortee! to -a. &ote that whe o operator is
preset, -and is implie! %y !e+a"lt.
-or
>atch i+ a test o either si!e o+ the operator is tr"e. >ay %e
shortee! to -o.
-not
>atch i+ the test +ollowig the operator is +alse. >ay %e
a%%re#iate! with a e3clamatio poit EGF.
N P
;ro"ps tests a! operators together to +orm larger
e3pressios. This is "se! to cotrol the prece!ece o+ the
logical e#al"atios. Jy !e+a"lt, find e#al"ates +rom le+t
to right. 9t is o+te ecessary to o#erri!e the !e+a"lt
e#al"atio or!er to o%tai the !esire! res"lt. E#e i+ ot
ee!e!, it is help+"l sometimes to icl"!e the gro"pig
characters to impro#e rea!a%ility o+ the comma!. &ote
that sice the paretheses characters ha#e special meaig
to the shell, they m"st %e ="ote! whe "sig them o the
comma! lie to allow them to %e passe! as arg"mets to
find. )s"ally the %ackslash character is "se! to escape
them.
With this list o+ operators i ha!, letIs !ecostr"ct o"r find comma!. Whe #iewe!
+rom the "ppermost le#el, we see that o"r tests are arrage! as two gro"pigs separate!
%y a -or operator,
N expression 1 P -or N expression 2 P
2.9
.8 8 Searchig 0or 0iles
This makes sese, sice we are searchig +or +iles with a certai set o+ permissios a!
+or !irectories with a !i++eret set. 9+ we are lookig +or %oth +iles a! !irectories, why
!o we "se -or istea! o+ -andC Jeca"se as find scas thro"gh the +iles a!
!irectories, each oe is e#al"ate! to see i+ it matches the speci+ie! tests. We wat to
kow i+ it is either a +ile with %a! permissios or a !irectory with %a! permissios. 9t
caIt %e %oth at the same time. So i+ we e3pa! the gro"pe! e3pressios, we ca see it
this way,
N file 9ith bad 7erms P -or N director1 9ith bad 7erms P
7"r e3t challege is how to test +or ?%a! permissios.@ Aow !o we !o thatC $ct"ally
we !oIt. What we will test +or is ?ot goo! permissios,@ sice we kow what ?goo!
permissios@ are. 9 the case o+ +iles, we !e+ie goo! as 0K00 a! +or !irectories, as
0/... The e3pressio that will test +iles +or ?ot goo!@ permissios is,
-t17e f -and -not -7erms (.((
a! +or !irectories,
-t17e d -and -not -7erms ()((
$s ote! i the ta%le o+ operators a%o#e, the -and operator ca %e sa+ely remo#e!, sice
it is implie! %y !e+a"lt. So i+ we p"t this all %ack together, we get o"r +ial comma!,
find ~ N -t17e f -not -7erms (.(( P -or N -t17e d -not
-7erms ()(( P
Aowe#er, sice the paretheses ha#e special meaig to the shell, we m"st escape them
to pre#et the shell +rom tryig to iterpret them. 6rece!ig each oe with a %ackslash
character !oes the trick.
There is aother +eat"re o+ logical operators that is importat to "!ersta!. *etIs say
that we ha#e two e3pressios separate! %y a logical operator,
expr1 -o7erator expr2
9 all cases, expr1 will always %e per+orme!H howe#er the operator will !etermie i+
expr2 is per+orme!. AereIs how it works,
Table 14-.: find )>-;&9 (ogic
1esults o" expr1 ?perator expr2 is...
Tr"e
-and
$lways per+orme!
0alse
-and
&e#er per+orme!
Tr"e
-or
&e#er per+orme!
0alse
-or
$lways per+orme!
220
+i! 8 0i! 0iles The Aar! Way
Why !oes this happeC 9tIs !oe to impro#e per+ormace. Take -and, +or e3ample. We
kow that the e3pressio expr1 -and expr2 caot %e tr"e i+ the res"lt o+ expr1
is +alse, so there is o poit i per+ormig expr2. *ikewise, i+ we ha#e the e3pressio
expr1 -or expr2 a! the res"lt o+ expr1 is tr"e, there is o poit i per+ormig
expr2" as we alrea!y kow that the e3pressio expr1 -or expr2 is tr"e.
7M, so it helps it go +aster. Why is this importatC 9tIs importat %eca"se we ca rely o
this %eha#ior to cotrol how actios are per+orme!, as we shall soo see..
'rede=ined A/tions
*etIs get some work !oeL Aa#ig a list o+ res"lts +rom o"r find comma! is "se+"l,
%"t what we really wat to !o is act o the items o the list. 0ort"ately, find allows
actios to %e per+orme! %ase! o the search res"lts. There are a set o+ pre!e+ie! actios
a! se#eral ways to apply "ser-!e+ie! actios. 0irst letIs look at a +ew o+ the pre!e+ie!
actios,
Table 14-2: 6redefined find )ctions
#ction Description
-delete
'elete the c"rretly matchig +ile.
-ls
6er+orm the e="i#alet o+ ls -dils o the matchig +ile.
7"tp"t is set to sta!ar! o"tp"t.
-7rint
7"tp"t the +"ll pathame o+ the matchig +ile to sta!ar!
o"tp"t. This is the !e+a"lt actio i+ o other actio is
speci+ie!.
-Buit
T"it oce a match has %ee ma!e.
$s with the tests, there are may more actios. See the find ma page +or +"ll !etails.
9 o"r #ery +irst e3ample, we !i! this,
find ~
which pro!"ce! a list o+ e#ery +ile a! s"%!irectory cotaie! withi o"r home !irectory.
9t pro!"ce! a list %eca"se the -7rint actio is implie! i+ o other actio is speci+ie!.
Th"s o"r comma! co"l! also %e e3presse! as,
22.
.8 8 Searchig 0or 0iles
find ~ -print
We ca "se find to !elete +iles that meet certai criteria. 0or e3ample, to !elete +iles
that ha#e the +ile e3tesio ?.J$M@ Ewhich is o+te "se! to !esigate %ack"p +ilesF, we
co"l! "se this comma!,
find ~ -t!pe f -name %*.<:X% -delete
9 this e3ample, e#ery +ile i the "serIs home !irectory Ea! its s"%!irectoriesF is searche!
+or +ileames e!ig i <H42. Whe they are +o"!, they are !elete!.
'arning5 9t sho"l! go witho"t sayig that yo" sho"l! use extreme caution whe
"sig the -delete actio. $lways test the comma! +irst %y s"%stit"tig the
-7rint actio +or -delete to co+irm the search res"lts.
Je+ore we go o, letIs take aother look at how the logical operators a++ect actios.
Cosi!er the +ollowig comma!,
find ~ -t!pe f -name %*.<:X% -print
$s we ha#e see, this comma! will look +or e#ery reg"lar +ile E-t17e fF whose ame
e!s with <H42 E-name %*<H42KF a! will o"tp"t the relati#e pathame o+ each
matchig +ile to sta!ar! o"tp"t E-7rintF. Aowe#er, the reaso the comma! per+orms
the way it !oes is !etermie! %y the logical relatioships %etwee each o+ the tests a!
actios. :emem%er, there is, %y !e+a"lt, a implie! -and relatioship %etwee each test
a! actio. We co"l! also e3press the comma! this way to make the logical
relatioships easier to see,
find ~ -t!pe f -and -name %*.<:X% -and -print
With o"r comma! +"lly e3presse!, letIs look at how the logical operators a++ect its
e3ec"tio,
Test8#ction Is Per"ormed ?nl, I"...
-7rint
-t17e f a! -name KE<H42K are tr"e
222
+i! 8 0i! 0iles The Aar! Way
-name aE<H42b
-t17e f is tr"e
-t17e f
9s always per+orme!, sice it is the +irst test-actio i a
-and relatioship.
Sice the logical relatioship %etwee the tests a! actios !etermies which o+ them are
per+orme!, we ca see that the or!er o+ the tests a! actios is importat. 0or istace, i+
we were to reor!er the tests a! actios so that the -7rint actio was the +irst oe, the
comma! wo"l! %eha#e m"ch !i++eretly,
find ~ -print -and -t!pe f -and -name %*.<:X%
This #ersio o+ the comma! will prit each +ile Ethe -7rint actio always e#al"ates to
tr"eF a! the test +or +ile type a! the speci+ie! +ile e3tesio.
User )e=ined A/tions
9 a!!itio to the pre!e+ie! actios, we ca also i#oke ar%itrary comma!s. The
tra!itioal way o+ !oig this is with the -exec actio. This actio works like this,
-exec command [\ T
where command is the ame o+ a comma!, cd is a sym%olic represetatio o+ the c"rret
pathame a! the semicolo is a re="ire! !elimiter i!icatig the e! o+ the comma!.
AereIs a e3ample o+ "sig -exec to act like the -delete actio !isc"sse! earlier,
-exec rm K[\K KTK
$gai, sice the %race a! semicolo characters ha#e special meaig to the shell, they
m"st %e ="ote! or escape!.
9tIs also possi%le to e3ec"te a "ser !e+ie! actio iteracti#ely. Jy "sig the -ok actio
i place o+ -exec, the "ser is prompte! %e+ore e3ec"tio o+ each speci+ie! comma!,
find ~ -t!pe f -name %foo*% -ok ls -l %9>% %#%
X ls <<< 6home6me6bin6foo D F !
-r9xr-xr-x # me me !!% !(()-#(-!0 #/:%% 6home6me6bin6foo
X ls <<< 6home6me6foo<txt D F !
-r9-r--r-- # me me ( !((/-(0-#0 #!:"$ 6home6me6foo<txt
22(
.8 8 Searchig 0or 0iles
9 this e3ample, we search +or +iles with ames startig with the strig ?+oo@ a! e3ec"te
the comma! ls -l each time oe is +o"!. )sig the -ok actio prompts the "ser
%e+ore the ls comma! is e3ec"te!.
Im5ro;ing 6==i/ien/y
Whe the -exec actio is "se!, it la"ches a ew istace o+ the speci+ie! comma!
each time a matchig +ile is +o"!. There are times whe we might pre+er to com%ie all
o+ the search res"lts a! la"ch a sigle istace o+ the comma!. 0or e3ample, rather
tha e3ec"tig the comma!s like this,
ls -l file1
ls -l file2
we may pre+er to e3ec"te it this way,
ls -l file1 file2
th"s ca"sig the comma! to %e e3ec"te! oly oe time rather tha m"ltiple times.
There are two ways we ca !o this. The tra!itioal way, "sig the e3teral comma!
xargs a! the alterate way, "sig a ew +eat"re i find itsel+. WeIll talk a%o"t the
alterate way +irst.
Jy chagig the trailig semicolo character to a pl"s sig, we acti#ate the a%ility o+
find to com%ie the res"lts o+ the search ito a arg"met list +or a sigle e3ec"tio o+
the !esire! comma!. ;oig %ack to o"r e3ample, this,
find ~ -t!pe f -name %foo*% -exec ls -l %9>% %#%
-r9xr-xr-x # me me !!% !(()-#(-!0 #/:%% 6home6me6bin6foo
-r9-r--r-- # me me ( !((/-(0-#0 #!:"$ 6home6me6foo<txt
will e3ec"te ls each time a matchig +ile is +o"!. Jy chagig the comma! to,
find ~ -t!pe f -name %foo*% -exec ls -l %9>% 5
-r9xr-xr-x # me me !!% !(()-#(-!0 #/:%% 6home6me6bin6foo
-r9-r--r-- # me me ( !((/-(0-#0 #!:"$ 6home6me6foo<txt
we get the same res"lts, %"t the system oly has to e3ec"te the ls comma! oce.
xargs
The xargs comma! per+orms a iterestig +"ctio. 9t accepts ip"t +rom sta!ar!
221
+i! 8 0i! 0iles The Aar! Way
ip"t a! co#erts it ito a arg"met list +or a speci+ie! comma!. With o"r e3ample,
we wo"l! "se it like this,
find ~ -t!pe f -name %foo*% -print 0 xargs ls -l
-r9xr-xr-x # me me !!% !(()-#(-!0 #/:%% 6home6me6bin6foo
-r9-r--r-- # me me ( !((/-(0-#0 #!:"$ 6home6me6foo<txt
Aere we see the o"tp"t o+ the find comma! pipe! ito xargs which, i t"r,
costr"cts a arg"met list +or ls comma! a! the e3ec"tes it.
1ote5 While the "m%er o+ arg"mets that ca %e place! ito a comma! lie is
="ite large, itIs ot "limite!. 9t is possi%le to create comma!s that are too log
+or the shell to accept. Whe a comma! lie e3cee!s the ma3im"m legth
s"pporte! %y the system, xargs e3ec"tes the speci+ie! comma! with the
ma3im"m "m%er o+ arg"mets possi%le a! the repeats this process "til
sta!ar! ip"t is e3ha"ste!. To see the ma3im"m siDe o+ the comma! lie,
e3ec"te xargs with the --sho9-limits optio.
Dealing +ith ;unn, ;ilenames
)i3-like systems allow em%e!!e! spaces Ea! e#e ewliesLF i +ileames.
This ca"ses pro%lems +or programs like xargs that costr"ct arg"met lists +or
other programs. $ em%e!!e! space will %e treate! as a !elimiter a! the
res"ltig comma! will iterpret each space-separate! wor! as a separate
arg"met. To o#ercome this, find a! xarg allow the optioal "se o+ a null
character as arg"met separator. $ "ll character is !e+ie! i $SC99 as the
character represete! %y the "m%er Dero Eas oppose! to, +or e3ample, the space
character, which is !e+ie! i $SC99 as the character represete! %y the "m%er
(2F. The find comma! pro#i!es the actio -7rint(, which pro!"ces "ll
separate! o"tp"t, a! the xargs comma! has the --null optio, which
accepts "ll separate! ip"t. AereIs a e3ample,
find ~ -iname KE<j7gK -7rint( Q xargs --null ls -l
)sig this techi="e, we ca es"re that all +iles, e#e those cotaiig em%e!!e!
spaces i their ames, are ha!le! correctly.
222
.8 8 Searchig 0or 0iles
A !eturn To The 'ayground
9tIs time to p"t find to some EalmostF practical "se. WeIll create a playgro"! a! try
o"t some o+ what we ha#e leare!.
0irst, letIs create a playgro"! with lots o+ s"%!irectories a! +iles,
[me@linuxbox ~]$ mkdir -p pla!ground/dir-9**91..D>;*91*..DD>;1**>
[me@linuxbox ~]$ touch pla!ground/dir-9**91..D>;*91*..DD>;1**>/file-
9:..@>
>ar#el i the power o+ the comma! lieL With these two lies, we create! a playgro"!
!irectory cotaiig oe h"!re! s"%!irectories each cotaiig twety-si3 empty +iles.
Try that with the ;)9L
The metho! we employe! to accomplish this magic i#ol#e! a +amiliar comma!
EmkdirF, a e3otic shell e3pasio E%racesF a! a ew comma!, touch. Jy
com%iig mkdir with the -7 optio Ewhich ca"ses mkdir to create the paret
!irectories o+ the speci+ie! pathsF with %race e3pasio, we were a%le to create oe
h"!re! !irectories.
The touch comma! is "s"ally "se! to set or "p!ate the access, chage, a! mo!i+y
times o+ +iles. Aowe#er, i+ a +ileame arg"met is that o+ a oe3istet +ile, a empty +ile
is create!.
9 o"r playgro"!, we create! oe h"!re! istaces o+ a +ile ame! file-4. *etIs +i!
them,
[me@linuxbox ~]$ find pla!ground -t!pe f -name %file-:%
&ote that "like ls, find !oes ot pro!"ce res"lts i sorte! or!er. 9ts or!er is
!etermie! %y the layo"t o+ the storage !e#ice. To co+irm that we act"ally ha#e oe
h"!re! istaces o+ the +ile we ca co+irm it this way,
[me@linuxbox ~]$ find pla!ground -t!pe f -name %file-:% 0 wc -l
#((
&e3t, letIs look at +i!ig +iles %ase! o their mo!i+icatio times. This will %e help+"l
whe creatig %ack"ps or orgaiDig +iles i chroological or!er. To !o this, we will +irst
create a re+erece +ile agaist which we will compare mo!i+icatio time,
22K
+i! 8 0i! 0iles The Aar! Way
[me@linuxbox ~]$ touch pla!ground/timestamp
This creates a empty +ile ame! timestam7 a! sets its mo!i+icatio time to the
c"rret time. We ca #eri+y this %y "sig aother ha!y comma!, stat, which is a
ki! o+ so"pe!-"p #ersio o+ ls. The stat comma! re#eals all that the system
"!ersta!s a%o"t a +ile a! its attri%"tes,
[me@linuxbox ~]$ stat pla!ground/timestamp
-ile: J7la1ground6timestam7K
*iCe: ( Hlocks: ( A Hlock: %(0. regular em7t1 file
'evice: /($h6!("#d Anode: #%!."(.# =inks: #
4ccess: N(.%%6-r9-r--r--P 3id: N #((#6 meP @id: N #((#6 meP
4ccess: !((/-#(-(/ #":#":$0<((((((((( -(%((
+odif1: !((/-#(-(/ #":#":$0<((((((((( -(%((
Change: !((/-#(-(/ #":#":$0<((((((((( -(%((
9+ we touch the +ile agai a! the e3amie it with stat, we will see that the +ileIs
times ha#e %ee "p!ate!.,
[me@linuxbox ~]$ touch pla!ground/timestamp
[me@linuxbox ~]$ stat pla!ground/timestamp
-ile: J7la1ground6timestam7K
*iCe: ( Hlocks: ( A Hlock: %(0. regular em7t1 file
'evice: /($h6!("#d Anode: #%!."(.# =inks: #
4ccess: N(.%%6-r9-r--r--P 3id: N #((#6 meP @id: N #((#6 meP
4ccess: !((/-#(-(/ #":!$:$$<((((((((( -(%((
+odif1: !((/-#(-(/ #":!$:$$<((((((((( -(%((
Change: !((/-#(-(/ #":!$:$$<((((((((( -(%((
&e3t, letIs "se find to "p!ate some o+ o"r playgro"! +iles,
[me@linuxbox ~]$ find pla!ground -t!pe f -name %file-<% -exec touch
%9>% %#%
This "p!ates all +iles i the playgro"! ame! file-H. &e3t weIll "se find to i!eti+y
the "p!ate! +iles %y comparig all the +iles to the re+erece +ile timestam7,
[me@linuxbox ~]$ find pla!ground -t!pe f -newer pla!ground/timestamp
22/
.8 8 Searchig 0or 0iles
The res"lts cotai all oe h"!re! istaces o+ file-H. Sice we per+orme! a touch
o all the +iles i the playgro"! ame! file-H a+ter we "p!ate! timestam7, they
are ow ?ewer@ tha timestam7 a! th"s ca %e i!eti+ie! with the -ne9er test.
0ially, letIs go %ack to the %a! permissios test we per+orme! earlier a! apply it to
7la1ground,
[me@linuxbox ~]$ find pla!ground K4 -t!pe f -not -perm *N** K6 -or K4
-t!pe d -not -perm *C** K6
This comma! lists all oe h"!re! !irectories a! twety-si3 h"!re! +iles i
7la1ground Eas well as timestam7 a! 7la1ground itsel+, +or a total o+ 2/02F
%eca"se oe o+ them meets o"r !e+iitio o+ ?goo! permissios.@ With o"r kowle!ge
o+ operators a! actios, we ca a!! actios to this comma! to apply ew permissios
to the +iles a! !irectories i o"r playgro"!,
[me@linuxbox ~]$ find pla!ground K4 -t!pe f -not -perm *N** -exec
chmod *N** %9>% %#% K6 -or K4 -t!pe d -not -perm *C11 -exec chmod
*C** %9>% %#% K6
7 a !ay-to-!ay %asis, we might +i! it easier to iss"e two comma!s, oe +or the
!irectories a! oe +or the +iles, rather tha this oe large compo"! comma!, %"t itIs
ice to kow that we ca !o it this way. The importat poit here is to "!ersta! how
the operators a! actios ca %e "se! together to per+orm "se+"l tasks.
<5tions
0ially, we ha#e the optios. The optios are "se! to cotrol the scope o+ a find search.
They may %e icl"!e! with other tests a! actios whe costr"ctig find e3pressios.
Aere is a list o+ the most commoly "se! oes,
Table 14-3: find &ptions
?ption Description
-de7th
'irect find to process a !irectoryIs +iles %e+ore the
!irectory itsel+. This optio is a"tomatically applie!
whe the -delete actio is speci+ie!.
-maxde7th levels
Set the ma3im"m "m%er o+ le#els that find will
!esce! ito a !irectory tree whe per+ormig tests a!
228
+i! 8 0i! 0iles The Aar! Way
actios.
-minde7th levels
Set the miim"m "m%er o+ le#els that find will
!esce! ito a !irectory tree %e+ore applyig tests a!
actios.
-mount
'irect find ot to tra#erse !irectories that are mo"te!
o other +ile systems.
-noleaf
'irect find ot to optimiDe its search %ase! o the
ass"mptio that it is searchig a )i3-like +ile system.
This is ee!e! whe scaig '7S-Wi!ows +ile
systems a! C'-:7>s.
;urther 1eading
The locate, u7datedb, find, a! xargs programs are all part the ;&)
6ro4ectIs findutils package. The ;&) 6ro4ect pro#i!es a we%site with e3tesi#e
o-lie !oc"metatio, which is ="ite goo! a! sho"l! %e rea! i+ yo" are "sig
these programs i high sec"rity e#iromets,
http,--www.g".org-so+tware-+i!"tils-
229
.9 8 $rchi#ig $! Jack"p
10 )rchiving )nd 6ac,u#
7e o+ the primary tasks o+ a comp"ter systemIs a!miistrator is keepig the systemIs
!ata sec"re. 7e way this is !oe is %y per+ormig timely %ack"ps o+ the systemIs +iles.
E#e i+ yo"Ire ot system a!miistrators, it is o+te "se+"l to make copies o+ thigs a!
to mo#e large collectios o+ +iles +rom place to place a! +rom !e#ice to !e#ice.
9 this chapter, we will look at se#eral commo programs that are "se! to maage
collectios o+ +iles. There are the +ile compressio programs,
gCi7 8 Compress or e3pa! +iles
bCi7! 8 $ %lock sortig +ile compressor
The archi#ig programs,
tar 8 Tape archi#ig "tility
Ci7 8 6ackage a! compress +iles
$! the +ile sychroiDatio program,
rs1nc 8 :emote +ile a! !irectory sychroiDatio
Compressing ;iles
Thro"gho"t the history o+ comp"tig, there has %ee a str"ggle to get the most !ata ito
the smallest a#aila%le space, whether that space %e memory, storage !e#ices or etwork
%a!wi!th. >ay o+ the !ata ser#ices that we take +or grate! to!ay, s"ch as porta%le
m"sic players, high !e+iitio tele#isio, or %roa!%a! 9teret, owe their e3istece to
e++ecti#e data compression techi="es.
'ata compressio is the process o+ remo#ig redundancy +rom !ata. *etIs cosi!er a
imagiary e3ample. Say we ha! a etirely %lack pict"re +ile with the !imesios o+ oe
h"!re! pi3els %y oe h"!re! pi3els. 9 terms o+ !ata storage Eass"mig twety-+o"r
%its, or three %ytes per pi3elF, the image will occ"py thirty tho"sa! %ytes o+ storage,
.00 U .00 U ( W (0,000
$ image that is all oe color cotais etirely re!"!at !ata. 9+ we were cle#er, we
co"l! eco!e the !ata i s"ch a way that we simply !escri%e the +act that we ha#e a %lock
2(0
Compressig 0iles
o+ thirty tho"sa! %lack pi3els. So, istea! o+ storig a %lock o+ !ata cotaiig thirty
tho"sa! Deros E%lack is "s"ally represete! i image +iles as DeroF, we co"l! compress
the !ata ito the "m%er (0,000, +ollowe! %y a Dero to represet o"r !ata. S"ch a !ata
compressio scheme is calle! run-length encoding a! is oe o+ the most r"!imetary
compressio techi="es. To!ayIs techi="es are m"ch more a!#ace! a! comple3 %"t
the %asic goal remais the sameBget ri! o+ re!"!at !ata.
%ompression algorithms Ethe mathematical techi="es "se! to carry o"t the compressioF
+all ito two geeral categories, lossless a! lossy. *ossless compressio preser#es all
the !ata cotaie! i the origial. This meas that whe a +ile is restore! +rom a
compresse! #ersio, the restore! +ile is e3actly the same as the origial, "compresse!
#ersio. *ossy compressio, o the other ha!, remo#es !ata as the compressio is
per+orme!, to allow more compressio to %e applie!. Whe a lossy +ile is restore!, it
!oes ot match the origial #ersioH rather, it is a close appro3imatio. E3amples o+
lossy compressio are J6E; E+or imagesF a! >6( E+or m"sic.F 9 o"r !isc"ssio, we
will look e3cl"si#ely at lossless compressio, sice most !ata o comp"ters caot
tolerate ay !ata loss.
gHi5
The gCi7 program is "se! to compress oe or more +iles. Whe e3ec"te!, it replaces the
origial +ile with a compresse! #ersio o+ the origial. The correspo!ig gunCi7
program is "se! to restore compresse! +iles to their origial, "compresse! +orm. Aere is
a e3ample,
[me@linuxbox ~]$ ls -l /etc & foo.txt
[me@linuxbox ~]$ ls -l foo.*
-r9-r--r-- # me me #")$/ !((/-#(-#% ():#" foo<txt
[me@linuxbox ~]$ g-ip foo.txt
[me@linuxbox ~]$ ls -l foo.*
-r9-r--r-- # me me $!$( !((/-#(-#% ():#" foo<txt<gC
[me@linuxbox ~]$ gun-ip foo.txt
[me@linuxbox ~]$ ls -l foo.*
-r9-r--r-- # me me #")$/ !((/-#(-#% ():#" foo<txt
9 this e3ample, we create a te3t +ile ame! foo<txt +rom a !irectory listig. &e3t, we
r" gCi7, which replaces the origial +ile with a compresse! #ersio ame!
foo<txt<gC. 9 the !irectory listig o+ +oo.U, we see that the origial +ile has %ee
replace! with the compresse! #ersio, a! that the compresse! #ersio a%o"t oe-+i+th
the siDe o+ the origial. We ca also see that the compresse! +ile has the same
permissios a! time stamp as the origial.
&e3t, we r" the gunCi7 program to "compress the +ile. $+terwar!, we ca see that
2(.
.9 8 $rchi#ig $! Jack"p
the compresse! #ersio o+ the +ile has %ee replace! with the origial, agai with the
permissios a! time stamp preser#e!.
gCi7 has may optios. Aere are a +ew,
Table 15-1: g7ip &ptions
?ption Description
-c
Write o"tp"t to sta!ar! o"tp"t a! keep origial +iles. >ay also %e
speci+ie! with --stdout a! --to-stdout.
-d
'ecompress. This ca"ses gCi7 to act like gunCi7. >ay also %e
speci+ie! with --decom7ress or --uncom7ress.
-f
0orce compressio e#e i+ compresse! #ersio o+ the origial +ile
alrea!y e3ists. >ay also %e speci+ie! with --force.
-h
'isplay "sage i+ormatio. >ay also %e speci+ie! with --hel7.
-l
*ist compressio statistics +or each +ile compresse!. >ay also %e
speci+ie! with --list.
-r
9+ oe or more arg"mets o the comma! lie are !irectories,
rec"rsi#ely compress +iles cotaie! withi them. >ay also %e
speci+ie! with --recursive.
-t
Test the itegrity o+ a compresse! +ile. >ay also %e speci+ie! with
--test.
-v
'isplay #er%ose messages while compressig. >ay also %e speci+ie!
with --verbose.
-number
Set amo"t o+ compressio. number is a iteger i the rage o+ .
E+astest, least compressioF to 9 Eslowest, most compressioF. The
#al"es . a! 9 may also %e e3presse! as --fast a! --best,
respecti#ely. The !e+a"lt #al"e is K.
;oig %ack to o"r earlier e3ample,
[me@linuxbox ~]$ g-ip foo.txt
[me@linuxbox ~]$ g-ip -tv foo.txt.g-
foo<txt<gC: 2
[me@linuxbox ~]$ g-ip -d foo.txt.g-
Aere, we replace! the +ile foo<txt with a compresse! #ersio, ame! foo<txt<gC.
2(2
Compressig 0iles
&e3t, we teste! the itegrity o+ the compresse! #ersio, "sig the -t a! -v optios.
0ially, we !ecompresse! the +ile %ack to its origial +orm.
gCi7 ca also %e "se! i iterestig ways #ia sta!ar! ip"t a! o"tp"t,
[me@linuxbox ~]$ ls -l /etc 0 g-ip & foo.txt.g-
This comma! creates a compresse! #ersio o+ a !irectory listig.
The gunCi7 program, which "compresses gDip +iles, ass"mes that +ileames e! i the
e3tesio <gC, so itIs ot ecessary to speci+y it, as log as the speci+ie! ame is ot i
co+lict with a e3istig "compresse! +ile,
[me@linuxbox ~]$ gun-ip foo.txt
9+ o"r goal were oly to #iew the cotets o+ a compresse! te3t +ile, we ca !o this,
[me@linuxbox ~]$ gun-ip -c foo.txt 0 less
$lterately, there is a program s"pplie! with gCi7, calle! Ccat, that is e="i#alet to
gunCi7 with the -c optio. 9t ca %e "se! like the cat comma! o gCi7 compresse!
+iles,
[me@linuxbox ~]$ -cat foo.txt.g- 0 less
Ti(5 There is a Cless program, too. 9t per+orms the same +"ctio as the pipelie
a%o#e.
bHi52
The bCi7! program, %y J"lia Sewar!, is similar to gDip, %"t "ses a !i++eret
compressio algorithm that achie#es higher le#els o+ compressio at the cost o+
compressio spee!. 9 most regar!s, it works i the same +ashio as gCi7. $ +ile
compresse! with bCi7! is !eote! with the e3tesio <bC!,
2((
.9 8 $rchi#ig $! Jack"p
[me@linuxbox ~]$ ls -l /etc & foo.txt
[me@linuxbox ~]$ ls -l foo.txt
-r9-r--r-- # me me #")$/ !((/-#(-#) #$:"# foo<txt
[me@linuxbox ~]$ b-ip2 foo.txt
[me@linuxbox ~]$ ls -l foo.txt.b-2
-r9-r--r-- # me me !)0! !((/-#(-#) #$:"# foo<txt<bC!
[me@linuxbox ~]$ bun-ip2 foo.txt.b-2
$s we ca see, bCi7! ca %e "se! the same way as gCi7. $ll the optios Ee3cept +or -
rF that we !isc"sse! +or gCi7 are also s"pporte! i bCi7!. &ote, howe#er, that the
compressio le#el optio E-numberF has a somewhat !i++eret meaig to bCi7!.
bCi7! comes with bunCi7! a! bCcat +or !ecompressig +iles.
bCi7! also comes with the bCi7!recover program, which will try to reco#er
!amage! <bC! +iles.
DonFt /e Compressi%e Compulsi%e
9 occasioally see people attemptig to compress a +ile, which has %ee alrea!y
compresse! with a e++ecti#e compressio algorithm, %y !oig somethig like
this,
$ g-ip picture.jpg
'oIt !o it. 5o"Ire pro%a%ly 4"st wastig time a! spaceL 9+ yo" apply
compressio to a +ile that is alrea!y compresse!, yo" will act"ally e! "p a larger
+ile. This is %eca"se all compressio techi="es i#ol#e some o#erhea! that is
a!!e! to the +ile to !escri%e the compressio. 9+ yo" try to compress a +ile that
alrea!y cotais o re!"!at i+ormatio, the compressio will ot res"lt i ay
sa#igs to o++set the a!!itioal o#erhea!.
#rchi%ing ;iles
$ commo +ile maagemet task "se! i co4"ctio with compressio is archiving.
$rchi#ig is the process o+ gatherig "p may +iles a! %"!lig them together ito a
sigle large +ile. $rchi#ig is o+te !oe as a part o+ system %ack"ps. 9t is also "se!
whe ol! !ata is mo#e! +rom a system to some type o+ log-term storage.
tar
9 the )i3-like worl! o+ so+tware, the tar program is the classic tool +or archi#ig +iles.
2(1
$rchi#ig 0iles
9ts ame, short +or tape archive, re#eals its roots as a tool +or makig %ack"p tapes.
While it is still "se! +or that tra!itioal task, it is e="ally a!ept o other storage !e#ices as
well. We o+te see +ileames that e! with the e3tesio <tar or <tgC which i!icate a
?plai@ tar archi#e a! a gDippe! archi#e, respecti#ely. $ tar archi#e ca cosist o+ a
gro"p o+ separate +iles, oe or more !irectory hierarchies, or a mi3t"re o+ %oth. The
comma! syta3 works like this,
tar mode[options] pathname<<<
where mode is oe o+ the +ollowig operatig mo!es Eoly a partial list is show hereH
see the tar ma page +or a complete listF,
Table 15-': tar !odes
>ode Description
c
Create a archi#e +rom a list o+ +iles a!-or !irectories.
x
E3tract a archi#e.
r
$ppe! speci+ie! pathames to the e! o+ a archi#e.
t
*ist the cotets o+ a archi#e.
tar "ses a slightly o!! way o+ e3pressig optios, so weIll ee! some e3amples to show
how it works. 0irst, letIs re-create o"r playgro"! +rom the pre#io"s chapter,
[me@linuxbox ~]$ mkdir -p pla!ground/dir-9**91..D>;*91*..DD>;1**>
[me@linuxbox ~]$ touch pla!ground/dir-9**91..D>;*91*..DD>;1**>/file-
9:-@>
&e3t, letIs create a tar archi#e o+ the etire playgro"!,
[me@linuxbox ~]$ tar cf pla!ground.tar pla!ground
This comma! creates a tar archi#e ame! 7la1ground<tar that cotais the etire
playgro"! !irectory hierarchy. We ca see that the mo!e a! the f optio, which is
"se! to speci+y the ame o+ the tar archi#e, may %e 4oie! together, a! !o ot re="ire a
lea!ig !ash. &ote, howe#er, that the mo!e m"st always %e speci+ie! +irst, %e+ore ay
other optio.
To list the cotets o+ the archi#e, we ca !o this,
2(2
.9 8 $rchi#ig $! Jack"p
[me@linuxbox ~]$ tar tf pla!ground.tar
0or a more !etaile! listig, we ca a!! the v E#er%oseF optio,
[me@linuxbox ~]$ tar tvf pla!ground.tar
&ow, letIs e3tract the playgro"! i a ew locatio. We will !o this %y creatig a ew
!irectory ame! foo, a! chagig the !irectory a! e3tractig the tar archi#e,
[me@linuxbox ~]$ mkdir foo
[me@linuxbox ~]$ cd foo
[me@linuxbox foo]$ tar xf ../pla!ground.tar
[me@linuxbox foo]$ ls
7la1ground
9+ we e3amie the cotets o+ ~6foo67la1ground, we see that the archi#e was
s"ccess+"lly istalle!, creatig a precise repro!"ctio o+ the origial +iles. There is oe
ca#eat, howe#er, "less yo" are operatig as the s"per"ser, +iles a! !irectories e3tracte!
+rom archi#es take o the owership o+ the "ser per+ormig the restoratio, rather tha
the origial ower.
$other iterestig %eha#ior o+ tar is the way it ha!les pathames i archi#es. The
!e+a"lt +or pathames is relati#e, rather tha a%sol"te. tar !oes this %y simply remo#ig
ay lea!ig slash +rom the pathame whe creatig the archi#e. To !emostrate, we will
recreate o"r archi#e, this time speci+yig a a%sol"te pathame,
[me@linuxbox foo]$ cd
[me@linuxbox ~]$ tar cf pla!ground2.tar ~/pla!ground
:emem%er, ~67la1ground will e3pa! ito 6home6me67la1ground whe we
press the eter key, so we will get a a%sol"te pathame +or o"r !emostratio. &e3t, we
will e3tract the archi#e as %e+ore a! watch what happes,
[me@linuxbox ~]$ cd foo
[me@linuxbox foo]$ tar xf ../pla!ground2.tar
[me@linuxbox foo]$ ls
home 7la1ground
[me@linuxbox foo]$ ls home
2(K
$rchi#ig 0iles
me
[me@linuxbox foo]$ ls home/me
7la1ground
Aere we ca see that whe we e3tracte! o"r seco! archi#e, it recreate! the !irectory
home6me67la1ground relati#e to o"r c"rret workig !irectory, ~6foo, ot relati#e
to the root !irectory, as wo"l! ha#e %ee the case with a a%sol"te pathame. This may
seem like a o!! way +or it to work, %"t itIs act"ally more "se+"l this way, as it allows "s
to e3tract archi#es to ay locatio rather tha %eig +orce! to e3tract them to their
origial locatios. :epeatig the e3ercise with the icl"sio o+ the #er%ose optio EvF
will gi#e a clearer pict"re o+ whatIs goig o.
*etIs cosi!er a hypothetical, yet practical e3ample, o+ tar i actio. 9magie we wat
to copy the home !irectory a! its cotets +rom oe system to aother a! we ha#e a
large )SJ har! !ri#e that we ca "se +or the tras+er. 7 o"r mo!er *i"3 system, the
!ri#e is ?a"tomagically@ mo"te! i the 6media !irectory. *etIs also imagie that the
!isk has a #ol"me ame o+ Hig'isk whe we attach it. To make the tar archi#e, we
ca !o the +ollowig,
[me@linuxbox ~]$ sudo tar cf /media/<ig1isk/home.tar /home
$+ter the tar +ile is writte, we "mo"t the !ri#e a! attach it to the seco! comp"ter.
$gai, it is mo"te! at 6media6Hig'isk. To e3tract the archi#e, we !o this,
[me@linuxbox! ~]$ cd /
[me@linuxbox! 6]$ sudo tar xf /media/<ig1isk/home.tar
WhatIs importat to see here is that we m"st +irst chage !irectory to 6, so that the
e3tractio is relati#e to the root !irectory, sice all pathames withi the archi#e are
relati#e.
Whe e3tractig a archi#e, itIs possi%le to limit what is e3tracte! +rom the archi#e. 0or
e3ample, i+ we wate! to e3tract a sigle +ile +rom a archi#e, it co"l! %e !oe like this,
tar xf archive.tar pathname
Jy a!!ig the trailig pathname to the comma!, tar will oly restore the speci+ie! +ile.
>"ltiple pathames may %e speci+ie!. &ote that the pathame m"st %e the +"ll, e3act
2(/
.9 8 $rchi#ig $! Jack"p
relati#e pathame as store! i the archi#e. Whe speci+yig pathames, wil!car!s are
ot ormally s"pporte!H howe#er, the ;&) #ersio o+ tar Ewhich is the #ersio most
o+te +o"! i *i"3 !istri%"tiosF s"pports them with the --9ildcards optio. Aere
is a e3ample "sig o"r pre#io"s 7la1ground<tar +ile,
[me@linuxbox ~]$ cd foo
[me@linuxbox foo]$ tar xf ../pla!ground2.tar --wildcards %home/me/pla
!ground/dir-*/file-:%
This comma! will e3tract oly +iles matchig the speci+ie! pathame icl"!ig the
wil!car! dir-E.
tar is o+te "se! i co4"ctio with find to pro!"ce archi#es. 9 this e3ample, we
will "se find to pro!"ce a set o+ +iles to icl"!e i a archi#e,
[me@linuxbox ~]$ find pla!ground -name %file-:% -exec tar rf
pla!ground.tar %9>% %5%
Aere we "se find to match all the +iles i 7la1ground ame! file-4 a! the,
"sig the -exec actio, we i#oke tar i the appe! mo!e ErF to a!! the matchig
+iles to the archi#e 7la1ground<tar.
)sig tar with find is a goo! way o+ creatig incremental backups o+ a !irectory tree
or a etire system. Jy "sig find to match +iles ewer tha a timestamp +ile, we co"l!
create a archi#e that oly cotais +iles ewer tha the last archi#e, ass"mig that the
timestamp +ile is "p!ate! right a+ter each archi#e is create!.
tar ca also make "se o+ %oth sta!ar! ip"t a! o"tp"t. Aere is a comprehesi#e
e3ample,
[me@linuxbox foo]$ cd
[me@linuxbox ~]$ find pla!ground -name %file-:% 0 tar cf - --files-
from$- 0 g-ip & pla!ground.tg-
9 this e3ample, we "se! the find program to pro!"ce a list o+ matchig +iles a! pipe!
them ito tar. 9+ the +ileame ?-@ is speci+ie!, it is take to mea sta!ar! ip"t or
o"tp"t, as ee!e! E%y the way, this co#etio o+ "sig ?-@ to represet sta!ar!
ip"t-o"tp"t is "se! %y a "m%er o+ other programs, too.F The --files-from optio
Ewhich may %e also %e speci+ie! as -TF ca"ses tar to rea! its list o+ pathames +rom a
2(8
$rchi#ig 0iles
+ile rather tha the comma! lie. *astly, the archi#e pro!"ce! %y tar is pipe! ito
gCi7 to create the compresse! archi#e 7la1ground<tgC. The <tgC e3tesio is the
co#etioal e3tesio gi#e to gDip-compresse! tar +iles. The e3tesio <tar<gC is
also "se! sometimes.
While we "se! the gCi7 program e3terally to pro!"ce! o"r compresse! archi#e,
mo!er #ersios o+ ;&) tar s"pport %oth gDip a! %Dip2 compressio !irectly, with the
"se o+ the C a! j optios, respecti#ely. )sig o"r pre#io"s e3ample as a %ase, we ca
simpli+y it this way,
[me@linuxbox ~]$ find pla!ground -name %file-:% 0 tar c-f
pla!ground.tg- -+ -
9+ we ha! wate! to create a %Dip2 compresse! archi#e istea!, we co"l! ha#e !oe this,
[me@linuxbox ~]$ find pla!ground -name %file-:% 0 tar cjf
pla!ground.tb- -+ -
Jy simply chagig the compressio optio +rom C to j Ea! chagig the o"tp"t +ileIs
e3tesio to <tbC to i!icate a %Dip2 compresse! +ileF we ea%le! %Dip2 compressio.
$other iterestig "se o+ sta!ar! ip"t a! o"tp"t with the tar comma! i#ol#es
tras+errig +iles %etwee systems o#er a etwork. 9magie that we ha! two machies
r"ig a )i3-like system e="ippe! with tar a! ssh. 9 s"ch a sceario, we co"l!
tras+er a !irectory +rom a remote system Eame! remote-s1s +or this e3ampleF to o"r
local system,
[me@linuxbox ~]$ mkdir remote-stuff
[me@linuxbox ~]$ cd remote-stuff
[me@linuxbox remote-stuff]$ ssh remote-s!s %tar cf - 1ocuments% 0 tar
xf -
me@remote-s1sbs 7ass9ord:
[me@linuxbox remote-stuff]$ ls
'ocuments
Aere we were a%le to copy a !irectory ame! 'ocuments +rom the remote system
remote-s1s to a !irectory withi the !irectory ame! remote-stuff o the local
system. Aow !i! we !o thisC 0irst, we la"che! the tar program o the remote system
"sig ssh. 5o" will recall that ssh allows "s to e3ec"te a program remotely o a
etworke! comp"ter a! ?see@ the res"lts o the local systemBthe sta!ar! o"tp"t
2(9
.9 8 $rchi#ig $! Jack"p
pro!"ce! o the remote system is set to the local system +or #iewig. We ca take
a!#atage o+ this %y ha#ig tar create a archi#e Ethe c mo!eF a! se! it to sta!ar!
o"tp"t, rather tha a +ile Ethe f optio with the !ash arg"metF, there%y trasportig the
archi#e o#er the ecrypte! t"el pro#i!e! %y ssh to the local system. 7 the local
system, we e3ec"te tar a! ha#e it e3pa! a archi#e Ethe x mo!eF s"pplie! +rom
sta!ar! ip"t Eagai, the f optio with the !ash arg"metF.
Hi5
The Ci7 program is %oth a compressio tool a! a archi#er. The +ile +ormat "se! %y
the program is +amiliar to Wi!ows "sers, as it rea!s a! writes <Ci7 +iles. 9 *i"3,
howe#er, gCi7 is the pre!omiat compressio program with bCi7! %eig a close
seco!.
9 its most %asic "sage, Ci7 is i#oke! like this,
Ci7 options zipfile file...
0or e3ample, to make a Dip archi#e o+ o"r playgro"!, we wo"l! !o this,
[me@linuxbox ~]$ -ip -r pla!ground.-ip pla!ground
)less we icl"!e the -r optio +or rec"rsio, oly the 7la1ground !irectory E%"t
oe o+ its cotetsF is store!. $ltho"gh the a!!itio o+ the e3tesio <Ci7 is a"tomatic
a, we will icl"!e the +ile e3tesio +or clarity.
'"rig the creatio o+ the Dip archi#e, Ci7 will ormally !isplay a series o+ messages
like this,
adding: 7la1ground6dir-(!(6file-I Nstored (5P
adding: 7la1ground6dir-(!(6file-U Nstored (5P
adding: 7la1ground6dir-(!(6file-V Nstored (5P
adding: 7la1ground6dir-(/)6 Nstored (5P
adding: 7la1ground6dir-(/)6file-* Nstored (5P
These messages show the stat"s o+ each +ile a!!e! to the archi#e. Ci7 will a!! +iles to
the archi#e "sig oe o+ two storage metho!s, either it will ?store@ a +ile witho"t
compressio, as show here, or it will ?!e+late@ the +ile which per+orms compressio.
The "meric #al"e !isplaye! a+ter the storage metho! i!icates the amo"t o+
compressio achie#e!. Sice o"r playgro"! oly cotais empty +iles, o compressio
is per+orme! o its cotets.
210
$rchi#ig 0iles
E3tractig the cotets o+ a Dip +ile is straight+orwar! whe "sig the unCi7 program,
[me@linuxbox ~]$ cd foo
[me@linuxbox foo]$ un-ip ../pla!ground.-ip
7e thig to ote a%o"t Ci7 Eas oppose! to tarF is that i+ a e3istig archi#e is
speci+ie!, it is "p!ate! rather tha replace!. This meas that the e3istig archi#e is
preser#e!, %"t ew +iles are a!!e! a! matchig +iles are replace!.
0iles may %e liste! a! e3tracte! selecti#ely +rom a Dip archi#e %y speci+yig them to
unCi7,
[me@linuxbox ~]$ un-ip -l pla!ground.-ip pla!ground/dir-MC/file-@
4rchive: <<67la1ground<Ci7
=ength 'ate Time Lame
-------- ---- ---- ----
( #(-("-(/ (0:!" 7la1ground6dir-/)6file-I
-------- -------
( # file
[me@linuxbox ~]$ cd foo
[me@linuxbox foo]$ un-ip ./pla!ground.-ip pla!ground/dir-MC/file-@
4rchive: <<67la1ground<Ci7
re7lace 7la1ground6dir-/)6file-IF [1]es> [n]o> [4]ll> [L]one>
[r]ename: !
extracting: 7la1ground6dir-/)6file-I
)sig the -l optio ca"ses unCi7 to merely list the cotets o+ the archi#e witho"t
e3tractig the +ile. 9+ o +ileEsF are speci+ie!, unCi7 will list all +iles i the archi#e. The
-v optio ca %e a!!e! to icrease the #er%osity o+ the listig. &ote that whe the
archi#e e3tractio co+licts with a e3istig +ile, the "ser is prompte! %e+ore the +ile is
replace!.
*ike tar, Ci7 ca make "se o+ sta!ar! ip"t a! o"tp"t, tho"gh its implemetatio is
somewhat less "se+"l. 9t is possi%le to pipe a list o+ +ileames to Ci7 #ia the -@ optio,
[me@linuxbox foo]$ cd
[me@linuxbox ~]$ find pla!ground -name Jfile-:J 0 -ip -V file-:.-ip
Aere we "se find to geerate a list o+ +iles matchig the test -name _file-4_, a!
pipe the list ito Ci7, which creates the archi#e file-4<Ci7 cotaiig the selecte!
+iles.
21.
.9 8 $rchi#ig $! Jack"p
Ci7 also s"pports writig its o"tp"t to sta!ar! o"tp"t, %"t its "se is limite! %eca"se #ery
+ew programs ca make "se o+ the o"tp"t. )+ort"ately, the unCi7 program, !oes ot
accept sta!ar! ip"t. This pre#ets Ci7 a! unCi7 +rom %eig "se! together to
per+orm etwork +ile copyig like tar.
Ci7 ca, howe#er, accept sta!ar! ip"t, so it ca %e "se! to compress the o"tp"t o+
other programs,
[me@linuxbox ~]$ ls -l /etc/ 0 -ip ls-etc.-ip -
adding: - Ndeflated /(5P
9 this e3ample we pipe the o"tp"t o+ ls ito Ci7. *ike tar, Ci7 iterprets the trailig
!ash as ?"se sta!ar! ip"t +or the ip"t +ile.@
The unCi7 program allows its o"tp"t to %e set to sta!ar! o"tp"t whe the -7 E+or
pipeF optio is speci+ie!,
[me@linuxbox ~]$ un-ip -p ls-etc.-ip 0 less
We to"che! o some o+ the %asic thigs that Ci7-unCi7 ca !o. They %oth ha#e a lot o+
optios that a!! to their +le3i%ility, tho"gh some are plat+orm speci+ic to other systems.
The ma pages +or %oth Ci7 a! unCi7 are pretty goo! a! cotai "se+"l e3amples.
Aowe#er, the mai "se o+ these programs is +or e3chagig +iles with Wi!ows systems,
rather tha per+ormig compressio a! archi#ig o *i"3, where tar a! gCi7 are
greatly pre+erre!.
S,nchroniGing ;iles #nd Directories
$ commo strategy +or maitaiig a %ack"p copy o+ a system i#ol#es keepig oe or
more !irectories sychroiDe! with aother !irectory Eor !irectoriesF locate! o either the
local system E"s"ally a remo#a%le storage !e#ice o+ some ki!F or with a remote system.
We might, +or e3ample, ha#e a local copy o+ a we% site "!er !e#elopmet a!
sychroiDe it +rom time to time with the ?li#e@ copy o a remote we% ser#er.
9 the )i3-like worl!, the pre+erre! tool +or this task is rs1nc. This program ca
sychroiDe %oth local a! remote !irectories %y "sig the rsync remote-update protocol,
which allows rs1nc to ="ickly !etect the !i++ereces %etwee two !irectories a!
per+orm the miim"m amo"t o+ copyig re="ire! to %rig them ito syc. This makes
rs1nc #ery +ast a! ecoomical to "se, compare! to other ki!s o+ copy programs.
rs1nc is i#oke! like this,
212
SychroiDig 0iles $! 'irectories
rs1nc options source destination
where source a! destination are oe o+ the +ollowig,
$ local +ile or !irectory
$ remote +ile or !irectory i the +orm o+ Fuser@Ghost:path
$ remote rsyc ser#er speci+ie! with a ):9 o+ rsync:;;Fuser@GhostF:portG;path
&ote that either the so"rce or !estiatio m"st %e a local +ile. :emote to remote copyig
is ot s"pporte!.
*etIs try rs1nc o"t o some local +iles. 0irst, letIs clea o"t o"r foo !irectory,
[me@linuxbox ~]$ rm -rf foo/*
&e3t, weIll sychroiDe the 7la1ground !irectory with a correspo!ig copy i foo,
[me@linuxbox ~]$ rs!nc -av pla!ground foo
WeI#e icl"!e! %oth the -a optio E+or archi#igBca"ses rec"rsio a! preser#atio o+
+ile attri%"tesF a! the -v optio E#er%ose o"tp"tF to make a mirror o+ the 7la1ground
!irectory withi foo. While the comma! r"s, we will see a list o+ the +iles a!
!irectories %eig copie!. $t the e!, we will see a s"mmary message like this,
sent #$")"0 b1tes received ")/)( b1tes $/)!"/<(( b1tes6sec
total siCe is $!$( s7eedu7 is (<(!
i!icatig the amo"t o+ copyig per+orme!. 9+ we r" the comma! agai, we will see a
!i++eret res"lt,
[me@linuxbox ~]$ rs!nc -av pla!gound foo
building file list <<< done
sent !!.$" b1tes received !( b1tes %"$#(<(( b1tes6sec
total siCe is $!$( s7eedu7 is (<#%
&otice that there was o listig o+ +iles. This is %eca"se rs1nc !etecte! that there were
o !i++ereces %etwee ~67la1ground a! ~6foo67la1ground, a! there+ore it
21(
.9 8 $rchi#ig $! Jack"p
!i!It ee! to copy aythig. 9+ we mo!i+y a +ile i 7la1ground a! r" rs1nc
agai,
[me@linuxbox ~]$ touch pla!ground/dir-*DD/file-@
[me@linuxbox ~]$ rs!nc -av pla!ground foo
building file list <<< done
7la1ground6dir-(006file-I
sent !!./" b1tes received %! b1tes %"%"%<(( b1tes6sec
total siCe is $!$( s7eedu7 is (<#%
we see that rs1nc !etecte! the chage a! copie! oly the "p!ate! +ile.
$s a practical e3ample, letIs cosi!er the imagiary e3teral har! !ri#e that we "se!
earlier with tar. 9+ we attach the !ri#e to o"r system a!, oce agai, it is mo"te! at 6
media6Hig'isk, we ca per+orm a "se+"l system %ack"p %y +irst creatig a !irectory,
ame! 6backu7 o the e3teral !ri#e a! the "sig rs1nc to copy the most importat
st"++ +rom o"r system to the e3teral !ri#e,
[me@linuxbox ~]$ mkdir /media/<ig1isk/backup
[me@linuxbox ~]$ sudo rs!nc -av --delete /etc /home /usr/local
/media/<ig1isk/backup
9 this e3ample, we copie! the 6etc, 6home, a! 6usr6local !irectories +rom o"r
system to o"r imagiary storage !e#ice. We icl"!e! the --delete optio to remo#e
+iles that may ha#e e3iste! o the %ack"p !e#ice that o loger e3iste! o the so"rce
!e#ice Ethis is irrele#at the +irst time we make a %ack"p, %"t will %e "se+"l o
s"%se="et copies.F :epeatig the proce!"re o+ attachig the e3teral !ri#e a! r"ig
this rs1nc comma! wo"l! %e a "se+"l Etho"gh ot i!ealF way o+ keepig a small
system %acke! "p. 7+ co"rse, a alias wo"l! %e help+"l here, too. We co"l! create a
alias a! a!! it to o"r <bashrc +ile to pro#i!e this +eat"re,
alias backu7MKsudo rs1nc -av --delete 6etc 6home 6usr6local
6media6Hig'isk6backu7K
&ow all we ha#e to !o is attach o"r e3teral !ri#e a! r" the backu7 comma! to !o
the 4o%.
211
SychroiDig 0iles $! 'irectories
Using rs1nc <;er A ,et%ork
7e o+ the real %ea"ties o+ rs1nc is that it ca %e "se! to copy +iles o#er a etwork.
$+ter all, the ?r@ i rs1nc sta!s +or ?remote.@ :emote copyig ca %e !oe i oe o+
two ways. The +irst way is with aother system that has rs1nc istalle!, alog with a
remote shell program s"ch as ssh. *etIs say we ha! aother system o o"r local
etwork with a lot o+ a#aila%le har! !ri#e space a! we wate! to per+orm o"r %ack"p
operatio "sig the remote system istea! o+ a e3teral !ri#e. $ss"mig that it alrea!y
ha! a !irectory ame! 6backu7 where we co"l! !eli#er o"r +iles, we co"l! !o this,
[me@linuxbox ~]$ sudo rs!nc -av --delete --rsh$ssh /etc /home
/usr/local remote-s!s2/backup
We ma!e two chages to o"r comma! to +acilitate the etwork copy. 0irst, we a!!e!
the --rshMssh optio, which istr"cts rs1nc to "se the ssh program as its remote
shell. 9 this way, we were a%le to "se a ssh ecrypte! t"el to sec"rely tras+er the
!ata +rom the local system to the remote host. Seco!, we speci+ie! the remote host %y
pre+i3ig its ame Ei this case the remote host is ame! remote-s1sF to the
!estiatio path ame.
The seco! way that rs1nc ca %e "se! to sychroiDe +iles o#er a etwork is %y "sig
a rysnc server. rs1nc ca %e co+ig"re! to r" as a !aemo a! liste to icomig
re="ests +or sychroiDatio. This is o+te !oe to allow mirrorig o+ a remote system.
0or e3ample, :e! Aat So+tware maitais a large repository o+ so+tware packages "!er
!e#elopmet +or its 0e!ora !istri%"tio. 9t is "se+"l +or so+tware testers to mirror this
collectio !"rig the testig phase o+ the !istri%"tio release cycle. Sice +iles i the
repository chage +re="etly Eo+te more tha oce a !ayF, it is !esira%le to maitai a
local mirror %y perio!ic sychroiDatio, rather tha %y %"lk copyig o+ the repository.
7e o+ these repositories is kept at ;eorgia TechH we co"l! mirror it "sig o"r local copy
o+ rs1nc a! their rsyc ser#er like this,
[me@linuxbox ~]$ mkdir fedora-devel
[me@linuxbox ~]$ rs!nc -av -delete rs!nc2//rs!nc.gtlib.gatech.edu/fed
ora-linux-core/development/iMN/os fedora-devel
9 this e3ample, we "se the ):9 o+ the remote rsyc ser#er, which cosists o+ a protocol
Ers1nc:66F, +ollowe! %y the remote host ame Ers1nc<gtlib<gatech<eduF,
+ollowe! %y the pathame o+ the repository.
212
.9 8 $rchi#ig $! Jack"p
;urther 1eading
The ma pages +or all o+ the comma!s !isc"sse! here are pretty clear a!
cotai "se+"l e3amples. 9 a!!itio, the ;&) 6ro4ect has a goo! olie ma"al
+or its #ersio o+ tar. 9t ca %e +o"! here,
http,--www.g".org-so+tware-tar-ma"al-i!e3.html
21K
20 8 :eg"lar E3pressios
20 /egular !"#ressions
9 the e3t +ew chapters, we are goig to look at tools "se! to maip"late te3t. $s we
ha#e see, te3t !ata plays a importat role o all )i3-like systems, s"ch as *i"3. J"t
%e+ore we ca +"lly appreciate all o+ the +eat"res o++ere! %y these tools, we ha#e to +irst
e3amie a techology that is +re="etly associate! with the most sophisticate! "ses o+
these toolsB regular e+pressions.
$s we ha#e a#igate! the may +eat"res a! +acilities o++ere! %y the comma! lie, we
ha#e eco"tere! some tr"ly arcae shell +eat"res a! comma!s, s"ch as shell
e3pasio a! ="otig, key%oar! shortc"ts, a! comma! history, ot to metio the #i
e!itor. :eg"lar e3pressios coti"e this ?tra!itio@ a! may %e Earg"a%lyF the most
arcae +eat"re o+ them all. This is ot to s"ggest that the time it takes to lear a%o"t them
is ot worth the e++ort. T"ite the cotrary. $ goo! "!ersta!ig will ea%le "s to
per+orm amaDig +eats, tho"gh their +"ll #al"e may ot %e imme!iately apparet.
What Are !eguar 6x5ressions?
Simply p"t, reg"lar e3pressios are sym%olic otatios "se! to i!eti+y patters i te3t.
9 some ways, they resem%le the shellIs wil!car! metho! o+ matchig +ile a! pathames,
%"t o a m"ch gra!er scale. :eg"lar e3pressios are s"pporte! %y may comma! lie
tools a! %y most programmig lag"ages to +acilitate the sol"tio o+ te3t maip"latio
pro%lems. Aowe#er, to +"rther co+"se thigs, ot all reg"lar e3pressios are the sameH
they #ary slightly +rom tool to tool a! +rom programmig lag"age to lag"age. 0or o"r
!isc"ssio, we will limit o"rsel#es to reg"lar e3pressios as !escri%e! i the 67S9G
sta!ar! Ewhich will co#er most o+ the comma! lie toolsF, as oppose! to may
programmig lag"ages Emost ota%ly 6erlF, which "se slightly larger a! richer sets o+
otatios.
grep
The mai program we will "se to work with reg"lar e3pressios is o"r ol! pal, gre7.
The ame ?grep@ is act"ally !eri#e! +rom the phrase ?glo%al reg"lar e3pressio prit,@ so
we ca see that gre7 has somethig to !o with reg"lar e3pressios. 9 essece, gre7
searches te3t +iles +or the occ"rrece o+ a speci+ie! reg"lar e3pressio a! o"tp"ts ay
21/
20 8 :eg"lar E3pressios
lie cotaiig a match to sta!ar! o"tp"t.
So +ar, we ha#e "se! gre7 with +i3e! strigs, like so,
[me@linuxbox ~]$ ls /usr/bin 0 grep -ip
This will list all the +iles i the 6usr6bin !irectory whose ames cotai the s"%strig
?Dip@.
The gre7 program accepts optios a! arg"mets this way,
gre7 [options] regex [file...]
where rege+ is a reg"lar e3pressio.
Aere is a list o+ the commoly "se! gre7 optios,
Table'=-1: grep &ptions
?ption Description
-i
9gore case. 'o ot !istig"ish %etwee "pper a! lower case
characters. >ay also %e speci+ie! --ignore-case.
-v
9#ert match. &ormally, gre7 prits lies that cotai a match.
This optio ca"ses gre7 to prit e#ery lie that !oes ot cotai a
match. >ay also %e speci+ie! --invert-match.
-c
6rit the "m%er o+ matches Eor o-matches i+ the -v optio is
also speci+ie!F istea! o+ the lies themsel#es. >ay also %e
speci+ie! --count.
-l
6rit the ame o+ each +ile that cotais a match istea! o+ the lies
themsel#es. >ay also %e speci+ie! --files-9ith-matches.
-=
*ike the -l optio, %"t prit oly the ames o+ +iles that !o ot
cotai matches. >ay also %e speci+ie! --files-9ithout-
match.
-n
6re+i3 each matchig lie with the "m%er o+ the lie withi the
+ile. >ay also %e speci+ie! --line-number<
-h
0or m"lti-+ile searches, s"ppress the o"tp"t o+ +ileames. >ay also
%e speci+ie! --no-filename.
9 or!er to more +"lly e3plore gre7, letIs create some te3t +iles to search,
218
grep
[me@linuxbox ~]$ ls /bin & dirlist-bin.txt
[me@linuxbox ~]$ ls /usr/bin & dirlist-usr-bin.txt
[me@linuxbox ~]$ ls /sbin & dirlist-sbin.txt
[me@linuxbox ~]$ ls /usr/sbin & dirlist-usr-sbin.txt
[me@linuxbox ~]$ ls dirlist*.txt
dirlist-bin<txt dirlist-sbin<txt dirlist-usr-sbin<txt
dirlist-usr-bin<txt
We ca per+orm a simple search o+ o"r list o+ +iles like this,
[me@linuxbox ~]$ grep b-ip dirlist*.txt
dirlist-bin<txt:bCi7!
dirlist-bin<txt:bCi7!recover
9 this e3ample, gre7 searches all o+ the liste! +iles +or the strig bCi7 a! +i!s two
matches, %oth i the +ile dirlist-bin<txt. 9+ we were oly itereste! i the list o+
+iles that cotaie! matches rather tha the matches themsel#es, we co"l! speci+y the -l
optio,
[me@linuxbox ~]$ grep -l b-ip dirlist*.txt
dirlist-bin<txt
Co#ersely, i+ we wate! oly to see a list o+ the +iles that !i! ot cotai a match, we
co"l! !o this,
[me@linuxbox ~]$ grep -T b-ip dirlist*.txt
dirlist-sbin<txt
dirlist-usr-bin<txt
dirlist-usr-sbin<txt
>etacharacters #nd Literals
While it may ot seem apparet, o"r gre7 searches ha#e %ee "sig reg"lar e3pressios
all alog, al%eit #ery simple oes. The reg"lar e3pressio ?%Dip@ is take to mea that a
match will occ"r oly i+ the lie i the +ile cotais at least +o"r characters a! that
somewhere i the lie the characters ?%@, ?D@, ?i@, a! ?p@ are +o"! i that or!er, with o
other characters i %etwee. The characters i the strig ?%Dip@ are all literal characters,
i that they match themsel#es. 9 a!!itio to literals, reg"lar e3pressios may also
219
20 8 :eg"lar E3pressios
icl"!e metacharacters that are "se! to speci+y more comple3 matches. :eg"lar
e3pressio metacharacters cosist o+ the +ollowig,
` $ < [ ] [ \ - F E 8 N P Q ]
$ll other characters are cosi!ere! literals, tho"gh the %ackslash character is "se! i a
+ew cases to create meta se<uences, as well as allowig the metacharacters to %e escape!
a! treate! as literals istea! o+ %eig iterprete! as metacharacters.
1ote5 $s we ca see, may o+ the reg"lar e3pressio metacharacters are also
characters that ha#e meaig to the shell whe e3pasio is per+orme!. Whe we
pass reg"lar e3pressios cotaiig metacharacters o the comma! lie, it is #ital
that they %e eclose! i ="otes to pre#et the shell +rom attemptig to e3pa! them.
The #n, Character
The +irst metacharacter we will look at is the !ot or perio! character, which is "se! to
match ay character. 9+ we icl"!e it i a reg"lar e3pressio, it will match ay character
i that character positio. AereIs a e3ample,
[me@linuxbox ~]$ grep -h %.-ip% dirlist*.txt
bunCi7!
bCi7!
bCi7!recover
gunCi7
gCi7
funCi7
g7g-Ci7
7reunCi7
7reCi7
7reCi7-bin
unCi7
unCi7sfx
We searche! +or ay lie i o"r +iles that matches the reg"lar e3pressio ?.Dip@. There are
a co"ple o+ iterestig thigs to ote a%o"t the res"lts. &otice that the Ci7 program was
ot +o"!. This is %eca"se the icl"sio o+ the !ot metacharacter i o"r reg"lar
e3pressio icrease! the legth o+ the re="ire! match to +o"r characters, a! %eca"se the
ame ?Dip@ oly cotais three, it !oes ot match. $lso, i+ there ha! %ee ay +iles i o"r
lists that cotaie! the +ile e3tesio <Ci7, they wo"l! ha#e also %ee matche! as well,
%eca"se the perio! character i the +ile e3tesio is treate! as ?ay character,@ too.
220
$chors
#nchors
The caret E`F a! !ollar sig E$F characters are treate! as anchors i reg"lar e3pressios.
This meas that they ca"se the match to occ"r oly i+ the reg"lar e3pressio is +o"! at
the %egiig o+ the lie E`F or at the e! o+ the lie E$F,
[me@linuxbox ~]$ grep -h %[-ip% dirlist*.txt
Ci7
Ci7cloak
Ci7gre7
Ci7info
Ci7note
Ci7s7lit
[me@linuxbox ~]$ grep -h %-ip3% dirlist*.txt
gunCi7
gCi7
funCi7
g7g-Ci7
7reunCi7
7reCi7
unCi7
Ci7
[me@linuxbox ~]$ grep -h %[-ip3% dirlist*.txt
Ci7
Aere we searche! the list o+ +iles +or the strig ?Dip@ locate! at the %egiig o+ the lie,
the e! o+ the lie, a! o a lie where it is at %oth the %egiig a! the e! o+ the lie
Ei.e., %y itsel+ o the lie.F &ote that the reg"lar e3pressio e`[I Ea %egiig a! a e!
with othig i %etweeF will match %lak lies.
# Cross4ord PuGGle 3elper
E#e with o"r limite! kowle!ge o+ reg"lar e3pressios at this poit, we ca !o
somethig "se+"l.
>y wi+e lo#es crosswor! p"DDles a! she will sometimes ask me +or help with a
partic"lar ="estio. Somethig like, ?whatIs a +i#e letter wor! whose thir! letter
is e4I a! last letter is erI that meas...C@ This ki! o+ ="estio got me thikig.
'i! yo" kow that yo"r *i"3 system cotais a !ictioaryC 9t !oes. Take a look
i the 6usr6share6dict !irectory a! yo" might +i! oe, or se#eral. The
!ictioary +iles locate! there are 4"st log lists o+ wor!s, oe per lie, arrage! i
alpha%etical or!er. 7 my system, the 9ords +ile cotais 4"st o#er 98,200
22.
20 8 :eg"lar E3pressios
wor!s. To +i! possi%le aswers to the crosswor! p"DDle ="estio a%o#e, we
co"l! !o this,
[me@linuxbox ~]$ grep -i %[..j.r3% /usr/share/dict/words
+ajor
major
)sig this reg"lar e3pressio, we ca +i! all the wor!s i o"r !ictioary +ile that
are +i#e letters log a! ha#e a ?4@ i the thir! positio a! a ?r@ i the last
positio.
/rac(et $xpressions #nd Character Classes
9 a!!itio to matchig ay character at a gi#e positio i o"r reg"lar e3pressio, we
ca also match a sigle character +rom a speci+ie! set o+ characters %y "sig bracket
e+pressions. With %racket e3pressios, we ca speci+y a set o+ characters Eicl"!ig
characters that wo"l! otherwise %e iterprete! as metacharactersF to %e matche!. 9 this
e3ample, "sig a two character set,
[me@linuxbox ~]$ grep -h %(bg)-ip% dirlist*.txt
bCi7!
bCi7!recover
gCi7
we match ay lie that cotais the strig ?%Dip@ or ?gDip@.
$ set may cotai ay "m%er o+ characters, a! metacharacters lose their special
meaig whe place! withi %rackets. Aowe#er, there are two cases i which
metacharacters are "se! withi %racket e3pressios, a! ha#e !i++eret meaigs. The
+irst is the caret E`F, which is "se! to i!icate egatioH the seco! is the !ash E-F, which
is "se! to i!icate a character rage.
,egation
9+ the +irst character i a %racket e3pressio is a caret E`F, the remaiig characters are
take to %e a set o+ characters that m"st ot %e preset at the gi#e character positio. We
!o this %y mo!i+yig o"r pre#io"s e3ample,
[me@linuxbox ~]$ grep -h %([bg)-ip% dirlist*.txt
bunCi7!
222
Jracket E3pressios $! Character Classes
gunCi7
funCi7
g7g-Ci7
7reunCi7
7reCi7
7reCi7-bin
unCi7
unCi7sfx
With egatio acti#ate!, we get a list o+ +iles that cotai the strig ?Dip@ prece!e! %y ay
character e3cept ?%@ or ?g@. &otice that the +ile Ci7 was ot +o"!. $ egate! character
set still re="ires a character at the gi#e positio, %"t the character m"st ot %e a mem%er
o+ the egate! set.
The caret character oly i#okes egatio i+ it is the +irst character withi a %racket
e3pressioH otherwise, it loses its special meaig a! %ecomes a or!iary character i
the set.
Traditiona Chara/ter !anges
9+ we wate! to costr"ct a reg"lar e3pressio that wo"l! +i! e#ery +ile i o"r lists
%egiig with a "pper case letter, we co"l! !o this,
[me@linuxbox ~]$ grep -h %[(:<=1G7QRPYXTO?SB\HF+E]WZ@^)% dirlist*.txt
9tIs 4"st a matter o+ p"ttig all twety-si3 "pper case letters i a %racket e3pressio. J"t
the i!ea o+ all that typig is !eeply tro"%lig, so there is aother way,
[me@linuxbox ~]$ grep -h %[(:-@)% dirlist*.txt
+42&'&;
Control:anel
@&T
S&4'
:*T
V
V##
Vorg
+42&-=::A&*
Let9ork+anager
Let9ork+anager'is7atcher
Jy "sig a three character rage, we ca a%%re#iate the twety-si3 letters. $y rage o+
22(
20 8 :eg"lar E3pressios
characters ca %e e3presse! this way icl"!ig m"ltiple rages, s"ch as this e3pressio
that matches all +ileames startig with letters a! "m%ers,
[me@linuxbox ~]$ grep -h %[(:-@a--*-D)% dirlist*.txt
9 character rages, we see that the !ash character is treate! specially, so how !o we
act"ally icl"!e a !ash character i a %racket e3pressioC Jy makig it the +irst character
i the e3pressio. Cosi!er these two e3amples,
[me@linuxbox ~]$ grep -h %(:-@)% dirlist*.txt
This will match e#ery +ileame cotaiig a "pper case letter. While,
[me@linuxbox ~]$ grep -h %(-:@)% dirlist*.txt
will match e#ery +ileame cotaiig a !ash, or a "pper case ?$@ or a "ppercase ?f@.
'<SIJ Chara/ter Casses
The tra!itioal character rages are a easily "!erstoo! a! e++ecti#e way to ha!le the
pro%lem o+ ="ickly speci+yig sets o+ characters. )+ort"ately, they !oIt always work.
While we ha#e ot eco"tere! ay pro%lems with o"r "se o+ gre7 so +ar, we might r"
ito pro%lems "sig other programs.
Jack i Chapter 2, we looke! at how wil!car!s are "se! to per+orm pathame e3pasio.
9 that !isc"ssio, we sai! that character rages co"l! %e "se! i a maer almost
i!etical to the way they are "se! i reg"lar e3pressios, %"t hereIs the pro%lem,
[me@linuxbox ~]$ ls /usr/sbin/(:<=1G7QRPYXTO?SB\HF+E]WZ^@)*
6usr6sbin6+42&-=::A&*
6usr6sbin6Let9ork+anager'is7atcher
6usr6sbin6Let9ork+anager
E'epe!ig o the *i"3 !istri%"tio, we will get a !i++eret list o+ +iles, possi%ly a
empty list. This e3ample is +rom )%"t"F This comma! pro!"ces the e3pecte! res"lt
Ba list o+ oly the +iles whose ames %egi with a "ppercase letter, %"t,
221
Jracket E3pressios $! Character Classes
[me@linuxbox ~]$ ls /usr/sbin/(:-@)*
6usr6sbin6biosdecode
6usr6sbin6chat
6usr6sbin6chg7ass9d
6usr6sbin6ch7ass9d
6usr6sbin6chroot
6usr6sbin6cleanu7-info
6usr6sbin6com7lain
6usr6sbin6console-kit-daemon
with this comma! we get a etirely !i++eret res"lt Eoly a partial listig o+ the res"lts
is showF. Why is thatC 9tIs a log story, %"t hereIs the short #ersio,
Jack whe )i3 was +irst !e#elope!, it oly kew a%o"t $SC99 characters, a! this
+eat"re re+lects that +act. 9 $SC99, the +irst thirty-two characters E"m%ers 0-(.F are
cotrol co!es Ethigs like ta%s, %ackspaces, a! carriage ret"rsF. The e3t thirty-two
E(2-K(F cotai prita%le characters, icl"!ig most p"ct"atio characters a! the
"merals Dero thro"gh ie. The e3t thirty-two E"m%ers K1-92F cotai the "ppercase
letters a! a +ew more p"ct"atio sym%ols. The +ial thirty-oe E"m%ers 9K-.2/F
cotai the lowercase letters a! yet more p"ct"atio sym%ols. Jase! o this
arragemet, systems "sig $SC99 "se! a collation order that looke! like this,
4HC'&-@SA?2=+L:ZR*T3;,VUIabcdefghijklmno7Brstuv9x1C
This !i++ers +rom proper !ictioary or!er, which is like this,
a4bHcCd'e&f-g@hSiAj?k2l=m+nLo7:BZrRs*tTu3v;9,xV1UCI
$s the pop"larity o+ )i3 sprea! %eyo! the )ite! States, there grew a ee! to s"pport
characters ot +o"! i ).S. Eglish. The $SC99 ta%le was e3pa!e! to "se a +"ll eight
%its, a!!ig characters "m%ers .28-222, which accommo!ate! may more lag"ages.
To s"pport this a%ility, the 67S9G sta!ar!s itro!"ce! a cocept calle! a locale, which
co"l! %e a!4"ste! to select the character set ee!e! +or a partic"lar locatio. We ca see
the lag"age settig o+ o"r system "sig this comma!,
[me@linuxbox ~]$ echo 3T:?Q
enO3*<3T--/
With this settig, 67S9G compliat applicatios will "se a !ictioary collatio or!er
rather tha $SC99 or!er. This e3plais the %eha#ior o+ the comma!s a%o#e. $ character
rage o+ [4-I] whe iterprete! i !ictioary or!er icl"!es all o+ the alpha%etic
characters e3cept the lowercase ?a@, hece o"r res"lts.
To partially work aro"! this pro%lem, the 67S9G sta!ar! icl"!es a "m%er o+
character classes which pro#i!e "se+"l rages o+ characters. They are !escri%e! i the
222
20 8 :eg"lar E3pressios
ta%le %elow,
Ta%le 20-2, 67S9G Character Classes
Character Class Description
[:alnum:]
The alpha"meric characters. 9 $SC99, e="i#alet to,
[4-Ia-C(-0]
[:9ord:]
The same as a,al"m,b, with the a!!itio o+ the "!erscore
EOF character.
[:al7ha:]
The alpha%etic characters. 9 $SC99, e="i#alet to,
[4-Ia-C]
[:blank:]
9cl"!es the space a! ta% characters.
[:cntrl:]
The $SC99 cotrol co!es. 9cl"!es the $SC99 characters Dero
thro"gh thirty-oe a! .2/.
[:digit:]
The "merals Dero thro"gh ie.
[:gra7h:]
The #isi%le characters. 9 $SC99, it icl"!es characters ((
thro"gh .2K.
[:lo9er:]
The lowercase letters.
[:7unct:]
The p"ct"atio characters. 9 $SC99, e="i#alet to,
[-G_^$5YKNPE8><6:TXMDF@[]]]]OJ[Q\~]
[:7rint:]
The prita%le characters. $ll the characters i [:gra7h:]
pl"s the space character.
[:s7ace:]
The whitespace characters icl"!ig space, ta%, carriage
ret"r, ewlie, #ertical ta%, a! +orm +ee!. 9 $SC99,
e="i#alet to,
[ ]t]r]n]v]f]
[:u77er:]
The "pper case characters.
[:xdigit:]
Characters "se! to e3press he3a!ecimal "m%ers. 9 $SC99,
e="i#alet to,
[(-04--a-f]
E#e with the character classes, there is still o co#eiet way to e3press partial rages,
s"ch as a$->b.
)sig character classes, we ca repeat o"r !irectory listig a! see a impro#e! res"lt,
22K
Jracket E3pressios $! Character Classes
[me@linuxbox ~]$ ls /usr/sbin/((2upper2))*
6usr6sbin6+42&-=::A&*
6usr6sbin6Let9ork+anager'is7atcher
6usr6sbin6Let9ork+anager
:emem%er, howe#er, that this is ot a e3ample o+ a reg"lar e3pressio, rather it is the
shell per+ormig pathame e3pasio. We show it here %eca"se 67S9G character classes
ca %e "se! +or %oth.
1e%erting To Traditional Collation ?rder
5o" ca opt to ha#e yo"r system "se the tra!itioal E$SC99F collatio or!er %y
chagig the #al"e o+ the =4L@ e#iromet #aria%le. $s we saw a%o#e, the
=4L@ #aria%le cotais the ame o+ the lag"age a! character set "se! i yo"r
locale. This #al"e was origially !etermie! whe yo" selecte! a istallatio
lag"age as yo"r *i"3 was istalle!.
To see the locale settigs, "se the locale comma!,
[me@linuxbox ~]$ locale
=4L@MenO3*<3T--/
=COCTU:&M_enO3*<3T--/_
=COL3+&RACM_enO3*<3T--/_
=COTA+&M_enO3*<3T--/_
=COC==4T&M_enO3*<3T--/_
=CO+L&T4RUM_enO3*<3T--/_
=CO+&**4@&*M_enO3*<3T--/_
=CO:4:&RM_enO3*<3T--/_
=COL4+&M_enO3*<3T--/_
=CO4''R&**M_enO3*<3T--/_
=COT&=&:SL&M_enO3*<3T--/_
=CO+&4*3R&+&LTM_enO3*<3T--/_
=COA'&LTA-AC4TALM_enO3*<3T--/_
=CO4==M
To chage the locale to "se the tra!itioal )i3 %eha#iors, set the =4L@ #aria%le
to :*AV,
[me@linuxbox ~]$ export T:?Q$BSFPZ
&ote that this chage co#erts the system to "se ).S. Eglish Emore speci+ically,
$SC99F +or its character set, so %e s"re i+ this is really what yo" wat.
22/
20 8 :eg"lar E3pressios
5o" ca make this chage permaet %y a!!ig this lie to yo" yo"r <bashrc
+ile,
ex7ort =4L@M:*AV
P?SIH /asic Vs. $xtended 1egular $xpressions
J"st whe we tho"ght this co"l!It get ay more co+"sig, we !isco#er that 67S9G also
splits reg"lar e3pressio implemetatios ito two ki!s, basic regular e+pressions
H891I a! e+tended regular e+pressions H191I. The +eat"res we ha#e co#ere! so +ar are
s"pporte! %y ay applicatio that is 67S9G-compliat a! implemets J:E. 7"r gre7
program is oe s"ch program.
WhatIs the !i++erece %etwee J:E a! E:EC 9tIs a matter o+ metacharacters. With
J:E, the +ollowig metacharacters are recogiDe!,
` $ < [ ] E
$ll other characters are cosi!ere! literals. With E:E, the +ollowig metacharacters Ea!
their associate! +"ctiosF are a!!e!,
N P [ \ F 8 Q
Aowe#er Ea! this is the +" partF, the ?E@, ?F@, ?c@, a! ?d@ characters are treate! as
metacharacters i J:E if they are escape! with a %ackslash, whereas with E:E,
prece!ig ay metacharacter with a %ackslash ca"ses it to %e treate! as a literal. $y
weir!ess that comes alog will %e co#ere! i the !isc"ssios that +ollow.
Sice the +eat"res we are goig to !isc"ss e3t are part o+ E:E, we are goig to ee! to
"se a !i++eret gre7. Tra!itioally, this has %ee per+orme! %y the egre7 program, %"t
the ;&) #ersio o+ gre7 also s"pports e3te!e! reg"lar e3pressios whe the -&
optio is "se!.
P?SIH
'"rig the .980Is, )i3 %ecame a #ery pop"lar commercial operatig system, %"t
%y .988, the )i3 worl! was i t"rmoil. >ay comp"ter ma"+act"rers ha!
licese! the )i3 so"rce co!e +rom its creators, $T^T, a! were s"pplyig
#ario"s #ersios o+ the operatig system with their systems. Aowe#er, i their
e++orts to create pro!"ct !i++eretiatio, each ma"+act"rer a!!e! proprietary
chages a! e3tesios. This starte! to limit the compati%ility o+ the so+tware.
228
67S9G Jasic Rs. E3te!e! :eg"lar E3pressios
$s always with proprietary #e!ors, each was tryig to play a wiig game o+
?lock-i@ with their c"stomers. This !ark time i the history o+ )i3 is kow
to!ay as ?the 8alkani7ation.@
Eter the 9EEE E9stit"te o+ Electrical a! Electroics EgieersF. 9 the
mi!-.980s, the 9EEE %ega !e#elopig a set o+ sta!ar!s that wo"l! !e+ie how
)i3 Ea! )i3-likeF systems wo"l! per+orm. These sta!ar!s, +ormally kow
as 9EEE .00(, !e+ie the application programming interfaces E$69sF, shell a!
"tilities that are to %e +o"! o a sta!ar! )i3-like system. The ame ?67S9G,@
which sta!s +or 6ortable &perating System *nterface Ewith the ?G@ a!!e! to the
e! +or e3tra sappiessF, was s"ggeste! %y :ichar! Stallma Eyes, that :ichar!
StallmaF, a! was a!opte! %y the 9EEE.
#lternation
The +irst o+ the e3te!e! reg"lar e3pressio +eat"res we will !isc"ss is calle! alternation,
which is the +acility that allows a match to occ"r +rom amog a set o+ e3pressios. J"st
as a %racket e3pressio allows a sigle character to match +rom a set o+ speci+ie!
characters, alteratio allows matches +rom a set o+ strigs or other reg"lar e3pressios.
To !emostrate, weIll "se gre7 i co4"ctio with echo. 0irst, letIs try a plai ol!
strig match,
[me@linuxbox ~]$ echo J:::J 0 grep :::
444
[me@linuxbox ~]$ echo J<<<J 0 grep :::
[me@linuxbox ~]$
$ pretty straight+orwar! e3ample, i which we pipe the o"tp"t o+ echo ito gre7 a!
see the res"lts. Whe a match occ"rs, we see it prite! o"tH whe o match occ"rs, we
see o res"lts.
&ow weIll a!! alteratio, sigi+ie! %y the #ertical %ar metacharacter,
[me@linuxbox ~]$ echo J:::J 0 grep -G %:::0<<<%
444
[me@linuxbox ~]$ echo J<<<J 0 grep -G %:::0<<<%
HHH
[me@linuxbox ~]$ echo J===J 0 grep -G %:::0<<<%
[me@linuxbox ~]$
229
20 8 :eg"lar E3pressios
Aere we see the reg"lar e3pressio K444QHHHK which meas ?match either the strig
444 or the strig HHH.@ &otice that sice this is a e3te!e! +eat"re, we a!!e! the -&
optio to gre7 Etho"gh we co"l! ha#e 4"st "se! the egre7 program istea!F, a! we
eclose! the reg"lar e3pressio i ="otes to pre#et the shell +rom iterpretig the
#ertical %ar metacharacter as a pipe operator. $lteratio is ot limite! to two choices,
[me@linuxbox ~]$ echo J:::J 0 grep -G %:::0<<<0===%
444
To com%ie alteratio with other reg"lar e3pressio elemets, we ca "se EF to separate
the alteratio,
[me@linuxbox ~]$ grep -Gh %[4b-0g-0-ip6% dirlist*.txt
This e3pressio will match the +ileames i o"r lists that start with either ?%D@, ?gD@, or
?Dip@. Aa! we le+t o++ the paretheses, the meaig o+ this reg"lar e3pressio ,
[me@linuxbox ~]$ grep -Gh %[b-0g-0-ip% dirlist*.txt
chages to match ay +ileame that %egis with ?%D@ or contains ?gD@ or contains ?Dip@.
Auanti"iers
E3te!e! reg"lar e3pressios s"pport se#eral ways to speci+y the "m%er o+ times a
elemet is matche!.
? B :at/h An 6ement Kero <r <ne Time
This ="ati+ier meas, i e++ect, ?make the prece!ig elemet optioal.@ *etIs say we
wate! to check a phoe "m%er +or #ali!ity a! we cosi!ere! a phoe "m%er to %e
#ali! i+ it matche! either o+ these two +orms,
NnnnP nnn-nnnn
nnn nnn-nnnn
where ?n@ is a "meral. We co"l! costr"ct a reg"lar e3pressio like this,
`]NF[(-0][(-0][(-0]]PF [(-0][(-0][(-0]-[(-0][(-0][(-0][(-0]$
9 this e3pressio, we +ollow the paretheses characters with ="estio marks to i!icate
that they are to %e matche! Dero or oe time. $gai, sice the paretheses are ormally
2K0
T"ati+iers
metacharacters Ei E:EF, we prece!e them with %ackslashes to ca"se them to %e treate!
as literals istea!.
*etIs try it,
[me@linuxbox ~]$ echo J4"""6 12-A"NCJ 0 grep -G %[K4_(*-D)(*-D)(*-D)
K6_ (*-D)(*-D)(*-D)3%
N"""P #!$-%".)
[me@linuxbox ~]$ echo J""" 12-A"NCJ 0 grep -G %[K4_(*-D)(*-D)(*-D)K6
_ (*-D)(*-D)(*-D)-(*-D)(*-D)(*-D)(*-D)3%
""" #!$-%".)
[me@linuxbox ~]$ echo J::: 12-A"NCJ 0 grep -G %[K4_(*-D)(*-D)(*-D)K6
_ (*-D)(*-D)(*-D)-(*-D)(*-D)(*-D)(*-D)3%
[me@linuxbox ~]$
Aere we see that the e3pressio matches %oth +orms o+ the phoe "m%er, %"t !oes ot
match oe cotaiig o-"meric characters.
L B :at/h An 6ement Kero <r :ore Times
*ike the F metacharacter, the E is "se! to !eote a optioal itemH howe#er, "like the F,
the item may occ"r ay "m%er o+ times, ot 4"st oce. *etIs say we wate! to see i+ a
strig was a seteceH that is, it starts with a "ppercase letter, the cotais ay "m%er
o+ "pper a! lowercase letters a! spaces, a! e!s with a perio!. To match this E#ery
cr"!eF !e+iitio o+ a setece, we co"l! "se a reg"lar e3pressio like this,
[[:u77er:]][[:u77er:][:lo9er:] ]E]<
The e3pressio cosists o+ three items, a %racket e3pressio cotaiig the [:u77er:]
character class, a %racket e3pressio cotaiig %oth the [:u77er:] a! [:lo9er:]
character classes a! a space, a! a perio! escape! with a %ackslash. The seco! elemet
is traile! with a E metacharacter, so that a+ter the lea!ig "ppercase letter i o"r
setece, ay "m%er o+ "pper a! lowercase letters a! spaces may +ollow it a! still
match,
[me@linuxbox ~]$ echo J+his works.J 0 grep -G %((2upper2))((2upper2)(
2lower2) )*K.%
This 9orks<
[me@linuxbox ~]$ echo J+his Works.J 0 grep -G %((2upper2))((2upper2)(
2lower2) )*K.%
This ,orks<
[me@linuxbox ~]$ echo Jthis does notJ 0 grep -G %((2upper2))((2upper2
)(2lower2) )*K.%
[me@linuxbox ~]$
2K.
20 8 :eg"lar E3pressios
The e3pressio matches the +irst two tests, %"t ot the thir!, sice it lacks the re="ire!
lea!ig "ppercase character a! trailig perio!.
M B :at/h An 6ement <ne <r :ore Times
The 8 metacharacter works m"ch like the E, e3cept it re="ires at least oe istace o+ the
prece!ig elemet to ca"se a match. Aere is a reg"lar e3pressio that will oly match
lies cosistig o+ gro"ps o+ oe or more alpha%etic characters separate! %y sigle
spaces,
`N[[:al7ha:]]8 FP8$
[me@linuxbox ~]$ echo J+his thatJ 0 grep -G %[4((2alpha2))5 _653%
This that
[me@linuxbox ~]$ echo Ja b cJ 0 grep -G %[4((2alpha2))5 _653%
a b c
[me@linuxbox ~]$ echo Ja b DJ 0 grep -G %[4((2alpha2))5 _653%
[me@linuxbox ~]$ echo Jabc dJ 0 grep -G %[4((2alpha2))5 _653%
[me@linuxbox ~]$
We see that this e3pressio !oes ot match the lie ?a % 9@ %eca"se it cotais a o-
alpha%etic characterH or !oes it match ?a%c !@ %eca"se more tha oe space character
separates the characters ?c@ a! ?!@.
N O B :at/h An 6ement A S5e/i=i/ ,umber <= Times
The [ a! \ metacharacters are "se! to e3press miim"m a! ma3im"m "m%ers o+
re="ire! matches. They may %e speci+ie! i +o"r possi%le ways,
Table '=-3: Specifying The >umber &f !atches
Speci"ier >eaning
[n\
>atch the prece!ig elemet i+ it occ"rs e3actly n times.
[n>m\
>atch the prece!ig elemet i+ it occ"rs at least n times, %"t o
more tha m times.
[n>\
>atch the prece!ig elemet i+ it occ"rs n or more times.
[>m\
>atch the prece!ig elemet i+ it occ"rs o more tha m times.
;oig %ack to o"r earlier e3ample with the phoe "m%ers, we ca "se this metho! o+
speci+yig repetitios to simpli+y o"r origial reg"lar e3pressio +rom,
2K2
T"ati+iers
`]NF[(-0][(-0][(-0]]PF [(-0][(-0][(-0]-[(-0][(-0][(-0][(-0]$
to,
`]NF[(-0][$\]PF [(-0][$\-[(-0][%\$
*etIs try it,
[me@linuxbox ~]$ echo J4"""6 12-A"NCJ 0 grep -G %[K4_(*-D)9>K6_ (*-
D)9>-(*-D)9A>3%
N"""P #!$-%".)
[me@linuxbox ~]$ echo J""" 12-A"NCJ 0 grep -G %[K4_(*-D)9>K6_ (*-D)
9>-(*-D)9A>3%
""" #!$-%".)
[me@linuxbox ~]$ echo J"""" 12-A"NCJ 0 grep -G %[K4_(*-D)9>K6_ (*-D
)9>-(*-D)9A>3%
[me@linuxbox ~]$
$s we ca see, o"r re#ise! e3pressio ca s"ccess+"lly #ali!ate "m%ers %oth with a!
witho"t the paretheses, while re4ectig those "m%ers that are ot properly +ormatte!.
Putting 1egular $xpressions To +or(
*etIs look at some o+ the comma!s we alrea!y kow a! see how they ca %e "se! with
reg"lar e3pressios.
?aidating A 'hone List With gre7
9 o"r earlier e3ample, we looke! at sigle phoe "m%ers a! checke! them +or proper
+ormattig. $ more realistic sceario wo"l! %e checkig a list o+ "m%ers istea!, so
letIs make a list. WeIll !o this %y recitig a magical icatatio to the comma! lie. 9t
will %e magic %eca"se we ha#e ot co#ere! most o+ the comma!s i#ol#e!, %"t worry
ot. We will get there i +"t"re chapters. Aere is the icatatio,
[me@linuxbox ~]$ for i in 91..1*># do echo J439H:?1SO2*2>6 39H:?1S
O2*2>-39H:?1SO2*2A>J && phonelist.txt# done
This comma! will pro!"ce a +ile ame! 7honelist<txt cotaiig te phoe
"m%ers. Each time the comma! is repeate!, aother te "m%ers are a!!e! to the list.
We ca also chage the #al"e #( ear the %egiig o+ the comma! to pro!"ce more or
+ewer phoe "m%ers. 9+ we e3amie the cotets o+ the +ile, howe#er, we see we ha#e a
pro%lem,
2K(
20 8 :eg"lar E3pressios
[me@linuxbox ~]$ cat phonelist.txt
N!$!P !0/-!!."
N.!%P $/#-#()/
N"%(P #!.-#0/(
N/)%P #.$-!//"
N!/.P !"%-!/.(
N!0!P #(/-"#/
N#!0P %%-#$)0
N%"/P !)$-#.%!
N./.P !00-/!./
N#0/P $()-!%%(
Some o+ the "m%ers are mal+orme!, which is per+ect +or o"r p"rposes, sice we will "se
gre7 to #ali!ate them.
7e "se+"l metho! o+ #ali!atio wo"l! %e to sca the +ile +or i#ali! "m%ers a! !isplay
the res"ltig list o the !isplay,
[me@linuxbox ~]$ grep -Gv %[K4(*-D)9>K6 (*-D)9>-(*-D)9A>3%
7honelist<txt
N!0!P #(/-"#/
N#!0P %%-#$)0
[me@linuxbox ~]$
Aere we "se the -v optio to pro!"ce a i#erse match so that we will oly o"tp"t the
lies i the list that !o ot match the speci+ie! e3pressio. The e3pressio itsel+ icl"!es
the achor metacharacters at each e! to es"re that the "m%er has o e3tra characters at
either e!. This e3pressio also re="ires that the paretheses %e preset i a #ali!
"m%er, "like o"r earlier phoe "m%er e3ample.
2inding Ugy 2ienames With find
The find comma! s"pports a test %ase! o a reg"lar e3pressio. There is a importat
cosi!eratio to keep i mi! whe "sig reg"lar e3pressios i find #ers"s gre7.
Whereas gre7 will prit a lie whe the lie contains a strig that matches a
e3pressio, find re="ires that the pathame e+actly match the reg"lar e3pressio. 9 the
+ollowig e3ample, we will "se find with a reg"lar e3pressio to +i! e#ery pathame
that cotais ay character that is ot a mem%er o+ the +ollowig set,
[-O<6(-0a-C4-I]
S"ch a sca wo"l! re#eal pathames that cotai em%e!!e! spaces a! other potetially
o++esi#e characters,
2K1
6"ttig :eg"lar E3pressios To Work
[me@linuxbox ~]$ find . -regex %.*([-../*-Da--:-@).*%
'"e to the re="iremet +or a e3act match o+ the etire pathame, we "se <E at %oth e!s
o+ the e3pressio to match Dero or more istaces o+ ay character. 9 the mi!!le o+ the
e3pressio, we "se a egate! %racket e3pressio cotaiig o"r set o+ accepta%le
pathame characters.
Sear/hing 2or 2ies With locate
The locate program s"pports %oth %asic Ethe --regex7 optioF a! e3te!e! Ethe --
regex optioF reg"lar e3pressios. With it, we ca per+orm may o+ the same
operatios that we per+orme! earlier with o"r dirlist +iles,
[me@linuxbox ~]$ locate --regex %bin/4b-0g-0-ip6%
6bin6bCcat
6bin6bCcm7
6bin6bCdiff
6bin6bCegre7
6bin6bCexe
6bin6bCfgre7
6bin6bCgre7
6bin6bCi7!
6bin6bCi7!recover
6bin6bCless
6bin6bCmore
6bin6gCexe
6bin6gCi7
6usr6bin6Ci7
6usr6bin6Ci7cloak
6usr6bin6Ci7gre7
6usr6bin6Ci7info
6usr6bin6Ci7note
6usr6bin6Ci7s7lit
)sig alteratio, we per+orm a search +or pathames that cotai either bin6bC, bin6
gC, or 6bin6Ci7.
Sear/hing 2or Text In less And vim
less a! vim %oth share the same metho! o+ searchig +or te3t. 6ressig the 6 key
+ollowe! %y a reg"lar e3pressio will per+orm a search. 9+ we "se less to #iew o"r
7honelist<txt +ile,
2K2
20 8 :eg"lar E3pressios
[me@linuxbox ~]$ less phonelist.txt
The search +or o"r #ali!atio e3pressio,
N!$!P !0/-!!."
N.!%P $/#-#()/
N"%(P #!.-#0/(
N/)%P #.$-!//"
N!/.P !"%-!/.(
N!0!P #(/-"#/
N#!0P %%-#$)0
N%"/P !)$-#.%!
N./.P !00-/!./
N#0/P $()-!%%(
~
~
~
/[K4(*-D)9>K6 (*-D)9>-(*-D)9A>3
less will highlight the strigs that match, lea#ig the i#ali! oes easy to spot,
N!$!P !0/-!!."
N.!%P $/#-#()/
N"%(P #!.-#0/(
N/)%P #.$-!//"
N!/.P !"%-!/.(
N!0!P #(/-"#/
N#!0P %%-#$)0
N%"/P !)$-#.%!
N./.P !00-/!./
N#0/P $()-!%%(
~
~
~
N&L'P
vim, o the other ha!, s"pports %asic reg"lar e3pressios, so o"r search e3pressio
wo"l! look like this,
6N[(-0]][$]\P [(-0]][$]\-[(-0]][%]\
We ca see that the e3pressio is mostly the sameH howe#er, may o+ the characters that
are cosi!ere! metacharacters i e3te!e! e3pressios are cosi!ere! literals i %asic
e3pressios. They are oly treate! as metacharacters whe escape! with a %ackslash.
2KK
6"ttig :eg"lar E3pressios To Work
'epe!ig o the partic"lar co+ig"ratio o+ vim o o"r system, the matchig will %e
highlighte!. 9+ ot, try this comma! mo!e comma!,
:hlsearch
to acti#ate search highlightig.
1ote5 'epe!ig o yo"r !istri%"tio, vim may or may ot s"pport te3t search
highlightig. )%"t", i partic"lar, s"pplies a #ery strippe!-!ow #ersio o+ vim
%y !e+a"lt. 7 s"ch systems, yo" may wat to "se yo"r package maager to istall
a more complete #ersio o+ vim.
Summing -p
9 this chapter, weI#e see a +ew o+ the may "ses o+ reg"lar e3pressios. We ca +i!
e#e more i+ we "se reg"lar e3pressios to search +or a!!itioal applicatios that "se
them. We ca !o that %y searchig the ma pages,
[me@linuxbox ~]$ cd /usr/share/man/man1
[me@linuxbox man#]$ -grep -Gl %regex0regular expression% *.g-
The Cgre7 program pro#i!es a +rot e! +or gre7, allowig it to rea! compresse! +iles.
9 o"r e3ample, we search the compresse! sectio oe ma page +iles locate! i their
"s"al locatio. The res"lt o+ this comma! is a list o+ +iles cotaiig either the strig
?rege3@ or ?reg"lar e3pressio@. $s we ca see, reg"lar e3pressios show "p i a lot o+
programs.
There is oe +eat"re +o"! i %asic reg"lar e3pressios that we !i! ot co#er. Calle!
back references, this +eat"re will %e !isc"sse! i the e3t chapter.
;urther 1eading
There are may olie reso"rces +or learig reg"lar e3pressios, icl"!ig #ario"s
t"torials a! cheat sheets.
9 a!!itio, the Wikipe!ia has goo! articles o the +ollowig %ackgro"! topics,
67S9G, http,--e.wikipe!ia.org-wiki-6osi3
$SC99, http,--e.wikipe!ia.org-wiki-$scii
2K/
2. 8 Te3t 6rocessig
21 Te"t 2rocessing
$ll )i3-like operatig systems rely hea#ily o te3t +iles +or se#eral types o+ !ata
storage. So it makes sese that there are may tools +or maip"latig te3t. 9 this
chapter, we will look at programs that are "se! to ?slice a! !ice@ te3t. 9 the e3t
chapter, we will look at more te3t processig, +oc"sig o programs that are "se! to
+ormat te3t +or pritig a! other ki!s o+ h"ma cos"mptio.
This chapter will re#isit some ol! +rie!s a! itro!"ce "s to some ew oes,
cat 8 Cocateate +iles a! prit o the sta!ar! o"tp"t
sort 8 Sort lies o+ te3t +iles
uniB 8 :eport or omit repeate! lies
cut 8 :emo#e sectios +rom each lie o+ +iles
7aste 8 >erge lies o+ +iles
join 8 Joi lies o+ two +iles o a commo +iel!
comm 8 Compare two sorte! +iles lie %y lie
diff 8 Compare +iles lie %y lie
7atch 8 $pply a !i++ +ile to a origial
tr 8 Traslate or !elete characters
sed 8 Stream e!itor +or +ilterig a! tras+ormig te3t
as7ell 8 9teracti#e spell checker
#pplications ?" Text
So +ar, we ha#e leare! a co"ple o+ te3t e!itors Enano a! vimF, looke! a %"ch o+
co+ig"ratio +iles, a! ha#e witesse! the o"tp"t o+ !oDes o+ comma!s, all i te3t.
J"t what else is te3t "se! +orC 0or may thigs, it t"rs o"t.
2K8
$pplicatios 7+ Te3t
)o/uments
>ay people write !oc"mets "sig plai te3t +ormats. While it is easy to see how a
small te3t +ile co"l! %e "se+"l +or keepig simple otes, it is also possi%le to write large
!oc"mets i te3t +ormat, as well. 7e pop"lar approach is to write a large !oc"met i
a te3t +ormat a! the "se a markup language to !escri%e the +ormattig o+ the +iishe!
!oc"met. >ay scieti+ic papers are writte "sig this metho!, as )i3-%ase! te3t
processig systems were amog the +irst systems that s"pporte! the a!#ace!
typographical layo"t ee!e! %y writers i techical !isciplies.
Web 'ages
The worl!Is most pop"lar type o+ electroic !oc"met is pro%a%ly the we% page. We%
pages are te3t !oc"mets that "se either ,T!( H,yperte+t !arkup (anguageI or J!(
H1+tensible !arkup (anguageI as mark"p lag"ages to !escri%e the !oc"metIs #is"al
+ormat.
6mai
Email is a itrisically te3t-%ase! me!i"m. E#e o-te3t attachmets are co#erte!
ito a te3t represetatio +or trasmissio. We ca see this +or o"rsel#es %y !owloa!ig
a email message a! the #iewig it i less. We will see that the message %egis with
a header that !escri%es the so"rce o+ the message a! the processig it recei#e! !"rig its
4o"rey, +ollowe! %y the body o+ the message with its cotet.
'rinter <ut5ut
7 )i3-like systems, o"tp"t !estie! +or a priter is set as plai te3t or, i+ the page
cotais graphics, is co#erte! ito a te3t +ormat page description language kow as
6ostScript, which is the set to a program that geerates the graphic !ots to %e prite!.
'rogram Sour/e Code
>ay o+ the comma! lie programs +o"! o )i3-like systems were create! to s"pport
system a!miistratio a! so+tware !e#elopmet, a! te3t processig programs are o
e3ceptio. >ay o+ them are !esige! to sol#e so+tware !e#elopmet pro%lems. The
reaso te3t processig is importat to so+tware !e#elopers is that all so+tware starts o"t as
te3t. Source code, the part o+ the program the programmer act"ally writes, is always i
te3t +ormat.
1e%isiting Some ?ld ;riends
Jack i Chapter / E:e!irectioF, we leare! a%o"t some comma!s that are a%le to
2K9
2. 8 Te3t 6rocessig
accept sta!ar! ip"t i a!!itio to comma! lie arg"mets. We oly to"che! o them
%rie+ly the, %"t ow we will take a closer look at how they ca %e "se! to per+orm te3t
processig.
cat
The cat program has a "m%er o+ iterestig optios. >ay o+ them are "se! to help
%etter #is"aliDe te3t cotet. 7e e3ample is the -4 optio, which is "se! to !isplay o-
pritig characters i the te3t. There are times whe we wat to kow i+ cotrol
characters are em%e!!e! i o"r otherwise #isi%le te3t. The most commo o+ these are ta%
characters Eas oppose! to spacesF a! carriage ret"rs, o+te preset as e!-o+-lie
characters i >S-'7S style te3t +iles. $other commo sit"atio is a +ile cotaiig
lies o+ te3t with trailig spaces.
*etIs create a test +ile "sig cat as a primiti#e wor! processor. To !o this, weIll 4"st
eter the comma! cat Ealog with speci+yig a +ile +or re!irecte! o"tp"tF a! type o"r
te3t, +ollowe! %y &nter to properly e! the lie, the Ctrl-d, to i!icate to cat that
we ha#e reache! e!-o+-+ile. 9 this e3ample, we eter a lea!ig ta% character a! +ollow
the lie with some trailig spaces,
[me@linuxbox ~]$ cat & foo.txt
The Buick bro9n fox jum7ed over the laC1 dog<
[me@linuxbox ~]$
&e3t, we will "se cat with the -4 optio to !isplay the te3t,
[me@linuxbox ~]$ cat -: foo.txt
`AThe Buick bro9n fox jum7ed over the laC1 dog< $
[me@linuxbox ~]$
$s we ca see i the res"lts, the ta% character i o"r te3t is represete! %y `A. This is a
commo otatio that meas ?Cotrol-9@ which, as it t"rs o"t, is the same as a ta%
character. We also see that a $ appears at the tr"e e! o+ the lie, i!icatig that o"r te3t
cotais trailig spaces.
2/0
:e#isitig Some 7l! 0rie!s
>SED?S Text Vs. -nix Text
7e o+ the reasos yo" may wat to "se cat to look +or o-pritig characters
i te3t is to spot hi!!e carriage ret"rs. Where !o hi!!e carriage ret"rs come
+romC '7S a! Wi!owsL )i3 a! '7S !oIt !e+ie the e! o+ a lie the
same way i te3t +iles. )i3 e!s a lie with a lie+ee! character E$SC99 .0F
while >S-'7S a! its !eri#ati#es "se the se="ece carriage ret"r E$SC99 .(F
a! lie+ee! to termiate each lie o+ te3t.
There are a se#eral ways to co#ert +iles +rom '7S to )i3 +ormat. 7 may
*i"3 systems, there are programs calle! dos!unix a! unix!dos, which ca
co#ert te3t +iles to a! +rom '7S +ormat. Aowe#er, i+ yo" !oIt ha#e
dos!unix o yo"r system, !oIt worry. The process o+ co#ertig te3t +rom
'7S to )i3 +ormat is #ery simpleH it simply i#ol#es the remo#al o+ the
o++e!ig carriage ret"rs. That is easily accomplishe! %y a co"ple o+ the
programs !isc"sse! later i this chapter.
cat also has optios that are "se! to mo!i+y te3t. The two most promiet are -n,
which "m%ers lies, a! -s, which s"ppresses the o"tp"t o+ m"ltiple %lak lies. We
ca !emostrate th"sly,
[me@linuxbox ~]$ cat & foo.txt
The Buick bro9n fox
jum7ed over the laC1 dog<
[me@linuxbox ~]$ cat -ns foo.txt
# The Buick bro9n fox
!
$ jum7ed over the laC1 dog<
[me@linuxbox ~]$
9 this e3ample, we create a ew #ersio o+ o"r foo<txt test +ile, which cotais two
lies o+ te3t separate! %y two %lak lies. $+ter processig %y cat with the -ns optios,
the e3tra %lak lie is remo#e! a! the remaiig lies are "m%ere!. While this is ot
m"ch o+ a process to per+orm o te3t, it is a process.
sort
The sort program sorts the cotets o+ sta!ar! ip"t, or oe or more +iles speci+ie! o
the comma! lie, a! se!s the res"lts to sta!ar! o"tp"t. )sig the same techi="e
2/.
2. 8 Te3t 6rocessig
that we "se! with cat, we ca !emostrate processig o+ sta!ar! ip"t !irectly +rom
the key%oar!,
[me@linuxbox ~]$ sort & foo.txt
c
b
a
[me@linuxbox ~]$ cat foo.txt
a
b
c
$+ter eterig the comma!, we type the letters ?c@, ?%@, a! ?a@, +ollowe! oce agai %y
Ctrl-d to i!icate e!-o+-+ile. We the #iew the res"ltig +ile a! see that the lies
ow appear i sorte! or!er.
Sice sort ca accept m"ltiple +iles o the comma! lie as arg"mets, it is possi%le to
merge m"ltiple +iles ito a sigle sorte! whole. 0or e3ample, i+ we ha! three te3t +iles
a! wate! to com%ie them ito a sigle sorte! +ile, we co"l! !o somethig like this,
sort file1.txt file2.txt file.txt & final.sorted.list.txt
sort has se#eral iterestig optios. Aere is a partial list,
Table '1-1: %ommon sort &ptions
?ption Long ?ption Description
-b --ignore-leading-blanks
Jy !e+a"lt, sortig is per+orme!
o the etire lie, startig with the
+irst character i the lie. This
optio ca"ses sort to igore
lea!ig spaces i lies a!
calc"lates sortig %ase! o the +irst
o-whitespace character o the
lie.
-f --ignore-case
>akes sortig case isesiti#e.
-n --numeric-sort
6er+orms sortig %ase! o the
"meric e#al"atio o+ a strig.
)sig this optio allows sortig to
%e per+orme! o "meric #al"es
2/2
:e#isitig Some 7l! 0rie!s
rather tha alpha%etic #al"es.
-r --reverse
Sort i re#erse or!er. :es"lts are
i !esce!ig rather tha
asce!ig or!er.
-k --ke1Mfield1[>field2]
Sort %ase! o a key +iel! locate!
+rom field1 to field' rather tha the
etire lie. See !isc"ssio %elow.
-m --merge
Treat each each arg"met as the
ame o+ a presorte! +ile. >erge
m"ltiple +iles ito a sigle sorte!
res"lt witho"t per+ormig ay
a!!itioal sortig.
-o --out7utMfile
Se! sorte! o"tp"t to file rather
tha sta!ar! o"tp"t.
-t --field-se7aratorMchar
'e+ie the +iel! separator
character. Jy !e+a"lt +iel!s are
separate! %y spaces or ta%s.
$ltho"gh most o+ the optios a%o#e are pretty sel+-e3plaatory, some are ot. 0irst, letIs
look at the -n optio, "se! +or "meric sortig. With this optio, it is possi%le to sort
#al"es %ase! o "meric #al"es. We ca !emostrate this %y sortig the res"lts o+ the du
comma! to !etermie the largest "sers o+ !isk space. &ormally, the du comma! lists
the res"lts o+ a s"mmary i pathame or!er,
[me@linuxbox ~]$ du -s /usr/share/* 0 head
!"! 6usr6share6aclocal
0. 6usr6share6ac7i-su77ort
/ 6usr6share6adduser
#0. 6usr6share6alacarte
$%% 6usr6share6alsa
/ 6usr6share6alsa-base
#!%// 6usr6share6anth1
/ 6usr6share6a7md
!#%%( 6usr6share6a77-install
%/ 6usr6share6a77lication-registr1
9 this e3ample, we pipe the res"lts ito head to limit the res"lts to the +irst te lies.
We ca pro!"ce a "merically sorte! list to show the te largest cos"mers o+ space this
way,
2/(
2. 8 Te3t 6rocessig
[me@linuxbox ~]$ du -s /usr/share/* 0 sort -nr 0 head
"(00%( 6usr6share6locale-lang7ack
!%!..( 6usr6share6doc
#0)".( 6usr6share6fonts
#)0#%% 6usr6share6gnome
#%.).% 6usr6share6m1s7ell
#%%$(% 6usr6share6gim7
#$"//( 6usr6share6dict
)."(/ 6usr6share6icons
./()! 6usr6share6a77s
.!/%% 6usr6share6foomatic
Jy "sig the -nr optios, we pro!"ce a re#erse "merical sort, with the largest #al"es
appearig +irst i the res"lts. This sort works %eca"se the "merical #al"es occ"r at the
%egiig o+ each lie. J"t what i+ we wat to sort a list %ase! o some #al"e +o"!
withi the lieC 0or e3ample, the res"lts o+ a ls -l,
[me@linuxbox ~]$ ls -l /usr/bin 0 head
total #"!0%/
-r9xr-xr-x # root root $%/!% !((/-(%-(% (!:%! [
-r9xr-xr-x # root root #(#"". !(()-##-!) (.:(/ a!7
-r9xr-xr-x # root root #$($. !((/-(!-!) (/:!! aconnect
-r9xr-xr-x # root root #(""! !(()-(/-#" #(:$% ac7i
-r9xr-xr-x # root root $/(( !((/-(%-#% ($:"# ac7iOfakeke1
-r9xr-xr-x # root root )"$. !((/-(%-#0 ((:#0 ac7iOlisten
-r9xr-xr-x # root root $"). !((/-(%-!0 ():") add7art
-r9xr-xr-x # root root !(/(/ !((/-(#-($ #/:(! addr!line
-r9xr-xr-x # root root %/0)(% !((/-#(-(0 #):(! ade7tObatch
9gorig, +or the momet, that ls ca sort its res"lts %y siDe, we co"l! "se sort to sort
this list %y +ile siDe, as well,
[me@linuxbox ~]$ ls -l /usr/bin 0 sort -nr -k " 0 head
-r9xr-xr-x # root root /!$%!#. !((/-(%-() #):%! inksca7e
-r9xr-xr-x # root root /!!!.0! !((/-(%-() #):%! inkvie9
-r9xr-xr-x # root root $)%."(/ !((/-($-() !$:%" gim7-!<%
-r9xr-xr-x # root root $."%(!( !((/-(/-!. #.:#. Buanta
-r9xr-xr-x # root root !0!/).( !((/-(0-#( #%:$# gdbtui
-r9xr-xr-x # root root !0!/)". !((/-(0-#( #%:$# gdb
-r9xr-xr-x # root root !.(!!$. !((/-#(-#( #!:". net
-r9xr-xr-x # root root !$(%./% !((/-#(-#( #!:". r7cclient
-r9xr-xr-x # root root !!%#/$! !((/-(%-(% (":". a7titude
-r9xr-xr-x # root root !!(!%). !((/-#(-#( #!:". smbcacls
2/1
:e#isitig Some 7l! 0rie!s
>ay "ses o+ sort i#ol#e the processig o+ tabular data, s"ch as the res"lts o+ the ls
comma! a%o#e. 9+ we apply !ata%ase termiology to the ta%le a%o#e, we wo"l! say that
each row is a record a! that each recor! cosists o+ m"ltiple fields, s"ch as the +ile
attri%"tes, lik co"t, +ileame, +ile siDe a! so o. sort is a%le to process i!i#i!"al
+iel!s. 9 !ata%ase terms, we are a%le to speci+y oe or more key fields to "se as sort keys.
9 the e3ample a%o#e, we speci+y the n a! r optios to per+orm a re#erse "merical sort
a! speci+y -k " to make sort "se the +i+th +iel! as the key +or sortig.
The k optio is #ery iterestig a! has may +eat"res, %"t +irst we ee! to talk a%o"t
how sort !e+ies +iel!s. *etIs cosi!er a #ery simple te3t +ile cosistig o+ a sigle lie
cotaiig the a"thorIs ame,
,illiam *hotts
Jy !e+a"lt, sort sees this lie as ha#ig two +iel!s. The +irst +iel! cotais the
characters,
c,illiamd
a! the seco! +iel! cotais the characters,
c *hottsd
meaig that whitespace characters Espaces a! ta%sF are "se! as !elimiters %etwee
+iel!s a! that the !elimiters are icl"!e! i the +iel! whe sortig is per+orme!.
*ookig agai at a lie +rom o"r ls o"tp"t, we ca see that a lie cotais eight +iel!s
a! that the +i+th +iel! is the +ile siDe,
-r9xr-xr-x # root root /!$%!#. !((/-(%-() #):%! inksca7e
0or o"r e3t series o+ e3perimets, letIs cosi!er the +ollowig +ile cotaiig the history
o+ three pop"lar *i"3 !istri%"tios release! +rom 200K to 2008. Each lie i the +ile has
three +iel!s, the !istri%"tio ame, #ersio "m%er, a! !ate o+ release i
>>-''-5555 +ormat,
*3*& #(<! #!6()6!((.
-edora #( ##6!"6!((/
*3*& ##<( (.6#06!((/
3buntu /<(% (%6!%6!((/
-edora / ##6(/6!(()
*3*& #(<$ #(6(%6!(()
2/2
2. 8 Te3t 6rocessig
3buntu .<#( #(6!.6!((.
-edora ) ("6$#6!(()
3buntu )<#( #(6#/6!(()
3buntu )<(% (%6#06!(()
*3*& #(<# ("6##6!((.
-edora . #(6!%6!((.
-edora 0 ("6#$6!((/
3buntu .<(. (.6(#6!((.
3buntu /<#( #(6$(6!((/
-edora " ($6!(6!((.
)sig a te3t e!itor Eperhaps vimF, weIll eter this !ata a! ame the res"ltig +ile
distros<txt.
&e3t, weIll try sortig the +ile a! o%ser#e the res"lts,
[me@linuxbox ~]$ sort distros.txt
-edora #( ##6!"6!((/
-edora " ($6!(6!((.
-edora . #(6!%6!((.
-edora ) ("6$#6!(()
-edora / ##6(/6!(()
-edora 0 ("6#$6!((/
*3*& #(<# ("6##6!((.
*3*& #(<! #!6()6!((.
*3*& #(<$ #(6(%6!(()
*3*& ##<( (.6#06!((/
3buntu .<(. (.6(#6!((.
3buntu .<#( #(6!.6!((.
3buntu )<(% (%6#06!(()
3buntu )<#( #(6#/6!(()
3buntu /<(% (%6!%6!((/
3buntu /<#( #(6$(6!((/
Well, it mostly worke!. The pro%lem occ"rs i the sortig o+ the 0e!ora #ersio
"m%ers. Sice a ?.@ comes %e+ore a ?2@ i the character set, #ersio ?.0@ e!s "p at the
top while #ersio ?9@ +alls to the %ottom.
To +i3 this pro%lem we are goig to ha#e to sort o m"ltiple keys. We wat to per+orm a
alpha%etic sort o the +irst +iel! a! the a "meric sort o the thir! +iel!. sort allows
m"ltiple istaces o+ the -k optio so that m"ltiple sort keys ca %e speci+ie!. 9 +act, a
key may icl"!e a rage o+ +iel!s. 9+ o rage is speci+ie! Eas has %ee the case with o"r
pre#io"s e3amplesF, sort "ses a key that %egis with the speci+ie! +iel! a! e3te!s to
the e! o+ the lie. Aere is the syta3 +or o"r m"lti-key sort,
2/K
:e#isitig Some 7l! 0rie!s
[me@linuxbox ~]$ sort --ke!$1;1 --ke!$2n distros.txt
-edora " ($6!(6!((.
-edora . #(6!%6!((.
-edora ) ("6$#6!(()
-edora / ##6(/6!(()
-edora 0 ("6#$6!((/
-edora #( ##6!"6!((/
*3*& #(<# ("6##6!((.
*3*& #(<! #!6()6!((.
*3*& #(<$ #(6(%6!(()
*3*& ##<( (.6#06!((/
3buntu .<(. (.6(#6!((.
3buntu .<#( #(6!.6!((.
3buntu )<(% (%6#06!(()
3buntu )<#( #(6#/6!(()
3buntu /<(% (%6!%6!((/
3buntu /<#( #(6$(6!((/
Tho"gh we "se! the log +orm o+ the optio +or clarity, -k #># -k !n wo"l! %e
e3actly e="i#alet. 9 the +irst istace o+ the key optio, we speci+ie! a rage o+ +iel!s
to icl"!e i the +irst key. Sice we wate! to limit the sort to 4"st the +irst +iel!, we
speci+ie! #># which meas ?start at +iel! oe a! e! at +iel! oe.@ 9 the seco!
istace, we speci+ie! !n, which meas that +iel! two is the sort key a! that the sort
sho"l! %e "meric. $ optio letter may %e icl"!e! at the e! o+ a key speci+ier to
i!icate the type o+ sort to %e per+orme!. These optio letters are the same as the glo%al
optios +or the sort program, b Eigore lea!ig %laksF, n E"meric sortF, r Ere#erse
sortF, a! so o.
The thir! +iel! i o"r list cotais a !ate i a ico#eiet +ormat +or sortig. 7
comp"ters, !ates are "s"ally +ormatte! i 5555->>-'' or!er to make chroological
sortig easy, %"t o"rs are i the $merica +ormat o+ >>-''-5555. Aow ca we sort
this list i chroological or!erC
0ort"ately, sort pro#i!es a way. The key optio allows speci+icatio o+ offsets withi
+iel!s, so we ca !e+ie keys withi +iel!s,
[me@linuxbox ~]$ sort -k .Cnbr -k .1nbr -k .Anbr distros.txt
-edora #( ##6!"6!((/
3buntu /<#( #(6$(6!((/
*3*& ##<( (.6#06!((/
-edora 0 ("6#$6!((/
3buntu /<(% (%6!%6!((/
-edora / ##6(/6!(()
3buntu )<#( #(6#/6!(()
*3*& #(<$ #(6(%6!(()
2//
2. 8 Te3t 6rocessig
-edora ) ("6$#6!(()
3buntu )<(% (%6#06!(()
*3*& #(<! #!6()6!((.
3buntu .<#( #(6!.6!((.
-edora . #(6!%6!((.
3buntu .<(. (.6(#6!((.
*3*& #(<# ("6##6!((.
-edora " ($6!(6!((.
Jy speci+yig -k $<) we istr"ct sort to "se a sort key that %egis at the se#eth
character withi the thir! +iel!, which correspo!s to the start o+ the year. *ikewise, we
speci+y -k $<# a! -k $<% to isolate the moth a! !ay portios o+ the !ate. We also
a!! the n a! r optios to achie#e a re#erse "meric sort. The b optio is icl"!e! to
s"ppress the lea!ig spaces Ewhose "m%ers #ary +rom lie to lie, there%y a++ectig the
o"tcome o+ the sortF i the !ate +iel!.
Some +iles !oIt "se ta%s a! spaces as +iel! !elimitersH +or e3ample, the 6etc67ass9d
+ile,
[me@linuxbox ~]$ head /etc/passwd
root:x:(:(:root:6root:6bin6bash
daemon:x:#:#:daemon:6usr6sbin:6bin6sh
bin:x:!:!:bin:6bin:6bin6sh
s1s:x:$:$:s1s:6dev:6bin6sh
s1nc:x:%:.""$%:s1nc:6bin:6bin6s1nc
games:x:":.(:games:6usr6games:6bin6sh
man:x:.:#!:man:6var6cache6man:6bin6sh
l7:x:):):l7:6var6s7ool6l7d:6bin6sh
mail:x:/:/:mail:6var6mail:6bin6sh
ne9s:x:0:0:ne9s:6var6s7ool6ne9s:6bin6sh
The +iel!s i this +ile are !elimite! with colos E:F, so how wo"l! we sort this +ile "sig a
key +iel!C sort pro#i!es the -t optio to !e+ie the +iel! separator character. To sort
the 7ass9d +ile o the se#eth +iel! Ethe acco"tIs !e+a"lt shellF, we co"l! !o this,
[me@linuxbox ~]$ sort -t %2% -k C /etc/passwd 0 head
me:x:#((#:#((#:+1self>>>:6home6me:6bin6bash
root:x:(:(:root:6root:6bin6bash
dhc7:x:#(#:#(!::6nonexistent:6bin6false
gdm:x:#(.:##%:@nome 'is7la1 +anager:6var6lib6gdm:6bin6false
h7li7:x:#(%:):S:=A: s1stem user>>>:6var6run6h7li7:6bin6false
klog:x:#($:#(%::6home6klog:6bin6false
messagebus:x:#(/:##0::6var6run6dbus:6bin6false
2/8
:e#isitig Some 7l! 0rie!s
7olkituser:x:##(:#!!::olic12it>>>:6var6run6:olic12it:6bin6false
7ulse:x:#():##.::ulse4udio daemon>>>:6var6run67ulse:6bin6false
Jy speci+yig the colo character as the +iel! separator, we ca sort o the se#eth +iel!.
uniB
Compare! to sort, the uniB program is a lightweight. uniB per+orms a seemigly
tri#ial task. Whe gi#e a sorte! +ile Eicl"!ig sta!ar! ip"tF, it remo#es ay !"plicate
lies a! se!s the res"lts to sta!ar! o"tp"t. 9t is o+te "se! i co4"ctio with sort
to clea the o"tp"t o+ !"plicates.
Ti(5 While uniB is a tra!itioal )i3 tool o+te "se! with sort, the ;&) #ersio
o+ sort s"pports a -u optio, which remo#es !"plicates +rom the sorte! o"tp"t.
*etIs make a te3t +ile to try this o"t,
[me@linuxbox ~]$ cat & foo.txt
a
b
c
a
b
c
:emem%er to type Ctrl-d to termiate sta!ar! ip"t. &ow, i+ we r" uniB o o"r
te3t +ile,
[me@linuxbox ~]$ uni, foo.txt
a
b
c
a
b
c
the res"lts are o !i++eret +rom o"r origial +ileH the !"plicates were ot remo#e!. 0or
uniB to act"ally !o its 4o%, the ip"t m"st %e sorte! +irst,
2/9
2. 8 Te3t 6rocessig
[me@linuxbox ~]$ sort foo.txt 0 uni,
a
b
c
This is %eca"se uniB oly remo#es !"plicate lies which are a!4acet to each other.
uniB has se#eral optios. Aere are the commo oes,
Table '1-': %ommon uni< &ptions
?ption Description
-c
7"tp"t a list o+ !"plicate lies prece!e! %y the "m%er o+ times the
lie occ"rs.
-d
7ly o"tp"t repeate! lies, rather tha "i="e lies.
-f n
9gore n lea!ig +iel!s i each lie. 0iel!s are separate! %y
whitespace as they are i sortH howe#er, "like sort, uniB has
o optio +or settig a alterate +iel! separator.
-i
9gore case !"rig the lie comparisos.
-s n
Skip EigoreF the lea!ig n characters o+ each lie.
-u
7ly o"tp"t "i="e lies. This is the !e+a"lt.
Aere we see uniB "se! to report the "m%er o+ !"plicates +o"! i o"r te3t +ile, "sig
the -c optio,
[me@linuxbox ~]$ sort foo.txt 0 uni, -c
! a
! b
! c
Slicing #nd Dicing
The e3t three programs we will !isc"ss are "se! to peel col"ms o+ te3t o"t o+ +iles a!
recom%ie them i "se+"l ways.
cut
The cut program is "se! to e3tract a sectio o+ te3t +rom a lie a! o"tp"t the e3tracte!
280
Slicig $! 'icig
sectio to sta!ar! o"tp"t. 9t ca accept m"ltiple +ile arg"mets or ip"t +rom sta!ar!
ip"t.
Speci+yig the sectio o+ the lie to %e e3tracte! is somewhat awkwar! a! is speci+ie!
"sig the +ollowig optios,
Table '1-3: cut Selection &ptions
?ption Description
-c char_list
E3tract the portio o+ the lie !e+ie! %y charlist. The list
may cosist o+ oe or more comma-separate! "merical
rages.
-f field_list
E3tract oe or more +iel!s +rom the lie as !e+ie! %y
fieldlist. The list may cotai oe or more +iel!s or +iel!
rages separate! %y commas.
-d delim_char
Whe -f is speci+ie!, "se delimchar as the +iel! !elimitig
character. Jy !e+a"lt, +iel!s m"st %e separate! %y a sigle ta%
character.
--com7lement
E3tract the etire lie o+ te3t, e3cept +or those portios
speci+ie! %y -c a!-or -f.
$s we ca see, the way cut e3tracts te3t is rather i+le3i%le. cut is %est "se! to e3tract
te3t +rom +iles that are pro!"ce! %y other programs, rather tha te3t !irectly type! %y
h"mas. WeIll take a look at o"r distros<txt +ile to see i+ it is ?clea@ eo"gh to %e
a goo! specime +or o"r cut e3amples. 9+ we "se cat with the -4 optio, we ca see i+
the +ile meets o"r re="iremets o+ ta% separate! +iel!s,
[me@linuxbox ~]$ cat -: distros.txt
*3*&`A#(<!`A#!6()6!((.$
-edora`A#(`A##6!"6!((/$
*3*&`A##<(`A(.6#06!((/$
3buntu`A/<(%`A(%6!%6!((/$
-edora`A/`A##6(/6!(()$
*3*&`A#(<$`A#(6(%6!(()$
3buntu`A.<#(`A#(6!.6!((.$
-edora`A)`A("6$#6!(()$
3buntu`A)<#(`A#(6#/6!(()$
3buntu`A)<(%`A(%6#06!(()$
*3*&`A#(<#`A("6##6!((.$
-edora`A.`A#(6!%6!((.$
-edora`A0`A("6#$6!((/$
28.
2. 8 Te3t 6rocessig
3buntu`A.<(.`A(.6(#6!((.$
3buntu`A/<#(`A#(6$(6!((/$
-edora`A"`A($6!(6!((.$
9t looks goo!. &o em%e!!e! spaces, 4"st sigle ta% characters %etwee the +iel!s. Sice
the +ile "ses ta%s rather tha spaces, weIll "se the -f optio to e3tract a +iel!,
[me@linuxbox ~]$ cut -f distros.txt
#!6()6!((.
##6!"6!((/
(.6#06!((/
(%6!%6!((/
##6(/6!(()
#(6(%6!(()
#(6!.6!((.
("6$#6!(()
#(6#/6!(()
(%6#06!(()
("6##6!((.
#(6!%6!((.
("6#$6!((/
(.6(#6!((.
#(6$(6!((/
($6!(6!((.
Jeca"se o"r distros +ile is ta%-!elimite!, it is %est to "se cut to e3tract +iel!s rather
tha characters. This is %eca"se whe a +ile is ta%-!elimite!, it is "likely that each lie
will cotai the same "m%er o+ characters, which makes calc"latig character positios
withi the lie !i++ic"lt or impossi%le. 9 o"r e3ample a%o#e, howe#er, we ow ha#e
e3tracte! a +iel! that l"ckily cotais !ata o+ i!etical legth, so we ca show how
character e3tractio works %y e3tractig the year +rom each lie,
[me@linuxbox ~]$ cut -f distros.txt 0 cut -c C-1*
!((.
!((/
!((/
!((/
!(()
!(()
!((.
!(()
!(()
!(()
!((.
282
Slicig $! 'icig
!((.
!((/
!((.
!((/
!((.
Jy r"ig cut a seco! time o o"r list, we are a%le to e3tract character positios /
thro"gh .0, which correspo!s to the year i o"r !ate +iel!. The )-#( otatio is a
e3ample o+ a rage. The cut ma page cotais a complete !escriptio o+ how rages
ca %e speci+ie!.
$xpanding Ta0s
7"r distros<txt +ile is i!eally +ormatte! +or e3tractig +iel!s "sig cut. J"t
what i+ we wate! a +ile that co"l! %e +"lly maip"late! with cut %y characters,
rather tha +iel!sC This wo"l! re="ire "s to replace the ta% characters withi the
+ile with the correspo!ig "m%er o+ spaces. 0ort"ately, the ;&) Core"tils
package icl"!es a tool +or that. &ame! ex7and, this program accepts either
oe or more +ile arg"mets or sta!ar! ip"t, a! o"tp"ts the mo!i+ie! te3t to
sta!ar! o"tp"t.
9+ we process o"r distros<txt +ile with ex7and, we ca "se the cut -c to
e3tract ay rage o+ characters +rom the +ile. 0or e3ample, we co"l! "se the
+ollowig comma! to e3tract the year o+ release +rom o"r list, %y e3pa!ig the
+ile a! "sig cut to e3tract e#ery character +rom the twety-thir! positio to the
e! o+ the lie,
[me@linuxbox ~]$ expand distros.txt 0 cut -c 2-
Core"tils also pro#i!es the unex7and program to s"%stit"te ta%s +or spaces.
Whe workig with +iel!s, it is possi%le to speci+y a !i++eret +iel! !elimiter rather tha
the ta% character. Aere we will e3tract the +irst +iel! +rom the 6etc67ass9d +ile,
[me@linuxbox ~]$ cut -d %2% -f 1 /etc/passwd 0 head
root
daemon
bin
s1s
s1nc
games
28(
2. 8 Te3t 6rocessig
man
l7
mail
ne9s
)sig the -d optio, we are a%le to speci+y the colo character as the +iel! !elimiter.
7aste
The 7aste comma! !oes the opposite o+ cut. :ather tha e3tractig a col"m o+ te3t
+rom a +ile, it a!!s oe or more col"ms o+ te3t to a +ile. 9t !oes this %y rea!ig m"ltiple
+iles a! com%iig the +iel!s +o"! i each +ile ito a sigle stream o sta!ar! o"tp"t.
*ike cut, 7aste accepts m"ltiple +ile arg"mets a!-or sta!ar! ip"t. To !emostrate
how 7aste operates, we will per+orm some s"rgery o o"r distros<txt +ile to
pro!"ce a chroological list o+ releases.
0rom o"r earlier work with sort, we will +irst pro!"ce a list o+ !istros sorte! %y !ate
a! store the res"lt i a +ile calle! distros-b1-date<txt,
[me@linuxbox ~]$ sort -k .Cnbr -k .1nbr -k .Anbr distros.txt & dis
tros-b!-date.txt
&e3t, we will "se cut to e3tract the +irst two +iel!s +rom the +ile Ethe !istro ame a!
#ersioF, a! store that res"lt i a +ile ame! distro-versions<txt,
[me@linuxbox ~]$ cut -f 1;2 distros-b!-date.txt & distros-versions.t
xt
[me@linuxbox ~]$ head distros-versions.txt
-edora #(
3buntu /<#(
*3*& ##<(
-edora 0
3buntu /<(%
-edora /
3buntu )<#(
*3*& #(<$
-edora )
3buntu )<(%
The +ial piece o+ preparatio is to e3tract the release !ates a! store them a +ile ame!
distro-dates<txt,
281
Slicig $! 'icig
[me@linuxbox ~]$ cut -f distros-b!-date.txt & distros-dates.txt
[me@linuxbox ~]$ head distros-dates.txt
##6!"6!((/
#(6$(6!((/
(.6#06!((/
("6#$6!((/
(%6!%6!((/
##6(/6!(()
#(6#/6!(()
#(6(%6!(()
("6$#6!(()
(%6#06!(()
We ow ha#e the parts we ee!. To complete the process, "se 7aste to p"t the col"m
o+ !ates ahea! o+ the !istro ames a! #ersios, th"s creatig a chroological list. This is
!oe simply %y "sig 7aste a! or!erig its arg"mets i the !esire! arragemet,
[me@linuxbox ~]$ paste distros-dates.txt distros-versions.txt
##6!"6!((/ -edora #(
#(6$(6!((/ 3buntu /<#(
(.6#06!((/ *3*& ##<(
("6#$6!((/ -edora 0
(%6!%6!((/ 3buntu /<(%
##6(/6!(() -edora /
#(6#/6!(() 3buntu )<#(
#(6(%6!(() *3*& #(<$
("6$#6!(() -edora )
(%6#06!(() 3buntu )<(%
#!6()6!((. *3*& #(<!
#(6!.6!((. 3buntu .<#(
#(6!%6!((. -edora .
(.6(#6!((. 3buntu .<(.
("6##6!((. *3*& #(<#
($6!(6!((. -edora "
join
9 some ways, join is like 7aste i that it a!!s col"ms to a +ile, %"t it "ses a "i="e
way to !o it. $ Aoin is a operatio "s"ally associate! with relational databases where
!ata +rom m"ltiple tables with a share! key +iel! is com%ie! to +orm a !esire! res"lt.
The join program per+orms the same operatio. 9t 4ois !ata +rom m"ltiple +iles %ase!
o a share! key +iel!.
To see how a 4oi operatio is "se! i a relatioal !ata%ase, letIs imagie a #ery small
!ata%ase cosistig o+ two ta%les each cotaiig a sigle recor!. The +irst ta%le, calle!
282
2. 8 Te3t 6rocessig
C)ST7>E:S, has three +iel!s, a c"stomer "m%er EC)ST&)>F, the c"stomerIs +irst
ame E0&$>EF a! the c"stomerIs last ame E*&$>EF,
C3*TL3+ -L4+& =L4+&
MMMMMMMM MMMMM MMMMMM
%./#0$% ?ohn *mith
The seco! ta%le is calle! 7:'E:S a! cotais +o"r +iel!s, a or!er "m%er
E7:'E:&)>F, the c"stomer "m%er EC)ST&)>F, the ="atity ET)$&F, a! the item
or!ere! E9TE>F.
R'&RL3+ C3*TL3+ Z34L AT&+
MMMMMMMM MMMMMMM MMMM MMMM
$(#%0"$$(" %./#0$% # Hlue ,idget
&ote that %oth ta%les share the +iel! C)ST&)>. This is importat, as it allows a
relatioship %etwee the ta%les.
6er+ormig a 4oi operatio wo"l! allow "s to com%ie the +iel!s i the two ta%les to
achie#e a "se+"l res"lt, s"ch as preparig a i#oice. )sig the matchig #al"es i the
C)ST&)> +iel!s o+ %oth ta%les, a 4oi operatio co"l! pro!"ce the +ollowig,
-L4+& =L4+& Z34L AT&+
MMMMM MMMMM MMMM MMMM
?ohn *mith # Hlue ,idget
To !emostrate the join program, weIll ee! to make a co"ple o+ +iles with a share!
key. To !o this, we will "se o"r distros-b1-date<txt +ile. 0rom this +ile, we will
costr"ct two a!!itioal +iles, oe cotaiig the release !ate Ewhich will %e o"r share!
key +or this !emostratioF a! the release ame,
[me@linuxbox ~]$ cut -f 1;1 distros-b!-date.txt & distros-names.txt
[me@linuxbox ~]$ 7aste distros-dates.txt distros-names.txt & distros-
ke!-names.txt
[me@linuxbox ~]$ head distros-ke!-names.txt
##6!"6!((/ -edora
#(6$(6!((/ 3buntu
(.6#06!((/ *3*&
("6#$6!((/ -edora
(%6!%6!((/ 3buntu
##6(/6!(() -edora
#(6#/6!(() 3buntu
#(6(%6!(() *3*&
("6$#6!(() -edora
(%6#06!(() 3buntu
a! the seco! +ile, which cotais the release !ates a! the #ersio "m%ers,
28K
Slicig $! 'icig
[me@linuxbox ~]$ cut -f 2;2 distros-b!-date.txt & distros-vernums.txt
[me@linuxbox ~]$ paste distros-dates.txt distros-vernums.txt & distro
s-ke!-vernums.txt
[me@linuxbox ~]$ head distros-ke!-vernums.txt
##6!"6!((/ #(
#(6$(6!((/ /<#(
(.6#06!((/ ##<(
("6#$6!((/ 0
(%6!%6!((/ /<(%
##6(/6!(() /
#(6#/6!(() )<#(
#(6(%6!(() #(<$
("6$#6!(() )
(%6#06!(() )<(%
We ow ha#e two +iles with a share! key Ethe ?release !ate@ +iel!F. 9t is importat to
poit o"t that the +iles m"st %e sorte! o the key +iel! +or join to work properly.
[me@linuxbox ~]$ join distros-ke!-names.txt distros-ke!-vernums.txt
0 head
##6!"6!((/ -edora #(
#(6$(6!((/ 3buntu /<#(
(.6#06!((/ *3*& ##<(
("6#$6!((/ -edora 0
(%6!%6!((/ 3buntu /<(%
##6(/6!(() -edora /
#(6#/6!(() 3buntu )<#(
#(6(%6!(() *3*& #(<$
("6$#6!(() -edora )
(%6#06!(() 3buntu )<(%
&ote also that, %y !e+a"lt, join "ses whitespace as the ip"t +iel! !elimiter a! a sigle
space as the o"tp"t +iel! !elimiter. This %eha#ior ca %e mo!i+ie! %y speci+yig optios.
See the join ma page +or !etails.
Comparing Text
9t is o+te "se+"l to compare #ersios o+ te3t +iles. 0or system a!miistrators a!
so+tware !e#elopers, this is partic"larly importat. $ system a!miistrator may, +or
e3ample, ee! to compare a e3istig co+ig"ratio +ile to a pre#io"s #ersio to !iagose
a system pro%lem. *ikewise, a programmer +re="etly ee!s to see what chages ha#e
%ee ma!e to programs o#er time.
28/
2. 8 Te3t 6rocessig
comm
The comm program compares two te3t +iles a! !isplays the lies that are "i="e to each
oe a! the lies they ha#e i commo. To !emostrate, we will create two early
i!etical te3t +iles "sig cat,
[me@linuxbox ~]$ cat & file1.txt
a
b
c
d
[me@linuxbox ~]$ cat & file2.txt
b
c
d
e
&e3t, we will compare the two +iles "sig comm,
[me@linuxbox ~]$ comm file1.txt file2.txt
a
b
c
d
e
$s we ca see, comm pro!"ces three col"ms o+ o"tp"t. The +irst col"m cotais lies
"i="e to the +irst +ile arg"metH the seco! col"m, the lies "i="e to the seco! +ile
arg"metH the thir! col"m cotais the lies share! %y %oth +iles. comm s"pports
optios i the +orm -n where n is either ., 2 or (. Whe "se!, these optios speci+y
which col"mEsF to s"ppress. 0or e3ample, i+ we oly wate! to o"tp"t the lies share!
%y %oth +iles, we wo"l! s"ppress the o"tp"t o+ col"ms oe a! two,
[me@linuxbox ~]$ comm -12 file1.txt file2.txt
b
c
d
diff
*ike the comm program, diff is "se! to !etect the !i++ereces %etwee +iles. Aowe#er,
288
Comparig Te3t
diff is a m"ch more comple3 tool, s"pportig may o"tp"t +ormats a! the a%ility to
process large collectios o+ te3t +iles at oce. diff is o+te "se! %y so+tware !e#elopers
to e3amie chages %etwee !i++eret #ersios o+ program so"rce co!e, a! th"s has the
a%ility to rec"rsi#ely e3amie !irectories o+ so"rce co!e o+te re+erre! to as source trees.
7e commo "se +or diff is the creatio o+ diff files or patches that are "se! %y
programs s"ch as 7atch Ewhich weIll !isc"ss shortlyF to co#ert oe #ersio o+ a +ile Eor
+ilesF to aother #ersio.
9+ we "se diff to look at o"r pre#io"s e3ample +iles,
[me@linuxbox ~]$ diff file1.txt file2.txt
#d(
X a
%a%
D e
we see its !e+a"lt style o+ o"tp"t, a terse !escriptio o+ the !i++ereces %etwee the two
+iles. 9 the !e+a"lt +ormat, each gro"p o+ chages is prece!e! %y a change command i
the +orm o+ range operation range to !escri%e the positios a! type o+ chages re="ire!
to co#ert the +irst +ile to the seco! +ile,
Table '1-$: diff %hange %ommands
Change Description
r1ar2
$ppe! the lies at the positio r' i the seco! +ile to the positio
r1 i the +irst +ile.
r1cr2
Chage EreplaceF the lies at positio r1 with the lies at the
positio r' i the seco! +ile.
r1dr2
'elete the lies i the +irst +ile at positio r1, which wo"l! ha#e
appeare! at rage r' i the seco! +ile
9 this +ormat, a rage is a comma separate! list o+ the startig lie a! the e!ig lie.
While this +ormat is the !e+a"lt Emostly +or 67S9G compliace a! %ackwar!
compati%ility with tra!itioal )i3 #ersios o+ diffF, it is ot as wi!ely "se! as other,
optioal +ormats. Two o+ the more pop"lar +ormats are the conte+t format a! the unified
format.
Whe #iewe! "sig the cote3t +ormat Ethe -c optioF, we will see this,
289
2. 8 Te3t 6rocessig
[me@linuxbox ~]$ diff -c file1.txt file2.txt
EEE file#<txt !((/-#!-!$ (.:%(:#$<((((((((( -("((
--- file!<txt !((/-#!-!$ (.:%(:$%<((((((((( -("((
EEEEEEEEEEEEEEE
EEE #>% EEEE
- a
b
c
d
--- #>% ----
b
c
d
8 e
The o"tp"t %egis with the ames o+ the two +iles a! their timestamps. The +irst +ile is
marke! with asterisks a! the seco! +ile is marke! with !ashes. Thro"gho"t the
remai!er o+ the listig, these markers will sigi+y their respecti#e +iles. &e3t, we see
gro"ps o+ chages, icl"!ig the !e+a"lt "m%er o+ s"rro"!ig cote3t lies. 9 the +irst
gro"p, we see,
EEE #>% EEE
which i!icates lies oe thro"gh +o"r i the +irst +ile. *ater we see,
--- #>% ---
which i!icates lies oe thro"gh +o"r i the seco! +ile. Withi a chage gro"p, lies
%egi with oe o+ +o"r i!icators,
Table '1-.: diff %onte+t Format %hange *ndicators
Indicator >eaning
blank
$ lie show +or cote3t. 9t !oes ot i!icate a !i++erece %etwee
the two +iles.
-
$ lie !elete!. This lie will appear i the +irst +ile %"t ot i the
seco! +ile.
8
$ lie a!!e!. This lie will appear i the seco! +ile %"t ot i the
+irst +ile.
G
$ lie chage!. The two #ersios o+ the lie will %e !isplaye!, each
i its respecti#e sectio o+ the chage gro"p.
The "i+ie! +ormat is similar to the cote3t +ormat, %"t is more cocise. 9t is speci+ie!
with the -u optio,
290
Comparig Te3t
[me@linuxbox ~]$ diff -u file1.txt file2.txt
--- file#<txt !((/-#!-!$ (.:%(:#$<((((((((( -("((
888 file!<txt !((/-#!-!$ (.:%(:$%<((((((((( -("((
@@ -#>% 8#>% @@
-a
b
c
d
8e
The most ota%le !i++erece %etwee the cote3t a! "i+ie! +ormats is the elimiatio o+
the !"plicate! lies o+ cote3t, makig the res"lts o+ the "i+ie! +ormat shorter tha the
cote3t +ormat. 9 o"r e3ample a%o#e, we see +ile timestamps like those o+ the cote3t
+ormat, +ollowe! %y the strig @@ -#>% 8#>% @@. This i!icates the lies i the +irst
+ile a! the lies i the seco! +ile !escri%e! i the chage gro"p. 0ollowig this are the
lies themsel#es, with the !e+a"lt three lies o+ cote3t. Each lie starts with oe o+ three
possi%le characters,
Table '1-2: diff 0nified Format %hange *ndicators
Character >eaning
blank
This lie is share! %y %oth +iles.
-
This lie was remo#e! +rom the +irst +ile.
8
This lie was a!!e! to the +irst +ile.
7atch
The 7atch program is "se! to apply chages to te3t +iles. 9t accepts o"tp"t +rom diff
a! is geerally "se! to co#ert ol!er #ersio o+ +iles ito ewer #ersios. *etIs cosi!er
a +amo"s e3ample. The *i"3 kerel is !e#elope! %y a large, loosely orgaiDe! team o+
cotri%"tors who s"%mit a costat stream o+ small chages to the so"rce co!e. The
*i"3 kerel cosists o+ se#eral millio lies o+ co!e, while the chages that are ma!e %y
oe cotri%"tor at oe time are ="ite small. 9it makes o sese +or a cotri%"tor to se!
each !e#eloper a etire kerel so"rce tree each time a small chage is ma!e. 9stea!, a
!i++ +ile is s"%mitte!. The !i++ +ile cotais the chage +rom the pre#io"s #ersio o+ the
kerel to the ew #ersio with the cotri%"tor<s chages. The recei#er the "ses the
7atch program to apply the chage to his ow so"rce tree. )sig diff-7atch o++ers
two sigi+icat a!#atages,
.. The !i++ +ile is #ery small, compare! to the +"ll siDe o+ the so"rce tree.
2. The !i++ +ile cocisely shows the chage %eig ma!e, allowig re#iewers o+ the
29.
2. 8 Te3t 6rocessig
patch to ="ickly e#al"ate it.
7+ co"rse, diff-7atch will work o ay te3t +ile, ot 4"st so"rce co!e. 9t wo"l! %e
e="ally applica%le to co+ig"ratio +iles or ay other te3t.
To prepare a !i++ +ile +or "se with 7atch, the ;&) !oc"metatio Esee 0"rther :ea!ig
%elowF s"ggests "sig diff as +ollows,
diff -Laur old_file new_file D diff_file
Where oldfile a! newfile are either sigle +iles or !irectories cotaiig +iles. The r
optio s"pports rec"rsio o+ a !irectory tree.
7ce the !i++ +ile has %ee create!, we ca apply it to patch the ol! +ile ito the ew +ile,
7atch X diff_file
WeIll !emostrate with o"r test +ile,
[me@linuxbox ~]$ diff -?aur file1.txt file2.txt & patchfile.txt
[me@linuxbox ~]$ patch / patchfile.txt
7atching file file#<txt
[me@linuxbox ~]$ cat file1.txt
b
c
d
e
9 this e3ample, we create! a !i++ +ile ame! 7atchfile<txt a! the "se! the
7atch program to apply the patch. &ote that we !i! ot ha#e to speci+y a target +ile to
7atch, as the !i++ +ile Ei "i+ie! +ormatF alrea!y cotais the +ileames i the hea!er.
7ce the patch is applie!, we ca see that file#<txt ow matches file!<txt.
7atch has a large "m%er o+ optios, a! there are a!!itioal "tility programs that ca
%e "se! to aalyDe a! e!it patches.
$diting ?n The ;l,
7"r e3periece with te3t e!itors has %ee largely interactive, meaig that we ma"ally
mo#e a c"rsor aro"!, the type o"r chages. Aowe#er, there are non-interactive ways to
e!it te3t as well. 9tIs possi%le, +or e3ample, to apply a set o+ chages to m"ltiple +iles
with a sigle comma!.
tr
The tr program is "se! to transliterate characters. We ca thik o+ this as a sort o+
292
E!itig 7 The 0ly
character-%ase! search-a!-replace operatio. Trasliteratio is the process o+ chagig
characters +rom oe alpha%et to aother. 0or e3ample, co#ertig characters +rom
lowercase to "ppercase is trasliteratio. We ca per+orm s"ch a co#ersio with tr as
+ollows,
[me@linuxbox ~]$ echo Jlowercase lettersJ 0 tr a-- :-@
=,&RC4*& =&TT&R*
$s we ca see, tr operates o sta!ar! ip"t, a! o"tp"ts its res"lts o sta!ar! o"tp"t.
tr accepts two arg"mets, a set o+ characters to co#ert +rom a! a correspo!ig set o+
characters to co#ert to. Character sets may %e e3presse! i oe o+ three ways,
.. $ e"merate! list. 0or e3ample, 4HC'&-@SA?2=+L:ZR*T3;,VUI
2. $ character rage. 0or e3ample, 4-I. &ote that this metho! is sometimes
s"%4ect to the same iss"es as other comma!s, !"e to the locale collatio or!er,
a! th"s sho"l! %e "se! with ca"tio.
(. 67S9G character classes. 0or e3ample, [:u77er:].
9 most cases, %oth character sets sho"l! %e o+ e="al legthH howe#er, it is possi%le +or
the +irst set to %e larger tha the seco!, partic"larly i+ we wish to co#ert m"ltiple
characters to a sigle character,
[me@linuxbox ~]$ echo Jlowercase lettersJ 0 tr (2lower2) :
444444444 4444444
9 a!!itio to trasliteratio, tr allows characters to simply %e !elete! +rom the ip"t
stream. Earlier i this chapter, we !isc"sse! the pro%lem o+ co#ertig >S-'7S te3t
+iles to )i3 style te3t. To per+orm this co#ersio, carriage ret"r characters ee! to %e
remo#e! +rom the e! o+ each lie. This ca %e per+orme! with tr as +ollows,
tr -d K]rK X dos_file D unix_file
where dosfile is the +ile to %e co#erte! a! uni+file is the res"lt. This +orm o+ the
comma! "ses the escape se="ece ]r to represet the carriage ret"r character. To see
a complete list o+ the se="eces a! character classes tr s"pports, try,
[me@linuxbox ~]$ tr --help
29(
2. 8 Te3t 6rocessig
1?T1!I The 7otESoESecret Decoder 1ing
7e am"sig "se o+ tr is to per+orm 9&T13 encoding o+ te3t. :7T.( is a tri#ial
type o+ ecryptio %ase! o a simple s"%stit"tio cipher. Callig :7T.(
?ecryptio@ is %eig geero"sH ?te3t o%+"scatio@ is more acc"rate. 9t is "se!
sometimes o te3t to o%sc"re potetially o++esi#e cotet. The metho! simply
mo#es each character thirtee places "p the alpha%et. Sice this is hal+ way "p
the possi%le twety-si3 characters, per+ormig the algorithm a seco! time o the
te3t restores it to its origial +orm. To per+orm this eco!ig with tr,
echo Jsecret textJ 0 tr a--:-@ n--a-m?-@:-O
fr7erg grkg
6er+ormig the same proce!"re a seco! time res"lts i the traslatio,
echo Jfrperg grkgJ 0 tr a--:-@ n--a-m?-@:-O
secret text
$ "m%er o+ email programs a! )SE&ET ews rea!ers s"pport :7T.(
eco!ig. Wikipe!ia cotais a goo! article o the s"%4ect,
http,--e.wikipe!ia.org-wiki-:7T.(
tr ca per+orm aother trick, too. )sig the -s optio, tr ca ?s="eeDe@ E!eleteF
repeate! istaces o+ a character,
[me@linuxbox ~]$ echo JaaabbbcccJ 0 tr -s ab
abccc
Aere we ha#e a strig cotaiig repeate! characters. Jy speci+yig the set ?a%@ to tr,
we elimiate the repeate! istaces o+ the letters i the set, while lea#ig the character
that is missig +rom the set E?c@F "chage!. &ote that the repeatig characters m"st %e
a!4oiig. 9+ they are ot,
[me@linuxbox ~]$ echo JabcabcabcJ 0 tr -s ab
abcabcabc
the s="eeDig will ha#e o e++ect.
291
E!itig 7 The 0ly
sed
The ame sed is short +or stream editor. 9t per+orms te3t e!itig o a stream o+ te3t,
either a set o+ speci+ie! +iles or sta!ar! ip"t. sed is a power+"l a! somewhat comple3
program Ethere are etire %ooks a%o"t itF, so we will ot co#er it completely here.
9 geeral, the way that sed works is that it is gi#e either a sigle e!itig comma! Eo
the comma! lieF or the ame o+ a script +ile cotaiig m"ltiple comma!s, a! it the
per+orms these comma!s "po each lie i the stream o+ te3t. Aere is a #ery simple
e3ample o+ sed i actio,
[me@linuxbox ~]$ echo JfrontJ 0 sed %s/front/back/%
back
9 this e3ample, we pro!"ce a oe wor! stream o+ te3t "sig echo a! pipe it ito sed.
sed, i t"r, carries o"t the istr"ctio s6front6back6 "po the te3t i the stream
a! pro!"ces the o"tp"t ?%ack@ as a res"lt. We ca also recogiDe this comma! as
resem%lig the ?s"%stit"tio@ Esearch a! replaceF comma! i vi.
Comma!s i sed %egi with a sigle letter. 9 the e3ample a%o#e, the s"%stit"tio
comma! is represete! %y the letter s a! is +ollowe! %y the search a! replace strigs,
separate! %y the slash character as a !elimiter. The choice o+ the !elimiter character is
ar%itrary. Jy co#etio, the slash character is o+te "se!, %"t sed will accept ay
character that imme!iately +ollows the comma! as the !elimiter. We co"l! per+orm the
same comma! this way,
[me@linuxbox ~]$ echo JfrontJ 0 sed %s.front.back.%
back
Jy "sig the "!erscore character imme!iately a+ter the comma!, it %ecomes the
!elimiter. The a%ility to set the !elimiter ca %e "se! to make comma!s more rea!a%le,
as we shall see.
>ost comma!s i sed may %e prece!e! %y a address, which speci+ies which lieEsF o+
the ip"t stream will %e e!ite!. 9+ the a!!ress is omitte!, the the e!itig comma! is
carrie! o"t o e#ery lie i the ip"t stream. The simplest +orm o+ a!!ress is a lie
"m%er. We ca a!! oe to o"r e3ample,
[me@linuxbox ~]$ echo JfrontJ 0 sed %1s/front/back/%
back
292
2. 8 Te3t 6rocessig
$!!ig the a!!ress # to o"r comma! ca"ses o"r s"%stit"tio to %e per+orme! o the +irst
lie o+ o"r oe-lie ip"t stream. 9+ we speci+y aother "m%er,
[me@linuxbox ~]$ echo JfrontJ 0 sed %2s/front/back/%
front
we see that the e!itig is ot carrie! o"t, sice o"r ip"t stream !oes ot ha#e a lie two.
$!!resses may %e e3presse! i may ways. Aere are the most commo,
Table '1-3: sed )ddress >otation
#ddress Description
n
$ lie "m%er where n is a positi#e iteger.
$
The last lie.
6regexp6
*ies matchig a 67S9G %asic reg"lar e3pressio. &ote that
the reg"lar e3pressio is !elimite! %y slash characters.
7ptioally, the reg"lar e3pressio may %e !elimite! %y a
alterate character, %y speci+yig the e3pressio with
]cregexpc, where c is the alterate character.
addr1>addr2
$ rage o+ lies +rom addr1 to addr', icl"si#e. $!!resses
may %e ay o+ the sigle a!!ress +orms a%o#e.
first~step
>atch the lie represete! %y the "m%er first, the each
s"%se="et lie at step iter#als. 0or e3ample .S2 re+ers to
each o!! "m%ere! lie, 2S2 re+ers to the +i+th lie a! e#ery
+i+th lie therea+ter.
addr1>8n
>atch addr1 a! the +ollowig n lies.
addrL >atch all lies e3cept addr, which may %e ay o+ the +orms
a%o#e.
WeIll !emostrate !i++eret ki!s o+ a!!resses "sig the distros<txt +ile +rom earlier
i this chapter. 0irst, a rage o+ lie "m%ers,
[me@linuxbox ~]$ sed -n %1;"p% distros.txt
*3*& #(<! #!6()6!((.
-edora #( ##6!"6!((/
*3*& ##<( (.6#06!((/
29K
E!itig 7 The 0ly
3buntu /<(% (%6!%6!((/
-edora / ##6(/6!(()
9 this e3ample, we prit a rage o+ lies, startig with lie oe a! coti"ig to lie
+i#e. To !o this, we "se the 7 comma!, which simply ca"ses a matche! lie to %e
prite!. 0or this to %e e++ecti#e howe#er, we m"st icl"!e the optio -n Ethe o a"to-
prit optioF to ca"se sed ot to prit e#ery lie %y !e+a"lt.
&e3t, weIll try a reg"lar e3pressio,
[me@linuxbox ~]$ sed -n %/FEFG/p% distros.txt
*3*& #(<! #!6()6!((.
*3*& ##<( (.6#06!((/
*3*& #(<$ #(6(%6!(()
*3*& #(<# ("6##6!((.
Jy icl"!ig the slash-!elimite! reg"lar e3pressio 6*3*&6, we are a%le to isolate the
lies cotaiig it i m"ch the same maer as gre7.
0ially, weIll try egatio %y a!!ig a G to the a!!ress,
[me@linuxbox ~]$ sed -n %/FEFG/Lp% distros.txt
-edora #( ##6!"6!((/
3buntu /<(% (%6!%6!((/
-edora / ##6(/6!(()
3buntu .<#( #(6!.6!((.
-edora ) ("6$#6!(()
3buntu )<#( #(6#/6!(()
3buntu )<(% (%6#06!(()
-edora . #(6!%6!((.
-edora 0 ("6#$6!((/
3buntu .<(. (.6(#6!((.
3buntu /<#( #(6$(6!((/
-edora " ($6!(6!((.
Aere we see the e3pecte! res"lt, all o+ the lies i the +ile e3cept the oes matche! %y the
reg"lar e3pressio.
So +ar, weI#e looke! at two o+ the sed e!itig comma!s, s a! 7. Aere is a more
complete list o+ the %asic e!itig comma!s,
29/
2. 8 Te3t 6rocessig
Table '1-4: sed 8asic 1diting %ommands
Command Description
M
7"tp"t c"rret lie "m%er.
a
$ppe! te3t a+ter the c"rret lie.
d
'elete the c"rret lie.
i
9sert te3t i +rot o+ the c"rret lie.
7
6rit the c"rret lie. Jy !e+a"lt, sed prits
e#ery lie a! oly e!its lies that match a
speci+ie! a!!ress withi the +ile. The !e+a"lt
%eha#ior ca %e o#erri!!e %y speci+yig the -n
optio.
B
E3it sed witho"t processig ay more lies. 9+
the -n optio is ot speci+ie!, o"tp"t the c"rret
lie.
Z
E3it sed witho"t processig ay more lies.
s6regexp6replacement6
S"%stit"te the cotets o+ replacement where#er
rege+p is +o"!. replacement may icl"!e the
special character Y, which is e="i#alet to the te3t
matche! %y rege+p. 9 a!!itio, replacement may
icl"!e the se="eces ]# thro"gh ]0, which are
the cotets o+ the correspo!ig s"%e3pressios
i rege+p. 0or more a%o"t this, see the !isc"ssio
o+ back references %elow. $+ter the trailig slash
+ollowig replacement, a optioal +lag may %e
speci+ie! to mo!i+y the s comma!Is %eha#ior.
16set16set2
6er+orm trasliteratio %y co#ertig characters
+rom set1 to the correspo!ig characters i set'.
&ote that "like tr, sed re="ires that %oth sets %e
o+ the same legth.

The s comma! is %y +ar the most commoly "se! e!itig comma!. We will
!emostrate 4"st some o+ its power %y per+ormig a e!it o o"r distros<txt +ile.
We !isc"sse! %e+ore how the !ate +iel! i distros<txt was ot i a ?comp"ter-
+rie!ly@ +ormat. While the !ate is +ormatte! >>-''-5555, it wo"l! %e %etter E+or
ease o+ sortigF i+ the +ormat were 5555->>-''. To per+orm this chage o the +ile
%y ha! wo"l! %e %oth time-cos"mig a! error proe, %"t with sed, this chage ca
298
E!itig 7 The 0ly
%e per+orme! i oe step,
[me@linuxbox ~]$ sed %s/K4(*-D)K92K>K6K/K4(*-D)K92K>K6K/K4(*-D)K9AK>K
63/K-K1-K2/% distros.txt
*3*& #(<! !((.-#!-()
-edora #( !((/-##-!"
*3*& ##<( !((/-(.-#0
3buntu /<(% !((/-(%-!%
-edora / !(()-##-(/
*3*& #(<$ !(()-#(-(%
3buntu .<#( !((.-#(-!.
-edora ) !(()-("-$#
3buntu )<#( !(()-#(-#/
3buntu )<(% !(()-(%-#0
*3*& #(<# !((.-("-##
-edora . !((.-#(-!%
-edora 0 !((/-("-#$
3buntu .<(. !((.-(.-(#
3buntu /<#( !((/-#(-$(
-edora " !((.-($-!(
WowL &ow that is a "gly lookig comma!. J"t it works. 9 4"st oe step, we ha#e
chage! the !ate +ormat i o"r +ile. 9t is also a per+ect e3ample o+ why reg"lar
e3pressios are sometimes 4okigly re+erre! to as a ?write-oly@ me!i"m. We ca write
them, %"t we sometimes caot rea! them. Je+ore we are tempte! to r" away i terror
+rom this comma!, letIs look at how it was costr"cte!. 0irst, we kow that the
comma! will ha#e this %asic str"ct"re,
sed %s/regexp/replacement/% distros.txt
7"r e3t step is to +ig"re o"t a reg"lar e3pressio that will isolate the !ate. Sice it is i
>>-''-5555 +ormat a! appears at the e! o+ the lie, we ca "se a e3pressio like
this,
(*-D)92>/(*-D)92>/(*-D)9A>3
which matches two !igits, a slash, two !igits, a slash, +o"r !igits, a! the e! o+ lie. So
that takes care o+ rege+p, %"t what a%o"t replacementC To ha!le that, we m"st itro!"ce
a ew reg"lar e3pressio +eat"re that appears i some applicatios which "se J:E. This
+eat"re is calle! back references a! works like this, i+ the se="ece \n appears i
replacement where n is a "m%er +rom oe to ie, the se="ece will re+er to the
299
2. 8 Te3t 6rocessig
correspo!ig s"%e3pressio i the prece!ig reg"lar e3pressio. To create the
s"%e3pressios, we simply eclose them i paretheses like so,
4(*-D)92>6/4(*-D)92>6/4(*-D)9A>63
We ow ha#e three s"%e3pressios. The +irst cotais the moth, the seco! cotais the
!ay o+ the moth, a! the thir! cotais the year. &ow we ca costr"ct replacement as
+ollows,
K-K1-K2
which gi#es "s the year, a !ash, the moth, a !ash, a! the !ay.
&ow, o"r comma! looks like this,
sed %s/4(*-D)92>6/4(*-D)92>6/4(*-D)9A>63/K-K1-K2/% distros.txt
We ha#e two remaiig pro%lems. The +irst is that the e3tra slashes i o"r reg"lar
e3pressio will co+"se sed whe it tries to iterpret the s comma!. The seco! is that
sice sed, %y !e+a"lt, accepts oly %asic reg"lar e3pressios, se#eral o+ the characters i
o"r reg"lar e3pressio will %e take as literals, rather tha as metacharacters. We ca
sol#e %oth these pro%lems with a li%eral applicatio o+ %ackslashes to escape the
o++e!ig characters,
sed %s/K4(*-D)K92K>K6K/K4(*-D)K92K>K6K/K4(*-D)K9AK>K63/K-K1-K2/% dis
tros.txt
$! there yo" ha#e itL
$other +eat"re o+ the s comma! is the "se o+ optioal +lags that may +ollow the
replacemet strig. The most importat o+ these is the g +lag, which istr"cts sed to
apply the search a! replace glo%ally to a lie, ot 4"st to the +irst istace, which is the
!e+a"lt. Aere is a e3ample,
[me@linuxbox ~]$ echo JaaabbbcccJ 0 sed %s/b/</%
aaaHbbccc
(00
E!itig 7 The 0ly
We see that the replacemet was per+orme!, %"t oly to the +irst istace o+ the letter ?%,@
while the remaiig istaces were le+t "chage!. Jy a!!ig the g +lag, we are a%le to
chage all the istaces,
[me@linuxbox ~]$ echo JaaabbbcccJ 0 sed %s/b/</g%
aaaHHHccc
So +ar, we ha#e oly gi#e sed sigle comma!s #ia the comma! lie. 9t is also
possi%le to costr"ct more comple3 comma!s i a script +ile "sig the -f optio. To
!emostrate, we will "se sed with o"r distros<txt +ile to %"il! a report. 7"r report
will +eat"re a title at the top, o"r mo!i+ie! !ates, a! all the !istri%"tio ames co#erte!
to "pper case. To !o this, we will ee! to write a script, so weIll +ire "p o"r te3t e!itor
a! eter the +ollowig,
U sed script to produce Tinux distributions report
1 iK
K
Tinux 1istributions HeportK
s/K4(*-D)K92K>K6K/K4(*-D)K92K>K6K/K4(*-D)K9AK>K63/K-K1-K2/
!/abcdefghijklmnop,rstuvwx!-/:<=1G7QRPYXTO?SB\HF+E]WZ^@/
We will sa#e o"r sed script as distros<sed a! r" it like this,
[me@linuxbox ~]$ sed -f distros.sed distros.txt
=inux 'istributions Re7ort
*3*& #(<! !((.-#!-()
-&'R4 #( !((/-##-!"
*3*& ##<( !((/-(.-#0
3H3LT3 /<(% !((/-(%-!%
-&'R4 / !(()-##-(/
*3*& #(<$ !(()-#(-(%
3H3LT3 .<#( !((.-#(-!.
-&'R4 ) !(()-("-$#
3H3LT3 )<#( !(()-#(-#/
3H3LT3 )<(% !(()-(%-#0
*3*& #(<# !((.-("-##
-&'R4 . !((.-#(-!%
-&'R4 0 !((/-("-#$
(0.
2. 8 Te3t 6rocessig
3H3LT3 .<(. !((.-(.-(#
3H3LT3 /<#( !((/-#(-$(
-&'R4 " !((.-($-!(
$s we ca see, o"r script pro!"ces the !esire! res"lts, %"t how !oes is !o itC *etIs take
aother look at o"r script. WeIll "se cat to "m%er the lies,
[me@linuxbox ~]$ cat -n distros.sed
# ^ sed scri7t to 7roduce =inux distributions re7ort
!
$ # i]
% ]
" =inux 'istributions Re7ort]
.
) s6]N[(-0]][!]\]P]6]N[(-0]][!]\]P]6]N[(-0]][%]\]P$6]$-]#-]!6
/ 16abcdefghijklmno7Brstuv9x1C64HC'&-@SA?2=+L:ZR*T3;,VUI6
*ie oe o+ o"r script is a comment. *ike may co+ig"ratio +iles a! programmig
lag"ages o *i"3 systems, commets %egi with the ^ character a! are +ollowe! %y
h"ma-rea!a%le te3t. Commets ca %e place! aywhere i the script Etho"gh ot withi
comma!s themsel#esF a! are help+"l to ay h"mas who might ee! to i!eti+y a!-or
maitai the script.
*ie two is a %lak lie. *ike commets, %lak lies may %e a!!e! to impro#e
rea!a%ility.
>ay sed comma!s s"pport lie a!!resses. These are "se! to speci+y which lies o+
the ip"t are to %e acte! "po. *ie a!!resses may %e e3presse! as sigle lie "m%ers,
lie "m%er rages, a! the special lie "m%er ?[@ which i!icates the last lie o+ ip"t.
*ies three thro"gh si3 cotai te3t to %e iserte! at the a!!ress ., the +irst lie o+ the
ip"t. The i comma! is +ollowe! %y the se="ece %ackslash-carriage ret"r to pro!"ce
a escape! carriage ret"r, or what is calle! a line continuation character. This
se="ece, which ca %e "se! i may circ"mstaces icl"!ig shell scripts, allows a
carriage ret"r to %e em%e!!e! i a stream o+ te3t witho"t sigalig the iterpreter Ei
this case sedF that the e! o+ the lie has %ee reache!. The i, a! likewise, the a
Ewhich appe!s te3t, rather tha isertig itF a! c Ewhich replaces te3tF comma!s,
allow m"ltiple lies o+ te3t as log as each lie, e3cept the last, e!s with a lie
coti"atio character. The si3th lie o+ o"r script is act"ally the e! o+ o"r iserte! te3t
a! e!s with a plai carriage ret"r rather tha a lie coti"atio character, sigalig
the e! o+ the i comma!.
(02
E!itig 7 The 0ly
1ote5 $ lie coti"atio character is +orme! %y a %ackslash +ollowe! immediately
%y a carriage ret"r. &o iterme!iary spaces are permitte!.
*ie se#e is o"r search a! replace comma!. Sice it is ot prece!e! %y a a!!ress,
each lie i the ip"t stream is s"%4ect to its actio.
*ie eight per+orms trasliteratio o+ the lowercase letters ito "ppercase letters. &ote
that "like tr, the 1 comma! i sed !oes ot s"pport character rages E+or e3ample,
[a-C]F, or !oes it s"pport 67S9G character classes. $gai, sice the 1 comma! is
ot prece!e! %y a a!!ress, it applies to e#ery lie i the ip"t stream.
People +ho Li(e sed #lso Li(e...
sed is a #ery capa%le program, a%le to per+orm +airly comple3 e!itig tasks to
streams o+ te3t. 9t is most o+te "se! +or simple oe lie tasks rather tha log
scripts. >ay "sers pre+er other tools +or larger tasks. The most pop"lar o+ these
are a9k a! 7erl. These go %eyo! mere tools, like the programs co#ere! here,
a! e3te! ito the realm o+ complete programmig lag"ages. 7erl, i
partic"lar, is o+te "se! i place o+ shell scripts +or may system maagemet a!
a!miistratio tasks, as well as %eig a #ery pop"lar me!i"m +or we%
!e#elopmet. a9k is a little more specialiDe!. 9ts speci+ic stregth is its a%ility to
maip"late ta%"lar !ata. 9t resem%les sed i that a9k programs ormally
process te3t +iles lie-%y-lie, "sig a scheme similar to the sed cocept o+ a
a!!ress +ollowe! %y a actio. While %oth a9k a! 7erl are o"tsi!e the scope
o+ this %ook, they are #ery goo! skills +or the *i"3 comma! lie "ser.
as7ell
The last tool we will look at is as7ell, a iteracti#e spellig checker. The as7ell
program is the s"ccessor to a earlier program ame! is7ell, a! ca %e "se!, +or the
most part, as a !rop-i replacemet. While the as7ell program is mostly "se! %y other
programs that re="ire spell checkig capa%ility, it ca also %e "se! #ery e++ecti#ely as a
sta!-aloe tool +rom the comma! lie. 9t has the a%ility to itelligetly check #ario"s
type o+ te3t +iles, icl"!ig AT>* !oc"mets, C-CVV programs, email messages a!
other ki!s o+ specialiDe! te3ts.
To spell check a te3t +ile cotaiig simple prose, it co"l! %e "se! like this,
(0(
2. 8 Te3t 6rocessig
as7ell check textfile
where te+tfile is the ame o+ the +ile to check. $s a practical e3ample, letIs create a
simple te3t +ile ame! foo<txt cotaiig some !eli%erate spellig errors,
[me@linuxbox ~]$ cat & foo.txt
+he ,uick brown fox jimped over the lax! dog.
&e3t weIll check the +ile "sig as7ell,
[me@linuxbox ~]$ aspell check foo.txt
$s as7ell is iteracti#e i the check mo!e, we will see a scree like this,
The Buick bro9n fox jim7ed over the lax1 dog<
#P jum7ed .P 9im7ed
!P gim7ed )P cam7ed
$P com7ed /P hum7ed
%P lim7ed 0P im7ede
"P 7im7ed (P um7ed
iP Agnore AP Agnore all
rP Re7lace RP Re7lace all
aP 4dd lP 4dd =o9er
bP 4bort xP &xit

F
$t the top o+ the !isplay, we see o"r te3t with a s"spicio"sly spelle! wor! highlighte!. 9
the mi!!le, we see te spellig s"ggestios "m%ere! Dero thro"gh ie, +ollowe! %y a
list o+ other possi%le actios. 0ially, at the #ery %ottom, we see a prompt rea!y to accept
o"r choice.
9+ we press the # key, as7ell replaces the o++e!ig wor! with the wor! ?4"mpe!@ a!
mo#es o to the e3t misspelle! wor! which is ?la3y.@ 9+ we select the replacemet
?laDy,@ as7ell replaces it a! termiates. 7ce as7ell has +iishe!, we ca e3amie
o"r +ile a! see that the misspelligs ha#e %ee correcte!,
(01
E!itig 7 The 0ly
[me@linuxbox ~]$ cat foo.txt
The Buick bro9n fox jum7ed over the laC1 dog<
)less tol! otherwise #ia the comma! lie optio --dont-backu7, as7ell creates
a %ack"p +ile cotaiig the origial te3t %y appe!ig the e3tesio <bak to the
+ileame.
Showig o++ o"r sed e!itig prowess, weIll p"t o"r spellig mistakes %ack i so we ca
re"se o"r +ile,
[me@linuxbox ~]$ sed -i %s/la-!/lax!/# s/jumped/jimped/% foo.txt
The sed optio -i tells sed to e!it the +ile ?i-place,@ meaig that rather tha se!ig
the e!ite! o"tp"t to sta!ar! o"tp"t, it will re-write the +ile with the chages applie!. We
also see the a%ility to place more tha oe e!itig comma! o the lie %y separatig
them with a semicolo.
&e3t, weIll look at how as7ell ca ha!le !i++eret ki!s o+ te3t +iles. )sig a te3t
e!itor s"ch as vim Ethe a!#et"ro"s may wat to try sedF, we will a!! some AT>*
mark"p to o"r +ile,
/html&
/head&
/title&Oispelled R+OT file//title&
//head&
/bod!&
/p&The Buick bro9n fox jim7ed over the lax1 dog<//p&
//bod!&
//html&
&ow, i+ we try to spell check o"r mo!i+ie! +ile, we r" ito a pro%lem. 9+ we !o it this
way,
[me@linuxbox ~]$ aspell check foo.txt
weIll get this,
XhtmlD
(02
2. 8 Te3t 6rocessig
XheadD
XtitleD+is7elled ST+= fileX6titleD
X6headD
Xbod1D
X7DThe Buick bro9n fox jim7ed over the lax1 dog<X67D
X6bod1D
X6htmlD
#P ST+= %P Samel
!P ht ml "P Samil
$P ht-ml .P hotel
iP Agnore AP Agnore all
rP Re7lace RP Re7lace all
aP 4dd lP 4dd =o9er
bP 4bort xP &xit
F
as7ell will see the cotets o+ the AT>* tags as misspelle!. This pro%lem ca %e
o#ercome %y icl"!ig the -S EAT>*F checkig mo!e optio, like this,
[me@linuxbox ~]$ aspell -R check foo.txt
which will res"lt i this,
XhtmlD
XheadD
XtitleD+is7elled ST+= fileX6titleD
X6headD
Xbod1D
X7DThe Buick bro9n fox jim7ed over the lax1 dog<X67D
X6bod1D
X6htmlD
#P +i s7elled .P +isa77lied
!P +i-s7elled )P +iscalled
$P +iss7elled /P Res7elled
%P 'is7elled 0P +iss7ell
"P *7elled (P +isled
iP Agnore AP Agnore all
rP Re7lace RP Re7lace all
(0K
E!itig 7 The 0ly
aP 4dd lP 4dd =o9er
bP 4bort xP &xit

F
The AT>* is igore! a! oly the o-mark"p portios o+ the +ile are checke!. 9 this
mo!e, the cotets o+ AT>* tags are igore! a! ot checke! +or spellig. Aowe#er, the
cotets o+ 4=T tags, which %ee+it +rom checkig, are checke! i this mo!e.
1ote5 Jy !e+a"lt, as7ell will igore ):*s a! email a!!resses i te3t. This
%eha#ior ca %e o#erri!!e with comma! lie optios. 9t is also possi%le to
speci+y which mark"p tags are checke! a! skippe!. See the as7ell ma page
+or !etails.
Summing -p
9 this chapter, we ha#e looke! at a +ew o+ the may comma! lie tools that operate o
te3t. 9 the e3t chapter, we will look at se#eral more. $!mitte!ly, it may ot seem
imme!iately o%#io"s how or why yo" might "se some o+ these tools o a !ay-to-!ay
%asis, tho"gh we ha#e trie! to show some semi-practical e3amples o+ their "se. We will
+i! i later chapters that these tools +orm the %asis o+ a tool set that is "se! to sol#e a
host o+ practical pro%lems. This will %e partic"larly tr"e whe we get ito shell scriptig,
where these tools will really show their worth.
;urther 1eading
The ;&) 6ro4ect we%site cotais may olie g"i!es to the tools !isc"sse! i this
chapter.
0rom the Core"tils package,
http,--www.g".org-so+tware-core"tils-ma"al-core"tils.htmlP7"tp"t-o+-etire-
+iles
http,--www.g".org-so+tware-core"tils-ma"al-core"tils.htmlP7peratig-o-
sorte!-+iles
http,--www.g".org-so+tware-core"tils-ma"al-core"tils.htmlP7peratig-o-+iel!s-
withi-a-lie
http,--www.g".org-so+tware-core"tils-ma"al-core"tils.htmlP7peratig-o-
characters
0rom the 'i++"tils package,
http,--www.g".org-so+tware-!i++"tils-ma"al-htmlOmoo-!i++.html
(0/
2. 8 Te3t 6rocessig
sed,
http,--www.g".org-so+tware-se!-ma"al-se!.html
as7ell,
http,--aspell.et-ma-html-i!e3.html
There are may other olie reso"rces +or sed, i partic"lar,
http,--www.grymoire.com-)i3-Se!.html
http,--se!.so"rce+orge.et-se!.lie.t3t
$lso try googlig ?se! oe liers@, ?se! cheat sheets@
$xtra Credit
There are a +ew more iterestig te3t maip"latio comma!s worth i#estigatig.
$mog these are, s7lit Esplit +iles ito piecesF, cs7lit Esplit +iles ito pieces %ase!
o cote3tF, a! sdiff Esi!e-%y-si!e merge o+ +ile !i++ereces.F
(08
22 8 0ormattig 7"tp"t
22 (or%atting 7ut#ut
9 this chapter, we coti"e o"r look at te3t relate! tools, +oc"sig o programs that are
"se! to +ormat te3t o"tp"t, rather tha chagig the te3t itsel+. These tools are o+te "se!
to prepare te3t +or e#et"al pritig, a s"%4ect that we will co#er i the e3t chapter. The
programs that we will co#er i this chapter icl"!e,
nl 8 &"m%er lies
fold 8 Wrap each lie to a speci+ie! legth
fmt 8 $ simple te3t +ormatter
7r 8 6repare te3t +or pritig
7rintf 8 0ormat a! prit !ata
groff 8 $ !oc"met +ormattig system
Simple ;ormatting Tools
WeIll look at some o+ the simple +ormattig tools +irst. These are mostly sigle p"rpose
programs, a! a %it "sophisticate! i what they !o, %"t they ca %e "se! +or small tasks
a! as parts o+ pipelies a! scripts.
nl A ,umber Lines
The nl program is a rather arcae tool "se! to per+orm a simple task. 9t "m%ers lies.
9 its simplest "se, it resem%les cat -n,
[me@linuxbox ~]$ nl distros.txt 0 head
# *3*& #(<! #!6()6!((.
! -edora #( ##6!"6!((/
$ *3*& ##<( (.6#06!((/
% 3buntu /<(% (%6!%6!((/
" -edora / ##6(/6!(()
. *3*& #(<$ #(6(%6!(()
) 3buntu .<#( #(6!.6!((.
(09
22 8 0ormattig 7"tp"t
/ -edora ) ("6$#6!(()
0 3buntu )<#( #(6#/6!(()
#( 3buntu )<(% (%6#06!(()
*ike cat, nl ca accept either m"ltiple +iles as comma! lie arg"mets, or sta!ar!
ip"t. Aowe#er, nl has a "m%er o+ optios a! s"pports a primiti#e +orm o+ mark"p to
allow more comple3 ki!s o+ "m%erig.
nl s"pports a cocept calle! ?logical pages@ whe "m%erig. This allows nl to reset
Estart o#erF the "merical se="ece whe "m%erig. )sig optios, it is possi%le to set
the startig "m%er to a speci+ic #al"e a!, to a limite! e3tet, its +ormat. $ logical page
is +"rther %roke !ow ito a hea!er, %o!y, a! +ooter. Withi each o+ these sectios, lie
"m%erig may %e reset a!-or %e assige! a !i++eret style. 9+ nl is gi#e m"ltiple +iles,
it treats them as a sigle stream o+ te3t. Sectios i the te3t stream are i!icate! %y the
presece o+ some rather o!!-lookig mark"p a!!e! to the te3t,
Table ''-1: nl !arkup
>ar(up >eaning
]:]:]:
Start o+ logical page hea!er
]:]:
Start o+ logical page %o!y
]:
Start o+ logical page +ooter
Each o+ the a%o#e mark"p elemets m"st appear aloe o its ow lie. $+ter processig
a mark"p elemet, nl !eletes it +rom the te3t stream.
Aere are the commo optios +or nl,
Table ''-': %ommon nl &ptions
?ption >eaning
-b stle
Set %o!y "m%erig to style, where style is oe o+ the +ollowig,
a W "m%er all lies
t W "m%er oly o-%lak lies. This is the !e+a"lt.
n W oe
7regexp W "m%er oly lies matchig %asic reg"lar e3pressio
rege+p.
-f stle
Set +ooter "m%erig to style. 'e+a"lt is EoeF.
-h stle
Set hea!er "m%erig to style. 'e+a"lt is EoeF.
(.0
Simple 0ormattig Tools
-i number
Set page "m%erig icremet to number. 'e+a"lt is oe.
-n format
Sets "m%erig +ormat to format, where +ormat is,
ln W le+t 4"sti+ie!, witho"t lea!ig Deros.
rn W right 4"sti+ie!, witho"t lea!ig Deros. This is the !e+a"lt.
rC W right 4"sti+ie!, with lea!ig Deros.
-7
'o ot reset page "m%erig at the %egiig o+ each logical page.
-s string
$!! string to the e! o+ each lie "m%er to create a separator.
'e+a"lt is a sigle ta% character.
-v number
Set +irst lie "m%er o+ each logical page to number. 'e+a"lt is
oe.
-9 width
Set wi!th o+ the lie "m%er +iel! to width. 'e+a"lt is si3.
$!mitte!ly, we pro%a%ly woIt %e "m%erig lies that o+te, %"t we ca "se nl to look
at how we ca com%ie m"ltiple tools to per+orm more comple3 tasks. We will %"il! o
o"r work i the pre#io"s chapter to pro!"ce a *i"3 !istri%"tios report. Sice we will
%e "sig nl, it will %e "se+"l to icl"!e its hea!er-%o!y-+ooter mark"p. To !o this, we
will a!! it to the sed script +rom the last chapter. )sig o"r te3t e!itor, we will chage
the script as +ollows a! sa#e it as distros-nl<sed,
^ sed scri7t to 7roduce =inux distributions re7ort
# i]
KK2KK2KK2K
]
=inux 'istributions Re7ort]
K
?ame ]er. HeleasedK
---- ---- --------K
KK2KK2
s6]N[(-0]][!]\]P]6]N[(-0]][!]\]P]6]N[(-0]][%]\]P$6]$-]#-]!6
3 iK
KK2K
K
Gnd Sf Heport
The script ow iserts the nl logical page mark"p a! a!!s a +ooter at the e! o+ the
report. &ote that we ha! to !o"%le "p the %ackslashes i o"r mark"p, %eca"se they are
ormally iterprete! as a escape character %y sed.
&e3t, weIll pro!"ce o"r ehace! report %y com%iig sort, sed, a! nl,
(..
22 8 0ormattig 7"tp"t
[me@linuxbox ~]$ sort -k 1;1 -k 2n distros.txt 0 sed -f distros-nl.s
ed 0 nl


=inux 'istributions Re7ort

Lame ;er< Released
---- ---- --------
# -edora " !((.-($-!(
! -edora . !((.-#(-!%
$ -edora ) !(()-("-$#
% -edora / !(()-##-(/
" -edora 0 !((/-("-#$
. -edora #( !((/-##-!"
) *3*& #(<# !((.-("-##
/ *3*& #(<! !((.-#!-()
0 *3*& #(<$ !(()-#(-(%
#( *3*& ##<( !((/-(.-#0
## 3buntu .<(. !((.-(.-(#
#! 3buntu .<#( !((.-#(-!.
#$ 3buntu )<(% !(()-(%-#0
#% 3buntu )<#( !(()-#(-#/
#" 3buntu /<(% !((/-(%-!%
#. 3buntu /<#( !((/-#(-$(

&nd f Re7ort
7"r report is the res"lt o+ o"r pipelie o+ comma!s. 0irst, we sort the list %y !istri%"tio
ame a! #ersio E+iel!s oe a! twoF, the we process the res"lts with sed, a!!ig the
report hea!er Eicl"!ig the logical page mark"p +or nlF a! +ooter. 0ially, we process
the res"lt with nl, which, %y !e+a"lt, oly "m%ers the lies o+ the te3t stream that
%elog to the %o!y sectio o+ the logical page.
We ca repeat the comma! a! e3perimet with !i++eret optios +or nl. Some
iterestig oes are,
nl -n r-
a!
nl -w -s % %
(.2
Simple 0ormattig Tools
fold A Wra5 6a/h Line To A S5e/i=ied Length
Folding is the process o+ %reakig lies o+ te3t at a speci+ie! wi!th. *ike o"r other
comma!s, fold accepts either oe or more te3t +iles or sta!ar! ip"t. 9+ we se!
fold a simple stream o+ te3t, we ca see how it works,
[me@linuxbox ~]$ echo J+he ,uick brown fox jumped over the la-! dog.J
0 fold -w 12
The Buick br
o9n fox jum7
ed over the
laC1 dog<
Aere we see fold i actio. The te3t set %y the echo comma! is %roke ito
segmets speci+ie! %y the -9 optio. 9 this e3ample, we speci+y a lie wi!th o+ twel#e
characters. 9+ o wi!th is speci+ie!, the !e+a"lt is eighty characters. &otice how the lies
are %roke regar!less o+ wor! %o"!aries. The a!!itio o+ the -s optio will ca"se
fold to %reak the lie at the last a#aila%le space %e+ore the lie wi!th is reache!,
[me@linuxbox ~]$ echo J+he ,uick brown fox jumped over the la-! dog.J
0 fold -w 12 -s
The Buick
bro9n fox
jum7ed over
the laC1
dog<
fmt A A Sim5e Text 2ormatter
The fmt program also +ol!s te3t, pl"s a lot more. 9t accepts either +iles or sta!ar! ip"t
a! per+orms paragraph +ormattig o the te3t stream. Jasically, it +ills a! 4ois lies i
te3t while preser#ig %lak lies a! i!etatio.
To !emostrate, weIll ee! some te3t. *etIs li+t some +rom the fmt i+o page,
JfmtK reads from the s7ecified -A=& arguments Nor standard in7ut
if none are givenP> and 9rites to standard out7ut<
H1 default> blank lines> s7aces bet9een 9ords> and indentation are
(.(
22 8 0ormattig 7"tp"t
7reserved in the out7utT successive in7ut lines 9ith different
indentation are not joinedT tabs are ex7anded on in7ut and introduced
on out7ut<
JfmtK 7refers breaking lines at the end of a sentence> and tries
to avoid line breaks after the first 9ord of a sentence or before the
last 9ord of a sentence< 4 _sentence break_ is defined as either the
end of a 7aragra7h or a 9ord ending in an1 of J<FGK> follo9ed b1 t9o
s7aces or end of line> ignoring an1 intervening 7arentheses or
Buotes< =ike TeV> JfmtK reads entire _7aragra7hs_ before choosing
line breaksT the algorithm is a variant of that given b1 'onald &<
2nuth and +ichael -< :lass in _Hreaking :aragra7hs Anto =ines_>
J*oft9are--:ractice Y &x7erienceK ##> ## NLovember #0/#P> ###0-##/%<
WeIll copy this te3t ito o"r te3t e!itor a! sa#e the +ile as fmt-info<txt. &ow, letIs
say we wate! to re+ormat this te3t to +it a +i+ty character wi!e col"m. We co"l! !o this
%y processig the +ile with fmt a! the -9 optio,
[me@linuxbox ~]$ fmt -w "* fmt-info.txt 0 head
JfmtK reads from the s7ecified -A=& arguments
Nor standard in7ut if
none are givenP> and 9rites to standard out7ut<
H1 default> blank lines> s7aces bet9een 9ords>
and indentation are
7reserved in the out7utT successive in7ut lines
9ith different indentation are not joinedT tabs
are ex7anded on in7ut and introduced on out7ut<
Well, thatIs a awkwar! res"lt. 6erhaps we sho"l! act"ally rea! this te3t, sice it
e3plais whatIs goig o,
?8y default" blank lines" spaces between words" and indentation are preserved in the
outputK successive input lines with different indentation are not AoinedK tabs are
e+panded on input and introduced on output#@
So, fmt is preser#ig the i!etatio o+ the +irst lie. 0ort"ately, fmt pro#i!es a
optio to correct this,
[me@linuxbox ~]$ fmt -cw "* fmt-info.txt
JfmtK reads from the s7ecified -A=& arguments
Nor standard in7ut if none are givenP> and 9rites
to standard out7ut<
(.1
Simple 0ormattig Tools
H1 default> blank lines> s7aces bet9een 9ords>
and indentation are 7reserved in the out7utT
successive in7ut lines 9ith different indentation
are not joinedT tabs are ex7anded on in7ut and
introduced on out7ut<
JfmtK 7refers breaking lines at the end of a
sentence> and tries to avoid line breaks after
the first 9ord of a sentence or before the
last 9ord of a sentence< 4 _sentence break_
is defined as either the end of a 7aragra7h
or a 9ord ending in an1 of J<FGK> follo9ed
b1 t9o s7aces or end of line> ignoring an1
intervening 7arentheses or Buotes< =ike TeV>
JfmtK reads entire _7aragra7hs_ before choosing
line breaksT the algorithm is a variant of
that given b1 'onald &< 2nuth and +ichael -<
:lass in _Hreaking :aragra7hs Anto =ines_>
J*oft9are--:ractice Y &x7erienceK ##> ##
NLovember #0/#P> ###0-##/%<
>"ch %etter. Jy a!!ig the -c optio, we ow ha#e the !esire! res"lt.
fmt has some iterestig optios,
Table ''-3: fmt &ptions
?ption Description
-c
7perate i crown margin mo!e. This preser#es the i!etatio o+
the +irst two lies o+ a paragraph. S"%se="et lies are alige! with
the i!etatio o+ the seco! lie.
-7 string
7ly +ormat those lies %egiig with the pre+i3 string. $+ter
+ormattig, the cotets o+ string are pre+i3e! to each re+ormatte!
lie. This optio ca %e "se! to +ormat te3t i so"rce co!e
commets. 0or e3ample, ay programmig lag"age or
co+ig"ratio +ile that "ses a ?P@ character to !elieate a commet
co"l! %e +ormatte! %y speci+yig -7 K^ K so that oly the
commets will %e +ormatte!. See the e3ample %elow.
-s
Split-oly mo!e. 9 this mo!e, lies will oly %e split to +it the
speci+ie! col"m wi!th. Short lies will ot %e 4oie! to +ill lies.
This mo!e is "se+"l whe +ormattig te3t s"ch as co!e where
4oiig is ot !esire!.
-u
6er+orm "i+orm spacig. This will apply tra!itioal ?typewriter
(.2
22 8 0ormattig 7"tp"t
style@ +ormattig to the te3t. This meas a sigle space %etwee
wor!s a! two spaces %etwee seteces. This mo!e is "se+"l +or
remo#ig ?4"sti+icatio,@ that is, te3t that has %ee pa!!e! with
spaces to +orce aligmet o %oth the le+t a! right margis.
-9 width
0ormat te3t to +it withi a col"m width characters wi!e. The
!e+a"lt is /2 characters. &ote, fmt act"ally +ormats lies slightly
shorter tha the speci+ie! wi!th to allow +or lie %alacig.
The -7 optio is partic"larly iterestig. With it, we ca +ormat selecte! portios o+ a
+ile, pro#i!e! that the lies to %e +ormatte! all %egi with the same se="ece o+
characters. >ay programmig lag"ages "se the po"! sig EPF to i!icate the
%egiig o+ a commet a! th"s ca %e +ormatte! "sig this optio. *etIs create a +ile
that sim"lates a program that "ses commets,
[me@linuxbox ~]$ cat & fmt-code.txt
^ This file contains code 9ith comments<
^ This line is a comment<
^ -ollo9ed b1 another comment line<
^ 4nd another<
This> on the other hand> is a line of code<
4nd another line of code<
4nd another<
7"r sample +ile cotais commets which %egi the strig ?P ? Ea P +ollowe! %y a spaceF
a! lies o+ ?co!e@ which !o ot. &ow, "sig fmt, we ca +ormat the commets a!
lea#e the co!e "to"che!,
[me@linuxbox ~]$ fmt -w "* -p %U % fmt-code.txt
^ This file contains code 9ith comments<
^ This line is a comment< -ollo9ed b1 another
^ comment line< 4nd another<
This> on the other hand> is a line of code<
4nd another line of code<
4nd another<
&otice that the a!4oiig commet lies are 4oie!, while the %lak lies a! the lies
that !o ot %egi with the speci+ie! pre+i3 are preser#e!.
(.K
Simple 0ormattig Tools
7r A 2ormat Text 2or 'rinting
The 7r program is "se to paginate te3t. Whe pritig te3t, it is o+te !esira%le to
separate the pages o+ o"tp"t with se#eral lies o+ whitespace, to pro#i!e a top a! %ottom
margi +or each page. 0"rther, this whitespace ca %e "se! to isert a hea!er a! +ooter
o each page.
WeIll !emostrate 7r %y +ormattig o"r distros<txt +ile ito a series o+ #ery short
pages Eoly the +irst two pages showF,
[me@linuxbox ~]$ pr -l 1" -w N" distros.txt
!((/-#!-## #/:!) distros<txt :age #
*3*& #(<! #!6()6!((.
-edora #( ##6!"6!((/
*3*& ##<( (.6#06!((/
3buntu /<(% (%6!%6!((/
-edora / ##6(/6!(()
!((/-#!-## #/:!) distros<txt :age !
*3*& #(<$ #(6(%6!(()
3buntu .<#( #(6!.6!((.
-edora ) ("6$#6!(()
3buntu )<#( #(6#/6!(()
3buntu )<(% (%6#06!(()
9 this e3ample, we employ the -l optio E+or page legthF a! the -9 optio Epage
wi!thF to !e+ie a ?page@ that is si3ty-+i#e col"ms wi!e a! +i+tee lies log. 7r
pagiates the cotets o+ the distros<txt +ile, separates each page with se#eral lies
o+ whitespace a! creates a !e+a"lt hea!er cotaiig the +ile mo!i+icatio time,
+ileame, a! page "m%er. The 7r program pro#i!es may optios to cotrol page
layo"t. WeIll take a look at more o+ them i the e3t chapter.
(./
22 8 0ormattig 7"tp"t
7rintf A 2ormat And 'rint )ata
)like the other comma!s i this chapter, the 7rintf comma! is ot "se! +or
pipelies Eit !oes ot accept sta!ar! ip"tF or !oes it +i! +re="et applicatio !irectly
o the comma! lie EitIs mostly "se! i scriptsF. So why is it importatC Jeca"se it is
so wi!ely "se!.
7rintf E+rom the phrase, ?prit +ormatte!@F was origially !e#elope! +or the C
programmig lag"age a! has %ee implemete! i may programmig lag"ages
icl"!ig the shell. 9 +act, i bash, 7rintf is a %"ilti.
7rintf works like this,
7rintf cformatd arguments
The comma! is gi#e a strig cotaiig a +ormat !escriptio which is the applie! to a
list o+ arg"mets. The +ormatte! res"lt is set to sta!ar! o"tp"t. Aere is a tri#ial
e3ample,
[me@linuxbox ~]$ printf JP formatted the string2 8sKnJ foo
A formatted the string: foo
The +ormat strig may cotai literal te3t Elike ?9 +ormatte! the strig,@F, escape
se="eces Es"ch as ]n, a ewlie characterF, a! se="eces %egiig with the 5
character, which are calle! conversion specifications. 9 the e3ample a%o#e, the
co#ersio speci+icatio 5s is "se! to +ormat the strig ?+oo@ a! place it i the
comma!Is o"tp"t. Aere it is agai,
[me@linuxbox ~]$ printf JP formatted %8s% as a string.KnJ foo
A formatted KfooK as a string<
$s we ca see, the 5s co#ersio speci+icatio is replace! %y the strig ?+oo@ i the
comma!Is o"tp"t. The s co#ersio is "se! to +ormat strig !ata. There are other
speci+iers +or other ki!s o+ !ata. This ta%le lists the commoly "se! !ata types,
Table ''-$: %ommon printf -ata Type Specifiers
Speci"ier Description
d
0ormat a "m%er as a sige! !ecimal iteger.
f
0ormat a! o"tp"t a +loatig poit "m%er.
o
0ormat a iteger as a octal "m%er.
(.8
Simple 0ormattig Tools
s
0ormat a strig.
x
0ormat a iteger as a he3a!ecimal "m%er "sig lowercase a-+ where
ee!e!.
V
Same as x %"t "se "pper case letters.
5
6rit a literal _ sym%ol Ei.e. speci+y ?__@F
WeIll !emostrate the e++ect each o+ the co#ersio speci+iers o the strig ?(80@,
[me@linuxbox ~]$ printf J8d; 8f; 8o; 8s; 8x; 8ZKnJ M* M* M* M*
M* M*
$/(> $/(<((((((> ")%> $/(> #)c> #)C
Sice we speci+ie! si3 co#ersio speci+iers, we m"st also s"pply si3 arg"mets +or
7rintf to process. The si3 res"lts show the e++ect o+ each speci+ier.
There are se#eral optioal compoets that may %e a!!e! to the co#ersio speci+ier to
a!4"st its o"tp"t. $ complete co#ersio speci+icatio may cosist o+ the +ollowig,
5[flags][width][.precision]conversionOs7ecification
>"ltiple optioal compoets, whe "se!, m"st appear i the or!er speci+ie! a%o#e to %e
properly iterprete!. Aere is a !escriptio o+ each,
Table ''-.: printf %onversion Specification %omponents
Component Description
flags
There are +i#e !i++eret +lags,
^ 8 "se the ?alterate +ormat@ +or o"tp"t. This #aries %y !ata
type. 0or o Eoctal "m%erF co#ersio, the o"tp"t is pre+i3e!
with (. 0or x a! V Ehe3a!ecimal "m%erF co#ersios, the
o"tp"t is pre+i3e! with (x or (V respecti#ely.
(WEa DeroF pa! the o"tp"t with Deros. This meas that the +iel!
will %e +ille! with lea!ig Deros, as i ?000(80@.
- 8 Ea !ashF le+t-alig the o"tp"t. Jy !e+a"lt, 7rintf right-
aligs o"tp"t.
e I 8 Ea spaceF pro!"ces a lea!ig space +or positi#e "m%ers.
8 8 Ea pl"s sigF sigs positi#e "m%ers. Jy !e+a"lt, 7rintf
(.9
22 8 0ormattig 7"tp"t
oly sigs egati#e "m%ers.
width
$ "m%er speci+yig the miim"m +iel! wi!th.
.precision
0or +loatig poit "m%ers, speci+y the "m%er o+ !igits o+
precisio to %e o"tp"t a+ter the !ecimal poit. 0or strig
co#ersio, precision speci+ies the "m%er o+ characters to
o"tp"t.
Aere are some e3amples o+ !i++eret +ormats i actio,
Table ''-2: print %onversion Specification 1+amples
#rgument ;ormat 1esult 7otes
$/( _5d_ $/(
Simple +ormattig o+ a
iteger.
$/( _5^x_ (x#)c
9teger +ormatte! as a
he3a!ecimal "m%er "sig
the ?alterate +ormat@ +lag.
$/( _5("d_ (($/(
9teger +ormatte! with
lea!ig Deros Epa!!igF
a! a miim"m +iel! wi!th
o+ +i#e characters.
$/( _5("<"f_ $/(<(((((
&"m%er +ormatte! as a
+loatig poit "m%er with
pa!!ig a! +i#e !ecimal
places o+ precisio. Sice
the speci+ie! miim"m
+iel! wi!th E2F is less tha
the act"al wi!th o+ the
+ormatte! "m%er, the
pa!!ig has o e++ect.
$/( _5(#(<"f_ ($/(<(((((
Jy icreasig the
miim"m +iel! wi!th to .0
the pa!!ig is ow #isi%le.
$/( _58d_ 8$/(
The V +lag sigs a positi#e
"m%er.
$/( _5-d_ $/(
The - +lag le+t aligs the
+ormattig.
(20
Simple 0ormattig Tools
abcdefghijk _"s_ abcedfghijk
$ strig +ormatte! with a
miim"m +iel! wi!th.
abcdefghijk _<"s_ abcde
Jy applyig precisio to a
strig, it is tr"cate!.
$gai, 7rintf is "se! mostly i scripts where it is employe! to +ormat ta%"lar !ata,
rather tha o the comma! lie !irectly. J"t we ca still show how it ca %e "se! to
sol#e #ario"s +ormattig pro%lems. 0irst, letIs o"tp"t some +iel!s separate! %y ta%
characters,
[me@linuxbox ~]$ printf J8sKt8sKt8sKnJ str1 str2 str
str# str! str$
Jy isertig ]t Ethe escape se="ece +or a ta%F, we achie#e the !esire! e++ect. &e3t,
some "m%ers with eat +ormattig,
[me@linuxbox ~]$ printf JTine2 8*"d 81".f Hesult2 851"dKnJ 1*C1
.1A1"N2D" 2"MD
=ine: (#()# $<#%! Result: 8$!"/0
This shows the e++ect o+ miim"m +iel! wi!th o the spacig o+ the +iel!s. 7r how a%o"t
+ormattig a tiy we% page,
[me@linuxbox ~]$ printf J/html&KnKt/head&KnKtKt/title&8s//title&Kn
Kt//head&KnKt/bod!&KnKtKt/p&8s//p&KnKt//bod!&Kn//html&KnJ JBage +it
leJ JBage =ontentJ
XhtmlD
XheadD
XtitleD:age TitleX6titleD
X6headD
Xbod1D
X7D:age ContentX67D
X6bod1D
X6htmlD
Document ;ormatting S,stems
So +ar, we ha#e e3amie! the simple te3t +ormattig tools. These are goo! +or small,
(2.
22 8 0ormattig 7"tp"t
simple tasks, %"t what a%o"t larger 4o%sC 7e o+ the reasos that )i3 %ecame a pop"lar
operatig system amog techical a! scieti+ic "sers Easi!e +rom pro#i!ig a power+"l
m"ltitaskig, m"lti"ser e#iromet +or all ki!s o+ so+tware !e#elopmetF is that it
o++ere! tools that co"l! %e "se! to pro!"ce may types o+ !oc"mets, partic"larly
scieti+ic a! aca!emic p"%licatios. 9 +act, as the ;&) !oc"metatio !escri%es,
!oc"met preparatio was istr"metal to the !e#elopmet o+ )i3,
The first version of 0>*J was developed on a 6-6-3 which was sitting around 8ell
(abs# *n 1531 the developers wanted to get a 6-6-11 for further work on the
operating system# *n order to Austify the cost for this system" they proposed that
they would implement a document formatting system for the )TLT patents division#
This first formatting program was a reimplementation of !c*llroyMs NroffM" written
by D# F# &ssanna#
There are two mai +amilies o+ !oc"met +ormatters that !omiate the +iel!, those
!esce!e! +rom the origial roff program icl"!ig nroff a! troff, a! those
%ase! o 'oal! M"thIs T
E
G Eproo"ce! ?tek@F typesettig system. $! yes, the
!roppe! ?E@ i the mi!!le is part o+ its ame.
The ame ?ro++@ is !eri#e! +rom the term ?r" o++@ as i, ?9Ill r" o++ a copy +or yo".@
The nroff program is "se! to +ormat !oc"mets +or o"tp"t to !e#ices that "se
moospace! +ots, s"ch as character termials a! typewriter-style priters. $t the time
o+ its itro!"ctio, this icl"!e! early all pritig !e#ices attache! to comp"ters. The
later troff program +ormats !oc"mets +or o"tp"t o typesetters, !e#ices "se! to
pro!"ce ?camera-rea!y@ type +or commercial pritig. >ost comp"ter priters to!ay are
a%le to sim"late the o"tp"t o+ typesetters. The roff +amily also icl"!es some other
programs that are "se! to prepare portios o+ !oc"mets. These icl"!e eBn E+or
mathematical e="atiosF a! tbl E+or ta%lesF.
The T
E
G system Ei sta%le +ormF +irst appeare! i .989 a! has, to some !egree,
!isplace! troff as the tool o+ choice +or typesetter o"tp"t. We woIt %e co#erig T
E
G
here, !"e %oth to its comple3ity Ethere are etire %ooks a%o"t itF a! to the +act that it is
ot istalle! %y !e+a"lt o most mo!er *i"3 systems.
Ti(5 0or those itereste! i istallig T
E
G, check o"t the texlive package
which ca %e +o"! i most !istri%"tio repositories, a! the *yG graphical cotet
e!itor.
groff
groff is s"ite o+ programs cotaiig the ;&) implemetatio o+ troff. 9t also
icl"!es a script that is "se! to em"late nroff a! the rest o+ the roff +amily as well.
(22
'oc"met 0ormattig Systems
While roff a! its !esce!ats are "se! to make +ormatte! !oc"mets, they !o it i a
way that is rather +oreig to mo!er "sers. >ost !oc"mets to!ay are pro!"ce! "sig
wor! processors that are a%le to per+orm %oth the compositio a! layo"t o+ a !oc"met
i a sigle step. 6rior to the a!#et o+ the graphical wor! processor, !oc"mets were
o+te pro!"ce! i a two-step process i#ol#ig the "se o+ a te3t e!itor to per+orm
compositio a! a processor, s"ch as troff to apply the +ormattig. 9str"ctios +or the
+ormattig program were em%e!!e! ito the compose! te3t thro"gh the "se o+ a mark"p
lag"age. The mo!er aalog +or s"ch a process is the we% page, which is compose!
"sig a te3t e!itor o+ some ki! a! the re!ere! %y a we% %rowser "sig AT>* as the
mark"p lag"age to !escri%e the +ial page layo"t.
WeIre ot goig to co#er groff i its etirety, as may elemets o+ its mark"p lag"age
!eal with rather arcae !etails o+ typography. 9stea! we will cocetrate o oe o+ its
macro packages that remais i wi!e "se. These macro packages co!ese may o+ its
low-le#el comma!s ito a smaller set o+ high-le#el comma!s that make "sig groff
m"ch easier.
0or a momet, letIs cosi!er the h"m%le ma page. 9t li#es i the 6usr6share6man
!irectory as a gCi7 compresse! te3t +ile. 9+ we were to e3amie its "compresse!
cotets, we wo"l! see the +ollowig Ethe ma page +or ls i sectio . is showF,
[me@linuxbox ~]$ -cat /usr/share/man/man1/ls.1.g- 0 head
<]_ ' LT +'A-U TSA* -A=&G At 9as generated b1 hel7!man #<$"<
<TS =* _#_ _47ril !((/_ _@L3 coreutils .<#(_ _3ser Commands_
<*S L4+&
ls ]- list director1 contents
<*S *UL:*A*
<H ls
[]fA:TAL]fR]<<< []fA-A=&]fR]<<<
<*S '&*CRA:TAL
<]_ 4dd an1 additional descri7tion here
<::
Compare! to the ma page i its ormal presetatio, we ca %egi to see a correlatio
%etwee the mark"p lag"age a! its res"lts,
[me@linuxbox ~]$ man ls 0 head
=*N#P 3ser Commands =*N#P
L4+&
ls - list director1 contents
(2(
22 8 0ormattig 7"tp"t
*UL:*A*
ls [:TAL]<<< [-A=&]<<<
The reaso this is o+ iterest is that ma pages are re!ere! %y groff, "sig the
mandoc macro package. 9 +act, we ca sim"late the man comma! with the +ollowig
pipelie,
[me@linuxbox ~]$ -cat /usr/share/man/man1/ls.1.g- 0 groff -mandoc -+
ascii 0 head
=*N#P 3ser Commands =*N#P
L4+&
ls - list director1 contents
*UL:*A*
ls [:TAL]<<< [-A=&]<<<
Aere we "se the groff program with the optios set to speci+y the mandoc macro
package a! the o"tp"t !ri#er +or $SC99. groff ca pro!"ce o"tp"t i se#eral +ormats.
9+ o +ormat is speci+ie!, 6ostScript is o"tp"t %y !e+a"lt,
[me@linuxbox ~]$ -cat /usr/share/man/man1/ls.1.g- 0 groff -mandoc 0
head
5G:*-4dobe-$<(
55Creator: groff version #<#/<#
55Creation'ate: Thu -eb " #$:%%:$) !((0
55'ocumentLeededResources: font Times-Roman
558 font Times-Hold
558 font Times-Atalic
55'ocument*u77liedResources: 7rocset gro7s #<#/ #
55:ages: %
55:agerder: 4scend
55rientation: :ortrait
We %rie+ly metioe! 6ostScript i the pre#io"s chapter, a! will agai i the e3t
chapter. 6ostScript is a page !escriptio lag"age that is "se! to !escri%e the cotets o+
a prite! page to a typesetter-like !e#ice. 9+ we take the o"tp"t o+ o"r comma! a! store
it to a +ile Eass"mig that we are "sig a graphical !esktop with a 'eskto7 !irectoryF,
(21
'oc"met 0ormattig Systems
[me@linuxbox ~]$ -cat /usr/share/man/man1/ls.1.g- 0 groff -mandoc &
~/1esktop/foo.ps
$ ico +or the o"tp"t +ile sho"l! appear o the !esktop. Jy !o"%le-clickig o the ico,
a page #iewer sho"l! start "p a! re#eal the +ile i its re!ere! +orm,
What we see is a icely typeset ma page +or lsL 9 +act, itIs possi%le to co#ert the
6ostScript +ile ito a 6'0 E6ortable -ocument FormatF +ile with this comma!,
[me@linuxbox ~]$ ps2pdf ~/1esktop/foo.ps ~/1esktop/ls.pdf
The 7s!7df program is part o+ the ghostscri7t package, which is istalle! o most
*i"3 systems that s"pport pritig.
Ti(, *i"3 systems o+te icl"!e may comma! lie programs +or +ile +ormat
(22
Figure $: Biewing 6ostScript &utput /ith ) 6age Biewer *n @>&!1
22 8 0ormattig 7"tp"t
co#ersio. They are o+te ame! "sig the co#etio o+ format2format. Try
"sig the comma! ls 6usr6bin6E[[:al7ha:]]![[:al7ha:]]E to
i!eti+y them. $lso try searchig +or programs ame! formattoformat.
0or o"r last e3ercise with groff, we will re#isit o"r ol! +rie! distros<txt oce
more. This time, we will "se the tbl program which is "se! to +ormat ta%les to typeset
o"r list o+ *i"3 !istri%"tios. To !o this, we are goig to "se o"r earlier sed script to
a!! mark"p to a te3t stream that we will +ee! to groff.
0irst, we ee! to mo!i+y o"r sed script to a!! the ecessary re="ests that tbl re="ires.
)sig a te3t e!itor, we will chage distros<sed to the +ollowig,
^ sed scri7t to 7roduce =inux distributions re7ort
1 iK
.+FK
center box#K
cb s sK
cb cb cbK
l n c.K
Tinux 1istributions HeportK
$K
?ame ]ersion HeleasedK
.
s6]N[(-0]][!]\]P]6]N[(-0]][!]\]P]6]N[(-0]][%]\]P$6]$-]#-]!6
3 aK
.+G
&ote that +or the script to work properly, care m"st %ee take to see that the wor!s
?&ame Rersio :elease!@ are separate! %y ta%s, ot spaces. WeIll sa#e the res"ltig +ile
as distros-tbl<sed. tbl "ses the <T* a! <T& re="ests to start a! e! the ta%le.
The rows +ollowig the <T* re="est !e+ie glo%al properties o+ the ta%le which, +or o"r
e3ample, are cetere! horiDotally o the page a! s"rro"!e! %y a %o3. The remaiig
lies o+ the !e+iitio !escri%e the layo"t o+ each ta%le row. &ow, i+ we r" o"r report-
geeratig pipelie agai with the ew sed script, weIll get the +ollowig ,
[me@linuxbox ~]$ sort -k 1;1 -k 2n distros.txt 0 sed -f distros-tbl
.sed 0 groff -t -+ ascii 2&/dev/null
8------------------------------8
Q Tinux 1istributions Heport Q
8------------------------------8
Q ?ame ]ersion Heleased Q
(2K
'oc"met 0ormattig Systems
8------------------------------8
Q-edora " !((.-($-!( Q
Q-edora . !((.-#(-!% Q
Q-edora ) !(()-("-$# Q
Q-edora / !(()-##-(/ Q
Q-edora 0 !((/-("-#$ Q
Q-edora #( !((/-##-!" Q
Q*3*& #(<# !((.-("-## Q
Q*3*& #(<! !((.-#!-() Q
Q*3*& #(<$ !(()-#(-(% Q
Q*3*& ##<( !((/-(.-#0 Q
Q3buntu .<(. !((.-(.-(# Q
Q3buntu .<#( !((.-#(-!. Q
Q3buntu )<(% !(()-(%-#0 Q
Q3buntu )<#( !(()-#(-#/ Q
Q3buntu /<(% !((/-(%-!% Q
Q3buntu /<#( !((/-#(-$( Q
8------------------------------8
$!!ig the -t optio to groff istr"cts it to pre-process the te3t stream with tbl.
*ikewise, the -T optio is "se! to o"tp"t to $SC99 rather tha the !e+a"lt o"tp"t me!i"m,
6ostScript.
The +ormat o+ the o"tp"t is the %est we ca e3pect i+ we are limite! to the capa%ilities o+ a
termial scree or typewriter-style priter. 9+ we speci+y 6ostScript o"tp"t a!
graphically #iew the res"ltig o"tp"t, we get a m"ch more satis+yig res"lt,
[me@linuxbox ~]$ sort -k 1;1 -k 2n distros.txt 0 sed -f distros-tbl
.sed 0 groff -t & ~/1esktop/foo.ps
(2/
22 8 0ormattig 7"tp"t
Summing -p
;i#e that te3t is so cetral to the character o+ )i3-like operatig systems, it makes
sese that there wo"l! %e may tools that are "se! to maip"late a! +ormat te3t. $s we
ha#e see, there areL The simple +ormattig tools like fmt a! 7r will +i! may "ses i
scripts that pro!"ce short !oc"mets, while groff Ea! +rie!sF ca %e "se! to write
%ooks. We may e#er write a techical paper "sig comma! lie tools Etho"gh there
are may people who !oLF, %"t itIs goo! to kow that we co"l!.
;urther 1eading
groff )serIs ;"i!e
http,--www.g".org-so+tware-gro++-ma"al-
/riting 6apers /ith nroff 0sing -me,
http,--!ocs.+ree%s!.org-11!oc-"s!-.9.memacros-paper.p!+
-me 9eference !anual,
(28
Figure .: Biewing The Finished Table
0"rther :ea!ig
http,--!ocs.+ree%s!.org-11!oc-"s!-20.mere+-paper.p!+
Tbl O ) 6rogram To Format Tables,
http,--pla9.%ell-la%s.com-.0thE!>a-t%l.p!+
$!, o+ co"rse, try the +ollowig articles at Wikipe!ia,
http,--e.wikipe!ia.org-wiki-TeG
http,--e.wikipe!ia.org-wiki-'oal!OM"th
http,--e.wikipe!ia.org-wiki-Typesettig
(29
2( 8 6ritig
23 2rinting
$+ter spe!ig the last co"ple o+ chapters maip"latig te3t, itIs time to p"t that te3t o
paper. 9 this chapter, weIll look at the comma! lie tools that are "se! to prit +iles a!
cotrol priter operatio. We woIt %e lookig at how to co+ig"re pritig, as that
#aries +rom !istri%"tio to !istri%"tio a! is "s"ally set "p a"tomatically !"rig
istallatio. &ote that we will ee! a workig priter co+ig"ratio to per+orm the
e3ercises i this chapter.
We will !isc"ss the +ollowig comma!s,
7r 8 Co#ert te3t +iles +or pritig
l7r 8 6rit +iles
a!7s 8 0ormat +iles +or pritig o a 6ostScript priter
l7stat 8 Show priter stat"s i+ormatio
l7B 8 Show priter ="e"e stat"s
l7rm 8 Cacel prit 4o%s
# /rie" 3istor, ?" Printing
To +"lly "!ersta! the pritig +eat"res +o"! i )i3-like operatig systems, we m"st
+irst lear some history. 6ritig o )i3-like systems goes way %ack to the %egiig o+
the operatig system itsel+. 9 those !ays, priters a! how they were "se! was m"ch
!i++eret +rom to!ay.
'rinting In The )im Times
*ike the comp"ters themsel#es, priters i the pre-6C era te!e! to %e large, e3pesi#e
a! cetraliDe!. The typical comp"ter "ser o+ .980 worke! at a termial coecte! to a
comp"ter some !istace away. The priter was locate! ear the comp"ter a! was "!er
the watch+"l eyes o+ the comp"terIs operators.
Whe priters were e3pesi#e a! cetraliDe!, as they o+te were i the early !ays o+
)i3, it was commo practice +or may "sers to share a priter. To i!eti+y prit 4o%s
((0
$ Jrie+ Aistory 7+ 6ritig
%elogig to a partic"lar "ser, a banner page !isplayig the ame o+ the "ser was o+te
prite! at the %egiig o+ each prit 4o%. The comp"ter s"pport sta++ wo"l! the loa! "p
a cart cotaiig the !ayIs prit 4o%s a! !eli#er them to the i!i#i!"al "sers.
Chara/terBbased 'rinters
The priter techology o+ the 80s was #ery !i++eret i two respects. 0irst, priters o+
that perio! were almost always impact printers. 9mpact priters "se a mechaical
mechaism which strikes a ri%%o agaist the paper to +orm character impressios o the
page. Two o+ the pop"lar techologies o+ that time were daisy wheel pritig a! dot-
matri+ pritig.
The seco!, a! more importat characteristic o+ early priters was that priters "se! a
+i3e! set o+ characters that were itrisic to the !e#ice itsel+. 0or e3ample, a !aisy wheel
priter co"l! oly prit the characters act"ally mol!e! ito the petals o+ the !aisy wheel.
This ma!e the priters m"ch like high-spee! typewriters. $s with most typewriters, they
prite! "sig moospace! E+i3e! wi!thF +ots. This meas that each character has the
same wi!th. 6ritig was !oe at +i3e! positios o the page, a! the prita%le area o+ a
page cotaie! a +i3e! "m%er o+ characters. >ost priters prite! te characters per
ich EC69F horiDotally a! si3 lies per ich E*69F #ertically. )sig this scheme, a )S
letter sheet o+ paper is eighty-+i#e characters wi!e a! si3ty-si3 lies high. Takig ito
acco"t a small margi o each si!e, eighty characters was cosi!ere! the ma3im"m
wi!th o+ a prit lie. This e3plais why termial !isplays Ea! o"r termial em"latorsF
are ormally eighty characters wi!e. 9t pro#i!es a /CS*/C@ E/hat Cou See *s /hat Cou
@etF #iew o+ prite! o"tp"t, "sig a moospace! +ot.
'ata is set to a typewriter-like priter i a simple stream o+ %ytes cotaiig the
characters to %e prite!. 0or e3ample, to prit a ?a@, the $SC99 character co!e 9/ is
set. 9 a!!itio, the low-"m%ere! $SC99 cotrol co!es pro#i!e! a meas o+ mo#ig
the priterIs carriage a! paper, "sig co!es +or carriage ret"r, lie +ee!, +orm +ee!, etc.
)sig the cotrol co!es, itIs possi%le to achie#e some limite! +ot e++ects, s"ch as %ol!
+ace, %y ha#ig the priter prit a character, %ackspace a! prit the character agai to get
a !arker prit impressio o the page. We ca act"ally witess this i+ we "se nroff to
re!er a ma page a! e3amie the o"tp"t "sig cat -4,
[me@linuxbox ~]$ -cat /usr/share/man/man1/ls.1.g- 0 nroff -man 0 cat
-: 0 head
=*N#P 3ser Commands =*N#P
$
$
$
L`SL4`S4+`S+&`S&$
ls - list director1 contents$
((.
2( 8 6ritig
$
*`S*U`SUL`SL`S:`S:*`S*A`SA*`S*$
l`Sls`Ss [O`SO`S:O`STO`SAO`SO`SL]<<< [O`S-O`SAO`S=O`S&]<<<$
The `S ECotrol-hF characters are the %ackspaces "se! to create the %ol!+ace e++ect.
*ikewise, we ca also see a %ackspace-"!erscore se="ece "se! to pro!"ce "!erliig.
+ra5hi/a 'rinters
The !e#elopmet o+ ;)9s le! to ma4or chages i priter techology. $s comp"ters
mo#e! to more pict"re-%ase! !isplays, pritig mo#e! +rom character-%ase! to graphical
techi="es. This was +acilitate! %y the a!#et o+ the low-cost laser priter which, istea!
o+ pritig +i3e! characters, co"l! prit tiy !ots aywhere i the prita%le area o+ the
page. This ma!e pritig proportioal +ots Elike those "se! %y typesettersF, a! e#e
photographs a! high ="ality !iagrams, possi%le.
Aowe#er, mo#ig +rom a character-%ase! scheme to a graphical scheme presete! a
+ormi!a%le techical challege. AereIs why, the "m%er o+ %ytes ee!e! to +ill a page
"sig a character-%ase! priter ca %e calc"late! this way Eass"mig si3ty lies per page
each cotaiig eighty charactersF,
K0 G 80 W 1800 %ytes
9 compariso, a three h"!re! !ot per ich E'69F laser priter Eass"mig a eight %y te
ich prit area per pageF re="ires,
E8 G (00F G E.0 G (00F - 8 W 900000 %ytes
>ay o+ the slow 6C etworks simply co"l! ot ha!le the early oe mega%yte o+ !ata
re="ire! to prit a +"ll page o a laser priter, so it was clear that a cle#er i#etio was
ee!e!.
That i#etio t"re! o"t to %e the page description language E6'*F. $ page !escriptio
lag"age is a programmig lag"age that !escri%es the cotets o+ a page. Jasically it
says, ?go to this positio, !raw the character eaI i te poit Ael#etica, go to this
positio...@ "til e#erythig o the page is !escri%e!. The +irst ma4or 6'* was
6ostScript +rom $!o%e Systems, which is still i wi!e "se to!ay. The 6ostScript
lag"age is a complete programmig lag"age tailore! +or typography a! other ki!s o+
graphics a! imagig. 9t icl"!es %"ilt-i s"pport +or thirty-+i#e sta!ar!, high-="ality
+ots, pl"s the a%ility to accept a!!itioal +ot !e+iitios at r" time. $t +irst, s"pport +or
6ostScript was %"ilt ito the priters themsel#es. This sol#e! the !ata trasmissio
pro%lem. While the typical 6ostScript program was #ery #er%ose i compariso to the
simple %yte stream o+ character-%ase! priters, it was m"ch smaller tha the "m%er o+
%ytes re="ire! to represet the etire prite! page.
$ 6ostScript printer accepte! a 6ostScript program as ip"t. The priter cotaie! its
((2
$ Jrie+ Aistory 7+ 6ritig
ow processor a! memory Eo+tetimes makig the priter a more power+"l comp"ter
tha the comp"ter to which it was attache!F a! e3ec"te! a special program calle! a
6ostScript interpreter, which rea! the icomig 6ostScript program a! rendered the
res"lts ito the priterIs iteral memory, th"s +ormig the patter o+ %its E!otsF that
wo"l! %e tras+erre! to the paper. The geeric ame +or this process o+ re!erig
somethig ito a large %it patter Ecalle! a bitmapF is raster image processor or :96.
$s the years wet %y, %oth comp"ters a! etworks %ecame m"ch +aster. This allowe!
the :96 to mo#e +rom the priter to the host comp"ter, which, i t"r, permitte! high
="ality priters to %e m"ch less e3pesi#e.
>ay priters to!ay still accept character-%ase! streams, %"t may low-cost priters !o
ot. They rely o the host comp"terIs :96 to pro#i!e a stream o+ %its to prit as !ots.
There are still some 6ostScript priters, too.
Printing +ith Linux
>o!er *i"3 systems employ two so+tware s"ites to per+orm a! maage pritig. The
+irst, C)6S ECommo )i3 6ritig SystemF pro#i!es prit !ri#ers a! prit 4o%
maagemet , a! the seco!, ;hostscript, a 6ostScript iterpreter, acts as a :96.
C)6S maages priters %y creatig a! maitaiig prit ="e"es. $s we !isc"sse! i
o"r history lesso a%o#e, )i3 pritig was origially !esige! to maage a cetraliDe!
priter share! %y m"ltiple "sers. Sice priters are slow %y at"re, compare! to the
comp"ters that are +ee!ig them, pritig systems ee! a way to sche!"le m"ltiple prit
4o%s a! keep thigs orgaiDe!. C)6S also has the a%ility to recogiDe !i++eret types o+
!ata Ewithi reasoF a! ca co#ert +iles to a prita%le +orm.
Preparing ;iles ;or Printing
$s comma! lie "sers, we are mostly itereste! i pritig te3t, tho"gh it is certaily
possi%le to prit other !ata +ormats as well.
7r A Con;ert Text 2ies 2or 'rinting
We looke! at 7r a little i the pre#io"s chapter. &ow we will e3amie some o+ its may
optios "se! i co4"ctio with pritig. 9 o"r history o+ pritig, we saw how
character-%ase! priters "se moospace! +ots, res"ltig i +i3e! "m%ers o+ characters
per lie a! lies per page. 7r is "se! to a!4"st te3t to +it o a speci+ic page siDe, with
optioal page hea!ers a! margis. Aere is a s"mmary o+ its most commoly "se!
optios,
(((
2( 8 6ritig
Table '3-1: %ommon pr &ptions
?ption Description
8first[:last]
7"tp"t a rage o+ pages startig with first a!, optioally,
e!ig with last.
-columns
7rgaiDe the cotet o+ the page ito the "m%er o+ col"ms
speci+ie! %y columns.
-a
Jy !e+a"lt, m"lti-col"m o"tp"t is liste! #ertically. Jy
a!!ig the -a EacrossF optio, cotet is liste! horiDotally.
-d
'o"%le-space o"tp"t.
-' cformatd
0ormat the !ate !isplaye! i page hea!ers "sig format. See
the ma page +or the date comma! +or a !escriptio o+ the
+ormat strig.
-f
)se +orm +ee!s rather tha carriage ret"rs to separate pages.
-h cheaderd
9 the ceter portio o+ the page hea!er, "se header rather the
ame o+ the +ile %eig processe!.
-l length
Set page legth to length. 'e+a"lt is KK E)S letter at K lies
per ichF
-n
&"m%er lies.
-o offset
Create a le+t margi offset characters wi!e.
-9 width
Set page wi!th to width. 'e+a"lt is /2.
7r is o+te "se! i pipelies as a +ilter. 9 this e3ample, we will pro!"ce a !irectory
listig o+ 6usr6bin a! +ormat it ito pagiate! three-col"m o"tp"t "sig 7r,
[me@linuxbox ~]$ ls /usr/bin 0 pr - -w N" 0 head
!((0-(!-#/ #%:(( :age #
[ a7turl bsd-9rite
%##to77m ar bsh
a!7 arecord btcflash
a!7s arecordmidi bug-budd1
a!7s-l7r-9ra77er ark buildhash
((1
6reparig 0iles 0or 6ritig
Sending # Print Jo0 To # Printer
The C)6S pritig s"ite s"pports two metho!s o+ pritig historically "se! o )i3-like
systems. 7e metho!, calle! Jerkeley or *6' E"se! i the Jerkeley So+tware
'istri%"tio #ersio o+ )i3F, "ses the l7r program, while the other metho!, calle!
SysR E+rom the System R #ersio o+ )i3F, "ses the l7 program. Joth programs !o
ro"ghly the same thig. Choosig oe o#er the other is a matter o+ persoal taste.
l7r A 'rint 2ies EBerkeey StyeF
The l7r program ca %e "se! to se! +iles to the priter. 9t may also "se! i pipelies,
as it accepts sta!ar! ip"t. 0or e3ample, to prit the res"lts o+ o"r m"lti-col"m
!irectory listig a%o#e, we co"l! !o this,
[me@linuxbox ~]$ ls /usr/bin 0 pr - 0 lpr
a! the report wo"l! %e set to the systemIs !e+a"lt priter. To se! the +ile to a !i++eret
priter, the -: optio ca "se! like this,
lpr -B printer_name
where printername is the ame o+ the !esire! priter. To see a list o+ priters kow to
the system,
[me@linuxbox ~]$ lpstat -a
Ti(5 >ay *i"3 !istri%"tios allow yo" to !e+ie a ?priter@ that o"tp"ts +iles i
6'0 E6orta%le 'oc"met 0ormatF +ormat, rather tha pritig o the physical
priter. This is #ery ha!y +or e3perimetig with pritig comma!s. Check
yo"r priter co+ig"ratio program to see i+ it s"pports this co+ig"ratio. 7 some
!istri%"tios, yo" may ee! to istall a!!itioal packages Es"ch as cu7s-7dfF to
ea%le this capa%ility.
Aere are some o+ the commo optios +or l7r,
((2
2( 8 6ritig
Table '3-': %ommon lpr &ptions
?ption Description
-^ number
Set "m%er o+ copies to number.
-7
6rit each page with a sha!e! hea!er with the !ate, time, 4o%
ame a! page "m%er. This so-calle! ?pretty prit@ optio
ca %e "se! whe pritig te3t +iles.
-: printer
Speci+y the ame o+ the priter "se! +or o"tp"t. 9+ o priter is
speci+ie!, the systemIs !e+a"lt priter is "se!.
-r
'elete +iles a+ter pritig. This wo"l! %e "se+"l +or programs
that pro!"ce temporary priter o"tp"t +iles.
l7 A 'rint 2ies ESystem ? StyeF
*ike l7r, l7 accepts either +iles or sta!ar! ip"t +or pritig. 9t !i++ers +rom l7r i
that it s"pports a !i++eret Ea! slightly more sophisticate!F optio set. Aere are the
commo oes,
Table '3-3: %ommon lp &ptions
?ption Description
-d printer
Set the !estiatio EpriterF to printer. 9+ o !
optio is speci+ie!, the system !e+a"lt priter is
"se!.
-n number
Set the "m%er o+ copies to number.
-o landsca7e
Set o"tp"t to la!scape orietatio.
-o fit7lot
Scale the +ile to +it the page. This is "se+"l whe
pritig images, s"ch as J6E; +iles.
-o scalingMnumber
Scale +ile to number. The #al"e o+ .00 +ills the
page. Ral"es less tha .00 are re!"ce!, while
#al"es greater tha .00 ca"se the +ile to %e prite!
across m"ltiple pages.
-o c7iMnumber
Set the o"tp"t characters per ich to number.
'e+a"lt is .0.
-o l7iMnumber
Set the o"tp"t lies per ich to number. 'e+a"lt
is K.
((K
Se!ig $ 6rit Jo% To $ 6riter
-o 7age-bottomMpoints
-o 7age-leftMpoints
-o 7age-rightMpoints
-o 7age-to7Mpoints
Set the page margis. Ral"es are e3presse! i
points, a "it o+ typographic meas"remet. There
are /2 poits to a ich.
-: pages
Speci+y the list o+ pages. pages may %e e3presse!
as a comma separate! list a!-or a rage. 0or
e3ample ?.,(,2,/-.0@
WeIll pro!"ce o"r !irectory listig agai, this time pritig twel#e C69 a! eight *69
with a le+t margi o+ oe hal+ ich. &ote that we ha#e to a!4"st the 7r optios to acco"t
+or the ew page siDe,
[me@linuxbox ~]$ ls /usr/bin 0 pr -A -w D* -l MM 0 lp -o page-left$N
-o cpi$12 -o lpi$M
This pipelie pro!"ces a +o"r col"m listig "sig smaller type tha the !e+a"lt. The
icrease! "m%er o+ characters per ich allows "s to +it more col"ms o the page.
Another <5tionQ a!7s
The a!7s program is iterestig. $s we ca s"rmise +rom it ame, itIs a +ormat
co#ersio program, %"t it also m"ch more. 9ts ame origially meat ?$SC99 to
6ostScript@ a! it was "se! to prepare te3t +iles +or pritig o 6ostScript priters. 7#er
the years, howe#er, the capa%ilities o+ the program ha#e grow, a! ow its ame meas
?$ythig to 6ostScript.@ While its ame s"ggest a +ormat co#ersio program, it is
act"ally a pritig program. 9t se!s its !e+a"lt o"tp"t to the systemIs !e+a"lt priter
rather tha sta!ar! o"tp"t. The programIs !e+a"lt %eha#ior is that o+ a ?pretty priter,@
meaig that it impro#es the appearace o+ o"tp"t. 9+ we "se the program to create a
6ostScript +ile o o"r !esktop,
[me@linuxbox ~]$ ls /usr/bin 0 pr - -t 0 a2ps -o ~/1esktop/ls.ps -T
NN
[stdin N7lainP: ## 7ages on . sheets]
[Total: ## 7ages on . sheets] saved into the file J6home6me6'eskto76
ls<7sK
Aere we +ilter the stream with 7r, "sig the -t optio Eomit hea!ers a! +ootersF a!
the with a!7s, speci+yig a o"tp"t +ile E-o optioF a! KK lies per page E-= optioF
((/
2( 8 6ritig
to match the o"tp"t pagiatio o+ 7r. 9+ we #iew the res"ltig +ile with a s"ita%le +ile
#iewer, we will see this,
$s we ca see, the !e+a"lt o"tp"t layo"t is ?two "p@ +ormat. This ca"ses the cotets o+
two pages to %e prite! o each sheet o+ paper. a!7s applies ice page hea!ers a!
+ooters, too.
a!7s has a lot o+ optios. Aere is a s"mmary,
Table '3-$: a'ps &ptions
?ption Description
--center-title text
Set ceter page title to te+t.
--columns number
$rrage pages ito number col"ms. 'e+a"lt
is 2.
((8
Figure 2: Biewing a'ps &utput
Se!ig $ 6rit Jo% To $ 6riter
--footer text
Set page +ooter to te+t.
--guess
:eport the types o+ +iles gi#e as arg"mets.
Sice a!7s tries to co#ert a! +ormat all
types o+ !ata, this optio ca %e "se+"l +or
pre!ictig what a!7s will !o whe gi#e a
partic"lar +ile.
--left-footer text
Set le+t page +ooter to te+t.
--left-title text
Set le+t page title to te+t.
--line-numbersMinterval
&"m%er lies o+ o"tp"t e#ery interval lies.
--listMdefaults
'isplay !e+a"lt settigs.
--listMtopic
'isplay settigs +or topic, where topic is oe
o+ the +ollowig, !elegatios Ee3teral
programs that will %e "se! to co#ert !ataF,
eco!igs, +eat"res, #aria%les, me!ia Epaper
siDes a! the likeF, pp! E6ostScript priter
!escriptiosF, priters, prolog"es Eportios o+
co!e that are pre+i3e! to ormal o"tp"tF, style-
sheets, a! "ser optios.
--7ages range
6rit pages i rage.
--right-footer text
Set right page +ooter to te+t.
--right-title text
Set right page title to te+t.
--ro9s number
$rrage pages ito number rows. 'e+a"lt is
oe.
-H
&o page hea!ers.
-b text
Set page hea!er to te+t.
-f size
)se si7e poit +ot.
-l number
Set characters per lie to number. This a! the
-= optio E%elowF ca %e "se! to make +iles
pagiate! with other programs, s"ch as 7r, +it
correctly o the page.
-= number
Set lies per page to number.
-+ name
)se me!ia ame. 0or e3ample ?$1@.
-n number
7"tp"t number copies o+ each page.
((9
2( 8 6ritig
-o file
Se! o"tp"t to file. 9+ file is speci+ie! as ?-@,
"se sta!ar! o"tp"t.
-: printer
)se printer. 9+ a priter is ot speci+ie!, the
system !e+a"lt priter is "se!.
-R
6ortrait orietatio.
-r
*a!scape orietatio.
-T number
Set ta% stops to e#ery number characters.
-u text
)!erlay EwatermarkF pages with te+t.
This is 4"st a s"mmary. a!7s has se#eral more optios.
1ote5 a!7s is still i acti#e !e#elopmet. '"rig my testig, 9 otice! !i++eret
%eha#ior o #ario"s !istri%"tios. 7 Cet7S 1, o"tp"t always wet to sta!ar!
o"tp"t %y !e+a"lt. 7 Cet7S 1 a! 0e!ora .0, o"tp"t !e+a"lte! to $1 me!ia,
!espite the program %eig co+ig"re! to "se letter siDe me!ia %y !e+a"lt. 9 co"l!
o#ercome these iss"es %y e3plicitly speci+yig the !esire! optio. 7 )%"t" 8.01,
it per+orme! as !oc"mete!.
$lso ote that there is aother o"tp"t +ormatter that is "se+"l +or co#ertig te3t
ito 6ostScript. Calle! enscri7t, it ca per+orm may o+ the same ki!s o+
+ormattig a! pritig tricks, %"t "like a!7s, it oly accepts te3t ip"t.
>onitoring #nd Controlling Print Jo0s
$s )i3 pritig systems are !esige! to ha!le m"ltiple prit 4o%s +rom m"ltiple "sers,
C)6S is !esige! to !o the same. Each priter is gi#e a print <ueue, where 4o%s are
parke! "til they ca %e spooled to the priter. C)6S s"pplies se#eral comma! lie
programs that are "se! to maage priter stat"s a! prit ="e"es. *ike the l7r a! l7
programs, these maagemet programs are mo!ele! a+ter the correspo!ig programs
+rom the Jerkeley a! System R pritig systems.
l7stat A )is5ay 'rint System Status
The l7stat program is "se+"l +or !etermiig the ames a! a#aila%ility o+ priters o
the system. 0or e3ample, i+ we ha! a system with %oth a physical priter Eame!
?priter@F a! a 6'0 #irt"al priter Eame! ?6'0@F, we co"l! check their stat"s like this,
(10
>oitorig $! Cotrollig 6rit Jo%s
[me@linuxbox ~]$ lpstat -a
:'- acce7ting reBuests since +on (/ 'ec !((/ ($:(":"0 :+ &*T
7rinter acce7ting reBuests since Tue !% -eb !((0 (/:%$:!! 4+ &*T
0"rther, we co"l! !etermie a more !etaile! !escriptio o+ the prit system co+ig"ratio
this way,
[me@linuxbox ~]$ lpstat -s
s1stem default destination: 7rinter
device for :'-: cu7s-7df:6
device for 7rinter: i77:667rint-server:.$#67rinters67rinter
9 this e3ample, we see that ?priter@ is the systemIs !e+a"lt priter a! that it is a
etwork priter "sig 9teret 6ritig 6rotocol Eipp,--F attache! to a system ame!
?prit-ser#er@.
The commoly "se+"l optios icl"!e,
Table '3-.: %ommon lpstat &ptions
?ption Description
-a [printer<<<]
'isplay the state o+ the priter ="e"e +or printer. &ote
that this is the stat"s o+ the priter ="e"eIs a%ility to accept
4o%s, ot the stat"s o+ the physical priters. 9+ o priters
are speci+ie!, all prit ="e"es are show.
-d
'isplay the ame o+ systemIs !e+a"lt priter.
-7 [printer<<<]
'isplay the stat"s o+ the speci+ie! printer. 9+ o priters
are speci+ie!, all priters are show.
-r
'isplay the stat"s o+ the prit ser#er.
-s
'isplay a stat"s s"mmary.
-t
'isplay a complete stat"s report.
l7B A )is5ay 'rinter Dueue Status
To see the stat"s o+ a priter ="e"e, the l7B program is "se!. This allows "s to #iew the
stat"s o+ the ="e"e a! the prit 4o%s it cotais. Aere is a e3ample o+ a empty ="e"e
+or a system !e+a"lt priter ame! ?priter@,
(1.
2( 8 6ritig
[me@linuxbox ~]$ lp,
7rinter is read1
no entries
9+ we !o ot speci+y a priter E"sig the -: optioF, the systemIs !e+a"lt priter is show.
9+ we se! a 4o% to the priter a! the look at the ="e"e, we will see it liste!,
[me@linuxbox ~]$ ls *.txt 0 pr - 0 lp
reBuest id is 7rinter-.($ N# fileNsPP
[me@linuxbox ~]$ lp,
7rinter is read1 and 7rinting
Rank 9ner ?ob -ileNsP Total *iCe
active me .($ NstdinP #(!% b1tes
l7rm - cancel A Can/e 'rint Gobs
C)6S s"pplies two programs "se! to termiate prit 4o%s a! remo#e them +rom the prit
="e"e. 7e is Jerkeley style El7rmF a! the other is System R EcancelF. They !i++er
slightly i the optios they s"pport, %"t !o %asically the same thig. )sig o"r prit 4o%
a%o#e as a e3ample, we co"l! stop the 4o% a! remo#e it this way,
[me@linuxbox ~]$ cancel N*
[me@linuxbox ~]$ lp,
7rinter is read1
no entries
Each comma! has optios +or remo#ig all the 4o%s %elogig to a partic"lar "ser,
partic"lar priter, a! m"ltiple 4o% "m%ers. Their respecti#e ma pages ha#e all the
!etails.
Summing -p
9 this chapter, we ha#e see how the priters o+ the past i+l"ece! the !esig o+ the
pritig systems o )i3-like machies, a! how m"ch cotrol is a#aila%le o the
comma! lie to cotrol ot oly the sche!"lig a! e3ec"tio o+ prit 4o%s, %"t also the
#ario"s o"tp"t optios.
;urther 1eading
$ goo! article o the 6ostScript page !escriptio lag"age,
(12
0"rther :ea!ig
http,--e.wikipe!ia.org-wiki-6ostScript
The Commo )i3 6ritig System EC)6SF,
http,--e.wikipe!ia.org-wiki-CommoO)i3O6ritigOSystem
http,--www.c"ps.org-
The Jerkeley a! System R 6ritig Systems,
http,--e.wikipe!ia.org-wiki-JerkeleyOpritigOsystem
http,--e.wikipe!ia.org-wiki-SystemOROpritigOsystem
(1(
21 8 Compilig 6rograms
2 Co%#iling 2rogra%s
9 this chapter, we will look at how to %"il! programs %y compilig so"rce co!e. The
a#aila%ility o+ so"rce co!e is the essetial +ree!om that makes *i"3 possi%le. The etire
ecosystem o+ *i"3 !e#elopmet relies o +ree e3chage %etwee !e#elopers. 0or may
!esktop "sers, compilig is a lost art. 9t "se! to %e ="ite commo, %"t to!ay, !istri%"tio
pro#i!ers maitai h"ge repositories o+ pre-compile! %iaries, rea!y to !owloa! a!
"se. $t the time o+ this writig, the 'e%ia repository Eoe o+ the largest o+ ay o+ the
!istri%"tiosF cotais almost twety-three tho"sa! packages.
So why compile so+twareC There are two reasos,
.. #vailability. 'espite the "m%er o+ pre-compile! programs i !istri%"tio
repositories, some !istri%"tios may ot icl"!e all the !esire! applicatios. 9
this case, the oly way to get the !esire! program is to compile it +rom so"rce.
2. Timeliness. While some !istri%"tios specialiDe i c"ttig e!ge #ersios o+
programs, may !o ot. This meas that i or!er to ha#e the #ery latest #ersio
o+ a program, compilig is ecessary.
Compilig so+tware +rom so"rce co!e ca %ecome #ery comple3 a! techicalH well
%eyo! the reach o+ may "sers. Aowe#er, may compilig tasks are ="ite easy a!
i#ol#e oly a +ew steps. 9t all !epe!s o the package. We will look at a #ery simple
case i or!er to pro#i!e a o#er#iew o+ the process a! as a startig poit +or those who
wish to "!ertake +"rther st"!y.
We will itro!"ce oe ew comma!,
make 8 )tility To >aitai 6rograms
+hat Is Compiling.
Simply p"t, compilig is the process o+ traslatig source code Ethe h"ma-rea!a%le
!escriptio o+ a program writte %y a programmerF ito the ati#e lag"age o+ the
comp"terIs processor.
The comp"terIs processor Eor %60F works at a #ery elemetal le#el, e3ec"tig programs
i what is calle! machine language. This is a "meric co!e that !escri%es #ery small
operatios, s"ch as ?a!! this %yte,@ ?poit to this locatio i memory,@ or ?copy this
(11
What 9s CompiligC
%yte.@ Each o+ these istr"ctios is e3presse! i %iary Eoes a! DerosF. The earliest
comp"ter programs were writte "sig this "meric co!e, which may e3plai why
programmers who wrote it were sai! to smoke a lot, !rik gallos o+ co++ee, a! wear
thick glasses.
This pro%lem was o#ercome %y the a!#et o+ assembly language, which replace! the
"meric co!es with EslightlyF easier to "se character mnemonics s"ch as C65 E+or copyF
a! >7R E+or mo#eF. 6rograms writte i assem%ly lag"age are processe! ito
machie lag"age %y a program calle! a assembler. $ssem%ly lag"age is still "se!
to!ay +or certai specialiDe! programmig tasks, s"ch as device drivers a! embedded
systems.
We e3t come to what are calle!, high-level programming languages. They are calle!
this %eca"se they allow the programmer to %e less cocere! with the !etails o+ what the
processor is !oig a! more with sol#ig the pro%lem at ha!. The early oes
E!e#elope! !"rig the .920sF icl"!e! Fortran E!esige! +or scieti+ic a! techical
tasksF a! %&8&( E!esige! +or %"siess applicatiosF. Joth are still i limite! "se
to!ay.
While there are may pop"lar programmig lag"ages, two pre!omiate. >ost programs
writte +or mo!er systems are writte i either % or %PP. 9 the e3amples to +ollow, we
will %e compilig a C program.
6rograms writte i high-le#el programmig lag"ages are co#erte! ito machie
lag"age %y processig them with aother program, calle! a compiler. Some compilers
traslate high-le#el istr"ctios ito assem%ly lag"age a! the "se a assem%ler to
per+orm the +ial stage o+ traslatio ito machie lag"age.
$ process o+te "se! i co4"ctio with compilig is calle! linking. There are may
commo tasks per+orme! %y programs. Take, +or istace, opeig a +ile. >ay
programs per+orm this task, %"t it wo"l! %e waste+"l to ha#e each program implemet its
ow ro"tie to ope +iles. 9t makes more sese to ha#e a sigle piece o+ programmig
that kows how to ope +iles a! to allow all programs that ee! it to share it. This is
accomplishe! %y what are calle! libraries. They cotai m"ltiple routines, each
per+ormig some commo task that m"ltiple programs ca share. 9+ we look i the 6lib
a! 6usr6lib !irectories, we ca see where may o+ them li#e. $ program calle! a
linker is "se! to +orm the coectios %etwee the o"tp"t o+ the compiler a! the li%raries
that the compile! program re="ires. The +ial res"lt o+ this process is the e+ecutable
program file, rea!y +or "se.
Are A 'rograms Com5ied?
&o. $s we ha#e see, there are programs s"ch as shell scripts that !o ot re="ire
compilig. They are e3ec"te! !irectly. These are writte i what are kow as scripting
or interpreted lag"ages. These lag"ages ha#e grow i pop"larity i recet years a!
(12
21 8 Compilig 6rograms
icl"!e 6erl, 6ython, 6,6, :uby a! may others.
Scripte! lag"ages are e3ec"te! %y a special program calle! a interpreter. $
iterpreter ip"ts the program +ile a! rea!s a! e3ec"tes each istr"ctio cotaie!
withi it. 9 geeral, iterprete! programs e3ec"te m"ch more slowly tha compile!
programs. This is !"e to the +act that each so"rce co!e istr"ctio i a iterprete!
program is traslate! e#ery time it is carrie! o"t, whereas with a compile! program, a
so"rce co!e istr"ctio is oly traslate! oce, a! this traslatio is permaetly
recor!e! i the +ial e3ec"ta%le +ile.
So why are iterprete! lag"ages so pop"larC 0or may programmig chores, the res"lts
are ?+ast eo"gh,@ %"t the real a!#atage is that it is geerally +aster a! easier to !e#elop
iterprete! programs tha compile! programs. 6rograms are "s"ally !e#elope! i a
repeatig cycle o+ co!e, compile, test. $s a program grows i siDe, the compilatio phase
o+ the cycle ca %ecome ="ite log. 9terprete! lag"ages remo#e the compilatio step
a! th"s spee! "p program !e#elopmet.
Compiling # C Program
*etIs compile somethig. Je+ore we !o that howe#er, weIre goig to ee! some tools
like the compiler, liker, a! make. The C compiler "se! almost "i#ersally i the *i"3
e#iromet is calle! gcc E;&) C CompilerF, origially writte %y :ichar! Stallma.
>ost !istri%"tios !o ot istall gcc %y !e+a"lt. We ca check to see i+ the compiler is
preset like this,
[me@linuxbox ~]$ which gcc
6usr6bin6gcc
The res"lts i this e3ample i!icate that the compiler is istalle!.
Ti(, 5o"r !istri%"tio may ha#e a meta-package Ea collectio o+ packagesF +or
so+tware !e#elopmet. 9+ so, cosi!er istallig it i+ yo" ite! to compile
programs o yo"r system. 9+ yo"r system !oes ot pro#i!e a meta-package, try
istallig the gcc a! make packages. 7 may !istri%"tios, this is s"++iciet to
carry o"t the e3ercise %elow.
<btaining The Sour/e Code
0or o"r compilig e3ercise, we are goig to compile a program +rom the ;&) 6ro4ect
calle! diction. This is a ha!y little program that checks te3t +iles +or writig ="ality
a! style. $s programs go, it is +airly small a! easy to %"il!.
(1K
Compilig $ C 6rogram
0ollowig co#etio, weIre +irst goig to create a !irectory +or o"r so"rce co!e ame!
src a! the !owloa! the so"rce co!e ito it "sig ft7,
[me@linuxbox ~]$ mkdir src
[me@linuxbox ~]$ cd src
[me@linuxbox src]$ ftp ftp.gnu.org
Connected to ft7<gnu<org<
!!( @L3 -T: server read1<
Lame Nft7<gnu<org:meP: anon!mous
!$( =ogin successful<
Remote s1stem t17e is 3LAV<
3sing binar1 mode to transfer files<
ft7D cd gnu/diction
!"( 'irector1 successfull1 changed<
ft7D ls
!(( :RT command successful< Consider using :4*;<
#"( Sere comes the director1 listing<
-r9-r--r-- # #(($ .""$% ./0%( 4ug !/ #00/ diction-(<)<tar<gC
-r9-r--r-- # #(($ .""$% 0(0") +ar (% !((! diction-#<(!<tar<gC
-r9-r--r-- # #(($ .""$% #%#(.! *e7 #) !(() diction-#<##<tar<gC
!!. 'irector1 send 2<
ft7D get diction-1.11.tar.g-
local: diction-#<##<tar<gC remote: diction-#<##<tar<gC
!(( :RT command successful< Consider using :4*;<
#"( 7ening HAL4RU mode data connection for diction-#<##<tar<gC
N#%#(.! b1tesP<
!!. -ile send 2<
#%#(.! b1tes received in (<#. secs N/%)<% kH6sP
ft7D b!e
!!# @oodb1e<
[me@linuxbox src]$ ls
diction-#<##<tar<gC
1ote5 Sice we are the ?maitaier@ o+ this so"rce co!e while we compile it, we
will keep it i ~6src. So"rce co!e istalle! %y yo"r !istri%"tio will %e istalle!
i 6usr6src, while so"rce co!e ite!e! +or "se %y m"ltiple "sers is "s"ally
istalle! i 6usr6local6src.
$s we ca see, so"rce co!e is "s"ally s"pplie! i the +orm o+ a compresse! tar +ile.
Sometimes calle! a ?tarball,@ this +ile cotais the source tree, or hierarchy o+ !irectories
a! +iles that comprise the so"rce co!e. $+ter arri#ig at the +tp site, we e3amie the list
o+ tar +iles a#aila%le a! select the ewest #ersio +or !owloa!. )sig the get
comma! withi ft7, we copy the +ile +rom the +tp ser#er to the local machie.
(1/
21 8 Compilig 6rograms
7ce the tar +ile is !owloa!e!, it m"st %e "packe!. This is !oe with the tar
program,
[me@linuxbox src]$ tar x-f diction-1.11.tar.g-
[me@linuxbox src]$ ls
diction-#<## diction-#<##<tar<gC
Ti(5 The diction program, like all ;&) 6ro4ect so+tware, +ollows certai
sta!ar!s +or so"rce co!e packagig. >ost other so"rce co!e a#aila%le i the
*i"3 ecosystem also +ollows this sta!ar!. 7e elemet o+ the sta!ar! is that %y
"packig the so"rce co!e tar +ile, a !irectory will %e create! which cotais the
so"rce tree, a! that this !irectory will %e ame! proAect-+#++, th"s cotaiig %oth
the pro4ectIs ame a! #ersio "m%er. This scheme allows easy istallatio o+
m"ltiple #ersios o+ the same program. Aowe#er, it is o+te a goo! i!ea to e3amie
the layo"t o+ the tree %e+ore "packig it. Some pro4ects will ot create the
!irectory, %"t istea! will !eli#er the +iles !irectly ito the c"rret !irectory. This
will make a mess i yo"r otherwise well-orgaiDe! src !irectory. To a#oi! this,
"se the +ollowig comma! to e3amie the cotets o+ the tar +ile,
tar tCvf tarfile Q head
6xamining The Sour/e Tree
)packig the tar +ile res"lts i the creatio o+ a ew !irectory, ame! diction-#<##.
This !irectory cotais the so"rce tree. *etIs look isi!e,
[me@linuxbox src]$ cd diction-1.11
[me@linuxbox diction-#<##]$ ls
config<guess diction<c geto7t<c nl
config<h<in diction<7ot geto7t<h nl<7o
config<sub diction<s7ec geto7tOint<h R&4'+&
configure diction<s7ec<in AL*T4== sentence<c
configure<in diction<texi<in install-sh sentence<h
C:UAL@ en +akefile<in st1le<#<in
de enO@H misc<c st1le<c
de<7o enO@H<7o misc<h test
diction<#<in geto7t#<c L&,*
9 it, we see a "m%er o+ +iles. 6rograms %elogig to the ;&) 6ro4ect, as well as may
(18
Compilig $ C 6rogram
others, will s"pply the !oc"metatio +iles R&4'+&, AL*T4==, L&,*, a! C:UAL@.
These +iles cotai the !escriptio o+ the program, i+ormatio o how to %"il! a!
istall it, a! its licesig terms. 9t is aways a goo! i!ea to care+"lly rea! the R&4'+&
a! AL*T4== +iles %e+ore attemptig to %"il! the program.
The other iterestig +iles i this !irectory are the oes e!ig with <c a! <h,
[me@linuxbox diction-#<##]$ ls *.c
diction<c geto7t#<c geto7t<c misc<c sentence<c st1le<c
[me@linuxbox diction-#<##]$ ls *.h
geto7t<h geto7tOint<h misc<h sentence<h
The <c +iles cotai the two C programs s"pplie! %y the package Est1le a!
dictionF, !i#i!e! ito mo!"les. 9t is commo practice +or large programs to %e %roke
ito smaller, easier to maage pieces. The so"rce co!e +iles are or!iary te3t a! ca %e
e3amie! with less,
[me@linuxbox diction-#<##]$ less diction.c
The <h +iles are kow as header files. These, too, are or!iary te3t. Aea!er +iles
cotai !escriptios o+ the ro"ties icl"!e! i a so"rce co!e +ile or li%rary. 9 or!er +or
the compiler to coect the mo!"les, it m"st recei#e a !escriptio o+ all the mo!"les
ee!e! to complete the etire program. &ear the %egiig o+ the diction<c +ile, we
see this lie,
^include _geto7t<h_
This istr"cts the compiler to rea! the +ile geto7t<h as it rea!s the so"rce co!e i
diction<c i or!er to ?kow@ whatIs i geto7t<c. The geto7t<c +ile s"pplies
ro"ties that are share! %y %oth the st1le a! diction programs.
$%o#e the include statemet +or geto7t<h, we see some other include statemets
s"ch as these,
^include Xregex<hD
^include Xstdio<hD
^include Xstdlib<hD
^include Xstring<hD
(19
21 8 Compilig 6rograms
^include Xunistd<hD
These also re+er to hea!er +iles, %"t they re+er to hea!er +iles that li#e o"tsi!e the c"rret
so"rce tree. They are s"pplie! %y the system to s"pport the compilatio o+ e#ery
program. 9+ we look i 6usr6include, we ca see them,
[me@linuxbox diction-#<##]$ ls /usr/include
The hea!er +iles i this !irectory were istalle! whe we istalle! the compiler.
Buiding The 'rogram
>ost programs %"il! with a simple two comma! se="ece,
<6configure
make
The configure program is a shell script which is s"pplie! with the so"rce tree. 9ts 4o%
is to aalyDe the build environment. >ost so"rce co!e is !esige! to %e portable. That
is, it is !esige! to %"il! o more tha oe ki! o+ )i3-like system. J"t i or!er to !o
that, the so"rce co!e may ee! to "!ergo slight a!4"stmets !"rig the %"il! to
accommo!ate !i++ereces %etwee systems. configure also checks to see that
ecessary e3teral tools a! compoets are istalle!. *etIs r" configure. Sice
configure is ot locate! where the shell ormally e3pects programs to %e locate!, we
m"st e3plicitly tell the shell its locatio %y pre+i3ig the comma! with <6 to i!icate
that the program is locate! i the c"rret workig !irectory,
[me@linuxbox diction-#<##]$ ./configure
configure will o"tp"t a lot o+ messages as it tests a! co+ig"res the %"il!. Whe it
+iishes, it will look somethig like this,
checking libintl<h 7resence<<< 1es
checking for libintl<h<<< 1es
checking for librar1 containing gettext<<< none reBuired
configure: creating <6config<status
(20
Compilig $ C 6rogram
config<status: creating +akefile
config<status: creating diction<#
config<status: creating diction<texi
config<status: creating diction<s7ec
config<status: creating st1le<#
config<status: creating test6rundiction
config<status: creating config<h
[me@linuxbox diction-#<##]$
WhatIs importat here is that there are o error messages. 9+ there were, the
co+ig"ratio +aile!, a! the program will ot %"il! "til the errors are correcte!.
We see configure create! se#eral ew +iles i o"r so"rce !irectory. The most
importat oe is +akefile. +akefile is a co+ig"ratio +ile that istr"cts the make
program e3actly how to %"il! the program. Witho"t it, make will re+"se to r".
+akefile is a or!iary te3t +ile, so we ca #iew it,
[me@linuxbox diction-#<##]$ less Oakefile
The make program takes as ip"t a makefile Ewhich is ormally ame! +akefileF,
which !escri%es the relatioships a! !epe!ecies amog the compoets that comprise
the +iishe! program.
The +irst part o+ the make+ile !e+ies #aria%les that are s"%stit"te! i later sectios o+ the
make+ile. 0or e3ample we see the lie,
CCM gcc
which !e+ies the C compiler to %e gcc. *ater i the make+ile, we see oe istace
where it gets "se!,
diction: diction<o sentence<o misc<o geto7t<o geto7t#<o
$NCCP -o $@ $N='-=4@*P diction<o sentence<o misc<o ]
geto7t<o geto7t#<o $N=AH*P
$ s"%stit"tio is per+orme! here, a! the #al"e $NCCP is replace! %y gcc at r" time.
>ost o+ the make+ile cosists o+ lies, which !e+ie a target, i this case the e3ec"ta%le
+ile diction, a! the +iles o which it is !epe!et. The remaiig lies !escri%e the
comma!EsF ee!e! to create the target +rom its compoets. We see i this e3ample that
(2.
21 8 Compilig 6rograms
the e3ec"ta%le +ile diction Eoe o+ the +ial e! pro!"ctsF !epe!s o the e3istece o+
diction<o, sentence<o, misc<o, geto7t<o, a! geto7t#<o. *ater o, i the
make+ile, we see !e+iitios o+ each o+ these as targets,
diction<o: diction<c config<h geto7t<h misc<h sentence<h
geto7t<o: geto7t<c geto7t<h geto7tOint<h
geto7t#<o: geto7t#<c geto7t<h geto7tOint<h
misc<o: misc<c config<h misc<h
sentence<o: sentence<c config<h misc<h sentence<h
st1le<o: st1le<c config<h geto7t<h misc<h sentence<h
Aowe#er, we !oIt see ay comma! speci+ie! +or them. This is ha!le! %y a geeral
target, earlier i the +ile, that !escri%es the comma! "se! to compile ay <c +ile ito a
<o +ile,
<c<o:
$NCCP -c $NC::-=4@*P $NC-=4@*P $X
This all seems #ery complicate!. Why ot simply list all the steps to compile the parts
a! %e !oe with itC The aswer to this will %ecome clear i a momet. 9 the
meatime, letIs r" make a! %"il! o"r programs,
[me@linuxbox diction-#<##]$ make
The make program will r", "sig the cotets o+ +akefile to g"i!e its actios. 9t
will pro!"ce a lot o+ messages.
Whe it +iishes, we will see that all the targets are ow preset i o"r !irectory,
[me@linuxbox diction-#<##]$ ls
config<guess de<7o en install-sh sentence<c
config<h diction enO@H +akefile sentence<h
config<h<in diction<# enO@H<mo +akefile<in sentence<o
config<log diction<#<in enO@H<7o misc<c st1le
config<status diction<c geto7t#<c misc<h st1le<#
config<sub diction<o geto7t#<o misc<o st1le<#<in
configure diction<7ot geto7t<c L&,* st1le<c
configure<in diction<s7ec geto7t<h nl st1le<o
C:UAL@ diction<s7ec<in geto7tOint<h nl<mo test
de diction<texi geto7t<o nl<7o
(22
Compilig $ C 6rogram
de<mo diction<texi<in AL*T4== R&4'+&
$mog the +iles, we see diction a! st1le, the programs that we set o"t to %"il!.
Cograt"latios are i or!erL We 4"st compile! o"r +irst programs +rom so"rce co!eL
J"t 4"st o"t o+ c"riosity, letIs r" make agai,
[me@linuxbox diction-#<##]$ make
make: Lothing to be done for JallK<
9t oly pro!"ces this strage message. WhatIs goig oC Why !i!It it %"il! the
program agaiC $h, this is the magic o+ make. :ather tha simply %"il!ig e#erythig
agai, make oly %"il!s what ee!s %"il!ig. With all o+ the targets preset, make
!etermie! that there was othig to !o. We ca !emostrate this %y !eletig oe o+ the
targets a! r"ig make agai to see what it !oes. *etIs get ri! o+ oe o+ the
iterme!iate targets,
[me@linuxbox diction-#<##]$ rm getopt.o
[me@linuxbox diction-#<##]$ make
We see that make re%"il!s it a! re-liks the diction a! st1le programs, sice they
!epe! o the missig mo!"le. This %eha#ior also poits o"t aother importat +eat"re
o+ make, it keeps targets "p to !ate. make isists that targets %e ewer tha their
!epe!ecies. This makes per+ect sese, as a programmer will o+te "p!ate a %it o+
so"rce co!e a! the "se make to %"il! a ew #ersio o+ the +iishe! pro!"ct. make
es"res that e#erythig that ee!s %"il!ig %ase! o the "p!ate! co!e is %"ilt. 9+ we "se
the touch program to ?"p!ate@ oe o+ the so"rce co!e +iles, we ca see this happe,
[me@linuxbox diction-#<##]$ ls -l diction getopt.c
-r9xr-xr-x # me me $)#.% !((0-($-(" (.:#% diction
-r9-r--r-- # me me $$#!" !(()-($-$( #):%" geto7t<c
[me@linuxbox diction-#<##]$ touch getopt.c
[me@linuxbox diction-#<##]$ ls -l diction getopt.c
-r9xr-xr-x # me me $)#.% !((0-($-(" (.:#% diction
-r9-r--r-- # me me $$#!" !((0-($-(" (.:!$ geto7t<c
[me@linuxbox diction-#<##]$ make
$+ter make r"s, we see that it has restore! the target to %eig ewer tha the
!epe!ecy,
(2(
21 8 Compilig 6rograms
[me@linuxbox diction-#<##]$ ls -l diction getopt.c
-r9xr-xr-x # me me $)#.% !((0-($-(" (.:!% diction
-r9-r--r-- # me me $$#!" !((0-($-(" (.:!$ geto7t<c
The a%ility o+ make to itelligetly %"il! oly what ee!s %"il!ig is a great %ee+it to
programmers. While the time sa#igs may ot %e #ery apparet with o"r small pro4ect, it
is #ery sigi+icat with larger pro4ects. :emem%er, the *i"3 kerel Ea program that
"!ergoes coti"o"s mo!i+icatio a! impro#emetF cotais se#eral million lies o+
co!e.
Instaing The 'rogram
Well-package! so"rce co!e will o+te icl"!e a special make target calle! install.
This target will istall the +ial pro!"ct i a system !irectory +or "se. )s"ally, this
!irectory is 6usr6local6bin, the tra!itioal locatio +or locally %"ilt so+tware.
Aowe#er, this !irectory is ot ormally writa%le %y or!iary "sers, so we m"st %ecome
the s"per"ser to per+orm the istallatio,
[me@linuxbox diction-#<##]$ sudo make install
$+ter we per+orm the istallatio, we ca check that the program is rea!y to go,
[me@linuxbox diction-#<##]$ which diction
6usr6local6bin6diction
[me@linuxbox diction-#<##]$ man diction
$! there we ha#e itL
Summing -p
9 this chapter, we ha#e see how three simple comma!s,
<6configure
make
make install
ca %e "se! to %"il! may so"rce co!e packages. We ha#e also see the importat role
that make plays i the maiteace o+ programs. The make program ca %e "se! +or
ay task that ee!s to maitai a target-!epe!ecy relatioship, ot 4"st +or compilig
so"rce co!e.
(21
0"rther :ea!ig
;urther 1eading
The Wikipe!ia has goo! articles o compilers a! the make program,
http,--e.wikipe!ia.org-wiki-Compiler
http,--e.wikipe!ia.org-wiki->akeOEso+twareF
The @>0 !ake !anual,
http,--www.g".org-so+tware-make-ma"al-htmlOo!e-i!e3.html
(22
6art 2 8 Writig Shell Scripts
'art & A Writing She S/ri5ts
(2/
22 8 Writig 5o"r 0irst Script
2& Writing 8our (irst Scri#t
9 the prece!ig chapters, we ha#e assem%le! a arseal o+ comma! lie tools. While
these tools ca sol#e may ki!s o+ comp"tig pro%lems, we are still limite! to ma"ally
"sig them oe %y oe o the comma! lie. Wo"l!It it %e great i+ we co"l! get the
shell to !o more o+ the workC We ca. Jy 4oiig o"r tools together ito programs o+
o"r ow !esig, the shell ca carry o"t comple3 se="eces o+ tasks all %y itsel+. We ca
ea%le it to !o this %y writig shell scripts.
+hat #re Shell Scripts.
9 the simplest terms, a shell script is a +ile cotaiig a series o+ comma!s. The shell
rea!s this +ile a! carries o"t the comma!s as tho"gh they ha#e %ee etere! !irectly o
the comma! lie.
The shell is somewhat "i="e, i that it is %oth a power+"l comma! lie iter+ace to the
system a! a scriptig lag"age iterpreter. $s we will see, most o+ the thigs that ca %e
!oe o the comma! lie ca %e !oe i scripts, a! most o+ the thigs that ca %e !oe
i scripts ca %e !oe o the comma! lie.
We ha#e co#ere! may shell +eat"res, %"t we ha#e +oc"se! o those +eat"res most o+te
"se! !irectly o the comma! lie. The shell also pro#i!es a set o+ +eat"res "s"ally E%"t
ot alwaysF "se! whe writig programs.
3o4 To +rite # Shell Script
To s"ccess+"lly create a! r" a shell script, we ee! to !o three thigs,
.. 'rite a scri(t. Shell scripts are or!iary te3t +iles. So we ee! a te3t e!itor to
write them. The %est te3t e!itors will pro#i!e synta+ highlighting, allowig "s to
see a color-co!e! #iew o+ the elemets o+ the script. Syta3 highlightig will help
"s spot certai ki!s o+ commo errors. vim, gedit, kate, a! may other
e!itors are goo! ca!i!ates +or writig scripts.
2. :ake the scri(t executable. The system is rather +"ssy a%o"t ot lettig ay ol!
te3t +ile %e treate! as a program, a! +or goo! reasoL We ee! to set the script
+ileIs permissios to allow e3ec"tio.
(28
Aow To Write $ Shell Script
(. Put the scri(t some+here the shell can "ind it. The shell a"tomatically searches
certai !irectories +or e3ec"ta%le +iles whe o e3plicit pathame is speci+ie!.
0or ma3im"m co#eiece, we will place o"r scripts i these !irectories.
Script ;ile ;ormat
9 keepig with programmig tra!itio, weIll create a ?hello worl!@ program to
!emostrate a e3tremely simple script. So letIs +ire "p o"r te3t e!itors a! eter the
+ollowig script,
UL/bin/bash
U +his is our first script.
echo %Rello WorldL%
The last lie o+ o"r script is pretty +amiliar, 4"st a echo comma! with a strig
arg"met. The seco! lie is also +amiliar. 9t looks like a commet that we ha#e see
"se! i may o+ the co+ig"ratio +iles we ha#e e3amie! a! e!ite!. 7e thig a%o"t
commets i shell scripts is that they may also appear at the e! o+ lies, like so,
echo KSello ,orldGK ^ This is a comment too
E#erythig +rom the ^ sym%ol owar! o the lie is igore!.
*ike may thigs, this works o the comma! lie, too,
[me@linuxbox ~]$ echo %Rello WorldL% U +his is a comment too
Sello ,orldG
Tho"gh commets are o+ little "se o the comma! lie, they will work.
The +irst lie o+ o"r script is a little mysterio"s. 9t looks like it sho"l! %e a commet,
sice it starts with ^, %"t it looks too p"rpose+"l to %e 4"st that. The ^G character
se="ece is, i +act, a special costr"ct calle! a shebang. The she%ag is "se! to tell the
system the ame o+ the iterpreter that sho"l! %e "se! to e3ec"te the script that +ollows.
E#ery shell script sho"l! icl"!e this as its +irst lie.
*etIs sa#e o"r script +ile as helloO9orld.
(29
22 8 Writig 5o"r 0irst Script
$xecuta0le Permissions
The e3t thig we ha#e to !o is make o"r script e3ec"ta%le. This is easily !oe "sig
chmod,
[me@linuxbox ~]$ ls -l hello.world
-r9-r--r-- # me me .$ !((0-($-() #(:#( helloO9orld
[me@linuxbox ~]$ chmod C"" hello.world
[me@linuxbox ~]$ ls -l hello.world
-r9xr-xr-x # me me .$ !((0-($-() #(:#( helloO9orld
There are two commo permissio settigs +or scriptsH /22 +or scripts that e#eryoe ca
e3ec"te, a! /00 +or scripts that oly the ower ca e3ec"te. &ote that scripts m"st %e
rea!a%le i or!er to %e e3ec"te!.
Script ;ile Location
With the permissios set, we ca ow e3ec"te o"r script,
[me@linuxbox ~]$ ./hello.world
Sello ,orldG
9 or!er +or the script to r", we m"st prece!e the script ame with a e3plicit path. 9+
we !oIt, we get this,
[me@linuxbox ~]$ hello.world
bash: helloO9orld: command not found
Why is thisC What makes o"r script !i++eret +rom other programsC $s it t"rs o"t,
othig. 7"r script is +ie. 9ts locatio is the pro%lem. Jack i Chapter .2, we !isc"sse!
the :4TS e#iromet #aria%le a! its e++ect o how the system searches +or e3ec"ta%le
programs. To recap, the system searches a list o+ !irectories each time it ee!s to +i! a
e3ec"ta%le program, i+ o e3plicit path is speci+ie!. This is how the system kows to
e3ec"te 6bin6ls whe we type ls at the comma! lie. The 6bin !irectory is oe o+
the !irectories that the system a"tomatically searches. The list o+ !irectories is hel!
withi a e#iromet #aria%le ame! :4TS. The :4TS #aria%le cotais a colo-
separate! list o+ !irectories to %e searche!. We ca #iew the cotets o+ :4TS,
(K0
Script 0ile *ocatio
[me@linuxbox ~]$ echo 3B:+R
6home6me6bin:6usr6local6sbin:6usr6local6bin:6usr6sbin:6usr6bin:6sbin:
6bin:6usr6games
Aere we see o"r list o+ !irectories. 9+ o"r script were locate! i ay o+ the !irectories i
the list, o"r pro%lem wo"l! %e sol#e!. &otice the +irst !irectory i the list,
6home6me6bin. >ost *i"3 !istri%"tios co+ig"re the :4TS #aria%le to cotai a
bin !irectory i the "serIs home !irectory, to allow "sers to e3ec"te their ow programs.
So i+ we create the bin !irectory a! place o"r script withi it, it sho"l! start to work
like other programs,
[me@linuxbox ~]$ mkdir bin
[me@linuxbox ~]$ mv hello.world bin
[me@linuxbox ~]$ hello.world
Sello ,orldG
$! so it !oes.
9+ the :4TS #aria%le !oes ot cotai the !irectory, we ca easily a!! it %y icl"!ig this
lie i o"r <bashrc +ile,
export B:+R$~/bin2J3B:+RJ
$+ter this chage is ma!e, it will take e++ect i each ew termial sessio. To apply the
chage to the c"rret termial sessio, we m"st ha#e the shell re-rea! the <bashrc +ile.
This ca %e !oe %y ?so"rcig@ it,
[me@linuxbox ~]$ . .bashrc
The !ot E<F comma! is a syoym +or the source comma!, a shell %"ilti which
rea!s a speci+ie! +ile o+ shell comma!s a! treats it like ip"t +rom the key%oar!.
1ote5 )%"t" a"tomatically a!!s the ~6bin !irectory to the :4TS #aria%le i+ the
~6bin !irectory e3ists whe the "serIs <bashrc +ile is e3ec"te!. So, o )%"t"
systems, i+ we create the ~6bin !irectory a! the log o"t a! log i agai,
e#erythig works.
(K.
22 8 Writig 5o"r 0irst Script
+ood Lo/ations 2or S/ri5ts
The ~6bin !irectory is a goo! place to p"t scripts ite!e! +or persoal "se. 9+ we write
a script that e#eryoe o a system is allowe! to "se, the tra!itioal locatio is
6usr6local6bin. Scripts ite!e! +or "se %y the system a!miistrator are o+te
locate! i 6usr6local6sbin. 9 most cases, locally s"pplie! so+tware, whether
scripts or compile! programs, sho"l! %e place! i the 6usr6local hierarchy a! ot i
6bin or 6usr6bin< These !irectories are speci+ie! %y the *i"3 0ilesystem Aierarchy
Sta!ar! to cotai oly +iles s"pplie! a! maitaie! %y the *i"3 !istri%"tor.
>ore ;ormatting Tric(s
7e o+ the key goals o+ serio"s script writig is ease o+ maintenanceH that is, the ease
with which a script may %e mo!i+ie! %y its a"thor or others to a!apt it to chagig ee!s.
>akig a script easy to rea! a! "!ersta! is oe way to +acilitate easy maiteace.
Long <5tion ,ames
>ay o+ the comma!s we ha#e st"!ie! +eat"re %oth short a! log optio ames. 0or
istace, the ls comma! has may optios that ca %e e3presse! i either short or log
+orm. 0or e3ample,
[me@linuxbox ~]$ ls -ad
a!,
[me@linuxbox ~]$ ls --all --director!
are e="i#alet comma!s. 9 the iterests o+ re!"ce! typig, short optios are pre+erre!
whe eterig optios o the comma! lie, %"t whe writig scripts, log optios ca
pro#i!e impro#e! rea!a%ility.
Indentation And Line Continuation
Whe employig log comma!s, rea!a%ility ca %e ehace! %y sprea!ig the
comma! o#er se#eral lies. 9 Chapter .8, we looke! at a partic"larly log e3ample o+
the +i! comma!,
(K2
>ore 0ormattig Tricks
[me@linuxbox ~]$ find pla!ground K4 -t!pe f -not -perm *N** -exec
chmod *N** `9>a `#a K6 -or K4 -t!pe d -not -perm *C11 -exec chmod
*C11 `9>a `#a K6
7%#io"sly, this comma! is a little har! to +ig"re o"t at +irst glace. 9 a script, this
comma! might %e easier to "!ersta! i+ writte this way,
find 7la1ground ]
]N ]
-t17e f ]
-not -7erm (.(( ]
-exec chmod (.(( a[\b aTb ]
]P ]
-or ]
]N ]
-t17e d ]
-not -7erm ()## ]
-exec chmod ()## a[\b aTb ]
]P
Jy "sig lie coti"atios E%ackslash-lie+ee! se="ecesF a! i!etatio, the logic o+
this comple3 comma! is more clearly !escri%e! to the rea!er. This techi="e works o
the comma! lie, too, tho"gh it is sel!om "se!, as it is #ery awkwar! to type a! e!it.
7e !i++erece %etwee a script a! the comma! lie is that the script may employ ta%
characters to achie#e i!etatio, whereas the comma! lie caot, sice ta%s are "se!
to acti#ate completio.
Con"iguring vim ;or Script +riting
The vim te3t e!itor has may, may co+ig"ratio settigs. There are se#eral
commo optios that ca +acilitate script writig,
2s!ntax on
t"rs o syta3 highlightig. With this settig, !i++eret elemets o+ shell syta3
will %e !isplaye! i !i++eret colors whe #iewig a script. This is help+"l +or
i!eti+yig certai ki!s o+ programmig errors. 9t looks cool, too. &ote that +or
this +eat"re to work, yo" m"st ha#e a complete #ersio o+ vim istalle!, a! the
+ile yo" are e!itig m"st ha#e a she%ag i!icatig the +ile is a shell script. 9+ yo"
ha#e !i++ic"lty with the comma! a%o#e, try 2set s!ntax$sh istea!.
(K(
22 8 Writig 5o"r 0irst Script
2set hlsearch
t"rs o the optio to highlight search res"lts. Say we search +or the wor!
?echo.@ With this optio o, each istace o+ the wor! will %e highlighte!.
2set tabstop$A
sets the "m%er o+ col"ms occ"pie! %y a ta% character. The !e+a"lt is eight
col"ms. Settig the #al"e to +o"r Ewhich is a commo practiceF allows log
lies to +it more easily o the scree.
2set autoindent
t"rs o the ?a"to i!et@ +eat"re. This ca"ses vim to i!et a ew lie the same
amo"t as the lie 4"st type!. This spee!s "p typig o may ki!s o+
programmig costr"cts. To stop i!etatio, type Ctrl-d<
These chages ca %e ma!e permaet %y a!!ig these comma!s Ewitho"t the
lea!ig colo charactersF to yo"r ~6<vimrc +ile.
Summing -p
9 this +irst chapter o+ scriptig, we ha#e looke! at how scripts are writte a! ma!e to
easily e3ec"te o o"r system. We also saw how we may "se #ario"s +ormattig
techi="es to impro#e the rea!a%ility Ea! th"s, the maitaia%ilityF o+ o"r scripts. 9
+"t"re chapters, ease o+ maiteace will come "p agai a! agai as a cetral priciple
i goo! script writig.
;urther 1eading
0or ?Aello Worl!@ programs a! e3amples i #ario"s programmig lag"ages,
see,
http,--e.wikipe!ia.org-wiki-AelloOworl!
This Wikipe!ia article talks more a%o"t the she%ag mechaism,
http,--e.wikipe!ia.org-wiki-She%agOE)i3F
(K1
2K 8 Startig $ 6ro4ect
2+ Starting ) 2ro9ect
Startig with this chapter, we will %egi to %"il! a program. The p"rpose o+ this pro4ect
is to see how #ario"s shell +eat"res are "se! to create programs a!, more importatly,
create good programs.
The program we will write is a report generator. 9t will preset #ario"s statistics a%o"t
o"r system a! its stat"s, a! will pro!"ce this report i AT>* +ormat, so we ca #iew it
with a we% %rowser s"ch as 0ire+o3 or Mo="eror.
6rograms are "s"ally %"ilt "p i a series o+ stages, with each stage a!!ig +eat"res a!
capa%ilities. The +irst stage o+ o"r program will pro!"ce a #ery miimal AT>* page that
cotais o system i+ormatio. That will come later.
;irst StageI >inimal Document
The +irst thig we ee! to kow is the +ormat o+ a well-+orme! AT>* !oc"met. 9t
looks like this,
XST+=D
XS&4'D
XTAT=&D:age TitleX6TAT=&D
X6S&4'D
XH'UD
:age bod1<
X6H'UD
X6ST+=D
9+ we eter this ito o"r te3t e!itor a! sa#e the +ile as foo<html, we ca "se the
+ollowig ):* i 0ire+o3 to #iew the +ile,
file:666home6username6foo<html
The +irst stage o+ o"r program will %e a%le to o"tp"t this AT>* +ile to sta!ar! o"tp"t.
We ca write a program to !o this pretty easily. *etIs start o"r te3t e!itor a! create a
ew +ile ame! ~6bin6s1sOinfoO7age,
(K2
2K 8 Startig $ 6ro4ect
[me@linuxbox ~]$ vim ~/bin/s!s.info.page
a! eter the +ollowig program,
UL/bin/bash
U Brogram to output a s!stem information page
echo J/R+OT&J
echo J /RG:1&J
echo J /+P+TG&Bage +itle//+P+TG&J
echo J //RG:1&J
echo J /<S1^&J
echo J Bage bod!.J
echo J //<S1^&J
echo J//R+OT&J
7"r +irst attempt at this pro%lem cotais a she%ag, a commet Ealways a goo! i!eaF a!
a se="ece o+ echo comma!s, oe +or each lie o+ o"tp"t. $+ter sa#ig the +ile, weIll
make it e3ec"ta%le a! attempt to r" it,
[me@linuxbox ~]$ chmod C"" ~/bin/s!s.info.page
[me@linuxbox ~]$ s!s.info.page
Whe the program r"s, we sho"l! see the te3t o+ the AT>* !oc"met !isplaye! o the
scree, sice the echo comma!s i the script se! their o"tp"t to sta!ar! o"tp"t.
WeIll r" the program agai a! re!irect the o"tp"t o+ the program to the +ile
s1sOinfoO7age<html, so that we ca #iew the res"lt with a we% %rowser,
[me@linuxbox ~]$ s!s.info.page & s!s.info.page.html
[me@linuxbox ~]$ firefox s!s.info.page.html
So +ar, so goo!.
Whe writig programs, itIs always a goo! i!ea to stri#e +or simplicity a! clarity.
>aiteace is easier whe a program is easy to rea! a! "!ersta!, ot to metio, it
ca make the program easier to write %y re!"cig the amo"t o+ typig. 7"r c"rret
#ersio o+ the program works +ie, %"t it co"l! %e simpler. We co"l! act"ally com%ie all
the echo comma!s ito oe, which will certaily make it easier to a!! more lies to the
programIs o"tp"t. So, letIs chage o"r program to this,
(KK
0irst Stage, >iimal 'oc"met
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
echo J/R+OT&
/RG:1&
/+P+TG&Bage +itle//+P+TG&
//RG:1&
/<S1^&
Bage bod!.
//<S1^&
//R+OT&J
$ ="ote! strig may icl"!e ewlies, a! there+ore cotai m"ltiple lies o+ te3t. The
shell will keep rea!ig the te3t "til it eco"ters the closig ="otatio mark. 9t works
this way o the comma! lie, too,
[me@linuxbox ~]$ echo J/R+OT&
D /RG:1&
D /+P+TG&Bage +itle//+P+TG&
D //RG:1&
D /<S1^&
D Bage bod!.
D //<S1^&
D //R+OT&J
The lea!ig ?X@ character is the shell prompt cotaie! i the :*! shell #aria%le. 9t
appears whee#er we type a m"lti-lie statemet ito the shell. This +eat"re is a little
o%sc"re right ow, %"t later, whe we co#er m"lti-lie programmig statemets, it will
t"r o"t to %e ="ite ha!y.
Second StageI #dding # Little Data
&ow that o"r program ca geerate a miimal !oc"met, letIs p"t some !ata i the
report. To !o this, we will make the +ollowig chages,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
echo _XST+=D
XS&4'D
XTAT=&DF!stem Pnformation HeportX6TAT=&D
(K/
2K 8 Startig $ 6ro4ect
X6S&4'D
XH'UD
XS#DF!stem Pnformation HeportX6S#D
X6H'UD
X6ST+=D_
We a!!e! a page title a! a hea!ig to the %o!y o+ the report.
Varia0les #nd Constants
There is a iss"e with o"r script, howe#er. &otice how the strig ?System 9+ormatio
:eport@ is repeate!C With o"r tiy script itIs ot a pro%lem, %"t letIs imagie that o"r
script was really log a! we ha! m"ltiple istaces o+ this strig. 9+ we wate! to
chage the title to somethig else, we wo"l! ha#e to chage it i m"ltiple places, which
co"l! %e a lot o+ work. What i+ we co"l! arrage the script so that the strig oly
appeare! oce a! ot m"ltiple timesC That wo"l! make +"t"re maiteace o+ the script
m"ch easier. AereIs how we co"l! !o that,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
title$JF!stem Pnformation HeportJ
echo _XST+=D
XS&4'D
XTAT=&D3titleX6TAT=&D
X6S&4'D
XH'UD
XS#D3titleX6S#D
X6H'UD
X6ST+=D_
Jy creatig a variable ame! title a! assigig it the #al"e ?System 9+ormatio
:eport,@ we ca take a!#atage o+ parameter e3pasio a! place the strig i m"ltiple
locatios.
So, how !o we create a #aria%leC Simple, we 4"st "se it. Whe the shell eco"ters a
#aria%le, it a"tomatically creates it. This !i++ers +rom may programmig lag"ages i
which #aria%les m"st %e e3plicitly declared or !e+ie! %e+ore "se. The shell is #ery la3
a%o"t this, which ca lea! to some pro%lems. 0or e3ample, cosi!er this sceario playe!
o"t o the comma! lie,
(K8
Raria%les $! Costats
[me@linuxbox ~]$ foo$J!esJ
[me@linuxbox ~]$ echo 3foo
1es
[me@linuxbox ~]$ echo 3fool
[me@linuxbox ~]$
We +irst assig the #al"e ?yes@ to the #aria%le foo, the !isplay its #al"e with echo.
&e3t we !isplay the #al"e o+ the #aria%le ame misspelle! as ?+ool@ a! get a %lak
res"lt. This is %eca"se the shell happily create! the #aria%le fool whe it eco"tere!
it, a! ga#e it the !e+a"lt #al"e o+ othig, or empty. 0rom this, we lear that we m"st
pay close attetio to o"r spelligL 9tIs also importat to "!ersta! what really
happee! i this e3ample. 0rom o"r pre#io"s look at how the shell per+orms e3pasios,
we kow that the comma!,
[me@linuxbox ~]$ echo $foo
"!ergoes parameter e3pasio a! res"lts i,
[me@linuxbox ~]$ echo 1es
Whereas the comma!,
[me@linuxbox ~]$ echo $fool
e3pa!s ito,
[me@linuxbox ~]$ echo
The empty #aria%le e3pa!s ito othigL This ca play ha#oc with comma!s that
re="ire arg"mets. AereIs a e3ample,
[me@linuxbox ~]$ foo$foo.txt
[me@linuxbox ~]$ foo1$foo1.txt
[me@linuxbox ~]$ cp 3foo 3fool
c7: missing destination file o7erand after Jfoo<txtK
(K9
2K 8 Startig $ 6ro4ect
Tr1 Jc7 --hel7K for more information<
We assig #al"es to two #aria%les, foo a! foo#. We the per+orm a c7, %"t misspell
the ame o+ the seco! arg"met. $+ter e3pasio, the c7 comma! is oly set oe
arg"met, tho"gh it re="ires two.
There are some r"les a%o"t #aria%le ames,
.. Raria%le ames may cosist o+ alpha"meric characters Eletters a! "m%ersF a!
"!erscore characters.
2. The +irst character o+ a #aria%le ame m"st %e either a letter or a "!erscore.
(. Spaces a! p"ct"atio sym%ols are ot allowe!.
The wor! ?#aria%le@ implies a #al"e that chages, a! i may applicatios, #aria%les are
"se! this way. Aowe#er, the #aria%le i o"r applicatio, title, is "se! as a constant. $
costat is 4"st like a #aria%le i that it has a ame a! cotais a #al"e. The !i++erece
is that the #al"e o+ a costat !oes ot chage. 9 a applicatio that per+orms geometric
calc"latios, we might !e+ie :A as a costat, a! assig it the #al"e o+ $<#%#",
istea! o+ "sig the "m%er literally thro"gho"t o"r program. The shell makes o
!istictio %etwee #aria%les a! costatsH they are mostly +or the programmerIs
co#eiece. $ commo co#etio is to "se "pper case letters to !esigate costats
a! lower case letters +or tr"e #aria%les. We will mo!i+y o"r script to comply with this
co#etio,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
+P+TGM_*1stem Anformation Re7ort 7or 3RSF+?:OG_
echo _XST+=D
XS&4'D
XTAT=&D$+P+TGX6TAT=&D
X6S&4'D
XH'UD
XS#D$+P+TGX6S#D
X6H'UD
X6ST+=D_
We also took the opport"ity to 4aDD "p o"r title %y a!!ig the #al"e o+ the shell #aria%le
S*TL4+&. This is the etwork ame o+ the machie.
(/0
Raria%les $! Costats
1ote, The shell act"ally !oes pro#i!e a way to e+orce the imm"ta%ility o+
costats, thro"gh the "se o+ the declare %"ilti comma! with the -r Erea!-
olyF optio. Aa! we assige! TAT=& this way,
declare -r TAT=&M_:age Title_
the shell wo"l! pre#et ay s"%se="et assigmet to TAT=&. This +eat"re is
rarely "se!, %"t it e3ists +or #ery +ormal scripts.
Assigning ?aues To ?ariabes And Constants
Aere is where o"r kowle!ge o+ e3pasio really starts to pay o++. $s we ha#e see,
#aria%les are assige! #al"es this way,
variableMvalue
where variable is the ame o+ the #aria%le a! value is a strig. )like some other
programmig lag"ages, the shell !oes ot care a%o"t the type o+ !ata assige! to a
#aria%leH it treats them all as strigs. 5o" ca +orce the shell to restrict the assigmet to
itegers %y "sig the declare comma! with the -i optio, %"t, like settig #aria%les
as rea!-oly, this is rarely !oe.
&ote that i a assigmet, there m"st %e o spaces %etwee the #aria%le ame, the
e="als sig, a! the #al"e. So what ca the #al"e cosist o+C $ythig that we ca
e3pa! ito a strig,
aMC ^ 4ssign the string _C_ to variable a<
bM_a string_ ^ &mbedded s7aces must be 9ithin Buotes<
cM_a string and $b_ ^ ther ex7ansions such as variables can be
^ ex7anded into the assignment<
dM$Nls -l foo<txtP ^ Results of a command<
eM$NN" E )PP ^ 4rithmetic ex7ansion<
fM_]t]ta string]n_ ^ &sca7e seBuences such as tabs and ne9lines<
>"ltiple #aria%le assigmets may %e !oe o a sigle lie,
aM" bM_a string_
'"rig e3pasio, #aria%le ames may %e s"rro"!e! %y optioal c"rly %races ?cd@.
This is "se+"l i cases where a #aria%le ame %ecomes am%ig"o"s !"e to its s"rro"!ig
(/.
2K 8 Startig $ 6ro4ect
cote3t. Aere, we try to chage the ame o+ a +ile +rom m1file to m1file#, "sig a
#aria%le,
[me@linuxbox ~]$ filename$Jm!fileJ
[me@linuxbox ~]$ touch 3filename
[me@linuxbox ~]$ mv 3filename 3filename1
mv: missing destination file o7erand after Jm1fileK
Tr1 Jmv --hel7K for more information<
This attempt +ails %eca"se the shell iterprets the seco! arg"met o+ the mv comma! as
a ew Ea! emptyF #aria%le. The pro%lem ca %e o#ercome this way,
[me@linuxbox ~]$ mv 3filename 39filename>1
Jy a!!ig the s"rro"!ig %races, the shell o loger iterprets the trailig # as part o+
the #aria%le ame.
WeIll take this opport"ity to a!! some !ata to o"r report, amely the !ate a! time the
report was create! a! the "ser ame o+ the creator,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
TAT=&M_*1stem Anformation Re7ort -or $S*TL4+&_
=EHHG?+.+POG$34date 5J8x 8r 8@J6
+POG.F+:OB$JQenerated 3=EHHG?+.+POG; b! 3EFGHJ
echo _XST+=D
XS&4'D
XTAT=&D$TAT=&X6TAT=&D
X6S&4'D
XH'UD
XS#D$TAT=&X6S#D
/B&3+POG.F+:OB//B&
X6H'UD
X6ST+=D_
3ere Documents
WeI#e looke! at two !i++eret metho!s o+ o"tp"ttig o"r te3t, %oth "sig the echo
(/2
Aere 'oc"mets
comma!. There is a thir! way calle! a here document or here script. $ here !oc"met
is a a!!itioal +orm o+ 9-7 re!irectio i which we em%e! a %o!y o+ te3t ito o"r script
a! +ee! it ito the sta!ar! ip"t o+ a comma!. 9t works like this,
command XX token
text
token
where command is the ame o+ comma! that accepts sta!ar! ip"t a! token is a strig
"se! to i!icate the e! o+ the em%e!!e! te3t. WeIll mo!i+y o"r script to "se a here
!oc"met,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
TAT=&M_*1stem Anformation Re7ort -or $S*TL4+&_
C3RR&LTOTA+&M$Ndate 8_5x 5r 5I_P
TA+&O*T4+:M_@enerated $C3RR&LTOTA+&> b1 $3*&R_
cat // .GS7.
XST+=D
XS&4'D
XTAT=&D$TAT=&X6TAT=&D
X6S&4'D
XH'UD
XS#D$TAT=&X6S#D
X:D$TA+&O*T4+:X6:D
X6H'UD
X6ST+=D
.GS7.
9stea! o+ "sig echo, o"r script ow "ses cat a! a here !oc"met. The strig O&-O
Emeaig ?E! 7+ 0ile,@ a commo co#etioF was selecte! as the toke, a! marks the
e! o+ the em%e!!e! te3t. &ote that the toke m"st appear aloe a! that there m"st ot
%e trailig spaces o the lie.
So whatIs the a!#atage o+ "sig a here !oc"metC 9tIs mostly the same as echo, e3cept
that, %y !e+a"lt, sigle a! !o"%le ="otes withi here !oc"mets lose their special
meaig to the shell. Aere is a comma! lie e3ample,
[me@linuxbox ~]$ foo$Jsome textJ
[me@linuxbox ~]$ cat // .GS7.
D 3foo
(/(
2K 8 Startig $ 6ro4ect
D J3fooJ
D %3foo%
D K3foo
D .GS7.
some text
_some text_
Ksome textK
$foo
$s we ca see, the shell pays o attetio to the ="otatio marks. 9t treats them as
or!iary characters. This allows "s to em%e! ="otes +reely withi a here !oc"met. This
co"l! t"r o"t to %e ha!y +or o"r report program.
Aere !oc"mets ca %e "se! with ay comma! that accepts sta!ar! ip"t. 9 this
e3ample, we "se a here !oc"met to pass a series o+ comma!s to the ft7 program i
or!er to retrie#e a +ile +rom a remote 0T6 ser#er,
^G6bin6bash
^ *cri7t to retrieve a file via -T:
-T:O*&R;&RMft7<nl<debian<org
-T:O:4TSM6debian6dists6lenn16main6installer-i$/.6current6images6cdrom
R&+T&O-A=&Mdebian-cdOinfo<tar<gC
ft7 -n XX O&-O
o7en $-T:O*&R;&R
user anon1mous me@linuxbox
cd $-T:O:4TS
hash
get $R&+T&O-A=&
b1e
O&-O
ls -l $R&+T&O-A=&
9+ we chage the re!irectio operator +rom ?YY@ to ?YY-@, the shell will igore lea!ig
ta% characters i the here !oc"met. This allows a here !oc"met to %e i!ete!, which
ca impro#e rea!a%ility,
^G6bin6bash
^ *cri7t to retrieve a file via -T:
-T:O*&R;&RMft7<nl<debian<org
(/1
Aere 'oc"mets
-T:O:4TSM6debian6dists6lenn16main6installer-i$/.6current6images6cdrom
R&+T&O-A=&Mdebian-cdOinfo<tar<gC
ft7 -n XX- O&-O
o7en $-T:O*&R;&R
user anon1mous me@linuxbox
cd $-T:O:4TS
hash
get $R&+T&O-A=&
b1e
O&-O
ls -l $R&+T&O-A=&
Summing -p
9 this chapter, we starte! a pro4ect that will carry "s thro"gh the process o+ %"il!ig a
s"ccess+"l script. We itro!"ce! the cocept o+ #aria%les a! costats a! how they ca
%e employe!. They are the +irst o+ may applicatios we will +i! +or parameter
e3pasio. We also looke! at how to pro!"ce o"tp"t +rom o"r script, a! #ario"s
metho!s +or em%e!!ig %locks o+ te3t.
;urther 1eading
0or more i+ormatio a%o"t AT>*, see the +ollowig articles a! t"torials,
http,--e.wikipe!ia.org-wiki-Atml
http,--e.wiki%ooks.org-wiki-AT>*O6rogrammig
http,--html.et-t"torials-html-
The bash ma page icl"!es a sectio etitle! ?AE:E '7C)>E&TS,@ which
has a +"ll !escriptio o+ this +eat"re.
(/2
2/ 8 Top-'ow 'esig
2. To#:*o5n *esign
$s programs get larger a! more comple3, they %ecome more !i++ic"lt to !esig, co!e
a! maitai. $s with ay large pro4ect, it is o+te a goo! i!ea to %reak large, comple3
tasks ito a series o+ small, simple tasks. *etIs imagie that we are tryig to !escri%e a
commo, e#ery!ay task, goig to the market to %"y +oo!, to a perso +rom >ars. We
might !escri%e the o#erall process as the +ollowig series o+ steps,
.. ;et i car.
2. 'ri#e to market.
(. 6ark car.
1. Eter market.
2. 6"rchase +oo!.
K. :et"r to car.
/. 'ri#e home.
8. 6ark car.
9. Eter ho"se.
Aowe#er, a perso +rom >ars is likely to ee! more !etail. We co"l! +"rther %reak !ow
the s"%task ?6ark car@ ito this series o+ steps,
.. 0i! parkig space.
2. 'ri#e car ito space.
(. T"r o++ motor.
1. Set parkig %rake.
2. E3it car.
K. *ock car.
The ?T"r o++ motor@ s"%task co"l! +"rther %e %roke !ow ito steps icl"!ig ?T"r
o++ igitio,@ ?:emo#e igitio key@ a! so o, "til e#ery step o+ the etire process o+
goig to the market has %ee +"lly !e+ie!.
This process o+ i!eti+yig the top-le#el steps a! !e#elopig icreasigly !etaile! #iews
o+ those steps is calle! top-down design. This techi="e allows "s to %reak large comple3
tasks ito may small, simple tasks. Top-!ow !esig is a commo metho! o+ !esigig
(/K
2/ 8 Top-'ow 'esig
programs a! oe that is well s"ite! to shell programmig i partic"lar.
9 this chapter, we will "se top-!ow !esig to +"rther !e#elop o"r report geerator
script.
Shell ;unctions
7"r script c"rretly per+orms the +ollowig steps to geerate the AT>* !oc"met,
.. 7pe page.
2. 7pe page hea!er.
(. Set page title.
1. Close page hea!er.
2. 7pe page %o!y.
K. 7"tp"t page hea!ig.
/. 7"tp"t time stamp.
8. Close page %o!y.
9. Close page.
0or o"r e3t stage o+ !e#elopmet, we will a!! some a!!itioal tasks %etwee steps /
a! 8. These will icl"!e,
System "ptime a! loa!. This is the amo"t o+ time sice the last sh"t!ow or
re%oot a! the a#erage "m%er o+ tasks c"rretly r"ig o the processor o#er
se#eral time iter#als.
'isk space. The o#erall "se o+ space o the systemIs storage !e#ices.
Aome space. The amo"t o+ storage space %eig "se! %y each "ser.
9+ we ha! a comma! +or each o+ these tasks, we co"l! a!! them to o"r script simply
thro"gh comma! s"%stit"tio,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
TAT=&M_*1stem Anformation Re7ort -or $S*TL4+&_
C3RR&LTOTA+&M$Ndate 8_5x 5r 5I_P
TA+&O*T4+:M_@enerated $C3RR&LTOTA+&> b1 $3*&R_
cat XX O&-O
XST+=D
XS&4'D
(//
2/ 8 Top-'ow 'esig
XTAT=&D$TAT=&X6TAT=&D
X6S&4'D
XH'UD
XS#D$TAT=&X6S#D
X:D$TA+&O*T4+:X6:D
34report.uptime6
34report.disk.space6
34report.home.space6
X6H'UD
X6ST+=D
O&-O
We co"l! create these a!!itioal comma!s two ways. We co"l! write three separate
scripts a! place them i a !irectory liste! i o"r 6$TA, or we co"l! em%e! the scripts
withi o"r program as shell functions. $s we ha#e metioe! %e+ore, shell +"ctios are
?mii-scripts@ that are locate! isi!e other scripts a! ca act as a"toomo"s programs.
Shell +"ctios ha#e two sytactic +orms,
function name [
commands
return
P
a!
name NP [
commands
return
\
where name is the ame o+ the +"ctio a! commands are a series o+ comma!s
cotaie! withi the +"ctio. Joth +orms are e="i#alet a! may %e "se!
iterchagea%ly. Jelow we see a script that !emostrates the "se o+ a shell +"ctio,
# ^G6bin6bash
!
$ ^ *hell function demo
%
" function funct [
. echo _*te7 !_
) return
/ \
0
#( ^ +ain 7rogram starts here
##
#! echo _*te7 #_
(/8
Shell 0"ctios
#$ funct
#% echo _*te7 $_
$s the shell rea!s the script, it passes o#er lies . thro"gh .., as those lies cosist o+
commets a! the +"ctio !e+iitio. E3ec"tio %egis at lie .2, with a echo
comma!. *ie .( calls the shell +"ctio funct a! the shell e3ec"tes the +"ctio
4"st as it wo"l! ay other comma!. 6rogram cotrol the mo#es to lie K, a! the
seco! echo comma! is e3ec"te!. *ie / is e3ec"te! e3t. 9ts return comma!
termiates the +"ctio a! ret"rs cotrol to the program at the lie +ollowig the
+"ctio call Elie .1F, a! the +ial echo comma! is e3ec"te!. &ote that i or!er +or
+"ctio calls to %e recogiDe! as shell +"ctios a! ot iterprete! as the ames o+
e3teral programs, shell +"ctio !e+iitios m"st appear i the script %e+ore they are
calle!.
WeIll a!! miimal shell +"ctio !e+iitios to o"r script,
^G6bin6bash
^ :rogram to out7ut a s1stem information 7age
TAT=&M_*1stem Anformation Re7ort -or $S*TL4+&_
C3RR&LTOTA+&M$Ndate 8_5x 5r 5I_P
TA+&O*T4+:M_@enerated $C3RR&LTOTA+&> b1 $3*&R_
report.uptime 46 9
return
>

report.disk.space 46 9
return
>
report.home.space 46 9
return
>
cat XX O&-O
XST+=D
XS&4'D
XTAT=&D$TAT=&X6TAT=&D
X6S&4'D
XH'UD
XS#D$TAT=&X6S#D
X:D$TA+&O*T4+:X6:D
$Nre7ortOu7timeP
$Nre7ortOdiskOs7aceP
(/9
2/ 8 Top-'ow 'esig
$Nre7ortOhomeOs7aceP
X6H'UD
X6ST+=D
O&-O
Shell +"ctio ames +ollow the same r"les as #aria%les. $ +"ctio m"st cotai at least
oe comma!. The return comma! Ewhich is optioalF satis+ies the re="iremet.
Local Varia0les
9 the scripts we ha#e writte so +ar, all the #aria%les Eicl"!ig costatsF ha#e %ee
global variables. ;lo%al #aria%les maitai their e3istece thro"gho"t the program. This
is +ie +or may thigs, %"t it ca sometimes complicate the "se o+ shell +"ctios. 9si!e
shell +"ctios, it is o+te !esira%le to ha#e local variables. *ocal #aria%les are oly
accessi%le withi the shell +"ctio i which they are !e+ie! a! cease to e3ist oce the
shell +"ctio termiates.
Aa#ig local #aria%les allows the programmer to "se #aria%les with ames that may
alrea!y e3ist, either i the script glo%ally or i other shell +"ctios, witho"t ha#ig to
worry a%o"t potetial ame co+licts.
Aere is a e3ample script that !emostrates how local #aria%les are !e+ie! a! "se!,
^G6bin6bash
^ local-vars: scri7t to demonstrate local variables
fooM( ^ global variable foo
functO# NP [
local foo ^ variable foo local to functO#
fooM#
echo _functO#: foo M $foo_
\
functO! NP [
local foo ^ variable foo local to functO!
fooM!
echo _functO!: foo M $foo_
\
echo _global: foo M $foo_
(80
*ocal Raria%les
functO#
echo _global: foo M $foo_
functO!
echo _global: foo M $foo_
$s we ca see, local #aria%les are !e+ie! %y prece!ig the #aria%le ame with the wor!
local. This creates a #aria%le that is local to the shell +"ctio i which it is !e+ie!.
7ce o"tsi!e the shell +"ctio, the #aria%le o loger e3ists. Whe we r" this script,
we see the res"lts,
[me@linuxbox ~]$ local-vars
global: foo M (
functO#: foo M #
global: foo M (
functO!: foo M !
global: foo M (
We see that the assigmet o+ #al"es to the local #aria%le foo withi %oth shell +"ctios
has o e++ect o the #al"e o+ foo !e+ie! o"tsi!e the +"ctios.
This +eat"re allows shell +"ctios to %e writte so that they remai i!epe!et o+ each
other a! o+ the script i which they appear. This is #ery #al"a%le, as it helps pre#et oe
part o+ a program +rom iter+erig with aother. 9t also allows shell +"ctios to %e
writte so that they ca %e porta%le. That is, they may %e c"t a! paste! +rom script to
script, as ee!e!.
=eep Scripts 1unning
While !e#elopig o"r program, it is "se+"l to keep the program i a r"a%le state. Jy
!oig this, a! testig +re="etly, we ca !etect errors early i the !e#elopmet process.
This will make !e%"ggig pro%lems m"ch easier. 0or e3ample, i+ we r" the program,
make a small chage, the r" the program agai a! +i! a pro%lem, itIs #ery likely that
the most recet chage is the so"rce o+ the pro%lem. Jy a!!ig the empty +"ctios,
calle! stubs i programmer-speak, we ca #eri+y the logical +low o+ o"r program at a
early stage. Whe costr"ctig a st"%, itIs a goo! i!ea to icl"!e somethig that
pro#i!es +ee!%ack to the programmer, which shows the logical +low is %eig carrie! o"t.
9+ we look at the o"tp"t o+ o"r script ow,
[me@linuxbox ~]$ s!s.info.page
XST+=D
XS&4'D
(8.
2/ 8 Top-'ow 'esig
XTAT=&D*1stem Anformation Re7ort -or t9in!X6TAT=&D
X6S&4'D
XH'UD
XS#D*1stem Anformation Re7ort -or linuxboxX6S#D
X:D@enerated ($6#06!((0 (%:(!:#( :+ &'T> b1 meX6:D
X6H'UD
X6ST+=D
we see that there are some %lak lies i o"r o"tp"t a+ter the time stamp, %"t we caIt %e
s"re o+ the ca"se. 9+ we chage the +"ctios to icl"!e some +ee!%ack,
re7ortOu7time NP [
echo J7unction report.uptime executed.J
return
\
re7ortOdiskOs7ace NP [
echo J7unction report.disk.space executed.J
return
\
re7ortOhomeOs7ace NP [
echo J7unction report.home.space executed.J
return
\
a! r" the script agai,
[me@linuxbox ~]$ s!s.info.page
XST+=D
XS&4'D
XTAT=&D*1stem Anformation Re7ort -or linuxboxX6TAT=&D
X6S&4'D
XH'UD
XS#D*1stem Anformation Re7ort -or linuxboxX6S#D
X:D@enerated ($6!(6!((0 (":#):!. 4+ &'T> b1 meX6:D
-unction re7ortOu7time executed<
-unction re7ortOdiskOs7ace executed<
-unction re7ortOhomeOs7ace executed<
X6H'UD
X6ST+=D
(82
Meep Scripts :"ig
we ow see that, i +act, o"r three +"ctios are %eig e3ec"te!.
With o"r +"ctio +ramework i place a! workig, itIs time to +lesh o"t some o+ the
+"ctio co!e. 0irst, the re7ortOu7time +"ctio,
re7ortOu7time NP [
cat //- .GS7.
/R2&F!stem Eptime//R2&
/BHG&34uptime6//BHG&
.GS7.
return
\
9tIs pretty straight+orwar!. We "se a here !oc"met to o"tp"t a sectio hea!er a! the
o"tp"t o+ the u7time comma!, s"rro"!e! %y X:R&D tags to preser#e the +ormattig
o+ the comma!. The re7ortOdiskOs7ace +"ctio is similar,
re7ortOdiskOs7ace NP [
cat //- .GS7.
/R2&1isk Fpace Etili-ation//R2&
/BHG&34df -h6//BHG&
.GS7.
return
\
This +"ctio "ses the df -h comma! to !etermie the amo"t o+ !isk space. *astly,
weIll %"il! the re7ortOhomeOs7ace +"ctio,
re7ortOhomeOs7ace NP [
cat //- .GS7.
/R2&Rome Fpace Etili-ation//R2&
/BHG&34du -sh /home/*6//BHG&
.GS7.
return
\
We "se the du comma! with the -sh optios to per+orm this task. This, howe#er, is ot
a complete sol"tio to the pro%lem. While it will work o some systems E)%"t", +or
e3ampleF, it will ot work o others. The reaso is that may systems set the permissios
o+ home !irectories to pre#et them +rom %eig worl!-rea!a%le, which is a reasoa%le
sec"rity meas"re. 7 these systems, the re7ortOhomeOs7ace +"ctio, as writte,
(8(
2/ 8 Top-'ow 'esig
will oly work i+ o"r script is r" with s"per"ser pri#ileges. $ %etter sol"tio wo"l! %e to
ha#e the script co"l! a!4"st its %eha#ior accor!ig to the pri#ileges o+ the "ser. We will
take this "p i the e3t chapter.
Shell ;unctions In :our .bashrc ;ile
Shell +"ctios make e3cellet replacemets +or aliases, a! are act"ally the
pre+erre! metho! o+ creatig small comma!s +or persoal "se. $liases are #ery
limite! i the ki! o+ comma!s a! shell +eat"res they s"pport, whereas shell
+"ctios allow aythig that ca %e scripte!. 0or e3ample, i+ we like! the
re7ortOdiskOs7ace shell +"ctio that we !e#elope! +or o"r script, we
co"l! create a similar +"ctio ame! ds +or o"r .bashrc +ile,
ds NP [
echo c'isk *7ace 3tiliCation -or $S*TL4+&d
df -h
\
Summing -p
9 this chapter, we ha#e itro!"ce! a commo metho! o+ program !esig calle! top-
!ow !esig, a! we ha#e see how shell +"ctios are "se! to %"il! the stepwise
re+iemet that it re="ires. We ha#e also see how local #aria%les ca %e "se! to make
shell +"ctios i!epe!et +rom oe aother a! +rom the program i which they are
place!. This makes it possi%le +or shell +"ctios to %e writte i a porta%le maer a!
to %e reusable %y allowig them to %e place! i m"ltiple programsH a great time sa#er.
;urther 1eading
The Wikipe!ia has may articles o so+tware !esig philosophy. Aere are a
co"ple o+ goo! oes,
http,--e.wikipe!ia.org-wiki-Top-!owO!esig
http,--e.wikipe!ia.org-wiki-S"%ro"ties
(81
28 8 0low Cotrol, Jrachig With i+
2- (lo5 Control; 6ranching With if
9 the last chapter, we were presete! with a pro%lem. Aow ca we make o"r report
geerator script a!apt to the pri#ileges o+ the "ser r"ig the scriptC The sol"tio to this
pro%lem will re="ire "s to +i! a way to ?chage !irectios@ withi o"r script, %ase! o a
the res"lts o+ a test. 9 programmig terms, we ee! the program to branch.
*etIs cosi!er a simple e3ample o+ logic e3presse! i pseudocode, a sim"latio o+ a
comp"ter lag"age ite!e! +or h"ma cos"mptio,
G W 2
9+ G W 2, the,
Say ?G e="als 2.@
7therwise,
Say ?G is ot e="al to 2.@
This is a e3ample o+ a %rach. Jase! o the co!itio, ?'oes G W 2C@ !o oe thig,
?Say G e="als 2,@ otherwise !o aother thig, ?Say G is ot e="al to 2.@
if
)sig the shell, we ca co!e the logic a%o#e as +ollows,
xM"
if [ $x M " ]T then
echo _x eBuals "<_
else
echo _x does not eBual "<_
fi
or we ca eter it !irectly at the comma! lie Eslightly shortee!F,
(82
28 8 0low Cotrol, Jrachig With i+
[me@linuxbox ~]$ x$"
[me@linuxbox ~]$ if ( 3x $ " )# then echo Je,uals "J# else echo Jdoes
not e,ual "J# fi
eBuals "
[me@linuxbox ~]$ x$*
[me@linuxbox ~]$ if ( 3x $ " )# then echo Je,uals "J# else echo Jdoes
not e,ual "J# fi
does not eBual "
9 this e3ample, we e3ec"te the comma! twice. 7ce, with the #al"e o+ x set to 2,
which res"lts i the strig ?e="als 2@ %eig o"tp"t, a! the seco! time with the #al"e o+
x set to 0, which res"lts i the strig ?!oes ot e="al 2@ %eig o"tp"t.
The if statemet has the +ollowig syta3,
if commandsT then
commands
[elif commandsT then
commands<<<]
[else
commands]
fi
where commands is a list o+ comma!s. This is a little co+"sig at +irst glace. J"t
%e+ore we ca clear this "p, we ha#e to look at how the shell e#al"ates the s"ccess or
+ail"re o+ a comma!.
$xit Status
Comma!s Eicl"!ig the scripts a! shell +"ctios we writeF iss"e a #al"e to the system
whe they termiate, calle! a e+it status. This #al"e, which is a iteger i the rage o+
0 to 222, i!icates the s"ccess or +ail"re o+ the comma!Is e3ec"tio. Jy co#etio, a
#al"e o+ Dero i!icates s"ccess a! ay other #al"e i!icates +ail"re. The shell pro#i!es a
parameter that we ca "se to e3amie the e3it stat"s. Aere we see it i actio,
[me@linuxbox ~]$ ls -d /usr/bin
6usr6bin
[me@linuxbox ~]$ echo 3_
(
[me@linuxbox ~]$ ls -d /bin/usr
ls: cannot access 6bin6usr: Lo such file or director1
[me@linuxbox ~]$ echo 3_
!
(8K
E3it Stat"s
9 this e3ample, we e3ec"te the ls comma! twice. The +irst time, the comma!
e3ec"tes s"ccess+"lly. 9+ we !isplay the #al"e o+ the parameter $F, we see that it is Dero.
We e3ec"te the ls comma! a seco! time, pro!"cig a error a! e3amie the
parameter $F agai. This time it cotais a 2, i!icatig that the comma! eco"tere!
a error. Some comma!s "se !i++eret e3it stat"s #al"es to pro#i!e !iagostics +or
errors, while may comma!s simply e3it with a #al"e o+ oe whe they +ail. >a pages
o+te icl"!e a sectio etitle! ?E3it Stat"s,@ !escri%ig what co!es are "se!. Aowe#er,
a Dero always i!icates s"ccess.
The shell pro#i!es two e3tremely simple %"ilti comma!s that !o othig e3cept
termiate with either a Dero or oe e3it stat"s. The true comma! always e3ec"tes
s"ccess+"lly a! the false comma! always e3ec"tes "s"ccess+"lly,
[me@linuxbox ~]$ true
[me@linuxbox ~]$ echo 3_
(
[me@linuxbox ~]$ false
[me@linuxbox ~]$ echo 3_
#
We ca "se these comma!s to see how the if statemet works. What the if statemet
really !oes is e#al"ate the s"ccess or +ail"re o+ comma!s,
[me@linuxbox ~]$ if true# then echo JPt%s true.J# fi
AtKs true<
[me@linuxbox ~]$ if false# then echo JPt%s true.J# fi
[me@linuxbox ~]$
The comma! echo _AtKs true<_ is e3ec"te! whe the comma! +ollowig if
e3ec"tes s"ccess+"lly, a! is ot e3ec"te! whe the comma! +ollowig if !oes ot
e3ec"te s"ccess+"lly. 9+ a list o+ comma!s +ollows if, the last comma! i the list is
e#al"ate!,
[me@linuxbox ~]$ if falseT trueT then echo _AtKs true<_T fi
AtKs true<
[me@linuxbox ~]$ if trueT falseT then echo _AtKs true<_T fi
[me@linuxbox ~]$
(8/
28 8 0low Cotrol, Jrachig With i+
test
Jy +ar, the comma! "se! most +re="etly with if is test. The test comma!
per+orms a #ariety o+ checks a! comparisos. 9t has two e="i#alet +orms,
test expression
a! the more pop"lar,
[ expression ]
where e+pression is a e3pressio that is e#al"ate! as either tr"e or +alse. The test
comma! ret"rs a e3it stat"s o+ Dero whe the e3pressio is tr"e a! a stat"s o+ oe
whe the e3pressio is +alse.
2ie 6x5ressions
The +ollowig e3pressios are "se! to e#al"ate the stat"s o+ +iles,
Table '4-1: test File 1+pressions
$xpression Is True I"I
file1 -ef file2
file1 a! file' ha#e the same io!e "m%ers Ethe two
+ileames re+er to the same +ile %y har! likigF.
file1 -nt file2
file1 is ewer tha file'.
file1 -ot file2
file1 is ol!er tha file'.
-b file
file e3ists a! is a %lock special E!e#iceF +ile.
-c file
file e3ists a! is a character special E!e#iceF +ile.
-d file
file e3ists a! is a !irectory.
-e file
file e3ists.
-f file
file e3ists a! is a reg"lar +ile.
-g file
file e3ists a! is set-gro"p-9'.
-@ file
file e3ists a! is owe! %y the e++ecti#e gro"p 9'.
-k file
file e3ists a! has its ?sticky %it@ set.
-= file
file e3ists a! is a sym%olic lik.
- file
file e3ists a! is owe! %y the e++ecti#e "ser 9'.
-7 file
file e3ists a! is a ame! pipe.
-r file
file e3ists a! is rea!a%le Ehas rea!a%le permissio +or
(88
test
the e++ecti#e "serF.
-s file
file e3ists a! has a legth greater tha Dero.
-* file
file e3ists a! is a etwork socket.
-t fd
fd is a +ile !escriptor !irecte! to-+rom the termial. This
ca %e "se! to !etermie whether sta!ar! ip"t-o"tp"t-
error is %eig re!irecte!.
-u file
file e3ists a! is set"i!.
-9 file
file e3ists a! is writa%le Ehas write permissio +or the
e++ecti#e "serF.
-x file
file e3ists a! is e3ec"ta%le Ehas e3ec"te-search
permissio +or the e++ecti#e "serF.
Aere we ha#e a script that !emostrates some o+ the +ile e3pressios,
^G6bin6bash
^ test-file: &valuate the status of a file
-A=&M~6<bashrc
if [ -e _$-A=&_ ]T then
if [ -f _$-A=&_ ]T then
echo _$-A=& is a regular file<_
fi
if [ -d _$-A=&_ ]T then
echo _$-A=& is a director1<_
fi
if [ -r _$-A=&_ ]T then
echo _$-A=& is readable<_
fi
if [ -9 _$-A=&_ ]T then
echo _$-A=& is 9ritable<_
fi
if [ -x _$-A=&_ ]T then
echo _$-A=& is executable6searchable<_
fi
else
echo _$-A=& does not exist_
exit #
fi
exit
(89
28 8 0low Cotrol, Jrachig With i+
The script e#al"ates the +ile assige! to the costat -A=& a! !isplays its res"lts as the
e#al"atio is per+orme!. There are two iterestig thigs to ote a%o"t this script. 0irst,
otice how the parameter $-A=& is ="ote! withi the e3pressios. This is ot re="ire!,
%"t is a !e+ese agaist the parameter %eig empty. 9+ the parameter e3pasio o+ $-A=&
were to res"lt i a empty #al"e, it wo"l! ca"se a error Ethe operators wo"l! %e
iterprete! as o-"ll strigs rather tha operatorsF. )sig the ="otes aro"! the
parameter is"res that the operator is always +ollowe! %y a strig, e#e i+ the strig is
empty. Seco!, otice the presece o+ the exit comma!s ear the e! o+ the script.
The exit comma! accepts a sigle, optioal arg"met, which %ecomes the scriptIs e3it
stat"s. Whe o arg"met is passe!, the e3it stat"s !e+a"lts to Dero. )sig exit i this
way allows the script to i!icate +ail"re i+ $-A=& e3pa!s to the ame o+ a oe3istet
+ile. The exit comma! appearig o the last lie o+ the script is there as a +ormality.
Whe a script ?r"s o++ the e!@ Ereaches e! o+ +ileF, it termiates with a e3it stat"s o+
Dero %y !e+a"lt, ayway.
Similarly, shell +"ctios ca ret"r a e3it stat"s %y icl"!ig a iteger arg"met to the
return comma!. 9+ we were to co#ert the script a%o#e to a shell +"ctio to icl"!e
it i a larger program, we co"l! replace the exit comma!s with return statemets
a! get the !esire! %eha#ior,
testOfile NP [
^ test-file: &valuate the status of a file
-A=&M~6<bashrc
if [ -e _$-A=&_ ]T then
if [ -f _$-A=&_ ]T then
echo _$-A=& is a regular file<_
fi
if [ -d _$-A=&_ ]T then
echo _$-A=& is a director1<_
fi
if [ -r _$-A=&_ ]T then
echo _$-A=& is readable<_
fi
if [ -9 _$-A=&_ ]T then
echo _$-A=& is 9ritable<_
fi
if [ -x _$-A=&_ ]T then
echo _$-A=& is executable6searchable<_
fi
else
echo _$-A=& does not exist_
return #
(90
test
fi
\
String 6x5ressions
The +ollowig e3pressios are "se! to e#al"ate strigs,
Table '4-': test String 1+pressions
$xpression Is True I"...
string
string is ot "ll.
-n string
The legth o+ string is greater tha Dero.
-C string
The legth o+ string is Dero.
string1 M string2
string1 !! string2
string1 a! string' are e="al. Sigle or !o"%le
e="al sigs may %e "se!, %"t the "se o+ !o"%le
e="al sigs is greatly pre+erre!.
string1 GM string2
string1 a! string' are ot e="al.
string1 D string2
string1 sorts a+ter string'.
string1 X string2
string1 sorts %e+ore string'.
'arning, the D a! X e3pressio operators m"st %e ="ote! Eor escape! with a
%ackslashF whe "se! with test. 9+ they are ot, they will %e iterprete! %y the
shell as re!irectio operators, with potetially !estr"cti#e res"lts. $lso ote that
while the bash !oc"metatio states that the sortig or!er co+orms to the
collatio or!er o+ the c"rret locale, it !oes ot. $SC99 E67S9GF or!er is "se! i
#ersios o+ bash "p to a! icl"!ig 1.0.
Aere is a script that icorporates strig e3pressios,
^G6bin6bash
^ test-string: evaluate the value of a string
4L*,&RMma1be
if [ -C _$4L*,&R_ ]T then
(9.
28 8 0low Cotrol, Jrachig With i+
echo _There is no ans9er<_ DY!
exit #
fi
if [ _$4L*,&R_ M _1es_ ]T then
echo _The ans9er is U&*<_
elif [ _$4L*,&R_ M _no_ ]T then
echo _The ans9er is L<_
elif [ _$4L*,&R_ M _ma1be_ ]T then
echo _The ans9er is +4UH&<_
else
echo _The ans9er is 3L2L,L<_
fi
9 this script, we e#al"ate the costat 4L*,&R. We +irst !etermie i+ the strig is
empty. 9+ it is, we termiate the script a! set the e3it stat"s to oe. &otice the
re!irectio that is applie! to the echo comma!. This re!irects the error message ?There
is o aswer.@ to sta!ar! error, which is the ?proper@ thig to !o with error messages. 9+
the strig is ot empty, we e#al"ate the #al"e o+ the strig to see i+ it is e="al to either
?yes,@ ?o,@ or ?may%e.@ We !o this %y "sig elif, which is short +or ?else i+.@ Jy
"sig elif, we are a%le to costr"ct a more comple3 logical test.
Integer 6x5ressions
The +ollowig e3pressios are "se! with itegers,
Table '4-3: test *nteger 1+pressions
$xpression Is True I"...
integer1 -eB integer2
integer1 is e="al to integer'.
integer1 -ne integer2
integer1 is ot e="al to integer'.
integer1 -le integer2
integer1 is less tha or e="al to integer'.
integer1 -lt integer2
integer1 is less tha integer'.
integer1 -ge integer2
integer1 is greater tha or e="al to integer'.
integer1 -gt integer2
integer1 is greater tha integer'.
Aere is a script that !emostrates them,
^G6bin6bash
(92
test
^ test-integer: evaluate the value of an integer<
ALTM-"
if [ -C _$ALT_ ]T then
echo _ALT is em7t1<_ DY!
exit #
fi
if [ $ALT -eB ( ]T then
echo _ALT is Cero<_
else
if [ $ALT -lt ( ]T then
echo _ALT is negative<_
else
echo _ALT is 7ositive<_
fi
if [ $NNALT 5 !PP -eB ( ]T then
echo _ALT is even<_
else
echo _ALT is odd<_
fi
fi
The iterestig part o+ the script is how it !etermies whether a iteger is e#e or o!!.
Jy per+ormig a mo!"lo 2 operatio o the "m%er, which !i#i!es the "m%er %y two
a! ret"rs the remai!er, it ca tell i+ the "m%er is o!! or e#e.
# >ore >odern Version ?" test
:ecet #ersios o+ bash icl"!e a compo"! comma! that acts as a ehace!
replacemet +or test. 9t "ses the +ollowig syta3,
[[ expression ]]
where, like test, e+pression is a e3pressio that e#al"ates to either a tr"e or +alse
res"lt. The [[ ]] comma! is #ery similar to test Eit s"pports all o+ its e3pressiosF,
%"t a!!s a importat ew strig e3pressio,
string1 M~ regex
which ret"rs tr"e i+ string1 is matche! %y the e3te!e! reg"lar e3pressio rege+. This
opes "p a lot o+ possi%ilities +or per+ormig s"ch tasks as !ata #ali!atio. 9 o"r earlier
e3ample o+ the iteger e3pressios, the script wo"l! +ail i+ the costat ALT cotaie!
aythig e3cept a iteger. The script ee!s a way to #eri+y that the costat cotais a
iteger. )sig [[ ]] with the M~ strig e3pressio operator, we co"l! impro#e the
(9(
28 8 0low Cotrol, Jrachig With i+
script this way,
^G6bin6bash
^ test-integer!: evaluate the value of an integer<
ALTM-"
if (( J3P?+J $~ [-_(*-D)53 ))# then
if [ $ALT -eB ( ]T then
echo _ALT is Cero<_
else
if [ $ALT -lt ( ]T then
echo _ALT is negative<_
else
echo _ALT is 7ositive<_
fi
if [ $NNALT 5 !PP -eB ( ]T then
echo _ALT is even<_
else
echo _ALT is odd<_
fi
fi
else
echo JP?+ is not an integer.J &'2
exit 1
fi
Jy applyig the reg"lar e3pressio, we are a%le to limit the #al"e o+ ALT to oly strigs
that %egi with a optioal mi"s sig, +ollowe! %y oe or more "merals. This
e3pressio also elimiates the possi%ility o+ empty #al"es.
$other a!!e! +eat"re o+ [[ ]] is that the MM operator s"pports patter matchig the
same way pathame e3pasio !oes. 0or e3ample,
[me@linuxbox ~]$ 7PTG$foo.bar
[me@linuxbox ~]$ if (( 37PTG $$ foo.* ))# then
D echo J37PTG matches pattern %foo.*%J
D fi
foo<bar matches 7attern Kfoo<EK
This makes [[ ]] "se+"l +or e#al"atig +ile a! path ames.
(91
EE FF - 'esige! 0or 9tegers
44 66 E Designed ;or Integers
9 a!!itio to the [[ ]] compo"! comma!, bash also pro#i!es the NN PP
compo"! comma!, which is "se+"l +or operatig o itegers. 9t s"pports a +"ll set o+
arithmetic e#al"atios, a s"%4ect we will co#er +"lly i Chapter (2.
NN PP is "se! to per+orm arithmetic truth tests. $ arithmetic tr"th test res"lts i tr"e i+
the res"lt o+ the arithmetic e#al"atio is o-Dero.
[me@linuxbox ~]$ if 44166# then echo JPt is true.J# fi
At is true<
[me@linuxbox ~]$ if 44*66# then echo JPt is true.J# fi
[me@linuxbox ~]$
)sig NN PP, we ca slightly simpli+y the test-integer! script like this,
^G6bin6bash
^ test-integer!a: evaluate the value of an integer<
ALTM-"
if [[ _$ALT_ M~ `-F[(-0]8$ ]]T then
if 44P?+ $$ *66T then
echo _ALT is Cero<_
else
if 44P?+ / *66T then
echo _ALT is negative<_
else
echo _ALT is 7ositive<_
fi
if 44 44P?+ 8 266 $$ *66T then
echo _ALT is even<_
else
echo _ALT is odd<_
fi
fi
else
echo _ALT is not an integer<_ DY!
exit #
fi
&otice that we "se less tha a! greater tha sigs a! that MM is "se! to test +or
e="i#alece. This is a more at"ral lookig syta3 +or workig with itegers. &otice
too, that %eca"se the compo"! comma! NN PP is part o+ the shell syta3 rather tha
(92
28 8 0low Cotrol, Jrachig With i+
a or!iary comma!, a! it !eals oly with itegers, it is a%le to recogiDe #aria%les %y
ame a! !oes ot re="ire e3pasio to %e per+orme!. WeIll !isc"ss NN PP a! the
relate! arithmetic e3pasio +"rther i Chapter (2.
Com0ining $xpressions
9tIs also possi%le to com%ie e3pressios to create more comple3 e#al"atios.
E3pressios are com%ie! %y "sig logical operators. We saw these i Chapter .8, whe
we leare! a%o"t the find comma!. There are three logical operatios +or test a!
[[ ]]. They are $&', 7: a! &7T. test a! [[ ]]"se !i++eret operators to
represet these operatios ,
Table '4-$: (ogical &perators
?peration
test
(( )) and 44 66
$&'
-a YY
7:
-o QQ
&7T
G G
AereIs a e3ample o+ a $&' operatio. The +ollowig script !etermies i+ a iteger is
withi a rage o+ #al"es,
^G6bin6bash
^ test-integer$: determine if an integer is 9ithin a
^ s7ecified range of values<
+ALO;4=M#
+4VO;4=M#((
ALTM"(
if [[ _$ALT_ M~ `-F[(-0]8$ ]]T then
if [[ ALT -ge +ALO;4= YY ALT -le +4VO;4= ]]T then
echo _$ALT is 9ithin $+ALO;4= to $+4VO;4=<_
else
echo _$ALT is out of range<_
fi
else
echo _ALT is not an integer<_ DY!
exit #
fi
(9K
Com%iig E3pressios
9 this script, we !etermie i+ the #al"e o+ iteger ALT lies %etwee the #al"es o+
+ALO;4= a! +4VO;4=. This is per+orme! %y a sigle "se o+ [[ ]], which icl"!es
two e3pressios separate! %y the YY operator. We co"l! ha#e also co!e! this "sig
test,
if ( $ALT -ge $+ALO;4= -a $ALT -le $+4VO;4= )T then
echo _$ALT is 9ithin $+ALO;4= to $+4VO;4=<_
else
echo _$ALT is out of range<_
fi
The G egatio operator re#erses the o"tcome o+ a e3pressio. 9t ret"rs tr"e i+ a
e3pressio is +alse, a! it ret"rs +alse i+ a e3pressio is tr"e. 9 the +ollowig script, we
mo!i+y the logic o+ o"r e#al"atio to +i! #al"es o+ ALT that are o"tsi!e the speci+ie!
rage,
^G6bin6bash
^ test-integer%: determine if an integer is outside a
^ s7ecified range of values<
+ALO;4=M#
+4VO;4=M#((
ALTM"(
if [[ _$ALT_ M~ `-F[(-0]8$ ]]T then
if [[ G NALT -ge +ALO;4= YY ALT -le +4VO;4=P ]]T then
echo _$ALT is outside $+ALO;4= to $+4VO;4=<_
else
echo _$ALT is in range<_
fi
else
echo _ALT is not an integer<_ DY!
exit #
fi
We also icl"!e paretheses aro"! the e3pressio, +or gro"pig. 9+ these were ot
icl"!e!, the egatio wo"l! oly apply to the +irst e3pressio a! ot the com%iatio
o+ the two. Co!ig this with test wo"l! %e !oe this way,
if [ G ]N $ALT -ge $+ALO;4= -a $ALT -le $+4VO;4= ]P ]T then
(9/
28 8 0low Cotrol, Jrachig With i+
echo _$ALT is outside $+ALO;4= to $+4VO;4=<_
else
echo _$ALT is in range<_
fi
Sice all e3pressios a! operators "se! %y test are treate! as comma! arg"mets %y
the shell E"like [[ ]] a! NN PP F, characters which ha#e special meaig to bash,
s"ch as X, D, N, a! P, m"st %e ="ote! or escape!.
Seeig that test a! [[ ]] !o ro"ghly the same thig, which is pre+era%leC test is
tra!itioal Ea! part o+ 67S9GF, whereas [[ ]] is speci+ic to bash. 9tIs importat to
kow how to "se test, sice it is #ery wi!ely "se!, %"t [[ ]] is clearly more "se+"l
a! is easier to co!e.
Porta0ilit, Is The 3o0go0lin ?" Little >inds
9+ yo" talk to ?real@ )i3 people, yo" ="ickly !isco#er that may o+ them !oIt
like *i"3 #ery m"ch. They regar! it as imp"re a! "clea. 7e teet o+ )i3
+ollowers is that e#erythig sho"l! %e ?porta%le.@ This meas that ay script yo"
write sho"l! %e a%le to r", "chage!, o ay )i3-like system.
)i3 people ha#e goo! reaso to %elie#e this. Aa#ig see what proprietary
e3tesios to comma!s a! shells !i! to the )i3 worl! %e+ore 67S9G, they are
at"rally wary o+ the e++ect o+ *i"3 o their %elo#e! 7S.
J"t porta%ility has a serio"s !owsi!e. 9t pre#ets progress. 9t re="ires that
thigs are always !oe "sig ?lowest commo !eomiator@ techi="es. 9 the
case o+ shell programmig, it meas makig e#erythig compati%le with sh, the
origial Jo"re shell.
This !owsi!e is the e3c"se that proprietary #e!ors "se to 4"sti+y their
proprietary e3tesios, oly they call them ?io#atios.@ J"t they are really 4"st
lock-i !e#ices +or their c"stomers.
The ;&) tools, s"ch as bash, ha#e o s"ch restrictios. They eco"rage
porta%ility %y s"pportig sta!ar!s a! %y %eig "i#ersally a#aila%le. 5o" ca
istall bash a! the other ;&) tools o almost ay ki! o+ system, e#e
Wi!ows, witho"t cost. So +eel +ree to "se all the +eat"res o+ bash. 9tIs really
porta%le.
(98
Cotrol 7perators, $other Way To Jrach
Control ?peratorsI #nother +a, To /ranch
bash pro#i!es two cotrol operators that ca per+orm %rachig. The YY E$&'F a! QQ
E7:F operators work like the logical operators i the [[ ]] compo"! comma!. This
is the syta3,
command1 YY command2
a!
command1 QQ command2
9t is importat to "!ersta! the %eha#ior o+ these. With the YY operator, command1 is
e3ec"te! a! command' is e3ec"te! i+, and only if, command1 is s"ccess+"l. With the
QQ operator, command1 is e3ec"te! a! command' is e3ec"te! i+, and only if,
command1 is "s"ccess+"l.
9 practical terms, it meas that we ca !o somethig like this,
[me@linuxbox ~]$ mkdir temp '' cd temp
This will create a !irectory ame! tem7, a! i+ it s"ccee!s, the c"rret workig !irectory
will %e chage! to tem7. The seco! comma! is attempte! oly i+ the mkdir
comma! is s"ccess+"l. *ikewise, a comma! like this,
[me@linuxbox ~]$ ( -d temp ) 00 mkdir temp
will test +or the e3istece o+ the !irectory tem7, a! oly i+ the test +ails, will the
!irectory %e create!. This type o+ costr"ct is #ery ha!y +or ha!lig errors i scripts, a
s"%4ect we will !isc"ss more i later chapters. 0or e3ample, we co"l! !o this i a script,
[ -d tem7 ] QQ exit #
9+ the script re="ires the !irectory tem7, a! it !oes ot e3ist, the the script will
termiate with a e3it stat"s o+ oe.
Summing -p
We starte! this chapter with a ="estio. Aow co"l! we make o"r s1sOinfoO7age
script !etect i+ the "ser ha! permissio to rea! all the home !irectoriesC With o"r
kowle!ge o+ if, we ca sol#e the pro%lem %y a!!ig this co!e to the
(99
28 8 0low Cotrol, Jrachig With i+
re7ortOhomeOs7ace +"ctio,
re7ortOhomeOs7ace NP [
if (( 34id -u6 -e, * ))# then
cat //- .GS7.
/R2&Rome Fpace Etili-ation 4:ll Esers6//R2&
/BHG&34du -sh /home/*6//BHG&
.GS7.
else
cat //- .GS7.
/R2&Rome Fpace Etili-ation 43EFGH6//R2&
/BHG&34du -sh 3RSOG6//BHG&
.GS7.
fi
return
\
We e#al"ate the o"tp"t o+ the id comma!. With the -u optio, id o"tp"ts the "meric
"ser 9' "m%er o+ the e++ecti#e "ser. The s"per"ser is always Dero a! e#ery other "ser
is a "m%er greater tha Dero. Mowig this, we ca costr"ct two !i++eret here
!oc"mets, oe takig a!#atage o+ s"per"ser pri#ileges, a! the other, restricte! to the
"serIs ow home !irectory.
We are goig to take a %reak +rom the s1sOinfoO7age program, %"t !oIt worry. 9t
will %e %ack. 9 the meatime, weIll co#er some topics that weIll ee! whe we res"me
o"r work.
;urther 1eading
There are se#eral sectios o+ the bash ma page that pro#i!e +"rther !etail o the topics
co#ere! i this chapter,
*ists Eco#ers the cotrol operators QQ a! YYF
Compo"! Comma!s Eco#ers [[ ]], NN PP a! ifF
C7&'9T97&$* EG6:ESS97&S
SAE** J)9*T9& C7>>$&'S Eco#ers testF
0"rther, the Wikipe!ia has a goo! article o the cocept o+ pse"!oco!e,
http,--e.wikipe!ia.org-wiki-6se"!oco!e
100
29 8 :ea!ig Mey%oar! 9p"t
20 /eading 1e$board In#ut
The scripts we ha#e writte so +ar lack a +eat"re commo i most comp"ter programsB
interactivity. That is, the a%ility o+ the program to iteract with the "ser. While may
programs !oIt ee! to %e iteracti#e, some programs %ee+it +rom %eig a%le to accept
ip"t !irectly +rom the "ser. Take, +or e3ample, this script +rom the pre#io"s chapter,
^G6bin6bash
^ test-integer!: evaluate the value of an integer<
ALTM-"
if [[ _$ALT_ M~ `-F[(-0]8$ ]]T then
if [ $ALT -eB ( ]T then
echo _ALT is Cero<_
else
if [ $ALT -lt ( ]T then
echo _ALT is negative<_
else
echo _ALT is 7ositive<_
fi
if [ $NNALT 5 !PP -eB ( ]T then
echo _ALT is even<_
else
echo _ALT is odd<_
fi
fi
else
echo _ALT is not an integer<_ DY!
exit #
fi
Each time we wat to chage the #al"e o+ ALT, we ha#e to e!it the script. 9t wo"l! %e
m"ch more "se+"l i+ the script co"l! ask the "ser +or a #al"e. 9 this chapter, we will
%egi to look at how we ca a!! iteracti#ity to o"r programs.
10.
29 8 :ea!ig Mey%oar! 9p"t
read 1ead Values ;rom Standard Input
The read %"ilti comma! is "se! to rea! a sigle lie o+ sta!ar! ip"t. This
comma! ca %e "se! to rea! key%oar! ip"t or, whe re!irectio is employe!, a lie o+
!ata +rom a +ile. The comma! has the +ollowig syta3,
read [-options] [variable<<<]
where options is oe or more o+ the a#aila%le optios liste! %elow a! variable is the
ame o+ oe or more #aria%les "se! to hol! the ip"t #al"e. 9+ o #aria%le ame is
s"pplie!, the shell #aria%le R&:=U cotais the lie o+ !ata.
Jasically, read assigs +iel!s +rom sta!ar! ip"t to the speci+ie! #aria%les. 9+ we
mo!i+y o"r iteger e#al"atio script to "se read, it might look like this,
^G6bin6bash
^ read-integer: evaluate the value of an integer<
echo -n JBlease enter an integer -& J
read int
if [[ _$int_ M~ `-F[(-0]8$ ]]T then
if [ $int -eB ( ]T then
echo _$int is Cero<_
else
if [ $int -lt ( ]T then
echo _$int is negative<_
else
echo _$int is 7ositive<_
fi
if [ $NNint 5 !PP -eB ( ]T then
echo _$int is even<_
else
echo _$int is odd<_
fi
fi
else
echo _An7ut value is not an integer<_ DY!
exit #
fi
We "se echo with the -n optio Ewhich s"ppresses the trailig ewlie o o"tp"tF to
!isplay a prompt, the "se read to ip"t a #al"e +or the #aria%le int. :"ig this
script res"lts i this,
102
rea! 8 :ea! Ral"es 0rom Sta!ar! 9p"t
[me@linuxbox ~]$ read-integer
:lease enter an integer -D "
" is 7ositive<
" is odd<
read ca assig ip"t to m"ltiple #aria%les, as show i this script,
^G6bin6bash
^ read-multi7le: read multi7le values from ke1board
echo -n _&nter one or more values D _
read var# var! var$ var% var"
echo _var# M K$var#K_
echo _var! M K$var!K_
echo _var$ M K$var$K_
echo _var% M K$var%K_
echo _var" M K$var"K_
9 this script, we assig a! !isplay "p to +i#e #al"es. &otice how read %eha#es whe
gi#e !i++eret "m%ers o+ #al"es,
[me@linuxbox ~]$ read-multiple
&nter one or more values D a b c d e
var# M KaK
var! M KbK
var$ M KcK
var% M KdK
var" M KeK
[me@linuxbox ~]$ read-multiple
&nter one or more values D a
var# M KaK
var! M KK
var$ M KK
var% M KK
var" M KK
[me@linuxbox ~]$ read-multiple
&nter one or more values D a b c d e f g
var# M KaK
var! M KbK
var$ M KcK
var% M KdK
var" M Ke f gK
10(
29 8 :ea!ig Mey%oar! 9p"t
9+ read recei#es +ewer tha the e3pecte! "m%er, the e3tra #aria%les are empty, while a
e3cessi#e amo"t o+ ip"t res"lts i the +ial #aria%le cotaiig all o+ the e3tra ip"t.
9+ o #aria%les are liste! a+ter the read comma!, a shell #aria%le, R&:=U, will %e
assige! all the ip"t,
^G6bin6bash
^ read-single: read multi7le values into default variable
echo -n _&nter one or more values D _
read
echo _R&:=U M K$R&:=UK_
:"ig this script res"lts i this,
[me@linuxbox ~]$ read-single
&nter one or more values D a b c d
R&:=U M Ka b c dK
<5tions
read s"pports the +ollowig optios,
Table '5-1: read &ptions
?ption Description
-a arra
$ssig the ip"t to array, startig with i!e3 Dero. We
will co#er arrays i Chapter (K.
-d delimiter
The +irst character i the strig delimiter is "se! to
i!icate e! o+ ip"t, rather tha a ewlie character.
-e
)se :ea!lie to ha!le ip"t. This permits ip"t e!itig
i the same maer as the comma! lie.
-n num
:ea! num characters o+ ip"t, rather tha a etire lie.
-7 prompt
'isplay a prompt +or ip"t "sig the strig prompt.
-r
:aw mo!e. 'o ot iterpret %ackslash characters as
escapes.
101
rea! 8 :ea! Ral"es 0rom Sta!ar! 9p"t
-s
Silet mo!e. 'o ot echo characters to the !isplay as
they are type!. This is "se+"l whe ip"ttig passwor!s
a! other co+i!etial i+ormatio.
-t seconds
Timeo"t. Termiate ip"t a+ter seconds. read ret"rs a
o-Dero e3it stat"s i+ a ip"t times o"t.
-u fd
)se ip"t +rom +ile !escriptor fd, rather tha sta!ar!
ip"t.
)sig the #ario"s optios, we ca !o iterestig thigs with read. 0or e3ample, with
the -7 optio, we ca pro#i!e a prompt strig,
^G6bin6bash
^ read-single: read multi7le values into default variable
read -7 _&nter one or more values D _
echo _R&:=U M K$R&:=UK_
With the -t a! -s optios we ca write a script that rea!s ?secret@ ip"t a! times o"t
i+ the ip"t is ot complete! i a speci+ie! time,
^G6bin6bash
^ read-secret: in7ut a secret 7ass 7hrase
if read -t #( -s7 _&nter secret 7ass 7hrase D _ secretO7assT then
echo -e _]n*ecret 7ass 7hrase M K$secretO7assK_
else
echo -e _]nAn7ut timed out_ DY!
exit #
fi
The script prompts the "ser +or a secret pass phrase a! waits te seco!s +or ip"t. 9+
the etry is ot complete! withi the speci+ie! time, the script e3its with a error. Sice
the -s optio is icl"!e!, the characters o+ the pass phrase are ot echoe! to the !isplay
as they are type!.
102
29 8 :ea!ig Mey%oar! 9p"t
I2S
&ormally, the shell per+orms wor! splittig o the ip"t pro#i!e! to read. $s we ha#e
see, this meas that m"ltiple wor!s separate! %y oe or more spaces %ecome separate
items o the ip"t lie, a! are assige! to separate #aria%les %y read. This %eha#ior is
co+ig"re! %y a shell #aria%le ame! A-* E+or 9teral 0iel! SeparatorF. The !e+a"lt
#al"e o+ A-* cotais a space, a ta%, a! a ewlie character, each o+ which will separate
items +rom oe aother.
We ca a!4"st the #al"e o+ A-* to cotrol the separatio o+ +iel!s ip"t to read. 0or
e3ample, the 6etc67ass9d +ile cotais lies o+ !ata that "se the colo character as a
+iel! separator. Jy chagig the #al"e o+ A-* to a sigle colo, we ca "se read to
ip"t the cotets o+ 6etc67ass9d a! s"ccess+"lly separate +iel!s ito !i++eret
#aria%les. Aere we ha#e a script that !oes 4"st that,
^G6bin6bash
^ read-ifs: read fields from a file
-A=&M6etc67ass9d
read -7 _&nter a user name D _ userOname
fileOinfoM$Ngre7 _`$userOname:_ $-A=&P
if [ -n _$fileOinfo_ ]T then
A-*M_:_ read user 79 uid gid name home shell XXX _$fileOinfo_
echo _3ser M K$userK_
echo _3A' M K$uidK_
echo _@A' M K$gidK_
echo _-ull Lame M K$nameK_
echo _Some 'ir< M K$homeK_
echo _*hell M K$shellK_
else
echo _Lo such user K$userOnameK_ DY!
exit #
fi
This script prompts the "ser to eter the "ser ame o+ a acco"t o the system, the
!isplays the !i++eret +iel!s +o"! i the "serIs recor! i the 6etc67ass9d +ile. The
script cotais two iterestig lies. The +irst is,
fileOinfoM$Ngre7 _`$userOname:_ $-A=&P
This lie assigs the res"lts o+ a gre7 comma! to the #aria%le fileOinfo. The
reg"lar e3pressio "se! %y gre7 ass"res that the "ser ame will oly match a sigle lie
10K
rea! 8 :ea! Ral"es 0rom Sta!ar! 9p"t
i the 6etc67ass9d +ile.
The seco! iterestig lie is this oe,
A-*M_:_ read user 79 uid gid name home shell XXX _$fileOinfo_
The lie cosists o+ three parts, a #aria%le assigmet, a read comma! with a list o+
#aria%le ames as arg"mets, a! a strage ew re!irectio operator. WeIll look at the
#aria%le assigmet +irst.
The shell allows oe or more #aria%le assigmets to take place imme!iately %e+ore a
comma!. These assigmets alter the e#iromet +or the comma! that +ollows. The
e++ect o+ the assigmet is temporaryH oly chagig the e#iromet +or the !"ratio o+
the comma!. 9 o"r case, the #al"e o+ A-* is chage! to a colo character. $lterately,
we co"l! ha#e co!e! it this way,
='OA-*M_$A-*_
A-*M_:_
read user 79 uid gid name home shell XXX _$fileOinfo_
A-*M_$='OA-*_
where we store the #al"e o+ A-*, assig a ew #al"e, per+orm the read comma!, the
restore A-* to its origial #al"e. Clearly, placig the #aria%le assigmet i +rot o+ the
comma! is a more cocise way o+ !oig the same thig.
The XXX operator i!icates a here string. $ here strig is like a here !oc"met, oly
shorter, cosistig o+ a sigle strig. 9 o"r e3ample, the lie o+ !ata +rom the
6etc67ass9d +ile is +e! to the sta!ar! ip"t o+ the read comma!. We might
wo!er why this rather o%li="e metho! was chose rather tha,
echo _$fileOinfo_ Q A-*M_:_ read user 79 uid gid name home shell
Well, thereIs a reaso...
:ou CanFt Pipe read
While the read comma! ormally takes ip"t +rom sta!ar! ip"t, yo" caot
!o this,
echo _foo_ Q read
We wo"l! e3pect this to work, %"t it !oes ot. The comma! will appear to
s"ccee! %"t the R&:=U #aria%le will always %e empty. Why is thisC
The e3plaatio has to !o with the way the shell ha!les pipelies. 9 bash Ea!
other shells s"ch as shF, pipelies create subshells. These are copies o+ the shell
10/
29 8 :ea!ig Mey%oar! 9p"t
a! its e#iromet which are "se! to e3ec"te the comma! i the pipelie. 9
o"r e3ample a%o#e, read is e3ec"te! i a s"%shell.
S"%shells i )i3-like systems create copies o+ the e#iromet +or the processes
to "se while they e3ec"te. Whe the processes +iishes the copy o+ the
e#iromet is !estroye!. This meas that a subshell can never alter the
environment of its parent process. read assigs #aria%les, which the %ecome
part o+ the e#iromet. 9 the e3ample a%o#e, read assigs the #al"e ?+oo@ to
the #aria%le R&:=U i its s"%shellIs e#iromet, %"t whe the comma! e3its,
the s"%shell a! its e#iromet are !estroye!, a! the e++ect o+ the assigmet is
lost.
)sig here strigs is oe way to work aro"! this %eha#ior. $other metho! is
!isc"sse! i Chapter (/.
Validating Input
With o"r ew a%ility to ha#e key%oar! ip"t comes a a!!itioal programmig challege,
#ali!atig ip"t. Rery o+te the !i++erece %etwee a well-writte program a! a poorly
writte oe is i the programIs a%ility to !eal with the "e3pecte!. 0re="etly, the
"e3pecte! appears i the +orm o+ %a! ip"t. WeI#e !oe a little o+ this with o"r
e#al"atio programs i the pre#io"s chapter, where we checke! the #al"e o+ itegers a!
screee! o"t empty #al"es a! o-"meric characters. 9t is importat to per+orm these
ki!s o+ programmig checks e#ery time a program recei#es ip"t, to g"ar! agaist
i#ali! !ata. This is especially importat +or programs that are share! %y m"ltiple "sers.
7mittig these sa+eg"ar!s i the iterests o+ ecoomy might %e e3c"se! i+ a program is to
%e "se! oce a! oly %y the a"thor to per+orm some special task. E#e the, i+ the
program per+orms !agero"s tasks s"ch as !eletig +iles, it wo"l! %e wise to icl"!e !ata
#ali!atio, 4"st i case.
Aere we ha#e a e3ample program that #ali!ates #ario"s ki!s o+ ip"t,
^G6bin6bash
^ read-validate: validate in7ut
invalidOin7ut NP [
echo _Anvalid in7ut K$R&:=UK_ DY!
exit #
\
108
Rali!atig 9p"t
read -7 _&nter a single item D _
^ in7ut is em7t1 NinvalidP
[[ -C $R&:=U ]] YY invalidOin7ut
^ in7ut is multi7le items NinvalidP
NN $Necho $R&:=U Q 9c -9P D # PP YY invalidOin7ut
^ is in7ut a valid filenameF
if [[ $R&:=U M~ `[-[:alnum:]]<O]8$ ]]T then
echo _K$R&:=UK is a valid filename<_
if [[ -e $R&:=U ]]T then
echo _4nd file K$R&:=UK exists<_
else
echo _So9ever> file K$R&:=UK does not exist<_
fi
^ is in7ut a floating 7oint numberF
if [[ $R&:=U M~ `-F[[:digit:]]E]<[[:digit:]]8$ ]]T then
echo _K$R&:=UK is a floating 7oint number<_
else
echo _K$R&:=UK is not a floating 7oint number<_
fi
^ is in7ut an integerF
if [[ $R&:=U M~ `-F[[:digit:]]8$ ]]T then
echo _K$R&:=UK is an integer<_
else
echo _K$R&:=UK is not an integer<_
fi
else
echo _The string K$R&:=UK is not a valid filename<_
fi
This script prompts the "ser to eter a item. The item is s"%se="etly aalyDe! to
!etermie its cotets. $s we ca see, the script makes "se o+ may o+ the cocepts that
we ha#e co#ere! th"s +ar, icl"!ig shell +"ctios, [[ ]], NN PP, the cotrol operator
YY, a! if, as well as a healthy !ose o+ reg"lar e3pressios.
>enus
$ commo type o+ iteracti#ity is calle! menu-driven. 9 me"-!ri#e programs, the
"ser is presete! with a list o+ choices a! is aske! to choose oe. 0or e3ample, we co"l!
imagie a program that presete! the +ollowig,
:lease *elect:
109
29 8 :ea!ig Mey%oar! 9p"t
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
&nter selection [(-$] D
)sig what we leare! +rom writig o"r s1sOinfoO7age program, we ca costr"ct a
me"-!ri#e program to per+orm the tasks o the a%o#e me",
^G6bin6bash
^ read-menu: a menu driven s1stem information 7rogram
clear
echo _
:lease *elect:
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
_
read -7 _&nter selection [(-$] D _
if [[ $R&:=U M~ `[(-$]$ ]]T then
if [[ $R&:=U MM ( ]]T then
echo _:rogram terminated<_
exit
fi
if [[ $R&:=U MM # ]]T then
echo _Sostname: $S*TL4+&_
u7time
exit
fi
if [[ $R&:=U MM ! ]]T then
df -h
exit
fi
if [[ $R&:=U MM $ ]]T then
if [[ $Nid -uP -eB ( ]]T then
echo _Some *7ace 3tiliCation N4ll 3sersP_
du -sh 6home6E
else
echo _Some *7ace 3tiliCation N$3*&RP_
du -sh $S+&
1.0
>e"s
fi
exit
fi
else
echo _Anvalid entr1<_ DY!
exit #
fi
This script is logically !i#i!e! ito two parts. The +irst part !isplays the me" a! ip"ts
the respose +rom the "ser. The seco! part i!eti+ies the respose a! carries o"t the
selecte! actio. &otice the "se o+ the exit comma! i this script. 9t is "se! here to
pre#et the script +rom e3ec"tig "ecessary co!e a+ter a actio has %ee carrie! o"t.
The presece o+ m"ltiple e3it poits i a program is geerally a %a! i!ea Eit makes
program logic har!er to "!ersta!F, %"t it works i this script.
Summing -p
9 this chapter, we took o"r +irst steps towar! iteracti#ityH allowig "sers to ip"t !ata
ito o"r programs #ia the key%oar!. )sig the techi="es presete! th"s +ar, it is
possi%le to write may "se+"l programs, s"ch as specialiDe! calc"latio programs a!
easy-to-"se +rot e!s +or arcae comma! lie tools. 9 the e3t chapter, we will %"il!
o the me"-!ri#e program cocept to make it e#e %etter.
6xtra Credit
9t is importat to st"!y the programs i this chapter care+"lly a! ha#e a complete
"!ersta!ig o+ the way they are logically str"ct"re!, as the programs to come will %e
icreasigly comple3. $s a e3ercise, rewrite the programs i this chapter "sig the
test comma! rather tha the [[ ]] compo"! comma!. Ait, "se gre7 to
e#al"ate the reg"lar e3pressios a! e#al"ate its e3it stat"s. This will %e goo! practice.
;urther 1eading
The 8ash 9eference !anual cotais a chapter o %"iltis, which icl"!es the
read comma!,
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.htmlPJash-J"iltis
1..
(0 8 0low Cotrol, *oopig With while - "til
30 (lo5 Control; <oo#ing With 5hile = until
9 the pre#io"s chapter, we !e#elope! a me"-!ri#e program to pro!"ce #ario"s ki!s
o+ system i+ormatio. The program works, %"t it still has a sigi+icat "sa%ility
pro%lem. 9t oly e3ec"tes a sigle choice a! the termiates. E#e worse, i+ a i#ali!
selectio is ma!e, the program termiates with a error, witho"t gi#ig the "ser a
opport"ity to try agai. 9t wo"l! %e %etter i+ we co"l! somehow costr"ct the program
so that it co"l! repeat the me" !isplay a! selectio o#er a! o#er, "til the "ser
chooses to e3it the program.
9 this chapter, we will look at a programmig cocept calle! looping, which ca %e "se!
to make portios o+ programs repeat. The shell pro#i!es three compo"! comma!s +or
loopig. We will look at two o+ them i this chapter, a! the thir! i a later oe.
Looping
'aily li+e is +"ll o+ repeate! acti#ities. ;oig to work each !ay, walkig the !og, slicig
a carrot are all tasks that i#ol#e repeatig a series o+ steps. *etIs cosi!er slicig a
carrot. 9+ we e3press this acti#ity i pse"!oco!e, it might look somethig like this,
.. get c"ttig %oar!
2. get ki+e
(. place carrot o c"ttig %oar!
1. li+t ki+e
2. a!#ace carrot
K. slice carrot
/. i+ etire carrot slice!, the ="it, else go to step 1
Steps 1 thro"gh / +orm a loop. The actios withi the loop are repeate! "til the
co!itio, ?etire carrot slice!,@ is reache!.
9hile
bash ca e3press a similar i!ea. *etIs say we wate! to !isplay +i#e "m%ers i
1.2
*oopig
se="etial or!er +rom oe to +i#e. a bash script co"l! %e costr"cte! as +ollows,
^G6bin6bash
^ 9hile-count: dis7la1 a series of numbers
countM#
9hile [ $count -le " ]T do
echo $count
countM$NNcount 8 #PP
done
echo _-inished<_
Whe e3ec"te!, this script !isplays the +ollowig,
[me@linuxbox ~]$ while-count
#
!
$
%
"
-inished<
The syta3 o+ the 9hile comma! is,
9hile commandsT do commandsT done
*ike if, 9hile e#al"ates the e3it stat"s o+ a list o+ comma!s. $s log as the e3it stat"s
is Dero, it per+orms the comma!s isi!e the loop. 9 the script a%o#e, the #aria%le
count is create! a! assige! a iitial #al"e o+ .. The 9hile comma! e#al"ates the
e3it stat"s o+ the test comma!. $s log as the test comma! ret"rs a e3it stat"s
o+ Dero, the comma!s withi the loop are e3ec"te!. $t the e! o+ each cycle, the test
comma! is repeate!. $+ter si3 iteratios o+ the loop, the #al"e o+ count has icrease!
to si3, the test comma! o loger ret"rs a e3it stat"s o+ Dero a! the loop
termiates. The program coti"es with the e3t statemet +ollowig the loop.
We ca "se a while loop to impro#e the read-menu program +rom the pre#io"s chapter,
^G6bin6bash
^ 9hile-menu: a menu driven s1stem information 7rogram
1.(
(0 8 0low Cotrol, *oopig With while - "til
1GT:^$ ^ Lumber of seconds to dis7la1 results
while (( 3HGBT^ L$ * ))# do
clear
cat XX- O&-O
:lease *elect:
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
O&-O
read -7 _&nter selection [(-$] D _
if [[ $R&:=U M~ `[(-$]$ ]]T then
if [[ $R&:=U MM # ]]T then
echo _Sostname: $S*TL4+&_
u7time
sleep 31GT:^
fi
if [[ $R&:=U MM ! ]]T then
df -h
sleep 31GT:^
fi
if [[ $R&:=U MM $ ]]T then
if [[ $Nid -uP -eB ( ]]T then
echo _Some *7ace 3tiliCation N4ll 3sersP_
du -sh 6home6E
else
echo _Some *7ace 3tiliCation N$3*&RP_
du -sh $S+&
fi
sleep 31GT:^
fi
else
echo _Anvalid entr1<_
sleep 31GT:^
fi
done
echo _:rogram terminated<_
Jy eclosig the me" i a while loop, we are a%le to ha#e the program repeat the me"
!isplay a+ter each selectio. The loop coti"es as log as R&:=U is ot e="al to ?0@ a!
the me" is !isplaye! agai, gi#ig the "ser the opport"ity to make aother selectio.
$t the e! o+ each actio, a slee7 comma! is e3ec"te! so the program will pa"se +or a
+ew seco!s to allow the res"lts o+ the selectio to %e see %e+ore the scree is cleare!
a! the me" is re!isplaye!. 7ce R&:=U is e="al to ?0,@ i!icatig the ?="it@ selectio,
1.1
*oopig
the loop termiates a! e3ec"tio coti"es with the lie +ollowig done.
/rea(ing ?ut ?" # Loop
bash pro#i!es two %"ilti comma!s that ca %e "se! to cotrol program +low isi!e
loops. The break comma! imme!iately termiates a loop, a! program cotrol
res"mes with the e3t statemet +ollowig the loop. The continue comma! ca"ses
the remai!er to the loop to %e skippe!, a! program cotrol res"mes with the e3t
iteratio o+ the loop. Aere we see a #ersio o+ the 9hile-menu program icorporatig
%oth break a! continue,
^G6bin6bash
^ 9hile-menu!: a menu driven s1stem information 7rogram
'&=4UM$ ^ Lumber of seconds to dis7la1 results
while true# do
clear
cat XX- O&-O
:lease *elect:
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
O&-O
read -7 _&nter selection [(-$] D _
if [[ $R&:=U M~ `[(-$]$ ]]T then
if [[ $R&:=U MM # ]]T then
echo _Sostname: $S*TL4+&_
u7time
slee7 $'&=4U
continue
fi
if [[ $R&:=U MM ! ]]T then
df -h
slee7 $'&=4U
continue
fi
if [[ $R&:=U MM $ ]]T then
if [[ $Nid -uP -eB ( ]]T then
echo _Some *7ace 3tiliCation N4ll 3sersP_
du -sh 6home6E
else
1.2
(0 8 0low Cotrol, *oopig With while - "til
echo _Some *7ace 3tiliCation N$3*&RP_
du -sh $S+&
fi
slee7 $'&=4U
continue
fi
if (( 3HGBT^ $$ * ))# then
break
fi
else
echo _Anvalid entr1<_
slee7 $'&=4U
fi
done
echo _:rogram terminated<_
9 this #ersio o+ the script, we set "p a endless loop Eoe that e#er termiates o its
owF %y "sig the true comma! to s"pply a e3it stat"s to 9hile. Sice true will
always e3it with a e3it stat"s o+ Dero, the loop will e#er e!. This is a s"rprisigly
commo scriptig techi="e. Sice the loop will e#er e! o its ow, itIs "p to the
programmer to pro#i!e some way to %reak o"t o+ the loop whe the time is right. 9 this
script, the break comma! is "se! to e3it the loop whe the ?0@ selectio is chose.
The continue comma! has %ee icl"!e! at the e! o+ the other script choices to
allow +or more e++iciet e3ec"tio. Jy "sig continue, the script will skip o#er co!e
that is ot ee!e! whe a selectio is i!eti+ie!. 0or e3ample, i+ the ?.@ selectio is
chose a! i!eti+ie!, there is o reaso to test +or the other selectios.
until
The until comma! is m"ch like 9hile, e3cept istea! o+ e3itig a loop whe a o-
Dero e3it stat"s is eco"tere!, it !oes the opposite. $ until loop coti"es "til it
recei#es a Dero e3it stat"s. 9 o"r 9hile-count script, we coti"e! the loop as log
as the #al"e o+ the count #aria%le was less tha or e="al to +i#e. We co"l! get the same
res"lt %y co!ig the script with until,
^G6bin6bash
^ until-count: dis7la1 a series of numbers
countM#
until ( 3count -gt " )T do
echo $count
1.K
Jreakig 7"t 7+ $ *oop
countM$NNcount 8 #PP
done
echo _-inished<_
Jy chagig the test e3pressio to $count -gt ", until will termiate the loop at
the correct time. The !ecisio o+ whether to "se the 9hile or until loop is "s"ally a
matter o+ choosig the oe that allows the clearest test to %e writte.
1eading ;iles +ith Loops
9hile a! until ca process sta!ar! ip"t. This allows +iles to %e processe! with
while a! "til loops. 9 the +ollowig e3ample, we will !isplay the cotets o+ the
distros<txt +ile "se! i earlier chapters,
^G6bin6bash
^ 9hile-read: read lines from a file
9hile read distro version releaseT do
7rintf _'istro: 5s]t;ersion: 5s]tReleased: 5s]n_ ]
$distro ]
$version ]
$release
done / distros.txt
To re!irect a +ile to the loop, we place the re!irectio operator a+ter the done statemet.
The loop will "se read to ip"t the +iel!s +rom the re!irecte! +ile. The read comma!
will e3it a+ter each lie is rea!, with a Dero e3it stat"s "til the e!-o+-+ile is reache!. $t
that poit, it will e3it with a o-Dero e3it stat"s, there%y termiatig the loop. 9t is also
possi%le to pipe sta!ar! ip"t ito a loop,
^G6bin6bash
^ 9hile-read!: read lines from a file
sort -k 1;1 -k 2n distros.txt 0 9hile read distro version releaseT do
7rintf _'istro: 5s]t;ersion: 5s]tReleased: 5s]n_ ]
$distro ]
$version ]
$release
done
1./
(0 8 0low Cotrol, *oopig With while - "til
Aere we take the o"tp"t o+ the sort comma! a! !isplay the stream o+ te3t. Aowe#er,
it is importat to remem%er that sice a pipe will e3ec"te the loop i a s"%shell, ay
#aria%les create! or assige! withi the loop will %e lost whe the loop termiates.
Summing -p
With the itro!"ctio o+ loops, a! o"r pre#io"s eco"ters with %rachig, s"%ro"ties
a! se="eces, we ha#e co#ere! the ma4or types o+ +low cotrol "se! i programs. bash
has some more tricks "p its slee#e, %"t they are re+iemets o these %asic cocepts.
;urther 1eading
The 8ash @uide for 8eginners +rom the *i"3 'oc"metatio 6ro4ect has some
more e3amples o+ while loops,
http,--tl!p.org-*'6-Jash-Jegiers-;"i!e-html-sectO09O02.html
The Wikipe!ia has a article o loops, which is part o+ a larger article o +low
cotrol,
http,--e.wikipe!ia.org-wiki-CotrolO+lowP*oops

1.8
(. 8 Tro"%leshootig
31 Troubleshooting
$s o"r scripts %ecome more comple3, itIs time to take a look at what happes whe
thigs go wrog a! they !oIt !o what what we wat. 9 this chapter, weIll look at
some o+ the commo ki!s o+ errors that occ"r i scripts, a! !escri%e a +ew "se+"l
techi="es that ca %e "se! to track !ow a! era!icate pro%lems.
S,ntactic $rrors
7e geeral class o+ errors is syntactic. Sytactic errors i#ol#e mis-typig some
elemet o+ shell syta3. 9 most cases, these ki!s o+ errors will lea! to the shell re+"sig
to e3ec"te the script.
9 the +ollowig the !isc"ssios, we will "se this script to !emostrate commo types o+
errors,
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
numberM#
if [ $number M # ]T then
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
$s writte, this script r"s s"ccess+"lly,
[me@linuxbox ~]$ trouble
Lumber is eBual to #<
1.9
(. 8 Tro"%leshootig
:issing Duotes
9+ we e!it o"r script a! remo#e the trailig ="ote +rom the arg"met +ollowig the +irst
echo comma!,
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
numberM#
if [ $number M # ]T then
echo J?umber is e,ual to 1.
else
echo _Lumber is not eBual to #<_
fi
watch what happes,
[me@linuxbox ~]$ trouble
6home6me6bin6trouble: line #(: unex7ected &- 9hile looking for
matching J_K
6home6me6bin6trouble: line #$: s1ntax error: unex7ected end of file
9t geerates two errors. 9terestigly, the lie "m%ers reporte! are ot where the missig
="ote was remo#e!, %"t rather m"ch later i the program. We ca see why, i+ we +ollow
the program a+ter the missig ="ote. bash will coti"e lookig +or the closig ="ote
"til it +i!s oe, which it !oes imme!iately a+ter the seco! echo comma!. bash
%ecomes #ery co+"se! a+ter that, a! the syta3 o+ the if comma! is %roke %eca"se
the fi statemet is ow isi!e a ="ote! E%"t opeF strig.
9 log scripts, this ki! o+ error ca %e ="ite har! to +i!. )sig a e!itor with syta3
highlightig will help. 9+ a complete #ersio o+ vim is istalle!, syta3 highlightig ca
%e ea%le! %y eterig the comma!,
2s!ntax on
:issing <r Unex5e/ted Tokens
$other commo mistake is +orgettig to complete a compo"! comma!, s"ch as if or
120
Sytactic Errors
9hile. *etIs look at what happes i+ we remo#e the semicolo a+ter the test i the if
comma!,
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
numberM#
if ( 3number $ 1 ) then
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
The res"lt is this,
[me@linuxbox ~]$ trouble
6home6me6bin6trouble: line 0: s1ntax error near unex7ected token
JelseK
6home6me6bin6trouble: line 0: JelseK
$gai, the error message poits to a error that occ"rs later tha the act"al pro%lem. What
happes is really pretty iterestig. $s we recall, if accepts a list o+ comma!s a!
e#al"ates the e3it co!e o+ the last comma! i the list. 9 o"r program, we ite! this list
to cosist o+ a sigle comma!, [, a syoym +or test. The [ comma! takes what
+ollows it as a list o+ arg"mets. 9 o"r case, three arg"mets, $number, M, a! ]. With
the semicolo remo#e!, the wor! then is a!!e! to the list o+ arg"mets, which is
sytactically legal. The +ollowig echo comma! is legal, too. 9tIs iterprete! as
aother comma! i the list o+ comma!s that if will e#al"ate +or a e3it co!e. The
else is eco"tere! e3t, %"t itIs o"t o+ place, sice the shell recogiDes it as a reserved
word Ea wor! that has special meaig to the shellF a! ot the ame o+ a comma!,
hece the error message.
Unanti/i5ated 6x5ansions
9tIs possi%le to ha#e errors that oly occ"r itermittetly i a script. Sometimes the script
will r" +ie a! other times it will +ail %eca"se o+ res"lts o+ a e3pasio. 9+ we ret"r
o"r missig semicolo a! chage the #al"e o+ number to a empty #aria%le, we ca
!emostrate,
12.
(. 8 Tro"%leshootig
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
number$
if [ $number M # ]T then
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
:"ig the script with this chage res"lts i the o"tp"t,
[me@linuxbox ~]$ trouble
6home6me6bin6trouble: line ): [: M: unar1 o7erator ex7ected
Lumber is not eBual to #<
We get this rather cryptic error message, +ollowe! %y the o"tp"t o+ the seco! echo
comma!. The pro%lem is the e3pasio o+ the number #aria%le withi the test
comma!. Whe the comma!,
[ $number M # ]
"!ergoes e3pasio with number %eig empty, the res"lt is this,
[ M # ]
which is i#ali! a! the error is geerate!. The M operator is a %iary operator Eit
re="ires a #al"e o each si!eF, %"t the +irst #al"e is missig, so the test comma!
e3pects a "ary operator Es"ch as -CF istea!. 0"rther, sice the test +aile! E%eca"se o+
the errorF, the if comma! recei#es a o-Dero e3it co!e a! acts accor!igly, a! the
seco! echo comma! is e3ec"te!.
This pro%lem ca %e correcte! %y a!!ig ="otes aro"! the +irst arg"met i the test
comma!,
[ _$number_ M # ]
122
Sytactic Errors
The whe e3pasio occ"rs, the res"lt will %e this,
[ __ M # ]
which yiel!s the correct "m%er o+ arg"mets. 9 a!!itio to empty strigs, ="otes
sho"l! %e "se! i cases where a #al"e co"l! e3pa! ito m"lti-wor! strigs, as with
+ileames cotaiig em%e!!e! spaces.
Logical $rrors
)like sytactic errors, logical errors !o ot pre#et a script +rom r"ig. The script
will r", %"t it will ot pro!"ce the !esire! res"lt, !"e to a pro%lem with its logic. There
are co"tless "m%ers o+ possi%le logical errors, %"t here are a +ew o+ the most commo
ki!s +o"! i scripts,
.. )ncorrect conditional ex(ressions. 9tIs easy to icorrectly co!e a i+-the-else
a! ha#e the wrog logic carrie! o"t. Sometimes the logic will %e re#erse! or it
will %e icomplete.
2. .;"" by one0 errors. Whe co!ig loops that employ co"ters, it is possi%le to
o#erlook that the loop may re="ire the co"tig start with Dero, rather tha oe,
+or the co"t to cocl"!e at the correct poit. These ki!s o+ errors res"lt i either
a loop ?goig o++ the e!@ %y co"tig too +ar, or else missig the last iteratio o+
the loop %y termiatig oe iteratio too soo.
(. -nantici(ated situations. >ost logic errors res"lt +rom a program eco"terig
!ata or sit"atios that were "+oresee %y the programmer. This ca also icl"!e
"aticipate! e3pasios, s"ch as a +ileame that cotais em%e!!e! spaces that
e3pa!s ito m"ltiple comma! arg"mets rather tha a sigle +ileame.
)e=ensi;e 'rogramming
9t is importat to #eri+y ass"mptios whe programmig. This meas a care+"l
e#al"atio o+ the e3it stat"s o+ programs a! comma!s that are "se! %y a script. Aere is
a e3ample, %ase! o a tr"e story. $ "+ort"ate system a!miistrator wrote a script to
per+orm a maiteace task o a importat ser#er. The script cotaie! the +ollowig
two lies o+ co!e,
cd $dirOname
rm E
12(
(. 8 Tro"%leshootig
There is othig itrisically wrog with these two lies, as log as the !irectory ame!
i the #aria%le, dirOname, e3ists. J"t what happes i+ it !oes otC 9 that case, the cd
comma! +ails, the script coti"es to the e3t lie a! !eletes the +iles i the c"rret
workig !irectory. &ot the !esire! o"tcome at allL The hapless a!miistrator !estroye!
a importat part o+ the ser#er %eca"se o+ this !esig !ecisio.
*etIs look at some ways this !esig co"l! %e impro#e!. 0irst, it might %e wise to make
the e3ec"tio o+ rm cotiget o the s"ccess o+ c!,
cd $dirOname YY rm E
This way, i+ the cd comma! +ails, the rm comma! is ot carrie! o"t. This is %etter, %"t
still lea#es ope the possi%ility that the #aria%le, dirOname, is "set or empty, which
wo"l! res"lt i the +iles i the "serIs home !irectory %eig !elete!. This co"l! also %e
a#oi!e! %y checkig to see that dirOname act"ally cotais the ame o+ a e3istig
!irectory,
[[ -d $dirOname ]] YY cd $dirOname YY rm E
7+te, it is %est to termiate the script with a error whe a sit"atio s"ch as the oe
a%o#e occ"rs,
if [[ -d $dirOname ]]T then
if cd $dirOnameT then
rm E
else
echo _cannot cd to K$dirOnameK_ DY!
exit #
fi
else
echo _no such director1: K$dirOnameK_ DY!
exit #
fi
Aere, we check %oth the ame, to see that it is that o+ a e3istig !irectory, a! the
s"ccess o+ the cd comma!. 9+ either +ails, a !escripti#e error message is set to sta!ar!
error a! the script termiates with a e3it stat"s o+ oe to i!icate a +ail"re.
121
*ogical Errors
?eri=ying In5ut
$ geeral r"le o+ goo! programmig is that i+ a program accepts ip"t, it m"st %e a%le to
!eal with aythig it recei#es. This "s"ally meas that ip"t m"st %e care+"lly screee!,
to es"re that oly #ali! ip"t is accepte! +or +"rther processig. We saw a e3ample o+
this i the pre#io"s chapter whe we st"!ie! the read comma!. 7e script cotaie!
the +ollowig test to #eri+y a me" selectio,
[[ $R&:=U M~ `[(-$]$ ]]
This test is #ery speci+ic. 9t will oly ret"r a Dero e3it stat"s i+ the strig ret"re! %y the
"ser is a "meral i the rage o+ Dero to three. &othig else will %e accepte!. Sometimes
these sorts o+ tests ca %e #ery challegig to write, %"t the e++ort is ecessary to pro!"ce
a high ="ality script.
Design Is # ;unction ?" Time
Whe 9 was a college st"!et st"!yig i!"strial !esig, a wise pro+essor state!
that the !egree o+ !esig o a pro4ect was !etermie! %y the amo"t o+ time gi#e
to the !esiger. 9+ yo" were gi#e +i#e mi"tes to !esig a !e#ice ?that kills
+lies,@ yo" !esige! a +lyswatter. 9+ yo" were gi#e +i#e moths, yo" might come
"p with a laser-g"i!e! ?ati-+ly system@ istea!.
The same priciple applies to programmig. Sometimes a ?="ick a! !irty@
script will !o i+ itIs oly goig to %e "se! oce a! oly "se! %y the programmer.
That ki! o+ script is commo a! sho"l! %e !e#elope! ="ickly to make the e++ort
ecoomical. S"ch scripts !oIt ee! a lot o+ commets a! !e+esi#e checks. 7
the other ha!, i+ a script is ite!e! +or production use" that is, a script that will
%e "se! o#er a! o#er +or a importat task or %y m"ltiple "sers, it ee!s m"ch
more care+"l !e#elopmet.
Testing
Testig is a importat step i e#ery ki! o+ so+tware !e#elopmet, icl"!ig scripts.
There is a sayig i the ope so"rce worl!, ?release early, release o+te,@ which re+lects
this +act. Jy releasig early a! o+te, so+tware gets more e3pos"re to "se a! testig.
E3periece has show that %"gs are m"ch easier to +i!, a! m"ch less e3pesi#e to +i3,
i+ they are +o"! early i the !e#elopmet cycle.
9 a pre#io"s !isc"ssio, we saw how st"%s ca %e "se! to #eri+y program +low. 0rom
122
(. 8 Tro"%leshootig
the earliest stages o+ script !e#elopmet, they are a #al"a%le techi="e to check the
progress o+ o"r work.
*etIs look at the +ile !eletio pro%lem a%o#e a! see how this co"l! %e co!e! +or easy
testig. Testig the origial +ragmet o+ co!e wo"l! %e !agero"s, sice its p"rpose is to
!elete +iles, %"t we co"l! mo!i+y the co!e to make the test sa+e,
if [[ -d $dirOname ]]T then
if cd $dirOnameT then
echo rm E U +GF+P?Q
else
echo _cannot cd to K$dirOnameK_ DY!
exit #
fi
else
echo _no such director1: K$dirOnameK_ DY!
exit #
fi
exit U +GF+P?Q
Sice the error co!itios alrea!y o"tp"t "se+"l messages, we !o<t ha#e to a!! ay. The
most importat chage is placig a echo comma! 4"st %e+ore the rm comma! to
allow the comma! a! its e3pa!e! arg"met list to %e !isplaye!, rather tha the
comma! act"ally %eig e3ec"te!. This chage allows sa+e e3ec"tio o+ the co!e. $t the
e! o+ the co!e +ragmet, we place a exit comma! to cocl"!e the test a! pre#et
ay other part o+ the script +rom %eig carrie! o"t. The ee! +or this will #ary accor!ig
to the !esig o+ the script.
We also icl"!e some commets that act as ?markers@ +or o"r test-relate! chages. These
ca %e "se! to help +i! a! remo#e the chages whe testig is complete.
Test Cases
To per+orm "se+"l testig, it<s importat to !e#elop a! apply goo! test cases. This is
!oe %y care+"lly choosig ip"t !ata or operatig co!itios that re+lect edge a!
corner cases. 9 o"r co!e +ragmet Ewhich is #ery simpleF, we wat to kow how the
co!e per+orms "!er three speci+ic co!itios,
.. dirOname cotais the ame o+ a e3istig !irectory
2. dirOname cotais the ame o+ a o-e3istet !irectory
(. dirOname is empty
Jy per+ormig the test with each o+ these co!itios, goo! test coverage is achie#e!.
12K
Testig
J"st as with !esig, testig is a +"ctio o+ time, as well. &ot e#ery script +eat"re ee!s
to %e e3tesi#ely teste!. 9t<s really a matter o+ !etermiig what is most importat. Sice
it co"l! %e so potetially !estr"cti#e i+ it mal+"ctioe!, o"r co!e +ragmet !eser#es
care+"l cosi!eratio !"rig %oth its !esig a! testig.
De0ugging
9+ testig re#eals a pro%lem with a script, the e3t step is !e%"ggig. ?$ pro%lem@
"s"ally meas that the script is, i some way, ot per+ormig to the programmers
e3pectatios. 9+ this is the case, we ee! to care+"lly !etermie e3actly what the script is
act"ally !oig a! why. 0i!ig %"gs ca sometimes i#ol#e a lot o+ !etecti#e work.
$ well !esige! script will try to help. 9t sho"l! %e programme! !e+esi#ely, to !etect
a%ormal co!itios a! pro#i!e "se+"l +ee!%ack to the "ser. Sometimes, howe#er,
pro%lems are ="ite strage a! "e3pecte! a! more i#ol#e! techi="es are re="ire!.
2inding The 'robem Area
9 some scripts, partic"larly log oes, it is sometimes "se+"l to isolate the area o+ the
script that is relate! to the pro%lem. This woIt always %e the act"al error, %"t isolatio
will o+te pro#i!e isights ito the act"al ca"se. 7e techi="e that ca %e "se! to
isolate co!e is ?commetig o"t@ sectios a script. 0or e3ample, o"r +ile !eletio
+ragmet co"l! %e mo!i+ie! to !etermie i+ the remo#e! sectio was relate! to a error,
if [[ -d $dirOname ]]T then
if cd $dirOnameT then
rm E
else
echo _cannot cd to K$dirOnameK_ DY!
exit #
fi
U else
U echo Jno such director!2 %3dir.name%J &'2
U exit 1
fi
Jy placig commet sym%ols at the %egiig o+ each lie i a logical sectio o+ a script,
we pre#et that sectio +rom %eig e3ec"te!. Testig ca the %e per+orme! agai, to see
i+ the remo#al o+ the co!e has ay impact o the %eha#ior o+ the %"g.
Tra/ing
J"gs are o+te cases o+ "e3pecte! logical +low withi a script. That is, portios o+ the
12/
(. 8 Tro"%leshootig
script are either e#er %eig e3ec"te!, or are %eig e3ec"te! i the wrog or!er or at the
wrog time. To #iew the act"al +low o+ the program, we "se a techi="e calle! tracing.
7e tracig metho! i#ol#es placig i+ormati#e messages i a script that !isplay the
locatio o+ e3ec"tio. We ca a!! messages to o"r co!e +ragmet,
echo Jpreparing to delete filesJ &'2
if [[ -d $dirOname ]]T then
if cd $dirOnameT then
echo Jdeleting filesJ &'2
rm E
else
echo _cannot cd to K$dirOnameK_ DY!
exit #
fi
else
echo _no such director1: K$dirOnameK_ DY!
exit #
fi
echo Jfile deletion completeJ &'2
We se! the messages to sta!ar! error to separate them +rom ormal o"tp"t. We also !o
ot i!et the lies cotaiig the messages, so it is easier to +i! whe itIs time to
remo#e them.
&ow whe the script is e3ec"te!, itIs possi%le to see that the +ile !eletio has %ee
per+orme!,
[me@linuxbox ~]$ deletion-scri7t
7re7aring to delete files
deleting files
file deletion com7lete
[me@linuxbox ~]$
bash also pro#i!es a metho! o+ tracig, implemete! %y the -x optio a! the set
comma! with the -x optio. )sig o"r earlier trouble script, we ca acti#ate
tracig +or the etire script %y a!!ig the -x optio to the +irst lie,
UL/bin/bash -x
^ trouble: scri7t to demonstrate common errors
numberM#
128
'e%"ggig
if [ $number M # ]T then
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
Whe e3ec"te!, the res"lts look like this,
[me@linuxbox ~]$ trouble
8 numberM#
8 K[K # M # K]K
8 echo KLumber is eBual to #<K
Lumber is eBual to #<
With tracig ea%le!, we see the comma!s per+orme! with e3pasios applie!. The
lea!ig pl"s sigs i!icate the !isplay o+ the trace to !istig"ish them +rom lies o+
reg"lar o"tp"t. The pl"s sig is the !e+a"lt character +or trace o"tp"t. 9t is cotaie! i
the :*% Eprompt strig 1F shell #aria%le. The cotets o+ this #aria%le ca %e a!4"ste! to
make the prompt more "se+"l. Aere, we mo!i+y the cotets o+ the #aria%le to icl"!e
the c"rret lie "m%er i the script where the trace is per+orme!. &ote that sigle ="otes
are re="ire! to pre#et e3pasio "til the prompt is act"ally "se!,
[me@linuxbox ~]$ export BFA$%3TP?G?S 5 %
[me@linuxbox ~]$ trouble
" 8 numberM#
) 8 K[K # M # K]K
/ 8 echo KLumber is eBual to #<K
Lumber is eBual to #<
To per+orm a trace o a selecte! portio o+ a script, rather tha the etire script, we ca
"se the set comma! with the -x optio,
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
numberM#
set -x U +urn on tracing
if [ $number M # ]T then
129
(. 8 Tro"%leshootig
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
set 5x U +urn off tracing
We "se the set comma! with the -x optio to acti#ate tracig a! the 8x optio to
!eacti#ate tracig. This techi="e ca %e "se! to e3amie m"ltiple portios o+ a
tro"%lesome script.
6xamining ?aues )uring 6xe/ution
9t is o+te "se+"l, alog with tracig, to !isplay the cotet o+ #aria%les to see the iteral
workigs o+ a script while it is %eig e3ec"te!. $pplyig a!!itioal echo statemets
will "s"ally !o the trick,
^G6bin6bash
^ trouble: scri7t to demonstrate common errors
numberM#
echo Jnumber$3numberJ U 1G<EQ
set -x ^ Turn on tracing
if [ $number M # ]T then
echo _Lumber is eBual to #<_
else
echo _Lumber is not eBual to #<_
fi
set 8x ^ Turn off tracing
9 this tri#ial e3ample, we simply !isplay the #al"e o+ the #aria%le "m%er a! mark the
a!!e! lie with a commet to +acilitate its later i!eti+icatio a! remo#al. This
techi="e is partic"larly "se+"l whe watchig the %eha#ior o+ loops a! arithmetic
withi scripts.
Summing -p
9 this chapter, we looke! at 4"st a +ew o+ the pro%lems that ca crop "p !"rig script
!e#elopmet. 7+ co"rse, there are may more. The techi="es !escri%e! here will
ea%le +i!ig most commo %"gs. 'e%"ggig is a +ie art that ca %e !e#elope!
thro"gh e3periece, %oth i kowig how to a#oi! %"gs Etestig costatly thro"gho"t
!e#elopmetF a! i +i!ig %"gs Ee++ecti#e "se o+ tracigF.
1(0
0"rther :ea!ig
;urther 1eading
The Wikipe!ia has a co"ple o+ short articles o sytactic a! logical errors,
http,--e.wikipe!ia.org-wiki-Syta3Oerror
http,--e.wikipe!ia.org-wiki-*ogicOerror
There are may olie reso"rces +or the techical aspects o+ bash programmig,
http,--woole!ge.org,8000-Jash6it+alls
http,--tl!p.org-*'6-a%s-html-gotchas.html
http,--www.g".org-so+tware-%ash-ma"al-htmlOo!e-:eser#e!-Wor!-9!e3.html
Eric :aymo!Is The )rt of 0ni+ 6rogramming is a great reso"rce +or learig the
%asic cocepts +o"! i well-writte )i3 programs. >ay o+ these i!eas apply
to shell scripts,
http,--www.+a=s.org-!ocs-art"-
http,--www.+a=s.org-!ocs-art"-ch0.s0K.html
0or really hea#y-!"ty !e%"ggig, there is the Jash 'e%"gger,
http,--%ash!%.so"rce+orge.et-
1(.
(2 8 0low Cotrol, Jrachig With case
32 (lo5 Control; 6ranching With case
9 this chapter, we will coti"e to look at +low cotrol. 9 Chapter 29, we costr"cte!
some simple me"s a! %"ilt the logic "se! to act o a "serIs selectio. To !o this, we
"se! a series o+ if comma!s to i!eti+y which o+ the possi%le choices has %ee
selecte!. This type o+ costr"ct appears +re="etly i programs, so m"ch so that may
programmig lag"ages Eicl"!ig the shellF pro#i!e a +low cotrol mechaism +or
m"ltiple-choice !ecisios.
case
The bash m"ltiple-choice compo"! comma! is calle! case. 9t has the +ollowig
syta3,
case word in
[pattern [Q pattern]<<<P commands TT]<<<
esac
9+ we look at the read-menu program +rom Chapter 29, we see the logic "se! to act o
a "serIs selectio,
^G6bin6bash
^ read-menu: a menu driven s1stem information 7rogram
clear
echo _
:lease *elect:
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
_
read -7 _&nter selection [(-$] D _
if (( 3HGBT^ $~ [(*-)3 ))# then
if (( 3HGBT^ $$ * ))# then
1(2
case
echo JBrogram terminated.J
exit
fi
if (( 3HGBT^ $$ 1 ))# then
echo JRostname2 3RSF+?:OGJ
uptime
exit
fi
if (( 3HGBT^ $$ 2 ))# then
df -h
exit
fi
if (( 3HGBT^ $$ ))# then
if (( 34id -u6 -e, * ))# then
echo JRome Fpace Etili-ation 4:ll Esers6J
du -sh /home/*
else
echo JRome Fpace Etili-ation 43EFGH6J
du -sh 3RSOG
fi
exit
fi
else
echo JPnvalid entr!.J &'2
exit 1
fi
)sig case, we ca replace this logic with somethig simpler,
^G6bin6bash
^ case-menu: a menu driven s1stem information 7rogram
clear
echo _
:lease *elect:
#< 'is7la1 *1stem Anformation
!< 'is7la1 'isk *7ace
$< 'is7la1 Some *7ace 3tiliCation
(< Zuit
_
read -7 _&nter selection [(-$] D _
case 3HGBT^ in
*6 echo JBrogram terminated.J
exit
##
1((
(2 8 0low Cotrol, Jrachig With case
16 echo JRostname2 3RSF+?:OGJ
uptime
##
26 df -h
##
6 if (( 34id -u6 -e, * ))# then
echo JRome Fpace Etili-ation 4:ll Esers6J
du -sh /home/*
else
echo JRome Fpace Etili-ation 43EFGH6J
du -sh 3RSOG
fi
##
*6 echo JPnvalid entr!J &'2
exit 1
##
esac
The case comma! looks at the #al"e o+ word, i o"r e3ample, the #al"e o+ the R&:=U
#aria%le, a! the attempts to match it agaist oe o+ the speci+ie! patterns. Whe a
match is +o"!, the commands associate! with the speci+ie! patter are e3ec"te!. $+ter a
match is +o"!, o +"rther matches are attempte!.
'atterns
The patters "se! %y case are the same as those "se! %y pathame e3pasio. 6atters
are termiate! with a ?F@ character. Aere are some #ali! patters,
Table3'- 1: case 6attern 1+amples
Pattern Description
aP
>atches i+ word e="als ?a@.
[[:al7ha:]]P
>atches i+ word is a sigle alpha%etic character.
FFFP
>atches i+ word is e3actly three characters log.
E<txtP
>atches i+ word e!s with the characters ?.t3t@.
EP
>atches ay #al"e o+ word. 9t is goo! practice to icl"!e this
as the last patter i a case comma!, to catch ay #al"es o+
word that !i! ot match a pre#io"s patterH that is, to catch ay
possi%le i#ali! #al"es.

Aere is a e3ample o+ patters at work,
1(1
case
^G6bin6bash
read -7 _enter 9ord D _
case $R&:=U in
[[:al7ha:]]P echo _is a single al7habetic character<_ TT
[4HC][(-0]P echo _is 4> H> or C follo9ed b1 a digit<_ TT
FFFP echo _is three characters long<_ TT
E<txtP echo _is a 9ord ending in K<txtK_ TT
EP echo _is something else<_ TT
esac
9t is also possi%le to com%ie m"ltiple patters "sig the #ertical %ar character as a
separator. This creates a ?or@ co!itioal patter. This is "se+"l +or s"ch thigs as
ha!lig %oth "pper a! lower case characters. 0or e3ample,
^G6bin6bash
^ case-menu: a menu driven s1stem information 7rogram
clear
echo _
:lease *elect:
:. 'is7la1 *1stem Anformation
<. 'is7la1 'isk *7ace
=. 'is7la1 Some *7ace 3tiliCation
\. Zuit
_
read -7 _&nter selection (:; <; = or \) D _
case $R&:=U in
,0\6 echo _:rogram terminated<_
exit
TT
a0:6 echo _Sostname: $S*TL4+&_
u7time
TT
b0<6 df -h
TT
c0=6 if [[ $Nid -uP -eB ( ]]T then
echo _Some *7ace 3tiliCation N4ll 3sersP_
du -sh 6home6E
else
echo _Some *7ace 3tiliCation N$3*&RP_
du -sh $S+&
fi
1(2
(2 8 0low Cotrol, Jrachig With case
TT
EP echo _Anvalid entr1_ DY!
exit #
TT
esac
Aere, we mo!i+y the case-menu program to "se letters istea! o+ !igits +or me"
selectio. &otice how the ew patters allow +or etry o+ %oth "pper a! lower case
letters.
Summing -p
The case comma! is a ha!y a!!itio to o"r %ag o+ programmig tricks. $s we will
see i the e3t chapter, itIs the per+ect tool +or ha!lig certai types o+ pro%lems.
;urther 1eading
The 8ash 9eference !anual sectio o Co!itioal Costr"cts !escri%es the
case comma! i !etail,
http,--tiswww.case.e!"-php-chet-%ash-%ashre+.htmlPSEC2.
The )dvanced 8ash-Scripting @uide pro#i!es +"rther e3amples o+ case
applicatios,
http,--tl!p.org-*'6-a%s-html-test%rach.html
1(K
(( 8 6ositioal 6arameters
33 2ositional 2ara%eters
7e +eat"re that has %ee missig +rom o"r programs is the a%ility to accept a! process
comma! lie optios a! arg"mets. 9 this chapter, we will e3amie the shell +eat"res
that allow o"r programs to get access to the cotets o+ the comma! lie.
#ccessing The Command Line
The shell pro#i!es a set o+ #aria%les calle! positional parameters that cotai the
i!i#i!"al wor!s o the comma! lie. The #aria%les are ame! ( thro"gh 0. They ca
%e !emostrate! this way,
^G6bin6bash
^ 7osit-7aram: scri7t to vie9 command line 7arameters
echo _
]$( M $(
]$# M $#
]$! M $!
]$$ M $$
]$% M $%
]$" M $"
]$. M $.
]$) M $)
]$/ M $/
]$0 M $0
_
$ #ery simple script that !isplays the #al"es o+ the #aria%les $(-$0. Whe e3ec"te!
with o comma! lie arg"mets,
[me@linuxbox ~]$ posit-param
$( M 6home6me6bin67osit-7aram
1(/
(( 8 6ositioal 6arameters
$# M
$! M
$$ M
$% M
$" M
$. M
$) M
$/ M
$0 M
E#e whe o arg"mets are pro#i!e!, $( will always cotai the +irst item appearig o
the comma! lie, which is the pathame o+ the program %eig e3ec"te!. Whe
arg"mets are pro#i!e!, we see the res"lts,
[me@linuxbox ~]$ posit-param a b c d
$( M 6home6me6bin67osit-7aram
$# M a
$! M b
$$ M c
$% M d
$" M
$. M
$) M
$/ M
$0 M
1ote5 5o" ca act"ally access more tha ie parameters "sig parameter
e3pasio. To speci+y a "m%er greater tha ie, s"rro"! the "m%er i %races.
0or e3ample [c.0d, [c22d, [c2..d a! so o.
)etermining The ,umber o= Arguments
The shell also pro#i!es a #aria%le, $^, that yiel!s the "m%er o+ arg"mets o the
comma! lie,
^G6bin6bash
^ 7osit-7aram: scri7t to vie9 command line 7arameters
echo _
1(8
$ccessig The Comma! *ie
?umber of arguments2 3U
]$( M $(
]$# M $#
]$! M $!
]$$ M $$
]$% M $%
]$" M $"
]$. M $.
]$) M $)
]$/ M $/
]$0 M $0
_
The res"lt,
[me@linuxbox ~]$ posit-param a b c d
Lumber of arguments: %
$( M 6home6me6bin67osit-7aram
$# M a
$! M b
$$ M c
$% M d
$" M
$. M
$) M
$/ M
$0 M
shift A +etting A//ess To :any Arguments
J"t what happes whe we gi#e the program a large "m%er o+ arg"mets s"ch as this,
[me@linuxbox ~]$ posit-param *
Lumber of arguments: /!
$( M 6home6me6bin67osit-7aram
$# M addresses<ldif
$! M bin
$$ M bookmarks<html
$% M debian-"((-i$/.-netinst<iso
$" M debian-"((-i$/.-netinst<jigdo
$. M debian-"((-i$/.-netinst<tem7late
$) M debian-cdOinfo<tar<gC
1(9
(( 8 6ositioal 6arameters
$/ M 'eskto7
$0 M dirlist-bin<txt
7 this e3ample system, the wil!car! E e3pa!s ito eighty-two arg"mets. Aow ca we
process that mayC The shell pro#i!es a metho!, al%eit cl"msy, to !o this. The shift
comma! ca"ses all o+ the parameters to ?mo#e !ow oe@ each time it is e3ec"te!. 9
+act, %y "sig shift, it is possi%le to get %y with oly oe parameter Ei a!!itio to $(,
which e#er chagesF,
^G6bin6bash
^ 7osit-7aram!: scri7t to dis7la1 all arguments
countM#
9hile [[ $^ -gt ( ]]T do
echo _4rgument $count M $#_
countM$NNcount 8 #PP
shift
done
Each time shift is e3ec"te!, the #al"e o+ $! is mo#e! to $#, the #al"e o+ $$ is mo#e!
to $! a! so o. The #al"e o+ $^ is also re!"ce! %y oe.
9 the 7osit-7aram! program, we create a loop that e#al"ates the "m%er o+
arg"mets remaiig a! coti"es as log as there is at least oe. We !isplay the
c"rret arg"met, icremet the #aria%le count with each iteratio o+ the loop to
pro#i!e a r"ig co"t o+ the "m%er o+ arg"mets processe! a!, +ially, e3ec"te a
shift to loa! $# with the e3t arg"met. Aere is the program at work,
[me@linuxbox ~]$ posit-param2 a b c d
4rgument # M a
4rgument ! M b
4rgument $ M c
4rgument % M d
Sim5e A55i/ations
E#e witho"t shift, itIs possi%le to write "se+"l applicatios "sig positioal
parameters. Jy way o+ e3ample, here is a simple +ile i+ormatio program,
110
$ccessig The Comma! *ie
^G6bin6bash
^ fileOinfo: sim7le file information 7rogram
:R@L4+&M$Nbasename $(P
if [[ -e $# ]]T then
echo -e _]n-ile T17e:_
file $#
echo -e _]n-ile *tatus:_
stat $#
else
echo _$:R@L4+&: usage: $:R@L4+& file_ DY!
exit #
fi
This program !isplays the +ile type E!etermie! %y the file comma! F a! the +ile
stat"s E+rom the stat comma!F o+ a speci+ie! +ile. 7e iterestig +eat"re o+ this
program is the :R@L4+& #aria%le. 9t is gi#e the #al"e that res"lts +rom the
basename $( comma!. The basename comma! remo#es the lea!ig portio o+ a
pathame, lea#ig oly the %ase ame o+ a +ile. 9 o"r e3ample, basename remo#es the
lea!ig portio o+ the pathame cotaie! i the $( parameter, the +"ll pathame o+ o"r
e3ample program. This #al"e is "se+"l whe costr"ctig messages s"ch as the "sage
message at the e! o+ the program. Jy co!ig it this way, the script ca %e reame! a!
the message a"tomatically a!4"sts to cotai the ame o+ the program.
Using 'ositiona 'arameters With She 2un/tions
J"st as positioal parameters are "se! to pass arg"mets to shell scripts, they ca also %e
"se! to pass arg"mets to shell +"ctios. To !emostrate, we will co#ert the
fileOinfo script ito a shell +"ctio,
fileOinfo NP [
^ fileOinfo: function to dis7la1 file information
if [[ -e $# ]]T then
echo -e _]n-ile T17e:_
file $#
echo -e _]n-ile *tatus:_
stat $#
else
echo _$-3LCL4+&: usage: $-3LCL4+& file_ DY!
return #
11.
(( 8 6ositioal 6arameters
fi
\
&ow, i+ a script that icorporates the fileOinfo shell +"ctio calls the +"ctio with a
+ileame arg"met, the arg"met will %e passe! to the +"ctio.
With this capa%ility, we ca write may "se+"l shell +"ctios that ca ot oly %e "se! i
scripts, %"t also withi the <bashrc +ile.
&otice that the :R@L4+& #aria%le was chage! to the shell #aria%le -3LCL4+&. The
shell a"tomatically "p!ates this #aria%le to keep track o+ the c"rretly e3ec"te! shell
+"ctio. &ote that $( always cotais the +"ll pathame o+ the +irst item o the
comma! lie Ei.e., the ame o+ the programF a! !oes ot cotai the ame o+ the shell
+"ctio as we might e3pect.
3andling Positional Parameters !n 'asse
9t is sometimes "se+"l to maage all the positioal parameters as a gro"p. 0or e3ample,
we might wat to write a ?wrapper@ aro"! aother program. This meas that we create
a script or shell +"ctio that simpli+ies the e3ec"tio o+ aother program. The wrapper
s"pplies a list o+ arcae comma! lie optios a! the passes a list o+ arg"mets to the
lower le#el program.
The shell pro#i!es two special parameters +or this p"rpose. They %oth e3pa! ito the
complete list o+ positioal parameters, %"t !i++er i rather s"%tle ways. They are,
Table 33-1: The Q )nd R Special 6arameters
Parameter Description
$E
E3pa!s ito the list o+ positioal parameters, startig with ..
Whe s"rro"!e! %y !o"%le ="otes, it e3pa!s ito a !o"%le
="ote! strig cotaiig all o+ the positioal parameters, each
separate! %y the +irst character o+ the 90S shell #aria%le E%y !e+a"lt
a space characterF.
$@
E3pa!s ito the list o+ positioal parameters, startig with ..
Whe s"rro"!e! %y !o"%le ="otes, it e3pa!s each positioal
parameter ito a separate wor! s"rro"!e! %y !o"%le ="otes.
Aere is a script that shows them i actio,
112
Aa!lig 6ositioal 6arameters E >asse
^G6bin6bash
^ 7osit-7arams$ : scri7t to demonstrate $E and $@
7rintO7arams NP [
echo _]$# M $#_
echo _]$! M $!_
echo _]$$ M $$_
echo _]$% M $%_
\
7assO7arams NP [
echo -e _]n_ K$E :KT 7rintO7arams $E
echo -e _]n_ K_$E_ :KT 7rintO7arams _$E_
echo -e _]n_ K$@ :KT 7rintO7arams $@
echo -e _]n_ K_$@_ :KT 7rintO7arams _$@_
\
7assO7arams _9ord_ _9ords 9ith s7aces_
9 this rather co#ol"te! program, we create two arg"mets, ?wor!@ a! ?wor!s with
spaces@ a! pass them to the 7assO7arams +"ctio. That +"ctio, i t"r, passes
them o to the 7rintO7arams +"ctio, "sig each o+ the +o"r metho!s a#aila%le with
the special parameters $G a! $@. Whe e3ec"te!, the script re#eals the !i++ereces,
[me@linuxbox ~]$ posit-param
$E :
$# M 9ord
$! M 9ords
$$ M 9ith
$% M s7aces
_$E_ :
$# M 9ord 9ords 9ith s7aces
$! M
$$ M
$% M
$@ :
$# M 9ord
$! M 9ords
$$ M 9ith
$% M s7aces
_$@_ :
$# M 9ord
11(
(( 8 6ositioal 6arameters
$! M 9ords 9ith s7aces
$$ M
$% M
With o"r arg"mets, %oth $G a! $@ pro!"ce a +o"r wor! res"lt,
9ord 9ords 9ith s7aces
_$E_ pro!"ces a oe wor! res"lt,
_9ord 9ords 9ith s7aces_
_$@_ pro!"ces a two wor! res"lt,
_9ord_ _9ords 9ith s7aces_
which matches o"r act"al itet. The lesso to take +rom this is that e#e tho"gh the
shell pro#i!es +o"r !i++eret ways o+ gettig the list o+ positioal parameters, _$@_ is %y
+ar the most "se+"l +or most sit"atios, %eca"se it preser#es the itegrity o+ each
positioal parameter.
# >ore Complete #pplication
$+ter a log hiat"s, we are goig to res"me work o o"r s1sOinfoO7age program.
7"r e3t a!!itio will a!! se#eral comma! lie optios to the program as +ollows,
;ut(ut "ile. We will a!! a optio to speci+y a ame +or a +ile to cotai the
programIs o"tp"t. 9t will %e speci+ie! as either -f file or --file file.
)nteractive mode. This optio will prompt the "ser +or a o"tp"t +ileame a!
will !etermie i+ the speci+ie! +ile alrea!y e3ists. 9+ it !oes, the "ser will %e
prompte! %e+ore the e3istig +ile is o#erwritte. This optio will %e speci+ie! %y
either -i or --interactive.
Hel(. Either -h or --hel7 may %e speci+ie! to ca"se the program to o"tp"t a
i+ormati#e "sage message.
Aere is the co!e ee!e! to implemet the comma! lie processig,
usage NP [
echo _$:R@L4+&: usage: $:R@L4+& [-f file Q -i]_
return
\
^ 7rocess command line o7tions
interactiveM
111
$ >ore Complete $pplicatio
filenameM
9hile [[ -n $# ]]T do
case $# in
-f Q --fileP shift
filenameM$#
TT
-i Q --interactiveP interactiveM#
TT
-h Q --hel7P usage
exit
TT
EP usage DY!
exit #
TT
esac
shift
done
0irst, we a!! a shell +"ctio calle! usage to !isplay a message whe the help optio is
i#oke! or a "kow optio is attempte!.
&e3t, we %egi the processig loop. This loop coti"es while the positioal parameter
$# is ot empty. $t the %ottom o+ the loop, we ha#e a shift comma! to a!#ace the
positioal parameters to es"re that the loop will e#et"ally termiate.
Withi the loop, we ha#e a case statemet that e3amies the c"rret positioal
parameter to see i+ it matches ay o+ the s"pporte! choices. 9+ a s"pporte! parameter is
+o"!, it is acte! "po. 9+ ot, the "sage message is !isplaye! a! the script termiates
with a error.
The -f parameter is ha!le! i a iterestig way. Whe !etecte!, it ca"ses a
a!!itioal shift to occ"r, which a!#aces the positioal parameter $# to the +ileame
arg"met s"pplie! to the -f optio.
We e3t a!! the co!e to implemet the iteracti#e mo!e,
^ interactive mode
if [[ -n $interactive ]]T then
9hile trueT do
read -7 _&nter name of out7ut file: _ filename
if [[ -e $filename ]]T then
read -7 _K$filenameK exists< ver9riteF [16n6B] D _
case $R&:=U in
112
(( 8 6ositioal 6arameters
UQ1P break
TT
ZQBP echo _:rogram terminated<_
exit
TT
EP continue
TT
esac
elif [[ -C $filename ]]T then
continue
else
break
fi
done
fi
9+ the interactive #aria%le is ot empty, a e!less loop is starte!, which cotais
the +ileame prompt a! s"%se="et e3istig +ile ha!lig co!e. 9+ the !esire! o"tp"t +ile
alrea!y e3ists, the "ser is prompte! to o#erwrite, choose aother +ileame, or ="it the
program. 9+ the "ser chooses to o#erwrite a e3istig +ile, a break is e3ec"te! to
termiate the loop. &otice how the case statemet oly !etects i+ the "ser chooses to
o#erwrite or ="it. $y other choice ca"ses the loop to coti"e a! prompts the "ser
agai.
9 or!er to implemet the o"tp"t +ileame +eat"re, we m"st +irst co#ert the e3istig page
writig co!e ito a shell +"ctio, +or reasos that will %ecome clear i a momet,
9riteOhtmlO7age NP [
cat XX- O&-O
XST+=D
XS&4'D
XTAT=&D$TAT=&X6TAT=&D
X6S&4'D
XH'UD
XS#D$TAT=&X6S#D
X:D$TA+&O*T4+:X6:D
$Nre7ortOu7timeP
$Nre7ortOdiskOs7aceP
$Nre7ortOhomeOs7aceP
X6H'UD
X6ST+=D
O&-O
return
\
^ out7ut html 7age
11K
$ >ore Complete $pplicatio
if [[ -n $filename ]]T then
if touch $filename YY [[ -f $filename ]]T then
9riteOhtmlO7age D $filename
else
echo _$:R@L4+&: Cannot 9rite file K$filenameK_ DY!
exit #
fi
else
9riteOhtmlO7age
fi
The co!e that ha!les the logic o+ the -f optio appears at the e! o+ the listig show
a%o#e. 9 it, we test +or the e3istece o+ a +ileame a!, i+ +o"!, a test is per+orme! to
see i+ the +ile is i!ee! writa%le. To !o this, a touch is per+orme!, +ollowe! %y a test to
!etermie i+ the res"ltig +ile is a reg"lar +ile. These two tests take care o+ sit"atios
where a i#ali! pathame is ip"t Etouch will +ailF a! i+ the +ile alrea!y e3ists, that
itIs a reg"lar +ile.
$s we ca see, the 9riteOhtmlO7age +"ctio is calle! to per+orm the act"al
geeratio o+ the page. 9ts o"tp"t is either !irecte! to sta!ar! o"tp"t Ei+ the #aria%le
filename is emptyF or re!irecte! to the speci+ie! +ile.
Summing -p
With the a!!itio o+ positioal parameters, we ca ow write +airly +"ctioal scripts.
0or simple, repetiti#e tasks, positioal parameters make it possi%le to write #ery "se+"l
shell +"ctios that ca %e place! i a "serIs <bashrc +ile.
7"r s1sOinfoO7age program has grow i comple3ity a! sophisticatio. Aere is a
complete listig, with the most recet chages highlighte!,
^G6bin6bash
^ s1sOinfoO7age: 7rogram to out7ut a s1stem information 7age
BHSQ?:OG$34basename 3*6
TAT=&M_*1stem Anformation Re7ort -or $S*TL4+&_
C3RR&LTOTA+&M$Ndate 8_5x 5r 5I_P
TA+&O*T4+:M_@enerated $C3RR&LTOTA+&> b1 $3*&R_
re7ortOu7time NP [
cat XX- O&-O
XS!D*1stem 37timeX6S!D
X:R&D$Nu7timePX6:R&D
11/
(( 8 6ositioal 6arameters
O&-O
return
\
re7ortOdiskOs7ace NP [
cat XX- O&-O
XS!D'isk *7ace 3tiliCationX6S!D
X:R&D$Ndf -hPX6:R&D
O&-O
return
\
re7ortOhomeOs7ace NP [
if [[ $Nid -uP -eB ( ]]T then
cat XX- O&-O
XS!DSome *7ace 3tiliCation N4ll 3sersPX6S!D
X:R&D$Ndu -sh 6home6EPX6:R&D
O&-O
else
cat XX- O&-O
XS!DSome *7ace 3tiliCation N$3*&RPX6S!D
X:R&D$Ndu -sh $S+&PX6:R&D
O&-O
fi
return
\
usage 46 9
echo J3BHSQ?:OG2 usage2 3BHSQ?:OG (-f file 0 -i)J
return
>
write.html.page 46 9
cat //- .GS7.
/R+OT&
/RG:1&
/+P+TG&3+P+TG//+P+TG&
//RG:1&
/<S1^&
/R1&3+P+TG//R1&
/B&3+POG.F+:OB//B&
34report.uptime6
34report.disk.space6
34report.home.space6
//<S1^&
//R+OT&
.GS7.
return
>
118
S"mmig )p
^ 7rocess command line o7tions
interactive$
filename$
while (( -n 31 ))# do
case 31 in
-f 0 --file6 shift
filename$31
##
-i 0 --interactive6 interactive$1
##
-h 0 --help6 usage
exit
##
*6 usage &'2
exit 1
##
esac
shift
done
^ interactive mode
if (( -n 3interactive ))# then
while true# do
read -p JGnter name of output file2 J filename
if (( -e 3filename ))# then
read -p J%3filename% exists. Sverwrite_ (!/n/,) & J
case 3HGBT^ in
^0!6 break
##
\0,6 echo JBrogram terminated.J
exit
##
*6 continue
##
esac
fi
done
fi
^ out7ut html 7age
if (( -n 3filename ))# then
if touch 3filename '' (( -f 3filename ))# then
write.html.page & 3filename
else
echo J3BHSQ?:OG2 =annot write file %3filename%J &'2
119
(( 8 6ositioal 6arameters
exit 1
fi
else
write.html.page
fi
WeIre ot !oe yet. There are still more thigs we ca !o a! impro#emets we ca
make.
;urther 1eading
The 8ash ,ackers /iki has a goo! article o positioal parameters,
http,--%ash-hackers.org-wiki-!ok".php-scriptig-posparams
The 8ash 9eference !anual has a article o the special parameters, icl"!ig
$E a! $@,
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.htmlPSpecial-6arameters
9 a!!itio to the techi="es !isc"sse! i this chapter, bash icl"!es a %"ilti
comma! calle! geto7ts, which ca also %e "se! +or process comma! lie
arg"mets. 9t is !escri%e! i the SAE** J)9*T9& C7>>$&'S sectio o+ the
bash ma page a! at the 8ash ,ackers /iki,
http,--%ash-hackers.org-wiki-!ok".php-howto-getoptsOt"torial
120
(1 8 0low Cotrol, *oopig With +or
3 (lo5 Control; <oo#ing With for
9 this +ial chapter o +low cotrol, we will look at aother o+ the shellIs loopig
costr"cts. The for loop !i++ers +rom the while a! "til loops i that it pro#i!es a meas
o+ processig se="eces !"rig a loop. This t"rs o"t to %e #ery "se+"l whe
programmig. $ccor!igly, the +or loop is a #ery pop"lar costr"ct i bash scriptig.
$ +or loop is implemete!, at"rally eo"gh, with the for comma!. 9 mo!er
#ersios o+ bash, for is a#aila%le i two +orms.
forI Traditional Shell ;orm
The origial for comma!Is syta3 is,
for variable [in words]T do
commands
done
Where variable is the ame o+ a #aria%le that will icremet !"rig the e3ec"tio o+ the
loop, words is a optioal list o+ items that will %e se="etially assige! to variable, a!
commands are the comma!s that are to %e e3ec"te! o each iteratio o+ the loop.
The for comma! is "se+"l o the comma! lie. We ca easily !emostrate how it
works,
[me@linuxbox ~]$ for i in : < = 1# do echo 3i# done
4
H
C
'
9 this e3ample, for is gi#e a list o+ +o"r wor!s, ?$,@ ?J,@ ?C,@ a! ?'.@ With a list o+
+o"r wor!s, the loop is e3ec"te! +o"r times. Each time the loop is e3ec"te!, a wor! is
assige! to the #aria%le i. 9si!e the loop, we ha#e a echo comma! that !isplays the
#al"e o+ i to show the assigmet. $s with the 9hile a! until loops, the done
keywor! closes the loop.
12.
(1 8 0low Cotrol, *oopig With +or
The really power+"l +eat"re o+ for is the "m%er o+ iterestig ways we ca create the
list o+ wor!s. 0or e3ample, thro"gh %race e3pasio,
[me@linuxbox ~]$ for i in 9:..1># do echo 3i# done
4
H
C
'
7r pathame e3pasio,
[me@linuxbox ~]$ for i in distros*.txt# do echo 3i# done
distros-b1-date<txt
distros-dates<txt
distros-ke1-names<txt
distros-ke1-vernums<txt
distros-names<txt
distros<txt
distros-vernums<txt
distros-versions<txt
7r comma! s"%stit"tio,
^G6bin6bash
^ longest-9ord : find longest string in a file
9hile [[ -n $# ]]T do
if [[ -r $# ]]T then
maxO9ordM
maxOlenM(
for i in $Nstrings $#PT do
lenM$Necho $i Q 9c -cP
if NN len D maxOlen PPT then
maxOlenM$len
maxO9ordM$i
fi
done
echo _$#: K$maxO9ordK N$maxOlen charactersP_
fi
shift
done
122
+or, Tra!itioal Shell 0orm
9 this e3ample, we look +or the logest strig +o"! withi a +ile. Whe gi#e oe or
more +iles ames o the comma! lie, this program "ses the strings program Ewhich
is icl"!e! i the ;&) %i"tils packageF to geerate a list o+ rea!a%le te3t ?wor!s@ i
each +ile. The for loop processes each wor! i t"r a! !etermies i+ the c"rret wor!
is the logest +o"! so +ar. Whe the loop cocl"!es, the logest wor! is !isplaye!.
9+ the optioal in words portio o+ the for comma! is omitte!, for !e+a"lts to
processig the positioal parameters. We will mo!i+y o"r longest-9ord script to "se
this metho!,
^G6bin6bash
^ longest-9ord! : find longest string in a file
for iT do
if [[ -r $i ]]T then
maxO9ordM
maxOlenM(
for j in $Nstrings $iPT do
lenM$Necho $j Q 9c -cP
if NN len D maxOlen PPT then
maxOlenM$len
maxO9ordM$j
fi
done
echo _$i: K$maxO9ordK N$maxOlen charactersP_
fi
done
$s we ca see, we ha#e chage! the o"termost loop to "se for i place o+ 9hile. Jy
omittig the list o+ wor!s i the for comma!, the positioal parameters are "se!
istea!. 9si!e the loop, pre#io"s istaces o+ the #aria%le i ha#e %ee chage! to the
#aria%le j. The "se o+ shift has also %ee elimiate!.
+h, i.
5o" may ha#e otice! that the #aria%le i was chose +or each o+ the for loop
e3amples a%o#e. WhyC &o speci+ic reaso act"ally, %esi!es tra!itio. The
#aria%le "se! with for ca %e ay #ali! #aria%le, %"t i is the most commo,
+ollowe! %y j a! k.
12(
(1 8 0low Cotrol, *oopig With +or
The %asis o+ this tra!itio comes +rom the 0ortra programmig lag"age. 9
0ortra, "!eclare! #aria%les startig with the letters 9, J, M, *, a! > are
a"tomatically type! as itegers while #aria%les %egiig with ay other letter are
type! as real E"m%ers with !ecimal +ractiosF. This %eha#ior le! programmers
to "se the #aria%les 9, J, a! M +or loop #aria%les, sice it was less work to "se
them whe a temporary #aria%le Eas loop #aria%les o+te areF was ee!e!.
9t also le! to the +ollowig 0ortra-%ase! witticism,
?;7' is real, "less !eclare! iteger.@
forI C Language ;orm
:ecet #ersios o+ bash ha#e a!!e! a seco! +orm o+ for comma! syta3, oe that
resem%les the +orm +o"! i the C programmig lag"age. >ay other lag"ages
s"pport this +orm, as well,
for NN expression1T expression2T expression3 PPT do
commands
done
where e+pression1, e+pression', a! e+pression3 are arithmetic e3pressios a!
commands are the comma!s to %e per+orme! !"rig each iteratio o+ the loop.
9 terms o+ %eha#ior, this +orm is e="i#alet to the +ollowig costr"ct,
NN expression1 PP
9hile NN expression2 PPT do
commands
NN expression3 PP
done
e+pression1 is "se! to iitialiDe co!itios +or the loop, e+pression' is "se! to !etermie
whe the loop is +iishe! a! e+pression3 is carrie! o"t at the e! o+ each iteratio o+ the
loop.
Aere is a typical applicatio,
^G6bin6bash
^ sim7leOcounter : demo of C st1le for command
for NN iM(T iX"T iMi8# PPT do
echo $i
done
121
+or, C *ag"age 0orm
Whe e3ec"te!, it pro!"ces the +ollowig o"tp"t,
[me@linuxbox ~]$ simple.counter
(
#
!
$
%
9 this e3ample, e+pression1 iitialiDes the #aria%le i with the #al"e o+ Dero, e+pression'
allows the loop to coti"e as log as the #al"e o+ i remais less tha +i#e, a!
e+pression3 icremets the #al"e o+ i %y oe each time the loop repeats.
The C lag"age +orm o+ for is "se+"l aytime a "meric se="ece is ee!e!. We will
see se#eral applicatios +or this i the e3t two chapters.
Summing -p
With o"r kowle!ge o+ the for comma!, we will ow apply the +ial impro#emets to
o"r s1sOinfoO7age script. C"rretly, the re7ortOhomeOs7ace +"ctio looks
like this,
re7ortOhomeOs7ace NP [
if [[ $Nid -uP -eB ( ]]T then
cat XX- O&-O
XS!DSome *7ace 3tiliCation N4ll 3sersPX6S!D
X:R&D$Ndu -sh 6home6EPX6:R&D
O&-O
else
cat XX- O&-O
XS!DSome *7ace 3tiliCation N$3*&RPX6S!D
X:R&D$Ndu -sh $S+&PX6:R&D
O&-O
fi
return
\
&e3t, we will rewrite it to pro#i!e more !etail +or each "serIs home !irectory, a! icl"!e
the total "m%er o+ +iles a! s"%!irectories i each,
re7ortOhomeOs7ace NP [
122
(1 8 0low Cotrol, *oopig With +or
local formatM_5/s5#(s5#(s]n_
local i dirOlist totalOfiles totalOdirs totalOsiCe userOname
if [[ $Nid -uP -eB ( ]]T then
dirOlistM6home6E
userOnameM_4ll 3sers_
else
dirOlistM$S+&
userOnameM$3*&R
fi
echo _XS!DSome *7ace 3tiliCation N$userOnamePX6S!D_
for i in $dirOlistT do

totalOfilesM$Nfind $i -t17e f Q 9c -lP
totalOdirsM$Nfind $i -t17e d Q 9c -lP
totalOsiCeM$Ndu -sh $i Q cut -f #P
echo _XS$D$iX6S$D_
echo _X:R&D_
7rintf _$format_ _'irs_ _-iles_ _*iCe_
7rintf _$format_ _----_ _-----_ _----_
7rintf _$format_ $totalOdirs $totalOfiles $totalOsiCe
echo _X6:R&D_
done
return
\
This rewrite applies m"ch o+ what we ha#e leare! so +ar. We still test +or the s"per"ser,
%"t istea! o+ per+ormig the complete set o+ actios as part o+ the if, we set some
#aria%les "se! later i a +or loop. We ha#e a!!e! se#eral local #aria%les to the +"ctio
a! ma!e "se o+ 7rintf to +ormat some o+ the o"tp"t.
;urther 1eading
The )dvanced 8ash-Scripting @uide has a chapter o loops, with a #ariety o+
e3amples "sig for,
http,--tl!p.org-*'6-a%s-html-loops..html
The 8ash 9eference !anual !escri%es the loopig compo"! comma!s,
icl"!ig for,
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.htmlP*oopig-Costr"cts
12K
(2 8 Strigs $! &"m%ers
3& Strings )nd Nu%bers
Comp"ter programs are all a%o"t workig with !ata. 9 past chapters, we ha#e +oc"se!
o processig !ata at the +ile le#el. Aowe#er, there are may programmig pro%lems that
ee! to %e sol#e! "sig smaller "its o+ !ata s"ch as strigs a! "m%ers.
9 this chapter, we will look at se#eral shell +eat"res that are "se! to maip"late strigs
a! "m%ers. The shell pro#i!es a #ariety o+ parameter e3pasios that per+orm strig
operatios. 9 a!!itio to arithmetic e3pasio Ewhich we to"che! "po i Chapter 8F,
there is a commo comma! lie program calle! bc, which per+orms higher le#el math.
Parameter $xpansion
Tho"gh parameter e3pasio came "p i Chapter 8, we !i! ot co#er it i !etail %eca"se
most parameter e3pasios are "se! i scripts rather tha o the comma! lie. We ha#e
alrea!y worke! with some +orms o+ parameter e3pasioH +or e3ample, shell #aria%les.
The shell pro#i!es may more.
Basi/ 'arameters
The simplest +orm o+ parameter e3pasio is re+lecte! i the or!iary "se o+ #aria%les.
0or e3ample,
$a
whe e3pa!e!, %ecomes whate#er the #aria%le a cotais. Simple parameters may also
%e s"rro"!e! %y %races,
$[a\
This has o e++ect o the e3pasio, %"t is re="ire! i+ the #aria%le is a!4acet to other
te3t, which may co+"se the shell. 9 this e3ample, we attempt to create a +ileame %y
appe!ig the strig ?O+ile@ to the cotets o+ the #aria%le a.
[me@linuxbox ~]$ a$JfooJ
[me@linuxbox ~]$ echo J3a.fileJ
12/
(2 8 Strigs $! &"m%ers
9+ we per+orm this se="ece, the res"lt will %e othig, %eca"se the shell will try to
e3pa! a #aria%le ame! aOfile rather tha a. This pro%lem ca %e sol#e! %y a!!ig
%races,
[me@linuxbox ~]$ echo b39a>.fileb
fooOfile
We ha#e also see that positioal parameters greater tha 9 ca %e accesse! %y
s"rro"!ig the "m%er i %races. 0or e3ample, to access the ele#eth positioal
parameter, we ca !o this,
$[##\
6x5ansions To :anage 6m5ty ?ariabes
Se#eral parameter e3pasios !eal with oe3istet a! empty #aria%les. These
e3pasios are ha!y +or ha!lig missig positioal parameters a! assigig !e+a"lt
#al"es to parameters.
$[parameter:-word\
9+ parameter is "set Ei.e., !oes ot e3istF or is empty, this e3pasio res"lts i the #al"e
o+ word. 9+ parameter is ot empty, the e3pasio res"lts i the #al"e o+ parameter.
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 39foo2-Jsubstitute value if unsetJ>
substitute value if unset
[me@linuxbox ~]$ echo 3foo
[me@linuxbox ~]$ foo$bar
[me@linuxbox ~]$ echo 39foo2-Jsubstitute value if unsetJ>
bar
[me@linuxbox ~]$ echo 3foo
bar
$[parameter:Mword\
9+ parameter is "set or empty, this e3pasio res"lts i the #al"e o+ word. 9 a!!itio,
the #al"e o+ word is assige! to parameter. 9+ parameter is ot empty, the e3pasio
res"lts i the #al"e o+ parameter.
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 39foo2$Jdefault value if unsetJ>
128
6arameter E3pasio
default value if unset
[me@linuxbox ~]$ echo 3foo
default value if unset
[me@linuxbox ~]$ foo$bar
[me@linuxbox ~]$ echo 39foo2$Jdefault value if unsetJ>
bar
[me@linuxbox ~]$ echo 3foo
bar
1ote, 6ositioal a! other special parameters caot %e assige! this way.
$[parameter:Fword\
9+ parameter is "set or empty, this e3pasio ca"ses the script to e3it with a error, a!
the cotets o+ word are set to sta!ar! error. 9+ parameter is ot empty, the e3pasio
res"lts i the #al"e o+ parameter.
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 39foo2_Jparameter is empt!J>
bash: foo: 7arameter is em7t1
[me@linuxbox ~]$ echo 3_
#
[me@linuxbox ~]$ foo$bar
[me@linuxbox ~]$ echo 39foo2_Jparameter is empt!J>
bar
[me@linuxbox ~]$ echo 3_
(
$[parameter:8word\
9+ parameter is "set or empty, the e3pasio res"lts i othig. 9+ parameter is ot
empty, the #al"e o+ word is s"%stit"te! +or parameterH howe#er, the #al"e o+ parameter is
ot chage!.
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 39foo25Jsubstitute value if setJ>
[me@linuxbox ~]$ foo$bar
[me@linuxbox ~]$ echo 39foo25Jsubstitute value if setJ>
substitute value if set
129
(2 8 Strigs $! &"m%ers
6x5ansions That !eturn ?ariabe ,ames
The shell has the a%ility to ret"r the ames o+ #aria%les. This is "se! i some rather
e3otic sit"atios.
$[GprefixE\
$[Gprefix@\
This e3pasio ret"rs the ames o+ e3istig #aria%les with ames %egiig with prefi+.
$ccor!ig to the bash !oc"metatio, %oth +orms o+ the e3pasio per+orm i!etically.
Aere, we list all the #aria%les i the e#iromet with ames that %egi with J$SA,
[me@linuxbox ~]$ echo 39L<:FR*>
H4*S H4*SO4R@C H4*SO4R@; H4*SOC++4L' H4*SOC+:=&TAL
H4*SOC+:=&TALO'AR H4*SO=AL&L H4*SO*3RC& H4*SO*3H*S&==
H4*SO;&R*AL- H4*SO;&R*AL
String <5erations
There is a large set o+ e3pasios that ca %e "se! to operate o strigs. >ay o+ these
e3pasios are partic"larly well-s"ite! +or operatios o pathames.
$[^parameter\
E3pa!s ito the legth o+ the strig cotaie! %y parameter. &ormally, parameter is a
strigH howe#er, i+ parameter is either @ or E, the the e3pasio res"lts i the "m%er o+
positioal parameters.
[me@linuxbox ~]$ foo$J+his string is long.J
[me@linuxbox ~]$ echo J%3foo% is 39Ufoo> characters long.J
KThis string is long<K is !( characters long<
$[parameter:offset\
$[parameter:offset:length\
This e3pasio is "se! to e3tract a portio o+ the strig cotaie! i parameter. The
e3tractio %egis at offset characters +rom the %egiig o+ the strig a! coti"es "til
the e! o+ the strig "less the length is speci+ie!.
[me@linuxbox ~]$ foo$J+his string is long.J
[me@linuxbox ~]$ echo 39foo2">
string is long<
1K0
6arameter E3pasio
[me@linuxbox ~]$ echo 39foo2"2N>
string
9+ the #al"e o+ offset is egati#e, it is take to mea it starts +rom the e! o+ the strig
rather tha the %egiig. &ote that egati#e #al"es m"st %e prece!e! %y a space to
pre#et co+"sio with the $[parameter:-word\ e3pasio. length, i+ preset, m"st
ot %e less tha Dero.
9+ parameter is @, the res"lt o+ the e3pasio is length positioal parameters, startig at
offset.
[me@linuxbox ~]$ foo$J+his string is long.J
[me@linuxbox ~]$ echo 39foo2 -">
long<
[me@linuxbox ~]$ echo 39foo2 -"22>
lo
$[parameter^pattern\
$[parameter^^pattern\
This e3pasio remo#es a lea!ig portio o+ the strig cotaie! i parameter !e+ie! %y
pattern. pattern is a wil!car! patter like those "se! i pathame e3pasio. The
!i++erece i the two +orms is that the ^ +orm remo#es the shortest match, while the ^^
+orm remo#es the logest match.
[me@linuxbox ~]$ foo$file.txt.-ip
[me@linuxbox ~]$ echo 39fooU*.>
txt<Ci7
[me@linuxbox ~]$ echo 39fooUU*.>
Ci7
$[parameter5pattern\
$[parameter55pattern\
These e3pasios are the same as the ^ a! ^^ e3pasios a%o#e, e3cept they remo#e
te3t +rom the e! o+ the strig cotaie! i parameter rather tha +rom the %egiig.
[me@linuxbox ~]$ foo$file.txt.-ip
[me@linuxbox ~]$ echo 39foo8.*>
file<txt
[me@linuxbox ~]$ echo 39foo88.*>
1K.
(2 8 Strigs $! &"m%ers
file
$[parameter6pattern6string\
$[parameter66pattern6string\
$[parameter6^pattern6string\
$[parameter65pattern6string\
This e3pasio per+orms a search a! replace "po the cotets o+ parameter. 9+ te3t is
+o"! matchig wil!car! pattern, it is replace! with the cotets o+ string. 9 the ormal
+orm, oly the +irst occ"rrece o+ pattern is replace!. 9 the 66 +orm, all occ"rreces are
replace!. The 6^ +orm re="ires that the match occ"r at the %egiig o+ the strig, a!
the 65 +orm re="ires the match to occ"r at the e! o+ the strig. -string may %e omitte!,
which ca"ses the te3t matche! %y pattern to %e !elete!.
[me@linuxbox ~]$ foo$YBQ.YBQ
[me@linuxbox ~]$ echo 39foo/YBQ/jpg>
j7g<?:@
[me@linuxbox ~]$ echo 39foo//YBQ/jpg>
j7g<j7g
[me@linuxbox ~]$ echo 39foo/UYBQ/jpg>
j7g<?:@
[me@linuxbox ~]$ echo 39foo/8YBQ/jpg>
?:@<j7g
6arameter e3pasio is a goo! thig to kow. The strig maip"latio e3pasios ca %e
"se! as s"%stit"tes +or other commo comma!s s"ch as sed a! cut. E3pasios
impro#e the e++iciecy o+ scripts %y elimiatig the "se o+ e3teral programs. $s a
e3ample, we will mo!i+y the longest-9ord program !isc"sse! i the pre#io"s
chapter to "se the parameter e3pasio $[^j\ i place o+ the comma! s"%stit"tio $
Necho $j Q 9c -cP a! its res"ltig s"%shell like so,
^G6bin6bash
^ longest-9ord$ : find longest string in a file
for iT do
if [[ -r $i ]]T then
maxO9ordM
maxOlenM
for j in $Nstrings $iPT do
len$39Uj>
if NN len D maxOlen PPT then
1K2
6arameter E3pasio
maxOlenM$len
maxO9ordM$j
fi
done
echo _$i: K$maxO9ordK N$maxOlen charactersP_
fi
shift
done
&e3t, we will compare the e++iciecy o+ the two #ersios %y "sig the time comma!,
[me@linuxbox ~]$ time longest-word2 dirlist-usr-bin.txt
dirlist-usr-bin<txt: Kscrollkee7er-get-extended-content-listK N$/
charactersP
real (m$<.#/s
user (m#<"%%s
s1s (m#<)./s
[me@linuxbox ~]$ time longest-word dirlist-usr-bin.txt
dirlist-usr-bin<txt: Kscrollkee7er-get-extended-content-listK N$/
charactersP
real (m(<(.(s
user (m(<(".s
s1s (m(<((/s
The origial #ersio o+ the script takes (.K.8 seco!s to sca the te3t +ile, while the ew
#ersio, "sig parameter e3pasio, takes oly 0.0K seco!sBa #ery sigi+icat
impro#emet.
#rithmetic $%aluation #nd $xpansion
We looke! at arithmetic e3pasio i Chapter 8. 9t is "se! to per+orm #ario"s arithmetic
operatios o itegers. 9ts %asic +orm is,
$NNexpressionPP
where e+pression is a #ali! arithmetic e3pressio.
This is relate! to the compo"! comma! NN PP "se! +or arithmetic e#al"atio Etr"th
testsF we eco"tere! i Chapter 28.
9 pre#io"s chapters, we saw some o+ the commo types o+ e3pressios a! operators.
Aere, we will look at a more complete list.
1K(
(2 8 Strigs $! &"m%ers
,umber Bases
Jack i Chapter .0, we got a look at octal E%ase 8F a! he3a!ecimal E%ase .KF "m%ers.
9 arithmetic e3pressios, the shell s"pports iteger costats i ay %ase.
Table 3.-1: Specifying -ifferent >umber 8ases
7otation Description
number
Jy !e+a"lt, "m%ers witho"t ay otatio are treate! as !ecimal
E%ase .0F itegers.
(number
9 arithmetic e3pressios, "m%ers with a lea!ig Dero are
cosi!ere! octal.
(xnumber
Ae3a!ecimal otatio
base^number
number is i base
Some e3amples,
[me@linuxbox ~]$ echo 344*xff66
!""
[me@linuxbox ~]$ echo 3442U1111111166
!""
9 the e3amples a%o#e, we prit the #al"e o+ the he3a!ecimal "m%er ff Ethe largest
two- !igit "m%erF a! the largest eight-!igit %iary E%ase 2F "m%er.
Unary <5erators
There are two "ary operators, the 8 a! -, which are "se! to i!icate i+ a "m%er is
positi#e or egati#e, respecti#ely. 0or e3ample, -".
Sim5e Arithmeti/
The or!iary arithmetic operators are liste! i the ta%le %elow,
Table 3.-': )rithmetic &perators
?perator Description
8
$!!itio
-
S"%tractio
1K1
$rithmetic E#al"atio $! E3pasio
E
>"ltiplicatio
6
9teger !i#isio
EE
E3poetiatio
5
>o!"lo Eremai!erF
>ost o+ these are sel+-e3plaatory, %"t iteger !i#isio a! mo!"lo re="ire +"rther
!isc"ssio.
Sice the shellIs arithmetic oly operates o itegers, the res"lts o+ !i#isio are always
whole "m%ers,
[me@linuxbox ~]$ echo 344 " / 2 66
!
This makes the !etermiatio o+ a remai!er i a !i#isio operatio more importat,
[me@linuxbox ~]$ echo 344 " 8 2 66
#
Jy "sig the !i#isio a! mo!"lo operators, we ca !etermie that +i#e !i#i!e! %y two
res"lts i two, with a remai!er o+ oe.
Calc"latig the remai!er is "se+"l i loops. 9t allows a operatio to %e per+orme! at
speci+ie! iter#als !"rig the loop<s e3ec"tio. 9 the e3ample %elow, we !isplay a lie
o+ "m%ers, highlightig each m"ltiple o+ +i#e,
^G6bin6bash
^ modulo : demonstrate the modulo o7erator
for NNi M (T i XM !(T i M i 8 #PPT do
remainderM$NNi 5 "PP
if NN remainder MM ( PPT then
7rintf _X5dD _ $i
else
7rintf _5d _ $i
fi
done
7rintf _]n_
1K2
(2 8 Strigs $! &"m%ers
Whe e3ec"te!, the res"lts look like this,
[me@linuxbox ~]$ modulo
X(D # ! $ % X"D . ) / 0 X#(D ## #! #$ #% X#"D #. #) #/ #0 X!(D
Assignment
$ltho"gh its "ses may ot %e imme!iately apparet, arithmetic e3pressios may per+orm
assigmet. We ha#e per+orme! assigmet may times, tho"gh i a !i++eret cote3t.
Each time we gi#e a #aria%le a #al"e, we are per+ormig assigmet. We ca also !o it
withi arithmetic e3pressios,
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 3foo
[me@linuxbox ~]$ if 44 foo $ " 66#then echo JPt is true.J# fi
At is true<
[me@linuxbox ~]$ echo 3foo
"
9 the e3ample a%o#e, we +irst assig a empty #al"e to the #aria%le foo a! #eri+y that
it is i!ee! empty. &e3t, we per+orm a if with the compo"! comma! NN foo M "
PP. This process !oes two iterestig thigs, .F it assigs the #al"e o+ +i#e to the
#aria%le foo a! 2F it e#al"ates to tr"e %eca"se the assigmet was s"ccess+"l.
1ote5 9t is importat to remem%er the e3act meaig o+ the M i the e3pressio
a%o#e. $ sigle M per+orms assigmet. foo M " says ?make foo e="al to
+i#e,@ while MM e#al"ates e="i#alece. foo MM " says ?!oes foo e="al +i#eC@
This ca %e #ery co+"sig %eca"se the test comma! accepts a sigle M +or
strig e="i#alece. This is yet aother reaso to "se the more mo!er [[ ]] a!
NN PP compo"! comma!s i place o+ test.
9 a!!itio to the M> the shell also pro#i!es otatios that per+orm some #ery "se+"l
assigmets,
Table 3.-3: )ssignment &perators
7otation Description
1KK
$rithmetic E#al"atio $! E3pasio
parameter M value
Simple assigmet. $ssigs value to parameter.
parameter 8M value
$!!itio. E="i#alet to parameter W parameter V
value
parameter -M value
S"%tractio. E="i#alet to parameter W parameter -
value
parameter EM value
>"ltiplicatio. E="i#alet to parameter W parameter
U value
parameter 6M value
9teger !i#isio. E="i#alet to parameter W
parameter - value
parameter 5M value
>o!"lo. E="i#alet to parameter W parameter _
value
parameter88
Raria%le post-icremet. E="i#alet to parameter W
parameter V . Ehowe#er, see !isc"ssio %elowF
parameter""
Raria%le post-!ecremet. E="i#alet to parameter W
parameter g .
88parameter
Raria%le pre-icremet. E="i#alet to parameter W
parameter V .
--parameter
Raria%le pre-!ecremet. E="i#alet to parameter W
parameter g .
These assigmet operators pro#i!e a co#eiet shortha! +or may commo arithmetic
tasks. 7+ special iterest are the icremet E88F a! !ecremet EeeF operators, which
icrease or !ecrease the #al"e o+ their parameters %y oe. This style o+ otatio is take
+rom the C programmig lag"age a! has %ee icorporate! %y se#eral other
programmig lag"ages, icl"!ig bash.
The operators may appear either at the +rot o+ a parameter or at the e!. While they %oth
either icremet or !ecremet the parameter %y oe, the two placemets ha#e a s"%tle
!i++erece. 9+ place! at the +rot o+ the parameter, the parameter is icremete! Eor
!ecremete!F %e+ore the parameter is ret"re!. 9+ place! a+ter, the operatio is per+orme!
after the parameter is ret"re!. This is rather strage. %"t it is the ite!e! %eha#ior.
Aere is a !emostratio,
[me@linuxbox ~]$ foo$1
[me@linuxbox ~]$ echo 344foo5566
#
[me@linuxbox ~]$ echo 3foo
1K/
(2 8 Strigs $! &"m%ers
!
9+ we assig the #al"e o+ oe to the #aria%le foo a! the icremet it with the 88
operator place! a+ter the parameter ame, foo is ret"re! with the #al"e o+ oe.
Aowe#er, i+ we look at the #al"e o+ the #aria%le a seco! time, we see the icremete!
#al"e. 9+ we place the 88 operator i +rot o+ the parameter, we get the more e3pecte!
%eha#ior,
[me@linuxbox ~]$ foo$1
[me@linuxbox ~]$ echo 34455foo66
!
[me@linuxbox ~]$ echo 3foo
!
0or most shell applicatios, pre+i3ig the operator will %e the most "se+"l.
The VV a! -- operators are o+te "se! i co4"ctio with loops. We will make some
impro#emets to o"r mo!"lo script to tighte it "p a %it,
^G6bin6bash
^ modulo! : demonstrate the modulo o7erator
for NNi M (T i XM !(T 55i PPT do
if NN4i 8 "6 MM ( PPT then
7rintf _X5dD _ $i
else
7rintf _5d _ $i
fi
done
7rintf _]n_
Bit <5erations
7e class o+ operators maip"lates "m%ers i a ""s"al way. These operators work at
the %it le#el. They are "se! +or certai ki!s o+ low le#el tasks, o+te i#ol#ig settig or
rea!ig %it-+lags.
Table 3.-$: 8it &perators
?perator Description
1K8
$rithmetic E#al"atio $! E3pasio
~
Jitwise egatio. &egate all the %its i a "m%er.
XX
*e+t %itwise shi+t. Shi+t all the %its i a "m%er to the le+t.
DD
:ight %itwise shi+t. Shi+t all the %its i a "m%er to the right.
Y
Jitwise $&'. 6er+orm a $&' operatio o all the %its i
two "m%ers.
Q
Jitwise 7:. 6er+orm a 7: operatio o all the %its i two
"m%ers.
`
Jitwise G7:. 6er+orm a e3cl"si#e 7: operatio o all the
%its i two "m%ers.
&ote that there are also correspo!ig assigmet operators E +or e3ample, YYWF +or all
%"t %itwise egatio.
Aere we will !emostrate pro!"cig a list o+ powers o+ two, "sig the le+t %itwise shi+t
operator,
[me@linuxbox ~]$ for 44i$*#i/M#55i66# do echo 3441//i66# done
#
!
%
/
#.
$!
.%
#!/
Logi/
$s we !isco#ere! i Chapter 28, the NN PP compo"! comma! s"pports a #ariety o+
compariso operators. There are a +ew more that ca %e "se! to e#al"ate logic. Aere is
the complete list,
Table 3.-.: %omparison &perators
?perator Description
XM
*ess tha or e="al to
DM
;reater tha or e="al to
X
*ess tha
1K9
(2 8 Strigs $! &"m%ers
D
;reater tha
MM
E="al to
GM
&ot e="al to
YY
*ogical $&'
QQ
*ogical 7:
expr1Fexpr2:expr3
Compariso EteraryF operator. 9+ e3pressio e+pr1
e#al"ates to %e o-Dero Earithmetic tr"eF the e+pr',
else e+pr3.
Whe "se! +or logical operatios, e3pressios +ollow the r"les o+ arithmetic logicH that is,
e3pressios that e#al"ate as Dero are cosi!ere! +alse, while o-Dero e3pressios are
cosi!ere! tr"e. The NN PP compo"! comma! maps the res"lts ito the shellIs
ormal e3it co!es,
[me@linuxbox ~]$ if 44166# then echo JtrueJ# else echo JfalseJ# fi
true
[me@linuxbox ~]$ if 44*66# then echo JtrueJ# else echo JfalseJ# fi
false
The stragest o+ the logical operators is the ternary operator. This operator Ewhich is
mo!ele! a+ter the oe i the C programmig lag"ageF per+orms a sta!aloe logical test.
9t ca %e "se! as a ki! o+ i+-the-else statemet. 9t acts o three arithmetic e3pressios
Estrigs woIt workF, a! i+ the +irst e3pressio is tr"e Eor o-DeroF the seco!
e3pressio is per+orme!. 7therwise, the thir! e3pressio is per+orme!. We ca try this
o the comma! lie,
[me@linuxbox ~]$ a$*
[me@linuxbox ~]$ 44a/1_55a2--a66
[me@linuxbox ~]$ echo 3a
#
[me@linuxbox ~]$ 44a/1_55a2--a66
[me@linuxbox ~]$ echo 3a
(
Aere we see a terary operator i actio. This e3ample implemets a toggle. Each time
the operator is per+orme!, the #al"e o+ the #aria%le a switches +rom Dero to oe or #ice
#ersa.
1/0
$rithmetic E#al"atio $! E3pasio
6lease ote that per+ormig assigmet withi the e3pressios is ot straight+orwar!.
Whe attempte!, bash will !eclare a error,
[me@linuxbox ~]$ a$*
[me@linuxbox ~]$ 44a/1_a5$12a-$166
bash: NN: aX#Fa8M#:a-M#: attem7ted assignment to non-variable Nerror
token is _-M#_P
This pro%lem ca %e mitigate! %y s"rro"!ig the assigmet e3pressio with
paretheses,
[me@linuxbox ~]$ 44a/1_4a5$1624a-$1666
&e3t, we see a more complete e3ample o+ "sig arithmetic operators i a script that
pro!"ces a simple ta%le o+ "m%ers,
^G6bin6bash
^ arith-loo7: scri7t to demonstrate arithmetic o7erators
finishedM(
aM(
7rintf _a]taEE!]taEE$]n_
7rintf _M]tMMMM]tMMMM]n_
until NNfinishedPPT do
bM$NNaEE!PP
cM$NNaEE$PP
7rintf _5d]t5d]t5d]n_ $a $b $c
NNaX#(F88a:NfinishedM#PPP
done
9 this script, we implemet a "til loop %ase! o the #al"e o+ the finished #aria%le.
9itially, the #aria%le is set to Dero Earithmetic +alseF a! we coti"e the loop "til it
%ecomes o-Dero. Withi the loop, we calc"late the s="are a! c"%e o+ the co"ter
#aria%le a. $t the e! o+ the loop, the #al"e o+ the co"ter #aria%le is e#al"ate!. 9+ it is
less tha te Ethe ma3im"m "m%er o+ iteratiosF, it is icremete! %y oe, else the
#aria%le finished is gi#e the #al"e o+ oe, makig finished arithmetically tr"e,
there%y termiatig the loop. :"ig the script gi#es this res"lt,
1/.
(2 8 Strigs $! &"m%ers
[me@linuxbox ~]$ arith-loop
a aEE! aEE$
M MMMM MMMM
( ( (
# # #
! % /
$ 0 !)
% #. .%
" !" #!"
. $. !#.
) %0 $%$
/ .% "#!
0 /# )!0
#( #(( #(((
bc #n #r0itrar, Precision Calculator Language
We ha#e see how the shell ca ha!le all maer o+ iteger arithmetic, %"t what i+ we
ee! to per+orm higher math or e#e 4"st "se +loatig poit "m%ersC The aswer is, we
caIt. $t least ot !irectly with the shell. To !o this, we ee! to "se a e3teral program.
There are se#eral approaches we ca take. Em%e!!ig 6erl or $WM programs is oe
possi%le sol"tio, %"t "+ort"ately, o"tsi!e the scope o+ this %ook.
$other approach is to "se a specialiDe! calc"lator program. 7e s"ch program +o"! o
most *i"3 systems is calle! bc.
The bc program rea!s a +ile writte i its ow C-like lag"age a! e3ec"tes it. $ bc
script may %e a separate +ile or it may %e rea! +rom sta!ar! ip"t. The bc lag"age
s"pports ="ite a +ew +eat"res icl"!ig #aria%les, loops, a! programmer-!e+ie!
+"ctios. We woIt co#er bc etirely here, 4"st eo"gh to get a taste. bc is well-
!oc"mete! %y its ma page.
*etIs start with a simple e3ample. WeIll write a bc script to a!! two pl"s two,
6E 4 ver1 sim7le bc scri7t E6
! 8 !
The +irst lie o+ the script is a commet. bc "ses the same syta3 +or commets as the C
programmig lag"age. Commets, which may spa m"ltiple lies, %egi with 6E a!
e! with E6.
1/2
%c 8 $ $r%itrary 6recisio Calc"lator *ag"age
Using bc
9+ we sa#e the bc script a%o#e as foo<bc, we ca r" it this way,
[me@linuxbox ~]$ bc foo.bc
bc #<(.<0%
Co71right #00#-#00%> #00)> #00/> !(((> !((%> !((. -ree *oft9are
-oundation> Anc<
This is free soft9are 9ith 4H*=3T&=U L ,4RR4LTU<
-or details t17e J9arrant1K<
%
9+ we look care+"lly, we ca see the res"lt at the #ery %ottom, a+ter the copyright message.
This message ca %e s"ppresse! with the -B E="ietF optio.
bc ca also %e "se! iteracti#ely,
[me@linuxbox ~]$ bc -,
2 5 2
%
,uit
Whe "se! iteracti#ely, we simply type the calc"latios we wish to per+orm, a! the
res"lts are imme!iately !isplaye!. The bc comma! Buit e!s the iteracti#e sessio.
9t is also possi%le to pass a script to bc #ia sta!ar! ip"t,
[me@linuxbox ~]$ bc / foo.bc
%
The a%ility to take sta!ar! ip"t meas that we ca "se here !oc"mets, here strigs,
a! pipes to pass scripts. This is a here strig e3ample,
[me@linuxbox ~]$ bc /// c252b
%
An 6xam5e S/ri5t
$s a real-worl! e3ample, we will costr"ct a script that per+orms a commo calc"latio,
1/(
(2 8 Strigs $! &"m%ers
mothly loa paymets. 9 the script %elow, we "se a here !oc"met to pass a script to
bc,
^G6bin6bash
^ loan-calc : scri7t to calculate monthl1 loan 7a1ments
:R@L4+&M$Nbasename $(P
usage NP [
cat XX- &-
3sage: $:R@L4+& :RALCA:4= ALT&R&*T +LTS*
,here:
:RALCA:4= is the amount of the loan<
ALT&R&*T is the 4:R as a number N)5 M (<()P<
+LTS* is the length of the loanKs term<
&-
\
if NN$^ GM $PPT then
usage
exit #
fi
7rinci7alM$#
interestM$!
monthsM$$
bc XX- &-
scale M #(
i M $interest 6 #!
7 M $7rinci7al
n M $months
a M 7 E NNi E NN# 8 iP ` nPP 6 NNN# 8 iP ` nP - #PP
7rint a> _]n_
&-
Whe e3ec"te!, the res"lts look like this,
[me@linuxbox ~]$ loan-calc 1"*** *.*CC" 1M*
#!)(<)!!!%0((((
This e3ample calc"lates the mothly paymet +or a [.(2,000 loa at /./2_ $6: +or .80
1/1
%c 8 $ $r%itrary 6recisio Calc"lator *ag"age
moths E.2 yearsF. &otice the precisio o+ the aswer. This is !etermie! %y the #al"e
gi#e to the special scale #aria%le i the bc script. $ +"ll !escriptio o+ the bc
scriptig lag"age is pro#i!e! %y the bc ma page. While its mathematical otatio is
slightly !i++eret +rom that o+ the shell Ebc more closely resem%les CF, most o+ it will %e
="ite +amiliar, %ase! o what we ha#e leare! so +ar.
Summing -p
9 this chapter, we ha#e leare! a%o"t may o+ the little thigs that ca %e "se! to get the
?real work@ !oe i scripts. $s o"r e3periece with scriptig grows, the a%ility to
e++ecti#ely maip"late strigs a! "m%ers will pro#e e3tremely #al"a%le. 7"r loan-
calc script !emostrates that e#e simple scripts ca %e create! to !o some really
"se+"l thigs.
$xtra Credit
While the %asic +"ctioality o+ the loan-calc script is i place, the script is +ar +rom
complete. 0or e3tra cre!it, try impro#ig the loan-calc script with the +ollowig
+eat"res,
0"ll #eri+icatio o+ the comma! lie arg"mets
$ comma! lie optio to implemet a ?iteracti#e@ mo!e that will prompt the
"ser to ip"t the pricipal, iterest rate, a! term o+ the loa.
$ %etter +ormat +or the o"tp"t.
;urther 1eading
The 8ash ,ackers /iki has a goo! !isc"ssio o+ parameter e3pasio,
http,--%ash-hackers.org-wiki-!ok".php-syta3-pe
The 8ash 9eference !anual co#ers this, too,
http,--www.g".org-so+tware-%ash-ma"al-%ashre+.htmlPShell-6arameter-
E3pasio
The /ikipedia has a goo! article !escri%ig %it operatios,
http,--e.wikipe!ia.org-wiki-JitOoperatio
a! a article o terary operatios,
http,--e.wikipe!ia.org-wiki-TeraryOoperatio
as well as a !escriptio o+ the +orm"la +or calc"latig loa paymets "se! i o"r
loan-calc script,
http,--e.wikipe!ia.org-wiki-$mortiDatioOcalc"lator
1/2
(K 8 $rrays
3+ )rra$s
9 the last chapter, we looke! at how the shell ca maip"late strig a! "m%ers. The
!ata types we ha#e looke! at so +ar are kow i comp"ter sciece circles as scalar
variablesK that is, #aria%les that cotai a sigle #al"e.
9 this chapter, we will look at aother ki! o+ !ata str"ct"re calle! a array, which hol!s
m"ltiple #al"es. $rrays are a +eat"re o+ #irt"ally e#ery programmig lag"age. The shell
s"pports them, too, tho"gh i a rather limite! +ashio. E#e so, they ca %e #ery "se+"l
+or sol#ig programmig pro%lems.
+hat #re #rra,s.
$rrays are #aria%les that hol! more tha oe #al"e at a time. $rrays are orgaiDe! like a
ta%le. *etIs cosi!er a sprea!sheet as a e3ample. $ sprea!sheet acts like a two-
dimensional array. 9t has %oth rows a! col"ms, a! a i!i#i!"al cell i the
sprea!sheet ca %e locate! accor!ig to its row a! col"m a!!ress. $ array %eha#es
the same way. $ array has cells, which are calle! elements, a! each elemet cotais
!ata. $ i!i#i!"al array elemet is accesse! "sig a a!!ress calle! a inde+ or
subscript.
>ost programmig lag"ages s"pport multi-dimensional arrays. $ sprea!sheet is a
e3ample o+ a m"lti-!imesioal array with two !imesios, wi!th a! height. >ay
lag"ages s"pport arrays with a ar%itrary "m%er o+ !imesios, tho"gh two a! three
!imesioal arrays are pro%a%ly the most commoly "se!.
$rrays i bash are limite! to a sigle !imesio. We ca thik o+ them as a sprea!sheet
with a sigle col"m. E#e with this limitatio, there are may applicatios +or them.
$rray s"pport +irst appeare! i bash #ersio 2. The origial )i3 shell program, sh,
!i! ot s"pport arrays at all.
Creating #n #rra,
$rray #aria%les are ame! 4"st like other bash #aria%les, a! are create! a"tomatically
whe they are accesse!. Aere is a e3ample,
1/K
Creatig $ $rray
[me@linuxbox ~]$ a(1)$foo
[me@linuxbox ~]$ echo 39a(1)>
foo
Aere we see a e3ample o+ %oth the assigmet a! access o+ a array elemet. With the
+irst comma!, elemet . o+ array a is assige! the #al"e ?+oo@. The seco! comma!
!isplays the store! #al"e o+ elemet .. The "se o+ %races i the seco! comma! is
re="ire! to pre#et the shell +rom attemptig pathame e3pasio o the ame o+ the
array elemet.
$ array ca also %e create! with the declare comma!,
[me@linuxbox ~]$ declare -a a
)sig the -a optio, this e3ample o+ declare creates the array a.
#ssigning Values To #n #rra,
Ral"es may %e assige! i oe o+ two ways. Sigle #al"es may %e assige! "sig the
+ollowig syta3,
name[subscript]Mvalue
where name is the ame o+ the array a! subscript is a iteger Eor arithmetic e3pressioF
greater tha or e="al to Dero. &ote that the +irst elemet o+ a array is s"%script Dero, ot
oe. value is a strig or iteger assige! to the array elemet.
>"ltiple #al"es may %e assige! "sig the +ollowig syta3,
nameMNvalue1 value2 <<<P
where name is the ame o+ the array a! value### are #al"es assige! se="etially to
elemets o+ the array, startig with elemet Dero. 0or e3ample, i+ we wate! to assig
a%%re#iate! !ays o+ the week to the array da1s, we co"l! !o this,
[me@linuxbox ~]$ da!s$4Fun Oon +ue Wed +hu 7ri Fat6
9t is also possi%le to assig #al"es to a speci+ic elemet %y speci+yig a s"%script +or each
#al"e,
[me@linuxbox ~]$ da!s$4(*)$Fun (1)$Oon (2)$+ue ()$Wed (A)$+hu
1//
(K 8 $rrays
(")$7ri (N)$Fat6
#ccessing #rra, $lements
So what are arrays goo! +orC J"st as may !ata maagemet tasks ca %e per+orme! with
a sprea!sheet program, may programmig tasks ca %e per+orme! with arrays.
*etIs cosi!er a simple !ata gatherig a! presetatio e3ample. We will costr"ct a
script that e3amies the mo!i+icatio times o+ the +iles i a speci+ie! !irectory. 0rom this
!ata, o"r script will o"tp"t a ta%le showig at what ho"r o+ the !ay the +iles were last
mo!i+ie!. S"ch a script co"l! %e "se! to !etermie whe a system is most acti#e. This
script, calle! hours, pro!"ces this res"lt,
[me@linuxbox ~]$ hours .
Sour -iles Sour -iles
---- ----- ---- -----
(( ( #! ##
(# # #$ )
(! ( #% #
($ ( #" )
(% # #. .
(" # #) "
(. . #/ %
() $ #0 %
(/ # !( #
(0 #% !# (
#( ! !! (
## " !$ (
Total files M /(
We e3ec"te the hours program, speci+yig the c"rret !irectory as the target. 9t
pro!"ces a ta%le showig, +or each ho"r o+ the !ay E0-2(F, how may +iles were last
mo!i+ie!. The co!e to pro!"ce this is as +ollows,
^G6bin6bash
^ hours : scri7t to count files b1 modification time
usage NP [
echo _usage: $Nbasename $(P director1_ DY!
\
1/8
$ccessig $rray Elemets
^ Check that argument is a director1
if [[ G -d $# ]]T then
usage
exit #
fi
^ AnitialiCe arra1
for i in [(<<!$\T do hours[i]M(T done
^ Collect data
for i in $Nstat -c 51 _$#_6E Q cut -c #!-#$PT do
jM$[i6^(\
NN88hours[j]PP
NN88countPP
done
^ 'is7la1 data
echo -e _Sour]t-iles]tSour]t-iles_
echo -e _----]t-----]t----]t-----_
for i in [(<<##\T do
jM$NNi 8 #!PP
7rintf _5(!d]t5d]t5(!d]t5d]n_ $i $[hours[i]\ $j $[hours[j]\
done
7rintf _]nTotal files M 5d]n_ $count
The script cosists o+ oe +"ctio EusageF a! a mai %o!y with +o"r sectios. 9 the
+irst sectio, we check that there is a comma! lie arg"met a! that it is a !irectory. 9+
it is ot, we !isplay the "sage message a! e3it.
The seco! sectio iitialiDes the array hours. 9t !oes this %y assigig each elemet a
#al"e o+ Dero. There is o special re="iremet to prepare arrays prior to "se, %"t o"r
script ee!s to es"re that o elemet is empty. &ote the iterestig way the loop is
costr"cte!. Jy employig %race e3pasio E[(<<!$\F, we are a%le to easily geerate a
se="ece o+ wor!s +or the for comma!.
The e3t sectio gathers the !ata %y r"ig the stat program o each +ile i the
!irectory. We "se cut to e3tract the two-!igit ho"r +rom the res"lt. 9si!e the loop, we
ee! to remo#e lea!ig Deros +rom the ho"r +iel!, sice the shell will try Ea! "ltimately
+ailF to iterpret #al"es ?00@ thro"gh ?09@ as octal "m%ers Esee Ta%le (2-.F. &e3t, we
icremet the #al"e o+ the array elemet correspo!ig with the ho"r o+ the !ay. 0ially,
we icremet a co"ter EcountF to track the total "m%er o+ +iles i the !irectory.
The last sectio o+ the script !isplays the cotets o+ the array. We +irst o"tp"t a co"ple
o+ hea!er lies a! the eter a loop that pro!"ces two col"ms o+ o"tp"t. *astly, we
o"tp"t the +ial tally o+ +iles.
1/9
(K 8 $rrays
#rra, ?perations
There are may commo array operatios. S"ch thigs as !eletig arrays, !etermiig
their siDe, sortig, etc. ha#e may applicatios i scriptig.
<ut5utting The 6ntire Contents <= An Array
The s"%scripts E a! @ ca %e "se! to access e#ery elemet i a array. $s with
positioal parameters, the @ otatio is the more "se+"l o+ the two. Aere is a
!emostratio,
[me@linuxbox ~]$ animals$4Ja dogJ Ja catJ Ja fishJ6
[me@linuxbox ~]$ for i in 39animals(*)># do echo 3i# done
a
dog
a
cat
a
fish
[me@linuxbox ~]$ for i in 39animals(V)># do echo 3i# done
a
dog
a
cat
a
fish
[me@linuxbox ~]$ for i in J39animals(*)>J# do echo 3i# done
a dog a cat a fish
[me@linuxbox ~]$ for i in J39animals(V)>J# do echo 3i# done
a dog
a cat
a fish
We create the array animals a! assig it three two-wor! strigs. We the e3ec"te +o"r
loops to see the a++ect o+ wor!-splittig o the array cotets. The %eha#ior o+ otatios
$[animals[E]\ a! $[animals[@]\ are i!etical "til they are ="ote!. The E
otatio res"lts i a sigle wor! cotaiig the arrayIs cotets, while the @ otatio
res"lts i three wor!s, which matches the arrays ?real@ cotets.
)etermining The ,umber <= Array 6ements
)sig parameter e3pasio, we ca !etermie the "m%er o+ elemets i a array i
m"ch the same way as +i!ig the legth o+ a strig. Aere is a e3ample,
180
$rray 7peratios
[me@linuxbox ~]$ a(1**)$foo
[me@linuxbox ~]$ echo 39Ua(V)> U number of arra! elements
#
[me@linuxbox ~]$ echo 39Ua(1**)> U length of element 1**
$
We create array a a! assig the strig ?+oo@ to elemet .00. &e3t, we "se parameter
e3pasio to e3amie the legth o+ the array, "sig the @ otatio. 0ially, we look at the
legth o+ elemet .00 which cotais the strig ?+oo@. 9t is iterestig to ote that while
we assige! o"r strig to elemet .00, %ash oly reports oe elemet i the array. This
!i++ers +rom the %eha#ior o+ some other lag"ages i which the ""se! elemets o+ the
array Eelemets 0-99F wo"l! %e iitialiDe! with empty #al"es a! co"te!.
2inding The Subs/ri5ts Used By An Array
$s bash allows arrays to cotai ?gaps@ i the assigmet o+ s"%scripts, it is sometimes
"se+"l to !etermie which elemets act"ally e3ist. This ca %e !oe with a parameter
e3pasio "sig the +ollowig +orms,
$[Garra[E]\
$[Garra[@]\
where array is the ame o+ a array #aria%le. *ike the other e3pasios that "se E a!
@> the @ +orm eclose! i ="otes is the most "se+"l, as it e3pa!s ito separate wor!s,
[me@linuxbox ~]$ foo$4(2)$a (A)$b (N)$c6
[me@linuxbox ~]$ for i in J39foo(V)>J# do echo 3i# done
a
b
c
[me@linuxbox ~]$ for i in J39Lfoo(V)>J# do echo 3i# done
!
%
.
Adding 6ements To The 6nd <= An Array
Mowig the "m%er o+ elemets i a array is o help i+ we ee! to appe! #al"es to the
e! o+ a array, sice the #al"es ret"re! %y the E a! @ otatios !o ot tell "s the
ma3im"m array i!e3 i "se. 0ort"ately, the shell pro#i!es "s with a sol"tio. Jy "sig
the 8M assigmet operator, we ca a"tomatically appe! #al"es to the e! o+ a array.
Aere, we assig three #al"es to the array foo a! the appe! three more.
18.
(K 8 $rrays
[me@linuxbox ~]$ foo$4a b c6
[me@linuxbox ~]$ echo 39foo(V)>
a b c
[me@linuxbox ~]$ foo5$4d e f6
[me@linuxbox ~]$ echo 39foo(V)>
a b c d e f
Sorting An Array
J"st as with sprea!sheets, it is o+te ecessary to sort the #al"es i a col"m o+ !ata. The
shell has o !irect way o+ !oig this, %"t it<s ot har! to !o with a little co!ig,
^G6bin6bash
^ arra1-sort : *ort an arra1
aMNf e d c b aP
echo _riginal arra1: $[a[@]\_
aOsortedMN$Nfor i in _$[a[@]\_T do echo $iT done Q sortPP
echo _*orted arra1: $[aOsorted[@]\_
Whe e3ec"te!, the script pro!"ces this,
[me@linuxbox ~]$ arra!-sort
riginal arra1: f e d c b a
*orted arra1: a b c d e f
The script operates %y copyig the cotets o+ the origial array EaF ito a seco! array
EaOsortedF with a tricky piece o+ comma! s"%stit"tio. This %asic techi="e ca %e
"se! to per+orm may ki!s o+ operatios o the array %y chagig the !esig o+ the
pipelie.
)eeting An Array
To !elete a array, "se the unset comma!,
[me@linuxbox ~]$ foo$4a b c d e f6
[me@linuxbox ~]$ echo 39foo(V)>
a b c d e f
182
$rray 7peratios
[me@linuxbox ~]$ unset foo
[me@linuxbox ~]$ echo 39foo(V)>
[me@linuxbox ~]$
unset may also %e "se! to !elete sigle array elemets,
[me@linuxbox ~]$ foo$4a b c d e f6
[me@linuxbox ~]$ echo 39foo(V)>
a b c d e f
[me@linuxbox ~]$ unset %foo(2)%
[me@linuxbox ~]$ echo 39foo(V)>
a b d e f
9 this e3ample, we !elete the thir! elemet o+ the array, s"%script 2. :emem%er, arrays
start with s"%script Dero, ot oeL &otice also that the array elemet m"st %e ="ote! to
pre#et the shell +rom per+ormig pathame e3pasio.
9terestigly, the assigmet o+ a empty #al"e to a array !oes ot empty its cotets,
[me@linuxbox ~]$ foo$4a b c d e f6
[me@linuxbox ~]$ foo$
[me@linuxbox ~]$ echo 39foo(V)>
b c d e f
$y re+erece to a array #aria%le witho"t a s"%script re+ers to elemet Dero o+ the array,
[me@linuxbox ~]$ foo$4a b c d e f6
[me@linuxbox ~]$ echo 39foo(V)>
a b c d e f
[me@linuxbox ~]$ foo$:
[me@linuxbox ~]$ echo 39foo(V)>
4 b c d e f
Summing -p
9+ we search the bash ma page +or the wor! ?array,@ we +i! may istaces o+ where
bash makes "se o+ array #aria%les. >ost o+ these are rather o%sc"re, %"t they may
pro#i!e occasioal "tility i some special circ"mstaces. 9 +act, the etire topic o+
arrays is rather "!er-"tiliDe! i shell programmig owig largely to the +act that the
18(
(K 8 $rrays
tra!itioal )i3 shell programs Es"ch as shF lacke! ay s"pport +or arrays. This lack o+
pop"larity is "+ort"ate %eca"se arrays are wi!ely "se! i other programmig lag"ages
a! pro#i!e a power+"l tool +or sol#ig may ki!s o+ programmig pro%lems.
$rrays a! loops ha#e a at"ral a++iity a! are o+te "se! together. The
for NNexprT exprT exprPP
+orm o+ loop is partic"larly well-s"ite! to calc"latig array s"%scripts.
;urther 1eading
http,--e.wikipe!ia.org-wiki-ScalarOEcomp"tigF
181
(/ 8 E3otica
3. !"otica
9 this, the +ial chapter o+ o"r 4o"rey, we will look at some o!!s a! e!s. While we
ha#e certaily co#ere! a lot o+ gro"! i the pre#io"s chapters, there are may bash
+eat"res that we ha#e ot co#ere!. >ost are +airly o%sc"re, a! "se+"l maily to those
itegratig bash ito a *i"3 !istri%"tio. Aowe#er, there are a +ew that, while ot i
commo "se, are help+"l +or certai programmig pro%lems. We will co#er them here.
6roup Commands #nd Su0shells
bash allows comma!s to %e gro"pe! together. This ca %e !oe i oe o+ two waysH
either with a group command or with a subshell. Aere are e3amples o+ the syta3 o+
each,
;ro"p Comma!,
[ command#T command!T [command$T <<<] \
S"%shell,
Ncommand#T command!T [command$T<<<]P
The two +orms !i++er i that a gro"p comma! s"rro"!s its comma!s with %races a! a
s"%shell "ses paretheses. 9t is importat to ote that, !"e to the way bash implemets
gro"p comma!s, the %races m"st %e separate! +rom the comma!s %y a space a! the
last comma! m"st %e termiate! with either a semicolo or a ewlie prior to the
closig %race.
So what are gro"p comma!s a! s"%shells goo! +orC While they ha#e a importat
!i++erece Ewhich we will get to i a mometF, they are %oth "se! to maage re!irectio.
*etIs cosi!er a script segmet that per+orms re!irectios o m"ltiple comma!s,
ls -l D out7ut<txt
echo _=isting of foo<txt_ DD out7ut<txt
cat foo<txt DD out7ut<txt
This is pretty straight+orwar!. Three comma!s with their o"tp"t re!irecte! to a +ile
182
(/ 8 E3otica
ame! out7ut<txt. )sig a gro"p comma!, we co"l! co!e this as +ollows,
[ ls -lT echo _=isting of foo<txt_T cat foo<txtT \ D out7ut<txt
)sig a s"%shell is similar,
Nls -lT echo _=isting of foo<txt_T cat foo<txtP D out7ut<txt
)sig this techi="e, we ha#e sa#e! o"rsel#es some typig, %"t where a gro"p comma!
or s"%shell really shies is with pipelies. Whe costr"ctig a pipelie o+ comma!s, it
is o+te "se+"l to com%ie the res"lts o+ se#eral comma!s ito a sigle stream. ;ro"p
comma!s a! s"%shells make this easy,
[ ls -lT echo _=isting of foo<txt_T cat foo<txtT \ Q l7r
Aere we ha#e com%ie! the o"tp"t o+ o"r three comma!s a! pipe! them ito the ip"t
o+ l7r to pro!"ce a prite! report.
'ro/ess Substitution
While they look similar a! ca %oth %e "se! to com%ie streams +or re!irectio, there is
a importat !i++erece %etwee gro"p comma!s a! s"%shells. Whereas a gro"p
comma! e3ec"tes all o+ its comma!s i the c"rret shell, a s"%shell Eas the ame
s"ggestsF e3ec"tes its comma!s i a chil! copy o+ the c"rret shell. This meas that the
e#iromet is copie! a! gi#e to a ew istace o+ the shell. Whe the s"%shell e3its,
the copy o+ the e#iromet is lost, so ay chages ma!e to the s"%shellIs e#iromet
Eicl"!ig #aria%le assigmetF is lost as well. There+ore, i most cases, "less a script
re="ires a s"%shell, gro"p comma!s are pre+era%le to s"%shells. ;ro"p comma!s are
%oth +aster a! re="ire less memory.
We saw a e3ample o+ the s"%shell e#iromet pro%lem i Chapter 29, whe we
!isco#ere! that a read comma! i a pipelie !oes ot work as we might it"iti#ely
e3pect. To recap, i+ we costr"ct a pipelie like this,
echo _foo_ Q read
echo $R&:=U
18K
;ro"p Comma!s $! S"%shells
The cotet o+ the R&:=U #aria%le is always empty %eca"se the read comma! is
e3ec"te! i a s"%shell, a! its copy o+ R&:=U is !estroye! whe the s"%shell termiates.
Jeca"se comma!s i pipelies are always e3ec"te! i s"%shells, ay comma! that
assigs #aria%les will eco"ter this iss"e. 0ort"ately, the shell pro#i!es a e3otic +orm
o+ e3pasio calle! process substitution that ca %e "se! to work aro"! this pro%lem.
6rocess s"%stit"tio is e3presse! i two ways,
0or processes that pro!"ce sta!ar! o"tp"t,
XNlistP
or, +or processes that itake sta!ar! ip"t,
DNlistP
where list is a list o+ comma!s.
To sol#e o"r pro%lem with read, we ca employ process s"%stit"tio like this,
read X XNecho _foo_P
echo $R&:=U
6rocess s"%stit"tio allows "s to treat the o"tp"t o+ a s"%shell as a or!iary +ile +or
p"rposes o+ re!irectio. 9 +act, sice it is a +orm o+ e3pasio, we ca e3amie its real
#al"e,
[me@linuxbox ~]$ echo XNecho _foo_P
6dev6fd6.$
Jy "sig echo to #iew the res"lt o+ the e3pasio, we see that the o"tp"t o+ the s"%shell
is %eig pro#i!e! %y a +ile ame! 6dev6fd6.$.
6rocess s"%stit"tio is o+te "se! with loops cotaiig read. Aere is a e3ample o+ a
read loop that processes the cotets o+ a !irectory listig create! %y a s"%shell,
^G6bin6bash
^ 7ro-sub : demo of 7rocess substitution
9hile read attr links o9ner grou7 siCe date time filenameT do
cat XX- &-
-ilename: $filename
18/
(/ 8 E3otica
*iCe: $siCe
9ner: $o9ner
@rou7: $grou7
+odified: $date $time
=inks: $links
4ttributes: $attr
&-
done X XNls -l Q tail -n 8!P
The loop e3ec"tes read +or each lie o+ a !irectory listig. The listig itsel+ is pro!"ce!
o the +ial lie o+ the script. This lie re!irects the o"tp"t o+ the process s"%stit"tio
ito the sta!ar! ip"t o+ the loop. The tail comma! is icl"!e! i the process
s"%stit"tio pipelie to elimiate the +irst lie o+ the listig, which is ot ee!e!.
Whe e3ec"te!, the script pro!"ces o"tp"t like this,
[me@linuxbox ~]$ pro.sub 0 head -n 2*
-ilename: addresses<ldif
*iCe: #%"%(
9ner: me
@rou7: me
+odified: !((0-(%-(! ##:#!
=inks: #
4ttributes: -r9-r--r--
-ilename: bin
*iCe: %(0.
9ner: me
@rou7: me
+odified: !((0-()-#( ():$#
=inks: !
4ttributes: dr9xr-xr-x
-ilename: bookmarks<html
*iCe: $0%!#$
9ner: me
@rou7: me
Traps
9 Chapter .., we saw how programs ca respo! to sigals. We ca a!! this capa%ility
to o"r scripts, too. While the scripts we ha#e writte so +ar ha#e ot ee!e! this
capa%ility Eowig to the +act that they ha#e #ery short e3ec"tio times, a! !o ot create
temporary +ilesF, larger a! more complicate! scripts may %ee+it +rom ha#ig a sigal
188
Traps
ha!lig ro"tie.
Whe we !esig a large, complicate! script, it is importat to cosi!er what happes i+
the "ser logs o++ or sh"ts !ow the comp"ter while the script is r"ig. Whe s"ch a
e#et occ"rs, a sigal will %e set to all a++ecte! processes a!, i t"r, the programs
represetig those processes ca per+orm actios to es"re a proper a! or!erly
termiatio o+ the program. *etIs say, +or e3ample, that we wrote a script that create! a
temporary +ile !"rig its e3ec"tio. 9 the co"rse o+ goo! !esig, we wo"l! ha#e the
script !elete the +ile whe the script +iishes its work. 9t wo"l! also %e smart to ha#e the
script !elete the +ile i+ a sigal is recei#e! i!icatig that the program was goig to %e
termiate! premat"rely.
bash pro#i!es a mechaism +or this p"rpose kow as a trap. Traps are implemete!
with the appropriately ame! %"ilti comma!, tra7. tra7 "ses the +ollowig syta3,
tra7 argument signal [signal<<<]
where argument is a strig which will %e rea! a! treate! as a comma! a! signal is the
speci+icatio o+ a sigal that will trigger the e3ec"tio o+ the iterprete! comma!.
Aere is a simple e3ample,
^G6bin6bash
^ tra7-demo : sim7le signal handling demo
tra7 _echo KA am ignoring 1ou<K_ *A@ALT *A@T&R+
for i in [#<<"\T do
echo _Ateration $i of "_
slee7 "
done
This script !e+ies a trap that will e3ec"te a echo comma! each time either the
S9;9&T or S9;TE:> sigal is recei#e! while the script is r"ig. E3ec"tio o+ the
program looks like this whe the "ser attempts to stop the script %y typig Ctrl-c,
[me@linuxbox ~]$ trap-demo
Ateration # of "
Ateration ! of "
A am ignoring 1ou<
Ateration $ of "
A am ignoring 1ou<
Ateration % of "
Ateration " of "
189
(/ 8 E3otica
$s we ca see, each time the "ser attempts to iterr"pt the program, the message is
prite! istea!.
Costr"ctig a strig to +orm a "se+"l se="ece o+ comma!s ca %e awkwar!, so it is
commo practice to speci+y a shell +"ctio as the comma!. 9 this e3ample, a separate
shell +"ctio is speci+ie! +or each sigal to %e ha!le!,
^G6bin6bash
^ tra7-demo! : sim7le signal handling demo
exitOonOsignalO*A@ALT NP [
echo _*cri7t interru7ted<_ !DY#
exit (
\
exitOonOsignalO*A@T&R+ NP [
echo _*cri7t terminated<_ !DY#
exit (
\
tra7 exitOonOsignalO*A@ALT *A@ALT
tra7 exitOonOsignalO*A@T&R+ *A@T&R+
for i in [#<<"\T do
echo _Ateration $i of "_
slee7 "
done
This script +eat"res two tra7 comma!s, oe +or each sigal. Each trap, i t"r,
speci+ies a shell +"ctio to %e e3ec"te! whe the partic"lar sigal is recei#e!. &ote the
icl"sio o+ a exit comma! i each o+ the sigal ha!lig +"ctios. Witho"t a
exit, the script wo"l! coti"e a+ter completig the +"ctio.
Whe the "ser presses Ctrl-c !"rig the e3ec"tio o+ this script, the res"lts look like
this,
[me@linuxbox ~]$ trap-demo2
Ateration # of "
Ateration ! of "
*cri7t interru7ted<
190
Traps
Temporar, ;iles
7e reaso sigal ha!lers are icl"!e! i scripts is to remo#e temporary +iles
that the script may create to hol! iterme!iate res"lts !"rig e3ec"tio. There is
somethig o+ a art to amig temporary +iles. Tra!itioally, programs o )i3-
like systems create their temporary +iles i the 6tm7 !irectory, a share! !irectory
ite!e! +or s"ch +iles. Aowe#er, sice the !irectory is share!, this poses certai
sec"rity cocers, partic"larly +or programs r"ig with s"per"ser pri#ileges.
$si!e +rom the o%#io"s step o+ settig proper permissios +or +iles e3pose! to all
"sers o+ the system, it is importat to gi#e temporary +iles o-pre!icta%le
+ileames. This a#oi!s a e3ploit kow as a temp race attack. 7e way to
create a o-pre!icta%le E%"t still !escripti#eF ame is to !o somethig like this,
tem7fileM6tm76$Nbasename $(P<$$<$R4L'+
This will create a +ileame cosistig o+ the programIs ame, +ollowe! %y its
process 9' E69'F, +ollowe! %y a ra!om iteger. &ote, howe#er, that the
$R4L'+ shell #aria%le oly ret"rs a #al"e i the rage o+ .-(2/K/, which is
ot a #ery large rage i comp"ter terms, so a sigle istace o+ the #aria%le is
ot s"++iciet to o#ercome a !etermie! attacker.
$ %etter way is to "se the mktem7 program Eot to %e co+"se! with the mktem7
sta!ar! li%rary +"ctioF to %oth ame a! create the temporary +ile. The
mktem7 program accepts a template as a arg"met that is "se! to %"il! the
+ileame. The template sho"l! icl"!e a series o+ ?G@ characters which are
replace! %y a correspo!ig "m%er o+ ra!om letters a! "m%ers. The loger
the series o+ ?G@ characters, the loger the series o+ ra!om characters. Aere is
a e3ample,
tem7fileM$Nmktem7 6tm76foobar<$$<VVVVVVVVVVP
This creates a temporary +ile a! assigs its ame to the #aria%le tem7file.
The ?G@ characters i the template are replace! with ra!om letters a! "m%ers
so that the +ial +ileame Ewhich, i this e3ample, also icl"!es the e3pa!e!
#al"e o+ the special parameter $$ to o%tai the 69'F might %e somethig like,
6tm76foobar<."0$<3Iuv+.."%
While the mktem7 ma page states that mktem7 makes a temporary +ileame,
mktem7 also creates the +ile as well.
0or scripts that are e3ec"te! %y reg"lar "sers, it may %e wise to a#oi! the "se o+
the 6tm7 !irectory a! create a !irectory +or temporary +iles withi the "serIs
home !irectory, with a lie o+ co!e s"ch as this,
19.
(/ 8 E3otica
[[ -d $S+&6tm7 ]] QQ mkdir $S+&6tm7
#s,nchronous $xecution
9t is sometimes !esira%le to per+orm more tha oe task at the same time. We ha#e see
how all mo!er operatig systems are at least m"ltitaskig i+ ot m"lti-"ser as well.
Scripts ca %e costr"cte! to %eha#e i a m"ltitaskig +ashio.
)s"ally this i#ol#es la"chig a script that, i t"r, la"ches oe or more chil! scripts
that per+orm a a!!itioal task while the paret script coti"es to r". Aowe#er, whe a
series o+ scripts r"s this way, there ca %e pro%lems keepig the paret a! chil!
coor!iate!. That is, what i+ the paret or chil! is !epe!et o the other, a! oe script
m"st wait +or the other to +iish its task %e+ore +iishig its owC
bash has a %"ilti comma! to help maage asynchronous e+ecution s"ch as this. The
9ait comma! ca"ses a paret script to pa"se "til a speci+ie! process Ei.e., the chil!
scriptF +iishes.
9ait
We will !emostrate the 9ait comma! +irst. To !o this, we will ee! two scripts, a
paret script,
^G6bin6bash
^ as1nc-7arent : 4s1nchronous execution demo N7arentP
echo _:arent: starting<<<_
echo _:arent: launching child scri7t<<<_
as1nc-child Y
7idM$G
echo _:arent: child N:A'M $7idP launched<_
echo _:arent: continuing<<<_
slee7 !
echo _:arent: 7ausing to 9ait for child to finish<<<_
9ait $7id
echo _:arent: child is finished< Continuing<<<_
192
$sychroo"s E3ec"tio
echo _:arent: 7arent is done< &xiting<_
a! a chil! script,
^G6bin6bash
^ as1nc-child : 4s1nchronous execution demo NchildP
echo _Child: child is running<<<_
slee7 "
echo _Child: child is done< &xiting<_
9 this e3ample, we see that the chil! script is #ery simple. The real actio is %eig
per+orme! %y the paret. 9 the paret script, the chil! script is la"che! a! p"t ito the
%ackgro"!. The process 9' o+ the chil! script is recor!e! %y assigig the 7id #aria%le
with the #al"e o+ the $G shell parameter, which will always cotai the process 9' o+ the
last 4o% p"t ito the %ackgro"!.
The paret script coti"es a! the e3ec"tes a 9ait comma! with the 69' o+ the chil!
process. This ca"ses the paret script to pa"se "til the chil! script e3its, at which poit
the paret script cocl"!es.
Whe e3ec"te!, the paret a! chil! scripts pro!"ce the +ollowig o"tp"t,
[me@linuxbox ~]$ as!nc-parent
:arent: starting<<<
:arent: launching child scri7t<<<
:arent: child N:A'M .)%#P launched<
:arent: continuing<<<
Child: child is running<<<
:arent: 7ausing to 9ait for child to finish<<<
Child: child is done< &xiting<
:arent: child is finished< Continuing<<<
:arent: 7arent is done< &xiting<
7amed Pipes
9 most )i3-like systems, it is possi%le to create a special type o+ +ile calle! a named
pipe. &ame! pipes are "se! to create a coectio %etwee two processes a! ca %e
"se! 4"st like other types o+ +iles. They are ot that pop"lar, %"t theyIre goo! to kow
a%o"t.
19(
(/ 8 E3otica
There is a commo programmig architect"re calle! client-server, which ca make "se o+
a comm"icatio metho! s"ch as ame! pipes, as well as other ki!s o+ interprocess
communication s"ch as etwork coectios.
The most wi!ely "se! type o+ cliet-ser#er system is, o+ co"rse, is a we% %rowser
comm"icatig with a we% ser#er. The we% %rowser acts as the cliet, makig re="ests
to the ser#er a! the ser#er respo!s to the %rowser with we% pages.
&ame! pipes %eha#e like +iles, %"t act"ally +orm +irst-i +irst-o"t E0907F %"++ers. $s with
or!iary E"ame!F pipes, !ata goes i oe e! a! emerges o"t the other. With ame!
pipes, it is possi%le to set "p somethig like this,
process1 D named_pipe
a!
process2 X named_pipe
a! it will %eha#e as i+,
process1 Q process2
Setting U5 A ,amed 'i5e
0irst, we m"st create a ame! pipe. This is !oe "sig the mkfifo comma!,
[me@linuxbox ~]$ mkfifo pipe1
[me@linuxbox ~]$ ls -l pipe1
7r9-r--r-- # me me ( !((0-()-#) (.:%# 7i7e#
Aere we "se mkfifo to create a ame! pipe calle! 7i7e#. )sig ls, we e3amie the
+ile a! see that the +irst letter i the attri%"tes +iel! is ?p@, i!icatig that it is a ame!
pipe.
Using ,amed 'i5es
To !emostrate how the ame! pipe works, we will ee! two termial wi!ows Eor
alterately, two #irt"al cosolesF. 9 the +irst termial, we eter a simple comma! a!
re!irect its o"tp"t to the ame! pipe,
[me@linuxbox ~]$ ls -l & pipe1
$+ter pressig the &nter key, the comma! will appear to hag. This is %eca"se there is
191
&ame! 6ipes
othig recei#ig !ata +rom the other e! o+ the pipe yet. Whe this occ"rs, it is sai! that
the pipe is blocked. This co!itio will clear oce we attach a process to the other e!
a! it %egis to rea! ip"t +rom the pipe. )sig the seco! termial wi!ow, we eter
this comma!,
[me@linuxbox ~]$ cat / pipe1
a! the !irectory listig pro!"ce! +rom the +irst termial wi!ow appears i the seco!
termial as the o"tp"t +rom the cat comma!. The ls comma! i the +irst termial
s"ccess+"lly completes oce it is o loger %locke!.
Summing -p
Well, we ha#e complete! o"r 4o"rey. The oly thig le+t to !o ow is practice, practice,
practice. E#e tho"gh we co#ere! a lot o+ gro"! i o"r trek, we %arely scratche! the
s"r+ace as +ar as the comma! lie goes. There are still tho"sa!s o+ comma! lie
programs le+t to %e !isco#ere! a! e4oye!. Start !iggig aro"! i 6usr6bin a!
yo"Ill seeL
;urther 1eading
The ?Compo"! Comma!s@ sectio o+ the %ash ma page cotais a +"ll
!escriptio o+ gro"p comma! a! s"%shell otatios.
The EG6$&S97&S sectio o+ the %ash ma page cotais a s"%sectio o+ process
s"%stit"tio.
The )dvanced 8ash-Scripting @uide also has a !isc"ssio o+ process s"%stit"tio,
http,--tl!p.org-*'6-a%s-html-process-s"%.html
(inu+ Dournal has two goo! articles o ame! pipes. The +irst, +rom Septem%er
.99/,
http,--www.li"34o"ral.com-article-2.2K
a! the seco!, +rom >arch 2009,
http,--www.li"34o"ral.com-cotet-"sig-ame!-pipes-+i+os-%ash
192
9!e3
Inde"
#
a2ps comma!...................................................((/
a%sol"te pathames...............................................K
alias comma!.............................................2K, .(0
aliases.....................................................18, 2K, .28
$merica &atioal Sta!ar!s 9stit"te Esee $&S9F
.............................................................................K1
$merica Sta!ar! Co!e +or 9+ormatio
9terchage Esee $SC99F.......................................21
achors...............................................................22.
aoymo"s 0T6 ser#ers.....................................201
$&S9...................................................................K1
$&S9 escape co!es.....................................K1, .K8
$&S9.S5S...........................................................K1
$pache we% ser#er..............................................2(
apropos comma!................................................2(
apt-cache comma!............................................/(
apt-get comma!............................................../2p.
aptit"!e comma!.............................................../2
archi#ig............................................................2(1
arithmetic e3pasio............../2, 80, (/., 12/, 1K(
arithmetic e3pressios................../2, 121, 1K(, 1KK
arithmetic operators...................................../2, 1K1
arithmetic tr"th tests...................................(92, 1K(
arrays........................................................................
appe! #al"es to the e!..............................18.
assigig #al"es............................................1//
creatig.........................................................1/K
!eletig.........................................................182
!etermie "m%er o+ elemets.....................180
+i!ig "se! s"%scripts.................................18.
i!e3.............................................................1/K
m"lti-!imesioal.........................................1/K
rea!ig #aria%les ito...................................101
sortig...........................................................182
s"%script.......................................................1/K
two-!imesioal...........................................1/K
$SC99.............................82, 8K, 222, 222, 2K/, ((/
%ell character.................................................K.
carriage ret"r..............................................2/.
collatio or!er..............................222, 22/, (9.
cotrol co!es..................................82, 222, ((.
gro++ o"tp"t !ri#er........................................(21
lie+ee! character.........................................2/.
"ll character................................................222
prita%le characters......................................222
te3t..................................................................21
aspell comma!.................................................(0(
assem%ler............................................................(12
assem%ly lag"age.............................................(12
assigmet operators..........................................1KK
asychroo"s e3ec"tio.....................................192
a"!io C's....................................................81, .92
$WM programmig lag"age....................(0(, 1/2
<
%ack re+ereces........................................2K/, 298p.
%ackslash escape se="eces.................................8(
%ackslash-escape! special characters..................K0
%ack"ps, icremetal..........................................2(8
%aseame comma!...........................................11.
%ash...............................................................0, .28
%ash..........................................................................
ma page........................................................2(
%asic reg"lar e3pressios 228, 2KKp., 29K, (00, (.0
%c comma!.......................................................1/2
Jerkeley So+tware 'istri%"tio.........................((2
%g comma!........................................................2.
%iary........................................98p., .02, (12, 1K1
%it mask...............................................................0.
%it operators.......................................................1K8
Jo"re, Ste#e..................................................0, .(
%race e3pasio....................................../K, 80, 122
%rachig............................................................(82
%reak comma!..........................................1.2, 11K
19K
9!e3
%roke liks..........................................................1K
JS' style.............................................................K
%"++erig..............................................................8K
%"gs............................................................122, 12/
%"il! e#iromet..............................................(20
%Dip2 comma!..................................................2((
!
C programmig lag"age...........(12, 121, 1K/, 1/0
CVV....................................................................(12
cal comma!.........................................................2
cacel comma!.................................................(12
carriage ret"r......22, 82p., .K., 222p., 2/0, (02p.,
((1
case compo"! comma!..................................1(2
cat comma!................................................K(, 2/0
c! comma!....................................................K, .8
C'-:7>s................................................8(p., .92
c!recor! comma!..............................................9K
c!rtools................................................................9K
character classes....(2p., 222, 221p., 22/, 2K., 29(,
(0(
character rages................................((, 22(p., (0(
chgrp comma!...................................................09
chil! process.........................................................(
chmo! comma!..................................9/, ..0, (K0
chow comma!.........................................0/, ..0
chroological sortig.........................................2//
clearte3t......................................................201, 20/
cliet-ser#er architect"re....................................191
C7J7* programmig lag"age........................(12
collatio or!er.....................(0, 222, 22/, 29(, (9.
$SC99...................................................22/, (9.
!ictioary......................................................222
tra!itioal.....................................................22/
comm comma!.................................................288
comma! history.............................................., 88
comma! lie...........................................................
arg"mets.....................................................1(/
e!itig........................................................., 81
e3pasio......................................................../2
history........................................................., 89
iter+aces....................................................(, (1
comma! optios.................................................2.
comma! s"%stit"tio............................/8, 80, 122
comma!s................................................................
arg"mets...............................................2., 1(/
!etermiig type.............................................19
!oc"metatio................................................20
e3ec"ta%le program +iles........................18, (12
e3ec"tig as aother "ser..............................01
log optios....................................................2.
optios............................................................2.
commets............................(2, .(8, (02, (29, 12/
Commo )i3 6ritig System.................(((, (1(
compariso operators.........................................1K9
compiler.............................................................(12
compilig...........................................................(11
completios..........................................................8K
compo"! comma!s..............................................
case...............................................................1(2
+or.................................................................12.
i+...................................................................(82
"til...............................................................1.K
while.............................................................1.(
EE FF................................................(92, 109, 1K(
aa bb........................................................(9(, 109
compressio algorithms.....................................2(.
co!itioal e3pressios..............................100, 12(
co+ig"ratio +iles..................................22, 2/, .28
co+ig"re comma!...........................................(20
costats.............................................................(/0
coti"e comma!.............................................1.2
cotrol characters........................................K., 2/0
cotrol co!es................................................82, 222
cotrol operators......................................................
^^........................................................(99, 109
ZZ....................................................................(99
cotrollig termial..............................................1
C7659&;..........................................................(19
copyig a! pastig.................................................
i #im.............................................................19
o the comma! lie......................................82
with G Wi!ow System....................................
core"tils package.........................2., 21p., 28(, (0/
co"tig wor!s i a +ile........................................K/
cp comma!...................................(1, 1., .(2, 2..
C6)...........................................................(p., (11
cro 4o%..............................................................2.2
crosswor! p"DDles..............................................22.
csplit comma!..................................................(08
C)6S..........................................................(((, (1(
c"rret workig !irectory.....................................2
c"rsor mo#emet..................................................81
c"t comma!..............................................280, 1K2
/
!aemo programs.........................................(, .2(
!ata compressio................................................2(0
!ata re!"!acy.................................................2(0
19/
9!e3
!ata #ali!atio....................................................(9(
!ate comma!.......................................................2
!ate +ormats........................................................2//
!! comma!........................................................91
'e%ia................................................................./0
'e%ia Style E.!e%F............................................./.
!e%"ggig...................................................(8., 12/
!e+esi#e programmig.............................12(, 12/
!elimiters..............................................8., 2/2, 2/8
!epe!ecies.............................................../2, (2(
!esig.........................................................122, 12/
!e#ice !ri#ers............................................../8, (12
!e#ice ames.......................................................8K
!e#ice o!es.........................................................2/
!+ comma!..................................................2, (8(
!ictio................................................................(1K
!ictioary collatio or!er...................................222
!i++ comma!.....................................................288
'igital :estrictios >aagemet E':>F.........../2
!irectories.................................................................
archi#ig.......................................................2(1
chagig..........................................................K
copyig...........................................................(1
creatig.....................................................(1, 10
c"rret workig...............................................2
!eletig.....................................................(/, 12
hierarchical......................................................1
home.................................................28, 92, (8(
listig..............................................................20
mo#ig......................................................(K, 12
a#igatig........................................................1
7*'O6W' #aria%le......................................(0
paret...............................................................2
6$TA #aria%le...............................................(0
6W' #aria%le................................................(.
remo#ig...................................................(/, 12
reamig...................................................(K, 12
root..................................................................1
share!............................................................09
sticky %it........................................................0(
sychroiDig...............................................212
tras+errig o#er a etwork..........................212
#iewig cotets..............................................2
!isk partitios......................................................8.
'9S6*$5 #aria%le..............................................(0
'olphi................................................................((
!os2"i3 comma!............................................2/.
!o"%le ="otes......................................................./9
!pkg comma!..................................................../2
!" comma!...............................................2/(, (8(
'yamic Aost Co+ig"ratio 6rotocol E'AC6F 20(
$
echo comma!...................................../2, .29, (KK
-e optio..........................................................8(
- optio.......................................................102
e!ge a! corer cases.........................................12K
E'9T7: #aria%le................................................(0
e++ecti#e gro"p 9'...............................................0(
e++ecti#e "ser 9'..........................................0(, ..1
eli+ statemet......................................................(92
email...................................................................2K9
em%e!!e! systems.............................................(12
empty #aria%les..................................................128
ecrypte! t"els...............................................2.0
ecryptio..........................................................291
e! o+ +ile.....................................................K1, (/(
e!less loop........................................................1.K
escript comma!..............................................(10
e#iromet........................................02, .28, 108
aliases............................................................28
esta%lishig....................................................(.
e3amiig......................................................28
logi shell......................................................(.
shell +"ctios...............................................28
shell #aria%les................................................28
start"p +iles....................................................(.
s"%shells.......................................................18K
#aria%les........................................................28
e= comma!.....................................................(22
e3ec"ta%le +iles...................................................(2.
e3ec"ta%le program +iles..............................18, (12
e3ec"ta%le programs................................................
!etermiig locatio.......................................19
6$TA #aria%le...............................................(0
e3it comma!........................................(, (90, 1..
e3it stat"s...................................................(8K, (90
e3pa! comma!...............................................28(
e3pasios............................................................/2
arithmetic........................../2, 80, (/., 12/, 1K(
%race................................................./K, 80, 122
comma! s"%stit"tio....................../8, 80, 122
!elimiters........................................................8.
errors res"ltig +rom.....................................12.
history.......................................................89, 9.
parameter..........................//, 80, (K9, (/2, 12/
pathame........................................../(, 80, 122
til!e.........................................................../1, 80
wor!-splittig............................................../9p.
e3pressios...............................................................
arithmetic......................../2, 121, 1K(, 1KK, 1//
co!itioal............................................100, 12(
198
9!e3
e3t(......................................................................92
e3te!e! reg"lar e3pressios.............................228
E3tesi%le >ark"p *ag"age............................2K9
9
+alse comma!...................................................(8/
+!+ormat comma!..............................................91
+!isk comma!....................................................89
+g comma!........................................................2.
0907...................................................................191
+ile comma!.......................................................2(
+ile !escriptor.......................................................K2
+ile system corr"ptio.........................................8K
0ile Tras+er 6rotocol E0T6F..............................201
+ileames............................................................222
case sesiti#e...................................................8
em%e!!e! spaces i.................................9, 2K1
e3tesios........................................................9
hi!!e..............................................................8
+iles...........................................................................
access..............................................................91
archi#ig...............................................2(1, 210
attri%"tes.........................................................92
%lock special...................................................9K
%lock special !e#ice.....................................2.K
chagig +ile mo!e.........................................9/
chagig ower a! gro"p ower.................0/
character special.............................................9K
character special !e#ice................................2.K
compressio..................................................2(0
co+ig"ratio..................................22, .28, 2K8
copyig.....................................................(1, 1.
copyig o#er a etwork................................20(
creatig empty................................................K.
!e%................................................................./0
!eletig.............................................(/, 12, 222
!etermiig cotets......................................2(
!e#ice o!es...................................................2/
e3ec"tio access.............................................92
e3pressios...................................................(88
+i!ig..........................................................2.(
hi!!e..............................................................8
iso image....................................................92p.
listig.........................................................2, 20
mo!e...............................................................9K
mo#ig......................................................(K, 12
ower..............................................................9/
permissios.....................................................91
rea! access......................................................92
reg"lar...........................................................2.K
remo#ig...................................................(/, 12
reamig...................................................(K, 12
rpm................................................................/0
share! li%rary..................................................28
start"p............................................................(.
sticky %it........................................................0(
sym%olic liks..............................................2.K
sychroiDig...............................................212
temporary.....................................................19.
te3t..................................................................21
tras+errig o#er a etwork..........20(, 2(9, 212
tr"catig........................................................K.
type.................................................................92
#iewig cotets.............................................21
write access....................................................92
+ilters....................................................................KK
+i! comma!............................................2.2, 2(8
+i!"tils package................................................229
0ire+o3................................................................(K2
+irewalls..............................................................200
+irst-i +irst-o"t...................................................191
+loppy !isks.........................................80, .8/, .9(
+low cotrol..............................................................
%rachig......................................................(82
case compo"! comma!............................1(2
eli+ statemet................................................(92
e!less loop..................................................1.K
+or compo"! comma!...............................12.
+or loop.........................................................12.
+"ctio statemet........................................(/8
i+ compo"! comma!.................................(82
loopig..........................................................1.2
me"-!ri#e.................................................109
m"ltiple-choice !ecisios.............................1(2
rea!ig +iles with while a! "til loops.......1./
termiatig a loop.........................................1.2
traps..............................................................189
"til loop......................................................1.K
while loop.....................................................1.(
+mt comma!.....................................................(.(
+oc"s policy...........................................................2
+ol! comma!....................................................(.(
+or compo"! comma!....................................12.
+or loop...............................................................12.
0oresight............................................................./0
0ortra programmig lag"age..................(12, 121
+ree comma!................................................(, .82
0ree So+tware 0o"!atio...................................Kp.
+sck comma!.....................................................9(
+tp comma!...............................201, 2.., (1/, (/1
0T6 ser#ers.................................................201, (/1
199
9!e3
0)&C&$>E #aria%le.......................................112
+"ctio statemet..............................................(/8
=
gcc......................................................................(1K
ge!it comma!.............................................9, .(2
geisoimage comma!........................................92
;etoo................................................................./0
getopts comma!...............................................120
;hostscript.........................................................(((
gi!........................................................................91
glo%al #aria%les..................................................(80
glo%%ig...............................................................(2
;&7>E............................0, ((, 1/, .0., .(2, 2.2
gome-termial.....................................................0
;&) %i"tils package........................................12(
;&) C Compiler...............................................(1K
;&) core"tils package...............2., 21p., 28(, (0/
;&) +i!"tils package......................................229
;&) 6ro4ect..................Kp., 2., 229, (0/, (1K, (18
;&) 6ro4ect.............................................................
i+o comma!.................................................21
;&)-*i"3........................................................2, /
graphical "ser iter+aces........................................(
grep comma!......................................K/, 21/, 10K
gro++....................................................................(22
gro"p comma!s................................................182
gro"ps...................................................................91
e++ecti#e gro"p 9'.........................................0(
gi!...................................................................91
primary gro"p 9'............................................91
setgi!.............................................................0(
;)9.................................(, .., ((, 1/, 81, .00, .(.
g"Dip comma!................................................2(.
gDip comma!..............................................22, 2(.
H
har! !isks............................................................80
har! liks..................................................(0, (9, 1(
creatig...........................................................1(
listig..............................................................11
hea! comma!.....................................................K8
hea!er +iles.........................................................(19
hello worl! program...........................................(29
help comma!......................................................20
here !oc"mets..................................................(/(
here strigs.........................................................10/
he3a!ecimal.................................................98, 1K1
hi!!e +iles......................................................8, /1
hierarchical !irectory str"ct"re.............................1
high-le#el programmig lag"ages....................(12
history......................................................................
e3pasio..................................................89, 9.
searchig.........................................................89
history comma!..................................................89
home !irectories...................................................28
root acco"t....................................................28
-etc-passw!.....................................................92
home !irectory..........................2, .8, /1, .02, .(0
A7>E #aria%le...................................................(0
host ame............................................................K.
AT>*........................2K9, (0(, (2(, (K2, (/2, (//
Ayperte3t >ark"p *ag"age.............................2K9
)
9-7 re!irectio Esee re!irectioF...........................29
i! comma!..........................................................91
9'E......................................................................8/
i+ compo"! comma!.......................((, 12., 1(2
90S #aria%le........................................................10K
9>C6 ECA7O:ET)EST..................................200
icremetal %ack"ps...........................................2(8
i+o +iles...............................................................21
iit........................................................................(
iit scripts.............................................................(
io!es...................................................................1(
9&ST$**...........................................................(19
istallatio wiDar!.............................................../.
itegers.....................................................................
arithmetic................................................/2, 1/2
!i#isio.................................................../K, 1K2
e3pressios...................................................(92
iteracti#ity........................................................10.
9teral 0iel! Separator......................................10K
iterprete! lag"ages.........................................(12
iterprete! programs..........................................(1K
iterpreter...........................................................(1K
iso images.........................................................92p.
iso9KK0........................................................81, .9K
>
4o% cotrol...........................................................20
4o% "m%ers.........................................................20
4o%spec................................................................2.
4oi comma!.....................................................282
Joliet e3tesios..................................................9K
Joy, Jill...............................................................1.
200
9!e3
3
kate comma!.....................................................(2
M'E..................................0, ((, 1/, .0., .(2, 2.2
ke!it comma!....................................................(2
kerel......2, K, 2/, 22, ..(, .2(, ./8, .8/, 29., (21
key +iel!s............................................................2/2
kill comma!......................................................22
killall comma!..................................................22
killig te3t............................................................82
M"th, 'oal!....................................................(22
Mo="eror....................................((, .0., 2.2, (K2
kosole..................................................................0
kwrite comma!...........................................9, .(2
L
*$&; #aria%le....................................(0, 222, 22/
less comma!.................................21, KK, 212, 2K2
l+tp comma!.....................................................20K
li%raries..............................................................(12
lie coti"atio character.........................(02, (K(
lie e!itors...........................................................1.
liker..................................................................(12
likig................................................................(12
liks..........................................................................
%roke.............................................................1K
creatig...........................................................(9
har!...........................................................(0, (9
sym%olic...................................................(0, 10
*i"3 comm"ity................................................/0
*i"3 !istri%"tios............................................../0
Cet7S................................................../., (10
'e%ia................................................/0p., (11
0e!ora.........................................2, 91, ./., (10
0oresight......................................................../0
;etoo.........................................................../0
*ispire........................................................../.
>a!ri#a......................................................./.
7peS)SE...............................................2, ./.
packagig systems........................................./0
6C*i"37S.................................................../.
:e! Aat Eterprise *i"3............................../.
Slackware....................................................../0
)%"t"...........................................2, ./0p., (10
Ga!ros........................................................./.
*i"3 0ilesystem Aierarchy Sta!ar!. . .2K, (0, (K2
*i"3 kerel. .2, K, 2/, 22, ..(, .2(, ./8, .8/, 29.,
(21
*i"3 kerel.............................................................
!e#ice !ri#ers................................................/8
literal characters.................................................219
li#e C's.................................................................2
l comma!....................................................(9, 1(
local #aria%les....................................................(80
locale..........................................222, 22/, 29(, (9.
locale comma!.................................................22/
localhost.............................................................20/
locate comma!.........................................2.(, 2K2
logical errors......................................................12(
logical operatios...............................................(9K
logical operators.................................................2.9
logical relatioships...................................2.8, 222
logi prompt..................................................(, 202
logi shell.............................................92, .02, .(.
log optios..........................................................2.
loop%ack iter+ace..............................................20(
loopig...............................................................1.2
loops...................................12(, 1K2, 1K8, 181, 18/
lossless compressio..........................................2(.
lossy compressio..............................................2(.
lp comma!........................................................((K
lp= comma!......................................................(1.
lpr comma!......................................................((2
lprm comma!...................................................(12
lpstat comma!..................................................(10
ls comma!.....................................................2, 20
log +ormat.....................................................22
#iewig +ile attri%"tes.....................................92
*"kyao#, $le3a!er.........................................20K
*R> E*ogical Rol"me >aagerF................80, .8(
:
machie lag"age...............................................(11
maiteace...............................(K2, (KK, (K8, (/K
make comma!..................................................(2.
>ake+ile.............................................................(2.
ma comma!......................................................2.
ma pages.....................................................2., (2(
mark"p lag"ages......................................2K9, (2(
memory....................................................................
assige! to each process.................................1
!isplayig +ree.................................................(
:esi!et Set SiDe............................................K
segmetatio #iolatio..................................21
"sage...............................................................K
#iewig "sage................................................2K
#irt"al..............................................................K
me"-!ri#e programs.......................................109
meta key...............................................................8K
meta se="eces...................................................220
metacharacters....................................................220
20.
9!e3
meta!ata....................................................../., ./(
mk!ir comma!.............................................(1, 10
mk+i+o comma!................................................191
mk+s comma!............................................92, .91
mkiso+s comma!...............................................9K
mktemp comma!..............................................19.
memoics.........................................................(12
mo!al e!itor........................................................11
moospace! +ots...............................................(((
>ooleaar, Jram.................................................1.
more comma!.....................................................2K
mo"t comma!..........................................82, .9K
mo"t poits.........................................28, .82, .81
mo"tig.............................................................8.
>6(.....................................................................09
m"lti-"ser systems...............................................9(
m"ltiple-choice !ecisios...................................1(2
m"ltitaskig..........................................9(, ..(, 192
m# comma!..................................................(K, 12
1
ame! pipes.......................................................19(
ao comma!....................................................10
&a"til"s................................................((, .0., 2.2
etstat comma!................................................202
etworkig..........................................................99
aoymo"s 0T6 ser#ers...............................201
!e+a"lt ro"te..................................................20(
'yamic Aost Co+ig"ratio 6rotocol E'AC6F
......................................................................20(
ecrypte! t"els..........................................2.0
e3amie etwork settigs a! statistics.......202
0ile Tras+er 6rotocol E0T6F........................201
+irewalls........................................................200
0T6 ser#ers...................................................201
*ocal $rea &etwork.....................................20(
loop%ack iter+ace........................................20(
ma i the mi!!le attacks.............................20/
ro"ters...........................................................202
sec"re comm"icatio with remote hosts... .20/
testig i+ a host is ali#e.................................200
tracig the ro"te to a host.............................20.
tras+errig +iles...........................................212
trasportig +iles...........................................20(
Rirt"al 6ri#ate &etwork................................2.0
ewlie character................................................K.
ewlies...............................................................8.
&EWS................................................................(19
l comma!........................................................(09
ro++ comma!...................................................(22
"ll character......................................................222
"m%er %ases......................................................1K1
;
octal......................................................98, 1K1, 1/9
7gg Ror%is..........................................................09
7*'O6W' #aria%le............................................(0
7pe7++ice.org Writer..................................Kp., 21
7peSSA............................................................20/
operators...................................................................
arithmetic................................................/2, 1K1
assigmet....................................................1KK
%iary............................................................122
compariso...................................................1K9
terary...........................................................1/0
owig +iles..........................................................91
P
package +iles......................................................../.
package maitaiers............................................/.
package maagemet........................................../0
!e%................................................................./0
'e%ia Style E.!e%F......................................../.
+i!ig packages............................................/(
high-le#el tools............................................../2
istallig packages......................................../(
low-le#el tools.............................................../2
package repositories....................................../.
:e! Aat Style E.rpmF...................................../.
remo#ig packages......................................../1
:6>............................................................../0
"p!atig packages........................................./2
packagig systems............................................../0
page !escriptio lag"age..................2K9, (21, ((2
6$;E: #aria%le..................................................(0
pagers...................................................................2K
parameter e3pasio..............................//, 80, 12/
paret !irectory.....................................................2
paret process.......................................................(
passw! comma!..................................................
passwor!s..............................................................
paste comma!...................................................281
6$T$...................................................................8/
patch comma!..................................................29.
patches................................................................289
6$TA #aria%le.............................(0, .((, (K0, (/8
pathame e3pasio.............................../(, 80, 122
pathames..........................................................2K1
pathames................................................................
202
9!e3
a%sol"te............................................................K
completio......................................................8K
relati#e.............................................................K
6'0............................................................(22, ((2
6erl programmig lag"age. 18, 21/, (0(, (1K, 1/2
permissios........................................................(28
6A6 programmig lag"age..............................(1K
pig comma!....................................................200
pipelies...............................................K2, 10/, 18/
pipelies...................................................................
i comma! s"%stit"tio................................/8
porta%ility...........................................(20, (81, (98
porta%le..............................................................(81
6orta%le 'oc"met 0ormat........................(22, ((2
6orta%le 7peratig System 9ter+ace.................229
positioal parameters......................1(/, 128p., 1K.
67S9G......................................9K, 222, 228p., (98
67S9G......................................................................
character classes......(2p., 221p., 22/, 2K., 29(,
(0(
6ostScript...........................2K9, (21, ((2, ((/, (12
pr comma!...............................................(./, (((
primary gro"p 9'.................................................91
prita%le characters............................................222
prite# comma!......................................./8, .28
priter %"++ers......................................................82
priters........................................................82, .8/
%"++erig o"tp"t.............................................82
cotrol co!es................................................((.
!aisy wheel...................................................((.
!e#ice ames.................................................8/
!ri#ers...........................................................(((
graphical.......................................................((2
impact...........................................................((.
laser..............................................................((2
prit+ comma!..........................................(.8, 12K
pritig.....................................................................
!etermiig system stat"s............................(10
history o+......................................................((0
9teret 6ritig 6rotocol.............................(1.
moospace! +ots.........................................((.
preparig te3t................................................(((
pretty.............................................................((/
prit ="e"es..................................................(10
proportioal +ots.........................................((2
="e"e............................................................(1.
spoolig........................................................(10
termiate prit 4o%s.......................................(12
#iewig 4o%s.................................................(1.
process 9'.............................................................1
process s"%stit"tio............................................18/
processes..............................................................(
%ackgro"!....................................................20
chil!................................................................(
cotrollig.......................................................8
+oregro"!.....................................................20
iterr"ptig.....................................................9
4o% cotrol.....................................................20
killig............................................................22
ice.................................................................2
paret..............................................................(
69'..................................................................1
process 9'.......................................................1
S9;9&T.........................................................189
sigals............................................................22
S9;TE:>....................................................189
sleepig...........................................................2
state.................................................................2
stoppig.........................................................2.
#iewig...................................................1, ..K
Dom%ie............................................................2
pro!"ctio "se....................................................122
programma%le completio...................................88
ps comma!.........................................................1
6S. #aria%le................................................(0, .K0
6S2 #aria%le.......................................................(K/
ps2p!+ comma!................................................(22
6S1 #aria%le.......................................................129
pse"!oco!e.................................................(82, 1.2
pstree comma!..................................................2K
6"TT5................................................................2.2
pw! comma!.......................................................2
6W' #aria%le......................................................(.
6ytho programmig lag"age..........................(1K
?
="otig................................................................./9
!o"%le ="otes................................................../9
escape character..............................................82
missig ="ote................................................120
sigle ="otes...................................................8.
R
:$9' E:e!"!at $rray o+ 9!epe!et 'isksF
.............................................................................80
raster image processor........................................(((
rea! comma!....................102, 1.., 1./, 122, 18K
:ea!lie...............................................................81
:E$'>E.....................................................22, (19
re!irectio................................................................
20(
9!e3
%locke! pipe.................................................192
gro"p comma!s a! s"%shells....................182
here !oc"mets.............................................(/(
here strigs...................................................10/
sta!ar! error..................................................K.
sta!ar! ip"t.........................................K(, (/1
sta!ar! o"tp"t...............................................K0
re!irectio operators................................................
^X...................................................................K(
Y......................................................................K2
YElistF............................................................18/
YY..............................................................(/(p.
YY-................................................................(/1
YYY...............................................................10/
X......................................................................K0
XElistF............................................................18/
XX...................................................................K.
Z.......................................................................K2
reg"lar e3pressios...............K/, 21/, 299, (9(, 10K
reg"lar e3pressios...................................................
achors.........................................................22.
%ack re+ereces..................................2K/, 298p.
%asic...........................228, 2KKp., 29K, (00, (.0
e3te!e!.......................................................228
relatioal !ata%ases............................................282
relati#e pathames................................................K
release early, release o+te.................................122
remo#ig !"plicate lies i a +ile.........................KK
:E6*5 #aria%le..........................................102, 18/
report geerator..................................................(K2
repositories........................................................../.
ret"r comma!.........................................(/9, (90
re"sa%le..............................................................(81
:96.....................................................................(((
rlogi comma!.................................................20/
rm comma!........................................................(/
:ock :i!ge e3tesios........................................9K
ro++......................................................................(22
:7T.( eco!ig................................................291
:6>..................................................................../0
rpm comma!...................................................../(
rsyc comma!..................................................212
rsyc remote-"p!ate protocol............................212
:"%y programmig lag"age.............................(1K
S
scalar #aria%les...................................................1/K
Schillig, Jorg....................................................9K
scp comma!......................................................2..
script comma!....................................................9.
scriptig lag"ages.......................................18, (12
s!i++ comma!....................................................(08
searchig a +ile +or patters..................................K/
searchig history..................................................89
Sec"re Shell.......................................................20/
se! comma!.....................................292, (2K, 1K2
set comma!...............................................28, 128
setgi!...................................................................0(
set"i!...........................................................0(, (89
Sewar!, J"lia....................................................2((
s+tp comma!.....................................................2..
share! li%raries.............................................28, ./2
she%ag......................................................(29, (K1
shell %"iltis.........................................................18
shell +"ctios..............................18, .28, (/8, 11.
shell prompts....0, .K, 90, .02, ..9, .(0, .K0, 208,
(K/
shell scripts.........................................................(28
SAE** #aria%le..................................................(0
shell #aria%les......................................................28
shi+t comma!............................................110, 112
S9;9&T..............................................................189
sigals................................................................188
sigle ="otes.........................................................8.
Slackware............................................................/0
sleep comma!...................................................1.1
so+t lik................................................................(0
sort comma!...............................................KK, 2/.
sort keys.............................................................2/2
so"rce co!e.............................../0p., ./8, 2K9, (11
so"rce comma!.........................................(9, (K.
so"rce tree..........................................................(1/
special parameters......................................112, 129
split comma!....................................................(08
SSA....................................................................20/
ssh comma!..............................................20/, 2(9
ssh program..........................................................9(
Stallma, :ichar!....................2, Kp., .(2, 229, (1K
sta!ar! error..............................................29p., K2
!isposig o+....................................................K(
re!irectig to a +ile.........................................K.
sta!ar! ip"t.......................................29, (/1, 102
sta!ar! ip"t...........................................................
re!irectig.......................................................K(
sta!ar! o"tp"t.....................................................29
appe!ig to a +ile..........................................K.
!isposig o+....................................................K(
re!irectig sta!ar! error to...........................K2
re!irectig to a +ile.........................................K0
start"p +iles..........................................................(.
stat comma!.....................................................22/
201
9!e3
sticky %it..............................................................0(
storage !e#ices....................................................80
a"!io C's..............................................81, .92
C'-:7>s..........................................8(p., .92
creatig +ile systems......................................89
!e#ice ames.................................................8K
!isk partitios................................................8.
0$T(2............................................................89
+loppy !isks...........................................8/, .9(
+ormattig......................................................89
*R> E*ogical Rol"me >aagerF..................8(
mo"t poits..........................................82, .81
partitios........................................................89
rea!ig a! writig !irectly..........................91
repairig +ile systems....................................9(
"mo"tig....................................................82
)SJ +lash !ri#es............................................91
stream e!itor.......................................................292
strigs.......................................................................
e3pressios...................................................(9.
e3tract a portio o+.......................................1K0
legth o+........................................................1K0
per+orm search a! replace "po.................1K2
remo#e lea!ig portio o+............................1K.
remo#e trailig portio o+............................1K.
[cparameter,o++set,legthd...........................1K0
[cparameter,o++setd......................................1K0
strigs comma!................................................12(
st"%s...........................................................(8., 122
style....................................................................(19
s" comma!........................................................01
s"%shells.....................................................10/, 182
s"!o comma!............................................01, .0K
S" >icrosystems...............................................1.
s"per"ser.........................................0, 92, .02, .22
sym%olic liks..........................................(0, 10, 11
creatig.....................................................11, 1/
listig..............................................................11
symlik.................................................................(0
syta3 errors.......................................................1.9
syta3 highlightig.....................................(28, (K(
T
ta%les..................................................................282
ta%"lar !ata.................................................2/2, (2.
tail comma!........................................................K8
tape archi#e........................................................2(2
tar comma!.......................................................2(1
tar%alls................................................................(1/
targets.................................................................(2.
Task >aager.......................................................8
Tatham, Simo...................................................2.2
t%l comma!...............................................(22, (2K
tee comma!........................................................K9
Teletype................................................................1
telet comma!..................................................20/
TE:> #aria%le....................................................(.
termial em"lators................................................0
termial sessios......................................................
cotrollig termial........................................1
e++ect o+ .%ashrc............................................(K.
e#iromet...................................................02
e3itig..............................................................(
logi shell..............................................02, .(.
TE:> #aria%le..............................................(.
"sig ame! pipes........................................191
#irt"al..............................................................(
with remote systems.......................................9(
termials..............................8K, 92p., .K(, (22, ((.
terary operator..................................................1/0
test cases.............................................................12K
test comma!.............................(88, (9(, 1.(, 122
test co#erage.......................................................12K
testig.........................................................122, 12/
TEG....................................................................(22
te3t........................................................................21
a!4"stig lie legth.....................................(.(
$SC99.............................................................21
carriage ret"r..............................................2/.
comparig.....................................................28/
co#ertig >S-'7S to )i3........................29(
co"tig wor!s...............................................K/
c"ttig...........................................................280
!eletig !"plicate lies.................................2/9
!eletig m"ltiple %lak lies........................2/.
!etectig !i++ereces.....................................288
!isplayig commo lies..............................288
!isplayig cotrol characters........................2/0
'7S +ormat..................................................2/.
E'9T7: #aria%le..........................................(0
e!itors....................................................(1, 2K8
e3pa!ig ta%s..............................................28(
+iles.................................................................21
+ilterig...........................................................KK
+ol!ig..........................................................(.(
+ormattig.....................................................(09
+ormattig +or typesetters.............................(22
+ormattig ta%les...........................................(2K
4oiig...........................................................282
lie+ee! character.........................................2/.
lowercase to "ppercase co#ersio..............29(
202
9!e3
"m%erig lies....................................2/., (09
pagiatig.....................................................(./
pastig..........................................................281
preparig +or pritig...................................(((
remo#ig !"plicate lies................................KK
re!erig i 6ostScript.................................(21
:7T.( eco!e!............................................291
searchig +or patters.....................................K/
sortig.....................................................KK, 2/.
spell checkig...............................................(0(
s"%stit"tig...................................................298
s"%stit"tig ta%s +or spaces...........................28(
ta%-!elimite!.................................................282
trasliteratig characters..............................292
)i3 +ormat..................................................2/.
#iewig with less......................................21, KK
te3t e!itors...........................................(1, 2K8, 292
emacs.............................................................(2
+or writig shell scripts.................................(28
ge!it.......................................................(2, (28
iteracti#e.....................................................292
kate........................................................(2, (28
ke!it...............................................................(2
kwrite.............................................................(2
lie.................................................................1.
ao.......................................................(2, .10
pico................................................................(2
stream...........................................................292
syta3 highlightig...............................(28, (K(
#i....................................................................(2
#im.................................................(2, (28, (K(
#is"al.............................................................1.
til!e e3pasio.............................................../1, 80
tloa! comma!....................................................2K
top comma!........................................................K
top-!ow !esig.................................................(/K
Tor#al!s, *i"s...................................................2, /
to"ch comma!.......................22Kp., 211, (2(, 11/
tr comma!........................................................292
tracero"te comma!...........................................20.
tracig................................................................128
trasliteratig characters....................................292
traps....................................................................189
tro++ comma!....................................................(22
tr"e comma!.....................................................(8/
TT5......................................................................1
type comma!......................................................19
typesetters..................................................(22, ((2
Tf #aria%le..........................................................(.
-
)%"t"..................................91, .0/, ./0, 221, (K.
"mask comma!..........................................0., ..0
"mo"t comma!................................................82
"alias comma!.................................................2/
"ary operator e3pecte!.....................................122
"ary operators...................................................1K1
"e3pa! comma!...........................................28(
"e3pecte! toke...............................................12.
"i= comma!..............................................KK, 2/9
)i3........................................................................(
)i3 System R...................................................((2
"i32!os comma!............................................2/.
"set comma!..................................................182
"til compo"! comma!..................................1.K
"til loop............................................................1.K
"Dip comma!..................................................21.
"p!ate!% comma!............................................2.2
"pstream pro#i!ers............................................../.
"ptime................................................................(//
"ptime comma!................................................(8(
)SJ +lash !ri#es.........................................80, .91
)SE&ET............................................................291
)SE: #aria%le............................................29, .(.
"sers.........................................................................
acco"ts..........................................................91
chagig i!etity...........................................01
chagig passwor!s.........................................
e++ecti#e "ser 9'....................................0(, ..1
home !irectory................................................92
i!etity............................................................91
passwor!.........................................................92
settig !e+a"lt permissios............................0.
set"i!.............................................................0(
s"per"ser................................92, 9/, .0(p., ..2
-etc-passw!.....................................................92
-etc-sha!ow.....................................................92
@
#ali!atig ip"t..................................................108
#aria%les...............................................//, (K8, 12/
assigig #al"es....................................(/., 1KK
costats.......................................................(/0
!eclarig...............................................(K8, (/.
e#iromet...................................................28
glo%al............................................................(80
local..............................................................(80
ames....................................................(/0, 1K0
scalar.............................................................1/K
shell...............................................................28
20K
9!e3
#+at......................................................................92
#i comma!.........................................................10
#im comma!.............................................2K/, (K(
#irt"al cosoles.....................................................(
Rirt"al 6ri#ate &etwork.....................................2.0
#irt"al termials....................................................(
#is"al e!itors.......................................................1.
#mstat comma!.................................................2K
'
wait comma!....................................................192
wc comma!........................................................K/
we% pages...........................................................2K9
wget comma!...................................................20K
What 5o" See 9s What 5o" ;et.........................((.
whatis comma!..................................................2(
which comma!.............................................19, /8
while compo"! comma!................................1.(
wil!car!s..................................(2, K1, /2, 21/, 221
wo!im comma!.................................................9/
wor!-splittig................................................../9pp.
worl!....................................................................91
W5S9W5;........................................................((.
A
G Wi!ow System...................................., 9(, 2.0
3args comma!..................................................221
3loa! comma!...................................................2K
3logo comma!....................................................9
G>*...................................................................2K9
B
yakig te3t..........................................................82
y"m comma!...................................................../(
C
Dgrep comma!..................................................2K/
Dip comma!......................................................210
Dless comma!.....................................................22
-
--help optio.........................................................2.
.
.-co+ig"re..........................................................(20
.%ashOhistory........................................................89
.%ashOlogi..........................................................(.
.%ashOpro+ile........................................................(.
.%ashrc.................................(2, .(1, (K., (81, 112
.pro+ile.................................................................(.
.ssh-kowOhosts...............................................209
*
EE FF compo"! comma!...........................1K(, 1K9
D
a comma!.........................................................12.
E
-............................................................................2/
-%i.......................................................................2/
-%oot.....................................................................2/
-%oot-gr"%-gr"%.co+............................................2/
-%oot-#mli"D.......................................................2/
-!e#.......................................................................2/
-!e#-c!rom..........................................................8/
-!e#-!#!...............................................................8/
-!e#-+loppy..........................................................8/
-!e#-"ll...............................................................K(
-etc........................................................................2/
-etc-%ash.%ashrc...................................................(2
-etc-crota%...........................................................2/
-etc-+sta%...............................................2/, .8., .9(
-etc-gro"p.............................................................92
-etc-passw!.............................2/, 92, 2/8, 28(, 10K
-etc-pro+ile...................................................(., .((
-etc-sha!ow..........................................................92
-etc-s"!oers.........................................................01
-li%........................................................................28
-lostV+o"!...........................................................28
-me!ia...................................................................28
-mt......................................................................28
-opt.......................................................................28
-proc.....................................................................28
-root..............................................................28, .02
-s%i......................................................................28
-tmp..............................................................28, 19.
-"sr........................................................................29
-"sr-%i.................................................................29
-"sr-li%..................................................................29
-"sr-local...............................................................29
-"sr-local-%i........................................29, (21, (K2
-"sr-local-s%i.....................................................(K2
-"sr-s%i................................................................29
20/
9!e3
-"sr-share..............................................................29
-"sr-share-!ict.....................................................22.
-"sr-share-!oc.................................................29, 22
-#ar.......................................................................29
-#ar-log.................................................................29
-#ar-log-messages...................................29, K9, .8/
F
[L................................................................11(, 19(
[EEe3pressioFF...................................................1K(
[cLarrayaNbd......................................................18.
[cLarrayaUbd........................................................18.
[cLpre+i3Nd.......................................................1K0
[cLpre+i3Ud.........................................................1K0
[cPparameterd....................................................1K0
[cparameter,-wor!d...........................................128
[cparameter,Cwor!d...........................................129
[cparameter,Vwor!d..........................................129
[cparameter,Wwor!d..........................................128
[cparameter--patter-strigd..............................1K2
[cparameter-Ppatter-strigd.............................1K2
[cparameter-_patter-strigd............................1K2
[cparameter-patter-strigd...............................1K2
[cparameterPPpatterd.......................................1K.
[cparameterPpatterd.........................................1K.
[cparameter__patterd....................................1K.
[cparameter_patterd.......................................1K.
[N..............................................................112, 120
[U................................................................112, 120
[P........................................................................1(8
[0........................................................................112
208