Unix Tutorial

By Joshua Lande
SASS
January 21

This is not a philosophy talk!

Doug McIlroy, the inventor of Uni !i!es and

one of the founders of the Uni tradition,
su""ari#ed the !hiloso!hy as follo$s%

This is the Unix philosophy: Write programs

that do one thing and do it well. Write
programs to work together. Write programs
to handle text streams, because that is a
universal interface.

&htt!%''en($i)i!edia(org'$i)i'Uni*!hiloso!hy+

The Basics

All co""and line !rogra"s have , "ain

co"!onents%

Command line arguments

Standard Input (stdin)

Standard Output (stdout)

By default, stdin is ty!ed fro" the ter"inal and

stdout is !rinted to the ter"inal

for hel! on any co""and%

$ man command

A e! "asic programs

echo - sends the co""and line argu"ents to

stdout

cat - reads file&s+ as co""and line argu"ents

and sends the lines to stdout( If no files
s!ecified, sends stdin to stdout(

tac - Just li)e cat .ut .ac)$ards

tee - $rites the in!ut .oth to the stdout and to a

file s!ecified as a co""and line argu"ent

#xample
$ sed 's/lame/awesome/g'
This example is lame
This example is awesome
^D

sed re!laces the first $ord $ith the second $ord

/s'la"e'a$eso"e'g/ is a co""and line argu"ent

0irst line is the stdin &I ty!ed+

Second line is the stdout &!rinted to screen+

1hen you are done sending stuff to stdin, ty!e

234L5D and the !rogra" $ill finish u!(
htt!%''$$$(caton"at(net'.log'sed5one5liners5e!lained5!art5one'

Sorting
$ sort -t ":" -n -k2
Ted:1000
John:1
Sally:100
Bob:10
John:1
Bob:10
Sally:100
Ted:1000

Sort is a !rogra" to
sort the lines of
standard in!ut

5t s!ecifies the field

se!erator

5n "eans nu"eric sort

5)2 "eans sort the

second colu"n

input$output redirection
$ cat > file.txt
Some random stuff...
^D

2hange $here stdin co"es fro" and stdout goes(

6nd your line $ith 7 to redirect stdout to a file(

Use 77 to a!!end to a file

Use 8 to read stdin fro" a file(

$ cat < file.txt
Some random stuff...

pipes
$ cat *.txt | sort | uniq > output.txt

In this ea"!le, cat out!uts all tet files, $hich

are sorted( All du!icates are than re"oved and
the out!ut is saved to a file(
$ somecommand | tee output.txt

9rints out!ut of a co""and to stdout and a file:

$ somecommand | less

9i!e to less for nice navigation(

3urn the stdout of one !rogra" to the stdin of

another using a !i!e ;

a!k

9o$erful !rogra""ing language

6asy to $hi! u! !o$erful scri!ts

3he general synta is an e!ression follo$ed

.y a co""and(

loo!s over stdin

#xample% second colu"n if the first colu"n is a

nu".er greater than 1<
$ awk '$1>10{print $2}' file.txt

a!k (more%)

9ut code you $ant to run .efore or after inside

B6=I> and 6>D .loc)s(

#xample& count nu".er of occurrences in file%

$ awk 'BEGIN {print "Analysis:" }
/foo/{++foobar }
END {print "foo appears
" foobar " times." }' file

a!k (again%)
BEGIN {FS=:}
BEGIN {OFS=;}

Set out!ut colu"n se!erator as se"icolons%

Divides each line into colu"ns

deault se!arator is s!aces

S!ecify the se!arator .et$een each colu"n%

a!k
$ ls -l
drwxr-xr-x 3 lande gl 2048 Dec 12 19:21 bin
drwx------ 2 lande gl 4096 Nov 20 15:59 mail
...

Su" total "e"ory

$ ls -l | awk '{s+=$5} END{print s}'
$ ls -l | awk '$6=="Dec"&&$7=="12"{print $0}'

9rint only files fro" Dec 12

(last a!k script)

4e!lace all colu"ns $ith their a.solute value%

$ awk '{ for (i = 1; i <= NF; i++)
if ($i < 0) $i = -$i; print $0}'

htt!%''$$$(caton"at(net'.log'a$)5one5liners5e!lained5!art5one'

htt!%''$$$(caton"at(net'.log'a$)5one5liners5e!lained5!art5t$o'

htt!%''$$$(caton"at(net'.log'a$)5one5liners5e!lained5!art5three'

So "any one liners

'o" Control

2ontrol5# sus!ends a currently running ?o.

3he ?o.s co""and sho$s you all the ?o.s

running in the ter"inal
$ jobs
[1]- Stopped yes
[2]+ Stopped yes

6ach ?o. given a nu".er( 4un the second ?o.

in the .ac)ground or foreground%
$ bg 2
$ fg 2

'o" Control

Begin ?o. in the .ac)ground

$ command &

List all ?o.s running on your "achine%

$ ps -u lande
PID TTY TIME CMD
19231 pts/21 00:00:00 vim
19233 pts/21 00:00:00 find

@ill any ?o. &.y 9ID or na"e+

$ kill 19231
$ killall find

ind (stu (uickly)

Synta% find !ath e!ression

Searches recursively through all su.folders

$ find /path/ -name file.txt
$ find . -type f \( -iname "*.sh" -or \
-iname "*.pl" \)

)iname for case insensitive search

)type finds only files and )type d only folders

6a"!le% find files ending $ith either /sh/ or /!l/%

Use a A to continue a long line

grep (is "eautiul)

Search through stdin for things

Sends to stdout lines "atched lines

$ grep tacos
this line has tacos
this line has tacos
this line dosen't
more tacos
more tacos

Bou can do the sa"e in a$) $ith

$ awk '/tacos/{print $0}'

grep
$ grep -B2
$ grep -A4
$ grep -C3

)B !rints lines .efore "atch

)A !rints lines after each "atch

)C !rints the lines .efore and after

)i case insenstive search

)* !rints lines $ith no "atch

)c !rints ?ust nu".er of "atches

))color highlights "atches

grep

0ancy regular e!ressions% 56

#xample& Match I9 range fro" 1C2(22(21(1 to

1C2(22(21(,D%
$ grep -E '172\.22\.21\.([1-9]|(1[0-9]|
2[0-9]|3[0-5])) ' hosts.txt

htt!%''unsta.le"e(.logs!ot(co"'2<<E'<C'"atch5
i!5range5using5egre!5.ash(ht"l

xargs

Ma)es stdin as a co""and line argu"ent

useful for running a co""and a .unch of ti"es

#xample& Search in all files for a varia.le na"e

$ find . -name *.cxx | xargs grep var

3his is eFuivalent to running gre! on all G(c

files in all su.directories(
$ grep *.cxx

3he a.ove $ould only search files in current

directory

xargs (is xtreme)

Use )I+, to re!lace all occurrences of HI in the

co""and $ith the standard in!ut(

#xample &I use all the ti"e+% 4un all the scri!ts
in all su.directories
$ find . -name "*.sh" | xargs -I{} sh {}
$ find . -name '*.dat' | xargs -I{} cp
{} /folder/

2o!y lots of files at once

Too many -o"s running%

@ill all ?o.s running in ter"inal

jobs -p | xargs -i kill -9 {}

?o.s 5! !rints all ?o. IDs(

)ill 5J )ills the ?o. $ith that ID(

xargs (to the rescue)

#xample% run cvs u!date in all su.folders%

find . -type d | xargs -i -t sh -c \
'cd {};cvs update'
htt!%''en($i)i!edia(org'$i)i'args

5t !rints out the co""and .efore eecuting &for

de.ugging+

par

4efor"ats tet

>ot installed .y default .ut easy to .uild(

$ par 30j
We the people of
the United States, in order to form a
more perfect
union, establish justice...
We the people of the United
States, in order to form a
more perfect union, establish
justice...

par (cont)
$ par 25
# one fish #
# two fish #
# red #
# fish blue fish #
# one fish two fish #
# red fish blue fish #

!ar can fi your code co""ents

htt!%''$$$(nice"ice(net'!ar'

paste
$ cat f1.txt
a
"
c
. cat /0txt
1
/
. paste 10txt /0txt
a 1
" /
c
htt!%''unsta.le"e(.logs!ot(co"'2<<J'<1'linu5
!aste5co""and5good5ea"!les5uses(ht"l

2arious stu

=o to !revious folder%
$ cd -

=et the !revious co""and%

$ file.txt
bash: file.txt: command not found
$ echo !!

!$ is the last !art of the last co""and

3a4y history

9rints the co""and to the screen

$ !comma
$ !comma:p
$ !comma

4uns !revious co""and .eginning $ith

co""a

5un stu
$ mkdir -p /home/make/all/of/these/dirs/
$ cp /path/filename1 /path/filename2

Instead%

2reates all su.directories(

$ cp /path/filename{1,2}
$ mkdir foo1 foo2 foo3 bar1 bar2 bar3
$ mkdir {foo, bar}{1, 2, 3}

6uess !ho%

1ho is on your "achine

send the" a "essage

$ who
lande pts/16 Jan 21 01:34
(rescomp-07-119188.stanford.edu)
...
$ write lande
What's up?

7uestions