Commands, editing, Variables

More commands, and more details

echo
grep
sed
cut

- uniq
- sort
- diff
- pr

- find

Editing
vi

- emacs

Shell Variables
Environment Variables
11

echo
to show stuff on the screen
Prints to standard output
echo hello
hello

echo A short

line > newfile

A short line

echo A short
A short

(in newfile)

line > newfile

line

(in newfile)

hello=Hi, there
Hi, there

echo \hello\
hello

22

Echo
(with variables and quotes)
echo $hello
blank

Define variable:
echo $hello

hello=Hi, there

Hi, there

echo $hello
Hi, there

echo \$hello
$hello

echo $hello
$hello

33

grep
Find Pattern in File(s)

grep pattern file(s)

grep -v pattern filename
grep ^pattern filename
grep pattern$ filename
grep pattern with spaces filename
fgrep -f patternfile filename
grep -i pattern filename
egrep pat1|pat2|pat3 filename
44

cut
To Get Pieces Of Each Line

cut
cut
cut
cut
cut

-c 5,6,7 file > newfile

-c 5-7 file > newfile
-c15-20,2-5,8-13 filename > newfile
-f3 d" " filename > newfile
-f1,5,6 -d: /etc/passwd
55

tr Examples
To translate single characters to single characters

tr abc xyz < filename

a x, by, cz

tr A-Z a-z
or tr A-Z a-z
or tr [A-Z] [a-z]
translates upper case to lower case

tr a-zA-Z n-za-mN-ZA-M < filename

"rot13: changes letters halfway up alphabet

tr -s \t :
replace multiple spaces+tabs with single colon
6

sed
To Substitute in a File
sed s/string/newstring/g file > newfile
sed s?relative/path/name?new/dir/name?
file > newfile
sed 5,10s/string/newstring/ file > newfile
sed /pat1/,/pat2/s/strg/newstrg/ file > nf
77

sed
Delete Lines
(You can also add lines, but >> is easier)
sed '5,10d' file > newfile
Deletes lines 5 through 10

sed /isaacs/d file > newfile

Deletes all lines containing "isaacs"

sed /here/,/there/d file > newfile

Finds first line containing "here", and deletes it
and all lines up to one containing "there"
88

uniq
Eliminate Duplicate Lines
uniq file > newfile
eliminate duplicates (must be adjancent)

sort file | uniq > newfile

sort first

uniq -c file | less

counts identical lines

99

sort
sort file > newfile

alphabetically from beginning of line

sort -k3 file > newfile

alphabetically from third field

sort -k 3n file > newfile

numeric order, third field

sort -rn -k 3 file > newfile

reverse numeric order

ls -l | sort -r -k 5n | less
show largest files)

sort -t: -k1,1 -k5,5 /etc/passwd | less

sort the password file, by account then name

11
00

Comparing Files
diff file1 file2

reports differences between 2 files

form allows changing one to the other
5,6c7,8 means lines 5-6 n file1 match lines 7-8 in file2
< is line from file1; > is line from file2

diff3 file1 file2 file3 compares 3 files

cmp file1 file2 compares binary files by byte (and
reports first different byte)
11
11

pr
Print Command
pr filename
format for printing

pr filename | lpr
actually print formatted file

flags:

+k
-k
-h
-t

start with page k

k-column output
make your own filename
dont print headers
11
22

find
find . -name filename -print
find a file with the name filename

find ~isaacs -mtime +6 -print

find files in my home, older than 6 days

find . -types f -exec ls -l {} \;

only do ls on regular files

find . -type f | xargs ls -l

11
33

awk
This is an extra. I will not test on awk, but
it may be useful in some of the quiz and
homework questions.
awk is more than a command, it is
essentially a language. But if you want the
full power, I would suggest learning and
using Perl instead. Though harder to learn,
it is much more powerful, and does all that
awk does.
1
1
44

awk definition
AWK is a language for processing text files. A file
is treated as a sequence of lines, called records.
Each line is broken up into a sequence of fields,
so we can think of the first word in a line as the
first field, the second word as the second field,
and so on. An AWK program is of a sequence of
pattern-action statements. AWK reads the input
a line at a time. A line is scanned for each
pattern in the program, and for each pattern
that matches, the associated action is executed.
- Alfred V. Aho (one of the inventors of awk,
quoted in Wikipedia, modified by me)

1
5

How awk works.

awk does actions (like arithmetic or
printing) for lines specified by patterns, or
before any lines are read ("BEGIN") or
after all lines are read ("END)
awk BEGIN {start-action} {action} END
{stop-action}
awk pattern1{action1} pattern2{action2}
pattern2{action3}
11
66

awk examples
1.
2.
3.
4.
5.

awk '{print $1}'

awk '/isaacs/ {print}'
awk '/isaacs/'
awk '{print $8 " - " $5 " bytes"}'
awk 'BEGIN{sum=0} {sum=sum+$5} END
{print sum}'
6. awk 'BEGIN {FS=":"} $5 ~ " - Fall13"
{split ($5,xx,/,/);print xx[1]}'

11
77

awk variables and actions

NR - number of the current line

NF - number of fields in current line
FS - Field separator
RS - Line (record) separator
print - print a line
length() - length of a string
split(str,array,sep) - split string into an array
tolower() - make a string lower case
1
8

Some extra commands

(Not in tests)
column - make a list into columns
ls | grep somefiles | column -c 5
locate - easier to use then "find"
locate filename
paste - join each line of two files together, one after
the other
paste file1 file2
join - If 2 files have a field in common, combine the
lines from both files if the fields match. (Must sort
on the field.)
1
9

Using vi To Create A File

vi file-name
i (or a)

enter vi
go into insert mode

<Enter as much text as you like. Backspace

will delete the last character.>

ESC
ZZ

(key)

get out of insert mode

save the file and exit

(uppercase)
22
00

vi Concepts
Modes
command (ESC)
insert (i, a, o, etc)
ex mode (:)

Screen, as a window in the text

Where is the cursor?
How to move around
Search for a line (/ and ?)
Exit, saving or not saving your changes
22
11

vi modes
Command mode
What you type is a command to vi
ESC will change you from insert to command

Insert (or replace) mode

Whatever you type will go into your file
Various commands put you in insert mode

ex mode
Commands to ex editor, start with :
On bottom line of screen
22
22

ex mode

s, d, undo, w, q
:undo undo last vi operation
:w <filename> write file (:w! to force)
:q quit (:q! to force)
:d delete current line
:5,10d delete lines 5 through 10

:s/old/new/g substitute new for old on current

line
/xyz/s/old/new/g sub. new for old on all lines with
xyz in them
:5,10s/old/new/ sub. new for 1st old in lines 5-10
22
33

addressing for ex

n,m (line n through line m)

.,$
(here to end)
1,.
(line 1 to here)
.,.+9 (next 10 lines)
.-2,.+2 (5 lines: 2 before through 2nd after)
/pat/ (next line containing pat)
/pat1/,/pat2/ (lines from next containing pat1
through one after containing pat2)
?pat? (previous line containing pat)
22
44

Search for a line

/abc
?abc
n

search forward for pattern abc

search backwards for pattern abc
repeat search (in same direction)

22
55

To exit vi
ZZ save file and exit
DO NOT confuse with ^Z, which puts it in the
background

:wq write file and exit

:w <filename> write new file
:w! <filename> overwrite old file

22
66

vi Inserting and Appending

O Open new
line above

I Insert at
beginning

o Open
new line
below

Cursor

A Append at
end

Inserting text commands.

Insert
before
cursor

Append
after
cursor
22
77

Viewing Text Within A File

Basic VI COMMANDS:
To enter VI type:
vi filename <cr>
To save a file type:
:w
To get out of VI type:

To save a file type:

:w
To get out of VI type:
ZZ

ZZ
INSERTING TEXT
I - insert text before
cursor
o - insert text below
current line
a - insert text after
cursor
O - insert text above
line

22
88

vi Cursor Movements
1G
K
O

Cursor Movements Commands

Cursor

[]
E
[] Try these,cursor,commands ,
in a

+
Return

file.

b l
h

G
2
9

vi Screen Movement
H

Basic VI COMMANDS:
To enter VI type:
vi filename <cr>

^b

To save a file type:

:w
To get out of VI type:

To save a file type:

:w
To get out of VI type:
ZZ

ZZ

^f

INSERTING TEXT
I - insert text before
cursor
o - insert text below
current line
a - insert text after
cursor
O - insert text above
line

L
33
00

vi Deleting Words and Lines

D or d$ Delete to
end of line
dw Delete word
Deleting text in a file
x Delete
Cursor

dd Delete
line

character
at cursor
3
1

emacs Concepts
Commands use special keys: control (CTRL),
escape (ESC) or meta or alt, function keys
Style modes (but not like vi)
Buffers (multiple windows into a file)
Extensible (in Lisp)
Unlimited undo
Keyboard macros; key binding
Files can be large (doesnt read them into core)
33
22

(emacs concepts)
Automatic backups
Can set automatic saving

email, directories, ftp, much more

Menu control (instead of commands)
Information bars

33
33

Starting and Ending emacs

emacs

(puts you in scratch buffer)

C-x C-f to read in file

emacs filename
C-x C-s
C-x s
C-c C-c

(edits filename)
save file and exit
save all files and exit
exit (will ask if you want to)

33
44

emacs help
C-h

usual help key

F1 (function 1 key) try if C-h doesnt work

C-h a apropos
C-h c what does a character do
C-h f what does a function do
C-h t tutorial
C-h b show all key bindings
C-h ? show possible help questions
33
55

Processes

Basic unit of multi-processing

ps to show your current processes
ps -l to show more about them (l=long)
ps -alwf shows all processes on system, in
long format, child procs indented, 2 lines if
necessary

33
66

Shell Variables
Name and Value
Examples
MyName=Isaacs
HOME=isaacs
fruit=apple
MONTH=April
fullname=Stan Isaacs
33
77

To Use or See Shell Variables

$NAME to use it
echo $NAME
echo I am going to eat an $fruit
IF fruit=apple
drink=crab$fruit
drink=${fruit}juice
echo Fruit is $fruit, drink is $drink38
3
8

USES
Environment Variables
to define your environment

Local Variables
inside scripts
long names

Parameters to shell scripts

pass arguments to shell procedures (a kind of
program)
33
99

Environment Variables
Usually all caps
Represent your personal environment
Used by various UNIX programs and commands

HOME
LOGNAME
PATH
EDITOR

LPDEST
PRINTER
TERM
PS1

44
00

Environment Variables
env to see them all
You can change some of them
frequently in startup scripts

export so everybody can see them

sub-shells
scripts

set shows local variables also

44
11

Example
d=/usr/isaacs/class/day1/exercises
cd $d
pr $d/ex3

44
22

Summary of Variables

Case sensitive
No spaces around =
Quote if there are spaces in value
Curly braces ({,}) to separate
Usually character strings
num=100.0 is different from num=100)

echo or env to see them

export
44
33