BASIC

UNIX
Version 1.0
 UNIX Operating System
Features :


Multiuser & Multiprocessor System
 Time Sharing Operating System

Written in “C” (HLL)
 has all advantages of HLL

performance low compared with HLL

Simple User Interface

Hierarchical File System

Consistent File Format - the Byte Stream

Supports Languages - FORTRAN, BASIC, PASCAL, Ada, COBOL, LISP,
PROLOG, C, C++

System V - about 64 system calls

Unix/Intro/01
 Architecture of UNIX System

Cpp Sh

wh
o
nroff
Kernel

A.cut
comp
Hardware da t
CC e
os

m
wc

ra
og
pr
vi

n
tio
ed grep

ica
pl
ap
er
th
O

Unix/Intro/02
 Unix O.S.

Services Provided :


Process Management - Creation, Termination or Suspension and

Communication


CPU Scheduling


Memory Management - Swapping & Paging


File Management


Peripheral Device controlled access to process - terminal, printer, tape

drives, disk drives and network devices

Unix/Intro/03
 File System

 A Hierarchical Structure

 Consistent Treatment of Data

 The Ability to Create and Delete Files

 Dynamic Growth of Files

 The Protection of File Data

 The Treatment of Peripheral Devices as Files

Unix/Intro/04
 File System
/ (Root)

bin etc usr unix dev fs

sh date who passwd src bin tty00 tty01 mjb maury

cmd

Date.c Who.c

Unix/Intro/05
 File System
File Types

 Directory File
 Regular File
 Device File

Classes of User

 Owner
 Group
 Others

Access Permission

 Read
 Write
 Execute

Unix/Intro/06
 File System
 Consists of Sequence of Logical Blocks

(Multiple of 512 k bytes)


An Installation may have

 several physical disk units


each containing one or more file system


File System Treated as Logical Device (Logical Device Number)

 Data from another File System can be Accessed by Mounting that File System

 Block- unit of data transfer

- can be assigned to only one file

Unix/Intro/07
 File System Layout
Boot Block
Super
Block

Inode List
Data
Blocks

Unix/Intro/08
 File System

Boot Block
 Beginning of File System (1st Sector)

Contains Bootstrap Code
 Every File System has (possibly empty) Boot Block

Super Block

Size
 # of Files that can be Stored
 Free Space Information
 Inode Information
 Inode List : Information about each inode e.g. access permissions, data/time,
phy. Addr. Etc.

Data Block : File Data & Administrative Data

Unix/Intro/09
 File System
Inode Table Fields :

 File Owner Identifier - Owner & Group id


File Type - Regular, Directory, Character / Block /Pipe

File Access Permissions
rwx rwx rwx

user group others


Date & Time - Last File/Inode Modified/Accessed
 No. of Links

Disk Addresses
 File Size

Unix/Intro/10
 File System
Directories

 Directory Entry Size is 16 bytes

 Two Fields
 Inode Number

 File Names
83 . Current Directory
2 .. Parent Directory
1798 Init
1799 mount
2114 passwd
:
0 crash

: :

Unix/Intro/11
 Building Block Primitives
1. Each Process has Access to 3 Files

standard input (“<“)

standard output (“>”)

standard error (“2>”)

e.g.

$ls

$ls>temp

$mail xyz <letter

$nroff - mm <file1> outfile 2> errfile

$cat >test

$cat ->test

$cat >>test

$catt ->>test

Unix/Intro/12
 Building Block Primitives
2. Pipe - allows stream of data to be passed between reader &
writer process

The data that the first processor writes into the

pipe is input to second process.

E.g. a) $grep main a.c. b.c. c.c. | wc-l

- counts the number of lines in the files

that contain the string “main”

b) $who| wc-l

c) $ls|wc-l

d) $ls|wc-l>fcount

Unix/Intro/13
 Building Block Primitives
3. Tee - standard output can be saved in a file as well as displayed

- can be placed anywhere in pipeline

e.g. $who | tee user.lst

kumar tty01 May 18 09 : 32

sharma tty02 May 18 11 : 18

tiwary tty03 May 18 13 : 21

e.g. $who | tee user.lst|wc-l

$_

Unix/Intro/14
 File System
The Structure of File System
Root (/)

bin dev etc mnt tmp unix usr

<Files> <Files> <Files> <Files> bin Kumar Sharma

<files> prog safe

<files> <files>

Unix/Intro/15
 File System
/ bincommonly used UNIX Command like

/usr/bin who, ls, cat, wc etc. are stored here

/dev- contains device files of all hardware devices

/etc - contains those utilities mostly used by system administrator

e.g. passwd, chmod, chown

/tmp - used by some UNIX utilities especially vi and by user to store

temporary files

/usr - contains all the files created by user, including login directory

/unix - kernel

Release V - does not contain / bin

- contains / home instead of /usr

Unix/Intro/16
Basic Commands - pwd, cd

1. Checking your directory - pwd command

$pwd

/usr/Kumar

2. Changing directories - cd commands

e.g. $pwd

/usr/Kumar

$cd Prog

$pwd

/usr/Kumar/Prog

e.g. $cd / bin

e.g. $cd .. / tmp

e.g. $cd

Unix/cmd/01
Basic Commands - logname, cat

3. Prompting login directory - logname command

$cd /usr/Sharma

$logname

Kumar

$_

4. Displaying and Creating Files - cat command

e.g. $cat dept.lst

01|accounts|6213

02|admin|5423

06|training|1006

$_

Unix/cmd/02
cat Command
 Scrolling of large files can be freezed by pressing <ctrl s> and resumed by
<ctrl q>

e.g. $cat chap1 chap2

-display contents of chap2 immediately
after displaying chap1

e.g. $cat>test
Type here the text contents
After typing the entire text
Press <ctrl d> to save and exit
<ctrl d>
$_

e.g. $cat pgm1 pgm2 > pgm3

e.g. $cat test test test

Unix/cmd/03
ls Command

5. Listing current directory contents - ls command

e.g. $ls

a.out

chap1

chap2

test

test.c

$_

Unix/cmd/04
ls Command
Options available in ls

Option Description

-x Displays multicolumnar output (prior to Release 4)

-F Marks executables with *and directories with /
-r Sorts files in reverse order (ASCII collating sequence
by default)
-l The long listing showing seven attributes of a file
-d Forces listing of a directory
-a Shows all files including ., .. And those beginning with
a dot
-t Sorts files by modification time
-R Recursive listing of all files in sub-directories
-u Sorts files by access time (when used with the -t
option)
-i Shows i-node number of a file
-c Sorts by time of change of the i-node
-s Displays number of blocks used by a file
-1 One file per line (Release 4)

Unix/cmd/05
ls Command
E.g.
$ls - l
total 8
_rw_rw_rw_ 1 Kumar group 44 May 9 09:08 dept.h
_rw_rw_rw_ 1 Kumar group 212 May 9 09:08 dept.q
_rw_rw_rw_ 1 Kumar group 154 May 9 09:08 emp.h
_rw_rw_rw_ 1 Kumar group 742 May 9 09:08 emp.q
$_

Consider first entry -

Field1 --> mode
- rwx rwx rwx

  
 File type
- --> regular file
d --> directory file
c --> character - read
b --> block read
  --> user permissions
 --> group permissions
 --> others permissions

Unix/cmd/06
ls Command

Field2 - indicates number of links

Field3 - File owner id

Field4 - Group id

Field5 - File size in bytes

Field6 - Date/time last altered

Field7 - Filename


The total field at the beginning indicates total no. of block occupied by these

files

Unix/cmd/07
mkdir Command

 Creates a directory

e.g. $mkdir doc

e.g. $mkdir doc doc/example doc/data

e.g. $mkdir doc/example doc

- will give error - Order important

Unix/cmd/08
rmdir Command

 Used to remove directory

 Only empty dir can be deleted


More than one dir can be deleted in single command


Command should be executed from at least one level above in the hierarchy

e.g. $rmdir doc

$rmdir doc/example doc

$rmdir doc doc/example

- will give error

Unix/cmd/09
chmod Command
 Used to alter file permissions

Format :

chmod <category> <operation> <permission> <filenames>

Category Operations Attribute

u-user +assigns permission r-read

g-group -remove permission w-write

o-others =assigns absolute x-execute

permission

a-all

Unix/cmd/10
chmod Command
E.g. $chmod u+x note
$ls - l note
-rwx r-- r -- 1 …… note

e.g. $chmod ugo+x note

$ls - l note
-rwxr-xr-x …… note

e.g. $chmod u-x, go+r note

$chmod u+x note note1 note2
$chmod o+wx note
$chmod ugo=r note

Unix/cmd/11
chmod Command
 Octal notation
 describes both category and permission
 similar to =operator (absolute assignment)
 read permission - assigned value is 4
write permission - assigned value is 2
Execute permission - assigned value is 1

e.g. $chmod 666 note

will assign read & write permission to all

e.g. $chmodd 777 note

will assign all permissions to all

e.g. $chmod 753 note

?

Unix/cmd/12
cp Command
 Copies a file or group of files

e.g. $cp chap1 temp/chap2

 Option - i (interactive)
$cp - i chap1 chap2
cp: overwrite chap2 ? y

 Option -r (recursive) to copy entire directory

$cp - r temp newtemp

Unix/cmd/13
rm Command
 To delete files
e.g. $rm chap1 chap2 chap3
$rm chap*
$rm *
Are you sure? y

 Option - i (interactive delete)

$rm - i chap1 chap2
chap1 : ? y
chap2: ? n

 Option - r (recursive delete)

$rm -r *
(Warning: Pl. do not use this option)

 Option -f (deletion of write protected files)

$rm -f chap1
$rm -r -f*

Unix/cmd/14
mv Command
 Used to rename file or group of files

e.g. $mv chap1 man1

 the destination file if existing gets overwritten

 dir can also be renamed

e.g. $mv temp doc

e.g. $mv chap1 chap2 chap3 man1

will move chap1, chap2 & chap3 to man1

directory

Unix/cmd/15
more Command
 Allows user to view a file one screen at a time

Syntax :
more <options <+linenumber> <+/pattern> <filename>


: appears after displaying screen

<spacebar> or <enter> - next screen
 q - quit
 n - switch to next file on command
 p - switch to previous file
 f - displays filename and line no.

/pattern - to locate line with pattern

! - to run command

Unix/cmd/16
more Command
Command Description

k:n Skips to kth next file that has been

specified in the command line
k:p Skips to kth previous file that has been
specified in the command line
:f Displays current filename and line
number
= Displays current line number
spacebar Displays next screen
ks Skips k lines forward
kf Skips k screens forward
, Reverts to beginning of file if no search
made, or to point where last search was
made
/pattern Searches for pattern forward
n Repeats last search forward
v Starts up the vi editor
. Repeats previous command
! cmd Executes the Unix command cmd
q Exits from more

Unix/cmd/17
wc Command
 Counts lines, words and character depending on option
 takes one or more filename as arguments
e.g. $wc infile
3 20 103 infile
$wc infile test
3 20 103 infile
10 100 180 test
13 120 283 total
$wc - l infile
3 infile
$wc - w infile
20 infile
$wc - c infile
103 infile
$wc - wl infile
20 3 infile

Unix/cmd/18
od Command

 Lists the octal value (ASCII) of any file content

 used to display non-printing character

 options

 b to display 16 byte of data on each line

 c to display character above octal value

e.g. $od - bc odfile

Unix/cmd/19
cmp Command

 To compare two files & echo first mismatch

 option -l gives detailed list of byte no. and different byte in octal

e.g. $cmp note1 note2

note1 note2 differ : char3, line1

$cmp - l note1 note2

3143 145

6170 167

7171 170

Unix/cmd/20
comm Command

 To compare two sorted files

 displays 3 - columnar output

 first column - lines unique to first file

second column - lines unique to second file

third column - line common to both files

 option -1, -2, -3 can be used to display specific column only

Unix/cmd/21
diff Command

 Display file differences

 display which lines in one file have to be changed to make both files identical

 option -e generates a script when applied to first file makes it identical to

second

 option -b consecutive blanks considered as one blank and trailing blanks are

ignored

Unix/cmd/22
banner Command

 Creates poster by blowing up its arguments

 maximum ten character in a line

e.g. $banner UNIX

$banner ‘UNIX’ ‘IS HERE’

Unix/cmd/23
cal Command

 Displays calendar of a particular month/ year

 any calendar from year 1 to 9999

e.g. $cal 1991

$cal 01 1991

Unix/cmd/24
date Command
 Displays system date
 number of options available
option m - month
h - month name
d - day of month
y - last two digit of the year
H - hour
M - minute
S - second
T - time in hh:mm:ss
a - day of week (Sun to Sat)
r - time in AM/PM
e.g. $date
Fri Dec 7 15:00:21 EST 1990
$date +%m
12
$date +”%h%m”
Dec 12
$date “+The day of week is %a and time y.r”
The day of week is Fri and time 03:23:42 PM

Unix/cmd/25
who Command

 Displays current users of the system

 displays logname, terminal no., login date and time

e.g. $wo

$who - H displays Header for all

columns

$who - Hu displays additional info. As

IDLE, PID, comments

$who am i logname of the user

Unix/cmd/26
Terminal Related Commands

 tty - displays device name of the terminal

 stty -a - displays current settings

 stty can also be used to change the setting

e.g. to use <ctrl c> as interrupt key instead

<Del> key

$stty intr \^c

Unix/cmd/27
head Command
 Displays the beginning of a file
 also works with multiple files
e.g. $head -3 emp.lst
 will display first 3 lines

 if not specified (default) 10 lines

$head -2 emp.lst dept.lst

tail command
 displays the end of a file
 default - display last 10 lines
 $tail -3 emp.lst
 displays last 3 lines
 $tail +50 emp.lst
 displays till end of file starting from line no. 50
 max. 4096 char segment can be displayed
 if size specified is greater than 4096 then last 4096 char are displayed

Unix/cmd/28
tail Command
$tail -15c emp.lst
 display last 15 characters

$tail -1b emp.lst

 block is used as unit of measurement

Unix/cmd/29
Cut & Paste Commands
Cut :
Allows you to slice the file vertically


 identifies both column and fields

e.g.$cut -c6-22, 24-32 emp.lst

cuts the file emp.lst vertically from column 6 to 22 and from column 24 to 32
$cut -c-5, 10-15, 25 - emp.lst

implies 1-5 implies 25 - end of file

 d option to specify delimiter can be used with -f option which specifies field no.
 default delimiter is tab
 cut selects entire line if not properly delimited

$cut -d “|” -f 2,3 emp.lst

$cut -d “|” -f 1,5- empl.lst | tee temp

Unix/cmd/30
Cut & Paste Commands
Paste :
paste more than one file laterally
$paste file1 file2
<…file1> <file2>
 delimiter can be inserted between two files

$paste -d “|” file1 file2

 Inserting field delimiter with cut and paste if file does not have delimiter
$temp =/tmp/Kumar
$cut -c1-4 emp.lst > $temp.01
$cut -c5-10 emp.lst > $temp.02
:
:
$cut -c73-80 emp.lst > $temp.06
$paste -d “|” $temp??

Unix/cmd/31
sort Command
 $sort <file> …. Sorts entire line
 reorders a line in a ASCII collating sequence
 sorting can be done on a field
 delimiter can be specified, -t option
$sort -t “|” +1 emp.lst

+1indicates sort after skipping first field i.e.

on 2nd field
 sorting in reverse order -r option
$sort -r+1 emp.lst
 output filename can be specified as argument
$sort -0 sortedf emp.lst
 -c (check) option to check if sort successful. No message indicates sort successful
 char position can be specified
$sort -t “|” +4.6 - 4.9 emp.lst
 will sort 7th & 8th column of field 5
 sorting on multiple fields
$sort -t “|” +2-3 +1 file1

Unix/cmd/32
sort Command
$sort -m file1 file2 file3

merges sorted files
 Numeric Sort
$sort -n file1
 Option -b ignores leading blank spaces
-d Alphanuemric sort
- f folds uppercase into lowercase

Uniq Command
 removes duplicate records (-d option)
$uniq dept.lst
$sort dept.lst | unique - uniqlst

-u option if only unique records to be listed

-d if only one copy of repeated records to be selected

Unix/cmd/33
nl Command
nl - line numbering command


$nl empl.lst
1 1001 …..
2 1002 …..
:
:
 width of line number can be specified
-w option

-s (separator) can be specified

-n leading zeros in line number
$nl -w2 -s “|” -nrz emplst

01 | 1001 .. Right justified

02 | 1002 ..
:
:

-v option to specify starting line number

-i option specifies increment

Unix/cmd/34
grep Command - Advanced Filter

Searches for a given pattern in the files specified
grep <options> <pattern> <filename(s)>

Options
-c displays count of occurrences
-l displays list of filenames only
-n displays line number along with lines
-v displays all but the line matching pattern
-i ignores case
-h omits filename
-f fname reads expression/pattern from file
(egrep & fgrep option)
-x displays lines match in entirety
(fgrep option)

Unix/cmd/35
grep Command
e.g. $grep sales emp.lst

- lists records with sales as dept

$grep ‘sales’ emp.lst

$str = ‘sales’

$grep -c “$str” emp.lst

 will count no. of directors in all emp?.lst
 display will be for each file separate

e.g. Find out designation of youngest person

who is not a director

Unix/cmd/36
Regular Expressions
 Used to specify pattern
 specified in single quotes
Expression
ch* - 0 or more occurrence of char ch
ch \ {m\} - matches m occurrence of char ch
ch \ {m,n\} - matches min. m & max. n occurrence
of char ch
ch \ {m,\} - matches min. m occurrence
[pqr] - matches single char p, q or r
[c1-c2] - match single char within ASCII range
specified
[^pqr] - matches single char which is not p, q
or r
\(ptn\) - matches pattern ptn enclosed in \(,\)
with the tag \n (n ranges 1 to 9)
e.g. ` \ (.. \) \1`

Unix/cmd/37
Expression
^ptn - matches pattern ptn at beginning of line
ptn$ - matches pattern ptn at end of line
e.g. Consider following R.E.
(i) ‘[cC]ho [wu]dh*ury’
(ii) ‘[7-9] …$’ char [7-9] followed by 3 more
chars rep by … at end of line
(iii) ‘director.*[7-9]…$’
(iv) ‘^[\t]*$’
(v) ‘^ ……… 03’
 .\{10\}
(vi) [0-9] [0-9] [0-9] [0-9]

Q. Write R.E. to locate Sengupta & Dasgupta

Q. Write R.E. to locate people having the same month

of birth as current month

Unix/cmd/38
egrep Command
egrep - Extending grep
 extends grep pattern matching capabilities
 Extending R.E. features
ch+ matches 1 or more char ch
ch? Matches  or 1 char ch
exp1/exp2 matches exp1 or exp2 expression
(x1/x2)x3 matches expression x1x3 or x2x3
x1/x2 matches expression x1 or x2

e.g. egrep ‘(sen/das)gupta’ emp.lst

egrep ‘[aA]gg?[ar]+wal’ emp.lst
egrep -f pat.lst emp.lst

Unix/cmd/39
fgrep Command

fgrep - multiple string searching

 only simple (pattern) string or group of strings can be searched

 faster than grep and egrep

e.g. $fgrep -f pat.lst emp.lst

$cat pat.lst

sales

admin

Unix/cmd/40
Process Status
 $ps - give pid, tty, time, command
 options
-f - full from (additional - user-id parents pid,
CPU time consumed, start time
cumulative process time)
-u - details of only users processes
-a - all processes details
-l - detailed listing
-e - system processes

Background Process
 $sort -0 emp.lst emp.lst &
$a.out &
 nohup (no hangup) - permits execution of process even if user has logged
off
$nohup sort emp.lst &
sends output to nohup.out

Unix/cmd/41
Process Related Commands
 Wait
 shell command
 waits for child process to complete
 $wait 138 - waits for background job with pid 138

 Kill
$kill 1005 (default signal 15)
 kills job with pid 1005
$kill -9 1005

sure killing of job

$kill 0 - kills all background process

 nice - job execution with low priority

 prefix the command with nice
$nice wc -l xxx
$nice -15 wc -l xxx

reduce priority by 15 units

Unix/cmd/42
 Editors

 Three Types

 Line Editor - ed, ex

 Screen Editor - vi

 Stream Editor - sed

Unix/vi/01
 vi Editor
 Three Modules :

Command
i, I, a, A, Mode
r, R, o, O, :
s, S
<Enter>

sh
<Esc>
<ctrl d>
Input or exit
ex
command
Mode
Mode

 :set showmode
 displays message when in input mode

Unix/vi/02
 vi Editor - Save & Quit


From input mode ZZ


From input mode to command mode press <Esc> and from command mode

To Save :w

To Quit :q

To save & quit :wq

or :x

Unix/vi/03
 vi Editor - Command Mode

Deletion

x - single char

4x - deletes 4 characters from cursor to right

dd - line delete

4dd - deletes 4 lines

Unix/vi/04
 vi Editor - Navigation

 Moving between lines

G command is its line address

e.g. 40G - move to 40th line

1G - move to beginning of file

G - move to end of file

Unix/vi/05
 vi Editor - Other Features


Joining line

J - to join current line with next line

4J - to join 4 lines from current line

 Marking Text

m <single char label> - to mark line

<single char label> - to locate marked

line

 Repeating last Instruction - . (dot key)

 Undo last Instruction - u


Reverse all changes made to current line - U

Unix/vi/06
 Input Mode Commands

Command Function

i Insert text to left of cursor

I Inserts text at beginning of line
a Appends text to right of cursor
A Appends text at the end of line
o Opens line below
O Opens line above
rch Replaces single character under
cursor with character ch (no<Esc>)
R Replaces text from cursor to right
s Replaces single character under
cursor with any number of
characters
S Replaces entire line

Unix/vi/07
 Cursor Movement Commands

Command Function

h or backspace Moves cursor left

j Moves cursor down
k Moves cursor up
I or spacebar Moves cursor right
^ Moves cursor to beginning of first word
of line (no repeat factor)
O or l Moves cursor to beginning of line (no
repeat factor with O)
$ Moves cursor to end of line
b Moves cursor backwards to beginning
of word
e Moves cursor forward to end of word
w Moves cursor forward to beginning of
word

Unix/vi/08
 Paging Functions

Command Function

<Control-f> Full page forward

<Control-b> Full page backward

<Control-d> Half page forward

<Control-u> Half page backward

<Control-l> Redraw page screen (no repeat

factor)

Unix/vi/09
 Search and Repeat Commands

Commands Functions

/patSearches forward for pat

?pat Searches backward for pattern pat
n Repeats search in the same direction along
which the previous search was made (no
repeat factor)
N Repeats search in a direction opposite to
that which the previous search was made
(no repeat factor)
fch Moves cursor forward to first occurrence of
character ch in the current line
tch Moves cursor forward onto but before first
occurrence of character ch in the current
line
; Repeats search in the same direction along
which the previous search was made with
f or t
, Repeats search in a direction opposite to
that along which the previous search was
made with f or t

Unix/vi/10
 A Few Specimen Operator-Command Combinations

Command Function

d$ or D Deletes from cursor to end of line

5ddDelete five lines
d/endif Deletes from cursor up to the first occurrence
of the string endif in the forward direction
d30G Deletes from cursor up to line number 30
df. Deletes from cursor to first occurrence of a
dot
c0 Changes from cursor to beginning of line
c$ or C Changes from cursor to end of line
3cw or c3w Changes three words
cc Changes current line
cG Changes from cursor to end of file
y$ or Y Yanks from cursor to end of line
3yw or y3w Yanks three words from cursor position
5yy Yanks five lines
y?case Yanks from cursor up to the first occurrence
of the string case in the reverse
!Gsort Sorts all lines from current position to end of
file

Unix/vi/11
 Command Function

:r note Reads file note into present cursor location

:w saves buffer and remains in editing mode
:w note1 Creates file note1 and writes buffer contents
to it
:w! note1 Overwrites contents of file note1 with
contents of buffer
:w>> note1 Appends buffer contents to file note1
:x Saves buffer and quits editing mode
:wq Saves buffer and quits editing mode
:q Quits editing mode without saving
:q! As above but after abandoning changes
:e note1 Stops editing current file, and edits file
note1
“e! note1 As above, but after abandoning changes
made to current file
:e! Loads last saved edition of current file
:e #Returns to editing the most recently edited
file
:n Edits next file (on the vi command line)
:rew Rewinds file list to start editing from first file
specified in the command line
100Cursor at start of 100th line

Unix/vi/12
 Set Options Used by ex/vi
Option Abbreviation Significance

autoindent ai Next line starts at the previous indented

level
autowrite aw Writes current file automatically when-
ever switching files with :n and escaping
to the shell with :sh
ignorecase ic Ignores case while searching for patterns
magic Treats the characters of the regular
expression set as special while searching
for patterns
number nu Displays line numbers on screen
showmatch sm Shows momentarily the mach to a ) & }
showmode Displays a message when vi is in input
mode
tabstop ts Sets tab for display (default : 8 spaces)
wrapscan ws Continues pattern search by moving to
the other end of a file

Unix/vi/13
Advanced Filter - awk
 Report writer
 named after Aho
Weinberger
Kernigham
 As powerful programming language
 awk pgm are slow
 It can access, transform and format individual fields in a record
 Syntax
awk <options ‘line specifier {action}’ <files>

I. Simple awk filtering

awk ‘/director/ {print}’ emp.lst

line specifier Action

(selects line (as has an effect of
for processing) printing all selected lines)
 $0 variable stores entire line
 print  print $0

Unix/awk/01
Advanced Filter - awk
 Line specifier and action option are optional, either of them needs to be
specified
 line specifier not specified indicates all lines to be selected
 {action} omitted indicates print (default)
 fields are identified by special variable
$1, $2, ….; default delimiter is contiguous string of spaces
 Explicit delimiter can be specified using -F option
e.g. awk -F “|” ‘/sales/{print $3, $4}’ emp.lst
 Regular expression of egrep can be used to specify pattern
 Line numbers can be selected using NR built-in variable
$awk -F “|” ‘NR ==3, NR ==6 {print NR, $0}’ emp.lst

Unix/awk/02
awk Command

Output can be formatted using printf
 similar to printf in ‘c’ but not a function
e.g. $awk -F “|” ‘/[aA]gg?[ar]+wal/{
>printf “%3d%-20s%-12s%d\n”, NR, $2,$3,$6}’
>emp.lst

Record no./
Line number
will be printed
 , (comma) to be used as delimiter in printf

Logical and Relational operators

Logical operator &&, ||
$awk - F “|” ‘$3 ==“director” || $3 == “chairman”{
>printf “%-20s”,$2}’ emp.lst

== tries to find perfect match but string may have trailing spaces to overcome this ~ and !~
(match & negate of match) can be used with R.E.

Unix/awk/03
awk Command
e.g.$awk -F “|” ‘$2~/director/||$~/g.m/{
>printf $0}’ emp.lst
e.g.$3 ~/^g.m/

Relational Operators : <, <=, ==, !=, >=, >, ~, !~

e.g.$awk -F “|” ‘$6>7500 {printf “%20s”, $2}’
emp.lst
e.g.$awk -F “|” ‘$3 == “director” || $6>7500 {
>print $0}’ emp.lst
Number Processing
 Computation on numbers can be done
+, -, *, /, % operator available
 no type declaration
 initialized to zero
e.g.$awk -F “|” ‘$3 == director” || $6>7500 {
>kount = kount+1
>printf “%3d%-20s\n”, kount,$2}’ emp.lst
e.g.valid expression
kount=kount+1
kount++; ++kount

Unix/awk/04
awk Command
 -f option
 awk program can be written in a separate file and used in awk

e.g. $awk -F “|” -f emp.awk emp.lst

single quoted contents are

written in this file without
quotes

 BEGIN and END Section

Format: (i) BEGIN {action}
(ii) END {action}
(iii)
awk<options> ‘BEGIN {action}
line specifier {action}
END {action}’ <files>

Unix/awk/05
awk Command
e.g. $cat emp.awk
BEGIN {
printf “\n\t Employee details \n\n”
}
$6>7500{
# increment sr. no. and sum salary
kount++; tot+=$6
printf “%d%-20s%d\n”, kount, $2, $6
}
END {
printf “\n The Avg. Sal. Is %6d\n”, tot/kount
}
$_
&awk -F “|” -f emp.awk emp.lst

Unix/awk/06
awk Command
 Positional parameters and shell variable
(i) Method I
 requires entire awk command should be in the shell script
 to differentiate positional parameter and field identifier, the positional
parameter should be single quoted in awk program
e.g. $3 > ‘$1’

(ii) Method II

store entire command sequence in separate script file

shell can pass values to awk
e.g. awk pgm
$6 > mpay { … }
$awk - F “|” -f emp.awk mpay = 7800 emp.lst

assignment should be immediately before filename

Unix/awk/07
awk Command
e.g. List the details of employee drawing salary
greater than avg. salary

$awk - F “|” ‘$6>x’ x = `awk - F “|”’ ‘{x+=$6}

END {print x/NR}’ emp.lst` emp.lst

Unix/awk/08
nawk - getline Statement
 awk uses i/p from standard input or file


nawk - i/p from standard input device & file possible (at beginning)

using getline statement

 when getline is execute the contents of next line gets stored in var.

e.g. getline x

getline x < “Emp.lst”

getline x < “/dev/tty”

Unix/awk/09
Built-in Variable
NR - Cumulative no. of records read

FS- The i/p field separator

OFS - The output field separator

NF- No. of fields in current record

FILENAME - The current i/p file

ARGC - No. of arguments in command

line

ARGV - List of organization

Unix/awk/10
Arrays
 Can handle single dimension array
 index can be anything even string
 no declaration required, initialized to zero automatically
e.g. totl[1] +=$6; tot[2]+=da; tot[3]+=HRA; temp[$3]

Functions

int(x) - Returns integer value of x

sqrt(x) - Returns square root of x
index(s1,s2) - Returns the position of string s2 in s1
length() - Returns length of the argument
substr(s1,s2,s3) - Returns portion of string of length - s3,
starting position s2 in string s1
split(s,a) - Split string s into array a optionally
returns no. of fields

Unix/awk/11
Interface with Shell
 Redirection > , piping | possible
 restriction - filenames, commands that follows these symbol must be in double quotes
 unix commands can be run within nawk using system function
e.g. BEGIN {system (“date”)}

Control FLOW :

(I) if … then …
if (conditions is true)
<statements>
else
<statements>

(II) For statement

for (i=1; i <= 9; i++)
{

Unix/awk/12
Interface with Shell
(iii) for loop for arrays

for (k in array)

<commands>

(iv) While loop

while (condition is true)

(v) break and continue

Unix/awk/13
Shell
 Shell - The agency that sits between user and

UNIX System

- much more than command processor

 Bourne Shell - named after its founder Steve Bourne

- widely used
 C Shell - product from Univ. of California, Berkeley

- advanced user interface and enhanced

features
 Korn Shell - by David Korn of Bell Lab.

Executables in /bin directory

 sh indicates - Bourne Shell

 csh if present indicates - C Shell
 ksh if present indicated - Korn Shell

Unix/sh/01
echo Command
 Displays argument
e.g. $echo GOOD Morning
GOOD Morning
$echo \*
*
$echo ‘*’
*
 escape sequence to manipulate cursor position
\t : tab
\f : formfeed
\n : newline
\c : cursor immediately after argument
e.g. $echo ‘Enter Your Name : \c’
Enter Your Name : $_
 accepts ASCII Octal values
e.g. $echo ‘\007’
<… beep hear…:

Unix/sh/02
Command Substitution

 Command enclosed in backquotes (`), the shell executes the command first,
and the enclosed command text is replaced by the output of the command

e.g. $echo The date today is `date`

The date today is Fri 27 00:12:55 EST 1990

$echo The date today is; date

- issues command echo and date

sequentially

e.g. $echo “There are `ls | wc - l` files in current

directory”

$cal `date “+%m 19%y”`

Unix/sh/03
Shell Variables
 User variables can be defined
 no type declaration or initialization

Format : variable=value
 Value is of string type
 No space preceding and following =operator
 Evaluated by prefixing variable name with $

multi word assignment must be quoted
e.g.1 $x = 37
$echo $ x
37
e.g.2 $msg = You have mail
have not found
e.g.3 $msg = ‘You have mail’
$echo $msg
You have mail

Unix/sh/04
Shell Variables
4. $msg = You\ have\ mail
$echo $msg
You have mail
5. $x = aa
$y = bb
$echo $x$y
aabb
6. $echo ‘Amount = $1000’
$echo Amount = \$1000
- both will display : Amount =$1000
$echo “Amount = $1000”
Amount=000
- here shell evaluated variable $1 as double
quoted

Unix/sh/05
Shell Variables

7. $p = `pwd`

$echo $p

8. $echo ${fname}

emp.sh

$echo ${fname}x

emp.shx

Unix/sh/06
Shell Variables

System Variables
 Set during - booting
- login
 script executed during login time - profile

profile alters operating environment of a user
 to display list of system variable $set

list of variable
PATH - list of dir. In order of precedence that have to be scanned for hunting
command
Home - dir. Named after Login name
IFS - string of char which are used as word separator in the command line
MAIL - mail directory
PS1 - Two prompts - primary and secondary
PS2 - (used by sed, awk)
TERM - Terminal type

Unix/sh/07
Shell Variables

DATA in the same file - The here document

 < redirection symbol, redirects the standard input to come from a file
 << here document symbol signifies data is here rather than in a Separate file

e.g. $grep director <<mark - delimiter to

indicate end of

document

>1001 | Sanjay | g.m. | 5

>5001 | Rahul | director | 7

>mark

Unix/sh/08
Read Statement
 Takes input from standard i/p
 can be used to read one or more variables

e.g. echo “\n Enter the Pattern : \c”

read patname
echo “\n Enter file name : \c”
read fname
grep “$patname” $fname

OR
echo “\n Enter pattern & file name : \c”
read patname fname

Unix/sh/09
Command Line Arguments
 Shell procedures arguments in command
 Arguments are specified along with the name of shell procedure
 Arguments are assigned to special variables (positional parameters) $1 - First
parameters
$2 - Second parameters
:
:

Special parameters also available
$0 - Name of executed command
$* - Complete set of positional parameters
$# - No. of arguments
$$ - PID of current shell
$! - PID of last background job
$? - Exit status of last command
$@ - Same as $* except when enclosed in double quotes

Unix/sh/10
Command Line Arguments
E.g.1 if [$# -nc 3]
then
:
:
fi
2 $kill - 9 $$

3 grep “$1” $2

4 grep director emp.lst >/dev/null;echo$?

Exit Status  indicates successful
1 indicates failure
2 indicates unreadable file

Unix/sh/11
Conditional Execution

Logical Operators && and ||

&& operator delimits two commands, the second command is executed only if first succeeds
 || operator delimits two commands, the second command is executed only if first fails

e.g.$grep `director` emp.lst && echo “patern found”

$grep `manager` emp.lst || echo “pattern not found”

Script Termination - exit statement


premature termination of program

control returned to the calling program

e.g.grep “$1” $2 || exit 2

echo “pattern found”
Return value
default zero

Unix/sh/12
if Statement
 Format :
(i) if <condition is true>
then
<execute commands>
else
(execute commands>
fi
(ii) if <condition is true>
then
<execute commands>
fi
(iii) if <condition is true>
then
<execute commands>
elif <condition is true>
then
<execute commands>
<…>
<…>
else
<execute commands>
fi

Unix/sh/13
if Statement
 Condition can be specified either using test or [cond]
 Relational Operator used
 eq Equal to
 nc Not equal to
 gtGreater than
 gc Greater than or equal to
 lt Less than

lc Less than or equal to

 File related tests with test

-a <file> Trace if <file> exists
-f <file> if exists & regular file
-r <file> if exists & readable
-w <file> if exists & writable
-x <file> if exists & executable
-d <file> if exists & is dir file
-s <file> if exists & size > 0

Unix/sh/14
Test - String Comparison
 String tests used by test

n str True, if str not a null string

z str True, if str is a null string

S1 = S2 True, if S1 = S2
 S1! = S2 True, if S1  S2
 str True, if str is assigned and not null


Logical Operators

a .AND.

O .OR.
 ! Not

e.g. If test $x -eq $y

 if [$x -eq $y]

e.g. If [! -f fname]
then
echo “file does not exists”
fi

Unix/sh/15
The Case Statement
 Format :
case <expression> in
<pattern 1> ) <execute commands> ;;
<pattern 2> ) <execute commands> ;;
<…>
<…>
esac

e.g.echo “\n Enter Option : \c”

read choice
case $choice in
1) ls -l ;;
2) ps -f ;;
3) date ;;
4) who ;;
5) exit ;;
esac

Unix/sh/16
The Case Statement
E.g.

Case `date | cat -d “ “ -fi ` in

Mon > <commands> ;;
Tue > <commands> ;;
:
:
esac

E.g.

Case “$ans” in
[yY] [eE] [sS] ;;
[nN] [oO] exit ;;
*) “invalid option” ;;

Unix/sh/17
While Statement
 Format :
while <condition is true>
do
<execute statements>
done

e.g. while [$x -gt 3]

do
ps -f
sleep 5
done

e.g. while true

do
ps -f
sleep 5
done

Unix/sh/18
While Statement
E.g. Script to edit, compile & execute a program

while true
do
cc $1
case $? In
o) echo “Compilation Successful”
echo “Executing a.out”
a.out ; exit ;;

*) echo “Compilation Error”

echo “Press <Enter> to edit”
read pause
vi $1 ;;
esac
done

Unix/sh/19
Break & Continue Statement
 Continue : Suspends execution of all statements
following it, and switches control to
top of loop for next iteration
 Break : causes control to break out of the loop

e.g. Validation of designation code

fname = desig.lst
while echo “Designation code : \c”
do
read desig
case “$desig” in
[0-9] [0_9] if grep “^$desig” $fname
>/dev/null
then
echo “code exists”
continue
fi ;;
*) echo “invalid code”
continue ;;
esac
done

Unix/sh/20
Until Statement
 Complement of while stm
 loop body is executed repeatedly as long as the condition remains
false

e.g. until false

do
ps -f
sleep 5
done

Unix/sh/21
for Statement
 Format :
for variable in list
do
<execute commands>
done

e.g. for x in 1 2 3
do
echo “The value of x is $x”
done
e.g.for var in $PATH $HOME $MAIL
do
echo “$var”
done
e.g.for file in *.c
do
cc $file
done

Unix/sh/22