The UNIX Operating System

UNIX Operating System

Objectives :
Learn about the Unix Operating System and develop the
skills
required to build software applications in the Unix Environment.

Contents :

1.

Introduction & History of UNIX

2.

File System

3.

Basic Utilities

4.

Shell Features

5.

Advanced Utilities

6.

Communication Features

7.

System Calls

Evolution

Primarily influenced by MULTICS

MULTICS
- Developed for GENERAL ELECTRIC
mainframe computer
- Interactive operating system
- Batch - mode characteristics

645

UNIX Operating System

- First Version developed in 1969 by
Ken Thompson on DEC PDP Series ( PDP - 7 )
- Business, Scientific and Industrial Users

Chapter 1
Introduction
of UNIX

and

History

Standard release - AT&T Versions

Version 3

Version 6

Version 7

Version 8

Written in C
Moved to PDP - 11/45 and
PDP - 11/70
Released in 1975
For use by universities only
Released in 1978
First commercial version but
primarly used in universities
32V - VAX version
UNIX Support Group(USG)
takes responsibility
Research Group still busy

Standard release - AT&T Versions

Contd...
System III

System V
Release 1
System V
Release 2
System V
Release 3
System V
Release 4.0

Released in 1981
Commercial orientation
Over 100 manufacturers of UNIX-based
micro , mini and mainframes
UNIX users > 100,000

Released in 1983
Enhancement of System III

Released in 1984
Enhancements of System V , Release 1

Released in 1988

BSD UNIX
(from Berkeley Software
Distributions)
* UNIX's entry into University of California, Berkeley Campus during
1976-77
* Inputs of UCB Faculty/researchers incorporated in System V of
AT&T UNIX
UNIX 3BSD

Released in 1978
Developed by Bill Joy and Ozalp Baboglu
Added Virtual memory, Demand paging
to the VAX version 32V

UNIX 4BSD

UNIX 4.1 BSD

DARPA - funded Development

Provided support for DARPA TCP/IP
C-Shell , ex/vi , new compilers for PASCAL
and LISP were added
Released in 1981

UNIX 4.2 BSD

Released in 1983

UNIX Features
Multitasking
Performing tasks simultaneously
rather than
sequentially
e.g., While printing a document , start
editing
another document
Each task is considered to be a process
Multi-user

Several users can use the same

computer

simultaneously
i.e , more than one keyboard and
terminal
can be connected to one computer
Highly cost-effective

Portability

Easier to modify the UNIX system

code for installation on a new
computer rather
than write from scratch a new operating

*
*

*
*
*
*

Available on micros, minis and mainframes

UNIX Communication and electronic mail
Communication between different
terminals
hooked to same computer
Communication between computers of
different
sizes and types in different locations as
far away
as other countries
Incorporation of third party application
programs
Hierarchical File System
On-Line Help facility
Example : man <Command>
Library of application software

The UNIX Operating System

Kernel( Heart of Unix Operating System)

Shell Interpreter
Tools and Applications

Kernel

Schedules tasks and manages data storage

Shell

A program that
interprets the user commands/requests
calls programs from memory and
executes them one at a time or in a

series
(called a pipe)
Tools &
OS

Specific capabilities added on to the

The UNIX Kernel

Networking
Services

File Management
And Security

Input / Output
Services

Date and Time

Services

UNIX
Signal
Handling

SYSTEM
KERNEL
System
Memory
Administration
Management
and Accounting

Process
Scheduling

The Programming Environment

*

A GOOD operating system !!

Simple yet powerful

Allows building of tools, writing of small

programs and putting them together

A rich & productive environment

A PORTABLE operating system !!

Written in C
Runs on a range of computers
Source code is available

Chapter 2
File System

The UNIX File System

1.
2.
3.
4.
5.
6.
7.
8.
9.

File System Layout

Concepts of file
Features of File system
Starting and Ending a session
File Management utilities
Directory Management
File operation
Mountable File System
File security

File Systems in UNIX

Disks are partitioned into File Systems.
Logical blocks of 512 bytes or multiples
Arranged to form a hierarchical structure
Convenient to manage data
File System Layout
Boot
block

Super
block

Inode
list

Data
block

Boot block
-

generally occupies the first sector

may contain bootstrap code

Information about the file system

How large, where free and other details

Contains a list of inodes

One inode is the root inode

Contains file and administrative data

Allocated data can belong to that block
only

Super block
Inode list
Data block

Classification of files in UNIX

Ordinary Files

Directory Files

Special Files

Standard Files

Concept of Ordinary Files

*

A File in UNIX is a Stream of bytes ( 8-bits)

Kernel does not impose a structure on files

File may contain text, data and even machine

language code

Examples

Text Files

Lines of ASCII characters

separated by a new-line

Commands

Sequence of commands
interpreted by UNIX text

Data

File containing data as

stream of bytes

Executable

File containing machine

language instructions

The files format remains unchanged but only the way

the program interprets it is different

Concept of Directory Files

*

Gathering together related files in a common place

Every file is assigned to a Directory

Directories have names (maximum of 14 characters)

A file within a Directory can itself be another Directory (A

Sub-directory)
Internally a Directory is just a file that contains
a list of file names
their Inode Numbers and
a pointer to the actual file on the disk

An upper limit of around 35 sub -directories

Examples
/ ( Forward slash )
Prime or Root Directory
Note - in DOS it is \ ( Backward slash )
/usr
Directory for the user
/usr/trg1
Directory usr

Sub-Directory for trg1 under

Concept of Special Files

* Used to reference Physical devices such as terminals, disks, floppy
drives and tape drives
*

Read and written like ordinary files

Requests cause activation of the associated Physical Device

Device Drivers associated with each file

* Types :
Character file
by
Block file
or 1024

Terminal (tty0, tty1 )

Disk Drives (hd0,fd0)

Transmits data Character

Character
Transmits data in 512
byte chunks

* Major and Minor numbers identify the hardware link

Major No.s : Indicates type of device
Minor No.s : Different instances of the device

Concept of Standard Files

*
*
*

Helps display information on the screen

Special Names for Communication channels
Keyboard input channel is called Standard
input
(stdin) file id is 0
Terminal Screen output is called Standard
output
(stdout) file id is 1
Diagnostic error messages (generated by a
program) are sent to Standard error
(stderr) file id is 2 ( shown on terminal
screen )
All three files are open by default at the time

File Names
*

A means for the system to identify a file

Note : Unix is case sensitive
* User accesses a file by a user-assigned
file
name
* Can be a sequence of characters (a
maximum of 14 )
*
May be identified by two or more names
(Multiple links to a file)
*
Internally assigned a unique inode
number
(for a File System)
* Different files can be grouped under a
directory

Path Names
Absolute Path name
*
e.g.,

A file is identified by the path name from the root

/usr/trg/c/test.c
where

test.c is an ordinary file

usr, trg, c are directories
trg is a sub-directory under usr

Relative path name

*
*
interpretation

UNIX keeps track of the user's current directory

If a "/" does not precede a file name then the name
begins with the current directory

e.g.,
If current directory is
/usr/trg
then the file could be just referenced as c/test.c

Features
* Hierarchical
* Security on each file
- Owner
- Group
- All others
*

Separate security for

- read
- write and
- execute

Removable

File Independence -

Time stamp on each file

- Modification time
- Access time

Unix
dev

bin

lib
/

bin

console
lp0
tty0

src

sh
csh
--Libc.a
---

uch

tmp

man
---

bin

local

lib
---

usr

include

spool
etc

spell
troff
---

*.h
tmac

passwd
group
init
---

lib

troff
---

tmp
---

Standard File Hierarchy

/bin

Basic Unix utilities

/dev

Special I/O device files

fd0, lp0

/etc

Administrative Programs

passwd, shutdown

/lib

Libraries used by Unix

libc.a

/usr/bin

Unix utilities

cal, bc,spell

/usr/adm
/tmp

Administrative commands
and files
Temporary files created
on error conditions

cp, mv, ln

adduser

/usr/games

Game Programs

/usr/include
procedure

Include files for language

Examples : C-header files
stdio.h, math.h
Archive libraries
Example : trof

/usr/lib

/usr/mail

Mail files
Example : mailbox

/usr/news

News files

/usr/spool

Spool files

/usr/tmp

Temporary files

Program Source Files

/usr/src

Security and Access Methods

*
*

Three levels of access

User/Owner, group, others
Three types of access on Files and Directories
Read, Write, Execute

Access Mode
Directory

Ordinary

Read

Examination of
File Contents

Listing of
files within
Directory

Write

Allows changing
of file contents

Creating new
files within
Directory

Execute

Executing file as
a command
allowed

Searching the
Directory

allowed

Starting and Ending a Session

login :
User can type his name and password to
identify himself
login command can be used as
$ exec login
to log-on onto another user account after identifying yourself
in response to prompts for user name and password
su

setuser

This is used to become another user or super-user provided

the password is known.
e.g.,
$su
Prompt the user for the superuser password $su - trg2
Prompt the user for the password of user trg2
"ls -l"

$su - trg2 -c

Temporarily changes to trg2 and executes the command ls -l

and
comes back to the original user

Starting And Ending A Session

Contd.
passwd
Change the password for the user
e.g., $ passwd
Prompt you for old password and new password
logout
user

This command exits or logs-out from the current

and executes the file .logout before coming out
e.g.,

$ logout
or
$ exit
or
$ <ctrl-d>

exits from the current login

File Management Utilities

Directory
Security
Management

Operation

cd
pwd
mkdir
rmdir
mvdir

File Comp.

cmp
comm

File
contents

File
compression

Mountable
file

cat
ls
wc
file

pack
unpack

mount
umount

passwd
crypt
chown
chgrp
umask
chmod

Copy, Move
Remove & Time
cp
ln
mv
rm
touch

mkdir

creates a new directory

rm

removes a file

rmdir

removes a directory

du

displays disk usage

df

displays number of free block

touch

updates the time of last modification

find

locates files that match certain area

file

displays the type of file

pwd

displays full pathname of current

directory

Command Structure
General Structure:
Command Name

[Options]

[Arguments]

E.g.,
Command Name
Tells the shell what to do
(in this case list files)
Options
Control how the command will work
(in this case ask for a long listing)
Arguments
on which the command works
(in this case the directory
usr/trg )

ls

-l

/usr/trg

Directory Management
cd
cd..
cd.
e.g.,
$ cd /usr/trg/c
$ cd ..
$ cd ./c
or $ cd c
$ cd
mkdir
$ mkdir pathname

Change working Directory

Parent Directory
Current Directory
( current Directory is c )
( current Directory is trg )
( current Directory is again c )
( home directory - in this case /usr/trg)
Make a Directory

Makes Directory in 777 mode

Write permission should at least be permitted for owner in parent Directory
e.g.,
$ mkdir /usr/trg2 ( makes directory trg2 )

rmdir
*
rm -r

Remove a Directory
$ rmdir pathname

Directory should be empty, or else

(recursively remove)

e.g.,
$ rmdir /usr/trg2

(removes directory

trg2)
pwd
Directory

Print Working

File Contents
cat

Concatenate & Print on screen or printer

$cat [Options] [Arguments]

Options -

take input from stdin

-n
no. of output lines
-s
squeeze adj. blank lines
-v
enable display of non-printing

characters
-b

used with -n to avoid numbering blank

lines
e.g.,
$ cat try.c
$ cat
displays on

Display the contents of try.c on the screen

Takes input from stdin i.e. keyboard and
screen

$ cat f1 > f2
$ cat f2 > f3
$ cat f4 >> f3
$ cat try[0-3] > final
$ cat test* > report

Takes input from file f1 & puts it

on file f2
f3 contains the contents of f1
Appends the contents of f4 to
file f3
The file final contains contents
of try0, try1, try2 try3
The file report contains all files
beginning with test

Is[Options]

List the Directory Contents

Options

-1

number one single column output

-l

long format (II also used)

-a

all entries including dot files

-s

gives no. of disk blocks

-i

inode no.

-t

ordered by modification time recent

first recursively display all directories,
starting specified or current directory

$ Is -I

List the files along with the protection bits and the user

$ Is -a

List the files starting with .and..also

$ Is -1

symtab.c
symtab.o
treegen
test

$ Is -I

-rw-rr 1 smj proj1 30766 Mar 3 15:08 symtab.c

-rw-rr 1 smj proj1 8759 Mar 3 15:12 symtab.o
-rwxr-xr-x 4 smj proj1 40743 Mar 3 15:23 treegen
drwxrwxr-x 1 smj proj1 53 Mar 1 09:15 test

$ Is -a

$ Is -iI

..
.profile
.cshrc
symtab.c
...

10936-rw-rrI smj proj1 3076 Mar 3 15:08 test.c

10936 - inode number of file test.c

wc

Word Count
$wc [Options] filename

Options

e.g.,

nl
the file

Display
-I
Display
-w
Display
-c
Display
$ wc test.c
20 200 5678
20 - lines
200 - words
5678- characters

no.
no.
no.
no.

of
of
of
of

lines, words, characters

lines
words
characters

no. of lines in the file and temply lists out

Similar to wc -l < filename >

file

Determine file types

$file [Options] [Arguments]

Options

-f
filelist
Normal File Types
C program text
assembler program text
commands text
ASCII text
English text
e.g.,

$ file test.c
C Program test

cp

copy a file
-i
e.g.,

In

- user interactive mode

$ cp test.c test.c.bak
test.c and test.c.bak contain the same contents
Extra disk storage
Create link

e.g.,
$ ln first.c second.c
The file is referenced by two different names
No Extra disk storage

mv

Moves or renames files and directories

-i

interactive mode

e.g.,
rm

$ mv old.c new.c
Renames the file old.c as new.c
Deletes the indicated file(s) files
rm
-i
-f
-r

touch

e.g.,

removes files and directories

remove interactively
forcible remove
remove recursively
Dangerous
used in conjunction with -i

Updates access, modification or change times of a file

-a
update access time
-m
update modification time
-c
prevents creating the file
$ touch f1
* The current system date & time stamp is put on the file f1
* If f1 does not exist then it is created with 0 bytes

File Comparison
cmp

Compare two files

If files are same no output is sent to the terminal, or else The

line number and
the byte at which the first difference occurs
is reported
-s

Outputs nothing Registers return code

Return code

0
1
2
e.g.,

if files are identical

if files are different
on error

$ cmp test1 test2

test1 and test2 differ in char 36 line 3
$ cmp -s test1 test2
$ echo $status
outputs 1 indicating that the files are different

diff

Reports more than one differences

-b
-e

convert

$diff [Options] file1 file2

Ignores trailing blanks
Gives a list of ed commands so as to
file1 into file2.

e.g.,
$ diff test1 test2
Outputs:

n1 a n3,n4
n1,n2 d n3
n1,n1 c n3,n4

where

*
*

respectively

n1 ,n2, n3 ,n4 are line numbers

a ,d, c means append, delete ,change

comm

Display common lines

$comm -[123] f1 f2
Prints a three column output:
- lines that occur only in f1
- lines that occur only in f2
- lines that occur in both
comm -12
- prints lines common to the two files
comm -23
- prints only lines in the first file but not in
the
second
comm -123
- prints nothing
e.g.,
$ comm test1 test2

Reports the common lines between

files test1, test2 and reports

the lines differing

$ comm -12 test1 test2 Prints line common to both

File Compression
pack

Compress the file

$ pack <filename>
e.g.,

$ pack try
- Creates a file try.z which is packed
- Normally the executables are packed
- The size is reduced by 25 - 40 %
unpack
or
pcat

e.g.,

$ unpack try.z
or
$ pcat try.z
unpacks the file try.z

Uncompress packed file

Mountable File System

mount
e.g.,

umount
e.g.,

Associates a directory with a device

Mounting a floppy on the root file system

Dissociates directory from the device

$ mount /dev/fd096 /mnt
$ umount /mnt

Mounts the floppy on

the directory /mnt
Dissociates /mnt from
the floppy

File Security
passwd

chown
file

To change the password

To change the ownership of the

$ chown owner filename

e.g., $ chown trg2 test.c

* Initially the owner is trg1
* Only the owner or the superuser
can
change the ownership
of the file

chmod

change the permissions of the file

$ chmod who op permission <filelist>

who
a, u, g, o
op
+, -, =
permission
r,w,x
execute

all, user, group, others

+ add, - remove, = set
r read, w write, x

e.g.,
$ chmod a=rw test.c
* users, group, others have read
and write permissions
$ chmod u+r, g+w, o+x test.c
* read for users write for groups execute for others
$ chmod 777 test.c
* Sets read, write, execute Permissions

umask

Set file creation mode mask

$ umask nnn
(nnn set file creation mode)
umask can also be set as a shell variable
e.g.,
umask 022
- Files normally created with 777 mode is assigned
755 permission
The value of each digit is subtracted from the corresponding "digit"
specified by the system for the creation of a file.

File Operation

tail

Displays the last lines of file

options : -n (n= no. of lines)
e.g., $ tail -30 test.c
Displays the last 30 lines of file test. c

head

Displays the top lines of file

e.g., $ head -10 test.c Displays the first 10 lines of test.c

split
lines

Splits the file into different files as

specified by the number of

e.g., $ split -20 test.c

Splits the file test.c in blocks of 20 lines and creates files xaa, xab,
xac
and so on, such that
xaa
has first 20 lines of test.c
xab
has the next 20 lines of test.c
...
The file test.c is unaffected
$ split-20 test.c try Generates files as tryaa , tryab , tryac
paste
terminal

Joins the two or more files horizontally

e.g., $ paste xaa xab
File xaa and xab are joined horizontally and output to the

Chapter 3
BASIC
UTILITIES

Basic Utilities

1.

Line Editor

Ed

2.

Visual Editor Vi

3.

Debuggers

4.

Profiling Tools

5.

C-Beautification

The Line Editor - ed

Invoking ed:
Prompt :
Append:
Write :

$ ed filename

Ed has no formal prompt

P - Prompts with a *
a

w
Exiting ed :
q
Example :

This is a toggle function

Appends given text after current line

Writing to the file

Quit from ed

$ ed test.c
? test.c
P
*a
<>
<>
<>
*w
*q
The edit buffer can be written to some other file also as *w filename

Pattern Matching
abc

abc

a*bc

abc,

aabc,

aaa...bc

aac,

acc,

axc,

abc,

axyc,

a$+pmc,

a[b2m]c

abc,

a2c,

amc

a[0-2b-d]c
adc

a0c,

a1c,

a2c,

a[^0-2]c

a3c,

a5c,

axc, ....

a[xy]*c

ac,

axc,

axxc,

a\{3,\}

aaa,

aaaa -

3 or more as

a\{3,5\}

aaa,

aaaa,
aaaaa
between 3 & 5 as

a.c
a.*c
abcdefgc

abc,

a$c

abc,

ayyyc

acc,

Adding and Replacing Text..

Command
Usage
p

Explanation
print current line (s)

list current line(s)

(display invisible

print line(s) with

1,3p
2,4l
characters like tabs etc.)
number before it

1,5n
a

append text after current line

i

3i

insert text before current line

4a

Traversing within a file

line

Command

Explanation

Pointer to current line

.=

Gives line number

Current line is n

+n

Examples
*-

*-

Current line

Move up by n lines

-n

Move down by n lines

points to the last line

short for 1,$

short for .,$

/.../

forward search for string

of characters enclosed between
slashes

Previous

*$= Prints
total no. lines
but does not
changecurrent
line

Modifying Text
Command

Explanation

Usage
d
2,5d
c

delete line(s) in text

move line(s) in text

copy line(s) in text

j
u

join lines in text

undo last command

change line(s) in text

2,4c
2,3m5
2,3t5
2,3j

Note : The above commands can be easily remembered by associating

them with the
first characters of their action.

Substitute Commands
(n,n)s/oldstr/newstr
replaces oldstr with newstr (n,n) range of
lines - optional
s/oldstr/newstr/g all oldstrs' on the current line
are replaced with newstr
s/oldstr/newstr/p the replacement is only effected in print but not
executed

Finding Text

/^ pattern/

searches for patterns at

beginning of line

/pattern $/
line

searches for pattern at end of

/pattern/

the pattern is searched forward

//
??

forward search
backward search

Command Execution

Command

Explanation

(n,n)g/findstr/commandlist Executes given commandlist for

every occurrence of findstr
(n,n)G/findstr/

Inputs one command to execute for

every occurrence of findstr

(n,n)v/findstr/commandlist Executes given commandlist for

every
non-occurrence of findstr
(n,n)V/findstr/

Inputs one command to execute for

every non-occurrence of findstr

Special ed Commands
Command

Explanation

print current filename

write contents of buffer into file

w file2

write contents of buffer to file 2

read current file

r file2

read contents of file2 after current line

e file2
!shellcmd

after current line

edit file2 > invoke ed on file2

Execute shell command by preceding with !
! ls l gives a listing of current directory

The ex editor

*
an improved version of ed
*
less terse
*
provides display options like
numbered
line
*
allows shorthand versions of
commands
*
clear response for error messages

The vi - Visual Editor

The

editor
*
*
*
*
*
*

Invoking

Powerful full screen editor

vi v/s ed, ex
Mostly single key stroke commands
Interface with ex
Macro facility
Ability to process text

$ vi filename
e.g., $ vi pgm.c

Modes :
*
*
*

Command mode
Insert mode
From Insert mode Pressing
<ESC> remitts Command mode

Types of commands
* vi-commands (invisible)
Command mode
Commands can be categorised as :
* Cursor movement
* Text manipulations
- insert, delete, copy, change
* Marking/Selecting, Positioning
* Search
Objects of interest recognized by vi:
* characters
* words
* lines
* block

Format of commands
[count] command [operand]
Use . to repeat last command
Use u to undo last command
Cursor Movement
Line Oriented :
^ or (zero)
$
Character oriented
h
l
j
k

beginning of line
end of line
move
move
move
move

left
right
down
up

Format of commands (contd.)

Word Oriented :
e

move to end of next word

move to beginning of word

move to beginning of previous word

move to end of next word

ignoring punctuation

move to beginning of word ignoring

punctuation

move to previous word ignoring

punctuation

Block Oriented :
)

move forward one sentence

move backward one sentence

move forward one para

move backward one para

move to find matching parenthesis

Rightly used in C- Programming for matching (

mark a particular line with a label.

e.g., ma (marks the line with label a)

return cursor to position (m)

e.g., "a moves the cursor to the line
which was marked with label a

^g

file information, line number

& )

Text Manipulation
Insert mode :
a
A
i
I
o
O
esc
Delete mode :
u
nx
ndw
ndd

append after cursor

at end of line
insert at cursor
at beginning of line
enter in new line after current line
enter in before current line
to exit insert mode

undo last command

delete n- next character
deleting n next words
deleting n lines current line and n-1 below current
line are deleted.
delete till end of line

Changing Text :

mode

nr
s

replace char
delete current char, enter insert

S
cw

delete line, enter insert mode

change word

Cut & Paste :

nyy

yank n line
Named Buffers : a to z ( 26 )
Unnamed Buffers : 1

print buffer

join next line to current EOL

Join :

Search
Searching for a character :
fc

search forward for character c

tc

similar to f, cursor placed 1 char left of c

Fc

search backward for character c

Tc

similar to F, cursor place 1 char right of c

continue search in same direction

continue

search

in

reverse direction

Search
File Related :
G

go to the end line

nG

go to the nth line

go to the middle line

go to the top line

go to the last line

Text Manipulation

Screen Adjustments :
^d

Scroll down half a screen

^u

Scroll up half a screen

^f

Page forward

^b

Page backward

^e

exposes one more line at bottom of

screen

^y

exposes one more line at the top of

screen

Text Manipulation
Searching for a string :
/string

for searching forward

?string

for searching backward

Use n to continue search in the same direction
Use N to continue search in the opposite direction

s/oldstr/newstr

commands can be used in vi

Common Set commands

set ai

Causes automatic indentation

set noai

Nullifies the effect of auto-indent

set nu

Causes line numbers to be displayed

set wrapmargin = n

Sets n column right margin

set bf

The beautify option Removes all

unimportant control characters

set

Causes a displays of current set options

Text Manipulation
:w!

file force write

:q

quit normally

:q!

quit, no write

:n

next file, for $ vi f1 f2 f3 f4

:n!

edit next file, discarding changes

to current file

:n args

specify new argument list

PROFILING TOOLS :
PROF

and

GPROF

Improves efficiency and debugging

gprof is available in BSD versions

*
The program should have been compiled with -p
option
for prof and -g option for gprof
*

prof - refers default input file mon.out

gprof - refers default input file gmon.out

*
Gives an idea as to which function used up the
maximum
time

Example of prof
$ cc -p test.c
$ a.out
$ prof a.out
Generates an output like....
%time
msec/call
91.2
7.2
1.0

cumsecs

name
9.34
9.34
10.08
0.74
10.18
0.10

0.0

10.24

%time

Percentage of time consumed by the function

cumsecs

Running sum or cumulative sum of the functions called

seconds

Seconds consumed by the function

0.06

seconds
20100

#calls

_func2
1
100
1

calls

Number of times the function was invoked

name

Function name

acnt
_func1
_main

Chapter 4
Shell Features

Shell Features

Shell Features
1.
2.
3.
4.
5.
6.
7.
8.

Various kind of shells

Shell commands
Shell Startup Dot files
Shell variables
C Shell
Test conditions
Shell programming - Bourne Shell
Shell programming - C Shell

Various types of Shells :

1.

Bourne Shell

sh

2.

C Shell

csh

3.

Korn Shell

ksh

4.

Restricted Shell

5.

Visual Shell

rsh
vsh

Shell Commands
Simple commands using shell Metacharacters
Exit status

Redirection

file
file

Pipeline

return value 0 - Successful

return value > 0 - Failure

Default
Default
<
>
>>
n>&m

input from Keyboard

output on the Screen
- Read from a file
- Redirect it to a file
- Append to a file
- Merge output from file descriptor n with
descriptor m
n<&m - Merge input from file descriptor n with
descriptor m
where n and m are file descriptors for the
stdin, stdout and stderr files
sequence of commands separated by |

sequential execution

&

asynchronous execution

&&
returns

following list executed if preceding pipeline

||
returns

following list executed if preceding pipeline

nonzero value

matches 0 or more characters

matches any single character in filenames

$var

value of shell variable var

zero value

var=variable assign variable to var

rest of the line is a comment

...

run command within backward quote

...

treats ... literally

...
... and \

treats ... literally except for $ and

e.g.,
$ (sleep 5 ; echo The date is `date` ) & date
The Output of the above command will be
6345
Wed Sep 23 10:20:45 EDT 1991
$The date is Wed Sep 23 10:20:50 EDT 1991

PATH
*

If full path is given while executing the command, the command is

executed from the path specified

Else it is searched in the Shell variable $PATH

Search order is defined by value of Shell variable Path

e.g.,
PATH = :.:/usr/bin:/bin:

Shell Startup Dot files

Helps in customizing UNIX System

Executes .profile for Bourne Shell when user logs on

Executes .login and .cshrc for C Shell when user logs on

Executes .logout for C Shell when user logs out from C-Shell

Keeps track of the history mechanism In C Shell through the .history

Shell Variables
User defined
assignment : name = value
e.g., (on Bourne shell)
$
$

x=Hello how are you

echo $x
Hello how are you

$
$
$

PATH=$PATH:/usr/games
export PATH
echo $PATH
:.:/usr/bin:/bin:/usr/games:

Positional Parameters
e.g.,
shellscript
arg2
$0

arg1
$1

$2

Evaluation of Shell Variables

* $var

value of var ; nothing if undefined

* ${var}
characters

same as $var ; useful if alphanumeric

follow variable name
- derefrencing variable

e.g.,
$

var=hello

varx=goodbye

echo $var
hello

echo $varx
goodbye

echo ${var}x
hellox

$ {var?message} If defined , $var

Otherwise print message
if message not defined print
default message
e.g.,
$
echo ${var?}
hello

echo ${junk?}
junk : parameter not set

echo ${junk?my message error}

junk : my message error

${var-thing}

Value of var if defined ,

else use thing
$var unaffected

* ${var=thing}
thing
e.g.,

Value of var if defined ; otherwise

$var is set to thing

echo $ {junk-Hi there}

Hi there

echo ${junk?}
junk : parameter not set

echo $ {junk=Hi there}

Hi there

echo ${junk?}
Hi there

* $ {var+thing}
nothing

If var defined then thing Otherwise

Command Environment
*

Variables & associated values

Shell 1
Shell 2
Export A

e.g.,
export A
*

Export from parent to child shell

Vice-versa not true

Predefined Special Parameters

$#

number of positional parameters

$?

exit value of last command

$$

process number of a process

$*

all the parameters

$!

process id of last command started with

asynchronous execution i.e., &

Variables Used by Bourne Shell in .profile

CDPATH

search path for cd

HOME

login dir

PATH

search path for commands

PS1

primary system prompt

PS2

secondary system prompt

IFS

internal field separator

MAIL

files containing mail messages

TERM

terminal type

EXINIT

list of set commands for vi

The C Shell

Developed by Bill Joy ( Berkeley University

More helpful interaction

Permits shorthand repetition

Permits aliasing

The variables can be defined using a set for

C-Shell variables
e.g., (on - C shell)
% set x=date
% echo $x
Wed Sep 23 12:15:20 EDT 1991

History
execute previous

Records previous commands ! is used to

command

Command Format :
! <event> [ <word> <modifier> ]
<event>
- absolute
- relative
- name
expr
<word>-

!1
!-3
!c
!?su?

absolute
0, 1, ...n
range
a-e
last
$
All arguments *

<modifier>

- Substitute
:s/trg1/trg
- Print only
:s/trg1/trg/p

History Mechanism
Keeps track of commands
% history

Lists previous commands

% ^old^new

Modify last command

e.g.,
% cd /usr/trg/SRC
% ^SRC^src
% cd /usr/trg/src
% !15
% !c
beginning with c

Run command 15
Run previous command

Changing

Commands

:S/pattern/replacement
e.g.,
% !cd:s/trg/trg1
cd /usr/trg1/src
% !ls : p
Prints the previous command
beginning with ls
but does not execute it

Using Arguments

!$

last argument

!^

Begin argument

!*

All arguments

!10:5-9
command 10
e.g.,
%ls !15:2-4

Arguments 5 to 9 for

Alias mechanism
Short names for frequently-used long commands
e.g.,
%
%
%

alias cdms cd /usr/man/man\!$

cdms 2
pwd
/usr/man/man2

% alias cd cd \!*;set prompt=`pwd`>

sets the prompt to the present working directory as and
when cd is used to change the working directory.

Filename Grouping
{

%cp

Attach filenames to common root

/usr/trg4/{ac,docs,test} .
%cp/usr/trg4/ac .
%cp /usr/trg4/docs .
%cp /usr/trg4/test .

%ls

{/usr/bin,/bin}/{ls,more}
/usr/bin/ls : not found
/bin/ls
/usr/bin/more
/bin/more

: not found

Using C - Shell
e.g.,
% cc test.c
% vi test.c
% !c
% a.out
% !v
% !c - o test
% test
% car /usr/trg2/document/report
% !! : s/car/cat
or
% ^car ^cat

Variables in .cshrc of C SHELL

% set variable=value
histchars :
% set histchars=";,"
will thereafter use
; as execution character
, as substitution character
history :
% set history=40
path:
% set path=:.:/bin:/usr/bin:
prompt :
% set prompt=\!>

Variables in .cshrc of C SHELL

that

setenv

used to make a variable as an environment variable so

whenever changes are made it is also reflected in
the environment like TERM , DISPLAY is used
% setenv TERM=AT386-M

filec
% set filec
Its usage :

:
file name completion
:
Predefined
% set variable
% unset variable

ignoreeof
% set ignoreeof
noclobber
:
redirection
% set noclobber
appends to

To ignore all eof character ^D

Abandons all command using output

which destroys an existing file and
non-existing file

Test Conditions
TESTing Strings
test - z $string String length equal to zero
test - n $string String length not equal to zero
test $str1 = $str2

str1 is equal to str2

test $str1 != $str2

str1 is not equal to str2

test $string

string is not a null string

To algebraically compare variable values

test $num1 -eq $num2
Other algebraic operators

-ne, -gt -ge -lt, -le

File related
$ test -f file

True if file exists and is an ordinary file

$ test -r file

True if file exists and is readable file

$ test -w file True if file exists and is writable file

$ test -s file

True if file exists and has size greater than zero

Shell Programming
- The Bourne Shell
Control flows

if command
then
commands
[ elif commands ] ...
[ else commands ]
fi
case word in,
pattern) commands ;;
pattern) commands ;;
...
esac

for name [ in word ...]

do
commands
done
while command
do
commands
done
until command
do
commands
done

break n
continue n

exit from for / while / until from level n

next iteration of for / while / until from level n

exit n

exit with value

export [name..]

export

test expr

evaluate conditions

read [arg]

read variables from stdin or Keyboard

shift n

shift positioned parameters w.r.t $1

Examples
$
>
>
>
>
>
>
>

for i in *.dat
do
size=wc -c $i
if test $size -eq 0
then
rm $i
fi
done

- The complete for loop can also be edited in a file and, by

granting the
execute permission on the file , the Shell program can be
invoked
- By default any shell program is parsed to Bourne shell
However, by mentioning #! /bin/sh or #! /bin/csh in the first
line of the
shell program, it can be parsed to either Bourne shell or C-shell
resply

Example

echo If you have a TTY vt100 enter vt100

echo If you have a TTY AT386-M enter AT386-M
echo If you have a TTY sma84 enter sma84
read term
case $term in
vt1001) TERM=vt100
;;
AT386-M) TERM=AT386-M
;;
sma84) TERM=sma84
;;
*)
echo Invalid TERM
;;
esac
# Making it is an environment variable
export TERM
echo end of program
This can be a part of your .profile

IF

Control Flows
if (expr) then
command;
[ else if ]
command;
[ else ]
command;
endif

SWITCH

switch (string)

command;
breaksw
case string2:
command;
breaksw
default :
command;
breaksw

case string1:

FOREACH

foreach name (word)

commands
end

WHILE

while ( expr )
commands
end

UNTIL

until ( expr )
commands
end

Chapter 5
Advanced Utilities

Advanced Utilities
1.
2.
3.
4.
5.
6.
7.
join, csplit
8.
9.
10.
11.
12.
13.
14.
15.
16.

Redirection
Pipes
Pipe fitting with tee utility
Filters
Find
Sort
Utilities that process tabular data- cut,
Ps and Kill
Timely Execution- nohup, at, time
User backup utilities- tar, cpio, dd , doscp
Output related commands- pr, lpr,lp
Other utilities- sleep, sync, wait, clear, tr
Introduction to Make
Lint
Lex
Yacc

Redirection

e.g.,

Metacharacter

Performs

<

input

>

output write

>>

output append

$ who
trg1
trg2

tty00
tty02

Apr 8
Apr 8

09:33
11:10

Apr 8
Apr 8

09:33
11:10

$ cat date_out
Fri
Apr 8

14 : 30 :

10 est

$ who >> date_out

$ cat date_out
Fri
Apr 8
trg1
tty00
trg2
tty02

14 : 30 : 10 est
Apr 8
09:33
Apr 8
11:10

$ who > who_out

$ cat who_out
trg1
tty00
trg2
tty02
$ date > date_out
1983

1983

Pipes :

Metacharacter
|

Performs
piping

e.g.,
Without Pipe

$ cat /etc/passwd > temp

$ sort < temp
$ rm temp

With Pipe

$ cat /etc/passwd | sort

* No need for creation/deletion of a file
* Sorts the file passwd as per the first
entry in passwd

A UNIX Pipeline
Standard Input

>Prgm #1

Prgm #2

Prgm #3

>Prgm #5

Standard Output<

Prgm #4

Prgm #6

The TEE Utility :

Helps in creating the intermediate file during the

pipe
operation
e.g.,
$ ls *.c | tee Cflst | sort
$ cat cflst
*
*

Creates the intermediate file Cflst

Cflst contains the list of c files

FILTERS :

grep

get regular expressions only

fgrep

fast, several simple strings at one

egrep
powerful

extended grep, can handle more

expressions like | - or operators

time

$ grep pattern filenames

Options :

-c

number of lines matched

-i

ignore case

-n

print line with line-number

-v

print lines which do not match

Searching Files with grep

e.g.,
$ grep -ni func1 *.c
that match
ignoring the case

Prints all the lines and line

numbers in files *.c
pattern func1

$ grep * *

Search for the pattern * in all the

files

$ ls -l | grep ^d

Searches for all subdirectories

e.g.,
$ egrep int|long test.c
Searches for all those lines
containing
either int or long in test.c
$ egrep (^[A-Z]) testfile

$ cat exprfile
commas
,.*,.*,

Searches for all the lines which

start with a capital letter

Searches for lines having at least 3

in file testfile using the

exprfile
$ egrep -n -f exprfile testfile

Find
To locate one or more files
find path-list cond-list action-list
path-list
cond-list
-type x

one or more directory names

-user name
-group name
-size n

for a specificed user name

for a specified group name
File size n. blocks
If n follows c then so many characters
locate for file with n links
files accessed during n days ago
modified n days ago
created n days ago
permission flags match nnn
with specified filenames

-links n
-atime n
-mtime n
-ctime n
-perm nnn
-name fname

x can be d, f, or c
d directory
f ordinary file
c character special file

actionlist
- print

display path names

- exec command

execute the unix command

- cpio device

copy the files on the tape in specified format

- ok command

like exec , but executed interactively

e.g.,
find / -atime 7 -print
find / -atime +7 -print
find / -atime -7 -print

will print files accessed in

exactly 7 days
will print files accessed
earlier than 7 days
will print files accessed
within 7 days

Examples
$
$
$
$
$

find
find
find
find
find

/ -atime +30 -print

/ -atime +30 -size +10 -print
/ -atime +30 -size +10 -ok rm -f {} \;
. -perm 0777 -print
. -perm 0777 -print -exec rm {} \;

find / -name passwd -print

Will find the passwd file

Sort

*
*
line.

Sort keys can be fields or lines

A field is a string of characters separated by a field separator or new

$ sort {-options} {+post1} {-post2} {files}

The sort key begins at post1 ands ends just before post2. There can
be several keys.
Options
b

ignore leading blanks and tabs

only check if input file is already sorted

dictionary order letters, digits, blanks

ignore case

ignore non-printable characters

n
r
tp
u
o filenameout

numeric sorting
reverse order
use p as separator
output only lines with unique keys
save output in filenameout

e.g.,
$ sort -t: +2n -3 /etc/passwd
Print the passwd file sorted by numeric user id
$ who | sort +4n
Sorts according to the login time stamp

of the user

uniq

Finds and eliminates duplicate lines in

a file and is often used with sort
$ sort <file> | uniq -c
Sorts and ouputs the number of count of lines
containing unique fields
$ sort <file> | uniq -d
Gives only the duplicated lines

tsort

Accepts as input a partial ordering and produces a

fully ordered list of the items.
$ tsort psortfile

Processing Tabular Data

Cut
*

Deletes columns from a file producing a new file

with shorter lines

Cuts out selected fields of each line in a file.

*
which

Cuts columns from a table or fields from a file

can be of type
-

Fixed length fields

or
Delimited by some character

Cut (contd.)
cut -c list { file1 file2 ...}
e.g.,
cut -c 5-70 file1
cut would pass 5-70 characters from file1
cut -flist {-d char } {file1 file2....}
e.g.,
who | cut -d -f1
gives a list of user login names
cut -d: -f 1,5 /etc/passwd
gives a list of user ID and names

JOIN
*

Combines corresponding lines in two files by relating the

contents of one or more columns.

*
files

Implements a relational data base join on two tabular

-jn m

join on the mth field of file n

-on.m

output mth field of nth file

n - file no.
m - field no.

-tc

use char c as separator

e.g.,
$ join -j1 4 -j2 3 -o 1.1 2.1 1.6 -t: etc/passwd
etc/group
-

joins field group id

outputs the following parameters

login
name

group
name

login
dir

csplit

context split

$ csplit [-k] [-f prefix] [-s] file name arg1 [..argn]

Reads file and separates it into n+1 section defined by arg1...argn
Options :
-s

Normally csplit prints the character counts for each file ,

-s is to suppress this

-k

csplit removes previously created files in case of error ,

-k is to avoid this
csplit normally creates file as xx00 , xx01 ... , xx99

xx

-f prefix creates file with that prefix instead of default

csplit(contd.)
e.g.,
$ csplit -f cobol inpfile /Procedure division/
/ Para5./ /Para6./ /Para 7./
*
*
*

Creates file as cobol 00 .. cobol 03

Edit these Cobol files
Can be recombined as
$ cat cobol 0[0-3] > file

csplit -k prog.c %main(% /^}/+1 {20}

*

Breaks file prog.c containing C routine

upto a maximum of 21

% expr % > no. file is created for this section

No csplit for the main routine %main%

PS, kill
$ somecommand &
5511 - pid
$ps
pid

tty

time

3432
5765
5511

2
2
2

0 : 24
0 : 03
0 : 51

tty

time

2
2

0 : 24
0 : 03

$ kill 5511
$ ps
pid
3432
5985

command
-sh
ps
somecommand

command
-sh
ps

$ stubborn-cmd &
pid
3432
6004
5995
$ kill 5995

tty
2
2
2

time
0 : 24
0 : 03
0 : 44

command
-sh
ps
stubborn-cmd

pid
tty
3432 2
6004 2
5995 2
$ kill -9 5995

time
0 : 24
0 : 03
0 : 44

command
-sh
ps
stubborn-cmd

time
0 : 24
0 : 03

command
-sh
ps

$ ps

$ ps

pid
3432
6103

tty
2
2

Timely Execution :

time

Time a command

$ time wc test.c > wc.test

real
2.0
user
0.4
sys
0.3
nohup

Protecting a process from hanging and quit

signals
or interrupts
The standard output is sent to nohup.out
e.g.,
$ nohup du / &
820
Sending output to nohup.out
$ logout

nice

*
*

Executes at specified priority

Default priority is 24

$ nice +n

Raise priority

$ nice -n

Lower priority

$ nice n

set priority

at

Executes process at specified time

$ at 5 pm
echo ^G ^G Time to logoff >/dev/tty04
^D
$

User Backup Utilities :

tar

tape archiver
Copies files on backup medium such as floppy
or tape in tar format

tar [options] devicename filelist

Options :
c
r

create a new tape backup tape old files are overwritten

append files to the tape

t
x

list the names of files from backup tape

extract files from the backup medium

update a tape, if the named files are not present or have

been modified later on
verbose ; provides informational messages, such as the
name of each file as and when it is encountered

v
f

devicename use device for backup medium

tar (example)
$ tar cvf /dev/fd096 *
directory
medium /dev/fd096

copies all files from the current

onto the backup

$ tar xvf /dev/fd096 Extracts all the files from the backup
medium
/dev/fd096 onto the
current directory
Creates necessary directories
tar xvf /dev/fd096 try.c
/dev/fd096
tar tvf /dev/fd096

Extracts the file try.c from

Generates a file list of /dev/fd096

find / -mtime -7 -exec tar uf /dev/fd096 {} \;

All the files modified last week is backed up onto /dev/fd096

cpio

*
Copy files archives in and out
*
Bundles all the files into one package
cpio [options] file list devicename
Options
-o

Copy these files onto tap

-i

Extract files from tape

-p

Read a list of file or path names from the standard input

-v

display a verbose set of cpio action

-c
character information in ASCII for portability
considerations
-t

For listing files from the backup medium

-O

Append files to the end of tape to be used with -A

-d

Creates necessary directories

-k

In case of error , ignore the error and continue

cpio (Examples)

$ ls *.c | cpio -ocBv > /dev/rmt/0n

Lists the files *.c and copy onto the device /dev/rmt/0n
$ cpio -ocBv < filelst > /dev/rct/c0s0
Copies all the files mentioned in filelst onto the catridge tape
$ cpio -icBdv < /dev/rct/c0s0
Extracts all the files from device c0s0 and creates necessary required
directories
$ cpio -itv < /dev/rmt/0m
List the files from the tape /dev/rmt/0m

dd

convert, reblock, translate, copy a tape file.

dd {option = value }
Options

Values

if
of
ibs
obs
cbs
skip
seek
count
conv

input file name

output file name
input block size (Default 512)
output block size(Default 512)
conversion block size
n records before copying from I/P file
write after n output records in O/P file
n records
To ASCII, EBCDIC, lcase, ucase (separated by
a comma)

e.g.,
$ dd if=/dev/rmt/0n of=x ibs=800 cbs=80 conv=ascii,lcase

df
clear
goes to
tr
e.g.,

displays no. of free blocks

df [option] file system

clears the screen and the prompt

top of screen
replaces specified characters with other
characters
$ tr [a-z] [A-Z] < lfile > ufile
Replaces all small case letters to upper case letters
$ tr -d < tstfl > tstfl2
Deletes all the blank characters
$ tr -s \012 < try > try2
Removes adjacent blank lines in file try

Chapter 6
Communication Features

Communication Features

1.

User communication commands

mail
write
talk
post
news
mesg

2.

Networking commands
uucp
uuto
rlogin
telnet
ftp
rcp
uux
ct
cu

User Communication Commands :

*

Mail

Write

Talk

Post

News

Mesg

Mail

Sends and receives mail messages between

users
Sending mail on local system
At the end of the message press ^D to exit
$ mail <loginnames>
Example : (Assume the current user is trg)
$ mail trg1
Sending mail on remote system
$ mail recipient@remote_system.domain_info

Receiving mail
$ mail

Shows the message received

?
q
s [file]
w [file]
d
n or +
r
m [user]
! cmd

Prompt
quit
save (defalut mbox)
write without header
delete
next message
previous
reply to the sender
mail to user
run the shell command

* The message could be saved or deleted

* Environment could be setup in .mailrc

User Communication commands Contd...

Write

Allows sending messages to a user terminal

from
another terminal
The message is ended by pressing ^D.
$ write <username>

Talk

Allows interactive dialogue between users at

two
different

terminals.

$ talk <username>

Post
Messages can be posted on a bulletin board for
previewing by all users.
News

first

*
Keeps the user informed of current events
Refers file /usr/news

e.g.,

$ news

Prints all the news , latest

$ news reorg Prints the news named reorg

mesg The terminal can be write protected to prevent other
users from
writing onto your terminal
e.g.,

$ mesg [option]
option : - y
write allowed
-n

mesg is y or n

$ mesg

remove write permission on the

terminal
prints the current status whether

Networking Commands
*

uucp

uuto

telnet

rlogin

ftp

rcp

uux

ct

cu

uucp

* allows copying files from one unix system

to
another
* Refers file uuname and uulog
uucp [options] source-files destination files
options

-d make necessary directory

-c do not copy local file to spool directory
-m send mail to the requester when the
copy is completed

$ uucp -d -m file? sys_2!/usr/spool/uucppublic

Copies the files file? creating
appropriate
directories onto system
sys_2 onto
/usr/spool/uucppublic

uuto

*
*

Simplified version of uucp

Sends file to /usr/spool/uucppublic

$ uuto filename(s) system!login

$ uuto -m test.c sys2!trg2
Sends a mail to the sender when the job is over
rlogin

*
Allows remote login over the network
*
Similar to telnet, but a much more flexible
facility.
$ rlogin system_name!username

Networking Commands Contd...

ftp

*
*

File transfer protocol

Transfers files from one system to another

$ ftp
ftp > open sys1

Name login:
passwd

ftp > ?

Gives you the help

ftp > get filename

Gets the file onto the current system

ftp > put filename

Puts the file onto the \

sys1

ftp > close

rcp

Similar to ftp, but a much more flexible

facility
uux
remote

* Unix to Unix command execution on

systems
* Gather files from various computers, run a
command on a specified computer and

redirect the
stdout to a file on a specified computer
$ uux [options] commandstring
All shell special characters must be quoted
$ pr minutes | uux -p host!lp
Command line queues the file minutes to be printed
on the area of the computer host

ct

*
*

Connect terminal to remote terminal

Connects your terminal to remote
terminal which is equipped with a

modem
This command dials the phone number of the modem
$ ct [options] telno
$ ct -h -w5 -s1200 9=5553497
1200
is not

ct will call modem using a dialer operating at a speed of

baud , wait for 5 minutes before quitting if dialer
available and use the -h option not to disconnect the

local

cu

*
*

connects local system to remote system

allows you to be logged on to both

simultaneously
Examples
cu -s2400 9=5557867

Connected
login :

~%take proposal
computer to local
~%put minutes minutes.bak
file minutes on the
minutes.bak

Takes proposal from remote

computer
Vice versa of take and puts the
remote computer as

~%!comma

run the command on local system

~%$command
output

run the command on local and send the

to the remote terminal

~%.

terminate the link