You are on page 1of 172

Overview of UNIX

1. “UNIX: Volume I and II” by Sumitabha Das,
2. “C and UNIX Programming: A Conceptual
Perspective” by Narayanaswamy Kutti, TMH.
3. “Linux Unleashed” by K. Husain, T. Parker,
Techmedia SAMS Publishing.
4. “The UNIX Programming Environment” by B.
W. Kernighan, Robert Pike, PHI.
G.Shyama Chandra Prasad 1
Overview of UNIX
 Session 1
UNIX as Operating System
(Slides 3 - 48)
 Session 2
Shell Programming and Shell Scripts
(Slides 49 - 75)
 Session 3
Communication Utilities
(Slides 76 - 92)
G.Shyama Chandra Prasad 2
Session 1
UNIX as Operating System
 Introduction to UNIX
 Structure of UNIX System
 The UNIX File system
 Filters and Pipes
 awk and vi Editor

G.Shyama Chandra Prasad 3

Introduction to UNIX
 It is a time-sharing, multi-user OS with
elegant, featureless file system, a
command interpreter (shell), and a set
of utilities (tools/commands, over 200
 Developed by Ken Thompson and
Ritchie originally in assembly, and later
in C, thus making it portable to other
 Supports C, Fortran, Basic, Pascal,
COBOL, Lisp, Prolog, Java, Ada
G.Shyama Chandra Prasad 4
Introduction to UNIX ...
 Facility of interconnecting commands
through pipes and filters permit the user
to create complex programs from simple
 Facility of background processing helps
the user in effective utilization of time
 It is an OS for Programmers - shell
provides the programming facility
 Security is in-built through the user
name and password, combined with the
access rights associated with files
 Types: Sco UNIX, Sun UNIX, Xenix, LINUX
G.Shyama Chandra Prasad 5
Structure of the UNIX System
 Shell and kernel together make UNIX system
Utilities or Shells
Other Application commands interact
& System Software user with users
with H/W user

G.Shyama Chandra Prasad 6
The UNIX Kernel
 Kernel is a collection of programs mostly written in
C, that runs directly on the hardware - so parts of
kernel must be customized to each system’s
hardware features.
 Kernel is directly loaded into memory when the
system is booted. Low level jobs are its basic
 System Initialization
 Process/Memory/File/I-O Management
 Programming Interface
 Communication Facilities
(The Block Diagram of a Kernel)
G.Shyama Chandra Prasad 7
The UNIX Process
 A process in UNIX is a program in execution with definite
life-time and well-defined hierarchy.
 The context of a process is a snapshot of its current run-
time environment that consists of the current state of the
processor registers and –
 User program image - Process execution on user data
structure that represents an application and is manipulated
by its own functions (user mode execution).
 System image - Process execution on system’s data
structures that represent the resources (memory, files,
devices) and supported by the kernel routines. Depending
upon resource requirement and availability, process’s
states are controlled by executing the kernel routines
accessed through system calls (system mode execution).
 The kernel maintains a process table to manage all
processes. The two data structures per process are the user
structure and the process structure.
(Structure details).
G.Shyama Chandra Prasad 8
The UNIX Process …
 The kernel process is the first (root) process that comes into
existence when the system is booted. Its process_id and
group_id are both 0.
 In establishing a multi-programming environment, the
kernel process then creates the init process with its
process_id of 1 and group_id, 0, showing that process 1 is
part of process 0. The init process creates and manages
terminal processes for active terminals in the system.
 At the time of creation, each terminal process belongs to
process group 0 and is made to execute a program called
getty. Once each terminal process (now called a getty
process) establishes communication with the associated
terminal, it displays a login message and waits for a user to
input a user name (Process group of the init process).
G.Shyama Chandra Prasad 9
The UNIX Process …
 When a user starts a dialogue, the getty process receives
the user name and leaves the task of validating the user’s
password to another program called login. The same
terminal process is now called a login process.
 The login process, after validating the user’s password,
calls a command line program, the login shell to run in the
same process. The same terminal process is now called a
shell process.
 Each shell process now establishes a new process group
and becomes ready to process the user commands. A shell
process is the initiating process by which each terminal
maintains the user session.
 While interpreting a command, the shell creates an
execution thread and then assigns the requested command
program to this new process.
 Both the shell and the new process proceed independently
in separate execution threads. The parent shell process
normally waits until child process completes its execution.
G.Shyama Chandra Prasad 10
Environment around the UNIX Process
 When a process is created, UNIX opens 3 streams
stdin/stdout/stderr for basic communication with respect to
the process control terminal. In addition, it knows the
current working directory for performing file I/O.
 Each login shell maintains a description of the
environment as a table of pointers to strings.
extern char USER=ashok
**environ LOGNAME=ashok
A UNIX process PATH=.:/bin:/usr/ahok/bin
 A global shell environment pointer called environ is
maintained by UNIX kernel and it can be used by a
process to access its own table
 The shell does not directly use this table, but it creates a
child process and calls exec() system call to execute a
command program that uses the table inherited from the
shell parent. G.Shyama Chandra Prasad 11
Environmental Variables (EVs)
 Children inherit the entire execution environment
from the parent. Some examples of the
environment variables are the USER, LOGNAME,
 The HOME Variable
It specifies an associated directory with every user
in a UNIX system. If the HOME variable for the
user Sita contains /usr/sita/stores, every time Sita
logs in, she is taken to the directory stores.
The variable HOME is referenced the same way:
$ echo ${HOME}<Enter>

G.Shyama Chandra Prasad 12

Environmental Variables (EVs) ...
 The PATH Variable
contains a list of all full path-names (separated by a
colon) of directories that are to be searched for an
executable program. For example, the command
$PATH=.:/usr/bin:/bin<Enter> specifies directories
to be searched for any executable file or a
command file (current directory, /usr/bin and /bin,
in that order).
 The PS1 Variable
The system prompt may be changed by setting the
value of this variable to the desired prompt:
$ PS1=“Hello>”<Enter>
Hello> #can be changed only at the UNIX
command line, not within a shell script.
 The PS2 Variable: prompt string for continued
command line (default ‘> ‘).
G.Shyama Chandra Prasad 13
Environmental Variables (EVs) ...
 The LOGNAME Variable
contains user’s login name. Its contents cannot be
changed by the user, but can be displayed:
echo “${LOGNAME}”
 The TERM Variable
Names the kind of terminal you are using; setting it
helps to manage your screen more effectively, say,
$ TERM=vt100<Enter>
 The PWD Variable
The current working directory can be displayed:
echo “${PWD}”
In fact the whole environment table can be
 $IFS: String of characters which are used as word
separators in command line ( space, tab, newline
G.Shyama Chandra Prasad 14
Environmental Variables (EVs) ...
 The MAIL Variable
Names the standard file where your mail is kept
 Variables for Abbreviation
It is useful if frequently referred names are
abbreviated: d=/very/long/directory/name
 The .profile File
Some of above variables like HOME and
LOGNAME are set automatically each time the
user logs in. The others, however, have to be set.
The .profile is used for this purpose as it is
executed as soon the user logs in. A sample .profile
file would look like:
G.Shyama Chandra Prasad 15
Utilities and Applications
 UNIX utilities or commands are
collection of about 200 programs (ed,
sed, awk, vi, grep, make, link, debug, etc.)
that service day-to-day processing
requirements. They are invoked through
the shell, which is itself another utility.
 More than a 1000 UNIX-based
application programs like DBMS, word
processors, language processors,
accounting software, etc. are available
from independent vendors
G.Shyama Chandra Prasad 16
The UNIX Shell
 It is the interpreter of UNIX to decipher and
execute commands - user interface to the kernel
for isolating the user from the knowledge of
kernel functions.
 Maintains interactive dialogue with the user and
is capable of Input/Output Redirection. Can do
background processing so that time-consuming,
non-interactive tasks can proceed side by side.
 Makes it possible to connect a couple of
commands with one command taking input from
another (pipes). The commands that can be
connected in this way are called filters, because
they filter or manipulate data in different ways.
Pipes and filters form the central feature of the
UNIX philosophy.
 Shell variables can control the behavior of shell as
well as other programs by storing data in them.
G.Shyama Chandra Prasad 17
The UNIX Shell ...
 Shell includes programming language features which can
be used to build shell scripts for performing complex
 Shell scripts are frequently used sequence of shell
commands stored in a file; file-name can be used later to
execute the stored sequence.
 Types of Shells - Bourne Shell - original command
interpreter developed at AT&T by Stephen R. Bourne;
fastest official shell distributed with UNIX systems
(executable filename sh)
 C Shell - developed by William Joy and others at UCB;
gets its name from C due to syntax resemblance of its
programming language (executable file name csh)
 Korn Shell - developed by David Korn, combines best
features of both shells, not popular (executable file name
 Restricted Shell - restricted version of Bourne shell,
typically used for guest logins and in secure installations
(executable file name).
G.Shyama Chandra Prasad 18
Some Shell Commands
 Command Function
 pwd print working directory
 cd change directory
 mkdir make directory
 rmdir remove directory
 ls list contents of directory
 cat display contents of files
 cp copy one file to another
 rm remove files
 mv change name of a file/directory
or move file from one directory to another
 date display system date
 tput clear clear screen
 who display logged-in users’ names
 echo display string; used for prompt
 ln create another link to a file
G.Shyama Chandra Prasad 19
The UNIX File System (UFS)
 It is a hierarchical collection of 3 types of files: ordinary,
directory and special files (device, pipe, fifo, socket).
 A UNIX file is featureless because it is simply an array of
 Dominant file type in UNIX is the text file.
 System related files are also stored in text form.
 Separate device can be added by creating a file for it.
 Root is the supremo and is represented by the ‘/’. Every
subdirectory must have a parent.
 File names can be up to 14 characters long; can contain
both upper and lower case alphabets, digits, a dot, hyphen
(-), underscore (_) anywhere; should not have a blank or
tab; are case-sensitive.
 Path names are a sequence of directory names separated
by ‘/’. They are used to access files.
 Absolute pathname - file location is determined with
respect to the root.
 Relative pathname - file location is determined with
respect to the current directory.
G.Shyama Chandra Prasad 20
The UNIX File System …
Though the UFS looks hierarchical, it is actually a directed
acyclic graph because files can be shared.
root ( / )

/proc /tmp /mnt /home
/bin /lib /sbin ntpuser1
vi/awk /etc /usr /var
/dev to 4
cat/cut audio aliases bin – sh lost+ lost+
chmod hosts etc found found
beep games
date/sort init.d crash ntpusr …
cpu dict ftp
echo fd0 to mail include lib
ed/sed fd7 mknod lib man mail cron
grep/tar motd sbin opt fax
ram share news
kill/pwd news tmp
ttyxx tmp … spool mail
ln/ping … passwd mqueue

mv/ps/ls uucp uucp
rm/rmdir … …
G.Shyama Chandra Prasad 21
… M.Tech.,(Ph.D)
Important Directories in Linux File System
/home – It holds user’s home directories. In other UNIX systems,
this can be /usr directory.
/bin – It holds many of the basic Linux programs; bin stands for
binaries, files that are executable.
/usr – It holds many user-oriented directories:
bin – It holds user-oriented Linux programs.
sbin – It holds system administration files.
spool – It has several subdirectories:
. mail holds mail files
. spool holds files to be printed
. uucp holds files copied between Linux machines.
docs – various documents including useful Linux info
man – man pages accessed by typing the man <command>
games – the fun stuff!
/sbin – It holds system files that are usually run automatically.
/etc – It and its subdirectories hold many of Linux config files.
/dev – It holds device files. All info sent to /dev/null is thrown into
trash. Your terminal is one of the /dev/tty files.
G.Shyama Chandra Prasad 22
The UNIX File System …
 The UFS resides on a single logical disk. A logical disk is a
disk partition comprising of a set of consecutive cylinders.
 UFS further subdivides a partition into one or more cylinder
groups and attempts to allocate inodes and related data
blocks from the same cylinder group, thus minimizing the
disk head movements.
 At the beginning of the logical disk lies the boot block of
UNIX operating system containing the bootstrap program.
 It is followed by repetitive cylinder groups each one
containing a super block, cylinder group block, inode list and
the data area.
 Each cylinder group contains a duplicate copy of the super
block. The super block contains the size of file system,
number of free blocks, index of next free block in free block
list, size of inode list, number of free inodes, index of next free
inode in free inode list.
 The cylinder group block contains a number of inodes and
corresponding data blocks for that cylinder group. The block
size is a power of 2 (>=4096).
G.Shyama Chandra Prasad 23
Internal File Maintenance - UNIX File System …
 For each file created in the system, an inode is also created.
Inode is a disk file record of 64 bytes that maintains the
permanent attributes of a file.
 An inode is permanent and it exists until the corresponding
file is removed from the system. Sample details of an inode –
 Owner and group identifiers
 File type and file size
 Number of links for this file
 Times of file creation, last file access and modification,
and last inode modification
 List of access rights – read/write/execute permissions
 Reference count showing number of times file is opened
 Physical address of file on the disk: array of 13 pointers
for data storage
 Whenever a file is opened, its inode is brought into main
memory. The active inode is kept there until the file is closed
and is used to locate the beginning of an open file on disk
and to verify that every I/O request comes from a valid user
as per specified access permissions.
G.Shyama Chandra Prasad 24
The inode - UNIX File System …
The logical disk - UFS
Boot block Cyl. Grp. Block Super block Inode list Data blocks …
direct0 direct1
Array of 13 pointers for

direct2 Assumptions:
direct3 direct4 Logical Block size = 1K bytes

Data Blocks
data storage in inode

If block number occupies 4 bytes,

direct6 number of blocks numbers = 256
direct7 Maximum number of bytes in file = (10 +
direct8 256 + 2562 + 2563) KB = over 16GB
If file size field in inode is 4 bytes long,
direct9 sigle- the file size is limited to 4GB.
indirect double-
indirect triple-

G.Shyama Chandra Prasad 25

File Access Permissions (FAP)
 FAP refer to permissions associated with a file with
respect to the following:
 File Owner, Group Owner, Other Users.
Read r Allows displaying /copying
Write w Allows editing/deleting
Execute x Allows execution of a file
 The protection on a file is referred to as its file mode
on UNIX systems. The long version of ls command
also displays FAP in addition to user and group
ownership. The first character indicates the file
type, a hyphen for a plain file, and a ‘d’ for a
directory; rest of the bytes may be rwxrwxrwx or a
‘-’ for r/w/x if that particular permission does not
G.Shyama Chandra Prasad 26
File Access Permissions (FAP) ...
 Meaning of 0123456789 FAP bytes of a
 No read does not allow list <directory> or remove
 No write does not allow copy files to directory,
remove files from directory, rename files in
directory, make a subdirectory, remove a
subdirectory, move files to/from directory.
 No execute does not allow display<directory> from
within directory, change to directory, display a file
in dir, copy a file to/from dir.
 Byte 0 is ‘b’ (block), ‘c’ (character) or ‘p’ (device)
for special files.
G.Shyama Chandra Prasad 27
Filters and Pipes
 Filter is a program that takes its input from standard input
file, processes it, and sends its output to standard output file
(simple filters: head, tail, cut, paste, sort, uniq, etc; advanced
filters: grep, pg, wc, tr, awk).
 The grep filter (global search for regular expressions)
searches files for a particular pattern. The format of grep
command is:
grep [options] reg-expr filenames
-n prints each line matching the pattern with line no.
-c prints only the count of lines that match the pattern.
-v prints out all those lines that do not match the pattern.
Specifying grep regular expressions
\c Backslash turns off any special meaning of character c.
^ Pattern following it must occur at the beginning of line.
$ Pattern preceding it must occur at the end of line.
. Matches any single character
[ … ] Matches any one of chars in …; ranges like a-z are legal
[^…] Matches any single char not in …; ranges are legal.
r1r2 Concatenation of regular expressions: r1 followed by r2
G.Shyama Chandra Prasad 28
Filters and Pipes ...
 pg displays a large file, a screen-full at a time.
 wc counts number of lines, words, characters
 tr is used to translate one set of characters to
another; does not accept a filename as an argument
but only the redirected file input.
 cut (vertical file slit) - extracts only the desired
columns from the output of certain commands.
 paste (vertical file paste) - what you cut with
previous cmd can be pasted back with the paste
cmd. But while cat pastes more than one file
horizontally, paste does it vertically.
 pipes (|)combine filters and commands such that
standard output of one can be sent as standard
input to another.
Advantage of pipes: UNIX tools (commands) can be
combined rather than be rewritten.
G.Shyama Chandra Prasad 29
Filters and Pipes - awk
Pattern Scanning and Processing Language
$ awk ‘program’ filenames … where program
pattern { action }
pattern { action }
… # By Aho, Weinberger, Kernighan
awk reads the input in the filenames one line at a
time. Each line is compared with each pattern in
order; for each pattern that matches the line, the
corresponding action is performed. Awk does not
alter its input files.
Patterns can be regular expressions exactly as in
egrep, or C-like complicated conditions.
G.Shyama Chandra Prasad 30
awk Built-in Variables
FILENAME name of current input file
FS field separator character
(default blank & tab)
NF number of fields in input
NR number of input record
OFMT output format for numbers
(default %g)
OFS output field separator string
(default blank)
ORS output record separator string
(default newline)
RS input record separator
character (default newline)
G.Shyama Chandra Prasad 31
awk Operators
= += -= *= /= %= Assignment operators
v op= expr is the same as v = v op (expr)
|| OR: expr1 || expr2 - true if either is true;
expr2 is not evaluated if expr1 is true.
&& AND: expr1 && expr2 - true if both are true;
expr2 is not evaluated if expr1 is false.
! Negate value of expression
> >= < <= == != ~ !~ relational operators;
~ and !~ are match and non-match embedded
string comparison operators for fields
+ - plus minus
* / % multiply, divide, remainder
++ -- increment, decrement (prefix or postfix)
^ $ are used in a regular expression for a pattern to
be found in the beginning or the end of a field.
G.Shyama Chandra Prasad 32
awk Built-in Functions
cos(expr) cosine of expr
exp (expr) exponentiation of expr: eexpr
getline() reads next input line; returns 0 if end
of file, -1 if error, 1 if successful read
index(s1,s2) position of string s2 in s1; returns 0 if
not present
int (expr) integer part of expr; truncates
towards 0
length(s) length of string s
log (expr) natural logarithm of expr
sin (expr) sine of expr
split(s,a,c) split s into a[1]…a[n] on character c;
return n
sprintf(fmt, …) format … according to fmt
substr(s,m,n) n-char substring of s beginning
at position m
G.Shyama Chandra Prasad 33
awk ...
Example 1
$ awk ‘/regular expression/ { print }’ filenames …
$ awk ‘/regular expression/’ filenames …
Both print every line that matches the regular expression
Example 2
$ awk ‘{ print }’ filenames …
does exactly what cat does, albeit more slowly.
Example 3
It is possible to present program from a file:
$ awk -f cmdfile filenames ...
Example 4
$ who | awk ‘{ print $1, $5 }’
prints names of logged-in users and time of login. Awk splits
each input line automatically into fields strings) separated by
blanks or tab.
G.Shyama Chandra Prasad 34
awk ...
Example 5
$ awk ‘{ print NR, $0 }’
adds line numbers to an input stream; field $0 is the entire
input line, unchanged; built-in variable NR is the number of
current input record or line.
Example 6
$ awk ‘{ printf “%4d %s\n”, NR, $0 }’
prints line numbers in a field 4 digits wide.
Example 7
$ awk -F ‘$2 == “”’ /etc/passwd
lists people with no passwords; encrypted password is the
second field in /etc/passwd file.
Example 8: Simple Data Validation
$ awk ‘NF % 2 != 0’ testfile
prints lines from testfile with odd number of fields (NF =
number of fields in input record).
G.Shyama Chandra Prasad 35
awk ...
Example 9: Special BEGIN Pattern
$ awk ‘BEGIN { FS = “:” }
> $2 == “”‘ /etc/passwd # No output
BEGIN actions are performed before first
input line has been read (initializing FS here).
Example 10: Special END Pattern
$ awk ‘END { print NR }’ ...
Prints number of lines of output. END actions
are done after the last line of input has been
Awk’s real strength lies in its ability to do
calculations on the input data besides text
G.Shyama Chandra Prasad 36
awk ...
Example 11
Compute separate sums of each of n columns
of input, plus the grand total.
Program addup in shell envelop invoked as:
$ addup 6
awk ‘{
for (i= 1; i <= n; i++) sum[i] += $i
END { for (i= 1; i <= n; i++) {
printf “%6g “, sum[i]
total += sum[i] }
printf “; total = %6g\n”, total
}’ n = $1 datafile
G.Shyama Chandra Prasad 37
The vi Editor
 It is a visual editor used to enter and edit text files
 Invoking vi with/without filename puts it in
command mode:
$ vi [<file name>]
 vi command mode syntax:
[<repeat-count>] <command>
 vi works in three different modes:
• Input - where any key is entered as text
• Command - where keys are used as commands
to act on text
• ex - where ex commands can be entered in last
line to act on text
G.Shyama Chandra Prasad 38
Relationship between 3 Modes

sS Mode

:< E


nt e





E nt


er >

Input ex
Mode Mode

G.Shyama Chandra Prasad 39

Input Mode Commands
Command Action
a Appends text to right of cursor
A Appends text at end of line
i Inserts text to left of cursor
I Inserts text at beginning of line
/ Inserts before current character
o Inserts blank line below + inserts text
O Inserts blank line above + inserts text
rx Replace current character with char x
Rtext<Esc> Replaces character with text
s Replaces character under cursor with
many characters
S Replaces entire line
20i- followed by <Enter> or <Esc> enters
20 hyphens (-) in one line
G.Shyama Chandra Prasad 40
Save and Exit from vi
Command Action
ZZ saves file only if vi started with a filename
Save and Exit Commands of ex Mode
:w save file and remain in edit mode
:wq save file and quit edit mode
:w <filename> write buffer to filename
:q quit editing mode when no changes
are made
:q! quit editing mode but after
abandoning changes
:x save file and quit editing mode
G.Shyama Chandra Prasad 41
Cursor Movement Commands
Command Action
h or backspace Moves Cursor Left
l or spacebar Moves Cursor Right
j Moves Cursor Down
k Moves Cursor Up
nG Go to line number n
^ / 0 / | Moves cursor to beginning of line
$ Moves cursor to end of line
w Moves cursor to next word
b Moves cursor back to previous word
e Moves cursor forward to end of word
Note: W, B, E perform functions similar to w,
b, e except that punctuation is skipped)
G.Shyama Chandra Prasad 42
Paging and Scrolling Commands
Command Action
^d Scroll down half screen
^u Scroll up half screen
^f or <PfDn> Full Page forward (12 lines by
^b or <PgUp> Full Page backward default)
^l Redraw Screen
[[ Scroll to beginning of document
]] Scroll to end of document
{ Scroll to previous paragraph
} Scroll to next paragraph
( or ) Scroll to previous or next sentence
Note: repeat factor can be used
G.Shyama Chandra Prasad 43
Search and Repeat-Search Commands
Command Action
/pat Search forward for pattern pat
?pat Search backward for pattern pat
n Repeat search in previous search
direction (no repeat factor)
N same as n but in opposite direction
fx or tx Move cursor forward to or before the
first occurrence of character x in
current line
Fx or Tx Same as above but move backward
(repeat factor works with all f/F and t/T)
; or , Repeat last character search in same
or opposite direction (by f/F/t/T) only
in current line (repeat factor works)
G.Shyama Chandra Prasad 44
Deletion and Modification Commands
Command Action
dw Delete Current Word
dd Delete Current Line
d$ or D Delete from cursor to end of line
cw Change Current Word
cc Change Current Line
c$ or C Change from cursor to end of line
x Delete character under cursor
X Delete character before cursor
J Join current, next lines (also nJ)
.(dot) Repeat last editing instruction
u Undo single last change
U Restore all changes to line since
cursor moved to it
G.Shyama Chandra Prasad 45
Commands to Move or Copy Lines
Cut (delete) n lines starting from current line
Copy n lines starting from current line
Put yanked text after current cursor position
Put yanked text before current cursor position
 Note: Named-buffer is useful for copying an
area from one file into another. Open some files
simultaneously by vi. Mark an area in one file
by dd or yy; move to another file (by :next) to
paste (by p or P) the named area; then say
:rewind to return to the parent file.
G.Shyama Chandra Prasad 46
Customizing vi
 vi can be customized as per users’ requirements using ex-
mode commands.
ex Commands Action
:set all Display all set options; those pre-fixed with no are not
:set autoindent (ai for short)
Extremely useful to programmers for
indentation of lines
:set number (nu for short)
Displays all lines with numbers which are not preserved on
saving file
:set nonumber (nonu for short) - Reverses number setting
:set tabstop=6 (ts for short)
Changes default tab setting (8 spaces)
:set ignorecase (ic for short)
ignores case while pattern matching
G.Shyama Chandra Prasad 47
Customizing vi ...
:set showmatch (sm for short)
Helps locate matching brackets when ) or } is entered in
input mode; beeps when no match found to alert for
:set autowrite (aw for short)
Writes current file automatically whenever switching files
with :n and escaping to shell with :sh
:set showmode - Displays a message when vi goes to input mode
:next (n for short) Moves to next file opened in vi
:rewind Comes back to parent file
 All sets can be stored in .exrc file used by vi for its startup
 Equivalently an environment variable, EXINIT can be
assigned to store the settings:
$EXINIT=“set nu tabstop=6 ignorecase”
 It is better to make this assignment in the .profile so that it is
available for all sessions.
G.Shyama Chandra Prasad 48
Session 2
Shell Programming
Shell Scripts

G.Shyama Chandra Prasad 49

Shell Programming and Shell Scripts
 UNIX shell can be considered as a master utility program that
enables a user to gain access to all other utilities and resources
of the computer.
 Frequently used UNIX commands can be stored in a file. Shell
can read the file and execute the commands in it. Such a file is
called a script file.
 The script file can be created by:
$ vi filename or
$ cat > filename <Enter>
 Executing a Shell Script:
$ sh filename <Enter> or $ filename <Enter>
But, remember to modify the FAP for the file containing the
shell script for execute permission: $ chmod u+x filename
 Pressing <Ctrl>u at any point, right up to before pressing
<Enter>, lets you clear everything on the command line.
G.Shyama Chandra Prasad 50
Shell Programming - Wildcards
 UNIX offers the facility to perform an operation on a set of
files (directories included) without having to specify all the
file names on which the operation has to be performed.
Some special characters are used for this purpose:
Wildcard Purpose
* Matches a string of none or any number of characters.
? Matches exactly one character.
[abcd] Matches exactly one of a specified set of characters
[a-d] [! or a range of characters
abcd] [! Matches exactly one character that is not in the specified
a-d] set of characters or a range of characters.
 Comment entries can be inserted in a script by prefixing
statements with the # symbol.
G.Shyama Chandra Prasad 51
Shell Programming - Variables
 User-defined shell variables can be created at any
point of time by a simple assignment of a value:
$ today=20 ; var=beta
$ name1=“Bharat Bhushan” or ‘Bharat Bhushan’
 NOTE: The assignment operator = must not have
surrounding spaces.
 Variables can be referenced as:
$ name2=$name1 or name2=${name1} <Enter>
 But see what happens when braces are not used:
X=$todayth # would not produce desired result
X=${today}th # would produce 20th
So, braces are optional if no concatenation is
 The echo command is used for displaying messages
on screen (possibly terminated by \c, \n, \007, etc).
G.Shyama Chandra Prasad 52
The Evaluation of Shell Variables
$var value of var; nothing if var undefined
${var}same; useful if alphanumerics follow
variable name
${var-str} value of var if defined; else
str; $var unchanged.
${var=str} value of var if defined; else
str; if undefined, $var set to str.
${var?str} if defined, $var; else, print
str and exit shell; if str empty, print:
var:parameter not set
${var+str} str if $var defined, else null.
G.Shyama Chandra Prasad 53
The Global and Local Variables
 When a variable is referenced, it is known only to the shell
which created it. A new shell created by typing sh, is born
unaware of the parent shell’s variables.
 The same variable name (as in the parent shell) can be given
a different value in the new shell. Such a variable is called a
local variable that is known only to the child shell.
 In many instances, it may be necessary for the child shell to
know about the parent shell’s variables. They are known as
global variables.
$ city=Bombay ; echo $city # Create variable; Response: Bombay
$ sh ; echo $city # create a child shell; No response
$ city=Delhi; echo $city # Create new variable; Response: Delhi
<Press ^d> # Return to parent
echo $city # Parent continues to have value Bombay
 The global variables must be exported so that the child shell
is aware of them as follows:
$ export city # only the copy passed to the child
G.Shyama Chandra Prasad 54
The alias and unalias Commands
 Alias commands are usually abbreviations of other
commands that are designed to save keystrokes.
 For example, if you enter the following command on regular
basis, you might be inclined to create an alias for it to save
yourself some typing:
cd /usr/X11/lib/alpha/sample.config
alias myconfig=‘cd /usr/X11/lib/alpha/sample.config’
 Now, until you exit from bash, the myconfig command will
cause the original longer command to be executed as if you
had just typed it.
 If you do not need an alias any longer, you can use the bash
unalias command to delete the alias:
unalias myconfig
 When the alias command is entered without any argument, it
will display all of the aliases that are already defined on
G.Shyama Chandra Prasad 55
Backward quotes and set & shift commands
 Recall the usage of pipes in joining commands. Other way is
through command substitution:
$ echo “The date is `date`” <Enter>
$ count=`cat fileA | wc -l` <Enter>
Single backward quotes enclose command(s) to be executed.
 The set command sets the values of $1 to $n to the words
given explicitly/implicitly as its arguments:
$ date <Enter>
Fri Sep 24 11:27:42 IST 2004
$ set `date` <Enter>
$ echo $1 $2 $3 <Enter>
Fri Sep 24
 The shift command transfers the contents of a positional
parameter to a lower numbered one as shown below:
$ shift 2 # Output: 24 11:27:42 IST
Note: shift and shift 1 have the same effect.
G.Shyama Chandra Prasad 56
Arithmetic Evaluation with expr
 Shell does not support numerical variables. All
variables are treated as character strings.
Simulation of numeric variables is possible by expr
command which is used to evaluate arithmetic
expressions without decimals.
Computing square of n:
echo “Square of $n is `expr ${n} \* ${n}`”
n=`expr ${n} + 1` ; echo $n
 The arithmetic operators (+, -, *, /)must be
surrounded by at least one space. The * should be
preceded by a backslash (\) else shell will interpret
it as a wildcard character.
 Unix understands only the whole numbers, so, 5.2
and 5.6 both are interpreted as 5.
G.Shyama Chandra Prasad 57
String Handling with expr
 The expr is quite handy for finding the length of a string and
extracting a sub-string:
$ str=abcdefghijk ; n=`expr "$str" : ‘.*’` ; echo $n
11 # expr gave how many times any character (.*) occurs.
# This feature is very useful in validating data entry.
 Extracting a sub-string:
$ expr “$str” : ‘……\(..\)’
Note that there are 6 dots preceding the sequence \(..\). This
advanced regular expression signifies that the first six
characters of the string are to be ignored and extraction
should start from the 7th character. Two dots inside \(..\)
suggests that this extraction is limited to two characters only
(backslashes override the usual interpretation of ‘()’).
 Location of the first occurrence of a character inside string:
$ expr "$str" : '[^d]*d‘
G.Shyama Chandra Prasad 58
Shell’s Built-in Variables and eval command
 Shell’s Built-in Variables
$# the number of arguments
$* all arguments to shell
$@ similar to $*
$- options supplied to the shell
$? Return value of the last command executed
$$ process-id of the shell
$! process-id of the last command started with &
 The eval command is used to execute commands
generated by multiple levels of indirection:
$ a=c; b=m; c=d; cmd=date
$ eval $`echo $a$b$c`
Tue Nov 23 16:45:43 EDT 2000
G.Shyama Chandra Prasad 59
Shell Programming – input/output
 The read command can be used to get run-time
input from the user (standard input). It reads :
$ read var1 var2 <Enter>
 A file can also be read sequentially through read:
$ read var1 var2 var3 < myfile
 One line is read at a time; the next word in the line
is assigned to each successive variable specified as
an argument of read; extra words are assigned to
the final argument of read.
 Screen display from echo can be redirected (>) or
appended (>>) to a file. The double-quoted string
formed by concatenating the fields together
separated by the desired delimiter forms echo’s
argument for standard or file output:
$ echo “$name | $code | $price | $qty” >> Salesfile
G.Shyama Chandra Prasad 60
Grouping Commands by ( ) and { }
 A pair of parentheses is used to enclose two or more
commands, and the aggregate standard output can be
redirected to a file:
$ ( ls –l ; who ) > <filename> <Enter>
 Basically the commands grouped with a pair of parentheses
are executed in a sub-shell:
$ pwd <Enter>
$ ( cd psdir ; pwd )
$ pwd
 However, the current directory of the father shell can be
affected by grouping the commands in a pair of braces {}. So
try the above code with {} instead of ().
 But, where does one use the command grouping features?
See exercise 5 at the end of session 2.
G.Shyama Chandra Prasad 61
Coding Branching/Looping Conditions
 The test command can be used to write conditions
that can be used in if, case, and while commands.
 String comparison operators: = != < <= > >=
 Example: To check if variables name and myname
have the same value, and variable uid‘s value is 10:
test ${name} = ${myname} -a ${uid} = “10”
Same as [ ${name} = ${myname} -a ${uid} = “10” ]
 Arithmetic comparison operators:
-eq -ne -gt -ge -lt -le
1. All comparison operators must be surrounded by at
least one space.
2. Multiple conditions in one test command can be
formed using –a / -o for Anding / Oring two truth
G.Shyama Chandra Prasad 62
Options for Testing File-types and Strings
 File test options:
Condition Return value when
test -f fname returns true if file exists and is ordinary
test -d fname returns true if file exists and is a directory
test -r fname returns true if file exists and is readable
test -w fname returns true if file exists and is writeable
test -x fname returns true if file exists and is executable
test -s fname returns true if file exists and is not empty
 String test options:
test str Returns true if str is not null
test –n str Returns true if length of str is greater than zero
test –z str Returns true if length of str is equal to zero
 The &&, ||, :, ; meta-characters of shell
C1 && C2 – Cmd C1 Executed; if success, then C2 Executed.
C1 || C2 – Cmd C1 Executed; if failure, then C2 Executed.
: - Do-nothing command - used mostly in shell scripts.
; - Command separator used to type multiple cmds in a line. 63
G.Shyama Chandra Prasad
Decision-making Constructs
1. if <condition(s)>
then <command(s)>
[else <command(s)>] # Optional else part
2. if <condition(s)>
then <command(s)>
elif <condition(s)> # Many elif-then allowed
then <command(s)>
else <command(s)>
fi echo “Want to quit (type Y or y): “
read ans
Example: if test $ans = ‘y’ –o $ans = ‘Y’
then exit
G.Shyama Chandra Prasad 64
Decision-making Constructs ...
3. case-esac command in shell scripts is a multiple-
choice construct:
case ${num} in
dozen) echo “12”;;
score) echo “20”;;
*) echo “num is neither a dozen nor a score”
NOTE: the last command in any choice must be
delimited by a pair of semi-colons to delimit it from
the commands of the next choice.
4. exit command is used to exit from the shell script
and return to the parent shell. It also takes an
argument whose value is 0 when not specified. It is
taken as the return value which is assigned to the
parameter $?.
G.Shyama Chandra Prasad 65
Constructs for Supporting Iteration
1. while <condition>
do # executed as long as the condition is true
2. until <condition> # complement of while
do # executed as long as the condition is false
n=0 $ cat file1 | while read x y z
while [ $n -lt 10 ] do echo $x $z
do n=`expr $n + 1`
echo “$n `expr $n \* $n`”
G.Shyama Chandra Prasad 66
Constructs for Supporting Iteration …
3(a) for n in 1 7 8 50
echo “Value of n is $n”
3(b) for filename in `ls tempdir`
echo “Displaying contents of ${filename}”
cat ${filename}
3(c) for itemlist; do commands; done
# executes once for each item in itemlist that contains all positional
# parameters passed in to shell program on the command line.

 For each iteration in both loops, variable n and filename

take on one value from their lists.
 Command break causes the termination of a loop and
continue resumes execution at its top.
G.Shyama Chandra Prasad 67
Background Processing
 UNIX provides a method of running time-consuming
commands in the background while the user continues
working in the foreground by typing ampersand (&) at the
end of line:
$ wc myfile & <Enter>
$ vi newfile <Enter>
 Here, user gives the wc command and then immediately
begins editing newfile without waiting for the completion of
the previous wc process.
 Well, the user must be careful that the background and
foreground processes do not interfere with each other. Any
screen output from the background process would disrupt
the screen in the middle of user’s work with the foreground
process. This should be avoided by output redirection:
$ wc myfile 1 > outfile 2 > errfile & <Enter>
$ vi newfile <Enter>
G.Shyama Chandra Prasad 68
Background Processing …
 Sometimes it is needed to check the current status of a
background process. The jobs command gives the job-
number of each active process. The ps command generates a
one-line entry for each of the active processes showing its
process-id, user’s terminal number, time for which the
process has been active, and the name of command. If the
background process is not listed in the output, it may be
completed already. The kill command can be used to stop
background process if it is not functioning as desired:
$ kill process-id <Enter> or, $ kill job-number <Enter>
 If a command issued in foreground is taking too long to
finish, and you want to execute it in background so that you
could use the system again, do the following:
$ <Ctrl> z bg
Pressing <Ctrl>z suspends a running process and bg restarts
it in background. <Ctrl> c kills a job running in foreground.
 Similarly, <Ctrl>z fg pid or <Ctrl>z fg %jobnumber forces a
suspended process to continue to execute in the foreground.
G.Shyama Chandra Prasad 69
Shell Scripts with Command-line Parameters
 Parameters are essentially used to create generalized shell
scripts. The command name (first word on command line) is
put into a variable called $0, the first argument of command
(second word) is put into $1, etc.
 UNIX shell creates a maximum of ten variables. The variables
$1 through $9 are also called the positional parameters of the
command line.
 A Shell Script ‘test’ with Parameters:
echo “Program name is $0”
echo “First argument is $1”
echo Number of arguments = $#”
echo “The arguments are $*”
 A Sample Run:
$ sh test arg1 arg2 arg3 <Enter>
Program name is test
First argument is arg1
Number of arguments = 3
The arguments are arg1 arg2 arg3
G.Shyama Chandra Prasad 70
Shell Scripts with Functions
 Shell enables you to define your own functions. Code written using
functions tends to be much easier to read and maintain and also tends to
be smaller.
Syntax: fname( ) { shell commands }
 The functions can be invoked as follows:
Syntax: fname [parm1 parm2 parm3 … ]
 Notice that you can pass any number of arguments to your function. If
passed, they are seen as positional parameters just like command-line
parameters and are referred to as $1, $2 etc.
 Example of a shell script using functions that convert file contents from
lower case to uppercase and vice versa:
upper() { for i # function definition - converts to upper case
do tr "a-z" "A-Z" < $1 > $1.out
rm $1 ; mv $1.out $1 ; shift # May delete first two commands
done; } # for distinct input/output files.
lower() { for i # function definition - converts to lower case
do tr "A-Z" "a-z" < $1 > $1.out
rm $1 ; mv $1.out $1 ; shift # May delete first two commands
done; } # for distinct input/output files.
upper low1 low2 # call of upper() with 2 file name arguments
lower upp1 upp2 # call of lower() with 2 file name arguments
G.Shyama Chandra Prasad 71
Example 1
Shell Scripts
echo “Enter the search string: \c”
read string
if [ -z “$string” ] # true if null string
# The above if same as: if test -z $string
echo “Not entered the string”
exit # Out to $ prompt
... # Rest of processing
Example 2
if [ -n “$pattern” -a -n “$filename” ]
then # Both operands not null strings
grep “$pattern” “$filename” || echo “Pattern not found”
else echo “At least one operand is a null string”
G.Shyama Chandra Prasad 72
Shell Scripts ...
Example 3
echo “enter the name of your file:\c”
read fname
if [ ! -f $fname ]
echo “file does not exist”
elif [ ! -r $fname ]
echo “ file is not readable”
echo “ file is readable”
G.Shyama Chandra Prasad 73
Shell Scripts ...
Example 4: Prepare and extend an inventory of food items with
their code and description.
while [ “$choice” = “y” ] # or until [ “$choice” = “n” ]
echo “Enter code and desc:\c”
read code
read desc
echo “$code | $desc” >> inventory
echo “More Items (Enter yes/no only)? \c”
read choice
case $choice in
Y*|y*) choice=y ;;
N*|n*) choice=n ;;
*) echo “Invalid answer - yes assumed”;
choice=y ;;
done … # Further Processing
G.Shyama Chandra Prasad 74
Shell Scripts ...
Example 5
if test $# -ne 3 # Three arguments not entered
then echo “Not keyed in 3 arguments”; exit
else if grep “$1” $2 > $3
then echo “Pattern found – Job over”
else echo “Pattern not found – Job over”
 The following can easily replace the opening session of
above script without creating a child shell:
[ $# -ne 3 ] && { echo
“You have not keyed in 3 arguments” ; exit }
 Well { } cannot execute a command unless it has execute
permission. The dot command (.) lets you execute a script,
however. So make changes to your .profile and execute it
with the . command without needing to log out and log in
again. Thereafter on issuing the set command you will find
all variables properly reset. $ . .profile ; set
G.Shyama Chandra Prasad 75
Session 3
UNIX Communication Tools
The objective of this session is to identify the
communication tools on our Linux system:
 write
 mesg
 talk
 mail
 wall
 finger
 ping
 ftp
 telnet
G.Shyama Chandra Prasad 76
The write Command
 It let’s you have a two-way communication with any person
who is currently logged in. One user writes his message and
then waits for the reply from the other. The conversation
continues until one or both the users decide to terminate it:
$ write ntpuser1
Have you completed exercise 1?
I have done mine – ntpuser3
^d # Ctrl-d is the end of file from terminal
 Here ntpuser1 is the login-id of the recipient as argument
and the text of the message comes from the standard input.
 It is equally possible to store this message in a file mymsg and
apply input redirection:
$ write ntpuser1 < mymsg
 The message appears on ntpuser1‘s terminal provided he is
logged in, else sender gets an error message response. If the
recipient is logged in more than one terminal, the message is
sent to the lowest numbered terminal unless you have
provided the terminal numbers as more arguments:
G.Shyama Chandra Prasad 77
The write … mesg Commands
 Once into the write program, you can execute any UNIX
command by preceding it with the operator ‘!’. The
command output is not transmitted.
 Since write ties you up in this interactive (and slow)
conversation, UNIX offers the mail command to permit
deferred communication so that your own work does not get
 Normally, you are not permitted to receive messages from
other users for security considerations. Such permissions are
granted by the mesg command to be discussed next.
 The mesg command can be used to prevent other people
from writing to your terminal as follows:
$ mesg n # default mode is y
 You can know the status of your terminal by typing:
$ mesg # y or n will be the response
 The message of today is stored in a file /etc/motd.
G.Shyama Chandra Prasad 78
The talk Command
 The write tool was nice for communication one at
a time. Simultaneous typing is enabled by the talk
tool. Entering mesg n similarly just turns talk off.
 talk is an interactive communication tool that
enables you to have an actual conversation in real-
time mode with a user who has to be logged in.
 You get an appropriate message if someone does
not want to talk to you, else your terminal shows a
divided screen. You can type your message on the
top half of the screen and it appears verbatim on
the other terminal at the bottom half of the screen.
 Both users can type messages simultaneously. The
connection can be closed by entering <Ctrl-C>.
G.Shyama Chandra Prasad 79
The mail Command
 Electronic mail (E-mail) has taken the world by
storm. It is stored in a file called system mail in a
mailbox that has the same name as that of the user.
For example, the user ashok’s mailbox under Linux
is /var/spool/mail/ashok.
 E-mail is used for non-interactive communication.
It originates in the form of a file on your computer
created by a Mail User Agent (MUA). E-mail is
then submitted to a mail router such as sendmail,
after which it is handed over to a Mail Transporter
Agent (MTA). E-mail is then appended to a
mailbox. The MTA delivers it to the final delivery
agent by traversing one or more hosts.
 The MUA is the user interface for the mail system
that is used to read and send mail. Simple MUAs
are mail or Mail. Mail is also called mailx or
capmail or Berkeley Mail. The sophisticated MUAs
are elm and pine.
G.Shyama Chandra Prasad 80
The mail … wall Commands
 elm stands for electronic mail and provides a full-screen
interface mail program with a good help feature. Pine, a
trademark of the University of Washington, stands for
Program for Internet News and E-mail. Pine is also capable
of sending documents and graphics.
 Like write, mail uses standard input but is capable of input
redirection from a file.
 The mail program when used for the purpose of viewing the
mail, features a prompt (&). There are a whole lot of
internal commands that you can enter at this prompt to
save, delete, ignore, reply the mail that you have received.
On-line help can be sought by typing ? at the prompt.
 Administrator can broadcast messages to all users (users can
also do so on some systems):
wall <Enter>
Machine will be shut down in 10 minutes. So log out, please!
^d # Ctrl-d is the end of file from terminal
G.Shyama Chandra Prasad 81
The mail - Internal Commands
 All the available options in the Mail can be listed by
typing a ? after the &. The listed options are:
t <message list> - type messages
n - go to and type next message
e <message list> - edit messages
f <message list> - give headlines of messages
d <message list> - delete messages
s <message list> file - append messages to file
u <message list> - undelete messages
R <message list> - reply to message senders
r <message list> - reply to message senders and all recipients
pre <message list> -
make messages go back to /usr/spool/mail
G.Shyama Chandra Prasad 82
The mail - Internal Commands …
m <user list> - mail to specific users
q - quit, saving unresolved messages in mbox
x - quit, do not remove system mailbox
h - print out active message headers
! - shell escape
cd [directory]
- chdir to directory or home if none given
 A <message list> consists of integers, ranges of
same, or user names separated by spaces. If
omitted, Mail uses the last message typed.
 A <user list> consists of user names or aliases
separated by spaces.
 File .mailrc in your home directory is invoked when
the mail command is executed. Aliases are defined
in this file.
G.Shyama Chandra Prasad 83
The finger Command
 This product from Berkeley, is more versatile compared to
‘who’, and produces an extremely readable output. finger,
when used without arguments - $ finger <Enter>
simply produces a list of all users who are currently logged in
(default output of 6 fields with headings).
 finger can also be followed by a list of login names. For each
name, it will display a detailed list, whether logged in or not.
$ finger ntpuser1 <Enter>
Login: ntpuser1 Name: (null)
Directory: /home/ntpuser1 Shell: /bin/bash
Never logged in.
No Plan.
 finger can be used with either the first or last name provided
the entry has been properly made in /etc/passwd file.
 It is often necessary to leave behind your schedule and other
important info for other to see. Since it is not possible to send
mail to all users, finger also shows the contents of two files
.plan and .project (only the first time), both in the home
directory. G.Shyama Chandra Prasad 84
The ping Command
 Can be used to check whether the machine is actually on
the net: $ ping <computer-name>
 Error message may be generated. Reasons for the same
could be any of the following:
• Wrong spelling
• One of the links between the source computer and
the remote computer may be down temporarily
• The destination computer may exist, but it may not
be on the internet
 Is ping a valid command ?
 Any of the following messages might be the answer:
 If the computer is on the internet and is responding, the
message would be: is alive
 If the computer is on the internet but is not responding,
the message would be: no answer from
 If the machine is not connected to the internet, the
message would be: ping : unknown host
G.Shyama Chandra Prasad 85
The ftp Command
 File Transfer Protocol (FTP) is a method of transferring
files from one computer to another. FTP provides the
capability of transferring files to and from a remote network
site as well as means for sharing public files. The ftp service is
the user interface to the FTP.
 Anonymous FTP enables users to access remote sites
without having an authorized user-id and password.
Generally the login-id is anonymous and password is guest.
Most current systems require your e-mail address as the
 The client host with which ftp is to communicate may be
specified on the command line:
$ ftp [option] hostname <Enter>
ftp immediately attempts to establish a connection to an FTP
server on that host. A hostname can be either a domain
address or an ip address. After this the remote FTP server
awaits your commands.
G.Shyama Chandra Prasad 86
The ftp - Internal Commands
? or help [command]
displays a list of ftp commands; if no argument is given,
displays a list of the known commands.
bell - Sounds a bell after each file transfer.
ascii – Sets the file transfer type to network ASCII (default).
binary or bin – Sets the file transfer type to binary mode.
quit or bye – Terminates the FTP session with the remote
computer and then exits.
cd <remote-directory>
Changes the working directory on the remote machine.
cdup – Goes to the parent of the current remote machine’s
working directory.
dir or ls [remote-directory]
List the contents of the remote-directory. Note that the ls
command in ftp behaves like the ls –al in Linux.
mdir [remote-files]
Like dir, except multiple remote files may be specified.
G.Shyama Chandra Prasad 87
The ftp - Internal Commands ...
Displays cur-work-directory name on remote m/c.
delete remote-file – Deletes remote file on remote m/c.
mdelete [remote-files] – Deletes the remote files on
remote machine.
rmdir directory-name
Deletes a directory on the remote machine.
lcd [directory] – Changes working directory on local
machine. If no directory is specified, user’s home
directory is used.
open host [port] – Establishes a connection to specified
host FTP server on the optional port number.
close or disconnect
Terminates the FTP session with the remote server.
G.Shyama Chandra Prasad 88
The ftp - Internal Commands …
get remote-file [local-file]
Retrieves the remote-file and stores it on local m/c with the
same or different name.
mget remote-files
Expands the remote files on the remote m/c and does a getfiles
based on regular expressions. For example, f*.tar will get all
the files starting with f.
put local-file [remote-file] - Stores a local file on the remote m/c.
mput local-files - Stores the local files (specified as a regular
expression) on the remote m/c with the same names.
 However, it is a good practice to set the directories on the
local machine and the FTP server before initiating the file
 The –I option turns off interactive prompting during multiple
file transfers.
 Usually <Ctrl-C> is used to abort a sending file transfer.
Receiving transfers are halted by sending an ftp ABOR
command to the remote server.
G.Shyama Chandra Prasad 89
The telnet Command
 It is used to communicate with another host using the
TELNET protocol. If telnet is invoked without the host
argument, it enters command mode. Normally you would
telnet hostname [port]
where hostname is the host you want to connect to and port
indicates a port number (an application’s address), else the
default telnet port is used.
 telnet can log you in either one of two modes: char-by-char
or line-by-line. In char-by-char mode, most text typed is
immediately sent to the remote host for processing. In line-
by-line mode, all text is echoed locally, and only completed
lines are sent to the remote host.
 While connected to a remote host, you can enter the telnet
command mode by typing the telnet escape character Ctrl-].
Normal terminal editing conventions are available in
command mode.
G.Shyama Chandra Prasad 90
The telnet Command ...
 The following commands are available under the command
open host [ [-l] user ] [-port]
Opens a connection to the named host; uses the default port
if port number not specified. The [-l] option may be used to
specify the user name.
close – Closes a telnet session and returns to cmd-mode
quit – Closes any open telnet session and exits telnet. An end of
file (in cmd-mode) will also close a session and exit.
? – Displays a list of telnet commands.
? [command]
– Displays a short summary of the specified command.
set <escape_value> – Specifies the character for switching into
telnet command mode.
status – displays the current status of telnet, including the name
of the remote computer the user is connected to.
G.Shyama Chandra Prasad 91
Key points covered in sessions 1 to 3:
 UNIX as Operating System
• Structure of UNIX
• UNIX File system
• Filters and Pipes
• vi Editor
 Shell Programming and Shell Scripts
 Communication Tools on Linux
 write
 mesg
 talk
 mail
 wall
 finger
 ping
 ftp
 telnet
G.Shyama Chandra Prasad 92
System calls

G.Shyama Chandra Prasad 93

System calls
1. Introduction

2. File related system calls


3. Process related system calls

getpid(), getppid(), getpgrp()

G.Shyama Chandra Prasad 94

* Library functions and system calls
* Both are C - functions
* Difference lies in their incorporation in the UNIX
* Library functions are referred to as add-ons
* System calls are part of the UNIX Kernel
* Library functions themselves use system calls and can
be expanded by the user
* System Calls generally common across UNIX versions
* System calls share the concept of fd -file descriptor
* File descriptor is G.Shyama Chandra Prasad
an integer used to identify a file

Basic tasks in file operations

- opening files
- reading files
- writing in files
- creating files
- changing the permission of files
- changing the owner and group of files
- seeking to file-location
- closing files
G.Shyama Chandra Prasad 96
open ()
* open() an existing file

int open (filename,mode)

char *filename;
int mode;

filename - character pointer to the name of the file

mode - integer signifying the mode

0 for read
1 for write
2 for read and write

open() returns the file descriptors on success and returns -1 on


Example :

Opening the file “test” in read mode and checking for the error

fd = open(“/usr/trg/test”,0);
if ( fd == -1 )
printf(“error in opening file test”);
G.Shyama Chandra Prasad
exit(1); 97
} M.Tech.,(Ph.D)
read ()
* File should be opened in read mode

* read() an opened file

int read (filedesc,buffer,nbytes)

int filedesc;
char *buffer;
int nbytes;

filedesc - File descriptor indicating which file to be read

buffer - An area of buffer storage for holding the

Characters read

nbytes - number of characters to be read at a time

read() returns the number of characters read and 0 in case ofend of

file (EOF) and returns -1 on error
G.Shyama Chandra Prasad 98
read () …Contd….
Example :

Reading the file “test” 100 characters at a time

while ( ( n = read(fd,buff,100)) > 0 )

printf(“file test has contents %s “,buff);
}if ( n == 0 )
printf ( “ End of file “);

if ( n == -1 )
printf (“Error in reading file test”);

When each read is finished the pointer advances by

100 bytes so that the next read picks from there

* If the number of characters left are less than

nbytes ( in this example - 100 ) then read()
will pick up what is G.Shyama
left overChandra Prasad 99
write ()
* File should be opened in write mode

* write() to an opened file

int write (filedesc, buffer, nbytes)

int filedesc;
char *buffer;
int nbytes;

filedesc - File descriptor indicating which file to be


buffer - The function takes from buffer and writes them

to indicated file

nbytes - number of characters to be written at a time

write() returns the number of characters written and

returns -1 on error
G.Shyama Chandra Prasad 100
write () …Contd...
Example :

while ( ( n = read (fd,buff,100)) > 0 )

n1 = write (1,buff,100);
/* writing to standard output */
/* file id - 1 is for stdout */

if ( n1 == -1 )
printf (“Error in writing on stdout");

if ( n == -1 )

printf (“Error in reading file test ");

G.Shyama Chandra Prasad 101
creat ()
* creat() creates a new file or overwrites on the existing

int creat(filename, mode)

char *filename;
int mode;

filename - character pointer to the name of

the file

mode - Integer signifying the mode

The mode is specified in octal

creat returns the file descriptor

G.Shyama Chandra Prasadon success and returns102
creat () …Contd...

Example :
fd = creat(“newfile”,0666);
if ( fd == -1 )
printf(“error in creating file newfile “);
Creates a file called “newfile” in mode 0666 i.e., read and write
permissions for owner, group, and others
Note : while creating a new file ensure umask is set to zero
If umask had been 0022 in the envirnoment variable, then the
effective permission would be
mode & ~0022
G.Shyama Chandra Prasad 103
chmod ()
* chmod() set permissions for the file

int chmod (filename, mode)

char *filename;
int mode;

filename - character pointer to the name of

the file

mode - Integer signifying the mode

The mode is specified in octal code

chmod() returns 0 on success

and returns -1 on G.Shyama

error Chandra Prasad 104
chmod () …Contd...
Example :

ret = chmod(“test.c”,0600);

if ( ret == -1 )

printf(“error in changing the file


Changes the permission of file test.c

i.e., read and write permissions for owner
G.Shyama Chandra Prasad 105
chown ()
* System call chown()

* chown() set ownership for the file

int chown (filename,owner ,group)

char *filename;
int owner , group ;

filename - character pointer to the name of

the file

owner - owner id

group - group id

chown() returns 0G.Shyama

on success and returns -1 on error
Chandra Prasad 106
chown () …Contd….
Example :

ret = chown(“test.c”,0,1);

if ( ret == -1 )

printf(“error in changing the owner and group

of the file”);

Changes the owner and group of the file test.c
as root and others respectively
G.Shyama Chandra Prasad 107
lseek ()
* lseek() changes the position of read-write pointer for
the file descriptor

int lseek(filedes, offset, origin);

int filedes, origin;
long offset;

lseek() returns new-value of the pointer on success

returns -1 on error

The value of the pointer depends on origin :

0 set the pointer to offset bytes from the begining of
the file
1 increment the current value of the pointer by offset
2 set the pointer to the Chandra
G.Shyama size Prasad
of the file plus offset bytes
lseek () …Contd...
Example :

ret = lseek(fd,1000,0);

if ( ret == -1 )

printf(“error in seeking to the 1000’th byte of

the file “);

lseek(fd,1000,0) skips the first 1000 bytes of the

file and starts reading fromPrasad
G.Shyama Chandra the 1001’th byte 109
fork ()
* fork() creates a new process which is a
child process

* Child process is a logical copy of the

parent process

* Parent’s return value is the process id of the


* Child’s return value is 0

G.Shyama Chandra Prasad 110

getpid () getppid () getpgrp ()

* getpid() returns the process id of the calling


* getppid() returns the parent process id of the

calling process

* getpgrp() returns the process group of the calling

G.Shyama Chandra Prasad 111
fork(), getpid(), getppid(), getpgrp()

#include <stdio.h>
int id ;
int pid , pgrp ;
int ppid ;
id = fork();
printf (“PPID - %d PID- %d id - %d \n”, getppid() ,
getpid() ,id );
printf (“PGRP - %d \n “,
G.Shyama getpgrp()
Chandra Prasad ); 112
fork () , getpid () , getppid (), getpgrp

PPID - 371 PID - 372 id - 0 - From child Process

PGRP - 136

PPID - 136 PID - 371 id - 372 - From Parent Process

PGRP - 136

PID - Process-ID

PPID - Parent Process - ID

ID - Returned value from fork()

PGRP - Process-group-ID

If PID is equal to the process PGRP then the process is the group
leader G.Shyama Chandra Prasad 113
wait () AND execl ()
* wait() causes a parent to stop running and await the termination of
a child process

* execl() overlays the original process with a new set of instructions

Example on execl()
#include <stdio.h>
int id;
printf ( “Parent process \n”);
if ( ( id = fork() ) == 0 )
printf(“Statement from child process\n”);
printf(“ Parent Chandraagain
process Prasad \n”); 114
EXAMPLE ON execl()
Parent Process
Statement from child process
Parent process again
Tue Sep 10 11:34:17 1991
Process forked two processes and parent process avoided execl() to print the
final statement i.e parent process did not wait for the child to finish
To make the parent wait for the child to finish - wait() can be used
The example on execl() gets modified as
#include <stdio.h>
int id;
printf ( “Parent process \n”);
if ( ( id = fork() ) == 0 )
printf(“Statement from child process\n”);
printf(“ Parent process again \n”);

G.Shyama Chandra Prasad 115


Parent Process

Statement from child process

waits for the
child to finish

Tue Sep 10 11:34:17 1991

Parent Process again

G.Shyama Chandra Prasad 116
Process Related System Calls

G.Shyama Chandra Prasad 117


Execve executes the program file filename, overlaying

the address space of the executing process.Argv is an
array of character string parameters to the execed
program and envp is an array of character strings that
are the environment of the new process
execve(filename, argv, envp)
char *filename
char *argv[]
char *envp[]

G.Shyama Chandra Prasad 118


Fork creates a new process. The child process is a logical

copy of the parent process, except that the parent’s return
value from the fork is the process ID of the child, and the
child’s return value is 0

G.Shyama Chandra Prasad 119


Wait causes the process to sleep until it discovers a

child process that had exited or a process asleep in trace
mode. If wait_stat is not 0, it points to an address that
contains status information on return from the call.
int *wait_stat

G.Shyama Chandra Prasad 120


Exit causes the calling process to terminate, reporting

the 8 low-order bits of status to its waiting parent. The
kernel may call exit internally, in response to certain
int status

G.Shyama Chandra Prasad 121

Some other System Calls
 getuid()  Returns the real user ID
 geteuid()  Returns the effective user ID
 getgid()  Returns the real group ID
 getegid()  Returns the effective group ID
 getpid()  Returns the process ID
 getppid()  Returns the parent process ID

G.Shyama Chandra Prasad 122


Software Interrupts

G.Shyama Chandra Prasad 123

Classification of Signals
 Termination of a process
 Process induced exceptions
 Unrecoverable conditions during system calls
 Unexpected error condition during a system call
 Tracing execution of a process

G.Shyama Chandra Prasad 124

Handling Signals
 Exit on receipt of a signal
 Ignore the signal
 Execute a particular function on receipt of a signal

G.Shyama Chandra Prasad 125

Signal Syntax

#include <signal.h>
int sig;
void (*func)()
Signal allows the calling process to control
signal processing.

G.Shyama Chandra Prasad 126

Signal Syntax

Some values of sig

SIGHUP hangup
SIGINT interrupt
SIGFPE floating point exception
SIGBUS bus error
SIGPIPE write on a pipe with no reader
SIGTERM software termination
G.Shyama Chandra Prasad 127
Signal Syntax

The function is interpreted as

SIG_DFL default operation. Process
terminates except for some
SIG_IGN ignore the signal.kernel calls a
function in the process with
signal number as the argument.
SIGKILL can’t be ignored

G.Shyama Chandra Prasad 128


Kill sends the signal sig to the process identified by pid

int pid,sig
pid > 0 send signal to process whose PID is pid
pid 0 send signal to processes whose process group ID is
pid of sender
pid -1 if effective UID of sender is of superuser send
signal to all processes otherwise to processes whose real
UID equals effective UID of the sender

G.Shyama Chandra Prasad 129

Inter Process Communication

G.Shyama Chandra Prasad 130

Types of IPC
 Pipes
 Named Pipes
 Messages
 Shared Memory
 Semaphores
 Sockets

G.Shyama Chandra Prasad 131

 Allow data transfer in a FIFO manner
 Allow synchronization of process execution
 Use file system for data storage
 Can be created using “pipe()” system call
 Pipe manipulation is done using regular system calls for files
e.g. read(), write()
 Only related processes can access a pipe

G.Shyama Chandra Prasad 132


pipe (file_descriptors)
where file_descriptors is a pointer to an array of 2 integers
which are the file descriptors for reading and writing the

G.Shyama Chandra Prasad 133

Named Pipes
 Similar to pipes
 Have directory entries
 “open” system call used to open named pipes
 “mknod” system call to create named pipes
 Unrelated processes can also communicate using named pipes

G.Shyama Chandra Prasad 134

 Allow processes to send data streams to arbitrary processes
 Four system calls cover all the operations
 msgget
 msgsend
 msgrec
 msgctl

G.Shyama Chandra Prasad 135


Returns the message queue identifier assocaited with the key

msgget(key, flag)
key is the name of the message queue
flag is the modes which can be passed

G.Shyama Chandra Prasad 136


msgsnd (id, message, size, flag)

Sends message to the queue associated with the message queue identifier specified
by id
message is of the form
struct msgbuf
long mtype;
char mtext[];
Flags can be
IPC_NOWAIT- Return -1 if the message cant be stored
MSG_NOERROR- Message bigger than size is shortened with no error returned
G.Shyama Chandra Prasad 137

msgsnd (id, message, size, type, flag)

Reads message from the queue associated with the message
queue identifier specified by id
message is of the form
struct msgbuf
long mtype;
char mtext[];
G.Shyama Chandra Prasad 138

Msgctl ( id, command, buffer )

Provides a variety of message control functions as specified by
Commands can be
IPC_STAT store information about queue in the buffer
IPC_SET Change the information in the buffer
IPC_RMID remove the message queue identifier specified
by id

G.Shyama Chandra Prasad 139

 Allows processes to share part of their virtual address space
 System calls similar to the ones in messages

G.Shyama Chandra Prasad 140


int shmget ( key, size, flag )

key is the name of the shared memory

size is the size of shared memory segment in bytes
flag is a combination of

G.Shyama Chandra Prasad 141


Inserts a shared memory segment into a process address space

char *shmat ( id, at_address, flag )

id is the shared memory segment identifier

size is the address in process address space where the shared
memory segment has to be attached
flag is a combination of
SHM_RDONLY read only
0 read,write

G.Shyama Chandra Prasad 142


Removes a shared memory segment into a process address space

int shmdt ( dt_address )

dt_address is the address in the process address space of a

shared memory segment to be detached

G.Shyama Chandra Prasad 143


shmctl ( id, cmd, buf)

Provides shared memory control operations as specified by the



G.Shyama Chandra Prasad 144

 Non-negative integer count used to coordinate access to the
 Initial semaphore count set to the number of free resources
 Count decremented or incremented by the threads or processes
as and when they acquire or free resources.
 Threads block at count zero till it becomes positive

G.Shyama Chandra Prasad 145


semget ( key, num, flag )

Returns or creates the semaphore identifier associated with the
key is the name of the semaphore set
number defines the number of semaphores in the set
flag is a combination of
G.Shyama Chandra Prasad 146

semop ( id, operations, number )

Performs operations on the semaphore
id is the semaphore identifier
operations is the array of semaphore operation structure
operation consists of
- semaphore number
number is the number of G.Shyama
in operations
Prasad 147

semctl ( id, semnum, cmd, ... )

Performs operations on the semaphore as specified by the cmd
id is the semaphore identifier
Fourth argument is optional, depending upon the operation
If required, then it’s a buffer.

G.Shyama Chandra Prasad 148


Creates an endpoint for communication and returns a descriptor

int socket ( domain, type, protocol )
domain specifies communication domain
type type of communication over socket (virtual circuit or
protocol protocol to control the communication

G.Shyama Chandra Prasad 149

Other Socket System Calls

G.Shyama Chandra Prasad 150


Closes the communication endpoint

close ( sd )
sd is the socket descriptor

G.Shyama Chandra Prasad 151


Assigns a name to an unnamed socket

int bind ( sd, sockname, namelength)
sd is the socket descriptor
sockname is the name of the socket
namelength is the length of the name

G.Shyama Chandra Prasad 152


Makes a connection to an existing socket

int connect ( sd, sockaddr, namelength)
sd is the socket descriptor
sockaddr is the name of the target socket

G.Shyama Chandra Prasad 153


Receives incoming requests for a connection

accept ( fd, addr, addrlen )
addr user data array which kernel fills with the address of the
connecting client

G.Shyama Chandra Prasad 154


Transmits data over a connected socket

send ( sd, msg, length, flag)
msg pointer to data being sent
length length of the data

G.Shyama Chandra Prasad 155


Receives data over a connected socket

recv ( sd, buf, length, flag)
buf pointer to data being sent
length length of the data

G.Shyama Chandra Prasad 156


Closes a socket connection

shutdown ( sd, mode )
mode specifies which side no longer permits data transmission

G.Shyama Chandra Prasad 157

System Administration
 File System Administration
 Machine Management
 System Tuning and Performance Management
 System Accounting
 User Management
 Backup/Restore Activities
 Setting up Peripheral Devices
 Routine Maintenance

G.Shyama Chandra Prasad 158

File System Administration
 Creation and Maintenance of File Systems
 File System consistency checks
 Monitoring of disk space
 Routine Maintenance such as removing inactive files

G.Shyama Chandra Prasad 159

File System Administration
 Check File System  fsck
 Display disk usage  df
 List files by size  du
 Identify FileSystem type  fstyp
 Create File System  mkfs
 Mount File System  mount
 Unmount File System  umount

G.Shyama Chandra Prasad 160

Machine Management Commands
 Set boot defaults  fltboot
 Change to firmware  shutdown
 Power Off  shutdown
 Reboot  shutdown
 Display Configuration  prtconf
 Display System  uname

G.Shyama Chandra Prasad 161

System States
 System State  Description
 0  Powerdown State
 s  Single user
 2  Multiuser
 3  Start Remote File Share
 5  Switch to firmware
 6  Halt & Reboot OS
 q  Reexamine (/etc/inittab)

G.Shyama Chandra Prasad 162

Changing System States

Switching from multiuser to single user

Switching to any state
INIT <System State>
Switching to multiuser
Turning system off
0 Chandra Prasad 163
User and Group Management
 Assigning and Maintaining logins and passwords
 Organization of system resources to suit particular needs of the
 Communicating with users

G.Shyama Chandra Prasad 164

User Management Commands
 Task  Command
 Add User  useradd
 Add Group  groupadd
 Change password  passwd
 Listing users and groups  logins, listusers
 Modify user attributes  usermod
 Modify group attributes  groupmod
 Remove user  userdel
 Remove group  groupdel

G.Shyama Chandra Prasad 165

Communicating with Users
 Keeping users informed about system status and servicing
 Using “Wall” to send messages to the users
 Using utilities like “mail”
 Message of the day facility using “/etc/motd”

G.Shyama Chandra Prasad 166


G.Shyama Chandra Prasad 167


Provides a method for maintaining an up-to-date version of

programs that consist of a number of files generated in a
variety of ways

G.Shyama Chandra Prasad 168


• Find the target in the description file

• Ensure that all the dependencies (files) of the target exist and
are up to date
• Create the target in case any of the dependencies have been

G.Shyama Chandra Prasad 169


Comments All characters after a ‘#’

Continuation lines Lines ending in ‘\’
Macro Definition Identifier followed by ‘=‘ sign

G.Shyama Chandra Prasad 170

Example MakeFile “makedemo”

prog : x.o y.o z.o
cc x.o y.o z.o -o prog
x.o : x.c defs.h
cc x.c
y.o : y.c defs.h
cc y.c
z.o : z.c defs.h
cc z.c
G.Shyama Chandra Prasad 171

Macro Definition
name = value

OBJECTS = x.o y.o z.o

prog : $(OBJECTS)
cc $(OBJECTS) -o prog

G.Shyama Chandra Prasad 172