Sreenivas Koppuravuri

Session Contents
              

History of Unix Flavors of Unix Applications of UNIX Unix Features Unix Architecture Relationship :Kernel, Shell, User Programs Various shells Unix File System Unix Process Management I/O Redirection and Pipes User and group management About vi editor Unix Commands Shell Scripting Sed ( Stream Editor) , Awk programming .

UNIX Introduction
 Developed in 1969 by Ken thompson
 Popular OS, available from micro to mainframe  Available for scientific as well as commercial use

History of Unix
 Originator : Ken Thompson
 Redesigned MULTICS SYSTEM and evolved early    

version of UNIX Developed on PDP-7 machine 1971 was written in B language 1973 rewrote the UNIX OS in C 1981 commercial version of UNIX released

Applications of UNIX
        

Business environment Word Processing Financial Accounting Database management Communication Graphics Software development Office automation Academic and scientific environment
 Operating system research  Prime operating system for conducting research work

Flavors of Unix:
   
 

 

UNIX is a registered trademark of The Open Group. FREENIX is a trademark of Applied Digital Arts. Linux is a Registered Trademark of Linus Torvalds. UnixWare and Open Server are trademarks of Santa Cruz Operation. Tru64 is a trademark of Compaq. AIX is a trademark of International Business Machines Corporation. Solaris is a trademark of Sun Microsystems. NetBSD is a free, highly portable UNIX-like operating system available for many platforms.

Falvours of Unix:
 HP-UX is a trademark of Hewlett-Packard Co.
 IRIX is a trademark of Silicon Graphics, Inc.  U/WIN is a trademark of Global Technologies Ltd.,

Inc.  A/UX and Mac OS X are trademarks of Apple Computer.  Free BSD and BSD/OS are trademarks of BSDi.  Minix is a Unix-like freely distributed operating system.

Unix Features
 Multi User
 Multi Tasking  Portability

 Security
 Modularity

 Multi user

Unix Features
server

-Several users can use the computer at the same time. Unix uses time slicing to enhance multi user environment.

Unix Features
 Multi Tasking

Multi Tasking - Perform several tasks at the same time. t 1 t 0 Some tasks are placed in the background while the user continues to work in the foreground. A background task is essentially a non - interactive task.

Unix Features
 Portability

Ability of the software to operate on any machine effectively without major changes to the software.

 

Porting the O/s - hardware independent
Porting the Data - utilities are available. Porting the Applications - transportation is done by recompiling the source codes.

Unix Features
Modularity The Modular design of the O/S allows various parts of the Unix system to be added or removed according to requirement without affecting the performance of other components.

Unix Architecture

Relationship :Kernel, Shell, Utilities and Apps
KSUA
User commands and data Prompt Shell --------------------------------Built-in Commands Output cat Transfer of control Request for services ls Output vi User Interactivec ommands

Unix Kernel and Device Drivers

Kernel
 Core of the UNIX system
 Interacts with the hardware directly  Insulates other parts of the UNIX from the hardware

 Perform all low level functions
 Part of kernel deals with the I/O devices, called Device

Drivers  All programs and applications interact with the kernel

UNIX Kernel
 Memory management
 Process scheduling  File management and security

 Error handling
 I/O services  Date and time services  System accounting

What is Shell?
 Shell is the agency between the user and the unix

operating system. Like command processors of other operating systems, the shell processes the instructions that you issue to the machine. These instructions are called COMMANDS.
 Simply stated, shell is a file that runs when a user logs

in and terminates when a user logs out.

Shell activities
 It issues the $ command and waits for user command.
 After the command is entered, the shell scans the

command line and and parses it.  The command is then passed to the kernel for execution and the shell waits for its completion.  Again $ prompt appears and the shell waits for the user to enter the next command.  When there is no input from user, the shell is said to be sleeping.

Various shells:
 Bourne (sh): this is the original Unix shell, available in

  

nearly every flavor of Unix. Korn (ksh): a backwards-compatible upgrade to the Bourne shell, developed by David G. Korn at AT&T Bell Laboratories. Bash (bash): Bourne-again shell, a public domain shell containing features of the Bourne, Korn and c-shell. C-Shell (csh) and T-C-Shell (tcsh): The c-shell is similar to C programming language; t-c-shell extends csh. Z-Shell (zsh): an open-source Unix shell.

Various shells Manual sties:
 Bourne shell programming  http://www.mcsr.olemiss.edu/unixhelp/scrpt/scrpt2.html
 The KornShell Command And Programming Language  http://www.kornshell.com/  Bash Reference Manual  http://theory.uwinnipeg.ca/localfiles/infofiles/bash/bashref_toc.html  The C Shell tutorial  http://www.eng.hawaii.edu/Tutor/csh.html

The Z Shell Manual  http://zsh.dotsrc.org/Doc/Release/zsh_toc.html

Various shells:
 You can see what shells are available on your system

like this:
 $ cat /etc/shells  You can see what shell you are currently using like this:  $ echo $SHELL

File System
 Provides a logical method for organising retrieving

and managing information. The structure of the file system is hierarchial , resembling an inverted tree.

File hierarchy

Software Directory

System Logs

Configuration files

Executables

Types of files
 Ordinary files
 Directory files  Special files (device files)  Character device files  Block device files  FIFO files

File naming conventions
 Max 14 characters long
 No concept of primary and secondary name  May contain alphabets,dots,digits and underscores

 No embedded spaces or tabs
 System commands cannot be used  Case sensitive

UNIX directories
 Directories : special files containing names of files and

other sub-dirs  Home directory
 Directory assigned by super-user in which a user

normally works

 Current directory  Directory from which user is currently issues commands

Standard directories of a UNIX system
 /etc - stores system administration utilities
 /bin - stores most commonly used UNIX commands  /usr - stores all user home directories and some UNIX

commands  /dev - stores all the device files  /lib - stores library files for C programming  /tmp - used for temporary storage

Files and processes
 Everything in UNIX is either a file or a process.
 A process is an executing program identified by a unique PID (process

identifier).
 A file is a collection of data. They are created by users using text

editors, running compilers etc.

File Access Permissions
 Groups :  Primary Group Membership (/etc/passwd)  Secondary Group Membership (/etc/group)  The ‘groups’ command  Every file belongs to one user and one group  Default user and group of a file when created  ‘chgrp’ and ‘chown’ commands

Access Rights
 ls –lag  chmod [options] file

list access rights for all files change access rights for named file

 Example:  -rwxrwxrwx
 - rw- --- ---

no-one mailbox file).

A file that everyone can read, write and execute (and delete). A file that only the owner can read and write - no-one else can read or write and has execution rights (e.g. your

Wildcards
*
  

Match any number of characters ? Match one character [ba] Match with one of the characters in the bracket [a-z] Match with one of the characters in between given two, both inclusive. [!abc] Match character not not appearing in bracket

Quotes
Quotes

 There are three types of quotes :
Double Quotes "Double Quotes" - Anything enclose in double quotes removed meaning of that characters (except \ and $).

Name

Meaning

"

'

Single quotes

'Single quotes' - Enclosed in single quotes remains unchanged.

`

Back quote

`Back quote` - To execute command

Unix Process Management
A process is program in execution  Starting a process 1)Foreground process 2)Background process  Process Handling  Moving foreground process to background nohup command  Listing running processes jobs ps  Killing processes

Processes Control Commands
 command &
 ^c  ^z   

run command in background kill the job running in the foreground suspend the job running in the
background the suspended job list current jobs foreground job number 1 kill job number 1 list current processes kill process number 1243


 

foreground bg jobs fg %1 kill %1 ps kill 1243

I/O Redirection and Pipes

Redirection
 Standard input :  Many commands take input from a file and sends output to

a terminal. These commands can also act effectively on a character stream, without knowing source of the stream.  The default source of this stream is keyboard. This stream is called the standard input stream.  The shell sets up the connection between command and the keyboard.  A stream can come from :
 The keyboard (default source)  A file (redirection)  Another program (pipeline)

Redirection : standard input.
 The shell can reassign the standard input from a file rather
   

than a keyboard. This connection is set by the shell using metachatacter ‘<‘. Eg. $ wc < infile Here the shell reassigns the input to the wc (word count) command to come from a disk file instead of keyboard. If input is taken from multiple sources, the ‘-’ symbol must be used to indicate sequence of taking input. Eg. $ cat – myfile <indicates first from standard input and then from ‘myfile’>

Redirection : standard output
 The standard output stream can also have three possible

destinations :

 The terminal ie. User screen (default destination)  File  Input to another program

 The default destination can be changed using ‘>’ operator.  Eg. $ wc infile > newfile  Here the results of the wc command will be sent to the disk

file ‘newfile’.  Eg. $ wc infile >> afile  The >> can be used to append the output to the file instead of overwriting it.

Redirection : standard error
 All error messages are written to the standard error stream.  Error messages such as ‘file not found’ are by default
   

displayed on terminal. We need to use the ‘2>’ descriptor to set the standard error; Eg: $ cat nofile 2> errorfile. Here if the file ‘nofile’ doesn’t exist in the directory then the error message ‘No such file or directory’ will be written to the file ‘errorfile’ instead of being displayed on the screen. The ‘2’ here actually represents the standard error file. In fact 0 and 1 are file descriptors for input and output streams but need not be explicitly specified with their corresponding redirection operators.

Input/ Output Redirection
I/O Abbreviation Stream Standard stdin Input Standard stdout Output Standard stderr Error Description Commands receive information from the user via standard input. By default, standard input is the keyboard. Commands send non-error output to the user via standard output. By default, standard output is the terminal (i.e. written to the screen). Commands send error messages to the user via standard error. By default, standard error is the terminal.

Simple Redirection
Example cmd1 | cmd2 cmd < file cmd << text cmd > file cmd >> file cmd 2>file cmd 2>>file Description Pipe stdout from one command to stdin of another. Read stdin from a file. Read stdin until reaching a line identical to text . Redirect stdout to a file. (overwrite if file exists) Append stdout to a file. Redirect stderr to a file. (overwrite if file exists) Append stderr to a file.

Multiple Redirection

Example Description cmd1 2>&1 | Pipe stdout and stderr from one command to stdin of another. cmd2 cmd < f1 > f2 Redirect stdin and stdout simultaneously. cmd > file 2>&1 Redirect both stdout and stderr to a file. (overwrite if file exists) cmd >> file Append both stdout and stderr to a file. 2>&1 cmd > f1 2> f2 Redirect stdout to the file f1 and stderr to the file f2 . (overwrite if the files exist) cmd >> f1 2>> Append stdout to the file f1 and append stderr to the file f2 . f2 cmd | tee f1 Redirect stdout to the screen and to the file f1 . (overwrites f1 if it exists) cmd | tee -a f1 Redirect stdout to the screen and append it to the file f1 .

Grouping command
Example cmd1 ; cmd2 { cmd1 ; cmd2 ( cmd1 ; cmd2 Description Execute multiple commands on the same line. ; Execute } multiple commands as a group in the current shell. ) Execute multiple commands as a group in a subshell. Environment changes and defined variables will not persist outside the subshell. cmd1 && cmd2 Execute cmd1 . Execute cmd2 only if cmd1 succeeds. && is called the AND operator. cmd1 || cmd2 Execute cmd1 . Execute cmd2 only if cmd1 fails. || is called the OR operator

 pipe – output of one command is supplied as input to

Pipes another

 tee – read standard input & writes to std o/p & the file of

user choice

User and group management

To create a Unix user id following information is required

User Management

1. User Name
2. User Id 3. Group Id

4. Home directory
5. Login shell

 User name – must be unique consisting of 2-8 letters

User Management and numerals

 User id – Unique user id ranging from 100 to 60,000.

User ID’s 0-100 reserved for system users  Group id – unique numerical id to which the user belongs. Ranges from 100-60,000  Home directory – identifies user’s home directory path  Login shell – identifies the user shell Various shells are: Bourne shell (sh), korn shell (ksh), C shell (csh), bash shell (bash)

The user and group information is stored in following files:

User & Group Information

/etc/passwd
/etc/shadow /etc/groups

vi editor

Operates in two modes  Insert mode

vi mode

 Command mode

Pressing [Esc] key returns the editor in command mode

 Insert before cursor, before line

Inserting text

i, I

 Append after cursor, after line
 Open new line after, line before  Replace one char, many char

a, A
o, O r, R

 Character to right, left

Deleting text

x, X

 To end of line
 Line  Word

D
dd dw

Copying is called yanking in vi

Copy and paste text
yy

Yanking Text  Line  Character yl
Pasting is referred to as Putting text in vi

Putting Text  Put after position or line  Put before position or line

p P

 Search forward

Search for strings

/string

 Search backward

?string

 Repeat search in same, reverse directions n, N

The search and replace command is accomplished with :s

Replace

The syntax is :%s/pattern/string/flags Or :start line no., end line no. s/pattern/string/flags

flags – g, c

 Exit saving changes

Quitting

:wq

 Quit (unless changes)
 Quit (force, even if unsaved)  Write (forcibly)

:q
:q! :w!

Unix Commands

The command
 The shell which processes commands is itself a program

which can be invoked by giving command ”sh”. This program is located in directory ‘/bin’.
 Whatever typed at the $ prompt is input to the sh program.
 To change the prompt so that it shows the path to your

current working directory, enter: export PS1='$PWD > '  To see the new prompt each time you login, place this in your .profile (login file).

Internal and External Commands
 Internal Commands :  These are built into the shell.  Eg.cd
 External Commands :  A separate process is spawned.  Steps followed :  Check if built-in command  Check if absolute path is given  Search in PATH – The environment variable containing a list of directories

COMMANDS
         

Simple unix commands Organizing a directory File management commands Security Commands Status information commands Working with text Controlling a running program Redirection commands communication commands Process management commands

Command Substitution
 This enables the argument of one command to be

obtained from the output of another command.
 Eg.  $ echo “Today is date” Output will be : today is date

 $ echo “Today is `date`” Output Today is Tue Dec 16 05:56:59 CST 2003

Help
Getting help :
 man command

for a  whatis command command

read the online manual page command brief description of a

Simple Unix Commands
 clear - clear the screen
 banner <text> - print in large letters  pwd

- prints the current working directory  echo <text> - echoes the text on the screen  cal - prints the calendar for the year.  cal [month] [year] - gives the calendar for the specified month & year.

Listing Files and Directories
 ls
 ls –a     

list files and directories list files and directories

including hidden mkdir make a directory cd directory change to named directory cd change to home-directory cd .. change to parent directory pwd display the path of the current directory

Copying Files
         

cp file1 file2 mv file1 file2 rm file rmdir directory cat file more file head file tail file grep keyword' file wc file in

copy file1 and call it file2 move or rename file1 to file2 remove a file remove a directory display a file display a file a page at a time display the first few lines of a file display the last few lines of a file search a file for keywords count number of lines/words/characters file

-e ListsUnix information about all process now running Basic Commands
 

ps – report process status
-f Full listing

echo – echo arguments
 

\c Continues the output on the same line echo “Hello World!”

chmod – change mode

chmod 755 <filename>

ls – directory listing
 

-a list all entries including those beginning with . -l long listing

Basic Unix Commands Contd. -b number the lines as in –n but omit numbers for blank line
 

cat – concatenate and display files

-s silent about non-existent files  cat <filename>

find – find files
-name True if pattern matches the current file name  -print causes current path name to be printed  find <pathname> -name <filename> -print

grep – search a file for a pattern
-l print only the name of the file with matching lines  -v print all lines except those that contain the pattern

date – write the date and time
date ‘+%m%d%y’  -u – display the date in GMT

Basic Unix -i interactive copy Commands Contd.
 

cp – copy files

-r copy files and subdirectories  cp <source_filename> <destination_filename>

mv – move files
-i asks for confirmation before moving if file already exists  -f move without prompting

 

rm – remove files

Same options as cp/mv

who – who is on the system
-m – output info about current terminal  -q – display only names and number of users currently logged in

whoami – display the effective current username

 

Basic Unix Commands Contd.
wc – line, word and character count in a file
-l – count number of lines -w – count number of words
 

pwd – print working directory

more – list file contents by screen
 

-c – clear before displaying -w – prompts and waits for key to be hit before exiting

diff – show difference between two files
  

-i – ignore the case of the letters -w – ignore all blanks and spaces diff –w <first filename> <second filename>

Basic Commands Contd. -c – listUnix following c specifies char positions eq –c1-72
 

cut

-d – delimiter  -f – show fields separated by delimiter  cut –c1-10 <filename>

paste
-d – delimiter character  -s – concatenate each separate line in a file into a single line  paste <first filename> <second filename> > <Out_filename>

sort – sort the file
-r – reverse the sorting direction  -n – restricts sort key to initial numeric value

-a – append the output to existing file Basic Unix Commands Contd.

tee – send output to terminal and file

uniq – show unique lines in a file
-d – suppress the writing of lines that are not repeated in the input  -u – suppress the writing of lines that are repeated in the input

 

expr – evaluate an expression head – output beginning of file

-n number – the first number of lines will be shown on std output

tail – output end of file

-f – link files without questioning the user Contd. Basic Unix Commands
 

ln – link to other file

-s – create a symbolic link

file – show the file type

-h – do not follow symbolic links


tr – translate characters

tr ‘[a-z]’ ‘[A-Z]’ < <input_filename>

shift

alias – display command aliases
 

Works in ksh and csh alias dir=‘ls –l|grep ^d’

Links
 Files with two or more names
 Hard Links  Can cross file systems  Not for directories  Syntax: ln filename linkname  Symbolic Links  Can cross file systems  Also for directories  Syntax: ln –s filename linkname

Other Useful Commands
       

quota df du compress gzip file history Recalling commands using ‘!’
 % !! (recall last command)
 % !-3 (recall third most recent command)  % !5 (recall 5th command in list)  % !grep (recall last command starting with grep)

Shell Scripting

 Shell Script is series of command written in plain text

What is Shell file. Shell script is justScript? like batch file is MS-DOS but
have more power than the MS-DOS batch file

 Need for shell script :  Shell script can take input from user, file and output them on screen.  Useful to create our own commands.  Save lots of time.  To automate some task of day to day life.  System Administration part can be also automated

Shell variables
     

No type declarations required. No initializations necessary before using them. Shell variables are assigned using = operator. Shell variables are evaluated using a $. All shell variables are initialized to null by default. Eg. $ x=10 # no white space on either side of the = $ echo $x # $ required at evaluation time.  Variables can be concatenated by placing then adjacent.  Eg $ z=$x$y

Predefined shell variables
 There is a set of predefined variables in the shell.

These variables are used to store values and also to change the behavior of called programs.  Eg. PATH he path variable controls, where the shell searches for commands, when you type them to at the prompt.  Another example of a variable is the HOME variable, which contains the path to your home directory
$ echo $HOME /users/smith

 Use any editor like vi to write shell script  Unix knows a file is a shell script if it starts with

How to write shell script

#!/bin/sh the path /bin/sh is the path of the shell program which will be used.  An example script :#!/bin/sh # Script to print user information who currently login , current date & time clear #clears the screen echo "Hello $LOGNAME" echo "Today is \c ";date echo "Number of user login : \c" ; who | wc -l echo "Calendar" cal exit 0

 After writing shell script set execute permission for your

Executing script as follows the script

 Eg. $ chmod +x your-script-name

or $ chmod 755 your-script-name

 Execute your script as  $ bash your-script-name $ sh your-script-name $ ./your-script-name

 Comments in shell script :

# followed by any text is considered as comment in a script file.

Command Line Arguments Telling the command/utility which option to use.
1. 2. Informing the utility/command which file or group of files to

Why needed?

process (reading/writing of files).

Eg: $ myshell foo bar

 

1 = Shell Script name i.e. myshell referred as $0

2 = First command line argument passed to myshell i.e. foo Referred as $1  3 = Second command line argument passed to myshell i.e. bar Referred as $2  $# is a built-in shell variable indicating no of arguments. Here $#=2

 Used to get input (data from user) from keyboard and

Read: Making Scripts Interactive store (data) to variable.

 Syntax:

read variable1, variable2,...variableN  Eg. $ read fname
 While executing the script if the number of arguments

supplied are less than the number of arguments required then leftover variables are left unassigned.  If arguments exceed number of variables, excess arguments are assigned to the last variable.

 If particular command/shell script is executed, it return

Exit Status a is command two type of valuesof which used to see whether

command or shell script executed is successful or not.
 If return value is zero (0), command is successful.  If return value is nonzero, command is not successful or

some sort of error executing command/shell script.

 To determine this exit Status you can use $? special

variable of shell.  Eg. $ ls $ echo $? It will print 0 to indicate command ‘ls’ is successful.

Special Parameters Used by the Shell Parameter Significance Shell
$1, $2, etc . $*
$@ $# $0 $? $$

Positional parameters. All positional parameters as a string. Command line arguments taken as single string
All positional parameters as a string. Command line arguments are taken individually. Number of arguments specified in command line. Name of executed command. Exit status of last command. PID of the shell.

Conditional Execution : && and ||
 The shell provides two operators to control execution of a

command depending on the success or failure of previous command.
 Syntax: $ command1 && command2

- Command2 is executed if, and only if, command1 returns an exit status of zero.
 Syntax: command1 || command2

- Command2 is executed if and only if command1 returns a non-zero exit status.

 Both can be used as follows :

Conditional Execution… Syntax: command1 && comamnd2 if exist status is zero || command3 if
exit status is non-zero

Example: grep ‘^name:’ /etc/passwd || useradd name If the grep command fails then only useradd command will be executed

Or grep –v ‘^name:’ /etc/passwd && useradd name Using both && and || in one statement grep –v ‘^name:’ /etc/passwd && useradd name || \ echo “`date`: useradd failed”

Script Termination
 The ‘exit’ statement is used to prematurely terminate a

  

program. When this statement is encountered in a script, execution is halted and control is returned to the calling program, in most cases the shell. Argument provided with ‘exit’ is optional. If you specify an argument, the script will terminate with a return value of the argument. If no argument is specified, the value returned will be zero.

if...else...fi
 If given condition is true then command1 is executed

otherwise command2 is executed Form 1: Form 2:  Syntax:
if condition is true then execute commands else execute commands fi if condition is true then execute commands fi

Multilevel if-then-else
 Syntax:

if condition then

condition is zero (true - 0) execute all commands up to elif statement elif condition1
then

elif
fi

condition1 is zero (true - 0) execute all commands up to elif statement else None of the above condtion,condtion1 are true execute all commands up to fi

test: Numeric comparison
 The ‘test’ statement is used as a control command for

the if statement.  Test uses certain operators to evaluate the condition on its right and returns true or false exit status. Relational operator Meaning  Numeric shelltois confined to integers -eq comparison inEqual only. -ne Not equal to
-gt -ge -lt -le Greater than Greater than or equal to Less than Less than or equal to

Test Exit status test: String comparison -n str -z str True if string str is not null. True if string str is null.

s1 = s2 True if string s1 = s2 s1 != s2 True if string s1 is not equal to s2 str True if string str is assigned and not null.

test: File Test Test
-s file -f file -d dir Non empty file

Meaning

Is File exist or normal file and not a directory Is Directory exist and not a file

-w file
-r file -x file

Is writeable file
Is read-only file Is file is executable

Logical Operators
 Logical operators are used to combine two or more

condition at a time
Operator ! expression Logical NOT Meaning

expression1 -a expression2 Logical AND expression1 -o expression2 Logical OR

case Statement
 Good alternative to Multilevel if-then-else-fi statement.
 Syntax :  case $variable-name in pattern1) execute commands ;; pattern2) execute commands ;; pattern3) execute commands ;; *) execute commands ..... esac  The default is *) and it is executed if no match is found.

case Statement: example
case $rental in "car") echo "For $rental Rs.20 per k/m";; "van") echo "For $rental Rs.10 per k/m";; "jeep") echo "For $rental Rs.5 per k/m";; "bicycle") echo "For $rental 20 paisa per k/m";; *) echo "Sorry, I can not get a $rental for you";; esac

while loop
 Loop is executed as long as given condition is true.  Syntax:  while [ condition ] do command1 command2 command3 ….. done  Creating an infinite loop with ‘true’ command  while true ; do command1 ….. done

while loop: example
while [ $i -le 10 ] #while value of i is less than 10 do echo “number is : $i” done

for Loop
 Syntax 1:

for { variable name } in { list } do

done  Syntax 2 : for (( expr1; expr2; expr3 )) do done

execute one for each item in the list until the list is not finished (And repeat all statement between do and done)

repeat all statements between do and done until expr2 is TRUE

Functions
 Functions are powerful features that aren't used often enough. Syntax is

name () { commands }
 Example :
purge() { if [ ! -d $1 ]; then echo $1: No such directory 1>&2 return fi etc... }

Functions contd.
 Within a function the positional parmeters $0, $1, etc. are the

arguments to the function  Within a function use return instead of exit.  Functions are good for encapsulations. You can pipe, redirect input, etc. to functions eg: # take standard input (or a specified file) and do it. if [ "$1" != "" ]; then cat $1 | do_file else do_file fi

I/O redirection
 Redirection simply means capturing output from a file,

program, command or script and sending it as input to another file, command or script
 0, 1 & 2 are reserved file descriptors for stdin, stdout,

stderr  Additional file descriptors can be opened from 3 to 9 The additional file descriptors can be used as temporary duplicate link to stdin, stdout, stderr. This simplifies restoration after complex redirection and reshuffling

I/O redirection contd…
 Redirecting stdin using exec

Sample Code: exec 6<&0 exec <data-file read a1

#Link file descriptor #6 with stdin #stdin replaced by data-file

echo $a1 exec 0<&6 6<&-

#Restores stdin and closes fd #6 to free #it for use with other process

I/O redirection contd…
 Redirecting stdout using exec

Sample Code: LOGFILE=logfile.txt exec 6>&1 #Link file descriptor #6 with stdout exec >$LOGFILE #stdout replaced by data-file echo “Logfile:” date exec 1>&6 6<&#Restores stdout and closes fd #6 to free #it for use with other process

Korn Shell

Matching Patterns
Pattern * ? […] Example Boo* Boo? [aeiou]* Matches Boot,boo,booth Boot Ark Booth Bark Not matched

[!…]
*(cc|cc)

Boo[!st]
Boo*(ze|r)

Boor
Boo,boor,booze,boozer

Boot
Boot

Matching Patterns contd…
Pattern +(cc|cc) ?(cc|cc) @(cc|cc) Example boo+(ze|r) boo?(ze|r) boo@(ze|r) Matches boor,booze,boozer boo,boor,booze booze,booth Not matched boo boozer boo

!(cc|cc)
{c,c,c}

boo!(ze|r)
a{b,c,d}e

booth,boo,boot
abe,ace,ade

booze,boor
axe

Conditional Statements
Numbers
Format (( _num1_ == _num2_ )) (( _num1_ != _num2_ )) (( _num1_ < _num2_ )) (( _num1_ > _num2_ )) (( _num1_ <= _num2_ )) (( _num1_ >= _num2_ )) true if numbers equal numbers not equal

Conditional Statements
Strings Format [[ _num1_ == _num2_ ]] [[ _num1_ != _num2_ ]] [[ _num1_ < _num2_ ]] [[ _num1_ > _num2_ ]] [[ _num1_ = _pattern_ ]] [[ _num1_ != _pattern_ ]] [[ -z _str_ ]] [[ -n _str_ ]] [ x=y –o k=j ] [x=y –a k=j ]
true if strings equal strings not equal

str is null str is not null or in expression and in expression

Flow control statements
If-then
If _expr_ then _cmd(s)_ elif _expr_ _cmd(s)_ else _cmd(s)_ fi

Flow control statements contd.
case
case _word_ in _pattern1_) _cmd(s)_;; _pattern2_) _cmd(s)_;; *) break;; esac

Flow control statements contd.
while
while _expr_ do _cmd(s)_ done

Flow control statements contd.
for
for _variable_ in _list_ do _cmd(s)_ done

Flow control statements contd.
until
until _expr_ do _cmd(s)_ done

Positional Parameter
Program, function or shell Argument 1 through 9 nth argument Number of positional parameter Every positional parameter Value returned by last executed cmd Pid of shell Pid of last background command $0
$1 .. $9 ${n} $# $@, $* $? $$ $!

Redirections
0 1 2 stdin stdout stderr
close stding close stdout open filename for read-write open 2 for write and dup as 1

<&>&<>filename 2>&1

Other functionalities
V1=${V2:=V3} Set V1 with value of V2 if this is set else set the variable V1 with value of V3 ${V1:?word} if V1 set & V1!= NULL return $V1 else print word and exit ${V1:=word} if V1 !set | V1 – null set V1=$word ${V1:-word} if V1 set & V1!=null ret $V1 else ret word ${V1:+word} if V1 set & V1!=null ret word else ret nothing ${V1#patt} if patt are found at the begin of V1 return V1 without patt else return V1 ${V1%patt} if patt are found at the end of V1 return V1 without patt else return V1

sed

sed – Quick Introduction
 sed is a non-interactive stream editor  It performs basic text transformation on input string  SED works by making only one pass over the input(s),

and is consequently more efficient  Syntax : sed options ‘address action’ file  Sed has two ways of addressing lines
 By line number  By specifying a pattern which occurs in a line

sed addressing by line number
 Examples :sed ‘3q’ file –-displays first 3 records sed –n ‘2,5p’ file --displays first 2 to 5 records sed –n ‘$p’ file --displays last record sed –n ‘3,$!p’ file --displays 1 & 2 records sed –n –e ‘1,5p’ ’10,15p’ ’20,$p’ file --displays 1-5, 10-15 & 20-last records

sed Context Addressing
 Examples :-

sed –n ‘/abc/p’ file – searches for string “abc” and displays the record if found sed –n ‘/abc/,/def/p’ file – searches for string “abc” and ‘def’ displays the record if found sed –n ‘1,/abc/p’ file – displays 1 record & searches for string “abc” displays the record if found sed –n ‘/[abc][xyz]/p file -searches for pattern abc & xyz and displays. sed ‘$a\ Record to append\ Next record ’ file1 > file2 - Append record in the standard output and redirects it to other file.

sed Context Addressing
Examples :-

sed ‘i\ ********************************** ‘ file - Insert * between each record sed ‘/abc/d’ file - Deletes the search string in the output sed ‘1,$s/string1/string2/’ file -substitutes string1 with string2 sed –n ‘/pattern1/w file1 /pattern2/w file2’ file -Write searched patterns in the files sed –n –f pattern file data file -Includes a pattern file

sed contd.
 Redirecting the output sed ‘s/<pattern>/<replace with>/g’ <filename> > <new file>
 Insert spaces at the starting lines of file
sed ‘s/^/ /’ <filename>

 Display lines between markers sed –n ‘/Unix is/,/Above me/p’ <filename>  Print selected lines of a file sed –n 6,10p <filename>

sed contd.
 Print lines apart from selected lines in a file sed 6,10d <filename>
 Make single spaced file double spaced sed G <filename>  Stop printing after specified line number sed 3q <filename>  Deleting blank lines sed ‘/^$/d’ <filename>

sed Functions d
n 1a\ <text> 2i\ <text> 2c\ <text>

Delete a line Next line Append Text same as a except inserts before matching line no. delete line selected by addr. and change with text

How to create sed scripts?
 Sed commands can be grouped in one text file, known

as sed script.
 TIME=`date | sed 's/.* .* .* \(.*\) .* .*/\1/'`

awk

awk
 awk utility is powerful data manipulation/scripting

programming language.
 Use awk to handle complex task such as calculation,

database handling, report creation etc.  It is a tool for report writing.  It is the combination of Shell script, grep, sed and ‘C’ Programming Language.  Syntax:
awk options ‘address { action }’ file(s)

awk metacharacters
Metacharacter Meaning  Following is the list of metacharacters in awk . (DOT) * ^ $ Match any character Match zero or more characters Match beginning of line Match end of line

\
[] {} + ?

Escape character following
List Match range of instance Match one or more preceding Match zero or one preceding

Predefined variables in awk
awk variable Awk var FILENAME RS OFS Meaning Name of current input file Input record separator character (Default is new line) Output field separator string (Blank is default)

ORS
NR NF OFMT FS

Output record separator string (Default is new line)
Number of input record Number of fields in input record Output format of number Field separator character (Blank & tab is default)

awk Built in functions
 awk
Function int (x) sqrt (x) length Description Returns integer value Returns the square root Returns the length of the record

length (x) substr(s1,s2, s3)
index(s1,s2) split(string, arrayname) system(“comma nd”)

Returns the length of the x Sub String
Index position of string s2 in s1 Split string and store in array Runs Unix command

awk contd.
 Arithmetic with awk
 User defined variables in awk  printf statement

 awk Options

-F -f

Field Separator Include awk script file

ARGC Argument Count , ARGV List of Arguments predefined variables

awk operators
 awk

Oper ator < <= > >= == !=

Significance Less than Less than equal to Greater than Greater than equal to Equates Not equal to

awk if condition
 General syntax of if condition

Syntx: if ( condition ) { Statement 1 Statement 2 Statement N if condition is TRUE } else { Statement 1 Statement 2 Statement N if condition is FALSE }

Loops in awk
 For and while loops are used for looping in awk
 Syntax for loop

for (expr1; condition; expr2) { Statement 1 Statement 2 Statement N }

Loops in awk contd…
 Syntax while loop

while (condition) { statement1 statement2 statementN Continue as long as given condition is TRUE }

Thank you

Sign up to vote on this title
UsefulNot useful