Lecture-9

Introduction to Shell Script,

Variables in Shell, Command line
Arguments, Positional Arguments
and Metacharacter in Shell
 The Shell
• The shell is the agency between the user and the
UNIX system.
• The shell is a command processor; it processes
the instructions you issue to the machine.
• The shell is both a command language and a
programming language that provides an interface
to the UNIX operating system.
• The shell itself is a program through which other
programs are invoked.
 The Shell as Command Processor
• When you log on to a UNIX machine, you see a prompt. A
UNIX command is running once you log in. This command is
the shell. It withers away when you log out.
• The shell accepts and interprets users requests. It examines
and rebuilds the command line and then leaves the execution
work to the kernel.
• The kernel handles the hardware on behalf of these
commands and all processes.
• The shell is generally sleeping. It wakes up when input is
keyed in at the prompt.
• Sleeping, waiting, and waking are the cycle of the shell.
 The Shell as Command Processor
• The following activities are performed by the
shell:
– It issues the prompt and sleeps till you enter a
command.
– After a command is entered, the shell scans the
command line for some special characters
(metacharacters) for expansion.
– It then passes the command to the kernel in a form
which the kernel can understand.
– After the job is complete, the prompt reappears
and the shell returns to its sleeping role to start the
next cycle.
 When to Use Shell Scripts
• Customizing your work environment. For example, every time you
log in you want to see the current date, a welcome message etc.
• Automating your daily tasks. For example you may want to back up
all your program at the end of the day.
• Automating repetitive tasks. For example repetitive task of
compiling a C program.
• Performing same operation on many files. For example you may
want to replace a string printf with a string myprintf in all the c
program in the present directory.
You should not use shell programming when the task:
• Is too complex, such as writing an entire billing system.
• Requires a high degree of efficiency.
• Requires a variety of software tools.
 Shell Types
• In UNIX there are two major types of shells:
• The Bourne shell.
– If you are using a Bourne-type shell, the default prompt is the $ character.
• The C shell.
– If you are using a C-type shell, the default prompt is the % character.
• There are again various subcategories for Bourne Shell which are listed as
follows −
– Bourne shell ( sh)
– Korn shell ( ksh)
– Bourne Again shell ( bash)
– POSIX shell ( sh)
• The different C-type shells follow −
– C shell ( csh)
– TENEX/TOPS C shell ( tcsh)
 Shell Type Con’t
• The original UNIX shell was written in the mid-1970s
by Stephen R. Bourne while he was at AT&T Bell Labs
in New Jersey.
• The Bourne shell was the first shell to appear on UNIX
systems, thus it is referred to as "the shell".
• The Bourne shell is usually installed as /bin/sh on
most versions of UNIX. For this reason, it is the shell of
choice for writing scripts to use on several different
versions of UNIX.
• we are going to cover most of the Shell concepts
based on Borne Shell.
 Shell Scripts
• The basic concept of a shell script is a list of commands,
which are listed in the order of execution. A good shell
script will have comments, preceded by a pound sign, #,
describing the steps.
• There are conditional tests, such as value A is greater
than value B, loops allowing us to go through massive
amounts of data, files to read and store data, and
variables to read and store data, and the script may
include functions.
• Shell scripts and functions are both interpreted. This
means they are not compiled.
 Example Script
• Assume we create a test.sh script.
• Note all the scripts would have .sh extension.
• Before you add anything else to your script, you need to alert the system
that a shell script is being started. This is done using the shebang construct.
• For example −
– #!/bin/sh
• This tells the system that the commands that follow are to be executed by
the Bourne shell. It's called a shebang because the # symbol is called a
hash, and the ! symbol is called a bang.
• To create a script containing these commands, you put the shebang line
first and then add the commands −
– #!/bin/bash
– pwd
– ls
 Shell Comments
• You can put your comments in your script as
follows −
#!/bin/bash
# Author : Arush
#script contains list of commands
# Script follows here:
pwd
ls
 How to Create a First Shell Script
• To successfully write a shell script, you have to do three
things:
– Write a script
– Give the shell permission to execute it
– Put it somewhere the shell can find it
• Writing a script
– A shell script is a file that contains ASCII text. To create a shell
script, you use a text editor. A text editor is a program, like a vi
editor, emacs etc.
• Setting permissions
– The next thing we have to do is give the shell permission to execute
your script. This is done with the chmod command as follows:
$ chmod 755 my_script
• Putting it in your path
– When you type in the name of a command, the system does not search the
entire computer to find where the program is located. The shell maintains a
list of directories where executable files (programs) are kept, and just searches
the directories in that list. This list of directories is called your path.
1. You can add directories to your path with the following command, where
directory is the name of the directory you want to add:
$ export PATH=$PATH:directory
2. Most modern Linux distributions encourage a practice in which each user
has a specific directory for the programs he/she personally uses. This
directory is called bin and is a subdirectory of your home directory.
• If you do not already have one, create it with the following command:
$ mkdir bin
• Move your script into your new bin directory and you're all set. Now you
just have to type:
• $ my_script
• and your script will run
 Shell Script
shell scripts are files:
¬ starting with:
#!/bin/sh
­containing shell commands, made executable by
chmod u+x
• executed using a copy of the shell
$ cat >my-first-script

#!/bin/sh
echo hello world
$ chmod u+x my-first-script
• Try to exeute it directly
$ my-first-script
hello world
$
 Command Arguments And Parameter
• A command is a program that tells the Unix system to do
something. It has the form: command [options]
[arguments]
• Where an argument indicates on what the command is
to perform its action, usually a file or series of files. An
option modifies the command, changing the way it
performs.
• When the shell executes a file of commands, each
occurrence of $1 is replaced by the first argument, each
$2 is replaced by the second argument, and so on
through $9.
 Command-Line Arguments
• The command-line arguments $1, $2, $3,...$9 are positional
parameters, with $0 pointing to the actual command,
program, shell script, or function and $1, $2, $3, ...$9 as the
arguments to the command.
• Script uses various special variables related to command line −
#!/bin/sh
echo "File Name: $0"
echo "First Parameter : $1"
echo "Second Parameter : $2"
echo "Quoted Values: $@"
echo "Quoted Values: $*"
echo "Total Number of Parameters : $#"
 Example
• Here is an example of using the cal command with zero and
two arguments.
• If you type as
$ cal
– with zero argument it print current month and year
• Now if you give 2 arguments
$cal 11 2016
– It will print cal of current month.
Special Parameters $* $# and $@:
• There are special parameters that allow accessing all of the
command-line arguments at once. $* and $@ both will act
the same unless they are enclosed in double quotes, "".
• the "$*" special parameter takes the entire list as one
argument with spaces between
• the "$@" special parameter takes the entire list and
separates it into separate arguments but if “ “ are not use it
behave same as $*.
• $# is a special parameter which can be use to see total
number of positional parameter were set.
 Exit Status
• The $? variable represents the exit status of the previous command.
• Exit status is a numerical value returned by every command upon its
completion. As a rule, most commands return an exit status of 0 if they
were successful, and 1 if they were unsuccessful.
• Some commands return additional exit statuses for particular reasons. For
example, some commands differentiate between kinds of errors and will
return various exit values depending on the specific type of failure.
• Following is the example of successful command −
$./test.sh Zara Ali
File Name : ./test.sh
First Parameter : Zara
Second Parameter : Ali
Total Number of Parameters : 2
$echo $?
0
$
 Some special Parameter
• $$ - PID of current shell
• $? - exit status of the last executed command.
• $! - PID of last background process.
• $#- Total number of positional parameter.
• $0 - Name of the command being executed.
• $* - List of all shell argument. Can’t yields each
argument separately.
• $@ - similar to $*, but yields each argument
separately when enclosed in double quotes.
 Examples
1. Let create our own command in which we will change its
mode to executable.
– $echo ‘chmod +x $1’ >cx create cx as a file
– $ sh cx cx (making cx as exectable)
2. Consider searching a personal telephone directory. File
name is phone_book. And its contents are as follows:
Dial-a-joke 212-976-3838
Dial-a-prayer 212-246-4200
Dial santa 212-976-3636
 Example continue
• Then we will use grep command to search for the
pattern given as the argument:
$ echo ‘grep $* phone-book’ >111
$ cx 111
$ 111 joke
Dial-a-joke 212-976-3838 (output)
$ 111 dial
Dial-a-joke 212-976-3838 (output)
Dial-a-prayer 212-246-4200
Dial santa 212-976-3636
 Program Output as Arguments
• The output of any program can be placed in a
command line by enclosing the invocation in
backquotes ‘….’.
• For example
• $ echo “Today’s date and time is ‘date’.”
• Today’s data and time is Sun dec 20 00:02:15
EDT 2013.
• $
 The Metacharacters
• Unix Shell provides various metacharacters which have
special meaning while using them in any Shell Script and
causes termination of a word unless quoted.
• For example ? matches with a single charater while
listing files in a directory and an * would match more
than one characters.
• Here is a list of most of the shell special characters (also
called metacharacters) −
• * ? [ ] ' " \ $ ; & ( ) | ^ < > new-line space tab A character
may be quoted (i.e., made to stand for itself) by
preceding it with a \.
 Meta Characters
• The shell recognizes a characters as special.
They are as follows:
1. * Matches any string or group of characters.
A null or no character at all may also qualify
for representation by a *.
– Use ls *.lst to list all files with extension .lst.
2. ? matches a single character.
– Use cp ???.c progs to copy files whose filename
has 3 characters and the .c extension.
Pattern Matching – The Wild Cards
3. [ccc] matches a single character – ranges like
0-9 or a-z are legal.
– [cut] matches a single character – either a ‘c’, ‘u’,
or ‘t’.
– [!cut] matches a single character that is not ‘c’, ‘u’
or ‘t’.
4. [x-z] matches a single character from x to z.
– Use rm note[01][0-9] to remove files note00,
note01, …., note18, note19.
Pattern Matching – The Wild Cards
5. [!q-z] matches a single character that is not
within q to z.
• These metacharacters lose their meaning
when placed in the wrong place.
• The * doesn’t match all filenames beginning
with a . (dot), or the / of a pathname.
• To list all the hidden files having at least three
characters after the dot, use
ls .???*
 Examples
Command Output
ls a* Lists all files beginning with character ‘a’
ls ?? Lists all files whose names are 2 characters long.
ls a?b? Lists all 4 characters filenames whose first character is ‘a’ and
third character is ‘b’
ls [kdgp]* Lists all files whose first character is ‘k’, ‘d’, ‘g’ or ‘p’.
ls [c-fmrv-z]* Lists all files whose first character is ‘m’ or ‘r’ or is in the
range c to f or v to z
ls [!d-m]* Lists all files whose first character is anything other than an
alphabet in the range d to m.
 Process Execution Metacharacter
• To run more than one command at the $ prompt in stroke, we
separate them with a semicolon. For example,
– $ ls ; who ; banner Hi
• This would result in the execution of ls first, then who, and , lastly the banner
command display a big Hi.
• To execute some command in in a sub-shell, we can enclose
them in parentheses.
– $ (cd mydir ;pwd )
– /usr/aaa/mydir
– $pwd
– /usr/aaa
• Parentheses causes a subshell to be invoked, in which a change of directory is
executed. Back in the current shell, we are still in /use/aaa
Conditional Execution Using && and ||

• These metacharacters can be used to optionally

execute a command depending on the success
or failure of the previous command.
• command1 && command2
– The second command will be executed only if the
first succeeds.
• Command1 || command2
– The second command to be carried out only if the
first fails.
 Example
• grep makeup paper > orignal && cat paper
– The command following the && is performed only
if the exit status of grep is 0, that is only when
‘makeup’ is found in the file paper.
• grep makeup paper > orignal || echo pattern
not present
– The command following the || is performed only
if the initial command (grep) fails, that is, has a
nonzero exit status.
 Example Cont’d
• Grep arush addresses || grep arush file1 && cat
file1
– The pattern ‘arush’ is searched first in the file
addresses. If the search fails then the pattern is
searched in the file file1. if this search is successful then
the file file1 is displayed on the screen.
– If the first grep is successful, the second grep is skipped
and the cat is executed.
– If both the grep fail, the cat is not executed.
• Note: the metacharacter which comes first enjoys
a higher priority.
 General Syntax
Command1 Command2 Command 3 Executed?
Fails Fails No
Fails Successful Yes
Successful Not executed yes
 Quoting Metacharacters
• The following characters come under this
category:
1. \ (Escaping Character Backslash)
2. “ “ (Double Quotes)
3. ‘ ‘ (back quotes)
4. ’ ’ (single quotes)
 Escape Character (Backslash)
• A backslash that is not quoted shall preserve the
literal value of the following character, with the
exception of a <newline>.
• It’s generally accepted principle that filenames
shouldn’t contain the shell metacharacters.
• Image a file named chap* created with the * symbol:
$ echo > chap*
• If you use rm chap*, you will remove all files
beginning with chap.
 Escape Character (Backslash)
• The way to remove the file chap* is to use the backslash (\) as
follows:
$ rm chap\*
• The use of the \ in removing any special character is called
escaping or despecializing.
• For example, to remove a filename chap0[1-3], use:
$ rm chap0\[1-3\]
• The \ is elided so that
• echo \?
– will echo a single ?, and
• echo \\
– will echo a single \.
 Escaping – The Backslash (\)
• Apart from the wild cards, some characters are also considered
special by the shell:
| ,<,>, ‘, “,
• The new line character is also special to the shell.
• When you enter a long chain of commands or a command with
numerous arguments, you can split the command line by hitting
[Enter] after the \.
• Example :$echo abc\
>def\
>ghi
abcdefghi
$
 Single Quoting
• When a command argument is enclosed in
quotes, the meanings of all enclosed special
characters are turned off:
$ echo ’*?[8-9]’
• The argument above is said to be quoted.
• Output will be *?[8-9]
 Single Quoting
• The space is another character that has a special
meaning to the shell.
• To remove a file with the space in the filename,
you can use quotes:
$ rm ’good morning’
• A string of characters may be quoted by enclosing
the string between single quotes. For example,
• echo xx'****'xx
– will echo xx****xx
 Example of Single Quotes
• When you need to quote several character at once, you
could use several backslashes:
– % echo a\ \ \ \ \ \ \ b
• (There are 7 spaces between 'a' and 'b'.) This is ugly but works. It is
easier to use pairs of quotation marks to indicate the start and end of
the characters to be quoted:
– % echo 'a b'
• Inside the single quotes, you can include almost all meta-characters:
– % echo 'What the *heck* is a $ doing here???'
– What the *heck* is a $ doing here???
• The above example uses asterisks, dollar signs, and question marks
meta-characters.
 Double Quotes
• Sometimes you want a weaker type of quoting: one
that doesn't expand meta-characters like "*" or
"?," but does expand variables and does command
substitution. This can be done with the double
quote characters:
• Example
– % echo "Is your home directory $HOME?"
• Is your home directory /home/kreskin/u0/barnett?
– % echo "Your current directory is `pwd`"
• Your current directory is /home/kreskin/u0/barnett
Escaping and Quoting in echo
• The \ is also used to emphasize a character.
The \ combines with some character to
represent an escape sequence.
• These are escape sequence accepted by
echo:
– \t – A tab
– \f – A formfeed (page skip)
– \n – A newline
 Backquotes
• The backquote character looks like the single
quote or apostrophe, but slants the other way
• It is used to capture the output of a UNIX utility
• A command in backquotes is executed and then
replaced by the output of the command
• Example: Execute these commands date
save_date=‘date ‘
echo The date is $save_date
echo prints the output of ‘date‘ command.
 Quotes with in Quotes
• You can put either quotes inside the other.
• If you want to quote single quotes, use double
quotes around it. To quote double quotes, use
single quotes. Heck, it's easier to show you:
• Example:
– % echo "Don't do that"
• Don't do that
– % echo 'The quote of the day is: "TGIF"'
• The quote of the day is: "TGIF" %
 Examples With Quotes
Example 1
• Bad command line:
grep dog.*cat file (Shell tries to expand dot.*cat as file name wildcard )
• Use single quotes to avoid translation
grep 'dog.*cat' file
• Single quotes OK in this case because we don't need variable translation
Example 2
• Read name and search file for name
read name
grep "$name" dbase
• Single quotes not OK because we need variable translation
 Examples With Quotes
Example 3
• Bad command line: search for dollar sign
grep "Gimme.*$20" file
• Problem: shell translates variable $20
• Solution: use single quotes
grep 'Gimme.*$20' file
 Revision Quotes
Provide control of collapsing of spaces and translation of variables
• No quotes (variables translated, spaces collapsed)
echo Home: $HOME
output: Home: /users/us/freddy

• Double quotes (no collapsing)

echo "Home: $HOME"
output: Home: /users/us/freddy

• Single quotes (no translation or collapsing)

echo 'Home: $HOME'
output: Home: $HOME

• back quotes within double quotes

echo "Home directory ‘$HOME’ is full..."
or echo "Home directory $HOME is full..."
output: Home directory /users/us/freddy is full…
 Quoting Description

All special characters between these

Single quote
quotes lose their special meaning.
•Most special characters between these
quotes lose their special meaning with
these exceptions:
$
Double quote •`
•\$
•\'
•\"
•\\
Backslash Any character immediately following the
backslash loses its special meaning.
Anything in between back quotes would
Back Quote be treated as a command and would be
executed.
 Revision of Metacharacter
Metacharcter Meaning
* Matches more than one characters
? Matches exactly one character
[abc] Matches one character in the set a, b, or c
[a–z] Matches one character in the range from a to z
[!a–z] Matches one character not in the range from a to z
\ Escapes or disables the metacharacter

• Characters with special meaning to shell " ' ` $ * [ ] ? ; > < & ( ) \
• Avoid special meaning with quoting
echo 'You have $20'
 Creating New Command
• Given a sequence of commands that is to be repeated more
than a few times, it would be convenient to make it into a
“new” command with its own name, so you can use it like a
regular command.
• To create new command we need to do following things
• Suppose you intent to count users frequently with the
pipeline $who|wc-l
• Now the first step is to create an ordinary file that contains
‘who|wc-l’
– $echo ‘who|wc-l’ >nu
 Creating New Command
• The shell is a program just like an who or wc, its name is sh. And since it’s
a program , you can run it and redirect its input.
• So run the shell with its input coming from the file nu instead of the
terminal
– $ cat nu
Who|wc-l
$ sh <nu
4
$
• The output is the same as it would have been if you had typed who|wc-l
at the terminal.
 Creating New Command
• If a file is executable and if it contains text, then
the shell assumes it to be of shell commands.
Such file is called a shell file.
• All you have to do is to make nu executable.
$ chmod u+x nu
OR
$ chmod 744 nu
And thereafter you can invoke it with
$nu