UNIX Basics & vi Editor Training

Purnima Talukdar
5th January 2012
Agenda

Unix Overview

Unix Filesystem and Permissions

Unix Most Commonly-used Commands

 ls  mkdir  more  sort  ps
 cp  rmdir  wc  head  kill
 mv  cd  cut  tail
 cat  diff  df  find
 rm  date  du  grep

vi - Unix File Editor

2
Unix Operating System Structure

OS mediates between the user and the computer

User

Application Programs

Shell
Kernel

Hardware
3
Kernel

Manages memory and allocates it to each process

Schedules work done by the CPU

Organizes transfer of data from one part of machine to another

Accepts instructions from shell and carries them out

Enforces access permission on the file system

4
Shell

Command interpreter

Create customized environments

Write shell scripts

Define command aliases

Manipulate command history

File and command completion

Edit the command line

5
File System

Logical method for organizing and storing large amounts of

information.

Easy to manage.

File: basic storage unit.

Types:
 ordinary file (stores info)

 directory (holds other files and directories)

 special file (represents physical devices like printers, terminals, etc)

 pipe (temporary file for command linkage)

6
UNIX: Multi-user Multi-tasking

More than one user can run at the same time and more than
one task can run at the same time

Unix is multiuser multitasking.

In Unix, each program is started as a process.

A process is a program in execution.

Usually only one copy of a program, but there may be many

processes running the same program.

To each interactive user (window):

 only one process in foreground

 may have several processes in background

7
Unix Process
Login process
 check password with the uid

 execute .profile or .login (depends on default shell)

 display shell prompt

Shell process (command line interpreter)

Shell prompt ($, %)

8
Shell: Command Interpreter
Bourne Shell: the default shell (sh)
 original unix shell

 does not have interactive features of newer shells

 widely used for writing shell scripts

 standard on Unix System V

C Shell (csh): available on BSD Unix and most other systems

 with syntax similar to the C language

 with many enhancement over the Bourne shell.

Korn Shell (ksh): AT&T’s answer to C Shell

 combine features of Bourne and C shells

 very efficient

Other shells: bash

9
Day-to-Day Use
Command Function Meaning
cat Display a file conCATenate
cp Copies a file CoPy
mv Renames a file or moves it MoVe
rm Delete files ReMove
lpr Sends a file to a printer Line Printer
lp (Sys V)
ls Lists the contents of a directory LiSt
chmod Changes the “mode” of permissions Change MODe
pwd Shows the current directory Print WorkingDir
cd Change current directory Change Dir
mkdir Create a directory MaKe DIR
rmdir Delete a directory ReMove DIR
ps Shows the processes on the system Process Status
man Shows info. about commands Manual
df Shows file system status Disk File
du Shows the space allocation of files Disk Utilization
10
Command Function Meaning
grep Search for patterns in files Search
more Read file content one screen at a time File Read
Pg Read files page by page File Read
head Displaying first part of file CustomDisplay
tail Displaying last part of the file Custom Diaplay
print custom print command Print
compress Compressing a file Compress
uncompress uncompressing a file UnCompress
gzip Zipping a file ZiP
gunzip Unzipping a file UnZiP
tar create an archive, add or extract files Archive
tee read from standard input and write to
standard output and files
awk Extracting fields from a file
sed substitute command changes all
occurrences of the regular expression into a new value
11
Standard Command Format
command [options] [arguments]
wc [-c | -m | -C] [-lw] [file …]

– stuff in brackets is optional

– boldface words are literals (must be typed as is)

– italicized (or <> enclosed) words are args (replace appropriately)

Options modify how the command works

Example
 ls [-alRF…] file-list

12
On-line Documentation
For shell command, system programs, and library functions.

%man [n] <command>

e.g.

%man wait %man 1 wait

%man man %man 1 man

%man -k <keywords>

Man(ual) page format

Name

Synopsis

Description (options, defaults, detail desc., examples)

Files

See Also

Bugs
13
I/O Redirection

Redirection and Pipe

“>“ redirects standard output (screen) to a file

E.g. ls > dirlist

“<“ redirects standard input (keyboard) from a file

E.g. sort < infile > outfile

“|” pipe output of program 1 to input of program 2

E.g. who | wc

Or getit < in | check_it | process_it | format_id > out

“>>“ appends output to a file

E.g. ls -l >> oldfile

14
File Name Expansion

Each shell program recognizes a set of special characters called

meta characters.

The metacharacters are used to create patterns to match

filenames and command names.

Bourne and Korn shell meta/wildcard characters

 * matches any string (including null)

 ? matches any one character

 [a-dA-D] matches any one character in the range

 [!AB123] matches a char not in this range

 \ escape

~<username> : (not bourne shell) the home dir of the user.

15
File Name Expansion

Assume we have the following files under the current directory:

120, 235, 2#5, a.out, c22, c*2, Doc.1, Doc.2, Doc.3, one.c,two.c,
three.c

ls *.c ls c*2

ls [a-z]*[!0-9] a.* *.*

ls ??? cd ~foo

ls *

16
Filters

Most UNIX utilities are filters

A filter is a program which

 reads input (text) from standard input only

 writes output to standard output only

 writes error to standard error only

 may use temporary files for intermediate results

Filters can be combined to work together using pipes

Pipe: takes stdout of one command and uses it as stdin of

another command

ls | wc

17
Command Alias

Assign your own name for a command

Syntax is shell dependent

alias ll ‘ls -l’ C shell

alias ll=‘ls -l’ Korn, Bourne shell

Displaying the value of an alias

alias ll (displays value)

18
Unix File Systems

File: a collection of data

 Just a sequence of bytes

 no record or block structure required

Directory
 A file that contains information for files

 distinction between a directory and a file

– system can alter the contents of a directory

 rooted tree file structure (inverted tree)

– directories can contain both files and other directories info.

19
 Unix File System Road Map

/dev /etc /var /bin /tmp /usr /mnt /home

passwd
hosts spool adm include etc bin lib 5bin sue john
…... printer messages
mail(in) wtmp
mail(out) …
uucp

Special files: /dev/* represents I/O devices.

20
Symbolic Links

Use ln command for creating symbolic links

Can have many links to the same file

ln oldfile newfile

21
Permissions

Every file has a set of permissions associated with it

 three types of permissions: read ( r), write (w), and execute (x)

 three sets of permission: user, group, world.

In Unix system, users are identified by numbers:uid, gid

ls -l

-rwxr-xr-x 1 root 3743 Jan 4 1970 test

user group others #links owner size (time of last mod) (file name)

22
Chmod

Change the access permissions of a file

chmod <permissions> <filename>

 permissions can be specified as 3 octal digits, <user,group,others>, the
three bits of an octal means r,w,x

 rwx = 4+2+1=7
– Example: chmod 755 Load.sh

 permissions can also be specified as u+x or g+r or o+x …

– Example: chmod u+rwx, g+rx, o+rx Load.sh

If a directory has “x” in its permision, the dir is searchable, ie.,

one can do “ls” on the directory.

23
Pathnames

Every file and directory in the file system can be identified by a

“full path name” (route from root to file)
/home/sue/email/f1
/
Relative path name
 location relative to current directory home
. Current directory

sue
.. Parent directory

 if cwd is /home/sue: fred

ls email

ls ./email
docs email
cd ..

ls ../fred

f1 f2

24
Basic Commands ...
ls [-alrtRF…] <file-list>
 a for listing all files including the dot files

 l for long format; r for sort in reverse order; t for sort by time

 R for recrusive. list the all subdirectories.

 F for listing directories with a trailing /

cp [-ir…] <file1> <file2>

cp [-ir…] <file-list> <directory>

 i for interactive. prompt use whenever a file will be overwritten

 r for recursive. copy a directory tree

mv [-i] <file1> <file2>

 rename file1 to file2

mv [-i] <file> <dir>

move file to a different directory 25
Basic Commands ...
rm –i <file-list>
 remove the files from a directory
 use -i for y/n cofirmation for each file to be deleted
pwd
 display the present working directory
who; who am i
 display the current user information
rmdir <dir>
 remove a directory if empty
mkdir <dir>
 create a sub-directory
cd <dir> | ~
 Go to a specific directory
 Use ~ to go to home directory
cat file1 file2...
 display the contents of file1, file2.
diff file1 file2
26
 compare file1 with file2 and display the differences
Basic Commands ...
date [+format]
 %a – Day of week (Sun, Mon, ...)
 %b – Abbreivated month name (Jan, Feb,...)

 %d – Day of month (01 to 31)

 %H – Hour in 24-hour format (00 to 23)
 %m – Month of the year (01 to 12)
 %M – Minutes (00 to 59)
 %S – Seconds (00 to 59)
 %Y - Year
 Examples
– date ‘+%h %d, 20%y’

Oct 01, 2005

– date ‘+%Y%m%d_%H%M%S’

Guess?

27
Useful Commands
wc [-lwc] <file-list>
 Count the number of lines, words and charaters in a file

more <file-list>
 Browse through text files one page at a time.

head [-n …] <file-list>

 Display the first n lines of the files (default=10)

tail [+n|-n| -f| …] <file-list>

 Display the last few lines in the files (default = 10)

 Example:

%tail +5 result.dat # display the last parf of result.dat starting from line 5

%tail -5 result.dat # display the last five lines of result.dat

%tail -f result.dat # wait and display the new lines appended to result.dat

%tail +30 result.dat | head -15 | more #display line 30-45 of result.dat
28
Useful Commands ...(2)
cut -c <list> <file>
 Cut out selected charaters or fields from each line of a file

 Examples:

%cut -c 1-5,15-20 offer.dat

# extract chars 1-5 and 5-20 from each line of offer.dat.

cut -f <list> [-d<Char>] <file>

 %cut -f 1,3 -d“ ” offer.dat # extract field 1 and 3 from each line of offer.dat.

ps [-ef | -p <processid>| -u <userid> ]

 -e list all processes

 -f full listing

 -p display only the listed process ids

 -u display only the processes belonging to the userid

29
Useful Commands ...(3)
grep, egrep, fgrep

grep [-nlv...] pattern [file-list]

 Search the input text files for lines matcing the pattern and display the
matched lines

 -i ignores the case (case insensitive search)

 -h prints the matched lines without the filename

 -l prints only the filename without the matched lines

 -n also prints line number

 -v inverts the search; prints all the non-matching lines

%grep Unix doc.1 # Display the lines in doc.1 that contains “Unix”

%grep -n Unix doc.* # Display the lines with line numbers

%grep -v Unix doc.1 # Display the lines which do not contain “Unix”

30
Useful Commands ...(4)
sort [-rnuf…] [-o outfile] [<field-list>] [file-list]
 sort the files

 -r reverse order

 -n numeric value order

 -u unique; remove duplicates

 -f folds (treats) lowercase to uppercase while sorting

%sort +1 list1 # sort list 1 starting from field 2 to the end of the line

%sort +2-3 list2 # sort list2 on the third field

%sort -n -o list4 list3 sort list3 numerically and place the output in list4

31
Useful Commands ...(5)
find <dir-name> <exp>
 Recursively search the directory tree rooted at <pathname> and find all files
whose names satisfy <exp>

The following options are available under <exp>

 -name “<filename>”
 -mtime n | +n | -n
 -user <uname>
 -print
 -exec command {} \;
 Examples:

%find . -name \*.doc -print # list all files names ending with .doc

%find /etc/source -atime 2 -print

# print the names of the files under /etc/source whose lst access
time was 2 days ago.

%find . -name “[a-z]*” -exec rm {} \;

# remove the files under the curent directory whose names begin with a lower case
letter.

%find / \(-name a.out -o -name “*.o” \) -atime +7 -exec {} \;

# remove the object and binary executable files under the root 32
directory which have not be accessed more than 7 days.
Useful Commands ...(6)
df [-k] [path]
 displays the amount of free disk space in the filesystem

 -k reports total in kilobytes

du [-k] [path]
 displays the amount of disk used by a specific directory or its sub-
directories

 -k reports total in kilobytes

 -s displays only the summary total

kill [-signal] pid

 Sending the specified singal to the process.

 Example:
– %kill -9 1234 # kill process 1234

33
Useful Commands ...(7)

awk
 Most of the time awk is used to extract fields from a text line. The default
field separator is space. To specify a different one use the option -F.
 Example

cat file.txt | awk -F, '{print $1 "," $3 }'

Here we use the comma (,) as field separator and print the first and third
($1 $3) columns.

If file.txt has lines like:

Adam Bor, 34, India
Kerry Miller, 22, USA

then this will produce:

Adam Bor, India
Kerry Miller, USA

34
Useful Commands ...(8)

Sed
 Sed has several commands, but most people only learn the substitute
command: s. The substitute command changes all occurrences of the
regular expression into a new value.
 A simple example is changing "day" in the "old" file to "night" in the "new"
file:
sed s/day/night/ <old >new

s Substitute command
/../../ Delimiter
day Regular Expression Pattern Search Pattern
night Replacement string

35
Useful Commands ...(9)
gzip/gunzip

 Unix provides the command for zipping and unzipping a file

 The following examples illustrate typical uses of the command gzip for
compressing (or "packaging") a file into an "archive file".
gzip archivefile1.txt
This command compresses the file archivefile1.txt and replaces it with the
compressed version named "archivefile1.txt.gz".
If you want to keep the uncompressed version, you can use the -c option,
which writes tostandard out, and then re-direct (">") standard outto a file like
this:
gzip -c archivefile1.txt > archivefile1.txt.gz

36
Useful Commands ...(10)

tar.
 Use create compressed archives of directories and files, and also to extract
directories and files from an archive.
 Example:
$ tar -tvzf foo.tar.gz
displays the file names in the compressed archive foo.tar.gz while
$ tar -xvzf foo.tar.gz
extracts the files

37
vi Editor - Introduction
Text File Editor

Opening vi editor
 vi [+n] filename

Two Modes of vi editor

 Command mode (ESC key)

 Insert mode (a, i, A, I ...)

Saving a file
 :w saves the file without closing it

 :wq saves the file and close it

 :q! quit without saving any changes

 :w! newfile saves all lines from the entire current file into the file
'newfile', overwriting any existing newfile

38
vi Editor – Insert or Command mode
Insert mode
 i insert before the cursor position

 I insert at the beginning of the current line

 a insert after the cursor position

 A insert at the end of the current line

 o insert characters on a new line below the current line

 O insert characters on a new line above the current line

Command Mode
 Esc will take you to the command mode

vi commands format
 [count] command

39
vi Editor
Cursor movement
 h moves the cursor 1 space left
 j moves the cursor 1 space down
 k moves the cursor 1 space up
 l moves the cursor 1 space right
 ^ moves the cursor to the start of the line
 $ moves the cursor to the end of the line
 b beginning of previous word
 w beginning of next word
 e end of next word
 Control-B moves the cursor one screen backward
 Control-F moves cursor one screen forward
 G moves the cursor to the end of the file
 :n moves the cursor to beginning of line n

40
vi Editor
Basic commands – Deleting Text
 x delete single character; 5x deletes 5 characters

 dw delete word; 5dw deletes 5 words

 dd delete line; 5dd deletes ...

 cw delete word, leaves you in insert mode (i.e. change word)

 cc change line -- delete line and start insert mode

 s change character -- delete character and start insert mode

 D delete from cursor to end of line

 C change from cursor to end of line -- delete and start insert mode

 u undo last change

 U undo all changes to current line

 J join current line with line that follows (press Enter in insert
mode to split line)

41
vi Editor
Cut, Copy and Paste
 yy yank (i.e. copy) one line into a general buffer (5yy to yank 5
lines)

 p put the default buffer back after the current line

 P put the default buffer back before the current line

 "ayy yank into the buffer named a

 "aP put from buffer a before current line

 xp Guess?

 ddp Guess?

42
vi Editor
Search
 /string search down for string
 ?string search up for string

Search & Replace

 %s/string1/string2 search the first occurence of string1 on
each line and replace with string2
 %s/string1/string2/g search all occurences of string1 on
each line and replace with string2
 %s/string1/string2/gc search all occurences of string1 on
each line and replace with string2 with a
confirmation

set commands
 :set showmode show when you are in insert mode
 :set ic ignore case when searching
 :set noic turn ignore case off
 :set nu turn on line numbering
 :set nonu turn line numbering off
43
Thank You !!!

44