You are on page 1of 35

Date The only thing more powerful than a great idea, is a great idea powerfully executed.

- William Blake Presenter

© Copyright 2002 Wipro Technologies


Unix Overview

For WASE & Team Rainbow 1 Day Session

© Copyright 2002 Wipro Technologies

Operating Systems

 What is an Operating System?  By operating system, we mean the suite of programs which make the computer work. These are the set of programs which are loaded during the startup of a computer and runs until the system is switched off.  It is the operating system, which loads and runs other applications like MS-office suite or other such user applications. Examples of popular operating systems  Windows  Novell Netware  Unix  Single-user operating systems, like DOS, or the Macintosh System
Wipro Confidential


Operating Systems
Single User Operating Systems  Such operating systems allows only one user to use the computer at a time. MS-DOS, Windows-95 etc are such operating systems.  Such OS are primarily used in personnel computers. Multi-User / Multi-Tasking Operating Systems  Such operating systems allows multiple users to use the computer at the same time.  Each user can run multiple programs at the same time.  Such Operating Systems has various level of user access privileges. Certain users (super users) will have complete control over the system, where as other users will have only limited access rights.  Such restrictions are essential to control the system and to ensure availability to the system users.  Examples of such OS are: Unix, DB2, Windows-NT etc..
Wipro Confidential


Unix Operating System

UNIX was created in the late 1960s, in an effort to provide a multiuser, multitasking system for use by programmers. The philosophy behind the design of UNIX was to provide simple, yet powerful utilities that could be pieced together in a flexible manner to perform a wide variety of tasks. The UNIX operating system comprises of three parts:  The kernel  The standard utility programs  The system configuration files Unix is mainly written in C and Assembly-Language. There are various flavors available in the market.

Wipro Confidential


Unix flavors

Every Unix-box hardware vendor come out with their version of Unix.  These versions has all Unix System V features tuned to work with their hardware architecture.         Available Unix Flavors Sequent Dynix/Ptx Sun Solaris HP-UX Sco Unix Digital Unix IBM AIX Linux

Wipro Confidential


Unix Architecture overview

Wipro Confidential


Unix Architecture overview

 Typical computer system consists of:  Hardware  Operating system  Applications and utilities  Operating system controls the computer hardware and interface user with the hardware  Typical OS includes a user interface or command interpreter.  Unix is somewhat unique in that the command interpreter is not integrated with the OS but is a separate program. This enable vendors / developers to develop custom / better command interpreters for Unix.

Wipro Confidential


Files and processes
 Everything in UNIX is either a file or a process.  A process is an instance of executing program .  Each process is identified by a unique PID (process identifier).  Processes can spawn other processes.  A file is a collection of data. They are created by users using text editors, running compilers etc. Examples of files:  A document (report, essay etc.)  The text of a program written in some high-level programming language  Instructions comprehensible directly to the machine and incomprehensible to a casual user, for example, a collection of binary digits (an executable or binary file);  A directory, containing information about its contents, which may be a mixture of other directories (subdirectories) and ordinary files.
Wipro Confidential


The Directory Structure
All the stored information on a UNIX computer is kept in a filesystem. Any time you interact with the UNIX shell, the shell considers you to be located somewhere within a filesystem. Although it may seem strange to be "located" somewhere in a computer's filesystem, the concept is not so different from real life. After all, you can't just be, you have to be somewhere. The place in the filesystem tree where you are located is called the current working directory. CONCEPT: The UNIX filesystem is hierarchical (resembling a tree structure). The tree is anchored at a place called the root, designated by a slash "/". Every item in the UNIX filesystem tree is either a file, or a directory. A directory is like a file folder. A directory can contain files, and other directories. A directory contained within another is called the child of the other. A directory in the filesystem tree may have many children, but it can have only one parent. A file can hold information, but cannot contain other files, or directories.
Wipro Confidential


The Directory Structure

File 1 Sub-dir 1 File 5 Sub-dir 3 File 6 File 7 File 8 Sub-dir 2 File 8 Sub-dir 3 File 9 File 3 File 4 File 10

Working Directory Current directory where user is placed Root Directory Unix directory structure starting point Home Directory Default directory for the user Changing current working directories cd – Set current directory to home directory cd / - Set current directory to root cd .. – Set current directory one level up cd ../.. – Set current directory two levels up cd ../<dir> - Set current directory to sub directory a level up pwd – Shows the current directory

Wipro Confidential


Accessing a UNIX System

The main mode of access to a UNIX machine is through a terminal, which usually includes a keyboard, and a video monitor. For each terminal connected to the UNIX system, the kernel runs a process called a tty that accepts input from the terminal, and sends output to the terminal. tty processes are general programs, and must be told the capabilities of the terminal in order to correctly read from, and write to, the terminal. Initially Unix supported only character terminals. However, later xWindows got evolved to provide a graphical user interface. There are terminal programs available in Windows, which mimic a character terminal, and through such terminal programs, a Unix system can be accessed from Windows.

Wipro Confidential


Logging In and Logging Out

 To access a Unix system, one has to start a user session from a terminal.  A user session is started by providing a valid Unix user id and password.  Upon providing the right user id and password, Unix starts a user session and a command interpreter is spawned.  Command interpreter waits for the user to enter a command in its prompt. Execute the command and waits for the next command.  To end the session, a command “exit” is entered in the command interpreter’s prompt. This will make the Unix to clean-up the user session, end all user processes and remove the user session from the system.

Wipro Confidential


Space Management / mount points
 A file system is created on a hard disk and same can be mounted on the Unix system.
 File System creation is the process of readying a portion of hard disk for Unix to keep its files.  Mounting is a process by which the file system is made available to the users.

 Once mounted, the file system is available under the directory structure as a directory.  There are commands to describe the mounted files systems / mount points. “df” is such a command available in Unix.

Filesystem kbytes used avail capacity Mounted on /dev/md/dsk/d10 6050182 5701570 288111 96% / /dev/dsk/c0t0d0s5 3009327 2852761 96380 97% /u01

Wipro Confidential


Backup & Recovery
 Backup
 Backup is the process in which data stored in a computer is copied to an alternative storage medium (e.g Tapes, CDs etc..)  Hot & Cold Backup
 Backups taken during the normal system operations are called hot backups. Here the files could be get modified while backing them up.  Cold backups are taken while system is not performing any file operations. All cold backups will be stable and are independent backups.

 Recovery
 Process by which system’s data is restored from backup device after a system/media crash.

 There are vendors who offer backup & recovery mechanism (software & hardware).

Wipro Confidential


UNIX Shell

A Unix shell is the command interpreter, the primary user interface to UNIX which interpret and execute commands There are number of shells available.
     /bin/sh - Bourne shell /bin/csh - C shell /bin/ksh – K shell /bin/bash - Bourne again shell /bin/tcsh - modified, updated C shell

A Shell can be “tuned” for each user by keeping the initialization commands, scripts in the shell startup file. Shell scripting allows users to write procedural programs using shell scripting language. These scripts can be executed within the shell. Each Unix command consists of a command name, followed by command options (if any are desired) and command arguments (if any are desired). The command name, options, and arguments, are separated by blank space. e.g: cd /

Wipro Confidential


Main memory / Shared memory
Kinds of Memory:
 Main - The physical Random Access Memory located on the CPU motherboard that most people think of when they talk about RAM. Also called Real Memory. This does not include processor caches, video memory, or other peripheral memory.  File System - Disk memory accessible via pathnames. This does not include raw devices, tape drives, swap space, or other storage not addressable via normal pathnames. It does include all network file systems.  Swap Space - Disk memory used to hold data that is not in Real or File System memory. Swap space is most efficient when it is on a separate disk or partition, but sometimes it is just a large file in the File System

Shared Memory
 Part of Main memory which is shared between processes across unix users. Typically each process get part of main memory as a private area, where only that process operates. Shared memory is the place where multiple processes has access to.

Wipro Confidential


Unix Processes
 Foreground / Interactive Processes
 Processes that are spawned from shell command prompt. Shell waits for the process to complete before returning the command prompt. Command / Process output will be written to user’s terminal.

 Background processes
 Processes that are run non-interactively. Such processes are not associated with any terminal. Kernel processes are examples of background processes. A user can submit such processes from command prompt by appending the command with “&” sign.

 Input / Output
 Each process has the following devices attached to it.
 Standard input – Device to get the input data. Normally the keyboard.  Standard output – Device to output data from the process. Normally the terminal.  Standard error – Device to output error messages. Normally the terminal.

Wipro Confidential


User Management
 User
 Unix users created to access the system. A password is always attached to each user id. The right combination of user id and password allows one to access the Unix system and create a Unix session.  Various level of privileges can be set at the user level.

 Groups
 Unix administrator can create Groups where one or more Unix users can made members.  Various level of privileges can be set to group, which will be applicable to all group members.  One unix user can be made members of multiple groups.

Wipro Confidential


File / Directory Permissions
 Each file / directory in Unix has access permissions associated to it.
 Read – Whether the file/directory can be read?  Write – Whether the file/directory can be modified? In case of directories, this means whether files can be created/modified under this directory.  Execute – Whether the file can be executed. Applicable for shell scripts and other executable programs.  Read / Write / Execute permissions are denoted by “rwx” flags.  For each file/directory, “r,w,x” permissions are attached to User, Group and Others level.
 User – Owner or an Administror  Group – Unix group(s) where the user is a member of  Others – Users who are not members of group(s) where the “User” is member of.

Wipro Confidential


Popular Unix Commands
 Help on Unix commands are available by typing the following command at the command prompt
 man <command Name>
User Commands NAME man - find and display reference manual pages SYNOPSIS man [ - ] [ -adFlrt ] [ -M path ] [ -T macro-package ] [-s section ] name ... man [ -M path ] -k keyword ... man [ -M path ] -f file ... DESCRIPTION The man command displays information from the reference manuals. It displays complete manual pages that you select by name, or one-line summaries selected either by keyword (-k), or by the name of an associated file (-f). If no manual page is located, man prints an error message. Source Format Reference Manual pages are marked up with either nroff(1) or sgml(5) (Standard Generalized Markup Language) tags. The man command recognizes the type of markup and processes the
Wipro Confidential



Popular Unix Commands - Navigation
 “cd” – Change working directories  “ls” – List directory contents  “pwd” – Show current working directory
$ $ pwd /u14/home/gie $ $ cd ../.. $ pwd /u14 $ ls GIE_DATA_CORR.fmb home TT_DB lost+found $ ls -l total 992 -rw-r--r-1 gie users drwxr-xr-x 2 root root drwxrwxrwx 10 root other drwx-----2 root root drwxr-xr-x 6 app1159 dba drwxr-xr-x 2 ora1159 dba drwxr-xr-x 204 app1159 dba drwxr-xr-x 9 app1159 dba $ $ ls -CF GIE_DATA_CORR.fmb home/ TT_DB/ lost+found/ $

patchsso testdata 483328 512 512 8192 512 512 3072 512 Jan Jan Dec Apr Oct Oct Jan Sep 7 23 1 16 15 7 19 3 17:20 20:10 14:57 2002 13:43 18:45 10:08 23:29

visappl viscomn GIE_DATA_CORR.fmb TT_DB home lost+found patchsso testdata visappl viscomn visappl/ viscomn/

patchsso/ testdata/

Wipro Confidential


Popular Unix Commands – File Operations
       “cp” – Copy files “mv” – Move / Rename files “rm” – Remove files “touch” – Create file with zero bytes or mark files modified “chmod” – Change file permissions “chown” – Set the owner for the file “chgrp” – Set the group for the file

$ touch TeamRainbow $ ls -l TeamRainbow -rw-r--r-1 gie users $ mv TeamRainbow Batch100 $ ls -l Batch100 -rw-r--r-1 gie users $ chmod g+w Batch100 $ ls -l Batch100 -rw-rw-r-1 gie users $ $ su Password: # chgrp other Batch100 # ls -l Batch100 -rw-rw-r-1 gie other # # chown root Batch100 # ls -l Batch100 -rw-rw-r-1 root other # # rm Batch100 # # ls -l Batch100 Batch100: No such file or directory #

0 Jan 26 14:45 TeamRainbow 0 Jan 26 14:45 Batch100 0 Jan 26 14:45 Batch100

0 Jan 26 14:45 Batch100

0 Jan 26 14:45 Batch100

Wipro Confidential


Popular Unix Commands – File Operations
    “mkdir” – Create Directory “rmdir” – Remove directories “cat” – Output file contents “pg” / “more” – Output file contents page by page

$ ls -l total 0 -rw-r--r-1 rainbow $ mkdir Mydir $ ls -l total 2 drwxr-xr-x 2 rainbow -rw-r--r-1 rainbow $ cd Mydir $ ls -l total 0 $ cd .. $ pwd /u02/home/rainbow $ rmdir Mydir $ ls -l total 0 -rw-r--r-1 rainbow $ $ cat test This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 $


0 Jan 26 14:52 test

rainbow rainbow

512 Jan 26 15:03 Mydir 0 Jan 26 14:52 test


0 Jan 26 14:52 test

Wipro Confidential


Popular Unix Commands
       “clear” – Clear screen “date” – Displays system date “echo” – Prints the argument values to screen “file” – Shows the type of file “passwd” – Command to change the user password “uid” – Shows the current user & group details “ps” – Shows the current processes running

$ date Mon Jan 26 15:14:18 IST 2004 $ $ echo This is a test This is a test $ echo $LOGNAME rainbow $ file * test: English text $ file /bin/at /bin/at: ELF 32-bit MSB executable SPARC Version 1, dynamically linked, stripped $ id uid=1099(rainbow) gid=1005(rainbow) $ $ passwd rainbow Enter login password: New password: Re-enter new password: passwd (SYSTEM): passwd successfully changed for rainbow $ ps PID TTY TIME CMD 11363 pts/7 0:00 ksh 10540 pts/7 0:00 sh $ ps -fu gie UID PID PPID C STIME TTY TIME CMD gie 9235 9233 0 14:30:08 pts/7 0:00 -ksh $ ps -l F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD 8 S 1099 11363 10540 0 51 20 627e21f8 207 627e2268 pts/7 0:00 ksh 8 S 1099 10540 9235 0 51 20 6344b668 121 6344b6d8 pts/7 0:00 sh $

Wipro Confidential


Popular Unix Commands
   “cal” – Calendar program “cut” – Shows file contents after vertical filtration “grep” – Shows file contents after horizontal (lines) filtration

$ cal January 2004 S M Tu W Th F 1 2 4 5 6 7 8 9 11 12 13 14 15 16 18 19 20 21 22 23 25 26 27 28 29 30

S 3 10 17 24 31

$ $ cat test This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 $ $ cut -c1-10 test This is li This is li This is li This is li This is li $ $ cut -d' ' -f3 test line line line line Line $ $ grep [35] test This is line 3 This is line 5 $ Wipro Confidential


Popular Unix Commands
   “head” – Shows the initial part of the files “tail” – Shows the trailing part of the file “sort” – Shows the file contents after sorting

$ cat test This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 $ $ head -2 test This is line 1 This is line 2 $ $ head -4 test This is line 1 This is line 2 This is line 3 This is line 4 $ $ tail -3 test This is line 3 This is line 4 This is line 5 $ $ cat test This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 $ $ sort -r test This is line 5 This is line 4 This is line 3 This is line 2 This is line 1 $ Wipro Confidential


Popular Unix Commands
 “diff” – Shows the difference in file contents  “find” – Command to search for file(s) in a directory structure

$ cat test This is line 1 This is line 2 This is line 3 This is line 4 This is line 5 $ $ cat test1 This is line 1 This is line 2 This is line 3 $ $ diff test test1 4,5d3 < This is line 4 < This is line 5 $ $ cd .. $ pwd /u02/home $ $ find . -name 'test' -print ./rainbow/test ./abshah/test find: cannot read dir ./easdba/bin: Permission denied $

Wipro Confidential


Pipe - Feature
Unix provides a powerful feature in which output from a command can be directed as input for another command. This way, commands can be joined to process data in a very efficient manner. Example: cat <file> | sort Here, output of the “cat” command (file contents) are fed to command “sort”. The final output of this command will be sorted file contents $ cat test John received 30 marks Peter received 25 marks Abraham received 15 marks $ $ cat test | cut -d' ' -f1,3 | sort Abraham 15 John 30 Peter 25 $ $ cat test | cut -d' ' -f1,3 | sort +1 Abraham 15 Peter 25 John 30 $

Wipro Confidential


vi – Unix Text Editor
“vi” – is the most popular text editor available in Unix. It operates in two modes, command & input mode.
 In the command mode, vi accepts commands.  In the input mode, whatever user types are taken as file contents.  While in input mode, pressing “Esc” will change the mode into command mode.

Popular “vi” commands
                 “i” – Goes into the input mode. Start insertion from the current cursor position. “a” – Goes into the input mode. Start insertion from the current cursor position + 1 (append) “A” – Start insertion from end of line “c” – Change mode. Start replacing from the current cursor position. “dd” – Deletes the current line “:w” – Perform a save operations “:wq” – Save and Exit “:q!” – Discard the changes and exit. “:w <file>” – Save into a different file name. “dw” – Delete word “d$” – Delete till end of line “/<string>” – Search for <string> “n” – Search for the next occurrence (find next) “N” – Search backwards (find previous) “:e <file>” – Edit the file name <file> “dG” – Deletes till end of file “u” – Undo the last command

Wipro Confidential

Shell Variables
Unix allows you to define variables within a Shell. Unlike other programming languages, Shell variables do not have any types. All values are stored as character strings. There are many variables defined in a user’s environment.
$ env _=/usr/bin/env HZ=100 PATH=/usr/bin::/usr/local/bin::/usr/opt/SUNWmd/sbin LOGNAME=rainbow SHELL=/bin/sh HOME=/u02/home/rainbow TERM=ansi PWD=/u02/home/rainbow TZ=IST $ $ TESTVAR="Hello World" $ $ echo $TESTVAR Hello World $ $ ksh $ $ echo $TESTVAR $ exit $ export TESTVAR $ $ ksh $ echo $TESTVAR Hello World $ exit $

Wipro Confidential


Introduction to Shell Scripting
Shell script is a script file which contains commands and other control statements. A shell script can be executed within a Unix shell. Unix shell will execute each line in the shell script like any other interpreter.
$ vi myshell.ksh ls -l date echo "Log Name is : " $LOGNAME ~ ~ ~ ~ ~ ~ ~ ~ ~ "myshell.ksh" [New file] 4 lines, 43 characters $ ls -l myshell* -rw-r--r-1 rainbow rainbow 43 Jan 26 16:31 myshell.ksh $ chmod +x myshell* $ ls -l myshell.ksh -rwxr-xr-x 1 rainbow rainbow 43 Jan 26 16:31 myshell.ksh $ myshell.ksh total 6 -rwxr-xr-x 1 rainbow rainbow 43 Jan 26 16:31 myshell.ksh -rw-r--r-1 rainbow rainbow 74 Jan 26 15:51 test -rw-r--r-1 rainbow rainbow 45 Jan 26 15:40 test1 Mon Jan 26 16:31:42 IST 2004 Log Name is : rainbow $
Wipro Confidential


Introduction to Shell Scripting
Shell also provides control/procedural statements for Shell Scripting.
    for identifier [ in word ... ] ; do list ; done select identifier [ in word ... ] ; do list ; done case word in [ pattern [ | pattern ] ) list ;; ] ... Esac if list ; then list ;
[ elif list ; then list ; ... ] [ else list ; ] fi

 while list ; do list ; done  until list ; do list ; done

Do a “man ksh” at the command prompt and go through the “ksh” user manual to understand the full capabilities of ksh and also shell scripting features.

Wipro Confidential


Class room Assignment

 Go through the “man” pages of the commands listed in the previous slides.  Practice them with different command options.  Write a shell script to satisfy the following requirement.
 File “mydata.txt” has the following content (create this file using vi):
Bill 10 Joe 243 Kirthi 54

 Write a Shell script to get an output as follows (lines sorted on marks):
Bill Received 10 marks. Kirthi Received 54 marks. Joe Received 243 marks.

Wipro Confidential


Thank You