Professional Documents
Culture Documents
net/publication/312970940
CITATIONS READS
0 1,768
1 author:
Boualem Merabet
University Mustapha Stambouli of Mascara
62 PUBLICATIONS 143 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Boualem Merabet on 27 January 2017.
1
important component: a working UNIX kernel. The GNU project had started work on an ambitious kernel
design, known as the GNU/HURD, based on the Mach microkernel.
In 1991, Linus Torvalds, a Finnish student (University of Helsinki) was inspired to write an opera-
ting system for his Intel 80386 PC. In his studies' course, He had come into contact with Minix, a small
UNIX-like operating system kernel developed in the mid-1980s by A. Tanenbaum, a university Prof. in
Holland.
The earliest distributions of Linux appeared in 1992, and included Manchester Computing Centre
(UK) Interim Linux, Texas A&M University, and Soft Landing Linux System.
In 1993, 'Slackware' the oldest surviving commercial distribution appeared; the noncommercial
Debian distribution appeared at ~ the same time, and SUSE and Red Hat soon followed.
In 2004, the currently very popular Ubuntu distribution first appeared.
Nowadays, many distribution companies also employ programmers who actively contribute to
existing free software projects or initiate new projects.
In 2008, a less wide-ranging revision of the standard was completed, yielding the combined POSIX
1003.1-2008 and SUSv4 standard. Unlike most commercial UNIX implementations, Linux separates
implementation from distribution: no single “official” Linux distribution. Each Linux distributor’s offering
consists of a snapshot of the current stable kernel, with various patches applied. LSB develops and
promotes a set of standards for Linux systems to ensure binary application compatibility across Linux
distributions, so that compiled applications should be able to run on any LSB-conformant system running
on the same hardware.
2
Now we have our Terminal open and it's in our dock so we can get to it.
* Look through the menu for our Win manager for anything named "Shell" or "Terminal."
On Win we're going to use PowerShell; people used to work with a program called cmd.exe, but it's not
nearly as usable as PowerShell.
*If we have Win7, Click Start. Type: powershell in "Search programs and files". Hit Enter. *If we don't
have Win7, we should seriously consider upgrading, or try installing it from Microsoft's download center.
Search online 'powershell downloads' for our version of Win.
A lot of documentation is installed on the machine itself: we can access this by moving the mouse towards
Ubuntu Desktop at the top left of the screen and clicking on the help menu that appears. the name of the
program we are currently using is displayed at the top-left of the screen and moving the mouse to top of
the screen will reveal the programs menus in a similar fashion to how they are displayed on the Mac.
*To get help, click on the circular symbol (a stylised picture of three people holding hands) at the top left
of the screen and type help in the search box that appears. For better names, refer to the people-holding-
hands button as the Ubuntu.
* how to get an open terminal? 'Bash' is the most important thing we must learn on Linux.
*'zsh' will give us 30 more IQ points and win us millions in the stock market: Right! but, our goal is to
get capable enough and at this level it doesn't matter which shell we use.
"rm -rf /" is a classic command that we must never type. If we need help, make sure we get it from
someone we trust and not from random idiots on the internet.
*
"prompt" is the command-line showing that the computer is ready to accept commands. prompt is text of
the form user@computer:directory$ : tim in the directory ~ on a computer called 'coffee-grinder'.
"prompt" is configurable and may vary between computers.
4
* pwd does not agree with the command prompt, instead printing /home/merabet. This is because ~ is a
synonym for /home/merabet
Basical Unix Commands
* whoami Our username
* uname Operating system (uname -a for the full details).
* pwd : print working directory
* hostname : my computer's network name (Name of machine being used)
* mkdir : make directory
* cd : change directory
* ls : list directory
* rmdir : remove directoryenv
* pushd : push directory
* popd : pop directory
* cp : copy a file or directory
* mv : move a file or directory
* less : page through a file
* cat : print the whole file
* xargs : execute arguments
* find : find files
* grep : find things inside files
* man : read a manual page
* apropos : find what man page is appropriate
* env : look at our environment
* echo : print some arguments
* export : export/set a new environment variable
* exit : exit the shellDANGER! become DANGER!
* sudo : DANGER! become super user root DANGER! pwd
Drill! until we can say such phrases right away when we see that word, & so that we read the phrase and
know what command will do that; we're building our vocabulary by doing this, but don't spend so much
time we go nuts and get bored.
5
*Where Unix differs from other operating systems is that the filesystem is used much more for organising
different types of files: essential system programs are all in /bin and their shared code (libraries) are in
/lib; similarly user programs are in /usr/bin, with libraries in /usr/lib & manual pages in /usr/share/man.
* How to specify the file or directory location ?
– The absolute path and the relative path from where we currently are (current working directory : pwd)
in the filesystem. The 1st path is one that starts at the root and does not depend on the location of the
current working directory. Starting with a / to signify the root, the path describes all the directories
(branches) we must traverse to get to the file, each directory name separated by a /. For
ex.,/home/user/Music/TheKinks/SunnyAfternoon.mp3
* the paths of web pages are also separated by forward-slashes, revealing their Unix origins as a path to a
file on a remote machine. A few directories have special symbols that are synonyms for them and the
most common are as listed below :
/ root directory (for going to top of pyramid)
. current directory
.. parent directory
~ home directory (a synonym for $home
* A file in the current directory can be referred to by its name, a file in a directory inside our working
directory can be referred to by directory/filename (and so on for files inside of directories inside of
directories inside of our working directory, etc...).
* Such paths are very similar to how we describe absolute paths except that they do not start with /
* Absolute paths are relative paths relative to the root; Initial / could be read as “goto root”
6
* The directory above the current one can be referred to as .. so, if the working directory is /home/user,
then the root directory can be referred to as ../.. (go up one directory, then go up another directory).
* The symbol .. can be freely mixed into paths: the directory 'examples' below the current directory could
have path 'examples/../examples/../examples' (recommended : simply using just 'examples').
7
for our version of Windows. We are on our own, though, since we don't have Windows XP, but hopefully
the PowerShell experience is the same.
* Now, let's practice!
whoiam
merabet
pwd
/home/merabet
ls
Desktop Downloads Pictures Templates Video
Documents examples.desktop Quran Public Unbutu One
# Change to a different directory - don't press Enter yet
cd D
# Pressing tab once has no effect since there are three possible options.
# Pressing tab again lists the three options, note that cd D remains on the
# command-line for further editing.
cd D <tab><tab>
Desktop/ Documents/ Downloads/
# Press e to disambiguate options, and tab again complete.
cd De <tab>
# Gives
cd Desktop/
* 'history' is a command used to list commands we have entered (record kept): 'Ctrl-e' starts a search;
typing 'enter' accepts the current line, typing 'Ctrl-r' again goes to the next match and 'Ctrl-g' cancels
search. History can also be referred to by entry number.
*Many commands for manipulating files are used; these have short names, two or three let-ters; errors
typing can easily have unintended & severe consequences – be careful what we write because Unix rarely
gives we a 2nd chance to correct mistakes (some Unix machines have the sl command to encourage
accurate typing). Most useful commands are in details:
– change the working directory :
cd path # Change working directory to path
cd # Change to home directory
cd - # Change to previous directory
– copy a file from place to another :
cp file1 file2 # copy file1 to file2
cp file1 directory/ # copy file1 into directory. The copy of the file has path directory/file1
cp file1 file2 directory/ # copy file1 and file2 to directory. When copying multiple files, the destination
must be a path to a directory.
– LiSt contents of directory :
ls # List files
ls -a # Also show hidden files (those whose name begins with a period).
8
ls -l # Show more information about each file (permissions, owner, group, time and date of last
modification).
– MaKe (create) a new DIRectory
mkdir path # Make directory described by path
mkdir -p directory1/directory2 # Make the directory described and all directories leading to it (its Parents)
if necessary.
– MoVe (rename) a file: exactly like cp except that the file is moved rather than copied.
mv file1 file2 # Rename file1 to file2
mv file1 directory/ # Move file to directory
mv file1 file2 directory/ # Move files to directory.
– ReMove (delete) a file. Deletion on Unix machines is permanent and instantaneous; no trash
directory to save we from our mistakes exist.
rm file # Remove file
rm directory # Fails, can't remove directories
rm -r directory # Recursively descend into directory and delete everything, including other directories
inside of it (hence the recursively). This will remove the directory
rm -f file # Force (ignore warnings) file's removal; ignoring warning includes read-only files.
– ReMove a DIRectory: Only empty directories can be removed (hidden files included).
rmdir directory
# How to remove files & directories?
pwd
/home/merabet/unix/MoveCopy
# Some files are not read-only (answer "y")
rm read-only
rm: remove write-protected regular empty file 'read-only'? y
# Some files are not ours
rm /dev/null
rm: remove write-protected regular file 'dev/null'? y
rm: cannot remove 'dev/null' : Permission denied
# rm will not delete directories
rm test
rm: cannot remove 'test' : Is a directory
# rmdir will only remove empty directories
rmdir test
rmdir: 'test': Directory not empty
ls test
english
rm test/english
rmdir test
ls tests
9
english subdirectory
# rm -r (-r =“recursive”) deletes everything, including subdirectories
# danger - a mistake using this option can result in a lot of work being
# accidentally deleted
rm -r tests
# How to Escape examples
pwd
/home/merabet/examples/unix/Escaping
ls
my sequences.fasta sequence_directory
# Incorrect version. Space in name treated as command-line argument separator.
# “my” and “sequences.fasta” treated as separate files.
mv my sequences.fasta sequence_directory
mv: cannot stat `my': No such file or directory
# The wrong file has been moved
ls sequence_directory
sequences.fasta
# Correct version, the space is escaped to remove its special meaning.
mv my\ sequences.fasta sequence_directory
ls sequence_directory
my sequences.fasta sequences.fasta
# If we use “tab completion” to complete names, spaces and other characters
# are automatically escaped for us.
* Hidden files are those important for computers or programs, containing configuration information not
intended for users; files that begin with . characters are by default hidden & won't appear in the ls output.
pwd
/home/merabet
# Show ordinary files
ls
doc-samples examples
# Show all files. Note the special directories . and .. are visible.
ls -a
. .bash_history .bashrc .directory .mpd.conf examples
.. .bash_logout .config .kde4 .ssh
.Xauthority .bash_profile .dbus .local doc-samples
# Show only hidden files. For more details, see Dealing with multiple files.
ls -d .*
. .Xauthority .bash_logout .bashrc .dbus .kde4 .mpd.conf
.. .bash_history .bash_profile .config .directory .local .ssh
10
write a file. 3 among these are basic: read, write & execute. 3 categories of user exist : owner u, group, &
other; permissions of each file are described as a string of 9 characters, 3 for each user category. Each
category has a triplet which is either a letter 'rwx' if a user in that category has the corresponding
permission or '-' if he don't. rwxr-x--- is a permission string meaning that owners have permission to read,
write or execute, users in the same group have read & execute permission, but other users have no
permissions.
* Let's practice!
pwd
/home/merabet
id
uid=512 (merabet) gid=100(users) groups=6(disk),7(lp),11(floppy), ↩
17(console),27(video),80(cdrw),100(users),512 (merabet),1002(boinc) ↩
,65533(nogroup),65534(nobody)
# User merabet has ID 512, in group users by default (Group ID 100).
# 'd' as an initial means that both 'doc-samples' and 'examples' are directories. The files' owner
is merabet and they are part of the 'users' group.
mkdir test
ls -l
drwxr-xr-x 14 merabet users 352 Mar 22 14:02 doc-samples
drwxr-xr-x 3 merabet users 72 Mar 22 17:42 examples
drwxr-xr-x 2 merabet users 48 Mar 26 14:09 tests
# merabet has read, write and execute permission for both directories; users in the
# group users have read and execute permission, as do any other users.
# Remove execute permission for all users (a means u, g and o)
chmod a-x test
# Now we can't open the directory.
cd test
-bash: cd: test: Permission denied
# Give ourselves permission to enter directory.
chmod u+x test
# This now succeeds but nobody else will be able to enter
cd test
# Make all files in and below a directory read-only. The -R flag means
# recursively
chmod -R a-w test
Remark : As file owners, we can change its permissions to be anything; some programs do automatically
this for us, giving impression that permission has been ignored; by default
rm will prompt to remove write-protected files, but the -f (force) flag turns tells it not to bother asking &
just remove the file; running rm -f is possibly the only time we might run into this behavior.
11
How about multiple files?
It's desirable, especially when running scripts or organizing files, to deal with multiple files at once;
instead of a filename, we give the computer a pattern (just a filename containing symbols that have a
special meaning) rather than typing each file name out explicitly; any filename is checked against the
pattern & it's automatically replaced by a list of matching files before running a command; for ex, * & a*b
match respectively anything & any filename (pattern) beginning with a & ending with b.
The following special symbols (useful for constructing patterns) match......:
* anything, including match “nothing”
\* a '*' character (any symbol here can be prevented from having a special meaning by
“escaping” it with a '\').
? any character exactly once (excludes matching “nothing”)
[abc] exactly one of 'a', 'b' or 'c'
[^abc] or [!abc] any character but 'a', 'b' or 'c'
[c-y] any character between 'c' and 'y'. [-a] is defined to mean a match with '-' or 'a'.
{pattern1,pattern2} combines pattern1 & pattern2 results together; a file matched by both patterns returned twice.
Before a command is run pattern matching occurs; the command never sees patterns, just match results; it
can have unintended consequences, which can be dangerous! Look at this ex.:
pwd
/tmp
Create a file called -rf *
touch -- -rf\ \*
* It will very bad to type rm -rf *, since * matches everything, r means recursively descend down all
directories matched, & f forces deletion even if the file is important or write-protected; other ways exist to
safely delete a file :
rm -- '-rf *': -rf * stop the filename being interpreted as a pattern; -- prevents rm (& most commands) from
interpreting anything after it as a flag, so -rf is just a name not the recursive & force flags.
12
kill %n Kill job number n
killall name Kill all processes called name
ps Show all running processes (distinct from jobs) regardless of how they were started
fg %n Bring job number n to foreground
bg %n Run job number n in background
program & Start program in background
nohup program Run program in background so it will not stop if we log out. stdout & stderr are redirected
to nohup.out file; advanced user might like looking at screen instead.
*The difference between 'jobs' & 'processes' is that a process is a single program running on the machine;
all processes we are running, including the command-line itself, can be listed using ps (or ps -a if we want
to see what all the other users of the machine are doing; a command-line is another process running on a
computer); while a process is the fundamental method of keeping track of what is running on a computer,
a job is things entered on a command-line; many jobs include more than 1 program logically connected
together by pipes to achieve a task, & a command-line splits jobs into several processes & runs them,
possibly simultaneously.
* Unix commands (where possible) written as filters, read from inputs, manipulate data & write outputs.
This enables simple commands to be combined to produce complex results. * All command read from
stdin and write to stdout (by default connected to the current command-line, thus results are displayed on
the screen) and can be redirected; '> filename' redirects stdout to the file specified for later perusal.
* A pipe, rather than redirecting to a file, can be used to connect stdout of 1 command to the stdin of
another, by chaining many simple commands together, thus complex transformations of the input can be
achieved. Let's take ex. below, showing how complex outputs could be achieved using a smaller steps' set :
pwd
/home/merabet/examples/unix/Pipes
cat transcripts.file.gz | gunzip |
grep '^>' |
cut -d '|' -f 1 |
sort | uniq -c > transcript_counts.txt
cat transcripts.file.gz | gunzip | grep '^>' | cut -d '|' -f 1 | ↩
sort | uniq -c > transcript_counts.txt
Before typing commands, comments are provided successively, as follow :
At the command-line, a pipe is represented by the character |. How many transcripts there are for each
gene? first cat reads the file and writes it to stdout, pipe into the decompression program gunzip. Another
pipe into grep, a tool that extracts lines matching a certain pattern (here, those starting with a >).Sequence
names in the file file are of the form >gene_ID|transcript_ID; cut splits a line to bits; use | as delimiter &
we want the first field. Single quotes prevent the '|' being interpreted as a pipe by the command-line. Sort
& count the unique entries (uniq requires sorted input) Output to file transcript-counts.txt. Or on one
line, redirecting stdin in a similar way to how we redirected stdout before. < transcripts.file.gz connects
stdin of gunzip to transcripts.file.gz file rather than using cat to read it and output it into a pipe.
13
Unix basic administration commands [Networking Unix commands–for Telecoms] : ping, ifconfig, ifup,
ifdown, netstat, rlogin, ssh; ping is used a lot to see if a machine in our network is up/down,
In the ex. above ping 192.168.0.1 # 'machine 192.168.0.1' is "up", it sends packets
14
ping -c4 192.168.0.1 -q #don't show what's happening, just tell me the result at the end.
result: PING 192.168.0.1 (192.168.0.1) 56(84) of data, thus what happens in between?
ping c4 192.168.0.1 -v
ifup lo # wireless line interface could be configured, but local loopback address in our offices is not.
We need superuer/password & type ifconfig again, it show us that local loopback is 'up'
Then, let's ping the local loopback address :
This file contains definitions of our interfaces; now, exit the superuer account, since it's not needed/clear.
Let's now assign an IP address in the netmask to our LAN or Ethernet interfaces, then do ifconfig
# It brings up the configuration informations
16
ifconfig followed by the interface, in our case "eth0"& provide an IP address to assign, then the netmask :
It needs the superuer account, & type the same thing again : ifconfig...
It shows us for 'eth0' the internet & netmask addresses does how we allocate the IP address netmak to our
machine (using command-line). exit superuer/clear. netstat (networks connections open on our machine)
17
But, it's replaced by the 'ssh' command (which gives us the same thing) :
18
rm *.file
# 'x' means extract
tar -xf sequences.tar
ls
chimp.file human.file macaque.file orangutan.file sequences.tar
tar have increased to make it more convenient
ls
chimp.file human.file macaque.file orangutan.faile sequences.tar
# Pack and gzip sequences simultaneously
tar -zcf sequences.tgz *.file
# List the contents without extracting
tar -ztf sequences.tgz
chimp.file
human.file
macaque.file
orangutan.file
# More recent versions of tar can also bzip2 files
tar -jcf sequences.tbz2 *.file
tar -jtf sequences.tbz2
chimp.file
human.file
macaque.file
orangutan.file
To summarize : Compressing, Uncompressing, Suffix, & Tar'd Suffix, 4 items that are (in 2 ex.) as follow :
gzip gunzip .gz .tgz; bzip2 bunzip2 .bz2 .tbz2
To edit the root repertory 'sbin' [all files strarting with 'if'], type cd /sbin
whitch
20
Let's go back to'home' by typing 'cd ~/'. ifconfig --help gives us the help about that command; let's type it
Note nothing helps than the 'menu pages' shortly well. 'man ifconfig' gives us any information we need.
21
'man man' gives us details of Linux manual pages.
We can get the 'man' of the 'man' itself by typing man man
22
It gives us more details on Linux manual pages :
Before configuring networks that widely use Linux, one should have an idea on the
Networking road map
* Having a look at the Transmission Control Protocol/Internet Protocol themselves, The protocol "suite"
TCP/IP is a set of network protocols designed to work smoothly together. It includes several components,
each defined by a standards-track "Requests For Comments" or series of RFCs, that are :
• IP which routes data packets from one computer to another (RFC791)
• Internet Control Message Protocol providing many low-level kinds of support for IP, including error
messages, routing assistance, and debugging help (RFC792)
• Address Resolution Protocol translating IP addresses to hardware addresses (RFC826)2
• User Datagram Protocol providing unverified, one-way data delivery (RFC768)
• Transmission Control Protocol implementing reliable, full duplex, flow-controlled, error-corrected
conversations (RFC793). All of these protocols are arranged in a stack (hierarchy), with the higher-level
23
protocols making use of the protocols beneath them. TCP/IP is conventionally described as a 5-layer
system, but the actual TCP/IP protocols inhabit only 3 of these layers.
* What's a Port? IP addresses identifying a network interfaces of a machine, are not specific enough to
address individual processes or services, many of which may be actively using the network at once. A port
is a 16-bit number that supplements an IP address to specify a particular communication channel, so as
TCP and UDP extend IP addresses with a concept known as a port. Standard services such as email, FTP,
and HTTP associate themselves with “well known” ports defined in /etc/services. To avoid impersonation
of these services, Linux restrict server programs from binding to port numbers under 1024 unless they are
run as root, so that Anyone can communicate with a server running on a low port number, and the
restriction applies only to the program listening on the port.
* Some of many broad types of address defined by IP layer have direct counterparts at the link layer:
• Broadcast – addresses including all hosts on the local subnet.
• Anycast – addresses resolving to any one of a group of hosts.
• Multicast – addresses simultaneously targeting a group of hosts.
These, though largely unused on today’s Internet, facilitate applications such as video conferencing in
which the same set of packets must be sent to all participants. An Internet Group Management Protocol
constructs and manages sets of hosts treated as one multicast destination.
* Internet addressing (IP addressing) is used in networks, so IP addresses and hardware are respectively
unique and independent. Mapping from IP addresses to hardware ones is implemented at the link layer of
the TCP/IP model. On networks like Ethernet supporting broadcasting (allowing packets to be addressed
to “all hosts on such physical network”), senders use Address Resolution Protocol to discover mappings
without requiring any assistance from a system administrator. For hostname addressing, operating systems
allow hostnames to be associated with IP addresses, though these numbers' sequences are hard for users
to remember, so as users can type rfc-editor.org instead of 128.9.160.27 for example. Under Linux,
mapping can be set up in many manners, ranging from a static file (/etc/hosts) to Lightweight Directory
Access Protocol database system to the worldwide Domain Name System. Hostnames are a convenient
shorthand for IP addresses referring to network interfaces rather than computers.
*ping: check seeing if a host is alive; in many situations it's the only command needed for network debug-
ging. It sends an ICMP ECHO_REQUEST packet to a target host & waits to see if the host answers back.
<host> is an IP address of a machine whose name is as follows, ex : inspiron.u-mascara.dz
nslookup/dig <host> returns the IP address of the machine (so as it works backward with an parmetering
IP address.
telnet <host> <port> establish the interactive connection with another computer on Internet network.
host-address corresponds to the machine name or its IP address.
ssh <[user@]host> as in Secure mode, option –X to allow windowing.
ftp <host> establish the connection in transferring files with another computer, help to obtain the list of
commands.
scp [[user1@]host1:]file1 [[user2@]host2:]file2 a secure copy through a network.
ifconfig c and configure network map.
route displays active routes (as netstat –r)
ping < host > to check connectivity.
traceroute < host > to check network-transit.
netstat displays informations on networks (open ports, ect), -t ; tcp, -u ;
tcpdump to sniff trafic on networks, ex :
tcpdump src 130.79.90.60 and dst 130.79.44.193 –ne // n : numeric
(ip + num port), e : adresses MAC
tcpdump src 130.79.90.60 and port 80 –ne
25
wget for web pages aspiring.
nmap <host> displays the list of open ports. Ex : nmap localhost (scan locally the ports 1- >1024 + those
mentioned in nmap-services), nmap localhost –p1-65535 (scan ports 1 to 65535).
References
1 Evi Nemeth, Garth Snyder, Trent R. Hein & Ben Whaley, Unix & Linux System Administration
26