You are on page 1of 132

Click to edit Master subtitle style The Linux Operating System and Open Source Management Souri Guha

4/23/10
souriguha@gmail.com souriguha.wordpress.com

reasons that we all are

here

Probably we are fed up using Windows! BSOD’s, Viruses, System issues (pretty much incurable stuff) and what not? We want to explore a brand new world We want to learn about Computers We want to have higher efficiency, faster computation, higher stability We all love to be free...

4/23/10

4/23/10

so what are we going to

do?

Go Open Source To get familiar with the Linux environment Get to know what Linux can do for you Get to know what you can do for Linux

4/23/10

our

Agenda

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Bash: Let’s get talking with the kernel Kernel: The heart of the system Shutting down Windows: forever! Applications Contribute!

4/23/10

the world of

Open Source

What is Open Source? The meaning of ‘free’ The benefits of Open Source GPL Freedom 0 GPL Freedom 1

4/23/10

the beginning of

Open Source

In 1983 Richard Sterling starts the GNU Project [HURD Kernel] BSD Berkeley Software Dist [By UC Berkeley] Minix a UNIX Clone Released in1987 by Andrew Tanenbaum Work on Linux Kernel Begins in 1991 by a student from Helsinki – Mr. Linus Torvalds

4/23/10

the open source

Community

What do we mean by the ‘community’? How they work? How they manage to keep it free? Profits, Goals and Motivations Long term vision

4/23/10

the open source

Community

How they Work?

4/23/10

the

difference

4/23/10

the linux enterprise

coordination

4/23/10

birth of Linux

• •

Minix [Credit to Mr. Tannenbaum] Mr. Linus Torvalds

4/23/10

major

Open Source Projects

Mozilla Firefox VLC Media Player OpenOffice Android Symbian Maemo

4/23/10

birth of FOSS

The GNU Project

Travel back 30 years AT&T Bell Labs Developed UNIX

4/23/10

The colors and flavours: DISTROs

What are distros exactly? Thr number of Distributions How to choose from them The differences: Debian,SusE,Redhat,Slackware...

4/23/10

Small and large

Distros for small scale applications Distros for large scale applications

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Bash: Let’s get talking with the kernel Kernel: The heart of the system Shutting down Windows: forever! Applications Contribute!

4/23/10

operating systems: The

basics

The need for operating systems Why cant we do without them

4/23/10

operating systems: The

basics

Architecture of OS[Linux]

4/23/10

operating systems: The

big names

Windows MacOS Linux Others

4/23/10

The OS that is called Linux

The difference POSIX Compliance Supported Architectures

4/23/10

choosing the

Distro

Select your requirements: Development, Learning, Servers, Computation etc. How the Distros are classified What on earth are SPINS? distrowatch.com

4/23/10

why choose

Fedora

The origin of Fedora Core: Redhat Corporaion The fedoraproject Current version:
• •

Fedora 12(Stable) Fedora 13(beta)

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Bash: Let’s get talking with the kernel Kernel: The heart of the system Shutting down Windows: forever! Applications Contribute!

4/23/10

installing

Linux: diving in

Simple! Download the distro Checksum(Thats realy optional) Burn the file onto your DVD All for free!
4/23/10

installing

Methods

CD’s and DVD’s Flash Drives Network boot: NFS, PXE Screenshots

4/23/10

booting

up

Boot methods Prerequisites for a succesful boot Debugging boot error messages Screenshots

4/23/10

starting up the

GUI

gui! The X-Window Project Debugging GUI probs Do we really need a GUI?

4/23/10

logging

IN

the login screen[terminal and X-Window] the login process spawning the other processes

4/23/10

types of

Users

Root (SuperUser) Normal Users Every process is just another user

4/23/10

types of

GUI’s

Two principle types of GUI’s spawned by two different projects The KDE Project The GNOME Project

4/23/10

what we’re using

Here

The GNOME environment

4/23/10

why

GNOME?

Simpler Lighter My personal favourite KDE4 is under development and sometimes unstable in a few hardwares

4/23/10

the Gnome

Interface

The Desktop Panels Launcher Menus Possible customizations Screenshots
4/23/10

the Gnome

Interface

The Desktop Panels Launcher Menus Screenshots
4/23/10

the customization

Possibilities

Every bit of user environment is customizable visit gnomelook.org

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Bash: Let’s get talking with the kernel Kernel: The heart of the system Shutting down Windows: forever! Applications Contribute!

4/23/10

preparing the system for

first Use

what we don’t get in Fedora why we don’t get these the way out:
• • •

Drivers (Specially wireless) Playing your favourite music Movie clips

4/23/10

the God of initial

configs

• • •

www.fedoraguide.info repos codecs

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Bash: Let’s get talking with the kernel Kernel: The heart of the system Shutting down Windows: forever! Applications Contribute!

4/23/10

let’s get

deeper

The filesystem Files are evrything! interacting with the kernel: BASH

4/23/10

classification of

Filesystems

ROMFS: Filesystem on ROM Live Filesystems Hard Disk Filesystems

Journaling Filesystems

ext3 ext4

4/23/10

everything and anything:

Files

Every possible stuff can be done through files What you can do:
• • •

Device Management System Configuration Server, User, System Management

4/23/10

a sneek peek at the

Filesystem[RootFS]

the importance of RootFS descriptions of all folders

4/23/10

a sneek peek at the

Filesystem[RootFS]

4/23/10

the linux Virtual Filesystem[VFS]

• •

the best thing about vfs! how we see it

4/23/10

use a filesystem:

Mount

what do we mean by mounting

4/23/10

the key

files

files in /etc files in /dev files in /bin and /sbin /home and /root
4/23/10

the key

files

files in /dev files in /bin and /sbin /home and /root files in /etc

4/23/10

files in

/etc

4/23/10

windows and linux: a

Comparison

The similarities

Users, Install locations, Apps, GUI

The dissimilarities

App Install, Filesystems,

Comparable apps
4/23/10

now we know the

basics

• • •

What you’ve learnt so far What we can do now Where to go from here

4/23/10

now start

something

What we need

an editor

gedit(Gnome),emacs,vi

internet

4/23/10

the basic file

operations

cd mv cp rm mkdir rmdir ls mount

4/23/10

installing

applications

package managers: rpm, apt the yum package manager what yum can do for you

4/23/10

configuring

yum

package managers: rpm, apt the yum package manager yum for proxy yum repositories what yum can do for you search and install
4/23/10

configuring

yum

package managers: rpm, apt the yum package manager yum for proxy yum repositories what yum can do for you search and install
4/23/10

yum

vs rpm

points where rpm wins over yum if you manage the repository, yum rocks!

4/23/10

compiling Apps from

scratch

compiling from source code debugging installation problems resolving dependencies community support

4/23/10

community support:

howto

what exactly we mean by community support better than support from companies? how to get it? extent of this support

4/23/10

end of

Basics

what have we learnt? what can we do now? where we can go from here?

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Kernel: The heart of the system Bash: Let’s get talking with the kernel Shutting down Windows: forever! Applications Contribute!

4/23/10

getting to the heart of

Linux

The Kernel What the kernel is all about How many lines of code?

4/23/10

understanding the kernel

Versions

how to read the version: what the numbers mean current stable version kernel.org 2.6.....

4/23/10

hwo the kernel became

what it is

Torvalds The father of the kernel 2% of the code is written by Torvalds The development trees Tracking the progress

4/23/10

hwo the kernel became

what it is

Torvalds The father of the kernel 2% of the code is written by Torvalds The development trees Tracking the progress

4/23/10

the kernel:

An Overview

the UI User Identification Processes File management System Calls Scheduler Drivers

4/23/10

the kernel:

Portability

92% of the kernel is hardware independent Rest is written in mixtures of C and assembly 8% of code is used to port systems: The Porting Layer Currently supported Architectures

x86 SPARC PowerPC any system with and MMU systems with virtual MMUs

4/23/10

the kernel: Exploring the code

Kernel Datatypes:

Linked Lists Trees Binary Trees

C
Assembly Inline Assembly Applications

Objdump/readelf Hexdump

objcopy 4/23/10

the kernel: Listening to the kernel

Printk() Dmesg()

Sytem Messages: sysklogd() Messages stored in /proc/kmsg

Conbination of syslogd() and klogd()

4/23/10

the kernel: Some miscellaneous Quirks

__init()

The __init macro tells the compiler that the associate function or variable is used only upon initialization If data is used during init: __initdata()

Likely() and unlikely() IS_ERR() and PTR_ERR()

/include/linux/err.h

4/23/10

the kernel: Some miscellaneous Quirks

__init()

The __init macro tells the compiler that the associate function or variable is used only upon initialization If data is used during init: __initdata()

Likely() and unlikely() IS_ERR() and PTR_ERR()

/include/linux/err.h

4/23/10

the kernel: process

Lifecycle

4/23/10

the kernel: Process

Creation

4/23/10

the kernel: Process

System Calls

Fork() :

The fork() function returns twice: once in the parent and once in the child process. If it returns in the child process, fork() returns 0. If it returns in the parent, fork() returns the child's PID. When the fork() function is called, the function places the necessary information in the appropriate registers, including the index into the system call table where the pointer to the system call resides.

Vfork():

The vfork() function is similar to the fork() function with the exception that the parent process is blocked until the child calls exit() or exec()

Clone() 4/23/10

the kernel: Process

State Transitions

4/23/10

the kernel: Process

Killing

Return from main() function Calling of exit() function

Exit functions

Sys_exit() function Do_exit() function

Parent Notifications
Sys_wait4() 4/23/10

the kernel: Process

Killing

Return from main() function Calling of exit() function

Exit functions

Sys_exit() function Do_exit() function

Parent 4/23/10 Notifications

the kernel: Memory Management

Pages :

As the basic unit of memory managed by the memory manager, a page has a lot of state that it needs to be kept track of.

Slab Allocator Memory Request Path Page Faults

Do_page_fault()

/arch/i386/mm/fault.c

4/23/10

the kernel: task

Scheduling

Open() Close() Read() Flushing Dirty Pages

pdflush

4/23/10

the kernel: task

Scheduling

Completely new scheduler since 2.6 kernel: the O(1) Scheduler

kernel/sched.c include/linux/sched.h

4/23/10

the kernel: task

Scheduling - Preemption

Preemption is the switching of one task to another Both user space programs and kernel space programs can be switched at various times

4/23/10

the kernel: system

Clock

Real-Time Clock (RTC)
/include/linux/rtc.h RTC provides timekeeping fucnctions with a small 114-byte user NVRAM Input to this device is a 32.768Mhz oscillator and a battery backup Newer RTC built into the Southbridge of a computer chipset

4/23/10

the kernel: Booting it

up

4/23/10

the kernel:

Bootloader

Program residing on the boot device of the computer MBR is on the first sector of boot device(sector0, cylinder 0, head 0)

4/23/10

the kernel:

Bootloader

MBR Components

0x00 0x1bd MBR Program Code 0x1be 0x40 Partition Table 0x1fe 0x2 Hex Marker/Signature 0xAA55

MBR 16byte entries

0x00 1 0x01 3 0x04 1 0x05 3 0x08 4 0x0c 4

Active Boot Partition Flag Starting Cylinder/Head Sector of Boot Partition Partition type(in linux=0x83) Ending Cylinder of Boot Partition Partition Starting in Sector Numbers Partition Length in Sectors

4/23/10

the kernel: Boot Memory at

bootup time

4/23/10

the kernel:

Bootloader- GRUB

The Grand Unified Bootloader www.gnu.org/software/grub/ Two Stage

Stage 1: Installed in MBR and is called by BIOS Stage 2: Partially loaded by Stage1 and finishes loading from filesystem

4/23/10

the kernel:

Bootloader- GRUB

Stage 1: Installed in MBR and is called by BIOS

Initialization Detect Loading drive Load the first Sector of Stage 2 Jump to Stage 2

Stage 2: Partially loaded by Stage1 and finishes loading from filesystem

Load rest of Stage 2 Jump to loaded code

/boot/menu.lst

4/23/10

the kernel:

Bootloader- LILO

Stage 1: Installed in MBR and is called by BIOS

Begins execution and displays "L." Detects disk geometry and displays "I." Loads Stage 2 code.

Stage 2: Partially loaded by Stage1 and finishes loading from filesystem

Begins execution and displays "L." Locates boot data and OS and displays "O." Determines which OS to start and jumps to it

/boot/menu.lst 4/23/10

the kernel:

Bootloader- initrd

The Initial RAM Disk The kernel initializes using the RAM disk, mounts the final root filesystem, and then removes the initrd Steps:

Configuring a kernel at boot time Keeping a small general-purpose kernel Having one kernel for several hardware configurations

4/23/10

the kernel:

Bootloader- the startup

start_kernel()

Spawns off Process 0 Process 0 spawns Process 1(init) /sbin/init/ Process 0 is now an idle thread

4/23/10

Building the

Kernel

What do we need

Toolchains (GUI versions alos known as SDK) Compilers Cross Compilers (Architecture Specific) Linkers A copy of the Source Code!

4/23/10

Building the

Kernel: The Toolchain

Set of programs necessary to create the linux kernel image Toolchain concept: Output of one tool becomes the input of another tool Our toolchain includes:

A compiler An Assembler A linker

4/23/10

Building the

Kernel: Cross Compilers

Building linux for other platforms Host PC: PC you are performing the Development Process Target: The device for which you are developing

4/23/10

Building the

Kernel: ELF Object Files

Format for Object Files vary from manufacturer to manufacturer Standard unix object file type: Executable andLining File format Types of ELF

Executable Relocatable Core files/ Shared Libraries

4/23/10

Building the

Kernel: ELF Object Files

Format for Object Files vary from manufacturer to manufacturer Standard unix object file type: Executable andLining File format Types of ELF

Executable Relocatable Core files/ Shared Libraries

4/23/10

Let’s

Build it!

Heavy involvement with the kernel Source Code Detailed howto www.tldp.org/HOWTO/Kernel-HOWTO/
Download: kernel.org Find it under: /usr/src/linux-<version>

4/23/10

Source code

Layout

4/23/10

Architecture independent

Subdirectories

Crypto: Drivers : Fs: Include: init: ipc:

Holds code for cryptographic API and various encrypting/decrypting algorithms Code for device drivers Code for vfs and all filesystems supported by Linux
The header files. This directory has a series of subdirectories starting with the prefix asm. These directories hold the architecture-specific header files.

The architecture-independent portion of the bootstrapping code and initialization code

code for interprocess communication (ipc) support

kernel: code for kernel specific code lib: mm: net : code for helper functions code for memory manager code to support various networking protocols

sound: code for sound system support

4/23/10

Architecture dependent

Subdirectories

/arch/x86

boot: kernel : lib: mach-x:

Source code related to the x86 bootstrapping and install process Source code for the kernel subsystem hardware dependencies Source code for x86-specific library files
Source code for the x86 subarchitecturess. Source code for x86 math-emulation functions

math-emu: mm: oprofile: pci: power:

Source code for the x86-specific parts of memory management Source code for the oprofile kernel profiling tool x86 PCI drivers Source code for x86 power management

4/23/10

Miscellaneous

Files

Copying: Credits:

The GPL license under which Linux is licensed List of contributors to the Linux project List of maintainers and instructions on submitting
Release notes. Describes the procedure for reporting bugs

Maintainers: Readme:

kernel changes

Reporting-Bugs: documentation/:

4/23/10

Directory with partial documentation on various aspects of the Linux kernel and source code. Great source of information, if sometimes slightly out of date.

building the

Kernel Image

kbuild highly dependent on the hierarchical strucure of filesystem generates the .config file

Take a look at it

The makefile

#make xconfig ARCH=ppc #make ARCH=ppc CROSS_COMPILE=ppc-linux-

4/23/10

makefile

relationships

4/23/10

what

next?

Let’s communicate with the Kernel

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Kernel: The heart of the system Bash: Let’s get talking with the kernel Shutting down Windows: forever! Applications Contribute!

4/23/10

what

bash?
translate
the user's command lines into operating

Bash: The Bourne-Against-Shell The shell's job, is to system instructions

sort -n phonelist > phonelist.sorted

4/23/10

history of

bash

named after Steve Bourne bash, intended to be the standard shell for the GNU system, was officially "born" on Sunday, January 10, 1988 In keeping with the GNU principles, all versions of bash since 0.99 have been freely available from the FSF it includes the best features of the C and Korn shells as well as several advantages of its own

4/23/10

bash

Characters

4/23/10

bash

Script basics

A script, which is a file that contains shell commands, is a shell program. Write the script <./my_script> Enable Permissions

$ chmod +x  scriptname

4/23/10

ways to run a

script

4/23/10

the shell

variables

bash derives much of its programming functionality from shell variables. $ hatter=mad$ ; echo "$hatter"mad positional parameters in functions

function alice

{   
echo "alice: $*"    echo "$0: $1 $2 $3 $4"    echo "$# arguments”

} 4/23/10

the shell

flow control

if else

if condition then   
statements

[elif condition   
then statements...]

[else     statements] fi

Error Processing

if command ran successfully then  
normal processing

else  

4/23/10

the shell

flow control

for

for name [in list] do    statements that can use $name... done

example

for dir in $PATH; do   

4/23/10 if [ -z "$dir" ]; then dir=.; fi    

the shell

flow control

case
pattern1 )         statements ;;    pattern2 )      

case expression in    

  ... esac

statements ;;   

4/23/10

the shell

flow control

while

while conditiondo     statements...done

until
until command; do    statements...done

Example

until cp $1 $2; do    echo 'Attempt to copy failed. waiting...'    sleep 5 done

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Kernel: The heart of the system Bash: Let’s get talking with the kernel Shutting down Windows: forever! Applications Contribute!

4/23/10

say goodbye to

Windows!

Higher freedom

Faster, Lighter

Flexible

No viruses! [Until you compile and install them ]

4/23/10

say goodbye to

Windows!

Higher freedom

Faster, Lighter

Flexible

No viruses! [Until you compile and install them ]

4/23/10

say goodbye to

Windows!

Find Comparable apps to Windows

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Kernel: The heart of the system Bash: Let’s get talking with the kernel Shutting down Windows: forever! Applications Contribute!

4/23/10

current

Applications

Home/Personal use Corporate Use Workstations Parallel Computing

Supercomputers

Handheld Devices – PDA’s, Cellphones Small gadgets:

Routers Access Points GPS Sat Nav Devices(Garmin)

4/23/10 Oscilloscopes

current

Applications

Examples:

Generation of 3d Animations in movies like Titanic, Shrek Avatar: The Movie NASA Supercomputer Grids 70% of all servers run on linux Your Motorola Cellphone (Open Moto) Cloud Computing Nokia phones running Linux [video] LHC [Hadron Collider Project]

4/23/10

current

Applications : Gaming

Transgaming

Used to port Windows Games to run under Linux Distros Almost every Game runs under Linux
• • • •

Call of Duty 4 Need for Speed [Most Wanted and Shift] Counter Strike etc.

Try them out. Today! Much better performance if you have proper graphics drivers installed 

4/23/10

current

Applications : Gaming

Transgaming

Used to port Windows Games to run under Linux Distros Almost every Game runs under Linux

Call of Duty 4 Need for Speed [Most Wanted and Shift] Counter Strike etc.

Try them out. Today! Much better performance if you have proper graphics drivers installed  Supports installation to Sony Playstation 3! [video]

4/23/10

Applications : Parallel Computing[GRIDS]
current

Try out this one at your home

Download Parallel Knoppix Burn it onto multiple cd’s Boot it up Connect it to LAN Configure DHCP Server You now

own a supercomputer!

That was too much simple!

4/23/10

where are

WE?

The World of Open Source Operating Systems: The basics Linux: Let’s get started Configure it for first use How to use it: Common problems, techniques, tips Kernel: The heart of the system Bash: Let’s get talking with the kernel Shutting down Windows: forever! Applications Contribute!

4/23/10

please

contribute!

Why contribute?

The foundation expects you to contribute back to the project for the reason you have used the project for your own benefits The project is worth a billion man years! That’s fascinating right? Contribute, Share, Develop GPL and LGPL Licences

4/23/10

howto

contribute!

Use Version Control Systems

GIT View a guide to GIT on my blog (souriguha.wordpress.com) How the community work

4/23/10

GIT

Basics

4/23/10

Any

Questions?
anubhav.seth@ieee.org souri.guha@ieee.org souriguha@gmail.com
kernel.org linux.com fedoraproject.org linuxfoundation.com

Contact:

Send a cc to:

Visit:

4/23/10

thank

you!

4/23/10