Algorithm Analysis and Design PDF

Algorithm Analysis and Design (R 606) 1
Department of Computer Science & Engineering SJCET, Palai

ALGORITHM ANALYSIS AND DESIGN

R606 3+1+0
Module 1 Introduction and Complexity
What is an algorithm – Properties of an Algorithm, Difference between Algorithm,

Computational Procedure and Program, Study of Algorithms; Pseudo-code
Conventions; Recursive Algorithms –Space and Time Complexity –Asymptotic
Notations – ‗Oh‘, ‗Omega‘, ‗Theta‘, Common Complexity Functions; Recurrence
Relations and Recurrence Trees for Complexity Calculations; Profiling. –
Deterministic and non - deterministic algorithms.
Module 2 Divide and Conquer
Control Abstraction, Finding Maximum and Minimum, Binary Search, Divide and
Conquer Matrix Multiplication, Stressen‘s Matrix Multiplication, Merge Sort, Quick
Sort.
Module 3 Greedy Strategy
Control Abstraction, General Knapsack Problem, Optimal Storage on Tapes,

Minimum Cost Spanning Trees – Prim‘s Algorithm, Kruskal‘s Algorithm – Job
sequencing with deadlines.
Module 4 Dynamic Programming
Principle of Optimality, Multi-stage Graph, All-Pairs Shortest Paths, Travelling

Salesman Problem.
Lower Bound Theory - Comparison Trees for Searching and Sorting, Oracles and
Adversary Arguments – Merging, Insertion & Selection Sort; Selection of ‗k‘th
Smallest Element.
Module 5 Backtracking
Control Abstraction - Bounding Functions, Control Abstraction, N-Queens Problem,

Sum of Subsets, Knapsack problem.
Branch and Bound Techniques – FIFO, LIFO, and LC Control Abstractions, 15-
puzzle, Travelling Salesman Problem.

Text Book
1. Fundamentals of Computer Algorithms - Horowitz and Sahni, Galgotia
References
1. Computer Algorithms – Introduction to Design and Analysis - Sara Baase & Allen Van
Gelder, Pearson Education
2. Data Structures algorithms and applications - Sahni, Tata McGrHill
3. Foundations of Algorithms - Richard Neapolitan, Kumarss N., DC Hearth & Company
4. Introduction to algorithm- Thomas Coremen, Charles, Ronald Rivest -PHI

TABLE OF CONTENTS
Module 1
What is an algorithm ……………………………………………………… 7
Properties of an Algorithm ……………………………………................ .. 7
Difference between Algorithm Computational
Procedure and Program …………………………………...................... 8
Study of Algorithms........................................................................ 10
Pseudocode convention………………………………………………….. 10
Recursive Algorithms …………………………………........................ 12
Space Complexity ………………………………………….................. 19
Time Complexity............................................................................. 20
Asymptotic Notations ………………………………………………....... 25
Oh……………………………………………...................................... 26
Omega........................................................................................... ......... 28
Theta............................................................................................ .......... 30
Common Complexity Functions……………………………………….... 33
Recurrence Relations....................................................................... 35
Recurrence Trees for Complexity Calculations…................................ 39
Profiling……………………………………………............................... 41
Deterministic and non - deterministic algorithms………………………… 41

Module 2
Control Abstraction ………………………………………………………. 44
Finding Maximum and Minimum ………………………………………... 45
Binary Search ………………………………………………………........ 51
Divide and Conquer Matrix Multiplication …………………................. 54
Stressen‘s Matrix Multiplication ……………………………………….. 56
Merge Sort ……………………………………………………………... 60
Quick Sort …………………………………………………………….... 69
Module 3
Greedy Strategy
Control Abstraction ……………………………………………………… 87
General Knapsack Problem ……………………………………………… 88
Optimal Storage on Tapes ……………………………………………… 90
Minimum Cost Spanning Trees ………………………………………... 91
Prim‘s Algorithm …………………………………………….............. 93
Kruskal‘s Algorithm …………………………………………………... 99
Job sequencing with deadlines……………………………………. 100
Module 4
Dynamic Programming
Principle of Optimality ……………………………………………… 105
Multi-stage Graph ………………………………………………....... 106

All-Pairs Shortest Paths …………………………………………….. 118
Lower Bound Theory
Comparison Trees for Searching and Sorting .................................. 127
Oracles and Adversary Arguments.................................................. 134
Merging…………………............................................................... 135
Insertion………………................................................................... 136
Selection Sort……………............................................................... 142
Selection of ‗k‘th Smallest Element……………………....................... 144
Module 5
Backtracking
Control Abstraction ………………………………………………….... 149
Bounding Functions ………………………………………………....... 153
Control Abstraction …………………………………………………….. 149
N-Queens Problem.......................................................................... 153
Sum of Subsets ……....................................................................... 158
Knapsack problem……………......................................................... 164
Branch and Bound Techniques
FIFO …........................................................................................... 172
LIFO…........................................................................................... ............ 177
LC Control Abstractions..................................................................... 181
15-puzzle….................................................................................... .......... 182
Travelling Salesman Problem…........................................................ 184

MODULE 1

WHAT IS AN ALGORITHM
Definition: An algorithm is a finite set of instructions that, if followed, accomplishes a

particular task.
PROPERTIES OF AN ALGORITHM
All algorithms must satisfy the following criteria:
1. Input. Zero or more quantities are externally supplied.

2. Output. At least one quantity is produced.
3. Definiteness. Each instruction is clear and unambiguous.
4. Finiteness. If we trace out the instructions of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
5. Effectiveness. Every instruction must be very basic so that it can be carried out, in
principle, by a person using only pencil and paper. It is not enough that each
operation be definite as in criterion 3; it also must be feasible.
An algorithm is composed of a finite set of steps, each of which may require one or more
operations. The possibility of a computer carrying out these operations necessitates that
certain constraints be placed on the type of operations an algorithm can include.
Criteria 1 and 2 require that an algorithm produce one or more outputs and have zero or more
inputs that are externally supplied. According to criteria 3, each operation must be definite,
meaning that it must be perfectly clear what should be done.
The fourth criterion for algorithms is that they terminate after a finite number of operations.
A related consideration is that the time for termination should be reasonably short.
Criteria 5 requires that each operation be effective; each step must be such that it can, at least
in principle, be done by a person using pencil and paper in a finite amount of time.
Performing arithmetic on integers is an example of an effective operation, but arithmetic with
real numbers is not, since some values may be expressible only by infinitely long decimal
expansion.

DIFFERENCE BETWEEN ALGORITHM, COMPUTATIONAL

PROCEDURE AND PROGRAM
COMPUTATIONAL PROCEDURE
Algorithms that are definite and effective are also called computational procedures. One
important example of computational procedures is the operating system of a digital computer.
This procedure is designed to control the execution of jobs, in such a way that when no jobs
are available, it does not terminate but continues in a waiting state until a new job is entered.
PROGRAM
To help us achieve the criterion of definiteness, algorithms are written in a

programming language. Such languages are designed so that each legitimate sentence has a
unique meaning. A program is the expression of an algorithm in a programming language.
Sometimes words such as procedure, function and subroutine are used synonymously for
program.
The study of algorithms includes many important and active areas of research. There
are four distinct areas of study:
1. How to devise algorithms: - Creating an algorithm is an art which may never

be fully automated. There are several techniques with which you can devise
new and useful algorithms. Dynamic programming is one such technique.
Some of the techniques are especially useful in fields other than computer
science such as operations research and electrical engineering.
2. How to validate algorithms: - Once an algorithm is devised, it is necessary to
show that it computes the correct answer for all possible legal inputs. This
process is referred to as algorithm validation. It is sufficient to state the
algorithm in any precise way and need not be expressed as a program. The
purpose of validation is to assure us that this algorithm will work correctly
independently of the issues concerning the programming language it will
eventually be written in. once the validity of the method has been shown, a
program can be written and a second phase begins. This phase is referred to as
program proving or program verification. A proof of correctness requires that

the solution be stated in two forms. One form is usually as a program which is
annotated by a set of assertions about the input and output variables of the
program. These assertions are often expressed in predicate calculus. The
second form is called a specification, and this may also be expressed in the
predicate calculus. A proof consists of showing that these two forms are
equivalent in that for every given legal input, they describe the same output. A
complete proof of program correctness requires that each statement of the
programming language be precisely defined and all basic operations be proved
correct.
3. How to analyze algorithms: - this field of study is called analysis of
algorithms. As an algorithm is executed, it uses the computer‘s central
processing unit (CPU) to perform operations and its memory (both immediate
and auxiliary) to hold the program and data. Analysis of algorithms or
performance analysis refers to the task of determining how much computing
time and storage an algorithm requires. An important result of this study is that
it allows you to make quantitative judgments about the value of one algorithm
over another. Another result is that it allows you to predict whether the
software will meet any efficiency constraints that exist. Questions such as how
well an algorithm performs in the best case, in the worst case, or on the
average are typical.
4. How to test a program: - testing a program consists of two phases: debugging
and profiling (or performance measurement). Debugging is the process of
executing programs on sample data sets to determine whether faulty results
occur and, if so to correct them. In cases in which we cannot verify the
correctness of output on sample data, the following strategy can be employed:
let more than one programmer develop programs for the same problem, and
compare the outputs produced by these programs. If the outputs match, then
there is a good chance that they are correct. A proof of correctness is much
more valuable than a thousand tests, since it guarantees that the program will
work correctly for all possible inputs. Profiling or performance measurement
is the process of executing a correct program on data sets and measuring the
time and space it takes to compute the results. These timing figures are useful
in that they may confirm a previously done analysis and point out logical
places to perform useful optimization.

STUDY OF ALGORITHM
An algorithm is a finite set of instructions that , if followed , accomplishes a particular task.
The study of algorithms includes many important and active areas of research. There are four
distinct areas of study
1. How to devise algorithms: Creating an algorithm is an art which may never be

fully automated.To study various design techniques that have proven to be useful in
that they have often yielded good algorithms.Dynamic programming is one such
technique.some important design techniques are linear,nonlinear and integer
programming .
2. How to validate algorithms: Once an algorithm is devised , it is necessary to show
that it computes the correct answer for all possible legal inputs.This process is
algorithm validation.The purpose of the validation is to assure us this algorithm will
work correctly independent of the issues concerning the programming language it will
eventually be written in.
3. How to analyze algorithms: This field of study is called analysis of algorithms. As
an algorithm is executed , it uses the computer‘s central processing unit to hold the
program and data.Analysis of algorithms refers to the task of determining how much
computing time and storage an algorithm requires.It allows you to predict whether
the software will meet any efficiency constraints that exits.
4. How to test a program: Testing a program consists of two phases, debugging and
profiling.Debugging is the process of executing programs on sample data sets to
determine whether faulty results occur and , if so, to correct them. Profiling is the
process of executing a correct program on data sets and measuring the time and space
it takes to compute the results.
PSEUDOCODE CONVENTIONS
We can describe an algorithm in many ways.We can use a natural language like
English, although I we select this option,we must make sure that the resulting instructions are
definite.
We can present most of our algorithms using a pseudocode that resembles c
1. Comments begin with // and continueuntill the end of line
Eg: count :=count+1;//count is global ;It is initially zero.

2. Blocks are indicated with matching braces: { and } .A compound statement

can be represent as a block.The body of a procedure also forms a block.Statements
are delimited by ;
Eg: for j:= 1 to n do
{
Count:=count+1;
C[i,j]:=a[i,j]+b[i,j];
Count:=count +1; }
3. An identifier begins with a letter. The data types of variables are not explicitly
declared. The types will be clear from the context .Whether a variable is global or
local to a procedure will also be evident from the context. Compound data types
can be formed with records.
Eg: node=record
{
datatype_1 data_1;
:
datatype_n data_n;
node *link;
}
4. Assignment of values to variables is done using the assignment statement
<variable> := <expression>;
Eg: count:= count+1;
5. There are two Boolean values true and false. In order to produce these values, the
logical operators and , or , and not and the relational relational operators
<,<=,=,!=,>= and > are provided.
Eg: if (j>1) then k:=i-1; else k:=n-1;
6. Elements of multidimentional arrays are accessed using [ and ].
For eg: if A is a two dimentional array , the (I,j) th element of the array is denoted as A[I,j].
Array indicates start at zero.
7. The following looping statements are employed: for,while and repeat until. The
while loop takes the following form.
While (condition) do
{
<statement 1>

:
:
<statement n>
}
8. A conditional statement has the following forms:
If < condition > then <statement>

If<condition> then <statement 1> else <statement 2>
Here < condition > is a Boolean expression and <statement>,<statement 1>, and
< statement 2> are arbitrary statements.
9. Input and output are done using the instructions read and write. No format is used
to specify the size of input or output quantities.
Eg: write (―n is even‖);
10. There is only one type of procedure: Algorithm. An algorithm consists of a
heading and a body. The heading takes the form
Algorithm Nmae (<parameter list>)
RECURSIVE ALGORITHMS
A function that calls itself repeatedly, satisfying some condition is called a Recursive
Function. The algorithm that does this is called a recursive algorithm. Using recursion, we
split a complex problem into its single simplest case. The recursive function only knows how
to solve that simplest case.
TYPES OF RECURSION:
Linear Recursion
A linear recursive function is a function that only makes a single call to itself each time the
function runs (as opposed to one that would call itself multiple times during its execution).
The factorial function is a good example of linear recursion. Another example of a linear
recursive function would be one to compute the square root of a number using Newton's
method (assume EPSILON to be a very small number close to 0):
double my_sqrt(double x, double a)

{
double difference = a*x-x;
if (difference < 0.0) difference = -difference;
if (difference < EPSILON) return(a);
else return(my_sqrt(x,(a+x/a)/2.0));
}
Tail recursive
Tail recursion is a form of linear recursion. In tail recursion, the recursive call is the last thing
the function does. Often, the value of the recursive call is returned. As such, tail recursive
functions can often be easily implemented in an iterative manner; by taking out the recursive
call and replacing it with a loop, the same effect can generally be achieved. In fact, a good
compiler can recognize tail recursion and convert it to iteration in order to optimize the
performance of the code.
A good example of a tail recursive function is a function to compute the GCD, or Greatest
Common Denominator, of two numbers:
int gcd(int m, int n)

{
int r;
if (m < n) return gcd(n,m);
r = m%n;
if (r == 0) return(n);
else return(gcd(n,r));
}
Binary Recursive
Some recursive functions don't just have one call to themself, they have two (or more).
Functions with two recursive calls are referred to as binary recursive functions. The
mathematical combinations operation is a good example of a function that can quickly be
implemented as a binary recursive function. The number of combinations, often represented

as nCk where we are choosing n elements out of a set of k elements, can be implemented as
follows:
int choose(int n, int k)

{ if (k == 0 || n == k) return(1);
else return(choose(n-1,k) + choose(n-1,k-1));
}
Exponential recursion
An exponential recursive function is one that, if you were to draw out a representation of all
the function calls, would have an exponential number of calls in relation to the size of the
data set (exponential meaning if there were n elements, there would be O(an) function calls
where a is a positive number). A good example an exponentially recursive function is a
function to compute all the permutations of a data set. Let's write a function to take an array
of n integers and print out every permutation of it.
void print_array(int arr[], int n)

{
int i;
for(i=0; i<n; i) printf("%d ", arr[i]);
printf("\n");
}
void print_permutations(int arr[], int n, int i)
{
int j, swap;
print_array(arr, n);
for(j=i+1; j<n; j) {
swap = arr[i]; arr[i] = arr[j]; arr[j] = swap;
print_permutations(arr, n, i+1);
swap = arr[i]; arr[i] = arr[j]; arr[j] = swap;
}
}

To run this function on an array arr of length n, we'd do print_permutations(arr, n, 0) where

the 0 tells it to start at the beginning of the array.
Nested Recursion
In nested recursion, one of the arguments to the recursive function is the recursive function
itself! These functions tend to grow extremely fast. A good example is the classic
mathematical function, "Ackerman's function. It grows very quickly (even for small values of
x and y, Ackermann(x,y) is extremely large) and it cannot be computed with only definite
iteration (a completely defined for() loop for example); it requires indefinite iteration
(recursion, for example).
Ackerman's function
int ackerman(int m, int n)
{
if (m == 0) return(n+1);
else if (n == 0) return(ackerman(m-1,1));
else return(ackerman(m-1,ackerman(m,n-1)));
}
Mutual Recursion
A recursive function doesn't necessarily need to call itself. Some recursive functions work in
pairs or even larger groups. For example, function A calls function B which calls function C
which in turn calls function A. A simple example of mutual recursion is a set of function to
determine whether an integer is even or odd.
int is_even(unsigned int n)

{
if (n==0) return 1;
else return(is_odd(n-1));
}
int is_odd(unsigned int n)
{
return (!iseven(n));
}

EXAMPLES OF RECURSIVE ALGORITHMS:
The Towers of Hanoi

The Towers of Hanoi puzzle (TOH) was first posed by a French professor, ´Edouard Lucas,
in 1883. Although commonly sold today as a children‘s toy, it is often discussed in discrete
mathematics or computer science books because it provides a simple example of recursion. In
addition, its analysis is straightforward and it has many variations of varying difficulty.
The object of the Towers of Hanoi problem is to specify the steps required to move the disks
or, as we will sometimes call them, rings) from pole r (r = 1, 2, or 3) to pole s (s = 1, 2, or 3;
s _= r), observing the following rules:
i) Only one disk at a time may be moved.
ii) At no time may a larger disk be on top of a smaller one.
The most common form of the problem has r = 1 and s = 3.
FIGURE
.
The Towers of Hanoi problem
Solution The algorithm to solve this problem exemplifies the recursive paradigm. We
imagine that we know a solution for n − 1 disks (―reduce to a previous case‖), and then we
use this solution to solve the problem for n disks. Thus to move n disks from pole 1 to pole 3,
we would:
1. Move n − 1 disks (the imagined known solution) from pole 1 to pole 2.
However we do this, the nth disk on pole 1 will never be in our way because any valid
sequence of moves with only n −1 disks will still be valid if there is an nth (larger) disk
always sitting at the bottom of pole 1 (why?).
2. Move disk n from pole 1 to pole 3.
3. Use the same method as in Step 1 to move the n −1 disks now on pole 2 to

pole 3.
Algorithm:
Input num [Number of disks]
Pinit [Initial pole; 1 ≤ Pinit ≤ 3]
Pfin [Final pole; 1 ≤ Pfin ≤ 3, Pinit _= Pfin]
Output The sequence of commands to the robot to move the disks from
pole Pinit to pole Pfin
Algorithm Hanoi
procedure H(in n, r, s) [Move n disks from pole r to pole s]
if n = 1 then robot(r → s)
else H(n−1, r, 6−r−s)
robot(r → s)
H(n−1, 6−r−s, s)
endif
endpro
H(num, Pinit, Pfin) [Main algorithm]
ALGORITHM DESIGN TECHNIQUES
For a given problem, there are many ways to solve them. The different methods are listed
below.
1. Divide and Conquer.
2. Greedy Algorithm.
3. Dynamic Programming.
4. Branch and Bound.
5. Backtracking Algorithms.
6. Randomized Algorithm.
Now let us discuss each method briefly.
1. Divide and Conquer

Divide and conquer method consists of three steps.

a. Divide the original problem into a set of sub-problems.
b. Solve every sub-problem individually, recursively.
c. Combine the solutions of the sub-problems into a solution of the whole original
problem.
2. Greedy Approach
Greedy algorithms seek to optimize a function by making choices which
are the best locally but do not look at the global problem. The result is a good solution but not
necessarily the best one. Greedy Algorithm does not always guarantee the optimal solution
however it generally produces solutions that are very close in value to the optimal solution.
3.Dynamic Programming.
Dynamic Programming is a technique for efficient solution. It is a
method of solving problems exhibiting the properties of overlapping sub problems and
optimal sub-structure that takes much less time than other methods.
4. Branch and Bound Algorithm.
In Branch and Bound Algorithm a given Algorithm

which cannot be bounded has to be divided into at least two new restricted sub-problems.
Branch and Bound Algorithm can be slow, however in the worst case they require efforts that
grows exponentially with problem size. But in some cases the methods converge with much
less effort. Branch and Bound Algorithms are methods for global optimization in non-convex
problems.
5.Backtracking Algorithm.
They try each possibility until they find the right one. It is a depth
first search of a set of possible solution. During the search if an alternative doesn‘t work, the
search backtrack to the choice point, the place which presented different alternatives and tries
the next alternative. If there are no more choice points the search fails.
6.Randomized Algorithm.
A Randomized Algorithm is defined as an algorithm that is
allowed to access a source of independent, unbiased, random bits, and it is then allowed to
use these random bits to influence its computation.

ALGORITHMIC COMPLEXITY
The time complexity of an algorithm is given by the number of steps taken by the
algorithm to compute the function it was written for. The number of steps is itself a function
of the instance characteristics. Althogh any specific instance may have several characteristics,
the number of steps computed as a function of some subset of these. Usually, we might wish
to know how the computing time increases as the number of inputs increase. In this case the
number of steps will be computed as a function of the number of inputs alone. For a different
algorithm , we might be interested in determining how the computing time increases as the
magnitude of one of the input increases. In this case the number of steps will be computed as
a function of the magnitude of this input alone. Thus before the step count of an algorithm
can be determined, we need to know which characteristics of the problem instance are to be
used. These define the variables in the expression for the step count. In the case of sum, we
chose to measure the time complexity as a function of the number n of elements being added.
For algorithm Add, the choice of characteristics was the number m of rows and the number n
of columns in the matrices being added.
SPACE COMPLEXITY
Algorithm abc computes a+b+b*c+(a+b-c)/(a+b)+4.0;
Algorithm Sum computes Ei to n = 1 a[i] iteratively, where the a[i]‘s are real numbers; and
RSum is a recursive algorithm that computes Ei n =1 a [i]
The space needed by each of these algorithms is seen to be the sum of the following
components:
1. A fixed part that is independent of the characteristics of the inputs and ouputs . This
part typically includes the instruction space, space for simple variables and fixed-size
component variables, space for constants.
2. A variable part that consists of the space needed by component variables whose size is
dependent on the particular problem instance being solved, the space needed by
reference variables and he recursion stack space.
The space requirement s(p) of any algorithm p may thereore be written as s(p) =c+Sp,
Where c is a constant.

TIME COMPLEXITY
COMPLEXITY OF SIMPLE ALGORITHMS
Time complexity of an algorithm is given by the number of steps taken

by the algorithm to compute the function it was written for. The time T(P) taken by a
program P is the sum of compile time and run time. The compile time does not depends the
instance characteristics. A compiled program will run several times without recompilation.
So we have to concern with the run time of the program which is denoted by tp (instance
characteristics).
If we know the characteristics of the compiler to be used ,we could proceed to determine the
number of additions, subtractions, multiplications, divisions, compares, loads, stores and so
on,that would be made the code for P. So we could obtain an expression for tp (n) of the
form
tp(n)=ca ADD(n) + cs SUB(n) + cm MUL(n) + cd DIV(n)+…….
Where n denotes the instance characteristics, and ca, cs , cm, cd, and so on, respectively, denote
the time needed for an addition, subtraction, multiplication, division ,and so on, and
ADD,SUB,MUL, DIV, and so on, are functions whose values are the numbers of additions,
subtractions, multiplications, divisions, and so on ,that are performed when the code for P is
used on an instance with characteristic n. The value of tp(n) for any n can be obtained only
experimentally.The program is typed,compiled and run on a particular machine.The
execution time is physically blocked,and tp(n) obtained.In a multi user system ,execution time
depends on factors such as system load and number of other programs running on the
computer at the time P is running,the characteristics of these other programs and so on.
A propgram step is loosely defined as a syntactically or semantically meaningful segment of

a program that has an execution time independent of the instance characteristics. For
example,consider the entire stataement
return a+b+b*c+(a+b-c)/(a+b)+4.0 of the program given below

Algorihm abc(a,b,c)
return a+b+b*c+(a+b-c)/(a+b)+4.0
The above line could be considered as a step since its execution time is independent of the the
instance characteristics.The number of steps any program statement is assigned depends on
the kind of statement.For example comments count as zero steps;an assignment statement
which does not involve any calls to other algorithms is counted as one step;in an iterative
statement such as the for ,while,and repeat until statements ,we consider the step count only
for the control part of the the statement.The control parts for for and while statements have
the following forms:
for i=(expr) to (expr1) do
while(expr) do
Each execution of the control part of a while statement is given by a step count equal to the
number of step counts assignable to(expr).The step count for each execution of the
controlpart of a for statement is one,unless the counts attributable to (expr) and (expr1) are
the functions of the instance characteristics. In the latter case,the first execution of the
control part the for has a step count equal to the sum of counts for (expr)
and (expr1).Remaining executions of the for statement have a step count of one;and o on.
We can determine the number of steps needed by a program to solve a

particular program instance in one of the two ways.In the first method we introduce a new
variable,count into the problem.This is a global variable with initial value equals
zero.Statement to increment count by appropriate amount are introduced into the
program.This is done so that each time a statement in the original program is executed,count
is incremented by the step count of that statement.

EXAMPLE FOR TIIME COMPLEXITY CALCULATION
Fibonacci series of numbers starts as 0,1,1,2,3,5,8,13,21,34,55…………..
Each new term is obtained by taking the sum of two previous terms.If we call the first term
of the sequence f0,then f0=0,f1=1,and in general
fn=fn-1+fn-2, n>=2
Algorithm Fibonacci(n)
// Compute the nth Fibonacci number
{
if (n<=1) then
Write(n);
else
{
fnm2:=0;fnm=1;
For I := 1 to n do
{
fn = fnm1+fnm2;
fnm2 =fnm1;fnm1=fn;
}
Write(fn)
}
}
To analyze the complexity of this algorithm we need to consider two cases (1)
n=0 or 1 and (2) n>1.When n=0 or 1 lines 4and 5 get executed once each.Since each line has
an s/e of 1,total step count for this case is 2.When n>1,lines 4, 8 and 14 are each executed
once.Line 9 gets executed n times ,and lines 11 and 12 get executed n-1 times each.Line 8 has
an s/e of 2,line 12 has an s/e of 2 and line 13 has s/e of 0.The remaining lines that get
executed have s/e‘s of 1 The total steps for the case n>1 is therefore 4n+1.

Example 1 Sum of n numbers
Algorithm with count statements added
Algorithm sum(a,n)
{ S:=0.0;
Count := count+1;//count is global; it is initially zero.
For i:=1 to n do
{ Count :=count + 1;//For for
S:= s+a[i];count:=count+1;//For assignment
Count :=count+1;//For last time of for
Count := count+1;//For the return
Return s;
Simplified version of algorithm
Algorithm Sum(a,n)
{ For i:=1 to n do count:= count+2;
Count:= count+3;}
Complexity calculation
tRSum(n)=2+tRSum(n-1) =2+2+tRSum(n-2)
=2(2)+tRSum(n-2)
=n(2)+tRSum(0) =2n+2, n>=0
Example 2 :Complexity of Fibonacci series

Algorithm Fibonacci(n)
//compute the nth Fibonacci number
If(n<=1) then
Write (n);
else
Fnm2:=0; fnm1:= 1;
for i:=2 to n do
fn:= fnm1+fnm2;
fnm2:=fnm1;fnm1:=fn;
Write (fn);
To analyse the time complexity of this algorithm, we need to consider the two cases (1) n=0
or 1 and(2) n>1.When n=0 or 1 , lines 4 and 5 get executed once each. Since each line has an
s/e of 1, the total step count for this case is 2.When n>1, lines 4,8 and 14 are each executed
once. Line 9 gets executed n times and lines 11 and 12 get executed n-1 times each.Line 8 has
an s/e of 2, line 12 has an s/e of 2,and line 13 has an s/e of 0. The remaning lines that get
executed have s/e of 1. The total steps for the case n>1 is therefore 4n+1.

ASYMPTOTIC NOTATION
Introduction
A problem may have numerous algorithmic solutions. In order to choose the best algorithm
for a particular task, you need to be able to judge how long a particular solution will take to
run. Or, more accurately, you need to be able to judge how long two solutions will take to
run, and choose the better of the two. You don't need to know how many minutes and
seconds they will take, but you do need some way to compare algorithms against one another.
Asymptotic complexity is a way of expressing the main component of the cost of an algorithm,
using idealized units of computational work. Consider, for example, the algorithm for sorting
a deck of cards, which proceeds by repeatedly searching through the deck for the lowest card.
The asymptotic complexity of this algorithm is the square of the number of cards in the deck.
This quadratic behavior is the main term in the complexity formula, it says, e.g., if you
double the size of the deck, then the work is roughly quadrupled.
The exact formula for the cost is more complex, and contains more details than are needed to
understand the essential complexity of the algorithm. With our deck of cards, in the worst
case, the deck would start out reverse-sorted, so our scans would have to go all the way to the
end. The first scan would involve scanning 52 cards, the next would take 51, etc. So the cost
formula is 52 + 51 + ... + 1. generally, letting N be the number of cards, the formula is 1 + 2
+ ... + N, which equals (N + 1) * (N / 2) = (N2 + N) / 2 = (1 / 2)N2 + N / 2. But the N^2 term
dominates the expression, and this is what is key for comparing algorithm costs. (This is in
fact an expensive algorithm; the best sorting algorithms run in sub-quadratic time.)
Asymptotically speaking, in the limit as N tends towards infinity, 1 + 2 + ... + N gets closer
and closer to the pure quadratic function (1/2) N^2. And what difference does the constant
factor of 1/2 make, at this level of abstraction. So the behavior is said to be O(n2).
Now let us consider how we would go about comparing the complexity of two algorithms.
Let f(n) be the cost, in the worst case, of one algorithm, expressed as a function of the input
size n, and g(n) be the cost function for the other algorithm. E.g., for sorting algorithms, f(10)
and g(10) would be the maximum number of steps that the algorithms would take on a list of
10 items. If, for all values of n >= 0, f(n) is less than or equal to g(n), then the algorithm with

complexity function f is strictly faster. But, generally speaking, our concern for
computational cost is for the cases with large inputs; so the comparison of f(n) and g(n) for
small values of n is less significant than the "long term" comparison of f(n) and g(n), for n
larger than some threshold.
Note that we have been speaking about bounds on the performance of algorithms, rather than
giving exact speeds. The actual number of steps required to sort our deck of cards (with our
naive quadratic algorithm) will depend upon the order in which the cards begin. The actual
time to perform each of our steps will depend upon our processor speed, the condition of our
processor cache, etc., etc. It's all very complicated in the concrete details, and moreover not
relevant to the essence of the algorithm.
BIG-O NOTATION
Definition
Big-O is the formal method of expressing the upper bound of an algorithm's running time. It's
a measure of the longest amount of time it could possibly take for the algorithm to complete.
More formally, for non-negative functions, f(n) and g(n), if there exists an integer n0 and a
constant c > 0 such that for all integers n > n0, f(n) ≤ cg(n), then f(n) is Big O of g(n). This is
denoted as "f(n) = O(g(n))". If graphed, g(n) serves as an upper bound to the curve you are
analyzing, f(n).
O-Notation (Upper Bound)
This notation gives an upper bound for a function to within a constant factor. We
write f(n) = O(g(n)) if there are positive constants n0 and c such that to the right of n0,
the value of f(n) always lies on or below cg(n).

Theory Examples
So, let's take an example of Big-O. Say that f(n) = 2n + 8, and g(n) = n2. Can we find a
constant c, so that 2n + 8 <= n2? The number 4 works here, giving us 16 <= 16. For any
number c greater than 4, this will still work. Since we're trying to generalize this for large
values of n, and small values (1, 2, 3) aren't that important, we can say that f(n) is generally
faster than g(n); that is, f(n) is bound by g(n), and will always be less than it.
It could then be said that f(n) runs in O(n2) time: "f-of-n runs in Big-O of n-squared time".
To find the upper bound - the Big-O time - assuming we know that f(n) is equal to (exactly)
2n + 8, we can take a few shortcuts. For example, we can remove all constants from the
runtime; eventually, at some value of c, they become irrelevant. This makes f(n) = 2n. Also,
for convenience of comparison, we remove constant multipliers; in this case, the 2. This
makes f(n) = n. It could also be said that f(n) runs in O(n) time; that lets us put a tighter
(closer) upper bound onto the estimate.
Practical Examples
O(n): printing a list of n items to the screen, looking at each item once. O(ln n): also "log n",
taking a list of items, cutting it in half repeatedly until there's only one item left. O(n2): taking
a list of n items, and comparing every item to every other item.

BIG-OMEGA NOTATION
For non-negative functions, f(n) and g(n), if there exists an integer n0 and a constant c > 0
such that for all integers n > n0, f(n) ≥ cg(n), then f(n) is omega of g(n). This is denoted as
"f(n) = Ω(g(n))".
This is almost the same definition as Big Oh, except that "f(n) ≥ cg(n)", this makes g(n) a
lower bound function, instead of an upper bound function. It describes the best that can
happen for a given data size.
Ω-Notation (Lower Bound)
This notation gives a lower bound for a function to within a constant factor. We write
f(n) = Ω(g(n)) if there are positive constants n0 and c such that to the right of n0, the
value of f(n) always lies on or above cg(n).
How asymptotic notation relates to analyzing complexity
Temporal comparison is not the only issue in algorithms. There are space issues as well.
Generally, a trade off between time and space is noticed in algorithms. Asymptotic notation
empowers you to make that trade off. If you think of the amount of time and space your
algorithm uses as a function of your data over time or space (time and space are usually
analyzed separately), you can analyze how the time and space is handled when you introduce
more data to your program.

This is important in data structures because you want a structure that behaves efficiently as
you increase the amount of data it handles. Keep in mind though that algorithms that are
efficient with large amounts of data are not always simple and efficient for small amounts of
data. So if you know you are working with only a small amount of data and you have
concerns for speed and code space, a trade off can be made for a function that does not
behave well for large amounts of data.
A few examples of asymptotic notation
Generally, we use asymptotic notation as a convenient way to examine what can happen in a
function in the worst case or in the best case. For example, if you want to write a function
that searches through an array of numbers and returns the smallest one:
function find-min(array a[1..n])

let j :=
for i := 1 to n:
j := min(j, a[i])
repeat
return j
end
Regardless of how big or small the array is, every time we run find-min, we have to initialize
the i and j integer variables and return j at the end. Therefore, we can just think of those parts
of the function as constant and ignore them.
So, how can we use asymptotic notation to discuss the find-min function? If we search
through an array with 87 elements, then the for loop iterates 87 times, even if the very first
element we hit turns out to be the minimum. Likewise, for n elements, the for loop iterates n
times. Therefore we say the function runs in time O(n).
What about this function:
function find-min-plus-max(array a[1..n])

// First, find the smallest element in the array
let j := ;
for i := 1 to n:
j := min(j, a[i])
repeat

let minim := j
// Now, find the biggest element, add it to the smallest and

j := ;
for i := 1 to n:
j := max(j, a[i])
repeat
let maxim := j
// return the sum of the two

return minim + maxim;
end
What's the running time for find-min-plus-max? There are two for loops, that each iterate n
times, so the running time is clearly O(2n). Because 2 is a constant, we throw it away and
write the running time as O(n). Why can you do this? If you recall the definition of Big-O
notation, the function whose bound you're testing can be multiplied by some constant. If
f(x) = 2x, we can see that if g(x) = x, then the Big-O condition holds. Thus O(2n) = O(n). This
rule is general for the various asymptotic notations.
THETA
Definition:
The function f(n)=theta(g(n))(read as ―f of n is theta of g of n‖) iff there exist

positive constants c1,c2, and n0 such that c1g(n) <=f(n)<=c2g(n) for all n, n>=n0.
Example:
The function 3n+2=theta(n) as 3n+2>=3n for all n>=2 and 3n+2<=4n for all
n>=2, so c1=3,c2=4 and n0=2. 3n+3= theta(n), 10n2+4n+2=theta(n2),6*2n+n2=theta(2n),
and 10* log n+4= theta (log n). 3n+2 !=theta(1), 3n+3 != theta(n2), 10n2+4n+2 !=theta(n),
10n2+4n+2 !=theta(1), 6*2n+n2 !=theta(n2), 6*2n +n2 !=theta(n100), and 6 * 2n +n2
!=theta(1).
The theta notation is more precise than both the big oh and big omega
notations. The function f(n)=theta(g(n)) iff g(n) is both lower and upper bound of f(n).

Little oh
Definition:
The function f(n)=0(g(n)) (read as ―f of n is little oh of g of n‖) iff
Lim f(n)/g(n)=0
n->infinity
Example:
The function 3n+2=o(n2) since lim n->infinity (3n+2)/n2=0. 3n+2=o(n log

n). 3n+2= o(n log log n). 6*2n+n2=o(3n). 6*2n+n2=o(2n log n). 3n+2 !=o(n). 6*2n+n2
!=o(2n).

Little omega
Definition:
The function f(n)=w(g(n)) (read as ―f of n is little omega of g of n‖) iff
Lim g(n)/f(n)=0
n->infinity
Example: Algorithm Sum(a,n)
{ S:=0.0;
for i=1 to n do
s:=s+a[i];
return s;
Alg 1. Iterative function for sum
For this algorithm Sum wec determined that tSum(n)=

2n+3. So, tSum(n)=theta(n)
Asymptotic Notation Properties
Let f(n) and g(n) be asymptotically positive functions. Prove or disprove each of the
following conjectures.

a. f(n)=O(g(n)) implies g(n)=O(f(n)).

b. f(n)+g(n)= (min(f(n),g(n))).
c. f(n)=O(g(n)) implies lg(f(n))=O(lg(g(n))),where lg(g(n))>=1 and f(n)>=1 for all
sufficiently large n.
d. f(n)=O(g(n)) implies 2^f(n)=O(2^g(n)).
e. f(n)=O((f(n))^2).
f. f(n)=O(g(n)) implies g(n) and Ω(f(n)).
g. f(n)= (f(n/2)).
h. f(n)+o(f(n))= (f(n)).
COMMON COMPLEXITY FUNCTIONS
Comparison of functions
Many of the relational properties of real numbers apply to asymptotic comparisons as well.
For the following, assume that f(n) and g(n) are asymptotically positive.
Transitivity:
f(n)= (g(n)) and g(n)= (h(n)) imply f(n)= (h(n)),
f(n)=O(g(n)) and g(n)=O(h(n)) imply f(n)=O(h(n)),
f(n)=Ω(g(n)) and g(n)=Ω(h(n)) imply f(n)=Ω(h(n)),
f(n)=o(g(n)) and g(n)=o(h(n)) imply f(n)=o(h(n)),
f(n)=ω(g(n)) and g(n)=ω(h(n)) imply f(n)=ω(h(n)).
Reflexivity:
f(n)= (f(n)),
f(n)=O(f(n)),
f(n)=Ω(f(n)).
Symmetry:
f(n)= (g(n)) if and only if g(n)= (f(n)).

Transpose symmetry
f(n)=O(g(n)) if and only if g(n)=Ω(f(n)),
f(n)=O(g(n)) if and only if g(n)=Ω(f(n)),
Because these properties hold for asymptotic notations, one can draw an analogy between the
asymptotic comparison of two functions f and g and the comparison of two real numbers a
and b:
f(n)=O(g(n)) similar to a<=b,
f(n)=Ω(g(n)) similar to a>=b,
f(n)= (g(n)) similar to a=b,
f(n)=o(g(n)) similar to a<b,
f(n)=ω(g(n)) similar to a>b.
We say that f(n) is asymptotically smaller than g(n) if f(n)=o(g(n)), and f(n) is asymptotically
larger than g(n) if f(n)=ω(g(n)).
One property of real numbers, however, does not carry over to asymptotic notations:
Trichotomy: For any two real numbers a and b, exactly one of the following must hold:
a<b, a=b, or a>b.
Although any two real numbers can be compared, not all functions are asymptotically
comparable. That is, for two functions f(n) and g(n), it may be the case that neither
f(n)=O(g(n)) nor f(n)=Ω(g(n)) holds. For example, the functions n and n^(1+sin n) cannot be
compared using asymptotic notation, since the value of the exponent in n^(1+sin n) oscillates
between 0 and 2, taking on all values in between.

RECURRENCE RELATIONS
The recurrence relation is an equation or inequality that describes a function in terms of its
values of smaller inputs. The main tool for analyzing the time efficiency of a recurrence
algorithm is to setup a sum expressing the number executions of its basic operation and
ascertain the solution‘s order of growth
To solve a recurrence relation means ,to obtain a function defined on natural numbers that
satisfies the recurrence.
Examble: tn = 2tn-1
T(n)= 3T(n/4)+(θn²)
Different methods for solving recurrence relation are:-
Substitution method
Itration method
Changing variables method
Recurrence tree
Characteristic equation method
n^α method
Master theorem
Solving recurrences by Substitution method
Idea: Make a guess for the form of the solution and prove by induction.
Can be used to prove both upper bounds O() and lower bounds Ω().
Let‘s solve T(n) = 2T(n/2) + n using substitution
– Guess T(n) ≤ cn log n for some constant c (that is, T(n) = O(n log n))
– Proof:
 Base case: we need to show that our guess holds for some base case (not
necessarily
n = 1, some small n is ok). Ok, since function constant for small constant n.
 Assume holds for n/2: T(n/2) ≤ c n∕2 log n∕2
Prove that holds for n: T(n) ≤cn log n
T(n)= 2T(n/2)+n

≤ 2(c n/2 logn/2) + n
= cn log n/2 +n
=cn log n-cn log 2+ n
=cn log n-cn + n
So ok if c ≥ 1
Similarly it can be shown that T(n) =Ω (n log n)
Similarly it can be shown that T(n) = T([n/2]) + T([n/2]) + n is θ(n lg n).
The hard part of the substitution method is often to make a good guess.
Solving Recurrences with the Iteration

In the iteration method we iteratively ―unfold‖ the recurrence until we ―see the
pattern‖.
The iteration method does not require making a good guess like the substitution
method (but
it is often more involved than using induction).
Example: Solve T(n) = 8T(n/2) + n² (T(1) = 1)
T(n) = n² + 8T(n/2)
= n² + 8(8T( n/2² ) + (n/2)²)
= n² + 8²T( n/2 ²) + 8(n²/4))
= n²+ 2n² + 8²T( n/2² )
= n² + 2n² + 8²(8T( n/2³ ) + ( n/2² )²)
= n² + 2n² + 8³T( n/2³ ) + 8²(n²/4² ))
= n² + 2n² + 2²n²+ 8³T( n/2³ )
=...
= n² + 2n² + 2²n²+ 2²n³ + . . .
Masters theorem
The master method is used for solving the following type of recurrence
T(n)=aT(n/b)+f(n),a 1 and b>1.In above recurrence the problem is divided in to ‗a‘

subproblems each of size atmost ‗n/b‘.The subproblems are solved recursively each in T(n/b)

time.The cost of split the problem or combine the solutions of subproblems is given by
function f(n).It should be note that the number of leaves in the recursion tree is where
E=
THEOREM
Let T(n) be defined on the non-negative integers by the recurrence

T(n)=aT(n/b)+f(n) where a 1,b>1 are constants and f(n) be a function.Then T(n) can be
asymtotically as,
CASE 1:
If f(n) O( for some e>0,then T(n)€ ( )
CASE 2:
If f(n) € ( ) then T(n) € (f(n).log n)
CASE 3:
If f(n) € Ω( for some e>0 and f(n) € O( for some e then T(n) € (f(n))

RECURRENCE PROBLEMS
Case 1 problem:
1. T(n)=4T(n/2)+n
E=log 4/log2=2
O(
O( =n
T(n) € ( )
Case 2 problem:
2.T(n)=4T(n/2)+
a=4, b=2
f(n)=
E=log 4/log 2=2
( )
T(n) € ( .log n)
Case 3 problem:
3.T(n)=4T(n/2)+
a=4, b=2
f(n)=
E=log 4/log 2=2
Ω( )
= Ω( )
= Ω( )
O( )
= O( )
= O( ), T(n) € ( )

RECURSION TREES FOR COMPLEXITY CALCULATIONS

A different way to look at the iteration method: is the recursion-tree
we draw out the recursion tree with cost of single call in each node—running time is
sum of
costs in all nodes
if you are careful drawing the recursion tree and summing up the costs, the recursion
tree is
a direct proof for the solution of the recurrence, just like iteration and substitution
Example: T(n) = 8T(n/2) + n² (T(1) = 1)
T(n)=n²+ 2n²+ 2²n²+ 2³n²+ .....+2^(log n-1) n²+8^(log n)

Changing variables
Sometimes reucurrences can be reduced to simpler ones by changing variables
Example: Solve T(n) = 2T(√n) + log n
Let m = log 2^m = n √n = 2^m/2

T(n) = 2T(√n) + log n ) T(2^m) = 2T(2^m/2) + m
Let S(m) = T(2^m)

T(2^m) = 2T(2^m/2) + m ) S(m) = 2S(m/2) + m)
S(m) = O(mlogm))
T(n) = T(2^m) = S(m) = O(mlogm) = O(log n log log n)
Other recurrences
Some important/typical bounds on recurrences not covered by master method:
Logarithmic: θ(log n)
– Recurrence: T(n) = 1 + T(n/2)
– Typical example: Recurse on half the input (and throw half away)
– Variations: T(n) = 1 + T(99n/100)
Linear: θ(N)
– Recurrence: T(n) = 1 + T(n − 1)
– Typical example: Single loop
– Variations: T(n) = 1+2T(n/2), T(n) = n+T(n/2), T(n) = T(n/5)+T(7n/10+6)+n
Quadratic: θ(n2)
– Recurrence: T(n) = n + T(n − 1)
– Typical example: Nested loops
Exponential: θ(2^n)
– Recurrence: T(n) = 2T(n − 1)

PROFILING
Profiling or performance measurement is the process of executing a correct program on data

set and measuring the time and space it takes to compute the result.
NONDETERMINISTIC ALGORITHMS
The notation of algorithm that we have been using has the property that the result of every
operation is uniquely defined.Algoithm with this property are termed deterministic
algorithms.
Such algorithms agree with the programs are executed on computer. In a theoretical
framework we can remove this rustication on the outcomes of every operation. We can allow
algorithms to be containing operations whose outcomes of every operation. We can allow
algorithm to contain operations whose outcomes are not uniquely defined but are limited to
specified set of possibilities.
The machine executing such operations is allowed to choose any one of these outcomes
subject to be a defined later. This leads to the concept of nondeterministic algorithms, we
introduce three functions.
1. Choice (S) arbitrarily chooses one of the elements of element of set S.
2. Failure () signals an unsuccessful completion.
3. Success () signals a successful completion.

The assignment statement x=Choice (1, n) could result in x being assigned any
one of the integers in the range [1,n].there is the rule of specifying how this choice is to be
made. The Failure () and Success () signals are used to define a computation of the algorithm.
These statements cannot be used to effect a return. Whenever there is a set of choices that
leads to a successful completion, then one such set of choices is always made and the
algorithm terminates successfully. A nondeterministic algorithm terminates unsuccessfully if
and only if there exists no set choices leading to a success signal. The computing times for.
Choice, Failure, and Success are taken to be O(1).A machine capable of executing a non

nondeterministic algorithm in this way is called a nondeterministic machine. Although

nondeterministic machines do not exist in practice, we see that they provide strong intuitive
reasons to conclude that certain problems cannot be solved by fast deterministic algorithms.
Example
Consider the problem of searching for an element x in a given set of elements A[1:n],n>1.We
are required to determine an index j such that A[j]=x or j=0 if x is not in A.
1. J:=Choice(1.n);
2. If A[j]=x then {write (j);Success();}
3. Write (0); Failure();
From the way a nondeterministic computation is defined, it follows that no is 0 can be the
output if and only if there is no j such that A[j]=x.
Complexity of nondeterministic search algorithms =O(1).
Since A is not ordered, every deterministic algorithm is of complexity Ω (n).
DETERMINISTIC ALGOITHM for n>p

The deterministic algorithm for a selection whose run time is O(n/p log log p+ log
n)The basic idea of this algorithm is same to sequential algorithm.
The sequential algorithm partitions the input into groups(of size,says,5),finds the median of
each group, and output recursively the median(call it M)of these group meadians.Then the
rank
rM of M in the input is computed, and as a result, all element from the input that are either
≤ M or >Mare dropped, depending on whether i >rM or i ≤ rM,
Respectively. Finally an appropriate selection is performed from the remaining keys

recursively.
We showed that run time of this algorithm was O(n).

MODULE 2

DIVIDE AND CONQUER
Given a function to compute on n inputs the divide and conquer strategy suggests splitting the
inputs into k distinct subsets,1<k<n,yielding k subproblems. These subproblems must be
solved,and then a method must be found to combine subsolutions into a solution of whole. If
the subproblems are still relatively large,then the divide and conquer strategy can possibly
reapplied.The principle is natrually expressed using recursive algorithm.that is smaller and
smaller subproblems of the same kind are generated untill eventually subproblems that are
small enough to be solved without splitting are produced.
CONTROL ABSTRACTION
Control abstraction means a procedures whose flowof control is clear but whose
primary operations are specified by other procedures whose precise meanings are left
defined.Below algorithmDAndC is initially invoked as DandC(p),where p is the problem to
be solved.
Small(p) is a boolean-valued function that determines whether the input size is small enough
that the answer can be computed without splitting.If this is so,the function S is
invoked.Otherwise the problem p is divided into smaller subproblems.These subproblems
P1,P2,----,Pkare solved by recursive applications of DandC.Combine is a function that
determines the solutions to P using the solutions to the k subproblems.
ALGORITHM
Algorithm DAndC(p)
{ if small(p) then return s(p);
else
{ divide p into smaller instances p1,p2,p3-----p(k),k>=1;
apply DandC to each of these subproblems;
return combine(DandC(p1),DandC(p2),-----,DandC(pk));

FINDING THE MAXIMUM AND MINIMUM
This is a simple problem that can be solved by the divide and-conquer

technique. The problem is to find the maximum and minimum items in a set of ‗n ‗elements.
ALGORITHM-1
Algorithm StraightMaxMin (a, n, max, min)
//Set ‗max’ to the maximum and ‗min‘ to the minimum of a[1:n];
{
max:=min:=a[1];
for i:=2 to n do
{
if (a[i]>max) then max:= a[i];
if a[i]<min) then min:= a[i];
}
}
This is a straight forward algorithm to accomplish the above problem. In
analyzing the time complexity of this algorithm, we concentrate on the number of element
comparisons. The frequency count for other operations in this algorithm is of the same order
as that for element comparisons. StraightMaxMin requires 2(n-1) element comparisons in
the best, average, and worst cases.
The best case occurs when the elements are in increasing order. The number
of element comparisons is (n-1). The worst case occurs when the elements are in the
decreasing order. In this case, the number of element comparisons is 2(n-1).
The average number of element comparisons is less than 2(n-1). On the
average, a[i] is greater than max, half the time, and so the average number of comparisons is
3n/2-1. The maximum and minimum are a[i] if n=1. If n=2, the problem can be solved by
making one comparison.

ALGORITHM-2
Algorithm MaxMin ( i, j, max, min )
//a[1:n] is a global array. Parameters I and j are integers, 1<=i<=j<=n. The effect is to set
//max and min to the largest and smallest values in a [i : j] ,respectively.
{ if (i=j) then max :=min :=a[i]; // Small(p)
else if (i=j-1) then //Another case of Small(p)
{ if( a[i] < a[j]) then
{ max:=a[j];
min:=a[i];
}
else
{ max:=a[i];
min:=a[j];
}
}
else
{
// If P is not small, divide P into subproblems. Find where to split
// mid := [(i+j)/2];
// Solve the subproblems.
MaxMin ( i, mid, max, min )
MaxMin ( mid+1, j, max, min)
if (max<max1)then max:=max1;
if (min>min1)then min:=min1;
}
}
MaxMin is a recursive algorithm that finds the maximum and minimum of
the set of elements {a(i),a(i+1),…………….a(j)}.The situation of set sizes one (i=j) and two
(i=j-1) are handled separately. For sets containing more than two elements, the midpoint is
determined (just as in binary search) and two new subproblems are generated. When the
maxima and minima of these subproblems are generated, the two maxima are compared and
the two minima are compared to achieve the solution for the entire set.

Eg:- Suppose we simulate MaxMin on the following nine elements:

a: [1] [2] [3] [4] [5] [6] [7] [8] [9]
22 13 -5 -8 15 60 17 31 47
A good way of keeping track of recursive calls is to build a tree by adding a
node each time anew call is made. For this algorithm each node has four items of
information: i, j, max, and min. On the array a[] above, the tree is produced.
Trees of recursive calls of MaxMin
1, 9,60,-8
1, 5, 22, -8 6,9,60,17
1, 3, 22, -5 4, 5, 15,-8 8, 9, 47, 31

6,7,60,17
1,2,22,13
3,3,-5,-5
We see that the root node contains 1 and 9 as the values of i and j corresponding to
the initial call to MaxMin. This execution produces new call to MaxMin where i and j have
the values 1 ,5 and 6,9, respectively, and thus split the set into two subsets of approximately
the same size. From the tree, we can immediately see that the maximum depth of recursion is
four(including the first call).
The order in which max and min are assigned values are follows: [1,2,22,13] [3,3,-5,-5]
[1,3,22,-5] [4,5,15,-8] [1,5,22,-8] [6,7,60,17] [8,9,47,31] [6,9,60,17] [1,9,60,-8].
Number of element comparisons needed for MaxMin:
If T(n) represents the number of element comparisons needed, then the
resulting recurrence relation is:
T ([n/2]) + T ([n/2]) +2 n>2

T (n) = 1 n=2
0 n=1

When n is a power of 2, n=2k we can solve this equation by successive substitutions:

T(n) = 2T(n/2)+2
= 2(2T(n/4)+2) +2
= 4T(n/4)+ 4 + 2
.
.
= 2k-1T(2) +∑ 1< = i < = k-1 2
i
= 2k-1 + 2k -2
= 3n/2 - 2
When n is a power of 2, the number of comparisons in the best , average and worst case is
3n/2-2.
COMPLEXITY ANALYSIS OF FINDING MAXIMUM&MINIMUM
Consider the following nine elements to simulate MaxMin
22 13 -5 -8 15 60 17 31 47
Fig below shows the tree constructed for the algorithm. Each node has four items of
information: i, j, max and min.
9
1, 9, 60,-8
5 8
1, 5, 22,-8 6, 9, 60,17
3 1, 3, 22,-5 4, 5, 15,-8
4 6,67, 60,17 1, 9, 60,-8
7
1, 2, 22, 13 3, 3, -5,-5
1 2
Consider the total number of element comparison needed for MaxMin? If T(n) represents this
number, then the resulting recurrence relation is
T (┌ n/2 ┐) + T(┌ n/2 ┐) + 2 n>2
T (n)= 1 n=2
0 n=1

When n is a power of 2, n=2k for some positive integer k, then

T(n) = 2T( n/2 ) + 2
= 2(2T (n/4) +2) +2
= 4T (n/4) +4+2
:
:
= 2k-1T (2) +∑1<=i<=k-1 2i
= 2k-1+2k-2
=3n/2-2
Therefore 3n/2 is the best, average and worst case number of comparisons when n is a power
of two.
Consider the count when element comparisons have the same cost as comparisons between i
and j. Let C (n) be this number.
Assuming n=2k for some positive integer k, we get
C(n) = 2C(n/2) + 3 n>2

2 n=2
C (n) = 2C (n/2) + 3
=4 C (n/4) + 6 + 3
:
:
=2k-1C (2)+3∑0k-2 2i
=2k+3*2k-1-3
=5n/2-3
if comparisons between array elements are costlier than comparisons of integer variable, then
divide and conquer method is more efficient
In both cases mentioned above the best case, average case and worst case complexity is θ(n)

RECURSION TREE
DEFINITION:
A recursion tree is a tree that depicts the entire recursion process.
Consider the following recurrence relation.
T (n) =3T (n/4)+ θ(n2)

= 3T (n/4) + cn2
cn2
c(n/4)2 c(n/4)2 c(n/4)2

:
T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) :
: : : :
2 2 2
c(n/16) c(n/16) c(n/16) : : : :
: : : : : :
T(n/16) T(n/16) T(n/16) : : : : : :
: : : : : : : : :
T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)
Sub problem size at depth i=n/4i
Boundary condition (n=1) :=(n/4i)=1 ie i=log4n
Depth of the tree=log4n+1 (0, 1, 2,………logn)
Cost of each node at level i=c(n/4)i

No: of node at level log4n=3log4n
Cost of each node=3log4n.T(1)=θ(3log4n)=θ(nlog43)
Total cost=cn2+3/16cn2+ (3/16)2cn2+ (3/16)log4n-1cn2
= (3/16)I cn2+ θ(nlog43)+ θ(nlog43)

= (3/16)I cn2+ θ(nlog43)

= 1/(1-3/16)cn2+ θ(nlog43)
= 16/13cn2+ θ (nlog43)
= O (n2)
BINARY SEARCH ALGORITHMS
The Binary search requires an ordered list.
Iterative Algorithm
int find (const list, int target)
// pre: list is sorted in ascending order
//post: ITERATIVE binary search will return the index of the target element, else -1
{
int mid;
int first = 0;
int last = list.length( ) -1;
while ( first <= last )
{
mid = (first + last) / 2;
if ( list[mid] == target )
return mid;
if ( list[mid] > target )
last = mid - 1;
else first = mid + 1;
}
return -1;
}
Recursive Algorithm
int find (const list, int target, int first, int last)
// pre: list is sorted in ascending order
//post: RECURSIVE binary search will return the index of the target element, else -1

{
if (first > last)
return -1;
int mid = (first + last) / 2;

if (list[mid] == target)
return mid;
if (list[mid] < target)
return find(list, target, mid+1, last);
return find(list, target, first, mid-1);
}
Complexity Analysis of Binary Search
For binary search, storage is required for the n elements of the array plus the variables
low, high, mid and x.
A binary search tree can be created to explain the binary search where the mid value
of each node is the value of the node. For example, if n=14, the resultant tree will be like the
one shown below.
7
3 11
1 5 9 13
2 4 6 8 10 12 14

The first comparison is s with a[7].if x<a[7], then the next comparison is with a[3];
similarly, if x>a[7], then the next comparison is with a[11]. If x is present, then the algorithm
will end at one of the circular nodes tat lists the index into the array where x was found. If x
is not present the algorithm will terminate at one of the square nodes.
Theorem:
If n is in the range [2k-1,2k), then binary search makes at most k element comparison
for a successful search and either k-1 or k comparison for an unsuccessful search. In other
words the time for a successful search is O (log n) and for an unsuccessful search is ϴ (log n).
E=I+2n
Where E=External path length (sum of the distance of all external nodes from the root)
I=Internal path length (sum of the distance of all internal nodes from the root).
Let As (n) is the average number of comparison in a successful search and Au (n) is
the average number of comparison in an unsuccessful search.
The number of comparison to find an element represented by an internal node is one
more than the distance of this node from the root.
As (n) =1+I/n
The number of comparisons on any path from the root to an external node is equal to
the distance between the root and the external node. Since every binary tree with n internal
nodes has n+1 external nodes
Au (n) =E/ (n+1)
As (n) =I/n+1
= (E-2n/n) + 1
=E/n-1
= (Au (n). (n+1))/n -1
= (n. Au (n) + Au (n))/n -1
= Au (n) + Au (n)/n -1
As (n) = Au (n) (1 +1/n) -1

Successful searches
Best-ϴ (1) (only one element is compared)
Average- ϴ (log n)
Worst- ϴ (log n)
Unsuccessful searches
Best, Average and Worst- ϴ (log n)
DIVIDE-AND-CONQUER MATRIX MULTIPLICATION ALGORITHM
The product of two n x n matrices X and Y is a third , n x n matrix Z=XY , with (i,j)th entry
n
Zij=∑Xik Ykj
k=1
In general XY, not the same as YX; matrix multiplication is not commutative.
The formula above implies an O (n3) algorithm for matrix multiplication: there are n2 entries
to be computed, and each takes linear time. For quite a while, this was widely believed
to be the best running time possible, and it was even proved that no algorithm which used
just additions and multiplications could do better. It was therefore a source of great
excitement when in 1969, Strassen announced a signi_cantly more ef_cient algorithm, based
upon divide-and-conquer.
Matrix multiplication is particularly easy to break into subproblems, because it can be
performed blockwise. To see what this means, carve X into four n/2 x n/2 blocks, and also Y:
A B E F
X= Y=
C D , G H
Then their product can be expressed in terms of these blocks, and is exactly as if the blocks
were single elements.
A B E F AE+BG AF+BH
XY= =

C D G H CE+DG CF+DH
We now have a divide-and-conquer strategy: to compute the size-n, product XY,

recursively compute eight size-n/2 products AE,BG,AF,BH,CE,DG,CF,DH and then do a
few O(n2) time additions. The total running time is described by the recurrence relation
T(n)= 8T(n/2) + O(n2),
which comes out to O(n3), the same as for the default algorithm. However, an improvement
in the time bound is possible, and as with integer multiplication, it relies upon algebraic
tricks. It turns out that XY can be computed from just seven sub problems.
P1 = A(F – H)
=
P2 (A + B)H
=
P3 (C + D)E
=
P4 D(G – E)
=
P5 (A + D) (E + H)
=
P6 (B - D) (G + H)
=
P7 (A - C) (E + F)
P +P – P +P P +P
XY = 5 4 2 6 1 2
P +P P +P – P – P
3 4 1 5 3 7
This translates into a running time of
T(n)= 7T(n/2) + O(n2),
which by the result is O(nlog2 7 )≈ O(n2.81).

STRESSEN’S MATRIX MULTIPLICATION
Strassen showed that 2x2 matrix multiplication can be accomplished in 7 multiplication and
18 additions or subtractions. This reduce can be done by Divide and Conquer
Approach.Divide the input data S in two or more disjoint subsets S1, S2. Solve the
subproblems recursively. Combine the solutions for S1, S2, …, into a solution for S. The base
case for the recursion are subproblems of constant size.Analysis can be done using recurrence
equations.Divide matrices in sub-matrices and recursively multiply sub-matrices
Let A, B be two square matrices over a ring R. We want to calculate the matrix product C as
If the matrices A, B are not of type 2n x 2n we fill the missing rows and columns with zeros.
We partition A, B and C into equally sized block matrices
with
then
With this construction we have not reduced the number of multiplications. We still need 8
multiplications to calculate the Ci,j matrices, the same number of multiplications we need
when using standard matrix multiplication.
Now comes the important part. We define new matrices

which are then used to express the Ci,j in terms of Mk. Because of our definition of the Mk we
can eliminate one matrix multiplication and reduce the number of multiplications to 7 (one
multiplication for each Mk) and express the Ci,j as
We iterate this division process n-times until the submatrices degenerate into numbers (group
elements).
Practical implementations of Strassen's algorithm switch to standard methods of matrix

multiplication for small enough submatrices, for which they are more efficient. The particular
crossover point for which Strassen's algorithm is more efficient depends on the specific
implementation and hardware. It has been estimated that Strassen's algorithm is faster for
matrices with widths from 32 to 128 for optimized implementations and 60,000 or more for
basic implementations.
Numerical analysis
The standard matrix multiplications takes approximately 2n3 arithmetic operations (additions
and multiplications); the asymptotic complexity is O(n3).

N
Ci , j ai ,k bk , j
k 1
N N N
Thus T ( N ) c cN 3 O( N 3 )
i 1 j 1 k 1
The number of additions and multiplications required in the Strassen algorithm can be
calculated as follows: let f(k) be the number of operations for a matrix. Then by
recursive application of the Strassen algorithm, we see that f(k) = 7f(k − 1) + l4k, for some
constant l that depends on the number of additions performed at each application of the
algorithm. Hence f(k) = (7 + o(1))k, i.e., the asymptotic complexity for multiplying matrices
of size n = 2k using the Strassen algorithm is
The reduction in the number of arithmetic operations however comes at the price of a
somewhat reduced numerical stability.
Algorithm
void matmul(int *A, int *B, int *R, int n)
if (n == 1)
(*R) += (*A) * (*B);
else
matmul(A, B, R, n/4);

matmul(A, B+(n/4), R+(n/4), n/4);
matmul(A+2*(n/4), B, R+2*(n/4), n/4);
matmul(A+2*(n/4), B+(n/4), R+3*(n/4), n/4);
matmul(A+(n/4), B+2*(n/4), R, n/4);
matmul(A+(n/4), B+3*(n/4), R+(n/4), n/4);
matmul(A+3*(n/4), B+2*(n/4), R+2*(n/4), n/4);
matmul(A+3*(n/4), B+3*(n/4), R+3*(n/4), n/4);

MERGE SORT
Merge sort is an example of divide and conquer algorithm. Its worst case complexity is
O(nlog n). To understand merge sort assume that elements are arranged in the non decreasing
order. Assume a sequence of n elements a[1],a[2],...a[n]. Split them into two sets like
a[1],a[2],..a[n/2] and a[(n/2)+1],...a[n]. Each set is individually sorted, and the resulting
sorted sequences are merged to produce a single sorted sequence of n elements. Merge-sort is
based on the divide-and-conquer paradigm.
The Merge-sort algorithm can be described in general terms as consisting of the following
three steps:
1. Divide Step: If given array A has zero or one element,it is already sorted. Otherwise,
divide A into two arrays, each containing about half of the elements of A.
2. Recursion Step: Recursively sort array A1 and A2.
3. Conquer Step: Combine the elements back in A by merging the sorted arrays A1 and A2
into a sorted sequence.
Consider the following example:
Consider the array of ten elements a [1:10] = (310,285,179,652,351,423,861,254,450,520)
The merge sort algorithm first divides the array as follows
a[1:5] and a[6:10] and then
a[1:3] and a[4:5] and a[6:8] and a[9:10] and then into
a[1:2] and a[3:3] and a[6:7] and a[8:8] and finally it will look like
a[1:1],a[2:2],a[3:3],a[4:4],a[5:5],a[6:6],a[7:7],a[8:8],a[9:9],a[10:10]
Pictorially the file can now be viewed as
(310|285|179|652,351|423,861,254,450,520)
Elements a[1] and a[2] are merged as
(285,310|179|652,351|423,861,254,450,520)

And then a[3] is merged with a[1:2]
(179,285, 310 |652,351|423,861,254,450,520) next a[4] and a[5]
(179,285, 310 |351,652|423,861,254,450,520)and then a[1:3] and a[4:5]
(179,285, 310,351,652|423,861,254,450,520)
Repeated recursive calls are invoked producing the following array
(179,285, 310 ,351,652|254,423, 450,520,861) and finally
(179,254,285,310, 351,423, 450,520,652,861)
At this point there are two sorted subarrays and final merge produces the fully sorted result
1, 10
1, 5 6, 10
1, 3 4, 5 6, 8 9, 10
1, 2 3, 3 4, 4 5, 5 6, 7 8, 8 9, 9 10, 10
1, 1 2, 2 6, 6 7, 7
If the time for the merging operation is proportional to n, then the computing time for merge
sort is described by the recurrence relation
a n= 1a a constant

T(n)=
2T(n/2)+cn n>1, c a constant
When n is a power of 2, n=2k we can solve this equation by successive substitutions:
T(n) = 2(2T(n4)+cn/2)+cn
= 4T(n/4)+2cn
= 4(2T(n/8)+cn/4) + 2cn
= 2kT(1) + kcn
= an+cn log n
It is easy to see that if 2k < n <2k+1, then T(n)<=T(2k+1). Therefore T(n)=O (nlogn).
24
MergeSort describes the process very succinctly using recursion and a function Merge which
merges two sorted sets. Before executing MergeSort , the n elements should be placed in
a[1:n]. Then MergeSort (1,n) causes the keys to be rearranged into nondecreasing order in a.
Algorithm MergeSort(low,high)
//a[low:high] is a global array to be sorted.
//Small(P) is true if there is only one element to sort.
//In this case the list is already sorted.
if(low<high) then // If there are more than one element.
//Divide P into subproblems.

//Find where to split the set .
mid:=[(low+high)/2];
//Solve the subproblems.
MergeSort(low,high);
MergeSort(mid+1,high);
//Combine the solutions.
Merge(low,mid,high);
Algorithm Merge(low,mid,high)
//a[low:high] is a global array containing two sorted subsets in
//a[low:mid] and in a[mid+1:high].The goal is to merge these two
// sets into single set residing in a[low:high].b[] is an auxiliary global
//array.
h:=low; i:=low; j:=mid+1;
while ((h ≤ mid) and (j ≥ high)) do
if (a[h] ≤ a[j]) then
b[i]:=a[h]; h:=h+1;
else

b[i]:=a[j]; j:=j+1;
i:=i+1;
if(h>mid) then
for k:=j to high do
b[i]:=a[k]; i:=i+1;
else
for k:=h to mid do
b[i]:=a[k]; i:=i+1;
for k:=low to high do a[k]:=b[k];

COMPLEXITY ANALYSIS OF MERGE SORT
Merge sort is an ideal example of the divide and conquer strategy in which the given set of
elements is split into two equal-sized sets and the combining operation is the merging of two
sorted sets into one.
Given a sequence of n elements a[1],.....,a[n], the general idea is to imagine them to split into
two sets a[1],...a[n/2] and a[n/2+1],...a[n].Each set is individually sorted and the resulting
sorted sequences are merged to produce a single sorted sequence of n elements.
MergeSort describes this process using recursion and a function Merge which merges two
sorted sets.
Example:
Consider an array of ten elements a[1:10] = (310,285,179,652,351,423,861,254,450,520).

Algorithm MergeSort begins by splitting a[] into two subarrays each of size five (a[1:5] and
a[6:10]). The elements in a[1:5] are then split into two subarrays of size three (a[1:3]) and
two (a[4:5]). Then the items in a[1:3] are split into subarrays of size two (a[1:2]) and one
(a[3:3]). The two values in a[1:2] are split final time into one-element subarrays , and now the
merging begins. A record of the subarrays is implicitly maintained by the recursive
mechanism.
Pictorially the file can be viewed as
( 310 | 285 | 179 | 652 , 351 | 423 , 861 , 254 , 450 , 520 ) where vertical bars indicate the
boundaries of subarrays. Elements a[1] and a[2] are merged to yield
( 285 , 310 | 179 | 652 , 351 | 423 , 861 , 254 , 450 , 520 )
Then a[3] is merged with a[1:2] and
( 179 , 285 , 310 | 652 , 351 | 423 , 861 , 254 , 450 , 520 ) is produced. Next elements a[4]
and a[5] are merged:
( 179 , 285 , 310 | 351 , 652 | 423 , 861 , 254 , 450 , 520 )
And then a[1:3] and a[4:5]:
( 179 , 285 , 310 , 351 , 652 | 423 , 861 , 254 , 450 , 520 )

At this point the algorithm has returned to the first invocation of MergeSort and is about to
process the second recursive call. Repeated recursive calls are invoked producing the
following subarrays:
( 179 , 285 , 310 , 351 , 652 | 423 | 861 | 254 | 450 , 520 )
Elements a[6] and a[7] are merged. Then a[8] is merged with a[6:7]:
( 179 , 285 , 310 , 351 , 652 | 254 , 423 , 861 | 450 , 520 )
Next a[9] and a[10] are merged and then a[6:8] and a[9:10]:
( 179 , 285 , 310 , 351 , 652 | 254 , 423 , 450 , 520 , 861 )
At this point there are two sorted subarrays and the final merge produces the fully sorted
result
( 179 , 254 , 285 , 310 , 351 , 423 , 450 , 520 , 652 , 861 )
Figure (1.1) is a tree that represents the sequence of recursive calls that are produced by
MergeSort when it is applied to ten elements. The pair of values in each node are the values
of parameters low and high. The splitting continues until the sets containing a single element
are produced.
Figure (1.2) is a tree representing the calls to produce Merge by MergeSort. For example,
the node containing 1 , 2 and 3 represents the merging of a[1:2] with a[3].

1,10
1,5 6,10
1,3 4,5 6,8 9,10
1,2 3,3 4,4 5,5 6,7 8,8 9,9 10,10
1,1 2,2 6,6 7,7
Tree of calls of Merge
( Fig 1.1 )
1,1,2 6,6,7
1,2,3 4,4,5 6,7,8 9,9,10
1,3,5 6,8,10
1,5,10
Tree of calls of MergeSort(1,10)
( Fig 1.2 )

If the time for the merging operation is proportional to n, then the computing time for merge
sort is described by the recurrence relation
T(n) = { a n=1 , a a constant
{ 2T(n/2) + cn n>1 , c a constant
When n is a power of 2, n = 2^k, we can solve this equation by successive substitutions:
T(n) = 2(2T(n/4)+ cn/2) +cn
= 4T(n/4) + 2cn
= 4(2T(n/8) + cn/4) + 2cn
= 2^kT(1) + kcn
= an + cn logn
It is easy to see that if 2^k < n < 2^(k+1) , then T(n) < T(2^k+1).
T(n) = O(n logn)

QUICK SORT
Quicksort is one of the fastest and simplest sorting algorithms . It works recursively by a
divide-and-conquer strategy.
Idea
First, the sequence to be sorted a is partitioned into two parts, such that all elements of the
first part b are less than or equal to all elements of the second part c (divide). Then the two
parts are sorted separately by recursive application of the same procedure (conquer).
Recombination of the two parts yields the sorted sequence (combine). Figure 1 illustrates this
approach.
Figure 1: Quicksort(n)
The first step of the partition procedure is choosing a comparison element x. All elements of
the sequence that are less than x are placed in the first part, all elements greater than x are
placed in the second part. For elements equal to x it does not matter into which part they
come. In the following algorithm it may also happen that an element equal to x remains
between the two parts.

After partitioning the sequence, quicksort treats the two parts recursively by the same
procedure. The recursion ends whenever a part consists of one element only.
Program
The following Java program implements quicksort.
void quicksort (int[] a, int lo, int hi)

{
// lo is the lower index, hi is the upper index
// of the region of array a that is to be sorted
int i=lo, j=hi, h;
int x=a[(lo+hi)/2];
// partition
do
{
while (a[i]<x) i++;
while (a[j]>x) j--;
if (i<=j)
{
h=a[i]; a[i]=a[j]; a[j]=h;
i++; j--;
}
} while (i<=j);
// recursion
if (lo<j) quicksort(a, lo, j);
if (i<hi) quicksort(a, i, hi);
}
Analysis
The best-case behavior of the quicksort algorithm occurs when in each recursion step the
partitioning produces two parts of equal length. In order to sort n elements, in this case the

running time is in Θ(n log(n)). This is because the recursion depth is log(n) and on each level
there are n elements to be treated (Figure 2 a).
The worst case occurs when in each recursion step an unbalanced partitioning is produced,
namely that one part consists of only one element and the other part consists of the rest of the
elements (Figure 2 c). Then the recursion depth is n-1 and quicksort runs in time Θ(n2).
In the average case a partitioning as shown in Figure 2 b is to be expected.
The choice of the comparison element x determines which partition is achieved. Suppose that
the first element of the sequence is chosen as comparison element. This would lead to the
worst
case behavior of the algorithm when the sequence is initially sorted. Therefore, it is better to
choose the element in the middle of the sequence as comparison element.
Even better would it be to take the n/2-th greatest element of the sequence (the median). Then
the optimal partition is achieved. Actually, it is possible to compute the median in linear time
[AHU 74]. This variant of quicksort would run in time O(n log(n)) even in the worst case.
However, the beauty of quicksort lies in its simplicity. And it turns out that even in its simple
form quicksort runs in O(n log(n)) on the average. Moreover, the constant hidden in the O-
notation is small. Therefore, we trade this for the (rare) worst case behavior of Θ(n2).
Proposition: The time complexity of quicksort is in
Θ(n log(n)) In the average case and in

Θ(n2) In the worst case
Conclusions
Quicksort turns out to be the fastest sorting algorithm in practice. It has a time complexity of
Θ(n log(n)) on the average. However, in the (very rare) worst case quicksort is as slow as
Bubblesort, namely in Θ(n2). There are sorting algorithms with a time complexity of O(n
log(n)) even in the worst case, e.g. Heapsort and Mergesort. But on the average, these
algorithms are by a constant factor slower than quicksort.
It is possible to obtain a worst case complexity of O(n log(n)) with a variant of quicksort (by
choosing the median as comparison element). But this algorithm is on the average and in the

worst case by a constant factor slower than Heapsort or Mergesort; therefore, it is not
interesting in practice.
The quicksort algorithm was invented by C.A.R. Hoare in 1962. I learned about it from "The
C Programming Language, Second Edition" by Brian W. Kernighan and Dennis M. Ritchie
(page 87). I found the K&R code a bit hard to follow; I believe I've come upon a better way
of explaining the algorithm, using a few figures of speech.
The point of any sort routine is to take an array of objects and arrange the objects in some
kind of sequence. We might, for example, want to take this collection of numbers:
06 34 69 33 75 64 04 74 25 95 15 58 78 36 51 73 13 27
and put them into numerical order, with the smallest number first, like this:
04 06 13 15 25 27 33 34 36 51 58 64 69 73 74 75 78 95
The general approach of the quick sort is to select an element from the middle (or from close
to the middle) of an array and then put all other elements which are less than or equal to the
selected element to its left and all elements which are greater than the selected element to its
right. The selected element is called the "pivot element" because the other elements,
figuratively speaking, turn around it. This crude "sorting" around the pivot element yields
two sub-arrays: a left one and a right one. If, for example, we start with the original array
above and use "25" as the pivot element, our first sorting yields:
[ 04 06 15 13 ] 25 [ 64 34 74 69 95 33 58 78 36 51 73 75 27 ]
The next step is for quicksort to call itself to have the left and right sub-arrays sorted. Of
course, a "sub array" with zero elements or only one element is already in proper order and
does not need to be sorted. The interesting part of the quick sort is how it comes up with the
sub arrays. I will illustrate.
From our original array, take out the element in (or close to) the middle of the array for a
pivot element. This, figuratively speaking, leaves a hole in the array:

06 34 69 33 75 64 04 74 __ 95 15 58 78 36 51 73 13 27
p.e. = 25
Now "move the hole" to the first position in the array by moving the current occupant of that
position into the current hole's position:
__ 34 69 33 75 64 04 74 06 95 15 58 78 36 51 73 13 27
p.e. = 25
Now we get to work. We start with the first element to the right of the hole. If it is greater
than the pivot element, we do nothing. If it is less than or equal to the pivot element, then we
want to put the element in the hole and then re-establish the hole immediately to the right of
the moved element. The first element of our sample array that needs to be moved is "04":
+-----------------+
| |
V |
04 34 69 33 75 64 __ 74 06 95 15 58 78 36 51 73 13 27
+--------------+
| |
| V
04 _ 69 33 75 64 34 74 06 95 15 58 78 36 51 73 13 27
p.e. = 25
After we similarly process "06" we will have:

04 06 __ 33 75 64 34 74 69 95 15 58 78 36 51 73 13 27
p.e. = 25
At the end of the process, we have:

04 06 15 13 __ 64 34 74 69 95 33 58 78 36 51 73 75 27
p.e. = 25
As you can see, all elements to the left of the hole are less than the pivot element; all
elements to the right of the hole are greater than the pivot element. To finish up, we sort the
left sub-array then the right sub-array, then we stuff the pivot element back into the hole and
we are done.
Algorithm partition (a,m,p)
//Within a[m], a[m+1],…,a[p-1]the elements are rearranged in such
//a manner that if initially t=a[m], then after completion a[q] =t for
//some q between m and p-1,a[k]<=t form<=k<q, and a[k]>=t for
//q<k<p. q is returned. Set a[p]=.
v :=a[m]; I :=m; j :=p;
repeat
repeat
I:=i+1;
until ( a[i]>=v);
repeat
j:=j-1;
until(a[j]<=v);

if(i<j) then interchange (a,i,j);
}until(i>=j);
a[m]:=a[j]; a[j]:=v; return j;
Algorithm interchange (a,i,j)
// exchange a[i] with a[j].
p :=a[i];
a[i] :=a[j]; a[j] :=p;
In quick sort, the division into sub arrays is made so that the
sorted sub arrays do not need to be merged later.
This is accomplished by rearranging the elements in a[1:n] such that a[i]<=a[j] for all I
between 1 and m and all j between m+1 and n
for some m,1<=m<=n. Thus the elements in a[1:m] and a[m+1:n] can be independently
sorted. No merge is needed. The rearrangement of
the element is accomplished by picking some element of a[],say t =a[s],and then reordering
the other elements so that all
elements appearing before t in a[1:n] are less than or equal to t and all elements appearing
after t are greater than or equal to t. This is rearranging is referred to as partitioning.
Function Partition of Algorithm accomplishes an in-place

partitioning of the elements of a[m:p-1].It is assumed that a[p]>=a[m] and that a[m] is the
partitioning elements. If m=1 and p-1=n, then a[n+1] must be defined and must be greater
than or equal to all elements in a[1:n]. The assumption that a[m] is the partition element is
merely for convenience; other choices for the partitioning element than the first item in the
set are better in practice. The function Interchange (a, i, j) exchanges a[i] with a[j].

Algorithm Quick Sort (p,q)
//Sorts the elements a[p],……,a[q] which reside in the global array //a[1:n] into ascending
order; a[n+1] is considered to be defined and //must be >= all the elements in a[1:n].
if (p<q) then //If there are more than one element
// divide P into two subproblems.
j:=Partition(a,p,q+1);
//j is the position of the partitioning element.
// Solve the subproblems.
QuickSort (p,j-1);
Quick Sort(j+1,q);
// There is no need for combining solutions.
Using Hoare‘s clever method of partitioning a set of elements about a chosen element, we
can directly devise a divide-and-conquer method for completely sorting n elements.
Following a call to the function Partition, two sets S1 and S2 are produced. All elements in
S1 are less than or equal to the element in S2. Hence S1 and S2 can be sorted independently.
Each set is sorted by reusing the function Partition

Quicksort
Quicksort
Quicksort in action on a list of numbers. The horizontal lines are pivot values.
Class Sorting algorithm
Data structure Varies
Worst case performance Θ(n2)
Best case performance Θ(nlogn)
Average case performance Θ(nlogn) comparisons
Worst case space complexity Varies by implementation
Optimal Sometimes
Quicksort is a well-known sorting algorithm developed by C. A. R. Hoare that, on average,

makes Θ(nlogn) (big O notation) comparisons to sort n items. However, in the worst case, it
makes Θ(n2) comparisons. Typically, quicksort is significantly faster in practice than other
Θ(nlogn) algorithms, because its inner loop can be efficiently implemented on most
architectures, and in most real-world data, it is possible to make design choices which
minimize the probability of requiring quadratic time.

Quicksort is a comparison sort and, in efficient implementations, is not a stable sort.
Algorithm
Quicksort sorts by employing a divide and conquer strategy to divide a list into two sub-lists.
The steps are:
1. Pick an element, called a pivot, from the list.

2. Reorder the list so that all elements which are less than the pivot come before the
pivot and so that all elements greater than the pivot come after it (equal values can go
either way). After this partitioning, the pivot is in its final position. This is called the
partition operation.
3. Recursively sort the sub-list of lesser elements and the sub-list of greater elements.
The base case of the recursion are lists of size zero or one, which are always sorted.
In simple pseudocode, the algorithm might be expressed as this:
function quicksort(array)
var list less, greater
if length(array) ≤ 1
return array
select and remove a pivot value pivot from array
for each x in array
if x ≤ pivot then append x to less
else append x to greater
return concatenate(quicksort(less), pivot, quicksort(greater))
Notice that we only examine elements by comparing them to other elements. This makes
quicksort a comparison sort. This version is also a stable sort (assuming that the "for each"
method retrieves elements in original order, and the pivot selected is the last among those of
equal value).
The correctness of the partition algorithm is based on the following two arguments:

At each iteration, all the elements processed so far are in the desired position: before
the pivot if less than or equal to the pivot's value, after the pivot otherwise (loop
invariant).
Each iteration leaves one fewer element to be processed (loop variant).
The correctness of the overall algorithm follows from inductive reasoning: for zero or one
element, the algorithm leaves the data unchanged; for a larger data set it produces the
concatenation of two parts, elements less than or equal to the pivot and elements greater than
it, themselves sorted by the recursive hypothesis.
The disadvantage of the simple version above is that it requires Ω(n) extra storage space,
which is as bad as merge sort. The additional memory allocations required can also
drastically impact speed and cache performance in practical implementations. There is a more
complex version which uses an in-place partition algorithm and can achieve the complete sort
using O(nlogn) space use on average (for the call stack):
function partition(array, left, right, pivotIndex)

pivotValue := array[pivotIndex]
swap array[pivotIndex] and array[right] // Move pivot to end
storeIndex := left
for i from left to right − 1
if array[i] ≤ pivotValue
swap array[i] and array[storeIndex]
storeIndex := storeIndex + 1
swap array[storeIndex] and array[right] // Move pivot to its final place
return storeIndex
In-place partition in action on a small list. The boxed element is the pivot element, blue
elements are less or equal, and red elements are larger.
This is the in-place partition algorithm. It partitions the portion of the array between indexes
left and right, inclusively, by moving all elements less than or equal to a[pivotIndex] to the
beginning of the subarray, leaving all the greater elements following them. In the process it
also finds the final position for the pivot element, which it returns. It temporarily moves the
pivot element to the end of the subarray, so that it doesn't get in the way. Because it only uses
exchanges, the final list has the same elements as the original list. Notice that an element may
be exchanged multiple times before reaching its final place.

This form of the partition algorithm is not the original form; multiple variations can be found
in various textbooks, such as versions not having the storeIndex. However, this form is
probably the easiest to understand.
Once we have this, writing quicksort itself is easy:
procedure quicksort(array, left, right)

if right > left
select a pivot index (e.g. pivotIndex := left)
pivotNewIndex := partition(array, left, right, pivotIndex)
quicksort(array, left, pivotNewIndex - 1)
quicksort(array, pivotNewIndex + 1, right)
However, since partition reorders elements within a partition, this version of quicksort is not
a stable sort.
Formal analysis
From the initial description it's not obvious that quicksort takes Θ(nlogn) time on average. It's
not hard to see that the partition operation, which simply loops over the elements of the array
once, uses Θ(n) time. In versions that perform concatenation, this operation is also Θ(n).
In the best case, each time we perform a partition we divide the list into two nearly equal
pieces. This means each recursive call processes a list of half the size. Consequently, we can
make only logn nested calls before we reach a list of size 1. This means that the depth of the
call tree is Θ(logn). But no two calls at the same level of the call tree process the same part of
the original list; thus, each level of calls needs only Θ(n) time all together (each call has some
constant overhead, but since there are only Θ(n) calls at each level, this is subsumed in the
Θ(n) factor). The result is that the algorithm uses only Θ(nlogn) time.
An alternate approach is to set up a recurrence relation for the T(n) factor, the time needed to
sort a list of size n. Because a single quicksort call involves Θ(n) factor work plus two
recursive calls on lists of size n / 2 in the best case, the relation would be:
The master theorem tells us that T(n) = Θ(nlogn).

In fact, it's not necessary to divide the list this precisely; even if each pivot splits the elements
with 99% on one side and 1% on the other (or any other fixed fraction), the call depth is still
limited to 100logn, so the total running time is still Θ(nlogn).
In the worst case, however, the two sublists have size 1 and n − 1 (for example, if the array
consists of the same element by value), and the call tree becomes a linear chain of n nested
calls. The ith call does Θ(n − i) work, and . The recurrence relation is:
T(n) = Θ(n) + T(0) + T(n − 1) = O(n) + T(n − 1)
This is the same relation as for insertion sort and selection sort, and it solves to T(n) = Θ(n2).
Given knowledge of which comparisons are performed by the sort, there are adaptive
algorithms that are effective at generating worst-case input for quicksort on-the-fly,
regardless of the pivot selection strategy.
Randomized quicksort expected complexity
Randomized quicksort has the desirable property that it requires only Θ(nlogn) expected
time, regardless of the input.
Suppose we sort the list and then divide it into four parts. The two parts in the middle will
contain the best pivots; each of them is larger than at least 25% of the elements and smaller
than at least 25% of the elements. If we could consistently choose an element from these two
middle parts, we would only have to split the list at most 2log2n times before reaching lists of
size 1, yielding an Θ(nlogn) algorithm.
A random choice will only choose from these middle parts half the time. However, this is
good enough. Imagine that you are flipping a coin over and over until you get k heads.
Although this could take a long time, on average only 2k flips are required, and the chance
that you won't get k heads after 100k flips is highly improbable. By the same argument,
quicksort's recursion will terminate on average at a call depth of only 2(2log2n). But if its
average call depth is Θ(logn), and each level of the call tree processes at most n elements, the
total amount of work done on average is the product, Θ(nlogn). Note that the algorithm does
not have to verify that the pivot is in the middle half - if we hit it any constant fraction of the
times, that is enough for the desired complexity.

The outline of a formal proof of the O(nlogn) expected time complexity follows. Assume that
there are no duplicates as duplicates could be handled with linear time pre- and post-
processing, or considered cases easier than the analyzed. Choosing a pivot, uniformly at
random from 0 to n − 1, is then equivalent to choosing the size of one particular partition,
uniformly at random from 0 to n − 1. With this observation, the continuation of the proof is
analogous to the one given in the average complexity section.

Average complexity
Even if pivots aren't chosen randomly, quicksort still requires only Θ(nlogn) time over all
possible permutations of its input. Because this average is simply the sum of the times over
all permutations of the input divided by n factorial, it's equivalent to choosing a random
permutation of the input. When we do this, the pivot choices are essentially random, leading
to an algorithm with the same running time as randomized quicksort.
More precisely, the average number of comparisons over all permutations of the input
sequence can be estimated accurately by solving the recurrence relation:
Here, n − 1 is the number of comparisons the partition uses. Since the pivot is equally likely
to fall anywhere in the sorted list order, the sum is averaging over all possible splits.
This means that, on average, quicksort performs only about 39% worse than the ideal number
of comparisons, which is its best case. In this sense it is closer to the best case than the worst
case. This fast average runtime is another reason for quicksort's practical dominance over
other sorting algorithms.
Space complexity
The space used by quicksort depends on the version used.
Quicksort has a space complexity of Θ(logn), even in the worst case, when it is carefully
implemented such that
in-place partitioning is used. This requires Θ(1).

After partitioning, the partition with the fewest elements is (recursively) sorted first,
requiring at most Θ(logn) space. Then the other partition is sorted using tail recursion
or iteration.
The version of quicksort with in-place partitioning uses only constant additional space before
making any recursive call. However, if it has made Θ(logn) nested recursive calls, it needs to
store a constant amount of information from each of them. Since the best case makes at most
Θ(logn) nested recursive calls, it uses Θ(logn) space. The worst case makes Θ(n) nested
recursive calls, and so needs Θ(n) space; Sedgewick's improved version using tail recursion
requires Θ(logn) space in the worst case.

We are eliding a small detail here, however. If we consider sorting arbitrarily large lists, we
have to keep in mind that our variables like left and right can no longer be considered to
occupy constant space; it takes Θ(logn) bits to index into a list of n items. Because we have
variables like this in every stack frame, in reality quicksort requires Θ((logn)2) bits of space
in the best and average case and Θ(nlogn) space in the worst case. This isn't too terrible,
though, since if the list contains mostly distinct elements, the list itself will also occupy
Θ(nlogn) bits of space.
The not-in-place version of quicksort uses Θ(n) space before it even makes any recursive
calls. In the best case its space is still limited to Θ(n), because each level of the recursion uses
half as much space as the last, and
Its worst case is dismal, requiring
space, far more than the list itself. If the list elements are not themselves constant size, the
problem grows even larger; for example, if most of the list elements are distinct, each would
require about ΘO(logn) bits, leading to a best-case Θ(nlogn) and worst-case Θ(n2logn) space
requirement.
Variants
There are three variants of quicksort that are worth mentioning:
Balanced quicksort: choose a pivot likely to represent the middle of the values to be
sorted, and then follow the regular quicksort algorithm.
External quicksort: The same as regular quicksort except the pivot is replaced by a
buffer. First, read the M/2 first and last elements into the buffer and sort them. Read
the next element from the beginning or end to balance writing. If the next element is
less than the least of the buffer, write it to available space at the beginning. If greater
than the greatest, write it to the end. Otherwise write the greatest or least of the buffer,
and put the next element in the buffer. Keep the maximum lower and minimum upper
keys written to avoid resorting middle elements that are in order. When done, write
the buffer. Recursively sort the smaller partition, and loop to sort the remaining
partition.

Three-way radix quicksort (also called multikey quicksort): is a combination of radix

sort and quicksort. Pick an element from the array (the pivot) and consider the first
character (key) of the string (multikey). Partition the remaining elements into three
sets: those whose corresponding character is less than, equal to, and greater than the
pivot's character. Recursively sort the "less than" and "greater than" partitions on the
same character. Recursively sort the "equal to" partition by the next character (key).
Comparison with other sorting algorithms
Quicksort is a space-optimized version of the binary tree sort. Instead of inserting items
sequentially into an explicit tree, quicksort organizes them concurrently into a tree that is
implied by the recursive calls. The algorithms make exactly the same comparisons, but in a
different order.
The most direct competitor of quicksort is heapsort. Heapsort is typically somewhat slower
than quicksort, but the worst-case running time is always Θ(nlogn). Quicksort is usually
faster, though there remains the chance of worst case performance except in the introsort
variant. If it's known in advance that heapsort is going to be necessary, using it directly will
be faster than waiting for introsort to switch to it.
Quicksort also competes with mergesort, another recursive sort algorithm but with the benefit
of worst-case Θ(nlogn) running time. Mergesort is a stable sort, unlike quicksort and
heapsort, and can be easily adapted to operate on linked lists and very large lists stored on
slow-to-access media such as disk storage or network attached storage. Although quicksort
can be written to operate on linked lists, it will often suffer from poor pivot choices without
random access. The main disadvantage of mergesort is that, when operating on arrays, it
requires Θ(n) auxiliary space in the best case, whereas the variant of quicksort with in-place
partitioning and tail recursion uses only Θ(logn) space. (Note that when operating on linked
lists, mergesort only requires a small, constant amount of auxiliary storage.)

MODULE 3

THE GREEDY METHOD – CONTROL ABSTRACTION
The greedy method suggests that one can devise an algorithm that works in stages,
considering one input at a time. At each stage, a decision is made regarding whether a
particular input is in an optimal solution. This is done by considering the inputs in an order
determined by some selection procedure. If the inclusion of the next input into the partially
constructed optimal solution will result in an infeasible solution, then this input is not assed to
the partial solution. Otherwise, it is added. The selection procedure itself is based on some
optimization measure. The measure may be the objective function. In fact, several different
optimization measures may be plausible for a given problem. Most of these, however, will
result in algorithms that generate suboptimal solutions. This version of the greedy technique
is called the subset paradigm.
The function Select selects an input from a[ ] and removes it. The selected input‘s value is
assigned to x. Feasible is a Boolean-valued function that determines whether x can be
included into the solution vector. The function Union combines x with the solution and
updates the objective function. The function Greedy describes the essential way that a greedy
algorithm will look, once a particular problem is chosen and the functions Select, Feasible,
and Union are properly implemented.
Algorithm Greedy (a,n)
//a[1 : n] contains the n inputs.
{ Solution :=ø;// Initialize the solution.
for i:=1 to n do
{ x:=Select(a);
if Fesible(solution,x) then
solution := Union(solution,x);
return solution;

GENERAL KAPSACK PROBLEM
In the knapsack problem, we have n objects and a knapsack or a bag. Object

i has a weight wi and the knapsack has a capacity m. If a fraction xi , 0 ≤ xi ≤ 1, of object i
is placed into the knapsack, then a profit of pixi is earned. The objective is to obtain a filling
of the knapsack that maximizes the total profit earned. Since the knapsack capacity is
m, we require the total weight of all chosen objects to be at most
m. Formally, the problem can be stated as
maximize ∑ pixi
1≤ i≤n
Subject to ∑ wixi ≤ m
1≤i≤ n
And 0 ≤ xi ≤ 1, 1≤i≤ n
The profit and weight are positive numbers. This is done by using greedy method.
ALGORITHM
ALGORITHM greedyknapsack(m, n )
/ / p[ 1.. n] , w[ 1.. n]
/ / pi/ wi ≥ pi+1/ wi+1
/ / x[1 …. n] solution
{
for i=1 to n do
x[1] =0.0 ;
u=m ;
for i= 1 to n do
{
If ( w[i] > u) then break ;
x[i] = 1.0 ;
u = u – w[i] ;
}
If ( i ≤ n ) then x[i] = u/ w[i] ;
}

If p1 / w1 ≥ p2 /w2 ≥ ……………. ≥ pn / wn , then greedy knapsack generate an

Optimum solution to the given instant of knapsack problem. All optimum solutions
will fill the knapsack exactly complexity of knapsack algorithm is 0( n).
Greedy algorithms
Similarly to dynamic programming, greedy algorithms are used to solve optimization
problems. In contrast to dynamic programming, no global optimal solution is computed, but
rather locally optimal decisions are taken. That is why the approach is called greedy.
Example: Knapsack Problem I

2 possible greedy-strategies: select the most valuable element in each step that still fits
into the knapsack: Knapsack capacity: 15
Object Size Value
g1 3 3
g2 4 5
g3 6 8
g4 7 9
choose: g4; value: 9;
remaining capacity: 8
choose: g4; value: 9;
remaining capacity: 1
y not optimal
Example: Knapsack Problem II

select the relatively most valuable element (max( v(gi )s(gi ) ) )
This will lead to the optimal solution in the case described above. However, this need not
always be the case!
Knapsack capacity: 50
Object Size Value vs
g1 30 90 3
g2 40 100 2,5
g3 50 110 2,2
The strategy will choose
g1 (value 90).

However, the optimal would be g3 (value 110) Greedy algorithms do not always provide the
optimal solution. In exchange, they are far less expensive than dynamic
programming. Complexity of the greedy-knapsack: O(C)
Optimal greedy algorithms
There are problems where greedy algorithms produce optimal solutions. Example: resource
planning problem. Given: set of activities S = {1, . . . , n} that want to use some resource, for
example a lecture hall. The resource can only be used by one activity at a time. Each activity i
has a starting time si and a completion time fi with si < fi . The activity takes place in the time
interval [si , fi ), i.e., the next activity can start at time fi . Compatibility of activities:
compatible(i , j) :, si _ fj _ sj _ fi . Problem: compute the largest possible set of compatible
Activitie
OPTIMAL STORAGE ON TAPES
Optimal storage on tapes I
Given: n programs P1, . . . , Pn with lengths l1, . . . , ln.
The programs are stored on a tape in order Pi1 , . . . , Pin .
Time to access Program Pij :
T(i1, . . . , in, j) = li1 + · · · + lij .
Average access time:
T(i1, . . . , in) = 1n
Xn
j=1
T(i1, . . . , in, j)
= 1n
Xn
j=1
Xj
i=1
li j
= 1n
Xn
j=1
(n − j + 1)lij
Optimal storage on tapes II
How to store the programs on the tape so that the average access time becomes minimal?

Let n be fixed; hence 1n can be ignored.

Example: Let P1, P2, P3 have lengths 17, 5, 10.
order acess time
P1P2P3 17 + (17 + 5) + (17 + 5 + 10) = 71
P1P3P2 17 + (17 + 10) + (17 + 10 + 5) = 76
P2P1P3 59
P3P1P2 69
P2P3P1 52
P3P2P1 52
Optimal storage on tapes III

Fact:
If for the lenghts l1, . . . , ln the relation l1 _ l2 _ · · · _ ln holds, then the average access time is
minimized by the order P1 . . . Pn. Hence, a greedy strategy that always selects the shortest
program leads to the minimal average access time.
What have we learnt on greedy algorithms?
Greedy algorithms treat optimization problems by taking locally optimal decisions. Hence,
they may fail to determine the global optimum. Greedy algorithms need less time than, e.g.,
dynamic programming. There are problems where a greedy strategy leads to an optimal
solution.
SPANNING TREES-MINIMUM COST SPANNING TREES
Definition: Let G= (V, E) be an undirected connected graph. A graph t= (V, E‘) of G is a

spanning tree of G if T is a tree.
Spanning trees have many applications. For example, they can be used to obtain
independent set of circuit equation for an electric network.
Another application of spanning trees arises from the property that a spanning tree is
a minimal subgraph G‘ of G such that V(G‘) =V(G) and G‘ is connected. A minimal
subgraph is one with fewest numbers of edges. Any connected graph with n vertices must
have least n-1 edges and all connected graphs with n-1 edges are trees. If the nodes of G
represent cities and the edges represent possible communication links connecting two cities,

then the minimum number of links needed to connect the cities is n-1. The spanning trees of
G represent all feasible choices.
In practical situations, the edges have weights assigned to them. These weights may
represent the cost of construction, the length of the link, and so on. Given such weighted
graph, one would then wish to select cities to have minimum total cost or minimum total
length. In either case the links selected have to form a tree. If this is not so, then selection of
links contain cycle. Removal of any of the links on this cycle results in a link selection of less
cost connecting all the cities. So that the minimum cost spanning tree of G can be obtained.
The cost of the spanning tree is the sum of all the edges in that tree.
1 28
8
2
10
14 16
7 3
6
24
18
25 12
5
4
22
(a)

2
10
14 16
7 3
6
25 12
5
4
22
(b)
Figure (a) represents a graph and (b) represents the minimum cost spanning tree.
PRIM’S ALGORITHM
At first a peak is chosen in random order ,which for simplicity we accept it as V(1).This way
two sets of pointers are initialized ,the 0={1} and P={2...n}.
The O set (the O is taken from the Greek word Oristiko which means Terminal), will always
contain the pointers of those peaks which are terminally attached in the T tree. The V(1) peak
has already been attached in the T tree. The P set( P is taken from the Greek word Prosorino
which means Temporary) contains the rest of the pointers for the peaks, P={1...n}-O which
are those pointers who have not been terminally connected with a node of T, that means they
are not attached in the tree.
In every execution of the Prim Algorithm a new peak will be connected to the T tree, not
always with their numbering order, for example the V(4) peak can be connected to the tree
before the V(2) peak. The corresponding pointer of the newly connected peak will be deleted

from P set and will be inserted to the O set. When all peaks are connected there will be
O={1,...n} and P=0.This of course means the end of the algorithm.
The new peak every time will be chosen by using greedy method ,among all sides of G which
connect peaks already inserted in the T (pointers in the O set ) tree with the rest of the peaks
(pointers in the P set ),we choose one with minimum cost. If the chosen one is e(ij) then i
belongs in the O set , V(i) peak is already in the T tree, j belongs in the P set , and V(j) peak
has not been attached in the T tree yet. We put V(j) in the T tree, we change the O set by
putting the j pointer, and we also change the P set by removing the j pointer.
This may seem to you extremely complicated but it is easily understood by a set of examples.
Pseudocode For The Prim Algorithm.
INPUT :n,c[e(ij)],i,j belonging to {1,...,n}.

OUTPUT :p(j) j=2,...,n (pointer of peaks j father in the T tree).
STEPS
1. :(initializations).
O={1} (V(1) root of the T tree).
P={2,...,n}
For every j belonging to P :e(j):=c[e(j1)] , p(j)=1
( all peaks connected to the root. By definition of the cost function :e(j)=infinite
when V(j) does not connect to V(1).).
2. Choose a k for which e(k)<=e(j) for every j belonging to P In case of tight choose the
smaller one.
Exchange the O set with the set produced by the union of the O set and {k} . Exchange
the P set with the set produced by the difference of the P set and {k} .(P<-P-{k}) If
P=0 then stop.
3. For every j belonging to P compare e(j) with c[e(kj)].
If e(j) >c[e(kj)] exchange e(j) <-c(e(kj)).Go back to Step 1.

An example for Prim‘s algorithm
 shortest paths from v0 to all destinations

7 3
6
(a)
2
10
7 3
6

2
10
7 3
6
12
5
(c)
2
10
14
7 3
6
12
5
(d)

2
10
14 16
7 3
6
12
5
(e)
2
10
14 16
7 3
6
12
5
4
22
(f)

Figure (a) shows the current graph with no edges selected. Edge (1, 6) is the first edge considered. It
is included in the spanning tree being built. This yields the graph of fig (b). Next edge (3, 4) is
selected and included in the tree (fig(c)). The next edge to be considered is (2, 7). Its inclusion in tree
is being built does not create a cycle, so we get the graph of fig (d). Edge (2, 3) is considered next and
included in the home fig (e). Of the edges not yet considered, (7, 4) has the least cost. It is considered
next. Its inclusion in the tree results in a cycle, so this edge is discarded. Edge (5,4) is next edge to be
added to the tree being built. This results in the configuration of fig (f). The next edge to be
considered is (7, 5). It is discarded as it creates a cycle. Finally, edge (6, 5) is considered and included
in the tree being built. This completes the spanning tree. The resulting tree has the cost 99.
KRUSKAL’S ALGORITHM
ALGORITHM
ALGORITHM Kruskal ( E, cost, n, t)
/ / E is the set of edges in G. G has n vertices. Cost [u, v] is the cost of
/ / edge (u, v). t is the set of edges in the minimum – cost spanning tree.
/ / the final cost is returned.
{
Construct a heap out of the edge costs using Heapify ;
for i := n do parent [i] := -1 ;
/ / Each vertex is in a different set.
i := 0 ; mincost := 0.0 ;
while ( i < n – 1 ) and ( heap not empty )) do
{
Delete a minimum cost edge (u, v) from the heap and
reheapify using Adjust ;
j := Find (u) ;
k := Find (v) ;
if (j ≠ k) then
{
i := i+1 ;
t[ i, 1] := u ; t [i, 2 ] := v ;
mincost := mincost + cost [u, v] ;

Union (j, k) ;
}
}
If ( i ≠ n – 1 ) then write (― no spanning tree‖) ;
else return mincost;
}
COMPLEXITY
The set t is the set of edges to be included in the minimum – cost spanning tree
and i is the number of edges in t. The set t can be represented as a sequential list using
a two – dimensional array t [ 1: n – 1, 1 : 2]. Edge (u, v) can be added to t by the
assignments t[ i, 1] := u ; and y[ i, 2] := v; . In the while loop of line 10, edges are
removed from the heap one by one in nondecreasing order of cost. Line 14 determines
the sets containing u and v. If j ≠ k then vertices u and v are in different sets and edge
(u, v) is included into t. The sets containing u and v are combined in line 20. If u = v
the edge (u, v) is discarded as its inclusions into t would create a cycle. Line 23
determines whether a spanning tree was found. It follows that i ≠ n – 1 iff the graph G
is not connected. The computing time is O( | E | log |E|), where e is the edge set of G.
JOB SEQUENCING WITH DEADLINES
If we are given a set of n jobs. Associated with job i is integer deadline di ≥0 and a profit
pi>0. For any job i the profit pi is earned if the job is completed by its deadline. To complete a
job one has to process the job on the machine for one unit of time. Only one machine is
available for processing jobs. A feasible solution for this problem is a subset J of such that
each job in this can be completed by its deadline. The value of a feasible solution J is the sum
of the profits of the jobs in J, or ∑i€J pi an optimal solution is a feasible solution with
maximum value.

Example:- Let n=4,(p1,p2,p3,p4)=(100,10,15,27) and (d1,d2,d3,d4)=(2,1,2,1). The feasible

solutions and their values are:
No Feasible solution Processing sequence Value

1. (1,2) 2,1 110
2. (1,3) 1,3 or 3,1 115
3. (1,4) 4,1 127
4. (2,3) 2,3 25
5. (3,4) 4,3 42
6. (1) 1 100
7. (2) 2 10
8. (3) 3 15
9. (4) 4 27
Solution 3 is optimal. In this solution only jobs 1 and 4 are processed and the value is 127.
These jobs must be processed in the order job 4 followed by job 1. Thus the processing of job
begins at time zero and that of job 1 is completed at time 2.
High level description of job sequencing algorithm
Algorithm Greedy job(d,J,n)
//J is a set of jobs that can be completed by their deadlines.
J:={1};
for i:=2 to n do
if(all jobs in J U {i} can be completed by their headlines) then J:=J U {i};

Greedy algorithm for sequencing unit time jobs with deadlines and profits
Algorithm JS(d,j,n)
//d[i]≥1, 1≤i≤n are the deadlines,n≥1. The jobs are ordered such that p[1]≥p[2]≥...≥p[n].
//J[i] is the ith job in the optimal solution,1≤i≤k.
//Also at termination d[J][i]]≤d[J[i+1]],1≤i≤k.
{ d[0] :=J[0]:=0;//initialize.
J[1]:=1;// Include job 1.
k:=1;
for i:=2 to n do
{ //consider jobs in nonincreasing order of p[i].
// Find position for i and check feasibility of insertion.
r:=k;
while((d[J[r]]>d[i]) and (d[J][r]]≠r)) do r:=r-1;
if((d[J[r]]≤d[i]) and (d[i]>r)) then
{ //Insert i into J[ ].
for q:=k to (r+1) step-1 do J[q+1]:=J[q];
J[r+1]:=i; k:=k+1;
return k;
The fuction JS is a correct implemention of the greedy-based method. Since d[i]≥1,the job
with largest pi will always be in the greedy solution. As the jobs are in the nonincreasing

order of pi‘s,line 6 in the above algorithm includes the job with largest pi. The for loop of the
line 8 considers the remaining job in the jobs in the order required by the greedy algorithm
method. At all times, the set of jobs already included in the solution is maintained in J. If
J[i],1≤i≤k, is the set already included,then J is such that d[J][i]]≤d[J[i+1]],1≤i≤k. When job i
is being considered, the while loop determines where in J this job has to be inserted.
The worst case computing time of Algorithm JS is θ(n2). The computing time of JS can be
reduced from θ(n2) to nearby O(n2) by using disjoint set union and find algorithms and a
different method to determine the feasibility of a partial solution.

MODULE 4

DYNAMIC PROGRAMMING
Dynamic programming is an algorithm design method that can be used when the solution to a
problem can be viewed as the result of a sequence of decisions.Below are some of the
examples of problems that can be viewed this way.
1)KNAPSACK-The solution to the knapsack problem can be viewed as the result of a

sequence of decisions.We have to decide the values of xi,1 ≤ i ≤ n.An optimal sequence of
decisions maximizes the objective function ∑pixi.It also satisfies the constraints ∑wixi≤m and
0 ≤ xi ≤1.
2)OPTIMAL MERGE PATTERNS-An optimal merge pattern tells us which pair of files
should be merged at each step.An optimal sequence of decisions is a least cost sequence.
3)SHORTEST PATH-One way to find a shortest path from vertex i to vertex j in a directed
graph G is to decide which vertex should be the second vertex,which the third and so on.An
optimal sequence of decisions is one that results in a path of least length.
For some of the problems that may be viewed in this way,an optimal sequence
of decisions can be found by making the decisions one at a time and never making an
erroneous decision.This is true for all problems solvable by the greedy method.For many
problems it is not possible to make stepwise decisions in such a manner that the sequence of
decisions made is optimal.
One way to solve problems for which it is not possible to make a sequence of
stepwise decisions leading to an optimal decision sequence is to try all possible decision
sequences.We could enumerate all decision sequences and then pick out the best.But the time
and space requirements may be prohibitive.Dynamic programming often drastically reduces
the amount of enumeration by avoiding the enumeration of some decision sequences that
cannot possibly be optimal.In dynamic programming an optimal sequence of decisions is
obtained by making explicit appeal to the principle of optimality.
PRINCIPLE OF OPTIMALITY
The principle of optimality states that an optimal sequence of decisions has the property that
whatever the initial state and decisions are,the remaining decisions must constitute an optimal
decision sequence with regard to the state resulting from the first decision.

Thus, the essential difference between the greedy method and dynamic
programming is that in the greedy method only one decision sequence is ever generated.In
dynamic programming ,many decision sequences may be generated.However,sequences
containing suboptimal subsequences cannot be optimal(if the principle of optimality
holds)and so will not be generated as far as possible.
Example:SHORTEST PATH
Consider the shortest path problem.Assume that i,i1,i2,.........ik,j is a shortest
path from i to j.Starting with the initial vertex i,a decision has been made to go to vertex
i1.Following this decision the problem state is defined by vertex i1 and we need to find a path
from i1 to j.It is clear that the sequence i1,i2,.......ik,j must constitute a shortest i1 to j path.If not
,let i1,r1,r2,........rq,j be a shortest i1 to j path.Then i,i1,r1,.......,rq,j is an i to j path that is shorter
than the path i,i1,i2,.......,ik,j.Therefore the principle of optimality applies for this problem.
Because of the use of principle of optimality , decision sequences containing

subsequences that are suboptimal are not considered. Although the total number of different
decision sequences is exponential in the number of decisions(if there are d choices for each of
the n decisions to be made then there are dn possible sequences),dynamic programming
algorithms often have a polynomial complexity.
Another important feature of dynamic programming approach is that optimal

solutions to subproblems are retained so as to avoid recomputing their values.The use of
these tabulated values make it natural to recast the recursive equations into an iterative
algorithm.
MULTISTAGE GRAPHS
EXPLANATION
Theory
A multistage graph G = (V, E) is a directed graph in which the vertices
are partitioned into k>=2 disjoint sets Vi, 1<=i<=k. If (u, v) is an edge in E, then u ε Vi and
v ε Vi+1 for some i, 1<=i<k. The sets V1 and Vk are such that | V1|=|Vk|=1. Let s and t be
the vertex in V1 and Vk. The vertex s is the source, and t the sink. Let c (i,j) be the cost of
edge (i, j).The cost of a path from s to t is the sum of the costs of the edges on the path.

The multistage graph problem is to find a minimum-cost path from s to t. Each set Vi defines
a stage in the graph. Because of the constraints on E, every path from s to t starts
in stage 1, goes to stage 2, then to stage 3, then stage 4, etc., and eventually terminates in
stage k.
A dynamic programming formulation for a k-stage graph problem is obtained
by using the principle of optimality. According to this principle a path (i, k) is said to be
optimal if and only if the intermediate paths (i, j) and (j, k) are also optimal.
The multistage graph problem has two approaches namely, forward and
backward . In forward approach the resulting path is obtained by moving in the direction of
destination from source, even though it appears in reverse. In the latter case it from
destination to source tracing the lowest cost path.
Equation
The cost of a path (i, j) is given by the equation
cost (i, j) = min {c (j, l) + cost (i+1, l)}

l ε Vi+1
(j, l) ε E
where j is a vertex in Vi.
Here cost (i, j) means i th level and the cost of edge from j th node to the last node.
EXAMPLE
Consider the following figure. The graph is a 5 stage graph, with 12 vertices and
the cost of each node is given. The aim is to find the minimum cost path from the first node
to the last node.
Figure
V1 V2 V3 V4 V5

4
2
6
2 6 9
9 1
5 4
7 3 2
7 4
2
1 3 12
3 7 10
7 00
4 11
7 0
5 5
2 11 6
8
8 11
5
Cost Calculation
cost (4, 9) =4
cost (4,10) =2
cost (4,11) =5
cost (3,6) =cost from 6 th to 12 th node = min { 5+ cost (4, 9),6+ cost (4, 10) }=7
cost (3,7) = min { 4+ cost (4, 9),3+ cost (4, 10) }=5
cost (3,8) = min { 5+ cost (4, 10),6+ cost (4, 11) }=7
cost (2,2) = min { 4+ cost (3, 6),2+ cost (3, 7), 1+ cost (3, 8) }=7
cost (2,3) = min { 2+ cost (3, 6),7+ cost (3, 7)}=9
cost (2,4) = 11+cost (3, 8)=18
cost (2,5) = min { 11+ cost (3, 7),8+ cost (3, 8)}=15
cost (1,1) = min { 9+ cost (2, 2),7+ cost (2,3), 11+ cost (2, 4) ,2+ cost (2, 5)}=16
By selecting the least cost edges from each group we can determine which all edges
will be involved in the minimum cost path.
Thus the minimum cost path in forward approach is (1, 2, 7, 10, 12) and in
backward approach it is (1, 3, 6, 10, 12).Both have a cost of 16 each.
FORWARD APPROACH
EXPLANATION:

A multistage graph G = (V, E) is a directed graph in which the vertices are partitioned into
k>=2 disjoint sets Vi, 1<=i<=k. If (u, v) is an edge in E, then u ε Vi and v ε Vi+1 for some i,
1<=i<k. The sets V1 and Vk are such that | V1|=|Vk|=1. Let s and t be the vertex in V1 and
Vk. The vertex s is the source, and t the sink. Let c (i,j) be the cost of edge (i, j).The cost of a
path from s to t is the sum of the costs of the edges on the path.
The multistage graph problem is to find a minimum-cost path from s to t. Each set Vi defines
a stage in the graph. Because of the constraints on E, every path from s to t starts
in stage 1, goes to stage 2, then to stage 3, then stage 4, etc., and eventually terminates in
stage k.
A dynamic programming formulation for a k-stage graph problem is obtained by using the
principle of optimality. According to this principle a path (i, k) is said to be optimal if and
only if the intermediate paths (i, j) and (j, k) are also optimal.
The multistage graph problem has two approaches namely, forward and backward . In
forward approach the resulting path is obtained by moving in the direction of destination
from source, even though it appears in reverse. In the latter case it from destination to source
tracing the lowest cost path.
By using the forward approach,we obtain the cost of a path (i, j) which is given by the
following equation:
cost (i, j) = min {c (j, l) + cost (i+1, l)}

l ε Vi+1
(j, l) ε E , where j is a vertex in Vi.
Here cost (i, j) means i th level and the cost of edge from j th node to the last node.
Consider the example given below. This is a 5 stage graph, with 12 vertices and the cost of
each node is given. The aim is to find the minimum cost path from the first node to the last
node. This can be done by using the forward approach as shown below:
EXAMPLE:

V1 V2 V3 V4 V5
4
2
6
2 6 9
9 1
5 4
7 3 2
7 4
2
1 3 12
3 7 10
2
7 00
4 11
7 0
5 5
2 11 6
8
8 11
5
CALCULATING THE MINIMUM COST:

cost (4, 9) = cost from 4th node to the 9th node = 4
cost (4,10) = 2
cost (4,11) =5
cost (3,6) = min { 6+ cost (4, 9), 6+ cost (4, 10) }= 7
cost (2,2) = min { 4+ cost (3, 6), 2+ cost (3, 7), 1+ cost (3, 8) }= 7
While calculating the value of cost (2,2) the values of cost (3,6), cost (3,7) and cost (3,8) have
been reused so as to avoid their re-computation.
cost (2,3) = min { 2+ cost (3, 6), 7+ cost (3, 7)}= 9
cost (2,4) = min{11+cost (3, 8)}= 18

cost (2,5) = min { 11+ cost (3, 7), 8+ cost (3, 8)}= 15
cost (1,1) = min { 9+ cost (2, 2), 7+ cost (2,3), 11+ cost (2, 4) , 2+ cost (2, 5)}= 16
A minimum cost s to t path has a cost of 16.

Thus, the cost of forward approach is= 16.
By selecting the least cost edges from each group we can determine which all edges will be
involved in the minimum cost path.
Thus the minimum cost path in forward approach is (1, 2, 7, 10, 12) which has the cost of 16.
It is represented using dotted lines.
This is the multistage graph forward approach problem.
BACKWARD APPROACH :
The Multistage graph problem can be solved using backward approach in the
following manner. Let bp(i,j) be a minimum-cost path from vertex s to a vertex j in Vi . let
bcost(i,j) be the cost of bp(i,j). From the backward approach we obtain
bcost(i,j) = min {bcost(i-1,l) + c(l,j)}

l Є Vi-1
<l,j> Є E
Algorithm in psedocode corresponding to obtain a minimum-cost s-t graph is

BGraph.(Algorithm 1.1) The first subscript on bcost, p and d are omitted for the same reasons
as before. This algorithm has the same complexity as FGraph provided G is now represented
by itsinverse adjacency lists (i.e., for each vertex v we have a list of vertices w such that
<w,v> Є E

ALGORITHM
1. Algorithm BGraph(G,k,n,p)
2. // Same function as FGraph
3. {
4. bcost[1] :=0.0; <r,j>
5. for j :=2 to n do
6. { / / Compute bcost[j].
7. Let r be such that is an edge of G is an edge of and bcost is an edge of
8. G and bcost[r] + c[r,j] is minimum;
9. bcost[j] :=bcost[r] + c[r,j];
10. d[j] := r;
11. }
12. / / Find a minimum-cost path.
13. p[1] := 1; p[k] :=n;
14. for j := k – 1 to 2 do p[j] := d[p[j+1]];
15. }
It should be easy to see that both FGraph and BGraph work correctly even on a more
generalized version of multistage graphs. In this generalization, the graph is permitted to have
edges <u,v> such that u Є Vi , v Є Vj and i< j.
In the pseudocodes FGraph and BGraph, bcost(i,j) is set to ∞ for any <i,j> Є E. When
programming these pseudocodes, one could use the maximum allowable floating point
number for ∞. If the weight of any such edge is added to some other costs, a floatin point
overflow might occur. Care should be taken to avoid such overflows.
Eg:
 d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
 d(C, T) = min{ 2+d(F, T) } = 2+2 = 4

 d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
 The above way of reasoning is called
backward reasoning.
9 D
d(D, T)
5 d(E, T)
B E T
d(F, T)
16
F
Backward approach
(forward reasoning)
 d(S, A) = 1
d(S, B) = 2
d(S, C) = 5
 d(S,D)=min{d(S, A)+d(A, D),d(S, B)+d(B, D)}
= min{ 1+4, 2+9 } = 5
d(S,E)=min{d(S, A)+d(A, E),d(S, B)+d(B, E)}
= min{ 1+11, 2+5 } = 7
d(S,F)=min{d(S, A)+d(A, F),d(S, B)+d(B, F)}
= min{ 2+16, 5+2 } = 7
 d(S,T) = min{d(S, D)+d(D, T),d(S,E)+d(E,T), d(S, F)+d(F, T)}
= min{ 5+18, 7+13, 7+2 }
=9

 Forward approach and backward approach:
 Note that if the recurrence relations are formulated using the forward approach
then the relations are solved backwards . i.e., beginning with the last decision
 On the other hand if the relations are formulated using the backward approach,
they are solved forwards.
Sample Graph and Simulation of the Algorithm
Stage Stage Stage Stage Stage Stage

I II III IV V VI
2 5 12
1 9
3 6 13 15
10
00
0
4 7 14
11
SIMULATION OF THE SOLUTION USING BACKWARD COSTS
Format: COST(stage, node) = minimum cost of traveling to the node in stage from the
source(node 1).
STEP I
COST(I,1) = 0
STEP II
COST(II,2) = COST(I,1) + cost(1,2) = 0 + 10 = 10

STEP III
COST(III,5) = min{COST(II,2) + cost(2,5),
COST(II,3) + cost(3,5),
COST(II,4) + cost(4,5)}
=min{10 +10, 20+40, 30+∞}
= 20---via the path 1-2-5
=min{10 +20, 20+∞, 30+40}
=min{10 +30, 20+50, 30+30}
STEP IV
COST(IV,8)=min{COST(III,5) + cost(5,8),
COST(III,6) + cost(6,8),
COST(III,7) + cost(7,8)}
=min{20+10, 20+∞ ,40+∞ }

=30---via the path 1-2-5-8
=min{20+20, 20+20 ,40+∞ }
=40---via the path 1-2-5-9 or via the path 1-3-6-9
=min{20+10, 20+∞ ,40+∞ }
=30---via the path 1-2-5-10
=min{20+30, 20+30 ,40+30 }
=50---via the path 1-2-5-11 or via the path 1-3-6-11
STEP V
COST(V,12)=min{COST(IV,8) + cost(8,12),
COST(IV,9) + cost(9,12),
COST(IV,10) + cost(10,12),
COST(IV,11) + cost(11,12)}
=min{30+10 ,40+∞ ,30+∞,50+∞}
= 40—via the path 1-2-5-8-12

COST(IV,10) + cost(10,13),
COST(IV,11) + cost(11,13)}
=min{30+20 ,40+20 ,30+10,50+10}
= 40—via the path 1-2-5-8-13
COST(IV,10) + cost(10,14),
COST(IV,11) + cost(11,14)}
=min{30+30 ,40+10 ,30+20,50+30}
=50---via the path 1-2-5-10-14 or via 1-2-5-9-14 or 1-3-6-9-14
STEP VI
COST(VI,15)=min{COST(V,12)+cost(12,15),
COST(V,13)+cost(13,15),
COST(V,14)+cost(14,15)}
=min{40+20 ,40+10 ,50+30}
=50---via the path 1-2-5-8-13-15

ALL PAIRS SHORTEST PATH
Finding all pairs shortest path consists of finding the shortest distance between every pair of
nodes in a possibly directed graph. Various means of doing so are known, and the following
list gives a few of the common methods:
Floyd-Warshall algorithm is an elegant, quickly implementable O(n3) algorithm

(Assumes absence of negatively-weighed cycles).
Johnson's algorithm is harder to implement, but might perform better for sparse
graphs.
Algorithms for single source problem might also be repeatedly used, although they
often perform worse or are harder to optimize
FLOYD-WARSHELL ALGORITHM
In computer science, the Floyd–Warshall algorithm (sometimes known as the WFI

Algorithm or Roy–Floyd algorithm, since Bernard Roy described this algorithm in 1959) is
a graph analysis algorithm for finding shortest paths in a weighted, directed graph. A single
execution of the algorithm will find the shortest paths between all pairs of vertices. The
Floyd–Warshall algorithm is an example of dynamic programming
Algorithm
The Floyd-Warshall algorithm compares all possible paths through the graph between each
pair of vertices. It is able to do this with only V3 comparisons. This is remarkable considering
that there may be up to V2 edges in the graph, and every combination of edges is tested. It
does so by incrementally improving an estimate on the shortest path between two vertices,
until the estimate is known to be optimal.
Consider a graph G with vertices V, each numbered 1 through N. Further consider a function
shortestPath(i,j,k) that returns the shortest possible path from i to j using only vertices 1
through k as intermediate points along the way. Now, given this function, our goal is to find
the shortest path from each i to each j using only nodes 1 through k + 1.
There are two candidates for this path: either the true shortest path only uses nodes in the set
(1...k); or there exists some path that goes from i to k + 1, then from k + 1 to j that is better.

We know that the best path from i to j that only uses nodes 1 through k is defined by
shortestPath(i,j,k), and it is clear that if there were a better path from i to k + 1 to j, then the
length of this path would be the concatenation of the shortest path from i to k + 1 (using
vertices in (1...k)) and the shortest path from k + 1 to j (also using vertices in (1...k)).
Therefore, we can define shortestPath(i,j,k) in terms of the following recursive formula
This formula is the heart of Floyd Warshall. The algorithm works by first computing
shortestPath(i,j,1) for all (i,j) pairs, then using that to find shortestPath(i,j,2) for all (i,j) pairs,
etc. This process continues until k=n, and we have found the shortest path for all (i,j) pairs
using any intermediate vertices.
Pseudocode
Conveniently, when calculating the kth case, one can overwrite the information saved from
the computation of k − 1. This means the algorithm uses quadratic memory. Be careful to
note the initialization conditions:
1 /* Assume a function edgeCost(i,j) which returns the cost of the edge from i to j
2 (infinity if there is none).
3 Also assume that n is the number of vertices and edgeCost(i,i)=0
4 */
5
6 int path[][];
7 /* A 2-dimensional matrix. At each step in the algorithm, path[i][j] is the shortest path
8 from i to j using intermediate vertices (1..k-1). Each path[i][j] is initialized to
9 edgeCost(i,j) or infinity if there is no edge between i and j.
10 */
11
12 procedure FloydWarshall ()
13 for k: = 1 to n

14 for each (i,j) in {1,..,n}2

15 path[i][j] = min ( path[i][j], path[i][k]+path[k][j] );
Analysis
To find all n2 of from those of requires 2n2 bit operations. Since we begin with
and compute the sequence of n zero-one matrices , , ..., ,
the total number of bit operations used is . Therefore, the complexity of
the algorithm is Θ(n3) and can be solved by a deterministic machine in polynomial time
Applications and generalizations
The Floyd–Warshall algorithm can be used to solve the following problems, among others:
Shortest paths in directed graphs (Floyd's algorithm).

Transitive closure of directed graphs (Warshall's algorithm). In Warshall's original
formulation of the algorithm, the graph is unweighted and represented by a Boolean
adjacency matrix. Then the addition operation is replaced by logical conjunction
(AND) and the minimum operation by logical disjunction (OR).
Finding a regular expression denoting the regular language accepted by a finite
automaton (Kleene's algorithm)
Inversion of real matrices (Gauss-Jordan algorithm).
Optimal routing. In this application one is interested in finding the path with the
maximum flow between two vertices. This means that, rather than taking minima as
in the pseudocode above, one instead takes maxima. The edge weights represent fixed
constraints on flow. Path weights represent bottlenecks; so the addition operation
above is replaced by the minimum operation.
Testing whether an undirected graph is bipartite.
Fast computation of Pathfinder Networks.
Maximum Bandwidth Paths in Flow Networks

JOHNSON’S ALGORITHM
Johnson's algorithm is a way to find shortest paths between all pairs of vertices in a sparse
directed graph. It allows some of the edge weights to be negative numbers, but no negative-
weight cycles may exist.
Johnson's algorithm consists of the following steps:
1. First, a new node q is added to the graph, connected by zero-weight edge to each other
node.
2. Second, the Bellman-Ford algorithm is used, starting from the new vertex q, to find
for each vertex v the least weight h(v) of a path from q to v. If this step detects a
negative cycle, the algorithm is terminated.
3. Next the edges of the original graph are reweighted using the values computed by the
Bellman-Ford algorithm: an edge from u to v, having length w(u,v), is given the new
length w(u,v) + h(u) −h(v).
4. Finally, for each node s, Dijkstra's algorithm is used to find the shortest paths from s
to each other vertex in the reweighted graph.
In the reweighted graph, all paths between a pair s and t of nodes have the same quantity
h(s) -h(t) added to them, so a path that is shortest in the original graph remains shortest in the
modified graph and vice versa. However, due to the way the values h(v) were computed, all
modified edge lengths are non-negative, ensuring the optimality of the paths found by
Dijkstra's algorithm. The distances in the original graph may be calculated from the distances
calculated by Dijkstra's algorithm in the reweighted graph by reversing the reweighting
transformation.
Analysis
The time complexity of this algorithm, using Fibonacci heaps in the implementation of
Dijkstra's algorithm, is O(V2log V + VE): the algorithm uses O(VE) time for the Bellman-Ford
stage of the algorithm, and O(V log V + E) for each of V instantiations of Dijkstra's algorithm.

Thus, when the graph is sparse, the total time can be faster than the Floyd-Warshall
algorithm, which solves the same problem in time O(V3).
Example
The first three stages of Johnson's algorithm are depicted in the illustration below.
The graph on the left of the illustration has two negative edges, but no negative cycles. At the
center is shown the new vertex q, a shortest path tree as computed by the Bellman-Ford
algorithm with q as starting vertex, and the values h(v) computed at each other node as the
length of the shortest path from q to that node. Note that these values are all non-positive,
because q has a length-zero edge to each vertex and the shortest path can be no longer than
that edge. On the right is shown the reweighted graph, formed by replacing each edge weight
w(u,v) by w(u,v) + h(u) −h(v). In this reweighted graph, all edge weights are non-negative,
but the shortest path between any two nodes uses the same sequence of edges as the shortest
path between the same two nodes in the original graph. The algorithm concludes by applying
Dijkstra's algorithm to each of the four starting nodes in the reweighted graph.

SHORTEST PATH PROBLEM
A graph with 6 vertices and 7 edges
In graph theory, the shortest path problem is the problem of finding a path between two
vertices (or nodes) such that the sum of the weights of its constituent edges is minimized. An
example is finding the quickest way to get from one location to another on a road map; in this
case, the vertices represent locations and the edges represent segments of road and are
weighted by the time needed to travel that segment.
Formally, given a weighted graph (that is, a set V of vertices, a set E of edges, and a real-
valued weight function f : E → R), and one element v of V, find a path P from v to each v' of
V so that
is minimal among all paths connecting v to v' .
The problem is also sometimes called the single-pair shortest path problem, to distinguish
it from the following generalizations:
The single-source shortest path problem, in which we have to find shortest paths
from a source vertex v to all other vertices in the graph.
The single-destination shortest path problem, in which we have to find shortest
paths from all vertices in the graph to a single destination vertex v. This can be
reduced to the single-source shortest path problem by reversing the edges in the
graph.

The all-pairs shortest path problem, in which we have to find shortest paths
between every pair of vertices v, v' in the graph.
These generalizations have significantly more efficient algorithms than the simplistic
approach of running a single-pair shortest path algorithm on all relevant pairs of vertices.
Algorithms
The most important algorithms for solving this problem are:
Dijkstra's algorithm solves the single-pair, single-source, and single-destination

shortest path problems.
Bellman-Ford algorithm solves single source problem if edge weights may be
negative.
A* search algorithm solves for single pair shortest path using heuristics to try to speed
up the search.
Floyd-Warshall algorithm solves all pairs shortest paths.
Johnson's algorithm solves all pairs shortest paths, and may be faster than Floyd-
Warshall on sparse graphs.
Perturbation theory finds (at worst) the locally shortest path.
Applications
Shortest path algorithms are applied to automatically find directions between physical
locations, such as driving directions on web mapping websites like Mapquest or Google
Maps.
If one represents a nondeterministic abstract machine as a graph where vertices describe

states and edges describe possible transitions, shortest path algorithms can be used to find an
optimal sequence of choices to reach a certain goal state, or to establish lower bounds on the
time needed to reach a given state. For example, if vertices represents the states of a puzzle
like a Rubik's Cube and each directed edge corresponds to a single move or turn, shortest
path algorithms can be used to find a solution that uses the minimum possible number of
moves.

In a networking or telecommunications mindset, this shortest path problem is sometimes

called the min-delay path problem and usually tied with a widest path problem. For example,
the algorithm may seek the shortest (min-delay) widest path, or widest shortest (min-delay)
path.
A more lighthearted application is the games of "six degrees of separation" that try to find the
shortest path in graphs like movie stars appearing in the same film.
Other applications include "operations research, plant and facility layout, robotics,
transportation, and VLSI design".
Related problems
For shortest path problems in computational geometry, see Euclidean shortest path.
The traveling salesman problem is the problem of finding the shortest path that goes through
every vertex exactly once, and returns to the start. Unlike the shortest path problem, this
problem is NP-complete and, as such, is believed not to be efficiently solvable (see P = NP
problem) . The problem of finding the longest path in a graph is also NP-complete.
The Canadian traveller problem and the stochastic shortest path problem are generalizations
where either the graph isn't completely known to the mover, changes over time, or where
actions (traversals) are probabilistic.
The problems of recalculation of shortest paths arises if some graph transformations (e.g.,
shrinkage of nodes) are made with a graph

LOWER BOUND THEORY
Our main task for each problem is to obtain a correct and efficient solution. If two algorithms
solving the same problem were discovered and their times differed by an order of magnitude,
then the one with the smaller order was generally regarded as superior. To establish a given
algorithm in the most efficient way, a function g(n) , that is a lower bound on the time, is
discovered. If we have an algorithm whose computing time is the same order as g(n), then we
know that asymptotically we can do no better.
There is a mathematical notation for expressing lower bounds. If f(n) is the time for some
algorithm, then we write f(n)=Ω(g(n)) to mean that g(n) is a lower bound for f(n). Formally
this equation can be written if there exist positive constants c and n0 such that |f(n)| ≥ c|g(n)|
for all n > n0. In addition to developing lower bounds to within a constant factor, we are also
concerned with determining more exact bounds.
Deriving good lower bounds is often more difficult than devising efficient algorithms.
Perhaps this is because a lower bound states a fact about all possible algorithms for solving a
problem. Usually we cannot enumerate and analyze all these algorithms, so lower bound
proofs are often hard to obtain.
However , for many problems it is possible to easily observe that a lower bound identical to n
exists, where n is the number of inputs( or possibly outputs) to the problem. For example,
consider all algorithms that find the maximum of an unordered set of n integers. Clearly
every integer must be examined at least once, so Ω(n) is a lower bound for any algorithm that
solves this problem. Or, suppose we wish to find an algorithm that efficiently multiplies two
n×n matrices. Then Ω(n2) is a lower bound on any such algorithm since there are 2n2 inputs
that must be examined and n2 outputs that must be computed. Bounds such as these are often
reffered to as trivial lower bounds because they are so easy to obtain. We know how to find
the maximum of n elements by an algorithm that uses only n-1 comparisons so there is no
gap between the upper and lower bounds for this problem. But for matrix multiplication the
best known algorithm requires O(n2+€) operations (€ > 0) , and so there is no reason to believe
that a better method cannot be found.

COMPARISON TREES FOR SEARCHING AND SORTING
A comparison tree for binary search on an array is a binary tree that depicts all possible
search paths.i.e it shows all different sequences of comparisons undertaken by binary search
when searching for keys that may or may not be present in the array.
X:A(1)
Failure
X: A(2)
Failure
X:A(n)
Failure Failure
Suppose that we are given a set S of distinct values on which an ordering relation < holds.
The sorting problem calls for determining a permutation of the integers 1 to n, say p(1) to
p(n), such that the n distinct values from S stored in A[1:n] satisfy A[p(1)] < A[p(2)] <. .. .<
A[p(n)]. The ordered searching problem asks whether a given element x E S occurs within
the elements in A[1:n], then we are to determine an I between 1 and n such that A[i] = x. The
merging problem assumes that two ordered sets of distinct inputs from S are given in A[1:m]
and B[1:n] such that A[1] < . . . < A[m] and B[1] < . . . B[n]; these m+n values are to be
rearranged into an array C[1:m+n] so that C[1] < . . . < C[m+n]. For all these problems we
restrict the class of algorithms we are considering to those which work solely by making
comparisons between elements. No arithmetic involving elements is permitted, though it is
possible for the algorithm to move elements around.Thes algorithms are referred to as

comparison based algorithms. We rule out algorithms such as radix sort that decompose the
values into subparts.
a) Ordered Searching
In obtaining the lower bound for the ordered searching problem, we consider
only those comparison based algorithms in which every comparison between two elements of
S is of the type ―compare x and A[i]‖. Each internal node in the binary tree represents a
comparison between x and an A[i], or x > A[i].If x=A[i],the algorithm terminates. The left
branch is taken if x < A[i], and the right is taken if x > A[i].If the algorithm terminates
following a left or right branch , then no I has been found such that x = A[i] and the algorithm
must declare the search successful.
b) Sorting
We can describe any sorting algorithm that satisfies the restrictions of the
comparison tree model by a binary tree. Consider the case in which the n numbers A[1:n] to
be sorted are distinct.Now,any comparison between A[i] and A[j] must result in one of two
possibilities: either A[i] < A[j] or A[i] > A[j].So, the comparison tree is a binary tree in which
each internal node is labeled by the pair i:j which represents the comparison of A[i] with
A[j].If A[i] < A[j] , then the algorithm proceeds down the left branch of the tree; otherwise
right branch. The external nodes represent termination of the algorithm. Associate with every
path from the root to an external node is a unique permutation.
c) Selection
Any comparison tree that models comparison-based algorithms for finding
the maximum of n elements has at least 2^ n-1 external nodes since each path from the root to
an external node must contain at least n-1 internal nodes. This implies at least n-1
comparisons for otherwise at least two of the input items never lose a comparison and the
largest is not yet found.
Comparison trees
comparison tree
n! leaves : every permutation must be a leaf
-case # comparisons = height of tree

a1a2a3 : a1< a2?
a1a2a3 : a2<a3 ? a2a1a3 : a1< a3?
a1a2a3 a1a3a2 : a1< a3? a2a1a3 a2a3a1 : a2< a3?
a1a3a2 a3a1a2 a2a3a1 a3a2a1

Comparison tree for insertion sort of three items
LOWER BOUND FOR COMPARISON-BASED SORTING
It is generally much more difficult to show that an algorithm is ‗‗best possible‘‘ for a certain
task, than to come up with a
‗‗good‘‘ algorithm for that task. In the latter case, one need only argue about one algorithm—
the one that is supposedly
‗‗good‘‘; in the former case, one must argue about all possible algorithms — even those
which one knows nothing about!
To answer the question ‗‗what‘s the best way to do a job‘‘ we must start by fixing the
set of tools that may be used to do the job. For the problem at hand, sorting, we will take
pair wise comparisons of keys as the available tools. Of course, this leaves out of
consideration
Bin sorting (which is not based on key comparisons). But, in a sense, we can justify
this on the ground that Bin sorting is not a ‗‗general‘‘ sorting method: it is applicable only
if keys have a particular form. At any rate, it‘s important to keep in mind that the result

we are about to present only applies to sorting algorithms based on comparisons.

Suppose we want to sort n keys K1 , K2 , . . . , Kn . Let‘s assume that all keys are
distinct, so that for any Ki , Kj where i j , either Ki < Kj or Ki > Kj. The main theoretical
device we‘ll use to analyze our problem is a decision (or comparison) tree. This is
a useful way of representing any comparison-based algorithm that sorts n keys (for any
given n). Before introducing the formal definition, let‘s develop some intuition about
decision trees by studying an example.
Example 1: Below is a decision tree that corresponds to one possible algorithm for sorting
3 keys, K1 , K2 , K3 .
1 1:2
<
>
2:3
< >
2:3
<
> 321
1:3
123
1:
3 < >
>
213 231
<
132
312

The internal nodes of this tree correspond to comparisons the algorithm makes. The labels in
those nodes, specify which keys are to be compared. For example the label "1:2" of the root
indicates that K1 is to be compared with K2. Depending on the outcome of the comparison in
a node, the left or the right branch out of that node is taken. The label of each leaf is the
permutation specifying how to rearrange the keys to get them sorted. For example, consider
the node with label "2,3,1". This means that K2<K3<K1. Note that the fact this is the order of
the three keys is implied by the outcomes of the comparisons made in the path from the root
to that leaf: To get to that leaf we must go from the root to the right (signifying that K1>K2);
then from node "2:3" to the left (signifying that K2<K3) and from node "1:3" to the right
(signifying that K1>K3). These three comparisons imply that K2<K3<K1. This is the case for
the path from the root to any -2- leaf.
Thus the decision tree specifies the sequence of comparisons that the algorithm will perform
to sort 3 keys. An execution of the algorithm for a specific input (of 3 keys) corresponds to a
path from root to a leaf. For example, suppose that the input keys are K c, K a, K3=b
(where a < b < c). The execution starts at the root of the tree. The node there specifies that K1
and K2 are to be compared. Since K1>K2, we take the right branch from the root and arrive at
the "2:3" node. This indicates we must compare K2 to K3 and since K2<K3, we take the left
branch and arrive at the "1:3" node. By comparing K1 and K3 we discover that K1>K3 and
therefore we take the right branch which leads us to the leaf labeled "2,3,1". This indicates
that the keys in sorted order are K2<K3<K1, as indeed is the case. A more programming
language-like description of the algorithm corresponding to the above decision tree is:
if K1<K2 then
if K2<K3 then sorted order of keys is K1 , K2 , K3
else {K2>K3}
else {K1>K3} sorted order of keys is K3 , K1 , K2
else {K1>K2}
if K2<K3 then
Using the intuition we have gained by studying this example, let us now give the formal

definition of a decision tree.

Definition 1: A decision tree of order n is a binary tree so that
(1) It has n! leaves, each labeled by a different permutation of 1, 2, . . . , n.
(2) Internal nodes are labeled by pairs of indices of the form ‗‗i : j i j n.
(3) In the path from the root to a leaf p (1) p (2) . . . p (n) (p is a permutation of
1, 2, . . . , n) there is either a node ‗‗p (i) : p (i — in which case the path goes
from that node to its left child — or ‗‗p (i i)‘‘ — in which case the path goes
from that node to its right child — i < n. (The path may contain other
nodes too but must contain at least these.)
The idea is that leaves correspond to the possible outcomes of sorting n distinct
keys. Internal node ‗‗i : j‘‘ corresponds to the comparison of Ki and Kj. If the outcome is
Ki < Kj then the left subtree of the node ‗‗i : j‘‘ contains the subsequent comparisons
made until the order of the keys is determined. Symmetrically, if the outcome is Ki > Kj ,
the right subtree of ‗‗i : j‘‘ contains the subsequent comparisons. Part (3) of the definition
essentially requires that every relationship determined by the algorithm must have been
established by actual comparisons: the algorithm cannot ‗‗guess‘‘.
Any comparison-based algorithm for sorting n keys corresponds to a decision tree of
order n. The execution of the algorithm on input sequence of keys K1, K2, . . . , Kn follows
the path from the root to the leaf labeled by the permutation p such that
Kp (1) < Kp (2) < . . . < Kp (n). Note that for a given sorting algorithm we need a different
-3-
decision tree for each n to represent all possible executions of that algorithm when the
input consists of n keys.
Example 2: Consider the algorithm for insertion sort:
INSERTION -SORT(A)
begin
for i := 2 to n do
for j := i down to 2 do
if A[ j] < A[ j 1] then A[ j A[ j 1]
else break
end

WORST CASE LOWER BOUND ON THE NUMBER OF

COMPARISONS
Let Cn denote the minimum number of comparisons required to sort n keys. For any decision
tree, the worst case number of comparisons required by the algorithm represented by that tree
is precisely the height of the tree.
Lemma: Any binary tree of height h has at most 2h leaves.
Proof: Trivial induction on h.
By definition, any decision tree of order n is a binary tree with n! leaves. Thus, by the
lemma, it must have height at least n Cn n
approximation
says that n n (n/e)n. Thus,
Cn n (n/e)n n/e)n = n log n n log e
from which it follows that Cn n log n).
AVERAGE CASE LOWER BOUND ON THE NUMBER OF

COMPARISONS
Consider a decision tree of order n representing some sorting algorithm. If we assume that all
initial arrangements of the keys to be sorted are equally likely, the average case number of
comparisons performed by that algorithm is equal to the external path length of the decision
tree divided by the number of leaves in the tree.† The following fact can be easily proved:
Fact 2: The tree that minimizes the external path length has all leaves in at most two
depths d and d 1, for some d.
By Facts 1 and 2 we may assume that in the decision tree that minimizes the average number
of comparisons, all leaves have depth d or d 1 for some d. Let Cn be the minimum average
case number of comparisons needed to sort n keys. Also, let Nd and Nd−1 be the number of
leaves in the corresponding decision tree at depth d and d 1 respectively.
Recall that the number of leaves in T is n!. Thus we have
Cn = [(d 1) Nd− d Nd ]/n! (1)
But we have:
Nd Nd−1 = n! (2)
and

Nd Nd−1 =2d (3)

Equation (2) says that the total number of leaves in T is n!. For equation (3) note that if
we gave 2 children to each leaf of depth d 1, we would get the maximum possible number
of nodes at depth d, which is 2d .
† Recall that the external path length of the tree is the sum of the depths of all leaves.
-5-
Solving (2) and (3) for Nd and Nd−1, we get:
Nd =2 n d (4)
and
Nd−1 =2d n! (5)
Substituting (4) and (5) in (1) we obtain:
Cn = [(d 1) Nd− d Nd ]/n!
= (d n n d )/n!
But d n = log n
Cn = (n! log n n n n! 2e )/n!
= log n
-
Cn n n(n/e)n n/e)n = n log n n log e.
Hence, Cn n log n).
Therefore, any comparison- n log n) comparisons
both in the worst and in the average case.
ORACLES AND ADVERSARY ARGUMENTS
One of the proof techniques that is useful for obtaining lower bounds consists of making use
of an oracle. The most famous oracle in the history was called the Delphic oracle, located in
Delphi in Greece. This oracle can still be found, situated in the side of a hill embedded in
some rocks. In olden times people would approach the oracle and ask it a question. After
some period of time elapsed, the oracle would reply and a caretaker would interpret the
oracle‘s answer.

A similar phenomenon takes place when we use an oracle to establish a lower bound. Given
some model of computation such as comparison trees, the oracle tells us the outcome of each
comparison. To derive a good lower bound, the oracle tries its best to cause the algorithm to
work as hard as it can. It does this by choosing as the outcome of the next test, the result that
causes the most work to be required to determine the final answer. And by keeping tracks of
the work that is done, a worst-case lower bound for the problem can be derived.
MERGING
Now we consider the merging problem. Given the sets A[l:m] and B[l:n], Where the item in
A and the item in B are sorted, we investigate lower bounds for algorithms that merge these
two sets to give a single sorted set. As was the case of sorting, we assume that all the m+n
elements are distinct and that A[1]<A[2]……..<A[m] and B[1]<B[2]…….<B[n]. It is
possible that after these two sets are merged, the n elements of B can be inter leaved within A
in every possible way. Elementary combinatorics tells us that there are ( ) ways that the
A‘s and B‘s can merge together while preserving the ordering within A and B. For example,
if m=3, n=2, A[i]=x,A[2]=y,A[3]=z,B[1]=u,B[2]=v, there are ( )=10 ways in which A and
B merge: u,v,x,y,z;u,x,v,y,z;u,x,y,v,z;u,x,y,z,u;x,u,v,y,z;x,u,y,v,z;x,u,y,z,v;x,y,u,v,z;x,y,u,z,v;
and x,y,u,v.
Thus if we use comparison trees as our model for merging algorithm, then there will be
external nodes, and therefore at least [log ] comparisons are required by any comparison-
based merging algorithm. The conventional merging algorithm takes m+n-1 comparisons. If
we let MERGE(m,n) be the minimum number of comarisons needed to merge m items, then
we have the inequality [log ] MERGE(m,n)-1
The exercises show that these upper and lower bounds can get arbitrarily far apart as m gets
much smaller than n. This should not be a surprise becoz the conventional algorithm is
designed to work best when m and n binary insertion would required the fewest number of
comparisons needed to merge A[1] into B[1],…..,B[n].
When m and n are equal, the lower bound given by the comparison tree model is too and the
number of comparisons for the conventional merging algorithm can be shown to be optimal.

Theorm: MERGE(m,m)=2m-1, for m≥1.
Proof: Consider any algorithm that merges the two sets A[1] <……..< A[m] and
B[1]<…….<B[m]. We already have an algorithm that requires 2m-1 comparisons. If we can
show that MERGE(m,m)≥2m-1, then the theorm follows. Consider any comparison-based
algorithm for solving A[1]<B[2]<A[2]<………<B[m]<A[m], that is, for which the B‘s and
A‘s alternate. Any merging algorithm must make each of the 2m-1 comparisons
B[1]:A[1],A[1]:B[2],B[2]:A[2],…..,B[m]:A[m] while merging the given input. To see this,
supose that a comparison of type B[i]:A[i] is not made for some i. Then the algorithm cannot
distinguish b/w the previous ordering and the one in which
B[1]<A[1]<……<A[i-1]<A[i]<B[i]<B[i+1]<…….<B[m]<A[m].
So the algorithm will not neccessarly merge the A‘s and B‘s properly. If a comparison of type
A[i]:B[i+1] is not made, then the algorithm will not be able to distinguish b/w the cases in
which B[1]<A[1]<B[2]<……<B[m]<A[m] and in which B[1]<A[1]<B[2]<A[2]……<A[i-
1]<B[i]<B[i+1]<A[i]<a[i+1]<…….<B[m]<A[m]. So any algorithm must make all 2m-1
comparisons to produce this final result. The theorm follows.
INSERTION SORT
If the first few objects are already sorted, an unsorted object can be inserted in the sorted set
in proper place. This is called insertion sort. An algorithm consider the elements one at a
time, inserting each in its suitable place among those already considered (keeping them
sorted).
Insertion sort is an example of an incremental algorithm; it builds the sorted sequence one
number at a time.
INSERTION_SORT (A)
1. For j = 2 to length [A] do

2. key = A[j]
3. {Put A[j] into the sorted sequence A[1 . . j-1]
4. i ← j -1
5. while i > 0 and A[i] > key do
6. A[i+1] = A[i]

7. i = i-1
8. A[i+1] = key
Analysis
Best-Case
The while-loop in line 5 executed only once for each j. This happens if given array A is
already sorted.
T(n) = an + b = O(n)
It is a linear function of n.
Worst-Case
The worst-case occurs, when line 5 executed j times for each j. This can happens if
array A starts out in reverse order
T(n) = an2 + bc + c = O(n2)
It is a quadratic function of n.

The graph shows the n2 complexity of the insertion sort.
Stability
Since multiple keys with the same value are placed in the sorted array in the same order that
they appear in the input array, Insertion sort is stable.
Extra Memory
This algorithm does not require extra memory.
For Insertion sort we say the worst-case running time is θ(n2), and the best-case
running time is θ(n).
Insertion sort use no extra memory it sort in place.
The time of Insertion sort is depends on the original order of a input. It takes a time in
Ω(n2) in the worst-case, despite the fact that a time in order of n is sufficient to solve
large instances in which the items are already sorted.

Implementation
void insertionSort(int numbers[], int array_size)

{ int i, j, index;
for (i=1; i < array_size; i++)
{ index = numbers[i];
j = i;
while ((j > 0) && (numbers[j-1] > index))
{ numbers[j] = numbers[j-1];
j = j - 1;
}
numbers[j] = index;
}
}

Adversary argument
• Suppose we have an algorithm we think efficient.
• Image an adversary tries to prove otherwise.

• At each point in the algorithm, whenever an decision (i.e., key comparison) is made, the
adversary tells us the result of the decision.
• The adversary chooses the answer which tries to force the algorithm work hard (i.e., do a lot
of decision, or to say, the answer releases as less new information as possible).
• You can think the adversary is constructing a ―bad‖ input while it is answering the
questions. The only requirement on the answers is that they must be internally consistent.
• If the adversary can force the algorithm to perform f(n) steps, then f(n) is the lower bound,
i.e, at least how many steps in the worst case.
Simply Put:
• Playing a guessing game between you and your friend.

– You are to pick up a date, and the friend will try to guess the date by asking YES/NO
questions.
– Your purpose is forcing your friend to ask as many questions as possible.
• To question ―is it in winter‖, your answer should be NO.
• To question ―is the first letter of the month‘s name in the first half of the alphabet‖? Your
answer should be YES.
• Idea:
– You did not pick up a date in advance at all, but
– Construct a date according to the questions. The
– The requirement is that the finally constructed date should be
consistent to all your answers to the questions.
– Looks like cheating, but it is a good way to find the lower bound.

Lower Bound for Algorithms that remove at most One Inversion per
Comparison
Insertion Sort, either does nothing or moves the key in the jth slot to the (j+1)st slot. By
moving the key in the jth slot up one slot, we have remedied the fact that ‗x‘ (number being
compared with) should come before the key. However, this is all we have accomplished. We
show that all sorting algorithms that sort only by comparisons of keys, and accomplish such a
limited amount of rearranging after each time, require at least quadratic time. We obtain our
results under the assumption that the keys to be sorted are distinct. Clearly, the worst case
bound still holds true with this restriction removed because a lower bound on the worst-case
performance from some inputs of some subsets is also a lower bound when all inputs are
considered.
In general, we are concerned with sorting n distinct keys that come from any ordered
set. However, without loss of generality, we can assume that the keys to be sorted are simply
the positive integers 1,2,…….,n, because we can substitute 1 for the smallest key, 2 for the
second smallest, and so on. For example, suppose we have alpha input, [Ralph, Clyde, Dave].
We can associate 1 with Clyde, 2 with Dave and 3 with Ralph. To obtain the equivalent input,
[3, 1, 2]. Any algorithm that sorts these integers only by comparison of keys would have to
do the same number of comparisons to sort the three names.
A permutation of the first n positive integers can be thought of as an ordering of those

integers. Because there are n! Permutations, of the first n positive integers. There are n!
different orderings of those integers. For example the following six permutations are all the
ordering of the first three positive integers:
[1,2,3] [1,3,2] [2,1,3] [2,3,1] [3,1,2] [3,2,1]
This means that n! Different inputs (to a sorting algorithm) containing n different
keys. These six permutations are the different inputs of size three (3).
We denote a permutation by [k1,k2,…………,kn]. That is ki is the integer at the ith position.

For the permutation [3, 1, 2], for example,
K1=3, k2=1, k3=2
R and s must be integers between 1 and n such that s > r.

Given a permutation, the pair (s, r) is an inversion in either the permutation or its transpose
but not both. Showing that there are n ( n - 1 ) / 2 such pairs of integers between 1 and n is
left as an exercise. This means that a permutation and its transpose have exactly, n ( n - 1 ) / 2
inversions between them. So the average no of inversions in a permutation and its transpose
is,
1 n(n 1) n(n 1)
*
2 2 4
Therefore, we consider all permutations equally probable for the input, the average number of
inversions in the input is also n (n-1) / 4. Because we assume that the algorithm removes at
most one inversion after each comparison, on an average it must do at least this many
comparisons to remove all inversion sand thereby sort the input.
Insertion Sort removes at most the inversion consisting of S[j] and x after each comparison,
and therefore this algorithm is in the class of algorithms addressed by Theorem.
Because Insertion Sort‘s worst case time complexity is n ( n – 1 ) / 2 and its average – case
time complexity is about n2/4, its about as good as we can hope to do (as far as comparisons
of keys are concerned) with algorithms that sort only by comparisons of keys and remove at
least one inversion after each comparison.
SELECTION SORT
This type of sorting is called "Selection Sort" because it works by repeatedly element. It
works as follows: first find the smallest in the array and exchange it with the element in the
first position, then find the second smallest element and exchange it with the element in the
second position, and continue in this way until the entire array is sorted.
SELECTION_SORT (A)
for i ← 1 to n-1 do
min j ← i;
min x ← A[i]
for j ← i + 1 to n do
If A[j] < min x then
min j ← j

min x ← A[j]
A[min j] ← A [i]
A[i] ← min x
Selection sort is among the simplest of sorting techniques and it work very well for
small files. Furthermore, despite its evident "naïve approach "Selection sort has a quite
important application because each item is actually moved at most once, Section sort is a
method of choice for sorting files with very large objects (records) and small keys.
The worst case occurs if the array is already sorted in descending order. Nonetheless, the time
require by selection sort algorithm is not very sensitive to the original order of the array to be
sorted: the test "if A[j] < min x" is executed exactly the same number of times in every case.
The variation in time is only due to the number of times the "then" part (i.e., min j ← j; min x
← A[j] of this test are executed.
The Selection sort spends most of its time trying to find the minimum element in the
"unsorted" part of the array. It clearly shows the similarity between Selection sort and Bubble
sort. Bubble sort "selects" the maximum remaining elements at each stage, but wastes some
effort imparting some order to "unsorted" part of the array. Selection sort is quadratic in both
the worst and the average case, and requires no extra memory.
For each i from 1 to n - 1, there is one exchange and n - i comparisons, so there is a total
of n -1 exchanges and (n -1) + (n -2) + . . . + 2 + 1 = n(n -1)/2 comparisons. These
observations hold no matter what the input data is. In the worst case, this could be quadratic,
but in the average case, this quantity is O(n log n). It implies that the running time of
Selection sort is quite insensitive to the input.
Implementation
void selectionSort(int numbers[], int array_size)

{
int i, j;
int min, temp;
for (i = 0; i < array_size-1; i++)

{
min = i;
for (j = i+1; j < array_size; j++)
{
if (numbers[j] < numbers[min])
min = j;
}
temp = numbers[i];
numbers[i] = numbers[min];
numbers[min] = temp;
}
}
SELECTION OF KTH SMALLEST ELEMENT
In computer science, a selection algorithm is an algorithm for finding the k-th smallest
number in a list (such a number is called the kth order statistic.) This includes the cases of
finding the minimum, maximum, and median elements. There are worst-case linear time
selection algorithms. Selection is a subproblem of more complex problems like the nearest
neighbor problem and shortest path problems.
A worst-case linear algorithm for the general case of selecting the kth largest element
was published by Blum, Floyd, Pratt, Rivest, and Tarjan in their 1973 paper Time bounds for
selection, sometimes called BFPRT after the last names of the authors. The algorithm that it
is based on was conceived by the inventor of quicksort,C.A.R.Hoare, and is known as Hoare's
selection algorithm or quickselect.
In quicksort, there is a subprocedure called partition that can, in linear time, group a list
(ranging from indices left to right) into two parts, those less than a certain element, and those
greater than or equal to the element. Here is pseudocode that performs a partition about the
element list[pivotIndex]:
Algorithm function partition(list, left, right, pivotIndex)

pivotValue := list[pivotIndex]
swap list[pivotIndex] and list[right] // Move pivot to end

storeIndex := left
for i from left to right-1
if list[i] < pivotValue
swap list[storeIndex] and list[i]
storeIndex := storeIndex + 1
swap list[right] and list[storeIndex] // Move pivot to its final place
return storeIndex
In quicksort, we recursively sort both branches, leading to best-case Ω(n log n) time.
However, when doing selection, we already know which partition our desired element lies in,
since the pivot is in its final sorted position, with all those preceding it in sorted order
preceding it and all those following it in sorted order following it. Thus a single recursive call
locates the desired element in the correct partition:
Algorithm function select(list, left, right, k)

select pivotIndex between left and right
pivotNewIndex := partition(list, left, right, pivotIndex)
if k = pivotNewIndex
return list[k]
else if k < pivotNewIndex
return select(list, left, pivotNewIndex-1, k)
else
return select(list, pivotNewIndex+1, right, k)
Note the resemblance to quicksort; indeed, just as the minimum-based selection algorithm is
a partial selection sort, this is a partial quicksort, generating and partitioning only O(log n) of
its O(n) partitions. This simple procedure has expected linear performance, and, like
quicksort, has quite good performance in practice. It is also an in –place algorithm, requiring
only constant memory overhead, since the tail recursion can be eliminated with a loop like
this:
Algorithm function select(list, left, right, k)

loop
select pivotIndex between left and right
pivotNewIndex := partition(list, left, right, pivotIndex)

if k = pivotNewIndex
return list[k]
else if k < pivotNewIndex
right := pivotNewIndex-1
else
left := pivotNewIndex+1
Like quicksort, the performance of the algorithm is sensitive to the pivot that is chosen. If bad
pivots are consistently chosen, this degrades to the minimum-based selection described
previously, and so can require as much as O(n2) time. David Musser describes a "median-of-3
killer" sequence that can force the well-known median-of-three pivot selection algorithm to
fail with worst-case behavior .
The algorithm works as follows:
1. Find the minimum value in the list

2. Swap it with the value in the first position
3. Repeat the steps above for remainder of the list (starting at the second position)
Effectively, we divide the list into two parts: the sublist of items already sorted, which we
build up from left to right and is found at the beginning, and the sublist of items remaining to
be sorted, occupying the remainder of the array.
Here is an example of this sort algorithm sorting five elements:
64 25 12 22 11
11 25 12 22 64
11 12 25 22 64
11 12 22 25 64

Selection sort can also be used on list structures that make add and remove efficient, such as a
linked list. In this case it's more common to remove the minimum element from the
remainder of the list, and then insert it at the end of the values sorted so far. For example:
64 25 12 22 11
11 64 25 12 22
11 12 64 25 22
11 12 22 64 25
11 12 22 25 64

MODULE 5

BACKTRACKING
Many problems with searching a set of solutions or which ask for an optimal solution
satisfying some constraints can be solved using the backtracking formulation. In many
applications of the backtrack method, the desired solution is expressible as an
n-tuple(x1,…..,xn), where the xi are chosen from some from finite set Si. Often the problem to
be solved calls for finding one vector that maximizes (or minimizes or satisfies) a criterion
function P(x1,…..,xn). Sometimes it seeks all vectors that satisfy P. The criterion function P is
the inequality a[xi]≤a[xi+1] for 1 ≤ i < n. The set Si is finite and includes the integers 1 through
n.
Many of the problems we solve using backtracking require that all the solutions satisfy a
complex set of constraints. For any problem these constraints can be classified into two
categories: explicit and implicit. The explicit constraints depend on the particular instance I
of the problem being solved. All tuples that satisfy the explicit constraints define a possible
solution space for I. the implicit constraints are the rules that determine which of the tuples in
the solution space of I satisfy the criterion function. Thus implicit constraints describe the
way in which xi must relate to each other.
CONTROL ABSTRACTION
(i) Recursive backtracking algorithm
Algorithm Backtrack(k)
//This schema describes the backtracking process using recursion. On entering, the first //k-1
values x[1], x[2],…..,x[k-1] of the solution vector x[1:n] have been assigned. x[ ] and //n are
global.
for (each x[k] Є T(x[1],…..,x[k-1]) do
if (Bk(x[1],x[2],…..,x[k])!=0) then

if (x[1],x[2],…..,x[k] is a path to an answer node)
then write (x[1:k]);
if (k<n) then backtrack(k+1);
Explanation
Let (x1,x2,…..xi) be a path from the root to a node in a state space tree. Let
T(x1,x2,…..xi) be the set of all possible values for xi+1 such that (x1,x2,…..xi+1) is also a path to
a problem state. T(x1,x2,…..xn) =Ф. We assume the existence of bounding function Bi+1
(expressed as predicates) such that if Bi+1(x1,x2,…..xi+1) is false for a path (x1,x2,…..xi+1) from
the root node to a problem state, then the path cannot be extended to reach an answer node.
Thus the candidates for position i+1 of the solution vector (x1,x2,…..xn) are those values
which are generated by T and satisfy Bi+1. The above recursive algorithm is initially invoked
by Backtrack(1);.
The solution vector (x1,x2,…..xn), is treated as a global array x[1:n]. All the possible
elements for the kth position of the tuple that satisfy Bk are generated, one by one, and
adjoined to the current vector (x1,x2,…..xk-1). Each time xk is attached, a check is made to
determine whether a solution has been found. Then the algorithm is recursively invoked.
When the ‗for‘ loop is exited, no more values for xk exist and the current copy of Backtrack
ends. The last unresolved call now resumes.
(ii)General iterative backtracking method
Algorithm IBacktrack(n)
//This schema describes the backtracking process.
//All solutions are generated in x[1:n] and printed as soon as they are determined.

k:= 1;
while(k≠0) do
if (there remains an untried x[k] Є T(x[1],x[2],…..,x[k-1]) and Bk (x[1],…..,x[k]) is true) then
if (x[1],….x[k] is a path to answer node)
then write (x[1:k]);
k:=k+1;//Consider the next set.
else k:=k-1; //Backtrack to the previous set.
Explanation
T() will yield the set of possible values that can be placed as the first component x[1]
of the solution vector. The component x[1] will take on those for which the bounding
function B1x(1) is true. The variable k is continually incremented and a solution vector is
grown until either a solution in found or no untried value of x k remains. When k is
decremented, the algorithm must resume the generation of possible elements for the kth
position that have not yet been tried.
Conclusion
The efficiency of both algorithms depends very much on four factors:
1. The time to generate next xk

2. The number of xk satisfying the explicit constraints
3. The time for the bounding functions Bk
4. The number of xk satisfying the Bk

Constraints
Solutions must satisfy a set of constraints
Explicit vs. implicit
Definition 1: Explicit constraints are rules that restrict each xi to take on values
only from a given set.
eg)
xi 0 or Si = { all nonnegative real numbers }
xi = 0 or 1 or Si = { 0, 1 }
li xi ui or Si = {a : li a ui }
All tuples satisfying the explicit constraints define a possible solution space for I
(I=problem instance)
Definition 2: The implicit constraints are rules that determine which of the tuples
in the solution space of I satisfy the criterion function. Thus implicit constrains
describe the way in which the xi must relate to each other.
Classic combinatorial problem
Place eight queens on an 8*8 chessboard so that no two attack
Without loss of generality, assume that queen i is placed on row i
All solutions represented as 8-tuples (x1,x2,…,x8) where xi is the column on which

queen i is placed
Constraints
• Explicit constraints
– Si={1,2,…,8}
– So, solution space consists of 88 8-tuples
• Implicit constraints
– No two xi‘s can be the same (By this, solution space reduced
from 88 to 8!)
– No two queens can be on the same diagonal

BOUNDING FUNCTIONS
The backtrack algorithm as its virtue the ability to yield the same answer with far
fewer than m trials. Its basic idea is to build solution vector, one competent at a time and to
use modified criterion functions Pi(x1,……,xn) called bounding functions, to test whether
the vector being formed has any chance of success. The major advantage of this method is
this: if it is realized that the partial vector (x1, x2,….., xi) can in no way lead to an optimal
solution, then mi+1……mn possible test vectors can be ignored entirely.
N-QUEENS PROBLEM
The n-queens problem is a generalization of the 8-queens problem. Here n queens are to be
placed on an nxn chessboard so that no two attack, that is no two queens are on the same row,
column, or diagonal. The solution space consists of all n! permutations of the n-tuple
(1,2,…,n). The tree is called permutation tree. The edges are labeled by possible values of xi.
Edges from level 1 to level 2 nodes specify the values for x1. Edges from level i to i+1 are
labeled with values of xi. The solution space is defined by all the paths from the root node to
a leaf node. For eg. If n=4, then there will be 4! =24 leaf nodes in the tree.
Eg: 8-queens problem
A classic problem is to place eight queens on an 8x8 chessboard so that no two

―attack,‖ that is, so that no two of them are on the same row, column, or diagonal.

• Bounding function
– No two queens placed on the same column  xi’s are distinct
– No two queens placed on the same diagonal  how to test?
• The same value of ―row-column‖ or ―row+column‖
• Supposing two queens on (i,j) and (k,l)
– i-j=k-l (i.e., j-l=i-k) or i+j=k+l (i.e., j-l=k-i)
• So, two queens placed on the same diagonal iff
|j-l| = |i-k|

This example shows how the backtracking works(4-Queens)

TREE STRUCTURE
Tree organization of the 4-queens solution space. Nodes are numbered as in depth first
search.
The n-queens problem is a generalization of the 8-queens problem. Now n queens are to be
placed on an n×n chess board so that no two attack; that is, no two queens are on the same
row, column or diagonal.Generalising our discussions, the solution space consists of all n!
permutations of n-tuple (1, 2.., n).The above figure shows a possible tree organization for the
case n=4. A tree such as this is called a permutation tree. The edges are labeled by possible
values of xi. Edges from level 1 to level 2 nodes specify the value for x1.Thus the leftmost
sub tree contains all solutions with x1=1 and x2=2,and so on. Edges from level to level
i+1are labeled with the value of xi.the solution space is defined by all paths from the root
node to a leaf node. There are 4! =24 nodes in the tree.
ALGORITHM
Algorithm Place (k, ί)
// Returns true if a queen can be placed in the kth row and
//ith column. Otherwise it returns false. x[] is a
//global array whose first (k –1) values have been set.
//Abs (p) returns the absolute value of p.

{
for j: =1 to k-1 do
if ((x [j] = ί) // Two in the same column
or (Abs (x [j] – ί ) = Abs (j-k)) )
// or in the same diagonal
then return false;
return true;
}
Algorithm Nqueens (k, n)

// Using backtracking, this procedure prints all
// possible placements of n queens on an n x n
// chessboard so that they are nonattacking.
{
for ί: =1 to n do
{
if Place (k, ί) then
{
x [k] : = ί;
if (k=n) then write (x[1:n]);
else Nqueens (k+1,n);
}
}
}

SUM OF SUBSETS
Given positive numbers wi, 1≤i≤n, and m, find all subsets of the wi whose sum is
m
eg) (w1,w2,w3,w4) = (11,13,24,7) and m=31
Solutions are (11,13,7) and (24,7)
Representation of solution vector
Variable-sized
By giving indices
In the above example, (1,2,4) and (3,4)
Explicit constraints: xi ∈ { j | j is integer and 1≤j≤n}
Implicit constraints: no two be the same, the sums are m
Fixed-sized
n-tuple (x1,x2,…,xn) where xi∈{0,1}, 1≤i≤n
In the above example, (1,1,0,1) and (0,0,1,1)
Variable tuple size
Suppose we are given n distinct positive numbers(usually called weights) and we desire to
find all combinations of these numbers whose sums are m.This is called sum of subsets
problem.
We could formulate this problem using either fixed-or variable-sized tuple
Given positive numbers wi,1<=i<=n, and m, this problem calls for finding all subsets of the
wi whose sums are m.For example if n=4,( w1,w2,w3,w4)=(11,13,24,7), and m=31,then the
desired subsets are (11,13,7) and (24,7).Rather than represent the solution vector by the wi
which sum to m we could represent the solution vector by giving the indices of these wi.Now
the two solutions are described by the vectors (1,2,4) and (3,4).In general all solutions are k-
tuples (x1,x2,.....xk), 1<=k<=n and different solutions may have different sized tuples.

• Solution space defined by all paths from the root to any node
Fixed tuple size:
• Solution space defined by all paths from the root to a leaf node
• Bounding function
• Bk(x1,..,xk)=true iff
– Assuming wi’s in nondecreasing order, (x1,..,xk) cannot lead to an answer node if

– So, the bounding function is
• Backtracking algorithm for the sum of sunsets

– Invoked by SumOfSubsets(0,1,∑i=1,nwi)

Example:n=6, w[1:6]={5,10,12,13,15,18}, m=30
• Terminology
– A node which has been generated and all of whose children have not yet been
generated is called a live node
– The live node whose children are currently being generated is called E-node
– A dead node is a generated node which is not to be expanded further or all of
whose children have been generated
• Two different ways of tree generation
• Backtracking
• Depth first node generation with bounding functions
– Branch-and-bound
– E-node remains E-node until it is dead
Four factors for efficiency of the backtracking algorithms
Time to generate the next xk
Number of xk satisfying the explicit constraints
Time for the bounding function Bk
Number of xk satisfying Bk

A solution space organization for the sum of subsets problem.Nodes are numbered in
BFS
* The first three relatively independent of the problem instance, and the last dependent
1
1
X1=1
X1=4
X1=2
3 4 5
2
3 4 5
2
X2=2
X2=4 X2=3
X X2=4
6 7 8 9 10 11
6 7 99 11
1
X3=3 X3=4 X3=4 X3=4
12 13 14 15
6 6
X4=4
16
Fig 1
The tree of fig 1 represents corresponds to variable tuple size formulation.The edges are
labelled such that an edge from a level i node to a level i+1 node represents a value for xi.At
each node, the solution space is partitioned into subsolution spaces.The solution space is
defined by all paths from the root node to any node in the tree,since any such path
corresponds to a subset satisfying the explicit constraints.Thr possible paths are
(1),(1,2),(1,2,3),(1,2,3,4),(1,2,4),(1,3,4),(2),(2,3) and so on.Thus the leftmost subtree defines
all subsets containing w1,the next subtree defines all subsets containing w2 but not w1 and so
on.

Another organization for the sum of subsets problem.Nodes are numbered in D-search
1
1 X1=0
X1=1
3
2 1
12
X2=0
X2=1 1 X2=0 X2=1
4 5
18 19 1 11
11 19
81 1 X3=0
X3=1 X3=0 X3=1
X3=1
X3=0 X3=1 13 6
21 1 12
20 11 11
26 27 11 11
20 X4=1 X4
22 12 2
1 X4=1 X4= X4= 11 =1 8
6 X4=1 1 X4=1 X4= 151 10
30 1 31 28 29 24 25 22 23 1 16 1 17 0 14 1 11
X4=0 1 11
X4=0 11 24 1 X 4=0
22 1 61 1 14
11 11 22 0
1 00001 1 1 1
8 1
==
1
FIG 2
The tree of fig 2 corresponds to the fixed tuple size formation.Edges from level i nodes level
i+l nodes are labelled with the value of xi w hich is either zero or one.All paths from root to a
leaf node defining the solution space .The left subtree of the root defines all subsets
containing w1,the right subtree defines all subsets not containing w1, and so on.Now there are
24 leaf nodes.

KNAPSACK PROBLEM
Explanation:
Given n positive weights wi , n positive profits pi, and a positive number m that is the
knapsack capacity, this problem calls for choosing a subset of the weights such that
∑ i≤i≤n wixi ≤ m and ∑ i≤i≤n pixi is maximized.
The xi‘s constitute a zero-one-valued vector. The solution space for this problem consists of
the 2n distinct ways to assign zero or one values to the x i‘s. Thus the solution space is the
same as that for the sum of subsets proble. Two possible tree organizations are possible. One
corresponds to the fixed tuple size formulation and the other to the variable tuple size
formulation. Backtracking algorithms for the knapsack problem can arrived by using either of
these two state space trees. Regardless of which is used, bounding functions are needed to
help kill some live nodes without expanding them. A bounding function for this problem is
obtained by using an upper bound on the value of the best feasible solution obtainable by
expanding the given live node and any of its descendants. If this upper bound is not higher
than the value of the best solution determined so far, then that live node can be killed.
Now by using the fixed tuple size formulation, if at node Z the values of xi, 1≤i≤k, have
already been determined, then an upper bound for Z can be obtained by relaxing the
requirement xi=0 or 1 to 0≤xi ≤1 for k+1≤i≤n and using the greedy algorithm to solve the
relaxed problem. Function Bound(cp,cw,k) determines an upper bound on the best solution
obtainable by expanding any node Z at level k+1 of the state space tree. The object weights
and profits are w[i] and p[i]. It is assumed that p[i]/w[i]≥p[i+1]/w[i+1], 1≤i<n.
From Bound it follows that the bound for a feasible left child of a node Z is the same as that
for Z. Hence, the bounding function need not be used whenever the backtracking algorithm
makes a move to the left child of a node. The resulting algorithm is BKnap. It was obtained
form the recursive backtracking schema. Initially set fp;=-1;. This algorithm is invoked as
BKnap(1,0,0);
When fp ≠ -1, x[i]. 1≤ i≤ n, is such that ∑ni=1 p[i]x[i] = fp. In lines 8 to 18 children are
generated. In line 20, Bound is used to test whether a right child should be generated. The

path y[i], 1≤ i ≤ k, is the path to the current node. The current weight cw = ∑k-1i=1 w[i]y[i] and
cp= ∑i=1k-1 p[i]y[i]. In lines 13 to 17 and 23 to 27 the solution vector is updated if need
be.
So far, all the backtracking algorithms have worked on a static state space tree.
Algorithm: Bounding Function
Algorithm Bound(cp,cw,k)
//cp is the current profit total, cw is the current
//weight total; k is the index of the last removed
//item; and m is the knapsack size.
{
b:= cp; c:=cw;
for i:=k+1 to n do
{
c:=c+w[i];
if(c<m) then b:=b+p[i];
else return b+(1-(c-m)/w[i])*p[i];
}
return b;
}
Algorithm: Backtracking solution to the 0/1 knapsack problem.
Algorithm Bknap(k,cp,cw)
//m is the size of the knapsack; n is the number of weights.
//and profits. w[] and p[] are the weights and profits.
//p[i]/w[i] ≥ p[i+1]/w[i+1]. fw is the final weight of
//knapsack; fp is the final maximum profit. x[k] = 0 if w[k]
//is not in the knapsack; else x[k]=1.
{
// Generate left child.
if (cw + w[k] ≤ m) then

{
y[k]:=1;
if(k<n) then BKnap(k+1,cp+p[k], cw +w[k]);
if ((cp+p[k] > fp) and (k=n)) then
{
fp:=cp +p[k]; fw:=cw+w[k];
for j:=1 to k do x[j]:=y[j];
}
}
// Generate right child.
if (Bound(cp,cw,k)≥fp) then
{
y[k] :=0; if (k<n) then BKnap(k+1,cp,cw);
if((cp>fp) and (k=n)) then
{
fp:=cp; fw:=cw;
for j:=1 to k do x[j]:=y[j];
}
}
}
ALGORITHM
// CP-Current Profit;
//K-index of last removed access
b=cp;c=cw;
for P=K+1 to n do
c=c+w[i];
if(c<n)the b=b+p[i];

else
return b+(1-(c-m)/w[i])+p[i];
Return(b);
Algorithm B knap(K,cp,cw)
If(w+w[K]<=m) then
Y[K]=1;
If(K<n)then
B knap(K+1,cp+p[K],cw+w[K]);
If((cp=p[k])>fp)and(K=n)then
fp=cp+p[K];fw=cw+w[K];
for j=1 to K do
x[j]=y[j];
If(Bound(cp,w,K)>=fp) then
Y[K]=0;
If(k<n)then

B knap(K+1,cp,cw);
If((cp>fp)and(k=n)) then
fp=cp;
fw=cw;
for j=1 to k do
x[j]=y[j];
BRANCH AND BOUND ALGORITHM TECHNIQUE
Introduction
Branch and bound is another algorithm technique that we are going to present in our multi-
part article series covering algorithm design patterns and techniques. B&B, as it is often
abbreviated, is one of the most complex techniques and surely cannot be discussed in its
entirety in a single article. Thus, we are going to focus on the so-called A* algorithm that is
the most distinctive B&B graph search algorithm.
If you have followed this article series then you know that we have already covered the most
important techniques such as backtracking, the greedy strategy, divide and conquer, dynamic
programming, and even genetic programming. As a result, in this part we will compare
branch and bound with the previously mentioned techniques as well. It is really useful to
understand the differences.
Branch and bound is an algorithm technique that is often implemented for finding the optimal
solutions in case of optimization problems; it is mainly used for combinatorial and discrete
global optimizations of problems. In a nutshell, we opt for this technique when the domain of

possible candidates is way too large and all of the other algorithms fail. This technique is
based on the en masse elimination of the candidates.
You should already be familiar with the tree structure of algorithms. Out of the techniques
that we have learned, both the backtracking and divide and conquer traverse the tree in its
depth, though they take opposite routes. The greedy strategy picks a single route and forgets
about the rest. Dynamic programming approaches this in a sort of breadth-first search
variation (BFS).
Now if the decision tree of the problem that we are planning to solve has practically
unlimited depth, then, by definition, the backtracking and divide and conquer algorithms are
out. We shouldn't rely on greedy because that is problem-dependent and never promises to
deliver a global optimum, unless proven otherwise mathematically.
As our last resort we may even think about dynamic programming. The truth is that maybe
the problem can indeed be solved with dynamic programming, but the implementation
wouldn't be an efficient approach; additionally, it would be very hard to implement. You see,
if we have a complex problem where we would need lots of parameters to describe the
solutions of sub-problems, DP becomes inefficient.
Branch and bound is a systematic method for solving optimization problems B&B is a rather
general optimization technique that applies where the greedy method and dynamic
programming fail. However, it is much slower. Indeed, it often leads to exponential time
complexities in the worst case. On the other hand, if applied carefully, it can lead to
algorithms that run reasonably fast on average. The general idea of B&B is a BFS-like search
for the optimal solution, but not all nodes get expanded (i.e., their children generated).
Rather, a carefully selected criterion determines which node to expand and when, and another
criterion tells the algorithm when an optimal solution has been found. The basic concept
underlying the branch-and-bound technique is to divide and conquer. Since the original
―large‖ problem is hard to solve directly,it is divided into smaller and smaller subproblems
until these subproblems can be conquered. The dividing (branching) is done by partitioning
the entire set of feasible solutions into smaller and smaller subsets.The conquering
(fathoming) is done partially by (i) giving a bound for the best solution in the subset;(ii)
discarding the subset if the bound indicates that it can‘t contain an optimal solution.These

three basic steps – branching, bounding, and fathoming – are illustrated on the following
example.
Branch-and-Bound Algorithms
A counter-part of the backtracking search algorithm which, in the absence of a cost criteria,
the algorithm traverses a spanning tree of the solution space using the breadth-first approach.
That is, a queue is used, and the nodes are processed in first-in-first-out order
Procedure Expand(E)
begin
/* Generate all the children of E; */
I := E->I;
X,p: nodepointer;
S[1:n]: Boolean;
/* S is a bitmap set initialized to 0*/
/* S will contain all the jobs that have been
assigned by the partial path from the root
to E */
p := E;
while (p is not the root) do
S[p->J] := 1;
p := p-> Parent;
endwhile
for job=1 to n do
if S[job] = 0 then
X := new(node);
X->I := I + 1;
X->J := job;
X->Parent := E;
X->CC := E->CC + AX->I,X->J-mX->I;
Insert(X,H);
endif
endfor

end
.
If a cost criteria is available, the node to be expanded next (i.e., the branch) is the one with
the best cost within the queue. In such a case, the cost function may also be used to discard
(i.e., the bound) from the queue nodes that can be determined to be expensive. A priority
queue is needed here.
Cost-Based Tree Traversal of branch and bound
A function can be considered to be a tree generator, if when given a node X and index i it
produces the i‘th child of the node.
The following function produces a complete binary tree of 11 nodes.
The recursive function provided for deriving permutations is another example of a function
that may be used to generate trees.
Besides for a tree generator function, we also need a cost function to decide in what order to
traverse the nodes when searching for a solution. The algorithm proceeds in the following
manner.
1. Initialization: The root of the of the tree is declared to be alive.

2. Visit: The cost criteria decides which of the live nodes is to process next.
3. Replacement: The chosen node is removed from the set of live nodes, and its children
are inserted into the set. The children are determined by the tree generator function.
4. Iteration: The visitation and replacement steps are repeated until no alive nodes are
left.

In the case of backtracking the cost criteria assumes a last-in-first-out (LIFO) function, which
can be realized with a stack memory. A first-in-first-out cost criteria implies the FIFO
branch-and-bound algorithm, and it can be realized with queue memory. A generalization to
arbitrary cost criteria is the basis for the priority branch-and-bound algorithm, and a priority
queue memory can be employed to realize the function.
FIFO BRANCH AND BOUND ALGORITHM
FIFO branch and bound algorithm for the job sequencing problem can begin with upper=
infinity. While implementing FIFO B&B algorithm it is not economical to kill live nodes
with c®> upper each time upper is updated. Each solution is assumed to be expressible as an
array X[1:n] (as was seen in Backtracking).
A predictor, called an approximate cost function CC, is assumed to have been defined.
A live node is a node that has not been expanded.A dead node is a node that has been
expanded. The expanded node (or E-node for short) is the live node with the best CC value.
Branch and Bound is a general search method. Starting by considering the root problem (the
original problem with the complete feasible region), the lower-bounding and upper-bounding
procedures are applied to the root problem.If the bounds match, then an optimal solution has
been found and the procedure terminates. Otherwise, the feasible region is divided into two or
more regions, these subproblems partition the feasible region. The algorithm is applied
recursively to the subproblems. If an optimal solution is found to a subproblem, it is a
feasible solution to the full problem, but not necessarily globally optimal. If the lower bound
for a node exceeds the best known feasible solution, no globally optimal solution can exist in
the subspace of the feasible region represented by the node. Therefore, the node can be
removed from consideration. The search proceeds until all nodes have been solved or pruned,
or until some specified threshold is met between the best solution found and the lower bounds
on all unsolved subproblems. A counter-part of the backtracking search algorithm which, in
the absence of a cost criteria, the algorithm traverses a spanning tree of the solution space
using the breadth-first approach. That is, a queue is used, and the nodes are processed in first-
in-first-out order. If a cost criteria is available, the node to be expanded next (i.e., the branch)
is the one with the best cost within the queue. In such a case, the cost function may also be
used to discard (i.e., the bound) from the queue nodes that can be determined to be expensive.
A priority queue is needed here.

The general FIFO B&B algorithm follows:
Procedure B&B()
begin
E: nodepointer;
E := new(node); -- this is the root node which
-- is the dummy start node
H: heap; -- A heap for all the live nodes
-- H is a min-heap for minimization problems,
-- and a max-heap for maximization problems.
while (true) do
if (E is a final leaf) then
-- E is an optimal solution
print out the path from E to the root;
return;
endif
Expand(E);
if (H is empty) then
report that there is no solution;
return;
endif
E := delete-top(H);
endwhile
end
Procedure Expand(E)
begin
- Generate all the children of E;
- Compute the approximate cost value CC of each child;
- Insert each child into the heap H;
end

We need to define the full record of a node .We need to fully implement the Expand
procedure
Every node corresponds to something like X[i]=j, which signifies that the job X[i] assigned to
person i is j. Every node must store its CC value. Each node must point to its parent so that
when an optimal leaf is generated, the path from that leaf to the root can be traced and printed
out as the optimal solution.Therefore, a node record structure should look like:
Record node
Begin
Parent: nodepointer;
I: integer; -- person I
J: integer; -- Job J is assigned to person I
CC: real;
End
Take the 2nd CC formula:

CC(X at level k) = cost so far + sumni=k+1mi
where mi is the minimum of row i.
observe that if X is a pointer to a node, then
X->CC = X->Parent->CC + AX->I,A->J - mX->I
Write a piece of code that computes the mis for i=1,2,...,n
Code for Expand(E):
Procedure Expand(E)
begin
/* Generate all the children of E; */
I := E->I;
X,p: nodepointer;
S[1:n]: Boolean;
/* S is a bitmap set initialized to 0*/
/* S will contain all the jobs that have been
assigned by the partial path from the root
to E */
p := E;
while (p is not the root) do

S[p->J] := 1;
p := p-> Parent;
endwhile
for job=1 to n do
if S[job] = 0 then
X := new(node);
X->I := I + 1;
X->J := job;
X->Parent := E;
X->CC := E->CC + AX->I,X->J-mX->I;
Insert(X,H);
endif
endfor
end.
Although a number of algorithms have been proposed for the integer linear programming
problem, the FIFO branch-and-bound technique has proven to be reasonably efficient on
practical problems, and it has the added advantage that it solves continuous linear programs
as sub problems.The technique is also used in a lot of software in global optimization.
Fifo B&B Cost-Based Tree Traversal
produces the i‘th child of the node.The following function produces a complete binary tree of
11 nodes.

manner.

3. Replacement: The chosen node is removed from the set of live nodes, and its
children are inserted into the set. The children are determined by the tree generator
function.
left.
In the case of backtracking the cost criteria assumes a last-in-first-out (LIFO) function,
which can be realized with a stack memory. A first-in-first-out cost criteria implies the
FIFO branch-and-bound algorithm, and it can be realized with queue memory. A
generalization to arbitrary cost criteria is the basis for the priority branch-and-bound
algorithm, and a priority queue memory can be employed to realize the function.
• Search the tree using a breadth-first search (FIFO branch and bound).
• Search the tree as in a bfs, but replace the FIFO queue with a stack (LIFO branch and
bound).
• Replace the FIFO queue with a priority queue (least-cost (or max priority) branch and
bound). The priority of a node p in the queue is based on an estimate of the likelihood
that the answer node is in the subtree whose root is p. FIFO branch and bound finds
solution closest to root.Backtracking may never find a solution because tree depth is
infinite (unless repeating configurations are eliminated).
• Least-cost branch and bound directs the search to parts of the space most likely to
contain the answer. So it could perform better than backtracking.

LIFO BRANCH AND BOUND ALGORITHM

LIFO branch and bound algorithm for the job sequencing problem can begin with upper=
infinity. While implementing LIFO B&B algorithm it is not economical to kill live nodes
with c®> upper each time upper is updated. Each solution is assumed to be expressible as an
array X[1:n] (as was seen in Backtracking).
A predictor, called an approximate cost function CC, is assumed to have been defined.
A live node is a node that has not been expanded.A dead node is a node that has been
expanded. The expanded node (or E-node for short) is the live node with the best CC value.
Branch and Bound is a general search method. Starting by considering the root problem (the
original problem with the complete feasible region), the lower-bounding and upper-bounding
procedures are applied to the root problem.If the bounds match, then an optimal solution has
been found and the procedure terminates. Otherwise, the feasible region is divided into two or
more regions, these subproblems partition the feasible region. The algorithm is applied
recursively to the subproblems. If an optimal solution is found to a subproblem, it is a
feasible solution to the full problem, but not necessarily globally optimal. If the lower bound
for a node exceeds the best known feasible solution, no globally optimal solution can exist in
the subspace of the feasible region represented by the node. Therefore, the node can be
removed from consideration. The search proceeds until all nodes have been solved or pruned,
or until some specified threshold is met between the best solution found and the lower bounds
on all unsolved subproblems. A counter-part of the backtracking search algorithm which, in
the absence of a cost criteria, the algorithm traverses a spanning tree of the solution space
using the breadth-first approach. That is, a queue is used, and the nodes are processed in first-
in-first-out order. If a cost criteria is available, the node to be expanded next (i.e., the branch)
is the one with the best cost within the queue. In such a case, the cost function may also be
used to discard (i.e., the bound) from the queue nodes that can be determined to be expensive.
A priority queue is needed here.
A general LIFO B&B algorithm :
Procedure B&B()
begin
E: nodepointer;
E := new(node); -- this is the root node which
-- is the dummy start node

H: heap; -- A heap for all the live nodes

-- H is a min-heap for minimization problems,
-- and a max-heap for maximization problems.
while (true) do
if (E is a final leaf) then
-- E is an optimal solution
print out the path from E to the root;
return;
endif
Expand(E);
if (H is empty) then
report that there is no solution;
return;
endif
E := delete-top(H);
endwhile
end
Procedure Expand(E)
begin
- Generate all the children of E;
- Compute the approximate cost value CC of each child;
- Insert each child into the heap H;
end
We need to define the full record of a node .We need to fully implement the Expand
procedure
Every node corresponds to something like X[i]=j, which signifies that the job X[i] assigned to
person i is j. Every node must store its CC value. Each node must point to its parent so that
when an optimal leaf is generated, the path from that leaf to the root can be traced and printed
out as the optimal solution.Therefore, a node record structure should look like:
Record node
Begin

Parent: nodepointer;
I: integer; -- person I
J: integer; -- Job J is assigned to person I
CC: real;
End
Take the 2nd CC formula:

CC(X at level k) = cost so far + sumni=k+1mi
where mi is the minimum of row i.
observe that if X is a pointer to a node, then

X->CC = X->Parent->CC + AX->I,A->J - mX->I
Write a piece of code that computes the mis for i=1,2,...,n
Although a number of algorithms have been proposed for the integer linear programming
problem, the LIFO branch-and-bound technique has proven to be reasonably efficient on
practical problems, and it has the added advantage that it solves continuous linear programs
as sub problems.The technique is also used in a lot of software in global optimization.
Lifo B&B Cost-Based Tree Traversal
produces the i‘th child of the node.The following function produces a complete binary tree of
11 nodes.

manner.

7. Replacement: The chosen node is removed from the set of live nodes, and its
children are inserted into the set. The children are determined by the tree generator
function.
left.
In the case of backtracking the cost criteria assumes a last-in-first-out (LIFO) function,
which can be realized with a stack memory. A last-in-first-out cost criteria implies the
LIFO branch-and-bound algorithm, and it can be realized with queue memory. A
generalization to arbitrary cost criteria is the basis for the priority branch-and-bound
algorithm, and a priority queue memory can be employed to realize the function.
• Search the tree using a breadth-first search (FIFO branch and bound).
• Search the tree as in a bfs, but replace the FIFO queue with a stack (LIFO branch and
bound).
• Replace the FIFO queue with a priority queue (least-cost (or max priority) branch and
bound). The priority of a node p in the queue is based on an estimate of the likelihood
that the answer node is in the subtree whose root is p. LIFO branch and bound finds
solution closest to root.Backtracking may never find a solution because tree depth is
infinite (unless repeating configurations are eliminated).
• Least-cost branch and bound directs the search to parts of the space most likely to
contain the answer. So it could perform better than backtracking.

LC CONTROL ABSTRACTION
listnode= record
{
Listnode *next,*parent;
Float cost;
}
Algorithm LCSearch(t)
//Search t for an answer node.
{
If *t is an answer node then output t and return
E:=t; //E-node
Initalize the list of live nodes to be empty;
Repeat
{
for each child x of E do
{
If x is an answer node then output the path from x to t and return;
Add(x);//x is a new live node.
(x->parent):=E;//Pointer for path to root
}
If there are no more live nodes then
{
Write(―No answer node‖);return;
}
E:=Least();
}until(false);
}

15- PUZZLE
The 15 puzzle consists of 15 squares numbered from 1 to 15 that are placed in a box
leaving one position out of the 16 empty. The goal is to reposition the squares from a given
arbitrary starting arrangement by sliding them one at a time into the configuration shown
above. For some initial arrangements, this rearrangement is possible, but for others, it is not.
The n-puzzle is known in various versions, including the 8 puzzle, the 15 puzzle, and with
various names. It is a sliding puzzle that consists of a frame of numbered square tiles in
random order with one tile missing. If the size is 3×3, the puzzle is called the 8-puzzle or 9-
puzzle, and if 4×4, the puzzle is called the 15-puzzle or 16-puzzle. The object of the puzzle is
to place the tiles in order (see diagram) by making sliding moves that use the empty space.
Theorem:
The goal state is reachavle from intial state iff ∑16 i=1less(i)+x is even.
For any state let less(i) be the number of tiles j such that j<I and position(j)>position(i).Let
position(i) be the position number in the intial state of the tile numbered i.

First ten steps in depth first search

TRAVELLING SALESMAN PROBLEM
The Travelling Salesman problem (TSP) is a problem in combinatorial optimization studied

in operations research and theoretical computer science. Given a list of cities and their
pairwise distances, the task is to find a shortest possible tour that visits each city exactly once.
It is used as a benchmark for many optimization methods.
The TSP has several applications even in its purest formulation, such as planning, logistics,
and the manufacture of microchips. Slightly modified, it appears as a sub-problem in many
areas, such as genome sequencing. In these applications, the concept city represents, for
example, customers, soldering points, or DNA fragments, and the concept distance represents
travelling times or cost, or a similarity measure between DNA fragments.
Asymmetric and symmetric
In the symmetric TSP, the distance between two cities is the same in each direction. Thus, the
underlying structure is an undirected graph between; especially, each tour has the same length
in both directions. In the asymmetric TSP, the distance from one city to the other need not
equal the distance in the other direction, in general, there may not even be a connection in the
other direction.
Problem:
You are given a list of n cities along with the distances between each pair of cities. The goal
is to find a tour which starts at the first city, visits each city exactly once and returns to the
first city, such that the distance traveled is as small as possible. This problem is known to be
NP-complete , i.e. no serial algorithm exists that runs in time polynomial in n, only in time
exponential in n, and it is widely believed that no polynomial time algorithm exists. In
practice, we want to compute an approximate solution, i.e. a single tour whose length is as
short as possible, in a given amount of time.
More formally, we are given a graph G=(N,V,W) consisting of a set N of n nodes (or cities),
a set of edges V = {(i,j)} connecting cities, and a set of nonnegative weights W = {w(i,j)}
giving the length of edge (i,j) (distance from city i to city j). The graph is directed, so that an

edge (i,j) may only be traversed in the direction from i to j, and edge (j,i) may or may not
exist. Similarly, w(i,j) does not necessarily equal w(j,i), if both edges exist.
Branch-and-Bound for TSP
A simple improvement on the algorithm prunes the search tree by observing that if a partial
tour is already longer than the best solution found so far, there is no reason to continue
searching that path.
Naive Branch-and-Bound Solution of TSP
w = w(1,2) + w(2,3) + w(3,4) + ... + w(n-1,n) + w(n,1)

Best_S_so_far = ( n, [ 1, 2, 3, ... , n-1, n ], w )
S = ( 1, [ 1 ], 0 )
Search( S, Best_S_so_far )
print Best_S_so_far
procedure Search( S, Best_S_so_far )
let ( k, [ i1, i2, ... , ik ], w ) = S

let ( n, [ i1B, i2B, ... , inB ], wB ) = Best_S_so_far
if k = n then
new_w = w + w(ik,i1)
if new_w < wB then
Best_S_so_far = ( k, [ i1, i2, ... , ik ], new_w )
end if
else
for all j not in [ i1, i2, ... , ik ]
new_w = w + w(ik,j)
if new_w < wB then
New_S = ( k+1, [ i1, i2, ... , ik, j ], new_w )
Search( New_S, Best_S_so_far )
end if
end for
endif
return
end
Better Branch and Bound Algorithm for TSP

Another strategy for searching the solution space is to repeatedly divide it into two parts:
those with a given edge and those without the edge. The search tree would unfold as follows:
-----------
| all solns |
-----------
/ \
---------------- -----------------
| solns with e_i | | solns w/out e_i |
---------------- -----------------
/ \ / \
----------- ---------- ---------- -----------
| with e_j | | w/out e_j | | with e_k | | w/out e_k |
----------- ---------- ---------- -----------
Bounding the solutions: Assume the input is given as a dense adjacency matrix. We will put
infinities, rather than zeros on the diagonal to avoid traversing these self-edges.
i\j 1 2 3 4 5 6 7
\ ________________________________________
1 |Inf 3 93 13 33 9 57
2 | 4 Inf 77 42 21 16 34
3 | 45 17 Inf 36 16 28 25
4 | 39 90 80 Inf 56 7 91
5 | 28 46 88 33 Inf 25 57
6 | 3 88 18 46 92 Inf 7
7 | 44 26 33 27 84 39 Inf
We can subtract a constant from a given row or column, as long as the values remain non-
negative. This changes the weight of each tour, but not the set of legal tours or their relative
weights. We therefore normalize the matrix by subtracting the minimum value in each row
from the row and the minimum value of each column from the column. This results in a
matrix with at least one zero in every row and column.
i\j 1 2 3 4 5 6 7
\ ________________________________________
1 |Inf 0 83 9 30 6 50
2 | 0 Inf 66 37 17 12 26
3 | 29 1 Inf 19 0 12 5

4 | 32 83 66 Inf 49 0 80
5 | 3 21 56 7 Inf 0 28
6 | 0 85 8 42 89 Inf 0
7 | 18 0 0 0 58 13 Inf
Any solution must use one entry from every row and every column, so the sum of the values
we just subtracted is a lower bound on the weight of solution. In this case, we subtracted [3 4
16 7 25 3 26] from the rows and then [0 0 7 1 0 0 4] from the columns, so the lower bound on
the weight of any solution is 96.
Representing the set of solutions: The adjacency matrix can be used to represent the set of
solutions. When an edge is chosen for the solution, the row and column containing that edge
is deleted. When an edge is eliminated from the solution, its entry is changed to infinity so
that it will never be chosen. In the above example, assume we choose to split the search space
on the edge from 4 to 6. For the right subtree, which represents all solutions not containing
(4,6), we replace the (4,6) entry by infinity. The minimum value in row 4 is now 32, so we
can renormalize the matrix and improve the lower bound to 96+32 = 128. Column 6 has
another 0 entry, so it remains unchanged. The matrix for the right subtree is:
i\j 1 2 3 4 5 6 7
\ ________________________________________
1 |Inf 0 83 9 30 6 50
2 | 0 Inf 66 37 17 12 26
3 | 29 1 Inf 19 0 12 5
4 | 0 51 34 Inf 17 Inf 48
5 | 3 21 56 7 Inf 0 28
6 | 0 85 8 42 89 Inf 0
7 | 18 0 0 0 58 13 Inf
The left subtree represents all solutions containing (4,6). We therefore delete row 4 and
column 6 from the matrix and renormalize. In addition, since we have used edge (4,6), edge
(6,4) is no longer usable, so we replace the (6,4) with infinity.The matrix can now be
renormalized, in this case by subtracting 3 from the row with i=5 (now the 4th row in the
smaller matrix), yielding a lower bound of 96+3=99 and the matrix:
i\j 1 2 3 4 5 7
\ __________________________________
1 |Inf 0 83 9 30 50

2 | 0 Inf 66 37 17 26
3 | 29 1 Inf 19 0 5
5 | 0 18 53 4 Inf 25
6 | 0 85 8 Inf 89 0
7 | 18 0 0 0 58 Inf
The process outlined so far can be used by repeatedly following these steps. Following the
left branches we are guaranteed to reach some solution in n levels. Following the right
branch, we will eventually have a matrix full of infinities, at which point the lower bound
Choice of the splitting edge: In general, the right branches represent a larger set of solutions
than the left branches. Therefore, in choosing the splitting edge, we look for something that
will raise the lower bound of the right-hand subtree as much as possible. In the example, edge
(4,6) was chosen because its value was zero and the next larger value in row 4 was 32. There
are other zero entries in the matrix, for example, (3,5), but the improvement in the lower
bound for that case would have been only 1 (for row 3) plus 17 (for column 5). Therefore, the
general rule is to search for the zero entry at (i,j) that maximizes the increase in the lower
bound (largest sum of minimum in row i and minimum in column j, not counting the zero at
(i,j)).
Insertion of infinities in left-branches: When expanding a left branch in the search tree for
edge (i,j), we noted that edge (j,i) should be made infinity. In general, we are trying to avoid
the creation of non-covering cycles in the tour, i.e., cycles that traverse only a subset of the
nodes. For the example above, assume that the left-most search path unfolds with the
following edge splits: (4,6), (3,5), and (2,1). At this point the partial solution contains three
disconnected paths; all cycles were prevented by marking (6,4), (5,3), and (1,2) as infinity.

-----------
| all solns |
-----------
/ \
------------ -------------
| with (4,6)| | w/out (4,6)|
------------ -------------
/ \
------------ -------------
| with (3,5)| | w/out (3,5)|
------------ -------------
/ \
------------ ------------
| with (2,1)| | w/out (2,1)|
------------ ------------
The next choice of splitting edge is (1,4), so according to our above rule (4,1) should be
changed to infinity. However, at this point the adjacency matrix does not contain a row for 4,
since it was removed with the (4,6) step. Moreover, the partial solution is now: 2-1-4-6 &3-5.

Algorithm Analysis and Design PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Algorithm Analysis and Design PDF

Uploaded by

Copyright:

Available Formats

Algorithm Analysis and Design (R 606) 1

Department of Computer Science & Engineering SJCET, Palai