Module-3

GREEDY METHOD
General Method:
Greedy technique is a most straightforward algorithm design technique, which can
apply to wide variety of problems. These problems have n inputs and require us to obtain a
subset that satisfies some constraints. Any subset that satisfies these constraints is called
feasible solution. We need to find a feasible solution that either maximizes or minimizes a
given objective function called an optimal solution.
The greedy method constructs an algorithm that works in stages considering one
input at a time. At each stage the decision is made regarding whether a particular input is
an optimal solution. If the inclusion of the next input into a partially constructed optimal
solution will result in an infeasible solution, then that input is not added to the partial
solution; otherwise it is added. The selection procedure is itself based on some
optimization measure; this measure is may be objective of the problem.
Different optimization measures may be possible for a given problem, these will
generate an algorithm that generate suboptimal solutions. This version of greedy technique
called Subset paradigm.
Algorithm Greedy(a,n)
// a[1:n] contains n inputs
{
solution=ф;
for i=1 to n do
{
x= Select(a);
if Feasible(solution, x) then
solution = Union(solution,x);
}
return solution;
}

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 1

The above algorithm shows the Greedy method control abstraction for the subset
paradigm. The algorithm works as follows:
The function Select selects an input from a[ ] and removes it. The selected input’s
value is assigned to x. Feasible is Boolean valued function that determines whether x can
be included into the solution subset. The function Union combines x with the solution and
updates the objective function.
The problems that do not call for selection of an optimal subset, in the greedy
method we make decisions by considering inputs in some order. Each decision is made
using an optimization criterion that can be computed using decisions already made. This
method of greedy approach is called Ordering paradigm.
The subset paradigm problems are:
 Knapsack problem
 Job sequencing with deadlines
 Minimum cost spanning trees:
o Prim’s algorithm & Kruskal’s algorithm
The ordering paradigm problems are:
 Single source shortest path problem.
Example 1: Change Making Problem
Assume a person buys few things in a store for Rs.61/- and he gives Rs. 100/- to
cashier. He has to return a change of Rs.39/-, the cashier constructs change in different
stages. At each stage he increase the total amount of change constructed as much as
possible.
He give a one Rs.20/- , then he add one Rs.10/-, then one Rs. 5/- and finally two Rs.
2/- are added. Here he can not give two Rs. 20/- because it exceeds Rs.39/-. He the greedy
method constructs the solution such that least number of currencies is given.
Example 2: Machine Scheduling
There are n tasks & infinite numbers of machines on which these tasks can be
performed are given. Each task has start time si finish time fi such that si<fi. [si , fi]is the
processing interval for task i. Two tasks i & j overlap if and only if their processing intervals
overlap at a point other than the interval start or end. For example the interval [2,6]
overlaps with [1,6], but not with [7,11].

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 2

 A feasible task to machine assignment is an assignment in which no machine is
assigned two overlapping tasks, so in a feasible assignment each machine works on at most
one task at any time. An optimal assignment is a feasible assignment that utilizes fewest
numbers of machines.
Suppose we have n=7 tasks with the start & finish times as shown below
Task A B C D E F G
Start 0 3 4 9 7 1 6
Finish 2 7 7 11 10 5 8
The task- to- machine assignment is a feasible assignment that utilizes seven machines
such that one machine to one task. But this assignment is not an optimal assignment
because other assignments use fewer machines. For example, we can assign the tasks A, B
& D to the same machine so it can reduce number of machines required.
The greedy technique to obtain optimal task assignment is to assign the tasks in
stages, one task per stage and in non decreasing order of task start times. A machine is old
if at least one of the tasks has been assigned to it; otherwise the machine is new.
The greedy technique will select the machine such that if an old machine becomes
available by the start time of the task to be assigned assign the task to this machine; if not
assign to new machine.
In the above example tasks in the non decreasing order of task start time are A, F, B,
C, G, E, D. the greedy algorithm assigns tasks to machine in this order only. The algorithm
has n=7 stages & in each stage one task is assigned to a machine.
 In stage one no old machine so task A is assigned to a new machine, let M1. This
machine is busy from time 0 to 2.
 In stage 2 task F is considered, since old machine M1 is busy, it is assigned to new
machine let it be M2.
 The task B is considered at stage 3, since its start time is 3 it is assigned to old
machineM1 since M1 is available.
 In stage 4 the task C is considered, its start time is 4, as the old machines are not
available, this task is assigned to new machine, let it be M3.
 In stage 5, the task G is considered, as its start time is 6, it assigned M2 as M2 is
available.

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 3

  In stage 6, the task E is considered, as its start time is 7, as both M1 & M2 are available,
its can be assigned either M1 or M2, we assume it is assigned to M1.
 In stage 7, the last task D is considered, its start time is 9; as both M2 & M3 available, it
can be assigned to either M2 or M3, we assume it is assigned to M3.
The above steps are shown in the below figure. So the greedy algorithm will do the
machine scheduling of above instance with three machines only
M3 C D

M2 F G

M1
A B E

0
1 2 3 4 5 6 7 8 9 10 11

Knapsack Problem
We are given with a n objects & a knapsack or bag of capacity m. Object i has weight
wi & profit pi . If a fraction xi(0≤ xi ≤1) is placed into the knapsack, then a profit of pixi is
obtained. The objective is to obtain a feasible solution of the knapsack that maximizes the
total profit earned. Since the knapsack capacity is m, the total weight of all chosen objects
should not be more than m. This problem can be stated as:

Maximize  px
1 i  n
i i  Eqn 1

Subject to wx  m
1i  n
i i  Eqn 2

and 0≤xi≤1, 1≤i≤n  Eqn 3

The feasible solution is any set {x1, x2, . . . , xn} Eqn2 & Eqn3. An optimal solution is a feasible
solution for which Eqn1 is maximized.

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 4

Example:
Consider the following instance of knapsack problem: n=3, m=20, (p1,p2,p3)=(25,24,15)
and (w1,w2,w3)=(18,15,10) find the optimal solution.

Weights: (w1,w2,w3)=(18,15,10)
Profits : (p1,p2,p3)=(25,24,15)
Profit/weight ratio:
p1 25 p 2 24 p 3 15
= =1.39 = =1.6 = =1.5
w1 18 w2 15 w3 10
pi
Arranging the in decreasing order 1.6>1.5>1.39
wi
i.e Weights: (w2,w3,w1)=(15,10,18)
Profits : (p2,p3,p1)=(24,15,25)
Let RC be the remaining capacity of the Knapsack after placing the i th item.
RC
Item Wi Pi X=1 or Profit=X*Pi RC=RC-Wi*X
Wi
- - - - 20
2 15 24 1 24 5
3 10 15 0.5 7.5 0
1 18 25 0 0 0

∴ Total Profit= 31.5

∴ (x2, x3, x1) = ( 1, 0.5, 0)
i.e (x1, x2, x3) = (0, 1, 0.5)

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 5

Algorithm GreedyKnapsack(m,n)
//Greedy algorithm for Knapsack problem.
//I/P: P[1:n] & W[1:n] contains the profits & weights of n items respectively & m is the
//capacity of the knapsack.
//O/P: X[1:n] is represents the solution vector.
{
for i=1 to n do
X[i]=0.0;
U=m;
for i=1 to n do
{
if(W[i]>U) then break;
x[i]=1.0;
U=U-W[i];
}
U
if (i≤n) then x[i]=
W [i ]
}

Theorem:
p1 p 2 pn
If ≥ ≥. . .≥ then GreedyKnapsack generates an optimal solution to the given
w1 w2 wn
instance of the knapsack problem.
Proof: Let x=(x1,x2, . . .,xn) be the solution generated by the GreedyKnapsack. If all the x i
equal to 1, then clearly the solution is optimal. Let j be the least index such that x j≠1. xi=1
for 1≤i<j , xi=0 for j<i≤n and 0≤xj<1.
Let y=(y1,y2,. . . yn) be an optimal solution. We know that all the optimal solutions fill
the knapsack exactly, so we can assume ∑wiyi=m.
Let k be the least index such that yk≠xk . It also follows that yk<xk. To see this consider the
three possibilities k<j, k=j, k>j.
i. If k<j, then xk=1. But yk≠xk, so yk<xk.

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 6

 ii. If k=j, then since ∑wixi=m and yi=xi for 1≤i<j, it follows that either y k<xk or
∑wiyi>m.
iii. If k>j, then ∑wiyi>m and this is not possible.
Now suppose we increase yk to xk and decrease as many of (yk+1 , yk+2. . ., yn) as necessary so
that the total capacity used still m . This results in a new solution z=(z 1,z2,. . . zn) with zi=xi,
1≤i≤k, and  w ( y  z ) = wk(zk-yk). Then for z we have:
k i  n
i i i

pk pi
 pizi =
1i  n
 piyi  ( zk  yk )wk
1i  n
  ( yi  zi ) wi
wk k in wi

pk
≥  py
1i  n
i i  [( zk  yk ) wk   ( y  z )w ] w
k i  n
i i i
k

=  py
1 i  n
i i

If ∑pizi >∑piyi , then y could not have been optimal solution. If these sums are equal,
then either z=x and x is optimal, or z≠x. In z≠x repeated use of the above argument will
either shows that y is not optimal, or transform y into x thus it shows that x is too optimal.

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 7

Job Sequencing with Deadlines:
We are given a set of n jobs(1≤i≤n), each job I is associated with an integer deadline
di≥0 and a profit pi>0. For any job i the profit pi is obtained if an only if the job is completed
by its deadline. To complete a job one has to process the job on a machine for one unit of
time. Only one machine is available for processing jobs.
The feasible solution for this problem is a subset J of jobs which are completed by
their deadlines. The value of a feasible solution J is the sum of the profits of jobs in J or

p
iJ
i . An optimal solution is a feasible solution with maximum value.

Example: Let n=4, (p1,p2,p3,p4)=(100,10,15,27) and (d1,d2,d3,d4)=(2,1,2,1)

The feasible solutions are:

Feasible Solution Processing Sequence Value

(1,2) 2,1 110
(1,3) 1,3 or 3,1 115
(1,4) 4,1 127
(2,3) 2,3 25
(3,4) 4,3 42
(1) 1 100
(2) 2 10
(3) 3 15
(4) 4 27

The optimal solution is (1,4). These jobs are processed in the sequence job 4 followed by
job 1 i.e job 4 is processed by deadline 1 & job 1 is processed by deadline 2 and the total
profit is 127(100+27).

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 8

The following is the algorithm for the high level description for Job Sequencing with
deadlines:
Algorithm GreedyJob(d,J,n)
// J is a set of jobs can be completed by their deadlines.
{
J={1}
for i=2 to n do
{
if ( all jobs in J  { i } can be completed by their deadlines)
then J=J  { i }
}
}

The detailed Greedy algorithm for Job Sequencing with deadlines and profit is given below:
Algorithm JS(d, J, n)
//d[i]≥1, 1≤i≤n are the deadlins, n≥1. The jobs are ordered such that p[1]≥p[2]≥. . . ≥p[n].
// J[i] is the ith job in the optimal solution, 1≤i≤k. Also at termination d[ J[i] ] ≤ d[ J[i+1] ].
{
d[0]=J[0]=0;
J[1]=1
k=1
for i=2 to n do
{
r=k
while ((d[ J[r] ]>d[i]) and (d[ J[r] ]≠r)) do
r=r-1;
if ((d[ J[r] ]≤d[i]) and d[i]>r) then
{
for q=k to (r+1) step-1 do
J[q+1]=J[q]
J[r+1]=i;

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 9

 k=k+1
}
}
return k
}

Minimum Cost Spanning Trees:

Let G=(V,E) be an undirected connected graph, a subgraph t=(V,E’) of G is a spanning
tree of G if and only if t is a tree.
or
Spanning tree of a given graph is a connected acyclic graph that contains the all the
vertices of graph. For the given below graph the different spanning tress are shown below.

Spanning trees have many applications:

i. They can be used to obtain an independent set of circuit equations for an electric
network.

ii. They can be used to check the cyclicity & connectivity property of the graph.

The minimum spanning tree of a given graph is a connected acyclic subgraph that contains
all the vertices of the graph such that the sum of the edge weights of the tree should be
minimum.
or
Let G=(V,E) be an undirected connected graph, a subgraph t=(V,E’) of G is a spanning tree of
G if and only if t is a tree such that the sum of the edge weights of the tree should be
minimum.

There are two algorithms to find the minimum spanning tree of the given graph:

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 10

 i. Prim’s Algorithm
ii. Kruskal’s Algorithm

Prim’s Algorithm:
The greedy method Prim’s algorithm to obtain a minimum cost spanning tree builds
the tree edge by edge. The next edge to include is chosen according to some optimization
criterion. The best criterion is to choose an edge that result in a minimum increase in
the sum of the costs of the edges so for included to tree.
If A is the set of edges selected so far, then A forms a tree. The next edge (u,v) to be
included in A is a minimum cost edge not in A with property that A  { (u,v) }is also a tree.
The following is the algorithm for Prim’s algorithm
Algorithm Prim (E, cost, n, t)
// E is the set of edges in G. cost[1:n,1:n] is the cost matrix of an n vertex graph such that
// cost[i, j] is either a positive real number or ∞ if no edge (i,j) exists.
// A minimum spanning tree is computed and stored as a set of edged in the array
// t[1:n-1, 1:2]. ( t[I,1], t[i,2] is an edge in the minimum cost spanning tree. The final cost is
//returned.
{
Let (k,l) be an edge of minimum cost in E
mincost= cost[ k, l]
t[1,1]=k; t[1,2]=l;
for i=1 to n do
if (cost[ i , l ] < cost[ i , k]) then
near[ i ]=l;
else
near[ i ]=k;
near[ k ]=near[ l ]=0;
for i=2 to n-1 do
{
Let j be an index such that near[ j ]≠0 and cost[j, near[ j ]] is minimum.
t[i,1]=j; t[i,2]=near[j];

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 11

 mincost = mincost+cost[j,near[ j ] ];
near[j]= 0;
for k=1 to n do
if (near[k]≠0) and (cost[k, near[k]> cost[k,j]))
then near[k]=j;
}
return mincost;
}
The following is the illustration for the same:

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 12

Kruskal’s Algorithm:
Kruskal’s algorithm is the other algorithm for finding the minimum spanning tree of
the given graph. In this algorithm the edges of the graph are considered in non decreasing
order of edge weights. The set t of edges so far selected for the spanning tree be such that it
is possible to complete t into a tree.
The high level algorithm for Kruskal’s algorithm is:
Algorithm Kruskal(G)
{
t= Ø;
while ( ( t has less than n-1 edges ) and (E ≠Ø) do
{
Choose an edge (v,w) from E of lowest cost;
Delete (v,w) from E;
if (v, w) does not create a cycle in t then add (v,w) to t;
else discard (v,w);
}
}

The detailed Kruskal’s algorithm is given below

Algorithm Kruskal (E, cost, n ,t)
// E is the set of edges in G. G has n vertices. cost[u,v] is the cost of edge (u,v).
// t is the set of edges in minimum cost spanning tree. The final cost is returned.
{
Construct a heap out of the edge cost using Heapify;
for i= 1 to n do
parent[i]=-1;
i=0; mincost=0.0;
while(( i<n-1) and heap not empty) do
{
Delete a minimum cost edge (u,v) from heap and rehepify using Adjust;
j= Find(u); k= Find(v);

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 13

 if ( j≠k ) then
{
i=i+1;
t[i,1]=u; t[i,2]=v;
mincost= mincost+cost[u,v]
Union(j,k);
}
}
if (i ≠ n-1) then Write( “No spanning Tree”);
else return mincost;
}

The illustration is shown below:

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 14

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 15
Single Source Shortest Paths (Dijikstra’s Algorithm):
The single source shortest path will ask us to find the shortest path from the given
single source node to the all other nodes called destination nodes.
In the graph there may be more than one path from source to destination, so we
need to find shortest path between that pair of vertices among all possible paths.
The algorithm for the same is given below:
Algorithm ShortestPaths( v, cost, dist,n)
// dist[j], 1≤j≤n, is set to the length of the shortest path from vertex v to vertex j in a
// digraph G with n vertices. dist[v] is set to zero. G is represented by its cost matrix
//cost[1:n, 1:n]
{
for i=1 to n do
{
S[i]=false;
dist[i]=cost[v,i];
}
S[v]=true;
dist[v]=0.0;
for num=2 to n do
{
Choose u from among the vertices not in S such that dist[u] is minimum;
S[u]=true;
for (each w adjacent to u with S[w]=false) do
if (dist[w]>dist[u]+cost[u,w])) then
dist[w]= dist[u]+cost[u,w];
}
}

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 16

The illustration of the algorithm is given below:

The shortest distances are:

Hemanth T D, Dept. of ISE, Acharya Institute of Technology. Page 17