GREEDY ALGORITHM
GREEDY ALGORITHMS
Greedy algorithms solve problems by making the choice that
seems best at the particular moment. Many optimization
problems can be solved using a greedy algorithm.
Some problems have no efficient solution, but a greedy
algorithm may provide a solution that is close to optimal.
Greedy Algorithm Design
Comparison:
Dynamic Programming Greedy Algorithms
At each step, the choice is At each step, we quickly make a
determined based on choice that currently looks best.
solutions of subproblems. --A local optimal (greedy) choice.
Sub-problems are solved first. Greedy choice can be made first
before solving further sub-problems.
Bottom-up approach Top-down approach
Can be slower, more complex Usually faster, simpler
Applications of Greedy algorithm:
1. Activity selection problem.
2. Knapsack fractional Problem
3. Job Scheduling problem
4. Minimum Spanning Tree problem
5. Single source shortest path problem
6. Huffman tree problem
Activity selection problem
1. Let S={1,2,3,…,n} be a set of ‘n’ proposed activities. The
activities share a resource, which can be used by only one
activity at a time. E.g. a Tennis court ,a Lecture hall etc.
2. Each activity ‘i’ has a start time ‘Si ‘ and a finish time ‘fi ‘.
Where Si <= fi
3. activities ‘i’ and ‘j’ are compatible if interval [Sj , fj] and [Si,
fi ] do not overlap
i.e si >= fj or sj >= fi
4. Activities selection problem select maximum set of
mutually compatible activities.
5. complexity of this algorithm is O(n2).
Activity-Selection Problem
For a set of proposed activities that wish to
use a lecture hall, select a maximum-size
subset of “compatible activities”.
Set of activities: S={a1,a2,…an}
Duration of activity ai: [start_timei, finish_timei)
Activities sorted in increasing order of finish time:
i 1 2 3 4 5 6 7 8 9 10 11
start_timei 1 3 0 5 3 5 6 8 8 2 12
finish_timei 4 5 6 7 8 9 10 11 12 13 14
Activity-Selection Problem
i 1 2 3 4 5 6 7 8 9 10 11
start_timei 1 3 0 5 3 5 6 8 8 2 12
finish_timei 4 5 6 7 8 9 10 11 12 13 14
time a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11
0
1
2
3 Compatible activities:
4 {a3, a9, a11},
5
6 {a1,a4,a8,a11},
7
8 {a2,a4,a9,a11}
9
10
11
12
13
14
Thus the final activity schedule is:
{a2,a4,a9,a11}
Activity-Selection Problem
Set of activities: S={a1,a2,…a10}
Si={1,2,3,4,7,8,9,9,11,12}
Fi={3,5,4,7,10,9,11,13,12,14}
Compute a schedule where the largest number of activities takes place.
FRACTIONAL KNAPSACK PROBLEM
Fraction of items can be taken rather than having to make a
binary (0-1) choice for each item.
Steps for solution:
Step 1: calculate value per weight ratio.
i.e. p i = v i / w i
Step 2: Now arrange the value of p i in decreasing order.
Step 3: Now fill the knapsack according to the decreasing value
of p i .
Example: consider 5 items along their respective weights and values
I={I1,I2,I3,I4,I5}
W={5,10,20,30,40}
V={30,20,100,90,160}
The capacity of knapsack W=60.
Find the solution of the fractional knapsack problem.
Sol:
Item Wi vi
I1 5 30
I2 10 20
I3 20 100
I4 30 90
I5 40 160
Taking value per weight ratio i.e. p i = v i / w i
Item Wi vi pi =vi / wi
I1 5 30 6.0
I2 10 20 2.0
I3 20 100 5.0
I4 30 90 3.0
I5 40 160 4.0
Now arrange the value of pi in decreasing order
Item Wi vi pi =vi / wi
I1 5 30 6.0
I3 20 100 5.0
I5 40 160 4.0
I4 30 90 3.0
I2 10 20 2.0
Now fill the knapsack according to the decreasing value of pi.
First we choose item I1 whose weight is 5, then choose item I3
whose weight is 20. now the total weight in knapsack is
5+20=25
Now , the next item is I5 and its weight is 40, but we want only
35 . So we choose fractional part of it. i.e
35
20
60
5
The value of fractional part of I5 is (160/40)* 35=140
Thus the maximum value is= 30+100+140= 270
Q1:For the given set of items and knapsack capacity = 60 kg,
find the optimal solution for the fractional knapsack problem
making use of greedy approach.
Item Weight Value
1 5 30
2 10 40
3 15 45
4 22 77
5 25 90
Q2: Define a knapsack problem and describe its formulation.
Find the optimal solution to the knapsack instance n=5,
W={20,30,40,10,7}
P={7,8,9,1,6}
And C=80, using Greedy method.
Job scheduling with deadlines
Problem Statement
In job sequencing problem, the objective is to find a sequence of
jobs, which is completed within their deadlines and gives
maximum profit.
Solution
Let us consider, a set of n given jobs which are associated with deadlines and
profit is earned, if a job is completed by its deadline. These jobs need to be
ordered in such a way that there is maximum profit.
It may happen that all of the given jobs may not be completed within their
deadlines.
Assume, deadline of ith job Ji is di and the profit received from this job is pi.
Hence, the optimal solution of this algorithm is a feasible solution with
maximum profit.
Question
N=5
Jobs J1 J2 J3 J4 J5
Profit 20 15 10 5 1
Deadlines 2 2 1 3 3
N=5
Solution Jobs J1 J2 J3 J4 J5
Profit 20 15 10 5 1
Deadlines 2 2 1 3 3
Job considered Slot assign Solution Profit
J1 [1,2] J1 20
J2 [1,2][0,1] J1, J2 20+15
J3 [1,2], [0,1] J1, J2 20+15
J4 [1,2],[0,1],[2,3] J1, J2, J4 20+15+5
J5 [1,2],[0,1],[2,3] J1, J2, J4 20+15+5
Job sequencing= J1->J2->J4
Total profit=40
Practice Question
N=7
Jobs J1 J2 J3 J4 J5 J6 J7
Profit 35 30 25 20 15 12 5
Deadlines 3 4 4 2 3 1 2
SPANNING TREE
A spanning tree is a sub-graph of an undirected connected graph,
which includes all the vertices of the graph with a minimum possible
number of edges. If a vertex is missed, then it is not a spanning tree.
The edges may or may not have weights assigned to them.
The total number of spanning trees with n vertices that can be
created from a complete graph is equal to n(n-2).
Spanning tree: A spanning tree of a connected graph G is a connect
sub graph of G having no cycle.
A B
Original Graph
C D
Spanning Trees
MINIMUM SPANNING TREE
A minimum spanning tree is a spanning tree in which the sum of
the weight of the edges is as minimum as possible.
Example of a Spanning Tree
Let's understand the above definition with the help of the example below.
The initial graph is:
The possible spanning trees from the above graph are:
Real Life Application of a MST
A cable TV company is laying cable in a new
neighborhood. If it is constrained to bury the cable
only along certain paths, then there would be a
graph representing which points are connected by
those paths. Some of those paths might be more
expensive, because they are longer, or require the
cable to be buried deeper; these paths would be
represented by edges with larger weights. A
minimum spanning tree would be the network with
the lowest total cost.
The minimum spanning tree from a graph is found using the
following algorithms:
1. Prim's Algorithm
2. Kruskal's Algorithm
Prim's Algorithm
It falls under a class of algorithms called greedy algorithms that find
the local optimum in the hopes of finding a global optimum.
We start from one vertex and keep adding edges with the lowest
weight until we reach our goal.
The steps for implementing Prim's algorithm are as follows:
1. Initialize the minimum spanning tree with a vertex chosen at
random.
2. Find all the edges that connect the tree to new vertices, find the
minimum and add it to the tree
3. Keep repeating step 2 until we get a minimum spanning tree
4. The time complexity of Prim's algorithm is O(E log V).
Example of Prim's algorithm
Start with a weighted graph
Choose a vertex
Choose the shortest edge from this vertex and add it
Choose the nearest vertex not yet in the solution
Choose the nearest edge not yet in the solution, if there are
multiple choices, choose one at random
Repeat until you have a spanning tree
KRUSKAL ALGORITHM
We start from the edges with the lowest weight and keep
adding edges until we reach our goal.
The steps for implementing Kruskal's algorithm are as
follows:
1.Sort all the edges from low weight to high
2.Take the edge with the lowest weight and add it to the
spanning tree. If adding the edge created a cycle, then
reject this edge.
3.Keep adding edges until we reach all vertices.
The time complexity Of Kruskal's Algorithm is: O(E log E)
Kruskal’s algorithm
25
B D
5
A 30 5
15 10
50
E
C
30
1. STEP 1: Draw all the nodes w/o any edges
B D
E
C
STEP 2: make a list of all the edges according to
their weights: Sort the list of edges
according to weights
Edge weight Edge weight
A->B 5 A->B 5
A->C 50 D->C 5
A->E 15 E->D 10
B->C 30 A->E 15
B->D 25 B->D 25
C->E 30 B->C 30
D->C 5 C->E 30
E->D 10 A->C 50
1. STEP 1: choose an edge with minimum cost: A-
>B (wts=5) draw an edge
B D
5
A
E
C
1. STEP 2: choose another edge with minimum cost
D->C (wts=5) draw an edge
B D
5
A 5
E
C
1. STEP 2: choose another edge with minimum cost
E->D (wts=10) draw an edge
B D
5
A
5 10
E
C
1. STEP 2: choose another edge with minimum
weight A->E (wts=15)
Total cost=5+5+15+10=35
B D
5
A
5 10
15
E
C
Practice Q: Generate minimum cost spanning tree for the following graph
using Prims and Kruskal Algorithm
Practice Q: Generate minimum cost spanning tree for the following
graph using Prims and Kruskal Algorithm
HUFFMAN CODE
Huffman coding is a lossless data compression algorithm. The
idea is to assign variable-length codes to input characters,
lengths of the assigned codes are based on the frequencies of
corresponding characters. The most frequent character gets the
smallest code and the least frequent character gets the largest
code.
There are mainly two major parts in Huffman Coding
1) Build a Huffman Tree from input characters.
2) Traverse the Huffman Tree and assign codes to characters.
Steps to build Huffman Tree
Input is array of unique characters along with their frequency of occurrences
and output is Huffman Tree.
1. Create a leaf node for each unique character and build a heap of all leaf
nodes
2. Extract two nodes with the minimum frequency from the min heap.
3. Create a new internal node with frequency equal to the sum of the two
nodes frequencies. Make the first extracted node as its left child and the other
extracted node as its right child. Add this node to the min heap.
4. Repeat steps#2 and #3 until the heap contains only one node. The remaining
node is the root node and the tree is complete.
Huffman Coding Complexity
5. The time complexity for encoding each unique character based on its
frequency is O(nlog n).
Message: BCCABBDDAECCBBAEDDCC
Length=20
1 character in ASCII code takes 8 bits
Hence 20 characters will take 20*8=160 bits
Now apply Huffman code
Character Count code
A 3
B 5
C 6
D 4
E 2
Character a b c d e f
frequency 5 9 12 13 16 45
Step 1. Build a heap that contains 6 nodes where each node represents root of a tree
with single node.
Step 2 Extract two minimum frequency nodes from min heap. Add a new internal
node with frequency 5 + 9 = 14
Remaining nodes are: 14, c12, d13, e16, f45
Step 3: Extract two minimum frequency nodes from heap. Add a new
internal node with frequency 12 + 13 = 25
Remaining nodes are: 14, 25, e16, f45
Step 4: Extract two minimum frequency nodes. Add a new internal node
with frequency 14 + 16 = 30
Remaining nodes are: 30, 25, f45
Step 5: Extract two minimum frequency nodes. Add a new internal
node with frequency 25 + 30 = 55
Remaining nodes are: 55, f45
Step 6: Extract two minimum frequency nodes. Add a new internal node
with frequency 45 + 55 = 100
Remaining nodes are: 100
Since the heap contains only one node, the algorithm stops here.
Steps to print codes from Huffman Tree:
Traverse the tree formed starting from the root. Maintain an auxiliary array. While
moving to the left child, write 0 to the array. While moving to the right child, write
1 to the array. Print the array when a leaf node is encountered.
The codes are as follows:
Character Code word
f 0
c 100
d 101
a 1100
b 1101
e 111
Question for practice:
Character f e c b d a
frequency 5 9 12 13 16 45
Dijkstra Algorithm
Single source shortest path
Dijkstra Algorithm-
•Dijkstra Algorithm is a very famous greedy
algorithm.
•It is used for solving the single source shortest path
problem.
•It computes the shortest path from one particular
source node to all other remaining nodes of the
graph.
Conditions-
It is important to note the following points regarding
Dijkstra Algorithm-
•Dijkstra algorithm works only for connected graphs.
•Dijkstra algorithm works only for those graphs that do not
contain any negative weight edge.
•The actual Dijkstra algorithm does not output the shortest
paths.
•It only provides the value or cost of the shortest paths.
•By making minor modifications in the actual algorithm,
the shortest paths can be easily obtained.
•Dijkstra algorithm works for directed as well as undirected
graphs.
Dijkstra Algorithm Analysis
The running time of Dijkstra’s algo on a graph with
edges E and vertices V can be expressed as a function
of |E| and |V| using Big-O notation. The simplest
implementation of this algo stores vertices of set Q in
an ordinary linked list or array, and operation Extract-
Min(Q) is simply a linear search through all vertices in
the Q, hence, the running time is O(|V|2 |+|E|)=O(V2)
Edge Relaxation
Consider the edge (a,b) in the following graph-
Here, d[a] and d[b] denotes the shortest path estimate for
vertices a and b respectively from the source vertex ‘S’.
Now,
If d[a] + w < d[b]
then d[b] = d[a] + w and Π[b] = a
This is called as edge relaxation.
Steps of the Dijkstra’s algorithm
1. Initializes the distance of source vertex to zero and remaining all other vertices to
infinity.
2. Set source node to current node and put remaining all nodes in the list of unvisited
vertex list. Compute the tentative distance of all immediate neighbour vertex of the current
node.
3. If the newly computed value is smaller than the old value, then update it.
For example, C is the current node, whose distance from source S is dist (S, C) = 5.
•Consider N is the neighbour of C and weight of edge
(C, N) is 3. So the distance of N from source via C would be 8.
•If the distance of N from source was already computed and if it is greater than 8 then relax
edge (S, N) and update it to 8, otherwise don’t update it.
4. When all the neighbours of a current node are explored, mark it as visited.
Remove it from unvisited vertex list. Mark the vertex from unvisited vertex list with
minimum distance and repeat the procedure.
5. Stop when the destination node is tested or when unvisited vertex list becomes
empty.
Solution
Solve yourself:
The Travelling Salesman
Problem
Overview
The goal of the Traveling Salesman Problem (TSP) is to find the “cheapest” tour of a
select number of “cities” with the following restrictions:
●
You must visit each city once and only once
●
You must return to the original starting point
Applications of the TSP
Routing around Cities
Computer Wiring - connecting together computer
components using minimum wire
length
Genome Sequencing - arranging DNA fragments in
sequence
Job Sequencing - sequencing jobs in order to
minimise total set-up time
between jobs
Wallpapering to Minimise Waste
68
Q: A newspaper agent daily drops the newspaper to the area
assigned in such a manner that he has to cover all houses in the
respective area with minimum travel cost. Compute the
minimum travel cost.
H1
7
5 4
H6
H2 6
3
3
H7
4 6
2
H5
2
7 4
H8
H3 H4
1
Sol: the cost-adjacency matrix of graph G is as follows:
COST IJ=
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
The tour starts from area H1 and then select the minimum cost area
reachable from H1
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H6 because it is the minimum cost area reachable from H1 then
select min cost area reachable from H6
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H7 because it is the minimum cost area reachable from H6 then
select min cost area reachable from H7
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H8 because it is the minimum cost area reachable from H7 then
select min cost area reachable from H8
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H5 then select min cost area reachable from H5
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H3 then select min cost area reachable from H3
H1 H2 H3 H4 H5 H6 H7 H8
H1 0 5 0 6 0 4 0 7
H2 5 0 2 4 3 0 0 0
H3 0 2 0 1 0 0 0 0
H4 6 4 1 0 7 0 0 0
H5 0 3 0 7 0 0 6 4
H6 4 0 0 0 0 0 3 0
H7 0 0 0 0 6 3 0 2
H8 7 0 0 0 4 0 2 0
Mark area H4 and select min cost area reachable from H4 it is H1
So, using the greedy strategy we get the following :
H1 -> H6 -> H7 -> H8 -> H5-> H2 -> H3 -> H4 -> H1
THE MINIMUM TRAVEL COST:
=4+3+2+4+3+2+1+6=25
