ADVANCED

ALGORITHMIC
ANALYSIS
Greedy Algorithms – Huffman Trees
and Codes
GREEDY ALGORITHMS
Greed, for lack of a better word, is good! Greed is right! Greed works!

• Algorithms for optimization problems typically go through

a sequence of steps, with a set of choices at each step.
• For many optimization problems, using dynamic
programming to determine the best choices is overkill;
simpler, more efficient algorithms will do.
• A greedy algorithm always makes the choice that looks best
at the moment.
2
 Insert Image

PROBLEMS

• Activity Selection Problem

• ATM – Change Making Problem
• Minimum Spanning Tree – Prims and Kruskal Algorithm
• Shortest Path Algorithm - Dijkstra’s Algorithm
• Huffman Trees and Codes
• Knapsack Problem

3
 Insert Image

HUFFMAN CODES
algorithm developed by David A. Huffman – 1952

Key Values a b c d e f
Frequency / 0.45 0.13 0.12 0.16 0.09 0.05
Probability

4
 Insert Image

HUFFMAN ALGORITHM
A tree constructed by the above algorithm is called a Huffman tree. It defines in
the manner described below a Huffman code.
• Step 1 Initialize n one-node trees and label them with the symbols of
the alphabet given. Record the frequency of each symbol in its tree’s
root to indicate the tree’s weight. (More generally, the weight of a
tree will be equal to the sum of the frequencies in the tree’s leaves.)
• Step 2 Repeat the following operation until a single tree is obtained.
Find two trees with the smallest weight (ties can be broken
arbitrarily). Make them the left and right subtree of a new tree and
record the sum of their weights in the root of the new tree as its
weight.
5
 f e c b d a
0.05 0.09 0.12 0.13 0.16 0.45

f e c b d a
0.05 0.09 0.12 0.13 0.16 0.45

0.14

f e
0.05 0.09

c b d a
0.12 0.13 0.14 0.16 0.45

f e
0.05 0.09
6
 c b d a
0.12 0.13 0.14 0.16 0.45

f e
0.05 0.09

d a
0.14 0.16 0.25 0.45

f e c b
0.05 0.09 0.12 0.13

7
 d a
0.14 0.16 0.25 0.45

f e c b
0.05 0.09 0.12 0.13

a
0.30 0.45
0.25

c b d
0.12 0.13 0.14 0.16

f e
0.05 0.09

8
 a
0.55
0.45

0.25 0.30

c b d
0.12 0.13 0.14 0.16

f e
0.05 0.09

9
HUFFMAN TREE
1.00

a
0.45 0.55

0.25 0.30

c b d
0.12 0.13 0.14 0.16

f e
0.05 0.09

10
HUFFMAN CODES a •0

0
1.00
1 b • 101

a
0.55
c • 100
0.45 1
0
d • 111
0.25 0.30
0 1 0 1
e • 1101
c b
0.12 0.13 0.14
d
0.16
f • 1100
0 1
f e
0.05 0.09
11
WORKOUT
Construct Huffman Tree and predict Huffman
code for the following set of keys and frequencies?
a:1, b:1, c:2, d:3, e:5, f:8, g:13, h:21
Construct Huffman Tree and predict Huffman
code for the following set of string:
A_DEAD_DAD_CEDED_A_BAD_BABE_A_B
EADED_ABACA_BED

12