20-02-2023

Adaptive Huffman Coding

Why Adaptive Huffman Coding?

 Huffman coding suffers from the fact that the
to compress a file some knowledge of the
probabilities of the symbols is required
 this can need more bit to encode the file
 if this information is unavailable compressing the file
requires two passes
 first pass: find the frequency of each symbol and
construct the huffman tree
 second pass: compress the file

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

1
 20-02-2023

The key idea

 The key idea is to build a Huffman tree
that is optimal for the part of the
message already seen, and to
reorganize it when needed, to maintain
its optimality

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Adaptive Huffman Coding

 Adaptive Huffman determines the mapping to
codewords using a running estimate of the
source symbols probabilities
 Effective exploitation of locality
For example suppose that a file starts out with a series of
a character that are not repeated again in the file. In static
Huffman coding, that character will be low down on the
tree because of its low overall count, thus taking lots of
bits to encode. In adaptive huffman coding, the character
will be inserted at the highest leaf possible to be decoded,
before eventually getting pushed down the tree by higher-
frequency characters

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

2
 20-02-2023

Adaptive Huffman Coding

 only one pass over the data
• overhead
In static Huffman, we need to transmit someway the
model used for compression
• sometimes encoding needs some more bits w.r.t.
static Huffman (without overhead)
But adaptive schemes generally compare well with static
Huffman if overhead is taken into account

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Adaptive Huffman Coding

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

3
 20-02-2023

Algorithm
32 11

11
21 10
f 9

10 7 11 8

5 5 6
5 4
c d 5 e 6
3

2 3
a b
1 2
 Note that node numbering corresponds to the order in which the
nodes are combined by Huffman’s algorithm, first nodes 1 and 2,
then nodes 3 and 4 ...
DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

4
 20-02-2023

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

5
 20-02-2023

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm
 The basis for algorithm FGK is the Sibling
Property (Gallager 1978)
 A binary code tree with nonnegative weights has the
sibling property if each node (except the root) has a
sibling and if the nodes can be numbered in order of
increasing weight with each node adjacent to its
sibling. Moreover the parent of a node is higher in
the numbering

 A binary prefix code is a Huffman code if and

only if the code tree has the sibling property

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

6
 20-02-2023

Algorithm(update procedure)

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm(update procedure)

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

7
 20-02-2023

Algorithm FGK
Algorithm(update procedure)
32 11
33 b
Suppose next


11
21 10
symbol is “b”
f 9 22
 if we update the
11
10 7 11 8 weigths...
 ... sibling
5 5 6
c
65
4 d 5 e 6 property is
3 violated!!
2 43  This is no more a
a b
1 2 Huffman tree
no more ordered by
nondecreasing weight

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm FGK
Algorithm(update procedure)
33 11
32 b
12
11
21 10
f 9
6 10 7 11 8

4 5 5 6
5 4
c d 5 e 6
3
2 3
a b
1 2

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

8
 20-02-2023

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

9
 20-02-2023

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

10
 20-02-2023

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

11
 20-02-2023

Algorithm FGK - summary

 The algorithm starts with only one leaf node,
the 0-node. As the symbols arrive, new leaves
are created and each time the tree is
recomputed
 Each symbol is coded with its codeword in the
current tree, and then the tree is updated
 Unseen symbols are coded with 0-node
codeword and some other bits are needed to
specify the symbol

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

Algorithm FGK
 Algorithm adaptive huffman coding
compares favourably with static
Huffman code, if we consider also
overhead costs (it is used in the Unix utility
compact)
 Exercise
 Construct the static Huffman tree and the FGK tree
(adaptive huffman tree)for the message .Also
generate th ecode and decode the data.
 e eae deedba

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

12
 20-02-2023

References:
 Data compression: Khalid Sayood.

 Animations

www.csie.mcu.edu.tw/~yklee/Courses/Compre
ssion/DC-CH03.pdf

DCE//SEM VI//Extc//Dr. Vishakha Kelkar

13