Compression

Safeen H. Rasool
Assist. Lecturer
 Introduction to Entropy
• The number of bits required to represent an
image can be made based on the information
content using an entropy (variable length
coding) approach such as a Huffman code.
• Highly probable symbols are represented by
short code-words while less probable symbols
are represented by longer code-words.
• The result is a reduction in the average
number of bits per symbol.
• Example: a Fixed length coding
• Average bits/symbol
• = 0.75*2 + 0.125*2 + 0.0625*2 + 0.0625*2
• =2.0 bits/pixel
• Entropy (Variable Length) Coding

• Average bits/symbol
• = 0.75*1 + 0.125*2 + 0.0625*3 + 0.0625*3
• = 1.375
• bits/pixel (A 30% saving with no loss)
Variable Length
 Shannon-Fano Coding
• The Shannon-Fano technique has as an
advantage its simplicity. The code is constructed
as follows:
• the source messages a(i) and their probabilities
p( a(i) ) are listed in order of nonincreasing
probability.
• This list is then divided in such a way as to form
two groups of as nearly equal total probabilities
as possible.
• Each message in the first group receives 0 as
the first digit of its codeword.
• the messages in the second half have
codewords beginning with 1.
• Each of these groups is then divided according
to the same criterion and additional code
digits are appended.
• Shannon-Fano algorithm is not guaranteed to
produce an optimal code
 Lempel–Ziv–Welch(LZW)
• One of the most common algorithms used in computer
graphics is the Lempel-Ziv-Welch, or LZW, compression
scheme.
• This lossless method of data compression is found in
several image file formats, such as GIF and TIFF.
• LZW is referred to as a substitutional or dictionary-based
encoding algorithm. The algorithm builds a data
dictionary (also called a translation table or string table) of
data occurring in an uncompressed data stream. Patterns
of data (substrings) are identified in the data stream and
are matched to entries in the dictionary.
 Lempel–Ziv–Welch(LZW)
s k entry/ code string s k Entry/ code string
output output
1 A ABB 1 A 11 ABBA
Initial dictionary entries 2 B A EOF
3 C
NIL 1 A
A 2 B 4 AB
B 4 AB 5 BA LZW Decompression of the code
AB
BA
5
2
BA
B
6
7
ABB
BAB
12452346
B 3 C 8 BC 1
C 4 AB 9 CA
AB 6 ABB 10 ABA Decoded as ABABBABCABABBA
 LZW Algorithm [6]
• Simple LZW Fails with input
ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A
BA B 5 7 BAB

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB
B C 2 8 BC

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC
C A 3 9 CA

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA
A B
AB B
ABB A 6 10 ABBA

ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABB A 6 10 ABBA
A B
AB B
ABB A
ABBA X 10 11 ABBAX
ABABBABCABBABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 4
ABBA X 10 11 ABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 410
ABBA X 10 11 ABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 4108
ABBA X 10 11 ABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 410810
ABBA X 10 11 ABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 4108102
ABBA X 10 11 ABBAX
 s k entry/ code string
s c output code string output
1 A 1 A
Initial dictionary entries 2 B Initial dictionary entries 2 B
3 C 3 C
A B 1 4 AB NIL 1 A
B A 2 5 BA A 2 B 4 AB
A B B 4 AB 5 BA
AB B 4 6 ABB AB 5 BA 6 ABB
B A BA 2 B 7 BAB
BA B 5 7 BAB B 3 C 8 BC
B C 2 8 BC C 6 ABB 9 CA
C A 3 9 CA ABB 10 ???
A B
AB B
ABABBABCABBABBAX
ABB A 6 10 ABBA
A B
AB B
ABB A 41081025
ABBA X 10 11 ABBAX