Image Compression (Chapter 8)

CS474/674 Prof. Bebis

Goal of Image Compression
Digital images require huge amounts of space for
storage and large bandwidths for transmission.
A 640 x 480 color image requires close to 1MB of space.
The goal of image compression is to reduce the amount
of data required to represent a digital image.
Reduce storage requirements and increase transmission rates.
Approaches
Lossless
Information preserving
Low compression ratios

Lossy
Not information preserving
High compression ratios

Trade-off: image quality vs compression ratio
Data Information
Data and information are not synonymous terms!

Data is the means by which information is conveyed.

Data compression aims to reduce the amount of data
required to represent a given quantity of information
while preserving as much information as possible.

Data vs Information (contd)
The same amount of information can be represented
by various amount of data, e.g.:

Your wife, Helen, will meet you at Logan Airport
in Boston at 5 minutes past 6:00 pm tomorrow
night

Your wife will meet you at Logan Airport at 5
minutes past 6:00 pm tomorrow night

Helen will meet you at Logan at 6:00 pm
tomorrow night
Ex1:
Ex2:
Ex3:
Data Redundancy
compression
Compression ratio:
Data Redundancy (contd)


Relative data redundancy:
Example:
Types of Data Redundancy
(1) Coding
(2) Interpixel
(3) Psychovisual

Compression attempts to reduce one or more of these
redundancy types.
Coding Redundancy
Code: a list of symbols (letters, numbers, bits etc.)
Code word: a sequence of symbols used to represent a
piece of information or an event (e.g., gray levels).
Code word length: number of symbols in each code
word
Coding Redundancy (contd)
N x M image
r
k
: k-th gray level
P(r
k
): probability of r
k
l(r
k
): # of bits for r
k

( ) ( )
x
E X xP X x = =

Expected value:
Coding Redundancy (cond)
l(r
k
) = constant length




Example:
Coding Redundancy (contd)
l(r
k
) = variable length
Consider the probability of the gray levels:
variable length
Interpixel redundancy
Interpixel redundancy implies that any pixel value can be
reasonably predicted by its neighbors (i.e., correlated).

( ) ( ) ( ) ( ) f x o g x f x g x a da

= +
}
autocorrelation: f(x)=g(x)
Interpixel redundancy (contd)
To reduce interpixel redundnacy, the data must be
transformed in another format (i.e., through a transformation)
e.g., thresholding, differences between adjacent pixels, DFT

Example:
original
thresholded
(profile line 100)
threshold
(1+10) bits/pair
Psychovisual redundancy
The human eye does not respond with equal sensitivity to
all visual information.

It is more sensitive to the lower frequencies than to the
higher frequencies in the visual spectrum.

Idea: discard data that is perceptually insignificant!

Psychovisual redundancy (contd)
256 gray levels 16 gray levels 16 gray levels
C=8/4 = 2:1

i.e., add to each pixel a
small pseudo-random number
prior to quantization
Example: quantization
How do we measure information?

What is the information content of a message/image?

What is the minimum amount of data that is sufficient
to describe completely an image without loss of
information?


Modeling Information
Information generation is assumed to be a
probabilistic process.

Idea: associate information with probability!




Note: I(E)=0 when P(E)=1
A random event E with probability P(E) contains:
How much information does a pixel contain?

Suppose that gray level values are generated by a
random variable, then r
k
contains:
units of information!
Average information content of an image:


units/pixel
1
0
( ) Pr( )
L
k k
k
E I r r

=
=

using
How much information does an image contain?
(assumes statistically independent random events)
Entropy

Redundancy:
Redundancy (revisited)
where:
Note: of L
avg
= H, the R=0 (no redundancy)
Entropy Estimation
It is not easy to estimate H reliably!






image
Entropy Estimation (contd)
First order estimate of H:


Estimating Entropy (contd)
Second order estimate of H:
Use relative frequencies of pixel blocks :
image
Estimating Entropy (contd)
The first-order estimate provides only a lower-
bound on the compression that can be achieved.

Differences between higher-order estimates of
entropy and the first-order estimate indicate the
presence of interpixel redundancy!

Need to apply transformations!
Estimating Entropy (contd)
For example, consider differences:
16
Estimating Entropy (contd)

Entropy of difference image:
However, a better transformation could be found since:
Better than before (i.e., H=1.81 for original image)
Image Compression Model
Image Compression Model (contd)





Mapper: transforms input data in a way that facilitates
reduction of interpixel redundancies.
Image Compression Model (contd)





Quantizer: reduces the accuracy of the mappers output in
accordance with some pre-established fidelity criteria.
Image Compression Model (contd)





Symbol encoder: assigns the shortest code to the most
frequently occurring output values.
Image Compression Models (contd)





Inverse operations are performed.

But quantization is irreversible in general.
Fidelity Criteria




How close is to ?

Criteria
Subjective: based on human observers
Objective: mathematically defined criteria

Subjective Fidelity Criteria
Objective Fidelity Criteria
Root mean square error (RMS)



Mean-square signal-to-noise ratio (SNR)
RMSE = 5.17 RMSE = 15.67
RMSE = 14.17
Objective Fidelity Criteria (contd)
Lossless Compression
Lossless Methods: Taxonomy
Huffman Coding (coding redundancy)





A variable-length coding technique.
Optimal code (i.e., minimizes the number of code
symbols per source symbol).

Assumption: symbols are encoded one at a time!
Huffman Coding (contd)
Forward Pass
1. Sort probabilities per symbol
2. Combine the lowest two probabilities
3. Repeat Step2 until only two probabilities remain.
Huffman Coding (contd)

Backward Pass
Assign code symbols going backwards


Huffman Coding (contd)
L
avg
using Huffman coding:




L
avg
assuming binary codes:

Huffman Coding/Decoding
After the code has been created, coding/decoding can
be implemented using a look-up table.
Note that decoding is done unambiguously.
Arithmetic (or Range) Coding
(coding redundancy)
No assumption on encode source symbols one at a
time.
Sequences of source symbols are encoded together.
There is no one-to-one correspondence between source
symbols and code words.


Slower than Huffman coding but typically achieves
better compression.


Arithmetic Coding (contd)
A sequence of source symbols is assigned a single
arithmetic code word which corresponds to a sub-
interval in [0,1].

As the number of symbols in the message increases,
the interval used to represent it becomes smaller.

Smaller intervals require more information units (i.e.,
bits) to be represented.
Arithmetic Coding (contd)
Encode message: a
1
a
2
a
3
a
3
a
4
0
1
1) Assume message occupies [0, 1)
2) Subdivide [0, 1) based on the probability of
i

3) Update interval by processing source symbols


Example
a
1
a
2
a
3
a
3
a
4
[0.06752, 0.0688)
or,
0.068
Encode
Example
The message a
1
a
2
a
3
a
3
a
4
is encoded using 3 decimal
digits or 3/5 = 0.6 decimal digits per source symbol.

The entropy of this message is:




Note: finite precision arithmetic might cause problems
due to truncations!
-(3 x 0.2log
10
(0.2)+0.4log
10
(0.4))=0.5786 digits/symbol
1.0
0.8
0.4
0.2
0.8
0.72
0.56
0.48
0.4
0.0
0.72
0.688
0.624
0.592
0.592
0.5856
0.5728
0.5664
a
3
a
3
a
1
a
2
a
4

0.5728
0.57152
056896
0.56768
0.56 0.56 0.5664
Decode 0.572
Arithmetic Decoding
a
1
a
2
a
3
a
4
LZW Coding (interpixel redundancy)
Requires no priori knowledge of pixel probability
distribution values.

Assigns fixed length code words to variable length
sequences.

Patented Algorithm US 4,558,302

Included in GIF and TIFF and PDF file formats

LZW Coding
A codebook (or dictionary) needs to be constructed.

Initially, the first 256 entries of the dictionary are assigned
to the gray levels 0,1,2,..,255 (i.e., assuming 8 bits/pixel)


Consider a 4x4, 8 bit image
39 39 126 126
39 39 126 126
39 39 126 126
39 39 126 126
Dictionary Location Entry
0 0
1 1
. .
255 255
256 -

511 -

Initial Dictionary
LZW Coding (contd)
39 39 126 126
39 39 126 126
39 39 126 126
39 39 126 126



- Is 39 in the dictionary..Yes
- What about 39-39.No
- Then add 39-39 in entry 256

Dictionary Location Entry
0 0
1 1
. .
255 255
256 -

511 -

39-39
As the encoder examines image
pixels, gray level sequences
(i.e., blocks) that are not in the
dictionary are assigned to a new
entry.
Example
39 39 126 126
39 39 126 126
39 39 126 126
39 39 126 126
Concatenated Sequence: CS = CR + P
else:
(1) Output D(CR)
(2) Add CS to D
(3) CR=P
If CS is found:
(1) No Output
(2) CR=CS
(CR) (P)
CR = empty
Decoding LZW
The dictionary which was used for encoding need not
be sent with the image.

Can be built on the fly by the decoder as it reads the
received code words.
Differential Pulse Code Modulation (DPCM)
Coding (interpixel redundancy)
A predictive coding approach.
Each pixel value (except at the boundaries) is predicted
based on its neighbors (e.g., linear combination) to get a
predicted image.
The difference between the original and predicted
images yields a differential or residual image.
i.e., has much less dynamic range of pixel values.
The differential image is encoded using Huffman
coding.
Run-length coding (RLC)
(interpixel redundancy)
Used to reduce the size of a repeating string of characters
(i.e., runs):

1 1 1 1 1 0 0 0 0 0 0 1 (1,5) (0, 6) (1, 1)
a a a b b b b b b c c (a,3) (b, 6) (c, 2)

Encodes a run of symbols into two bytes: (symbol, count)
Can compress any type of data but cannot achieve high
compression ratios compared to other compression methods.
Bit-plane coding (interpixel redundancy)
An effective technique to reduce inter pixel redundancy
is to process each bit plane individually.

(1) Decompose an image into a series of binary images.



(2) Compress each binary image (e.g., using run-length
coding)

Combining Huffman Coding
with Run-length Coding
Assuming that a message has been encoded using
Huffman coding, additional compression can be
achieved using run-length coding.
e.g., (0,1)(1,1)(0,1)(1,0)(0,2)(1,4)(0,2)
Lossy Compression
Transform the image into a domain where compression
can be performed more efficiently (i.e., reduce interpixel
redundancies).
~ (N/n)
2
subimages
Example: Fourier Transform
The magnitude of the
FT decreases, as u, v
increase!
K-1 K-1
K << N
Transform Selection



T(u,v) can be computed using various transformations,
for example:
DFT
DCT (Discrete Cosine Transform)
KLT (Karhunen-Loeve Transformation)
DCT
if u=0
if u>0
if v=0
if v>0
forward
inverse
DCT (contd)
Basis set of functions for a 4x4 image (i.e.,cosines of
different frequencies).





DCT (contd)
DFT WHT DCT
RMS error: 2.32 1.78 1.13
8 x 8 subimages


64 coefficients
per subimage


50% of the
coefficients
truncated
DCT (contd)
DCT minimizes "blocking artifacts" (i.e., boundaries
between subimages do not become very visible).

DFT

i.e., n-point periodicity
gives rise to
discontinuities!

DCT
i.e., 2n-point periodicity
prevents
discontinuities!
DCT (contd)


Subimage size selection:
2 x 2 subimages
original
4 x 4 subimages 8 x 8 subimages
JPEG Compression
JPEG is an image compression standard which was
accepted as an international standard in 1992.
Developed by the Joint Photographic Expert Group of
the ISO/IEC for coding and compression of color/gray
scale images.
Yields acceptable compression in the 10:1 range.
A scheme for video compression based on JPEG
called Motion JPEG (MJPEG) exists

JPEG Compression (contd)
JPEG uses DCT for handling interpixel redundancy.

Modes of operation:
(1) Sequential DCT-based encoding
(2) Progressive DCT-based encoding
(3) Lossless encoding
(4) Hierarchical encoding
JPEG Compression
(Sequential DCT-based encoding)
Entropy
encoder
Entropy
decoder
JPEG Steps
1. Divide the image into 8x8 subimages;

For each subimage do:
2. Shift the gray-levels in the range [-128, 127]
- DCT requires range be centered around 0
3. Apply DCT (i.e., 64 coefficients)
1 DC coefficient: F(0,0)
63 AC coefficients: F(u,v)

Example
(i.e., non-centered
spectrum)
JPEG Steps
4. Quantize the coefficients (i.e., reduce the amplitude of
coefficients that do not contribute a lot).
Q(u,v): quantization table
Example
Quantization Table Q[i][j]
Example (contd)
Quantization
JPEG Steps (contd)
5. Order the coefficients using zig-zag ordering
- Place non-zero coefficients first
- Create long runs of zeros (i.e., good for run-length encoding)


Example
JPEG Steps (contd)

6. Form intermediate symbol sequence and encode coefficients:


6.2 AC coefficients: variable length coding
6.1 DC coefficients: predictive encoding
Intermediate Coding
symbol_1 (SIZE) symbol_2 (AMPLITUDE)
DC
DC (6) (61)

AC (0,2) (-3)
end of block
symbol_1 (RUN-LENGTH, SIZE) symbol_2 (AMPLITUDE)
SIZE: # bits for encoding amplitude
RUN-LENGTH: run of zeros
DC/AC Symbol Encoding
DC encoding



AC encoding
symbol_1 symbol_2
(SIZE) (AMPLITUDE)
If RUN-LENGTH > 15, use symbol (15,0) , i.e., RUN-LENGTH=16
0 0 0 0 0 0 476
(6,9)(476)
[-2048, 2047]=

predictive
coding:
= [-2
10
, 2
10
-1]
1 SIZE10


[-2
11
, 2
11
-1]
1 SIZE11
Entropy Encoding (e.g, variable length)
Entropy Encoding (e.g, Huffman)
Symbol_1
(Variable Length Code (VLC))
Symbol_2
(Variable Length Integer (VLI))
(1,4)(12) (111110110 1100)
VLC VLI
Effect of Quality
90 (58k bytes) 50 (21k bytes) 10 (8k bytes)
best quality,
lowest compression
worst quality,
highest compression
Effect of Quality (contd)
Example 1: homogeneous 8 x 8 block
Example 1 (contd)
Quantized
De-quantized
Example 1 (contd)
Reconstructed Error
Example 2: less homogeneous 8 x 8 block
Example 2 (contd)
Quantized De-quantized
Example 2 (contd)
Reconstructed spatial
Error
JPEG for Color Images
Could apply JPEG on R/G/B components .

It is more efficient to describe a color in terms of its
luminance and chrominance content separately, to enable
more efficient processing.
YUV

Chrominance can be subsampled due to human vision
insensitivity


JPEG for Color Images
Luminance: Received brightness of the light
(proportional to the total energy in the visible band).

Chrominance: Describe the perceived color tone of the
light (depends on the wavelength composition of light
Hue: Specify the color tone (i.e., depends on the peak
wavelength of the light).
Saturation: Describe how pure the color is (i.e., depends on
the spread or bandwidth of the light spectrum).

YUV Color Space
YUV color space
Y is the components of luminance
Cb and Cr are the components of chrominance
The values in the YUV coordinate are related to the values
in the RGB coordinate by:

0.299 0.587 0.114 0
0.169 0.334 0.500 128
0.500 0.419 0.081 128
Y R
Cb G
Cr B
| | | || | | |
| | | |
= +
| | | |
| | | |

\ . \ .\ . \ .
JPEG for Color Images
Image
R
G
B
YVU color
coordinate
Chrominance
Downsampling
(4:2:2 or 4:2:0)
8 X 8
FDCT
Quantizer
Quantization
Table
zigzag
Differential
Coding
Huffman
Encoding
Huffman
Encoding
Bit-stream
Decoded
Image
R
G
B
Chrominance
Upsampling
(4:2:2 or 4:2:0)
8 X 8
IDCT
YVU color
coordinate
Dequantizer
Quantization
Table
De-zigzag
De-DC
coding
Huffman
Decoding
Huffman
Decoding
Bit-stream
Encoder
Decoder
JPEG Modes
JPEG supports several different modes
Sequential Mode
Progressive Mode
Hierarchical Mode
Lossless Mode

Sequential is the default mode
Each image component is encoded in a single left-to-right,
top-to-bottom scan.
This is the mode we have been describing.

Progressive JPEG
The image is encoded in multiple scans, in order to
produce a quick, rough decoded image when
transmission time is long.

Sequential
Progressive
Progressive JPEG (contd)
Send DCT coefficients in multiple scans:
(1) Progressive spectral selection algorithm
(2) Progressive successive approximation algorithm
(3) Hybrid progressive algorithm

Progressive JPEG (contd)
(1) Progressive spectral selection algorithm
Group DCT coefficients into several spectral bands
Send low-frequency DCT coefficients first
Send higher-frequency DCT coefficients next

Progressive JPEG (contd)
(2) Progressive successive approximation algorithm
Send all DCT coefficients but with lower precision.
Refine DCT coefficients in later scans.


Progressive JPEG (contd)
(3) Hybrid progressive algorithm
Combines spectral selection and successive approximation.

Results using spectral selection
Results using successive approximation
Example using successive approximation
after 0.9s after 1.6s
after 3.6s after 7.0s
Hierarchical JPEG
Hierarchical mode encodes the image at several
different resolutions.

Image is transmitted in multiple passes with increased
resolution at each pass.

Hierarchical JPEG (contd)
N x N
N/2 x N/2
N/4 x N/4
f
f
2

f
4

Hierarchical JPEG (contd)
Hierarchical JPEG (contd)
Lossless JPEG
Uses predictive coding (see later)
Lossy Methods: Taxonomy

Lossless Differential Pulse Code
Modulation (DPCM) Coding
Each pixel value (except at the boundaries) is predicted
based on certain neighbors (e.g., linear combination) to
get a predicted image.
The difference between the original and predicted
images yields a differential or residual image.
Encode differential image using Huffman coding.
x
m

Predictor
Entropy
Encoder
p
m

d
m

Lossy Differential Pulse Code
Modulation (DPCM) Coding
Similar to lossless DPCM except that (i) it uses
quantization and (ii) the pixels are predicted from the
reconstructed values of certain neighbors.

Block Truncation Coding
Divide image in non-overlapping blocks of pixels.
Derive a bitmap (0/1) for each block using thresholding.
e.g., use mean pixel value in each block as threshold.
For each group of 1s and 0s, determine reconstruction
value
e.g., average of corresponding pixel values in original block.
Subband Coding
Analyze image to produce components containing
frequencies in well defined bands (i.e., subbands)
e.g., use wavelet transform.
Optimize quantization/coding in each subband.

Vector Quantization
Develop a dictionary of fixed-size vectors (i.e., code
vectors), usually blocks of pixel values.
Partition image in non-overlapping blocks (i.e., image
vectors).
Encode each image vector by the index of its closest
code vector.


Fractal Coding
What is a fractal?
A rough or fragmented geometric shape that can be split into
parts, each of which is (at least approximately) a reduced-size
copy of the whole.
Idea: store images as collections of transformations!
Fractal Coding (contd)
Generated by 4 affine
transformations!
Fractal Coding (contd)
Decompose image into segments (i.e., using standard
segmentations techniques based on edges, color, texture,
etc.) and look them up in a library of IFS codes.

Best suited for textures and natural images.
Fingerprint Compression
An image coding standard for digitized fingerprints,
developed and maintained by:
FBI
Los Alamos National Lab (LANL)
National Institute for Standards and Technology (NIST).

The standard employs a discrete wavelet transform-
based algorithm (Wavelet/Scalar Quantization or
WSQ).
Memory Requirements
FBI is digitizing fingerprints at 500 dots per inch with
8 bits of grayscale resolution.
A single fingerprint card turns into about 10 MB of
data!
A sample fingerprint image
768 x 768 pixels =589,824 bytes
Preserving Fingerprint Details
The "white" spots in the middle of
the black ridges are sweat pores

Theyre admissible points of
identification in court, as are the little
black flesh islands in the grooves
between the ridges
These details are just a couple pixels wide!
What compression scheme should be used?
Better use a lossless method to preserve every
pixel perfectly.

Unfortunately, in practice lossless methods
havent done better than 2:1 on fingerprints!

Would JPEG work well for fingerprint
compression?
Results using JPEG compression
file size 45853 bytes
compression ratio: 12.9
The fine details are pretty much
history, and the whole image has
this artificial blocky pattern
superimposed on it.


The blocking artifacts affect the
performance of manual or
automated systems!
Results using WSQ compression
file size 45621 bytes
compression ratio: 12.9
The fine details are preserved
better than they are with JPEG.

NO blocking artifacts!
WSQ Algorithm
Varying compression ratio
FBIs target bit rate is around 0.75 bits per pixel (bpp)
i.e., corresponds to a target compression ratio of 10.7
(assuming 8-bit images)
This target bit rate is set via a knob on the WSQ
algorithm.
i.e., similar to the "quality" parameter in many JPEG
implementations.
Fingerprints coded with WSQ at a target of 0.75 bpp
will actually come in around 15:1

Varying compression ratio (contd)
Original image 768 x 768 pixels (589824 bytes)
Varying compression ratio (contd)
0.9 bpp compression
WSQ image, file size 47619 bytes,
compression ratio 12.4
JPEG image, file size 49658 bytes,
compression ratio 11.9
Varying compression ratio (contd)
0.75 bpp compression
WSQ image, file size 39270 bytes
compression ratio 15.0
JPEG image, file size 40780 bytes,
compression ratio 14.5
Varying compression ratio (contd)
0.6 bpp compression
WSQ image, file size 30987 bytes,
compression ratio 19.0
JPEG image, file size 30081 bytes,
compression ratio 19.6