Lossy Compression schemes

Scalar Quantization (simplest lossy scheme)

 Many of the fundamental ideas of quantization

and compression are easily introduced in the
simple context of scalar quantization.
 An example: any real number x can be rounded
off to the nearest integer, say
q(x) = round(x)
 Maps the real line R (a continuous space) into a
discrete space.
 Quantizer
 The design of the quantizer has a significant impact on
the amount of compression (i.e., rate) obtained and
loss(distortion) incurred in a lossy compression scheme.
 Quantizer: encoder mapping and decoder mapping.
 Encoder mapping

 – The encoder divides the range of source into a number

of intervals
 – Each interval is represented by a distinct codeword

 Decoder mapping

 – For each received codeword, the decoder generates a

reconstruct value
 Components of a Quantizer
1. Encoder mapping: Divides the range of values that
the source generates into a number of intervals.
Each interval is then mapped to a codeword.

2. It is a many-to-one irreversible mapping. The code

word only identifies the interval, not the original
value.

3. If the source or sample value comes from a analog

source, it is called a A/D converter.
Mapping of a 3-bit Encoder

Codes

000 001 010 011 100 101 110 111

-3.0 -2.0 -1.0 0 1.0 2.0 3.0 input

Mapping of a 3-bit D/A Converter

Input Codes Output

000 -3.5
001 -2.5
010 -1.5
011 -0.5
100 0.5
101 1.5
110 2.5
111 3.5
 Components of a Quantizer

2. Decoder: Given the code word, the decoder gives an

estimated value that the source might have generated.

Usually, it is the midpoint of the interval but a more

accurate estimate will depend on the distribution of the
values in the interval.

In estimating the value, the decoder might generate

some errors.
Step Encoder
 Resulting quantization error (‘noise’)   q( x)  x so that
 Quantization operation:
 – Let M be the number of reconstruction levels

where the decision boundaries are

and the reconstruction levels are

Quantization Problem
 MSQE (mean squared quantization error)
 If the quantization operation is Q

 Suppose the input is modeled by a random variable X with pdf fX(x).

The MSQE is
Quantization Problem
 Rate of the quantizer
 The average number of bits required to represent a

single quantizer output

 –For fixed-length coding, the rate R is:

 For variable-length coding, the rate will depend on the

probability of occurrence of the outputs
Quantization Problem

 Quantizer design problem

 Fixed -length coding

 Variable-length coding
If li is the length of the codeword corresponding to the output yi,
and the probability of occurrence of yi is:

The rate is given by:

Quantizer Design Problem
 Given an input pdf fx(x) and the number of
levels M in the quantizer, find the decision
boundaries {bi} and the reconstruction levels
{yi} so as to minimize the MSQE

 Find the optimum partitions, codes and

representation levels
 Problem
 Given a distortion constraint 2
q  D *

 Find the decision boundaries, reconstruction

levels, and binary codes that minimize the rate,
for fixed length coding and

for variable length

coding
while satisfying the distortion constraint given
above.
 OR
Given a rate constraint R  R
*


find the decision boundaries,

reconstruction levels, and binary codes
that minimize the distortion given by
 UNIFORM QUANTIZER

 Simplest Quantizer
 All intervals are of the same size say ,

except for the two outer intervals.

 ie., the decision boundaries are spaced evenly.

 The reconstruction values are also spaced

evenly, with the same spacing as decision

boundaries.
 They are the midpoints of the decision

boundaries except in the outer intervals

 Midrise quantizer is usually used – since the
number of intervals is even.
 Assumption: Input distribution is symmetric
about the origin and the quantizer is also
symmetric.
 The design of a uniform quantizer consists of
finding step size , that minimizes the
distortion for a given input process and
number of decision levels.
Uniform Quantization of A Uniformly
Distributed Source
Uniform Quantization of A Uniformly
Distributed Source
Example – Image compression
 Assume Image pixels are uniformly
distributed between 0 & 255.
 1 bit/pixel – [0,255] is divided into two
intervals [0,127] and [128,255]
 Reconstruction levels – midpoints of intervals
– {64, 196}.
 2 bits/pixel – Four intervals –
[0,64,128,196,255]boundaries
 Reconstruction levels – {32,96,160,224}
Image Compression

Original 8bits/pixel 3bits/pixel

Image Compression

2bits/pixel 1bit/pixel
The lower rate images are darker than the original, and the lowest-rate
reconstructions are the darkest.
The reason for for this is that the quantization process usually results in
scaling down of the dynamic range of the input.
For example, in the 1-bit-per-pixel reproduction, the highest value is 196
as opposed to 255 for the original image.
As higher gray values represent lighter shades, there is a corresponding
darkening of the reconstruction.
The other thing to notice in the low rate reconstruction is that wherever
there were smooth changes in gray values, there are now abrupt
transitions. This is especially evident in the face and neck area. This is
because a range of values is being mapped to the same value.
 Uniform Quantization of Uniformly
Distributed Source

Summary:
If the distortion constraint is given as D*, then step size
can be calculated directly, since

2
D* =
12
M = (xmax – xmin)/

Fixed Length Coding :

Contd…
 If the rate constraint is given as R*,
then M can be calculated, hence  can
be calculated.
2
 Then distortion is D =
12
 Uniform Quantization of Nonuniform Sources

 Even if the sources are bounded, simply dividing the range of

the input by the number of quantization levels does not produce
a very good design.
 Ex: Input fall within the interval [-1,1] with probabilty 0.95 and
fall within the intervals [-100,1) & (1,00] with prob: 0.5.
 If an 8 – level quantizer is to be designed, if we follow previous
procedure – Total Range : -100 to 100. 200/8 = 25 is the step
size.
 This will cause the inputs in the interval [-1,0) –
[-12.5] and inputs in the interval [0,1) by 12.5.
 At least 95% of the time error will be approximately 12.5.
 Therefore this is not a good design.
 But if the step size that we select is
small say 0.3, then 95% of the time
error will be less. But the rate will be
very large.
 Therefore if we have a non-uniform
distribution, we should include the pdf
of the source to determine the step size
of a Uniform Quantizer.
 The decision boundaries are integral multiples of , and
the representation level for the interval [(k-1) , k ) is
simply ((2k-1)/2) . Therefore, the expression for MSQE
becomes
 To find the optimal value of , we simply
take the derivative of the equation and set it
equal to zero.
 Given the pdf fx(x), it is easy to solve using one of a
number of numerical techniques In Table 8.3, list of step
sizes obtained by solving previous equation for nine
different alphabet sizes and three different distributions.
 We can see that in the inner intervals the error is still
bounded by /2; however,the quantization error in the
outer intervals is unbounded. These two types of
quantization errors are given different names. The bounded
error is called granular error or granular noise, while the
unbounded error is called overload error or overload
noise.
 In the expression for the MSQE the first term represents
the granular noise, while the second term represents the
overload noise.
 The probability that the input will fall into the overload
region is called the overload probability
 The nonuniform sources we deal with have
probability density functions that are
generally peaked at zero and decay as we
move away from the origin.
 Therefore, the overload probability is
generally much smaller than the probability
of the input falling in the granular region.
 We can see from previous Equation that an increase in the size of the
M
step size  will result in an increase in the value of 2  1) , which
(
in turn will result in a decrease in the overload probability and the
second term in Equation.

 However, an increase in the step size  will also increase the

granular noise, which is the first term in Equation.

 The design process for the uniform quantizer is a balancing of these

two effects.

 Hence, the selection of  is a balance between the overload and

granular errors.
 The Laplacian distribution has more of its probability mass
away from the origin in its tails than the Gaussian distribution.
This means that for the same step size and number of levels
there is a higher probability of being in the overload region if
the input has a Laplacian distribution than if the input has a
Gaussian distribution.
 The uniform distribution is the extreme case, where the
overload probability is zero.
 For the same number of levels, if we increase the step size, the
size of the overload region (and hence the overload
probability) is reduced at the expense of granular noise.
Therefore, for a given number of levels, if we were picking the
step size to balance the effects of the granular and overload
noise, distributions that have heavier tails will tend to have
larger step sizes.