Data Compression Seminar John Kieffer

2 Scalar Quantizer Design

We remind the reader of the block diagram of a lossy compression system:

X → quantizer → X̂ → encoder → B(X̂) → decoder → X̂

The data vector to be compressed is X = (X1 , X2 , . . . , Xn ). We suppose that the data samples Xi are
real-valued. In principle, there are infinitely many possibilities for the data vector X. The purpose of the
quantizer is to furnish the encoder a vector X̂ = (X̂1 , . . . , X̂n ) which comes from a finite set of vectors C of
size no more than 2nR , where R is the desired compression rate in codebits per data sample. The set C is
called the quantizer codebook. The quantizer distortion is defined by
h i
∆
D = n−1 (X1 − X̂1 )2 + . . . + (Xn − X̂n )2 (1)

In quantizer design, the design goals are the following:

• Low Descriptional Complexity—The quantizer should be simple to describe, so that the overhead
needed to identify the quantizer for the decoder is much smaller than nR codebits.

• Low Computational Complexity—The quantization rule should allow for fast implementation.
• Low Distortion—Ideally, the quantizer distortion should be at least as low as that arising from other
quantizers of the same complexity that yield the same compression rate R.

There are two types of quantizers. Scalar quantizers quantize the data sample by sample. Vector quantiz-
ers quantize the data block by block (for some fixed block length). Scalar quantizers have the merit of being
simple, but their distortion performance would not be as good as the more complicated vector quantizers.
In these notes, we shall be concerned with scalar quantizers.
Let us describe the general form of a scalar quantizer. Let N be a positive integer. An N -level scalar
quantizer for quantizing the data vector X consists of:

(a) An interval [a, b] containing the data samples in X.

(b) A partition of [a, b] into N subintervals.
(c) A quantization level for each subinterval in (b).

For each data sample Xi in the data vector X, the N -level quantizer quantizes Xi into the real number
X̂i which is the quantization level for the subinterval of [a, b] in which Xi lies. The optimum choice of
quantization level for a given quantizer subinterval in (b) can be shown to be the average of those samples
in X falling in that subinterval.
How does the encoder in the above block diagram then operate on X̂i ? Let us assume that the encoder
operates in fixed-rate mode, meaning that each quantized data sample is assigned the same number of
codebits R, where N = 2R . The encoder assigns to each quantizer subinterval in (b) an R-bit binary
address. If X̂i falls in a given quantizer subinterval, then X̂i is encoded into the R-bit address for that
subinterval. The compression rate of this fixed-rate lossy compression scheme is then R codebits per data
sample.
In scalar quantizer design, it must be decided what to take as the partition into subintervals in (b) which
defines the quantizer. We shall see in the ensuing notes various ways in which this can be done.

1
2.1 The variance quantizer
The simplest quantizer for quantizing the data samples in the data vector X = (X1 , X2 , . . . , Xn ) is the
variance quantizer. Let X̄ be the mean of the data samples:
∆
X̄ = n−1 [X1 + X2 + . . . + Xn ]
Then the variance quantizer quantizes each Xi into the quantized sample X̂i = X̄.
The distortion (1) resulting from the variance quantizer is

D = n−1 (X1 − X̄)2 + . . . + (Xn − X̄)2

 

This is precisely the variance σ 2 of the data samples. This is how the variance quantizer gets its name.
The compression rate arising from the use of the variance quantizer is equal to 1/n times the number
of codebits that have to be sent to the decoder to allow the decoder to reconstruct the mean X̄. From the
mean the decoder can reconstruct the quantized vector as

(X̂1 , X̂2 , . . . , X̂n ) = (X̄, X̄, . . . , X̄)

EXAMPLE 1. Suppose the data to be compressed is the following 4 × 4 image:

146 149 152 156

 
 97 122 144 147 
(2)
89 90 135 145
 
85 92 99 120
The mean of the 16 pixel values is easily worked out to be 123. If we assume that the decoder knows
that it takes eight bits to represent each pixel value in the image, then the encoder output can be taken to
be the eight bits needed to tell the decoder that the mean is 123. The quantized image reconstructed by the
decoder is then

123 123 123 123

 
 123 123 123 123 
123 123 123 123
 
123 123 123 123
The compression rate is 8/16 = 0.5 codebits per pixel. The following lines of MATLAB code compute the
distortion D and also verify the value of the mean:

>> x=[146 149 152 156 97 122 144 147 89 90 135 145 85 92 99 120];
>> xbar=mean(x)

xbar =

123

>> D=mean((x-xbar).^2)

D =

668.2500

2.2 Uniform quantizers

The uniform scalar quantizers are the simplest quantizers. Here’s how you design one. Select the design
parameter R, which is the desired number of codebits per data sample—R must be a positive integer. Choose
an interval [a, b] which contains all of the samples Xi in your data vector. Let N = 2R . Form the N -level
uniform quantizer by partitioning the interval [a, b] into N equal subintervals. Quantize each sample Xi into

2
the midpoint X̂i of the subinterval in which it lies. For the uniform quantizer, one sees that the quantizer
distortion given by (1) satisfies:
 2
b−a
D≤
2N
One can evaluate the uniform quantizer performance (or the performance of any quantizer) by seeing how
much lower the quantizer distortion D is than the variance of the data samples σ 2 . The ratio of σ 2 to D
is called the signal to quantizing noise ratio (SQNR). The bigger the SQNR is, the better the quantizer is
doing in terms of distortion control.
EXAMPLE 2. In the following MATLAB demo, we generate 1000 data samples pseudorandomly
uniformly distributed in the interval [0, 1]. Then we quantize the 1000 samples using a uniform quantizer
on the interval [0, 1], quantizing at the rate of 3 codebits per sample. Since 23 = 8, we shall use an 8-level
quantizer. We see that the SQNR is about 18 decibels. (One converts SQNR to decibel units by taking
10 ∗ log 10(SQNR).)

%Generate the data

x=rand(1,1000);
%Generate the quantization intervals
j1=find(x<1/8);
j2=find(x>=1/8 & x<2/8);
j3=find(x>=2/8 & x<3/8);
j4=find(x>=3/8 & x<4/8);
j5=find(x>=4/8 & x<5/8);
j6=find(x>=5/8 & x<6/8);
j7=find(x>=6/8 & x<7/8);
j8=find(x>=7/8 & x<1);
%Generate the quantized samples
y(j1)=1/16*ones(size(j1));
y(j2)=1/2*(1/8+2/8)*ones(size(j2));
y(j3)=1/2*(2/8+3/8)*ones(size(j3));
y(j4)=1/2*(3/8+4/8)*ones(size(j4));
y(j5)=1/2*(4/8+5/8)*ones(size(j5));
y(j6)=1/2*(5/8+6/8)*ones(size(j6));
y(j7)=1/2*(6/8+7/8)*ones(size(j7));
y(j8)=1/2*(7/8+1)*ones(size(j8));
%Compute the distortion in decibels
D=10*log10(mean((x-y).^2));
%Compute the variance in decibels
sigma2=10*log10(mean((x-mean(x)).^2));
%Compute the SQNR
SQNR = sigma2-D
SQNR =

18.0176

EXERCISE. For a data vector consisting of a large number of pseudorandom uniformly distributed
samples, show that the approximate SQNR arising from using a N = 2R level uniform quantizer is about
6R decibels.

2.3 Variable-rate scalar quantizers

The uniform quantizer is a fixed-rate quantizer. This means that if the desired encoding rate is R codebits
per sample, every quantized data sample will be encoded using R codebits. On the other hand, a variable-
rate quantizer will allocate more than R codebits to some data samples, and less than R codebits to other

3
data samples, so that the overall average number of codebits per data sample is R. In variable-rate scalar
quantization, the designed compression rate R need not be an integer. Variable-rate scalar quantizers can
yield a better distortion performance than fixed-rate quantizers.
Here is one approach towards designing a variable rate quantizer. Partition the n data samples into
“bins” of size n1 , n2 , . . . , nk , where n1 + n2 + . . . + nk = n. Suppose it works out that there are integers
R1 , R2 , . . . , Rk so that
• (n1 /n)R1 + (n2 /n)R2 + . . . + (nk /n)Rk = R
• At least one Ri is bigger than R and at least one Ri is smaller than R.
For each bin, construct a uniform quantizer. The uniform quantizer for the i-th bin must employ 2Ri
subintervals. One now has a variable-rate quantizer that quantizes as follows. To quantize a data sample,
see which bin it is in. If it is in the i-th bin, then the sample is quantized using the uniform quantizer for
the i-th bin, which means that Ri codebits will be allocated to that sample.
EXAMPLE 3. Suppose it is desired to quantize a data vector consisting of 1000 samples in the interval
[0, 1], so that encoding can be done at a rate of 1.5 codebits per data sample. To accomplish this, one can
partition the samples into two bins of 500 samples each, uniformly quantizing the samples in one of the
bins using 1 codebit per sample, and uniformly quantizing the samples in the other bin using 2 codebits per
sample. The one bit per sample bin is split into 21 = 2 equal subintervals, and the two bit per sample bin
is split into 22 = 4 equal subintervals. This yields 6 subintervals for our overall variable-rate quantizer. The
MATLAB demo below computes the SQNR for this 1.5 bit per sample quantizer as applied to 1000 samples
of nonuniform pseudorandom data in [0, 1]. We obtained an SQNR of about 12 decibels.
%Generate nonuniformly pseudorandom samples
u=rand(1,1000);
x=u.^2;
%On either side of the median you get 500 samples
m=median(x);
%Generate the 2 bit per sample subintervals
j1=find(x<m/4);
j2=find(x >= m/4 & x < 2*m/4);
j3=find(x>=2*m/4 & x<3*m/4);
j4=find(x>=3*m/4 & x<m);
%Generate the one bit per sample subintervals
k1=find(x>=m & x<(1+m)/2);
k2=find(x>=(1+m)/2);
%Generate the vector of quantized samples
y(j1)=(m/8)*ones(size(j1));
y(j2)=.5*(m/4+2*m/4)*ones(size(j2));
y(j3)=.5*(2*m/4+3*m/4)*ones(size(j3));
y(j4)=.5*(3*m/4+m)*ones(size(j4));
y(k1)=.5*(m+.5*(1+m))*ones(size(k1));
y(k2)=.5*(.5*(1+m)+1)*ones(size(k2));
%Compute the SQNR
D=10*log10(mean((x-y).^2));
sigma2=10*log10(mean((x-mean(x)).^2));
SQNR=sigma2-D
SQNR =

12.0822

2.4 Block truncation coding

In block truncation coding of an image, the image is partitioned into blocks of the same size (usually 4 × 4
blocks), and then the pixel values in each block are quantized using a 1 bit per pixel quantizer (i.e., a 2-level

4
quantizer) expressly designed for that particular block.
For example, suppose we wish to perform block truncation coding on a large 256 level image using 4 × 4
sub-blocks of the image. Suppose one of these blocks is the block given in (2). The first step of block
truncation coding is to partition the 16 pixel values in (2) into two bins. Let Bin #1 consist of all those
pixel values lying above a certain threshold, and let Bin #2 consist of the pixel values falling below the
threshold. We take the threshold to be the mean of the 16 pixel values, which in this case works out to be
123. Numbering the pixel positions from left to right and top down, we see that the pixel values in (2) in
positions 1, 2, 3, 4, 7, 8, 11,and 12 comprise Bin #1. Each pixel value in Bin #1 is to be quantized into the
mean of the pixel values in Bin #1—this mean works out to be 146.75. Each pixel value in Bin #2 is to be
quantized into the mean of the pixel values in Bin #2—this mean works out to be 99.25. Let us round these
two means off to 147 and 99, respectively. The matrix (2) is therefore quantized into the matrix

147 147 147 147

 
 99 99 147 147 
(3)
99 99 147 147
 
99 99 99 999
The encoder encodes this as

1 1 1 1
 
0 0 1 1
(4)
0 0 1 1
 
0 0 0 0
prefixed by 16 codebits giving the two 8 bit representations of the integers 99, 147. From the 16 codebits in
(4) together with this 16 bit prefix describing the means of the two bins, the decoder reconstructs (3).
From the preceding example, the reader sees that block truncation coding, applied to 256 level images,
yields a compression rate of 16 + 16 = 32 codebits per 4 × 4 image sub-block, which converts to a compression
rate of 32/16 = 2 codebits per pixel.
The following lines of MATLAB code allow us to compute the SQNR arising from block truncation coding
of the block (2):

>> x=[146 149 152 156 97 122 144 147 89 90 135 145 85 92 99 120];
>> xhat=[147 147 147 147 99 99 147 147 99 99 147 147 99 99 99 99];
>> sigma2=10*log10(mean((x-mean(x)).^2));
>> D=10*log10(mean((x-xhat).^2));
>> SQNR=sigma2-D

SQNR =

8.0686

We see that the SQNR is 8.0686 decibels.

2.5 Lloyd’s quantizer design algorithm

Let N be a positive integer greater than one. Lloyd’s algorithm is an iterative technique for designing an
N -level scalar quantizer with low distortion. An N -level quantizer can be described in terms of an interval
[a, b] containing the data samples in the given data vector X and N − 1 thresholds y1 , y2 , . . . , yN −1 in [a, b]
satisfying

y1 < y2 < y3 < . . . < yN −1

(Let us also define y0 = a and yN = b.) If a data sample Xi falls between the thresholds yj−1 and yj ,
then the sample is quantized into the average value of all samples in X falling in the quantizer subinterval
[yj−1 , yj ].

5
 If the quantizer defined by the thresholds {yj } is used to quantize the data vector X = (X1 .X2 , . . . , Xn ),
there will result N bins consisting of subsets of the data samples. The j-th bin consists of all the data
samples Xi that fall between the thresholds yj−1 and yj . Let M1 , M2 , . . . , MN be the mean values for the
N bins, respectively. In the 1950’s, S. Lloyd noticed that if new thresholds are formed according to:

ỹj = (1/2)(Mj + Mj+1 ), j = 1, 2, . . . , N − 1 (5)

then the quantizer formed from the new thresholds yields lower distortion when used to quantize X than
does the original quantizer.
Verbally, (5) says that, for the new quantizer, the threshold between two consecutive quantizer subinter-
vals should be taken to be the average of the means of the corresponding pair of bins for the old quantizer.
This is one iteration step of Lloyd’s algorithm. One can start Lloyd’s algorithm with any initial choice of
quantizer (such as the uniform quantizer). After a few iterations, further iterations will not appreciably
lower the quantizer distortion, at which point you stop iterating and take the quantizer at that point for
your design.
You should apply Lloyd’s algorithm to the data vector X only when X has a large number of samples.
If it doesn’t, the practice is to apply Lloyd’s algorithm instead to a “training vector” consisting of a large
number of previously observed samples of the same type as the samples comprising the data vector.
EXAMPLE 4. We use the MATLAB function “lloyd(x,a,b,N,k)” (given in the last section) to design
a quantizer for a pseudorandomly generated data vector. The argument x is the data vector to be quantized,
the arguments a,b are the endpoints of the interval [a, b] containing the data samples, the argument N is the
desired number of levels, and the argument k is the number of iterations of Lloyd’s algorithm that are to be
performed.
Here is the record of our MATLAB session:

%Generate nonuniform pseudorandom data

>> u=rand(1,100000);
>> x=u.^2;
%Apply Lloyd’s algorithm
>> lloyd(x,0,1,4,20);
SQNR y1 y2 y3
12.1536 0.2252 0.4943 0.7465
12.3966 0.2126 0.4835 0.7431
12.5089 0.2041 0.4734 0.7384
12.5764 0.1981 0.4651 0.7334
12.6209 0.1935 0.4582 0.7289
12.6525 0.1900 0.4524 0.7246
12.6736 0.1872 0.4478 0.7210
12.6895 0.1848 0.4436 0.7177
12.7020 0.1827 0.4399 0.7148
12.7106 0.1810 0.4370 0.7124
12.7161 0.1797 0.4346 0.7105
12.7194 0.1786 0.4328 0.7091
12.7216 0.1777 0.4313 0.7080
12.7231 0.1770 0.4300 0.7070
12.7242 0.1764 0.4290 0.7062
12.7249 0.1759 0.4281 0.7055
12.7254 0.1755 0.4274 0.7050
12.7257 0.1751 0.4268 0.7045
12.7259 0.1749 0.4263 0.7041
12.7261 0.1747 0.4260 0.7038

For each iteration, you will see displayed on the screen the thresholds y1 , y2 , . . . , yN −1 for the quantizer
resulting from that iteration, along with the SQNR for that quantizer in the left column. Notice that in our
example here, we have found after 20 iterations of Lloyd’s algorithm a 4-level quantizer with subintervals

6
[0, .1747], [.1747, .4260], [.4260, .7038], and [.7038, 1]. The resulting SQNR is 12.7261 decibels. Notice in the
left column how the SQNR’s increase from iteration to iteration. This is guaranteed by Lloyd’s algorithm.
We used nonuniform data in our example, since the best we can do for data that is nearly uniformly
distributed is to use a uniform quantizer, which would not have been very interesting.
EXERCISE. Show that the data samples in the above example are approximately distributed according
to the probability density (0.5)x−0.5 in the interval (0, 1).
EXAMPLE 5. We again design a block truncation code for the 4 × 4 block (2). Unlike Section 10.4,
we optimize the choice of 2-level quantizer using Lloyd’s algorithm:

>> x=[146 149 152 156 97 122 144 147 89 90 135 145 85 92 99 120];
>> a=min(x);
>> b=max(x);
>> lloyd(x,a,b,2,10);
SQNR y1
8.1954 120.0000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000
8.6281 116.8000

This tells us we can use a threshold of 116.8 and achieve a higher SQNR than in Section 10.4 (where an
SQNR of 8.0686 was obtained). We will not achieve precisely the SQNR of 8.6281, because some rounding
off has to be done in the quantizer design. We have to partition the 16 pixel values in (2) into the two
bins consisting of the values above and below the threshold of 116.8. The means of these two bins are then
computed and rounded off to the nearest integer:

>> j1=find(x<116.8);
>> bin1=x(j1);
>> j2=find(x>116.8);
>> bin2=x(j2);
>> mean(bin1)

ans =

92

>> mean(bin2)

ans =

141.6000

>> xhat(j1)=92*ones(size(j1));
>> xhat(j2)=142*ones(size(j2));
>> D=10*log10(mean((x-xhat).^2));
>> sigma2 = 10*log10(mean((x-mean(x)).^2));
>> SQNR=sigma2-D

SQNR =

7
8.6233

Notice that the means of the two bins are 92 and 141.6. We rounded off the mean of the second bin to 142.
This yielded a SQNR of 8.6233 decibels for the resulting 2-level quantizer.

2.6 MATLAB program

Here is the m-file “lloyd.m” via which iterations of Lloyd’s quantizer design algotihm can be performed:

%Name this m-file lloyd.m

%x = data vector
%a,b = endpoints of interval [a,b] containing data
%N = number of levels of quantizer
%k = number of iterations
%A = matrix whose rows give SQNR and thresholds on
%each Lloyd algorithm iteration
function A = lloyd(x,a,b,N,k)
delta=(b-a)/N;
variance=mean((x-mean(x)).^2);
i=0:N-1;
y=a+delta*i;
quantizer=[];
SQNR=[];
r=1;
while r<=k
for j=1:N-1;
m1=find(x>=y(j) & x<y(j+1));
if length(m1)>0
M(j)=mean(x(m1));
D(j)=sum((x(m1)-M(j)).^2);
else
M(j)=0;
D(j)=0;
end
end
m2=find(x>=y(N));
if length(m2)>0
M(N)=mean(x(m2));
D(N)=sum((x(m2)-M(N)).^2);
else
M(N)=0;
D(N)=0;
end
SQNR=[SQNR; 10*log10(variance/(sum(D)/length(x)))];
y=(1/2)*(M(1:N-1)+M(2:N));
quantizer=[quantizer; y];
y=[a y];
r=r+1;
end
A=[SQNR quantizer];
for i=1:N;
if i==1
fprintf(’ SQNR ’)
elseif i<N

8
fprintf(’y%1.0f ’,i-1)
else
fprintf(’y%1.0f \n’,i-1)
end
end
for i=1:k;
for j=1:N;
if j<N
fprintf(’%8.4f ’,A(i,j))
else
fprintf(’%8.4f \n’,A(i,j))
end
end
end