Data Compression Seminar John Kieffer

11 Vector Quantizer Design

Let C be the quantization codebook for a k-dimensional vector quantizer. The set C consists of 2kR vectors
in k-dimensional Euclidean space, where R is the desired compression rate in codebits per data sample.
The resulting k-dimensional vector quantizer quantizes a data vector X of length n a multiple of k in the
following way: The data vector is partitioned into blocks of length k, and then each block is quantized into
a closest vector in C in the Euclidean distance. When the quantized blocks are concatenated together, the
quantized data vector X̂ of length n results:

X = (X1 , X2 , . . . , Xn ) = data vector

C = {v1 , v2 , . . . , vN } = quantizer codebook, N = 2kR

X̂ = (X̂1 , X̂2 , . . . , X̂n ) = quantized data vector

(X̂(i−1)k+1 , . . . , X̂ik ) = i-th quantized block =

arg min d v, (X(i−1)k+1 , . . . , Xik ) (d = Euclidean distance)
v∈C

The vector quantizer design problem is to find an optimal k-dimensional vector quantizer, a quantizer
for which the distortion D is minimized, where

D = (X1 − X̂1 )2 + (X2 − X̂2 )2 + . . . + (Xn − X̂n )2

The vector quantizer design problem as stated above is an intractable computational problem, because
of the great variety of potential codebooks that could be used. One can make the design problem easier by
attempting to find a locally optimal quantizer instead—this is what the LBG algorithm, to be covered in
these notes, will do for us.
One can also make the design problem easier by imposing structural constraints, and optimizing within
the class of quantizers satisfying the constraints. Structurally constrained quantizers that we shall discuss
in these notes are the tree-structured vector quantizers and the lattice vector quantizers. Structurally
constrained quantizers will not yield optimum distortion performance. But, the performance may be good
enough for the lossy compression application that one has in mind.
EXAMPLE 1. Consider the 2-D vector quantizer with codebook

√ √ √ √ √ √ √ √
C = {(r, 0), (r/ 2, r/ 2), (0, r), (−r/ 2, r/ 2), (−r, 0), (−r/ 2, −r/ 2), (0, −r), (r/ 2, −r/ 2)}

where r is a positive parameter. This quantizer yields a compression rate of R = log2 (8)/2 = 1.5 codebits per
data sample. This quantizer is a simple example of a structured quantizer—the 8 vectors in the quantization
codebook have been constrained to be equally spaced about a circle centered at the origin. It is a simple
problem in one-variable calculus to find the value of r that will minimize the quantizer distortion D for a
given data vector of even length. (See below.)

1
11.1 Voronoi regions
It is helpful to view the optimal quantizer design problem in terms of the Voronoi regions of a quantizer. Let
C = {v1 , v2 , . . . , vN } be the quantization codebook for a vector quantizer. For each k-dimensional vector x,
let Q(x) be the vector in C assigned to x by the vector quantizer. The i-th Voronoi region R(vi ) for the
vector quantizer consists of all x for which Q(x) = vi . The Voronoi regions {R(vi ) : i = 1, 2, . . . , N } partition
up k-dimensional Euclidean space, and the boundaries between these regions are contained in hyperplanes.
EXAMPLE 2. If C = {(1, 1), (−1, 1), (1, −1), (−1, −1)}, the Voronoi regions are the four quadrants in
the xy-plane.
EXAMPLE 3. We specify the Voronoi regions in Example 1. Let R1 be the region contained in the
first and fourth quadrants of the xy-plane which lies between the two straight lines y = ± cos(π/8)x. (Let us
suppose that R1 contains its upper boundary but not its lower boundary.) Notice that the region R1 is a cir-
cular sector of width 45 degrees. By rotating R1 , you can produce 7 other sectors R2 , R3 , R4 , R5 , R6 , R7 , R8 ,
such that the sectors R1 − R8 partition up the xy-plane. The eight regions R1 − R8 are the Voronoi regions
for the 2-D vector quantizer in Example 1. One can now write the distortion D arising from the quantization
of the data vector (X1 , X2 , . . . , Xn ) as

X X √ √
nD = {(X2j−1 − r)2 + X2j
2
}+ {(X2j−1 − r/ 2)2 + (X2j − r/ 2)2 }+
(X2j−1 ,X2j )∈R1 (X2j−1 ,X2j )∈R2

X
2
X √ √
{X2j−1 + (X2j − r)2 } + {(X2j−1 + r/ 2)2 + (X2j − r/ 2)2 }+
(X2j−1 ,X2j )∈R3 (X2j−1 ,X2j )∈R4
X X √ √
{(X2j−1 + r)2 + X2j
2
}+ {(X2j−1 + r/ 2)2 + (X2j + r/ 2)2 }+
(X2j−1 ,X2j )∈R5 (X2j−1 ,X2j )∈R6
X
2
X √ √
{X2j−1 + (X2j + r)2 } + {(X2j−1 − r/ 2)2 + (X2j + r/ 2)2 }
(X2j−1 ,X2j )∈R7 (X2j−1 ,X2j )∈R8

It is easy to take the derivative of this expression with respect to r, set the derivative equal to zero, and
solve for the optimum r.
EXERCISE. Write a MATLAB program which will find the optimum quantizer structured as in Exam-
ple 1, for any data vector of even length. Try your program on a long pseudorandom Gaussian data vector
generated using the MATLAB number generator “randn”. Compute the SQNR for the resulting optimum
quantizer.
If C = {v1 , v2 , . . . , vN } is the codebook of an optimum quantizer, it is not hard to see that each vi must
be the mean of those data samples falling in the Voronoi region R(vi ). This is a necessary condition for a
quantizer to be optimum, but it is not a sufficient condition. One can construct quantizers satisfying this
condition which are not optimal. This necessary condition is the basis for the LBG algorithm, to be discussed
next.

11.2 LBG algorithm

The LBG algorithm is an iterative algorithm for vector quantizer design, which is a generalization of the
Lloyd algorithm for scalar quantizer design. The LBG algorithm starts with the quantization codebook of
an arbitrary k-dimensional vector quantizer, and generates a new codebook on each iteration.
Linde, Buzo, and Gray, the developers of the LBG algorithm, noticed the following. Let C = {v1 , v2 , . . . , vN }
be any quantization codebook, and let {R(v1 ), . . . , R(vN )} be the corresponding Voronoi regions. Let vi0 be
the mean of the data samples falling in R(vi ). Then, the codebook C 0 = {v10 , v20 , . . . , vN
0
} yields lower
distortion than the codebook C.
The procedure described in the previous paragraph represents one iteration of the LBG algorithm. One
iterates until the distortion is not appreciably decreased by further iteration. If the number of data samples
in the data vector to be quantized is large, then the LBG algorithm can be applied directly to the data

2
vector itself. Otherwise, one should form a training sequence consisting of a large number of data samples of
the same type as those appearing in the data vector, and apply the LBG algorithm to the training sequence.
We give the m-file “LBG.m” in the last section. This m-file creates a MATLAB function “LBG(C0,u)”
via which the LBG algorithm may be implemented. The variable “C0” denotes the codebook at the start
of the LBG algorithm iteration—it is a matrix whose rows are the vectors in the codebook. The variable
“u” denotes the training sequence or data vector to which the LBG algorithm is applied. If we execute the
MATLAB command
C1 = LBG(C0,u)
we will see printed out on the screen a matrix C1 whose rows are the vectors in the codebook obtained
from the codebook represented by C0 in one iteration of the LBG algorithm. To see how well the quantizers
generated via the LBG algorithm perform, the reader can use the MATLAB function SQNR(C,x), created
by the m-file SQNR.m given in the last section. If C is a matrix whose rows represent some codebook, and x
is the data vector of interest, then the number SQNR(C,x) is the distortion in decibels resulting from the use
of the quantizer given by C on the data vector x.

11.2.1 Case Study: LBG algorithm on Gaussian data

Here is the diary of a MATLAB session in which the LBG algorithm was used to design for a Gaussian data
vector a 2-D vector quantizer of rate R = 1 codebit per data sample.

>> x=randn(1,1000);
%initial codebook
>> codebook_0 =[1 1; -1 1; -1 -1; 1 -1];

codebook_0 =

1 1
-1 1
-1 -1
1 -1

>> distortion_0 = SQNR(codebook_0,x)

distortion_0 =

3.8024

%Generate new codebook and compute distortion

>> codebook_1=LBG(codebook_0,x)

codebook_1 =

0.7891 0.8876
-0.7996 0.7731
-0.8066 -0.7963
0.7771 -0.7372

>> distortion_1 = SQNR(codebook_1,x)

distortion_1 =

4.2814

%Generate new codebook and compute distortion

>> codebook_2 = LBG(codebook_1,x)

3
codebook_2 =

0.7708 0.9258
-0.7936 0.7533
-0.8355 -0.8159
0.7705 -0.7042

>> distortion_2 = SQNR(codebook_2,x)

distortion_2 =

4.2906

%Generate new codebook and compute distortion

>> codebook_3 = LBG(codebook_2,x)

codebook_3 =

0.7736 0.9326
-0.7897 0.7248
-0.8529 -0.8463
0.7571 -0.6958

>> distortion_3 = SQNR(codebook_3,x)

distortion_3 =

4.2990

We started with an initial codebook of {(1, 1), (−1, 1), (−1, −1), (1, −1)}, for which the resulting vector
quantizer yields a distortion of 3.8024 decibels. After three iterations of the LBG algorthm, the codebook
{(.7736, .9326), (−.7897, .7248), (−.8529, −.8463), (.7571, −.6958)} was obtained, for which the resulting dis-
tortion is 4.2990 decibels. After about 10 more iterations, we obtained a vector quantizer with distortion
4.3846 decibels.
Notice how the distortion got better from iteration to iteration. This is guaranteeed by the LBG algo-
rithm.
EXERCISE. Find a 3-dimensional quantization codebook of size 8 which quantizes long pseudorandom
Gaussian data vectors (i.e., data vectors generated using the MATLAB pseudorandom number generator
“randn”) at a distortion level which is at least 4.41 decibels. (You are finding a 3-D one codebit per data
sample quantizer for pseudorandom Gaussian data. The best such quantizers operate at a distortion level
of about 4.49 decibels, but it takes a lot of “number crunching” to find one of these optimum quantizers.
It will take you less number crunching to find a 4.41 decibel quantizer. The reader should be warned that
symmetric codebooks of the form

{(r, r, r), (−r, r, r), (r, −r, r), (r, r, −r), (r, −r, −r), (−r, r, −r), (−r, −r, r), (r, r, r)}

will not yield the solution to this exercise since the optimum distortion performance for such codebooks is
only 4.3964 decibels.)

11.3 Tree-structured vector quantizers

A k-dimensional tree-structured vector quantizer of rate R is defined by means of a full rooted balanced
binary tree of depth kR. Each branch of the tree is labelled with a k-dimensional vector. The labels on

4
the bottom branches form the vector quantizer codebook. For example, if R = 1.5 and k = 2, the depth
of the tree with be kR = 3. There will be two branches of the tree of depth 1, four branches of the tree of
depth 2, and 8 branches of the tree of depth 3—it is these 8 branches at the bottom whose labels form the
quantization codebook of size eight.
Here is how a tree-structured vector quantizer quantizes a given k-dimensional data block X. Let v0 and
v1 be the labels on the two branches extending from the root node. Follow the branch labelled v0 if X is
closer to v0 than to v1 , and generate a codebit of 0—otherwise, follow the branch labelled v1 and generate
a codebit of 1. As a result of this first step, we have followed a branch labelled vi , where i = 0 or i = 1,
stopping at the node Ni at the end of this branch. On the next step, one follows one of the two branches
emanating from Ni . Let us suppose one of the branches is labelled vi0 and the other vi1 . Follow the branch
labelled vi0 if X is closer to vi0 than to vi1 , and generate a codebit of 0—otherwise follow the other branch
and generate a codebit of 1. Iterating this procedure, one will arrive at a terminal node of the tree after kR
steps. The vector X is quantized into the label on the last branch traversed, and the encoder output is the
sequence of kR bits generated along the way.
A rate R tree-structured vector quantizer allows for fast quantization. Notice that only 2kR distances
between the data block X and the labels on the branches are computed while following the path through
the tree to the branch at the bottom labelled with the quantizer output for X. For a general quantizer, 2kR
distances have to be computed, which takes much longer.
Tree-structured vector quantizers are much easier to design than are general vector quantizers. We
discuss a tree-structured vector quantizer design method. The method starts with a training sequence
S = {u1 , u2 , . . . , ur } of k-dimensional vectors, where r is very large. The sequence S is split into two
subsequences S0 and S1 by seeing which members of S lie on either side of some hyperplane. (We discuss
ways of choosing this hyperplane in the next paragraph.) Let v0 be the mean of S0 and let v1 be the mean of
S1 , and take the labels on the two top branches of the tree to be v0 and v1 . For each i = 0, 1, the subsequence
Si is split into two subsequences Si0 and Si1 by means of some hyperplane, and the two branches descending
from the vi branch are labelled vi0 and vi1 , where vi0 and vi1 are the means of Si0 and Si1 , respectively.
Continuing this iterative process, one furnishes a label to each branch of the tree—each label is the mean of
one of two subsequences obtained by partitioning a subsequence from the preceding step.
When one is to determine the labels on the two branches from a given node, there will be a sequence S 0
of training vectors in S that will have “survived” down to that node. The labels on the two branches are
the means of the subsequences S00 and S10 of S 0 into which S 0 is split via some hyperplane. Let us call this
hyperplane the splitting hyperplane for S 0 . The splitting hyperplane can be taken as the hyperplane which
minimizes the distortion
X X
(v − v00 )2 + (v − v10 )2 (1)
v∈S00 v∈S10

where v00 is the mean of S00 and v10 is the mean of S10 . This is equivalent to finding the optimum quantization
codebook {v0∗ , v1∗ } of size 2 for quantizing the sequence S 0 . (It can be shown that the hyperplane which is the
perpendicular bisector of the straight line joining v0∗ and v1∗ will minimize (1).) A computationally simpler
method for selecting the splitting hyperplane is to first use the LBG algorithm to find a locally optimum
quantization codebook of size 2 for quantizing S 0 —the splitting hyperplane is then taken as the perpendicular
bisector induced by this codebook. By consulting the literature, one will discover other methods for selecting
the splitting hyperplane.

11.4 Lattice vector quantizers

Lattice vector quantizers are the natural generalization of the 1-D uniform quantizers to higher dimensions.
Using a lattice, one can partition k-dimensional Euclidean space into congruent regions (such as rectangles
or hexagons in 2-D). This partition induces the lattice vector quantizer which quantizes each data block from
the data vector into the lattice point lying at the center of the region containing the data block. The rate
and distortion of the lattice quantizer are controlled by scaling of the lattice.
Let {u1 , u2 , . . . , uk } be a set of basis vectors for k-dimensional Euclidean space. The lattice spanned by
these vectors is the set of all vectors which are linear combinations

5
 n1 u1 + n2 u2 + . . . + nk uk
with integer coefficients n1 , n2 , . . . , nk .
EXAMPLE 4. The basis vectors (1, 0), (0, 1) in 2-D span the lattice of all points (n1 , n2 ) in which n1
and n2 are integers. We shall call this lattice the rectangular lattice.
√
EXAMPLE 5. The basis vectors (1, 0), (1/2, 3/2) span the 2-D lattice called the hexagonal lattice.
Let L be a lattice in d-dimensional Euclidean space Rd . Let ρL be the mapping which maps each k-
dimensional vector into the nearest point in L (if there are two or more such points, ρL selects one of them).
The mapping ρL is called the nearest neighbor mapping for the lattice L. For each lattice point u in L, let
R(u) be the region {x ∈ Rd : ρL (x) = u}. The regions {R(u) : u ∈ L} are called the nearest neighbor regions
for the lattice L. The nearest neighbor regions partition up Rd , so that each k-dimensional vector lies in
one and only one nearest neighbor region. The region R(u) is the unique nearest neighbor region containing
the lattice point u. We shall say that u is the center of R(u). The nearest neighbor regions of a lattice are
congruent. In fact, the nearest neighbor region R(0) centered at the zero vector 0 generates all the nearest
neighbor regions by translation: R(u) = u + R(0).
EXAMPLE 6. For the rectangular lattice, the nearest neighbor region R(u) is the rectangular region
whose vertices are u + (1/2, 1/2), u + (−1/2, 1/2), u + (−1/2, −1/2), u + (1/2, −1/2).
EXAMPLE 7. For the√hexagonal lattice, √ the nearest neighbor
√ region R(u) is
√ the hexagonal √ region
whose vertices
√ are u + (1/2, 3/6), u + (0, 3/3), u + (−1/2, 3/6), u + (−1/2, − 3/6), u + (0, − 3/3),
u + (1/2, − 3/6).
If a is a positive real number, and L is a lattice, let aL be the lattice in which every lattice point in L is
multiplied by the scalar a. Let the radius r(L) of the lattice L be the distance from the center of a nearest
neighbor region to a farthest
√ point on its boundary. Notice that r(aL) = √ ar(L). If L is the rectangular
lattice, then r(aL) = a 2/2. If L is the hexagonal lattice, then r(aL) = a 3/3.
Let us examine the rate and distortion performance of lattice vector quantizers. Let k be fixed, and
fix a d-dimensional lattice L. Let X be a data vector of length a large multiple of k. The k-dimensional
lattice vector quantizer induced by L quantizes X into a reproduction vector X̂. Here is how X̂ is obtained.
Partition X into data blocks of length k. Quantize each data block into the closest point in L. (Equivalently,
quantize each data block into the center of the nearest neighbor region in which it lies.) Concatenation of
the quantized blocks yields X̂. The lattice quantizer just described yields a rate R and a distortion D when
applied to the data vector X. Let N be the total number of nearest neighbor regions that contain data
blocks of length k in the partition of X. Then

R = dlog2 N e/k (2)

We also have the following upper bound on D:

D ≤ r(L)2 (3)
Which is better for constructing 2-D vector quantizers, the rectangular lattice or the hexagonal lattice?
We argue that the hexagonal lattice is better. Let L1 denote the rectangular lattice and let L2 denote
the hexagonal lattice. We use the lattice a1 L1 for quantization of the data vector X and we use a2 L2 for
quantization of the data vector X, where the scaling parameters a1 and a2 are selected so that the two
quantizers yield the same encoding rate R. Suppose that the data blocks of length 2 in X fill up a region in
for the lattice a1 L1 is A1 = (a1 )2 and the area
the plane of area A. The area of each nearest neighbor region √
of each nearest neighbor region for the lattice a2 L2 is A2 = 3(a2 )2 /2. Assuming that A is large relative
to A1 and A2 (this assumption is valid if R is large), it takes roughly A/A1 of the a1 L1 nearest neighbor
regions to cover the data blocks of length 2 in X and roughly A/A2 of the a2 L2 nearest neighbor regions to
cover these same blocks. Because of (2),

A/A1 ≈ A/A2

6
which tells us that the relationship between a1 and a2 is
√
(a1 )2 ≈ 3(a2 )2 /2 (4)
Using (3) and (4), one is assured that with the a1 L1 lattice vector quantizer,
√ √
distortion ≤ r(a1 L1 )2 = (a1 2/2)2 ≈ 3(a2 )2 /4 (5)
and with the a2 L2 lattice vector quantizer, one is assured that
√
distortion ≤ r(a2 L2 )2 = (a2 3/3)2 = (a2 )2 /3 (6)
The right side of (6) is smaller than the right side of (5), so the vector quantizer based on the hexagonal
lattice is better.
The hexagonal lattice is the best lattice to use to design high rate 2-D lattice vector quantizers. If one
increases the dimension, one can find better vector lattice quantizers. The best lattice vector quantizers so
far discovered are 24-dimensional and are based on the 24-dimensional Leech lattice.

11.5 MATLAB programs

%Name this m-file LBG.m
%u = training sequence
%Length of u must be a multiple of the dimension of quantizer
%oldcodebook = matrix whose rows are the codebook at the
%start of the iteration
%newcodebook = matrix whose rows are the codebook at the
%end of the iteration
function newcodebook = LBG(oldcodebook,u)
C=oldcodebook;
[N,k]=size(C);
%N is the size of the codebook
%k is the dimension of the quantizer
m=length(u)/k;
%m is number of blocks in training sequence
for i=1:m;
datablock=u((i-1)*k+1:i*k);
M(i,:)=datablock;
dist=[ ];
for j=1:N;
v=C(j,:);
S=sum((datablock-v).^2);
dist=[dist S];
end
[y,ind]=min(dist);
Q(i)=ind;
end
for j1=1:N;
r=(Q==j1);
if length(r)>0
C(j1,:)=r*M/sum(r);
else
end
end
newcodebook=C;

7
%Name this m-file SQNR.m
%C = matrix whose rows form the codebook of a given quantizer
%x = data vector to which the quantizer is applied
%SQNR(C,x) = distortion in decibels resulting from using
%the given quantizer to quantize x
function y = SQNR(C,x)
[N,k]=size(C);
D=0;
M=ones(size(C));
m=length(x)/k;
for i=1:m;
datablock=x((i-1)*k+1:i*k);
A(i,:)=datablock;
dev1=M*diag(datablock)-C;
Q1=dev1.^2;
S1=Q1*ones(k,1);
D=D+min(S1);
end
u=ones(1,m)*A/m;
dev2=ones(size(A))*diag(u)-A;
Q2=dev2.^2;
S2=ones(1,m)*Q2*ones(k,1);
y=10*log10(S2/D);