Prof. A.

Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

LECTURE 3:
MD SAMPLING AND CONVERSION
• MD Sampling
– Sampling on a Lattice, Sampling Matrix
– Spectrum of Signals Sampled on a Lattice, Reciprocal Matrix
– Nyquist Criterion for Sampling on a Lattice
– Anti-Alias Filtering
• MD Sampling Structure Conversion
– Problem Statement, Sum and Intersection of Lattices
– Filtering for Sampling Structure Conversion
– Examples

Chapter 1 Multi-dimensional Signals and Systems 1

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

MD Lattice
• A lattice Λ ∈ R is the set of all linear combinations of v1, v2, …, vM with
integer coefficients given by
Λ ⋯ | , , … , є
• The set of vectors v1, v2, …, vM is called a basis for the lattice Λ , which
defines an MD arbitrary periodic sampling structure. In vector-matrix
notation, a lattice Λ is the set of points defined by
Λ є
where V is called a sampling matrix given by
V = [ v1 | v2 | … | vM ] and nT =[ n1, n2, …, nM ].
• The basis, and thus the sampling matrix is not unique. In particular, for every
sampling matrix , , where is an integer matrix with det 1,
forms another sampling matrix for Λ . However, det , which denotes
the reciprocal of the sampling density, is unique.
Chapter 1 Multi-dimensional Signals and Systems 2
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

A/D Conversion: Sampling on a Lattice

• The sampled MD signal can be expressed as
, ∈
, ∈ Λ

2D sampling lattice (M=2, sampling matrix V is 2×2)

Chapter 1 Multi-dimensional Signals and Systems 3
Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

2D Rectangular (Progressive) Sampling

Sampling 2D still images Sampling in analog video

∆ or ∆
∆ ∆

Chapter 1 Multi-dimensional Signals and Systems 4

Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

2D Interlaced Sampling
Sampling 2D still images

Sampling in 2:1 interlaced analog video

Chapter 1 Multi-dimensional Signals and Systems 5

Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

3D Sampling (2D+t)
Progressive
p p p p p
p p p p p
p p p p p
p p p p p
p p p p p

Interlaced
o o o o o
e e e e e
o o o o o
e e e e e
o o o o o
Chapter 1 Multi-dimensional Signals and Systems 6
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Spectrum of Signals Sampled on a Lattice

• The Fourier transform of a continuous signal in vector notation

… and … , ∞ ∞
• The Fourier transform of a discrete signal in vector notation

∑
… and … ,

• The inverse Fourier transform of a discrete signal can be expressed as

/


/

Chapter 1 Multi-dimensional Signals and Systems 7

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Reciprocal Lattice and the Unit Cell

• Given a lattice Λ , the set of all vectors r such that is an integer for
∗
all є Λ is called the reciprocal lattice Λ of Λ .
∗
• A basis for Λ is the set of vectors , , …, determined by
, , 1,2, … ,
or, equivalently,

where is an identity matrix.
• The unit cell of a lattice is not unique. We define the Voronoi cell of a
lattice, which is the set of all points that are closer to the origin than to
any other sample as a unit cell. It corresponds to the fundamental period
of the lattice.
Chapter 1 Multi-dimensional Signals and Systems 8
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Spectrum of Sampled Signals (cont’d)

• The sampled signal
• After a change of variables , we have
1

|det |
∗
where is the sampling matrix of the reciprocal lattice Λ and
|det | .
• Writing the double integral over the entire plane as a sum of integrations over
the unit cell squares 1/2,1/2 1/2,1/2 , we obtain
/
1

|det |
/

where 1 for an integer vector (by definition of the reciprocal
lattice).
Chapter 1 Multi-dimensional Signals and Systems 9
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Spectrum of Sampled Signals (cont’d)

• Finally, comparing this expression with the inverse Fourier transform
/
expression /
, we conclude

1

det
where U is the periodicity matrix in the frequency domain which satisfies
and is the identity matrix. The periodicity matrix
can be expressed as | | … | , where u1, …, uM are basis
vectors of the reciprocal lattice.
• The Fourier transform of a signal sampled on a lattice Λ consists of sum
of periodic replications of the analog signal spectrum on the reciprocal
∗
lattice Λ .
Chapter 1 Multi-dimensional Signals and Systems 10
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Spectrum of Sampled Signals: Nyquist Theorem

• Alternatively, the MD signal sampled on a lattice can be expressed in
terms of continuous signal notation as
∑ є ∑ є

• The Fourier transform, of the sampled signal in terms of that

of the analog signal can be obtained by a change of variables
1

det
where the frequency variable (in terms of the normalized variable).

• Nyquist Theorem: The bandwidth of the analog signal should be limited

to the unit cell of the reciprocal lattice of the sampling lattice.

Chapter 1 Multi-dimensional Signals and Systems 11

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Example: 2D Rectangular Sampling

No aliasing

Chapter 1 Multi-dimensional Signals and Systems 12

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Example: 2D Rectangular Sampling (cont’d)

Aliasing

Proper anti-alias filtering is required prior to sampling. The pass-band of

the anti-alias filter should be limited to the unit cell of the reciprocal lattice.
Chapter 1 Multi-dimensional Signals and Systems 13
Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Example: 2D Arbitrary Periodic Sampling

Chapter 1 Multi-dimensional Signals and Systems 14

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

D/A Conversion: Ideal Reconstruction

• The reconstructed time-varying image , can be obtained through the ideal
low-pass filtering operation
det for

0 otherwise
• Taking the inverse Fourier transform, we have the reconstructed time-varying
image
, ∑ , ,

,
,
where

, det
is the impulse response of the ideal bandlimited spatio-temporal interpolation filter.
Chapter 1 Multi-dimensional Signals and Systems 15
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Analog Interpolation Filtering

, Change to , Analog ,
analog interpolation
notation filter ,

• Analog reconstruction of progressive (rectangularly) sampled signals

sin ∆ /∆ sin ∆ /∆ sin ∆ /∆
,
∆ /∆ ∆ /∆ ∆ /∆

Chapter 1 Multi-dimensional Signals and Systems 16

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Sampling Structure Conversion

• Sum of Lattices: We define the sum of two lattices as
| є and є
The sum of two lattices can be found by adding each point in one lattice to every
point of the other lattice.
• Intersection of Lattices: We can define the intersection of two lattices as
∩ | є and є
• The intersection ∩ is the largest lattice that is a sublattice of both and
, and the sum is the smallest lattice that contains both and .

Chapter 1 Multi-dimensional Signals and Systems 17

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Sampling Structure Conversion (cont’d)

• Upconversion from to is defined as follows:
ϵ

∉ and ∈

• Downconversion from to as:

∈
• The low-pass filter is applied on the lattice , which has a higher
sampling density than both and .
• By definition, this filter will be shift-invariant if the output gets shifted by a
vector when the input is shifted by . Thus, we need ∈ ∩ . This
condition is satisfied if ∩ is a lattice; that is, is a matrix of rational
numbers. This requirement is the counterpart of / to be a rational number in
the case of 1D sampling rate change problems.

Chapter 1 Multi-dimensional Signals and Systems 18

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Sampling Structure Conversion (cont’d)

• The linear, shift-invariant filtering operation on can be expressed as

• By definition, the upsampling operator for and zero

otherwise


• After downsampling,

• The frequency response of the filter is periodic with the main period equal to unit
∗
cell of . In order to avoid aliasing, the passband of the interpolation/
∗ ∗
anti-alias filter is restricted to the smaller of the Voronoi cells of and .
Chapter 1 Multi-dimensional Signals and Systems 19
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Sampling Structure Conversion: Example 1

• Consider a 2D sampling
lattice conversion from
the input lattice to
the output lattice .
• The sampling densities of
and are inversely
proportional to the
determinants of V1 and V2,
det 2∆ ∆ and
det 4∆ ∆
• Hence, this is a down-
sampling problem.
Chapter 1 Multi-dimensional Signals and Systems 20
 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Example 1 (cont’d)

∗
The frequency-response of the filter is restricted to the Voronoi cell of ,
which is determined by the matrix .

Chapter 1 Multi-dimensional Signals and Systems 21

 Prof. A. Murat Tekalp Digital Video Processing, 2E, Prentice Hall, 2015

Sampling Structure Conversion: Example 2

De-interlacing refers to conversion from an interlaced sampling grid (input) to
a progressive grid (output). The sampling matrices for the input and output grids are
∆ 0 0 ∆ 0 0
0 2∆ ∆ and 0 ∆ 0
0 0 ∆ 0 0 ∆
Note |det | 2 |det |; hence, this is spatio-temporal interpolation by a factor 2.

Chapter 1 Multi-dimensional Signals and Systems 22