Chapter 5 : Image Transforms

(from Anil. K. Jain)

CCU, Taiwan
Wen-Nung Lie

2-D orthogonal and unitary

transforms

Orthogonal series expansion for an NN image

u(m, n)
v(k , l ) =

N 1

u(m, n)a
m =0

u ( m, n ) =

N 1

( m, n )

0 k,l N 1

v ( k , l )a k*,l ( m, n )

0 m, n N 1

k ,l

n =0

N 1

k =0

N 1

l =0

v(k, l)s are the transform coefficients, V {v(k , l )}

represents the transformed image
{a k ,l ( m, n )} is a set of orthonormal functions,

representing the image transform

CCU, Taiwan
Wen-Nung Lie

5-1

Orthonormality and
completeness

{a k ,l ( m, n )} must satisfies
orthonormality :

completeness :

CCU, Taiwan
Wen-Nung Lie

N 1

N 1

m =0

n =0

N 1

N 1

k =0

l =0

a k ,l ( m, n )a k*,l ( m, n ) = ( k k , l l )

a k ,l ( m, n )a k*,l ( m , n ) = ( m m , n n )

5-2

Matrix representation of image

transform

v(k , l ) =

N 1

N 1

u(m, n)a
m =0

u ( m, n ) =

N 1

CCU, Taiwan
Wen-Nung Lie

k =0

*
k ,l

( m, n )

l =0

N 1

v(k , l ) A*k ,l

l =0

v(k , l ) =< U, A*k ,l >

N 1

f (m, n) g (m, n)
*

m=0

U=

N 1

N 1

v(k , l )a

< F, G >=

( m, n )

n =0

k =0

k ,l

N 1

is the matrix inner product

n =0

Image U can be described as a linear combination

*
of N 2 matrix A k ,l , k, l = 0,...,N-1
A*k ,l are called the basis images
v(k, l) can be considered as the projection of u(m, n) on
the (k, l)-th basis image
5-3

CCU, Taiwan
Wen-Nung Lie

Basis images

5-4

2-D Separable image

transformation

{ak ,l (m, n)} is separable

a k ,l (m, n) = a k (m)bl (n) a ( k , m)b(l , n)
v( k , l) =

u(m, n) =

N 1 N 1

a ( k , m)u(m, n)a (l , n)

V = AUAT

m= 0 n = 0
N 1 N 1

*T
*
*
*

U
=
A
VA
a
(
k
,
m
)
v
(
k
,
l
)
a
(
l
,
n
)

k =0 l =0

CCU, Taiwan
Wen-Nung Lie

A {a(k , m)} B {b(l , n)} are unitary matrices

Consider the above as transforming columns of U by
A, then transforming rows of the result by A T
5-5

Example of image transform

Given

A=

1 1 1
,

2 1 1

compute transform
Basis images

A 0, 0
*

1 2

U =
3 4

1 1 1 1 2 1 1 5 1

V =
2 1 1 3 4 1 1 2 0

1 1 1
1 1

= (1 1) =
2 1 1
2 1

1 1 1
1 1
= A1*T, 0
A 0 ,1 = (1 1) =
2 1 1
2 1
*

1 1 1 5 11 1 1 2

A VA =
2 1 1 2 0 1 1 3 4
*T

CCU, Taiwan
Wen-Nung Lie

1 1 1

A1,1 =
2 1 1
*

5-6

Properties of unitary transform

Given v = Au (A is a unitary transform)

v = u
2

CCU, Taiwan
Wen-Nung Lie

, i.e., energy-conserved

a unitary transformation is a rotation of the basis

coordinates , or a rotation of the vector u in NDimensional vector space
Unitary transform tends to concentrate most of the
signal energy on a small set of transform
coefficients

v = A u

R v = AR u A*T
5-7

Properties of unitary transform

(cont)

CCU, Taiwan
Wen-Nung Lie

The transform coefficients tends to be decorrelated,

i.e., the off-diagonal elements of R v are
approaching zero
The entropy of a random vector is preserved under
a unitary transformation (i.e., information
preserved)

5-8

1-D DFT

1-D DFT pair (unitary transformation)

v(k ) =

N 1

u ( n)WNkn

k = 0,..., N 1

v(k )WNkn

n = 0,..., N 1

n =0

u ( n) =

N 1

k =0

j 2
WN exp

NN transform matrix F for v = Fu

1

kn
F=
WN ,
N

CCU, Taiwan
Wen-Nung Lie

0 k, n N 1
5-9

Properties of DFT

Symmetry F = F T F 1 = F * (unitary : F 1 = F *T )
FFT needs O ( N log 2 N ) operations (DFT needs O ( N 2 ) )
Real DFT is conjugate symmetrical about N/2
v* (

N
N
k ) = v( + k )
2
2

x 2 ( n ) is the circular convolution between h (n ) and x1 ( n )

DFT {x 2 ( n )}N = DFT {h ( n )}N DFT {x1 ( n )}N

CCU, Taiwan
Wen-Nung Lie

Extend the length of h (n ) ( N ) and x1 ( n ) (N) with zeros to have the

same length ( M N + N 1 ), the above equation can be used to
compute linear convolution

5-10

2-D DFT

2-D DFT is a separable transform

1
v(k , l ) =
N
1
u ( m, n ) =
N

N 1

N 1

m =0

N 1

u ( m, n )WNkmWNln

n =0

N 1

k =0

0 k,l N 1

v ( k , l )WNkmWNln

0 m, n N 1

l =0

In matrix form
V = FUF T = FUF

CCU, Taiwan
Wen-Nung Lie

5-11

Properties of 2-D DFT

Symmetric and unitary F = F and F = F

Periodic v (k + N , l + N ) = v (k , l )
V = FUF needs 2N times of 1-D FFT
DFT of real images is conjugate symmetrical with
respect to ( N , N )
1

v(k , l ) = v * ( N k , N l )

N 2 basis images
1
{WN( km+ln) ,0 m, n N 1}
N

CCU, Taiwan
Wen-Nung Lie

0 k,l N 1
5-12

Plots of DFT

CCU, Taiwan
Wen-Nung Lie

5-13

Cosine transform

CCU, Taiwan
Wen-Nung Lie

DCT (discrete cosine transform)

1

,
k = 0,0 n N 1

N
c ( k , n) =
2 cos (2n + 1)k , 1 k N 1,0 n N 1
N
2N

88 2-D DCT

5-14

Properties of DCT

DCT is real and orthogonal C = C and C = C

DCT Real {DFT}
DCT can be calculated via FFT
*

v ( k ) = Re[ ( k )W2kN/ 2 DFT {u~( n )}N ]

u~( n ) = u ( 2n )

N
0

(
) 1
~
2
(

1
)
=
(
2
+
1
)
u
N
n
u
n

CCU, Taiwan
Wen-Nung Lie

For highly correlated data, DCT has good energy

compaction. That is, for first-order stationary Markov
sequence with correlation 1, DCT is approximate
to KL transform
5-15

Sine transform

DST (discrete sine transform)

(k , n) =

( k + 1)( n + 1)
2
,
sin
N +1
N +1

0 k, n N 1

Properties
DST is real, symmetric, and orthogonal. = =
DST and DST-1 are the same in the form (cf. DCT)
DST Imagery {DFT}
For first-order stationary Markov sequence with
correlation (-0.5, 0.5), the energy compaction of
DST is approximate to KL transform
*

CCU, Taiwan
Wen-Nung Lie

= 1

5-16

(Walsh-) Hadamard transform

Kronecker
operator

Hadamard matrix

1 H n1

H n = H n1 H1 =
2 H n1

1 1 1

H1 =
2 1 1

Components of HT vector contain only 1 and -1

CCU, Taiwan
Wen-Nung Lie

H n1

H n1

The number of transitions from 1 to -1 is called

sequency (like in the continuous case)
1

1
1

1 1
H3 =
8 1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1
1
1
1
1
1
1

1
1

1
1

1
1
1

0
7 sequency
3
Not in order
4
1
6
2
5
5-17

(Walsh-) Hadamard transform

(cont.)

Hadamard transform pair :

v(k ) =

v = Hu

u = Hv

N 1

u ( m )( 1)

b ( k ,m )

0 k N 1

m =0

u(m) =

N 1

v ( k )( 1)

b ( k ,m )

0 m N 1

k =0

b( k , m ) =

n 1

k m
i

i =0

CCU, Taiwan
Wen-Nung Lie

k i , mi = 0,1

Notice the disordering effect in the sequency in

HT spectrum when filtering will be performed

5-18

Properties of Hadamard
transform

Real, symmetric, and orthogonal H = H = H = H

Fast computation (only addition is needed)
Can be decomposed into product of sparse matrix
*

~
H = Hn = Hn

1 1

0 0
. .

~
1 0 0
H=
2 1 1
0 0
. .

0 0

CCU, Taiwan
Wen-Nung Lie

. . .

0 . .

. . .

. . 1

. . .

1 1 0 . .
.

. . .

. . 1

n = log 2 N
.

.
.

1
.

.
.
1

~n
~~ ~
v = Hu = H u = HH ...Hu

For highly correlated images, Hadamard transform also has

good energy compaction
5-19

Plot of Hadamard transform

CCU, Taiwan
Wen-Nung Lie

5-20

KL transform of 1-D vector

For a real N1 random vector u with autocorrelation matrix

R, KL transform of u is defined as
v = *T u

u = v

where R k = k k , 0 k N 1 , i.e., ks are orthonormalized

eigenvectors of R and k is the k-th column of
reduces R to its diagonal form, i.e.,
*T R = = Diag{k }

CCU, Taiwan
Wen-Nung Lie

If we replace R with R 0 R T, then eigenmatrix of R 0

is the KL transform of u-
KL transform is independent of the image data themselves,
but related to its 2nd-order statistics
5-21

KL transform of 2-D image

FFT, DCT, DST all are special cases of KL transform

For an image u(m, n) of NN pixels, its autocorrelation
function denoted as r ( m, n; m , n ) (or an N N R matrix),
the basis images of the KL transform are k ,l ( m, n ) :
2

N 1

N 1

r ( m, n; m , n ) k ,l ( m , n ) = k ,l k ,l ( m, n )

m=0 n=0

or

Ri = ii

0 k,l N 1
0 m, n N 1

i = 0,..., N 2 1

i is the N 1 vector representation of k ,l ( m, n )

2

CCU, Taiwan
Wen-Nung Lie

5-22

KL transform of 2-D image

(cont.)

If R matrix is separable, then = {i } of N N is also

separable
Kronecker
R = R1 R 2
= 1 2
operator
2

where

j R j*jT = j ,

j = 1,2

*T
v
=

u or V = U
KL transform :

Advantages of this separability : reduce the eigenvalue

problem of R matrix ( N N ) into two eigenvalue
problems of R1, R2 ( N N )
2

CCU, Taiwan
Wen-Nung Lie

*T

5-23

Properties of the KL transform

Decorrelation
{v ( k ), k = 0,..., N 1} are uncorrelated and have zero mean
is not unique, not necessarily be the eigenmatrix of R

Minimum basis restriction error

u

A
NN

Im
1mN

B
NN

Signal u is transformed via A to be v, clip the first m values by Im

to form w, and transformed back to z via BThe basis restriction
error is defined as

1
J m E
N
CCU, Taiwan
Wen-Nung Lie

N 1

n =0

2
1
u (n) z (n) = Tr[ E{(u z )(u z )*T }]
N

5-24

Properties of the KL transform

(cont.)

Find A KL and B KL (KL transform) such that J m is minimized for

each value of m [1, N ] . (KL is effective in compacting the energy)
Select A = *T and B = , AB = I , and columns of are sorted
according to their corresponding eigenvalues 0 1 2 ..... N 1

Minimum transmission rate at a given distortion

In a noisy channel, given a distortion specification,

D=

1
E[(u u. )*T (u u. )]
N

KL transform can, among all unitary transformation A (include A

= I ), achieve the minimum transmission rate
R (*T ) R ( A )
u
CCU, Taiwan
Wen-Nung Lie

A
(unitary)

v.

v
channel

u.
A*T
5-25

An example of KL-transform

A 21 vector u, whose covariance matrix is

1
R=
<1
,
1
The KL transform is

1 1 1
= =

1
2

E{[v(0)]2 } = 0 = 1 + , E{[v(1)]2 } = 1
v = u
1
1
1+
1
1
) + max(0, log
)]
R() = [max(0, log

2
2
2
comparing R(I ) = 1 [2 log ],
0 < <1
4
*T

CCU, Taiwan
Wen-Nung Lie

R () < R (I )

Let be small, say < 1

5-26

Comparison of energy
distribution between transforms

CCU, Taiwan
Wen-Nung Lie

For first-order stationary Markov process with

large , DCT is comparable to KL
With small , DST is comparable to KL
For any , we can find a faster sinusoidal
transform (DCT or DST) to substitute for the
optimal KL transform which needs covariance
matrix to compute transform basis vectors

5-27

CCU, Taiwan
Wen-Nung Lie

5-28

CCU, Taiwan
Wen-Nung Lie

5-29

Singular Value Decomposition

(SVD)

Um=km

For an NM matrix U (M N), we can find r orthogonal

M1 eigenvector { m } by using U T U , or find r orthogonal
N1 eigenvector { m } by using UUT (r is the rank of U)
UT U m = m m ,
m = 1,..., r UUT and UTU have the

UU T m = m m ,

m = 1,..., r

Matrix U can be described as

U = =
1/ 2

CCU, Taiwan
Wen-Nung Lie

same eigenvalues

1/ 2 =

.
0

.
.

m m Tm

Note. In KL, R is
symmetrical, but
U is not

m =1

Called the spectral representation,

outer product expansion, or
singular value decomposition
(SVD) of U
5-30

Singular Value Decomposition

(cont)
Note: U is not symmetrical

SVD of U (diagonalization) : 1/ 2 = T U

U can be described as the composition of r matrix (corresponding

to different m , called Us singular values)

If U is an NM image, U can be described as an NM1

vector, also as
{1m/ 4 m , 1m/ 4 m ; m = 1,..., r}
needing totally (M+N)r data and be more efficient than
NM or NM1 if r << M

CCU, Taiwan
Wen-Nung Lie

5-31

Properties of SVD

We can compute

U m , m = 1,...r

, when knowing

U = 1 / 2 T

due to the same eigenvalue

Energy conservation
E = u 2 ( m, n ) = m
m

T
Denote U k m m m , k r, as Us k partial-sum ( m is
m =1
ordered)
U k is the optimal rank-k description, with approximation error
k

k = u ( m, n ) uk ( m, n ) = m ,
M

m =1

n =1

m = k +1

The energy of each SVD spectral band

CCU, Taiwan
Wen-Nung Lie

k = 1,..., r

m m Tm is m
5-32

Properties of SVD (cont)

For a given image, SVD can compact the largest energy

for a specified number of transform coefficients (optimized
independently)

CCU, Taiwan
Wen-Nung Lie

The maximal number of basis images for SVD is N, while for other
transforms, it is N2

For KL transform, it optimizes one set of ensemble with

the same covariance matrix, resulting in the largest
average energy in probabilistic sense

5-33

Properties of SVD (cont)

Example :
1 2
U = 2 1
1 3

4.25 0
SVD : =

0
1
.
39

1 = 18.06

2 = 1.94

0.8649
0.5019
1 =

=
,
2
0.5019

0
.
8649

1.120 1.94
U 1 = 11 T = 0.935 1.62
1.549 2.70
1

2
1
Using DCT : V = C 3 UCT2 =
3
12
1

CCU, Taiwan
Wen-Nung Lie

10 2
2
2 1 2

1 1
1
=
0 3 2 1
3

1
1

12

1
2
1 1 3

2 2

3
5

v 2 ( k , l ) = 1 + 2
5-34