You are on page 1of 109

Wavelets (Chapter 7)

CS474/674 Prof. Bebis



STFT (revisited)
Time/Frequency localization depends on window size.
Once you choose a particular window size, it will be the same
for all frequencies.
Many signals require a more flexible approach - vary the
window size to determine more accurately either time or
frequency.

The Wavelet Transform
Overcomes the preset resolution problem of the STFT by
using a variable length window:

Use narrower windows at high frequencies for better time
resolution.

Use wider windows at low frequencies for better frequency
resolution.

The Wavelet Transform (contd)
Wide windows do not provide good localization
at high frequencies.
The Wavelet Transform (contd)
Use narrower windows at high frequencies.
The Wavelet Transform (contd)
Narrow windows do not provide good localization
at low frequencies.
The Wavelet Transform (contd)
Use wider windows at low frequencies.
What are Wavelets?
Wavelets are functions that wave above and below the
x-axis, have (1) varying frequency, (2) limited duration,
and (3) an average value of zero.

This is in contrast to sinusoids, used by FT, which have
infinite energy.
Sinusoid Wavelet
What are Wavelets? (contd)
Like sines and cosines in FT, wavelets are used as basis
functions
k
(t) in representing other functions f(t):




Span of
k
(t): vector space S containing all functions f(t)
that can be represented by
k
(t).
( ) ( )
k k
k
f t a t =

What are Wavelets? (contd)



There are many different wavelets:





Morlet
Haar Daubechies
(dyadic/octave grid)
What are Wavelets? (contd)
( )
jk
t =
What are Wavelets? (contd)
time localization
scale/frequency
localization
( )
/2
( ) 2 2
j j
jk
t t k =
j
Continuous Wavelet Transform (CWT)
( )
1
( , )
t
t
C s f t dt
s
s
t
t
-

| |
=
|
\ .
}
Continuous Wavelet Transform
of signal f(t)
translation parameter,
measure of time
scale parameter
(measure of frequency)
Mother wavelet
(window)
normalization
constant
Forward
CWT:
Scale = 1/j = 1/Frequency
CWT: Main Steps
1. Take a wavelet and compare it to a section at the start
of the original signal.

2. Calculate a number, C, that represents how closely
correlated the wavelet is with this section of the
signal. The higher C is, the more the similarity.

CWT: Main Steps (contd)
3. Shift the wavelet to the right and repeat steps 1 and 2 until
you've covered the whole signal.





CWT: Main Steps (contd)
4. Scale the wavelet and repeat steps 1 through 3.







5. Repeat steps 1 through 4 for all scales.


Coefficients of CTW Transform
( )
1
( , )
t
t
C s f t dt
s
s
t
t
-

| |
=
|
\ .
}
Wavelet analysis produces a time-scale view of the input
signal or image.
Continuous Wavelet Transform (contd)
Inverse CWT:


1
( ) ( , ) ( )
s
t
f t C s d ds
s
s
t
t
t t

=
} }
double integral!
FT vs WT
weighted by F(u)
weighted by C(,s)
Properties of Wavelets
Simultaneous localization in time and scale
- The location of the wavelet allows to explicitly represent
the location of events in time.
- The shape of the wavelet allows to represent different
detail or resolution.


Properties of Wavelets (contd)
Sparsity: for functions typically found in practice,
many of the coefficients in a wavelet representation
are either zero or very small.



Linear-time complexity: many wavelet
transformations can be accomplished in O(N) time.

1
( ) ( , ) ( )
s
t
f t C s d ds
s
s
t
t
t t

=
} }
Properties of Wavelets (contd)
Adaptability: wavelets can be adapted to represent a
wide variety of functions (e.g., functions with
discontinuities, functions defined on bounded domains
etc.).
Well suited to problems involving images, open or closed
curves, and surfaces of just about any variety.
Can represent functions with discontinuities or corners more
efficiently (i.e., some have sharp corners themselves).

Discrete Wavelet Transform (DWT)
( ) ( )
jk jk
k j
f t a t =

( )
/2
( ) 2 2
j j
jk
t t k =
(inverse DWT)
(forward DWT)
where
*
( ) ( )
jk
jk
t
a f t t =

DFT vs DWT
FT expansion:




WT expansion
or
one parameter basis
( ) ( )
l l
l
f t a t =

( ) ( )
jk jk
k j
f t a t =

two parameter basis


Multiresolution Representation using
( ) ( )
jk jk
k j
f t a t =

( ) f t
( )
jk
t
j
fine
details
coarse
details
wider, large translations
Multiresolution Representation using
( ) ( )
jk jk
k j
f t a t =

( ) f t
( )
jk
t
j
fine
details
coarse
details
Multiresolution Representation using
( ) ( )
jk jk
k j
f t a t =

( ) f t
( )
jk
t
j
fine
details
coarse
details
narrower, small translations
Multiresolution Representation using
high resolution
(more details)
low resolution
(less details)

( ) ( )
jk jk
k j
f t a t =

( ) f t
1

( ) f t
2

( ) f t

( )
s
f t
( )
jk
t
j
Prediction Residual Pyramid (revisited)
In the absence of quantization errors, the approximation
pyramid can be reconstructed from the prediction residual
pyramid.

Prediction residual pyramid can be represented more
efficiently.
(with sub-sampling)
Efficient Representation Using Details
details D
2
L
0
details D
3
details D
1
(no sub-sampling)
Efficient Representation Using Details (contd)
representation: L
0
D
1
D
2
D
3
A wavelet representation of a function consists of
(1) a coarse overall approximation
(2) detail coefficients that influence the function at various scales.
(decomposition
or analysis)
in general: L
0
D
1
D
2
D
3
D
J
Reconstruction (synthesis)
H
3
=L
2
+D
3
details D
2
L
0
details D
3
H
2
=L
1
+D
2
H
1
=L
0
+D
1
details D
1
(no sub-sampling)
Example - Haar Wavelets
Suppose we are given a 1D "image" with a resolution
of 4 pixels:
[9 7 3 5]

The Haar wavelet transform is the following:


L
0
D
1
D
2
D
3
(with sub-sampling)
Example - Haar Wavelets (contd)
Start by averaging the pixels together (pairwise) to get
a new lower resolution image:


To recover the original four pixels from the two
averaged pixels, store some detail coefficients.


1
Example - Haar Wavelets (contd)
Repeating this process on the averages gives the full
decomposition:






1
Example - Haar Wavelets (contd)
The Harr decomposition of the original four-pixel
image is:


We can reconstruct the original image to a resolution
by adding or subtracting the detail coefficients from
the lower-resolution versions.



2
1 -1
Example - Haar Wavelets (contd)
Note small magnitude
detail coefficients!
D
j
D
j-1
D
1
L
0
How to
compute D
i
?
Multiresolution Conditions
If a set of functions can be represented by a weighted
sum of (2
j
t - k), then a larger set, including the
original, can be represented by a weighted sum of
(2
j+1
t - k):



time localization
scale/frequency
localization
low
resolution
high
resolution
j
Multiresolution Conditions (contd)
If a set of functions can be represented by a weighted
sum of (2
j
t - k), then a larger set, including the
original, can be represented by a weighted sum of
(2
j+1
t - k):



V
j
: span of (2
j
t - k):
( ) ( )
j k jk
k
f t a t =

V
j+1
: span of (2
j+1
t - k):
1 ( 1)
( ) ( )
j k j k
k
f t b t
+ +
=

1 j j
V V
+
_
Nested Spaces V
j

(t - k)
(2t - k)
(2
j
t - k)

V
0
V
1
V
j
V
j
: space spanned by (2
j
t - k)

Multiresolution conditions nested spanned spaces:


( ) ( )
jk jk
k j
f t a t =

f(t) V
j
Basis functions:
i.e., if f(t) V
j
then f(t) V
j+1
1 j j
V V
+
c
How to compute D
i
? (contd)
( ) ( )
jk jk
k j
f t a t =

f(t) V
j
IDEA: define a set of basis
functions that span the
differences between V
j
Orthogonal Complement W
j

Let W
j
be the orthogonal complement of V
j
in V
j+1

V
j+1
= V
j
+ W
j
How to compute D
i
? (contd)
If f(t) V
j+1
, then f(t)

can be represented using basis
functions (t) fromV
j+1
:






1
( ) (2 )
j
k
k
f t c t k
+
=

( ) (2 ) (2 )
j j
k jk
k k
f t c t k d t k = +

V
j+1
= V
j
+ W
j
Alternatively, f(t) can be represented using two basis functions,
(t) from V
j
and (t) from W
j
:

V
j+1
Think of W
j
as a means to represent the parts of a function
in V
j+1
that cannot be represented in V
j
1
( ) (2 )
j
k
k
f t c t k
+
=

( ) (2 ) (2 )
j j
k jk
k k
f t c t k d t k = +

V
j
, W
j
How to compute D
i
? (contd)
differences
between
V
j
and V
j+1
How to compute D
i
? (contd)
using recursion on V
j
:

( ) ( ) (2 )
j
k jk
k k j
f t c t k d t k = +

V
0
W
0
, W
1
, W
2
,

basis functions basis functions
V
j+1
= V
j-1
+W
j-1
+W
j
= = V
0
+ W
0
+ W
1
+ W
2
+ + W
j
if f(t) V
j+1
, then:
V
j+1
= V
j
+ W
j
Summary: wavelet expansion (Section 7.2)
Wavelet decompositions involve a pair of waveforms
(mother wavelets):



The two shapes are translated and scaled to produce
wavelets (wavelet basis) at different locations and on
different scales.
(t) (t)
(t-k) (2
j
t-k)
encode low
resolution info
encode details or
high resolution info
Summary: wavelet expansion (contd)
f(t) is written as a linear combination of (t-k) and
(2
j
t-k) :
( ) ( ) (2 )
j
k jk
k k j
f t c t k d t k = +

scaling function wavelet function
1D Haar Wavelets
Haar scaling and wavelet functions:
computes average
computes details
(t) (t)
1D Haar Wavelets (contd)
Think of a one-pixel image as a function that is constant
over [0,1)


We will denote by V
0
the space of all such functions.



Example:
0 1
1D Haar Wavelets (contd)
Think of a two-pixel image as a function having two
constant pieces over the intervals [0, 1/2) and [1/2,1)



We will denote by V
1
the space of all such functions.

Note that

Examples:
0 1
0 1
V V c
=
+
1D Haar Wavelets (contd)
V
j
represents all the 2
j
-pixel images
Functions having constant pieces over 2
j
equal-sized
intervals on [0,1).



Note that





Examples:
width: 1/2
j

V
j

V
j

1 j j
V V

c
1D Haar Wavelets (contd)
V
0
, V
1
, ..., V
j
are nested
i.e.,
V
J


V
2
V
1
coarse details
fine details
1 j j
V V
+
c
1D Haar Wavelets (contd)
Mother scaling function:



Lets define a basis for V
j
:

0 1
note alternative notation:
( ) ( )
j
i ji
x x
1
1D Haar Wavelets (contd)
1D Haar Wavelets (contd)
Suppose W
j
is the orthogonal complement of V
j
in V
j+1

1D Haar Wavelets (contd)
Mother wavelet function:




Note that (x) . (x) = 0 (i.e., orthogonal)
1
-1
0 1/2 1
0 1
.
= 0
1
-1
0 1/2 1
1
1D Haar Wavelets (contd)
Mother wavelet function:




Lets define a basis
j
i
for W
j
:

1
-1
0 1
( ) ( )
j
i ji
x x
note alternative notation:
1D Haar Wavelets (contd)
j=1
basis W
1
:
basis for V
1
:
Note
that inner
product
is zero!
1D Haar Wavelets (contd)
Basis functions
j
i
of W
j

Basis functions
j
i
of V
j

form a basis in V
j+1
1D Haar Wavelets (contd)
V
3
= V
2
+ W
2
1D Haar Wavelets (contd)
V
2
= V
1
+ W
1
1D Haar Wavelets (contd)
V
1
= V
0
+ W
0
1D Haar Wavelets- Summary
(t)
(t)
Example - Haar basis (revisited)






1
Decomposition of f(x)
V
2

0,2
(x)

1,2
(x)

2,2
(x)

3,2
(x)
f(x)=
Decomposition of f(x) (contd)
V
1
and W
1

V
2
=V
1
+W
1

0,1
(x)

1,1
(x)

0,1
(x)

1,1
(x)
Example - Haar basis (revisited)






1
Decomposition of f(x) (contd)
Decomposition of f(x) (contd)
V
2
=V
1
+W
1
=V
0
+W
0
+W
1

V
0
,W
0
and W
1

0,0
(x)

0,0
(x)

0,1
(x)

1,1
(x)
Example - Haar basis (revisited)






1
Example
Example (contd)
Filter banks (analysis)
The lower resolution coefficients can be calculated
from the higher resolution coefficients by a tree-
structured algorithm (filter bank).



a
0k
(j=0)
a
1k
(j=1)
(t - k)
0 0 0

( ) ( )
k k
k j
f t a t =

1 1 1

( ) ( )
k k
k j
f t a t =

(2t - k)
Example:
Filter banks (analysis) (contd)
The lower resolution coefficients can be calculated
from the higher resolution coefficients by a tree-
structured algorithm (filter bank).

h
0
(-n) is a lowpass filter and h
1
(-n) is a highpass filter
Subband
encoding!
Example - Haar basis (revisited)
[9 7 3 5]
low-pass,
down-sampling
high-pass,
down-sampling
(9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2
V
1
basis functions
Filter banks (analysis) (contd)
Example - Haar basis (revisited)
[9 7 3 5]
high-pass,
down-sampling
low-pass,
down-sampling
(8+4)/2 (8-4)/2
V
1
basis functions
Convention for illustrating
1D Haar wavelet decomposition
x x x x x x x x
detail
average

re-arrange:
re-arrange:
V
1
basis functions
Convention for illustrating
1D Haar wavelet decomposition (contd)
x x x x x x x x
detail
average

Orthogonality and normalization
The Haar basis forms an orthogonal basis
It can become orthonormal through the following
normalization:

since ( ) 2 , ( ) 2
j j
ji ji
x x

= =
/ 2
/ 2
( ) 1/ 2 (2 ) 2 (2 )
( ) 1/ 2 (2 ) 2 (2 )
j j j j j
i
j j j j j
i
x x i x i
x x i x i
| | |

= =
= =
Examples of lowpass/highpass
analysis filters
Daubechies
Haar
h
0
h
1
h
0
h
1
Filter banks (synthesis)
The higher resolution coefficients can be calculated from
the lower resolution coefficients using a similar structure.
Filter banks (synthesis) (contd)
Examples of lowpass/highpass
synthesis filters
Daubechies
Haar (same as
for analysis):
+
g
0
g
1
g
0
g
1
2D Haar Wavelet Transform
The 2D Haar wavelet decomposition can be computed
using 1D Haar wavelet decompositions (i.e., 2D Haar
wavelet basis is separable).
Two decompositions
Standard decomposition
Non-standard decomposition
The two decompositions correspond to a different set
of 2D basis functions.
Standard Haar wavelet decomposition
Steps

(1) Compute 1D Haar wavelet decomposition of each row of
the original pixel values.

(2) Compute 1D Haar wavelet decomposition of each
column of the row-transformed pixels.
Standard Haar wavelet decomposition
(contd)
x x x x
x x x x

.
x x x ... x
(1) row-wise Haar decomposition:

detail
average

.


.
re-arrange terms
Standard Haar wavelet decomposition
(contd)
(1) row-wise Haar decomposition:

detail
average


.

.

row-transformed result
Standard Haar wavelet decomposition
(contd)
(2) column-wise Haar decomposition:

detail
average


.



.

row-transformed result
column-transformed result
Example



.
row-transformed result

.
re-arrange terms
Example (contd)



.
column-transformed result
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

0,0
(x)

0,0
(x)

0,1
(x)

1,1
(x)
V
2
=V
0
+W
0
+W
1

Example:
2D Haar basis for standard decomposition
To construct the standard 2D Haar wavelet basis, consider
all possible outer products of 1D basis functions.

00
(x),
00
(x)

00
(x),
00
(x)

01
(x),
00
(x)
( ) ( )
j
i ji
x x ( ) ( )
j
i ji
x x
2D Haar basis of standard decomposition
( ) ( )
j
i ji
x x
( ) ( )
j
i ji
x x
V
2

Non-standard Haar wavelet decomposition
Alternates between operations on rows and columns.

(1) Perform one level decomposition in each row (i.e., one
step of horizontal pairwise averaging and differencing).

(2) Perform one level decomposition in each column from
step 1 (i.e., one step of vertical pairwise averaging and
differencing).

(3) Repeat the process on the quadrant containing averages
only (i.e., in both directions).
Non-standard Haar wavelet decomposition
(contd)
x x x x
x x x x

.
x x x . . . x
one level, horizontal
Haar decomposition:


.



.
one level, vertical
Haar decomposition:

Note: averaging/differencing
of detail coefficients shown as
Non-standard Haar wavelet decomposition
(contd)
one level, horizontal
Haar decomposition
on green quadrant
one level, vertical
Haar decomposition
on green quadrant


.


re-arrange terms



.

Example


.


re-arrange terms
Example (contd)



.
2D Haar basis for non-standard decomposition

Defined through 2D scaling and wavelet functions:

( , ) ( ) ( )
( , ) ( ) ( )
( , ) ( ) ( )
( , ) ( ) ( )
x y x y
x y x y
x y x y
x y x y
|| | |
| |
| |

=
=
=
=
0
00
( , ) ( , )
( , ) 2 (2 , 2 )
( , ) 2 (2 , 2 )
( , ) 2 (2 , 2 )
j j j j
kf
j j j j
kf
j j j j
kf
x y x y
x y x k y f
x y x k y f
x y x k y f
|| ||
| |
| |

=
=
=
=
2D Haar basis for non-standard decomposition
(contd)


0
00
( , ) ( , )
( , ) 2 (2 , 2 )
( , ) 2 (2 , 2 )
( , ) 2 (2 , 2 )
j j j j
kf
j j j j
kf
j j j j
kf
x y x y
x y x k y f
x y x k y f
x y x k y f
|| ||
| |
| |

=
=
=
=
LL
LH: intensity variations along
columns (horizontal edges)
HL: intensity variations along
rows (vertical edges)
HH: intensity variations along
diagonals
LL: average
Detail coefficients
Three sets of detail coefficients (i.e., subband encoding)
2D Haar basis for non-standard decomposition
(contd)
( ) ( )
j
i ji
x x
( ) ( )
j
i ji
x x
V
2

Forward/Inverse DWT
(using textbooks notation)

H LH
V HL
D HH
LL
i=H,V,D
2D DWT using filter banks (analysis)
H LH
V HL
D HH
LH HH
LL HL
Illustrating 2D wavelet decomposition
The wavelet transform can be applied again on the lowpass-lowpass
version of the image, yielding seven subimages.
LL LH
HL
HH
LL
HH
HL
LH
2D IDWT using filter banks (synthesis)
H LH
V HL
D HH
Wavelets Applications
Noise filtering
Image compression
Fingerprint compression
Image fusion
Recognition
G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by
Fusing Thermal Infrared and Visible Imagery", Image and Vision
Computing, vol. 24, no. 7, pp. 727-742, 2006.
Image matching and retrieval
Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast
Multiresolution Image Querying", SIGRAPH, 1995.


Image Querying Using Wavelets
query by content or
query by example

Typically, the K best
matches are reported.
Fast Multiresolution Image Querying
painted low resolution target
queries