MULTIRESOLUTION

ANALYSIS
Image Representation

• Wavelet Pyramids
•Gaussian pyramids
• Laplacian Pyramids
• Applications
What is a good representation
for image analysis?
• Pixel domain representation tells you
“where” (pixel location), but not
“what”.
– In space, this representation is too localized
• Fourier transform domain tells you
“what” (textural properties), but not
“where”.
– In space, this representation is too spread out.
• Want an image representation that gives
you a local description of image events
—what is happening where.
– That representation might be “just right”.
Image representation
Fourier Analysis
 Breaks down a signal into constituent
sinusoids of different frequencies

In other words: Transform the view of the

signal from time-base to frequency-base.
What’s wrong with Fourier?

 By using Fourier Transform , we loose

the time information : WHEN did a
particular event take place ?
 FT can not locate drift, trends, abrupt
changes, beginning and ends of
events, etc.
 Calculating use complex numbers.
 Time and Space
definition

Time – for one dimension •

waves we start point shifting
from source to end in time
. scale
Space – for image point •
. shifting is two dimensional
. Here they are synonyms •
Space-Frequency Tiling
Freq.
Standard basis

Spatial
Freq.
Fourier basis

Spatial
Freq.
Wavelet basis

Spatial
Short Time Fourier Analysis
 In order to analyze small section of a
signal, Denis Gabor (1946), developed a
technique, based on the FT and using
windowing : STFT
STFT (or: Gabor Transform)

 A compromise between time-based

and frequency-based views of a signal.
 both time and frequency are
represented in limited precision.
 The precision is determined by the size
of the window.
 Once you choose a particular size for
the time window - it will be the same
for all frequencies.
What’s wrong with Gabor?

 Many signals require a more flexible

approach - so we can vary the window
size to determine more accurately
either time or frequency.
Space-Frequency Tiling
Freq.
Standard basis

Spatial
Freq.
Fourier basis

Spatial
Freq.
Wavelet basis

Spatial
 Wavelet's properties

Short time localized waves with •

.zero integral value

.Possibility of time shifting •

.Flexibility •
Scaling
 Wavelet analysis produces a time-
scale view of the signal.
 Scaling means stretching or
compressing of the signal.
 scale factor (a) for sine waves:

f ( t ) sin( t ) ; a 1
f ( t ) sin( 2t ) ; a  1 2

f ( t ) sin( 4t ) ; a  1 4
Scaling (Cont’d)

 Scale factor works exactly the same

with wavelets:

f ( t )  (t ) ; a 1
f ( t )  ( 2 t ) ; a  1 2

f ( t )  ( 4 t ) ; a  1 4
 Wavelet function

b – shift •

a , b x  1
a
 x b 
a
coefficient
a – scale •
coefficient

a , bx , by x , y   1a  
x  bx
a , y  by
a  2D function •

 a , bx , by x , y   
a , bx , by x , y  
1
1a  x  bx
x abx ,
,
y  by
y aby 
a a a
Various Wavelet basis
Wavelet - Frequency domain
Wavelet bands are split recursively

image

H L

H L

H L
 Discrete Wavelet Transform

D DWT for Image-2 •

Discrete Wavelet Transform
 Discrete Wavelet Transform

D DWT for Image-2 •

 Wavelet
Decomposition:Example

LENA

LH

HL HH
 Wavelet - Frequency domain
Wavelet decomposition - 2D

Frequency domain

Horizontal high pass Vertical high pass

Horizontal low pass Vertical low pass

 Wavelet - Frequency domain
Apply the wavelet transform separably in both dimensions

Horizontal high pass, Horizontal high pass,

vertical high pass vertical low-pass

Horizontal low pass, Horizontal low pass,

vertical high-pass Vertical low-pass
 JPEG2000
v.s. JPEG

low bit-rate performance

 Image Pyramids
Image features at different resolutions require filters at
different scales.

Edges (derivatives):

f(x)

f (x)
Space Required for Pyramids
 Image Pyramids

Image Pyramid = Hierarchical representation of an image

No details in image -
Low (blurred image)
Resolution low frequencies

High Details in image -

Resolution low+high frequencies

A collection of images at different resolutions.

Image Pyramid

Low resolution

High resolution
 Image Pyramid
Frequency Domain

Low resolution

High resolution
Image Blurring = low pass filtering

*
=

~
=
 *
=

*
=
 Image Pyramid

Low resolution

High resolution
 Gaussian pyramid construction

filter mask

Repeat
• Filter
• Subsample
Until minimum resolution reached
• can specify desired number of levels (e.g., 3-level pyramid)

The whole pyramid is only 4/3 the size of the original image!
Gaussian Pyramid
Level n
1X1

Level 1
2n-1 X 2n-1

Level 0
2n X 2n
 Image Pyramids

Known as a Gaussian Pyramid [Burt and Adelson, 1983]

• In computer graphics, a mip map [Williams, 1983]
• A precursor to wavelet transform
 The Gaussian Pyramid

Low resolution G4 (G3 * gaussian)  2

blur )  2 down-sample
G3 (G2 * gaussian
down-sam
blur ple
G2 (G1 * gaussian)  2
dow
n-sa
mple
blur
G1 (G0 * gaussian)  2
do
wn
-sa
plem
G0 Image
blur

High resolution
Gaussian Pyramid
 Image pyramids

• Gaussian Pyramids
• Laplacian Pyramids
What does blurring take
?away

smoothed (5x5 Gaussian)

High-Pass filter

smoothed – original
Band-pass filtering

Gaussian Pyramid (low-pass images)

 Laplacian Pyramid

Need this!

Original
image

How can we reconstruct (collapse) this •

?pyramid into the original image
 Laplacian Pyramid

Motivation = Compression, redundancy removal.

compression rates are higher for predictable values.
e.g. values around 0.

G0, G1, .... = the levels of a Gaussian Pyramid.

Predict level Gl from level Gl+1 by Expanding Gl+1 to G’l

Gl+1
Expand
Reduce

Gl G’l

Denote by Ll the error in prediction:

Ll = Gl - G’l

L0, L1, .... = the levels of a Laplacian Pyramid.

?What does blurring take away

original
?What does blurring take away

smoothed (5x5 Gaussian)

?What does blurring take away

smoothed – original
 Laplacian Pyramid

Gaussian Laplacian
Pyramid Pyramid

expa
nd

ex
p
- =
an
d

ex
p
- =
an
d

- =
 The Laplacian Pyramid
Li Gi  expand( Gi 1 )
Gaussian Pyramid Gi Li  expand( Gi 1 ) Laplacian Pyramid

Gn Ln Gn
ex pa
nd
G2
an-
L2
exp
d
=
G1 L1
ex
pa -
nd =
G0 L0

- =
 Image Pyramids - Comparison

Image pyramid levels = Filter then sample.

Filters:

Gaussian Pyramid

Laplacian Pyramid

Wavelet Pyramid
Gaussian
Frequency
Laplacian
Pyramid Domain
Pyramid
Laplace Pyramid -
No scaling

50

100
50
150
100
200
150
50
250
50 100 150 200 250
200
100

250
150
50 100 150 200 250

200

250
50 100 150 200 250
50

100

50
150

100
200

150
250
50 100 150 200 250
200

250
50 100 150 200 250
from: B.Freeman
 Reconstruction of the original image
from the Laplacian Pyramid

Laplacian Gl = Ll + G’l
Pyramid

expand

+ =
expand

+ =
expand

+ =

Original
= Image
Multiresolution Spline - Example

Original - Left Original - Right

Glued Splined
laplacian level 4

laplacian level 2

laplacian level 0

left pyramid right pyramid blended pyramid

Multiresolution Spline
Image Blending
Blending
 Multiresolution Spline

When splining two images, transition from one image to

the other should behave:

High Frequencies

Middle Frequencies

Low Frequencies
Multiresolution Spline

High Frequencies

Middle Frequencies

Low Frequencies
Multiresolution Spline - Example

Left Image Right Image

Left + Right Narrow Transition

Wide Transition

(Burt & Adelson)

Multiresolustion Spline - Using Laplacian Pyramid
Multiresolution Spline - Example

Left Image Right Image

Left + Right Narrow Transition

Wide Transition Multiresolution Spline

(Burt & Adelson)

Multiresolution Spline - Example
 The colored version

© prof. dmartin
 Pyramids in Frequency
Domain
Gaussian Pyramid Laplacian Pyramid
Wavelet Decomposition

Fourier Space
 Image Fusion

Multi-scale Transform (MST) = Obtain Pyramid from Image

Inverse Multi-scale Transform (IMST) = Obtain Image from Pyramid