CSE397/497-011 Real-time Image Processing

for Autonomous Robot Systems

Lecture 12: Image Processing

in the Frequency Domain

Department of
Computer Science & Engineering

P.C. Rossin
College of Engineering
and Applied Science

CSE397/497 Real-time Image Processing

Class Objectives/Announcements
Review of RIP06 Grand Challenge Rules
Questions?

Objectives
Investigate image representation, analysis and processing
techniques in the frequency domain
Review of Fourier Transforms
Questions about homework assignment?

JR Spletzer

CSE397/497 Real-time Image Processing

References

Gonzalez and Woods, Digital Image Processing,

Chapter 4

HIPR2, Fourier Transform,

http://homepages.inf.ed.ac.uk/rbf/HIPR2/fourier.htm

CVOnline, Fourier Methods,

http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MARSHALL/node15.html

JR Spletzer

CSE397/497 Real-time Image Processing

Things weve forgotten from Calculus

Eulers Identity

e it = cos(t ) + i sin(t )

e it + e it
cos(t ) =
2
e it e it
sin(t ) =
2i

JR Spletzer

CSE397/497 Real-time Image Processing

Definitions

Spatial Domain (I)

Normal image space

Changes in pixel positions correspond to changes in the scene

Distances in I correspond to real distances

Frequency Domain (F)

Changes in image position correspond to changes in the spatial

frequency

This is the rate at which image intensity values are changing in

the spatial domain image I

JR Spletzer

CSE397/497 Real-time Image Processing

Image Processing

Spatial Domain (I)

Directly process the input image pixel array

Frequency Domain (F)

Transform the image to its frequency representation

Perform image processing

Compute inverse transform back to the spatial domain

JR Spletzer

CSE397/497 Real-time Image Processing

Frequencies in an Image

Any spatial or temporal signal has an equivalent

frequency representation

What do frequencies mean in an image ?

High frequencies correspond to pixel values that change

rapidly across the image (e.g. text, texture, leaves, etc.)

Strong low frequency components correspond to large scale

features in the image (e.g. a single, homogenous object that
dominates the image)

We will investigate Fourier transformations to obtain

frequency representations of an image

JR Spletzer

CSE397/497 Real-time Image Processing

The Fourier Series

Periodic functions can be expressed as the sum of sines

and/or cosines of different frequencies each multiplied
by a different coefficient

JR Spletzer

CSE397/497 Real-time Image Processing

The Fourier Transform

Functions that are NOT periodic BUT with finite area under the
curve can be expressed as the integral of sines and/or cosines
multiplied by a weight function
Fourier
Transform

f ( x )e

F (u ) =
Fourier
Synthesis

j 2ux

dx
Fourier
Analysis

f ( x) =

F (u )e

j 2ux

du

Inverse
Fourier
Transform
JR Spletzer

CSE397/497 Real-time Image Processing

The Fourier Transform

The FT transforms can be easily extended to 2 (and n

dimensions)
F (u, v) =

f ( x , y )e

j 2 ( ux + vy )

dxdy

f ( x, y ) =

F (u, v)e

j 2 ( ux + vy )

dudv

The Fourier transform for f(x) exists iff

1.

f(x) is piecewise continuous on every finite interval

2.

f(x) is absolutely integrable

JR Spletzer

CSE397/497 Real-time Image Processing

Properties of the Fourier Transform

The FT is a linear operator

F (af + bg ) = aF ( f ) + bF ( g )

Some other useful properties include

Discrete

Periodic

Real

Symmetric

f+g

F+G

f*g

FG
JR Spletzer

CSE397/497 Real-time Image Processing

Some Fundamental Transform Pairs

Box

F (u ) =

Sinc

j 2xu
dx
f ( x )e j 2xu dx = e
1

1 j 2u j 2u
e
e
j 2u
sin 2u
=
u

JR Spletzer

CSE397/497 Real-time Image Processing

Some Fundamental Transform Pairs

Gaussian

F (u ) =

f ( x )e

Gaussian

j 2xu

dx = e ax (cos 2ux j sin 2ux )dx

2

= e ax cos 2uxdx j e ax sin 2uxdx

2

2 2

u /a

JR Spletzer

CSE397/497 Real-time Image Processing

The Discrete Fourier Transform

Since we are dealing with images, we will be more

interested in the discrete Fourier Transform (DFT)

For a function f(x), x=0,1,,M-1 we have

1
F (u ) =
M
M 1

iDFT

M 1

f ( x )e

j 2ux / M

u = 0,..., M 1

x =0

f ( x) = F (u )e j 2ux / M ,

x = 0,..., M 1

x =0

For discrete functions, the DFT and iDFT always exist

JR Spletzer

CSE397/497 Real-time Image Processing

The Discrete Fourier Transform

Recall Eulers Formula

e j = cos + j sin
from which we obtain
F (u ) =

1
M

M 1

f ( x) cos
x =0

2ux
2ux
j sin

M
M

for u = 0,,M-1

Each term is composed of ALL values of f(x)

The values of u are the frequency domain

Each F(u) is a frequency component of the transform

JR Spletzer

CSE397/497 Real-time Image Processing

The 2-D Discrete Fourier Transform

Since our images are nothing more than 2-D discrete

functions, we are interested in the 2-D DFT

1
F (u , v ) =
MN

M 1 N 1

f ( x, y)e

j 2 ( ux / M + vy / N )

x =0 y = 0

for u=0,,M-1 and v=0,,N-1 and the iDFT is defined

as
M 1 N 1

f ( x, y ) = F (u, v)e j 2 ( ux / M + vy / N )
x =0 y =0

JR Spletzer

CSE397/497 Real-time Image Processing

Some Fundamental Transform Pairs

Comb

Comb

w
2/w

JR Spletzer

CSE397/497 Real-time Image Processing

Sampling

Sampling is equivalent to multiplying with a comb

filter in the spatial domain

This corresponds to convolving with a comb filter in

the frequency domain

Fine sampling in spatial Spread out filter in the

frequency domain

Coarse sampling in Spatial Close filter in the

frequency domain

JR Spletzer

CSE397/497 Real-time Image Processing

Sampling Limits

Shannons Sampling Theorem states that if a function

f is sampled at a rate twice its highest frequency, f
can be completely recovered from its samples

This is known as the Nyquist Frequency Limit

The sampling rate in images is the pixels/unit distance

JR Spletzer

CSE397/497 Real-time Image Processing

Aliasing

For an image sampled spatially, the shortest

wavelength that be represented is 2 pixels

When a signal is under-sampled, aliasing can result

Aliasing is when a high frequency signal masquerades

as a lower frequency signal

JR Spletzer

10

CSE397/497 Real-time Image Processing

Aliasing Examples
75 dpi
Scan

600 dpi
Scan

How might we
handle this effect?
Moire
Pattern

* Wikipedia

JR Spletzer

CSE397/497 Real-time Image Processing

DFT (Continued)

Since the values of the FT are complex numbers, it is

sometimes more convenient to express F(u) in terms of
polar coordinates

F (u , v ) = F (u , v ) e j ( u , v )
where the magnitude or spectrum is denoted by

F (u , v ) = R 2 (u , v ) + I 2 (u, v)

1
2

and the phase angle by

I (u , v)

R (u , v)

(u, v) = tan 1

JR Spletzer

11

CSE397/497 Real-time Image Processing

DFT (Continued)
F (u , v) =

1
MN

M 1 N 1

f ( x, y)e

j 2 ( ux / M + vy / N )

x =0 y = 0

At u=v=0, the FDT reduces to

F (0,0) =

1
MN

M 1 N 1

f ( x, y)
x =0 y =0

This is nothing more than the average grayscale level

of the image

This is often referred to as the DC Component (0

frequency)
JR Spletzer

CSE397/497 Real-time Image Processing

DFT (Continued)

The FT has the following translation property

f ( x, y )e j 2 (u0 x / M + v0 y / N ) F (u u0 , v v0 )
which for u0=M/2 and v0=N/2 we see that

e j 2 (ux / M +vy / N ) = e j ( x + y ) = (1) x + y

F ( f ( x, y )(1) x + y ) = F (u M / 2, v N / 2)

In image processing, it is common to multiply the input

image by (-1)x+y prior to computing F(u,v)

This has the effect of centering the transform since

F(0,0) is now located at u=M/2, v=N/2
JR Spletzer

12

CSE397/497 Real-time Image Processing

FFT Example

For an image sampled spatially, the highest frequency

that can be represented is image size/2

To demonstrate this, we look at an image with 1 pixel

wide vertical bars

Note that the dominant frequency component in the

FT is at the max frequency 100/2 = 50 Hz

JR Spletzer

CSE397/497 Real-time Image Processing

FFT Example

If the bars are twice as wide, then we should expect

that the dominant frequency will drop off accordingly

Again, looking at the FT we see (as expected) that the

dominant harmonic component is at 25

JR Spletzer

13

CSE397/497 Real-time Image Processing

FFT Example

Now lets look at a checkerboard pattern

We now pick up the horizontal, vertical and diagonal

frequency components

JR Spletzer

CSE397/497 Real-time Image Processing

Properties of the Fourier Transform

The FT is a linear operator

F (af + bg ) = aF ( f ) + bF ( g )

Some other useful properties include

Discrete

Real
Basis for
filtering in the
f+g
frequency domain!
f*g

Periodic
Symmetric
F+G
FG
JR Spletzer

14

CSE397/497 Real-time Image Processing

Procedure for Filtering in the Frequency Domain

1.

Multiply the input image by (-1)x+y to center the

transform

2.

Compute the DFT F(u,v) of the resulting image

3.

Multiply F(u,v) by a filter G(u,v)

4.

Computer the inverse DFT transform h*(x,y)

5.

Obtain the real part h(x,y) of 4

6.

Multiply the result by (-1)x+y

JR Spletzer

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel

Traditionally, we would just convolve the image

with the a gaussian kernel

1 4 6 4 1
4 16 24 16 4

6 24 36 24 6 / 256

4 16 24 16 4
1 4 6 4 1

Instead, we will perform multiplication in the

frequency domain to achieve the same effect
JR Spletzer

15

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel
1.

Multiply the input image by (-1)x+y to center the

transform

JR Spletzer

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel
2. Compute the DFT F(u,v) of the resulting image

log transform

JR Spletzer

16

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel
3. Multiply F(u,v) by a filter G(u,v)

1 4 6 4 1
4 16 24 16 4

6 24 36 24 6 / 256

4 16 24 16 4
1 4 6 4 1

g(x,y)
G(u,v)

JR Spletzer

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel
3. Multiply F(u,v) by a filter G(u,v)

JR Spletzer

17

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel
4.

Computer the inverse DFT transform h*(x,y)

5.

Obtain the real part h(x,y) of 4

6.

Multiply the result by (-1)x+y

JR Spletzer

CSE397/497 Real-time Image Processing

Filtering Example
Smooth an Image with a Gaussian Kernel

f ( x, y ) g ( x, y )

F (u, v)G (u, v)

JR Spletzer

18

CSE397/497 Real-time Image Processing

Other Applications of the DFT

Geometrical Information

At times, we can also infer geometric information

about the image from the DFT

The DFT (and its log transform) show after thresholding

the vertical orientation of the banding in this image

* HIPR2

JR Spletzer

CSE397/497 Real-time Image Processing

Geometrical Information

If we rotate the image, the primary frequency

components are similarly aligned

The secondary axis is the result of the black triangles

in the image

* HIPR2
JR Spletzer

19

CSE397/497 Real-time Image Processing

Summary

Images have both spatial and frequency representations

We use the DFT to obtain the latter

High texture results in high frequency components, while solid,

homogenous objects correspond to low frequency components

Aliasing is when high frequency components masquerade as low

frequency ones, and can result from improper image sampling

The effect can be eliminated by filtering out high frequency

components of the signal

Convolution and multiplication are duals in the spatial/frequency

domains

Although multiplication is MUCH more efficient than convolution,

there is a high cost in computing the DFT and iDFT

As such, most convolutions are carried out in the spatial domain

(unless the kernel is sufficiently large)
JR Spletzer

20