COL LEGE OF E N GI N E ER I N G ROOR K EE

Established in 1998

Digital Image Processing (TCS-071)

Session: 2020-21

Digital Image Processing

 COURSE OBJECTIVE

The aim of this course is,

 Describe and explain basic principles of digital image
processing.
 Design and implement algorithms that perform basic image
processing (e.g. noise removal and image enhancement).
 Design and implement algorithms for advanced image
analysis (e.g. image compression, image segmentation).
 Assess the performance of image processing algorithms and
systems.

Digital Image Processing

 Unit 2
Image Enhancement in Frequency Domain

Fourier Transform and the Frequency Domain, Basis of

Filtering in Frequency Domain, Filters – Low-pass, High-
pass; Correspondence Between Filtering in Spatial and
Frequency Domain; Smoothing Frequency Domain
Filters – Gaussian Lowpass Filters; Sharpening Frequency
Domain Filters – Gaussian Highpass Filters;
Homomorphic Filtering.

Digital Image Processing

Digital Image Processing
Image Enhancement in the
Frequency Domain

Digital Image Processing

Image Processing in Frequency Domain

• Restoration / Enhancement
• Inverse Filtering
• Match Filtering / Pattern Detection
• Tomography
 Enhancement v.s. Restoration
• Image Enhancement:
– A process which aims to improve bad images so
they will “look” better.
• Image Restoration:
– A process which aims to invert known degradation
operations applied to images.
 Enhancement vs. Restoration
• “Better” visual • Remove effects of
representation sensing environment

• Subjective • Objective

• No quantitative • Mathematical, model

measures dependent quantitative
measures
 Typical Degradation Sources

Low Illumination
Optical distortions
(geometric, blurring)

Sensor distortion
(quantization, sampling,
sensor noise, spectral sensitivity,
Atmospheric attenuation
de-mosaicing)
(haze, turbulence, …)
 Image Preprocessing

Enhancement Restoration

• Denoising
• Inverse filtering
Spatial Freq.
• Wiener filtering
Domain Domain

Filtering
Point operations Spatial operations
Examples

Hazing
Echo image Motion Blur
Blurred image Blurred image + additive white noise
Reconstruction as an Inverse Problem
n noise
Original image Distortion
f g Hf n
H
measurements

fˆ
Reconstruction
g Algorithm
 So what is the problem?

g H 1
g  n fˆ

• Typically:
– The distortion H is singular or ill-posed.
– The noise n is unknown, only its statistical
properties can be learnt.
Key point: Stat. Prior of Natural Images

likelihood prior
MAP estimation:
fˆ  arg max P  f g   arg max P g f P  f 
x x
 Bayesian Reconstruction (MAP)
• From amongst all possible solutions, choose the one that
maximizes the a-posteriori probability:

P(f | g)P(g | f) P(f)

Most probable solution

P(f)
measurements

P(g|f)

Image space
 Bayesian Denoising
• Assume an additive noise model :
g=f + n
• A MAP estimate for the original f:
fˆ  arg max P  f | g 
f

• Using Bayes rule and taking the log likelihood :

ˆf  arg max P g | f P  f   arg min  log P g | f   log P  f 

f P g  f
 Bayesian Denoising
• If noise component is white Gaussian distributed:
g=f + n where n is distributed ~N(0,)


ˆf  arg min g  f 2   R  f 
f

data term prior term

R(f) is a penalty for non probable f

 Inverse Filtering
• Degradation model:

g(x,y) = h(x,y)*f(x,y)

G(u,v)=H(u,v)F(u,v)

F(u,v)=G(u,v)/H(u,v)
 Inverse Filtering (Cont.)
Two problems with the above formulation:
1. H(u,v) might be zero for some (u,v).
2. In the presence of noise the noise might be amplified:

F(u,v)=G(u,v)/H(u,v) + N(u,v)/H(u,v)

Solution: Use prior information

F  arg min HF  G   R F 

ˆ 2
F
data term prior term
 Option 1: Prior Term
• Use penalty term that restrains high F values:

Fˆ  arg min E F 
F

E F   HF  G    F
2 2
where

• Solution:
E F 
 2 H HF  G   2F  0
*

F
*
H H (u , v )  1  Fˆ  G H
Fˆ  * G
H H  H (u , v )  1  Fˆ  0
Degraded Image (echo)

F=G/H
 *
H
Fˆ  G
H H 
*
Degraded Image (echo+noise)
 *
H
Fˆ  G
H H 
*
• The inverse filter is C(H)= H*/(H*H+ )
• At some range of (u,v):
S(u,v)/N(u,v) < 1 noise
amplification.
18

16

14

12

10
C(H)

8 =10-3
6

0
-1 -0.5 0 0.5 1 1.5 2
H
 Option 2: Prior Term
1. Natural images tend to have low energy at high frequencies
2. White noise tend to have constant energy along freq.

where
Fˆ  arg min E F 
F


E F   HF  G    u 2  v 2 F 2
2

240

220

200

180

160

140

120

100

80

60

40

50 100 150 200 250

 E F 
• Solution:  2 H HF  G   2 u 2  v 2 F  0
*

F

*
H
Fˆ  G
H H  u v
*
 2 2

• This solution is known as the Wienner Filter
• Here we assume N(u,v) is constant.
• If N(u,v) is not constant:
*
H
Fˆ  * G

H H   u  v  N (u , v)
2 2

Degraded Image (echo+noise)
Wienner Filtering
Wienner Previous
Degraded Image (blurred+noise)
Inverse Filtering
Using Prior (Option 1)
Wienner Filtering
 Matched Filter in Freq. Domain
• Pattern Matching:
– Finding occurrences of a particular pattern in an image.

• Pattern:
– Typically a 2D image fragment.
– Much smaller than the image.
 Image Similarity Measures
• Image Similarity Measure:
– A function that assigns a nonnegative real value to two
given images.
– Small measure  high similarity
– Preferable to be a metric distance (non-negative, identity,
symmetric, triangular inequality)

d( - )≥0
• Can be combined with thresholding:
1 d ( P, Q)  threshold 
f ( P, Q )   
 0 otherwise 
 The Matching Approach
• Scan the entire image pixel by pixel.
• For each pixel, evaluate the similarity between its local
neighborhood and the pattern.
 The Euclidean Distance
as a Similarity Measure
• Given:
– k×k pattern P
– n×n image I
– kxk window of image I located at x,y - Ix,y
• For each pixel (x,y), we compute the distance:
1
d I , P x , y
2
2
E  2 I x, y  P 
k
k 1
1
 2  I  x  i , y  j   P i , j 2

k i , j 0

• Complexity O(n2k2)
 FFT Implementation
1
d I , P x , y
2
2
E  2 I x, y  P
k
k 1
  I 2
 x  i , y  j  P 2
i, j   2 I x  i, y  j Pi, j 
i , j 0

I 2 *  k  k mask of 1's  Fixed 2( I * P)

• Convolution can be applied rapidly using FFT.

• Complexity: O(n2 log n)
 Naïve vs. FFT

:Performance table for a 1024×1024 image, on a 1.8 GHz PC

Naïve FFT
Time Complexity O(n 2 k 2 ) O(n 2 log n)
Space n2 n2
Integer Arithmetic Yes No
Run time for 16×16 .Sec 1.33 .Sec 3.5
Run time for 32×32 .Sec 4.86 .Sec 3.5
Run time for 64×64 .Sec 31.30 .Sec 3.5
 7
x 10

1.5

0.5

0
 Normalized Cross Correlation

• NCC:
– A similarity measure, based on a normalized cross-
correlation function.
– Maps two given images to [0,1] (absolute value).
– Measures the angle between vectors Ix,y and P
– Invariant to intensity scale and offset.

d NC I , P  
I x, y  I 1 P  P1
x, y
I x , y  I 1 P  P1
 Efficient Implementation
• Note that X  X 1 Y  Y 1  X  Y  nXY
• Thus, d NCI , P  
I  I 1 P  P1
x, y x, y

x, y
I x , y  I x , y 1 P  P1

I x, y  P  k 2 I x, y  P

I x, y  I x, y  k 2 2
I x ,
y P  P  k 2 2
P 
• The above expression can be implemented efficiently using 3
convolutions.
 7
x 10
1.5
2

1.5
1

0.5
0.5

0 0

Euclidean distance similarity NCC similarity measure

measure
 6
x 10
60 60
11

10 1.2
50 50
9
1
8
40 40
7
0.8

30 6 30
5 0.6

20 4 20
0.4
3
10 10
2
0.2
1
10 20 30 40 50 60 70 10 20 30 40 50 60 70

Euclidean distance similarity NCC similarity measure

measure
Image Processing for MRI
 Patient -> Image -> Beyond

•signal reception
•FFT
•indices, coordinates
•File formats
•DICOM – 3D positions
•window levels
 16 bits are not enough precision
• ~16 bit analogue to digital converter
– integers 0 – 65535.
• True dynamic range is higher (variable
attenuators in receiver chain).
• Complex FFT and other processing.
• Use “intensity” image (floating point values),
rather than indexed or rgb.
– avoid read/writing intermediate files in tiff, gif.
FAST Fourier Transform Convention
 FFT on 2D k-space

k-space FFT
 FFT Recipe

1 shift

2 FFT

3 unshift
 FFT other tips
• 1/N scaling between forward and reverse.
• Lack of convention for forward/reverse.
• Don’t 3D FFT multi-slice data.
• Beware of indices and the location of DC.
0

N/2 + 1

N/2
Vertical axis reflection

Care mapping:
• spatial
transformations
• k-space
frequencies
 2D data in memory and files
2D data stored sequentially in 1D memory/file

MATLAB matrix(row, column)

IDL matrix(column, row)
C array [ index1 ] [ index2 ]

index is written out most rapidly

Symptoms: rotated (transposed) images.

 File Formats
• Files contain: Information and pixel values.
• Information in either:
– fixed length header (gipl)
– fixed length separate file (Analyze)
– tags (ACR/NEMA, tiff, DICOM, SPI)
 Endian (not indian)
Order in which multiple byte variables are stored

Big endian Little endian

• “big end first” • “little end first”
• Unix • Intel (Windows and
• Motorola Mac Linux on a PC)
 Endian (cont)

Problems reading file previously written on a

different system.
Most standard image file types and readers cope.
MATLAB – specify within fopen
IDL - byte swapping commands.
Do not just swap every pair of bytes blindly.
 3D Information in DICOM
right-handed coordinate system

ImagePositionPatient +z Head
(x,y,z) r
centre of top-left pixel

+y Posterior
ImageOrientationPatient
(rx, ry, rz, cx, cy, cz) c
image row and column directions
+x Patient Left
 Example Example

•restrict FOV
•surf command
•vertices specified
•no edge color

!! relies on radiographer correctly entering

supine/prone, head-foot !!
 Decoding a DICOM directory

• Thousands of files often with meaningless names.

– Set up DICOM node to use sensible file names.
– Examine the slice separations in 3D space using
ImagePositionPatient.
– for cardiac phases, use TriggerDelay?

• Future standard: Multi-frame DICOM objects for

MR. Also allow annotation for tractography, fMRI
etc.
 Coordinates

}?
(0,0)
(1,1)
(0.5,0.5)

Tip: generate a 4x4 image, plot a point.

To find rotation centres, create a cross test object.
Image Presentation

same image
 Window levels
•Alarming differences in gamma
• CRT, LCD
• Sun, Mac, PC.
• printer, toner level.
• projector, room brightness.
•Fix lower range at 0, adjust upper based on
inspection (e.g. exclude fat). Guess gamma.
Present with a colour bar.
•Future: support for .png gamma correction.
 Misc.
• In subtraction images, beware of negative values
wrapping round the integer range.
• signed vs. unsigned integers
• MR data is complex: transpose, complex conjugate
and Hermitian transpose are not the same.
• http://www.itk.org
– software
– test data (BrainWeb, Visible Human)
– software links (Image Magick, MRIConvert).
 misc. (cont)

• vtk good for surface visualisation.

• Data is discrete, not continuous.
• Voxels usually anisotropic.
• Care: voxel sizes (distortion) and resolution
(reduced phase encodes).
• sinc interpolation is best on bandlimited
data – but you may need the raw data, not
a truncated image.
Computer Tomography using FFT
 CT Scanners
• In 1901 W.C. Roentgen won the Nobel Prize (1st in physics)
for his discovery of X-rays.

Wilhelm Conrad Röntgen

 CT Scanners
• In 1979 G. Hounsfield & A. Cormack, won the Nobel Prize for
developing the computer tomography.
• The invention revolutionized medical imaging.

Allan M. Cormack

Godfrey N. Hounsfield
Tomography: Reconstruction from Projection

1

f(x,y)
2
 Projection & Sinogram

• Projection: All ray-sums in a direction

• Sinogram: collects all projections

y
P(t)
t
p
 x

f(x,y)
t
X-rays Sinogram
 CT Image & Its Sinogram

K. Thomenius & B. Roysam

 The Slice Theorem
Fourier
y Transform v

1 1
x u
f(x,y)

spatial domain frequency domain

 The Slice Theorem
f(x,y) = object
g(x) = projection of f(x,y) parallel to the y-axis: g(x) = f(x,y)dy

Fourier Transform of f(x,y): F(u,v) =   f(x,y) e -2pi(ux+vy)

dxdy

Fourier Transform at v=0 : F(u,0) =   f(x,y) e -2piux

dxdy

= [  f(x,y)dy] e -2piux
dx

=  g(x) e -2piux dx = G(u)

The 1D Fourier Transform of g(x)

 Interpolation Method
• Interpolate (linear, quadratic etc) in the frequency space to
obtain all values in F(u,v).
• Perform Inverse Fourier Transform to obtain the image f(x,y).

v
F(u,v)

u
THANK YOU!

Digital Image Processing