05 - Spatial Filtering

Image & Video Processing
Image Enhancement
(Spatial Filtering)
2 Contents
In this lecture we will look at spatial filtering
techniques:
– Neighbourhood operations
– Spatial filtering : Correlation and convolution
– Smoothing operations
– Sharpening filters
• 1st derivative filters
• 2nd derivative filters
– Combining filtering techniques
Image Processing and Computer Vision by Dr. Praveen Kumar @ CSE, VNIT
3 Neighbourhood Operations
Neighbourhood operations simply operate on a larger
neighbourhood of pixels than point operations
Origin x
Main idea
– Define a neighborhood of a pixel
– Define an operation on pixels in the
neighborhood (x, y)
Neighbourhood
– Output pixel value = result of the
operation
y Image f (x, y)
5 The Spatial Filtering Process
Origin x
a b c r s t
d e f u v w
g h i * x y z
Original Image Filter
Simple 3*3 e 3*3 Filter Pixels
Neighbourhood
eprocessed = v*e +
r*a + s*b + t*c +
y Image f (x, y) u*d + w*f +
x*g + y*h + z*i
This is repeated for every pixel in the original image to generate the filtered image
6 Linear Spatial Filtering
• Create a filter or mask w, size m x n
• Apply to image f, size M x N
• Compute the sum of products of mask coefficients with corresponding
pixels under mask
• Slide mask over image, apply at each point
m/ 2 n/ 2
g(x, y) =   w(s, t) f (x + s, y + t)
s=−m/ 2 t =−n/ 2
Also called “cross-correlation”
= w(x, y)  f (x, y)
• It is a linear operation
w(x, y)  ( a f (x, y)) = a w(x, y)  f (x, y)
w(x, y) ( f (x, y) + g(x, y)) = w(x, y)  f (x, y) + w(x, y)  g(x, y)
8 Smoothing Spatial Filters
One of the simplest spatial filtering operations we can
perform is a smoothing operation
– Simply average all of the pixels in a neighbourhood
around a central value
– Especially useful 1/
9
1/
9
1/
9
in removing noise
from images 1/
9
1/
9
1/
9 Simple
– Also useful for 1/ 1/ 1/
averaging
highlighting gross
9 9 9
‘box’ filter
detail
9 Averaging in 1D
Replace each pixel with an average of all the values
in its neighborhood – a moving average:
10 Averaging in 2D
11 What Happens At The Edges!
At the edges of an image we are missing
pixels to form a neighbourhood
Origin x
e e
e
e
e e e
y Image f (x, y)
12 What Happens At The Edges! (cont…)
There are a few approaches to dealing with missing edge
pixels:
– Omit missing pixels
• Can add extra code and slow down processing
– Pad the image
• Typically with either all white or all black pixels
– Replicate border pixels
– Truncate the image
– Allow pixels wrap around the image
• Can cause some strange image artefacts
13
What Happens At The Edges!
Averaged Image:
Zero Padding
Original Averaged Image:

Image Replicate Edge Pixels
Averaged Image:
Wrap Around Edge Pixels
14 Image Smoothing Example
The image at the top left
is an original image of
size 500*500 pixels
The subsequent images
show the image after
filtering with an averaging
filter of increasing sizes
– 3, 5, 9, 15 and 35
Notice how detail begins
to disappear
15 Another Smoothing Example
By smoothing the original image we get rid of lots
of the finer detail which leaves only the gross
features for thresholding
Original Image Smoothed Image Thresholded Image
16 Averaging as noise removal
Why Averaging
would help to
remove noise?
Original Image Image After

With Noise Averaging Filter
Assumptions:
1. The true value of the pixel is similar to the true value of the
nearby pixels
2. Noise is random and independent
17 Quiz
Q: If noise is just a function added to an image, we
could remove the noise by subtracting the noise
again – that is the operation is reversible?
a) True
b) True but we don’t know the noise function so we
can’t actually do the subtraction.
c) False. Additive noise destroys information in the
image and there is no way to recover it.
18 Weighted Smoothing Filters
More effective smoothing filters can be generated by
allowing different pixels in the neighbourhood different
weights in the averaging function
– Pixels closer to the 1/
16
2/
16
1/
16
central pixel are more 2/ 4/ 2/

16 16 16
important
– Often referred to as a 1/
16
2/
16
1/
16
weighted averaging
Weighted
averaging filter
19 Gaussian – an important smoothing filter
Notes:
• 1D
1
0.9
0.8  is the standard
2 0.7
deviation
h(x) = 1 e − 2x 2 0.6
0.5
2
0.4 Values should sum to 1
0.3
0.2 Mask size should be at
least ±3
0.1
0
-10 -8 -6 -4 -2 0 2 4 6 8 10
• 2D − x +y2
2 2
h(x, y) = 1 e 2 0.01
2 2 0.008
0.006
0.004
0.002
0 30 40
40 20 30
20 Smoothing with a Gaussian
21 Smoothing with not a Gaussian
22 Gaussian filters
Variance (𝝈𝟐) or standard deviation (𝝈) – determines
extent of smoothing
σ = 2 with 30 x 30 kernel σ = 5 with 30 x 30 kernel

Note: Size of kernel or mask is not necessarily variance
23 Matlab
>> fsize = 31;
>> sigma = 5;
>> h = fspecial(‘gaussian’, fsize, sigma);
>> surf(h);
>> imagesc(h);
im outim
>> outim = imfilter(im, h);
>> imshow(outim);
24 Varying Sigma
for sigma=1:3:10
h = fspecial('gaussian’, fsize, sigma);
out = imfilter(im, h);
imshow(out); pause;
end
25 Quiz
When filtering with a Gaussian, which is true:
A.The sigma is most important – it defines the blur
kernel’s scale with respect to the image.
B.The kernel size is most important – because it
defines the scale.
C.Altering the normalization coefficient does not
effect the blur, only the brightness.
D. A and C.
26
Filters as Templates
• Another use of filters can be as templates of certain
“interesting” regions in an image (like some object,
shape etc)
• By doing a spatial correlation of the filter/template
with the image one can localize/detect those
“interesting” areas in an image by looking at strong
responses of the filter at various locations in the
image
27 Vector representation of correlation
• Correlation is a sum of m/ 2 n/ 2
products of c(x, y) =   w(s,t) f (x + s, y + t)

s=−m/ 2 t =−n/ 2
corresponding terms
= w(x, y)  f (x, y)
• We can think of
correlation as a dot
product of vectors w c = w1 f1 + w2 f 2 + +w f
mn mn
and f mn
=  wk f k = w T f
• If images w and f are k=1
similar, i.e vectors are
aligned, then the
correlation score c is high
28 Template matching - A toy example
Template (mask)
Scene Correlation map
29 Weaknesses of correlation
Suppose we correlate a 1D filter:
With the 1D signal:
We get the result:
What is the problem?
30 Solution
• SSD – take sum of squared difference
• Normalized cross correlation

wT f
c= where w = w12 + w2 + w 3 ++ w N
2 2 2
w f
• result is between 0..1 (for positive valued images)

• can get better precision by subtracting off means, then
result is between -1 .. +1
31 Normalized cross correlation
c = normxcorr2(onion,peppers);
figure, surf(c), shading flat;
32 Line Detection
How can we detect straight lines in a binary image ?
Following masks can extract lines that are one pixel

thick and in a particular direction
33 Line Detection (cont…)
Binary image of a wire
bond mask
After Result of
processing thresholding
with -45° line filtering result
detector
34 Template matching
What if the template is not identical to

some sub-image in the scene?
Template
Scene
35 Template matching
Match can be meaningful, if general

appearance, scale and orientation is right
Template
Scene
36 Quiz
Which of the following case would template matching work?
37 Correlation and Convolution
• Correlation
m/2 n/2
g(x, y) =  
s=−m/ 2 t=−n/ 2
w(s, t) f (x + s, y + t)
= w(x, y)  f (x, y)
• Convolution is very similar
• Convolution to correlation, except that
m/ 2 n/ 2 we flip one of the functions
g(x, y) =   w(s,t) f (x − s, y − t)
• For symmetric filters it
s=−m/ 2 t=−n/ 2
= w(x, y)  f (x, y) makes no difference

– Note that • Also a linear operation
m/2 n/2 m/2 n/2
 
s=−m/ 2 t=−n/ 2
w(s,t) f (x − s, y − t) =  
s=−m/2 t=−n/ 2
w(−s, −t) f (x + s, y + t)
– So to implement convolution we can reflect w(x,y) and then do correlation

38 Properties
• Convolution is
– Commutative
f*g=g*f
– Associative
f * (g * h) = (f * g) * h
– Distributive
f * (g + h) = f * g + f * h
• Correlation is distributive but not commutative or associative

• Convolution of a filter h(x,y) with an impulse (x,y) yields h(x,y)
39 Convolving with two Gaussians
• You can convolve a function with a Gaussian, then convolve again
with a Gaussian. The effect is the same as convolving with a single,
larger Gaussian
• Why? Since convolution is associative
g1 ( g2  f ) = (g1  g2 ) f
− x2 − x2
• and if
g1 = 1 e 212
, g2 = 1 e 2 22
2 1 2 2 i.e., it is a Gaussian
•
2
then − x with standard
e ( )
1  12+ 22
g1  g 2 =
2 deviation
2 ( +  2
1
2
2 )  new = 12 +  22
40 Computational Complexity
• If an image is NxN and a kernel (filter) is WxW,
how many multiplies do you need to compute a
convolution?
• You need O(N*N*W*W) = O(N2W2 )
• This can get huge if W is big enough
41 Separable kernels (filter masks)
• If a 2D convolution kernel can be separated into two 1D kernels,
the operation is much faster
– If h(x, y) = h1(x)h2 ( y)
– then h(x, y)  f (x, y) = h1 (x)  (h2 ( y)  f (x, y) )
• Namely, do a 1D correlation with h2(y) along the individual columns
of the input image, then do a 1D correlation with h1(x) along the
rows of the result from the previous step
• If h(x,y) is nxn, and f(x,y) is NxN
– Cost of a 2D correlation?
– Cost of two 1D correlations?
42 Non Linear Filters
Spatial filters that are based on ordering of the pixel

values that make up the neighbourhood operated
on by the filter
Useful spatial filters include
– Max and min filter
– Median filter
– Adaptive median filter
43 Max and Min Filter
Max Filter:
fˆ ( x, y) = max {g ( s, t )}
( s ,t )S xy
Min Filter:
fˆ ( x, y ) = min {g ( s, t )}
( s ,t )S xy
Max filter is good for pepper noise and min is

good for salt noise
44 Noise Removal Examples (cont…)
Image Image
Corrupted Corrupted
By Pepper By Salt
Noise Noise
Result Of Result Of
Filtering Filtering
Above Above
With A 3*3 With A 3*3
Max Filter Min Filter
45 Median Filter
Median Filter:
fˆ ( x, y) = median{g ( s, t )}
( s ,t )S xy
Excellent at removing salt and pepper noise without much

modifying the original pixel values as other smoothing filters
46 Median Filter
Median Filter:
– Doesn’t blur sharp edges in the image
12 12
10
10
8
8
6
6
4
2
2
0
-2 0
0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50
median filtered, filter width=7
47 Averaging Filter Vs. Median Filter Example
Original Image Image After Image After

With Noise Averaging Filter Median Filter
48 Noise Removal Examples
49 Noise Removal Examples
50 Noise Removal Examples (contd)
Image Result of 1
Corrupted Pass With A
By Salt And 3*3 Median
Pepper Noise Filter
Result of 2 Result of 3
Passes With Passes With
A 3*3 Median A 3*3 Median
Filter Filter
51 Adaptive Median Filtering
• The median filter performs relatively well on
impulse noise as long as the spatial density of the
impulse noise is not large
• The adaptive median filter can handle much more
spatially dense impulse noise with less distortion
• The key insight in the adaptive median filter is that
the filter size changes depending on the
characteristics of the image region
52 Adaptive Median Filtering (cont…)
The adaptive median filter has following purposes:
– Remove spatially dense impulse noise
– Reduce distortion
First examine the following notation:
– zmin = minimum grey level in Sxy
– zmax = maximum grey level in Sxy
– zmed = median of grey levels in Sxy
– zxy = grey level at coordinates (x, y)
– Smax =maximum allowed size of Sxy
53 Adaptive Median Filtering (cont…)
Level A: A1 = zmed – zmin
A2 = zmed – zmax
If A1 > 0 and A2 < 0, Go to level B
Else increase the window size
If window size ≤ Smax repeat level A
Else output zmed
Level B: B1 = zxy – zmin
B2 = zxy – zmax
If B1 > 0 and B2 < 0, output zxy
Else output zmed
54 Simple Adaptive Median Filtering Example
Using only
level B and
filter 7 x 7
55 Adaptive Median Filtering Examples
Image corrupted by salt Result of filtering with a 7 Result of adaptive median

and pepper noise with * 7 median filter filtering with Smax = 7
probabilities Pa = Pb=0.25
56 Image sharpening
How about sharpening the image to highlight fine detail
– Remove blurring from images
– Highlight discontinuities (edges etc.)
How can you accomplish Sharpening by spatial filters?
57 Unsharp masking & Highboost filtering
58 Spatial Differentiation
• Image differentiation can help enhance edges
and other discontinuities (such as noise)
• Differentiation measures the rate of change of a
function.
• Thus, the strength of a derivative operator at any
location should be proportional to the degree of
discontinuity of the image at that location.
59 Spatial Differentiation
For 2D function, f(x,y), the partial derivative is defined as:
 f ( x , y) f (x +,y) − f(x ,y )
= lim
x  →0

For discrete data, we can approximate using finite
differences:
f (x, y)
 f (x +1, y) − f (x, y)
• Right difference x
f (x, y)
 f (x , y) − f (x-1, y)
• Left difference x
f
• Central difference  ( f (x +1) − f (x −1)) / 2
x
60 2nd Derivative
The formula for the 2nd derivative of a function is as
follows:
2 f
= f ( x + 1, y ) + f ( x − 1, y ) − 2 f ( x, y )
 x
2
61 1D example
Let’s consider a
simple 1
dimensional
example
63 The Laplacian
The first sharpening filter we will look at is the Laplacian
which is defined as follows:  2
f  2
f
2
 f = +
 x2
2 y
where the partial 2nd order derivative in the x direction is
defined as follows:
2 f
= f ( x + 1, y ) + f ( x − 1, y ) − 2 f ( x, y )
 x
2
and in the y direction as follows:

2 f
= f ( x, y + 1) + f ( x, y − 1) − 2 f ( x, y )
 y Image Processing and Computer Vision by Dr. Praveen Kumar @ CSE, VNIT
2
64 The Laplacian as a Filter
So, the Laplacian can be given as follows:
2 f = [ f ( x + 1, y) + f ( x − 1, y)
+ f ( x, y + 1) + f ( x, y − 1)]
− 4 f ( x, y)
We can easily build a filter based on this
0 1 0
1 -4 1
0 1 0
65 Variants On The Simple Laplacian
There are lots of slightly

different versions of the
Laplacian that can be
used:
66 The Laplacian (cont…)
Applying the Laplacian to an image we get a new
image that highlights edges and other discontinuities
Original Laplacian Laplacian

Image Filtered Image Filtered Image
Scaled for Display
67 But That Is Not Very Enhanced!
We have to do more work in order to
get our final image
Subtract the Laplacian result from the
original image to generate our final
sharpened enhanced image
Laplacian
g ( x, y) = f ( x, y) −  f 2
Filtered Image
Scaled for Display
Q: Why Subtract and not add?

68 Laplacian Image Enhancement
- =
Original Laplacian Sharpened
Image Filtered Image Image
Important to keep in mind which variant of Laplacian

is used to decide whether to add or subtract
69 Laplacian Image Enhancement
70 Simplified Image Enhancement
The entire enhancement can be combined into a
single filtering operation
g ( x, y) = f ( x, y) −  f 2
= f ( x, y) − [ f ( x + 1, y) + f ( x − 1, y)
+ f ( x, y + 1) + f ( x, y − 1)
− 4 f ( x, y)]
= 5 f ( x, y) − f ( x + 1, y) − f ( x − 1, y)
− f ( x, y + 1) − f ( x, y − 1)
71 Simplified Image Enhancement (cont…)
This gives us a new filter which does the whole job for
us in one step
0 -1 0
-1 5 -1
0 -1 0
72 Simplified Image Enhancement (cont…)
73 Summary
• We discussed various spatial domain image
enhancement techniques for
- contrast stretching
- removal of different types of noise
- Image smoothing and sharpening
• We also discussed using spatial filters as templates
for detecting objects and interesting shapes
• Next we will discuss methods for detecting edges
for image representation

05 - Spatial Filtering

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

05 - Spatial Filtering

Uploaded by

Copyright:

Available Formats

Image & Video Processing

Original Averaged Image:

Original Image Smoothed Image Thresholded Image

Original Image Image After

central pixel are more 2/ 4/ 2/

σ = 2 with 30 x 30 kernel σ = 5 with 30 x 30 kernel

products of c(x, y) =   w(s,t) f (x + s, y + t)

Scene Correlation map

With the 1D signal:

We get the result:

What is the problem?

• Normalized cross correlation

• result is between 0..1 (for positive valued images)

Following masks can extract lines that are one pixel

What if the template is not identical to

Match can be meaningful, if general

= w(x, y)  f (x, y) makes no difference

– So to implement convolution we can reflect w(x,y) and then do correlation

• Correlation is distributive but not commutative or associative

Spatial filters that are based on ordering of the pixel

Max filter is good for pepper noise and min is

Excellent at removing salt and pepper noise without much

median filtered, filter width=7

Original Image Image After Image After

Image corrupted by salt Result of filtering with a 7 Result of adaptive median

and in the y direction as follows:

There are lots of slightly

Original Laplacian Laplacian

Q: Why Subtract and not add?

Important to keep in mind which variant of Laplacian

You might also like