IT5409 Ch3 Filters 20200228 v3 Part1

Computer Vision
Chapter 3: Image Processing

Computer Vision
Chapter 3. Image Processing
2
Content
• Remind: digital image representation
• Point Operators
• Image Filtering
3
Digital images ?
• What we can see on the picture?

‒ A car?
• What does the machine see?
‒ Image is a matrix of pixels
‒ Image N x M : N xM matrix
‒ 1 pixel (gray levels):
• A intensity value: 0-255
• Black: 0
• White: 255
4
Digital images ?
• For an image I x
‒ Index (0,0): Top left corner

‒ I(x,y): intensity of pixel at
the position (x,y) y
5
Digital images ?
• Principal type of images

• Binary image:
- I(x,y) Î {0 , 1}
- 1 pixel: 1 bit
• Gray image:
- I(x,y) Î [0..255]
- 1 pixel: 8 bits (1 byte)
• Color image
- IR(x,y), IG(x,y) IB(x,y) Î [0..255]
- 1 pixel: 24 bits (3 bytes )
• Other : multi-spectre, depth image,…
6
Color image in RGB space
It exists other color spaces:

Lab, HSV, …
7
Images are sampled and quantized
• An image contains discrete number of pixels

• A simple example 75
• Pixel value:
‒ “grayscale”
(or “intensity”): [0,255]
231
148
8
Images are sampled and quantized
• An image contains discrete number of pixels

• A simple example
• Pixel value: [90, 0, 53]
‒ “grayscale”
(or “intensity”): [0,255]
‒ “color”
[249,
• RGB: [R, G, B] 215,
• Lab: [L, a, b] 203]
• HSV: [H, S, V] [213, 60, 67]
9
Modification of images values
• Isolated transformations (point operators/point processing) of the

pixels in an image
‒ Read the value of a pixel ® replace it by another
‒ Ex.: contrast enhancement, histogram egalization
• There are also local transformations
‒ Read the values of few neighboring pixels ® compute a new value for a pixel
‒ Convolutions, correlations, …
• …and global transformations
‒ Read the values of all pixels in the image ® compute a new value for one pixel
‒ FFT, …
10
Pixel transformation
Source : Caroline Rougier. Traitement d'images (IFT2730). Univ. de Montréal.

11
Content
• Rappel: digital image representation
• Point Processing
‒ Point operators
‒ Histogram and histogram egalization
• Image Filtering
12
Point processing
• The output at one point depends only on that point
but not other neighborhoods
s = T(r)
§ r = f(x, y)
§ s = g(x, y)
§ T: transformation function
13
Homogenous transformation
s=T(r)
light
The output is
T(r) identical as the
input : s = r
m
r
Dark
m
Dark Light
14
Linear / Non-linear transformation
s=T(r) s=T(r)
light
light
T(r) T(r)
n n
r Dark r
Dark
m m
Dark Light Dark Light
15
Contrast stretching
s=T(r)
Light
T(r) • The values of r below m are

compressed by T into a narrow
range of s, toward black.
m
• The opposite effect takes place for
values of r above m.
r
Dark
m
Dark Light
16
Thresholding function
s=T(r)
1
T(r)
Thresholding function T(r)
outputs the image as binary
image:
• s = 0 if r <m
• s = 1 if r >= m
0 r
m
17
Some Basic Gray Level Transformations
• Image Negatives
• Log Transformations
• Power-Law Transformations
• Piecewise-Linear Transformation Functions
• Histogram Processing
18
Image Negatives
• The negative of an image with gray levels in the range
[0, L-1] is obtained by s = L - 1 – r
L-1
3L/4
Ouput gray level, s
L/2 Negative
L/4
Input image Negative of the

0 L/4 L/2 3L/4 L-1
input image
Input gray level, r
19
Log Transformations
• The general form of the log transformation:

s = c ´ log(1 + r)
c: constant
L-1
r³0
3L/4
Output gray leve, s
Log
L/2
L/4 Inverse
log
0 L/4 L/2 3L/4 L-1
Input gray level r

20
Log Transformations
• This transformation maps
‒ a narrow range of low gray-level values in the input
image into a wider range of output levels.
‒ The opposite is true of higher values of input levels.
• Use this Log to expand the values of dark pixels in

an image while compressing the higher-level
values.
21
Inverse Log Transformations
• This transformation maps
‒ a wide range of low gray-level values in the input image
into a more narrow range of output levels.
‒ The opposite is true of higher values of input levels.
• Use this Inverse Log if you want to compress the
values of dark pixels in an image while expanding
the higher-level values.
22
Power-Law (Gamma) Transformations
• The general form of power-law
transformation is:
𝑠 = 𝑐. 𝑟 g
‒ g > 1: compress values in dark
area, while expanding values in
light area
‒ g < 1 : expand values in dark
area, while compressing values
in light area
‒ r : normalized values to [0, 1]
‒ c : scaling constant c
corresponding to the bit size
used
23
Power-Law (Gamma) Transformations
• γ1 = 1; γ1 = 3; γ2 = 4; γ3 = 5;
Increasing
the dynamic
ranges of
high
intensities
24
Gama correction
In Practice with grey level image :

k = 255
n = r/255
25
Gama correction
Image: Wikipedia
26
Piecewise-Linear Transformation
• Contrast stretching:
L-1
‒ the idea behind contrast
stretching is to increase the
(r2, s2)
dynamic range of the gray 3L/4
levels in the image being
processed T(r)
L/2
• Output:
‒ Output values of input ones from (0,
0) to (r1, s1) decrease L/4
‒ Output values of input ones from (r1, s1)
(r1, s1) đến (r2, s2) increase.
0 L/4 L/2 3L/4 L-1
27
L-
L-1
1
• If r1 = s1, r2 = s2, 3L/4/

4
‒ the function becomes L/
L/2
2 T(r)
Homogenous transformation (r2, s2)
L/ s2) (r1, s1)
L/4
4 s1)
Cấp xám
0 L/4L/ L/2
L/
đầu vào3L
3L/4 L-1
L-
4 2 /4 1
L-1
• If s1 = 0, s2 = L-1, (r2, s2)
3L/4
‒ the function become
thresholding function T(r)
L/2
‒ Output is a binary image
L/4
(r1, s1)
0 L/4 L/2 3L/4 L-1
28
(r2, s2)
255
• If (r1, s1) = (rmin, 0) ;

(r2, s2) = (rmax, 255) T(r)
‒ Linear stretching
0 (r1, s1)
0 rmi rsmin rmax
• If (r1, s1) = (rsmin, 0), rsmin > rmin 255
n
(r2, s2)
(r2, s2) = (rsmax, 255) rsmax < rmax
‒ Linear stretching with saturation
T(r)
0 (r1, s1)
0 rmin rsmin rsmax rmax
29
Original image, low Contrast sketching Thresholding

contrast (r1, s1) = (rmin, 0) (r1, s1) = (m, 0)
(r2, s2) = (rmax, L-1) (r2, s2) = (m, L-1)
m: mean of intensities
30
Image histogram
• What is histogram?
‒ Histogram of an image with gray level range [0, L-1] is a
discrete function: ℎ 𝑟! = 𝑛!
• 𝑟! is kth gray level
• 𝑛! : number of pixels that have level 𝑟!
‒ Histogram is a graphical representation of the repartition
of colours among the pixels of a digital image
31
Image histogram
• Normalized histogram?
‒ The histogram should be normalized by dividing all elements
"
to total number of pixels in the image (n) : ℎ 𝑟! = "
"
Image dynamic range = [min intensity, max intensity]
32
Image histogram
• Properties?
‒ Only statistic information
‒ No indication about the location of pixel (no spatial
information)
‒ Different images can have the same histogram
33
Image Brightness
• Brightness of a grayscale image is the average intensity
of all pixels in an image
‒ refers to the overall lightness or darkness of the image
34
Contrast
• The contrast of a grayscale image indicate how easily
object in the image can be distinguished
• Many different equations for contrast exist
‒ Standard deviation of intensity values of pixels in the image
& (
1 )
𝐶= ( ( 𝐼 𝑢, 𝑣 − 𝑚𝑒𝑎𝑛
𝑤ℎ
#$% '$%
‒ Difference between intensity value maximum et minimum
max 𝐼(𝑢, 𝑣) − min(𝐼(𝑢, 𝑣))

𝐶=
max 𝐼(𝑢, 𝑣) + min(𝐼(𝑢, 𝑣))
35
Contrast
¡ Contrast vs histogram
36
Histogram and Brightness
Dark image
Light image
37
Histogram and Contrast
Low- contrast
image
High- contrast
image
38
Histogram
• If we can somehow equalize the histogram of an
image à we can make it higher contrast.
39
Histogram Equalization
40
Histogram Equalization
41
Discrete Implementation
42
Histogram equalization
Ex. 1. Input image I has 10 levels.
Equalize the histogram of I # pixels
2 3 3 2 6
4 2 4 3 5 Histogram
3 2 3 5 4
2 4 2 4 3
2
k 0 1 2 3 4 5 6 7 8 9
1
nk 0 0 6 5 4 1 0 0 0 0 Gray level
0 1 2 3 4 5 6 7 8 9
43
k 0 1 2 3 4 5 6 7 8 9
nk 0 0 6 5 4 1 0 0 0 0
! 𝑛! 0 0 6 11 15 16 16 16 16 16
!"#
𝑇(𝑘) 0 0 6/16 11/16 15/16 16/16 16/16 16/16 16/16 16/16
𝑠! = 9𝑇(𝑘) 3.3 6.1 8.4

0 0 9 9 9 9 9
»3 »6 »8
44
k sk 2 3 3 2 # pixcels
0 0 6
4 2 4 3
1 0 5
3 2 3 5
2 3 4
2 4 2 4
3 6
3
4 8
2
5 9
3 6 6 3
6 9 1 Gray
8 3 8 6 level
7 9
8 9 6 3 6 9 0 1 2 3 4 5 6 7 8 9
9 9 3 8 3 8 Histogram after equalization
45
46
If we take the same image with different contrasts, histogram equalization
will give the same results for all images
47
• Discussion
‒ Compare between histogram equalization and
Piecewise-Linear Transformation Functions
‒ Is histogram equalization always good?
48
• Change histogram of modified image into uniform distribution
• No parameters. OpenCV: cv2.equalizeHist(img)
49
52
53
Histogram matching/specification
• In some cases: uniform histogram does not have a
best result
àSpecify a good histogram
àTransform the image to have the specific histogram
So, we have histogram matching or histogram

specification problem
54
Histogram matching
Desired histogram
è Additional lecture
55
Contrast enhancement for color
images ?
• Histogram egalization of color images ?
‒ Apply separately the procedure to each of the three
channels
• will yield in a dramatic change in the color balance
• NOT RECOMMENED unless you have a good reason for
doing that.
‒ Recommended method
• Convert the image into another color space such as the
Lab color space or HSL/HSV color space
• Apply the histogram equalization on the Luminance
channel
• Convert back into the RGB color space
63
Content
• Rappel: digital image representation
• Point Processing
• Image filtering
64
Image filtering
Filtering:
– Forming a new image whose pixel values are
transformed from original pixel values
Goals:
• Goal is to extract useful information from
images, or transform images into another
domain where we can modify/enhance image
properties
• Features (edges, corners, blobs…)
• super-resolution; in-painting; de-noising
65
Super-resolution
De-noising
In-painting
Bertamio et al
66
Image filtering
• Image filters in spatial domain

– Filter is a mathematical operation of a grid of numbers
– Smoothing, sharpening, measuring texture
• Image filters in the frequency domain

– Filtering is a way to modify the frequencies of images
– Denoising, sampling, image compression
67James Hays
Image filtering
• Image filtering:
‒ Compute function of local neighborhood at each
position
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l
68James Hays
Image filtering
position
h=output f=filter I=image
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l
2d coords=k,l 2d coords=m,n
[ ] [ ]
[ ]
69
Example: box filter
f [× ,× ]
1 1 1
1 1 1
1 1 1
Slide credit: David Lowe

70 (UBC)
Image filtering 1 1 1
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0
0 0
0 0
0 90
90 0
0 90
90 90
90 90
90 0
0 0
0
0
0 0
0 0
0 90
90 90
90 90
90 90
90 90
90 0
0 0
0
0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0
0
0 0
0 90
90 0
0 0
0 0
0 0
0 0
0 0
0 0
0
0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
71S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
72S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
73S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
74S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
75S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
?
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
76S. Seitz
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
?
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0 50
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
77S. Seitz
Image filtering f [× ,× ] 1 1 1
1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30
0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
78S. Seitz
Box Filter
f [× ,× ]
What does it do?
• Replaces each pixel with 1 1 1
an average of its
neighborhood 1 1 1
1 1 1
• Achieve smoothing effect
(remove sharp features)
Slide credit: David Lowe (UBC)

79
Box Filter
f [× ,× ]
What does it do?
• Replaces each pixel with 1 1 1
an average of its
neighborhood 1 1 1
1 1 1
• Achieve smoothing effect
(remove sharp features)
• Why does it sum to one?
Slide credit: David Lowe (UBC)

80
Smoothing with box filter
James Hays
81
Image filtering
position
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l
• Really important!
‒ Enhance images
• Denoise, resize, increase contrast, etc.
‒ Extract information from images
• Texture, edges, distinctive points, etc.
‒ Detect patterns
• Template matching
82James Hays
Think-Pair-Share time
0 0 0
1. 0 1 0
0 0 0
0 0 0
2. 0 0 1
0 0 0
1 0 -1
3. 2 0 -2
1 0 -1
0 0 0 1 1 1
4. 0
0
2
0
0
0
- 1
1
1
1
1
1
83
1. Practice with linear filters
0 0 0
0 1 0 ?
0 0 0
Original
84 Lowe
Source: D.
0 0 0
0 1 0
0 0 0
Original Filtered
(no change)
85 Lowe
Source: D.
0 0 0
0 0 1 ?
0 0 0
Original
86 Lowe
Source: D.
0 0 0
0 0 1
0 0 0
Original Shifted left

By 1 pixel
87 Lowe
Source: D.
1 0 -1
2 0 -2
1 0 -1
Sobel
Vertical Edge
(absolute value)
88David Lowe
1 2 1
0 0 0
-1 -2 -1
Sobel
Horizontal Edge
(absolute value)
89David Lowe
0 0 0 1 1 1
0
0
2
0
0
0
- 1
1
1
1
1
1
?
(Note that filter sums to 1)
Original
90 Lowe
Source: D.
0 0 0 1 1 1
0
0
2
0
0
0
- 1
1
1
1
1
1
Original
Sharpening filter
- Accentuates differences with local
average
91 Lowe
Source: D.
92 Lowe
Source: D.
Correlation and Convolution
• 2d correlation
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l
h=filter2(f,I); or h=imfilter(I,f);
• 2d convolution
h[m, n] = å f [k , l ] I [m - k , n - l ]
k ,l
h=conv2(f,I); or h=imfilter(I,f,’conv’);
Correlation and convolution are identical when the filter is symmetric.
93James Hays
1D Discrete convolution (symbol: )
We are going to convolve a function f with a filter h.
h[k,l] f[k,l]
94
1D Discrete convolution (symbol: )
We first need to calculate h[n-k, m-l]

h[-k]
h[k] h[n-k]
95
Discrete convolution (symbol: )
h[-2-k]
f[k] g[-2]
96
h[-1-k]
f[k] g[-1]
97
h[-k]
f[k] g[0]
98
h[1-k]
f[k] g[1]
99
h[2-k]
f[k] g[2]
100
h[n-k]
f[k] g[n]
101
2D convolution
2D convolution is very similar to 1D.
• The main difference is that we now have to iterate over 2 axis instead of 1.
f [n, m] ⇤ h[n, m]
Assume we have a filter(h[,]) that

is 3x3. and an image (f[,]) that is
7x7.
102
2D convolution
f [n, m] ⇤ h[n, m]

7x7.
103
2D convolution
f [n, m] ⇤ h[n, m]

7x7.
104
2D convolution
f [n, m] ⇤ h[n, m]

7x7.
105
2D convolution
f [n, m] ⇤ h[n, m]

7x7.
106
2D convolution
f [n, m] ⇤ h[n, m]

7x7.
107
2D convolution example
Slide credit: Song Ho Ahn
108
109
110
111
112
113
114
Convolution in 2D - examples
•0 •0 •0
* •0 •1 •0
= ?
•0 •0 •0
Original
Courtesy of D Lowe
115
•0 •0 •0
* •0 •1 •0
•0 •0 •0
=
Original Filtered
Courtesy of D Lowe
(no change)
116
•0 •0 •0
* •0 •0 •1
= ?
•0 •0 •0
Original
Courtesy of D Lowe
117
•0 •0 •0
* •0 •0 •1
•0 •0 •0
=
Original Shifted right
Courtesy of D Lowe
By 1 pixel
118
•1 •1 •1
* •1 •1 •1
•1 •1 •1
= ?
Original
Courtesy of D Lowe
119
•1 •1 •1
* •1 •1 •1
•1 •1 •1
=
Original Blur (with a
Courtesy of D Lowe
box filter)
120
•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1
=?
(Note that filter sums to 1)
Original
“details of the image”
•0 •0 •0 •0 •0 •0 •1 •1 •1
Courtesy of D Lowe
•0 •1 •0
•0 •0 •0
+ •0 •1 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1
121
• What does blurring take away?
– =
original smoothed (5x5) detail
• Let’s add it back:
+a =
original detail sharpened
122
Convolution in 2D – Sharpening
filter
•0 •0 •0 •1 •1 •1
•0 •2 •0
•0 •0 •0
- •1 •1 •1
•1 •1 •1 =
Original
Courtesy of D Lowe
Sharpening filter: Accentuates differences with local average
123
Convolution properties
Commutative: a * b = b * a
‒ Conceptually no difference between filter and signal
‒ But particular filtering implementations might break this
equality, e.g., image edges
Associative: a * (b * c) = (a * b) * c
‒ Often apply several filters one after another: (((a * b1) * b2) *
b3)
‒ This is equivalent to applying one filter: a * (b1 * b2 * b3)
Source: S.124
Lazebnik
Proof of associativity of convolution
125
Commutative: a * b = b * a
‒ But particular filtering implementations might break this
equality, e.g., image edges
Associative: a * (b * c) = (a * b) * c
‒ Often apply several filters one after another: (((a * b1) * b2) *
b3)
‒ Correlation is _not_ associative
‒ Why important?
Source: S.126
Lazebnik
• Commutative: a * b = b * a
‒ But particular filtering implementations might break this equality,
e.g., image edges
• Associative: a * (b * c) = (a * b) * c
‒ Often apply several filters one after another: (((a * b1) * b2) * b3)
‒ Correlation is _not_ associative (rotation effect)
‒ Why important?
• Distributes over addition: a * (b + c) = (a * b) + (a * c)
• Scalars factor out: ka * b = a * kb = k (a * b)

Source: S.127
Lazebnik
Important filter: Gaussian
• Weight contributions of neighboring pixels by nearness
0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013
y 0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003
x 5 x 5, s = 1
y
Slide credit: Christopher Rasmussen

128
James Hays
Smoothing with Gaussian filter
129
James Hays
Smoothing with box filter
130
Gaussian filters
• Remove “high-frequency” components from the image
(low-pass filter)
‒ Images become more smooth
• Gaussian convolved with Gaussian…
…is another Gaussian
– So can smooth with small-width kernel, repeat, and get same
result as larger-width kernel would have
– Convolving two times with Gaussian kernel of width σ is same
as convolving once with kernel of width σ√2
• Separable kernel
– Factors into product of two 1D Gaussians
Source: K.131
Grauman
Separability of the Gaussian filter
132D. Lowe
Source:
Separability example
2D convolution =
(center location only)
The filter factors

into a product of 1D
filters:
Perform convolution
along rows: * =
Followed by convolution =
along the remaining column: *
Source: K. 133
Grauman
Separability
Why is separability useful in practice?
134
Separability
Why is separability useful in practice?
MxN image, PxQ filter

• 2D convolution: ~MNPQ multiply-adds
• Separable 2D: ~MN(P+Q) multiply-adds
Speed up = PQ/(P+Q)
9x9 filter = ~4.5x faster
135
Practical matters
How big should the filter be?
• Values at edges should be near zero
• Gaussians have infinite extent…
• Rule of thumb for Gaussian: set filter half-width to
about 3 σ
136James Hays
Practical matters
• What about near the edge?
‒ the filter window falls off the edge of the image
‒ need to extrapolate
‒ methods:
• clip filter (black)
• wrap around
• copy edge
• reflect across edge
137
Source: S. Marschner
Convolution in Convolutional Neural
Networks
• Convolution is the basic operation in CNNs
• Learning convolution kernels allows us to learn
which `features’ provide useful information in
images.
138
Non-linear filters
139
Median filters
• Operates over a window by selecting the median
intensity in the window.
• ‘Rank’ filter as based on ordering of gray levels
‒ E.G., min, max, range filters
Steve Seitz, Steve Marschner

140
Image filtering - mean 1 1 1
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
?
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
141S. Seitz
Image filtering - mean 1 1 1
f [× ,× ] 1 1 1
1 1 1
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0 50
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
h[m, n] = å f [k , l ] I [m + k , n + l ]
k ,l Credit:
142S. Seitz
Median filter?
I [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
?
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Credit:
143S. Seitz
Median filters
• What advantage does a median filter have over a
mean filter?

144
Noisy Jack – Salt and Pepper
145
Mean Jack – 3 x 3 filter
146
Very Mean Jack – 11 x 11 filter
147
Noisy Jack – Salt and Pepper
148
Median Jack – 3 x 3
149
Very Median Jack – 11 x 11
150
Median filters
• What advantage does a median filter have over a
mean filter?
• Is a median filter a kind of convolution?

151
Review: questions
1. Write down a 3x3 filter that both:
- Returns a positive value if the average value of the 4-
adjacent neighbors is less than the center,
- Returns a negative value otherwise.
Slide:
152 Hoiem
Review: questions
1. Write down a 3x3 filter that both:
- Returns a positive value if the average value of the 4-
adjacent neighbors is less than the center,
[ 0 -1/4 0;
- Returns a negative value otherwise. -1/4 1 -1/4;
0 -1/4 0 ]
Slide:
153 Hoiem

IT5409 Ch3 Filters 20200228 v3 Part1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IT5409 Ch3 Filters 20200228 v3 Part1

Uploaded by

Copyright:

Available Formats

Computer Vision

Chapter 3: Image Processing

• What we can see on the picture?

‒ Index (0,0): Top left corner

• Principal type of images

• Other : multi-spectre, depth image,…

It exists other color spaces:

• An image contains discrete number of pixels

• An image contains discrete number of pixels

• Isolated transformations (point operators/point processing) of the

Source : Caroline Rougier. Traitement d'images (IFT2730). Univ. de Montréal.

T(r) • The values of r below m are

Input image Negative of the

• The general form of the log transformation:

0 L/4 L/2 3L/4 L-1

Input gray level r

• Use this Log to expand the values of dark pixels in

In Practice with grey level image :

0 L/4 L/2 3L/4 L-1

• If r1 = s1, r2 = s2, 3L/4/

• If (r1, s1) = (rmin, 0) ;

Original image, low Contrast sketching Thresholding

Image dynamic range = [min intensity, max intensity]

‒ Difference between intensity value maximum et minimum

max 𝐼(𝑢, 𝑣) − min(𝐼(𝑢, 𝑣))

𝑇(𝑘) 0 0 6/16 11/16 15/16 16/16 16/16 16/16 16/16 16/16

𝑠! = 9𝑇(𝑘) 3.3 6.1 8.4

• No parameters. OpenCV: cv2.equalizeHist(img)

So, we have histogram matching or histogram

• Image filters in spatial domain

• Image filters in the frequency domain

Slide credit: David Lowe

Slide credit: David Lowe (UBC)

• Why does it sum to one?

Slide credit: David Lowe (UBC)

Original Shifted left

Correlation and convolution are identical when the filter is symmetric.

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

We first need to calculate h[n-k, m-l]

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

We are going to convolve a function f with a filter h.

Assume we have a filter(h[,]) that

Assume we have a filter(h[,]) that

Assume we have a filter(h[,]) that

Assume we have a filter(h[,]) that

Assume we have a filter(h[,]) that

Assume we have a filter(h[,]) that

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

Slide credit: Song Ho Ahn

original smoothed (5x5) detail

• Let’s add it back:

original detail sharpened

• Distributes over addition: a * (b + c) = (a * b) + (a * c)