Image Enhancement &

Restoration
Computer Vision
 CONTENTs

• Point operators & Histogram equalization

• Image Filter
• Image Enhancement
• Image Restoration
• Image Pyramids

Computer Vision 2
 CONTENTs

•Point operators & Histogram equalization

• Image Filter
• Image Enhancement
• Image Restoration
• Image Pyramids

Computer Vision 3
 Image transformations
What types of image transformations can we do?

Warping Filtering

changes pixel locations changes pixel values

Computer Vision 4
 …Image transformations
What types of image transformations can we do?

Point
Operation
point processing

Neighborhood
Operation
“filtering”

Computer Vision 5
 Point operators

• Reverse Image
• Image Thresholding
• Logarithmic Transformations
• Power-law (Gamma) Transformations
• Contrast Stretching
• Grey level Slicing
• Bit-Plane Slicing
Image Enhancement
Computer Vision 6
 …Point operators

•The smallest possible neighborhood is of size 1x1

 g depends only on the value of f at a single point (x,y)

T becomes an intensity (gray-level) transformation function of the form

s=T(r)
• s is the output image
• r is an input image

Computer Vision
 …Point operators
•Image negatives (Reverse Image)
• Cải thiện các chi tiết màu trắng hay màu xám nằm trong vùng tối của ảnh
s = L-1- r = intensitymax - r

Original Negative
s = 1.0 - r
Image Image

Computer Vision
 …Point operators

•Image Thresholding
•Hữu ích với phân đoạn ảnh: cô lập một đối tượng so với nền

1.0 r > threshold

s=
0.0 r <= threshold

Computer Vision
 …Point operators

Chuyển đổi độ sáng

 …Point operators

•Có nhiều kiểu chuyển đổi độ xám khác nhau

•Ba kiểu hay dùng

• Tuyến tính
 Negative / Identity
• Logarithmic
 Log / Inverse log
• Luật hàm mũ
 nth power / nth root

Computer Vision
 …Point operators

Logarithmic transforms

•Dạng tổng quát của biến đổi log:

s = c * log(1 + r)

Biến đổi log ánh xạ vùng hẹp các mức sáng đầu vào thành vùng rộng
các mức sáng đầu ra
 Biến đổi log ngược thực hiện chuyển đổi ngược lại

Computer Vision
 …Point operators

Logarithmic transforms
•Các hàm log rất hữu ích khi các giá trị độ xám của ảnh đầu vào có
miền giá trị rộng.
s = log(1 + r)
•ví dụ, biến đối Fourier của một ảnh được chuyển sang biến đổi log
để làm rõ hơn các chi tiết.

s = log(1 + r)

Computer Vision
 …Point operators
Biến đổi hàm mũ

•Các biến đổi lũy thừa có dạng:

s=c*rγ

• Ánh xạ phạm vi hẹp các giá trị độ xám

đầu vào sang phạm vi rộng hơn các giá trị
đầu ra hoặc ngược lại
• Thay đổi  cho ta họ đường cong
• Thông thường, thường thiết lập c=1
• Các mức độ xám trong phạm vi [0.0, 1.0]
Computer Vision
 …Point operators
Biến đổi hàm mũ
γ = 0.6
1
Transform ed Intensities

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Old Intensities
 …Point operators
Biến đổi hàm mũ
γ = 0.4
T ra n s fo rm e d In te n s itie s

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
 …Point operators
Biến đổi hàm mũ
γ = 0.3
T ra n s fo rm e d In te n s i ti e s

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
 …Point operators
Biến đổi hàm mũ

s = r 0.6

•Ẩnh cộng hưởng từ của

s=
bệnh nhân bị gãy cột sống

s = r 0.4
r 0 .3

•Các đường cong làm nổi

các chi tiết khác nhau

Computer Vision
 …Point operators
Biến đổi hàm mũ
 …Point operators
Biến đổi hàm mũ

γ = 5.0
T ra n s fo rm ed In ten sities

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 0.2 0.4 0.6 0.8 1
Original Intensities
 …Point operators
Biến đổi hàm mũ

s = r 3.0
•Ảnh đường băng chụp từ
trên không

s = r 4.0
s=
r
•Biến đổi hàm mũ được sử dụng 5.0

để làm tối ảnh

•Các γ khác khau hiển thị các
chi tiết khác nhau

Computer Vision
 …Point operators

Grey level Slicing (cắt lát độ xám)

•Thay vì sử dụng các hàm toán học thông dụng, ta có thể sử dụng biến đổi
bất kỳ do người dùng định nghĩa
•Ảnh dưới đây mô tả biến đổi tuyến tính mở rộng độ tương phản để cộng độ
tương phản vào ảnh chất lượng kém

Computer Vision
 …Point operators

Grey level Slicing

•Làm nổi một vùng độ xám
• Tương tự như cắt ngưỡng
• Các mức khác có thể bị loại bỏ hoặc giữ nguyên
• Hữu ích để làm nổi các chi tiết trong ảnh

Computer Vision
 …Point operators

Bit Plane Slicing

•Cô lập các bit đặc biệt của các giá trị pixel trong ảnh để làm nổi
các khía cạnh quan tâm của ảnh
• Các bit thứ tự cao thường chứa thông tin trực quan quan trọng
• Các bit thứ tự thấp thường chứa các chi tiết tinh tế

Computer Vision
 …Point operators
Bit Plane Slicing

[10000000] [01000000]

[00100000] [00001000]

[00000100] [00000001]

Computer Vision
 …Point operators
Bit Plane Slicing

Computer Vision
 …Point operators
Bit Plane Slicing

Reconstructed image using

only bit planes 8 and 7

Reconstructed image using

only bit planes 8, 7 and 6

Reconstructed image using

only bit planes 7, 6 and 5

Computer Vision
 …Point operators
original darken lower contrast non-linear lower contrast

invert lighten raise contrast non-linear raise contrast

Computer Vision 28
 Histogram equalization

• Hist. của ảnh chỉ ra phân bố mức xám trên ảnh

• Hữu ích trong việc xử lý ảnh, đặc biệt trong phân đoạn ảnh
Frequencies

Grey Levels

Computer Vision
 …Histogram equalization

Computer Vision
…Histogram equalization
…Histogram equalization
 …Histogram equalization

Computer Vision
 …Histogram equalization

Computer Vision
 …Histogram equalization

Ảnh có độ tương phản cao

 hist phân bố đều
Computer Vision
 …Histogram equalization

Nới rộng tần số trong ảnh (hay cân bằng ảnh) là kỹ thuật đơn giản
để cải thiện ảnh tối hoặc các ảnh rửa trôi

Công thức để cân bằng hist: sk  T ( rk )

• rk: độ sáng đầu vào k
• sk:
• k:
độ sáng đầu ra
phạm vi độ sáng
 p
j 0
r (rj )
(e.g 0.0 – 1.0) k nj
• nj:
• n:
tần suất của độ sáng j
tổng các tần suất
 
j 0 n
…Histogram equalization
 …Histogram equalization

Các hàm được dùng để cân bằng ảnh

Computer Vision
 …Histogram equalization

Các hàm được dùng để cân bằng ảnh

Computer Vision
 …Histogram equalization

Computer Vision
 …Histogram equalization

Các hàm được dùng để cân bằng ảnh trong ví dụ

Computer Vision
 Image & Operators

Basic reading:

Szeliski textbook, Section 3.1 & 3.3

Computer Vision 42
 CONTENTs

• Point operators & Histogram equalization

•Image Filter
• Image Enhancement
• Image Restoration
• Image Pyramids
Computer Vision 43
 Image filter

• Image filtering
• Compute function of local neighborhood at each position
• Replace each pixel with a weighted average of its neighborhood
• The weights are called the filter kernel

• Really important!
• Enhance images g[ , ]
 Denoise, resize, increase contrast, etc.
1 1 1
• Extract information from images
 Texture, edges, distinctive points, etc. 1 1 1
• Detect patterns
 Template matching
1 1 1

Example: box filter

Computer Vision 44
 Image filter

Computer Vision 45
 Linear filter

Cross-correlation filtering (Linear filter)

• Let’s write this down as an equation. Assume the averaging window is
(2k+1)x(2k+1):

• We can generalize this idea by allowing different weights for different neighboring
pixels:

Computer Vision 46
 Linear filter
Cross-correlation filtering (Linear filter)
• We can generalize this idea by allowing different weights for different neighboring
pixels:

• This is called a cross-correlation operation and written:

• The above allows negative filter indices. When you implement need to use:
H[u+k,v+k] instead of H[u,v]

a b c r s t eprocessed = v*e +
d e f u v w r*a + s*b + t*c +
g h i x y z u*d + w*f +
Original Image Pixels Filter
x*g + y*h + z*i
Computer Vision 47
 …Linear filter

Key properties of linear filters

•Linearity:
filter(f1 + f2) = filter(f1) + filter(f2)

•Shift invariance: same behavior regardless of pixel location

filter(shift(f)) = shift(filter(f))

•Any linear, shift-invariant operator can be represented as a convolution

Computer Vision 48
 …Linear filter
…More properties
• Commutative: a * b = b * a
• Conceptually no difference between filter and signal

• Associative: a * (b * c) = (a * b) * c
• Often apply several filters one after another: (((a * b1) * b2) * b3)
• This is equivalent to applying one filter: a * (b1 * b2 * b3)

• Distributes over addition: a * (b + c) = (a * b) + (a * c)

• Scalars factor out: ka * b = a * kb = k (a * b)

• Identity: unit impulse e = [0, 0, 1, 0, 0], a * e = a

Computer Vision 49
 …Linear filter
Practice with linear filters

0 0 0
0 1 0
0 0 0

Original Filtered (no change)

0 0 0
0 0 1
0 0 0

Original Shifted left by 1 pixel

Computer Vision
 …Linear filter
Practice with linear filters

1 1 1
1 1 1
1 1 1

Original Blur (with a box filter)

-
0 0 0 1 1 1
0 2 0 1 1 1
0 0 0 1 1 1

Sharpening filter
Original - accentuates differences with local average
- stress intensity peaks
Computer Vision
 …Linear filter
Smoothing with box filter

Computer Vision 52
 …Linear filter
…Sharpening

Computer Vision
 Convolution filter

Convolution (tích chập)

A convolution operation is a cross-correlation where the filter is flipped
both horizontally and vertically before being applied to the image:

It is written: G  H *F

a b c r s t eprocessed = v*e +
d e f * u v w z*a + y*b + x*c +
w*d + u*f +
g h i x y z
Original Image Pixels Filter
t*g+ s*h + r*i
Computer Vision 54
 …Convolution filter

Filtering vs. Convolution

• 2D linear filter

• 2D convolution filter

• Most of the time won’t matter, because our kernels will be symmetric.
• Will be important when we discuss frequency-domain filtering.
Computer Vision 55
 …Convolution filter

Filtering vs. Convolution

• Cross-correlation/convolution is useful for, e.g.,

• Blurring
• Sharpening
• Edge Detection
• Interpolation

• Convolution has a number of nice properties

• Commutative, associative
• Convolution corresponds to product in the Fourier domain

Computer Vision 56
 CONTENTs

• Point operators & Histogram equalization

• Image Filter

•Image Enhancement
• Image Restoration
• Image Pyramids

Computer Vision 57
 Gaussian filter

• Most obvious difference is that a single point of light viewed in a defocused

lens looks like a fuzzy blob;
• …but the averaging process would give a little square

Computer Vision 58
 …Gaussian filter

• Better idea: to eliminate edge effects, weight contribution of neighborhood pixels

according to their closeness to the center, like so:

“fuzzy blob”
kernel
Important filter: Gaussian filter 1 2 1
1
A Gaussian kernel gives less weight to pixels further 2 4 2
16
from the center of the window 1 2 1

Computer Vision 59
 …Gaussian filter

• Gaussian Kernel

• Weight of neighboring pixels fall off

with distance from center pixel

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013
0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003

5 x 5,  = 1
Computer Vision 60
 …Gaussian filter

Smoothing with Gaussian filter

 …Gaussian filter

Gaussian filter
vs. Box filter
 …Gaussian filter
Gaussian vs Box filtering

7x7 Gaussian

original

Gaussian filter remove “high-frequency”

components from the image (low-pass filter)

 Images become more smooth

7x7 box
Computer Vision 63
 …Gaussian filter

• Gaussian filter removes “high-frequency” components from the image

(low-pass filter)
• Images become more smooth

• Convolution with self is another Gaussian

• So can smooth with small-width kernel, repeat, and get same result as larger-
width kernel would have
• Convolving two times with Gaussian kernel of width σ is same as convolving
once with kernel of width σ√2

Computer Vision 64
 Derivative filter

What are image edges?

Very sharp
discontinuities
in intensity.

grayscale image

domain

Computer Vision 65
 … Derivative filter

Edge detection
Goal: Identify sudden changes (discontinuities) in an image

• Detecting edges in an image (i.e., discontinuities in a function) ?

 take derivatives: derivatives are large at discontinuities.

• Differentiate a discrete image (or any other discrete signal) ?

 use finite differences.

Computer Vision 66
 … Derivative filter

Characterizing edges
• An edge is a place of rapid change in the image intensity function
intensity function
image (along horizontal scanline) first derivative

edges correspond to
extrema of derivative
Computer Vision 67
 … Derivative filter

Finite differences
• Definition of the first-order derivative using forward difference

• Alternative: use central difference

• For discrete signals: Remove limit and set h = 2

Computer Vision 68
 … Derivative filter
8
Finite differences 7
6
f(x)
5
4

• first-order finite difference 3

2
A 1
0

5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7

0 -1 -1 -1 -1 -1 0 0 6 -6 0 0 0 1 2 -2 -1 0 0 0 7 0 0 0
8
6
4
f’(x)
 sự khác nhau của các giá 2

trị liền kề  tốc độ thay đổi 0

-2

của hàm -4
-6
-8

Computer Vision 69
 … Derivative filter
8
Finite differences 7
6
f(x)
5
4

• second-order finite difference 3

2
1
0

5 5 4 3 2 1 0 0 0 6 0 0 0 0 1 3 1 0 0 0 0 7 7 7 7

-1 0 0 0 0 1 0 6 -12 6
2nd 0 0 1
Derivative 1 -4 1 1 0 0 7 -7 0 0

10

5 f’’(x)
0

 sự khác nhau của giá trị -5

trước, sau và giá trị hiện tại -10

-15

Computer Vision 70
 … Derivative filter

Finite differences

first-order 1D derivative filter

finite difference 1 0 -1

second-order Laplace filter

finite difference ? 1
1 -2

Computer Vision 71
 … Derivative filter

Several derivative filters

1 0 -1 1 2 1 3 0 -3 3 10 3
Sobel 2 0 -2 0 0 0 Scharr 10 0 -10 0 0 0
1 0 -1 -1 -2 -1 3 0 -3 -3 -10 -3

1 0 -1 1 1 1
0 1 1 0
Prewitt 1 0 -1 0 0 0 Roberts
-1 0 0 -1
1 0 -1 -1 -1 -1

Computer Vision 72
 … Derivative filter

Image gradient
• The gradient of an image:

• The gradient points in the direction

of most rapid increase in intensity

1 0 -1 1 2 1
2 0 -2 0 0 0
1 0 -1 -1 -2 -1
• Convolve with the image to compute derivatives.

Computer Vision 73
 … Derivative filter

Image gradient
• The gradient of an image:

• Form the image gradient, and compute its direction and

amplitude.
• Direction

• Amplitude

Computer Vision 74
 … Derivative filter
Image gradient

vertical
original
derivative

gradient horizontal
amplitude derivative

Computer Vision 75
 … Derivative filter

Effects of noise
• Consider a single row or column of the image
• Plotting intensity as a function of position gives a signal

(Intensity)

(derivative)
Where is the edge?
Computer Vision 76
 … Derivative filter

Effects of noise: Differentiation is very sensitive to noise

• Finite difference filters respond strongly to noise
• Image noise results in pixels that look very different from their neighbors
• Generally, the larger the noise the stronger the response
• What is to be done?
• Smoothing the image should help, by forcing pixels different to their neighbors
(=noise pixels?) to look more like neighbors

Computer Vision 77
 … Derivative filter
Solution: blur/smooth first

input

Gaussian g

blurred f*g

derivative of 
( f  g) How much
blurred x
should we blur?

( f  g)
x
Computer Vision  To find edges, look for peaks in 78
 … Derivative filter

Derivative of Gaussian (DoG) filter

• Derivative theorem of convolution:

input
f

derivative of 
h
Gaussian x

output (same as 
before) ( h)  f
x

Computer Vision 79
 … Derivative filter

Derivative of Gaussian filter

* [1 0 -1] =

• Is this filter separable?

Computer Vision 80
 … Derivative filter

Derivative of Gaussian filter

x-direction y-direction

Computer Vision 81
 Laplace filter

• Laplacian được viết lại dạng

2 2
2  f  f
 f  2  2
 x  y
• Trong đó đạo hàm thành phần bậc 1 theo phương x, y:
2 f
2
 f ( x  1, y )  f ( x  1, y )  2 f ( x, y )
 x

2 f
2
 f ( x, y  1)  f ( x, y  1)  2 f ( x, y )
 y
Computer Vision 82
 …Laplace filter

• 2
 f  [ f ( x  1, y )  f ( x  1, y )
 f ( x, y  1)  f ( x, y  1)]
 4 f ( x, y )

• 0 1 0

1 -4 1

0 1 0
Original Laplacian
Image Filtered Image
Computer Vision 83
 …Laplace filter

• Kết quả của lọc Laplacian chưa phải là một ảnh cải thiện
Trừ ảnh ban đầu cho ảnh Laplacian để được ảnh cải thiện sắc nét
2
g ( x, y )  f ( x, y )   f

- =
Original Laplacian Sharpened
Image Filtered Image Image
Computer Vision 84
 …Laplace filter

Computer Vision 85
 …Laplace filter

• Đơn giản hóa việc cải thiện ảnh

2
g ( x, y )  f ( x , y )   f
 f ( x, y )  [ f ( x  1, y )  f ( x  1, y )
 f ( x, y  1)  f ( x, y  1)
 4 f ( x, y )]
0 -1 0
 5 f ( x, y )  f ( x  1, y )  f ( x  1, y )
-1 5 -1
 f ( x, y  1)  f ( x, y  1)
0 -1 0

Computer Vision 86
 …Laplace filter

0 -1 0
-1 5 -1
0 -1 0

Computer Vision 87
 …Laplace filter

• Biến thể của Laplacian

0 1 0 1 1 1
Simple Variant of
1 -4 1 1 -8 1
Laplacian Laplacian
0 1 0 1 1 1

-1 -1 -1
-1 9 -1
-1 -1 -1

Computer Vision 88
 …Laplace filter

• Differentiation is very sensitive to noise

When using derivative filters, it is critical to blur first!

input

Gaussian

blurred

derivative of How much

blurred should we blur?

Computer Vision 89
 …Laplace filter
• Derivative of Gaussian filter (DoG)
Derivative theorem of convolution:

input

derivative of
Gaussian

• How many operations

output did we save?
(same as before) • Any other advantages
beyond efficiency?
Computer Vision 90
 …Laplace filter
• Laplacian of Gaussian filter (LoG)

As with derivative, we can combine Laplace filtering with Gaussian filtering

input

Laplacian of
Gaussian

output ? “zero crossings” at edges

Computer Vision 91
 …Laplace filter

• Laplace vs LoG filtering examples

Laplacian of Gaussian filtering Laplace filtering

Computer Vision 92
 …Laplace filter

• Laplacian of Gaussian vs Derivative of Gaussian

Laplacian of Gaussian filtering Derivative of Gaussian filtering

Computer Vision 93
 …Laplace filter

• Laplacian of Gaussian vs Derivative of Gaussian

zero-crossing peak

Laplacian of Gaussian filtering Derivative of Gaussian filtering

Zero crossings are more accurate at localizing edges (but not very convenient).
Computer Vision 94
 …Laplace filter

Gaussian

Derivative of Gaussian

how does this relate to this

lecture’s cover picture?

Laplacian of Gaussian
Computer Vision 95
 Sobel filter

Separable filters
• a 2D filter is separable if it can be written as the product of a “column” and a “row”.
1 1 1 1 1 1 1
example: =
box filter
1 1 1 1 * row
1 1 1 1
column
• a 2D separable filter is equivalent to two 1D convolutions
(with the “column” and “row” filters).
• If the image has M x M pixels and the filter kernel has size N x N:
• What is the cost of convolution with a non-separable filter?  M2 x N2
• What is the cost of convolution with a separable filter?  2 x N x M2

Computer Vision 96
 …Sobel filter

2D convolution
(center location only)

The filter factors

into a product of 1D filters

Perform convolution
along rows

Followed by convolution
along the remaining column

Computer Vision
 …Sobel filter

1 0 -1 1 1 0 -1
2 0 -2 = 2
* 1D derivative
1 0 -1 1 filter
Sobel filter Blurring

• In a 2D image, Sobel filter responses along horizontal or vertical lines

 Các bộ lọc Sobel thường được sử dụng để phát hiện nếp gấp

Computer Vision 98
 …Sobel filter

• In a 2D image, Sobel filter responses along horizontal or vertical lines

Horizontal Sober filter:

1 0 -1 1
2 0 -2 = 2
* 1 0 -1
1 0 -1 1

Vertical Sobel filter:

1 2 1 1
1 2 1
0 0 0 = 0
*
-1 -2 -1 -1

Computer Vision 99
 …Sobel filter

1 0 -1
2 0 -2
1 0 -1
Horizontal Sober filter

Vertical Edge
(absolute value)
Computer Vision
 …Sobel filter

1 2 1
0 0 0
-1 -2 -1
Vertical Sobel filter:

Horizontal Edge
(absolute value)
Computer Vision
 Image Filter

Basic reading:

Szeliski textbook, Sections 3.2 - 3.5

Computer Vision 102

 CONTENTs

• Point operators & Histogram equalization

• Image Filter
• Image Enhancement

•Image Restoration
• Image Pyramids

Computer Vision 103

 Image Restoration

• Nhằm khôi phục ảnh bị biến đổi

• Xác định quá trình biến đổi và cố gắng làm quá trình ngược lại
• Tương tự quá trình cải thiện ảnh nhưng nhiều mục đích

Computer Vision 104

 Image Restoration

Noise
• Nhiễu xuất hiện trong quá trình thu nhận ảnh, số hóa và truyền
• Cảm biến ảnh có thể bị ảnh hưởng bởi các điều kiện môi trường
• Nhiễu có thể can thiệp vào ảnh trong quá trình truyền
• Ảnh nhiễu được mô phỏng như sau:
g ( x, y )  f ( x , y )   ( x , y )
f(x, y) là ảnh ban đầu
η(x, y) là nhiễu
g(x, y) ảnh sau khi bị nhiễu tác động
Nếu xác định mô hình nhiễu, ta có thể tách nhiễu để phục hồi ảnh

Computer Vision
 Image Restoration
Common Types of Noise
Gaussian noise: variations in intensity drawn from a
Gaussian normal distribution

Computer Vision 106

 Image Restoration
Common Types of Noise
Impulse noise: random occurrences of white pixels

Computer Vision 107

 Image Restoration
Common Types of Noise
Salt and pepper noise: random occurrences of black and white pixels

Computer Vision 108

 Image Restoration
Ví dụ:
Histogram to go here

Image Histogram

Histogram to go here

Gaussian Rayleigh Erlang Exponential Uniform Impulse

Computer Vision
nhiễu với các mô hình khác nhau
 Image Restoration

Mô hình nhiễu
nhiều mô hình nhiễu
η(x, y) khác nhau:
• Gaussian
 thông dụng
• Rayleigh
• Erlang
• Hàm mũ
• Đồng nhất
• Xung
 Nhiễu muối,
nhiễu hạt tiêu

Computer Vision
 Image Restoration
• Dùng lọc không gian đơn giản để loại nhiễu:
• Trung bình số học
• Trung bình hình học
• Trung bình Harmonic
• Trung bình Contraharmonic

• Trung bình số học

/9
1 1
/9 /9
1

ˆ 1
f ( x, y )   g ( s, t )
mn ( s ,t )S xy /9
1 1
/9 /9
1

/9
1 1
/9 /9
1

 Lọc làm mịn đơn giản, làm mờ ảnh để loại nhiễu

Computer Vision
 Image Restoration

• Trung bình hình học 1

  mn
fˆ ( x, y )    g ( s, t ) 

( s ,t )S xy 

 Làm mịn như trung bình số học nhưng ít làm mất chi tiết ảnh

• Trung bình Harmonic

mn
fˆ ( x, y ) 
1

( s ,t )S xy g ( s, t )

 Làm việc tốt với nhiễu muối nhưng không tốt với nhiễu hạt tiêu
Computer Vision
 Image Restoration

• Trung bình Contraharmonic

 g ( s, t ) Q 1

fˆ ( x, y ) 
( s ,t )S xy

 g ( s ,
( s ,t )S xy
t ) Q

Q là bậc của bộ lọc; thay đổi Q sẽ làm thay đổi hành vi của bộ lọc
Q >0  loại nhiễu hạt tiêu
Q <0  loại nhiễu muối

Computer Vision
 Image Restoration

Image
Corrupted
Original By Gaussian
Image Noise

After A 3*3 After A 3*3

Arithmetic Geometric
Mean Filter Mean Filter
 Image Restoration

Image Image
Corrupted Corrupted
By Pepper By Salt
Noise Noise

Result of
Result of Filtering Above
Filtering Above With 3*3
With 3*3 Contraharmonic
Contraharmonic Q=-1.5
Q=1.5
 Image Restoration

Chọn không đúng giá trị của Q khi sử dụng lọc contraharmonic
có thể mạng lại những kết quả không mong muốn

Computer Vision
 Image Restoration

• Dùng lọc không gian dựa trên giá trị pixel lân cận của bộ lọc (lọc thống kê)
• Lọc trung vị
• Lọc Max, Min
• Lọc Midpoint
• Lọc cắt Alpha

• Lọc trung vị
fˆ ( x, y )  median{g ( s, t )}
( s ,t )S xy

không gây hiệu ứng làm mịn ảnh như các lọc trung bình,
Tốt cho lọc nhiễu muối và hạt tiêu
Computer Vision
 Image Restoration

• Lọc Max
fˆ ( x, y )  max {g ( s, t )}
( s ,t )S xy

• Lọc Min

fˆ ( x, y )  min {g ( s, t )}
( s ,t )S xy

 Lọc Max tốt cho loại bỏ nhiễu hạt tiêu

 Lọc Min tốt cho loại bỏ nhiễu muối

Computer Vision
 Image Restoration

• Lọc Midpoint
1
f ( x, y )   max { g ( s, t )}  min { g ( s, t )}
ˆ

2 ( s ,t )S xy ( s ,t )S xy 
Tốt loại bỏ nhiễu Gaussian và nhiễu đồng nhất

• Alpha-Trimmed
1
fˆ ( x, y )  g r ( s, t )
mn  d ( s ,t )S xy

 loại bỏ d/2 mức xám lớn nhất và d/2 mức xám bé nhất
 gr(s, t) biễu diễn phần còn lại của mn – d pixels
Computer Vision
 Image Restoration

Image Result of 1
Corrupted Pass With A
By Salt And 3*3 Median
Pepper Noise Filter

Result of 2 Result of 3
Passes With Passes With
A 3*3 Median A 3*3 Median
Filter Filter
 Image Restoration

Image Image
Corrupted Corrupted
By Pepper By Salt
Noise Noise

Result Of Result Of
Filtering Filtering
Above Above
With A 3*3 With A 3*3
Max Filter Min Filter
 Image Restoration

Image Image Further

Corrupted Corrupted
By Uniform By Salt and
Noise Pepper Noise

Filtered By Filtered By
5*5 Arithmetic 5*5 Geometric
Mean Filter Mean Filter

Filtered By Filtered By
5*5 Median 5*5 Alpha-Trimmed
Filter Mean Filter
 CONTENTs

• Point operators & Histogram equalization

• Image Filter
• Image Enhancement
• Image Restoration

•Image Pyramids
Computer Vision 123
 Image downsampling

• Why does a lower resolution image still make sense to us? What do we lose?

Computer Vision 124

This image is too big to fit on the screen.
How would you reduce it to half its size?
 …Image downsampling
Naïve image downsampling
Throw away half the rows and columns

delete even rows delete even rows

delete even columns delete even columns

1/8
1/4

1/2 What is the problem with this approach?

Computer Vision
 …Image downsampling
Naïve image downsampling

1/2 1/4 (2x zoom) 1/8 (4x zoom)

What is the 1/8 image so pixelated (and do you know what this effect is called)?
Computer Vision
 …Image downsampling
Sampling
Very simple example: a sine wave

Computer Vision
 …Image downsampling
Undersampling

Very simple example: a sine wave

Unsurprising effect: information is lost.

Computer Vision
 …Image downsampling
Undersampling

Very simple example: a sine wave

Unsurprising effect: information is lost.

Surprising effect: can confuse the signal with one of lower frequency.
Computer Vision
 …Image downsampling
Undersampling

Very simple example: a sine wave

Unsurprising effect: information is lost.

Surprising effect: can confuse the signal with one of lower frequency.
Note: we could always confuse the signal with one of higher frequency.
Computer Vision
 …Image downsampling
Nyquist-Shannon Sampling Theorem
• When sampling a signal at discrete intervals, the sampling frequency must be  2  fmax
• fmax = max frequency of the input signal
• This will allows to reconstruct the original perfectly from the sampled version

v v v good

bad

Computer Vision
 Gaussian image pyramid

The name of this sequence of subsampled images

 …Gaussian image pyramid
Gaussian
Filter Sample
Low-Pass Low-Res
Image
Filtered Image Image

Algorithm sample
repeat:
filter
filter
sample
subsample
filter
until min resolution reached
sample
Question: How much bigger than the original image is the whole pyramid?
Answer: Just 4/3 times the size of the original image!
Computer Vision
 …Gaussian image pyramid

Apply a smoothing filter first, then throw away half the

rows and columns

Gaussian filter Gaussian filter

delete even rows delete even rows
delete even columns delete even columns
1/8
1/4

1/2 Better image downsampling

Computer Vision
 …Gaussian image pyramid

1/2 1/4 (2x zoom) 1/8 (4x zoom)

Naïve image downsampling
Computer Vision
 …Gaussian image pyramid

1/2 1/4 (2x zoom) 1/8 (4x zoom)

Better image downsampling with Gaussian pre-filtering
Computer Vision
 …Gaussian image pyramid
Some properties of the Gaussian pyramid

What happens to the details of the image?

• They get smoothed out as we move to
higher levels.

What is preserved at the higher levels?

• Mostly large uniform regions in the
original image.

How would you reconstruct the original

image from the image at the upper level?
• That’s not possible.

Computer Vision
 Laplacian image pyramid
Blurring is lossy

- =

level 0 level 1 (before downsampling) residual

Can we make a pyramid that is lossless?

Computer Vision
 …Laplacian image pyramid

At each level, retain the residuals instead

of the blurred images themselves.

Can we reconstruct the original image

using the pyramid?
• Yes we can!

What do we need to store to be able to

reconstruct the original image?

Computer Vision
 …Laplacian image pyramid

= +

level 0 level 1 (upsampled) residual

Does this mean we need to store both residuals and the blurred copies of the original?

Computer Vision
 …Laplacian image pyramid

Algorithm
repeat:
filter
compute residual
subsample
until min resolution reached
Computer Vision
 …Laplacian image pyramid

It’s a Gaussian
pyramid.

Algorithm
repeat:
filter
compute residual
subsample
until min resolution reached
Computer Vision
 …Laplacian image pyramid
What do we need to construct the original image?

Computer Vision
 …Laplacian image pyramid
What do we need to construct the original image?

(2) smallest
image

(1) residuals

Computer Vision
 …Laplacian image pyramid
Reconstructing the original image

Algorithm
repeat:
upsample

sum with residual

until orig resolution reached

Computer Vision
 Gaussian vs Laplacian Pyramid
Shown in opposite
order for space.

Which one takes

more space to store?
Computer Vision
 Why is it called a Laplacian pyramid?

- =

unit Gaussian Laplacian

Difference of Gaussians approximates the Laplacian

Computer Vision
 What are image pyramids used for?
image compression multi-scale image blending
texture mapping

focal stack compositing denoising

multi-scale detection multi-scale registration

Computer Vision
 Image Filter

Basic reading:

Szeliski textbook, Sections 3.2 - 3.5

Computer Vision 150

 Image Enhancement & Restoration

Computer Vision 151

 Computer Vision

Thank You…!
Computer Vision 152