11/14/2018

Suggested readings

• Chapter 2

David A. Forsyth and Jean Ponce, “Computer Vision A Modern Approach”, 2nd
edition, Prentice Hall, Inc., 2003.

• For image processing details

Image processing basics Rafael C. Gonzalez, Richard E. Woods, “Digital Image Processing”, 3rd edition,
Prentice Hall, 2007
Lecture 02 (Contd.) Computer Vision

Acknowledgements Images

• Dr George Stockman
Professor Emeritus, Michigan State University

• Dr Mubarak Shah
Professor, University of Central Florida
Binary Gray Scale Color
• The Robotics Institute
Carnegie Mellon University

Image as a Function Binary Images

• We can think of an image as a function, f, • Im ages with only two values (0 or 1)

• Simple to process and analyze

• f: R2  R
• Very useful for industrial applications
– f (x, y) gives the intensity at position (x, y)
• Obtained from gray-level (or color)
• A color image is just three functions pasted together. We can write this im age f(x, y) by thresholding
as a “vector-valued” function:
• Characteristic Function
r(x, y) b(x, y ) = 1 if f(x, y) < T
f (x, y) = g(x, y) 0 if f(x, y ) >= T
b (x, y)

1
 11/14/2018

Binary Images Gray scale image

Y
Row 1 1 1 1 10 5 9
1

X
0: Black Row q 100
1: White 0 0 0 0 0

Red, Green, Blue channels Image thresholding

Color Image Region has brighter or darker color, etc.

If pixel > threshold

then pixel = 1

else pixel = 0

Image thresholding Image thresholding

clc; clear all; close all
sign = imread('StopSign.jpg','jpg');
red = (sign(:,:,1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200;
imwrite(out, 'SegStopSign.jpg', 'jpg');
subplot(1,2,1); imshow('StopSign.jpg'); title('Stop Sign - Original');
subplot(1,2,2); imshow('SegStopSign.jpg'); title('Stop Sign - Segmented');
clc; clear all; close all
sign = imread('Coins.jpg','jpg');
red = (sign(:,:,1)>80) & (sign(:,:,2)<240) & (sign(:,:,3)<140); out = red*200;
imwrite(out, 'SegCoins.jpg', 'jpg');
subplot(1,2,1); imshow('Coins.jpg'); title('Coins - Original');
subplot(1,2,2); imshow('SegCoins.jpg'); title('Coins - Segmented');

2

Image histogram
Image noise
• I(x,y) : the true pixel values
• n(x,y) : the noise at pixel (x,y)
Iˆ  x, y   I  x, y   n  x, y 
Salt and Pepper noise

 I  x, y 
Iˆ  x, y   
 smin  r  smax  smin 
 pl
11/14/2018
Image noise
• Light Variations
• Camera Electronics
• Surface Reflectance
• Lens
Gaussian noise
n 2
n  x, y   e 2
2
Image derivatives
pl
• Derivative: Rate of change
– Speed is a rate of change of a distance
– Acceleration is a rate of change of speed
• p is uniformly distributed random
variable • Average (Mean)
• l is threshold
• smin and smax are constant – Dividing the sum of N values by N
3
 11/14/2018

Derivative Examples

df f ( x)  f ( x  x) y  x2  x4
 lim x0  f ( x)  f x
dx x dy
 2x  4x3
dx
ds
v Speed
dt
dv y  sin x  e  x
a Acceleration
 cos x  (1)e  x
dt dy
dx

Discrete derivative Discrete derivative – Finite difference

df f ( x)  f ( x  x) df
 lim x0  f ( x)  f ( x)  f ( x  1)  f ( x) Backward difference
dx x dx

df
 f ( x)  f ( x  1)  f ( x) Forward difference
df f ( x)  f ( x  1) dx
  f ( x)
dx 1 Take value of f(x) at
present instant and
df
subtract from the  f ( x  1)  f ( x  1)  f ( x) Central difference
previous dx
df
 f ( x)  f ( x  1)  f ( x)
dx

Example Derivatives in 2-D

Given function f ( x, y) Image is 2D so we will
f ( x)  10 15 10 10 25 20 20 20 have 2 partial derivatives
f ( x)  0 5  5 0 15  5 0 0  f ( x, y ) 
   fx 
f ( x)  0 5  10 5 15 20 5 0 Gradient vector f ( x, y )   f (xx, y )    
   fy 
 y 
Derivative Masks

Gradient magnitude f ( x, y)  f x2  f y2
Backward difference [-1 1]
Forward difference [1 -1]
fx
Central difference [-1 0 1] Gradient direction   tan 1
fy

4
 11/14/2018

Derivatives of images Derivatives of images

 1 0 1 1 1 1
f x   1 0 1 f y   0 0 0  10 10 20 20 20 0 0 0 0 0
1 1
Derivative masks
3 3 10 10 20 20 20 0 0 0 0 0
 1 0 1  1 1 1  
I  10 10 20 20 20 I y  0 0 0 0 0
Averaging is one way to get rid of noise in any pixel    
10 10 20 20 20 0 0 0 0 0
10 10 20 20 20 0 0 0 0 0  10
10 10 20 20 20 0 0 0 0 0
 10 20 20 20 0 10 10 0 0
 
I  10 10 20 20 20 I x  0 10 10 0 0
    1 1 1
10 10 20 20 20 0 10 10 0 0
f y   0 0 0 
1
10 10 20 20 20 0 0 0 0 0
3
 1 1 1
(-10+0+20) + (-10+0+20) + (-10+0+20) = 30/3 = 10

Gaussian Filter Properties of Gaussian

• Most common natural model

• Smooth function, it has infinite number of derivatives

• Fourier Transform of Gaussian is Gaussian

• Convolution of a Gaussian with itself is a Gaussian

 x2  ( x2  y 2 ) • There are cells in eye that perform Gaussian filtering
g ( x)  e 2
2
 1 g ( x, y)  e 2 2

g ( x)  .011 .13 .6 1 .6 .13 .011

x=-3 x=-2 x=-1 x=0 x=1 x=2 x=3

Averages Correlation

• Mean f  h   f (k , l )h(i  k , j  l ) Go through each row

k l
Given a row, go through each column

I  I  In I i
f = image
h = kernel/filter
I 1 2  i 1
n n

• Weighted mean f1 f2 f3 h1 h2 h3
f h  f1h1  f 2 h2  f 3h3
n f4 f5 f6  h4 h5 h6  f 4 h4  f5 h5  f 6 h6

w1 I1  w2 I 2    wn I n 
wi I i  f 7 h7  f8 h8  f9 h9
f7 f8 f9 h7 h8 h9
I  i 1
n n

5
 11/14/2018

Convolution Convolution
f * h   f (k , l )h(i  k , j  l ) f h  f ( x  1, y  1)h(1,1)  f ( x, y  1)h(0,1)  f ( x  1, y  1)h(1,1) 
k l
f ( x  1, y )h(1,0)  f ( x, y )h(0,0)  f ( x  1, y )h(1,0)
h7 h8 h9 h1 h2 h3 f ( x  1, y  1)h(1,1)  f ( x, y  1)h(0,1)  f ( x  1, y  1)h(1,1)
f = image X-flip
h4 h5 h6 h4 h5 h6
h = kernel/filter f h
1 1
f  h    f ( x  i, y  i)h(i, j )
h1 h2 h3 h7 h8 h9
* x
i  1 j  1
Y-flip If h is symmetric here eg. Gaussian, correlation x
and convolution will have same results
f1 f2 f3 h9 h8 h7
f *h  f1h9  f 2 h8  f 3h7 -1,1 0,1 1,1

f4 f5 f6  h6 h5 h4  f 4 h6  f5 h5  f 6 h4 -1,0 0,0 1,0

Coordinates
f7 f8 f9 h3 h2 h1
 f 7 h3  f8 h2  f9 h1
-1,-1 0,-1 1,-1

Convolution example Convolution example

f
f f*h
h 1 1 1
? ? ? ?
f*h
2 2 2 3 h
1 -1 -1 -1 2 1
2 2 2 3 5 ? ? ?
2 1 3 3 ? ? ? ? 1 -1 -1
1 2 -1 -1 -1 1
2 1 3 3 ? ? ? ?
2 2 1 2 ? ? ? ? 1 2 -1
1 1 1
2 2 1 2 ? ? ? ?
1 3 2 2 ? ? ? ? 1 1 1
Rotate 1 3 2 2 ? ? ? ?
Rotate
1 1 1
Apply
-1 2 1 1 1 1
Apply
-1 2 1
-1 -1 1 2*2 + 1*2 + (-1)*2 + 1*1 = 5
-1 -1 1

Convolution example Convolution example

f f
1 1 1 1 1 1
f*h f*h
h -1 2 1 h -1 2 1
2 2 2 3 5 4 ? ? 2 2 2 3 5 4 4 ?
1 -1 -1 1 -1 -1
-1 -1 1 -1 -1 1
2 1 3 3 ? ? ? ? 2 1 3 3 ? ? ? ?
1 2 -1 1 2 -1
2 2 1 2 ? ? ? ? 2 2 1 2 ? ? ? ?
1 1 1 1 1 1
1 3 2 2 ? ? ? ? 1 3 2 2 ? ? ? ?
Rotate Rotate

1 1 1 1 1 1
Apply Apply
-1 2 1 -1 2 1
2*2+1*2+(-1)*2+1*1 = 5 2*2+1*2+(-1)*2+1*1 = 5
-1 -1 1 -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1 -1 1 -1*2+2*2+1*2-1*2-1*1+1*3= 4
-1*2+2*2+1*3-1*1-1*3+1*3= 4

6
 11/14/2018

Convolution example Convolution example

f f
1 1 1
f*h f*h
h -1 2 1 h 1 1 1
2 2 2 3 5 4 4 -2 2 2 2 3 5 4 4 -2
1 -1 -1 1 -1 -1
-1 -1 1 -1 2 1
2 1 3 3 ? ? ? ? 2 1 3 3 9 ? ? ?
1 2 -1 1 2 -1
-1 -1 1
2 2 1 2 ? ? ? ? 2 2 1 2 ? ? ? ?
1 1 1 1 1 1
1 3 2 2 ? ? ? ? 1 3 2 2 ? ? ? ?
Rotate Rotate

1 1 1 1 1 1
Apply Apply
-1 2 1 -1 2 1
2*2+1*2+(-1)*2+1*1 = 5 2*2+1*2+(-1)*2+1*1 = 5
-1 -1 1 -1*2+2*2+1*2-1*2-1*1+1*3= 4 -1 -1 1 -1*2+2*2+1*2-1*2-1*1+1*3= 4
-1*2+2*2+1*3-1*1-1*3+1*3= 4 -1*2+2*2+1*3-1*1-1*3+1*3= 4
-1*2+2*3-1*3-1*3 = -2 -1*2+2*3-1*3-1*3 = -2
1*2+1*2+2*2+1*1-1*2+1*2= 9

Convolution example Boarder handling

And so on ……..

for interior pixels where but what values do we

there is full overlap, we use for pixels that are
know what to do. “off the image” ?

Boarder handling Boarder handling

• One of the simplest methods is zero-padding • Replication – replace each off-image pixel with the value
from the nearest pixel that is in the image.

7
 11/14/2018

Boarder handling Boarder handling

• Wraparound – when going off the right border of the image, you wrap
• Reflection – reflect pixel values at the border (as if there around to the left border. Similarly, when leaving the
was a little mirror there) bottom of the image you reenter at the top. Basically,
the image is a big donut

Filtering Examples Filtering Examples

0 0 0 0 0 0

* 0 1 0  0 0 1 
*
0 0 0 0 0 0

No effect

Filtering Examples Filtering Examples

1 1 1 1 1
1 1 1 1 1 1 1 1
1  1 
* 1 1 1 * 1 1 1 1 1
9 25
1 1 1 1 1 1 1 1
1 1 1 1 1
Averaging

8
 11/14/2018

Blurring Examples Filtering Gaussian

2.4
0.3

0
Original Pixel offset Filtered
* 

8 8
6
4.8
4 4
0.3

0
Original Pixel offset Filtered

Gaussian vs Basic Averaging Noise Filtering

Gaussian Noise After Averaging After Gaussian Smoothing

Can model noise by Gaussian/Normal distribution

Gaussian Smoothing Smoothing by Averaging
Mimic the effect of many random processes that occur in nature
Weighted averaging Constant weight

Noise Filtering Images in Fourier domain

Magnitude of the FT

Salt & Pepper Noise After Averaging After Gaussian Smoothing

Does not look anything like what we have seen

9
 11/14/2018

Images in Fourier domain Conv. is Mult. in Fourier Domain

Magnitude of the FT f(x,y) |F(sx,sy)|

h(x,y) |H(sx,sy)|

g(x,y) |G(sx,sy)|
Does not look anything like what we have seen

Low-pass Filtering High-pass Filtering

Original image FFT of original image Low-pass filter Original image FFT of original image High-pass filter

Let the low frequencies pass and eliminating the high frequencies. Lets through the high frequencies (the detail), but eliminates the low
Low-pass image FFT of low-pass image frequencies (the overall shape). It acts like an edge enhancer.
High-pass image FFT of high-pass image

Generates image with overall

shading, but not much detail

Boosting High Frequencies Most information at low frequencies!

Original image FFT of original image High-boost filter

High boosted image FFT of high-boosted image

10
 11/14/2018

Fun with Fourier Spectra Some basic MATLAB functions

• conv: 1-D Convolution

C = conv(A, B) convolves vectors A and B

• conv2: Two dimensional convolution

C = conv2(A, B) performs the 2-D convolution

of matrices A and B.

Some basic MATLAB functions Some basic MATLAB functions

• filter2: Two-dimensional digital filter • gradient: Approximate gradient

Y = filter2(B,X) filters the data in X with the 2-D [FX,FY] = gradient(F) returns the numerical gradient of the
FIR filter in the matrix B matrix F. FX corresponds to dF/dx, FY corresponds to dF/dy

The result, Y, is computed using 2-D correlation and is • mean: Average or mean value
the same size as X

For vectors, mean(X) is the mean value (average) of the

filter2 uses CONV2 to do most of the work. 2-D correlation
elements in X
is related to 2-D convolution by a 180 degree rotation of
the filter matrix

Some basic MATLAB functions

• special: Create predefined 2-D filters

H = fspecial(TYPE) creates a two-dimensional filter H of the

specified type. Possible values for TYPE are:

'average' averaging filter;

'gaussian' Gaussian low-pass filter
'laplacian’ filter approximating the 2-D Laplacian operator
'log' Laplacian of Gaussian filter
'prewitt' Prewitt horizontal edge-emphasizing filter
'sobel' Sobel horizontal edge-emphasizing filter

• Example: H=fspecial('gaussian',7,1)
Creates a 7x7 Gaussian filter with variance 1

11