Preview

What is an image Segmentation?

Image Segmentation  Segmentation is to subdivide an image

into its constituent regions or objects.

 At which level subdivision stop

 Application Dependent

 Segmentation should stop when the

objects of interest in an application have
been isolated.
2

3
of
20
The Segmentation Problem
Segmentation attempts to partition the pixels
of an image into groups that strongly
correlate with the objects in an image
Typically the first step in any automated
computer vision application

1
 Example :
Example : Regions Straight Lines

8 9

Different Approaches for

Image Segmentation
 Segmentation algorithms generally are based on one of
2 basis properties of intensity values
 1.discontinuity:
To partition an image based on abrupt changes in intensity
(such as edges)
Edges are found based on DIFFERENCES between values of
adjacent pixels.

 2.similarity:
To partition an image into regions that are similar according to
a set of predefined criteria.
Regions are found based on SIMILARITIES between values of
adjacent pixels.

10 11

2
 Background

 First-order derivative
f
 f '( x)  f ( x  1)  f ( x)
x

 Second-order derivative
 f
2
 f ( x  1)  f ( x  1)  2 f ( x)
x 2

12 13

Characteristics of First and Second

Order Derivatives Detection of Discontinuities
 First-order derivatives generally produce thicker  detect the three basic types of gray-
edges in image
level discontinuities
 Second-order derivatives have a stronger response to  points , lines , edges
fine detail, such as thin lines, isolated points, and noise
 the common way is to run a mask through
 Second-order derivatives produce a double-edge the image
response at ramp and step transition in intensity

 The sign of the second derivative can be used to

determine whether a transition into an edge is from
light to dark or dark to light
14 15

3
 Detection of Isolated Points Detection of Isolated Points

 The Laplacian 2  a point has been detected at the location

 f 2 f
 2 f ( x, y )   on which the mask is centered if
x 2 y 2
 f ( x  1, y )  f ( x  1, y )  f ( x, y  1)  f ( x, y  1) |R|  T
4 f ( x, y )  where
9  T is a nonnegative threshold
1 if | R( x, y) | T R   wk zk
g ( x, y)   k 1  R is the sum of products of the coefficients
0 otherwise with the gray levels contained in the region
encompassed by the mask.
16 17

Point Detection Example

 Note that the mask is the same as the
mask of Laplacian Operation
 The only differences that are considered
of interest are those large enough (as
determined by T) to be considered
isolated points.
|R|  T

18 19

4
 Line Detection
 Second derivatives to result in a stronger response and
to produce thinner lines than first derivatives
 Double-line effect of the second derivative must be
handled properly
20
Line Detection
 Apply every masks on the image
 let R1, R2, R3, R4 denotes the response of
the horizontal, +45 degree, vertical and
-45 degree masks, respectively.
 if, at a certain point in the image
|Ri| > |Rj|,
 for all j  i, that point is said to be more likely associated
with a line in the direction of mask i.
22
Line Detection
 Horizontal mask will result with max response when a
line passed through the middle row of the mask with a
constant background.
 the similar idea is used with other masks.
 note: the preferred direction of each mask is weighted
with a larger coefficient (i.e.,2) than other possible
directions.
21
Line Detection
 Alternatively, if we are interested in detecting
all lines in an image in the direction defined by a
given mask, we simply run the mask through the
image and threshold the absolute value of the
result
 The points that are left are the strongest
responses, which, for lines one pixel thick,
correspond closest to the direction defined by
the mask
23
5
 Edge Detection
 the most common approach for detecting
meaningful discontinuities in gray level.
 we discuss approaches for implementing
 first-order derivative (Gradient operator)
 second-order derivative (Laplacian operator)
 Here, we will talk only about their
properties for edge detection.
 we have introduced both derivatives

24 25

Basic Formulation Ideal and Ramp Edges

 an edge is a set of connected pixels that
lie on the boundary between two regions.
 an edge is a “local” concept whereas a
region boundary, owing to the way it is
defined, is a more global idea.

because of optics,
sampling, image
acquisition
26 imperfection 27

6
 Thick edge First and Second derivatives
 The slope of the ramp is inversely proportional to the
degree of blurring in the edge.
 We no longer have a thin (one pixel thick) path.
 Instead, an edge point now is any point contained in the
ramp, and an edge would then be a set of such points
that are connected.
 The thickness is determined by the length of the ramp.
 The length is determined by the slope, which is in turn
determined by the degree of blurring.
the signs of the derivatives
 Blurred edges tend to be thick and sharp edges
would be reversed for an edge
tend to be thin
that transitions from light to
dark
28 29

Second derivatives Zero-crossing

 produces 2 values for every edge in an  quite useful for locating the centers of
image (an undesirable feature) thick edges
 an imaginary straight line joining the
extreme positive and negative values of
the second derivative would cross zero
near the midpoint of the edge. (zero-
crossing property)

30 31

7
 Noise Images Keep in mind
First column: images and

gray-level profiles of a  fairly little noise can have such a
ramp edge corrupted by significant impact on the two key
random Gaussian noise of
mean 0 and  = 0.0, 0.1, derivatives used for edge detection in
1.0 and 10.0,
respectively. images
 Second column: first-
derivative images and
 image smoothing should be serious
gray-level profiles. consideration prior to the use of
 Third column : second- derivatives in applications where noise is
derivative images and
gray-level profiles. likely to be present.
32 33

Edge point
 to determine a point as an edge point  Basic Edge Detection by Using First-
 the transition in grey level associated with Order Derivative
the point has to be significantly stronger
than the background at that point.
 use threshold to determine whether a value
is “significant” or not.
 the point’s two-dimensional first-order
derivative must be greater than a specified
threshold.

34 35

8
  f 
Gx   x 
f      f 
Gradient Operator G y   
 y 
Gradient direction
 first derivatives are implemented using  Let  (x,y) represent the direction angle
the magnitude of the gradient. of the vector f at (x,y)

 (x,y) = tan-1(Gy/Gx)
1
f  mag (f )  [Gx2  G y2 ] 2

commonly approx.
 f  2  f  2 
1
2  The direction of an edge at (x,y) is
       perpendicular to the direction of the
 x   y   gradient vector at that point
f  Gx  G y
the magnitude becomes nonlinear
36 37

38 39

9
 Example

40 42

Edge Detection Steps Using

Example Gradient

(i.e., sqrt is costly!)

43

10
Example (using Prewitt
operator) Another Example

d
I
dx
Note: in this example, the
divisions by 2 and 3 in the
computation of fx and fy
are done for normalization
purposes only
d
I
dy

Isotropic property of
Another Example (cont’d) gradient magnitude
22
d   d 
22

   I   I  • The magnitude of the gradient detects edges in all directions.

 dx   dy 
22
 dd   dd 
22
dd dd
II II   II  II 
dx
dx dy
dy dx
dx  dy
dy 

  Threshold  100

11
 Practical Issues Practical Issues (cont’d)
 Noise suppression-localization tradeoff. • Choice of threshold.
 Smoothing depends on mask size (e.g., depends gradient magnitude
on σ for Gaussian filters).
 Larger mask sizes reduce noise, but worsen

localization (i.e., add uncertainty to the location

of the edge) and vice versa.
smaller mask larger mask
low threshold high threshold

Practical Issues (cont’d)

Laplacian
• Edge thinning and linking. Laplacian operator
 2 f ( x, y )  2 f ( x, y )
(linear operator) 2 f  
x 2 y 2
 2 f  [ f ( x  1, y)  f ( x  1, y)
 f ( x, y  1)  f ( x, y  1)  4 f ( x, y)]

53

12
 positive central term
surrounded by an adjacent negative region (a function of distance)
zero outer region
Laplacian of Gaussian Mexican hat
 Laplacian combined with smoothing as a
precursor to find edges via zero-
crossing.
where r2 = x2+y2, and
r2  is the standard deviation

h( r )  e 2 2

r2
 r 2   2   2 2
 2 h( r )    e
 
4
 54
the coefficient must be sum to zero
55

Linear Operation Example

 second derivation is a linear operation
a). Original image
 thus, 2f is the same as convolving the b). Sobel Gradient
image with Gaussian smoothing function c). Spatial Gaussian
smoothing function
first and then computing the Laplacian of d). Laplacian mask
the result e). LoG
f). Threshold LoG
g). Zero crossing

56 57

13
 Zero crossing & LoG Zero crossing vs. Gradient
 Approximate the zero crossing from LoG  Attractive
image  Zero crossing produces thinner edges
 to threshold the LoG image by setting all  Noise reduction
its positive values to white and all  Drawbacks
negative values to black.  Zero crossing creates closed loops.
 the zero crossing occur between positive (spaghetti effect)
and negative values of the thresholded  sophisticated computation.
LoG.  Gradient is more frequently used.

58 59

Advanced Techniques for Edge

Detection
 The Marr-Hildreth edge detector
x2  y 2

G ( x, y )  e 2 2
,  : space constant.
Laplacian of Gaussian (LoG)
 2 G ( x, y )  2 G ( x , y )
 2 G ( x, y )  
x 2 y 2
  x      y  x  y2 
x2  y 2 2 2

  e 2 2
   2 e 2 
x   2  y   
x2  y 2 x2  y 2
 x2 1    y2 1  
  4  2 e 2 2
  4  2 e 2 2

     
x2  y 2
 x2  y2   2  
 e
2 2

 4  60 61

14
 Marr-Hildreth Algorithm
1. Filter the input image with an nxn Gaussian lowpass
filter.

2. Compute the Laplacian of the image resulting from

step1

3. Find the zero crossing of the image from step 2

g ( x, y)  2 G( x, y) f ( x, y)
62 63

The Canny Edge Detector The Canny Edge Detector: Algorithm (1)
 Optimal for step edges corrupted by white noise. Let f ( x, y ) denote the input image and
G ( x, y ) denote the Gaussian function:
 The Objective
x2  y 2

1. Low error rate G ( x, y )  e 2 2

The edges detected must be as close as possible to the true

edge

2. Edge points should be well localized

We form a smoothed image, f s ( x, y ) by
The edges located must be as close as possible to the true edges convolving G and f :
3. Single edge point response f s ( x, y )  G ( x, y ) f ( x, y )
The number of local maxima around the true edge should be
minimum
64 65

15
 The Canny Edge Detector: Algorithm(2)
Compute the gradient magnitude and direction (angle):
M ( x, y )  g x 2  g y 2
and
 ( x, y )  arctan( g y / g x )
where g x  f s / x and g y  f s / y
Note: any of the filter mask pairs in Fig.10.14 can be used
to obtain g x and g y
66
The Canny Edge Detector: Algorithm(3)
The gradient M ( x, y ) typically contains wide ridge around
local maxima. Next step is to thin those ridges.
Nonmaxima suppression:
Let d1 , d 2 , d3 , and d 4 denote the four basic edge directions for
a 3  3 region: horizontal, -45o , vertical,+45 , respectively.
1. Find the direction d k that is closest to  ( x, y ).
2. If the value of M ( x, y ) is less than at least one of its two
neighbors along d k , let g N ( x, y )  0 (suppression);
otherwise, let g N ( x, y )  M ( x, y )
67

The Canny Edge Detector: Algorithm(4)

The final operation is to threshold g N ( x, y) to reduce
false edge points.

Hysteresis thresholding:
g NH ( x, y )  g N ( x, y )  TH
g NL ( x, y )  g N ( x, y )  TL

68 69

16
 The Canny Edge Detector: Algorithm(5) The Canny Edge Detection: Summary
Depending on the value of TH , the edges in g NH ( x, y )  Smooth the input image with a Gaussian filter
typically have gaps. Longer edges are formed using
the following procedure:  Compute the gradient magnitude and angle images
(a). Locate the next unvisited edge pixel, p, in g NH ( x, y ).
(b). Mark as valid edge pixel all the weak pixels in g NL ( x, y )  Apply nonmaxima suppression to the gradient
magnitude image
that are connected to p using 8-connectivity.
(c). If all nonzero pixel in g NH ( x, y ) have been visited go to  Use double thresholding and connectivity analysis to
step (d), esle return to (a). detect and link edges
(d). Set to zero all pixels in g NL ( x, y ) that were not marked as
valid edge pixels. 70 71

TL  0.04; TH  0.10;  4 and a mask of size 25  25

TL  0.05; TH  0.15;  2 and a mask of size 1313
72 73

17
 74 75

Edge Linking and Boundary

Detection Local Processing
 edge detection algorithm are followed by  analyze the characteristics of pixels in a
linking procedures to assemble edge small neighborhood (say, 3x3, 5x5) about
pixels into meaningful edges. every edge pixels (x,y) in an image.
 Basic approaches  all points that are similar according to a
 Local Processing set of predefined criteria are linked,
 Global Processing forming an edge of pixels that share
those criteria.

76 77

18
 Criteria
1. the strength of the response of the
gradient operator used to produce the
edge pixel
 an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar
in magnitude to the pixel at (x,y) if
Criteria
2. the direction of the gradient vector
 an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar
in angle to the pixel at (x,y) if

|(x,y) -  (x0,y0) | < A

|f(x,y) - f (x0,y0) |  E

78 79

find rectangles whose sizes makes

them suitable candidates for license
Criteria Example plates

 A point in the predefined neighborhood of (x,y)

is linked to the pixel at (x,y) if both magnitude
and direction criteria are satified.
 the process is repeated at every location in the
image • use horizontal and
 a record must be kept vertical Sobel operators
 simply by assigning a different gray level to • eliminate isolated short
each set of linked edge pixels. segments
• link conditions:
gradient value > 25
80 gradient direction differs < 15 81

19
 Hough Transformation (Line) Accumulator cells
 (amax, amin) and (bmax, bmin) are
the expected ranges of slope
and intercept values.
 all are initialized to zero
 if a choice of ap results in
solution bq then we let
A(p,q) = A(p,q)+1
xy-plane ab-plane or parameter space  at the end of the procedure,
yi =axi + b b = - axi + yi value Q in A(i,j) corresponds
to Q points in the xy-plane
all points (xi ,yi) contained on the same line must have lines in lying on the line y = aix+bj b = - axi + yi
parameter space that intersect at (a’,b’) 82 83

-plane

-plane

x cos + y sin = 
 = 90 measured with respect to x-axis
 problem of using equation y = ax + b is that value of a is infinite
for a vertical line.
 To avoid the problem, use equation x cos + y sin  =  to
represent a line instead.
 vertical line has  = 90 with  equals to the positive y-intercept
or  = -90 with  equals to the negative y-intercept
84 85

20
 Generalized Hough
Transformation
 can be used for any function of the form
g(v,c) = 0
 v is a vector of coordinates
 c is a vector of coefficients
86
Edge-linking based on Hough
Transformation
1. Compute the gradient of an image and
threshold it to obtain a binary image.
2. Specify subdivisions in the -plane.
3. Examine the counts of the accumulator
cells for high pixel concentrations.
4. Examine the relationship (principally for
continuity) between pixels in a chosen
cell.
88
Hough Transformation (Circle)
 equation:
(x-c1)2 + (y-c2)2 = c32
 three parameters (c1, c2, c3)
 cube like cells
 accumulators of the form A(i, j, k)
 increment c1 and c2 , solve of c3 that satisfies
the equation
 update the accumulator corresponding to the
cell associated with triplet (c1, c2, c3)
87
Continuity
 based on computing the distance between
disconnected pixels identified during
traversal of the set of pixels
corresponding to a given accumulator cell.
 a gap at any point is significant if the
distance between that point and its
closet neighbor exceeds a certain
threshold.
89
21
 Thresholding
image with dark image with dark
background and background and
a light object two light objects

link criteria:
1). the pixels belonged to one of the set of pixels
linked according to the highest count
2). no gaps were longer than 5 pixels
90 91

Multilevel thresholding Thresholding

 a point (x,y) belongs to • When T depends only on f(x,y) 
 to an object class if T1 < f(x,y)  T2 global threshold
 to another object class if f(x,y) > T2
 to background if f(x,y)  T1 • When T depends on both f(x,y) and p(x,y) 
 T depends on local threshold
 only f(x,y) : only on gray-level values  Global
threshold • When T depends on x and y (in addition) 
 both f(x,y) and p(x,y) : on gray-level values and its dynamic threshold
neighbors  Local threshold

92

22
 easily use global thresholding
object and background are separated

The Role of Illumination Illumination compensation

f(x,y) = i(x,y) r(x,y)  compensate nonuniformity by projecting
the illumination pattern onto a constant,
a). computer generated white reflective surface.
reflectance function
b). histogram of  thus,
reflectance function g(x,y) = ki(x,y)
c). computer generated
illumination function  where k is a constant that depends on
(poor) the surface and i(x,y) is the illumination
d). product of a). and c). pattern
e). histogram of product
image
difficult to segment 94 95

Illumination compensation Basic Global Thresholding

 for any image f(x,y) = i(x,y) r(x,y)
obtained with the same illumination
function
 simply dividing f(x,y) by g(x,y)
 yields a normalized function h(x,y)
h(x,y) = f(x,y)/g(x,y) = r(x,y)/k use T midway
 thus, we can use a single threshold T/k between the max
and min gray levels
to segment h(x,y)
generate binary image
96 97

23
 Basic Global Thresholding Example: Heuristic method
 based on visual inspection of histogram note: the clear valley
1. Select an initial estimate for T. of the histogram and
2. Segment the image using T. This will produce two the effective of the
groups of pixels: G1 consisting of all pixels with gray segmentation
level values > T and G2 consisting of pixels with gray between object and
level values  T
background
3. Compute the average gray level values 1 and 2 for
the pixels in regions G1 and G2
4. Compute a new threshold value
T = 0.5 (1 + 2) T0 = 0
5. Go to [2] until the difference in values of T is small 3 iterations
enough. with result T = 125

98 99

Basic Adaptive Thresholding Example : Adaptive Thresholding

 subdivide original image into small areas.
 utilize a different threshold to segment
each subimages.
 since the threshold used for each pixel
depends on the location of the pixel in
terms of the subimages, this type of
thresholding is adaptive.

100 101

24
 Optimal Global and Adaptive
Further subdivision Thresholding
a). Properly and improperly
segmented subimages from previous
example
b)-c). corresponding histograms
d). further subdivision of the
improperly segmented subimage.
e). histogram of small subimage at
top
f). result of adaptively segmenting d).

p( z )  P1 p1 ( z )  P2 p2 ( z )
102
P1  P2  1 103

Probability of erroneously Minimum error

T
E1 (T )   p ( z )dz

2
Differentiating E(T) with respect to T (using Leibniz’s
rule) and equating the result to 0

dE (T ) d ( P2 E1 (T )  P1 E2 (T ))
T  0
E2 (T )   p ( z )dz

1
dT dT if P = P then 1 2
the optimum threshold
is where the curve
p1(z) and p2(z) intersect
E(T )  P2 E1 (T )  P1E2 (T ) find T which makes

P1 p1 (T )  P2 p2 (T )
104 105

25
 Gaussian density Quadratic equation
Example: use PDF = Gaussian density : p1(z) and p2(z)
AT 2  BT  C  0
where A   12   22
p( z )  P1 p1 ( z )  P2 p2 ( z )
( z  1 ) 2 ( z 2 )2 B  2( 1 22   2 12 )
 
P1 P2
 e 2 12
 e 2 22
C   12  22   22 12  2 12 2 22 ln(  2 P1 /  1 P2 )
where 2  1 2  2 if P1 = P2 or  = 0
• 1 and 12 are the mean and variance of the Gaussian 1   2 2 P  then the optimal
density of one object T  ln  2  threshold is the
• 2 and 22 are the mean and variance of the Gaussian 2 1   2  P1  average of the
density of the other object 106 means 107

Example Histogram of the example

108 109

26
 Boundary Characteristic for
Example: Histogram Improvement and Local
Boundary superimposed Thresholding
light object of dark background

0 if f  T

s ( x , y )   if f  T and  2 f  0
 if f  T and  2 f  0

 local processing due to using gradient of each small area
 (…)(-,+)(0 or +)(+,-)(…)
 all pixels that are not on an edge are labeled 0
 all pixels that are on the dark side of an edge are
labeled +
 all pixels that are on the light side an edge are labeled -
110 111

Example Histogram of Gradient

select T at the midpoint of the valley

112 113

27
 Result of applying T Region-Based Segmentation
 Basicn Formulation
(a)  Ri  R
i 1

(b) Ri is a connected region, i  1, 2, ..., n

(c) Ri  R j   for all i and j, i  j
(d ) P(Ri )  TRUE for i  1, 2, ..., n
(e) P(Ri  R j )  FALSE for i  j
P(Ri) is a logical predicate property defined over the points in set R i
114 ex. P(Ri) = TRUE if all pixel in Ri have the same gray level 115

select all seed points with gray level 255

Region Growing Region Growing

 start with a set of “seed” points criteria:
 growing by appending to each seed those 1. the absolute gray-
level difference
neighbors that have similar properties between any pixel
such as specific ranges of gray level and the seed has to
be less than 65
2. the pixel has to be
8-connected to at
least one pixel in
that region (if more,
the regions are
116 merged) 117

28
 Histogram of fig 10.40 a) Region splitting and merging

used to find the

criteria of the
difference gray-level Quadtree
between each pixels 1. Split into 4 disjoint quadrants any region R i for which
and the seeds P(Ri) = FALSE
2. Merge any adjacent region Rj and Rk for which
P(Ri  Rk ) = TRUE
3. Stop when no further merging or splitting is possible.
118 119

Example

P(Ri) = TRUE if at least 80% of the pixels in Ri have the

property |zj-mi|  2i,
where
zj is the gray level of the jth pixel in Ri
mi is the mean gray level of that region
i is the standard deviation of the gray levels in R i
120

29