Chapter10 2

Digital Image Processing
Chapter 10:
Image Segmentation
Element of Image Analysis
Preprocess
Image acquisition, restoration, and enhancement
Intermediate process
Image segmentation and feature extraction
High level process

Image interpretation and recognition
ISAN-DSP GROUP
Importance of Image Segmentation
Image segmentation is used to separate an image into constituent
parts based on some image attributes. Image segmentation is an
important step in image analysis
Benefit
1. Image segmentation reduces huge amount of unnecessary
data while retaining only importance data for image analysis
2. Image segmentation converts bitmap data into better
structured data which is easier to be interpreted
Image Attributes for Image Segmentation
. Similarity properties of pixels inside the object are used to group
ixels into the same set.
. Discontinuity of pixel properties at the boundary between object
nd background is used to distinguish between pixels belonging to
he object and those of background.
Discontinuity:
Intensity change
at boundary
Similarity:
Internal
pixels share
the same
intensity
Spatial Filtering Application to Shape Detection
 One application of spatial filtering is shape detection: finding

locations of objects with the desired shape.
 Unlike frequency selective masks that are designed based

on the concept of frequency, shape detection masks are
derived from the shapes to be detected themselves.
 A mask for shape detection usually contains the shape or a part

of the shape to be detected.
 The location that is most correlated to the mask is the

location where the highest filter response occurs. The
shape is most likely to exist there.
Detection of Discontinuities
• There are three kinds of discontinuities of intensity: points,

lines and edges.
• The most common way to look for discontinuities is to scan a
small mask over the image. The mask determines which kind
of discontinuity to look for.
9
R  w1 z1  w2 z 2  ...  w9 z9   wi zi
i 1
Point Detection
 We can use Laplacian masks
-1 -1 -1 0 -1 0
for point detection.
-1 8 -1 -1 4 -1
 Laplacian masks have the largest -1 -1 -1 0 -1 0
coefficient at the center of the mask
while neighbor pixels have an
opposite sign.
 This mask will give the high response to the object that has the
similar shape as the mask such as isolated points.
 Notice that sum of all coefficients of the mask is equal to zero.

This is due to the need that the response of the filter must be zero
inside a constant intensity area
Point Detection
Point detection can be done by applying the thresholding function:
1 f ( x , y )  T
g ( x, y )  
0 otherwise
Location of
porosity
X-ray image of the Laplacian image After thresholding
turbine blade with
porosity (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Line Detection
 Similar to point detection, line detection can be performed

using the mask the has the shape look similar to a part of a line
 There are several directions that the line in a digital image can be.
 For a simple line detection, 4 directions that are mostly used are
Horizontal, +45 degree, vertical and –45 degree.
Line detection masks

(Images from Rafael C. Gonzalez and Richard E.
Line Detection Example
Binary wire
bond mask
image
Absolute value
of result after Result after
processing with thresholding
-45 line detector
Notice that –45 degree

lines are most sensitive
(Images from Rafael C.

Gonzalez and Richard E.
Wood, Digital Image
Processing, 2nd Edition.
Edges
Ideal step edge Ideal ramp edge
Blurred edge
Generally, objects and background have different intensities. Therefore,

Edges of the objects are the areas where abrupt intensity changes occur.

Ideal Ramp Edges and its Derivatives
Edge
Original image

Smoothed Step Edge and Its Derivatives
1.2
Edge
Intensity
1
0.8
Edge
Gray level profile 0.6
0.4
0.2
0
-0.2
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
0.06 Minimum
0.04
0.02 point
The 1st derivative 0
-0.02
-0.04 Maximum
-0.5point
-0.06
-1.5 -1 0 0.5 1 1.5 2 2.5 3 3.5
x 10-3 Zero
5
4
3
crossing
2
1
+ +
The 2nd derivative 0
-1 - -
-2
-3
-4
-5
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
Derivative Based Edge Detection
 From the previous slide, we can conclude that:

Local maxima of the absolute of the 1st derivative and Zero crossing
of the 2nd derivative occur at edges.
 Therefore, for detecting edges, we can apply zero crossing detection

to the 2nd derivative image or thresholding the absolute of the
1st derivative image.
 Nevertheless, derivative operator is very sensitive to noise as we

will see in the next slide.
Noisy Edges and Derivatives
df d2 f Derivative operator is
f(x) a highpass filter and
dx dx 2
thus enhances noise.
AWGN = 0.1
Edge responses
AWGN = 1.0
are buried by
noise.
AWGN = 10
Gradient Operators
• First-order derivatives:
– The gradient of an image f(x,y) at location (x,y) is defined
as the vector:
G x   f
x

f      f 
G y   y 
– The magnitude of this vector: f  mag(f )  G  G  2
x
2
y 
1
2
 Gx 
– The direction of this vector:  ( x, y )  tan  
1
 Gy 
Masks for Estimating Partial Derivatives
Normally, the mask for estimating partial derivative is anti-

symmetry with respect to the orthogonal axis
For example, the Sobel mask for
computing f is anti-symmetry
x
with respect to the y-axis. It has the
positive sign on the right side and
negative sign on the left side.
Notice that sum of all coefficients is

equal to zero to make sure that the
response of a constant intensity area
is zero.
Masks for Detecting Diagonal Edges
The mask for detecting -45-degree

edges is anti-symmetry with respect to
the –45-degree lines while the mask for
detecting 45-degree edges is
anti-symmetry with respect to the
45-degree lines.

Example of Image Gradient
f ( x, y ) f
y
f f f

x x y

Example of Image Gradient
f ( x, y ) f
y
f f f

x x y
Note: the original image is smoothed by a 5x5 moving average

mask first. (Images from Rafael C. Gonzalez and Richard E.
Example of Diagonal Edges
Using -45-degree mask Using 45-degree mask
Note: the original image is smoothed by a 5x5 moving average

mask first. (Images from Rafael C. Gonzalez and Richard E.
Laplacian Masks
The Laplacian masks are used to estimate the Laplacian image:
2P 2P
 P 2  2
2
x y
Ideally, the Laplacian mask must be directional invariant: symmetry in

all direction (radially symmetry ). However, for 3x3 masks, there are
Only 8 possible directions. Hence, we can use the following masks:

Laplacian Masks
For a large scale Laplacian mask, we can use a Laplacian of Gaussian
(LOG) as a mask:
 x2  y2 
 x  y  
2 2 2 
 2
2


 G ( x, y )   
2
 e 
  4

Surface plot of
LOG, Looks like
a “Mexican hat” LOG image
Cross section 5x5 LOG

of LOG mask
Example of Laplacian Image
The angiogram image
(blood vessels)
P
We can compute the
Laplacian image by:
1. Smooth the image
by the Gaussian mask 1
2
2. Compute the Laplacian
image using the mask
Threshold of Zero crossing
The left image
 2G * P

Pixel Oriented Image Segmentation: Thresholding
It is a way of separating the elements of an image by looking at
the similarity of the internal pixel properties. Space alone like the way
Intensity Thresholding
1 f ( x, y )  T
g ( x, y )  
0 f ( x, y )  T
T = 102
500
Background
400
300
bacteria
200
100
0 50 100 150 200 250

After thresholding
Intensity Thresholding Example

Automatic Threshold Level Selection
The major problem of intensity thresholding is to find a good
threshold level
Algorithm: effective for bimodal histogram
1. Set initial value of T
2. T1  Average ( p( x, y ) p( x, y )  T )
3. T2  Average ( p( x, y ) p( x, y )  T )
T1  T2
4. T 
2
5. Repeat step 2
Automatic Threshold Level Selection Example

Multilevel Intensity Thresholding
In some cases, there may be multiple Threshold Levels.

Histogram
1500
T1 = 158
1000
T2 = 196
500
T3 = 228
0
0 50 100 150 200 250
T1< P <T2 T2< P <T3 P > T3

Noise Problem
When used with scrambled images. It will make a discrete image
peak unclear
Image degraded by Histogram
Gaussian noise ( =12) 250
200
T1 = 158
150
T2 = 196
100
50
T3 = 228
0
0 50 100 150 200 250
T1< P <T2 T2< P <T3 P > T3

Nonuniform Illumination Problem
An image can be expressed as
Reflectance
f ( x, y )  i ( x, y ) r ( x, y )
Function r(x,y)
i(x,y) = illumination component
Histogram r(x,y) = reflectance component
Illumination
Function i(x,y)
Image histogram
f(x,y)

Nonuniform Illumination and Global Thresholding
Global threshold level

Global thresholding of nonuniform
illumination image can cause huge
errors!
Histogram
Nonuniform illumination Global thresholding

image result
Nonuniform Illumination and Local Thresholding
Local thresholding:
. Divide an image into subimages.
. Threshold each subimage independently
2.1 Compute histogram of each subimage and select a suitable
threshold value for each subimage
2.2 threshold each subimage using a threshold value in 2.1
2.3 Combine all local thresholding results
Error
16 subimages Result of local thresholding

Histogram of Subimages and Local Thresholding
If areas of object and
background are not
balanced, a histogram
will be unimodal.
If areas of object and

background are nearly
equal, a histogram will
be bimodal

Example of Nonuniform Illumination Problem
When using images, the brightness is not the same throughout
the image. The results will be very wrong.
900
800
700
600
500
400
300
200
100
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
T=0.4
Error Edge of image

ISAN-DSP GROUP
Optimum Thresholding
Background
p1(z) = PDF of object pixels
Object p2(z) = PDF of background pixels
Error due to background pixels E1 (T )   p ( z )dz

2
classified as object pixels is : 

Error due to object pixels
E (T )   p1 ( z )dz
classified as background pixels is: 2
T
Total error = E (T )  P2 E1 (T )  P1E2 (T )
P1 = Probability of occurrence of object pixels

P2 = Probability of occurrence of background pixels


Edge Linking and Boundary Detection
Local Processing
• Two properties of edge points are useful for edge linking:

– the strength (or magnitude) of the detected edge points
– their directions (determined from gradient directions)
• This is usually done in local neighborhoods.
• Adjacent edge points with similar magnitude and direction are
linked.
• For example, an edge pixel with coordinates (x0,y0) in a
predefined neighborhood of (x,y) is similar to the pixel at (x,y)
if
f ( x, y )  ( x0 , y0 )  E , E : a nonnegative threshold
 ( x, y )   ( x0 , y0 )  A, A : a nonegative angle threshold
Local Processing: Example
In this example,
we can find the
license plate
candidate after
edge linking
process.
Global Processing via the Hough Transform
• Hough transform: a way of finding edge points in an image

that lie along a straight line.
• Example: xy-plane v.s. ab-plane (parameter space)
yi  axi  b
Global Processing via the Hough Transform
• The Hough transform consists of

finding all pairs of values of 
and  which satisfy the equations
that pass through (x,y).
• These are accumulated in what is
basically a 2-dimensional
histogram.
• When plotted these pairs of  and
 will look like a sine wave. The
process is repeated for all x cos   y sin   
appropriate (x,y) locations.
Hough Transform Example
The intersection of the
curves corresponding
to points 1,3,5
2,3,4
1,4
Hough Transform Example
Region-Based Segmentation
• Edges and thresholds sometimes do not give good

results for segmentation.
• Region-based segmentation is based on the
connectivity of similar pixels in a region.
– Each region must be uniform.
– Connectivity of the pixels within the region is very
important.
• There are two main approaches to region-based
segmentation: region growing and region splitting.
Basic Formulation
• Let R represent the entire image region.

• Segmentation is a process that partitions R into subregions,
R1,R2,…,Rn, such that
n
(a)  Ri  R
i 1
(b) Ri is a connected region, i  1,2,..., n
(c) Ri  R j   for all i and j , i  j
(d) P ( Ri )  TRUE for i  1,2,..., n
(e) P ( Ri  R j )  FALSE for any adjacent regions Ri and R j
where P(Rk): a logical predicate defined over the points in set Rk
For example: P(Rk)=TRUE if all pixels in Rk have the same gray
level.
Region Growing
Region Growing
• Fig. 10.41 shows the histogram of Fig. 10.40 (a). It is difficult

to segment the defects by thresholding methods. (Applying
region growing methods are better in this case.)
Figure 10.40(a) Figure 10.41

Region Splitting and Merging
• Region splitting is the opposite of region growing.

– First there is a large region (possible the entire image).
– Then a predicate (measurement) is used to determine if the
region is uniform.
– If not, then the method requires that the region be split into
two regions.
– Then each of these two regions is independently tested by
the predicate (measurement).
– This procedure continues until all resulting regions are
uniform.
Region Splitting
• The main problem with region splitting is determining where to

split a region.
• One method to divide a region is to use a quadtree structure.
• Quadtree: a tree in which nodes have exactly four descendants.
Region Splitting and Merging
• The split and merge procedure:

– Split into four disjoint quadrants any region Ri for which
P(Ri) = FALSE.
– Merge any adjacent regions Rj and Rk for which P(RjURk) =
TRUE. (the quadtree structure may not be preserved)
– Stop when no further merging or splitting is possible.
Segmentation by Morphological Watersheds
• The concept of watersheds is based on visualizing an image in

three dimensions: two spatial coordinates versus gray levels.
• In such a topographic interpretation, we consider three types
of points:
– (a) points belonging to a regional minimum
– (b) points at which a drop of water would fall with
certainty to a single minimum
– (c) points at which water would be equally likely to fall to
more than one such minimum
• The principal objective of segmentation algorithms based on
these concepts is to find the watershed lines.
Example
Example
Example
The Use of Motion in Segmentation
• ADI: accumulative difference image

The Use of Motion in Segmentation
Region Growing Image Segmentation Example
Histogram
X-ray image of defective weld

Region Growing Image Segmentation Example

Quadtree for Region Splitting Representation

Region Splitting Algorithm
Region Splitting
ISAN-DSP GROUP
Region Splitting and Merging Example

Smoothed Step Edge and Its Derivatives
1.2
Edge
Intensity
1
0.8
Edge
Gray level profile 0.6
0.4
0.2
0
-0.2
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
0.06 Minimum
0.04
0.02 point
The 1st derivative 0
-0.02
-0.04 Maximum
-0.5point
-0.06
-1.5 -1 0 0.5 1 1.5 2 2.5 3 3.5
x 10-3 Zero
5
4
3
crossing
2
1
+ +
The 2nd derivative 0
-1 - -
-2
-3
-4
-5
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5
Derivative Based Edge Detection
 From the previous slide, we can conclude that:

Local maxima of the absolute of the 1st derivative and Zero crossing
of the 2nd derivative occur at edges.
 Therefore, for detecting edges, we can apply zero crossing detection

to the 2nd derivative image or thresholding the absolute of the
1st derivative image.
 Nevertheless, derivative operator is very sensitive to noise as we

will see in the next slide.
Masks for Estimating Partial Derivatives
Normally, the mask for estimating partial derivative is anti-

symmetry with respect to the orthogonal axis
For example, the Sobel mask for
computing f is anti-symmetry
x
with respect to the y-axis. It has the
positive sign on the right side and
negative sign on the left side.
Notice that sum of all coefficients is

equal to zero to make sure that the
response of a constant intensity area
is zero.
Boundary Extraction Based on Particle motion in a Vector Field
Gradient field Tangential velocity field
Gradient Vector Field Edge Vector Field

(Hamiltonian Gradient Field)
P ˆ P ˆ P ˆ P ˆ
P  i j P   i j
x y y x
Edge Linking

Chapter10 2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter10 2

Uploaded by

Copyright:

Available Formats

Digital Image Processing

High level process

 One application of spatial filtering is shape detection: finding

 Unlike frequency selective masks that are designed based

 A mask for shape detection usually contains the shape or a part

 The location that is most correlated to the mask is the

• There are three kinds of discontinuities of intensity: points,

 Notice that sum of all coefficients of the mask is equal to zero.

 Similar to point detection, line detection can be performed

Line detection masks

Notice that –45 degree

(Images from Rafael C.

Generally, objects and background have different intensities. Therefore,

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

 From the previous slide, we can conclude that:

 Therefore, for detecting edges, we can apply zero crossing detection

 Nevertheless, derivative operator is very sensitive to noise as we

AWGN = 0.1

Normally, the mask for estimating partial derivative is anti-

Notice that sum of all coefficients is

The mask for detecting -45-degree

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

Note: the original image is smoothed by a 5x5 moving average

Using -45-degree mask Using 45-degree mask

Note: the original image is smoothed by a 5x5 moving average

Ideally, the Laplacian mask must be directional invariant: symmetry in

(Images from Rafael C. Gonzalez and Richard E.

Cross section 5x5 LOG

(Images from Rafael C. Gonzalez and Richard E.

0 50 100 150 200 250

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

In some cases, there may be multiple Threshold Levels.

0 50 100 150 200 250

T1< P <T2 T2< P <T3 P > T3

0 50 100 150 200 250

T1< P <T2 T2< P <T3 P > T3

(Images from Rafael C. Gonzalez and Richard E.

Global threshold level

Nonuniform illumination Global thresholding

16 subimages Result of local thresholding

If areas of object and

(Images from Rafael C. Gonzalez and Richard E.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Error Edge of image

Error due to background pixels E1 (T )   p ( z )dz

Total error = E (T )  P2 E1 (T )  P1E2 (T )

P1 = Probability of occurrence of object pixels

(Images from Rafael C. Gonzalez and Richard E.

(Images from Rafael C. Gonzalez and Richard E.

• Two properties of edge points are useful for edge linking:

• Hough transform: a way of finding edge points in an image

• The Hough transform consists of

• Edges and thresholds sometimes do not give good

• Let R represent the entire image region.

• Fig. 10.41 shows the histogram of Fig. 10.40 (a). It is difficult

Figure 10.40(a) Figure 10.41

• Region splitting is the opposite of region growing.

• The main problem with region splitting is determining where to

• The split and merge procedure:

• The concept of watersheds is based on visualizing an image in