Digital Image Processing Chapter 8 Image Analysis and Pattern Recognition

Digital image processing Chapter 8 Image analysis and pattern recognition
IMAGE ANALYSIS AND PATTERN RECOGNITION

Introduction
Feature extraction:
- spatial feature extraction
- transform feature extraction
- edge feature extraction; edge detection
Objects representation by their boundaries:
- contour extraction
- contour descriptors
Objects representation by their regions:
- region extraction
- region representation
Shapes and structures for region-based object representation:
- object skeletons
- binary morphology
- shape descriptors (numerical shape descriptors)
Textures; texture analysis
Image segmentation
Grey level based segmentation/color based segmentation
Connected components analysis
Contour-based segmentation
Region-based segmentation
Mixed techniques
Introduction Conclusion
Data analysis
of analysis
Input Feature Segmentation; ROI feature
Preprocessing extraction Classification;
image ROI selection extraction description
Image analysis
Image understanding
Symbolic Interpretation
representation & description
Fig. 8.1 Image analysis system – block diagram
Example image analysis tasks:
License plate recognition

Introduction
Text detection & recognition or removal
Trafic sign recognition Calcite localization/assessment Detect tissue on the slide

Introduction
Single object images; defect detection on the object
Ways to approach the image analysis task?

- “straight forward” approach: analyze the content of the whole image => one will
eventually find the entire information, including the information of interest – at too
high computational complexity/cost
- “smart” approach: focus strictly on the information you’re looking for! and extract
only this information from the image  focus on the part of the image containing the
information of interest.
• How?
1) describe somehow the main characteristics of the information of interest
 describe the discriminative features of the region containing the
information you look for, which are not present in the other regions =>
represent the image in the best feature space, by a feature map
E.g. on the images with apples – only interested in the defect area (if there is), and for this
defect – it appears darker grey than the normal apple coloration, but not black => design a
grey scale slicing to set to black everything else
New feature space
2) Analyze only the information in the region of interest , using the

previously defined feature space/feature map
e.g. Apples sort: no defects (defect area <2% of the apple); average defects;
large defects…
Feature extraction:
Spatial features extraction:
• Amplitude features: e.g. the brightness levels can identify regions of interest in the image:
Tissue identification
by color coding
(e.g. violet)
Can measure
afterwards the area,
describe the shape,
etc.
•Amplitude features may be discriminative enough if intensity is enough to distinguish wanted

info from the rest of the scene
=> defining the best parameters of the transformation for feature extraction – most difficult
=> amplitude feature space representation is not necessarily binary; just that unwanted
parts of the scenes should be represented uniquely (i.e. black) in the feature space
=> sometimes adaptive thresholding/adaptive grey scale slicing is needed.
• Histogram based features:

Local histogram = a local statistical description of the image;
If u = an image pixel; x=a grey level => pu(x)=the probability of appearance of the grey
level x in the image region = a value in the normalized histogram
=> One can compute: the standard deviation; the entropy; the median; percentiles, of pu(x).
Region Measurements
of interest
(ROI) ROI histogram
Tissue of interest
is well discriminated from
the microscopic slide by
the standard deviation
of the local histogram
Transform features extraction
Input Forward unitary v(k,l) v(k,l) Inverse unitary u(m,n)

x
image transform transform
Mask
g(k,l)
Fig.8.2 Transform features extraction

Edge features extraction. Edge detection
2 2
g(m,n) = g1 (m,n)+ g2 (m,n)
(u, H ) m,n    h( i, j ) u( i  m, j  n)  u( m, n)  h(  m, n)  g (m,n) = arctg
g 2 (m,n)
i j g1 (m,n)
g1(m,n) Magnitude Thresholding

h1(-m,-n) g  g12  g 22 1 Edge
u(m,n) g(m,n)
g 
 g  tan 1  2  0 t
detection
h2(-m,-n)  g1  Direction
g2(m,n)
g(m,n)
Fig. 8.3 Edge detection with gradient operators
1 0 -1 -1 - 2 -1
1 1
H x,S  2 0 -2, H y,S  0 0 0 (8.5)
4 4
1 0 -1 1 2 1
 0 1 1 0 
H1   , H2    (8.6)
 1 0 0  1
u(m,n) gk(m,n) Max{1 1} gradient g(m,n) Edge magnitude edge

hk(m,n)
k thresholding
detection
Fig. 8.4 Edge detection by compass operators
1 1 1 5 5 5 1 1 1 1 2 1
 1 2 1  3 0 3 0 0 0 0 0 0
       
1 1 1 3 3 3 1 1 1 1 2 1
a b c d
Fig. 8.5 Compass edge detectors (North direction)
1 1 1 1 1 0 1 0  1 0  1  1
       
0 0 0 1 0  1 1 0  1 1 0  1
 1  1  1 0  1  1 1 0  1 1 1 0 
N NW W SW
 1  1  1   1  1 0  1 0 1  0 1 1
0 0 0  1 0 1  1 0 1   1 0 1
       
 1 1 1   0 1 1  1 0 1  1  1 0
S SE E NE
Fig. 8.6 Compass operators on different directions

gk(m,n) –compass gradient on the direction k0,...,7,  k    k  ,

2 4
The gradient in the spatial position (m,n) is defined as:
g (m, n) = max{ g k (m, n) } (8.7)
k
Laplace operators and the zero-crossings edge localization method:
2 f 2 f
 f 
2
 (8.8)
 x2  y2
0 1 0   1 1  1  1 2 1 
(1) 1 4  1(2) 1 8  1(3) 2 4  2
 0 1 0   1 1  1  1 2 1 
df Thresholding Double contour

f(x) dx
d2 f
dx 2
Zero
a b crossing
c
Fig. 8.7 Edge detection by Laplace operators (the 1-D case)
The Gaussian filter:

1  m2  n2 
h(m, n)  exp  
 2 2 
2 2  
The Laplacian of Gaussian operator (LoG)

 ( m 2  n 2 )  22   m2  n 2 
h(m , n )  c exp  
  4
  22 
   
Gaussian filter The 2nd derivative

The derivative of the Gaussian
(Laplacian of Gaussian)
Discrete implementations:
Edge detection by different operators – comparison:
Original image Sobel edge detection LoG edge detection; sigma=5
Roberts
edge detection LoG edge
detection;
Sigma=10
Objects representation by their boundaries:

Contour extraction: 1
C
B
A A 2 4
3
a) b) c)
Fig. 8.8 4-connectivity; 8-connectivity
The Hough transform: s  x  sin  y  cos  s  x  cos  y  sin
y
s
s
θ Φ
x
The Hough transform of lines/random curves:
s s s
Approx. convergence =>

Curve convergence No convergence point =>
approx. straight line
on horizontal axis=> The points are not on
Straight line through the origin a straight line
Applications to contour extraction formed by line segments:
Edge detection
(gradient
Apply Hough
operator)
transform:
+ thresholding
Extracting the
=> Binary edge
contour of the
map
triangle
Input image Edge map; Lower left object =

Contours not yet extracted/ Defined by the 3 line
not yet labeled intersection points
8-connectivity:
Correct labeling
Contour extraction
in 8-connectivity
4-connectivity:
The contour is
“broken” during
labeling
Contour representation/contour descriptors:
• Goal: for a given object, described by its contour, find a compact description, by numerical
attributes, able to:
-Represent the contour with no significant loss of information (regenerative descriptors)
-Generate (by a subset of attributes) descriptions of the contour/shape invariant to: scaling;
rotation; translation; mirroring; projection distortions; small (limited) variations of shape
(among different individual representations of the same shape)
Using the contour descriptors, one can recognize the shape by template matching or shape
classification (contour descriptors classification)
• General assumption: single object contour (all edge pixels connected); 1-pixel width; no junction points
Chain codes and polygonal approximations:

2 D
3 1 C 4 B C
5 3
D A A
4 0
6 2
E H
5 7 E
7 1
6 F G B
0
a) b)
Fig. 8.9 Contour representation by chain codes Fig. 8.10 Polygonal approximation of the contours
Start point
Encode this contour

by a chain code with
8 directions,
Consider origin= the
upper left point; contour
following direction =
arbitrary; the only
possible positions are
on the grid corners
Original contour Contour approximation (“quantization”)

on a rectangular grid
Fourier descriptors:
(8.12)
u(n)  x(n)  j  y(n) for n  0,1,..., N  1
1 N 1  j 2 kn  (8.13)
u ( n)   a (k )exp ,0  n  N  1
N k 0  N 
(8.14)
N 1   j 2 kn 
a (k )   u (n)exp ,0  n  N  1
n 0  N 
(8.15)
u0  x 0  j y 0
Ax  By  C  0 (8.16)
leading to a new contour x'(n) y'(n), given by:

u'( n )  u *( n )e j 2  2 (8.17)
( A  jB)C ( A  jB) 2
where   , exp( j 2  )  (8.18)
A2  B 2 A2  B 2
Table 8.1
Transform The transformed contour The Fourier descriptors
Identity u(n) a(k)
Translation u'(n)u(n)u0 a'(k)a(k) u0 (k)
Scaling u'(n)u(n) a'(k)a(k)
Origin u'(n)u(n-n0) a'(k)a(k)e-j2n0k/N
Rotation u'(n)u(n)ej0 a'(k)a'(k)ej0
Symmetry towards
a straight line u'(n)u*(n)ej202 a'(k)a*(-k)ej22 (k)
Shape reconstruction from its Fourier descriptors:
Original Reconstruction using Reconstruction using

the first 2 descriptors only the first 6 descriptors
Reconstruction using Reconstruction using

the first 10 descriptors the first 20 descriptors
N 1 2
d ( u0 , , 0 , n0 )  min  u( n )   v ( n  n0 )e j 0  u0 
u0 , ,n0 , 0
 n 0 
 u( n)   v(n)  0
Model
shape Macthing results: d<Thd
,
then for a given translation u0, the distance d is minimal when:

u0  0
 c( k )cos( k  k  0 )
  k
 b( k )
2
k and
 c( k )  sin( k  k )
tg 0  k
 c( k )  cos(
k
k  k )
 2
d  min d ( )  min  a( k )   b( k ) exp j ( k   0 ) 
 
 k 
where a(k)b*(k)c(k)ejk,  -2n0/N and c(k) is a real valued term.
Shapes dictionary
Objects representation by their regions:

Region extraction: same as for contours (use connected components analysis, in 4- or 8- connectivity)
Region representation:
1 ,if m,n  
u(m,n)  
0 , otherwise
4 1
G
3 2
A
F A
I H
E B I H G E D C B
F
D C
4 3 2 1
Fig. 8.11 Quad-tree region representation
=> the quad-tree code : gbgbwwbwgbwwgbwwb ; decoding: g(bg(bwwb)wg(bwwg(bwwb)))

Shapes and structures for region-based object representation:

 Regions skeletons; medial axis transforms:
uk (m, n)  u0 (m, n)  min uk 1(i, j ); (i, j ):d (m, n; i, j )  1k  1,2,... 8.23)
d ( m, n;i , j )
u0 (m, n)  u (m, n)
(m,n) : uk(m,n)  uk(i,j) , d (m,n;i,j) 1 (8.24)
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 2 2 2 2 2 1 1 2 2 2 2 2 1 1 2 2 2 2 2 1
1 1 1 1 1 1 1 k
 
1 1 2 2 2 2 2 1 k
 
2 1 2 3 3 3 2 1 k
 
 3,4,5 1 2 3 3 3 2 1 

       
1 1 1 1 1 1 1 1 2 2 2 2 2 1 1 2 2 2 2 2 1 1 2 2 2 2 2 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
u0 (m, n) u1 (m, n) u 2 (m, n) u3 , u4 , u5
1 1  
2 2  
3 3 3   
2 2  
1 1  
Skeleton
a b c
Fig. 8.12 Skeleton extraction Fig. 8.13 Examples skeletons

P3 P2 P9 1 1 0 0 0 0 1 0 1
P4 P1 P8 1 P1 1 1 P1 0 0 P1 0
P5 P6 P7 0 0 0 0 0 0 1 1 1
a b
a. b. c. d.
Fig. 8.14 2-pass contour thinning: 2 logic rules: Fig. 8.15 Thinning result
R1: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P4·P6==0 && P2·P4·P8==0
R2: P1==1 && N(P1)==1 && 2<=NT(P1)<=6 && P2·P6·P8==0 && P4·P6·P8==0
 Morphological processing. Binary image morphology

Object Structuring Dilation
Object Structuring Erosion element
element
Origin
Origin
Object Structuring Contour Object Structuring Opening

element extraction element
Origin
Origin
Object Structuring Closing Object Structuring Skeletization

element element
Origin
Origin G
Object Element Thinning

Object Element Cleaning
structural
Origin Origin
structural
Object Structuring Thickening

element
Origin
Fig. 8.16 Morphological operations

Shape descriptors
Applications: shape recognition; quantitative measurements
Def.: The shape of an object = the profile of the object + its physical structure => “shape descriptors”
Classification:
(1) regenerative descriptors (contours; regions; high order statistics; structural and syntactic descriptors)
(2) geometrical shape descriptors (area, perimeter, max-min radii, eccentricity, corners, roundness, symmetry)
(3) moments
y
Geometrical features (geometrical descriptors):

2 2
  dx(t )   dy (t ) 
T   
Perimeter:
  dt
 dt   dt 
t – some contour parameter dy
y(t)
- Discrete => T = count of contour pixels
y(t+1)
x(t) x(t+1)
dx x
 Area:
dx (t ) dy( t )
A    dxdy   y( t )  dt   x ( t )  dt
R R dt R dt
where: R and R – the object region and the object contour
- Discrete => A = count of pixels inside the object region
 Min-max radii, Rmin and Rmax – the minimum and maximum distances from the center of mass of the object
region to its contour (the Rmax/ Rmin ratio – gives a measure of the eccentricity or elongation of the object)
Rmin Rmax
Compact
compact Non-compact
necompact
 Roundness or compactness:
( perimeter) 2 T2
  
4 (area) 4 A
For a disc -  is minimum, =1.
 Symmetry: 2 types of shape symmetry: rotational and mirror
Moment-based features:
 Center of mass:
1 1
m
N
 m ,
( m , n )R
n
N
n
( m , n )R
 The (p,q) order central moments:

 p ,q   (m - m )
( m,n )R
p
( n  n) q
 Orientation = the angle of the axis of the smallest moment of inertia
– found by minimizing:
I ( )     (m, n)  [(n  n)cos  (m  m)sin ]
( m, n )R
2
( m, n )R
2
with respect to :
1  2 1,1 
   arctg  
2   2, 0   0, 2 
 
Textures
 The texture = the periodic repetition of some basic structures in an image area; the basic image
structure is called texel
Artificial textures Natural textures
Texture analysis methods: statistical classification; structural classification

 Statistical classification techniques:
 The auto-correlation function (ACF): the spatial dimensions of the texels are proportional to the width of the auto-
correlation functions::
au ,u (k , l )  m2 (k , l ) / m2 (0,0) where m2 (k , l )    u(m, n)u(m  k , n  l )
m,n W
 Several measures used to evaluate the distribution of the ACF to describe the texture:
M ( k , l )    ( m   1 ) k ( n   2 ) l r ( m, n )
m n
 1    mr ( m, n ) ,  2    nr ( m, n ) ,
m n m n
ACF for “Sand” ACF for “Wool”
 Image transforms based approaches:
z i ( k , l )  v ( k , l )  gi ( k , l ) (8.41)
2 2 2
1
1 1
2
2 2
1 1
1
Fig. 8.18 Various masks in the frequency domain used for texture analysis
 The edge density – as texture classification feature:
Sobel Sobel
Sand Raffia
 Histogram features for texture analysis: => the co-occurrence histogram:

p u ( x1 , x 2 )  f ( r ,  ; x1 , x 2 )
=> various features can be extracted from the co-occurrence histogram:
R4, 0
- Inertia: I ( r, )   x  x2 f ( r ,  ; x1 x 2 )
2
1
x1 , x 2
- The mean of the distribution:
1
 ( r;x1 ,x 2 ) 
N0


f ( r ,  ;x1 x 2 )
N0 – the total number of possible orientations. Grass
- The variance of the distribution:

1
 2 ( r;x1 ,x 2 ) 
N0
 [ f ( r, ;x x 1 2 )   ( r;x1 ,x 2 )]2
- The spread of the distribution:
( r;x1 ,x 2 )  max{ f ( r, ;x1 ,x 2 )}  min{ f ( r, ;x1 ,x 2 )}

 Random texture models

u( m, n)  a( m, n)   ( m, n) (8.47)
ACF
computation
Feature Texture
u(m,n) extraction features
Texture
Decorrelation
(m,n) Histogram
filter
A(x1,x2) computation
white (m,n) 1 u(m,n)

noise A(x1,x2)
Fig. 8.19 Texture analysis model

Image segmentation
• Image segmentation = the decomposition of the image into disjoint

homogeneous regions
• “Good segmentation” (acc. to Haralick si Shapiro) :
• Decomposition into uniform homogeneous regions in respect to the selected visual
features
• Simple regions, without many holes inside
• The regions boundaries should be smooth, simple, well delimited
• In the adjacent regions, the values of the visual features should be significantly
different from a region to the other
• Formal definition of image segmentation:

Let I – the set of image pixels;
=> segmentation of I = finding a partition P of N sub-sets Rk, having H – a
homogeneity predicate, so that:
N
 Rk  I; Rk  Rl  , k  l; H Rk   true, k ; H Rk  Rl   false, k  l adiacent .
k 1
Simple homogeneity predicates: brightness, color, texture

Image segmentation
Amplitude thresholding/ grey level window slicing
1) Local maxima detection
Object
2) Local minima detection
Objects are found between the local minima
Component labeling
 Pixel labeling. Region growing
C A D
B X
seed segmentation with threshold =4 new seed segmented image

Boundary-based image segmentation
Object Boundary-
Edge Feature Analysis and
based extraction classification
detection
segmentation
Fig. 8.23 Segmentation algorithm based on boundary detection
Region-based segmentation; segmentation based on regions and boundaries
R 1 R
R 1 '=
R 1  R 2  R
2
4
R R 3
R R 4
3 R 5
5
a b
Fig. 8.24 Region merging
1 2
A B A B R2
Split D C D C Merge
A B R1
D C
4 3
1 2 3 4 1(A,B,D)
1C, 2D, 3A
2(A,B,C)
A B D C A B C D A B C D 3(B,C,D)
4
b
c
Fig. 8.25 Segmentation by split and merge algorithm: a. input

b. region splitting by quad-trees c. segmented regions
a) Original image b) Region-based image

segmentation
c) Main objects detected, marked on

the original image by their contour

Digital Image Processing Chapter 8 Image Analysis and Pattern Recognition

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Digital Image Processing Chapter 8 Image Analysis and Pattern Recognition

Uploaded by

Copyright:

Available Formats

Digital image processing Chapter 8 Image analysis and pattern recognition

IMAGE ANALYSIS AND PATTERN RECOGNITION

Fig. 8.1 Image analysis system – block diagram

Example image analysis tasks:

License plate recognition

Example image analysis tasks:

Text detection & recognition or removal

Trafic sign recognition Calcite localization/assessment Detect tissue on the slide

Single object images; defect detection on the object

Ways to approach the image analysis task?

New feature space

2) Analyze only the information in the region of interest , using the

•Amplitude features may be discriminative enough if intensity is enough to distinguish wanted

• Histogram based features:

Transform features extraction

Input Forward unitary v(k,l) v(k,l) Inverse unitary u(m,n)

Fig.8.2 Transform features extraction

Edge features extraction. Edge detection

g1(m,n) Magnitude Thresholding

u(m,n) gk(m,n) Max{1 1} gradient g(m,n) Edge magnitude edge

Fig. 8.4 Edge detection by compass operators

Fig. 8.5 Compass edge detectors (North direction)

Fig. 8.6 Compass operators on different directions

gk(m,n) –compass gradient on the direction k0,...,7,  k    k  ,

Laplace operators and the zero-crossings edge localization method:

df Thresholding Double contour

The Gaussian filter:

The Laplacian of Gaussian operator (LoG)

Gaussian filter The 2nd derivative

Edge detection by different operators – comparison:

Original image Sobel edge detection LoG edge detection; sigma=5

Objects representation by their boundaries:

Fig. 8.8 4-connectivity; 8-connectivity

The Hough transform: s  x  sin  y  cos  s  x  cos  y  sin

The Hough transform of lines/random curves:

Approx. convergence =>

Applications to contour extraction formed by line segments:

Input image Edge map; Lower left object =

Contour representation/contour descriptors:

Chain codes and polygonal approximations:

Encode this contour

Original contour Contour approximation (“quantization”)

leading to a new contour x'(n) y'(n), given by:

Shape reconstruction from its Fourier descriptors:

Original Reconstruction using Reconstruction using

Reconstruction using Reconstruction using

then for a given translation u0, the distance d is minimal when:

Objects representation by their regions:

Fig. 8.11 Quad-tree region representation

=> the quad-tree code : gbgbwwbwgbwwgbwwb ; decoding: g(bg(bwwb)wg(bwwg(bwwb)))

Shapes and structures for region-based object representation:

Fig. 8.12 Skeleton extraction Fig. 8.13 Examples skeletons

 Morphological processing. Binary image morphology

Object Structuring Contour Object Structuring Opening

Object Structuring Closing Object Structuring Skeletization

Object Element Thinning

Object Structuring Thickening

Fig. 8.16 Morphological operations

Geometrical features (geometrical descriptors):

- Discrete => A = count of pixels inside the object region

For a disc -  is minimum, =1.

 Symmetry: 2 types of shape symmetry: rotational and mirror

 The (p,q) order central moments: