Image Processing with Scilab

and Image Processing Design

Toolbox
Copyright by Dr. Eng. (J !arald "alda# $%&&
Contents
1 Introduction.........................................................................................................................4
2 Characteristic Properties of Images....................................................................................5
3 Basic Operations on Images.............................................................................................13
3.1 Introduction................................................................................................................13
3.2 Thresholding..............................................................................................................13
3.3 Blob nal!sis..............................................................................................................1"
3.4 #iltering......................................................................................................................2$
3.4.1 Introduction.........................................................................................................2$
3.4.2 %inear #iltering....................................................................................................2$
3.4.3 &edian #iltering..................................................................................................22
3.4.4 &orphological #iltering.......................................................................................23
3.5 'atershed Transform and (istance Transform.........................................................2)
4 *o+ to (etect Ob,ects in Images......................................................................................31
4.1 Introduction................................................................................................................31
4.2 Thresholding and Blob nal!sis................................................................................31
4.3 -dge (etection and 'atershed Transform...............................................................35
.eferences...........................................................................................................................41
& Introd'ction
There are /arious applications of image processing0 e. g. detection of surface defects in in1
dustrial 2ualit! control0 detection of anatomical landmar3s in surger!0 counting cells in bio1
technolog! and classification of regions in remote sensing. Images are generated b! opti1
cal cameras0 ultrasound0 41ra! machines and other imaging de/ices.
'hen processing an image +ith a computer0 it must be digiti5ed or created in a digital for1
mat. There are some basic methods to distinguish bet+een ob,ects and bac3ground and to
describe regions in digital images. This document e4plains ho+ digital images can be re1
presented mathematicall! and basic methods for finding ob,ects and describing regions.
6cientists0 engineers and medical doctors +ho intend to anal!5e images usuall! 3no+ ho+
their images +ere generated and ho+ the ob,ects of interest loo3 li3e. *o+e/er0 these ob1
,ects must be described mathematicall! +hen the! are to be detected automaticall! +ith a
computer. The goal of this document is introducing these mathematical descriptions in a
+a! easil! to understand.
The remaining part of the document is organi5ed as follo+s7 In chapter 20 a mathematical
representation of digital images and characteristic properties of images are introduced. In
chapter 30 basic image processing operations such as segmentation into connected regi1
ons and filtering are introduced. In chapter 4 methods for ob,ect detection are presented.
.eferences are gi/en in the last chapter.
-4amples are implemented +ith 6cilab 5.3.1 and Image Processing (esign Toolbo4 8IP(9
".$. The photograph used in this document is ta3en from IP(. 6cilab is a/ailable for do+n1
load at +++.scilab.org. IP( can be do+nloaded from atoms.scilab.org:toolbo4es:IP(.
#igure 1.17 This image sho+s a ;apanese tea cup and a can for tea po+der. The author
too3 this photograph at home. This image is used in the further chapters.
4
$ Characteristic Properties o( Images
2( digital image consists of a finite number of points aligned as ro+s and columns.
point of a digital image is called <pi4el=. <Pi4el= means <picture element=. t each pi4el there
can be a scalar gra! /alue or a /ector of color components or gra! /alues. In 6cilab >1?
there are the follo+ing t!pes of images7
@ra! /alue images0 also called <gra! scale= images or <intensit! images=7 gra! /a1
lue image is a matri4 of gra! /alues. In 6cilab a gra! /alue image is a 2( arra!.
gra! /alue is usuall! an integer scalar bet+een $ and 255 or a real number bet+een
5ero and one. n e4ample is displa!ed in #ig. 2.2 b.
Pseudo color images0 also called <inde4ed images=7 -ach pi4el of a pseudo color
image corresponds to an item in a list of colors. list of colors is called <color map=.
color map is a matri4. The ro+s of this matri4 correspond to color /ectors and the
columns correspond to color channels. n e4ample is sho+n in #ig. 2.2 c.
Color images7 t each pi4el of a color image there is a /ector of color components0
e. g. red A green A blue. &athematicall! a color image can be described as a triple
of matrices. -ach matri4 corresponds to a color channel. In 6cilab0 color images are
represented as 3( arra!s. The first dimension corresponds to the ro+s0 the second
dimension corresponds to the columns and the third dimension corresponds to the
color channel. color component is usuall! an integer scalar bet+een $ and 255 or
a real number bet+een 5ero and one. color image can be transformed to a gra!
/alue image b! calculating the scalar product or each color /ector and a constant
/ector. n e4ample can be seen in #ig. 1.1 and 2.2 a.
%ogical images0 also called <binar! images=7 logical image is a matri4 of boolean
/alues. 'hen a logical image is /isuali5ed0 false is displa!ed in blac3 and true is
displa!ed in +hite. %ogical images can be generated comparing each pi4el of a gra!
/alue image to a threshold. Pi4els +ith gra! /alues at least as high as the threshold
are mapped to true +hereas pi4els +ith gra! /alues lo+er than the threshold get
mapped to false.
Ob,ect images0 also called <label images=7 n ob,ect image consists of ob,ects and
bac3ground. t each ob,ect image there is a number greater than 5ero. Pi4els +ith
the same number belong to the same ob,ect. Pi4els +ith different numbers belong
to different ob,ects. Bac3ground pi4els ha/e the /alue 5ero. n ob,ect image can be
created searching connected areas in logical images. Ob,ect images can be /isuali1
5ed as pseudo color images.
#ig. 2.1 sho+s ho+ a color image0 a gra! /alue image and a pseudo color image can be
generated and displa!ed in 6cilab. The color image is read from dis3 and con/erted to a
gra! le/el image. The color image is displa!ed. The gra! le/el images is displa!ed as both
gra! le/el image and color image. #ig. 2.2 sho+s the resulting images.
n important propert! of a gra! le/el image is its gra! le/el histogram. #ig. 2.3 sho+s ho+
the histogram of a gra! le/el image is calculated and displa!ed in 6cilab. #ig. 2.4 sho+s
the histogram of the gra! le/el image displa!ed in #ig. 2.2 b. The +hole image is /er!
bright so there are a lot of pi4els +ith gra! le/els BC 2$$. There are fe+ pi4els +ith a gra!
/alue lo+er than 1$$. The t+o pea3s in the range bet+een $ and 255 correspond to the
+hite tea cup segments and the bac3ground. The lo+ pea3s bet+een 1$$ and 15$ corres1
pond to the comparati/el! dar3 blue and red areas.
5
#igure 2.17 color image is read from dis3 and con/erted to a gra! le/el image. The color
image is displa!ed. The gra! le/el images is displa!ed as both gra! le/el image and color
image.
#ig. 2.2 a7 color image
sa/ed on hard dis3.
#ig. 2.2 b7 .esult of transfor1
ming color image to gra!
le/el image
#ig. 2.2 c7 gra! le/el image
displa!ed as a pseudo color
image.
D
#ig. 2.37 *o+ to calculate and displa! a histogram in 6cilab
#ig. 2.47 The histogram of the image sho+n in #ig. 2.2 b. The t+o high pea3s bet+een 2$$
and 255 correspond to the +hite tea cup segments and the bac3ground. The three lo+
pea3s bet+een 1$$ and 15$ correspond to the comparati/el! dar3 blue and red regions.
)
In 6cilab images can be anal!5ed interacti/el!. This is sho+n in #ig. 2.5.
#ig. 2.5 a7 The function ImageTool for interacti/e image anal!sis is called.
#ig. 2.5 b7 The @EI for interacti/e image anal!sis. 'hen the mouse pointer is inside the
image0 the position and gra! /alue of the pi4el the mouse pointer resides at is displa!ed.
The user can select a single pi4el0 a line bet+een t+o points or a rectangular area.
"
#igure 2.D sho+s one rectangular area in the bac3ground0 one in a blue region and one in
the red region and the histogram of each region.
#ig. 2.D a7 rea in the +hite bac3ground. #ig. 2.D b7 *istogram of the +hite area.
#ig. 2.D c7 rea in a blue region. #ig. 2.D d7 *istogram of the blue area.
#ig. 2.D e7 rea in the red region. #ig. 2.D f7 *istogram of the red area.

F
s can be seen in #ig. 2.D0 the gra! le/el ranges of the red and blue regions partiall! o/er1
lap. The range of the selected +hite region does not o/erlap at all +ith the ranges of the
red and blue regions. In this image0 it is possible to distinguish bet+een ob,ects and bac31
ground based on gra! /alue ranges.
In the color image in 2.2 a there are blue0 red0 !ello+ and +hite regions. These colors are
easil! to distinguish +hen loo3ing at the image. One might e4pect that the red and blue re1
gions are clearl! /isible in the red and blue channel0 respecti/el!. *o+e/er0 this is not the
case as can be seen in #ig. 2.). In the red channel0 the middle and right parts of the red
are lighter than the bac3ground +hereas the left part is dar3er. In the blue channel0 the
blue areas are dar3er than +hite. &oreo/er0 the !ello+ region is brighter than the red one0
e/en though !ello+ is certainl! not more similar to blue than red.
The red A green A blue color space0 also called <.@B=0 is used for storing color image files
and for displa!ing color images on screens. *o+e/er0 this color space does not seem to
be appropriate for calculations on images. There are other color spaces more suitable for
this purpose.
#ig. 2.) a7 .ed channel. #ig. 2.) b7 @reen channel. #ig. 2.) c7 Blue channel.
#ig. 2." sho+s ho+ the channels of a color image can be displa!ed in 6cilab.
#ig. 2."7 The 6cilab commands for displa!ing color channels are sho+n.
1$
One of these color spaces is %GaGbG0 also called <CI-H%B= >2?. The %GaGbG is implemented
in 6cilab. #ig. 2.F sho+s ho+ an .@B image is con/erted to %GaGbG and ho+ the compo1
nents and their histograms can be /isuali5ed. The color channels are normali5ed so the
minimum /alue is 5ero and the ma4imum /alue is one. This is done for the purpose of dis1
pla! and histogram calculation.
The results are sho+n in #ig. 2.1$. The %G channel and its histogram loo3 /er! similar to
the gra! le/el image in #ig. 2.2 b and its histogram sho+n in #ig. 2.5. The blue segments
appear dar3 in both aG and bG channels +hereas the red area appears bright in both chan1
nels. Both blue and red areas can be distinguished clearl! from bac3ground. The !ello+
area appears dar3 in aG and bright in bG. The histograms of aG and bG ha/e high pea3s that
correspond to the bac3ground and lo+ broad ma4ima corresponding to ob,ects.
In the %GaGbG space the different colors of this image can be distinguished from each other
and from bac3ground better than in .@B space. It is important to choose an appropriate
color space +hen +or3ing +ith color images.
#ig. 2.F7 Con/ersion of an .@B image to %GaGbG and /isuali5ation of the color channels
and their histograms.
11
#ig. 2.1$ a7 Iormali5ed %G. #ig. 2.1$ b7 *istogram of normali5ed %G.
#ig. 2.1$ c7 Iormali5ed aG. #ig. 2.1$ d7 *istogram of normali5ed aG.
#ig. 2.1$ e7 Iormali5ed bG. #ig. 2.1$ f7 *istogram of normali5ed bG.
12
) *asic +perations on Images
).& Introd'ction
In this chapter basic image processing operations are introduced. These operations are
useful for detecting ob,ects in single channel images0 i. e. in gra! le/el0 logical and ob,ect
images. The follo+ing operations are described7
Thresholding7 The gra! le/el of each pi4el is compared to a threshold. If the gra! le1
/el is e2ual to or greater than the threshold0 the pi4el is mapped to true. If the gra!
le/el is belo+ the threshold0 the pi4el is mapped to false. The result is a logical ima1
ge >30 4?.
Blob anal!sis0 also called <connected component anal!sis=7 In a logical image the
connected regions of true pi4els are searched. The pi4els of each connected region
are mapped to a number greater than 5ero. ll false pi4els are mapped to 5ero. The
result is an ob,ect image. #or each ob,ect characteristic properties such as the cen1
troid or the area gi/en in pi4els can be calculated >4?.
#iltering7 #or each pi4el a ne+ gra! /alue0 logical /alue or ob,ect label is calculated
from an area surrounding it. This can emphasi5e ob,ects of interest or remo/e irre1
le/ant ob,ects or noise. &oreo/er0 the gra! le/el gradient can be appro4imated and
used for edge detection. #iltering can be applied to gra! le/el images0 logical ima1
ges and ob,ect images. The result is of the same t!pe as the original image >30 4?.
'atershed transform7 6tarting from some seed points0 unlabeled pi4els are assig1
ned to regions of pi4els that are alread! labeled and ha/e a lo+er or the same gra!
/alue >3?.
(istance transform7 #or each pi4el the distance to the ne4t bac3ground pi4el is cal1
culated. 'atershed segmentation can be applied to the result image >3?.
).$ Thresholding
gra! le/el image can be segmented b! comparing the gra! /alue +ith a threshold at
each pi4el. In 6cilab0 pi4els that ha/e a gra! /alue greater than or e2ual to the threshold
are mapped to true0 +hereas all other pi4els are mapped to false.
Thresholding helps to find ob,ects in an image if these ob,ects are significantl! brighter or
dar3er than the bac3ground. In this case there is a gra! le/el range t!pical for ob,ect pi4els
and another gra! le/el range t!pical for bac3ground pi4els. gra! le/el bet+een these ran1
ges can be a threshold. good candidate for a threshold is a local minimum of the histo1
gram.
#ig. 3.1 sho+s ho+ a threshold can be determined manuall!. The ob,ects are dar3er than
the bac3ground in the original image. Therefore0 the image is in/erted. Then the histogram
is calculated and displa!ed. The data tip mode is toggled and a local minimum of the histo1
gram is searched.
The threshold is applied to the in/erted image. The original image0 the in/erted image and
the result are displa!ed in #ig. 3.2. The bright regions of the resulting image correspond
/er! +ell to the dar3 regions of the original image.
13
#ig. 3.1 a7 The image is in/erted0 because the ob,ects are dar3 and the bac3ground is
bright. The histogram of the in/erted image is displa!ed.

#ig. 3.1 b7 The histogram of the in/erted image is sho+n. data tip is placed on a local
minimum of the histogram.
14
#ig. 3.2 a7 The in/erted image is segmented b! the threshold.
#ig. 3.2 b7 Original image. #ig. 3.2 c7 In/erted image. #ig. 3.3 d7 .esulting image.
threshold can be calculated automaticall!0 too. +ell 3no+n e4ample for automated
threshold calculation is the Otsu method7 The /alue k that ma4imi5es the bet+een1class
/ariance

between
is chosen as threshold. The bet+een1class /ariance is defined b! the
follo+ing e2uations7

between
2
=
dark
(
dark

image
)
2
+
bright
(
bright

image
)
2
().& a

dark
=

q=0
k1
p(r
q
) ().& b

bright
=

q=k
L1
p(r
q
) ().& c
15

dark
=

q=0
k1
r
q
p(r
q
)

dark
().& d

bright
=

q=k
L1
r
q
p(r
q
)

bright
().& e

image
=

q=0
L1
r
q
p(r
q
) ().& (
p(r
q
)=
h(r
q
)
N
().& g
N is the number of pi4els0 L is the number of gra! /alues0
r
q
is the qth gra! /alue0
h(r
q
)
is the histogram /alue of
r
q
>3?.
The Otsu method is implemented in 6cilab. The calculation and the result are sho+n in
#ig. 3.3.
#ig. 3.3 a7 Calculation of threshold using OtsuJs method.
#ig. 3.3 b7 Original image. #ig. 3.3 c7 In/erted image. #ig. 3.3 d7 .esulting image.
1D
The result is not /er! accurate0 because shado+s and comparati/el! dar3 regions in the
left part of the image ha/e a gra! /alue higher than the calculated threshold.
In #ig. 3.4 the bac3ground is anal!5ed b! calling the function ImageTool890 selecting a pi4el
and displa!ing a ro+ profile. ll pi4els on this ro+ belong to the bac3ground. The gra! /a1
lue strongl! /aries on the ro+ the selected pi4el belongs to. The bac3ground brightness is
inhomogeneous because the illumination +as inhomogeneous +hen the original photo1
graph +as ta3en. Therefore0 it is difficult to calculate a threshold automaticall! for the gra!
le/el image.
#ig. 3.4 a7 The function ImageTool89 is called for anal!5ing the image.
#ig. 3.4 b7 pi4el is selected. #ig 3.4 c7 @ra! /alue profile of the ro+ the
selcted pi4el belongs to.
In the follo+ing e4ample0 the image is cropped and the threshold is calculated on the crop1
ped in/erted image. The threshold is close to the manuall! determined one and the result
is 2uite accurate as +e can see in #ig. 3.5. The histogram of the in/erted image has a /er!
high pea3 as can be seen in #ig. 3.1. This pea3 ends at the threshold calculated on the ori1
ginal gra! le/el image. Contrasting this0 the cropped in/erted image does not ha/e a pea3
that is much higher than all other pea3s. The bac3ground brightness does not /ar! as
strongl! in the cropped image as in the original one.
The Otsu method is eas! to implement and fast. *o+e/er0 it is important to consider bac31
1)
ground brightness /ariations +hen tr!ing to segment an image using this method. Bac31
ground /ariations can be caused b! inhomogeneous illumination or b! specular reflection.
#ig. 3.5 a7 The image is cropped and the threshold is calculated on the cropped in/erted
image.
#ig. 3.5 b7 The result of thresholding. #ig. 3.5 c7 The histogram of the cropped
in/erted image.
).) *lob ,nalysis
Ob,ects can be found in a logical image b! searching the connected areas of true pi4els.
The pi4els of each connected area are mapped to an integer number greater than 5ero. ll
pi4els of the same area ha/e the same number +hereas pi4els belonging to different areas
ha/e different numbers. ll false pi4els are mapped to 5ero.
n algorithm for searching the connected components is described in >4? and this algo1
rithm is implemented in 6cilab. #ig. 3.D sho+s ho+ a blob anal!sis can be done in 6cilab.
1"
#ig. 3.D a7 blob anal!sis is performed using the function 6earchBlobs89. The result is
sho+n b! 6ho+Image89. The color map has as man! entries as the resulting image has
ob,ects.
#ig. 3.D b7 The logical image. #ig. 3.D c7 The result of blob anal!sis.
1F
).- .iltering
).-.& Introd'ction
#iltering means that a mas3 is placed on each pi4el and a ne+ gra! /alue or logical /alue
is calculated from the pi4els belo+ the mas3. Ob,ects of interest can be emphasi5ed and
irrele/ant ob,ects can be remo/ed appl!ing a filter.
In this document the coordinate s!stem of a filter mas3 ( is defined as sho+n in #ig. 3.)7
f8110 119 f8110 $9 f8110 19
f8$0 119 f8$0 $9 f8$0 19
f810 119 f810 $9 f810 19
#ig. 3.)7 The origin of a filter mas3 coordinate s!stem is on the central element.
The follo+ing t!pes of filtering are described in this document7
%inear filtering7 The ne+ gra! /alue is a +eighted sum of the gra! /alues of the
pi4els belo+ the mas3.
&edian filtering7 The median is calculated from the gra! /alues or logical /alues of
the pi4els belo+ the mas3.
&orphological filtering7 &orphological filters can e4pand or shrin3 ob,ects and con1
nect or di/ide areas.
).-.$ /inear .iltering
'hen an image I is filtered +ith a mas3 ( that has N ro+s and M columns0 the gra! /alue
I
filtered
(i , j )
at the pi4el (i , j ) is calculated in the follo+ing +a!7
I
filtered
(i , j )=

n=n
min
n
max

m=m
min
m
max
f (n , m) I (in, j m) ().$ a
knk , N=2 k+1
k+1nk , N=2k
().$ b
kmk , M=2k+1
k+1mk , M=2k
().$ c
n e4ample is sho+n in #ig. 3."7 The original image consist of an upper region +ith gra!
/alue 5ero and a lo+er region +ith gra! /alue one. The matri4
f =
1
3
(
1 1 1
0 0 0
1 1 1
)
().)
is used as filter mas3. In the filtered image0 the pi4els in the lo+est ro+ of the upper half
and the highest ro+ of the lo+er half ha/e the gra! /alue one +hereas all other pi4els ha1
/e the gra! /alue 5ero. The mas3 used in this e4ample emphasi5es hori5ontal edges.
2$
#ig. 3."7 n image is filtered +ith a mas3 that emphasi5es hori5ontal edges.
The matri4 of the pre/ious e4ample can be e4pressed at the product of a matri4 +ith onl!
one column and a matri4 +ith onl! one ro+7
f
column
=
1
3
(
1
0
1
)
().- a
f
row
=
1
3
( 1 1 1)
().- b
f =f
column
f
row
().- c
In other +ords0 this matri4 is separable. #ig. 3.F sho+s ho+ an image can be filtered +ith a
separable matri4. The column filter is applied to the columns and after this the ro+ filter is
applied to the ro+s or the other +a! around. This reduces the number of multiplications
and additions +hen compared to using a matri4 +ith se/eral ro+s and columns and there1
fore speeds up the computation.
21
#ig. 3.F7 The mas3 can be represented as a product of a ro+ filter and a column filter. The
function 6eparable#ilter89 applies both filters to the image. The result is the same as in #ig.
3.".
).-.) 0edian .iltering
'hen a median filter is applied to a gra! le/el image0 the median of the gra! /alues of the
pi4els belo+ the mas3 is calculated for each pi4el. 'hen a median filter +ith an odd num1
ber of elements is applied to a logical image0 the pi4el in the center is mapped to true0 if
there are more true than false pi4els belo+ the mas3.
The list of the gra! /alues at the pi4els belo+ the mas3 must be sorted and the median
must be searched if the number of elements is odd or e/en be calculated if the number of
elements is e/en. Therefore0 median filtering is /er! time consuming +hen compared to
other filtering methods.
&edian filtering can be applied for remo/ing small noise ob,ects as can be seen in #ig.
3.1$.
22
#ig. 3.1$7 In the original image there are a blob of nine ob,ect pi4els and a single ob,ect pi1
4el. The median filter remo/es the ob,ect consisting of onl! one pi4el. The other ob,ect is
still there e/en though some of its pi4els become bac3ground pi4els.
).-.- 0orphological .iltering
&orphological filters e4pand or shrin3 ob,ects and connect or disconnect areas in an ima1
ge. The mas3 of a morphological filter is a matri4 of logical /alues. If an element of this
matri4 is true0 the pi4el belo+ it is included into the calculation of the ne+ gra! /alue0 logi1
cal /alue or ob,ect inde4. If the element is false0 the pi4el belo+ it is not included into the
calculation. The set of true pi4els of the filter mas3 is also called <structuring element=.
The basic morphological operations are dilation and erosion. (ilation maps the gra! /alue
to the ma4imum of the gra! /alues blo+ the structuring element +hereas erosion maps
the gra! /alue to the minimum. &athematicall!0 the dilation and erosion of an image I +ith
a mas3 ( that has I ro+s and & columns are defined as follo+s7
23
I
dilated
( i , j)= max
f (n ,m)=true
I (i+n , j+m)
().1 a
I
eroded
(i , j )= min
f (n ,m)=true
I (i+n , j+m)
().1 b
knk , N=2 k+1
k+1nk , N=2k
().1 c
kmk , M=2k+1
k+1mk , M=2k
().1 d
#urther morphological operations are7
Closing7 dilation follo+ed b! erosion
Opening7 erosion follo+ed b! dilation
Top *at7 emphasi5es bright ob,ects on a dar3 bac3ground0
I
tophat
=I I
opened
Bottom *at7 emphasi5es dar3 ob,ects on a bright bac3ground0
I
bottomhat
=I
closed
I
#ig. 3.11 sho+s the effects of dilation0 erosion0 closing0 opening0 top hat and bottom hat.
&orphological operations are e4plained in detail in >3?.
#ig. 3.11 a7 The image and the structuring element of the follo+ing e4amples are defined.
24
#ig. 3.11 b7 (ilation ma3es the bright line broader +hereas erosion remo/es the bright line.
25
#ig. 3.11 c7 Closing lea/es the bright unchanged0 because the effect of dilation is canceled
out b! the follo+ing erosion. Contrasting this0 opening remo/es the bright line. Ob,ects that
are completel! remo/ed b! erosion can not restored b! dilation.
2D
#ig. 3.11 d7 The top hat filter lea/es the bright line intact +hereas the bottom hat filter com1
pletel! remo/es it.
).1 2atershed Trans(orm and Distance Trans(orm
6tarting from some seed points0 unlabeled pi4els are assigned to regions of pi4els that are
alread! labeled and ha/e a lo+er or the same gra! /alue. #ig. 3.12 sho+s ho+ the +ater1
shed transform +or3s.
#ig. 3.127 'ater fills se/eral basins. There are +atersheds +here the +ater le/el e4ceeds
the depth of basins. The bet+een +atersheds correspond to regions of an image segmen1
ted b! +atershed transform.
Ob,ects can be detected b! appl!ing a gradient filter and selecting seed points in regions
+here the gradient is lo+. #ig. 3.13 sho+s the results for manuall! selected seed points.
2)
la3e 1 la3e 2 la3e 3
#ig. 3.13 a7 The gradient is calculated and seed points are selected. Then the +atershed
transform is calculated.
2"
#ig. 3.13 b7 Original image. #ig. 3.13 c7 @radient image. #ig. 3.13 d7 .esult image.
possible +or3 flo+ for automated seed point selection is sho+n in #ig. 3.14.
#ig. 3.14 a7 6egmentation of an image using distance transform and +atershed transform.
2F
#ig. 3.14 b7 Original image. #ig. 3.14 c7 @radient image. #ig. 3.14 d7 In/erted edge
image.
#ig. 3.14 e7 (istance Image. #ig. 3.14 f7 &ar3er image. #ig. 3.14 g7 .esult image.
gradient image is calculated from the gra! le/el image. The gra! le/el image is threshol1
ded and in/erted. In the in/erted image0 bac3ground pi4els belong to edges and ob,ect pi1
4els are non1edge pi4els. The distance transform is calculated so pi4els as far as possible
from edge pi4els are found. The distance transform result is thresholded and blobs in the
thresholded distance transform image get mar3ers for +atershed transform.
similar algorithm is described in >3?.
3$
- !ow to Detect +b3ects in Images
-.& Introd'ction
The goal of ob,ect detection is demarcating each ob,ect of interest against the bac3ground
and to represent it b! one area that co/ers it as +ell as possible. In this chapter t+o me1
thods to achie/e this goal are introduced7
Thresholding and blob anal!sis7 The image is segmented b! a threshold to find
areas +ith gra! /alues t!pical for the ob,ects. Then the ob,ects are detected using
blob anal!sis. #ragmented ob,ects are connected b! morphological filtering and
small noise ob,ects are filtered out.
-dge detection and +atershed transform7 The edges are detected and seed points
for +atershed transform are determined. The biggest ob,ect after +atershed trans1
form is the bac3ground so the bac3ground is filtered out. Ob,ects that are fragmen1
ted are connected b! morphological filtering.
fter the ob,ects are detected0 their bounding bo4es are determined and dra+n into the ori1
ginal image.
-.$ Thresholding and *lob ,nalysis
In the image used in the pre/ious chapters0 the tea cup on the left has areas dar3er than
the bac3ground and the red tea po+der can is entirel! dar3er than the bac3ground. There1
fore0 these t+o ob,ects can be detected b! searching these comparati/el! dar3 regions.
This can be done the follo+ing +a!7 The image is con/erted from .@B to a gra! le/el ima1
ge. The gra! le/el image is in/erted and segmented using a manuall! selected threshold.
Then the connected areas are searched. This process and its result are sho+n in #ig. 4.1.
In the original image t+o ob,ects are /isible. *o+e/er0 more than t+o connected areas are
found as can be seen in #ig. 4.1 b. Therefore0 all ob,ects e4cept for the t+o biggest are fil1
tered out. #or this purpose0 a cumulated histogram of area si5es is calculated. The cumu1
lated histogram /alue of a specific si5e sho+s the percentage of connected areas that ha1
/e a number of pi4els less than or e2ual to this specific si5e. This is sho+n in #ig. 4.2.
The resulting image contains t+o ob,ects that co/er partiall! the ob,ects in the original ima1
ge. The bounding bo4es of these ob,ects are searched and dra+n into the gra! le/el ima1
ge. This is sho+n in #ig. 4.3.
s this e4ample sho+s0 it is important to distinguish not onl! bet+een ob,ects and bac31
ground0 but also bet+een the ob,ects of interest and noise ob,ects +hen +or3ing +ith thre1
sholding and blob anal!sis.
It is possible to merge the fragments of an ob,ect appl!ing a closing filter to the logical ima1
ge or the blob image. The t+o ob,ects to be detected are far from each other so the! are
still separate as can be seen in #ig. 4.4. *o+e/er0 if the ob,ects to be detected are nearer
to each other than the fragments belonging to the same ob,ects0 the! are merged. There1
fore0 it is necessar! to consi1der distances bet+een ob,ects +hen merging fragments.
31
#ig. 4.1 a7 The color image is transformed to a gra! le/el image. The gra! le/el image is
in/erted and segmented using a manuall! determined threshold. Then the connected
areas are searched.
#ig. 4.1 b7 The t+o biggest connected areas correspond to the ob,ects to be detected.
32
#ig. 4.2 a7 The cumulated si5e histogram of the ob,ect image is calculated and a minimum
si5e is determined from it. Connected areas smaller than this minimum si5e are filtered
out.
#ig. 4.2 b7 The cumulated si5e histogram of
the connected areas.
#ig. 4.2 c7 ll ob,ects e4cept for the t+o big1
gest are filtered out.
33
#ig. 4.3 a7 The bounding bo4es of the t+o detected ob,ects are determined and dra+n into
the original image.
#ig. 4.3 b7 The original image +ith the bounding bo4es of the detected ob,ects.
34
#ig. 4.4 a7 The logical image is filtered b! a closing filter +ith a structuring element of 21 G
21 pi4els. Then the connected areas in the filtered logical image are searched.
#ig. 4.4 b7 %ogical image. #ig. 4.4 c7 #iltered logical
image.
#ig. 4.4 c7 T+o connected
areas are found.
-.) Edge Detection and 2atershed Trans(orm
The ob,ects in the e4ample image can be demarcated against the bac3ground b! their ed1
ges. It is possible to detect the edges0 select points +ithin the areas surrounded b! edges
and appl! +atershed transform to detect the areas bet+een the edges.
*o+e/er0 there are also edges +ithin the ob,ects. Therefore0 it is necessar! to select seed
points for +atershed transform in se/eral areas. The biggest area found then is the bac31
ground. The bac3ground must be filtered out and then the remaining areas can be mer1
35
ged.
The seeds for +atershed transform are selected in the follo+ing +a!7 The edge image is
in/erted and a distance transform is carried out. The result of distance transform is thre1
sholded and the connected areas in the resulting logical image are searched. These areas
get the seeds of +atershed transform.
The process from edge detection to distance transform is sho+n in #ig. 4.5.
#ig. 4.5 a7 The edges are detected0 the edge image is in/erted and a distance transform is
applied to the in/erted edge image.
#ig. 4.5 b7 Original image. #ig. 4.5 c7 In/erted edge
image.
#ig. 4.5 d7 (istance trans1
form of in/erted edge image
3D
The result of distance transform is thresholded. The threshold is selected manuall!. #ig.
4.D sho+s the result of thresholding the distance transform image and +atershed trans1
form.
#ig. 4.D a7 The result of distance transform is thresholded and the connected areas are
used as mar3ers for +atershed transform.
#ig. 4.D b7 @radient image. #ig. 4.D c7 &ar3er Image. #ig. 4.D d7 .esult of +ater1
shed transform.
In the image resulting from +atershed transform the biggest blob is the bac3ground. The1
re are edge pi4els bet+een the blobs. The bac3ground and edges are set to 5ero and the
image is transformed to a logical image b! thresholding +ith the /alue one. This logical
image is filtered b! a closing filter +ith a 3G3 s2uare as structuring element. The blobs in
the resulting image are searched. This process is sho+n in #ig. 4.).
3)
#ig. 4.) a7 The bac3ground pi4els and edges in the ob,ect image are set to 5ero. Then the
image is transformed to a logical image0 filtered +ith a closing filter and transformed bac3
to an ob,ect image again. There are e4actl! t+o regions corresponding to the ob,ects to be
detected.
3"
#ig. 4.) b7 Ob,ect image after setting edges
and bac3ground to 5ero.
#ig. 4.) c7 %ogical image.
#ig. 4.) d7 %ogical image after closing. #ig. 4.) e7 .esult of blob anal!sis.
The blobs of the result image co/er the images to be detected. The bounding bo4es are
determined and dra+n into the original image 8s. #ig. 4."9.
'hen +or3ing +ith edge detection and +atershed transform0 it is important to select suita1
ble seed points and to detect and filter out the bac3ground in the image resulting from +a1
tershed transform.
The distance transform can help to find seed points for +atershed transform. *o+e/er0 it is
necessar! to find a threshold for segmenting the result of distance transform. This re2uires
some 3no+ledge concerning the smallest fragments +here seed points ha/e to reside.
3F
#ig. 4." a7 Bounding bo4es are determined and dra+n into the original color image.
#ig. 4." b7 The original color image +ith superimposed bounding bo4es.
4$
4e(erences
>1? 6tephen %. Campbell0 ;ean1Philippe Chancelier0 .amine Ii3ou3hah0 Modeling and
Simulation in Scilab/Scicos0 6pringer0 Ie+ Kor30 2$$D
>2? *arald @alda0 Development of a segmentation method for dermoscopic images based
on color clustering0 Lobe Eni/ersit!0 ugust 2$$3
>3? .afael C. @on5ales0 .ichard -. 'oods0 6te/en %. -ddins0 Digital Image Processing
Using MA!A"0 Pearson Prentice *all0 Epper 6addle .i/er0 Ie+ ;erse!0 2$$4
>4? %inda @. 6hapiro0 @eorge C. 6toc3man0 #omputer $ision0 Prentice1*all Inc.0 Epper
6addle .i/er0 Ie+ ;erse!0 2$$1
41