Contour Extraction Using Particle Filters

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

1
2

Dept. of Electronics and Info. Eng., Huazhong Univ. of Scie. and Tech., China
Dept. of Computer and Info. Sciences, Temple University, Philadelphia, USA
luchengen@gmail.com, latecki@temple.edu, gxzhu@hust.edu.cn

Abstract. This paper describes a novel approach to extract object region from an image by tracking the enclosing contour. We assume that
the image is not complex, and it can be roughly partitioned into two
parts with an intensity threshold. A lot of images (for example medical
images) are in accord with this assumption. Global constraint (threshold) and local constraint (gradient) are integrated in a particle filter
framework. We utilize the filter to track the optimal contour path pixel
by pixel. The processing time depends only on the contour length and
the number of particles used. Thus the proposed method is significantly
faster than the very popular and time consuming method: Active Contour Models (Snakes). Both Snakes and our method are targeted for
similar applications. Experimental results illustrate the validity and advantages of our method.

Introduction

The problem of segmenting an image into meaningful regions is one of the most
significant problems in the computer vision area. One of the normal approaches
to represent these regions is to use their enclosing contours. Thus, the issue
of how to find the enclosing contours of these regions is very critical. Since
object detection from cluttered images clearly need prior knowledge (e.g., shape
model) of object region, here, we only focus on images with simple intensity
distribution. We assume that the background or the foreground of an image has
relative homogenous intensity distribution. Actually, a lot of images (such as
medical images) are in accord with this assumption. Currently, there are mainly
two kinds of methods: region-based methods and edge-based methods.
Region-based methods rely on the homogeneity of spatially localized features
such as gray level intensity and other pixel statistics, and they also integrate some
edge features such as gradient feature. Region-based methods yield enclosing
contours of the regions as simple closed curves, like watershed [1], Active Contour
Models [25], etc.
Edge-based methods [68] primarily utilize gradient operator to locate object boundaries. Gradient information could be based on intensity, color and
texture, etc. Gradient operator is a local descriptor, and in most cases we need
global constraint to describe the object region. Boundary growing direction is
determined by the orthogonal direction of gradient. But, gradient direction is

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

very sensitive to noise and gradient is ambiguous at corners , e.g., see Fig. 1(a).
This is especially dangerous when tracking object contour with a single path,
any error can lead to the failure of entire system. Thus, edge-based methods are
usually used as edge detectors and they usually do not yield the enclosing object
contour without further processing.

(a)

(b)

Fig. 1. The difference between single path tracking and multiple path tracking: (a)
Tracking with only one path fails at a corner; (b) Multiple paths search for the correct
contour point.

The theory of particle filter [9] helps us solve the problem. A significant
advantage of particle filter is that it can keep multiple contour hypotheses simultaneously. Each particle is one contour path in our application.
Some efforts have been dedicated to put the edge-extracting method into
the framework of particle filter. Blake et al [10] first utilized particle filter for
contour extraction, but their method only considers local gradient constraint,
which makes their method very easy to fail while tracking object boundary
(see Fig. 2(b)). In some papers, the prior knowledge of shape model [1113] is
introduced to obtain object region in cluttered background.
This paper combines global and local image constraint smoothly by particle
filter and focuses on contour extraction based on low-level knowledge of image.
Firstly, benefitting from the homogeneity of the image, we use a global threshold
to roughly partition it into two parts. Secondly, we introduce multiple paths at
each tracking step with local gradient operator, and use particle filter to find the
optimal path (see Fig. 1(b)). In order to be tolerant to noise, we introduce two
different scales of gradient operator, one controlling the contour direction, and
the other providing local constraint.
Either global constrains or multiple paths can only partly solve the problem
of edge-based method. In Fig. 2, we give an example to illustrate that both global
intensity threshold in (a), and particle filter without global constraints [10] in
(b) cannot process well. But if they are combined together in a proper way, we
are able to obtain a correct contour (c).
The combination of global intensity, gradient operator and particle filter
is suitable to process the images with homogenous background. Compared to
Snakes, which are widely applied in these images, our method is more efficient. The processing time of the proposed method is only decided by the length

Lecture Notes in Computer Science

(a)

(b)

(c)

Fig. 2. (a) Thresholded image; (b) Gradient tracking by [10]; (c) Both methods combined yield a correct result.

of contour and the particle number used, while it is well known that Snakes
are very time consuming methods.
The rest of the paper is organized as follows. In Section 2, we describe the
algorithm in details. Extensive experimental results and comparison with other
approaches are presented in Section 3. Finally, the conclusion and discussion is
given in Section 4.

Algorithm descriptions

The basic structure of our algorithm is similar to the general contour tracking
algorithms. That is, first specify a start point, then start the contour tracking
process pixel by pixel. When ending condition is satisfied, the tracking process
is terminated as illustrated in Fig. 3.
End point

Start point

(a)

(b)

(c)

Fig. 3. An illustration of our approach: (a) Original image; (b) Contour extraction
result; (c) Result of removing extra points in (b).

We utilize particle filter to get the optimal contour. The growing contour
is represented by an ordered sequence x0:t (x0 , . . . , xt ) with the start point
x0 . Given the observations z1:t (z1 , . . . , zt ), The posterior p(x0:t+1 |z1:t ) can
be used to estimate the next contour point xt+1 . Contrary to standard tracking
problems where data arrives one bit after another as time passes by, the whole set

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

of data Z is standing there at once in our case. Let {xit , i = 0, , Ns } denote

a set of samples or particles and {wtj , j = 0, , Ns } denote the associated
weights. The weight of a particle at time step t can be approximated as [9]


p xit |z1:t


wti
(1)
q xit |z1:t
Rewritten in the form of recursive update formula as:




p zt |xit p xit |xit1
i
i


wt wt1
q xit |xt1 , z1:t

(2)

The proposal distribution is usually chosen to be the prior probability [9]

q (xt |xt1 , z1:t ) = p (xt |xt1 )

(3)

Then we get
i
wti wt1
p zt |xit

(4)

Finally, to avoid the degeneracy of particles, a resampling step is added,

which is called Sampling Importance Resampling (SIR) filter [9]. The likelihood


p zt |xit = pg pl
(5)
is composed by two parts: global pg and local pl , which are defined below.

2.1

Global and local constraints

In this paper, we utilize the threshold of global intensity as global constraint. It

is easy to get optimal intensity threshold Io with Ostus between-class variance
method [14], which is described as follow.
In the case of bi-level thresholding of an image, the pixels are divided into
two classes, C1 with gray levels {1, . . . , j} and C2 with gray levels {j + 1, . . . , L}.
The gray level probability distributions for the two classes are 1 (j) and 2 (j),
and the mean intensities for each class are 1 (j) and 2 (j). Let uT indicate mean
intensity of the whole image. Then define between-class variance value as:
2

2
B
(j) = 1 (j) [1 (j) T ] + 2 (j) [2 (j) T ]

The optimal threshold Io that maximizes equation (6) is given by:

2
Io = arg max B
(j)

(6)

(7)

For some images, if the intensity distribution is not simple and the optimal
threshold is not easy to obtain, we also can specify the threshold Io manually.
Global constraint is defined as:

2
i
I

I
o


xt
pg xit = exp
(8)
2I2

Lecture Notes in Computer Science

where Ixit means the intensity at pixel xit .

Local constraint is generated by gradient defined as:


1
pl xit =

2

1 + (xit , L )

(9)

Where (xit , L ) indicates the gradient at pixel xit , and L indicates the
scale of the operator, which is very small in order to guarantee the accuracy of
localization. Finally we obtain
 

2 
exp Ixit Io
2I2


p zt |xit =
(10)

2

1 + (xit , L )

The value of serves to characterize the relative strength between pg and pl .

Essentially the proper value of represents good balance between the gradient
and intensity information.
The interaction of pg and pl can be explained more clearly by the experimental results in Fig. 4, where (a) is the source image, (b) presents the result
of Canny operator and (c) provides the thresholded result. It can be seen from
(b) that the edge detection resulted from Canny detector includes many disconnected edges, which indicates that it is not easy to extract the object contour by
gradient information only. This claim is confirmed by the result in (d). (d) is the
result with a constant pg , which means that only the local gradient constraint
pl takes effect. On the other hand, (e) is the result with a constant pl , which
means that only pg , i.e., the global threshold constraint is considered. In (e),
a rather good outline is obtained from the thresholded image (c). However, the
outline is not the actual boundary of the plane. When pg and pl are combined
in a proper way, we can obtain a significantly better contour as shown in (f).
2.2

Contour direction

At each tracking step, we consider the tracking direction as the three adjacent
directions of gradients orthogonal direction, as Fig. 1(b) illustrates. The orthogonal direction of gradient xit is given by:


xit = arg max (cos , sin ) (xit , H )

(11)

Where {0, /4, . . . , 2} means the 8 neighborhood directions of pixel xt .

And (xit , H ) is the gradient operator with scale H at pixel xit , which is large
enough to guarantee the stability of gradient direction. Then, we get xit as the
most proper contour direction. To solve the problem displayed in Fig. 1(a), we
introduce multiple paths as Fig. 1(b) illustrates. Instead
n of one settled tracking o
path, we consider three directions as candidates: xit xit /4, xit , xit + /4 ,
which naturally makes the proposal distribution:



1 xit+1 N (xit , xit )
p xit+1 |xit
(12)
xit+1
/ N (xit , xit )

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

(a) Original image.

(b) Canny result.

(c) Region map.

(d) Constant pg result.

(e) Constant pl result.

(f) Optimal result.

Fig. 4. Performance of our algorithm with various parameters. The results of our algorithm are shown in (d) to (f). (d) is the result by fixing the global constraint pg ,
and only the local gradient constraint is considered; (e) is the result by fixing the local
constraint pl , and only the global threshold constraint is considered; In (f), we consider
both constraints, and then a better contour is extracted.

where N (xit , xit ) means the neighbor pixels of xit with direction belonging
to xit and is a very small constant.
2.3

Starting and ending conditions

The start point can be chosen as the pixel that maximizes equation (10). In
some cases where the intensity distribution is very complex, we still need to
choose the start point manually. Once a tracking path forms a closed contour,
the contour sequence is long enough, and the corresponding particle owns the
highest weight, we end the tracking processing and the path is selected as the
final contour path.
2.4

Summary

Initialize N particles with the weights w01 , w02 , . . . , w0N equal to 1/N . To maintain
the diversity of contour paths, in the intermediate process we always keep N
particles, and each particle represents an independent contour path. From the
i
old sample set {(xit1 , wt1
) i = 1, . . . , N } at time t 1, construct a new
i
i
sample set {(xt , wt ) i = 1, . . . , N } for time t. The procedure is as follow:
while(1)
for i = 1 to N
Prediction by Sampling: extend each particle to three particles,
i
xk1:t = {xi1:t1 , xkt } , where xkt N (xit1 , t1
).

Lecture Notes in Computer Science

i
Importance Weighting: Compute weights: wtk = p(zt |xkt )wt1
.
end of for
Subsampling: Draw N particles from current set of 3N particles by
Sampling Importance Resampling (SIR) filter [9].
if(ending condition is satisfied(see section: 2.3)) exit(while).
N
P
normalize weights:
wti = 1 and update status: t t + 1.
i

end of while

3
3.1

Experimental results
Comparison with JetStream [10]

The results in Fig. 5 are the comparisons between JetStream [10] and our approach. In some images (image: elephant, plane and running woman),
the start point and intensity threshold are manually specified for the proposed
method. There is no human interaction for both methods once the tracking process starts. It is obvious that the proposed approach is able to generate better
results than JetStream in these applications, which proves the effect of global
constrains.

(a) Original
image.
(a) Original image

(b) JetStream
results.
(b) Jeststream results

(c) Our algorithm results.

Fig. 5. Comparisons with JetStream. Our algorithm have better performance for images with homogenous background(image: hand, plane, toy and ice crystal).

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

3.2

Contour extraction from noisy images

Comparison experiments to examine the respective tolerance to noise of our algorithm and several popular Active Contour Model (Snake) methods are given
in Fig. 6. Impulse noise is added to the original image with intensity varying
from 10% to 60%, as shown in Fig. 6(a). Fig. 6(b) to (d) provide the segmentation results of the standard Snake, GGVF [4] and RAGS [5] respectively. Fig.
6(e) shows the results of the proposed approach. The comparison illustrates the
proposed approach can track the correct contour from noisy images, which is
more robust than the other three methods.

(a) Original image with various levels of added Gaussian noise [0%, 10%,. . . ,60%].

(b) Standard geometric snake results.

(c) GGVF snake results.

(d) RAGS snake results.

(e) Our algorithm results.

Fig. 6. Shape recovery from noisy images.

3.3

Results from images with intensity inhomogeneous background

To further test the performance of the proposed approach, we perform several

experiments for intensity-inhomogeneous images, in comparison with the other

Lecture Notes in Computer Science

two algorithms of snakes type, namely PS model [2] and LBF model [3], both
of which are suited to the above images. This kind of intensity-inhomogeneity
widely exists in medical images. From the results shown in Fig. 7((a) and (b) are
quoted from [3]), it can be seen that our algorithms have comparable performance
with the other two algorithms, while our approach takes much less processing
time, as shown in Table 1 . A few more results of our method on actual medical
images can be found in Fig. 8.

(a)LBF model [3] results.

(b)PS model [2] results.

(c)Our algorithm results.

Fig. 7. Results from images with intensity inhomogeneity background.

Table 1. Comparison of time consumption (in sec.) with 2.8G CPU computer.
Image 1 Image 2
LBF Model
1.78
2.42
PS Model
39.92
101.89
Our Algorithm 0.02
0.05

Image 3
11.85
203.03
4.23

Conclusion and future work

This paper presents a novel edge-based image segmentation algorithm using the
framework of particle filter. We combine gradient operators of two scales and
global threshold constraint to guarantee the robustness and accuracy of edge

10

ChengEn Lu1 , Longin Jan Latecki2 , and Guangxi Zhu1

Fig. 8. More results on medical images.

locating. The proposed method has very comparable performance with the popular method Snakes, and our method shows the advantage of less processing
time. In addition, this algorithm can be extended to contour extraction based
on high-level knowledge, which we plan to incorporate in our future work.

References
1. Vincent, L., Soille, P.: Watersheds in digital spaces - an efficient algorithm based
on immersion simulations. PAMI 13 (1991) 583598
2. Vese, L., Chan, T.: A multiphase level set framework for image segmentation using
the mumford and shah model. Intl J. Comp. Vis. 50 (2002) 271293
3. Li, C., Kao, C.Y., Gore, J.C., Ding, Z.: Implicit active contours driven by local
binary fitting energy. CVPR (2007)
4. Xu, C., Prince, Jerry, L.: Generalized gradient vector flow external forces for active
contours. Signal Process 71 (1998) 131139
5. Xie, X., Mirmehdi, M.: Rags: Region-aided geometric snake. IEEE Transactions
on Image Processing 13 (2004) 640652
6. Canny, J.: A computational approach to edge detection. PAMI 8 (1986) 679698
7. Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. PAMI 26 (2004) 530549
8. Dollar, P., Tu, Z., Belongie, S.: Supervised learning of edges and object boundaries.
CVPR (2006)
9. Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for on-line non-linear/non-gaussian bayesian tracking. IEEE Trans. Signal
Processing 50 (2002) 174188
10. Prez, P., Blake, A., Gangnet, M.: Jetstream: Probabilistic contour extraction with
particles. ICCV (2001) 524531
11. Zhang, J., Collins, R., Liu, Y.: Representation and matching of articulated shapes.
CVPR 2 (2004) 342349
12. Fan, X., Qi, C., Liang, D., Huang, H.: Probabilistic contour extraction using
hierarchical shape representation. ICCV 1 (2005) 302308
13. Bai, X., Yang, X.W., Latecki, L.J.: Detection and recognition of contour parts
based on shape similarity. Pattern Recognition 41 (2008) 21892199
14. Ostu, N.: A threshold selection method from gray-level histogram. IEEE Transactions on System Man Cybernetics 9 (1979) 6266