Segmentation

Kyongil Yoon
 Segmentation
Obtain a compact representation of what is
helpful (in the image)
No comprehensive theory of segmentation
Human vision: Grouping and Gestalt
– Proximity, similarity, common fate, common
region, parallelism, closure, symmetry, continuity,
familiar configuration
 Segmentation by clustering
Partitioning vs. grouping
Applications
– Background subtraction
– Shot boundary detection
Image segmentation by clustering pixels
– Using simple clustering
Agglomerative clustering (clustering by merging)
Divisive clustering (clustering by splitting)
– K-means
– Using graph-theoretic clustering
Affinity measure
Normalized cut
– cut(A,B)/assoc(A,V) + cut(A,B)/assoc(B,V)
 K-Means
Choose k data points to act as cluster centers
Until the cluster centers are unchanged
Allocate each data point to cluster whose center is
nearest
Now ensure that every cluster has at least one data
point; possible techniques for doing this include
supplying empty clusters with a point chosen at
random from points far from their cluster center.
Replace the cluster centers with the mean of the
elements in their clusters.
end
 Graph Eigenvectors
Construct an affinity matrix
Compute the eigenvalues and eigenvectors of the affinity m
atrix
Until there are sufficient clusters
Take the eigenvector corresponding to the largest unprocessed eig
envalue; zero all components corresponding to elements that ha
ve already been clustered, and threshold the remaining compon
ents to determine which element belongs to this cluster, choosin
g a threshold by clustering the components, or using a threshold
fixed in advance.
If all elements have been accounted for, there are sufficient cluster
s
end
Segmentation by fitting a model
To assert that pixels belong together to conform to some mode
l
– Large scale explicit model
Hough transform
– Three problems:
What is the line?
Which points belong to which line?
How many lines?
– Point space <-> line space
x*cos(t) + y*sin(t) + r = 0, (t, r) line space
Half-infinite cylinder
– Quantization errors, difficulties with noise
Fitting lines, fitting curves
– Least square
– Total least square
Segmentation by fitting a model(2)
Two big problems
– Robustness: what if one data point is FAR, and all others fill well?
– Missing data: which point is noise and which point is not?
Robustness
– Outliers: Improve the model either by giving the noise “heavier tails”
or allowing an explicit outlier model
– M-estimators
Assuming that somewhere in the collection of process close to our model
is the real process, and it just happens to be the one that makes the
estimator produce the worst possible estimates

   r (x
i
i i ,  );  

u2
 (u;  )  2 2
 u
Segmentation by fitting a model(3)
RANSAC (RAMdom SAmple Consensus)
– Searching for a random sample that leads to a fit on which many of the data points agree
– Determine
n : the smallest # of points required
k : the # of iterations required
t : the threshold used to identify a point that fits well
d : the # of nearby points required
Until k iterations have occurred
Pick n sample points
Fit to that set of n points
For each data point outside the sample
Test distance; if the distance < t, it is close
If there are d or more points close, this is a good fit.
Refit the line using all these points
End
Use the best fit from this collection, using the fitting error as a criterion
– Need to choose 3 parameters
# of samples required (n)
Telling whether a point is close (t)
# of points that must agree (d)
 Segmentation
E. Borenstein and S. Ullman. Class-specific, top-down segmentation, In Proc. 7th Europ. Conf. Comput. Vision, May 2
002
J. Freixenet, X. Munoz, D. Raba, J. Marti, and X. Cufi, Yet another survey on image segmentation: region and bounda
ry information integration, University of Girona, Institute of Information and Applications, ECCV 2002, LNCS 2352,
pp. 408-422, 2002
Harwood, D., Chang, S., Davis, L.S., Interpreting aerial photographs by segmentation and search, IUW(87), pp. 507-5
20
D. C. Alexander and B. F. Buxton. Statistical modeling of colour data, International Journal of Computer Vision, 44
(2):87--109, September 2001.
Friedman, N. and Russell, S. 1997. Image segmentation in video sequences: A probabilistic approach, In Proceedings 1
3. Conf. on Uncertainty in Articial Intelligence
Ahmed Elgammal, David Harwood, Larry Davis, Non-parametric model for background subtraction, 6th European C
onference on Computer Vision. Dublin, Ireland, June/July 2000
T. H. Chalidabhongse, K. Kim, D. Harwood and L. Davis, A perturbation method for evaluating background subtract
ion algorithms, Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking
and Surveillance, Nice, France, Oct. 11-12, 2003 (in conjunction with ICCV'03)
J. Shi and J. Malik. Normalized cuts and image segmentation. In Proceedings of the IEEE Conference on Computer V
ision and Pattern Recognition (CVPR'97), pages 731--737, 1997
J. Shi and J. Malik, "Motion segmentation and tracking using normalized cuts", in International Conference on Com
puter Vision, January 1998, Bombay, India
E. Sharon, A. Brandt and R. Basri, Fast Multiscale Image Segmentation, Proceedings IEEE Conference on Computer
Vision and Pattern Recognition, pp. 70--77, 2000
C. Stauffer and W.E.L. Grimson. Adaptive background mixture models for real-time tracking. In CVPR99, pages II:2
46--252, 1999
Weiss, Y., Segmentation using eigenvectors: A unifying view, Proc. 7th Int. Conf. Computer Vision, 1999, 975-982