# The multiRANSAC Algorithm And Its Application to Detect Planar Homographies

by:
M. Zuliani, C. Kenney and B. S. Manjunath Vision Research Lab Department of Electrical and Computer Engineering University of California, Santa Barbara

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 1/22

Motivation - I
Estimation of model parameters in presence of noise and outliers is a crucial task in image processing, computer vision, patter recognition. . .
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2 −4 x Inliers Outliers Unbiased estimate (RANSAC) Biased estimate (least squares)

Least squares produce biased estimates in presence of outliers Robust statistic can tolerate up to 50% outliers

−3

−2

−1 y

0

1

2

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 2/22

Motivation - I
Estimation of model parameters in presence of noise and outliers is a crucial task in image processing, computer vision, patter recognition. . .
Affine model 1

0.5

x

0

Outliers Inliers

RANSAC algorithm [Fischler 1981] RANdom SAmple Consensus

−0.5

−1 −1.5 −1 −0.5 0 y 0.5 1 1.5

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 2/22

Motivation - II
What happens if there are multiple models?

0.8 0.6 y 0.4 0.2

0

0.2

0.4 x

0.6

0.8

1

(a)
(a) Multiple segments, (b) Multiple planes

(b)

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 3/22

Motivation - II
Traditional RANSAC-based approaches: 1. apply standard RANSAC 2. remove the detected set of inliers 3. go back to 1

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 3/22

II Traditional RANSAC-based approaches: 1. remove the detected set of inliers 3. go back to 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Motivation . apply standard RANSAC 2. 3/22 .

apply standard RANSAC 2. go back to 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. remove the detected set of inliers 3.II Traditional RANSAC-based approaches: 1. 3/22 .Motivation .

Motivation . apply standard RANSAC 2.II Traditional RANSAC-based approaches: 1. 3/22 . remove the detected set of inliers 3. go back to 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

apply standard RANSAC 2. 3/22 . remove the detected set of inliers 3.II Traditional RANSAC-based approaches: 1. go back to 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Motivation .

3/22 . apply standard RANSAC 2.II Traditional RANSAC-based approaches: 1. go back to 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Motivation . remove the detected set of inliers 3.

0.8 0.8 1 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.III Inaccurate inlier detection for the initial (or subsequent) parameter estimation contributes heavily to the instability of the estimates of the parameters for the remaining models.Motivation .4 0.2 0 0.2 0.6 0.6 y 0.4 x 0. 4/22 .

5/22 .Goals Extend RANSAC to multiRANSAC Present a theoretical analysis of multiRANSAC Show its effectiveness on synthetic and real data sets The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 6/22 .

6/22 .Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

. Do this a sufﬁciently large number of times (more to come.From RANSAC to multiRANSAC .I RANSAC key ideas: Estimate the model parameters using the minimum number of data possible Check which of the remaining data points ﬁt the model instantiated with the estimated parameters. . ) RANSAC iteration: The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 7/22 .

multiRANSAC iteration: The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.From RANSAC to multiRANSAC . 8/22 .II multiRANSAC key ideas: At each iteration instantiate Ï models and ﬁnd the corresponding CSs Fuse the new CSs with the previously detected ones in a sensible way.

II multiRANSAC key ideas: At each iteration instantiate Ï models and ﬁnd the corresponding CSs Fuse the new CSs with the previously detected ones in a sensible way.From RANSAC to multiRANSAC . multiRANSAC iteration: The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 8/22 .

How Many Iterations? .I What is the probability Õ that we draw Ï MSSs only composed by inliers? The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 9/22 .

How Many Iterations? .I What is the probability Õ that we draw Ï MSSs only composed by inliers? It can be shown that: Õ where: É Ï Û ´ È ½´ ÆÁ Û Û Û µ Æ Ï Û ½ µ Ï is the number of models Æ is the total number of data points ÆÁ Û is the number of inliers for the ÛØ model Ø model Û is the cardinality of the MSS for the Û The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 9/22 .

How Many Iterations? . 10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

How Many Iterations? . 10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The probability that this happens for trials is ´½   Õ µ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

How Many Iterations? . 10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The probability that this happens for Goal: make sure that ´½   Õ µ trials is ´½   Õ µ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The probability that this happens for Goal: make sure that ´½   Õ µ Then: trials is ´½   Õ µ ÌØ Ð Ö ÐÓ ÐÓ ´½ Õ µ Ñ   The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.How Many Iterations? .

How Many Iterations? . 10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The probability that this happens for Goal: make sure that ´½   Õ µ Then: trials is ´½   Õ µ ÌØ Ð Ö ÐÓ ÐÓ ´½ Õ µ Ñ   Problem: a priori we don’t know ÆÁ ½ ÆÁ Ï ! The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

How Many Iterations? . The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 10/22 .II The probability that Ï MSSs are not entirely composed by inliers is ½   Õ The probability that this happens for Goal: make sure that ´½   Õ µ Then: trials is ´½   Õ µ ÌØ Ð Ö ÐÓ ÐÓ ´½ Õ µ Ñ   Problem: a priori we don’t know ÆÁ ½ ÆÁ Ï ! Solution: the estimate for ÆÁ Û is the cardinality of largest ÛØ CS found so far.

Fusing the CSs Observation: the CSs obtained in current iteration can be fused with the CSs obtained in the previous iterations to produce “better” CSs. More speciﬁcally: “Better” is quantiﬁed in terms of a ﬁtness function Cardinality MSE Simpliﬁcation: just look at the previous iteration Greedy algorithm The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 11/22 .

The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 11/22 .Fusing the CSs Observation: the CSs obtained in current iteration can be fused with the CSs obtained in the previous iterations to produce “better” CSs.

12/22 .Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 12/22 .

8 0. The stair data set 0.I Toy problem: Identify the lines containing the segments.6 y 0.4 0. 13/22 .8 1 0 0.6 0.8 1 Wrong estimate The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.2 0.Experiments: Detecting Lines .4 x 0.2 y 0.4 0.2 0.2 0 0.8 0.4 x 0.6 0.6 0.

4 x 0.8 1 Correct estimate The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.2 y 0.6 0.8 0.4 x 0.4 0.6 0. The stair data set 0.2 0.I Toy problem: Identify the lines containing the segments.2 0 0.8 0.6 0.2 0.6 y 0.4 0.8 1 0 0.Experiments: Detecting Lines . 13/22 .

II Deﬁnitions: Set of correctly detected inliers for the model Û: the detected set of inliers Percentage of correctly detected inliers: Û Á Û Á Û where Á Û is Û Û Á Û ½¼¼ models Ú is the averaged value of Û over 50 trials and Ï The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 14/22 .Experiments: Detecting Lines .

5¡½¼ ¿ 6. 14/22 .0¡½¼ ¿ 7.01% 7499 7626 8194 8699 8110 93.22% 90.29% 2016 2049 2080 2103 2103 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.5¡½¼ ¿ 7.13% 86.5¡½¼ ¿ multiRANSAC sequential RANSAC Ö Ú ÆØ Ú ÆØ Ö 95.00% 67.II Deﬁnitions: Set of correctly detected inliers for the model Û: the detected set of inliers Percentage of correctly detected inliers: Û Á Û Á Û where Á Û is Û Û Á Û ½¼¼ models Ú is the averaged value of Û over 50 trials and Ï Noise std 5.0¡½¼ ¿ 6.96% 95.08% 90.80% 86.14% 47.Experiments: Detecting Lines .92% 37.

15/22 .Experiments: Detecting Homographies Æ ¾ ¾. ¾ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

ÆÁ ¾ . Æ ¾ ¾.ÆØ Ö ¿ ¿ multiRANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Experiments: Detecting Homographies Ï ¿. 15/22 .

ÆØ Ö ½¼¼ sequential RANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 15/22 . ÆÁ ¾ . Æ ¾ ¾.Experiments: Detecting Homographies Ï ¿.

¾ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Experiments: Detecting Homographies Æ . 16/22 .

ÆÁ .ÆØ Ö ¾ ¼¾ multiRANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Æ . 16/22 .Experiments: Detecting Homographies Ï .

ÆÁ .Æ . 16/22 .Experiments: Detecting Homographies Ï .ÆØ Ö ¾¼¼ sequential RANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

17/22 . ¾ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Experiments: Detecting Homographies Æ .

17/22 . ÆÁ ¾ .ÆØ Ö ½¿¼ ¿ multiRANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.Æ ¾ ¿.Experiments: Detecting Homographies Ï .

Experiments: Detecting Homographies Ï . 17/22 . ÆÁ ¾ .Æ ¾ ¿.ÆØ Ö ¾¼¼ sequential RANSAC The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

18/22 .Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

Presentation Overview Description of multiRANSAC algorithm Experimental Results Conclusions and Future Work The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 18/22 .

better performance than sequential RANSAC Synthetic data (quantitative results) Real data Future Work Quantitative experiments on real imagery Improved CSs fusion strategy Can we detect the number of models? Explore waiting time between updates of the CSs The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 19/22 .Conclusion & Future Work We presented the multiRANSAC algorithm On average.

20/22 . Special thanks to Dr. E. Rose for the helpful discussion The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. D. Fedorov and prof.The End Thanks for your attention. K. Bober. M. Drelie.

How Many Iterations? .I What is the probability Õ that we draw Ï MSSs only composed by inliers? The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 21/22 .

I What is the probability Õ that we draw Ï MSSs only composed by inliers? Think in terms of ratio of number of favorable to number of possible The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 21/22 .How Many Iterations? .

I What is the probability Õ that we draw Ï MSSs only composed by inliers? The chance of drawing only inliers is given by:   È Æ È   Æ È Ï Û Ï Û ½ Á Û Û ¡ ½ ¡ Ï Û ½ Û The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 21/22 .How Many Iterations? .

21/22 .I What is the probability Õ that we draw Ï MSSs only composed by inliers? The chance of having the inliers belonging to the correct model:   ¡ ÉÏ Û È   È ½ ÆÁ Û Û Û Ï Û Ï Û ½ ÆÁ ½ ¡ Û The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.How Many Iterations? .

21/22 .How Many Iterations? .I What is the probability Õ that we draw Ï MSSs only composed by inliers? Putting things together we have:   Õ È Æ È   Æ È Ï Û Ï Û ½ Á Û Û ¡ ½ ¡ Ï Û ½ Û Û ¡  È È ÉÏ ½  ÆÁ Û ¡ Û Û Ï Û Ï Û ½ ÆÁ ½ ¡ Û The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.

How Many Iterations? .I What is the probability Õ that we draw Ï MSSs only composed by inliers? Putting things together we have: Õ É Ï Û ´ È ½´ ÆÁ Û Û Û µ Æ Ï Û ½ µ The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. 21/22 .

RANSAC vs.½ Ï The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p. multiRANSAC iterations How big is Ì Ø Ö in a typical problem? ¾ ¼ Number of data Æ Number of models Ï Cardinality of a MSS Probability threshold ½¼ ¿ ÖÁ Æ Ï Total percentage of inliers ÖÁ Inliers for each class ÆÁ . 22/22 .

85 0. 22/22 .7 0.75 r I 0. multiRANSAC iterations How big is Ì Ø Ö in a typical problem? N=2 5 0 6 W =4 RANSAC multiRANSAC W =3 10 10 ˆ T iter 4 W =4 W =3 W =2 W =2 10 2 W =1 W =1 0.6 0.RANSAC vs.9 The multiRANSAC Algorithm And Its Application to Detect Planar Homographies – p.8 0.65 0.