2010 Sixth International Conference on Natural Computation (ICNC 2010

)

Mean Shift Based Orientation and Location Tracking of Targets
Wang Changjun
College of computer science Hangzhou Dianzi University Hangzhou, P. R. China
Abstract—The mean shift based tracking algorithm has achieved considerable success in tracking targets’ location due to its simplicity and robustness. It finds local minima of a similarity measure between the color histograms or kernel density estimates of the model and target image. However, it can’t track the targets’ orientation. This paper proposed a novel mean shift based tracking method that can track both location and orientation of targets. It was realized by proposing an orientation tracking method that utilized the probability density distribution of the target gradient angle as the feature and constructed a similarity function that can be optimized by mean shift method, thus orientation tracking was transformed into an optimization problem. Thanks to the fast convergence of mean shift, this method can be run in real-time. A complete tracking method was constructed by using alternate iteration of the orientation tracking and Meer's location tracking algorithm. Keywords-gradient angle distribution; orientation and location tracking; mean shift; video tracking

Zhang Li
College of computer science Hangzhou Dianzi University Hangzhou, P. R. China

II.

MEAN SHIFT BASED LOCATION TRACKING

A. Mean shift Mean shift, proposed by Fukunaga, is a kind of nonparametric density estimation method, which was expected to be applied to pattern recognition. The successful application was done by Cheng [6] and Comaniciu [2]. Mean shift algorithm can be described as follows. R , the multivariate kernel density estimation with kernel k ( ⋅ ) and window radius h, computed in the point x is given by
d

Given a set {x i }i =1...n of n points in the d-dimensional space

ˆ (x ) = 1 f nh d

⎛ x−x i k⎜ ⎜ h2 i =1 ⎝
n

2

⎞ ⎟. ⎟ ⎠

(1)

I.

INTRODUCTION

Video based target tracking is to track one or more features of targets. In general, features can be divided into two categories. One is edge related, such as shape, the other is region related such as pixels or their statistics. Since the region related features contain more information than edge related, they perform more robust against noise. Mean shift based tracking method proposed by Meer [1] [2] proved to be an efficient and robust tracking algorithm, which utilizes probability density function of pixels within the target region as the feature and transforms tracking to a optimization problem of a similarity function that can be done by mean shift algorithm. Since mean shift has a good convergence speed, this method also has a fast convergence and can be used in realtime. However, this feature is lack of geometric information of targets so it can only track location. But it is necessary for some applications [3] [4] [5] to track the orientation of targets. Hence, this paper proposed a novel mean shift based tracking method that can track both location and orientation. Orientation tracking is realized by introducing another probability density function, which is the gradient angle of the target region. And just like Meer’s method, a similar similarity function was constructed, which can be optimized by mean shift method. The complete method iterates alternately location and orientation tracking method to realize two movements tracking.
This work was supported by a grant from Science and Technology Planning Project of Zhejiang Province, China (No. 2008C11108-1)

The minimization of the average global error between the estimate and the true density yields the multivariate Epanechnikov kernel

⎧ 1 −1 ⎪ c (d + 2)(1 − x ) if x < 1 k (x ) = ⎨ 2 d ⎪0 otherwise ⎩ where cd is the volume of the unit d-dimension sphere. The value of x that maximizes ˆ (x ) can be made by the f iteration below [1].


y i+1 =


where g (⋅) = −k ′(⋅) .

⎛ y −x 2 ⎞ i ⎜ j ⎟ x g 2 ⎟ i =1 i ⎜ h ⎜ ⎟ ⎝ ⎠ , j = 1,2,... ⎛ y −x 2 ⎞ n i ⎜ j ⎟ g 2 ⎟ i =1 ⎜ h ⎜ ⎟ ⎝ ⎠
n

B. Location tracking

978-1-4244-5961-2/10/$26.00 ©2010 IEEE

3593

which can be optimized by mean shift method... And just like Meer’s method.. this method has two disadvantages. such as color or texture and denote by q z the density distribution of the target model and by p z (y ) that of the candidate at location y in next video frame. the other is that it has no capability to track target orientation. { } target model centered at 0 and {x } are those of the candidate Similar with Meer’s method. ˆ u (ˆ 0 ) p y Using now (2) we have where ⎛ θ − θu ˆ Pu (θ ) = cp B(θ u )k ⎜ ⎜ π2 ⎝ 2 ⎞ ⎟ ⎟ ⎠ (7) 3594 . We measure the similarity by 1 ρ (y ) ≈ 2 ∑ u =1 m 1 ˆ u (y 0 )qu + p ˆ ˆ 2 ˆ qu .. ⎛θ 2 ˆ Qu = cB(θ u )k ⎜ u 2 ⎜ π ⎝ ⎞ ⎟ ⎟ ⎠ (6) ρ (y ) ≈ 1 2 ∑ u =1 m ˆ u (y 0 )qu + p ˆ ˆ 1 2 p ˆ ∑ ˆ (y ) u u =1 m ˆ qu . The second term has the same ⋅ form as (1) so mean shift method can be used to solve the value of y corresponding to maximum of the second term.Let us denote by z the feature of the target. m The first term of (3) is independent on y so maximum of the second item maximizes ρ [] . b(x ) is the index of histogram bin corresponding to the color of the pixel at location x . x* are the pixel locations of the i i centered at y . a normalized histogram is used to approximate the continuous density distribution.l . a similar similarity function is constructed.π ] ) are normalization constants. For computation. The iteration is ∑ m u =1 ˆ u (y ) = 1 . while it is necessary for some applications.2. denoted by { u }u =1.. So the tracking problem can be transformed to solving y to maximize the similarity of p z (y ) and q z .m p z (y ) = {ˆ u (y )}u =1.l . j = 1. we construct kernel based histograms... This paper proposed a mean shift based orientation tracking method. However. One is that it may fail when the tracking window can’t manage to cover the main part of the target due to the target rotation. ˆ u (ˆ 0 ) p y ∑ ⎛ y−x ⎜ i wi k ⎜ h2 ⎜ u =1 ⎝ m 2 ⎞ ⎟ ⎟ ⎟ ⎠ (4) where wi = δ b x i − u [( ) ] ρ (y ) = ∫ p z (y )qz dz .. p ˆ Now we introduce the kernel function to define new qu and ˆ u (y ) as follows. Given the y initial location of the target ˆ 0 , according to Taylor’s formula.m p ˆ where ∑u =1 qu = 1 . It uses the gradient orientation distribution of the target as the feature.. ORIENTATION TRACKING The above mentioned location tracking method perform robust against noise and rotation caused by the target.. p ⎜ ∑ k⎛ x ⎝ i =1 n 2 ∑ ˆ y i +1 = ∑ ˆ qu = C * i ⎞δ b x* − u ⎟ i ⎠ [( ) ] 2 (2) ⎛ y −x 2 ⎞ i ⎜ ˆj ⎟ xw g ⎟ 2 i =1 i i ⎜ h ⎜ ⎟ ⎝ ⎠ .. The θ corresponding distribution function (histogram) is denoted by B(θ u )u =1. ˆ q z = {qu }u =1. 1 2 ⎟ ⎜ ∑i=1 k ⎛ x*i ⎞ ⎠ ⎝ n and Ch = 1 ⎛ y−x ⎜ i k⎜ 2 i =1 ⎜ h ⎝ nh 2 ∑ ⎞ ⎟ ⎟ ⎟ ⎠ We divide the gradient angle of target image (with range into l discrete values. (− π . 2 ⎛ y −x ⎞ n ⎜ ˆj ⎟ i g⎜ ⎟ i =1 h2 ⎜ ⎟ ⎝ ⎠ n (5) ˆ u (y ) = Ch p ∑ i =1 nh ⎛ y−x ⎜ i k⎜ h2 ⎜ ⎝ ⎞ ⎟ ⎟δ b x i − u ⎟ ⎠ [( ) ] (3) where C= III.

In terms of orientation tracking.01 0. Distribution of target model gradient angle We use face tracking and hand tracking experiments to validate the method proposed.96 0.5 0 0.02 0.9 -2 -1.94 According to Taylor’s formula. initial ˆ location ˆ 0 and gradient angle θ 0 are known.92 0. ˆ θ Pu ˆ0 Theta Figure 3. y ˆ Estimate the corresponding qu according to (2). Qu is the feature distribution ˆ of target model (Figure 1) and P (θ ) is that of the target u Angle /rad Figure 2. EXPERIMENTS -2 -1 0 1 2 3 4 Angle /rad Figure 1..015 0.5 1 1.c= Probablity density ∑ and cp = 1 ⎛ θ 2⎞ l k ⎜ u2 ⎟ u =1 ⎜ π ⎟ ⎝ ⎠ 1 0. Similarity function ( ) Since the second term of (8) has the same form. y y y ˆ Finish if Δˆ j +1 < ε y and Δθ j +1 < ε y . Distribution of the candidate gradient angle (Figure 2). 1) Given a target region (a rectangle or an ellipse). The detail is as follows. ∑ ˆ θ j +1 = (9) ∑ Probablity density 0. 3595 .5 -1 -0.015 0. We denote similarity (Figure 3) by ˆ ˆ ρ (θ ) = ∑ Pu (θ )Qu u =1 l Simularity 2 1 0.025 0.98 0. Since faces and hands are targets with skin color..5 2 where wu = ˆ Qu .03 0. otherwise go to y 2). 2 ⎛ θ −θ ⎞ ˆ ⎜ j ⎟ u l g⎜ ⎟ 2 u =1 π ⎜ ⎟ ⎝ ⎠ l IV. Sobel operator is used to calculate the horizontal and vertical gradients. The iteration is ⎛ θ −θ 2 ⎞ ˆ ⎜ j ⎟ u θu g⎜ ⎟ 2 u =1 π ⎜ ⎟ ⎝ ⎠ .01 ∑ ⎛ θ −θ u k⎜ u =1 ⎜ π2 ⎝ l 2 ⎞ ⎟ ⎟ ⎠ 0. we have l ρ (θ ) ≈ 1 2 ∑ u =1 ⎛ θ −θ 1 u ˆ ˆ ˆ Pu θ 0 Qu + wu k ⎜ ⎜ π2 2 u =1 ⎝ ( ) ∑ l ⎞ ⎟ ⎟ ⎠ (8) 0..03 0. p y Compute ˆ u ˆ j and ˆ j +1 according to (5).2. we use mean shift iteration to make the value of θ corresponding to maximum of ρ (θ ) . y ˆ ˆ Compute pu θ j and θ j +1 according to (9).005 0 -4 -3 -2 -1 0 1 2 3 4 ˆ are the normalization constants. j = 1.005 0 -4 -3 2) 3) 4) 5) ( ) ( ) V. LOCATION AND ORIENTATION TRACKING This paper proposed a method with the capability of tracking both location and orientation by alternate iteration of these two methods. we use the value of H component in HSV color space [7] as the pixel value.025 0. ˆ ˆ ˆ Compute Δˆ j +1 = ˆ j +1 − ˆ j and Δθ j +1 = θ j +1 − θ j .02 0.

” International Conference on Robotics and Automation (ICRA). Figure 5 and Figure 6 show the effect of the method this paper proposed. V. 24. New Orleans. Y. “Real time hand tracking by combining particle filtering and mean shift. 2002. That’s because the target window can’t keep covering the target due to the target’s rotation. By that way. “Color Image Segmentation Using Global Information and Local Homogeneity. H. 2003. Mode Seeking and Clustering. an alternate iteration was used to combine these two tracking methods to obtain a complete tracking method. it converged to the background region whose H component values are close to those of the hand. Meer. Hager and P. [5] [6] [7] Figure 4 Location tracking of a hand Figure 5 Location and orientation tracking of a face Figure 6 Location and orientation tracking of a hand 3596 . pp: 229–674. which guaranteed the correct features. which can characterize the orientation of targets and constructed mean shift base iteration method to tracking this angle. Y. L. 790–799. Lerasle and J. “Real-time tracking of nonrigid objects using mean shift. 5.Figure 4 shows the effect of the method that Meer proposed. Meer. the target window kept covering the target. FG04. Hilton Head. instead.. vol. Aug. [2] [3] [4] Considering that the mean shift base target tracking method Meer proposed can’t track both location and orientation of targets. Wang and D. the density distribution of gradient angle.” IEEE Transactions On pattern analysis and machine intelligence. 1998: pp. 1995: pp. 7th Conf. orientation tracking was realized successfully. C. Sydney. pp. Comaniciu and P. IEEE Conf. Ramesh. 142– 149. Brethees. 1025–1039. Machine Intell. The elliptical target window failed to track the target. pp. “Efficient Region Tracking With Parametric Models of Geometry and Illumination. Finally. 17(8). Pattern Anal. no. SC. Tan and F. Cheng. Oct. and P. vol. on Computer Vision and Pattern Recognition. June 2000. Sixth IEEE Automatic Face and Gesture Recognition. Hayet. pp. Dec. “Mean shift: A robust approach toward feature space analysis. F. Wei.” IEEE Trans.N. which not only meets the need of some application but also improves the robustness of the Meer proposed method. this paper proposed a novel mean shift based target tracking method that can solve this problem successfully. DISCUSSION AND CONCLUSIONS REFERENCES [1] D. VI. Digital Image Computing: Techniques and Applications. II. Shan. Since the method succeeded in tracking the orientation of the target. Hence. Menezes. “Face tracking and hand gesture recognition for human robot interaction.” in Proc. G.” IEEE Transactions on Pattern Analysis and Machine Intelligence .Z.” Proc.D. 2004. 2004. this method gets more satisfactory effects. D. 1901–1906. Comaniciu. Belhumeur. “Mean Shift. This method introduced a new feature. Suter.” Proc. P. 20(10).Ojardias. T. 603–619.