413 views

Uploaded by pi194043

Mean Shift Algorithm for finding modes of probability density fucntion

save

You are on page 1of 8

Pi19404

September 23, 2013

Contents

Contents

Mean Shift Algorithm

0.1 Introduction . . . . . . . . . . . . . . . . . . 0.2 Kernel density Estimation . . . . . . . 0.3 Mean Shift . . . . . . . . . . . . . . . . . . . 0.3.1 Modes of Smooth function . 0.3.2 Using the Gradient . . . . . . . 0.3.3 Local Maxima . . . . . . . . . . . . . 0.4 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 3 4 7 8 8

2|8

0.1 Introduction

In the article we will look at the basics of Mean Shift Algorithm.

let us first consider a univariate gaussian PDF and sampled data from the PDF. The kernel density estimation uses fact that the density of samples about a given point is proportional to its probability. It approximates the probability density by estimating the local density of points as seen in figure ?? is resonable. Large density of points are observed near the maximum of PDF. The KDE estimate the PDF by superposition of kernel at each data point,which is equivalent to convolving the data points with a gaussian kernel.

Mean shift is a tool for finding the modes in a set of data samples which is sampled from underlying PDF.The aim of the mean shift algorithm is to find the densest region in given set of data samples. Data points density implies PDF value . Let us consider a 2D region.The points in the 2D region are sampled from a underlying unknown PDF. Let

X

= [x; y ]be

P (X )

.

P (X )

H = [xH ; y H ]

3|8

Figure 1: Density estimation

For example let us consider a multi-variate gaussian distribution where the random variables x and y take values in the range -3 to 3.

Let us say we want to find the modes of PDF.The PDF is approximated using kernel density estimation.Modes are the points at which PDF exhibits local maximum . Dense regions in PDF corresponds to modes or local maxima. Since the kernel is smooth,its differentiable.It gives to a smooth

4|8

^ (x) = f h 1

n

n X i=1

Kh (x

xi )

= 1

1

nh

n x xi X

K

r r r ^h(

f x)

^ (x) = f h

n X i=1

i=1

nh

x xi

h

^ (x) = f h

C nh

n X i=1

r

2

exp

(x xi )2

2

C nh

n X

h

exp

(x xi )2

2

K

i=1

1

^ (x) = f h

nh

n x xi X

h

((

x

xi))

i=1 n X

K

((

x x xi)

xi))

xH xi

h

n xH xi X

K

i=1

h

(

K

=0

i=1

n x xi X

H x =

The estimate is xH

=

m(x)

Pn

i=1

K

xi

Pn

i=1

x xi h

xi

i=1 K

xxi h

This will always be biased towards region with high density. Thus if we were to move along the vector m(x) x,we would reach the region with higher density.The density at m(x) will be greater than density at x. This forms the basis of mean shift algorithm. The vector m(x) x is called the mean shift vector which always points in the direction of local maximum or mode.

m(x)

Pn

x

i=1 K

Pn

xxi (xi h

i=1 K

x

m(x)

2r

h

xxi h

x)

fh (x)

fh (x)

This is a estimate of normalize gradient of fh (x) Given any point x,we can take a small step in the direction of vector m(x) x to reach the local maximum.

5|8

Let us consider that the present estimate of the mode is we compute m(x) at this point.

, then

For examples let initial estimate of the location of mode be (0:96; 2:01) The density at this point can be approximated by interpolation or computed again using non parametric density estimation The plot for this is show in 2.The estimate clearly does not lie at maximum. To find the direction of the mean shift vector we find the gradient of the normalize density estimate and take a smalll step in that direction.This is perform till gradient magnitude is very small A video for mean shift algorithm using KDE is shown in https:

//googledrive.com/host/0B-pfqaQBbAAtNkg2bUJvWERmNFU/a2.avi

In this case we scale the gradient by the estimated PDF values to obtain normalize gradient values.

m(x)

2r

h

fh (x)

fh (x)

This enabled us to adaptively change the step size based on estimated PDF value.The step size magnitude is iversly proportional to estimated PDF values. if the estimated PDF values is small,we are far away from the maximum and the step size will be large. If the estimate PDf value is large,we are close to maximum and the step size will be small.

6|8

to find the modes of the PDF,we do not actually required to estimate the PDF,we require just the gradient of the PDF to move in the direction of the mean shift vector. The gradient of superposition of kernels centered at each data point is equivalent to convolving the data points with gradient of the kernel. Instead of convolving with gaussian kernel,we can convolve with gradient of gaussian kernel.

k (X )

=

x

C exp k (x);

+ y2

k (X )

2

y

i

k (x)

Thus given a intial point X ,we estimate the value at X using the x k (x) and x k (x) which gives us the direction of gradient kernels h h at the point X Since we do not actually estimate the PDF at a point,but estimate the gradient of PDF each time during the mean shift iteration we need to take a step in direction of mean shift vector,in the earlier case ,we used the scale the gradient by the estimated PDF values to obtained a normalized measure. However in the present case we do not adaptively change the step size but take a step of fixed size in direction of the gradient. This still incorporates some adaptive behavior,since mean shift vector magnitude depends on the gradient magnitude. If gradient magnitude is large,step size take will be large else step take will be small and refined ,near the maximum. video of mean shift algorithm using gradient estimates is shown in https://googledrive.com/host/0B-pfqaQBbAAtNkg2bUJvWERmNFU/a3.avi This iterative algorithm is a standard gradient descent algorithm and the convergence is guranteed for infinately small step size. Since the algorithm depends on kernel density estimate, the band-

7|8

Mean Shift Algorithm with of kernel will play a important role in mean shift algorithm as well.

If we reach a region,where local density is flat or we have reached a local maximum.The algorithm will terminate. this is a problem in case of all algorithms trying to reach a global maximum.The animation for the same is shown in https://googledrive. com/host/0B-pfqaQBbAAtNkg2bUJvWERmNFU/a1.avi

Figure 3: Mean shift

0.4 Code

The Code is written in matlab and available in repository https:// github.com/pi19404/m19404/tree/master/meanshift the file mean_shift.m is the main file.The file kgde2 implements kernel density estimator using bivariate gaussian windows for 2D distributions.The file kgde2x implements estimation of gradient on KDE .The dim parameter decides the computation of gradient along x and y directions.

8|8

- Discuss Problems 1Uploaded bygowthamkurri
- Probability And QUeuing Theory formulaUploaded byRavichandran Damodaran
- soa103.pdfUploaded byAnonymous 0B4Duoq
- Exam P SampleUploaded bybezner_paul
- A simple color balance algorithmUploaded bypi194043
- Integral Image for Computation of Mean And VarianceUploaded bypi194043
- question.pdfUploaded byAngela Velez
- Who Has a More Accurate Shot-18Aayushk Boys or GIrlsUploaded byAayushkogar
- HackUploaded byBalaiyyappan Venkatesh
- Edu Exam p Sample QuestUploaded byMim Atcharee
- Assignment 1Uploaded byNurhashimah Za'ba
- Error analysis lecture 3Uploaded byOmegaUser
- Monte Carlo ProbabilityUploaded byanalese1
- Chapter 7Uploaded byAnonymous NUn6MESx
- Key Renew TheoryUploaded byDimas Adip
- S1 CompilationUploaded byAngel Wasi
- pertemuan4_statprobUploaded byBrigitta Angelina
- Probability May 12Uploaded byMaos Wu
- practicetest1-08fUploaded byTam D. Phan
- Patrice Meunier and Emmanuel Villermaux- Enhanced Mixing by VorticesUploaded byPlamcfe
- Chapter9Slide HandUploaded byAnkur Dalal
- A Course in Mathematical StatisticsUploaded byManuel Carmo
- Failure RatesUploaded byaakashtrivedi
- IHistorian Excel Add-InUploaded byimidken

- Modified Canny Edge DetectionUploaded bypi194043
- Polynomial Approximation of 2D image patch -Part 2Uploaded bypi194043
- Compiling Native C/C++ library for AndroidUploaded bypi194043
- Multi Class Logistic Regression Training and TestingUploaded bypi194043
- Gaussian Multivariate Distribution -Part 1Uploaded bypi194043
- ARM Neon Optimization for image interleaving and deinterleavingUploaded bypi194043
- A linear channel filterUploaded bypi194043
- Adaptive Skin Color DetectorUploaded bypi194043
- OpenVision Library Gaussian Mixture Model ImplementationUploaded bypi194043
- Continuous Emission Hidden Markov Model for sequence classificationUploaded bypi194043
- Fast 2D Separable Symmetric/Anti-Symmmetric ConvolutionUploaded bypi194043
- Dense optical flow expansion based on polynomial basis approximationUploaded bypi194043
- Implementation of discrete hidden markov model for sequence classification in C++ using EigenUploaded bypi194043
- Markov chain implementation in C++ using EigenUploaded bypi194043
- C++ InheritanceUploaded bypi194043
- Local Binary PatternUploaded bypi194043
- polynomial approximation of a 2D signalUploaded bypi194043
- C++ Const,Volatile Type QualifiersUploaded bypi194043
- Polynomial Approximation of 1D signalUploaded bypi194043
- Random Ferns for Patch DescriptionUploaded bypi194043
- Integral Image for Computation of Mean And VarianceUploaded bypi194043
- Embedded Systems Programming with ARM on Linux - Blinking LEDUploaded bypi194043
- C++ virtual functions and abstract classUploaded bypi194043
- C++ static members and functionUploaded bypi194043
- Uniform Local Binary Pattern and Spatial Histogram ComputationUploaded bypi194043
- Tan and Triggs Illumination normalizationUploaded bypi194043
- Normalized convolution for image interpolationUploaded bypi194043

- Measures of Central Tendency and Other Positional MeasuresUploaded bybv123bv
- Chapter 3Uploaded byYou Sothea
- Metodologia QuantitativaUploaded byhudson_lima_15
- Design Verification Humased 20Uploaded byRKM
- MMM ManualUploaded byHernan Marin
- Educational StatisticsUploaded byRico A. Regala
- Sop Ngcm Revised 2014 Updated (1)Uploaded byMukteshwar Mishra
- Calidad en El MantenimientoUploaded byDiiana Liidiia
- 2008-teacher_20080925_1435Uploaded byboomjosh12345
- Student Critical Thinking and Reading ComprehensionUploaded byAnggia
- Capitulo4 -PrecipitacaoAtmosfericaUploaded byClelfa Monteiro
- 1a. Ch 1&2 Worksheet AnswersUploaded byMansi Baxi
- CMA 01Uploaded byAdil Anwar
- Organização e tratamento de dados - 8º anoUploaded byAlmartins
- Apostila estatística descriticaUploaded byLuiz Paulo Vieira
- Graphics in C LanguageUploaded byZarnigar Altaf
- Assignment 1Uploaded byAbbaas Alif
- Stats FormulaUploaded bymnadeem_248131
- Arti Dan Kegunaan DataUploaded byPatrick Zwingly
- 261_0305 - Texto 05 - Médidas de Dispersão, exercicios de estatisticaUploaded byJosilda Dias Palmeira
- Dis ModUploaded byDaniela Agundis
- SatasticsUploaded byMuthu Ranganath
- Spring 12 ECON-E370 IU Exam 1 ReviewUploaded byTutoringZone
- PEGEUploaded byMercedes Rossa
- QNT 351.docUploaded byaamto
- Statistics.docxUploaded byJanaka Wanigasekera
- Web Aula 2 EstatisticaUploaded byjana souza
- Probabilidade IUploaded byjeffcan
- Trends in the Statistical Assessment of Reliability.pdfUploaded byPravin Sasatte
- Module10, Data Analysis and InterpretationUploaded byapi-3728125