3.1K views

Uploaded by pi194043

Dense optical flow expansion based on polynomial basis approximation

save

- OpenVision Library Gaussian Mixture Model Implementation
- Adaptive Skin Color Detector
- ARM Neon Optimization for image interleaving and deinterleaving
- A linear channel filter
- Fast 2D Separable Symmetric/Anti-Symmmetric Convolution
- Implementation of discrete hidden markov model for sequence classification in C++ using Eigen
- Normalized convolution for image interpolation
- Polynomial Approximation of 1D signal
- Modified Canny Edge Detection
- polynomial approximation of a 2D signal
- Polynomial Approximation of 2D image patch -Part 2
- Image enhancement using Fusion
- OpenCL Image Convolution Filter - Box Filter
- Gaussian Multivariate Distribution -Part 1
- Multi Class Logistic Regression Training and Testing
- Control Limited Adaptive Histogram Equalization for Image Enhancement
- Continuous Emission Hidden Markov Model for sequence classification
- Spatio Temporal Feature extraction using harris 3d corner detector
- OpenCL Heterogenenous program for Image Processing - ColorSpace conversion BGR-HSV,HSV-BGR,BGR-GRAY
- A detailed descriptions and results for different color constancy algorithms
- Markov chain implementation in C++ using Eigen
- Fast 3D Extension of Fast Feature Detection for SpatioTemporal Corner Detection
- shape classification using Histogram of oriented gradients
- OpenCL 2D Convolution Using Separable Filters -Box Filter
- Android OpenCV Face detection
- Gesture Recognition using Hidden Markov Mode
- temporal filters
- Feature Detection - Overview of Harris Corner Feature Detection
- SubPixel Corner Localization Algorithm
- Fast Asymmetric Learning for Cascade Face Detection Training/Testing Utility
- Continuous Emission Hidden Markov Model for sequence classification
- Implementation of discrete hidden markov model for sequence classification in C++ using Eigen
- Markov chain implementation in C++ using Eigen
- Polynomial Approximation of 1D signal
- Modified Canny Edge Detection
- polynomial approximation of a 2D signal
- Polynomial Approximation of 2D image patch -Part 2
- Gaussian Multivariate Distribution -Part 1
- Multi Class Logistic Regression Training and Testing
- Compiling Native C/C++ library for Android
- C++ virtual functions and abstract class
- C++ static members and function
- Uniform Local Binary Pattern and Spatial Histogram Computation
- Mean Shift Tracking
- Normalized convolution for image interpolation
- Tan and Triggs Illumination normalization
- Mean Shift Algorithm
- Local Binary Pattern
- C++ Inheritance
- C++ Const,Volatile Type Qualifiers
- Integral Image for Computation of Mean And Variance
- Random Ferns for Patch Description
- Embedded Systems Programming with ARM on Linux - Blinking LED
- C++ Class Members and friends
- Smoke and Fire Detection
- HDLogix 3D IQ Overview 001
- Finger Mouse Project Report
- Detection and Tracking of Moving Object: A Survey
- Gi 3511181122
- DETECTION & TRACKING OF MOVING OBJECT
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks.pdf
- A Survey on Approaches for Object Tracking
- Face Liveness Detection for Biometric Antispoofing Applications using Color Texture and Distortion Analysis Features
- tmp8828.tmp
- Report Drdo
- Learning to track for spatio-temporal action localization
- OpenCV C# Wrapper Based Video Enhencement Using Different Optical Flow Methods in the Super Resolution
- Ego-motion estimation using optical flow
- 84 1512499393_05-12-2017.pdf
- Bio Inspired Flow for Vis Guidance of Micro Uav Sys
- The Autonomous Detection and Tracking of Moving Objects - A Survey Work
- Virtual Room Visualizer
- Vision-based Robotics a Challenge for Artificial Intelligence
- Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
- Real Time Stabilization Turbulent Video Jr Tip
- hw4
- LITERATURE SURVEY On Moving Object Detection.docx
- The State of VR in 2017
- Control of two robotic platforms using multi-agent systems
- OpenCV By Example - Sample Chapter
- A Bayesian Approach for Image-based Underwater Target Tracking and Navigation
- Eye blink
- Driver Drowsiness Detection
- Poster ThucTrinhLE Videoinpainting ICIP

You are on page 1of 12

Pi19404

March 5, 2014

Contents

Contents

Dense Motion Estimation based on Polynomial expansion

0.1 Introduction 0.2 Introduction 0.3 Conclusion . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

3 3 11 11

2 | 12

0.1 Introduction

In this article we will look at dense motion estimation based on polymonial repsentation of image.The polynomial basis representation of the image is obtained by approximating the local neighborhood of image using quadratic polynomial basis.The displacement between adjacent frames can be obtained by equating the coefficients of the basis.

0.2 Introduction

This article describes a fast dense optical flow computation algorithm by Farnebck, 2002 . In the earlier articles it was seen that a local neighborhood of image can be represented using polynomial basis. Using this representation estimation of dense optical flow is obtained at each point in the image.

Assuming how a polynomial transforms under translation using the polynomial expansion coefficients derived from current and previous frames and estimate of displacement vector is obtained. The idea of polynomial expansion is to approximate a neighborhood of a point in a 2D function with a polynomial.Considering quadratic polynomial basis 1; x2 ; y 2 ; x; y; xy ,pixel values in a neighborhood of image is represented by

f x

( ) xT Ax + bT x + c

The coefficients can be estimated by weighted least square estimate of pixel values about neighborhood as seen in the earlier

3 | 12

Figure 1: Polynomial basis

4 | 12

Dense Motion Estimation based on Polynomial expansion article. As with all optical flow algorithm,the brightness constancy assumption is made. The brighness of a path of image in adjacent frames is constant.

Consider a translational motion d ecountered at point (x; y) in the image.

( ) = xT A1 x + bT 1 x + c1 T T f2 (x) = f1 (x d) = (x d) A1 (x d) + b1 (x d) + c1 T T f2 (x) = f1 (x d) = (x) A1 (x) + (b1 2A1 d) (x) + dT Ad bT 1 d + c1 T T f2 (x) = f1 (x d) = (x) A1 (x) + b2 (x) + c2

f1 x

A2 b2 c2

= A1 = (b1 2A1 d) = dT Ad bT 1 d + c1

A

Assuming

d

is non-singular

1 A 1(b2 b1 ) 2

Thus by equating the coefficients of the polynomial the displacement vector can be obtained at each point in the image assuming there is overlap between the region of interest ie image neighborhood in adjacent frames.

We extract Let us say we have the estimate of displacement d the ROI about the neighborhood at point P (x; y ) and point P (x + d:x; y + d:y ) The polynomial basis are extracted and the computation that is shown above is performed.

The total displacement can be estimated as

d

= 0:5 A1 (b2 + b2 b1 )

Thus an iterative scheme can be used where in every successive iteration a better estimate of displacement vector is obtained.The iterations can be terminated when change is displacement vector is below is threshold in successive iterations or

5 | 12

Dense Motion Estimation based on Polynomial expansion specific number of iterations have been completed. The intial estimate of displacement vector is assumed to be (0; 0) Thus ROI or the image patch in the current and previous frames are at the same.

6 | 12

/** * @brief updatePoly : The function computes the optical flow displacement * @param ptr1 : pointer to array containing the polynomial basis componen * @param ptr2 : pointer to array containing the polynomial basis componen * @param d : estimate of displacement vector at the current point * @param flag : the flag indicating if the point is border pixels or not * @param M : pointer to output array returning the computed coefficien * @param x : co-ordinate at the current point where the coefficients i * @param y : * @param s : windows size for averaging */ void updatePoly(const float *ptr1,const float *ptr2,Point2f d,bool flag, float *M,Point p,Size s) { int x=p.x; int y=p.y; const int BORDER = 5; static const float border[BORDER] = {0.14f, 0.14f, 0.4472f, 0.4472f, 0 float r[5]; float r2,r3,r4,r5,r6; if(flag==true) { //average A_1 and A_2 r4=(ptr1[2]+ptr2[2])*0.5; r5=(ptr1[3]+ptr2[3])*0.5; r6=(ptr1[4]+ptr2[4])*0.25; } else { r[0]=0.f; r[1]=0.f; r[2]=ptr1[2]; r[3]=ptr1[3]; r[4]=ptr1[4]*0.5; r2=r[0]; r3=r[1]; r4=r[2]; r5=r[3]; r6=ptr1[4]*0.5; }

7 | 12

//computing -(b1-b2) r2=(ptr1[0]-ptr2[0])*0.5; r3=(ptr1[1]-ptr2[1])*0.5; //sum for iterative estimation b2=b2+\bar{b2} //r2 += r4*d.y + r6*d.x; ///r3 += r6*d.y + r5*d.x; r2 += r4*d.x + r6*d.y; r3 += r6*d.x + r5*d.y; if( (unsigned)(x - BORDER) >= (unsigned)(s.width - BORDER*2) || (unsigned)(y - BORDER) >= (unsigned)(s.height - BORDER*2)) { float scale = (x < BORDER ? border[x] : 1.f)* (x >= s.width - BORDER ? border[s.width - x - 1] : 1.f)* (y < BORDER ? border[y] : 1.f)* (y >= s.height - BORDER ? border[s.height - y - 1] : 1.f); r2 *= scale; r3 *= scale; r4 *= scale; r5 *= scale; r6 *= scale;

//computing final displacement d=A^-1(b2-b1)*0.5 M[0] = r4*r4 + r6*r6; // G(1,1) M[1] = (r4 + r5)*r6; // G(1,2)=G(2,1) M[2] = r5*r5 + r6*r6; // G(2,2) M[3] = r4*r2 + r6*r3; // h(1) M[4] = r6*r2 + r5*r3; // h(2)

The method EstimateFlow computes the coefficients A; b2 ; b1 required for displacentt field computation.The EstimateFlow functions call the method UpdatePoly for each pixel in the image. The displacement field obtained may be discontinuous and contain noise and other atrifacts.Since it is reasonable to assume that if motion is encounted at a point, the neighborhood pixels also encounter the same motion. The displacement vector can be averaged over a neighborhood to get a better estimate of the displacement field.

8 | 12

The method AverageFlow computes the average of coefficients A1 ; b2 ; b1 and then computes the displacement flow field.

/** * @brief AverageFlow * @param _R0 : Polynomial basis of prev frame * @param _R1 : Polynomial basis coefficients of current frame * @param _flow :estimate of current displacement field * @param matM :containing coefficients of polynomial basis for computing */ void AverageFlow( const Mat& _R0, const Mat& _R1,Mat& _flow, Mat& matM) { int x, y, i, width = _flow.cols, height = _flow.rows; //computing the average flow field cv::GaussianBlur(matM,matM,Size(15,15),sigma); for( y = 0; y < height; y++ ) { double g11, g12, g22, h1, h2; float* flow = (float*)(_flow.data + _flow.step*y); float *coeff=(float *)(matM.data+matM.step*y); // update borders for( x = 0; x < width; x++ ) { g11 = coeff[x*5]; g12 = coeff[x*5+1]; g22 = coeff[x*5+2]; h1 = coeff[x*5+3]; h2 = coeff[x*5+4]; //computing determinant for inverse double idet = 1./(g11*g22 - g12*g12 + 1e-3); //computing displacement flow field flow[x*2] = (float)((g11*h2-g12*h1)*idet); flow[x*2+1] = (float)((g22*h1-g12*h2)*idet); } } //calling EstimateFlow for updading coefficients //for computation of next iteration EstimateFlow(_R0, _R1, _flow, matM,0,flow.rows);

9 | 12

}

This approach may in case of large displaement.Hence a multi scale estimation is performed. The estimation of flow field is performed as the smallest resolution.The displacement computed at the lower resolution is used as estimate for peform displacement field computation at higher resolution. Dense optial flow computed for two frames is shown in figure 2c

(a) Frame 1

(b) Frame 2

Figure 2: Displacement field

10 | 12

0.3 Conclusion

This article describes the theory and implementation details of the dense optical flow algorithm based on paper by Farnebck, 2002.This code for the algorithm can be found at github repository https://github.com/pi19404/OpenVision in files DenseOf.cpp and DenseOf.hpp files. In the future article we will look at optimizing the code using SSE,NEOM and OpenCL optimizations to enable real time computation of the dense optical flow fields

11 | 12

Bibliography

Bibliography

[1] Kenneth Andersson and Hans Knutsson. Continuous normalized convolution. In: ICME (1). IEEE, 2002, pp. 725728.

dblp.uni-trier.de/db/conf/icmcs/icme2002-1.html#AnderssonK02.

[2]

isbn:

0-7803-7304-9.

url: http://

Kenneth Andersson, Carl-Fredrik Westin, and Hans Knutsson. Prediction from o-grid samples using continuous normalized convolution. In: Signal Processing 87.3 (Mar. 22, 2007), pp. 353365.

EX-1596. MA thesis. SE-581 83 Linkping, Sweden: Linkping University, 1996.

[3]

[4]

Gunnar Farnebck. Polynomial Expansion for Orientation and Motion Estimation. Dissertation No 790, ISBN 91-7373-475-6. PhD thesis. SE-581 83 Linkping, Sweden: Linkping University, Sweden, 2002.

[5]

Gunnar Farneback. Two-Frame Motion Estimation Based on Polynomial Expansion. In: SCIA. LNCS 2749. Gothenburg, Sweden, 2003, pp. 363370.

12 | 12

- OpenVision Library Gaussian Mixture Model ImplementationUploaded bypi194043
- Adaptive Skin Color DetectorUploaded bypi194043
- ARM Neon Optimization for image interleaving and deinterleavingUploaded bypi194043
- A linear channel filterUploaded bypi194043
- Fast 2D Separable Symmetric/Anti-Symmmetric ConvolutionUploaded bypi194043
- Implementation of discrete hidden markov model for sequence classification in C++ using EigenUploaded bypi194043
- Normalized convolution for image interpolationUploaded bypi194043
- Polynomial Approximation of 1D signalUploaded bypi194043
- Modified Canny Edge DetectionUploaded bypi194043
- polynomial approximation of a 2D signalUploaded bypi194043
- Polynomial Approximation of 2D image patch -Part 2Uploaded bypi194043
- Image enhancement using FusionUploaded bypi194043
- OpenCL Image Convolution Filter - Box FilterUploaded bypi194043
- Gaussian Multivariate Distribution -Part 1Uploaded bypi194043
- Multi Class Logistic Regression Training and TestingUploaded bypi194043
- Control Limited Adaptive Histogram Equalization for Image EnhancementUploaded bypi194043
- Continuous Emission Hidden Markov Model for sequence classificationUploaded bypi194043
- Spatio Temporal Feature extraction using harris 3d corner detectorUploaded bypi194043
- OpenCL Heterogenenous program for Image Processing - ColorSpace conversion BGR-HSV,HSV-BGR,BGR-GRAYUploaded bypi194043
- A detailed descriptions and results for different color constancy algorithmsUploaded bypi194043
- Markov chain implementation in C++ using EigenUploaded bypi194043
- Fast 3D Extension of Fast Feature Detection for SpatioTemporal Corner DetectionUploaded bypi194043
- shape classification using Histogram of oriented gradientsUploaded bypi194043
- OpenCL 2D Convolution Using Separable Filters -Box FilterUploaded bypi194043
- Android OpenCV Face detectionUploaded bypi194043
- Gesture Recognition using Hidden Markov ModeUploaded bypi194043
- temporal filtersUploaded bypi194043
- Feature Detection - Overview of Harris Corner Feature DetectionUploaded bypi194043
- SubPixel Corner Localization AlgorithmUploaded bypi194043
- Fast Asymmetric Learning for Cascade Face Detection Training/Testing UtilityUploaded bypi194043

- Continuous Emission Hidden Markov Model for sequence classificationUploaded bypi194043
- Implementation of discrete hidden markov model for sequence classification in C++ using EigenUploaded bypi194043
- Markov chain implementation in C++ using EigenUploaded bypi194043
- Polynomial Approximation of 1D signalUploaded bypi194043
- Modified Canny Edge DetectionUploaded bypi194043
- polynomial approximation of a 2D signalUploaded bypi194043
- Polynomial Approximation of 2D image patch -Part 2Uploaded bypi194043
- Gaussian Multivariate Distribution -Part 1Uploaded bypi194043
- Multi Class Logistic Regression Training and TestingUploaded bypi194043
- Compiling Native C/C++ library for AndroidUploaded bypi194043
- C++ virtual functions and abstract classUploaded bypi194043
- C++ static members and functionUploaded bypi194043
- Uniform Local Binary Pattern and Spatial Histogram ComputationUploaded bypi194043
- Mean Shift TrackingUploaded bypi194043
- Normalized convolution for image interpolationUploaded bypi194043
- Tan and Triggs Illumination normalizationUploaded bypi194043
- Mean Shift AlgorithmUploaded bypi194043
- Local Binary PatternUploaded bypi194043
- C++ InheritanceUploaded bypi194043
- C++ Const,Volatile Type QualifiersUploaded bypi194043
- Integral Image for Computation of Mean And VarianceUploaded bypi194043
- Random Ferns for Patch DescriptionUploaded bypi194043
- Embedded Systems Programming with ARM on Linux - Blinking LEDUploaded bypi194043
- C++ Class Members and friendsUploaded bypi194043

- Smoke and Fire DetectionUploaded byNum Gip
- HDLogix 3D IQ Overview 001Uploaded byBurak Şekercioğlu
- Finger Mouse Project ReportUploaded byanupamdubey
- Detection and Tracking of Moving Object: A SurveyUploaded byAnonymous 7VPPkWS8O
- Gi 3511181122Uploaded byAnonymous 7VPPkWS8O
- DETECTION & TRACKING OF MOVING OBJECTUploaded byIJIRAE
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks.pdfUploaded bypatilrushal824
- A Survey on Approaches for Object TrackingUploaded byjournal
- Face Liveness Detection for Biometric Antispoofing Applications using Color Texture and Distortion Analysis FeaturesUploaded byEditor IJRITCC
- tmp8828.tmpUploaded byFrontiers
- Report DrdoUploaded bysaurabhkumar_
- Learning to track for spatio-temporal action localizationUploaded bySatyam JA
- OpenCV C# Wrapper Based Video Enhencement Using Different Optical Flow Methods in the Super ResolutionUploaded byAnonymous rp8A9j
- Ego-motion estimation using optical flowUploaded byDhillonv
- 84 1512499393_05-12-2017.pdfUploaded byAnonymous lPvvgiQjR
- Bio Inspired Flow for Vis Guidance of Micro Uav SysUploaded byAni Dalal
- The Autonomous Detection and Tracking of Moving Objects - A Survey WorkUploaded byInternational Journal of Computer Science and Mobile Computing - IJCSMC
- Virtual Room VisualizerUploaded byIJSTE
- Vision-based Robotics a Challenge for Artificial IntelligenceUploaded byHimanshu gupta
- Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine TransformationUploaded byNaachiz Faqeer
- Real Time Stabilization Turbulent Video Jr TipUploaded byAnonymous FGY7go
- hw4Uploaded bySanjay Shah
- LITERATURE SURVEY On Moving Object Detection.docxUploaded byChetan Turkane
- The State of VR in 2017Uploaded byTom Ffiske
- Control of two robotic platforms using multi-agent systemsUploaded byJaime Andres Rincon Arango
- OpenCV By Example - Sample ChapterUploaded byPackt Publishing
- A Bayesian Approach for Image-based Underwater Target Tracking and NavigationUploaded byBogdan Boc
- Eye blinkUploaded byRekha Manjunath
- Driver Drowsiness DetectionUploaded byLaveen Prabhu S
- Poster ThucTrinhLE Videoinpainting ICIPUploaded byDung Nguyen