Motion Estimation Techniques

Attribution Non-Commercial (BY-NC)

352 views

Motion Estimation Techniques

Attribution Non-Commercial (BY-NC)

- Codecs
- Motion Estimation
- Motion Estimation
- Video COdec Design Developing Image and Video Compression Systems - Iain Richardson
- Line buffer
- DigitalVideoforTeachersCourseAdvertSL0
- 06.07.Fast 3D Stabilization and Mosaic Construction
- k3ccdtools_manual
- 23. Electronics - IJECE - A Digital images Steganography.pdf
- 11g_vldb
- Avigilon Product Brochure
- DIGIZON_UserManual_200703e
- 2011-02-19 Nokia C2-01
- Evolutionary Synthesis of Lossless Compression Algorithms the GP-zip Family
- Document 2
- PowerDirector14 Tutorial Book Enu
- icip95.pdf
- 300-011-114
- Notebook HPg42 340br
- Guelph Standards

You are on page 1of 4

in video processing

By Milind Phadtare

System Architect, Video

NXP Semiconductors India Pvt. Ltd

form the core of video com-

pression and video processing

applications. Motion estimation

extracts motion information from

the video sequence. The motion

is typically represented using a

motion vector (x,y). The motion

vector indicates the displacement

of a pixel or a pixel block from the

current location due to motion.

Motion information is used in

video compression to find best Figure 1: Block matching algorithm.

matching block in reference

frame to calculate low energy in figure 1. The current frame other criteria also available such compression. But, the required

residue, used in scan rate con- is divided into pixel blocks and as cross correlation, maximum computations are prohibitively

version to generate temporally motion estimation is performed matching pixel count etc. high due to the large amount of

interpolated frames. It is also used independently for each pixel The reference pixel blocks are candidates to evaluate. The num-

in applications such motion block. Motion estimation is done generated only from a region ber of candidates to evaluate are

compensated de-interlacing, by identifying a pixel block from known as the search area. Search (2Sx+1)*(2Sy+1). Hence, full search

video stabilisation, motion track- the reference frame that best region defines the boundary for is typically not used. Also, it does

ing etc. matches the current block, whose the motion vectors and limits the not guarantee consistent motion

Varieties of motion estimation motion is being estimated. The number of blocks to evaluate. The vectors required for video process-

techniques are available. There reference pixel block is generated height and width of the search ing applications.

are pel-recursive techniques, by displacement from the current region is dependant on the There are several other fast

which derive motion vector for block’s location in the reference motion in video sequence. The block-matching algorithms,

each pixel. There is the phase frame. The displacement is pro- available computing power also which reduce the number of eval-

plane correlation technique, vided by the Motion Vector (MV). determines the search range. Big- uated candidates yet try to keep

which generates motion vectors MV consists of is a pair (x, y) of ger search region requires more a good block matching accuracy.

via correlation between current horizontal and vertical displace- computation due to increase in Note that since these algorithms

frame and reference frame. 2D ment values. number of evaluated candidates. test only limited candidates,

DFT is used for this. However, the There are various criteria avail- Typically the search region is they might result in selecting a

most popular technique is Block able for calculating block match- kept wider (i.e. width is more candidate corresponding to local

Matching Algorithm. This article ing. Two popular criteria are listed than height) since many video minima, unlike full search, which

mainly discusses this technique. bellow sequences often exhibit pan- always results in global minima.

Sum of Square Error (SSE) = ning motion. The search region Some of the algorithms are

Block Matching Algorithm can also be changed adaptively listed bellow.

Block Matching Algorithm (BMA) is depending upon the detected

the most popular motion estima- motion. The horizontal and verti- Three step search

tion algorithm. Sum of Absolute Difference cal search range, Sx & Sy, define In a three-step search (TSS) algo-

BMA calculates motion vec- (SAD) = the search area (+/-Sx and +/- Sy) rithm, the first iteration evaluates

tor for an entire block of pixels as illustrated in figure 1. nine candidates as shown in

instead of individual pixels. The figure 2. The candidates are cen-

same motion vector is applicable Full search block matching tred around the current block’s

to all the pixels in the block. This SSE provides a more accu- Full search block matching algo- position. The step size for the fist

reduces computational require- rate block matching, however rithm evaluates every possible iteration is typically set to half the

ment and also results in a more requires more computations. pixel block in the search region. search range. During the next it-

accurate motion vector since SAD provides fairly good match Hence, it can generate the best eration, the search centre is shifted

the objects are typically a cluster at lower computational require- block matching motion vec- to the best matching candidate

of pixels. ment. Hence it is widely used for tor. This type of BMA can give from the first iteration. Also, the

BMA algorithm is illustrated block matching. There are various least possible residue for video step size is reduced by half. The

selected for the current block. The

number of evaluated candidate

is variable for the 2D logarithmic

search. However, the worst case

and best case candidates can be

calculated.

Algorithm

The one at a time search algorithm

estimates the x-component and

the y-component of the motion

vector independently. The can-

didate search is first performed

along the x-axis. During each itera-

tion, a set of three neighbouring

candidates along the x-axis are

tested. The three-candidate set is

shifted towards the best matching

candidate, with the best matching

candidate forming the centre of

the set for the next iteration. The

process stops if the best match-

ing candidate happens to be the

centre of the candidate set.

The location of this candidate

on the x-axis is used as the x-com-

ponent of the motion vector. The

search now continues parallel to

the y-axis. A procedure similar to

x-axis search is followed to esti-

mate y-component of the motion

Figure 2: Fast block matching algorithms. vector. One-step at a time search

on average tests less number of

same process continues till the the second iteration, the centre iteration. Hence, there is no need candidates. However, the motion

step size becomes equal to one of the diamond is shifted to the for block matching calculation for vector accuracy is poor.

pixel. This is the last iteration of best matching candidate. The these candidates during the sec- The required computations

the three-step search algorithm. step size is reduced by half only if ond iteration. The results from the are significantly reduced due

The best matching candidate from the best candidate happens to be first iteration can be used for these to the above fast algorithms.

this iteration is selected as the the centre of the diamond. If the candidates. The process continues However, for higher resolutions,

final candidate. The motion vector best candidate is not the diamond till the step size becomes equal it might still be necessary to fur-

corresponding to this candidate is centre, same step size is used even to one pixel. For this iteration all ther reduce computations to fit

selected for the current block. The for second iteration. In this case, eight surrounding candidates real time requirements. Typically

number of candidates evaluated some of the diamond candidates are evaluated. The best matching this is achieved by using sub-

during three-step search is very are already evaluated during first candidate from this iteration is sampled SAD calculation and

less compared to the full search

algorithm. The number of evalu-

ated candidate is fixed depending

upon the step size set during the

first iteration.

2D logarithmic search

2D Logarithmic search is an-

other algorithm, which tests lim-

ited candidates. It is similar to the

three-step search. During the first

iteration, a total of five candidates

are tested. The candidates are

centred around the current block

location in a diamond shape. The

step size for first iteration is set

equal to half the search range. For Figure 3: Sub-pixel motion estimation.

stabilisation, it is more useful to

find global motion rather than

local motion. In global motion,

the same type of motion is ap-

plicable to each pixel in the video

frame. Some examples of global

motion are panning, tilting and

zoom in/out. In all these motion,

each pixel is moving using the

same global motion model. The

motion vectors for each pixel or

pixel block can be described using

following parametric model with

four parameters

pixel or pixel block is given a

Figure 4: Hierarchical block matching algorithm. only q0 and q1 are non-zero i.e.

constant motion vector for the

skipping alternate blocks for mo- sub-pixel grid. The sub-pixel grid the highest resolution is reduced entire video frame. For pure zoom

tion estimation. For sub-sampled can be either at half pixel resolu- to (N/2)x(N/2) in the next resolu- in/out, only p0 and p1 will be non-

SAD calculation only alternate tion or quarter pixel resolution. tion level. Similarly, the search zero. However a combination of all

samples in vertical and horizon- The reference block at sub-pixel range is also reduced. The mo- the parameters is usually present.

tal direction are used. It reduces grid is generated using either tion estimation process starts at Global motion estimation involves

the SAD computation by ¼. The bi-linear interpolation or a more the lowest resolution. Typically, calculation of the four parameters

motion estimation can be per- sophisticated six-tap filter as used full search motion estimation is in the model (p0,p1,q0,q1). The

formed only for alternate blocks. in H.264 standard. Normally, the performed for each block at the parameters can be calculated by

The motion vectors for missing integer motion vector is further lowest resolution. Since the block treating them as four unknowns.

blocks can be generated via inter- refined to first half pixel by testing size and the search range are Hence, ideally sample motion

polation from the neighbouring the eight neighbouring half pixel reduced, it does not require large vectors at four different locations

block’s motion vectors. locations and further to quarter computations. The motion vectors can be used to calculate the four

The choice of block size also pixel by testing the neighbouring from lowest resolution are scaled unknown parameters. In prac-

determines the trade off between eight quarter pixel locations. This and passed on as candidate mo- tice though, more processing is

required computation and accu- is illustrated in figure 3. tion vectors for each block to next needed to get good estimate for

racy of motion vectors. Smaller level. At the next level, the motion the parameters. Also, note that still

block size can accurately describe Hierarchical block matching vectors are refined with a smaller local motion estimation, at least

motion of even smaller objects Hierarchical Block Matching Al- search area. A simpler motion at four locations, is essential to

but will need higher computa- gorithm is a more sophisticated estimation algorithm and a small calculate the global motion estima-

tion. E.g. H.264 video coding stan- motion estimation technique. Hi- search range is enough at close tion parameters. However, there

dard allows a block size of 4x4. erarchical Block Matching motion to highest resolution since the are algorithms for global motion

Typical block sizes are 16x16, 8x8 estimation provides consistent motion vectors are already close estimation, which do not rely on

and 4x4. Note that a block size motion vectors by successively to accurate motion vectors. local motion estimation. The above

lower than 4x4 may not contain refining the motion vector at dif- parametric model with four param-

sufficient texture, hence, it may ferent resolutions. In Hierarchical Global motion estimation eters cannot fit rotational global

not be possible to accurately motion estimation, a pyramid of There is another type of motion motion. For rotational motion a

predict block matching. reduced resolution video frame is estimation technique know as six-parameter model is needed.

formed. The original video frame global motion estimation. The However, the same four-parameter

Sub-pixel motion estimation forms the highest resolution im- motion estimation techniques model concepts can be extended

The actual motion in the video se- age and the other images in the discussed so far are useful in es- to the six-parameter model.

quence can be much finer. Hence, pyramid are formed by down timating local motion (i.e. motion

the resulting object might not lie sampling the original image. A of objects within the video frame). True motion estimation

on the integer pixel grid. To get a simple bi-linear down sampling However, the video sequence can For video compression applica-

better match, the motion estima- can be used. This is illustrated in also contain global motion. For tions it is enough to get a motion

tion needs to be performed on a figure 4. The block size of NxN at some applications, such as video vector corresponding to best

consists of only spatial & temporal

neighbouring motion vectors. This

results in a very consistent motion

vector field giving true motion. To

kick-start the algorithm a random

motion vector is also used as a

candidate. This is illustrated in

figure 5.

So far the reference frame is as-

sumed to be from the past and the

motion estimation is performed in

forward direction. However, the

motion estimation can also be

performed in backward direction

using a future frame as a reference.

This is useful in video sequences

where new objects enter the

scene or background is being

uncovered. In both the cases the

matching content is available only

in the future video frames. The oc-

clusion problem can be avoided

by using both forward and back-

ward motion estimation.

Object-based estimation

Finally, the object based motion

Figure 5: Three dimensional recursive search (3DRS) algorithm. estimation provides the most

advanced motion estimation.

match. This in turns results in best possible match. True motion – objects are larger than block Here, the video sequence is seg-

lower residual energy and better estimation can be achieved via size and objects have inertia. The mented into objects instead of

compression. However, for video both post-processing the motion first assumption suggests that pixel blocks. Motion of each object

processing applications, especially vectors to get smooth motion the neighbouring block’s motion is independently estimated. This

for scan rate conversion, true mo- vector field as well as building the vectors can be used as candidates provides the best possible true

tion estimation is desired. In True consistency measures in motion for the current block. However, motion estimation helping both

Motion estimation, the motion estimation algorithm itself. Three for neighbouring blocks ahead video compression and scan

vectors should represent true Dimensional Recursive Search in raster scan, there is no motion rate conversion. However, object

motion of the objects in the video (3DRS) is one such algorithm vectors calculated yet. Here, the segmentation remains the most

sequence rather than providing where the consistency assump- second assumption is applied complex processing step, which

best block match. Hence, it is tion is inbuilt into the motion and motion vectors from previous caused the object based motion

important to achieve a consistent estimation. The algorithm works frame are for these blocks. 3DRS estimation to be still out of most

motion vector field rather that on two important assumptions motion estimator’s candidate set of the consumer devices.

- CodecsUploaded byarpan47
- Motion EstimationUploaded byDavid Reeves
- Motion EstimationUploaded byIvan Rodrigo Molano García
- Video COdec Design Developing Image and Video Compression Systems - Iain RichardsonUploaded byphaneendra208
- Line bufferUploaded byTuan Huynh
- DigitalVideoforTeachersCourseAdvertSL0Uploaded byRon Belshaw
- 06.07.Fast 3D Stabilization and Mosaic ConstructionUploaded byAlessio
- k3ccdtools_manualUploaded bycvincenzi440
- 23. Electronics - IJECE - A Digital images Steganography.pdfUploaded byiaset123
- 11g_vldbUploaded byDudi Kumar
- Avigilon Product BrochureUploaded byDan Birla
- DIGIZON_UserManual_200703eUploaded byAnnie Li
- 2011-02-19 Nokia C2-01Uploaded byv_sap
- Evolutionary Synthesis of Lossless Compression Algorithms the GP-zip FamilyUploaded byInno Odira
- Document 2Uploaded byAnonymous uUFj1s48n
- PowerDirector14 Tutorial Book EnuUploaded by하준호
- icip95.pdfUploaded bynhanlh
- 300-011-114Uploaded bylinuxir
- Notebook HPg42 340brUploaded byBeto Quirino
- Guelph StandardsUploaded bySamantha Nacion
- Compression and Information Leakage of PlaintextUploaded byandygen21
- Compression summaryUploaded byMaria Deborah Vivas Baluran
- Νέο Microsoft Office Word DocumentUploaded byChristina Tsekoura
- Www.samsungpresenter.comUploaded byRaghvendra Mani
- SNO-1080RUploaded bynk1969
- 013233 FL - Photographic DocumentationUploaded byAyman Kandeel
- 36594.7678329Uploaded byjraz141
- Amr CodecsUploaded byUmar Pervaiz
- p12Uploaded byKamarul Nizam
- Thermocom V52Uploaded byknightfelix12

- Black Magic JohnsonUploaded bycpsrinivasu1221569
- Power EstimationUploaded bychandra sekhar
- SPI SlidesUploaded bychandra sekhar
- 8. 16- Bit RISC Processor Design for Convolution Application Using Verilog HDLUploaded bychandra sekhar
- cpld_n_fpga_1Uploaded bynavneetan89
- 4. High Speed of Complex Multiplier Based on Urdhva-tiryakbyham Sutra Verilog HDLUploaded bychandra sekhar
- 1.an Efficient Implementation of Floating Point MultiplierUploaded byfusionmaster
- PAP2_UserGuideUploaded bychandra sekhar
- Introduction to CPLD&FPGAUploaded bychandra sekhar
- Paton - Fundamentals of Digital Electronics With LabviewUploaded bylakicar2
- Matlab NotesUploaded bychandra sekhar
- MPEG Video CodingUploaded bychandra sekhar
- NelsonF09xUploaded bychandra sekhar
- Bahari09Uploaded bychandra sekhar
- nps29Uploaded bycruzzader
- Lecture NotesUploaded bytoymaker09
- H.264 MPEG4 TutorialUploaded bychandra sekhar
- VHDL CookbookUploaded byRoshan
- Collaborative Fingerprint Authentication by Smart Card and a Trusted HostUploaded bychandra sekhar
- Text-To-Speech Technology-Based Programming Tool Final DocUploaded bychandra sekhar
- Video Compression—From Concepts to theUploaded bychandra sekhar
- Soumyendu Steganography Steganalysis Different ApproachesUploaded bychandra sekhar
- Domino LogicUploaded bySathish Kumar
- QuotationUploaded bychandra sekhar

- Ch9.docUploaded byAnonymous XR7iq7
- An exercise in color, or "reinventing the wheel"Uploaded byPaula Wirth
- assessment 1Uploaded byapi-302170302
- Design of an Algorithm for Video Mosaicing Using Cross-CorrelationUploaded byInternational Journal for Scientific Research and Development - IJSRD
- assignment 1 part 2Uploaded byapi-297329449
- Experiment 7Uploaded byAnish Bansal
- Diagnosis of Vertical Root Fracture Using Digital Radiography, Helical Computed Tomography And Cone Beam Computed Tomography.Uploaded byIOSRjournal
- A Dualistic Sub-Image Histogram Equalization Based Enhancement and Segmentation Techniques with NN for Medical ImagesUploaded byinventy
- Panasonic TC-P50U50 CNET review calibration reportUploaded byDavid Katzmaier
- Image Enhancement using CLAHE-DWT TechniqueUploaded byIJRASETPublications
- A Robust Feature-based Registration Method of Multimodal Image Using Phase Congruency and Coherent Point DriftUploaded byRahman Khan
- An Effective Implementation of Configurable Motion Estimation Architecture for Block Matching AlgorithmUploaded byInternational Journal for Scientific Research and Development - IJSRD
- Indian Currency Denomination IdentificationUploaded byJegnaw Fentahun
- lung cancer dtection using DIPUploaded bytarun
- Lung Cancer Detection on CT ImagesUploaded byAndreea Speriatu
- Image CompressionUploaded bynam_pic
- EC2029 DIP SyllabusUploaded bySebastin Suresh
- glspec40.core.20100311Uploaded bygennoveus
- ImageProcessing1-Introduction.pptUploaded bySamreen Zeeshan
- Object Detection using the Canny Edge Detector.docUploaded byArivazhagan Art
- Detection of Malignant Tissues by Segmentation of Histology Images using Histograms of Color and Filter ResponsesUploaded byIJIRST
- 70184306-ec2029Uploaded byaishuvc1822
- Assignment Photoshop MOHAMMAD NAZRUL HADIUploaded byMd Nzrul Hadi
- Image Processing Toolbox - Texture Segmentation Using Texture Filters Demo - MathWorks IndiaUploaded byChiranjit Ghosh
- Spatial Filtering Image ProcessingUploaded bySankalp_Kallakur_402
- CompressionUploaded bySreejith Km
- [2006] Perceptually-Inspired and Edge-Directed Color Image Super-ResolutionUploaded byThais Pedruzzi
- MGE Rev PreviewUploaded byJesús Palacios Serna
- syllabus-2013 medical imagingUploaded byBrian Liang
- DIP QuestionsUploaded byNithish Kumar