Attribution Non-Commercial (BY-NC)

97 views

Attribution Non-Commercial (BY-NC)

- Scene Change Detection Algorithms & Techniques: A Survey
- Motion Estimation
- Motion Estimation Techniques
- Extc- Sem Viii
- Real Time YCoCg DXT Compression
- 2.Cotii Paper Cotii 2014 Ec 091 Final
- IC-COD-REP020 Understanding H.264 Video
- Comp Slides8
- reference for text on dsp
- 2183201
- beee_thesis.pdf
- Project
- How to Add and Burn Subtitles to Avi Permanently
- IJAIEM-2014-04-30-101
- 12-JCIT1-804299[1]
- Homework 2 CSE 101
- Multimedia Production and Web Authoring Day1
- 85 Video Compress
- dsp ex2print
- Network engineer or wireless engineer or system engineer or tech

You are on page 1of 29

Outline

Motion estimation for video coding Block-based motion estimation:

EBMA algorithm, Integer accuracy EBMA algorithm, Half-pel accuracy HBMA

Frame t-1

Frame t

Adjacent frames are similar and changes are due to object or camera motion --- Temporal correlation

Yao Wang, 2003 EE4414: Motion Estimation Basics 3

Frame 66

Frame 69

Abs olute Difference w/o Motion Compens ation

Abs olute Difference with Motion Compens ation

Predict a new frame from a previous frame and only code the prediction error Prediction error will be coded using an image coding method (e.g., DCT-based as in JPEG) Prediction errors have smaller energy than the original pixel values and can be coded with fewer bits Those regions that cannot be predicted well will be coded directly using DCT-based method Use motion-compensated prediction to account for object motion Work on each macroblock (MB) (16x16 pixels) independently for reduced complexity

Motion compensation done at the MB level DCT coding of error at the block level (8x8 pixels)

EBMA/ HBMA

MV

Search Region

Yao Wang, 2003 EE4414: Motion Estimation Basics

7

For each MB in a new (predicted) frame

Search for a block in a reference frame that has the lowest matching error

Using sum of absolute errors between corresponding pels Search range: depends on the anticipated motion range Search step-size: integer-pel or half-pel

EDFD (d m ) =

xBm

2 ( x + d m ) 1 ( x)

min

Displacement between the current MB and the best matching MB is the MV Current MB is replaced by the best matching MB (motioncompensated prediction or motion compensation)

Yao Wang, 2003 EE4414: Motion Estimation Basics 8

Target frame Rx Ry dm Bm Best match Search region Bm Current block Anchor frame

%f1: anchor frame; f2: target frame, fp: predicted image; %mvx,mvy: store the MV image %widthxheight: image size; N: block size, R: search range for i=1:N:height-N, for j=1:N:width-N %for every block in the anchor frame MAD_min=256*N*N;mvx=0;mvy=0; for k=-R:1:R, for l=-R:1:R %for every search candidate MAD=sum(sum(abs(f1(i:i+N-1,j:j+N-1)-f2(i+k:i+k+N-1,j+l:j+l+N-1)))); % calculate MAD for this candidate if MAD<MAX_min MAD_min=MAD,dy=k,dx=l; end; end;end; fp(i:i+N-1,j:j+N-1)= f2(i+dy:i+dy+N-1,j+dx:j+dx+N-1); %put the best matching block in the predicted image iblk=(floor)(i-1)/N+1; jblk=(floor)(j-1)/N+1; %block index mvx(iblk,jblk)=dx; mvy(iblk,jblk)=dy; %record the estimated MV end;end;

Note: A real working program needs to check whether a pixel in the candidate matching block falls outside the image boundary and such pixel should not count in MAD. This program is meant to illustrate the main operations involved. Not the actual working matlab script. Yao Wang, 2003 EE4414: Motion Estimation Basics 10

Frame 66

Frame 69

P redicted Frame69

11

Frame 69

P redicted Frame69

12

Assumption

Image size: MxM Block size: NxN Search range: (-R,R) in each dimension Search stepsize: 1 pixel (assuming integer MV)

Each candidate position: N^2 Each block going through all candidates: (2R+1)^2 N^2 Entire frame: (M/N)^2 (2R+1)^2 N^2=M^2 (2R+1)^2 Independent of block size!

Total operation count = 2.85x10^8/frame =8.55x10^9/second

Regular structure suitable for VLSI implementation Challenging for software-only implementation

EE4414: Motion Estimation Basics 13

Real MV may not always be multiples of pixels. To allow subpixel MV, the search stepsize must be less than 1 pixel Half-pel EBMA: stepsize=1/2 pixel in both dimension

Move the search block pixel to left/bottom each time

Difficulty:

Target frame only have integer pels

Solution:

Interpolate the target frame by factor of two before searching Bilinear interpolation is typically used

Complexity:

4 times of integer-pel, plus additional operations for interpolation.

Fast algorithms:

Search in integer precisions first, then refine in a small search region in half-pel accuracy.

14

dm

Bm : matching block

15

Bilinear Interpolation

(x,y) (x+1,y) (2x,2y) (2x+1,2y)

(2x,2y+1) (2x+1,2y+1)

(x,y+!)

(x+1,y+1)

16

Show matlab script for half-pel EBMA

17

Yao Wang, 2003 EE4414: Motion EstimationEBMA Basics Example: Half-pel 18

anchor frame

Problems with BMA

Unless exhaustive search is used, the solution may not be global minimum Exhaustive search requires extremely large computation Block wise translation motion model is not always appropriate

Multiresolution approach

Aim to solve the first two problems First estimate the motion in a coarse resolution over low-pass filtered, down-sampled image pair

Can usually lead to a solution close to the true motion field

Then modify the initial solution in successively finer resolution within a small search range

Reduce the computation

Can be applied to different motion representations, but we will focus on its application to BMA -> Hierarchical BMA (HBMA)

Yao Wang, 2003 EE4414: Motion Estimation Basics 19

1,1

2,1 d1

d2 1,2 2,2 d2 q2

d3

d3

From [Wang02]

(a)

(b)

(c)

(d)

(e)

(f)

From [Wang02]

Yao Wang, 2003 EE4414: Motion EstimationEBMA Basics Example: Half-pel 22

anchor frame

Show matlab script for HBMA

23

Blocking effect (discontinuity across block boundary) in the predicted image

Because the block-wise translation model is not accurate Fix: Deformable BMA (next lecture)

because MVs are estimated independently from block to block Fix 1: Mesh-based motion estimation (next lecture) Fix 2: Imposing smoothness constraint explicitly

because motion is indeterminate when spatial gradient is near zero

Because its simplicity and optimality in minimizing prediction error

24

VcDemo Example

VcDemo: Image and Video Compression Learning Tool Developed at Delft University of Technology http://www-ict.its.tudelft.nl/~inald/vcdemo/ Use the ME tool to show the motion estimation results with different parameter choices

25

EBMA/ HBMA

26

Uni-directional Motion Compensation:

f$ ( t , m, n ) = f ( t 1, m d x , n d y )

Bi-directional Motion Compensation

Can handle better covered/uncovered regions

f$ ( t , m, n ) = wb f ( t 1, m d b , x , n d b , y ) + w f f ( t + 1, m d f , x , n d f , y )

Two sets of motion vectors can be estimated independently, using BMA algorithms twice For better results, the two motion vectors should be searched simultaneously, by minimizing the bi-directional prediction error

27

How does EBMA works? What is the effect of block size, search range, search accuracy (integer vs. half-pel) in terms of prediction accuracy and computation time? What is the benefit of hierarchical block matching method? How is BMA used in video coding?

28

References

Y. Wang, J. Ostermann, Y. Q. Zhang, Video Processing and Communications, Prentice Hall, 2002. chapter 6 Y. Wang, EL514 Lab Manual on Video Coding.

29

- Scene Change Detection Algorithms & Techniques: A SurveyUploaded byijcsis
- Motion EstimationUploaded byDavid Reeves
- Motion Estimation TechniquesUploaded bychandra sekhar
- Extc- Sem ViiiUploaded byRudolf Dsilva
- Real Time YCoCg DXT CompressionUploaded byterminatory808
- 2.Cotii Paper Cotii 2014 Ec 091 FinalUploaded bykhananu
- IC-COD-REP020 Understanding H.264 VideoUploaded byschumangel
- Comp Slides8Uploaded byAjay Kumar
- reference for text on dspUploaded bychetansrinidhi
- 2183201Uploaded byShivam Rajgor
- beee_thesis.pdfUploaded byzavirk
- ProjectUploaded byBashir Khan
- How to Add and Burn Subtitles to Avi PermanentlyUploaded byArmando Martin
- IJAIEM-2014-04-30-101Uploaded byAnonymous vQrJlEN
- 12-JCIT1-804299[1]Uploaded bypriyakavukalai
- Homework 2 CSE 101Uploaded byKel
- Multimedia Production and Web Authoring Day1Uploaded bytechtrainphp
- 85 Video CompressUploaded byvaibhav0206
- dsp ex2printUploaded byzunaidur_rashid
- Network engineer or wireless engineer or system engineer or techUploaded byapi-121391088
- AJA and Adobe Mercury EngineUploaded byZigund
- An H.264 Video Encryption Algorithm Based On Entropy CodingUploaded byhendra
- Minggu3-Lesson03_2Uploaded byIka Juniar Kristiani Hutapea
- Image Compression Part1Uploaded byMaheswaran Umaiyorupagan
- Time Domain Methods1 (1)Uploaded byMala Manjunatha
- Assignment_2 Video Phone System Design_vaibhav Mathur (Sc08b107)Uploaded byVaibhav Mathur
- 01031943_2Uploaded byalireza17sh1
- Hv 2514131415Uploaded byAnonymous 7VPPkWS8O
- Data Hiding JUploaded byshanmugaraja85
- Data Hiding J.docxUploaded byshanmugaraja85

- Server IronUploaded byHugo Cesar Flores Ortiz
- Putty.log Csmsapp1Uploaded byRinkesh Tripathi
- Serv Dev Sidhi SadhnaUploaded bysandy_108
- Assigment Industrial Instrumentation - Capacitive Comparison BridgeUploaded byAriff Ikhzam Mohd Mufaradzi
- Introduction to SafeNet Presentation EnUploaded byAkin Akinmosin
- MECHANICAL DRAFTINGUploaded bysurper
- RFC 3986 - Uniform Resource Identifier (URI)_ Generic Syntax (RFC3986)Uploaded bybelacqua16
- DOC-20180131-WA0002.pdfUploaded byIchigo
- StatoilOffshoreCranes.pdfUploaded byfazeli
- PH4500-AR_BN68-05796A-04ENG-0602.pdfUploaded bymasnoen randi
- 189 Ideas to improve your photography business photographer - Photography Business and Marketing Ideas.pdfUploaded byLuis D. Mendoza Pérez
- anchorsUploaded byneelchandhh
- Aspen Cloud Installation Guide_CCP_v1.9.4Uploaded byAnonymous GK9mHwnr
- Examples of Table Borders and RulesUploaded byporur
- TechGdeUploaded byRomi Fadly
- Online Banking GHuaUploaded bydanielbillion
- handbook polytechnicUploaded byYuriShibuya1993
- How to Install Business Object 11Uploaded byFanoZalfano
- Be124 Centurion Generator Control System USER ManualUploaded byChris Antoniou
- 3_dynatest_fwd_hwdUploaded byDidik Sulistya
- 334906844-E-b-o-o-k-0976836491-Basics-Retaining-Wall-Design-10th.pdfUploaded bybenedictoespinozaqui
- Engineering_Drawing.pdfUploaded bySajad ahmad sheikh
- term project - eportfolioUploaded byapi-251981709
- Q meterUploaded byscbpathak
- Research CapsuleUploaded byHazel Paguio-Lastrollo
- Sudoku SolutionUploaded byAlan Veeck
- Online Travel Services: The Antitrust Assault on Orbitz--and on Consumers, Cato Policy Analysis No. 441Uploaded byCato Institute
- CS467-textbook-Machine Learning-ktustudents.in.pdfUploaded byAmmu Mole
- TOSHIBA_Q9PLUS_VFD.pdfUploaded byMeagan Jones
- Building a Bitx20 Version 3Uploaded byDjordje Mandic