Fast Block Matching Algorithms for Motion Estimation

Soo-Mok Jung, Sung-Chul Shin, Hyunki Baik, Myong-Soon Park

Internet Computing Laboratory, Department of Computer Science and Engineering Korea University, An-Am Dong, Sung-Buk Gu, Seoul, 136-701, Korea {jsm, scshin, gibson, myongsp}@iLab.korea.ac.kr

Abstract - In this paper, we present fast algorithms to reduce the computation cost of block matching algorithms for motion estimation in video coding. Fast Block Matching

motion vector for each block is estimated by finding the best matching block of pixels within the search window in the previous frame according to matching criteria. Although Full Search (FS) algorithm finds the optimal motion vector by searching exhaustively for the best matching block within the search window, its high computation cost limits its practical applications. To reduce computation cost of FS, many fast block matching algorithms such as three step search, 2-D log search, orthogonal search, cross search, one-

Algorithms for Motion Estimation are based on Multilevel Successive Elimination Algorithm for Block Matching Motion Estimation [1] and Successive Elimination Algorithm [2]. Fast Block Matching Algorithms consist of two

algorithms. One of these algorithms is useful not only for Multilevel Successive Elimination Algorithm but also for all kinds of block matching algorithms. The efficiency of the proposed algorithms is verified by experimental results.

dimensional full search, variation of three-step search, unrestricted center-biased diamond search, and circular

Introduction
There is considerable temporal redundancy in Motion estimation and

zonal search have been developed.

As described in [6],

these algorithms rely on the assumption that the motion-compensated residual error surface is a convex function of the displacement motion vectors, but this assumption is rarely true [7]. Therefore, the best match obtained by these fast algorithms is basically a local optimum. In other words, most fast algorithms reduce computation cost at the expense of the accuracy of the motion estimation. Without this convexity assumption, Successive Elimination Algorithm (SEA) proposed by Li and Salari [2] reduces the computation cost of the FS. To reduce the computation cost of SEA, X. Q. Gao etc. proposed Multilevel Successive Elimination Algorithm (MSEA) [1]. This paper presents Fast Block Matching
Algorithms (FBMA).

consecutive video frames.

compensation techniques have been widely used in image sequence coding schemes to remove temporal redundancy. estimation The accuracy and efficiency of motion affects the efficiency of temporal

redundancy removal. Motion estimation methods are classified into two classes of block matching algorithms (BMA)[3] and pel-recursive algorithms (PRA)[4]. Due to their

implementation simplicity, block matching algorithms have been widely adopted by various video coding standards such as CCITT H.261, ITU-T H.263, and MPEG. In BMA, the current image frame is The

partitioned into fixed-size rectangular blocks.

The motion estimation accuracy

of FBMA is identical to that of

FS and the

SAD(x,y) = ||Bc(i,j) Bp(i,j,x,y)||1 =c

N-1 N-1

computation cost of MSEA is reduced by using FBMA.

|Bc(i,j)(m,n) Bp(i,j,x,y)(m,n)|

m n

2 Multilevel Successive Elimination Algorithm

Let fc(i,j) and fp(i,j) denote the intensity of the pixel with coordinate (i,j) in the current frame and the previous frame respectively. Assume that the size of a block (Y component of the macro block in H.263) is NxN pixels, the search window size is (2M+1)x(2M+1) pixels, and the matching criteria function is Sum of Absolute Difference (SAD) which is the distortion between two blocks. Let Bc(i,j) and Bp(i,j,x,y) denote the

R and M(x,y) are sum norms and are pre-computed using the efficient procedure described in [1]. In MSEA, each block is partitioned into several subblocks. First, the block is partitioned into four sub-

blocks with size N/2 x N/2. Then each sub-block is partitioned into four sub-blocks with size N/4 x N/4. This procedure can be repeated until the size of the subblocks become 2x2. The maximum level of such partition is Lmax=log2N-1 for the blocks with size N x N. The MSEA with level-L partition, 0^ L ^Lmax, is called level-L MSEA, and the SEA[2] corresponds to the level-0 MSEA. At lth level of the level-L partition, where 0 ^ l ^L, the number of the sub-blocks is Sl = 22l, and each sub-block is of size Nl x Nl, where Nl = N/2l.
2 -1 2 -1
l l

target block and compared block in the current frame and the previous frame with the top left corners at (i,j) and (i-x, j-y) respectively. which requires motion vector. Bc(i,j) (m,n)= fc(i+m, j+n) Bp(i,j,x,y) (m,n) = fp(i-x+m, j-y+n) motion vector, -M (x,y) M and 0 (m,n) N-1. SAD between the two blocks is defined as: SAD(x,y) (1) (2) Bc(i,j) is the target block

If we denote (6)

Where x and y represent two components of a candidate The

SAD_SBl = |Rl(u,v) Ml(u,v)(x,y)|

u=0 v=0 Nl-1 Nl-

where Rl(u,v) =

Bc(m+uNl, n+vNl),
m=0 n=0 Nl-1 Nl-1 m=0 n=0

c c
m=0

N-1

N-1

Ml(u,v) (x,y)= (m,n) | (3)

Bp(m+uNl, n+vNl)

| Bc

(i,j)

(m,n) - Bp

(i,j,x,y)

n=0

The goal of motion estimation is to find the best pair of indices (x,y) so that the following sum of absolute difference is minimized, as follows: d = min SAD(x,y) x,y (4)

[1] shows that equation (5) can be expressed as equation (7) for level-L MSEA. SAD_SB0 = | R-M(x,y) | SAD_SBl-1 SAD_SBl

SAD_SBL SAD(x,y)
where 0 l L.

(7)

Applying mathematical inequality | ||X||1 - ||Y||1 | ||X Y||1 [8] for X= Bc(i,j) and Y= Bp(i,j,x,y) gives | R-M(x,y) | SAD(x,y) where, R = ||Bc ||1 =c
(i,j)
N-1 N-1

level-L MSEA procedure is as follows: 1 select initial candidate motion vector within the search window in the previous frame. 2 calculate SAD at the selected point, current minimum SAD(curr_min_SAD) = SAD 3 select another candidate motion vector among the rest of the search points 4.0 calculate SAD_SB0 at the selected search point if (curr_min_SAD ) SAD_SB0) goto 7

(5)
(i,j)

m n

c Bc

(m,n),

M(x,y) = ||Bp(i,j,x,y)||1 =c

N-1 N-1

m n

c Bp(i,j,x,y)(m,n),

4.1 calculate SAD_SB1 at the selected search point if (curr_min_SAD ^) SAD_SB1) goto 7 4.L calculate SAD_SBL at the selected search point if (curr_min_SAD ^) SAD_SBL) goto 7 5 6 calculate SAD at the selected search point if (SAD < curr_min_SAD) curr_min_SAD=SAD

minimum SAD, the point (x,y) cannot be the optimum motion vector and the remainder of the sum does not need to be calculated. We denote our PDE technique used in SAD_SBl calculation as PDEsb to differentiate from PDE technique used in SAD calculation (symbolized: PDEsad). While it is not efficient to test the partial sum against the current minimum SAD every time an additional term is added, a reasonable compromise is to perform the test after each sub-block row as shown in table 1. In level-3 MSEA, a block consists of 8 sub-block rows and each sub-block row consists of 8 sub-blocks. So, 8 times PDE test is

7 if (all the search points in the search window are not tested?) 8 goto 3

minimum SAD=curr_min_SAD, calculate motion vector

The MSEA speeds up the process of finding the motion vector by eliminating hierarchically impossible candidate motion vectors in the search window before their SAD calculation.

executed in SAD_SB3 calculation.

3.1.1

Experimental Results

The standard video sequence, salesman.qcif, is used in experiment and we tested 100 frames of the

Fast Block Matching Algorithms for Motion Elimination

sequence.

The block size is 16x16 pixels (N=16). The

size of search window is 31x31 pixels (M=15) and only integer values for the motion vectors are considered.

3.1 Further Computational Reductions Partial distortion elimination (PDE) [9] is an effective speedup technique used in vector quantization to find the best reconstruction vector from a set of vector code-words. PDE technique in SAD

All our experiments(3.1.1, 3.2.1) are executed under these conditions. Experimental results are shown in

table 1 and table2. In table 1, (x row) means that the PDEsb test is executed at each x-th sub-block row and simple notation MSEAL means level-L MSEA. In

calculation is the strategy employed in the full search performed in the Telenor implementation of Test Model 1.4a distributed as part of the H.263 standardization effort. In MSEA, PDE technique was used in SAD calculation. But, PDE technique was not used in

tables, m.e. means matching evaluation that require SAD calculation, avg. # of rows means the number of calculated row in SAD calculation before partial distortion elimination. Overhead(in rows) is the sum of all the computations such as the sum norm

computations by using the efficient method described in [1], the computations of step 4, and the computations of PDEsb etc. but except for SAD calculation. In tables, in rows means that the computations are represented in order of 1 row SAD computations. It is important to notice that with the MSEA, the efficiency of the

SAD_SBl calculation. So, PDE technique can be used with the level-L MSEA to reduce the computations further where SAD_SBl must be computed. In

equation (6) and step 4.x in MSEA, if at any sub-blocks the partially evaluated sum exceeds the current

procedure depends on the order in which the candidate motion vectors are searched, and that the most likely candidates should be tested first. This eliminates the maximum number of candidates. In our experiment, we used circular zonal search [5] to find the motion vector. The method MSEA+PDEsb, which incorporates the MSEA with PDEsb, reduces the computations of MSEA by 0.4%, 2.8%, 6.2% for MSEA1, MSEA2, and MSEA3 respectively for this fairly large search window.

absolute difference vales are centered together. This property is shown in table 3. If we calculate SAD

from large absolute difference vales to small absolute difference values, partial distortion elimination in SAD calculation can be done very early. So, the

computations of SAD calculation can be reduced. To calculate SAD Adaptively for each block, we divided each block into 4 sub-blocks with size of 8x8 pixels and sampled 8 points at each sub-block and calculated the sum of absolute difference for the

Table 1. Computations of the MSEA3+PDEsb for several PDEsb test points

Algorithm MSEA3+PDEsb (1 row) MSEA3+PDEsb(2 row) MSEA3+PDEsb(4 row) Avg. # of m.e./frame 242.3 242.3 242.3 Avg. # of rows/m.e. 14.39 14.39 14.39 Overhead (in rows) 27,479.6 27,719.9 28,277.9 Total (in rows) 30,966.3 31,206.6 31,764.6

sampled pixels for each sub block as shown in Fig. 1. Sampling points at each sub-block must be chosen to represent the SAD of its sub-blocks The sum of

sampled absolute difference values at each sub-blocks are sorted to determine the order of sub-blocks to be used in SAD calculation. First of all, we calculate

Table 2. Computations of the FS algorithm and MSEA with PDEsb

Algorithm FS FS+PDEsad MSEA0(SEA) MSEA1 MSEA1+PDEsb MSEA2 MSEA2+PDEsb MSEA3 MSEA3+PDEsb Avg. # of m.e./frame 77,439.0 77,439.0 24,995.6 8,769.7 8,769.7 1,589.1 1589.1 242.3 242.3 Avg. # of rows/me 16.00 4.36 5.79 7.84 7.84 10.33 10.33 14.39 14.39 Overhead (in rows) 0.00 6135.0 8653.9 14,851.5 14,530.3 23,552.8 22,434.5 29,510.6 27,479.6 Total (in rows) 1,239,024.0 343,596.8 153,378.4 83,605.9 83,284.7 39,968.2 38,849.9 32,997.3 30,966.3 6.2% 2.8% 0.4% reduction

SAD at the sub-block of which the sum of sampled absolute difference values is greatest, and then we calculated SAD at the sub-blocks following the ordered sequence.
X X X X X X

3.2

Adaptive SAD Calculation calculation requires very intensive 3.2.1

Fig. 1. Block division and sampling

SAD

computations. SAD calculation must be done at many matching evaluation points per frame as shown in table 2. To find the method reducing the computations of

Experimental results

As maximum to minimum SAD ratio increases, the partial distortion elimination is achieved early in Adaptive SAD Calculation algorithm. Table 3 shows that a considerable portion of the blocks have a ratio that is very large. The total blocks of table 3 that are checked for 100 frames are 7,734,000. Adaptive SAD Calculation algorithm(symbolized: SADadap) reduces

SAD calculation, we investigated absolute difference values of |Bc(i,j)(m,n) Bp(i,j,x,y)(m,n)| for 0 ^ m,n ^ N1. There are 251 absolute difference values at each block. These values range from 0 to maximum pixel intensity value. One outstandi`ng feature is that large

the computations of MSEA by 10.6%, 7.2%, 6.2%, 3.0%, 0.4% for FS+PDEsad, MSEA0, MSEA1, MSEA2, and MSEA3 respectively.

good coding quality.

References
1. X. Q. Gao, C.J. Duanmu, and C.R. Zou: A Multilevel Successive Elimination Algorithm for Block Matching Motion Estimation. IEEE Trans. Image Processing, Vol. 9, No.3, pp.501-504, Mar.

Table 3. The distribution of blocks according to the ratio of (maximum SAD of sub-blocks/minimum SAD of sub-blocks)












2.1

1.6

3.2

9.1

41.6

10.1

11.2

11.3

8.0

1.8

2000. 2. W. Li and E. Salari: Successive elimination algorithm for motion estimation. IEEE Trans. Image Processing, vol. 4, No.1, pp. 105-107, Jan.

(unit: %) Table 4. Computations of Adaptive SAD Calculation algorithm

Algorithm FS+SADadap MSEA0+SADadap MSEA1+SADadap MSEA2+ SADadap MSEA3+ SADadap Avg. # of m.e./frame 77,439.0 24,995.6 8769.7 1,589.1 242.3 Avg. # of rows/m.e . 3.90 5.27 7.16 9.50 13.87 overhead (in rows) 5,095.4 10,631.2 15,603.2 23,673.2 29,521.0 Total (in rows) 307,107.5 142,358.0 78,394.3 38,769.7 32,881.7 reduction 10.6% 7.2% 6.2% 3.0% 0.4%

1995. 3. J. R. Jain and A. K. Jain: Displacement measurement and its application in interframe image coding. IEEE Trans. Commun., vol. COMM-29, pp. 1799-1808, 4. Dec. 1981.

A. N. Netravali and J. D. Robbins: Motion compensated television coding: Part I. Bell Syst. Tech. J., vol. 58, pp. 631-670, Mar. 1979.

Adaptive

SAD

Calculation

algorithm

can be

combined with not only MSEA but also all kinds of block matching algorithms to reduce the computations of motion estimation. 5.

J. N. Kim and T. S. Choi: A fast motion estimation for software based real-time video coding. IEEE Trans. on Consumer Electronics, vol.

4. Conclusions Fast Block Matching of the Algorithms, Multilevel which are 6.

45, no. 2, pp. 417-425, May 1999. B. Liu and A. Zaccarin: New fast algorithms for the estimation of block motion vectors. IEEE Trans. Circuits Syst. Video Technol., vol. 3, no. 2, pp. 148-157, Apr. 1993. 7. K. H. K. Chow and M. L. Liou: Genetic motion search algorithm for video compression. IEEE Trans. Circuits Syst. Video Technol., vol. 3, pp. 440-445, Dec. 1993. 8. T. M. Apostol: Mathematical Analysis. Reading, MA: Addison-Wesley, 1975. 9. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1991. Successive

improved

version

Elimination Algorithm, have been proposed to reduce the computations of block matching algorithms for motion estimation in video coding. Further

Computational Reduction is only useful in MSEA, but Adaptive SAD Calculation algorithm can be combined with not only the MSEA but also all kinds of block matching algorithms to reduce computations. This

FBMA can provide computations reduction over the MSEA, while keeping the same motion estimation accuracy as the FS. FBMA are very efficient solution for video coding applications that require both very low bit-rate and