You are on page 1of 6

KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 617

DIGITAL IMAGE STABILIZING ALGORITHMS BASED ON BIT-PLANE MATCHING
Sung-Jea KO,Senior Member IEEE, Sung-Hee Lee and Kyung-Hoon Lee
Department of Electronics Engineering, Korea University
5-1 Anam-Dong, Sungbuk-Ku, Seoul 136-701, Korea
Email: sjko@dali.korea.ac.kr

Abntract-In this paper, we present a new digital image The full-search(FS) BMA under the mean absolute dif-
stabilization (DIS) scheme based on bit-plane matching ference (MAD) and mean square error (MSE) criteria
(BPM). The proposed DIS system performs motion es-
timation using 1-bit planes which are extracted from a
can be considered as an optimal solution for motion es-
video sequence. This motion estimation technique can be timation [6],[7]. However, the FS BMA requires large
realized using only Boolean functions which have signif- amount of computations which causes time delay, and
icantly reduced computational complexity, while the ac-
requires complex hardware architecture [7]-[9].In this
curacy of motion estimation is maintained. In the sec-
ond part of this paper, a median-based motion correction paper, we present a new motion estimation technique
scheme is proposed which is robust to various irregular based on the bit-plane matching (BPM) for the DIS sys-
conditions such as moving objects and intentional pan- tem. The proposed algorithm performs binary motion
ning. Simulation results show that the proposed DIS al-
gorithm exhibits better performance compared with ex-
estimation using 1-bit planes which are extracted from
isting other algorithms when applied to real video signal. a video sequence. This motion estimation technique can
be realized using only Boolean functions which have sig-
nificantly reduced computational complexity, while the
I. INTRODUCTION accuracy of motion estimation is maintained. Simulation
Image stabilization is the process of generating a com- results show that the performance of the BPM-based mo-
pensated video sequence where image motion by the tion estimation algorithm is comparable to that of the FS
camera’s undesirable shake or jiggle is removed [1]-[5]. BMA.
The recent DIS systems are realized using digital im- Various algorithms have been developed to estimate
age processing techniques instead of mechanical motion the global motion of a frame from local motion vectors
detection techniques using gyro sensors or fluid prism [3]-[7],[11]. Most of these algorithms are complicated,
PI-[51. and thus are not simple to implement. In this paper, we
The image stabilization task can be subdivided into propose a simple and robust decision algorithm for deter-
two basic systems, namely: i) the motion estimation sys- mining the global motion vector. In the proposed algo-
tem and ii) the motion correction system. In general, the rithm, the global motion vector of a frame is determined
motion estimation system generates several local motion based on the order statistics of current local motion vec-
vectors from subimages in the different position of the tors and past global motion vectors. It is shown that the
frame using a block matching algorithm (BMA). The DIS system using the proposed motion estimation and
motion correction system determines the global motion correction algorithms is less sensitive to irregular condi-
of a frame by appropriately processing these local mo- tions such as moving objects and intentional panning.
tion vectors, and decides whether the motion of a frame This paper is orgainzed as follows: The proposed mo-
is caused by undesirable fluctuation of the camera or in- tion estimation and correction algorithms are presented
tentional panning. The stabilized image is generated by in Sections I1 and 111, respectively. Simulation results
reading out the proper block of fluctuated image in the are given in Section 1V and concluding remarks are in
frame memory [3]-[5]. Section V.

This work was supported by SAMSUNG Electronics Co.

Manuscript received June 17, 1998 0098 3063/98 $10.00 1998 IEEE

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
618 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

Fig. 1. Example of generating bit-planes from a gray-sclae image, (a) an original image, (b) eight bit-plane images.

11. MOTIONESTIMATION USINGBIT-PLANE IMAGES the bit-plane as shown in Fig. 2. Each motion vector of a
Before describing the proposed method to estimate lo- subimage in the current bit-plane image is determined by
cal motion vectors, we introduce the bit-plane decompo- evaluating bit-plane matching over subimages in the pre-
sition of a gray-scale image. vious bit-plane and selecting the subimage which yields
the closest matching. This approach assumes that all
A . B i t - p l a n e Decomposition of a Grayscale I m a g e pixels within the subimage have uniform motion and the
Let the graylevel of the pixel at location (z,y) in the range of the motion vector is constrained by the search
t-th image frame with 2K graylevels be represented as window.
Let the size of each subimage be M x N and a search
+
window be ( M 2p) x ( N + 2q). For bit-plane matching,
we define the correlation measure given by
where a k , O 5 h 5 K - 1, is either 0 or 1. Let the k-
th order bit-plane image be denoted by b t k ( z , y ) . This ~ ( m n),= C
b k t ( z , y) CB bkt-'(z m, y n) (2) + +
plane contains all the k-th order (ak)bits. For the case (",Y)ES.

of the 8-bit image, an image is composed of eight 1-
where bkt(a,y) and bkt-'(z, y), respectively, are the cur-
bit planes bk(z,y) N b$(z,y), ranging from plane 0 t o
rent and previous k-th order bit-planes, and @ is the
plane 7. Fig. 1 shows eight bit-planes decomposed from
exclusive-OR operation.
a grayscale image. bi(z,y) contains all the least signif-
At each ( m , n ) ,-p 5 m 5 p and -q 5 n 5 q, within
icant (lowest order) bits comprising pixels in the image
the search range, the proposed matching method calcu-
and b$(a, y) contains all the most significant (highest-
lates C i ( m , n )which is the number of unmatched bits
order) bits. Note that only the higher order bit-plane im-
between the reference subimage in the current bit-plane
ages contain visually significant data whereas the other
and the compared subimage in the previous bit-plane.
bit-planes contribute to more subtle details within the
The smallest Ci(m,n) yields the best matching for Si,
image.
and thus local motion vector V; from Si is obtained as
B. M o t i o n E s t i m a t i o n Based o n t h e Bit-plane M a t c h i n g
vit = arg min{Ci(m, n),- p 5 m 5 p , -q 5 n 5 q } . (3)
Local motion vectors are estimated from four subim-
ages (SI,S2, Ss, 5 4 ) placed in appropriate positions in

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 619

! preuioiis bit-plane image
I-_-._ i
_ _ _ _ - - _ _ I - _ _ _

Fig. 2. Estimation of local motion vectors from four subimages in a bit-plane.

This motion estimation technique can replace the and the motion compensation unit. The motion correc-
arithmetic calculations of BMA's based on conventional tion system determines the global motion of a frame by
MAD and MSE criteria with simple Boolean exclusive- appropriately processing local motion vectors, and de-
OR operations, and thus has significantly reduced com- cides whether the motion of a frame is caused by un-
putational complexity. desirable fluctuation of the camera or intenti6nal pan-
Since the proposed DIS system performs motion esti- ning. The stabilized image is generated by reading out
mation using a single bit-plane, it is important to select the proper block of fluctuated image in the frame mem-
an appropriate bit-plane for bit-plane matching. In this ory.
paper, the 4-th order bit-plane, b;(z,y), is utilized to In an image with motion, some subimages with mov-
estimate the local motion vector since it contains both ing objects can produce motion vectors which are signif-
the global information and details of the original image. icantly different from the other motion vectors. Fig. 5
To show that b:(z, y) is suitable for motion estimation shows an image which has moving objects in some subim-
based on BPM, an example for BPM using 1-D signals ages. Fig. 6(a) and (b) show the correlation measures
is presented in Fig. 3.. Fig. 3(a) shows an 1-D signal calculated using (2) from subimage SI which has no
which is one horizontal scan line of a real image with moving object and subimage S4 with moving objects,
256 graylevels, and Fig. 3(b) is the same scan line from
the previous frame. Using (2), we computed the correla-
tion measure between two binary signals from the scan
*
respectively. In Fig. 6, for display, the correlation mea-
sures are normalized using l - where C,,, is the
maximum Cj(m,n) within the search range. It is seen
lines at each bit-level. Fig. 3(c) illustrates the simulation that there does not exist a distinct maximal correlation
results. It is seen that the correlation measure from the value in S4.
4-th bit-level exhibits the steepest gradient around the In general, motion vectors from the subimages with
minimum point. moving objects are not reliable and should be excluded
from the global motion decision process. Moreover, since
111. MOTIONCORRECTION UNDER IRREGULAR
the hand movement is relatively slow than the frame rate
CONDITIONS of the video camera, two successive frames fluctuated by
In this section, we introduce the motion correction sys- camera's shake should have similar global motion.
tem to cope with irregular conditions such as moving Based on these properties of camera's motion, we pro-
objects and intentional panning that degrade the perfor- pose a simple and robust motion correction scheme where
mance of the DIS system. global motion decision is performed using current local
Fig. 4 shows the proposed motion correction system motion vectors (V:, Vi, Vi,V:) and the previous global
which consists of the decision unit, the integration unit,

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.
620 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

-8 .o -4 -2 0 2 4 0 B

horizontal displacement

(c>
Fig. 3. An example of BPM using 1-D signals, (a) a 1-D signal which is one horizontal scan line of a real image, (b) the same scan line
from the previouse frame, ( c ) the correlation measure between two binary signals from the scan lines at each bit-level. (+: bit-level
6, 0: bit-level 5, *: bit-level 4, 0 : bit-level 3, X : bit-level 2).

Fig. 4. Basic structure of the motion correction system.

motion vector V,"-'. In the proposed algorithm, the It is known that the median filter is very effective in elim-
global motion vector is obtained by inating impulses. Therefore, the median-based method
in (4) can exclude such abrupt local motion vectors and
Vi = median{Vi, Vi, V i ,V i ,v,"-'} (4) produce a global motion vector similar to the previous
one.
Here the median of vectors is determined by seperately After determining the global motion vector, the mo-
selecting medians of each vector elements. tion correction system decides whether the motion of a
Local motion vectors affected by undesirable condi- frame is caused by camera's motion or intentional pan-
tions such as moving objects can be viewed as impulses. ning. For this decision, the global motion vector of a
KO, Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 62 1

Fig. 5. An image with moving objects.

(4 (b)
Fig. 6. Correaltion measures, (a) from subimage S1 which has no motion, and (b) from subimage Sb which contains moving objects.

frame is integrated with a damping coefficient, and the TABLE I
integrated motion vector designates the final motion vec- RMSE’S ASSOCIATED WITH LOCAL MOTION YECTORS.

tor of a frame for motion correction. The integrated mo-
Sequence number RMSE
tion vector V, for estimating intentional panning is given
I
RPM I
I
EPM I BPM
I
by 1 I 0.0334 I 0.3285 1 0.0205
vat = DIVat-l + V,t (5) 2 0.0373 0.4886 0.0333
3 0.0296 0.3000 0.0219
where Q t is a global motion vector and D1(0 < D1 < 1) 0.0518 0.3409 0.0362
4
is a damping coefficient for smooth panning. I

1 0.0373 1 0.5002 1 0.0229
5

IV. SIMULATION
RESULTS
To evaluate the motion estimation performance of the
proposed BPM algorithm, we compare it with two exist-
ing DIS algorithms, namly, representative point match-
ing (RPM) [3] and edge pattern matching (EPM) [4]. 150 frames are utilized for simulation.
The performance is evaluated using the root mean square The RMSE’s associated with the local motion vector
error (RMSE) based on the FS BMA under the MAD are summarized in Table I. It is seen that BPM exhibits
criterion. The RMSE is given by better performance than RPM and EPM.
Table I1 summarizes the RMSE’s associated with
global motion vectors. It is interesting to observe that
the RMSE’s of BPM are always smaller than those of
RPM and EPM.
where (zn,yn) is the motion vector from the FS BMA, These experimental results indicate that the proposed
and (&, ijn) is that from aformentioned algorithms. Five BPM-based DIS system exhibits good performance com-
real image sequences with a resolution of 640 x 240 and parable to the FS BMA.
622 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

TABLE I1 o n Circuits and Systems for Video Technology, vol. 7, no. 2,
RMSE’S ASSOCIATED WITH GLOBAL MOTION VECTORS. pp. 429-433, Apr. 1992.
[9] H. Gharavi and M. Mills, “Blodunatchingmotion estimation
Sequence number algorithms-new results,” IEEE B a n d . o n Circuits and Sys-
RMSE
tems, vol. 37, no. 5, pp. 649-651, May 1990.
RPM EPM BPM
[lo] Y. T.Tse and R. L. Baker, “Global zoom/pan estimation and
1 0.0285 0.1223 0.0178 compensation for video compression,” Proceedings of ICASSP
2 0.0308 0.2082 0.0260 91, Toronto, Ontario, Canada, pp. 2725-2728, May 1991.
3 0.0212 0.0585 0.0164 [113 M. Okada, “Video camera and video signal reproducing ap-
paratus with shake detection and correlation operation,” U.S.
4 I
0.0349 I
0.0827 I
0.0242
Patent, no. 5,502,484, Mar. 1996.
5 1 0.0374 I 0.1690 1 0.0242

S u n g - J e a K Oreceived the Ph.D. degree in
1988 and the M.S. degree in 1986, both in
V. CONCLUSION Electrical and Computer Engineering, from
State University of New York at Buffalo,
In this paper, we proposed a BPM-based DIS system and the B.S. degree in Electronic Engineer-
which performs motion estimation using 1-bit planes. i r g at Korea University in 1980. In 1992,
It was shown experimentally that the performance of he joined the Department of Electronic En-
gineering a t Korea University where he is
BPM is very close to that of the FS BMA in terms of currently an Associate Professor and Grad-
RMSE. Moceover, BPM can be realized using only sim- uate Chairman. From 1988 to 1992, he was an Assistant Pro-
fessor of Electrical and Computer Engineering at the University
ple Boolean functions, and thus is more suitable for VLSI
of Michigan-Dearborn. From 1981 to 1983, he was with Daewoo
implementations than existing algorithms. It was also Telecom Corporation where he was involved in research and de-
shown that the median-based motion correction scheme velopment on data communication systems. He received the Hae-
Dong best paper award from the Institute of Electronics Engineers
is robust to irregular conditions such as moving objects.
of Korea (1997), and the best paper award from the IEEE Asia
Simulation results show that the proposed DIS algorithm Pacific Conference on Circuits and Systems (1996). Dr. KO is cur-
is a computationally efficient alternative to existing DIS rently a Senior Member in the IEEE and a programs chair (1996
algorithms.
- Present) of the IEEE Seoul Section of Korea Council. His cur-
rent research interests are in the areas of digital signal and image
processing, and multimedia communications.
REFERENCES
M. Oshima, T. Hayashi, S. Fujioka and T. Inaji, “VHS cam-
S u n g - H e e Lee received the B.S. degree
corder with electronic image stabilizer,” IEEE Trans. on Con-
in information engineering and M.S. degree
aumer Electronics, vol. 35, no. 4, pp. 749-758, Nov. 1989.
in computer science form Korea University
K. Sato, S . Ishizuka, A. Nikami, and M. Sato, “Control tech- in 1993 and 1995, respectively. He is now
niques for optical image stabilizaingsystem,” IEEE Trans. on a Ph.D. candidate in electronic engineer-
Consumer Electronics, vol. 39, no. 3, pp. 461-466, Aug. 1993. ing with the Department of Electronic En-
K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Ki- gineering a t Korea University. In 1995, he
tamura, “Automatic image stabilizing system by full-digital joined the Research Institute for Informa-
signal processing,” IEEE Trans. on Consumer Electronics, tion and Communication Technology, where
vol. 36, no. 3, pp. 510-519, Aug. 1990. he is currently a research engineer. His research interests are in
3.-K. Paik, Y.-C. Park, and De-W. Kim, “An adaptivemotion the areas of wavelets and nonlinear signal and image processing.
decision system for digital image stabilizer based on edge pat-
tern matching,” IEEE Trans. on Consumer Eleclronics, vol.
38, no. 3, pp. 607-615, Aug. 1992. K y u n g - H o o n Lee received the B.S. degree
T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T. and M.S. degree in electronic engineering
Imaide, “Electronic image stabilizer for video camera use,” form Korea University in 1992 and 1994,
IEEE Trans. o n Consumer Electronics, vol. 36, no. 3, pp. respectively. He is now a Ph.D. candidate
520-525, Aug. 1990. in electronic engineering with the Depart-
H. G. Musmann, P. Pirsch, and H. J. Gralleer, “Advances in ment of Electronic Engineering at Korea
picture coding,” Proc. IEEE, vol. 73, no. 4,pp. 523-548, Apr. University. In 1993, he joined the Research
1985. Institute for Information and Communica-
B. Liu and A. Zaccarin, “New fast algorithms for the estima- tion Technology, where he is currently a re-
tion of block motion vectors,” IEEE Trans. on Circuits and searcher. His research interests are in the areas of nonlinear signal
Sydems for Video Technology, vol. 3, no. 2 , pp. 148-157, Apr. and image processing including morphological and order statistics
1993. filtering.
3. Lu and M. L. Liou, “A simple and efficient search algo-
rithm for block-matching motion estimation,” IEEE Trans.