125 views

Uploaded by Alessio

save

You are on page 1of 6

**DIGITAL IMAGE STABILIZING ALGORITHMS BASED ON BIT-PLANE MATCHING
**

Sung-Jea KO,Senior Member IEEE, Sung-Hee Lee and Kyung-Hoon Lee

Department of Electronics Engineering, Korea University

5-1 Anam-Dong, Sungbuk-Ku, Seoul 136-701, Korea

Email: sjko@dali.korea.ac.kr

Abntract-In this paper, we present a new digital image The full-search(FS) BMA under the mean absolute dif-

stabilization (DIS) scheme based on bit-plane matching ference (MAD) and mean square error (MSE) criteria

(BPM). The proposed DIS system performs motion es-

timation using 1-bit planes which are extracted from a

can be considered as an optimal solution for motion es-

video sequence. This motion estimation technique can be timation [6],[7]. However, the FS BMA requires large

realized using only Boolean functions which have signif- amount of computations which causes time delay, and

icantly reduced computational complexity, while the ac-

requires complex hardware architecture [7]-[9].In this

curacy of motion estimation is maintained. In the sec-

ond part of this paper, a median-based motion correction paper, we present a new motion estimation technique

scheme is proposed which is robust to various irregular based on the bit-plane matching (BPM) for the DIS sys-

conditions such as moving objects and intentional pan- tem. The proposed algorithm performs binary motion

ning. Simulation results show that the proposed DIS al-

gorithm exhibits better performance compared with ex-

estimation using 1-bit planes which are extracted from

isting other algorithms when applied to real video signal. a video sequence. This motion estimation technique can

be realized using only Boolean functions which have sig-

nificantly reduced computational complexity, while the

I. INTRODUCTION accuracy of motion estimation is maintained. Simulation

Image stabilization is the process of generating a com- results show that the performance of the BPM-based mo-

pensated video sequence where image motion by the tion estimation algorithm is comparable to that of the FS

camera’s undesirable shake or jiggle is removed [1]-[5]. BMA.

The recent DIS systems are realized using digital im- Various algorithms have been developed to estimate

age processing techniques instead of mechanical motion the global motion of a frame from local motion vectors

detection techniques using gyro sensors or fluid prism [3]-[7],[11]. Most of these algorithms are complicated,

PI-[51. and thus are not simple to implement. In this paper, we

The image stabilization task can be subdivided into propose a simple and robust decision algorithm for deter-

two basic systems, namely: i) the motion estimation sys- mining the global motion vector. In the proposed algo-

tem and ii) the motion correction system. In general, the rithm, the global motion vector of a frame is determined

motion estimation system generates several local motion based on the order statistics of current local motion vec-

vectors from subimages in the different position of the tors and past global motion vectors. It is shown that the

frame using a block matching algorithm (BMA). The DIS system using the proposed motion estimation and

motion correction system determines the global motion correction algorithms is less sensitive to irregular condi-

of a frame by appropriately processing these local mo- tions such as moving objects and intentional panning.

tion vectors, and decides whether the motion of a frame This paper is orgainzed as follows: The proposed mo-

is caused by undesirable fluctuation of the camera or in- tion estimation and correction algorithms are presented

tentional panning. The stabilized image is generated by in Sections I1 and 111, respectively. Simulation results

reading out the proper block of fluctuated image in the are given in Section 1V and concluding remarks are in

frame memory [3]-[5]. Section V.

This work was supported by SAMSUNG Electronics Co.

Manuscript received June 17, 1998 0098 3063/98 $10.00 1998 IEEE

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.

618 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

Fig. 1. Example of generating bit-planes from a gray-sclae image, (a) an original image, (b) eight bit-plane images.

**11. MOTIONESTIMATION USINGBIT-PLANE IMAGES the bit-plane as shown in Fig. 2. Each motion vector of a
**

Before describing the proposed method to estimate lo- subimage in the current bit-plane image is determined by

cal motion vectors, we introduce the bit-plane decompo- evaluating bit-plane matching over subimages in the pre-

sition of a gray-scale image. vious bit-plane and selecting the subimage which yields

the closest matching. This approach assumes that all

A . B i t - p l a n e Decomposition of a Grayscale I m a g e pixels within the subimage have uniform motion and the

Let the graylevel of the pixel at location (z,y) in the range of the motion vector is constrained by the search

t-th image frame with 2K graylevels be represented as window.

Let the size of each subimage be M x N and a search

+

window be ( M 2p) x ( N + 2q). For bit-plane matching,

we define the correlation measure given by

where a k , O 5 h 5 K - 1, is either 0 or 1. Let the k-

th order bit-plane image be denoted by b t k ( z , y ) . This ~ ( m n),= C

b k t ( z , y) CB bkt-'(z m, y n) (2) + +

plane contains all the k-th order (ak)bits. For the case (",Y)ES.

**of the 8-bit image, an image is composed of eight 1-
**

where bkt(a,y) and bkt-'(z, y), respectively, are the cur-

bit planes bk(z,y) N b$(z,y), ranging from plane 0 t o

rent and previous k-th order bit-planes, and @ is the

plane 7. Fig. 1 shows eight bit-planes decomposed from

exclusive-OR operation.

a grayscale image. bi(z,y) contains all the least signif-

At each ( m , n ) ,-p 5 m 5 p and -q 5 n 5 q, within

icant (lowest order) bits comprising pixels in the image

the search range, the proposed matching method calcu-

and b$(a, y) contains all the most significant (highest-

lates C i ( m , n )which is the number of unmatched bits

order) bits. Note that only the higher order bit-plane im-

between the reference subimage in the current bit-plane

ages contain visually significant data whereas the other

and the compared subimage in the previous bit-plane.

bit-planes contribute to more subtle details within the

The smallest Ci(m,n) yields the best matching for Si,

image.

and thus local motion vector V; from Si is obtained as

B. M o t i o n E s t i m a t i o n Based o n t h e Bit-plane M a t c h i n g

vit = arg min{Ci(m, n),- p 5 m 5 p , -q 5 n 5 q } . (3)

Local motion vectors are estimated from four subim-

ages (SI,S2, Ss, 5 4 ) placed in appropriate positions in

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.

KO,Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 619

**! preuioiis bit-plane image
**

I-_-._ i

_ _ _ _ - - _ _ I - _ _ _

Fig. 2. Estimation of local motion vectors from four subimages in a bit-plane.

This motion estimation technique can replace the and the motion compensation unit. The motion correc-

arithmetic calculations of BMA's based on conventional tion system determines the global motion of a frame by

MAD and MSE criteria with simple Boolean exclusive- appropriately processing local motion vectors, and de-

OR operations, and thus has significantly reduced com- cides whether the motion of a frame is caused by un-

putational complexity. desirable fluctuation of the camera or intenti6nal pan-

Since the proposed DIS system performs motion esti- ning. The stabilized image is generated by reading out

mation using a single bit-plane, it is important to select the proper block of fluctuated image in the frame mem-

an appropriate bit-plane for bit-plane matching. In this ory.

paper, the 4-th order bit-plane, b;(z,y), is utilized to In an image with motion, some subimages with mov-

estimate the local motion vector since it contains both ing objects can produce motion vectors which are signif-

the global information and details of the original image. icantly different from the other motion vectors. Fig. 5

To show that b:(z, y) is suitable for motion estimation shows an image which has moving objects in some subim-

based on BPM, an example for BPM using 1-D signals ages. Fig. 6(a) and (b) show the correlation measures

is presented in Fig. 3.. Fig. 3(a) shows an 1-D signal calculated using (2) from subimage SI which has no

which is one horizontal scan line of a real image with moving object and subimage S4 with moving objects,

256 graylevels, and Fig. 3(b) is the same scan line from

the previous frame. Using (2), we computed the correla-

tion measure between two binary signals from the scan

*

respectively. In Fig. 6, for display, the correlation mea-

sures are normalized using l - where C,,, is the

maximum Cj(m,n) within the search range. It is seen

lines at each bit-level. Fig. 3(c) illustrates the simulation that there does not exist a distinct maximal correlation

results. It is seen that the correlation measure from the value in S4.

4-th bit-level exhibits the steepest gradient around the In general, motion vectors from the subimages with

minimum point. moving objects are not reliable and should be excluded

from the global motion decision process. Moreover, since

111. MOTIONCORRECTION UNDER IRREGULAR

the hand movement is relatively slow than the frame rate

CONDITIONS of the video camera, two successive frames fluctuated by

In this section, we introduce the motion correction sys- camera's shake should have similar global motion.

tem to cope with irregular conditions such as moving Based on these properties of camera's motion, we pro-

objects and intentional panning that degrade the perfor- pose a simple and robust motion correction scheme where

mance of the DIS system. global motion decision is performed using current local

Fig. 4 shows the proposed motion correction system motion vectors (V:, Vi, Vi,V:) and the previous global

which consists of the decision unit, the integration unit,

Authorized licensed use limited to: BIBLIOTECA D'AREA SCIENTIFICO TECNOLOGICA ROMA 3. Downloaded on October 8, 2009 at 04:15 from IEEE Xplore. Restrictions apply.

620 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

-8 .o -4 -2 0 2 4 0 B

horizontal displacement

(c>

Fig. 3. An example of BPM using 1-D signals, (a) a 1-D signal which is one horizontal scan line of a real image, (b) the same scan line

from the previouse frame, ( c ) the correlation measure between two binary signals from the scan lines at each bit-level. (+: bit-level

6, 0: bit-level 5, *: bit-level 4, 0 : bit-level 3, X : bit-level 2).

Fig. 4. Basic structure of the motion correction system.

motion vector V,"-'. In the proposed algorithm, the It is known that the median filter is very effective in elim-

global motion vector is obtained by inating impulses. Therefore, the median-based method

in (4) can exclude such abrupt local motion vectors and

Vi = median{Vi, Vi, V i ,V i ,v,"-'} (4) produce a global motion vector similar to the previous

one.

Here the median of vectors is determined by seperately After determining the global motion vector, the mo-

selecting medians of each vector elements. tion correction system decides whether the motion of a

Local motion vectors affected by undesirable condi- frame is caused by camera's motion or intentional pan-

tions such as moving objects can be viewed as impulses. ning. For this decision, the global motion vector of a

KO, Lee and Lee: Digital Image Stabilizing Algorithms Based on Bit-Plane Matching 62 1

Fig. 5. An image with moving objects.

(4 (b)

Fig. 6. Correaltion measures, (a) from subimage S1 which has no motion, and (b) from subimage Sb which contains moving objects.

**frame is integrated with a damping coefficient, and the TABLE I
**

integrated motion vector designates the final motion vec- RMSE’S ASSOCIATED WITH LOCAL MOTION YECTORS.

**tor of a frame for motion correction. The integrated mo-
**

Sequence number RMSE

tion vector V, for estimating intentional panning is given

I

RPM I

I

EPM I BPM

I

by 1 I 0.0334 I 0.3285 1 0.0205

vat = DIVat-l + V,t (5) 2 0.0373 0.4886 0.0333

3 0.0296 0.3000 0.0219

where Q t is a global motion vector and D1(0 < D1 < 1) 0.0518 0.3409 0.0362

4

is a damping coefficient for smooth panning. I

**1 0.0373 1 0.5002 1 0.0229
**

5

IV. SIMULATION

RESULTS

To evaluate the motion estimation performance of the

proposed BPM algorithm, we compare it with two exist-

ing DIS algorithms, namly, representative point match-

ing (RPM) [3] and edge pattern matching (EPM) [4]. 150 frames are utilized for simulation.

The performance is evaluated using the root mean square The RMSE’s associated with the local motion vector

error (RMSE) based on the FS BMA under the MAD are summarized in Table I. It is seen that BPM exhibits

criterion. The RMSE is given by better performance than RPM and EPM.

Table I1 summarizes the RMSE’s associated with

global motion vectors. It is interesting to observe that

the RMSE’s of BPM are always smaller than those of

RPM and EPM.

where (zn,yn) is the motion vector from the FS BMA, These experimental results indicate that the proposed

and (&, ijn) is that from aformentioned algorithms. Five BPM-based DIS system exhibits good performance com-

real image sequences with a resolution of 640 x 240 and parable to the FS BMA.

622 IEEE Transactions on Consumer Electronics, Vol. 44, No. 3, AUGUST 1998

**TABLE I1 o n Circuits and Systems for Video Technology, vol. 7, no. 2,
**

RMSE’S ASSOCIATED WITH GLOBAL MOTION VECTORS. pp. 429-433, Apr. 1992.

[9] H. Gharavi and M. Mills, “Blodunatchingmotion estimation

Sequence number algorithms-new results,” IEEE B a n d . o n Circuits and Sys-

RMSE

tems, vol. 37, no. 5, pp. 649-651, May 1990.

RPM EPM BPM

[lo] Y. T.Tse and R. L. Baker, “Global zoom/pan estimation and

1 0.0285 0.1223 0.0178 compensation for video compression,” Proceedings of ICASSP

2 0.0308 0.2082 0.0260 91, Toronto, Ontario, Canada, pp. 2725-2728, May 1991.

3 0.0212 0.0585 0.0164 [113 M. Okada, “Video camera and video signal reproducing ap-

paratus with shake detection and correlation operation,” U.S.

4 I

0.0349 I

0.0827 I

0.0242

Patent, no. 5,502,484, Mar. 1996.

5 1 0.0374 I 0.1690 1 0.0242

**S u n g - J e a K Oreceived the Ph.D. degree in
**

1988 and the M.S. degree in 1986, both in

V. CONCLUSION Electrical and Computer Engineering, from

State University of New York at Buffalo,

In this paper, we proposed a BPM-based DIS system and the B.S. degree in Electronic Engineer-

which performs motion estimation using 1-bit planes. i r g at Korea University in 1980. In 1992,

It was shown experimentally that the performance of he joined the Department of Electronic En-

gineering a t Korea University where he is

BPM is very close to that of the FS BMA in terms of currently an Associate Professor and Grad-

RMSE. Moceover, BPM can be realized using only sim- uate Chairman. From 1988 to 1992, he was an Assistant Pro-

fessor of Electrical and Computer Engineering at the University

ple Boolean functions, and thus is more suitable for VLSI

of Michigan-Dearborn. From 1981 to 1983, he was with Daewoo

implementations than existing algorithms. It was also Telecom Corporation where he was involved in research and de-

shown that the median-based motion correction scheme velopment on data communication systems. He received the Hae-

Dong best paper award from the Institute of Electronics Engineers

is robust to irregular conditions such as moving objects.

of Korea (1997), and the best paper award from the IEEE Asia

Simulation results show that the proposed DIS algorithm Pacific Conference on Circuits and Systems (1996). Dr. KO is cur-

is a computationally efficient alternative to existing DIS rently a Senior Member in the IEEE and a programs chair (1996

algorithms.

- Present) of the IEEE Seoul Section of Korea Council. His cur-

rent research interests are in the areas of digital signal and image

processing, and multimedia communications.

REFERENCES

M. Oshima, T. Hayashi, S. Fujioka and T. Inaji, “VHS cam-

S u n g - H e e Lee received the B.S. degree

corder with electronic image stabilizer,” IEEE Trans. on Con-

in information engineering and M.S. degree

aumer Electronics, vol. 35, no. 4, pp. 749-758, Nov. 1989.

in computer science form Korea University

K. Sato, S . Ishizuka, A. Nikami, and M. Sato, “Control tech- in 1993 and 1995, respectively. He is now

niques for optical image stabilizaingsystem,” IEEE Trans. on a Ph.D. candidate in electronic engineer-

Consumer Electronics, vol. 39, no. 3, pp. 461-466, Aug. 1993. ing with the Department of Electronic En-

K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Ki- gineering a t Korea University. In 1995, he

tamura, “Automatic image stabilizing system by full-digital joined the Research Institute for Informa-

signal processing,” IEEE Trans. on Consumer Electronics, tion and Communication Technology, where

vol. 36, no. 3, pp. 510-519, Aug. 1990. he is currently a research engineer. His research interests are in

3.-K. Paik, Y.-C. Park, and De-W. Kim, “An adaptivemotion the areas of wavelets and nonlinear signal and image processing.

decision system for digital image stabilizer based on edge pat-

tern matching,” IEEE Trans. on Consumer Eleclronics, vol.

38, no. 3, pp. 607-615, Aug. 1992. K y u n g - H o o n Lee received the B.S. degree

T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T. and M.S. degree in electronic engineering

Imaide, “Electronic image stabilizer for video camera use,” form Korea University in 1992 and 1994,

IEEE Trans. o n Consumer Electronics, vol. 36, no. 3, pp. respectively. He is now a Ph.D. candidate

520-525, Aug. 1990. in electronic engineering with the Depart-

H. G. Musmann, P. Pirsch, and H. J. Gralleer, “Advances in ment of Electronic Engineering at Korea

picture coding,” Proc. IEEE, vol. 73, no. 4,pp. 523-548, Apr. University. In 1993, he joined the Research

1985. Institute for Information and Communica-

B. Liu and A. Zaccarin, “New fast algorithms for the estima- tion Technology, where he is currently a re-

tion of block motion vectors,” IEEE Trans. on Circuits and searcher. His research interests are in the areas of nonlinear signal

Sydems for Video Technology, vol. 3, no. 2 , pp. 148-157, Apr. and image processing including morphological and order statistics

1993. filtering.

3. Lu and M. L. Liou, “A simple and efficient search algo-

rithm for block-matching motion estimation,” IEEE Trans.

- ISC CS Practical Specimen PaperUploaded bySayak Kolay
- instructionsUploaded byapi-337885130
- Sinfra 2009 Shahid Chaumont PuechUploaded byAmmara Majeed
- po2 ec2ttUploaded bypriyasrinivasan24
- COMP 122 Lab 2 Lab Report and Source CodeUploaded bystephanievillalvazoab
- Scimakelatex.27142.NoneUploaded byAnonymous wAuWRLxf
- Dirac Software Alg0.2Uploaded byAshish Kumar
- bitszg553mid2Uploaded byShivam Shukla
- MultimediaUploaded bySenthil R
- Advances in Multi-Sensor Data Fusion- Algorithms and ApplicationsUploaded byPankaj Pratap Singh
- Robust Video Watermarking using Multi-Band Wavelet TransformUploaded byIJCSI Editor
- tmpC6BF.tmpUploaded byFrontiers
- Program 2Uploaded byChad Mcdonald
- Ovation Functions GuideUploaded bymatt hopkins
- Best Settings for CompressionUploaded bynoescalante
- INTEGRATIVE CELL FORMATION IN CELLULAR MANUFACTURING SYSTEMSUploaded byIJIERT-International Journal of Innovations in Engineering Research and Technology
- End Sem Schedule M13Uploaded byatharvawazurkar
- techiiifinallessonplanUploaded byapi-302393999
- Quiz2.sp12Uploaded byJoe Park
- ProgrammingUploaded bySumit Saurabh
- 3Uploaded byParul Kaushik
- 04350758_hit_and_missUploaded bycris9
- Algorithm = Logic + ControlUploaded byirmuhidin
- COMPUTATIONAL TOOLS FOR IMPROVING ROUTE PLANNING IN AGRICULTURAL.pdfUploaded bysunilbhol
- SPA Syllabus Semester 1 - Mumbai UniversityUploaded bykhadye_ashwini
- !VFFTUploaded byІван Дідюк
- Circulating LoadUploaded bycuberbill1980
- cpphtp4_PPT_02.pptUploaded byClebermar Lourenço de Lanes
- IJETTCS-2014-02-25-123Uploaded byAnonymous vQrJlEN
- GT-M8910_UM_EU_Eng_Rev.1.2_090820_cmsUploaded byMonde Petrovski

- 09.21.The mathematics of moving contact lines in thin liquid ﬁlmsUploaded byAlessio
- 09.18.Nonlinear Total Variation Based Noise Removal AlgorithmsUploaded byAlessio
- 09.14.Scale-Space and Edge Detection Using Ani So Tropic DiffusionUploaded byAlessio
- 08.21.What Went WhereUploaded byAlessio
- 09.22.LCIS_A Boundary Hierarchy for Detail-preserving Contrast ReductionUploaded byAlessio
- 09.17.Explicit Algorithms for a New Time Dependent Model Based on Level Set Motion for Nonlinear Debluring and Noise RemovalUploaded byAlessio
- 08.20.Layered Representation for Motion AnalysisUploaded byAlessio
- 09.23.Mathematical Models for Local Deter Minis Tic In PaintingsUploaded byAlessio
- 09.15.Image Selective Smoothing and Edge Detection by Nonlinear DiffusionUploaded byAlessio
- 08.17.Layered Depth ImagesUploaded byAlessio
- 09.16.Fronts Propagating With Curvature Dependent Speed_algorithms Based on Hamilton-Jacobi FormulationsUploaded byAlessio
- 09.13.A new image processing primitive_reconstructing images from modiﬁed ﬂow ﬁeldsUploaded byAlessio
- 09.04.Detection of Missing Data in Image SequencesUploaded byAlessio
- 09.20.a Variational Level-set Approach to Multi Phase MotionUploaded byAlessio
- 09.05.Interpolation of Missing Data in Image SequencesUploaded byAlessio
- 09.07.Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise RemovalUploaded byAlessio
- 06.09.Camera Stabilization Based on 2.5D Motion Model Estimation and Inertial Motion FilteringUploaded byAlessio
- 01.20.Texture Synthesis by Non-parametric SamplingUploaded byAlessio
- 09.10.Texture characterization via joint statistics of wavelet coefﬁcient magnitudesUploaded byAlessio
- 06.11.Fast Software Image Stabilization With Color RegistrationUploaded byAlessio
- 09.24.Filling-in by joint interpolation of vector ﬁelds and grey levelsUploaded byAlessio
- 08.14.Jetstream Probabilistic Contour Extraction With ParticlesUploaded byAlessio
- 09.08.Texture Synthesis by Non-parametric SamplingUploaded byAlessio
- 06.02.Real-Time Scene Stabilization and Mosaic ConstructionUploaded byAlessio
- 09.09.Pyramid Based Texture Analysis_synthesisUploaded byAlessio
- Image Motion DeblurringUploaded byAlessio
- 06.12.Image Stabilization by Features TrackingUploaded byAlessio
- 06.07.Fast 3D Stabilization and Mosaic ConstructionUploaded byAlessio
- 09.12.Level-Lines Based Dis OcclusionUploaded byAlessio

- Gandhi VirajUploaded bySUPERGEEK
- A More Secure Steganography Method in Spatial DomainUploaded byAnusha Sree
- Seminar StatisticsUploaded byCristina Tomiță
- Structured LightUploaded byKopkoH
- 1Uploaded bynayanasnair_20
- Vernam Conjugated Manipulation of Bit-Plane Complexity SegmentationUploaded byAndysah Putra Utama Siahaan
- nkp gray code for image sms.pdfUploaded byBinoj V Janardhanan
- Dip Image EnhanceUploaded byapi-3823773
- AMCS-053Uploaded byKeerthana Sudarshan
- ssg_m2l5Uploaded byDickson Xavier
- Ip Lab ManualUploaded bynandkishor joshi
- A More Secure SteganographyUploaded byMichael Benilan
- DIPQNAUNITVIIIUploaded byJalagam Sailaja
- DIP Image Enhancement 1Uploaded byapi-3711506
- EBCOT ExampleUploaded bygrailu