2012 Fifth International Symposium on Computational Intelligence and Design
Parallel Algorithm for Moving Foreground Detection in Dynamic Background
YI YANG, WENJIE CHEN
School of Automation, Beijing Institute of Technology, Beijing, 100081
Beijing Key Laboratory of Automatic Control System (Beijing Institute of Technology)
Beijing, China
e-mail: yyhengwuyuan73@yahoo.com.cn, chen.wenjie@163.com
Abstract—Foreground detection in dynamic background has
become a hot topic in video surveillance in recent years. In this
paper we propose a new foreground detection approach based
on GPU in dynamic background. With the proposed method,
SIFT features are first extracted from two adjacent frames in
video sequences, which can be utilized to compute the
parameters of affine transform model and to solve global
motion compensation. Then improving background
subtraction approach with dynamic background updating
module is adopted to detect foreground objects. GPU method is
used to improve application performance. Combined with
CUDA, three mainly algorithm modules, which are so called
Global Motion Compensation Module, Updating Background
Module and Foreground Detection Module, are improved. In
this paper, GPU and CPU are used as a combined computing
unit, which makes good use of strong parallel computing
ability. The effectiveness of the method has been proved.
Finally, the contrasting experiments on processing time show
that the proposed algorithm based on GPU is better in speed.
Keywords-dynamic background; affine transform;
foreground detection; GPU parallel computing
I. INTRODUCTION
At present, there are two dominant methods for moving
foreground detection, optical flow method and motion
compensation method. Unfortunately, with high
computational complexity and poor anti noise capability,
optical flow method can only be used with special hardware
device to fit the need of real time. Among all of the motion
compensation based methods, global motion compensation
method is further helpful. The main ideal of this method is to
estimate the motion parameter of a camera through image
matching. But complex computation and excessive
processing bring serious problems to this approach[1,2].
With the rapid development of parallel computation,
GPU (Graphic Processing Unit) becomes more and more
attractive all over the world due to its powerful parallel
computing ability. CUDA (Compute Unified Device
Architecture) is a GPU general computing product produced
by NVIDIA Company in 2007. It develops a new structure to
eliminate a great deal of constraints caused by GPU
computing model.
Considering the problems above, we propose a parallel
algorithm for moving foreground detection in dynamic
background. It consists of four modules: image matching
between frames, global motion compensation, background
updating and moving foreground detection. We combine
last three algorithm modules with CUDA to improve the
978-0-7695-4811-1/12 $26.00 © 2012 IEEE 444
443
442
DOI 10.1109/ISCID.2012.270
detection performance. The strong parallel computing
ability is fully used. Meanwhile, the computation of CPU
and GPU are divided wisely.
We adopt function-structured and modularized design
concept to develop our moving foreground detection based
on CPU and GPU in dynamic background. The complete
system is designed as below.
CPU GPU CPU CPU
Video
input Background Image Global moving
Updating matching Motion foreground
between compensation detection
frames
Figure 1. The flow diagram of detection system in this paper
II. ALGORITHM FOR MOVING FOREGROUND DETECTION
IN DYNAMIC BACKGROUND
The algorithm in this paper consists of four parts as
Firgure.1. The complete algorithm is shown in Firgure.2.
A. Image Matching between Frames
SIFT ˄Scale Invariant Feature Transform˅algorithm is
applied to finish image feature extraction. As a feature point
based on matching algorithm, SIFT can process the matching
problem between two images with translation, rotation and
affine transformation. Strong matching ability and good
robustness make the SIFT algorithm widely used in image
match area. Principal Steps of SIFT algorithm are extreme
points detection ˈ confirming key-points in scale space,
direction parameters specified for each key-point and key
descriptor generation. Euclidean distance is utilized to
calculate the distance between two SIFT feature vectors. In
order to obtain the accurate parameters of affine
transformation model, RANSAC method is used to eliminate
the abnormal feature points[3,4].
B. Global Motion Compensation
Global Motion Compensation is an essential part of
moving target detection in dynamic background. In this
paper, we adopt affine model with six parameters to describe
the image global motion. This model is widely used, Because
of its invariance under motion and variation, such as
translation, scaling and 2D-rotation[5]. 
 ­0, f t ( x, y ) − F ( Bt ( x, y )) < Th
M ( x, y ) = ® ˄3˅
last frame Current frame ¯1, f t ( x, y ) − F ( Bt ( x, y )) ≥ Th
SIFT extraction Where M ( x, y ) represents the result in which 0 means
and matching this pixel should belong to background area, while 1 means
foreground area. Th is the threshold.
f t ( x, y ) is current
Global motion Background Frame differencing
compensation image
input frame, and
Bt ( x, y) is the corresponding background
Background image.
updating Foreground object
module
III. ALGORITHM OPTIMIZATION BASED ON GPU

Show image
Figure 2. The complete algorithm in this paper
The affine model with six parameters is shown as below.
ª f xt º ª a1 f xt −1 + a2 f yt −1 + a3 º
« t» = « t −1 t −1 »
¬« f y ¼» ¬«a4 f x + a5 f y + a6 ¼»
˄1˅
f xt
where represents x coordinate of a pixel at some
f yt
point in the image, represents
(a1 , a2 , a4 , a5 ) represents the parameters
coordinate,
( a3 , a 6 )
described image rotation and scaling, and
represents the parameters described translation.
C. Background Updating Model and Foreground
Detection
In order to adapt dynamic background, we propose a new
background updating model which is different from
traditional background subtraction. Background updating
model is a crucial component of our improved background
subtraction.
In this paper, the new background is made up of two
parts. One is the background area of current frame, and the
other is the foreground of current frame after affine
transformation which is the estimation of foreground motion.
The updating model is shown as below.
Bt +1 ( x, y ) = bk t ( x, y ) + F ( frt ( x, y ))
Bt+1 ( x, y ) represents the new background image
Where
bkt ( x, y ) represents the background area
after updating,
frt ( x, y ) represents the foreground
of current frame,
ground area of current frame, and F (•) represents affine
transformation.
After obtaining new background image which can be
used in our improved background subtraction, we can detect
the foreground of current input frame. The method we
adopted is shown as below.
Based on the above analysis, there are a large amount of
cumulative and matrix calculations in the global motion
compensation model, the background update model, and
foreground detection model. The following part will
transplant these three modules to the CUDA platform for
optimization according to this feature.
A. The division of tasks
CUDA is a hardware and software system which regards
GPU as a data parallel computing device. The CUDA
programming model considers the CPU as the host (Host),
and the GPU as a co-processor or device (Device). In this
y
model, the CPU and GPU work together to take their
responsibilities The CPU takes charge of the highly logical
transaction processing and serial computing, while the GPU
is responsible for the execution of highly threaded parallel
processing tasks. Each of them has independent memory
address space: The memory of the host side and device-side
memory. Memory operation of this algorithm is basically the
same as general C program, and memory operations need to
call the memory management functions of CUDA API.
These management operations include opening up, releasing
and initializing memory space, as well as transmutes data
between the host and the device. For better utilization of the
GPU's parallel processing capabilities, the task is divided as
shown in Figure 3.
B. arithmetic transplant
The kernel (kernel function) running on GPU is
organized in the form of thread grid (Grid). Each thread grid
˄2˅
is composed of a number of thread blocks (Block), and each
thread block consists of many threads (thread), which is the
smallest execution unit of the kernel function. The threads in
the same block can be quickly synchronized and they share a
limited size of shared memory. The thread number of each
block is limited (GPU dependable). Blocks which execute
the same procedure can be composed of grid [6-8].
Affine transformation in the global motion compensation
module, frame difference in the prospective detection, and
key calculation steps of background update shown as (1-3)
are typical image manipulation and typical matrix operations
as well. They occupy a lot of processing time. The larger the
image, the longer the processing time, and the more
difficulties to obtain assurance for real-time video
processing. Regarding of the three large amount of matrix

443
444
445
computation of the above three models, this paper come up Figure 5. This total contains 640 * 480 threads, each thread
with using GPU to achieve parallel computing acceleration. only needs to calculate a corresponding element. Elements to
The serial part of the programming will not be repeated here. be processed by the thread can be easily collected by unique
The parallel algorithm procedure used in the three models' identifier of each thread.
computation is shown in Figure 4. GPU kernel
pretreatment
HOST
DEVICE
Allocate Calculate thread
Create frames, Graphic memory
initialization and index
data transmission
Affine Transmit data Affine Frame difference or
transformation; from main transformation updating
memory to background
feature extraction and global motion
graphic memory
and matching compensation
between frames Calculate Summation
new or
Foreground coordinate subtraction
detection:
Frame difference
Transmit new
Updating coordinate from
background device to host
model

Show images Release graphic

memory
Figure 3. The division of tasks between GPU and CPU
Pretreatment contains a series of preparatory work after Figure 4. The flow diagram of parallel algorithm
starting CUDA, including defining of the number of thread Affine transformation part, according to the six-
blocks in the grid, defining the number of threads in each parameter affine model definition (1), in the serial code,
thread block and allocating memory size for data . Next, every time the program executes an affine transformation , it
memory allocates in the host side and memory allocates in need four multiplication and addition operations. For a
the device side. After the data is received, it is passed to the frame, it will be 4 * 640 * 480 multiplication and 4 * 640 *
CUDA program. Using the CUDA API, data in memory is 480 addition operations. Therefore, using of the GPU parallel
read into memory, and then memory data passes to the kernel mode, defining the same above-mentioned two-dimensional
function. The kernel function executes appropriate grid (10,120,1) and two-dimensional threads (64,4), multiple
processing on the GPU and copies the results from the threads work simultaneously to reduce computing time and
memory back to the main memory. The CPU is responsible to succeed accelerating.
for displaying the result image. The work of the CPU serial 10*64=640
code includes data preparation and device initialization
before the kernel start, including the creation and Grid Block˄119ˈ1˅
initialization of video image for each frame, image feature 120 Block Block Block thread thread thread
matching, some serial computation, de-noising operation and * (0,0) (0,1) (0,9) (0,0) (0,1) (0,63)
the final displayed image. 4= Block thread
Operation of the frame difference method to detect 480 (1,0) (1,0)
motion foreground (2) and the background model of the
background model update (3) are image subtraction, so they Block Block thread thread thread
(119,0) (119,9) (3,0) (3,1) (3,63)
can be regarded as typical matrix operations. The frame size
of experiment is 640 * 480, so that under the CPU mode,
every image processing operation or subtraction should be
640 * 480 times. Taking into account the experiment using
Figure 5. The define the layout of grid and threads on GPU
NVIDIA GeForce 9800 GT GPU, the computing power is
1.1. Therefore, when using GPU to achieve optimizations
and improvement, two-dimensional grid is defined to be IV. MAIN EXPERIMENTS AND RESULTS
(10,120,1) and two-dimensional thread is defined to be
A. effectiveness verification of Detection method in
(64,4). In other words, two-dimensional grid has10 blocks
each row, and 120 blocks each column, and the thread has 64 dynamic background
thread threads in each row and four threads each column. First, we verify the effectiveness of our algorithm on
The product of the three dimensions of the threads is 64 * 4 CPU by indoor experiment of pedestrian detection. The
= 256 which is less than upper limit 768 of the computing resolution ratio of moving camera in this experiment is 640
power for the 1.1 hardware. The grid definition is shown in h 0.Our method is performed on PC with Intel(R)

444
445
446
Core(TM)2 Duo CPU 2.4GHz, VC2010, Intel OpenCV, and
CUDA toolkit 4.0. The graphics card is NVIDIA GeForce
9800 GT and its computational capabilities is 1.1. The
threshold of foreground is 15.
The result of indoor experiment of pedestrian detection is
shown in Figure 6. Figure 6(a) shows four original images
extracted from original video sequence, and Figure 6˄b˅
are results of foreground detection. The results show that our
method is effective on moving foreground detection under
dynamic background. SIFT features matching and affine
transformation based global motion compensation serve the
detection in dynamic background pretty well. The detection
system can perform well in dynamic background detection,
which consists of four parts: image matching between
frames, global motion compensation, background updating
and moving foreground detection.
 (a) (b)
Figure 6. (a) Original video sequences; (b) results of detection
B. Performance testing of GPU optimizing
Table.I. THE RESULT OBTAINED BY CONTRASTIVE EXPERIMENTS
The  Before After optimizing
average optimizing˄s˅ ˄s˅
processi
ĉ  
ng time
of three  
parts ċ  
Č  
č
 
The experiment environment is the same as the above.
We make a comparison of processing time before and after
GPU optimization of three parts which are global motion
compensation, background updating and moving foreground
detection. The result is shown as table 1.The average
processing time is calculated according to the following
steps.1.Picking 10 frames randomly from the video sequence.
2. Counting the processing time of those three parts for each
frame. 3. Calculating the average. After that, the speed-up
ratio is calculated. This is one group of experiment. We have
conduct five groups of such experiments all together.
The result obtained by contrastive analysis shows that the
detection method proposed in this paper makes full use of
GPU powerful parallel computing ability. The moving
foreground detection in dynamic background is successfully
accomplished.
V. CONCLUSION
In this paper, we propose a new foreground detection
approach based on GPU in dynamic background. The
proposed method first extracts SIFT features of two adjacent
frames in video sequences. Then we compute the parameters
of affine transform model which are utilized in global motion
compensation. At last we adopt improving background
subtraction approach to detect foreground objects. The
updating background model is used which can leads a more
accurate detection performance. Combined with GPU, three
mainly algorithm modules, which are so called Global
Motion Compensation Module, Updating Background
Module and Foreground Detection Module have been
optimized. In this paper, GPU and CPU are used as a
combined computing unit, which makes a good use of
powerful parallel computing ability. In the end, the
effectiveness of the method and optimizing about speed up
by GPU has been proved. How to further improve the ability
in real-time processing is the new job pathway in the future.
ACKNOWLEDGEMENTS
We would like to thank Professor LiHua Dou for
discussions. This work is sponsored by Ordinary University
Key Laboratory of Beijing, SYS100070417.
REFERENCES
[1] Bo Gao,Guobin Bao. Based on the dynamic image processing
technology, the dynamic object detection and tracking[J].
Photoelectric technology, 2010,25(4),pp.73-76.
[2] Jun Lu㧘Fengling Li, Mai Jiang. Camera movement under dynamic
object detection and tracking[J]. Journal of Harbin engineering
Speed-up
university,2008,29(8),pp.831-835.
ratio
[3] Hua Nian.GPU general computing and SIFT the image matching

features based on parallel algorithms [D]. Xian:Xian master thesis,
 university of electronic science and technology,2010.
 [4] David G Lowe. Distinctive Image Features from Scale-Invariant
 Keypoints[J]. International Journal of Computer Vision, 2004, 60(3),
pp. 91-110.

[5] Xiaofeng Yang,Guilin Zhang.A target tracking algorithm based on an
affine transformation model[J]. The computer and digital project,
2005,33(12),pp.30-34.
[6] Feng Cheng,Dehua Li.The Adaboost algorithm based on CUDA and
parallel implementation[J]. Computer engineering and science, 2011,
33(2), pp.118-123.
[7] 3Hao Deng, Fei Yang, Xudong Pan, Anqing You. An implement with
CUDA of real-time target tracking based on upper machine[J].
Information and electronic engineering, 2010, 3(6), pp.368-371.
[8] Qing Zhong,Shungang Hua. GPU acceleration calculation of complex
grid model affine transformation[J]. Photoelectric technology
application,2011,26(1),pp.59-62.
445
446
447