2015 11th International Conference on Signal-Image Technology & Internet-Based Systems

An Adaptive Codebook Model for Change

Detection with Dynamic Background
Tapas Badal , Neeta Nain , Mushtaq Ahmed and Vishakha Sharma
Department of Computer Science and Engineering
MNIT, Jaipur, India 302017
Email: tapasbadal@gmail.com, nnain.cse@mnit.ac.in, mahmed.cse@mnit.ac.in, 2014pcp5168@mnit.ac.in
AbstractCodebook model is a widely used method for
segmenting foreground pixels. However, it is often generate
erroneous positive result in case of dynamic background. This
paper proposed an adaptive codebook model for change detection
to disparate static background from dynamic background. To
eliminate shadow/illumination effects cone-shaped color distance
map is utilized in lieu of cylindrical. Moreover multi-layer
codebook model is proposed containing codebooks for each
pixel that is used to separate dynamic background from static
background region. Proposed method reduces erroneous positive
foreground pixels detected conventionally when pixel belongs to
background shows dynamic behaviour. During experimentation
proposed method is tested over numerous videos with complex
illumination and background situations. The experimental result
shows improvement over basic codebook model and other stateof-art background subtraction model.

I. I NTRODUCTION
Change detection is primary and important task in many
computer vision understanding, video analysis and smart
surveillance systems. Many methods have been proposed for
change detection and they are able to segment changing
region successfully with static background in real time. The
task of moving object detection and segmentation remains
challenging when there are moving/changing background,
variation in illumination, shadow effects and multiple moving
objects(crowded sequence). The detail description of challenges in detecting changes between frame sequence can be
found in [1].
Many methods for change detection have been proposed in
literature to deals with situation specic issues. Background
subtraction is widely used approach for detection of changes
between frames by subtracting target frame from background
model of a scene. While existing methods are able to detect
changes with static background and they can resolve problem
specic issues, they are not effective for many practical situations where background is not static. It generates false positive
set of pixels when background show dynamic characteristics
that decline the result of further processing steps like tracking
and activity recognition.
This paper proposed a adaptive codebook model by estimating the variation in decision threshold used for segmenting
dynamic pixels and spatial context for pixels is utilized to
disparate uncovered static background from dynamic background. Proposed method can reduce erroneous positive pixels
detected conventionally as ghost region while pixel belongs
978-1-4673-9721-6/15 $31.00 2015 IEEE
DOI 10.1109/SITIS.2015.89

110110

to background that suddenly start moving or shows random

movement in background.
In section II it discusses widely followed methods for background subtraction available in literature. Section III gives
description about fundamental codebook model. Section IV
explains proposed technique and improvement over basic
codebook model. Section V discuss experimental results and
outcomes compared with other state-of-art algorithms followed
by Section VI that conclude this paper.
II. R ELATED W ORK
Background subtraction is a extensively followed technique
for moving object detection where changes present in a scene
is consider as an area of interest. It detects moving pixels
by subtracting current frame from reference frame. Numerous
methods are proposed in literature for modeling the background during training, subtracting reference background from
current image and updation of reference frame.
It was rst introduce as unimodal approach where background
is represented as a single gaussian model by Wren et.al [2].
Further improvement using multiple weighted Gaussians with
adaptive learning is present by grimson et.al [3] and it was
widely followed by many researchers [4], [5] etc. Although
adaptive gaussian mixture modal can detect moving object in
real time but it can not deal with shadow. The parameters
estimation is another important issue to deal with in gaussian
when operate on varying illumination environment.
Non-parametric approach is presented by researchers to model
background using kernel density estimation [6], [7]. These
method estimate background probability density function using histogram of previous frames. Although it avoid estimating
parameters to generate appropriate shape of density function
it is questionable that what learning rate is appropriate in
dynamic environment. As it update background model in order
of arrival of pixels.
Kim et al. [9] proposed compressed representation of background using codebook. Codewords comprises information
about color and intensity of background pixel in limited
amount of memory. Improvements in codebook model as
multi-layer and integrating it with spatial and temporal information of pixel has been presented in [10].
Random updation policy in color space is introduced by
Barnich [8], [12] that provide satisfying performance with
speed and detection. However, static background in training

codeword ck , k = {1, 2, ....m} made upof a color vector Vk =


{rk , gk , bk } and a 6 tuple auxk = Ik , Ik , fk , k , pk , qk
where Ik and Ik denotes minimum and maximum brightness
respectively, fk is used to represent the frequency with which
codeword k has occurred k is maximum negative run length
and pk and qk are rst and last access time of codeword k.
Fig. 1graphically represents tuple structure of codebook of a
pixels and Fig. 2 shows color vector using three color channels
R, G and B.
Initially, rst 80 frames are used for training where
each pixel value xt is compared to nd match with
codeword present in current codebook using color
distance measure and logical brightness function. Each
pixel x = (R, G, B) is compared against codeword
in M using color distance measure express as:

period is essential requirement in this method. Also change

in static part of background result as false detection which is
concerning factor while tracking objects.
Method proposed in this paper deals primarily with pixels
detected falsely(ghost region) with dynamic background. Here
varying decision threshold is generated to deal with background with multiple moving objects and varying illumination
effects. Multi-layer approach is followed by introducing layers
for foreground pixels and static permanent background.
III. C ODEBOOK MODEL
The fundamental codebook model detects moving object
using innovative color distance measure. The codebook model
represents background model as codebook consisting one or
more codewords as M = {c1 , c2 , ...., cm } for each pixel. Each



2


rk R + g k G + bk B
colordist(xt , vi ) = (R2 + G2 + B 2 )
rk2 + gk2 + b2k

and

logical

brightness

function

is

dened



I
=
brightness I, I,

as

true

ifIk  xt   min{ IK , Ik /},

f alse

otherwise

where and are xed parameters taken as 0.5 and 1.3

respectively.
The pixel is classied as background if it matches with
codeword present in background model of a pixel. The
matching operation include subtraction operation BGSx for
incoming pixel based on two conditions:

(1)

colordist(x,cm )  
I
= true
brightness I, I,

Where is detection threshold taken as 25. After training only

codewords having frequency greater than threshold T ha dd
shifted to M. Further improvement in codebook model suggested as an addition of another codebook H, which is used
in layered codebook model to update non-background pixels.
Due to the compressed representation of background model
codebook has attracted the attention of many researchers [10]
for improvement. The performance of codebook deteriorates
when background contains objects that show dynamic characteristics.
IV. P ROPOSED A PPROACH
In fundamental codebook model constant value is used as
the color distance measure for segmenting foreground pixels.
By analysing various videos, it has been found that constant
value as distance measure leads to false detection in many
situation like a dynamic background and different lighting

111111

(2)

conditions. In the following subsection we introduce adaptive

decision threshold that is use for foreground segmentation.
A. Adaptive Decision Threshold
In basic codebook model detection threshold is kept
constant for all video sequences. As different video sequence
may have different lightning conditions depend upon different
locations and time that sequence is captured. In between
indoor and outdoor video there are signicant difference
in color distance over a pixel in successive frames in the
same way video obtained during day or night. Ideally the
detection threshold should be increased for the highly dynamic
background like water surface, waving tree leaves, lightning
change as to not include these objects in the foreground.
For static background, low value should be assigned to
to allow detection of even small changes in a scene. To
allow decision threshold needs to be adaptive according to
the background situations adaptive color model is followed
to habituate variation in the background of a scene. Given a
codeword c1 belongs to pixel x at frame sequence t the mean
value of color distance measure can be represents as follows:
2

2
m,t = m,t1 + (1 )m,t

(3)

Where m,t denotes mean vector of color distance

over pixel x at time t and m,t represent current value of

Fig. 1: Codebook of a pixel consisting multiple codewords.

color distance calculated in Eq. 1. The denotes learning rate.

Different lighting condition and global illumination varia2
over color
tion over a pixel results in deviation in variance m,t
distance that may cause false detection or miss actual target.
2
is calculated as the square of the difference
The variance m,t
between current color distance and mean color value over pixel
xt as follows:
2
m,t
= ( m,t m,t )2

(4)

2
is curWhere m,t denotes mean color distance and m,t
rent color distance value belongs to codeword cm at pixel
xt . To adaptively update variance to match codeword with
2
of each codeword is formulated
background model the m,t
as given in Eq. 5.

Fig. 2: Color vector for the different channel.

The decision threshold value T Hcm is dynamically adapted

as follows:
T hcm = + cm

(7)

Here is xed parameter. Furthermore, the upper and lower

bound TLower T h TU pper are taken such that the value
of decision threshold cannot reach beyond specic bounds.
As shown in in Fig 5 the decision threshold varies across
mean color distance value is proportional to . In a case of
highly dynamic environment due to high variation in color
distance the decision threshold will also increases and reduces
false positive foreground pixel. Also in situation with low
lighting conditions where there is low variation occur even
with moving objects the decision parameter with low value
will detect change even with low color distance value.
B. Multi-Layer Codebook Model

2
m,t

= (1

2
),t1

2
m,t

(5)

As different lighting condition and global effects shows illumination variation and background dynamics may show high
variation in color distance. It may cause false detection even
without any moving object. To make model adaptive the color
distance formula is modied as given in Eq. 1.
colordist(xt , vi ) =

abs(i )

(6)

Where m is the variation in color distance vector of a pixel

xt belongs to background model codeword cm and m is color
distance calculated in Eq. 1. It has been observed that with
highly dynamic environment or illumination variation there are
signicant color difference occur in background leads to false
detection. Here by dividing color difference with it stabilised
color distance. In situations with low lightning conditions and
static background when there are low difference is found in
color distance measure low value of sigma does not affect
decision threshold value very much.
Instead of using constant distance parameter that build
cylinder around mean color vector adapting decision threshold
T hcm is used in this paper. It creates cone with center as m
and radius at the section is depend upon decision threshold
T hcm as shown in Fig. 4.

112112

It has been observed that uncovered background region

shows color similarity with their neighbouring pixels over
spatial domain. For example when an moving object become
stationary it covers the portion of the actual background. Depending upon learning rate of codebook model the codeword
of that actual background will be removed when maximum
negative run length exceeds the threshold value. Later on when
this temporary background leaves that space, it creates set of
holes as falsely detected foreground. In this subsections, we
explain improvements over fundamental codebook models that
shows an increase in performance.
As basic codebook model does not allow updation of background model by creating new codewords, it generates false
foreground result if there occur changes in background. A
multi-layered modeling approach that include uncovered background codeword in background model is propose in this
paper. The detail description of implementing multi-layer in
codebook model is given in this section. After the initialization
period of codebook construction for background model, for
every new frame, the observed pixel color value xt at each
pixel position is compared against each codeword present in
background codebook model M. If pixel xt match with any
of the codeword in M, it is classied as background and
corresponding codeword is updated. If none of the codeword
in M match with current pixel it is classied as foreground

Fig. 3: Cylindrical color distortion measure

Fig. 4: Conical color distortion measure

Fig. 5: Adaptive parameters for pixel specied in image across the frame sequence.

and check for matching codeword in H and if match found

it update that codeword. If no match found create a new
codeword in H. In a dynamic environment where the number
of moving objects are large in number the size of H keep
increasing. In fundamental approach to maintaining algorithm
memory efcient codewords are delete or shift from H to M
if they satisfy following conditions:
H H {hi |hi H, if hi TH }

(8)

Here Eq. 8 is used for deletion of codeword from H if it does

not match for long duration i.e. exceeds threshold TH .
M M {hi |hi H, if f reqhi Tadd }

(9)

113113

In Eq. 9 codeword hi is shifted from H to M if it nds a

match with the pixel in more than Tadd consecutive frames
i.e. f reqhi reached up to Tadd .
M M {ci |ci M, if ci Tdelete }

(10)

The codeword is deleted from M using Eq. 10.

To include uncovered background region codeword into the
background codebook, the decision is taken on two conditions.
First each codeword is checked if it matches with background
color for long and frequency belongs to that codeword reach up
to the threshold value. Second it shares color similarity with its
neighbours. The procedure of including codeword belongs to

uncovered background region in background model is outline

in algorithm 1: Here it is considered that each uncovered
Algorithm 1 Codeword belongs to uncovered background
region
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:

for each input pixel x at frame t, xt = (R, G, B), x =

R2 + G2 + B 2 do
Find a matching codeword to xt in background codebook M
if found then
Match=1, then update the codeword
if f reqm T hH igh then
for each codeword Mt {Mt |t Nx } do
Try to nd matching codeword in Mt with xt
if found then
Add match codeword in uncovered background layer U .
end if
end for
end if
end if
end for

background pixel share color similarity with its neighbour and

remain visible for minimum time duration which is represent
by T hH igh. Here Mt denote the codebook belongs to background model of pixel t Nx , Where Nx are the neighbouring
pixels of x. In this work four-connected neighbourhood is
considered for keeping our method computationally efcient.

art methods. Proposed approach is compared with ve

other background subtraction methods as basic codebook
model(CB Kim), Adaptive background mixture models
(GM M Stauf f er), Improved adaptive Gaussian
mixture model (GM M Zivkovic), Multi-scale spatio
temporal background model(M S ST BM ), Kernel density
estimation(KDE M ittal) and (V ibes Barnich). The
sequences used in this work are downloaded from standard
datasets change detection CDnet that are publicly available
over internet [13]. As proposed method in this paper is pixel
based method and results of other state-of-art methods used
for evaluation is post processed using Median smoothing
lter of size 3 3.
The analysis over six video sequences belongs to dynamic
background categories in CDnet dataset are performed. In
dynamic background category each frame sequence presents
challenges due to the dynamic background. Fig.6 and Fig. 7
presents precision and recall scatter plot obtained by proposed
model and other state of art methods over frame sequence
belongs to dynamic background category in CDnet dataset.
The proposed method overcome other background subtraction
in dynamic background environment is conrmed by high
value over different frame sequence used for evaluation.
For comparison qualitative results Fig. 8 shows the change
detection result over frame number 1500 belongs to video
fall generate by proposed methods and other background
subtraction methods.

V. E XPERIMENTAL R ESULTS
In this section, we show results of the proposed
approach and its comparison with other state of the

Fig. 6: The precision scatter plot for dynamic background category.

114114

VI. C ONCLUSION

R EFERENCES

This paper proposed an approach that deals primarily with

pixels detected falsely due to the dynamic background. The
uncovered background is represented by codeword using the
spatial context of a pixel with its neighbours. Improvement
in codebook proposed in this paper is estimation policy
for decision threshold value depends upon background
dynamics. The cone-shape color distance measure is used
to deal with global illumination changes. The proposed
method eliminate most of the falsely detected region. The
quantitative and qualitative analysis over different frame
sequence conrmed that proposed method perform well in
the dynamic background environment.

[1] Toyama, K.; Krumm, J.; Brumitt, B.; Meyers, B., Wallower: principles
and practice of background maintenance, Computer Vision, 1999. The
Proceedings of the Seventh IEEE International Conference on , vol.1,
no., pp.255,261 vol.1, 1999.
[2] Wren, Christopher Richard, et al. Pnder: Real-time tracking of the
human body. Pattern Analysis and Machine Intelligence, IEEE Transactions on 19.7 (1997): 780-785.
[3] Stauffer, Chris, and W. Eric L. Grimson. Adaptive background mixture
models for real-time tracking. Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on.. Vol. 2. IEEE, 1999.
[4] Zivkovic, Zoran. Improved adaptive Gaussian mixture model for background subtraction. Pattern Recognition, 2004. ICPR 2004. Proceedings
of the 17th International Conference on. Vol. 2. IEEE, 2004.
[5] KaewTraKulPong, Pakorn, and Richard Bowden. An improved adaptive
background mixture model for real-time tracking with shadow detection. Video-based surveillance systems. Springer US, 2002. 135-144.
[6] Elgammal, Ahmed, David Harwood, and Larry Davis. Non-parametric
model for background subtraction. Computer VisionECCV 2000.
Springer Berlin Heidelberg, 2000. 751-767.
[7] Mittal, Anurag, and Nikos Paragios. Motion-based background subtraction using adaptive kernel density estimation. Computer Vision and
Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE
Computer Society Conference on. Vol. 2. IEEE, 2004.
[8] Barnich, Olivier, and Marc Van Droogenbroeck. ViBe: A universal
background subtraction algorithm for video sequences. Image Processing, IEEE Transactions on 20.6 (2011): 1709-1724.
[9] Kim, Kyungnam, et al. Real-time foregroundbackground segmentation
using codebook model. Real-time imaging 11.3 (2005): 172-185.
[10] Sigari, Mohamad Hoseyn, and Mahmood Fathy. Real-time background
modeling/subtraction using two-layer codebook model. Proceedings of
the International MultiConference of Engineers and Computer Scientists.
Vol. 1. 2008.
[11] Guo, Jing-Ming, et al. Hierarchical method for foreground detection
using codebook model. Circuits and Systems for Video Technology,
IEEE Transactions on 21.6 (2011): 804-815.
[12] Van Droogenbroeck, Marc, and Olivier Paquot. Background subtraction: Experiments and improvements for ViBe. Computer Vision and
Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on. IEEE, 2012.
[13] Goyette, N.; Jodoin, P.; Porikli, F.; Konrad, J.; Ishwar, P., Changedetection.net: A new change detection benchmark dataset, Computer Vision
and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer
Society Conference on , vol., no., pp.1,8, 16-21 June 2012.

Fig. 7: The recall scatter plot for dynamic background category.

115115

Fig. 8: The qualitative results for frame number 1500 of fall video.

116116