Professional Documents
Culture Documents
Shadow Detection PDF
Shadow Detection PDF
(2017) 2017:49
EURASIP Journal on Image
DOI 10.1186/s13640-017-0198-x and Video Processing
Abstract
Many surveillance and forensic applications face problems in identifying shadows and their removal. The moving
shadow points overlap with the moving objects in a video sequence leading to misclassification of the exact
object. This article presents a novel method for identifying and removing moving shadows using stationary wavelet
transform (SWT) based on a threshold determined by wavelet coefficients. The multi-resolution property of the
stationary wavelet transform leads to the decomposition of the frames into four different bands without the loss of
spatial information. The conventional discrete wavelet transform (DWT), which has the same property, suffers from
the problem of shift invariance due to the decimation operation leading to a shift in the original signal during
reconstruction. Since SWT does not have the decimation operation, the problem of shift invariance is solved which
makes it feasible for change detection, pattern recognition and feature extraction and retrieves the original signal
without the loss of phase information also. For detection and removal of shadow, a new threshold in the form of a
variant statistical parameter—“skewness”—is proposed. The value of threshold is determined through the wavelet
coefficients without the requirement of any supervised learning or manual calibration. Normally, the statistical
parameters like mean, variance and standard deviation does not show much variation in complex environments.
Skewness shows a unique variation between the shadow and non-shadow pixels in various environments than the
previously used thresholds—standard deviation and relative standard deviation. The experimental results prove that
the proposed method works better than other state-of-art-methods.
Keywords: Object tracking, Shadow detection, Skewness, SWT, Video forensics and surveillance, Wavelet transform
© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made.
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 2 of 11
algorithms consider the colour-based methods [2–8] and shadow regions decides whether they are really formed
texture-based [9–12] methods to differentiate shadows or manipulated. The inconsistency of the texture be-
from the foreground objects. They are categorized under tween the shadow and background regions reveal the
property-based techniques. tampered content from the images [24] since it is known
The colour-based methods use the following facts about that the shadow does not change the texture of the back-
the shadow and background. The red-green-blue (RGB) ground [12, 14]. Farid [25] gave three geometric tech-
values, hue and saturation in hue-saturation-value (HSV) niques for detecting the traces of digital manipulation:
colour space and grey level of a shadow are inferior to the vanishing points, reflections and shadows. The author
background in the corresponding pixel [12, 13]. The vari- reported with both real-time image and video forgery by
ation between those values of shadow and background analysing the shadows in them. This supports the need
show a gradual growth between adjacent pixels [14]. of detection of shadow in image and video forensics by
The shadow and the background have similar texture. which we were motivated to propose a moving shadow
Texture-rich object has a texture-less shadow [12, 14]. detection method. All the methods discussed so far dealt
The entropy value or the Gaussian filter derivatives are the shadow detection or removal process in spatial do-
helpful in extracting the texture of the image segments main techniques that suffer from inaccurate segmentation
[15]. There is a difference in the illumination of shadow due to the non-removal of noise or failure to detect new
and background which also helps to differentiate shadow appearance automatically. Only few methods are available
pixels. Lesser boundary when compared to background in wavelet domain that rectifies the problems in spatial
and not many interior edges when compared to objects domain techniques for a better identification of the mov-
are used to identify shadow pixels. But still, the shadows ing cast shadows.
remain attached to their objects [16]. Even though object Guan [26] explored the properties of the HSV colour
and shadow have the same movement, their position model for shadow detection and removal using dyadic
separates them [17]. Skewness is a variant statistical fea- multi-scale wavelet transform. The standard deviation of
ture with respect to shadow and non-shadow areas for the wavelet coefficients from the value component helps
locating shadows provided a proper edge detection algo- to identify the shadow pixels from the foreground pixels.
rithm is used [15]. An invariant feature across shadow In a similar fashion, Khare et al. [27] used relative stand-
boundary is the image gradient. ard deviation of the wavelet coefficients to separate the
For majority of the moving shadow detection methods, shadows from the foreground pixels. Since the wavelet
chromatic analysis is the first step. The hue to intensity transforms decompose the input into high- and low-
ratio [18] is used to categorize shadow and non-shadow frequency values, the threshold values get updated auto-
pixels. Also, c1c2c3 colour space is used to distinguish matically. Combining the saturation component with the
shadow areas by introducing a variation in the colour value component, the threshold identifies the region of
space transformation. But still, if the RGB values remain the shadows to be removed. The algorithms discussed so
the same, the variation in the transformation formula far still have the deficiency of the accurate shadow de-
will not show any difference. Colour space-based methods tection and removal.
[19] such as HSI, HSV, YIQ and YCbCr analyse the inten- In the proposed approach, we have also used the HSV
sity and colour properties of shadows in aerial images and colour space as it corresponds closely to the human per-
calculate a hue to intensity ratio for each pixel. The ception of colour and separates chromaticity and lumi-
shadow regions are extracted based on a threshold, which nosity easily. We also used variant statistical parameter
is not clearly classifying the dark blue and dark green “skewness” of the wavelet coefficients to detect and re-
surfaces. To improve the shadow detection results, a move the shadows.
consecutive thresholding [20] can be done. Based on In the rest of the paper, Section 2 describes stationary
the lighting conditions, an invariant colour model [21] wavelet transform (SWT); Sections 3 and 4 explains the
which identifies the shadow can be used, but this may threshold selection and the proposed method respectively.
not work for all types of images. HSI colour space along Section 5 explains the experimental results, and Section 6
with the colour attenuation relationship [22] is analysed gives the comparisons of the proposed method and other
to detect the shadow in colour aerial images. existing and state-of-art-methods [1, 7, 12, 26, 27]. Finally,
The shadow properties discussed so far can be utilized discussions and conclusions of the work are given in
in image and video forensics to expose the falsification Sections 7 and 8.
of photos and videos by detecting the shadows using
their geometric and shading information [23]. Those im- 2 Stationary wavelet transform (SWT)
ages and videos can then be properly annotated for fu- In this section, we have briefly discussed SWT and the
ture forensic analysis after the removal of fake content. reasons for using SWT for detection and removal of
In image and video forensics field, the detection of shadow from moving objects. Guan [26], Khare et al.
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 3 of 11
Fig. 1 a Negative skewness—the left tail is longer and the mass of the distribution is concentrated on the right of the figure. b Positive skewness—the
right tail is longer and the mass of the distribution is concentrated on the left of the figure
3 Threshold selection
An optimal threshold has to be determined for the ac-
curate shadow detection process. Guan [26] used stand-
ard deviation (σ) as the threshold, Khare et al. [27] used Fig. 2 a Example Image1 having similar patterns. b Histogram of
images in a. c Example Image2 having similar patterns. d Histogram
relative standard deviation (σ/μ) as the threshold and we
of images in c
have chosen skewness as the threshold in SWT domain.
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 4 of 11
For a sample of n values, skewness (Skew) parameter is where x is the one of the sample value, μ is the sample
defined as: mean and σ is the standard deviation of the n values.
Skewness is a measure of the degree of asymmetry of a
x−μ3 distribution. The skewness value can be positive, nega-
Skewness ¼ E ð1Þ tive or even undefined [29]. A negative skewness indi-
σ
cates that the tail on the left side of the probability
Fig. 3 Visualized comparison results of the proposed method with others. a Input frame from Highway I, Intelligent Room, and Hallway. b Ground
truths of the respective frames from video sequence. c Results of Sanin et al. [12]. d Results of Cucchiara et al. [7]. e Results of Guan et al. [26]. f Results
of the proposed method
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 5 of 11
Fig. 4 Plot of a vertical, b horizontal and c diagonal wavelet coefficients of frame no. 100 from Highway I video sequence: The grey colour line
shows the original shadow pixels, and the arrow mark in grey colour indicates the end of the shadow points. The blue colour square indicates the
detected shadow pixels using skewness, and the arrow mark in blue colour indicates the end of the shadow points. The black and red squares
indicate the detected shadow pixels using standard deviation [26] (σ) and relative standard deviation [27] (σ/μ)
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 6 of 11
Fig. 5 Plot of a vertical, b horizontal and c diagonal wavelet coefficients of frame no. 130 from Intelligent Room video sequence: The grey colour
line shows the original shadow pixels, and the arrow mark in grey colour indicates the end of the shadow points. The blue colour square indicates
the detected shadow pixels using skewness, and the arrow mark in blue colour indicates the end of the shadow points. The black and red squares
indicate the detected shadow pixels using standard deviation [26] (σ) and relative standard deviation [27] (σ/μ)
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 7 of 11
density function is longer than the right side and the Table 2 Shadow discrimination rate (ε) for representative video
bulk of the values lie to the right of the mean. A positive sequences
skewness indicates that the tail on the right side is lon- Method used Shadow discrimination rate (ε)
ger than the left side and the bulk of the values lie to the Highway I Intelligent Room Hallway
left of the mean as shown in Fig. 1. Sanin et al. 95.57 95.76 95.92
The motivation for selecting skewness as one of the Cucchiara et al. 74.08 90.74 95.39
distinguishing features to represent shadow is it always
Guan et al. 83.25 95.36 85.11
shows a variation from the skewness of the creating ob-
ject and the casted background [30]. For an image hav- Khare et al., 89.31 92.19 87.91
ing homogeneous reflective surface, the skewness of the Proposed method 96.31 98.58 94.15
luminance histogram and sub-band histograms are cor-
related with the shadows regions present in it [29]. In new threshold for detection and removal of shadow in
Eq. 1, μ is the average luminance of the object and σ is SWT domain. Guan [26] and Khare et al. [27] applied the
the standard deviation of the luminance of the object thresholds to the discrete wavelet coefficients (DWT) that
along with its neighbours in wavelet domain. The acceler- losses the phase information. The loss of phase informa-
ation of (x − μ) with the cubic value enhances the edges tion in DWT and the difficulty faced when there are simi-
around the object present in a scene; the summation and lar patterns of input frames are rectified by applying SWT
normalization make the edges of shadows blurry since and skewness as a threshold respectively.
they are soft and fasten the process of shadow detection.
So, the skewness of the wavelet coefficients has the
boundary information of the objects which helps us to 4 Proposed algorithm
clearly segregate them from the shadows. Also, the exist- The proposed approach considers a reference frame as
ing thresholds may not show any differentiation of either the background model from the video sequence and the
as object or shadow for those pixels whose values of consecutive frames are processed one by one to detect
saturation and hue are undefined. The mean μ and the foreground and shadow pixels. The reference frame
standard deviation σ of such patterns as shown in Fig. 2 and the current frame are converted from RGB colour
are identical leading to misclassification. They differ only space to HSV colour space. The absolute difference of
in the sign of skewness because of the cubic acceleration the two frames is taken with respect to hue, saturation
and normalization of the luminance value of the object. and value component. They are represented as ΔH, ΔV
Therefore, skewness is selected as a stable threshold in and ΔS respectively. The stationary wavelet transform is
our work for classification of moving shadow pixels from applied to the absolute difference components of value
the objects. Though the thresholds σ and (σ/μ) are able (ΔV) and saturation (ΔS) to get the wavelet coefficients
to detect and remove shadow, their performance de- denoted as WΔV and WΔS. Also, the variant statistical
grades when the foreground object and the background parameter skewness is calculated for the wavelet coeffi-
share the same or dark colour. cients for the value and saturation component denoted
In the proposed method, we have applied the threshold as (Skew)WΔV and (Skew)WΔS respectively. They act as
on stationary wavelet coefficients of value component of an automatic threshold to classify the moving pixels as
HSV to detect moving object with shadow. For removal of foreground or shadow pixels.
shadow, we have applied logical AND on thresholded sta- The pixels having a greater value than the automatic
tionary wavelet coefficients of value component and those threshold “variant statistical value − skewness” of the value
of saturation component. Instead of manual calibration of component are categorized as the moving pixels which
threshold which needs some predefined parameters, we include the foreground and shadow pixels. Equation 2
propose a variant statistical parameter, i.e. skewness as a represents this situation.
Fig. 6 Shadow discrimination rate of various videos using skewness (proposed method), standard deviation [26] and relative standard deviation
[27]. a Shadow discrimination rate in Highway I sequence. b Shadow discrimination rate in Shopping Mall. c Shadow discrimination rate in “Sarvam”
regional movie song. d Shadow discrimination rate in Lobby Hall
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 9 of 11
Fig. 7 Visualized comparison results of the proposed method with the state-of-art-methods. The first frame is the input, second is the ground
truth, third is the proposed method result, fourth is the Guan [26] method result and fifth is the Khare et al. [27] method result. a Highway I video
sequence and Lobby Hall video sequence. b “Sarvam” regional movie song—long and short shadows
Nagarathinam and Kathavarayan EURASIP Journal on Image and Video Processing (2017) 2017:49 Page 10 of 11