Smoke Detection in Video Surveillance Using Optical Flow and Green’s

Theorem
Melih Altun and Mehmet Celenk
School of Electrical Engineering and Computer Science
Stocker Center, Ohio University
Athens, OH 45701 USA
{ma231709, celenk}@ohio.edu
http://www.ohio.edu

Abstract - Finding smoke in surveillance videos
can be crucial in early detection of fire
emergencies. Such early detections improve
damage prevention and control by enabling the
authorities to take the necessary precautionary
steps. This paper describes a smoke detection
technique developed for videos taken in visual
band. The method makes use of optical flow and
color filtering to detect smoke covered regions and
the associated smoke sources. Next it extracts
dynamic smoke features such as average upwards
motion above the source and divergence around the
source via Green’s theorem. This determines
whether the selected region contains smoke. In
turn, the extracted dynamic characteristics of the
smoke pattern greatly improve detection accuracy
of the method and produce highly robust results as
demonstrated in the experimental results.
Keywords: Smoke detection, video surveillance,
optical flow, Green’s theorem
1 Introduction
This paper presents a smoke detection method
based on video processing. Detection of smoke in
videos is particularly useful in surveillance and
automatic event detection applications.
Traditional smoke detectors measure certain
chemicals and particles in the air. In small closed
spaces these particles rapidly reach a high
concentration so detectors can generate early
warnings. However, they do not function as well in
large spaces and they are simply not applicable to
outdoor environments.
Video based fire and smoke detectors are known to
overcome such difficulties by their remote sensing
capabilities. Moreover, video based systems are
also capable of obtaining information such as the
exact location and the progress of fire.
Fire detection methods use visual and infrared (IR)
cameras to find fire sources. They use color, shape
and dynamic features such as flickering [1,2]. The
disadvantage of both visual and IR methods is that
they need direct visual of fire. In cases where fire is
not directly seen or it is at an early phase where
only smoke is visible, smoke detection systems
provide an earlier response.
Smoke detection systems mainly use methods based
on wavelet [3] information. [4] combines wavelet
methods with Hidden Markov trees to account for
the dynamic characteristics of smoke. Optical flow
[5] and other dynamic features such as velocity
histograms are also used [6]. [7,8] use color
information and motion estimation methods for
detecting smoke covered regions in a video frame.
Although color information is vital for eliminating
non-smoke objects, it is not sufficient for detection
due to the presence of other objects with colors
similar to smoke. Therefore, it is used in
combination with other methods to reduce false
detections. The idea behind our approach is similar
to the method developed in [8]. However, instead of
simple motion detection we use optical flow to
model smoke behavior more precisely.
In this paper a smoke detection method based on
optical flow and dynamic characteristics of smoke
is described. Color filtering is also applied to
reduce the possible smoke sources. After obtaining
an optical flow vector field and determining a
candidate smoke source, Green’s theorem is applied
around the selected source to test the divergent
behavior of smoke. Moreover, since heat
convection above the source causes the smoke to
rise, average upward motion above the source is
also calculated to improve the confidence of
detection. The remainder of this paper is organized
as follows: In section II the details of the smoke
detection method is described. Experimental results
are presented in section III. The last section
contains conclusions.
2 Description of the Method
Color Filter: Smoke color can vary greatly from
very dark to almost white shades. To select those
gray levels and eliminate image segments with
other colors we convert RGB values to HSL values
and consider S value for selection. Since gray
pixels have low saturation we take pixels with S
values less than a certain threshold as a potential
smoke pixel. As it is stated in [8], sometimes smoke
can have a bluish tone. So, for pixels where blue
value is higher than red and green we use a slightly
higher saturation threshold to allow for bluish type
of smoke. After selection of possible smoke pixels
a Basic Sequential Algorithmic Scheme (BSAS) [9]
clusters nearby smoke colored pixels together.
However, most of these smoke colored clusters are
usually stationary background areas. Using optical
flow, which will be discussed in the next
subsection, we are able to select dynamic regions
and eliminate those stationary areas.
2.1 Optical Flow
To obtain the foreground objects in a video and
accurately measure their motions optical flow is
utilized. A fast and efficient optical flow method is
essential for the real time requirements of detection.
Our optical flow calculation is mostly parallel to the
method proposed in [10]. First a spatiotemporal
gradient of the video sequence I(x,y,t) is obtained.
Then a 3D Gaussian filter is applied with linear
convolution to smooth the gradient as in equations
(1) and (2).
, , , ,
, ,
, , , , ∗ , ,
where H is the 3D Gaussian function and * is the
convolution operation. Then the smoothed gradient
vector is multiplied with its transpose to obtain the
tensor matrix T.
′
⋅ (3)
Following the calculation of tensors, the velocity
vectors are calculated using parameters t1 through
t6, which are the elements of the symmetric tensor
matrix.
t 6t 4  t5t 2 and t t t t
vx  v y  5 4 62 1
t1t 2  t 42 t1t 2  t 4
Finally the velocity vectors for each pixel
coordinate are smoothed by a 2D median filter [11]
to obtain the final x and y component values of
optical flow vector field. Optical flow for the
sample frames given in Figure 1 can be seen in
Figure 2.
Since we are interested in the motion of smoke and
not the other objects, we mask our optical flow map
with the smoke colored clusters found by our color
filter. So we obtain a vector map showing only the
motion of smoke colored foreground regions. And
since these regions are clustered we can detect the
appearance of a cluster and mark it as a smoke
source. Green rectangle in Figure 3 and in other
figures showing the detection results denotes the
detected smoke source.
2.2 Application of Green’s Theorem
After finding a potential smoke source with color
filtering and obtaining the optical flow vector field
with the methods described in previous subsections
we are able to test the dynamic characteristics of
smoke. The first dynamic characteristic we test is
the divergent behavior of smoke around its source.
One way to achieve this is to apply Green’s
Theorem. Green’s Theorem states that outward flux
of a vector field across a closed boundary is equal
to the divergence of the vector field integrated over
the region enclosed by the boundary. So if we
define a simple closed curve, such as a small
rectangle around the smoke source, a continuous
positive outward flux across this rectangle indicates
a divergent source within it.
We already have the horizontal and vertical
components of the optical flow field. Divergence of
the field is simply given by the equation 6.
, , ′
(1) (5)
(2) ∙ (6)
General equation of Green’s Theorem is given in
equation 7.
∮ ∙ ∬ (7)
Here, is the vector field, is the normal to the
closed curve C, s is an infinitesimal line segment
over C and R is the region surrounded with C. We
use the left side of this equation. Instead of
calculating the divergence over the entire source
region and summing it, we calculate the outward
(4) flux simply by taking the inner product of motion
vectors and the normal vector over the rectangle
surrounding the potential smoke source and sum the
values. Divergent smoke sources are expected to
produce positive results.
2.3 Average upwards motion above the
source
Another dynamic characteristic of smoke is that it
rises right above its source due to heat convection.
We only consider the area above the source because
as the smoke spreads away from the source it cools
down and starts moving with the air currents in that
environment. It even displays random Brownian
motion if there is no dominant air current.
However, its motion is predictable when it is close
to the heat source. Therefore, we consider a
rectangular region above the possible smoke source
and examine the optical flow in this area. We take
the average of vertical component of flow vectors
in that region. If the result shows consistent
upwards motion it is strong evidence that the
possible source is an actual smoke source.
Consistent upwards motion in conjunction with
divergent behavior suggests that the selected cluster
in the video frame is real smoke.

Figure 3: Smoke detection result where smoke is marked

3 Experimental Results
For testing our smoke detection method we used
sample videos from Bilkent University Signal and
Image Processing Group [12] and Octec dataset
from [13].
To demonstrate optical flow two consecutive
frames are shown in Figure 1. Figure 2 shows the
optical flow result in which, motions of rising
smoke and running person are clearly seen. The
result of smoke detection method is displayed in
Figure 3
by the red closed curve and smoke source is marked by
the green rectangle.
In Figure 4 average of upwards optical flow vectors
for the smoke source in Figure 1 is shown. Figure 5
displays the total outward flux around the smoke
source in each frame of the video. Note that both
average upwards motion and total outwards flux
consistently maintain positive values and support
our assertion.

Figure 1: Consecutive frames from a sample video

Figure 4: Average upwards motion above the smoke

source for the video sequence given in Figure 1

Figure 2: Optical flow results. Notice that hue represents

direction as indicated by the circle on bottom right and
saturation represents magnitude of the motion
Figure 5: Total outward flux around the smoke source for
the video sequence given in Figure 1. Notice the dip
around frame #41. It corresponds to the time the person
crosses in front of the smoke source and occludes it.
In order to measure our smoke detection rate, we
also visually selected smoke covered areas in the
video frames and established a ground truth. Then
we compared the smoke regions detected by our
algorithm to this ground truth. Figure 6 shows the
rate of detection obtained by this comparison.

Figure 8: Average upwards motion above the smoke

source for the video sequence given in Figure 7

Figure 6: Rate of smoke detection for the first sample

video sequence

Figure 7 shows another detection result from frame

sequence with the appearance of smoke. Results of
dynamic feature analysis are depicted in Figures 8
and 9. Again, average upwards motion and total
outwards flux value remain consistently positive
after smoke appears and source is detected.
Detection ratio obtained by comparing smoke areas
Figure 9: Total outward flux around the smoke source for
detected by our algorithm to ground truth also the video sequence given in Figure 7
remains high as seen in Figure 10.

Figure 10: Rate of smoke detection for the second sample

Figure 7: Smoke detection results for the Octec Dataset video sequence
 Figure 12: A frame from the video sequence with no
smoke occurrence

Figure 13: Average upwards motion above the detected

source for the video sequence given in Figure 12

Figure 11: Further detection results from Bilkent smoke

video dataset

Other detection results from different video

sequences are presented in Figures 11 a, b and c.
In Figure 12, a case without any smoke is shown.
Even though an area on the left is selected as a
possible source, total outwards flux and average
upwards motion (Figures 13,14) show no signs of
dynamic smoke behavior. Hence, there is no
detection.

Figure 14: Total outward flux around the detected source

for the video sequence given in Figure 12
4 Conclusion
In this paper we have presented a new method for
detecting smoke with color filtering, optical flow
and utilizing Green’s Theorem. At first, by using a
color filter possible smoke covered regions are
found. Then, optical flow enables detection of
motions in the video. Using the magnitude and the
directionality of these motions, total outward flux
and average upwards motion features are tested in
order to confirm the presence of a divergent smoke
source and a heat source in that selected region.
Extracting these dynamic smoke features produces
highly reliable results as it is demonstrated with the
test cases.
[9] S. Theodoridis and K. Koutroumbas, Pattern
Recognition, 4th ed. Elsevier, 2009.
[10] Z. Wei, D.J. Lee, B.E. Nelson, J.K. Archibald,
and B.B. Edwards, FPGA-Based Embedded Motion
Estimation Sensor, Int. Journal of Reconfigurable
Computing, vol. 2008, no. 636145, p. 8, 2008.
[11] J. S. Lim, Two-Dimensional Signal and Image
Processing, Prentice Hall, 1990
[12] Bilkent EE Signal Processing group,
http://signal.ee.bilkent.edu.tr/
[13] D. Dwyer, Octec Limited,
http://www.octec.co.uk/

5 References
[1] B.U. Toreyin, R.G. Cinbis, Y. Dedeoglu, A.E.
Cetin, Fire detection in infrared video using wavelet
analysis, Optical Engineering, vol. 46(6), pp.
67204-1 – 67204-9, 2007

[2] G. Marbach, M. Leopfe, T. Brupbacher, An

image processing technique for fire detection in
video images, Fire Safety Journal, vol. 41, pp 285-
289, 2006

[3] B.U. Toreyin, Y. Dedeoglu, A.E. Cetin, Contour

based smoke detection in video using wavelets,
14th European Signal Processing Conference
EUSIPCO, 2006

[4] R.J. Ferrari, H. Zhang, C.R. Kube, Real-time

detection of steam in video images. Pattern
Recognition, vol. 40(3), pp. 1148-1159, 2007

[5] I. Kolesov, P. Karasev, A. Tannenbaum, E.

Haber, Fire and smoke detection in video with
optimal mass transport based optical flow and
neural networks, 17th IEEE International
Conference on Image Processing (ICIP), pp. 761-
764, 2010

[6] J. Vicente, P. Guillemant, An image processing

technique for automatically detecting forest fire,
International Journal of Thermal Sciences, vol.
41(12), pp. 1113-1120, 2002

[7] D. Kim, Y. Wang. Smoke detection in video,

IEEE WRI World Congress on Computer Science
and Information Engineering, Vol. 5, pp. 759-763,
2009

[8] F. Yuan, A fast accumulative motion orientation

model based on integral image for video smoke
detection. Pattern Recognition Letters, vol. 29(7),
pp. 925-932, 2008