Fire Detection System with Utilization of Industrial

Video Surveillance System

SABARI V VIGNESHWARAN P PRUSHOTHAMAN

Electrical and Electronics Engineering
Kalaignarkarunanidhi Institute of
Technology
(Affiliated to Anna University)
Coimbatore,India
Sabariveerasekar2004@gmail.com
Electrical and Electronics Engineering
Kalaignarkarunanidhi Institute of
Technology
(Affiliated to Anna University)
Coimbatore,India
vwrv8825@gmail.com
Electrical and ElectronicsEngineering
Kalaignarkarunanidhi Institute of
Technology
(Affiliated to Anna University)
Coimbatore,India
Prushothaman2001@gmail.com

Abstract - Fire occurrence is the most destructive hazard for
production, as it causes irreparable damage to the enterprise,
equipment, health and life of people, as well as the environment.
Therefore, recognizing fires and preventing them in a timely
manner is very important. Modern production facilities are
equipped with widely distributed video surveillance systems. By
analyzing the transmitted images, the algorithm is able to
recognize the source of fire and activate an alarm, thereby
ensuring the safety of the enterprise. The algorithm deploys
various characteristics of flame images during data processing.
The mixed Gaussian model is used to distinguish the dynamic
area from the static background, and the color characteristics of
pixels in the RGB model are analyzed to detect fire. In order to
increase the accuracy of the results obtained, the frequency of
changes in the contour of the dynamic area (flickering) in the
target area of fire is checked.
Keywords - industrial video surveillance, petrochemical
industry, fire·detection, color model, gaussian distribution
I. INTRODUCTION
Industrial video cameras of explosion-proof design are
installed at certain oil refining facilities, which are most often
prone to fire or emergency situations. These facilities include:
 Pump stations;
 Compressor stations;
 Process heaters;
 Reactors;
 Fractionators;
 Feedstock tank parks.
The main task of video surveillance is to record emergency
situations in order to further clarify the causes of their
occurrence. However, due to the human factor, it is not always
possible to respond to emergencies in a timely manner and
avoid major damage or casualties.
Within this project, various methods of image recognition
were studied and a software package was developed for
automated recognition of open fire or ignition of oil refining
equipment.
The software product was developed and implemented in
the high-level general-purpose programming language Python
with additional libraries for image processing.
Most of the equipment of a typical industrial oil refinery
faces strict operating conditions and a high risk of fire and open
fire. The main risk is related to the delayed response of service
personnel to emergency situations. This happens usually due to
the absence of the operator at the workplace or his inattentive
observation of the process. For immediate response of
operators and special services, it is better to automate the
process of fire recognition. If a fire is detected, the system's
response must be accompanied by a sound alarm and a
message sent to the security services.
It is not always possible to use smoke detectors or gas
analyzers on industrial sites, especially in open spaces.
Therefore, the task of monitoring the state of the process is
assigned to the operator. This work is routine and requires
constant presence of the operator at the workplace. The human
factor impact, for example, inattention, fatigue or drowsiness of
the employee on night duty should also be kept in mind.
The main goals of human replacement in the process safety
monitoring are as follows:
 Releasing the operator from monotonous routine
operations to solve other more important tasks;
 Reducing material losses through timely response;
 Improving the speed of emergency response.
The implementation of a fire detection software package
will reduce the response time to local fires and mitigate
material and human losses. The implementation proposed in
the project will also reduce the operator's activity, leaving him
only partial control over the system itself as a whole.
II. MATERIALS AND METHODS
Let us imagine the structure of the recognition system. This
system processes and analyzes images received from
optoelectronic systems. Given the nature of the task, the
following limitations and features must be considered:
 Fire can occur at any time of the day, so there may be
errors in the form of sun glares or lanterns;
  Due to the limited time to make decisions and the
presence of a closed loop of traffic control, processing, analysis
and recognition must be performed in the real time;
 The system must operate in an independent mode.
Recognition will take place in several stages, the main ones
of which are searching for changes in the moving pixels,
determining the found pixels by color, and checking the
movement of the checked area of fire (flickering).
In order to determine the fire, two main approaches to the
algorithm formation can be used. The first approach pertains to
initial identification of colors similar to fire in the image, and
then search for dynamic areas, i.e. areas where movement
occurs or there is a presumable flame.
The second approach provides for the initial search for
dynamic areas, and after that, the identification of the area of
the presumable fire by the number of pixels similar to the fire
in color in different stages of its activity.
The selection of the implemented approach should be made
based on the characteristics of the controlled area. For example,
if the scenery has a lot of dynamic objects, but the color
scheme does not have colors similar to the flame, then the first
approach should be selected, because this solution would
reduce the computational time of the process, completing the
selection at the first stage. Conversely, if the controlled area
has a lot of colors similar to the flame in color, for example,
illuminating lamps, then the second approach should be
implemented.
First, let us consider the stage of determining the color
characteristics of pixels. The method is based on the analysis of
the space of color mathematical models. Four color models can
be distinguished.
The RGB model that is the most common. Color
reproduction is performed by adding three main colors: red,
green, and blue.
YCbCr color space. It is one of the variations of the RGB
model, where Y-brightness, Cb and Cr are the blue and red
chrominance components. A fuzzy logic system is used to
describe the fire image. The image is fuzzified - it is converted
into a fuzzy set. Then the degree of membership of the
elements in the fuzzy set changes in accordance with the fuzzy
rules.
HSL color model. A model in which color coordinates are
described by three components – tone, saturation, and intensity,
respectively.
HSV color model. Analogous to the HSL model, only the
last coordinate expresses the color value.
The parameters of HSL and HSV models represent a
knowledge base that the operator creates by specifically
these methods, a large knowledge base, which is not always
highlighting
convenient, istherequired,
flame inbecause
the image. However,
there in order
is an infinite to apply
number of
the flame variations.
The main part of the fire recognition algorithm is connected
with the background detection among moving areas or dynamic
objects, such as flame.
One approach to implementation comprises the use of a
neural network that receives the spatial and temporal
characteristics of fire as input data. The flame area is
computationally cut out of the image using the statistical
parameters of the color models described above. Then the
defined contour is linked to the polar coordinates of the initial
image to transform a series of frames into a series of
fluctuations in time. In order to obtain the frequency spectrum,
the received signal is converted into Fourier space. The
frequency spectrum is transmitted to a trained neural network.
A training table should be created at the development stage for
the respective training process. The described method has a
high accuracy of the results obtained, but due to the high
computational complexity, it is not applicable in this work.
Another way to separate dynamic and static objects is to
use the mixed Gaussian model. It is simple and effective at the
stage of modeling a static background, by tracking changes in
all pixels over time and mixing К normal distributions for a
sequence of frames in the video stream. К-distributions are
divided into two groups: the foreground group and the
background group. If a pixel belongs to the background
distribution, it is considered background. The same is true for
the group of foreground pixels. This method can analyze the
background of any complexity, but it has a low calculation
speed, since the analysis takes place at the pixel level and at
high video resolution, the implementation of the method
requires a lot of time.
Let us consider an improved method based on the Gaussian
model. We denote the vector of pixel color characteristics in
the RGB color model in coordinates (x, y)
X xy,t  R xy,t , Gxy,t , Bxy,t ,
T
where t is any moment in time.
Over time, a series of observations is formed as
 
X xy,1 , X,..., X xy,t , which is represented as a stochastic
xy,2
process relative to other pixels. The generated series has a
probability distribution density function. In subsequent
analysis, if each pixel is described by this function sufficiently,
the area will be considered background. It ought to be noted
that pixel parameters are subject to complex distributions,
which makes it more difficult to select and adapt the
distribution density function described above. Therefore, over
time, the pixel is described К by normal distributions. With К
increasing, the complexity of calculations increases as well.
We define the probability of observing a vector of color
characteristics as
K
 xy,k ,t

P( X xy,t )  w ( X xy,t , u xy,t , xy,k ,t )
wxy ,k ,t k 1
where is the weight factor, which is the probability
that the observation belongs to k-th component;
 u xy ,t – vector of mathematical expectations;
 xy,k ,t – covariance matrix of the k-th normal distribution at
the moment of time t;
 - normal distribution density function.
1
( X xy ,t , u xy ,t , xy ,k ,t )  n 1
(2 ) 2 xy ,k ,t
where n is the number of channels, since we use the RGB
model, it is equal to three.
In order to reduce the calculation time, we will assume the
red, green, and blue color channels for each pixel as
independent and having the same dispersion.
  diag 2 , 2 , 2   2
xy,k ,t R G
where I is the unity matrix;
 – standard deviation (mean square deviation).
The greater the value of the weight (w) of the normal
distribution, the more observations will fit the normal
distribution. The stability of the distribution ensures a small
deviation  . The ratio w /  determines the status of the pixel,
whether it belongs to the background or to the foreground, by
K-distribution. The dynamic area is described by several
distributions with small weights. Therefore, the first В in the
distribution of the total number of К are determined for the
background assessment, while the rest ones as the foreground.
Bxy ,t  arg min B  wxy,k ,t  T 
B
 k 1
The value T indicates the threshold, the minimum part of
which is considered the background. For each new pixel, T can
match one of the largest components of K-distribution. This
match is possible if the distance between X
than the threshold value  .
To find the first matching distribution, each new pixel
observation is compared with the existing K-distribution
component.
1, при X xy,t  u xy,t  
M  

xy,k ,t 
0, при X xy,t  u xy,t   xy,k ,t ,

where M xy,k ,t is the matching function of the k-th
distribution at the moment of time t;
 - a number in the range {2.5; 3} to achieve an accuracy
level of 95%.
If the analyzed pixel matches B-distribution, it is considered
background. Otherwise, it will be a foreground pixel.
For practical use, the change in illumination of the
controlled object should be taken into account. The parameters
of the mixed model should be updated. If there are matches
with K-distribution at the moment of time t, then u xy ,t and
 xy ,k ,t are updated according to the current values X xy,t , but
remain unchanged for non-matching distributions. The
described adaptation is implemented by the following
equations:
e
1/ 2( X xy ,t uxy ,k ,t )T 1
xy , k ,t
( X xy ,t uxy , k ,t )
wxy ,k ,t1  (1   )wxy ,k ,t  M xy ,k ,t ,
2 
 u xy ,k ,t 1  (1   xy ,k ,t )u xy ,k ,t   xy ,k ,t X xy,t ,
 2 xy ,k ,t 1  (1   )2 xy ,k ,t   ( X xy ,t  u xy,k ,t 1 )T ( X xy,t  u xy,k ,t 1 ),
 xy ,k ,t xy ,k ,t


 xy ,k ,t   ( X , u , 2
xy ,k ,t I ).
xy ,t xy,k ,t
where α is the background update rate, varying from 0 to 1.
If there are no matches for the current observation vector
I, X xy,t with K-distribution, the component with a small value
B xy,k ,t
w /  is updated in accordance with the expectation set by
X xy,t large variation, and low weight.
In order to improve the accuracy of the results, the flicker
detection method will be applied. Flickering, or flickering
movement of the flame is one of the characteristics of fire that
allows distinguishing the flame even against objects close to it
in color. The phenomenon of flickering is a random change in
the boundaries of the flame from frame to frame. Flickering of
the flame occurs on average with a frequency of 10 Hz. Thus,
by selecting pixels with a given color change frequency, more
accurate data on the presence of fire can be obtained.
III. RESULTS
. As a result of the methods described above, the fire
recognition algorithm is shown in Figure 1.

As a result of the algorithm, the image of the video stream
is displayed on the screen. If the program detects a fire, its
presumable area is highlighted with a rectangle. Various flares,
reflections, and direct sunlight can be mistaken for flames, so
xy ,t and u xy,t is less the program processes only flickering light sources. Also, to
filter out small noises, the minimum threshold area of the
dynamic area was set.
In order to test the program, video files with images of the
fire, which are freely available, were used. The results of the
program processing are shown in Figures 2 and 3.
xy,k ,t ,

Fig. 2. Results of the software product testing
 IV. DISCUSSION
Получение на вход
Image obtained The implemented program can process a single video
изображения
at input
stream at any frame rate. For the future use in enterprises, the
program must be updated by increasing the amount of input
data. Reporting on detected fires should be planned as well.
During testing, the filter range of the color model and the
Поиск
Search for
динамических flickering frequency range of the fire were adjusted. In addition
dynamic areas
областей to the ability to process video recordings, signal processing has
been implemented using the RTSP Protocol.
At present, only one program presented in the consumer
market can recognize fire, this is a program created by the
No
Нет Динамическая Macroscop company. There are no other registered programs
Dynamic area found?
область найдена? with implementation based on our algorithm. Accordingly, the
developed solution can be considered new in the recent times.
ДаYes
The developed program has demonstrated its efficiency in
practice and coped with the assigned task. The program
recognizes flame in the video sequence and does not react to
Извлечение
Extractingцветовых
color
характеристик
characteristics other dynamic objects. The presented software product
interacts with only one video stream, but in the future, it is
planned to expand its functionality to operate with multiple
video streams, since one computer is most often connected to a
multi-stream industrial video surveillance system in the
No
Нет Обнаружен цвет
Color similar to
production conditions.
подходящий
flame found?под
пламя? For its operation, the program does not require high power
and is suitable for computers with average computing power.
ДаYes
ACKNOWLEDGMENT
Оценить
Assess
мерцание
flame The work was carried out with the financial support of the
пламени
flickering Russian Foundation for Basic Research, Project No. 19-08-
00261.

No
REFERENCES
Нет Мерцание
Flickering
подтверждено?
confirmed? [1] O.E. Bandeev, “Classification of flame detection methods by video
sequences,” Actual problems of aviation and astronautics, no. 13, 2017,
ДаYes pp. 91-93.
[2] HSL. (n.d.). Retrieved from https://ru.wikipedia.org/wiki/HSL.html
[3] HSV (color model). (n.d.). Retrieved from
Включить тревогу,
Activate alarm, https://ru.wikipedia.org/wiki/HSV_(цветовая_модель).html
выдать сообщение
send message to
оператору [4] G.A. Polte, and A.P. Saenko, “Quantitative evaluation of image quality
operator
using fuzzy logic methods. Instrumentation,” vol. 54, no. 3, pp. 32-36,
2011.
Fig. 1. Block diagram of algorithm implementation [5] Industrial and process television systems. (n.d.). Retrieved from
http://www.itmicro.ru/industrial-tv.html
[6] Z. Zivkovic, “Improved adaptive Gaussian mixture model for
background subtraction,” Proceedings of the 17th International
Conference on. IEEE, vol. 2, pp. 28-31, 2004.
[7] T. Celik, Y. Demirel, H. Ozkaramanli, and M. Uyguroglu, “Fire
detection using statistical color model in video sequences,” IEEE
International Conference on Acoustics, Speech and Signal Processing,
vol. 2, pp. 213-216, 2006.
[8] Macroscop. (n.d.). Retrieved from https://macroscop.com
[9] SecurOS video management integration platform. (n. d.). Retrieved from
https://iss.ru/products/securos-vms.html
[10] G. Bradski, Learning Open CV, O’Reilly Media, 2008, p. 576.
[11] P. Piccinini, S. Calderara, R. Cucchiara, “Reliable smoke detection
Fig. 3. Results of the software product testing system in the domains of image energy and color,” in 15ᵗʰ International
Conference on Image Processing. San Diego, California, USA, 2008, pp.
1376-1379.
[12] B. U. Toreyin, “Wavelet based real-time smoke detection in video,”
Signal Processing: Image Communication, EURASIP, Elsevier, vol. 20,
pp. 255-26, 2005.
[13] D.K. Kim, Y.-F. Wang, “Smoke Detection in Video”, World Congress
on Computer Science and Information Engineering. Los Angeles, USA,
2009, pp. 759-763.
[14] B. Ugur Toreyin, Yigithan Dedeoglu, A. Enis Cetin “Contour based
smoke detection in video using wavelets”, In European Signal
Processing Conferenc. Florence, Italy, 2006, p. 5.
[15] F. Comez-Rodriuezl, “Smoke Monitoring and measurement Using
Image Processing. Application to Forest Fires”, Automatic Target
Recognation XIII, Proceedings of SPIE, vol. 5094, pp. 404-411, 2003.
[16] D. Krstinić, T. Jakovčević, D. Stipaničev, “Histogram-Based Smoke
Segmentation in Forest Fire Detection System,” Information
Technology and Control, vol.38, no. 3, pp. 237-244, 2009.
[17] Turgay Çelik, Hüseyin Özkaramanli, and Hasan Demirel, “Fire and
smoke detection without sensors: Image processing based approach”,
15th European Signal Processing Conference. Poznan, Poland, 2007, pp.
1794 – 1798.
[18] R. Yasmin, “Detection of Smoke Propagation Direction Using Color
Video Sequences,” International Journal of Soft Computing, vol. 4, no.
1, pp.45-48, 2009.
[19] M.M. Kaziev, E.V. Zubkova, and V.I. Bezborodov, “Features of the fire
protection transparent constructions with water irrigation.
Pozharovzryvobezopasnost,” Fire and Explosion Safety, vol. 25, no. 8,
pp. 27–33, 2016. [DOI: 10.18322/PVB.2016.25.08.27-33].
[20] M.M. Kaziev, A.M. Podgrushnyj, and A.V. Dudunov, “A pane of glass
destruction under thermal influence in the conditions of a fire,” Fire and
Emergencies: Prevention, Elimination, 2009, no. 2, pp. 5–10.
[21] A.V. Yaroslavtsev and O.V. Matukhina, “Creating the Web Service of
the Laboratory Information and Management System for Analytical
Quality Control,” Lecture Notes in Networks and Systems, vol. 115,
Industry Competitiveness: Digitalization, Management, and Integration.
Switzerland: Springer, vol. 1, pp. 643-649, 2020.