Slide 1 :

Good morning ,
Mr the president of the jury (assmo ) and its member , mr (assmo) , our supervisors (assmawathom) ,
welcome to the thesis defense which holds the theme of “Design and implementation of a
forest fire detection and localization system using UAVs” .Presented by “ Bouzidi Farah and Charef
wided “ , students of aeronautics , avionics department .
We would like to thank you for your attendance and for showing interest to our work.
we would also like to express our gratitude to our families and to our far coming friends and to
whoever attended our presentation .

Slide 2:
As a start , we are going to present our work plan which includes :

 A brief introduction on our theme.

 The state of art including most important reviews that shares our interest .
 Our methodology which is divided into three parts that are : Fire detection using computer
vision , localization using stereo vision and the design and control of a quadcopter using
PIXHAWK autopilot explained sequentially
 A conclusion that summaries our research with a hint to a future work propositions.

Slide 3 hata 8:
Over the last decades , environmental changes are large , frequent natural disasters has caused a
great impact. forest fires are one of them , to a certain point , affect people’s living , as happened in
Algeria for many years .
Forest fires , are highly complex ,not structured environments where the use of multiple sources of
information at different locations is essential .Wildfires represents an appropriate scenario for the
UAV capabilities and performance in fire detection , localization and prevention .
Due to that , it Is necessary to provide a UAV based forest fire detection and localization system ,
Based on : computer vision object detection algorithms using CNN network with aerial images,
localization using stereo-vision , a designed and controlled PIXHAWK quad copter.
The given solutions to the dilemma we introduced are minor portion of bigger world .

Slide 9 hata 11:

We will first cover briefly some of the latest papers that talked about fire detection methods and
localization using UAVs
To begin with :

1. DeepFire: A Novel Dataset and Deep Transfer Learning Benchmark for Forest Fire Detection,
By Ali Khan, April 2022.they proposed a robust forest fire detection system that requires precise and
efficient classification of forest fire imagery against no-fire. They had created a novel dataset
(DeepFire) containing diversified real-world forest imagery with and without fire , a training
based on transfer learning to achieve improved prediction accuracy was made with a comparison of
the performance of several machine learning approaches.
After that
1. FOREST FIRE DETECTION USING DRONE , Dr. S. Mary Praveena 2021.
Summary : proposed to develop a real-time forest fire monitoring system employing a UAV and
remote sensing technique. The drone is provided with sensors, a mini processor and camera. Data
from different sensors and therefore the images taken by the camera are processed on-board.

2. A Vision-Based Detection and Spatial Localization Scheme for Forest Fire Inspection
from UAV,  by a group of researchers, februrary 2022 .this paper presented a lightweight model,
NanoDet, which was applied as a detector to identify and locate fires in the vision field. After
capturing 2D images with fires from the detector, the binocular stereo vision is applied to calculate
the depth map, where some algorithms were proposed to eliminate the interference values when
calculating the depth of the fire area.

3. Design, Modelling, Localization, and Control for Fire-Fighting Aerial Vehicles, 2022 ,In this paper,
the overall dynamic model of the system is developed . The aerial vehicle is designed with vertical
and lateral rotors. A model-based controller is developed to guarantee stable flight of the aerial
vehicle. A localization method is proposed, where the aerial vehicle’s relative position with respect to
its base is computed .

Slide 12 -13
The next point we like to address is our methodology that is divided into 3 major titles , fire detection
using CV as the first key component, localization using stereo vision as the second and finally
designing and control of a quadcopter using pixhawk autopilot.

Slide 14
Traditional forest fire recognition algorithm depends heavily on man-made flame characteristics,
such as color, shape and motion ….However, there are many missed and false detection
cases due to the interference because of complex and changeable forest scenes. In recent years,
convolutional neural networks (CNN) have been widely used in the image field .Unlike traditional
methods, CNN can automatically learn features through convolution operation with more robust
performance and higher recognition accuracy, and were used widely in forest fire detection. This part
is divided into :
 Data selection and preprocessing
 CNN architecture including transfer learning.

Slide 15
Data selection and preprocessing :
Data selection :
A large amount of forest fire aerial videos and images are available on the internet , we have chosen
the most close environment to our local Terrain .So , we extracted images from the FLAME dataset of
a 2906 images containing flames to be labeled .

Slide 16
Data preprocessing:
the process in this part was divided mainly into 3 subsets :
slide 17
1. Image labeling : Manual labeling by creating bounding boxes over the Area that represents
fire from the upper left corner , which is the region of interest , the labeling was made using
MATLAB APP image labeler

Slide 18
2. Data splitting : the algorithm uses random splitting to divide the dataset into 2 subset which
includes the TRAINING DATA (80%of labeled data) which is used for network learning , and a
TEST DATA( the rest 20%) to extract the accuracy of the used method when the training
ends.

Slide 19-22

3. Data augmentation : this step is very important due to the necessity of big data for in deep
learning use. Augmentation is applied to enlarge the data-set. The used methods are :
 Jitter image color : changing in HUE , Contrast and saturation .
 Random horizontal flipping : the idea of chosen the random horizontal flip came
since the fire is generally burning upwards
 Random X/Y scaling: The image is scaled outward and inward along X/Y respecttively.
An object in new image can be smaller or bigger than in the original image by
scaling.

Convolutianal neural network architectures:

Slide 24

YOLO improves the accuracy of detection objects by referring to the idea of residual network. And
its one-stage strategy performs excellently on detection speed. It uses pre trained network to
generate a small-scale feature map, which is 32 times down sampled from the original image. The
small-scale feature map is used to detect large objects.YOLO generates a large-scale feature map by
up sampling the small-scale feature map and concatenating with a feature map from an earlier layer.
This large-scale feature map having the location information of the earlier layers and complex
features of deeper layers is used to detect small objects. YOLO uses independent sigmoid functions
to predict multilabel classification for per bounding box. That is, per bounding box could belong to
multiple categories like smoke and fire. This design is useful for detecting the regions where smoke
and fire appear simultaneously.

Faster-RCNN uses as the feature extraction network and SSD use the Inception Resnet -v2 as the
feature extraction network. YOLO v3 uses its own tiny-coco.

Slide 25

SSD is a one-stage object detection network. It predicts the object class and location by a single
forward CNN. SSD is mainly divided into three parts: (1) The basic convolutional layers consist of the
feature extraction network like ResNet50. The intermediate convolutional layer of this part generates
a large-scale feature map, which could be divided into more cells and has smaller size of receptive
fields, to detect smaller objects. (2) The additional convolutional layers connect to the last layer of
the basic convolutional network. This part of layers generates multi-scale feature maps having larger
size of receptive fields for larger object detection. (3) The prediction convolutional layers using a
small convolutional kernel predict bounding box locations and confidences for multiple categories.

Slide26

There are two stages in Faster-RCNN setting . In the first stage, feature maps of the original image are
generated by feature extraction networks And the feature map from some selected intermediate
convolutional layer is used to predict proposal regions with objectness scores and locations by
Region Proposal Network (RPN). This stage just output scores that estimate the probability of object
or not object and box regression for each proposal by a twoclass softmax layer. In the second stage,
the locations of the proposal regions are used to crop features from the same intermediate feature
map by ROI pooling. And the regional feature map for each proposal region is fed to the remainder of
the network to predict class-specific scores and refine box locations. This network achieves sharing
part of the computation by cropping proposals from the feature map generated by the same
intermediate convolutional layer in the first stage, the detection speed depends on the number of
proposal regions from the RPN.

Results discussion :

Slide27

1. confidence score:
The confidence score is predicted by the classifier and represents the probability that there is an
object in the anchor box.
For the same image , we notice that :
Yolov2 to the left : as it is showing the confidence score is between 0.56 and 0.76
Yolov4 to the right : the confidence score is between 0.55 and 0.98
Faster RCNN in the middle : express a good confidence score over 0.7 but fails in multiple object
detection.
SSd :we had a problem of overlapping and low confidence rate that went under 0.5 .

with the evolution of the prediction confidence score .Higher confidence rates tend to have higher
precision in their predictions, but a lower recall rate.
Slide 28
2. Average precision :

In object classification, the average accuracy (AP) is the most commonly used metric. The AP
measures how accurate the model is on a specific label (class). For all models, we want to calculate
the AP for the label ”Fire”, which is based on the precision-recall curve.
Average precision for yolov4: 0.86
Average precision for yolov2: 0.85
Average precision for Faster RCNN: 0.66
Average precision for SSD: 0.42
Using all the predictions for detecting fire in images, a smooth precision × recall curve was built.
both yolo models, had near a 100% recall rate, but in this stage, the precision was near 85%. The best
performing model was the one with the highest Area Under the Curve (AUC) . Therefore, from
Figures, YOLOv4 Tiny and YOLOv2 had similar results and were the best performing models.
However, the low precision at higher recall rates mean that the models have much prediction noise
and many false positives (knowing that SSD ResNet50 has the worst results). Therefore, while
considering all the model predictions, using the confidence score , the AP as a balanced metric
between the recall and the precision, and to the training time added , YOLOv4 was the best
performing model.

Chapter 03:
Slide 33-34 :
Now that we have discussed how to detect the fire, let’s talk about the Localization using stereo
vision.

This diagram that we see present the important steps performed

 Starting with the camera calibration process that should be performed carefully to extract
the camera parameters.
 Going to The pair images collected from stereo system created by two identical USB
cameras , will be processed by undistortion algorithm identified by MATLAB
 An object detection algorithm developed in the previous part was used inorder to identify
“fire “
 After defining the bounding boxes in each image , the coordinates of each center of the
boundingboxes are calculated and extracted in order to be used with the triangulation
formula that computesdepth
 finnaly we obtain an calculated coordinates of the object relatively to the camera.

Slide 35:
Before getting to the steps lets define what is stereo system,
As the figure shows, the simpliest model is that of two identical cameras separated only in the x
direction by a baseline distance b And only One element of the scene is seen by the cameras at
different positions in the image plane
And at least two images are taken with an offset from the baseline, the object whose depth we want
to know will have moved in the different images and assuming that we have calibrated cameras and
with restoring the image coordinates to the world coordinates it give us a clue to the depth.

Slide 36:
turning to the calibration process, is based on a MATLAB applicationand a chessboard as an input for
camera calibration
From the simulation results, camera parameters including focal length, principal point, radial
distortion, mean projection error and intrinsic parameters matrix are calculated.

Slide 38:
Second step is depth extraction:
Image undisortion In order eliminate the deformation of the image due to the lenses which
would make the depth estimation more accurate.
Computing the center coordinates (X,Y) of each bounding boxes.
we used the trained network mentioned before in Chapter 2 to perform fire detection
Using the “triangulate” MATLAB function to compute distance and to display the values as follow:

Slide 41-42:

As the picture shows, the result of depth is 0.56 meters, the measured real distance was 0.65 meters,
the error was of 9 cm which is considered as an acceptable result due to the camera resolution,
camera calibration error and camera orientation.

Slide 43:

As a summary, to get the coordinates we need the camera calibration parameters,

CHAPTER 04

Slide 45:
Finaly , we present in this part the design and control of a quadcopter using pixhawk
autopilot,

Slide 46:
Any project involving an autonomous vehicle will typically include the following parts:

Slide 47:
Hardware: a set of sensors, controllers, and output devices that enable the drone to perceive
its environment and take appropriate action in light of the circumstances. It essentially
consists of every part that makes up the drone physically.
Slide 49:
The main part of the quadcopter is the pixhawk autopilot that is widely used , it is integrated
with internal multiple sensors which is practical and avoid excessive weight and extra wires
on the drone, the pixhawk is compatible with its own software QGC and with MATLAB with
the need to extra configuration.

Slide 50:
Software: This is the controller's user interface, also known as the Ground Control Station
(GCS) .we used MATLAB for our academic research , based on PX4 UAV toolbox from matlab
we were able to deploy a personal firmware on the pixhawk 2.4.8 autopilot for the control
aim.
Firmware: the main goal was to design a mostly stable quadcopter using an attitude
controller model from the matlab toolbox .The model mainly implements a Proportional-
Integral-Derivative (PID) controller to control the attitude of an X type quadrirotor aerial
vehicle. At each time step, the algorithm adjusts rotational speeds of different rotors from
current angles to track the stability state , based on input from the radio control transmitter.
The input from the throttle stick is the base value of the rotational speed of all the rotors.
According to the angular rate demands, a small fraction is added to or subtracted from the
base value. The To Actuator subsystem in the model contains a Mixer matrix block, and the
structure of the mixer matrix depends on the arrangement of motors with respect to the
frame . The PX4 PWM Output block takes the individual rotational speed values and sends it
to a particular actuator , which leads to the change in speed when moving the throttle stick.
A PID tuning procedures were made in order to control the quadcopter to an almost stable
state for implementing the fire detection system of our research.
Slide 51 52 53:
The following slides represent some tests indoor and outdoor after completing the hardware
building and software implementation.