Ramrao Adik Institute of Technology

Department of Computer Engineering

TE MINI Project - Mock 1 Presentation
On
“Object Detection and Classification from Thermal Images
Using Region based Convolutional Neural Network”
By
Roll No. Name of Students:
20CE1119 Chandrangsu Biswas
20CE1056 Devansh Tank
20CE1331 Aaditya Patil
20CE1330 Aakash Patil
Guided by
Smita Bharne
Outline

 Introduction
 Literature Survey of the existing systems
 Limitations of the existing systems
 Problem statement
 Proposed Methodology/ Techniques
 References

2 TE Mini Project Mock 1 Presentation

Introduction

In computer vision, the process of scanning and searching for an object in an image
or a video is known as detection of objects. People can easily recognize and
distinguish objects present in a picture.

• Motivation – Our motivation is to build a machine learning classification system

that detects objects from thermal images captures by thermal cameras which is
better than normal RGB detection systems.

• Objective - Our objective is to build a system that performs better at night on

thermal images in place of RGB images.

3 TE Mini Project Mock 1 Presentation

 Literature Survey of the existing systems

Year Author(s) Image Approach Results and

Type Discussion
1. 2016 Zhang et RGB Deep neural network Deep neural network
al images was used. gives 3.34% error rate
as compared to a give
traditional neural
network which 6.67%.
2. 2016 Harsha RGB Alex Net, SIFT, Enhanced Combination of
and Anne images GMM and background enhanced GMM with
subtraction Alex Net DNN at layers
used for vehicle FC6 and FC7 is more
detection. accurate for detection
and classification.

4 TE Mini Project Mock 1 Presentation

 Literature Survey of the existing systems

Year Author(s) Image Type Approach Results and

Discussion
3. 2017 Bhartee et al. Thermal Utilizes heat Beneficial for the
images signatures security purpose. and
generated by the for finding and
objects. locating objects
during the time of
natural calamities. It is
useful in both day and
night vision.
Literature Survey of the existing systems

Year Author(s) Image Type Approach Results and

Discussion

4. 2018 Rodin et al Thermal GMM and CNN for sea The accuracy of
Images surface detection and 92.5% over a testing
classification of dataset
objects were used

5. 2018 Nam and Both visible For detection and The accuracy for
Nam and thermal classification GMM visible spectrum
images are was used images and thermal
used images were 92.7%
and 65.8%
respectively.

6 TE Mini Project Mock 1 Presentation

Limitations of Existing Systems

 In the work done by Zhang et al [2016](1), the used RGB image dataset which

doesn’t work as good during night.

 In the work done by Bhartee et al [2017](3), the used both RGB and thermal

images but it was for finding and locating objects during the time of natural

calamities. However, it is not optimal for on-road objects.

 In the work done by Nam and Nam [2018](5), GMM was used for detection and

classification which cannot do region-based classification or selective search.

Also, the prediction accuracy achieved was only 62.7% for thermal images.

7 TE Mini Project Mock 1 Presentation

Problem statement

Our problem statement is a comparative analysis using thermal images and visual
spectrum images. The motivation behind the experiment is to evaluate the results
and performance of deep learning algorithms on thermal images compared to visual
spectrum images. Here we will be using deep learning based faster R-CNN algorithm
on the thermal images and visible spectrum images

8 TE Mini Project Mock 1 Presentation

Proposed methodology /Techniques

First the input image is segmented into 2000 bounding boxes using Selective
Search algorithm. Selective search clusters the input pixels and bound them into
tight boxes containing probable classes of objects. Then the bounding boxes are
classified into the respective probable classes by using class-specific SVM. After
this classification the class-wise CNN layers are used to extract the features and
detect the probability of the object class.

9 TE Mini Project Mock 1 Presentation

Process Flow

10 TE Mini Project Mock 1 Presentation

Architecture Diagram

11 TE Mini Project Mock 1 Presentation

Abstract Diagram

12 TE Mini Project Mock 1 Presentation

Dataset

 FLIR Thermal Dataset (Version 1.3 August 16, 2019)

 The FLIR Thermal Starter Dataset provides an annotated thermal image and non-
annotated RGB image set for training and validation of object detection neural
networks. The dataset was acquired via a RGB and thermal camera mounted on a
vehicle. The dataset contains a total of 14,452 annotated thermal images with
10,228 images sampled from short videos and 4,224 images from a continuous
144-second video. All videos were taken on the streets and highways in Santa
Barbara, California, USA from November to May. Videos were taken under
generally clear-sky conditions at both day and night.

 The majority of 10,228 thermal images were sampled at a rate of two images per
second (native videos were 30 frames per second of video). A minority of images
that were acquired in less object-rich environments were sampled at a rate of
one image per second.

13 TE Mini Project Mock 1 Presentation

References

• Zhang, Z., C. Xu and W. Feng, 2017. Road vehicle detection and classification based on
Deep Neural Network. Proceedings of the 7th IEEE International Conference on Software
Engineering and Service Science, Aug. 26-28, IEEE Xplore Press: Beijing, China, pp: 675-
678. DOI: 10.1109/ICSESS.2016.7883158
• Harsha, S.S. and K.R. Anne, 2016. Gaussian mixture model and deep neural network
based vehicle detection and classification. Int. J. Advance. Computer Sci. Applica., 7: 17-
25. DOI: 10.14569/IJACSA.2016.070903
• Bhartee, A.K., K.M. Srivastava and T. Sharma, 2017. Object Identification using thermal
image processing. Int. J. Eng. Sci. Computing,
• Rodin, C.D., L.N. Lima, F.A. Andrade, D.B. Haddad and T.A. Johansen, 2018. Object
classification in thermal images using convolutional neural networks for search and
rescue missions with unmanned aerial systems. International Joint Conference, Neural
Networks, pp: 1-8. DOI: 10.1109/IJCNN.2018.8489465
• Nam, Y. and Y.C. Nam, 2018. Vehicle classification based on images from visible light and
thermal cameras. Eurasip J. Image Video Proc. DOI: 10.1186/s13640-018-0245-2

14 TE Mini Project Mock 1 Presentation

Thank You