Space Dogs

High-precision weather change detection using UVA

Integrants:

José del Carmen Peñalba

Tomas Karl Blas Strandgard Gonzalez

Pedro Emanuel Flores JImenes

Ramón Escobar

2022
 High-precision weather change detection using UVA

Project Summary

The team wants to provide key data to more accurately track and project how Earth's regions are
changing. Data from low-flying drones can help scientists measure this changing thickness more
accurately as Arctic sea ice fluctuates with the seasons.

A drone equipped with a camera would be used, in addition to using python applications with
image processing through the use of convoluted neural networks oriented to artificial vision, a
voice assistant module will be used that will notify the results, thus becoming an interface graphic
with user interaction

1.1 Preface

There are many standards with which we can work to develop artificial vision applications, and
one of them is YOLOv5. YOLO, whose acronym in English stands for "You Only Look Once"
(reference) is an open source framework used for real-time object detection.

1.2 Motivation

The clearest reasons are to try to develop not only robust data processing but also to provide data
that can provide a set of data and reports that, using the data offered by the NASA space agency,
can be projected towards other possible studies with the same set of data and reports, apply the
studies to possible consequences in agriculture and/or other economic sectors.

1.3 Objectives

You want to design an application that uses aerial data from one of five NASA campaigns (Delta-X,
OLYMPEX, ABoVE, FIREX-AQ, or NAAMES) and any additional data you want to educate the public
about an issue associated with El Earth's changing climate.

The environmental data collected by the different aerial means of land reconnaissance will be
applied to carry out both current situations as well as future projections of some environmental
repercussions of the study areas.

1.4 Structure

 The Theoretical Foundation: Where the most elementary fundamentals of artificial

intelligence are addressed until reaching the convolutional neural networks on which
YOLOv5 is based.
 YOLOv5 Dependencies: Here we will find all the Python libraries that it uses YOLOv5 to
offer the Artificial Vision service.
 Application: For the implementation of the features of the YOLOv5 software, an
application has been developed in Python. The architecture will be explained, as well as
the software engineering techniques to carry out the development, along with its
 operation step by step starting from a model trained in the proposed environment.
Application: For the implementation of the features of the YOLOv5 software, an
application has been developed in Python. The architecture will be explained, as well as
the software engineering techniques to carry out the development, along with its
operation step by step starting from a model trained in the proposed environment.
 Training, testing and validation of the model trained in the cloud and the model, in order
to contrast the benefits, strengths and weaknesses of both training environments.

2. YoloV5 Dependencies

We will address all the necessary technical aspects that the Yolov 5 software depends on. We will
start by describing the technical features

2.2 Dependencies

In this section we will describe the dependencies of YOLOv5, as well as its usefulness within the
YOLOv5 itself framework.

2.1.1 COCO Datasets

COCO (for its acronym in English, Common Objects in Context,) is a set of reference data for the

object detection. With COCO we will have access to the following functionalities:

1) Object segmentation.

2) Recognition of the context.

3) Entity segmentation using superpixel.

4) More than 300,000 images (of which 220,000 are categorized).

5) 1.5 million object instances.

6) 80 categories of objects.

7) 91 entity classifications.

8) 5 possible titles per image.

9) 250,000 people with points of interest.

In short, COCO will be the component with the data set previously trained to reduce the

object detection latency.

2.1.2 Matplotlib

This Python module contains all the necessary libraries to create static or animated displays in
Python. It will be used mainly to represent in the graphs the performance of the neural network
according to its stage.

2.1.3 Numpy
It is a widely used library that supports all the mathematical tools, especially for matrix operations,
with which our convolutional neural networks will work. You can find more information in the
reference

2.1.4 OpenCV-Python

This module contains all the libraries to work with in Python with this artificial vision software; It
will be the module in charge of generating the colored boxes with which we capture the objects in
an image or video frame with which the inference is made for the classification according to the
trained model.

2.1.5 Pillow

This cross-platform library contains all the tools for editing and manipulating images in Python. It
works as a direct dependency of OpenCV, in addition to serving as a driver to store the images
generated in the inference phase inside the internal memory of the device that we are using.

2.1.6 PyYAML

This module contains all the libraries to apply the data serialization format in a simpler way. In
fact, it will be an essential library when we use our dataset for the training phase, since it will
provide a manifest of the set of images used.

2.1.7 Scipy

This set of libraries provides a set of mathematical and algorithmic tools on which our neural
network will be based.

2.1.8 Torches

We will use this open source library for machine learning tasks. It is the core module on which the
YOLOv5 architecture is built. Replaces darknet in older versions of YOLO.

2.1.9 Torch vision

Complementing the previous module, this set of libraries will provide a reinforcement in machine
learning oriented to artificial vision.

2.1.10 Tqdm

This module is a visual aid that shows a progress bar when using loops. It will be used when the
model training phase is carried out, showing the stages, along with relevant information such as
the expected time to completion, the batch number, and the necessary internal memory used.

2.1.11 Tensor Board

Set of libraries with which we will keep track of machine learning, displaying graphs, histograms
with the weights or metrics of our neural network. In later sections of the document we will see
illustrations of the web interface used by this module, along with the results of the training phase.

2.1.12 Seaborne
This Matplotlib-based module provides a high-level interface for the presentation of information
in a more attractive way for the user.

2.1.13 Pandas

This library that is formed as an extension of NumPy is used for the analysis and manipulation of
data in Python. It is widely used for managing CSV files.

2.1.14 Top

This module will provide us with information about the number of floating point operations
(FLOPs) that our model will use during its training and during the inference phase.

2.1.15 Pycoco tools

This set of libraries serves as a logical interface for mapping the COCO data models to YAML.

3. Component Structure

The previously described dependencies have a relative relevance in terms of their use. Some are
the basis of the central operations of the entire convergence process of a convolutional neural
network, and others are graphical interfaces for obtaining and interpreting the results regarding
the parameters and statistics extracted during execution.

With all this, YOLOv5 manages to provide the Artificial Vision service

The purpose of the application is to offer an interface to the user with which to interact with the
image detection software, allowing the option of loading images from the computer and making
an inference with a certainty calculated at run time.

4. Component Architecture

It is developed in the YOLOv5 software to facilitate the integration with the YOLOv5 architecture,
and therefore, the internal communication and the Development tasks by not having to use a
middleware.

The central component of the application is made up of the Main, which requests the services of
the voice assistant, inference and graphical interface to the corresponding components. These use
their artifacts and the classes developed within these files to carry out their implementation, using
third-party modules as an application programming interface.

5. Training, Testing and Validation

We will see the procedures to train our image detector, defining a data model from a set of
images. This part is essential for image detection, since the quality of the inference is contingent
on building a sufficiently large data set. In turn, the training convergence time and the quality of
the inference will also depend on the YOLO training archetype chosen.

Summary of the features of the different models

val test val
Model Size mAP mAP mAP Velocity Parameters Flops
(Pixels) 0.5:0.95 0.5:0.95 0.5:0.95 V100(ms) (M) (floating
 point
operations
per
second)
YOLOv5s6 1280 43.3 43.3 61.9 4.3 12.7 17.4
YOLOv5m6 1280 50.5 50.5 68.7 8.4 35.9 52.4
YOLOv5l6 1280 53.4 53.4 71.1 12.3 77.2 117.7
YOLOv5x6 1280 54.4 54.4 72.0 22.4 141.8 229.9
Table : YOLOv5 Archetype Features

There are three columns with the metric “mAP” (mean Average Precision) which is nothing more
than a way to quantify the accuracy of a detector. To explain this concept, we will first look at the
concepts of False Positive, False Negative, True Positive, and True Negative.

• A True Positive is the result of a prediction in which it is determined to be valid because it

coincides with the reference from which the model starts, and, indeed, it is true.

• A False Positive on the contrary is the determination that a prediction is valid when it is not.

• A True Negative is the result of a prediction that is determined to be invalid because it does not
match the reference from which the model starts, hitting its criteria.

• A False Negative is the result of discarding a sample when it is determined that it does not fit the
model, when it really does.

Next, we will model and define Precision and Completeness:

• Precision is defined mathematically as the ratio between True Positives and the sum of true
positives and false positives. That is, it is a way of measuring how many times a correct prediction
has been made. The falser positives there are, the larger the divisor, causing the value to fluctuate
between 0 and 1.

VP
Precision= ( 1−1)
VP+VF
Precision Equation

• Recall is a stricter parameter than the previous one, which quantifies the goodness of a
prediction by also counting false negatives:

VP
Recall= (1−2)
VP+VN
Model Recall

Then, to determine the amount of mAP value during training, exhaustivity is averaged, and the
statistic is accumulated to finally assign a value between 0 and 100. If we look at the table again,
we can see that there is a correlation between speed of an archetype, its computational load and
the accuracy of the final model. For the tests that will be covered in this document we will use the
fastest version of the software, which uses the YOLOv5s6 archetype (usually abbreviated in
implementation terms as YOLOv5s).
Once the features have been addressed, we will use the YOLOv5 software on a laptop with a
Windows 10 operating system, using the Google Collaboratory cloud together with the YOLOv5
Python module.

The need to use the cloud as a means to train our model will also be highlighted, exposing the
convergence times for training on each device.

6. Cloud training (Collaboratory-Google)

We will deal with all aspects related to the Google Collaboratory cloud, a user can access
(authenticate with their Google account) the YOLOv5 project template and simply change the
input parameters to train the model.

6.1. Environment and tools

We will see two fundamental tools to be able to carry out training in the cloud: On the one hand,
the Google Colab cloud, and on the other hand, the roboflow web application to be able to
categorize and order our datasets.

6.1.1. Colab-Google Cloud

Google Collaboratory is a Notebook-based cloud service (like Jupyter) that enables Tensor
Processing Units (TPUs) and Graphics Processing Units (GPUs) to enable the development of open
source software, open standards, and computational services.

In order to use this interface parameterizing it with our dataset, we will need to use roboflow,
which will inject from a repository previously conditioned by the user, the set of images on which
to perform the training.

6.1.2. Roboflow

Roboflow is a computer vision development environment that offers a data collection and
classification service, as well as model training, using preprocessing techniques.

After uploading a dataset, we can classify each image with a graphical tool, apply transformations
to each image to extend our data model and give more variety and richness to the inference. The
more complex and elaborate the image transformation process, the longer it will take for the
model to converge. Finally, it allows us to condition the dataset to be received by a training
environment like YOLOv5, in which we will need 3 types of image directories:

• Training

• Valid

• Testing

The choice of the name of these directories is not accidental: roboflow works internally with the
same nomenclature used by YOLOv5, then these must be the names so that there is a one-to-one
correspondence when executing the relevant commands during image sampling.

6.1.3. Sample Image Collection

As we have already done in previous sections, it is important to develop a good set of image
samples to strengthen the inference of our image detection model. We will apply this procedure
to our image set, to finally export the dataset with a link to embed in our Google Colab template.

With all this, we are ready to proceed to training. The provided link contains a command to
download web content using the HTTPS protocol. Said content will be a zip with the dataset and a
preprocessing file with a YAML extension, which is what YOLOv5 needs to start the training. This
file will be named “data.yaml”.

6.1.4. Model Training

After conditioning our template by downloading all the necessary dependencies and executing the
relevant operations in the terminal, we proceed to the training of our model. With this we give
way to the training process in which the necessary stages for the convergence of the model will be
calculated, the use of hard disk space as well as the memory used.

6.1.5. Inference

Once the model is obtained, we will proceed with a test to see if it works, we specify the path to
the weights, and the path to the test images on which we want to make the inference.

References

Websites

https://science-data.larc.nasa.gov/naames-data/data2015.html

https://ciencia.nasa.gov/la-nasa-ayuda-volar-drones-en-el-artico

https://above.nasa.gov/index.html

https://asdc.larc.nasa.gov/project/NAAMES

https://power.larc.nasa.gov/data-access-viewer/