Deep Learning in Weather

Forecasting
CW 2
Declaration

I hereby declare that this particular project was completed on my own and that this project
is my original work. Furthermore, I hereby declare that this report has not been published
or submitted previously for any academic programme.

R.M.C.S. Rathnayake

Page 1 of 23
Acknowledgement

I would like to sincerely express my gratitude to my module lecturer who carefully and
proficiently supported me to complete this project in an effective manner.

Next, it should be mentioned that the successful and timely completion of this analytical
research would not have been a possible if not for the encouragement and guidance I
received from all the members of the Board of Directors, the institute’s Management,
and Academic Faculty of the Cardiff Metropolitan University of the United Kingdom
and ICBT Campus of Colombo, Sri Lanka.

Finally, I acknowledge the immensely valuable support provided by my family members

and colleagues as their constant support has been beneficially impactful in completing
this project successfully.

Page 2 of 23
Table of Contents

List of Figures ............................................................................................................... 4

List of Tables ................................................................................................................ 5

Abstract ............................................................................................................................ 6

1. Overview and Significance of Weather Forecasting .................................................... 7

2. Critical Analysis of Deep Learning .............................................................................. 8

2.1. Definition, Evolution, and Review of Deep Learning ........................................... 8

2.2. Justifying the Value of Deep Learning .................................................................. 8

2.3. Standard Mode of Deep Learning to Forecast Weather ........................................ 9

3. Evaluation of Techniques and Architectures in Weather Forecasting ....................... 10

3.1. Variables and Inputs for Deep Learning .............................................................. 10

3.2. Deep Hybrid Model for Weather Forecasting ..................................................... 11

3.2.1. Overview and Structure of Variables and Dataset ........................................ 11

3.2.2. Algorithms and Core Mechanism ................................................................. 11

3.2.3. Result Accuracy and Efficiency .................................................................... 12

3.2.4. Benefits, Drawbacks and Overall Success of DHM ..................................... 13

3.3. Multilayer Perception for Weather Forecasting................................................... 14

3.3.1. Overview and Structure of Variables and Dataset ........................................ 14

3.3.2. Benefits, Drawbacks and Overall Success of MLP ....................................... 16

3.4. RNN for Weather Forecasting ............................................................................. 17

3.4.1. Structure of the Mechanism, Variables and Dataset ..................................... 17

3.4.2. Benefits, Drawbacks and Overall Success of RNN ...................................... 18

3.5. Comparative Analysis of Various Architectures ................................................. 19

4. Research Conclusion .................................................................................................. 20

5. Research References .................................................................................................. 21

Page 3 of 23
List of Figures

Figure 1 - Standard mechanism of deep learning in weather forecasting ....................... 9

Figure 2 - Wind data is calculated as a two-dimensional vector in deep learning ......... 11

Figure 3 - Algorithm for train the deep learning model ................................................. 12

Figure 4 - Results of accuracy for various factors and time spans ................................. 12

Figure 5 - A typical representation of an MLP structure ............................................... 14

Figure 6 - Photovoltaic system used for academic purposes ......................................... 15

Figure 7 - Diagram of how a PV cell operates ............................................................... 15

Figure 8 - Structure of a typical Recurrent Neural Network .......................................... 17

Page 4 of 23
List of Tables

Table 1 - Common variable inputs for weather forecasting ........................................... 10

Table 2 - Overall comparative analysis of the deep learning techniques ....................... 19

Page 5 of 23
Abstract

Deep learning is at the helm of the modernistic technologies that drive innovation and
even the traditional sector of weather forecasting has witnessed this innovation. This
research-driven report attempts to scrutinise and evaluate multiple approaches of deep
learning associated with accurate, data-backed, and constantly enhancing weather
forecasting. Three reputed and published research papers in conjunction with other
supportive researches were utilised for this report.

While various deep learning mechanisms were evaluated, it is evident from the findings
of this particular research that certain mechanisms yield a more accurate end-result and
more importantly, a more practical approach that is useful in real world scenarios. In
addition, it was uncovered that swiftly-moving technology will subsequently lead to
improved, less error-prone, faster, and more effective weather prediction, weather
analysis, and data-driven results.

Page 6 of 23
1. Overview and Significance of Weather Forecasting

As explained by the journal research of Cao, Ewing, and Thompson (2012), a plethora
of businesses, organisations, and individuals rely on accurate weather forecasting for
their daily activities as well as operations that contribute significantly to the national
economy. For instance, the electric power industry, including solar power stations and
wind turbine plants, benefit from the application of deep learning techniques that utilise
weather data to forecast short-term weather behaviours. Biswas et. al. (2014) have further
explained that weather forecasting acts as a warning to the general public while being
important for agriculture and commodity market trading.

The relatively older research on weather prediction techniques by Cano et. al. (2002) is
valuable as it provides a wide array of insights that discuss both the benefits and
drawbacks of using deep learning models. Their research has highlighted the leading
methods of weather forecasting utilised since the late 20th century:

1. Numerical Atmospheric Circulation Models (ACMs)

2. Analog techniques
3. Neural network techniques
4. Bayesian networks for probabilistic weather prediction.

The insightful journal research formulated by Biswas et. al. (2014) has identified the two
possible modes of predicting the weather conditions:

1. Empirical approach - Based on the findings of comparable cases, i.e., similar

locations and time points, this approach is useful for predicting weather
conditions if historical data is abundant and reliable.

2. Dynamical approach – This mode depends on the equations of the atmosphere

and involves computer modelling, which in turn makes it useful only for
predicting large-scale weather phenomena, including the general wind
direction over a large region.

Page 7 of 23
2. Critical Analysis of Deep Learning

2.1. Definition, Evolution, and Review of Deep Learning

The origin of the term “deep learning” was defined decades ago, in 1986 when Rina
Dechter, professor of computer science, introduced it to the machine learning
community. The research book authored by Schmidhuber (2015) have distinguished the
concept of “deep learning” by comparing it with its opposite concept – “shallow
learning”. According to Schmidhuber (2015), deep learning has a higher depth in the
involved credit assignment paths, with are potentially causal connections between the
data inputs (or actions) and outputs (or effects) and from which the model can learn.

The research paper formulated by Greenspan et. al. (2016) has defined deep learning as
an improvement on artificial neural networks and as models that comprise a higher
number of layers that allow a higher level of abstraction. This, results in improved
predictions from data. Greenspan et. al. (2016) have further explained that deep learning
is one of the most prominent machine-learning tools in many applicable domains.

The popularity and applicability of deep learning rose to elevated heights in around 2010
and 2012, where deep artificial neural networks were able to win multiple contests in
pattern recognition as well as machine learning (Schmidhuber, 2015).

2.2. Justifying the Value of Deep Learning

While it is evident that using computing power and automation paves the way for lower
human oversigts, faster data processing, and other benefits, the specific advantages of
applying deep learning for weather forecasting should be examined in order to arrive at
a cohesive and accurate understanding of its value.

1. As highlighted by Biswas et. al. (2014), deep learning trains or learns from data
samples or examples and gathers the existing subtle functional relationships
between the data. This is potentially true even in cases where the underlying
relationships are intricately complex and unknown.

Page 8 of 23
 2. Deep learning, as pointed out by Biswas et. al. (2014), is used to generalise
information. This includes presenting data to a sample after the learning process
and reliably infer the unseen portion of a population. It should also be noted that
this can be applied in cases where the sample data may contain noisy
information.

3. Another use of deep learning for prediction purposes is that it is able to operate
as a functional approximator to arrive at largely accurate prediction results.

2.3. Standard Mode of Deep Learning to Forecast Weather

The standard approach of how deep learning is applied to weather forecasting is

presented in the subsequent diagram. As clearly outlined, multiple processes have to be
carried out on data to enhance accuracy. The training operations play a pivotal role in
deep learning mechanisms for predicting the weather.

Figure 1 - Standard mechanism of deep learning

in weather forecasting

Page 9 of 23
3. Evaluation of Techniques and Architectures in Weather
Forecasting

3.1. Variables and Inputs for Deep Learning

As explained by the journal paper of Biswas et. al. (2014), any deep or machine learning
mechanism requires scientific processing and a numerical modelling of meteorological
data for weather forecasting.

The investigative research of Ganshin (2019) has affirmed that atmospheric data is the
only primary input for deep learning systems of weather forecasting. The atmospheric
data includes multiple variables that have been used in weather forecasting for decades,
and the chief variables are listed subsequently.

Table 1 - Common variable inputs for weather forecasting

Factor Unit of Measurement

Atmospheric temperature degrees Celsius (°C)

Surface temperature degrees Celsius (°C)

Miles per hour (MPH)

Wind speed
(or nautical miles per hour)

Direction of wind Standard direction

Humidity Grams per cubic meter (g/m3)

Precipitation Millimetre (mm)

Atmospheric pressure Pascals (Pa)

Type and magnitude of cloud cover Category and percentage of cloud cover

Page 10 of 23
3.2. Deep Hybrid Model for Weather Forecasting

The published paper by Grover, Kapoor, and Horvitz (2015), who are researchers at
Microsoft, has successfully defined and analysed the deep learning “Deep Hybrid
Model” for forecasting weather in an efficient manner.

3.2.1. Overview and Structure of Variables and Dataset

The dataset utilised for this model is the dataset created by weather balloons located in
different regions across the world. These weather balloons observe, measure, and
transmit data on atmospheric conditions and other variables, including spatiotemporal
and geopotential factors (e.g. the geopotential height).

The data in this model is eventually integrated in to a large dataset that includes both
current or updated information as well as historical data. The variables are represented
in compatible datatypes. For example, variables related to the wind are represented as a
two-dimensional vector and the atmospheric temperature is a scalar.

Figure 2 - Wind data is calculated as a two-dimensional vector in deep learning

3.2.2. Algorithms and Core Mechanism

• Temporal mining
Identify and learn from recurring weather patterns over time in order to be more
accurate and efficient in the future on a continuous basis.

• Spatial interpolation
Due to the dynamic and unpredictable nature of the climate, the influence of any
applicable atmospheric laws on the weather phenomena should be taken in to account
by the model. This will ensure that this model will be more productive than
comparative deep learning models

• Inter-variable interactions

Page 11 of 23
 All interdependencies between the weather variables, including minor entities,
should be assessed by the model in order to achieve higher accuracy levels.

The following figure depicts the pseudocode that trains the Deep Hybrid Model to predict
the weather. As evident, the training mechanism is based on the variables and locations
with a focus on historical data. This allows the model to learn in an expansive manner
and be accurate in prediction future weather patterns.

Figure 3 - Algorithm for train the deep learning model

3.2.3. Result Accuracy and Efficiency

Figure 4 - Results of accuracy for various factors and time spans

Page 12 of 23
Overall, Grover, Kapoor, and Horvitz (2015) have concluded that the Deep Hybrid
Learning Model is effective in most cases, especially in cases where the given time is
high. For example, within a 24-hour cycle, the model is more accurate with a lower error
of margin than standard prediction models and the industry-accepted baseline.

3.2.4. Benefits, Drawbacks and Overall Success of DHM

One of the most prominent advantages of the Deep Hybrid Model is that it operates and
predicts in real-time with high levels of accuracy. In addition to this, the rate of
inaccuracy or the margin error is reduced by implementing multiple cycles of quality
assurance that resolve potential inconsistencies between the variables.

When assessing the involved disadvantages, it is clearly shown by Grover, Kapoor, and
Horvitz (2015) that the Deep Hybrid Model requires an extensive duration of time to
gather, study, and use historical data in order to arrive at a reliable prediction. As the
modern world requires information in an instant and as this phenomenon heightens with
time, the Deep Hybrid Model could be unsuitable at its current state.

Page 13 of 23
3.3. Multilayer Perception for Weather Forecasting

The comprehensive research propounded by Isa et. al. (2010) has analysed the use of
“multilayer perceptron (MLP) network activation functions” for weather automation and
classification purposes.

3.3.1. Overview and Structure of Variables and Dataset

The structure of the deep learning technique is based on the concepts of the multilayer
perceptron network approach, which is comprised of numerous neurons and layers. The
research of Isa et. al. (2010) has specifically focussed on the following two MLP
functions:

1. Hyperbolic tangent
2. Sigmoid.

Figure 5 - A typical representation of an MLP structure

It is interesting to note that the solutions discussed by Isa et. al. (2010) are able to
integrate a higher number of variables, especially variables that are complex in nature.
For example, the following data inputs are utilised in the MLP technique:

1. Solar Radiation
2. Ambient Temperature Current

Page 14 of 23
 3. Surface Temperature
4. Voltage.

The MLP system collects the necessary data automatically from a photovoltaic (PV)
mechanism. An actual PV system as well as a PV model that closely resembles this
mechanism is presented subsequently. The meteorological data collected from the PV is
then inputted to the MLP network, which then works to classify the prevailing weather
condition.

Figure 6 - Photovoltaic system used for academic purposes

Figure 7 - Diagram of how a PV cell operates

However, the study of Isa et. al. (2010) has only investigated four types of weather
classification – rain, cloudy, dry day, and storm. This is a manageable scope but limits
the applicability of the research findings.

Page 15 of 23
3.3.2. Benefits, Drawbacks and Overall Success of MLP

After assessing the deep learning techniques used for weather forecasting as evaluated
by Isa et. al. (2010), it is evident that the major involved benefit is the capability of
advanced and expansive data collection and data input. For example, the inclusion of
ambient temperature current and solar radiation level adds another dimension in weather
reporting and prediction activities.

The most significant drawback associated with this particular approach is that a more
accurate, future-proof, and reliable weather classification and forecasting outcome is
possible only if the number as well as the complexity of the layers are high. This makes
MLP-based weather forecasting complex, expensive, and challenging to maintain.

Page 16 of 23
3.4. RNN for Weather Forecasting

The research study of Cao, Ewing, and Thompson (2012) have focussed predominantly
on forecasting the wind speed while giving slight emphasis of other factors. The weather
forecasting mechanisms discussed by Cao, Ewing, and Thompson (2012) is largely based
on the RNN model.

3.4.1. Structure of the Mechanism, Variables and Dataset

The acronym “RNN” stands for “Recurrent Neural Network” and it is classified as a type
of artificial neural network where the output from the previous step is fed as an input to
the current step (Cao, Ewing, and Thompson, 2012). As opposed to other similar artificial
neural networks, RNNs operate with a “memory”, that is able to remember and use
information about the entities that have been calculated.

Four distinct types of layers are present in an RNN and the following diagram by Cao,
Ewing, and Thompson (2012) depicts these layers.

Figure 8 - Structure of a typical Recurrent Neural Network

Page 17 of 23
RNNs operate by calculating the current state, applying the activation function, and
calculating the output. Deep learning through RNN is possible by following this
operational framework and where the output is compared with the actual output in order
to arrive at the error margin. Afterwards, the error is back-propagated in to the RNN
network in order to update the weights and this allows the deep learning model to train
and improve.

3.4.2. Benefits, Drawbacks and Overall Success of RNN

The primary disadvantageous result of adopting RNNs as the deep learning model for
weather forecasting is the inability of the technique to process extensive or lengthy
sequences. Additionally, the time requirement for training an RNN model on an
extensive weather sequence is exorbitantly high due to the complex nature and the higher
margin of error.

However, if given sufficient time, RNN models will be able to train in an efficient fashion
and provide a solid result with extremely low margins of error and high reliability for
weather prediction purposes.

Page 18 of 23
3.5. Comparative Analysis of Various Architectures

The preceding report segments identified, defined, and evaluated three main types of
deep learning techniques that are utilised in weather forecasting. Each technique carries
with it common as well as unique advantages and shortcomings that affect the accuracy
of the final weather prediction results.

Listed and concisely discussed, based on numerous reputed journal papers and other
researches, is a comparative evaluation of the selected three primary deep learning
architectures as related to weather forecasting. Disadvantageous impacts are colour
coded in red while benefits are colour coded in green.

Table 2 - Overall comparative analysis of the deep learning techniques

Multilayer Recurrent Neural

Factor Deep Hybrid Model
Perception (MLP) Network (RNN)

Time required
Lengthy spans of Able to work
for effective Extensive time
time are required for without extensive
learning / required for learning
training time requirements
training

Needs to be highly
Less complicated
Complexity and complex for
Relatively easy to and comparatively
practicality of effective prediction
manage and apply convenient to
application and maintenance is
implement
advanced

Handling Able to handle

Performs well with Limited to medium-
complex advanced datasets
only limited data sized datasets and
meteorological and multivariate
inputs sequences
data inputs

Could be less
Overall margin High level of Comparatively low
accurate with
of error and accuracy despite margin of error and
slightly higher
accuracy margins of error high accuracy
margins of error

Page 19 of 23
4. Research Conclusion

Three reputed published research papers as well as other supporting researches were
assessed to arrive at a proper understanding of the related research domains, types of
deep learning mechanisms utilised, structures of the training models, algorithms,
involved meteorological variables, risks, drawbacks, as well as benefits.

This research concludes that the application of deep learning techniques to arrive at an
enhanced version of weather forecasting carries benefits as well as drawbacks. However,
this particular research identified that the benefits outweighed the involved
disadvantages. For example, despite requiring extensive spans of time to train using the
Deep Hybrid Model, the margin of error in prediction the wind speed, atmospheric
temperature, geopotential height, and other factors is lower when compared with
traditional prediction models and the baseline utilised in the weather forecasting sector.

As the most commonly encountered drawback of utilising deep learning for weather
forecasting is the extensive time durations necessary for training and the involved
complexity, it is safe to assume that with the rapidly advancing nature of modern
technology, these issues will be resolved in the near future.

However, the it is imperative that while the currently used techniques, including RNNs,
Deep Hybrid Models, and multiplayer perceptron networks, are used, other alternative
and potentially more effective solutions should be discovered. Further research on
potential alternatives could pave the way for a more competitive environment where the
most compatible, efficient, and practically applicable method is identified.

Page 20 of 23
5. Research References

1. Biswas, S.K., Sinha, N., Purkayastha, B. and Marbaniang, L., 2014. Weather
prediction by recurrent neural network dynamics. International Journal of
Intelligent Engineering Informatics, 2(2-3), pp.166-180.

2. Cano, R., Cofino, A.S., Sordo, C.M. and Gutiérrez, J., 2002. Bayesian
networks for probabilistic weather prediction.

3. Cao, Q., Ewing, B.T. and Thompson, M.A., 2012. Forecasting wind speed
with recurrent neural networks. European Journal of Operational
Research, 221(1), pp.148-154.

4. Greenspan, H., Van Ginneken, B. and Summers, R.M., 2016. Guest editorial
deep learning in medical imaging: Overview and future promise of an exciting
new technique. IEEE Transactions on Medical Imaging, 35(5), pp.1153-1159.

5. Grover, A., Kapoor, A. and Horvitz, E., 2015, August. A deep hybrid model
for weather forecasting. ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining (pp. 379-386). ACM.

6. Isa, I.S., Omar, S., Osman, M.K., and Saad, Z., 2010. Performance comparison
of different multilayer perceptron network activation functions in automated
weather classification. 2010 Fourth Asia International Conference on
Mathematical/Analytical Modelling and Computer Simulation (pp. 71-75).
IEEE.

7. Schmidhuber, J., 2015. Deep learning in neural networks: An overview.

Neural networks, 61, pp.85-117.

Page 21 of 23
Page 22 of 23
Page 23 of 23