Renewable Energy 196 (2022) 298e318

Contents lists available at ScienceDirect

Renewable Energy
journal homepage: www.elsevier.com/locate/renene

Machine learning in photovoltaic systems: A review

ez , Camilo Guillen , Luis Felipe Giraldo *,
Jorge Felipe Gaviria , Gabriel Narva
Michael Bressan

D.C., 111711, Colombia
Department of Electric and Electronic Engineering, Universidad de Los Andes, Bogota

a r t i c l e i n f o a b s t r a c t

Article history: This paper presents a review of up-to-date Machine Learning (ML) techniques applied to photovoltaic
Received 27 December 2021 (PV) systems, with a special focus on deep learning. It examines the use of ML applied to control,
Received in revised form islanding detection, management, fault detection and diagnosis, forecasting irradiance and power gen-
16 June 2022
eration, sizing, and site adaptation in PV systems. The contribution of this work is three fold: first, we
Accepted 21 June 2022
Available online 1 July 2022
review more than 100 research articles, most of them from the last five years, that applied state-of-the-
art ML techniques in PV systems; second, we review resources where researchers can find open data-
sets, source code, and simulation environments that can be used to test ML algorithms; third, we pro-
Keywords:
Machine learning
vide a case study for each of one of the topics with open-source code and data to facilitate researchers
Deep learning interested in learning about these topics to introduce themselves to implementations of up-to-date ML
Photovoltaic systems techniques applied to PV systems. Also, we provide some directions, insights, and possibilities for future
Neural networks development.
Reinforcement learning © 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND
Review license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
2. Techniques in machine learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
3. ML in control within PV systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
3.1. ML for maximum power point tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
3.2. ML for voltage and frequency control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
3.2.1. Centralized voltage and frequency control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
3.2.2. Decentralized voltage and frequency control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
4. Machine learning for management with PV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
5. ML for faults in PV systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
5.1. ML for fault detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
5.2. ML for PV diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
5.3. ML for islanding detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
6. ML for irradiance forecasting and PV output power estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
7. ML in sizing methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
8. ML in site-adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9. Open resources for ML research in PV systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.1. Simulation environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.2. Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
10. Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
10.1. Case study: maximum power point tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
10.2. Case study: islanding detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

* Corresponding author.
E-mail addresses: jf.gaviria@uniandes.edu.co (J.F. Gaviria), ge.narvaez@uniandes.
edu.co (G. Narva
ez), c.guillen@uniandes.edu.co (C. Guillen), lf.giraldo404@
uniandes.edu.co (L.F. Giraldo), m.bressan@uniandes.edu.co (M. Bressan).

https://doi.org/10.1016/j.renene.2022.06.105
0960-1481/© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

10.3. Case study: management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

10.4. Case study: fault detection and PV diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
10.5. Case study: irradiance forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
10.6. Case study: sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
10.7. Case study: site adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
11. Conclusions and future research directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
CRediT authorship contribution statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Declaration of competing interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

Nomenclature LSTM Long Short-Term Memory

MAE Mean Average Error
R2 Coefficient of Determination MPPT Maximum Power Point Tracking
ANN Artificial Neural Networks PCC Point of Common Coupling
CNN Convolutional Neural Networks PV Photovoltaic
DDPG Deep Deterministic Policy Gradient RF Random Forest
DG Distributed Generation RL Reinforcement Learning
DL Machine Learning RMSE Root Mean Square Error
DQL Deep Q-Learning RNN Recurrent Neural Networks
DQN Deep Q-Networks SAC Soft-Actor-Critic
EMS Energy Management System SVM Support Vector Machines
ESS Energy Storage System TD3 Twin-delayed Deep Deterministic Policy Gradient
GRUs Gated Recurrent Units

1. Introduction Fig. 1: control, management, faults in PV systems, irradiance fore-

casting, sizing, and site adaptation. We consider that these cate-
Among the renewable energy sources, solar generation is gories encompass most of the problems in PV systems where ML
perhaps one of the most widely used. For example, it currently have been applied to. Challenges involving cybersecurity in PV
corresponds to produce 11% of the total renewable generation in systems was not covered since it was thoroughly reviewed by
2017 in the US, and it is expected to increase to 48% by 2050 [9]. Kurukuru et al. [67]. Also, we present case studies for each of the
Moreover, the global solar photovoltaic (PV) capacity is estimated described challenges, including the source code and data that can
to increase from 593.9 GW in 2019 to 1582.9 GW in 2030, following be found in an open repository. Each portion of the open-source
capacity additions by China, India, Germany, the US, and Japan [37]. code is based, to some extent, on one representative reported
However, implementing PV systems still implies high costs and research work that proposes a solution to a challenge. Our goal is to
efficiency issues that need to be resolved. Efforts are still being allow researchers in the field to not only understand the problem
made to decrease the costs of implementing PV systems while and state-of-the-art solutions that have been proposed, but also to
increasing their efficiency, easing their implementation and quickly introduce themselves to implementations with real-world
coupling to electric grids. and simulated data. We also overview some of the most impor-
To address those challenges, machine learning (ML) algorithms tant resources that provide open datasets, source code, and simu-
have arisen as an alternative to traditional methods to provide lation environments to test ML algorithms in PV systems.
solutions that improve the performance of PV systems through the Very relevant and interesting reviews on applications of artifi-
integration of different techniques and modeling of complex dy- cial intelligence on energy systems have been done, such as the
namics. In particular, the advances of deep learning (DL) in PV ones proposed by Youssef et al. [125], Tina et al. [110], Kurukuru
systems during the last five years ago has allowed research in ML et al. [67], and Forootan et al. [43]. However, they either review a
and PV systems to move at a rapid pace since then, now involving wider range of topics in energy systems, or do not focus on state-of-
more powerful models to analyze structured information ranging the-art DL models in PV systems, or do not provide open source-
from images and video to multidimensional time series. This makes code and data. Our contribution in this paper is to complement
it imperative to review new methodologies that address problems previous review papers and to present a review of research work
in PV systems using state-of-the-art models in the field of ML. that have addressed different challenges in PV systems using
In this paper, we present a review of research work that has modern techniques in ML, in particular DL-based models, along
been reported mostly during the last five years in this topic, with a with open-source and data, to evaluate such models in the context
particular focus on applications of DL. For the literature review step, of PV systems. Due to the nature of the research topic, having a
the engines used to search the articles were: Scopus/ScienceDirect, review of the current advances along with open-source code and
IEEEXplore, ResearchGate, and GoogleScholar with keywords data is key to fuel the generation of new ideas for technology
related to machine learning, artificial intelligence, and photovoltaic development in the area.
systems. We focus on six important challenges in the imple- This paper is organized as follows: We first provide a brief
mentation and management of PV systems, as it is illustrated in introduction of the main ML techniques being applied in PV

299

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
Fig. 1. Problem categories in PV systems in which ML has been applied to.
Fig. 2. Some of representative techniques for supervised learning and reinforcement learning applied to PV systems in the research work reviewed.
systems (Section 2). Then, we review recently reported work in ML
applied to PV systems according to Fig. 1 (Sections 3 to 8). After-
ward, we provide a summary of some of the main open resources,
such as source code and data, to work in ML applied to PV systems,
(Section 9). We then present each of the study cases developed
(Section 10). We end this paper with some conclusions and a dis-
cussion about the possibilities for future development in the field of
ML in PV systems (Section 11).
2. Techniques in machine learning
In this section we introduce some of the most representative
techniques of ML that have been applied to PV systems with a
special focus on DL algorithms. The categorization of these tech-
niques is shown in Fig. 2.
Artificial Neural Networks (ANN) are models that are inspired in
the fact that neurons, as processing units, are interconnected in the
human brain [50]. A ANN defines a function that maps a n-
dimensional input to a r-dimensional output, and that it is con-
structed as the parallel and serial interconnection of basic pro-
cessing units, called neurons. Each neuron computes a weighted
sum of its own inputs that is then transformed using an activation
function. The learning process refers to the process of finding the
weights for each neuron in a way that the ANN model is able to
accurately predict the output for a given input [53].
300
Renewable Energy 196 (2022) 298e318
Reinforcement Learning (RL) is a technique that enables an
agent to learn control policies by trial and error using rewards
provided by an interacting environment, which is described as a
Markov Decision Process [28]. Essential elements of an RL problem
include the state (the current situation of the agent), the reward
(the feedback provided by the environment), and the policy (the
method that defines the relationship between the agent's states
and its actions). The primary goal of the algorithm must be to find a
set of actions that maximizes the reward received by interactions of
the agent with the environment. Some of the currently used
techniques to conduct the learning process in RL are Q-learning,
SARSA, and their variants [106]. Some approaches divide the
continuous states and action-space into a finite number of regions,
known as Q-table. Another family of methods use DL models where
ANNs are used to approximate a policy function. This allows for
continuous state and action-spaces [41], where the Q-table is
replaced by a nonlinear continuous function modeled by an ANN
[1]. Some of these approaches include Soft-Actor Critic (SAC), Deep
Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3),
and Deep Q-Learning (DQL).
Convolutional Neural Networks (CNN) are a type of ANN that
extract important features of a 1D, 2D or 3D array through filters of
a certain length in a hierarchical manner.
Recurrent Neural Networks (RNN) are a class of networks that
can work on data sequences of arbitrary length to predict future

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
outcomes or to classify events. This is done by inferring relation-
ships between the frame through its connections that point back-
wards [47]. Each neuron in an RNN layer receives an element of the
analyzed frame as input as well as the output of the previous
prediction (hidden state) that was based on the last analyzed
element of the frame. This structure implies that RNNs allow pre-
vious outputs to be used as inputs, while having hidden units to
take into account historical information through hidden states [16].
Gated Recurrent Units (GRU) aim to remember past information
for long periods of time [63]. This is done by assigning a set of
parameters in between edges of adjacent hidden units in RNNs.
This parameter would act as a gate, regulating the amount of in-
formation that the network can remember over time. If the value of
this parameter is close to one, more memory is retained, if it is close
to zero, the memory of previous steps would be forgotten.
Long Short-Term Memory (LSTM) is a special kind of RNN that is
capable of learning long-term dependencies [84]. It remembers
information for long periods of time due to a memory cell that
stores previous information, called cell state, which is updated
through 2 different parameters that are similar to the one find in
GRUs. These parameters allow previous information to be
remembered or forgotten, and new information to be added to a
certain degree. The cell state is then used to update the hidden state
of the common RNN by using another parameter that lets certain
parts of its information flow through the hidden unit. These pa-
rameters are the output of gates (forget gate, input gate and output
gate) that are simple ANNs whose input is the previous hidden
state.
Support Vector Machines (SVM) is a technique that finds a linear
model for classification [47]. The criterion of the SVM is based on
the concept of maximum margin, which refers to the distance from
the separating hyperplane to the closest observations in either
class.
Random Forest (RF) is a model based on decision trees that are
constructed in parallel using random subsets of input variables.
Once the decision trees are trained, a RF model takes an example,
passes it through all trees, and assigns the class with the most votes
[124].
3. ML in control within PV systems
There is a need for advanced control techniques to ensure a
reliable integration of PV systems to the grid. As stated by Glavic
Fig. 3. PV array power curve under partial shading conditions.
301
Renewable Energy 196 (2022) 298e318
[48], the implementation of advanced communication infrastruc-
ture and powerful electronic devices open up the possibility to
implement advanced control techniques in the PV-based power
system. In this section, we review different control techniques
based on ML applied to the problems of maximum power point
tracking (MPPT) with charge controllers and voltage/frequency
control for PV electric systems. Since applications on smart grids
that combine PV and other renewable energies have already been
covered in specific reviews such as the one by Rangel-Martinez
et al. [95], these topics are not entirely covered in this review.
3.1. ML for maximum power point tracking
The total produced power by a PV system is influenced primarily
by two elements: the solar irradiance and the cell temperature of
the solar panel arrays. To consistently obtain the highest efficiency
in solar power production under changing conditions, the voltage
of the system is controlled to keep the system at the MPP. This
problem is known as MPPT control. Some traditional methods for
MPPT control are Perturb-and-Observe and the Incremental-
Conductance methods, which are easy to implement [36]. How-
ever, these methods usually need certain calibration levels to pre-
vent high oscillations around the MPP during its search.
Furthermore, when partial shading conditions are present, these
algorithms tend to stay at local maxima, leading to lower energy
conversions. This issue is due to the nonlinear and time-varying
characteristics of the PV array affected by changes in the atmo-
spheric and load conditions [40]. Fig. 3 shows the power-vs-voltage
curve in a system with 3 PV modules where partial shading con-
ditions are present. Note that there are several local maxima for
different values of voltage.
Recently, ML techniques have been applied to solve these
problems, outperforming traditional methods. A review presenting
intelligent techniques applied for MPPT control in photovoltaic
systems was published in 2014 by El Telbany et al. [40]. However,
the application of techniques such as RL and ANNs were not
included in the review. Here, we review recent studies that have
proposed the implementation of more ML algorithms to address
the problem of MPPT control. A case study is introduced in Section
10.1.
Messalti et al. [79] obtained data from a standard Perturb-and-
Observe algorithm to train a ANN that learns when to increase or
decrease the voltage. The input variables are the output power
derivative and the voltage derivative. The results showed a high
tracking accuracy, high response time, and low overshoot. Similar
algorithms were proposed by Arjun and Zubin [19] and Agha et al.
[12], in which they achieved higher response times than conven-
tional methods.
Another DL algorithm that has provided excellent results for
MPPT control is RL. One of the first uses of RL in MPPT was
implemented by Kofinas et al. [62]. Arguing that RL has high
convergence stability with shorter computational time than other
meta-heuristic methods, the authors proposed a Q-Learning
tabular approach to track the MPP. The state-space was defined by
the voltage, the normalized current, and an angle relating to cur-
rent and voltage changes. The action-space was defined as a list of
steps to change the duty cycle, and the reward was based on pos-
itive or negative changes in the output power. The results obtained
showed that the algorithm designed outperformed the conven-
tional Perturb-and-Observe method in three different scenarios, in
which irradiance and temperature were changed. This research
leads to other similar tabular Q-learning methods applied to MPPT
such as one made by Aurobinda et al. [20], where their main con-
tributions were comparing the obtained RL control agent to fuzzy-
logic-sliding mode control and incremental conductance-sliding

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
Table 1
Comparative table of reports of ML applied to MPPT.
Articles Programming Language
[12,19,79] Python
[21,22]
[90] Matlab
[26,56,62]
[20,34]
mode control, achieving better results with the RL algorithm. They
also conducted experiments on real testbeds with good results.
However, both of these papers did not include comparisons be-
tween different RL algorithms. This is why Bavarinos et al. [26]
implemented the Q-learning and SARSA algorithms for MPPT
control, while also comparing them. The variables used as the state-
space were the open-circuit voltage, the short circuit current, and
the MPP under standard test conditions. The action-space used was
defined as discrete changes in the voltage duty-cycle.
Kalogerakis et al. [56] proposed a tabular Q-learning algorithm
for tracking the global MPP at partial shading conditions. A Boltz-
mann exploration policy was used with a similar reward policy as
previous mentioned papers. The state-space included current duty
cycle value, the power, and the previous value of the duty cycle. The
results achieved were compared to particle a swarm optimization
algorithm and showed that the RL algorithm reduced between
80.5% and 98.3% the time required for detecting the global MPP
with respect to other algorithms based on RL.
Although previous authors achieved remarkable results, none of
them utilized DL methods applied to RL. In the work made by Phan
et al. [90], a deep Q-learning (DQL) agent and a deep deterministic
policy gradient (DDPG) agent [68] were proposed as ways of con-
ducting an MPPT control, focusing primarily on partial shading
conditions. This methodology allowed the agents to deal with
continuous state-space. The state-space was defined as the com-
bination of voltage, current, duty cycle, and current perturbation.
The action-space for the DQL agent was defined as a set of discrete
steps that changed the duty cycle while the DDPG handled
continuous action-spaces. The reward function was defined using
the changes in power from one step to another, the MPP at standard
test conditions, and a penalty if the agent actions caused the duty
cycle to step out of bounds. The results obtained by the models
were compared to the commonly used Perturb-and-Observe
method, evidencing better results in the eight validation sce-
narios. The agents tracked the global MPP in seven out of eight
partial shading scenarios. Chou et al. [34] made comparisons be-
tween Q-table and Q-network algorithms for MPPT. The RL-Q-
network MPPT achieved a higher average power than the RL-Q-
table MPPT method, while the former provided smaller oscilla-
tions. Both methods were then compared to the conventional
method Perturb-and-Observe. The RL-based methods provided
smaller ripples and faster tracking in the simulated scenarios.
However, in experimental results, the Perturb-and-Observe
method obtained a better performance than the proposed
methods.
Although Matlab provides an environment for training and
testing the RL agents while also being highly used, it is not an open-
source software that every researcher can access. This is why Avila
et al. [21,22] constructed a DL model-free algorithm based on RL,
which was trained and tested in a custom developed OpenAI Gym
environment in Python [85]. The algorithm proposed is based on
DDPG in [21]. Moreover, the inverted-gradient and the TD3 algo-
rithm were implemented to solve the problem in [22]. The
continuous state-space for the algorithm proposed in these papers
was the voltage, power, and the difference between the actual
302
Renewable Energy 196 (2022) 298e318
Experimental Results ML Techniques Used
No ANNs
Deep RL
Tabular RL
Yes Tabular & Deep RL
power and the previous power. The action-space was defined as the
disturbance applied to the controllable variable of the voltage. It is
important to note that the authors did not simulate a complete
boost converter model in OpenAI Gym for the environment. The
reward was defined as the current power divided by a normaliza-
tion factor if the change of power was positive and minus one if the
change of power was negative. An epsilon-greedy exploration
strategy with linear decay was also used. The results obtained by
the authors showed that the maximum operating power point
using the algorithm was less than 1% of the theoretical MPP.
In Table 1 we show some of the characteristics identified in the
presented papers. First, most of the papers work using Matlab as
programming environment. One of the main reasons for this is that
Matlab facilitates the integration of ML algorithms into a wide va-
riety of applications in PV systems. There is a need to develop more
benchmark scenarios in free programming languages such as Py-
thon. Also, we identify a lack of experimental evaluation of RL al-
gorithms in real test-beds. We consider that more experiments
should be conducted on real testbeds for MPPT tracking consid-
ering mainly deep RL methods. Deep RL methods such as DDPG,
SAC and TD3 have not been yet tested on real experiments, and
considerations on implementations for real-time processing can
potentially arise new research challenges.
3.2. ML for voltage and frequency control
The integration of PV in distributed networks generates voltage
swings due to the rapid power fluctuations generated by PV sys-
tems. This voltage swing generates voltage quality degradation and
voltage stability issues, which may breach established regulations
[39]. Intelligent voltage control models have been recently imple-
mented to maintain the voltage magnitudes of buses within a
desirable defined range. This regulation is made by applying the
control algorithms to the DC/AC inverter responsible for generating
the three-phase AC voltage [125]. In a review made by Guo et al.
[49], two voltage control categories are defined: centralized control
and decentralized control, which will be further analyzed in the
following sections.
3.2.1. Centralized voltage and frequency control
Centralized voltage control schemes use sophisticated
communication networks to regulate voltage using a system
operator [113,117]. It requires highly reliable communication
schemes using protocols such as DNP 3.0 or IEC 61850 [49]. This
control is conducted using only one agent or multiple agents that
can regulate multiple zones in a grid with an injection of PV energy.
Li et al. [71] used a DDPG algorithm to coordinate multiple PV
smart inverters that regulate the voltage in a PV-grid system. The
authors described in great detail the characteristics of the agent
and how it was trained. The state-space were the voltage magni-
tudes at each bus, real and reactive power generation/consumption,
and actual values of the loads. The action-space was considered as
each inverter's generated reactive power. The rewards consisted of
large penalties due to breaching voltage limits and a negative
reward proportional to the total reactive power dispatched to

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

minimize the PV production reduction. The algorithm was trained believe that there is an opportunity to evaluate the use of a full-
and tested in the IEEE 37 bus system. To simulate the power flow, scale grid using only RL algorithms for both MPPT and voltage/
the solver OpenDSS [4] was used. Compared to the benchmark frequency control in uncertain scenarios.
method of Volt-Var, the DDPG was able to utilize reactive power
more efficiently with much less curtailment incurred. The DDPG
4. Machine learning for management with PV
generated a power loss of just 4.1% of the Volt-Var method losses
that resulted from reactive power generation while also avoiding
Energy management systems (EMS) have become increasingly
over or under voltages in all nodes.
helpful for consumers to reduce their electricity bills and maintain
A similar approach was proposed by Wang et al. [120] were a
efficiency. This improvement is possible due to the insertion of
multi-agent DDPG method was tested in the Illinois 200-Bus sys-
smart grid technologies such as smart meters and demand-in-
tem with PV integration considering load generation changes, N
1
response algorithms. The main objective of an EMS is to schedule,
contingencies, and a weak centralized communication environ-
in real-time, the system's energy flows by minimizing a defined
ment. Here, 3 agents were used to control certain parts of the 200-
objective function while maintaining reliable, secure, and safe op-
Bus system, which was divided into three zones. The state-space
erations of the system according to Zia et al. [126]. This objective
was defined as system-wide bus voltage magnitudes, phase an-
function is defined in a way that a desired demand/response is
gles, generations, and power flow. The action-space was a vector of
controlled through incentives or penalties. Several studies have
desired bus voltage magnitudes. The reward function took into
been conducted on the EMS optimization formulation. In this sec-
account a motivation to reduce the deviation of bus voltage mag-
tion, we will focus on some of the ML approaches proposed to
nitudes from a reference value and a penalization when a voltage
achieve this goal, as the Fig. 4 shows. We distinguish management
breach occurs in the system.
from control since the latter refers to keeping a specific process at a
set-point, while management refers to keeping the process's set-
3.2.2. Decentralized voltage and frequency control
point at a profitable point [80]. A case study is introduced in Sec-
In this type of control scheme, several agents have their own
tion 10.3.
local control policies [61]. This scheme implies that the coordina-
One of the most used ML techniques for solving this problem is
tion between agents is done in an automated manner, locally in
RL. Lee and Choi [69] used RL to manage the optimal management
each bus where it is needed, and without a centralized operator,
for residential houses that use a rooftop solar PV system, an energy
which may optimize local grid operation [98,116]. Cao et al. [31]
storage system (ESS), and smart home appliances. The authors used
used a multi-agent deep RL-based approach for a distribution
a Q-learning table method to control the energy consumption by
system with high penetration of PV. The main idea in this work was
appliances consumption such as air conditioner or washing ma-
that the agents interacted with each other by modeling each other's
chine, and the charging and discharging events in the energy
policies during training. The state-space of the agents consisted of
storage system. Also, the indoor temperature is predicted using an
the active and reactive power of load demand where the agent was
ANN to assist the Q-learning algorithm. The algorithm focuses on
connected and the active power injection of PV. Moreover, the
reducing the electricity bill using solar PV systems and the energy
action-space of each agent was the reactive power value of the
storage system while also reducing an established dissatisfaction
corresponding PV inverter to which the agent was associated.
cost. The state-space for the proposed problem was based on the
Lastly, the reward was defined as the negative sum of the absolute
energy consumption of both the washing machine and the air
value of the voltage deviation at each bus. The validation of the
conditioners, and the state of energy of ESS batteries at a given
algorithm was done using the IEEE 33 bus system [15]. The pro-
time. The action-spaces were: 1) turning on/off the washing ma-
posed algorithm was compared to the droop control method and
chine; 2) the ten different levels in which the system can set the AC
achieved shorter voltage fluctuations while also preventing over/
energy consumption of the air conditioner; 3) the ESS nine levels
under-voltage.
A similar approach was designed by Sun and Qiu [104], where
they used a multi-agent DDPG algorithm to mitigate voltage vio-
lations with centralized training and decentralized application. The
authors determined the dispatch of the on-load tap changers and
capacitor banks day-ahead, based on the hourly PV and load pre-
dictions. Then, the agents were trained offline using this informa-
tion as their input. The state-space was obtained from on-load tap
changers and capacitors banks. The action-space was defined as the
difference of PV reactive power output between two neighboring
time sections. The agents were trained in a centralized way while
its implementation was decentralized. The algorithm was tested in
multiple scenarios. It was demonstrated that it can potentially
mitigate voltage violations and reduce power loss better than other
methods. Another decentralized method was proposed by
Takayama and Ishigame [109]. The authors implemented a
multiple-agent RL algorithm with a radial basis function to deter-
mine the reactive power output of the PV inverters. The results
were compared to an algorithm that used particle swarm optimi-
zation, achieving a better performance.
Based on the reported research, there is a need to provide a
thorough comparison between centralized and decentralized
voltage control methods based on ML where practical aspects of
implementation can be assessed such as scalability, convergence,
communication topology, and real-time processing. Moreover, we Fig. 4. Diagram of the application of ML on EMS.

303

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
that allow the system to discharge and charge the batteries. The
reward evaluates the agents performance in terms of cost, the
consumers satisfaction with the current operation, the electric cost,
and energy under-utilization. The previously mentioned ANN al-
lows the system to accurately calculate the dissatisfaction of the
consumer. The algorithm was able to minimize the electricity bill
through the energy consumption scheduling while also maintain-
ing the consumer comfort level and appliance operation
characteristics.
Multi-agent-based RL techniques with Q-tables have also been
used to address this problem as shown by Xu et al. [122]. In that
paper, the authors used a multi-agent RL with Extreme Learning
Machine for implementing a home EMS. The objective of the agent
was to reduce the energy electricity bill and the demand response
induced dissatisfaction cost. The structure of the EMS included a
refrigerator, an alarm system, air conditioner system, heating,
lighting, washing machine, dishwasher, and an electric vehicle. The
state-space corresponded to both the electricity price and the PV
output in an hour ahead slot. The action-space corresponded to the
energy consumption scheduling of each home appliance and
charging/discharging of the electric vehicle. The actions on an
appliance depended on its type, which could be non-shift-able,
power-shift-able, or time-shift-able. Furthermore, the authors
used Extreme Learning Machines for predicting the future trends of
the electricity price and PV generations and used this as part of the
state-space. The algorithm was able to reduce the electricity cost by
45% when compared to scenarios where the algorithm was not
being used.
A DRL agent was proposed in the work made by François-Lavet
et al. [44], where the management problem considered efficiently
operating storage devices in a microgrid featuring panels with
short and long-term storage capacities. The RL technique used was
the DQL algorithm. The state-space was composed of the amount of
energy in the batteries and the amount of energy in a hydrogen
tank. The action-space consisted of the amount of energy trans-
ferred in or out from the storage system. The reward function was
composed of the instantaneous operational revenues, which
considered the electric power generated by the PV system. The
results gathered showed that the ANN representation of the value
generalized the policy in an efficient manner.
Although these papers achieved remarkable results, the lack of
standardization in EMS makes it difficult to compare the different
proposed algorithms. This is why some studies have implemented
their solutions on open environments that allow researches to
share, replicate, and compare their designed models. An example
includes the work made by Kathirgamanathan et al. [58], where a
centralized Soft Actor-Critic (SAC) algorithm was implemented to
flatten and smooth the aggregated curve of the electrical demand of
a district using the CityLearn environment [119]. The proposed
multi-objective cost function consisted of the peak electricity de-
mand, the average daily electricity peak demand, ramping, the load
factor, and the net electricity consumption of the district over the
evaluation period. The state-space included hour, month, day,
direct solar radiation, non-shift-able load, solar generation, cooling
storage SOC, domestic hot water stock SOC, and the temperature
outside of the buildings. The reward function was designed to
penalize peak consumption based on the total electricity con-
sumption. Moreover, the reward considered the charging and dis-
charging of batteries during day and night.
Other approach to tackle this problematic is based on using
optimization algorithms for managing the EMS that are not mainly
based on an ML algorithm. This is done by Leonori et al. [70] where
the authors made a comparison between several ML methods was
made. Several EMS models were created using ML techniques and
compared using six different simulation scenarios. The algorithms
304
Renewable Energy 196 (2022) 298e318
used included the Mamdani-type FIS, adaptive neuro-fuzzy inter-
face system, support vector regression, echo state network, and
multi-layer perceptron models. The models were compared as well
with solutions obtained through rolling-time-horizon strategies.
Also, an approach for representing the potential microgrid energy
flows as a single point in a planar domain is proposed to assess the
microgrid operational configuration. A mathematical formulation
of the microgrid model is highly described with the formulation of
the objective function. The results obtained showed that the rolling
time horizon strategy led to the best performance followed by the
neuro-fuzzy interface system, which in one scenario outperformed
the former strategy. EMS management based on multi-objetive
predictive energy management strategies were also made by Shi-
vam et al. [101]. They used mainly CNNs to predict energy pro-
duction and electric load as one of the three levels of the control
strategy.
Based on this review, we consider that more efforts should be
made to create or improve the current scenarios where researchers
are testing their algorithms. Simulation of EMS need to be stan-
dardized in platforms such as OpenAI to make research de-
velopments comparable between each other. Although the
CityLearn environment tries to standardize the tests of different ML
algorithms implemented over an EMS, it lacks of a variety of EMS
scenarios that researchers could encounter in real experiments.
Furthermore, based on the simulated scenarios created for testing
the algorithms, experimental results should be conducted and re-
ported to test the ML algorithms in real scenarios.
5. ML for faults in PV systems
One of the main challenges that the PV industry faces is its
vulnerability to faults due to its exposure to harsh environmental
conditions [66]. Fault detection and diagnosis is essential to opti-
mize energy conversion efficiency and improving the life span of
the PV system. In this section, we identified three subcategories
were ML algorithms are used: fault detection based on online
voltage and current data, fault diagnosis over PV modules, and
islanding detection. It is worth to mention that in this review we
did not cover fault detection approaches based on ML for grid
connected PV inverters since this topic has been widely covered by
Malik et al. [73].
5.1. ML for fault detection
There are different types of faults that can be classified based on
what causes them. For example, incipient faults are low in
magnitude and last only for a few microseconds, making them very
difficult to detect [97]. Abrupt faults refer to faults that occur
instantaneously, often attributed to the results of line-to-line faults
or line-to-ground short circuits. Intermittent faults refer to faults
that are cleared over time, such as partial shading [51]. Line to line
faults occurs when an unexpected short circuit happens between
two lines in a PV array [123], whereas the line to ground happens
when there is a short circuit between a point of the array and
ground. These are types of short circuit (SC) faults in which
abnormal connections between two points of different potentials
occur [11]. Hot spot faults are caused by partial or complete shading
of a PV module due to dust, which may increase the localized
temperature in a cell and damage the module [38]. Open circuit
(OC) fault occurs when in a PV array, an accidental disconnection
happens [57]. Finally, arc faults are generated because of a loose
connection or a cable insulation failure [103]. In recent years, ML
techniques have been highly studied for fault classification and, in
some cases, to identify the specific location of the fault. In this
section, we will study some of these papers.

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
Appiah et al. [18] used LSTM networks for fault classification and
detection. The faults considered by the authors only included line-
to-line faults and hot spot faults. The data was collected using a
simulated model of 5.3 kW PV array with added noise to simulate
real-world scenarios. The classification accuracy for the data was
100% for scenarios with no faults, 100% for scenarios with hot spot
faults, and 99.03% for the line-to-line faults. When adding the noise,
the accuracy of the models was reduced to 99.23%, 98.78%, and
97.66%, respectively.
An RF classifier was used by Chen et al. [33] for the detection and
diagnosis of PV array early faults. A dataset was generated including
line-line faults, degradation, open circuit, and partial shading in
simulation. The features considered for training the model were the
operating voltage and the string of current of the PV array. The
results showed that the performance of a random forest out-
performed other methods. A 99% accuracy was obtained in the
classification and detection tasks in both the simulated and
experimental results. Pahwa et al. [86] made a comparison between
decision trees, XGBoost, RF, and ANNs. Six different faults that
include partial shading, bypass diode fault, bridging fault, temper-
ature fault, complete shading fault, and short circuit faults were
included in the dataset with a total of 1200 readings. The results
presented showed that the ANNs achieved the highest validation
performance with an accuracy of 99.91%. A similar work was pre-
sented by Haji et al. [51], where a principal component analysis was
used to reduce the number of features before classification.
CNNs have also been used to address this problem. Aziz et al.
[24] used a 2D CNN to extract features from scalograms generated
from PV system data and classify faults. Five different faulty sce-
narios were used to train the model. The data was collected with
the incorporation of an MPPT algorithm in the simulated system.
The features transformed to a scalogram were the irradiance, the
temperature, the short circuit current, open-circuit voltage, the PV
current, the MPP and boost converter current, voltage, and power.
One of the main contributions of [24] was that the different values
of irradiance considered for the fault classification implied changes
in the MPPT model to generate broader types of scenarios for the
predictions. A CNN was also used by Lu et al. [72]. In that paper, the
faulty time-series data was analyzed and then used as input fault
Table 2
ML for fault detection articles.
Articles Faults Studied Experimental Results
[18] Line-to-line, No
Hot Spots
[33] Line-to-line,
Module Degradation,
OC,
Partial Shading
[86, 51] Partial shading,
Bypass diode fault,
Temperature fault,
Complete shading,
OC
[107] SC,
Partial Shading,
Abnormal Aging,
Hybrid Faults
[24, 72] Partial Shading,
Line-to-line,
OC,
Arc fault,
Hybrid Faults,
MPPT variations
[64] Yes
305
Renewable Energy 196 (2022) 298e318
features to the CNN. The features used for training were the PV
voltage and current transformed into a 2-Dimension electrical time
series graph and used as inputs to the designed CNN.
One of the main contributions made by Krizhevsky et al. [64]
was the design and implementation of an experimental platform to
verify the proposed CNN model. The researchers obtained 1400
graphs from the experimental platform that were divided into
training and testing datasets. Different sliding windows and graph
sizes were tested to obtain the best results. The proposed algorithm
was then compared to classifiers that used Inception-V3 [107] and a
wavelet-transform using SVM. Compared to the other methods, the
proposed method achieved higher accuracy under different con-
ditions. The authors achieved a maximum accuracy of 99.51% using
the created model and a sliding window of 10 s.
Furthermore, Gao and Wai [45] proposed another fault PV array
identification technique based on CNN. Instead of using a 2D-CNN
as in previous papers, the authors used a 1D-CNN and a fully con-
nected module for classification. The faulty scenarios considered for
this research included short-circuit faults, partial shading faults,
abnormal aging faults, and hybrid faults. To verify the effectiveness
of the proposed method, the authors used the t-distributed sto-
chastic neighbor embedding [114] to visualize the models distri-
bution effect over the features extracted. The results achieved in the
testing data showed a 100% accuracy. The overall accuracy of the
model using the data gathered through the experimental platform
was 98.41%.
Table 2 compares the described articles. We highlight the lack of
both experimental results on real testbeds, and the assessment of
new ML methods such as vision transformers applied to scalo-
grams. It has been shown that vision transformers can potentially
outperform other models such as CNNs. Also, open datasets of real
experimental results should be made available across researchers
for comparison purposes.
5.2. ML for PV diagnostics
The decrease in efficiency due to abnormal operating conditions
generated because of dirt on the surface of modules, cell breakage,
delamination or hot stops raises the need for diagnostics to PV
Features Used Main ML Technique
Current, LSTM
Voltage,
Power
RF
OC voltage, ANNs
SC current,
MPP data,
Temperature,
Irradiance,
Fill factor
IeV curve, 1D-CNN
Irradiance,
Temperature
Scalogram based on: 2D CNN
- Irradiance,
- Temperature,
- SC current,
- OC voltage,
-PV, Boost & MPP current & voltage
2D CNN
Inception-V3

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
modules to be done with ML techniques by analyzing images of
panels. Cipriani et al. [35] used CNN to automatically classify
thermographic images. The aim of the model was to distinguish
between hot spot conditions from those of dust. The dataset was
constructed using different thermal camera technologies, resolu-
tion, and color scales with legacy PV systems from different char-
acteristics through the Italian territory. Several techniques were
applied for pre-processing the images, which included normaliza-
tion, homogenization of the number of pixels, grayscaling, thresh-
olding, box blur, and Sobel-Feldman filters. The results showed that
a maximum accuracy of 98% was achieved with an actuation time
between milliseconds and about 2 min. A similar approach was
performed by Rico Espinosa et al. [96] where they proposed a
method based on semantic segmentation and the use of CNN for
failure mode classification. The authors first used semantic seg-
mentation to identify the PV modules in an image, obtaining its
mask. The obtained mask and the original image were then used as
input for a classification network that used CNN. The classification
networks considered two classification scenarios: binary classifi-
cation and quaternary classification. The binary classification and
qua-ternary classification model achieved an accuracy of 75% and
70%, respectively.
Pierdicca et al. [91] implemented an anomaly cells detection
model based on thermal infrared PV images collected by aerial
vehicles. The model was based on the mask region-based CNN ar-
chitecture because of its ability to make both object detection and
instance segmentation. Furthermore, the researchers collected a
publicly available dataset, which was used to train the proposed
model. The intersection over union and the dice coefficient metrics
showed the effectiveness of the proposed models. Moradi Sizkouhi
et al. [82] proposed a deep convolutional encoder-decoder archi-
tecture for autonomous fault detection of PV plants using multi-
copters. The paper focuses especially on detection of bird's drops
over PV modules. The average accuracy reached over the testing set
was 93%. A more in depth review on ML methods applied to PV
modules diagnostics was recently reported by Berghout et al. [27].
The authors discuss several ML tools for classifications including
conventional ML and DL. They included several papers that discuss
the detection of faults using electroluminescence images and
thermographic images.
Based on the reported results in this topic, we identify an op-
portunity to collect and make datasets available in which new ML
models for PV diagnostics can be tested. Also, ML methods such as
generative adversarial networks could be applied to create gener-
ators of artificial thermal images and have a better understanding
of the phenomena being studied. Furthermore, future research can
involve the development of a pipeline for deploying the ML algo-
rithms for real time PV diagnostics using performance- and power-
constrained devices and Tiny-ML technologies [121].
5.3. ML for islanding detection
Islanding is the process through which an on-grid distributed
generation (DG) system is disconnected from the grid while the DG
system continues to power the load. When unplanned islanding
events occur, due to line tripping or equipment errors, it may
generate power quality problems on account of voltage/frequency
instability and even damage electrical instruments. For this reason,
detecting unplanned islanding conditions is critical for DG reliance.
According to standards such as the IEEE Std. 1537e2018 [8] and IEC
62116:2014 [7], an unplanned islanding detection condition must
be detected and stopped within 2 s since the formation of the event.
As described by Manikonda and Gaonkar [78], there are six types of
islanding detection techniques: passive, active, hybrid, remote,
signal processing methods, and intelligent methods. Passive
306
Renewable Energy 196 (2022) 298e318
methods tend to fail due to the need to set up a threshold value that
may omit certain islanding scenarios. Active methods need to inject
disturbances into the grid to detect the event that generates power
quality issues in the system. Intelligent methods eliminate the need
to use a threshold value and to inject disturbances into the system
while also presenting low detection times. In this section, we will
focus on presenting some of the ML techniques applied to islanding
detection. A case study is introduced in Section 10.2.Kumar et al.
[65] proposed an islanding detection technique based on the
wavelet transform for feature extraction and an ANN for classifi-
cation. This technique was simulated in Matlab under different
scenarios considering all positive switching transients, islanding
events, and faults from the grid side. The proposed algorithm
achieved a mean accuracy of 98%. The algorithm was also tested
under noisy conditions achieving an accuracy of 88.5% in the
noisiest scenario. A similar approach was conducted by Puthen-
purakel and Subadhra [94]. Features were extracted using a wavelet
transform applied to the negative sequence voltage measurement
at the point of common coupling (PCC) of a DG system with a PV
system and a wind-power-plant. These features, along with the
total harmonic distortion, were used to train an ANN. An accuracy
of 95% was obtained. The main challenge in this problem was
making voltage swell (heavy load switching or capacitor load
switching events) predictions. Ahmadipour et al. [13] used a
Ridgelet Probabilistic ANN to detect islanding events. They simu-
lated two islanding scenarios and four none islanding scenarios to
extract features using the Slantlet Transform [99]. Then, the best of
these variables were selected to train the ANN using an evolu-
tionary search algorithm. The proposed method was compared
with other classifiers and achieved better performance.
Manikonda et al. have published several articles regarding
islanding detection in the past years. The use of CNN can be found
in [75,76]. In both of them, the authors collected data based on a
simulated 100 KW grid-connected PV system. The voltage phase
signals from the PCC were measured, concatenated, and then used
to generate images using the wavelet transformation method. In
[75], the authors used a transfer learning approach with AlexNet
[64] to classify islanding and non-islanding events with a SVM used
in the classification layer. On the other hand, in [76], the authors
used a custom-designed CNN. The results showed a 98.78% accu-
racy in both papers presented. Noise was also added to the data,
and the models achieved the same performance showing that the
proposed models were resilient to noise.
SVM methods have also been playing a major role in islanding
event predictions. Baghaee et al. [25] proposed a SVM algorithm
that was able to classify islanding and grid fault events. This work,
unlike previous work, studied the classification problem using real-
world data from a custom-made intelligent electronic device from a
micro-grid along with data from simulations. Seven Gaussian SVM
classifiers were trained, whose inputs included frequency, active
and reactive power of loads at PCC and total harmonic distortion
values of voltage and current. The results obtained in that work
showed high precision and effectiveness. Gayathry and Sujith [46]
proposed a SVM with a Radial-Basis-Function-Kernel for islanding
detection. The training and testing were conducted on a dataset
with different faults. The accuracy of the validations was 100%, but
it is worth noting that very few data were used for validation. A
SVM method was also proposed by Manikonda and Gaonkar [77].
The authors extracted features from the images using a histogram-
of-oriented-gradient-features. The dataset contained 1720 images,
where 860 of them were from islanding scenarios, and 860 were
different grid-connected events. Tree-based classifiers have also
been used to detect islanding scenarios as proposed by Khan et al.
[59]. The results show that the model achieved an accuracy of
97.6%, detecting the fault within 0.2s.

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

Table 3 shows a comparison between the reported articles. From
this analysis, we can see that very little has been done to report
experimental results on real testbeds. There is an interesting
pipeline proposed by Baghaee et al. [25] to conduct experimental
tests and also to compare the multiple ML methods in this field of
study.
6. ML for irradiance forecasting and PV output power
estimation
PV energy production is highly dependent on weather condi-
tions such as solar irradiance and temperature. Therefore, pro-
duction levels of this energy source fluctuate, making it difficult for
power companies to balance the production and consumption of
electricity when using PV systems. Hence, several ML algorithms
have been implemented to forecast solar irradiation and the output
power from PV systems. A case study is introduced in Section 10.5.
Miranda et al. [81] compared multiple ML algorithms, including
ANN, for the prediction of half-hourly diffuse horizontal solar
irradiance from only the global horizontal irradiance and a
geographic coordinate. Data taken from the national solar radiation
database (NSRDB) [100] for 6 different sites in Colombia was used.
The authors achieved a coefficient of determination ranging form
0.9974 to 0.9983 over the different sites. Furthermore, the article
includes a github repository with the work made. Ahn and Park [14]
used a RNN as a PV power short-term forecaster. This model was
trained using power data collected in real-time. The collected data
was the solar irradiation, the module temperature, the ambient
temperature, the wind speed, and the humidity. The forecasting
was conducted for 5min, 15min, 1h and 3h time horizons. The ex-
periments showed that the proposed model achieved higher pre-
diction accuracy than autoregressive integrated moving average
and support vector regression with RF models for the short-term
forecast. The RNN model achieved an accuracy of normalized
mean average of 99.1% and 98.6% for 5 min and 15 min forecasting,
and 97.4% and 96.2% for the 1h and 3h time horizons. The same
problem was solved by Hossain and Mahmood [55] using LSTM
networks. However, they also used a K-nearest-neighbors algo-
rithm to classify historical irradiance data into different types of sky
groups. The authors showed that the proposed model
outperformed models such as RNN, general regression neural
network, and extreme learning machines. Another approach using
LSTMs is presented by Pan et al. [87], where they used a short-term
solar generation forecasting method based on LSTM with temporal
attention mechanism. Narvaez et al. [83] evaluated Encoder-
Decoder LSTM and GRU networks for daily and weekly time hori-
zons predictions. The LSTMs outperformed the GRU networks both
in daily and weekly horizons.
Suresh et al. [105] had the goal to forecast the solar PV output
using CNN. The approach included using regular CNN, multi-
headed CNN, and a CNN-LSTM for short-term and medium-term
forecasting. The models were then compared to multiple linear
regression and an auto-regressive moving average models. The
features used for prediction included solar irradiation, wind speed,
ambient temperature, and the PV module temperature measured in
15-min windows. These features were chosen according to the IEA
report in [89]. The results in that work showed that a simple CNN
and a CNN along with a LSTM network performed well for 1h, 1-
day, and 1 week prediction time horizons. A similar approach was
conducted by Tovar et al. [112], where they used a five-layer CNN-
LSTM model for PV power forecast using data from Temixo [5,111].
The forecasting horizons ranged from 10 min to 180 min.
Table 4 describes the reported papers for PV output power
estimation and irradiance forecasting. This table shows that the
forecasting algorithms are mainly based on recurrent units. There is
a need to assess more recent techniques for time-series prediction
such as transformers architectures, which are based solely on
attention mechanism. Also, none of the articles described a pipeline
for the deployment of these models in real-time scenarios, which is
a challenging task that can define future research directions.
7. ML in sizing methods
ML techniques have been used to define the optimal number of
panels, storage capacity of batteries, tilt, and azimuth angles
required in PV systems. Moreover, several strategies have been
developed to size PV systems installed by residential costumers. A
case study is introduced in Section 10.6.
Kumar et al. [65] used ANNs to estimate the PV size, tilt, and
azimuth, using behind-the-meter data. The dataset used originated

Table 3
ML for islanding articles.

Articles Faults Studied Experimental Results Feature Extraction Main ML Technique

[65, 94] Positive switching No Wavelet ANN

transients, Transform
Islanding events,
Faults from grid side,
Voltage swell
[75, 76] Islanding, CNN
Power mismatch AlexNet
between DG
source generation,
Load connected to
PCC near zero,
Sudden switching of
capacitors and inductance
Loads
[59] Islanding, Decision Tree
Grid Faults
[13] Two islanding scenarios, Stantlet Ridgelet
4 none islanding scenarios Transform Probabilistic
ANN
[46, 77] Islanding, None SVM
Grid Faults,
Capacitor Switching
[25] Yes

307

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
Table 4
ML for irradiance forecasting and PV output power estimation.
Articles Power Forecast Algoritm Used
[81] e ANN
[14] RNN
[55] LSTM
[87] LSTM-attention
[83] e LSTM-GRU
[105, 112] CNN
from the Pecan Street dataset [88], which collects the behavior of
more than 1300 customer loads over one year. PV generation was
simulated using the system advisor model simulator [6] and the
PCLib toolbox. The input values were the minimum yearly day and
night net load values. The models used to predict PV size were
tested under different generated scenarios created by the authors,
including PV estimations of 1000 test customers, estimation with
varying tilt and azimuth considering errors in the dataset, different
net load data resolution, and mislabeled data added on purpose.
The results showed an improvement in accuracy for the neural
networks approach compared to the linear regression model. A
generalized RNN was used by Khatib and Elmenreich [60], where
the main objective was to estimate the PV array and battery sizing
ratio. This sizing was computed using the loss of load probability
index, latitude, and longitude. Using a simulation of hourly solar
radiation and load demand, the model was validated, achieving a
mean absolute percentage error of 0.6%.
An ML algorithm to map PV arrays in high-resolution overhead
imagery by identifying individual PV arrays, their size, and the
power generation capacities over large geographical areas was
proposed by Malof et al. [74]. It was created using CNN for semantic
segmentation, which provides pixel-wise labels of an input image.
It was called SolarMapper [10] and is publicly available. It was
trained on the Duke California Solar Array dataset [29], which
contains 400 km2 imagery and has 16000 hand-labeled solar ar-
rays. An estimate to calculate the installed solar capacity of the
panels is also proposed based on the identification realized by the
classifier. The model was accurate with a precision of 0.76 in the
object-based performance metric. The technique used to assess the
pixel-wise performance was the intersection-over-union (IOU)
metric, which was 0.67. To calculate the installed solar capacity, the
researchers first estimated the surface area of the PV array installed
based on the segmentation process. Then, the parameters of a
simple linear regression based on the surface area to predict the PV
array capacity installed was used. The correlation coefficient ach-
ieved by the model was 0.91, using color imagery to estimate the
parameters for each array.
8. ML in site-adaptation
Site-adaptation is the process in which satellite-based mea-
surements are calibrated with in-situ measurements that were
collected at a specific site. Most of the common approaches to solve
this problem are statistical analyses that attempt to fit satellite
estimates to in-situ data according to Polo et al. [93]. To the authors
knowledge, only one article has addressed this problem using ML
techniques. Narvaez et al. [83] tested several ML regression tech-
niques to construct a model that could accurately capture the point-
to-point relationship between meteorological station measure-
ments and satellite data. The input variables used were the
satellite-based data, while the output was the estimation of in-situ
global horizontal irradiance measurements. The input variables
were the diffuse horizontal irradiance, the global normal irradi-
ance, the direct normal irradiance, the solar zenith angle, the
308
Renewable Energy 196 (2022) 298e318
Irradiance Forecast Algorithm Used
e
K-nearest-neighbor
LSTM-attention
e
temperature, the wind speed, and the time of day. To collect sat-
ellite data, the authors used the NSRDB [100]. The models tested
included linear regressions, ANN, RF, and AdaBoost. To compare the
results of the different methods, the authors computed the site-
adaptation method quantile mapping [92], which is a classical
method for site-adaptation. When comparing the RMSE, R2, and
MAE metrics of the models, it was clear that almost every ML model
outperformed quantile mapping. The model with the best perfor-
mance was the random forest for all the performance indicators.
A case study is introduced in Section 10.7. Since ML applied
towards site-adaptation has not been yet further studied, other
techniques for regression and data preprocessing can be explored
to improve the performance of the site-adaptation algorithms.
9. Open resources for ML research in PV systems
9.1. Simulation environments
An OpenAI gym environment was created to evaluate MPPT
algorithms [21,22] and it is available in [23]. The state-space in-
cludes the current values of voltage, power, and the difference
between the actual and previous power. The action-space consists
of disturbances applied to the output voltage. For the simulation of
management systems, there are two main sources from which a
simulation of an environment can be found to easily compare
designed models. One is CityLearn, created with the OpenAI Gym
environment that allows the implementation of centralized or
multi-agent control for building energy coordination and demand
response in cities [118,119]. Another useful environment would be
pymgrid [54], which is an open-source package to generate and
simulate a large number of micro-grids.
9.2. Datasets
Open datasets are available to test and validate ML techniques
that are applied to different problems in PV systems. Some of them
include:
C Duke California Solar Array Dataset [29]: Over 400 km2 of
imagery and 16000 hand-labeled solar arrays.
C Desert Knowledge Australia Center Dataset [3]: It unifies
multiple real life data of PV technologies spanning many
types, ages, models and configurations.
C ESOLMET-IER Dataset [5]: The Institute of Renewable En-
ergies UNAM makes available solarmetric and meteorolog-
ical data taken from the station “ESOLMET-IER”.
C The National Solar Radiation Data Base (NSRDB) [100]: It is a
publicly available dataset that consists of solar radiation and
meteorological data over the US and surrounding countries
during the last 23 years.
C Photovoltaic Thermal Images Dataset [91]: Aerial thermal
images of PV arrays with the presence of one or more
anomaly cells and their respective masks. The images are
from a 66 MW PV plant in Tomboruke.

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

Fig. 6. PV power and duty cycle at G1 ¼ 1000W/m2, G2 ¼ 1000W/m2, G3 ¼ 1000W/

m2 and T ¼ 25 deg.C by using the Deep Q network agent and the Perturb and Observe
algorithm.

Fig. 5. Episode Reward for MPPT agent implemented with DQN.

Ptþ1
C Pecan Street Dataset [88]: Collects the behavior of more than r4 ¼ (6)
PMPP;STC
1300 customer loads over one year.
where d1, d2 and d3 are small constants, PMPP,STC is the MPP at
10. Case studies standard test conditions, and PMax is the MPP achieved in a defined
interval of time, which is restarted to the current power when the
In this section we present a case study for each of the problems signal (PG) of a pulse generator is 1, which does this every 0.504s.
in Fig. 1 with a link to open source code. PMax is initialized with a value of zero. A decaying epsilon greedy
strategy is used to explore the environment. The parameters used
for training the algorithm are similar to the ones presented in [90],
10.1. Case study: maximum power point tracking however, the researchers in that paper did not use Eq. (2) nor Eq.
(3). These parameters, along with the network architecture and
For this particular case study, the objective was to replicate the further explanation regarding the use of the algorithm, can be
results obtained by Phan et al. [90] where the authors used DQL. We found in the GitHub repository created.1
used the same PV system as the one present in the paper. The al- As it can be seen in Fig. 5, the DQN agent training curve con-
gorithm was trained and tested under different conditions, verges in about 2000 episodes. Multiple environmental conditions
including partial shading. The state-space was defined as the were used for this training process, including partial shaded and
voltage and current generated by the PV, the duty cycle, and the nonpartial shading conditions, different initial temperatures, and
duty cycle perturbation, while the action-space was defined as 9 different initial duty cycle values. Some of the results of the
discrete changes in the duty cycle. The discrete changes range from designed model against the commonly used Perturb-and-Observe
very-high-positive to very-high-negative-changes to the duty cycle, algorithm in three different environmental scenarios can be seen
including a value of no change. The reward function that we pro- in Figs. 6e8. These figures show that the DQN model outperformed
posed changes from the one presented in [90] as: the commonly used Perturb-and-Observe algorithm by generating
more power in each scenario.
r ¼ r1 þ r2 þ r3 þ r4 (1)


Ptþ1 if Ptþ1
PMax  d1 or PG ¼ 1 10.2. Case study: islanding detection
PMax ¼ (2)
PMax if Ptþ1
PMax < d1 and PGs1
In this case study, we replicated, to some extent, the work
8
>
<0 if Ptþ1
PMax  d2 presented by Manikonda and Gaonkar [76]. We generated our own
  dataset from a simulated 100 kW grid-connected PV system based
r1 ¼ Ptþ1
PMax (3)
>
:5 otherwise on [2]. The system included an active load and a capacitor bank
PMPP;STC connected to the PCC. To collect the data, the voltage at each phase
8 in the PCC was measured and concatenated in a single array. Then,
> 2  2 these time series data were converted into images using the
>
< Ptþ1 Ptþ1
if Ptþ1
Pt  d3
wavelet transform with a Morlet wavelet and re-scaled into
r2 ¼ PMPP;STC PMPP;STC (4)
>
> 224  224 RGB images. The scenarios included in the dataset can be
:
otherwise seen in Table 5.
The dataset was constructed by changing the active power of a


0 if 0  D  1
1
r3 ¼ (5)
otherwise
1
MPPT Case Study: https://github.com/SmartSystems-UniAndes/PV_MPPT_
Control_Based_on_Reinforcement_Learning.

309

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

Fig. 7. PV power and duty cycle at G1 ¼ 900W/m2, G2 ¼ 900W/m2, G3 ¼ 300W/m2

and T ¼ 25 deg.C by using the Deep Q network agent and the Perturb and Observe
algorithm. Fig. 9. Scalogram of the grid connected with a capacitor and load at 90 kW (scalogram
of class 0).

Fig. 8. PV power and duty cycle at G1 ¼ 1000W/m2, G2 ¼ 500W/m2, G3 ¼ 300W/m2

and T ¼ 25 deg.C by using the Deep Q network agent and the Perturb and Observe
algorithm.
Fig. 10. Scalogram of an islanding event with the capacitor connected and load at
90 kW (scalogram of class 4).
Table 5
Dataset composition for Islanding detection.
note that every image collected in the dataset was generated from
- On Grid On Grid Switch Switch Island. Island. LL LN
C Load C Load ON C OFF C C Load C Load Fault Fault measurements that took place exactly at the beginning of each
ON OFF Load Load ON OFF event (0.2se0.35s). Therefore, we encourage the reader to generate
Events 100 100 100 100 100 100 100 100
a more realistic dataset based on the system that we provide,
Class 0 1 2 3 4 5 6 7 adding noise to the data and collecting data in different time
windows.

load connected to the PCC from values between 90 kW and 110 kW
for each one of the events. An example of the scalograms generated
can be seen in Fig. 9 and Fig. 10.
The results obtained over the validation set are shown in the
confusion matrix presented in Fig. 11, which shows a 100% accuracy.
The dataset, along with the code to generate it, and the code
implemented for the CNN model are publicly available.2
Although the model achieved a 100% accuracy, it is important to
2
Case Study Islanding Detection: https://github.com/SmartSystems-UniAndes/
Islanding_Detection.
310
10.3. Case study: management
In this case study, we used the CityLearn environment and
replicated the methodology presented by Kathirgamanathan et al.
[58]. However, instead of using a centralized SAC agent, we
implemented a centralized DDPG agent. The environment involved
one building, which had a solar power capacity of 40 kW, an annual
cooling demand of 280, 239 kWh, and an annual non-shift-able
electrical demand of 36, 978 kWh. Moreover, we used the climate
zone type 1. The action-space is composed of changes that increase/
decrease the cooling energy stored in the storage device of the
building. The state-space variables used for our centralized agent

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

were day, month, year, the outside dry-bulb temperature, the direct Table 6
solar radiation, the non-shift-able electricity load of the building, Hyperparameters for the DDPG agent training.

and the state of charge of the buildings cooling storage. The Symbol Description Value
implementation of the DDPG agent was primarily based on a Replay buffer size 1e5
e
GitHub repository created by Tabor [108]. The hyperparameters Minibatch size 1024
e
used for training the DDPG algorithm can be seen in Table 6. L Discount factor 0.99
The reward function was designed based on the peak con- h1 Actor learning rate 0.001
sumption penalization as presented in [58], with incentive charging h2 Critic learning rate 0.002
during the nights. This reward function is defined based on Eq 7 Τ Target smoothing coefficient 0.0003
Hidden layers sizes 800
and Eq 8. e

ðe2demand Þ*b þ rnight

r¼ (7)
5000

8 5000 if 3am  hour  4am and csSOC < 0:9and 0:1 < a < 0:3
<
rnight ¼ (8)
:
5000 if 3am  hour  4am and csSOC < 0:9and a < 00 otherwise

where b is a weighting coefficient, edemand is the electricity demand,

csSOC is the state of charge of the cooling storage, and a is the action
chosen by the agent. The training process of the centralized DQN
agent can be seen in Fig. 12.
When the trained agent is tested, a reduction of around 16% over
the energy consumption occurs compared to an environment with
no agent. The building electricity demand profile using the imple-
mented DDPG agent and without using it throughout a year can be
seen in Fig. 13. Furthermore, in Fig. 14, a smaller interval of the
building electricity profile is presented to show the performance of
the created model.
The multi-objective cost function scores are presented in Table 7.
When compared to the results obtained in [58], the only metric in

Fig. 12. Episode Reward for Management agent implemented with DDPG in CityLearn.

which our model surpassed the SAC was in the net electricity
consumption. Also, the results obtained in the former paper used
more buildings than this case study. Therefore, we encourage the
reader to modify the DDPG algorithm implemented to be tested
with more buildings while also checking if, by changing hyper-
parameters of the model, it is possible to surpass the average score
obtained in [58]. Adjustments to the reward function are also
encouraged since the ramping score show that the implemented
model produces too many fluctuations over the building energy
demand. The current version of the model suffers from randomness
when training, which sometimes leads to a local maxima that does
not allow the agent to learn. This repository is publicly available.3

10.4. Case study: fault detection and PV diagnostics

For this case study, the main goal is to replicate, to some extent,
the results achieved by Rico Espinosa et al. [96]. In their paper, four

3
Case Study Management: https://github.com/SmartSystems-UniAndes/
Fig. 11. Confusion matrix on the validation dataset for islanding detection. Management_City_Learning.

311

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

Fig. 13. One year of electricity demand profile of building 4 without using i) PV generations, ii) with PV generation and iii) with PV generation and the implemented agent.

Fig. 14. Two hundred hours of electricity demand profile of building 4 i) without using PV generations, ii) with PV generation and iii) with PV generation and the implemented
agent.

Table 7
Results obtained for management case study.

Climate Ramping Load Avg. Daily Peak Net. Elect. Consumption Avg. Score
Factor Peak Demand

1 1.80 1.03 1.34 1.25 0.96 1.27

Fig. 15. Architecture of the classifier based on VGG-16.

failure signatures are classified based on RGB images. The main goal
is to use the augmented dataset with the masks shared in [17] to
create a quaternary classification model. However, in contrast to
the one presented in that paper, our model is generated using
transfer learning techniques. Transfer learning is a technique in
which a model that has been trained to solve a specific task is
reused as the starting point for a model that solves another task
[30]. We used the VGG16 as the pre-trained network [52,102]. Here,
we first segment the PV modules in the image, to then classify the
failure signature. The different classes of images can be seen in
Fig. 16. The structure of the classifier can be seen in Fig. 15.
The results were obtained on 400 images. The confusion matrix

312

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

Table 8
Layer configuration of the model.

Layer Configuration Activation Function

1D Convolution filters ¼ 16, kernel ¼ 16 ReLU

LSTM Enconder Units ¼ 24 Tanh, Sigmoid
Repeat Vector repetition factor ¼ 24 e
LSTM Decoder Units ¼ 24 Tanh, Sigmoid
Time Distributed Units ¼ 1 Linear

Table 9
CNN-LSTM Encoder-Decoder performance in daily forecasting.

Dataset R2 RMSE MAE

Training 0.837 110.5 59.86

Validation 0.85 100.9 55.08
Testing 0.846 106.5 58.72

Fig. 16. Fault signature categories that are intended to be detected: (a) Glass Damage,
(b) No Faults, (c) Dust and (d) Shadow.

Fig. 18. Daily forecasting of irradiance in 200 h.

Fig. 17. Confusion matrix for failure mode detection.

Fig. 19. Site-adaptation with SVR

313

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
Fig. 20. Sizing case study's example of original image, label and prediction based on the model created.
Table 10
SVR performance metrics of site adaptation according to cross-validation.
Model Hyperparameters R2 RMSE
SVR kernel: radial basis function 0.74 0.5
is presented in Fig. 17. Note that the classification accuracy is
around 86.75%, surpassing the accuracy obtained in [42]. The
implemented model along with the dataset are publicly available.4
10.5. Case study: irradiance forecasting
In this case study, we intended to replicate the methods applied
in [111] to the dataset used in [83]. This dataset was adjusted by the
site-adaptation models used in the [83]. With the improved data-
set, we trained a daily forecast model on an hourly sampling time.
The dataset was divided into three sets: 70% for training, 15% for
validation, and 15% for testing. We used a CNN-LSTM encoder-
decoder network model with an input size of 168 h of global hor-
izontal irradiance. The layer configuration of the proposed model
can be seen in Table 8.
The performance obtained after training the model through 17
epochs can be seen in Table 9, where R2, RMSE and MAE stand for
coefficient of determination, root mean square error and mean
average error respectively. In Fig. 18, the results of using the daily
forecasting model over a period of 200 h can be seen more clearly.
The model created along with the data are publicly available.5
10.6. Case study: sizing
For this case study, we intended to reproduce, to some degree,
the methodology applied by Malof et al. [74] to map PV arrays in
high-resolution overhead imagery. To do this, we used transfer
learning techniques with DeepLabv3 network [32], which uses
deep CNN in the application of semantic segmentation. We trained
and tested the algorithm using four images of 5000  5000 reso-
lution in [29], divided into images of size 500  500 and used as
input for the model. After training the model through 30 epochs,
we managed to achieve an inspection over union performance of
0.689 and 0.61 over the training and testing data, respectively.
An example of the results achieved by the model is shown in
Fig. 20. It is important to note that the images gathered in the
4 6
Case Study Fault Detection and PV Diagnostics: https://github.com/
SmartSystems-UniAndes/Failure_Detection_for_PV.
5 7
Case Study Irradiance Forecasting: https://github.com/SmartSystems-
UniAndes/Solar_Radiation_Forecasting.
Renewable Energy 196 (2022) 298e318
dataset are not completely well labeled. The metrics show that the
tendency of the model was to over-fit with more epochs. Therefore,
MAE we encourage the reader to test different models and try to obtain
0.27
better results. The repository created is publicly available.6
10.7. Case study: site adaptation
In this case study, we intended to test a support vector regres-
sion model over the dataset used by Narvaez et al. [83] for site
adaptation. The repository created is publicly available.7 The input
data used for the model was the same used by the researchers. As a
pre-processing technique, the measurements were all standard-
ized. A 4-cross-validation scheme was used to evaluate the per-
formance of the site adaptation technique. The results obtained can
be seen in Table 10.
Furthermore, an image representing the adapted site data
compared to the satellite and in-situ measurements is shown in
Fig. 19.
11. Conclusions and future research directions
We presented a thorough review of the recent advances in ML
applied to PV systems, where most of the papers considered were
published within the time period from 2014 to 2021. We also pre-
sented resources that are available to validate ML techniques in PV
systems, and provided study cases with links to open source code.
There are still a variety of challenges in the implementation and
analysis of PV systems that can potentially be addressed using ML.
One of the key aspects of the success of this type of techniques is
the availability of data that can provide information that is useful to
find those patterns that make it possible an automatic learning
process. Therefore, more open datasets with real data from PV
systems should be shared in the research community. Also, we have
identified several problems that can be considered for future
research in this field of study:
C More experimental tests on real testbeds should be con-
ducted while also making more comparisons between agent-
critic RL methods for MPPT. Moreover, RL techniques for
MPPT and voltage/frequency control should be tested
together.
Case Study Sizing: https://github.com/SmartSystems-UniAndes/Sizing_
DeepLabV3_Semantic_Segmentation.
Case Study Site Adaptation: https://github.com/SmartSystems-UniAndes/
Machine_Learning_for_Site_Adaptation.
314

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
C In forecasting methods, new techniques such as transformers
that are based solely on attention mechanism [115] should be
tested.
C Since the applications of ML techniques towards site-
adaptation are recently being developed, more data pro-
cessing techniques should be tested.
C For islanding detection, more physical trials should be con-
ducted to test the new proposed models.
C For fault detection and classification, there is a need to study
the behavior of RL-based MPPT algorithms in the presence of
faults. Also, new image recognition algorithms such as vision
transformers can be tested.
C For diagnosis and anomaly detection using thermal images,
the use of generative adversarial networks can be explored to
expand the current datasets available.
C Little has been done to propose ML algorithms for real-time
processing and decision-making. There is a need to propose
pipelines that describe the deployment of models for real-
time processing using TinyML or EdgeIA.
CRediT authorship contribution statement
Jorge Felipe Gaviria: Conceptualization, Methodology, Soft-
ware, Validation, Investigation, Data curation, Writing e original

ez: Conceptualization, Writing e
draft, Visualization. Gabriel Narva
review & editing, Supervision. Camilo Guillen: Methodology,
Software, Writing e review & editing. Luis Felipe Giraldo: Super-
vision, Writing e review & editing, Project administration. Michael
Bressan: Supervision, Writing e review & editing.
Declaration of competing interest
The authors declare that they have no known competing
financial interests or personal relationships that could have
appeared to influence the work reported in this paper.
Acknowledgement
This work was supported by Centro de Desarrollo Sostenible
para Ame
rica Latina through Convocatoria para financiacio
n de
proyectos de investigacio
n relacionados con el alcance de los
objetivos de desarrollo sostenible.
Appendix A. Supplementary data
Supplementary data to this article can be found online at
https://doi.org/10.1016/j.renene.2022.06.105.
References
[1] Deep deterministic policy gradient d spinning up documentation, URL,
https://spinningup.openai.com/en/latest/algorithms/ddpg.html.
[2] Detailed model of a 100-kW grid-connected PV array - MATLAB & simulink,
URL, https://www.mathworks.com/help/physmod/sps/ug/detailed-model-
of-a-100-kw-grid-connected-pv-array.html.
[3] DKASC, Alice springs | DKA solar centre, URL, http://dkasolarcentre.com.au/
locations/alice-springs.
[4] EPRI | smart grid resource center simulation tool e OpenDSS, URL, https://
smartgrid.epri.com/SimulationTool.aspx.
[5] ESOLMET-IER instituto de energías renovables, URL, http://esolmet.ier.unam.
mx/Tipos_consulta.php.
[6] Home - system advisor model (SAM), URL, https://sam.nrel.gov/.
[7] IEC 62116:2014 | IEC webstore | invertor, smart city, LVDC. https://webstore.
iec.ch/publication/6479.
[8] Ieee 1547-2018 - IEEE standard for interconnection and interoperability of
distributed energy resources with associated electric power systems in-
terfaces, URL, https://standards.ieee.org/standard/1547-2018.html.
[9] Renewable energy, URL, https://www.c2es.org/content/renewable-energy/.
315
Renewable Energy 196 (2022) 298e318
[10] Solar energy environmental mapper (solar mapper) web-based GIS appli-
cation, URL, https://solarmapper.anl.gov/.
[11] Types of faults in electrical power systems, URL, https://www.
electronicshub.org/types-of-faults-in-electrical-power-systems/.
[12] Haider Saif Agha, Zafar-ullah Koreshi, and Mohammad Bilal Khan. Artificial
neural network based maximum power point tracking for solar photovol-
taics. In 2017 International Conference on Information and Communication
Technologies (ICICT), pages 150-155. IEEE. ISBN 978-1-5386-2186-8. doi:
10.1109/ICICT.2017.8320180. URL http://ieeexplore.ieee.org/document/
8320180/.
[13] Masoud Ahmadipour, Hashim Hizam, Mohammad Lutfi Othman, Mohd
Amran Mohd Radzi, Avinash Srikanta Murthy, Islanding detection technique
using slantlet transform and ridgelet probabilistic neural network in grid-
connected photovoltaic system, 231:645e659. ISSN 03062619, https://
linkinghub.elsevier.com/retrieve/pii/S0306261918314570.
[14] Hyung Keun Ahn, Neungsoo Park, Deep RNN-based photovoltaic power
short-term forecast using power IoT sensors, 14(2):436, https://www.mdpi.
com/1996-1073/14/2/436. Number: 2 Publisher: Multidisciplinary Digital
Publishing Institute.
[15] Ikbal Ali, Mini Thomas, and Dr Pawan Kumar. Distributed Resource Planning
for Improved Voltage Stability of Radial Distribution System. 2:95e101.
[16] Afshine Amidi, Recurrent neural networks cheatsheet, URL, https://stanford.
edu/shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks.
[17] AndresFlorez-Git, AndresFlorez-git/proyecto_electronica, URL, https://
github.com/AndresFlorez-Git/Proyecto_Electronica. original-date: 2020-04-
16T19:27:14Z.
[18] A. Y. Appiah, X. Zhang, B. B. K. Ayawli, and F. Kyeremeh. Long short-term
memory networks based automatic feature extraction for photovoltaic
array fault diagnosis. 7:30089e30101. ISSN 2169-3536. doi: 10.1109/AC-
CESS.2019.2902949. Conference Name: IEEE Access.
[19] M. Arjun and J. B. Zubin. Artificial neural network based hybrid MPPT for
photovoltaic modules. In 2018 International CET Conference on Control,
Communication, and Computing (IC4), pages 140e145. doi: 10.1109/
CETIC4.2018.8530922.
[20] Bag Aurobinda, Bidyadhar Subudhi, and Pravat Kumar. A combined rein-
forcement learning and sliding mode control scheme for grid integration of a
PV system. ISSN 20960042, 20960042. doi: 10.17775/CSEEJPES.2017.01000.
URL https://ieeexplore.ieee.org/stamp/stamp.jsp?tp¼&amp;
arnumber¼8928283.
[21] L. Avila, M. De Paula, I. Carlucho, and C. Sanchez Reinoso. MPPT for PV sys-
tems using deep reinforcement learning algorithms. 17(12):2020e2027, .
ISSN 1548-0992. doi: 10.1109/TLA.2019.9011547. Conf. Name: IEEE Latin
Am. Transact..
[22] Luis Avila, Mariano De Paula, Maximiliano Trimboli, Ignacio Carlucho, Deep
reinforcement learning approach for MPPT control of partially shaded PV
systems in smart grids, 97:106711, . ISSN 15684946, https://linkinghub.
elsevier.com/retrieve/pii/S1568494620306499.
[23] Luis Avila, Mariano De Paula, Maximiliano Trimboli, Ignacio Carlucho, loa-
vila/mppt-gym. https://github.com/loavila/mppt-gym original-date: 2019-
04-10T13:39:40Z.
[24] F. Aziz, A. Ul Haq, S. Ahmad, Y. Mahmoud, M. Jalal, and U. Ali. A Novel
Convolutional Neural Network-Based Approach for Fault Classification in
Photovoltaic Arrays. 8:41889e41904. ISSN 2169-3536. doi: 10.1109/AC-
CESS.2020.2977116. Conference Name: IEEE Access.
[25] Hamid Reza Baghaee, Dragan Mlakic, Srete Nikolovski, Tomislav Dragicevic,
Support vector machine-based islanding and grid fault detection in active
distribution networks, 8(3): 2385e2403. ISSN 2168-6777, 2168-6785,
https://ieeexplore.ieee.org/document/8714012/.
[26] Kostas Bavarinos, Anastasios Dounis, Panagiotis Kofinas, Maximum power
point tracking based on reinforcement learning using evolutionary optimi-
zation algorithms, 14(2):335, https://www.mdpi.com/1996-1073/14/2/335.
Number: 2 Publisher: Multidisciplinary Digital Publishing Institute.
[27] Tarek Berghout, Benbouzid Mohamed, Toufik Bentrcia, Xiandong Ma,
Sinisa Djurovi
c, Leïla-Hayet Mouss, Machine learning-based condition
monitoring for PV systems: state of the art and future prospects, 14(19):
6316. ISSN 1996-1073, https://www.mdpi.com/1996-1073/14/19/6316.
[28] Shweta Bhatt, Reinforcement learning 101, URL, https://towardsdatascience.
com/reinforcement-learning-101-e24b50e1d292.
[29] Kyle Bradbury, Raghav Saboo, Jordan Malof, Timothy Johnson, Arjun
Devarajan, Wuming Zhang, Collins Leslie, and Richard Newell. Distributed
Solar Photovoltaic Array Location and Extent Data Set for Remote Sensing
Object Identification. doi: 10.6084/m9.figshare.3385780.v1. URL/articles/
dataset/Distributed_Solar_Photovoltaic_Array_Location_and_Extent_Data_
Set_for_Remote_Sensing_Object_Identification/3385780/1. Publisher: fig-
share type: dataset.
[30] Jason Brownlee, A gentle introduction to pooling layers for convolutional
neural networks. https://machinelearningmastery.com/pooling-layers-for-
convolutional-neural-networks/.
[31] Di Cao, Weihao Hu, Junbo Zhao, Qi Huang, Zhe Chen, Frede Blaabjerg,
A multi-agent deep reinforcement learning based voltage regulation using
coordinated PV inverters, 35(5):4120e4123. ISSN 0885-8950, 1558-0679.
https://ieeexplore.ieee.org/document/9113746/.
[32] Liang-Chieh Chen, Papandreou George, Florian Schroff, Hartwig Adam,
Rethinking atrous convolution for semantic image segmentation. http://
arxiv.org/abs/1706.05587.

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
[33] Zhicong Chen, Fuchang Han, Lijun Wu, Jinling Yu, Shuying Cheng, Peijie Lin,
Huihuang Chen, Random forest based intelligent fault diagnosis for PV arrays
using array voltage and string currents, 178:250e264, . ISSN 0196-8904,
https://www.sciencedirect.com/science/article/pii/S0196890418311415.
[34] Kuan-Yu Chou, Shu-Ting Yang, Yon-Ping Chen, Maximum power point
tracking of photovoltaic system based on reinforcement learning, 19(22):
5054, https://www.mdpi.com/1424-8220/19/22/5054. Number: 22 Pub-
lisher: Multidisciplinary Digital Publishing Institute.
[35] Giovanni Cipriani, Antonino D'Amico, Stefania Guarino, Donatella Manno,
Marzia Traverso, Vincenzo Di Dio, Convolutional neural network for dust and
hotspot classification in PV modules, 13(23):6357. ISSN 1996-1073, https://
www.mdpi.com/1996-1073/13/23/6357.
[36] M.A. Danandeh, S.M. Mousavi G, Comparative and comprehensive review of
maximum power point tracking methods for PV cells, 82:2743e2767. ISSN
13640321, https://linkinghub.elsevier.com/retrieve/pii/
S1364032117313813.
[37] G. Data, Global solar photovoltaic (PV) market update, 2019 with historic
(2006-2018) and forecast (2019-2030), URL, https://www.businesswire.com.
[38] Mahmoud Dhimish, Ghadeer Badran, Photovoltaic hot-spots fault detection
algorithm using fuzzy systems, 19(4):671e679. ISSN 1530-4388, 1558-2574,
https://ieeexplore.ieee.org/document/8854326/.
[39] Ruisheng Diao, Zhiwei Wang, Di Shi, Qianyun Chang, Jiajun Duan,
Xiaohu Zhang, Autonomous voltage control for grid operation using deep
reinforcement learning, URL, http://arxiv.org/abs/1904.10597.
[40] Mohamed E. El Telbany, Ayman Youssef, and Abdelhalim Abdelnaby Zekry.
Intelligent techniques for MPPT control in photovoltaic systems: a compre-
hensive review. In 2014 4th International Conference on Artificial Intelli-
gence with Applications in Engineering and Technology, pages 17e22. IEEE.
ISBN 978-1-4799-7910-3. doi: 10.1109/ICAIET.2014.13. URL https://
ieeexplore.ieee.org/document/7351807/.
[41] Jianqing Fan, Zhaoran Wang, Yuchen Xie, Zhuoran Yang, A theoretical
analysis of deep q-learning, URL, http://arxiv.org/abs/1901.00137.
[42] Andre
s Flo

rez, Luis Felipe Giraldo, and Michael Bressan. Portable Real-Time
Failure Mode Classification on PV Modules Based on RGB Images. page 10.
[43] Mohammad Mahdi Forootan, Iman Larki, Rahim Zahedi, Abolfazl Ahmadi,
Machine learning and deep learning in energy systems: a review, 14(8):
4832. ISSN 2071-1050, https://www.mdpi.com/2071-1050/14/8/4832.
Number: 8 Publisher: Multidisciplinary Digital Publishing Institute.
[44] Vincent François-Lavet, David Taralla, Damien Ernst, and Raphael Fonteneau.
Deep Reinforcement Learning Solutions for Energy Microgrids Management.
page 7.
[45] W. Gao and R.-J. Wai. A novel fault identification method for photovoltaic
array via convolutional neural network and residual gated recurrent unit. 8:
159493e159510. ISSN 2169-3536. doi: 10.1109/ACCESS.2020.3020296.
Conference Name: IEEE Access.
[46] V. Gayathry and M. Sujith. Machine learning based synchrophasor data
analysis for islanding detection. In 2020 International Conference for
Emerging Technology (INCET), pages 1e6. doi: 10.1109/
INCET49848.2020.9154089.
[47] Aurelien Geron. Hands-On Machine Learning with Scikit-Learn, Keras, and
TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems.
O'Reilly Media, second ed. edition..
[48] Mevludin Glavic. (deep) Reinforcement Learning for Electric Power System
Control and Related Problems: A Short Review and Perspectives. 48:22e35.
ISSN 13675788. doi: 10.1016/j.arcontrol.2019.09.008. URL https://
linkinghub.elsevier.com/retrieve/pii/S1367578819301014.
[49] Qinglai Guo, Junjian Qi, Venkataramana Ajjarapu, Richard Bravo, Joe Chow,
Zhengshuo Li, Rohit Moghe, Ehsan Nasr-Azadani, Ujjwol Tamrakar, G.N.
Taranto, Reinaldo Tonkoski, Gustavo Valverde, Qiuwei Wu, and Guangya
Yang. Review of Challenges and Research Opportunities for Voltage Control
in Smart Grids. 34:2790e2801. doi: 10.1109/TPWRS.2019.2897948.
[50] Kevin Gurney. Introduction to Neural Networks. ISBN: 9780203451519
Place: Oxford OCLC: 892785047.
[51] Mansour Hajji, Mohamed-Faouzi Harkat, Abdelmalek Kouadri,
Kamaleldin Abodayeh, Majdi Mansouri, Hazem Nounou, Nounou Mohamed,
Multivariate feature extraction based supervised machine learning for fault
detection and diagnosis in photovoltaic systems, page S0947358019304054.
ISSN 09473580, https://linkinghub.elsevier.com/retrieve/pii/
S0947358019304054.
[52] Muneeb Hassan, VGG16 - convolutional network for classification and
detection, URL, https://neurohive.io/en/popular-networks/vgg16/.
[53] Robert Hecht-Nielsen. Theory of the Backpropagation Neural Network. page
13.
[54] Gonzague Henri and Tanguy Levent. Pymgrid: an Open-Source python
Microgrid Simulator for Applied Artificial Intelligence Research. page 7..
[55] Mohammad Safayet Hossain, Hisham Mahmood, Short-term photovoltaic
power forecasting using an LSTM neural network and synthetic weather
forecast, 8: 172524e172533. ISSN 2169-3536, https://ieeexplore.ieee.org/
document/9200614/.
[56] Christos Kalogerakis, Eftichis Koutroulis, Michail G. Lagoudakis, Global MPPT
based on machine-learning for PV arrays operating under partial shading
conditions, 10(2):700, https://www.mdpi.com/2076-3417/10/2/700. Num-
ber: 2 Publisher: Multidisciplinary Digital Publishing Institute.
[57] R. Kase and S. Nishikawa. Fault detection of bypass circuit of PV module d
detection technology of open circuit fault location. In 2016 19th
316
Renewable Energy 196 (2022) 298e318
International Conference on Electrical Machines and Systems (ICEMS), pages
1e4.
[58] Anjukan Kathirgamanathan, Kacper Twardowski, Eleni Mangina, Donal Finn,
A centralised soft actor critic deep reinforcement learning approach to dis-
trict demand side management through CityLearn. http://arxiv.org/abs/2009.
10562.
[59] Mohammed Ali Khan, Ahteshamul Haque, and Varaha Satya Bharath. Ma-
chine Learning Based Islanding Detection for Grid Connected Photovoltaic
System. page 6..
[60] Tamer Khatib, Wilfried Elmenreich, An Improved Method for Sizing Stand-
alone Photovoltaic Systems Using Generalized Regression Neural Network,
2014. URL, https://www.hindawi.com/journals/ijp/2014/748142/. ISSN:
1110-662X Pages: e748142 Publisher: Hindawi Volume.
[61] Aristides Kiprakis and A.R. Wallace. Maximising Energy Capture from
Distributed Generators in Weak Networks. 151:611e618. doi: 10.1049/ip-
gtd:20040697.
[62] P. Kofinas, S. Doltsinis, A.I. Dounis, G.A. Vouros, A Reinforcement Learning
Approach for MPPT Control Method of Photovoltaic Sources, 108:461e473.
ISSN 09601481, https://linkinghub.elsevier.com/retrieve/pii/
S0960148117301891.
[63] Simeon Kostadinov, Understanding GRU networks. https://
towardsdatascience.com/understanding-gru-networks-2ef37df6c9be.
[64] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classifi-
cation with deep convolutional neural networks. 60(6):84e90. ISSN 0001-
0782, 1557-7317. doi: 10.1145/3065386. URL https://dl.acm.org/doi/
10.1145/3065386..
[65] S. Ananda Kumar, M.S.P. Subathra, Nallapaneni Manoj Kumar,
Maria Malvoni, N.J. Sairamya, S. Thomas George, Easter S. Suviseshamuthu,
Shauhrat S. Chopra, A novel islanding detection technique for a resilient
photovoltaic-based distributed power generation system using a tunable-q
wavelet transform and an artificial neural network, 13(16):4238. ISSN
1996-1073, https://www.mdpi.com/1996-1073/13/16/4238.
[66] C. Kuo, J. Chen, S. Chen, C. Kao, H. Yau, and C. Lin. Photovoltaic energy
conversion system fault detection using fractional-order color relation
classifier in microdistribution systems. 8(3):1163e1172. ISSN 1949-3061.
doi: 10.1109/TSG.2015.2478855. Conf. Name: IEEE Trans. Smart Grid..
[67] Varaha Satra Bharath Kurukuru, Ahteshamul Haque, Mohammed Ali Khan,
Subham Sahoo, Azra Malik, Frede Blaabjerg, A review on artificial intelli-
gence applications for grid-connected solar photovoltaic systems, 14(15):
4690. ISSN 1996-1073, https://www.mdpi.com/1996-1073/14/15/4690.
Number: 15 Publisher: Multidisciplinary Digital Publishing Institute.
[68] Maxim Lapan. Deep Reinforcement Learning Hands-On: Apply Modern RL
Methods, with Deep Q-Networks, Value Iteration, Policy Gradients, TRPO,
AlphaGo Zero and More. Packt Publishing Ltd. ISBN 978-1-78883-930-3.
Google-Books-ID: xKdhDwAAQBAJ..
[69] Sangyoon Lee, Dae-Hyun Choi, Reinforcement learning-based energy man-
agement of smart home with rooftop solar photovoltaic system, energy
storage system, and home appliances, 19(18):3937. ISSN 1424-8220, https://
www.mdpi.com/1424-8220/19/18/3937.
[70] Stefano Leonori, Alessio Martino, Fabio Massimo Frattale Mascioli, and
Antonello Rizzi. Microgrid Energy Management Systems Design by Compu-
tational Intelligence Techniques. 277:115524. ISSN 03062619. doi: 10.1016/
j.apenergy.2020.115524. URL https://linkinghub.elsevier.com/retrieve/pii/
S0306261920310369.
[71] Changfu Li, Chenrui Jin, Ratnesh Sharma, Coordination of PV smart inverters
using deep reinforcement learning for grid voltage regulation. http://arxiv.
org/abs/1910.05907.
[72] Xiaoyang Lu, Peijie Lin, Shuying Cheng, Yaohai Lin, Zhicong Chen, Lijun Wu,
Qianying Zheng, Fault diagnosis for photovoltaic array based on convolu-
tional neural network and electrical time series graph, 196:950e965. ISSN
01968904, https://linkinghub.elsevier.com/retrieve/pii/
S0196890419307332.
[73] Azra Malik, Ahteshamul Haque, V.S. Bharath Kurukuru, Mohammed Ali Khan,
Frede Blaabjerg, Overview of fault detection approaches for grid connected
photovoltaic inverters, 100035. ISSN 2772-6711, https://www.sciencedirect.
com/science/article/pii/S2772671122000079.
[74] Jordan M Malof, Boning Li, Bohao Huang, Kyle Bradbury, and Artem Stretslov.
Mapping Solar Array Location, Size, and Capacity Using Deep Learning and
Overhead Imagery. page 6.
[75] Santhosh K G Manikonda and Dattatraya N Gaonkar. A new islanding
detection method using transfer learning technique. In 2018 8th IEEE India
International Conference on Power Electronics (IICPE), pages 1e6. IEEE, .
ISBN 978-1-5386-4996-1. doi: 10.1109/IICPE.2018.8709431. URL https://
ieeexplore.ieee.org/document/8709431/.
[76] Santhosh K.G. Manikonda, Dattatraya N. Gaonkar, IDM based on image
classification with CNN (10):7256e7262, . ISSN 2051-3305, 2051-3305,
https://onlinelibrary.wiley.com/doi/10.1049/joe.2019.0025, 2019.
[77] Santhosh K.G. Manikonda, Dattatraya N. Gaonkar, Islanding detection
method based on image classification technique using histogram of oriented
gradient features, 14 (14):2790e2799, . ISSN 1751-8695, 1751-8695, https://
onlinelibrary.wiley.com/doi/10.1049/iet-gtd.2019.1824.
[78] Santhosh K.G. Manikonda and Dattatraya Narayan Gaonkar. Compreh. rev.
IDMs in DG sys.. 2(1):11e24, . ISSN 2515-2947, 2515-2947. doi: 10.1049/iet-
stg.2018.0096. URL https://onlinelibrary.wiley.com/doi/10.1049/iet-stg.
2018.0096.

ez, C. Guillen et al.
J.F. Gaviria, G. Narva
[79] Sabir Messalti, Abd Ghani Harrag, and Abd Elhamid Loukriz. A new neural
networks MPPT controller for PV systems. In IREC2015 the Sixth Interna-
tional Renewable Energy Congress, pages 1e6. IEEE. ISBN 978-1-4799-7947-
9. doi: 10.1109/IREC.2015.7110907. URL https://ieeexplore.ieee.org/
document/7110907.
[80] Dan Miklovic, The difference between process control & management, URL,
https://www.controlglobal.com/blogs/manufacturing2020/the-difference-
between-process-control-and-management/.
[81] Elieser Miranda, Jorge Felipe Gaviria Fierro, Narv
aez Gabriel, Luis
Felipe Giraldo, Michael Bressan, Prediction of site-specific solar diffuse
horizontal irradiance from two input variables in Colombia, 7(12):e08602.
ISSN 2405-8440, https://www.sciencedirect.com/science/article/pii/
S2405844021027055.
[82] Amirmohammad Moradi Sizkouhi, Mohammadreza Aghaei, Sayyed
Majid Esmailifar, A deep convolutional encoder-decoder architecture for
autonomous fault detection of PV plants using multi-copters, 223:217e228.
ISSN 0038-092X, https://www.sciencedirect.com/science/article/pii/
S0038092X21003935.
[83] Gabriel Narvaez, Luis Felipe Giraldo, Michael Bressan, and Andres Pantoja.
Machine Learning for Site-Adaptation and Solar Radiation Forecasting. 167:
333e342. ISSN 09601481. doi: 10.1016/j.renene.2020.11.089. URL https://
linkinghub.elsevier.com/retrieve/pii/S0960148120318395.
[84] OLah, Understanding LSTM networkds. https://web.stanford.edu/class/
cs379c/archive/2018/class_messages_listing/content/Artificial_Neural_
Network_Technology_Tutorials/OlahLSTM-NEURAL-NETWORK-TUTORIAL-
15.pdf.
[85] OpenAI. Gym, A toolkit for developing and comparing reinforcement
learning algorithms, URL, https://gym.openai.com.
[86] K. Pahwa, M. Sharma, M. S. Saggu, and A. Kumar Mandpura. Performance
evaluation of machine learning techniques for fault detection and classifi-
cation in PV array systems. In 2020 7th International Conference on Signal
Processing and Integrated Networks (SPIN), pages 791e796. doi: 10.1109/
SPIN48934.2020.9071223.ISSN:2688-769X.
[87] Cheng Pan, Jie Tan, Dandan Feng, and Yi Li. Very short-term solar generation
forecasting based on LSTM with temporal attention mechanism. In 2019 IEEE
5th International Conference on Computer and Communications (ICCC),
pages 267e271. IEEE. ISBN 978-1-72814-743-7. doi: 10.1109/
ICCC47050.2019.9064298. URL https://ieeexplore.ieee.org/document/
9064298/.
[88] PecanStreet. PecanStreet inc. dataport load data, URL, https://www.
pecanstreet.org/dataport/.
[89] Sophie Pelland, Jan Remund, Jan Kleissl, Takashi Oozeki, and Karel De Bra-
bandere. Photovoltaic and Solar Forecasting: State of the Art. ISBN 978-3-
906042-13-8.
[90] Bao Chau Phan, Ying-Chih Lai, Chin E. Lin, A deep reinforcement learning-
based MPPT control for PV systems under partial shading condition,
20(11):3039, https://www.mdpi.com/1424-8220/20/11/3039. Number: 11
Publisher: Multidisciplinary Digital Publishing Institute.
[91] Roberto Pierdicca, Marina Paolanti, Andrea Felicetti, Fabio Piccinini,
Primo Zingaretti, Automatic faults detection of photovoltaic farms: solAIr, a
deep learning-based system for thermal images, 13(24):6496, https://www.
mdpi.com/1996-1073/13/24/6496. Number: 24 Publisher: Multidisciplinary
Digital Publishing Institute.
[92] Jesus Polo, Carlos Fern
andez-Peruchena, Vasileios Salamalikis, Luis Mazorra-
Aguiar, Mathieu Turpin, Luis Martín-Pomares, Andreas Kazantzidis,
Philippe Blanc, Remund Jan, Benchmarking on improvement and site-
adaptation techniques for modeled solar radiation datasets, 201:469e479, .
ISSN 0038-092X, https://www.sciencedirect.com/science/article/pii/
S0038092X20302784.
[93] Jesus Polo, Stefan Wilbert, Jose
A. Ruiz-Arias, Richard Meyer,
Christian Gueymard, Marcel Suri, Luis Martin, Theresa Mieslinger,
Philippe Blanc, Ian Grant, John Boland, Pierre Ineichen, Jan Remund,
Rodrigo Escobar, Alberto Troccoli, Manajit Sengupta, Kristian Pagh Nielsen,
David Renne
, Norbert Geuder, Tomas Cebecauer, Preliminary survey on site-
adaptation techniques for satellite-derived and reanalysis solar radiation
datasets, 132:25e37, https://hal-mines-paristech.archives-ouvertes.fr/hal-
01297310.
[94] Sleeba Paul Puthenpurakel and P. R. Subadhra. Identification and classifica-
tion of microgrid disturbances in a hybrid distributed generation system
using wavelet transform. In 2016 International Conference on Next Gener-
ation Intelligent Systems (ICNGIS), pages 1e5. IEEE. ISBN 978-1-5090-0870-
4. doi: 10.1109/ICNGIS.2016.7854066. URL http://ieeexplore.ieee.org/
document/7854066/.
[95] Daniel Rangel-Martinez, K. D. P. Nigam, and Luis A. Ricardez-Sandoval. Ma-
chine Learning on Sustainable Energy: A Review and Outlook on Renewable
Energy Systems, Catalysis, Smart Grid and Energy Storage. 174:414e441.
ISSN 0263-8762. doi: 10.1016/j.cherd.2021.08.013. URL https://www.
sciencedirect.com/science/article/pii/S0263876221003312.
[96] Alejandro Rico Espinosa, Michael Bressan, Luis Felipe Giraldo, Failure
signature classification in solar photovoltaic plants using RGB images and
convolutional neural networks, 162:249e256. ISSN 0960-1481, https://
www.sciencedirect.com/science/article/pii/S0960148120312301.
317
Renewable Energy 196 (2022) 298e318
[97] Mehdi Salay Naderi, Gevork B. Gharehpetian, M. Abedi, and Trevor Black-
burn. Modeling and Detection of Transformer Internal Incipient Fault during
Impulse Test. 15: 284e291. doi: 10.1109/T-DEI.2008.4446762.
[98] T. Sansawatt, J. O'Donnell, L. F. Ochoa, and G. P. Harrison. Decentralised
voltage control for active distribution networks. In 2009 44th International
Universities Power Engineering Conference (UPEC), pages 1e5.
[99] Ivan Selesnick. Slantlet transform.. 47:1304e1313. doi: 10.1109/78.757218.
[100] Manajit Sengupta, Yu Xie, Anthony Lopez, Aron Habte, Galen Maclaurin,
Shelby James, The national solar radiation data base (NSRDB), 89:51e60.
ISSN 1364-0321, https://www.sciencedirect.com/science/article/pii/
S136403211830087X.
[101] Shivam Kumar, Jong-Chyuan Tzou, Shang-Chen Wu, A multi-objective pre-
dictive energy management strategy for residential grid-connected PV-bat-
tery hybrid systems based on machine learning technique, 237:114103. ISSN
0196-8904, https://www.sciencedirect.com/science/article/pii/
S019689042100279X.
[102] Karen Simonyan, Andrew Zisserman, Very deep convolutional networks for
large-scale image recognition, URL, http://arxiv.org/abs/1409.1556.
[103] C. Strobl and P. Meckler. Arc faults in photovoltaic systems. In 2010 Pro-
ceedings of the 56th IEEE Holm Conference on Electrical Contacts, pages
1e7. doi: 10.1109/HOLM.2010.5619538.ISSN:2158-9992.
[104] X. Sun and J. Qiu. Two-stage volt/var control in active distribution networks
with multi-agent deep reinforcement learning method. pages 1e1. ISSN
1949-3061. doi: 10.1109/TSG.2021.3052998. Conference Name: IEEE Trans-
actions on Smart Grid.
[105] Vishnu Suresh, Przemyslaw Janik, Jacek Rezmer, and Zbigniew Leonowicz.
Forecasting Solar PV Output Using Convolutional Neural Networks with a
Sliding Window Algorithm. page 15.
[106] Richard S. Sutton and Andrew G. Barto. Reinforcement learning: an Intro-
duction. Adaptive Computation and Machine Learning Series. The MIT Press,
second ed. edition. ISBN 978-0-262-03924-6.
[107] Christian Szegedy, Vanhoucke Vincent, Sergey Ioffe, Jonathon Shlens,
Zbigniew Wojna, Rethinking the inception architecture for computer vision,
URL, http://arxiv.org/abs/1512.00567.
[108] Phil Tabor, philtabor/actor-critic-methods-paper-to-code. https://github.
com/philtabor/Actor-Critic-Methods-Paper-To-Code original-date: 2020-
05-26T23:40:09Z.
[109] Satoshi Takayama, Atsushi Ishigame, Autonomous decentralized control of
distribution network voltage using reinforcement learning**this study was
supported by research grant from Japan power academy, 51 (28):209e214.
ISSN 2405-8963, https://www.sciencedirect.com/science/article/pii/
S2405896318334220.
[110] Giuseppe Marco Tina, Cristina Ventura, Sergio Ferlito, and Saverio De Vito. A
state-of-art-review on machine-learning based methods for PV. 11(16):
7550. ISSN 2076-3417. doi: 10.3390/app11167550. URL https://www.mdpi.
com/2076-3417/11/16/7550. Number: 16 Publisher: Multidisciplinary Digi-
tal Publishing Institute..
[111] Mario Tovar, Esolmet, URL, https://github.com/mariotovarrosas/
ESOLMET2019. original-date: 2020-09-21T22:54:55Z.
[112] Mario Tovar, Miguel Robles, and Felipe Rashid. PV power prediction, using
CNN-LSTM hybrid neural network model. case of study: Temixco-morelos,
mexico. 13(24):6512. ISSN 1996-1073. doi: 10.3390/en13246512. URL
https://www.mdpi.com/1996-1073/13/24/6512.
[113] G. Valverde and T. Van Cutsem. Model predictive control of voltages in active
distribution networks. 4(4):2152e2161. ISSN 1949-3061. doi: 10.1109/
TSG.2013.2246199. Conf. Name: IEEE Trans. Smart Grid..
[114] Laurens van der Maaten and Geoffrey Hinton. Viualizing Data Using T-SNE.
9: 2579e2605.
[115] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones,
Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention is all you need,
URL, http://arxiv.org/abs/1706.03762.
[116] Jan Von Appen, Martin Braun, Thomas Stetz, Konrad Diwold, and Dominik
Geibel. Time in the Sun: the Challenge of High PV Penetration in the German
Electric Grid. 11: 55e64. doi: 10.1109/MPE.2012.2234407.
[117] P. N. Vovos, A. E. Kiprakis, A. R. Wallace, and G. P. Harrison. Centralized and
distributed voltage control: impact on distributed generation penetration.
22(1): 476e483. ISSN 1558-0679. doi: 10.1109/TPWRS.2006.888982. Conf.
Name: IEEE Trans. Power Syst...

R. Va
[118] Jose
szquez-Canteli, Je
ro
^me K€ampf, G. Henze, Zolt
an Nagy, intelligent-
environments-lab/CityLearn, original-date: 2019-06-30T02:41:48Z, https://
github.com/intelligent-environments-lab/CityLearn.

R V

[119] Jose azquez-Canteli, Sourav Dey, Gregor Henze, and Zoltan Nagy. Cit-
yLearn: Standardizing Research in Multi-Agent Reinforcement Learning for
Demand Response and Urban Energy Management. page 11..
[120] Shengyi Wang, Jiajun Duan, Di Shi, Chunlei Xu, Haifeng Li, Ruisheng Diao,
and Zhiwei Wang. A data-driven multi-agent autonomous voltage control
framework using deep reinforcement learning. 35(6):4644e4654. ISSN
0885-8950, 1558-0679. doi: 10.1109/TPWRS.2020.2990179. URL https://
ieeexplore.ieee.org/document/9076841/.
[121] Pete Warden and Daniel Situnayake. TinyML: Machine Learning with Ten-
sorFlow Lite on Arduino and Ultra-low-power Microcontrollers. ”O'Reilly
Media, Inc.”. ISBN 978-1-4920-5201-2. Google-Books-ID: tn3EDwAAQBAJ..

ez, C. Guillen et al.
J.F. Gaviria, G. Narva Renewable Energy 196 (2022) 298e318

[122] Xu Xu, Youwei Jia, Yan Xu, Zhao Xu, Songjian Chai, and Chun Sing Lai. A [125] Ayman Youssef, Mohammed El-Telbany, Abdelhalim Zekry, The role of
multi-agent reinforcement learning-based data-driven method for home artificial intelligence in photo-voltaic systems design and control: a review,
energy management. 11(4):3201e3211. ISSN 1949-3053, 1949-3061. doi: 78:72e79. ISSN 13640321, https://linkinghub.elsevier.com/retrieve/pii/
10.1109/TSG.2020.2971427. URL https://ieeexplore.ieee.org/document/ S1364032117305555.
8981876/.. [126] Muhammad Fahad Zia, Elhoussin Elbouchikhi, and Mohamed Benbouzid.
[123] S.-M Xue and C. Liu. Line-to-line Fault Analysis and Location in a VSC-Based Microgrids Energy Management Systems: A Critical Review on Methods,
Low-Voltage DC Distribution Network. 11. doi: 10.3390/en11030536.. Solutions, and Prospects. 222:1033e1055. doi: 10.1016/
[124] Tony Yiu. Understanding random forest. URL https://towardsdatascience. j.apenergy.2018.04.103.
com/understanding-random-forest-58381e0602d2.

318