Fault classification for Photovoltaic Modules Using
Thermography and Machine Learning Techniques
V S Bharath. Kurukuru, Ahteshamul Haque,
Department of Electrical Department of Electrical
Engineering, Faculty of Engineering, Faculty of
Engineering & Technology Engineering & Technology
Jamia Millia Islamia (Central Jamia Millia Islamia (Central
University) University)
Delhi, India Delhi, India
kvsb272@gmail.com. ahaque@jmi.ac.in
Abstract— Fault classification and detection are necessary
for safety, efficiency and reliability of photovoltaic systems.
Regardless of the fact that PV system requires low
maintenance due to the absence of moving parts, they are still
vulnerable to many faults. Particularly for PV modules, it’s
problematic to shut them down completely during faults as it
will affect their performance and result in early degradation.
Additionally, the inability of conventional fault detection
methods in early detection of faults leads to higher risk of
failure. In this paper thermography and machine learning
based PV module fault classification is developed. Texture
feature analysis is adapted to study the features of different
faulty panel thermal images. The extracted features are trained
by implementing Artificial Neural network classifier to develop
the fault classifier. The developed algorithm depicted 93.4%
training efficiency and 91.7% testing efficiency which is better
when compared with the conventional classification techniques.
Keywords— Photovoltaic Modules, Thermography, Grey
Level Co-occurrence Matrix (GLCM), Artificial Neural Network.
I. INTRODUCTION
Fault diagnosis is necessary for safety, efficiency and
reliability of photovoltaic systems [1]. Regardless of the fact
that PV system requires low maintenance due to the absence
of moving parts, they are still vulnerable to many faults.
Particularly for PV modules, it’s problematic to shut them
down completely during faults as it will affect their
performance and result in early degradation. Major faults
that can affect the operation and efficiency of PV modules
are frame defects [2], glass defects [3], encapsulant defects
[4], back sheet defects [4] and other cell defects like snail
trail [5], hot spot [6], cracked cell [7], broken cell [8] and
faulty interconnections [9]. In addition, conventional series-
parallel PV configurations increase voltage and current
ratings, leading to higher risk of large fault currents or dc
arcs. In general, defects on the modules are usually not
visible by the naked eye and special methods are needed to
detect them.
Conventional detection methods deal with diagnostics
using radiation, static characteristics measurement, and
dynamic parameters measurement (Impedance
Spectroscopy). Serigu et al. [3] developed a fault
classification method for PV module cells using
electroluminescence images. Places, which are affected by
some damage, and with high density of defects, are seen as
dark places in the electroluminescence images. Raghavi et
al. [10] observed photoluminescence imaging of PV
modules with sunlight excitation. In this process a strong
radioactive recombination is stimulated which can help in
the process of PV cell diagnostics. Dezso [11] presented a
978-1-5386-8125-1/19/$31.00 ©2019 IEEE
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
Mohammed Ali Khan, Arun Kumar Tripathy,
Department of Electrical National Institute of Solar Energy
Engineering Haryana, India
Faculty of Engineering & aktripathi@nic.in
Technology
Jamia Millia Islamia (Central
University)
Delhi, India
mak1791@gmail.com
fault detection method for PV modules by measuring the
series resistance and comparing them with reference values
or datasheet. John et al. reviewed the use of thermography in
fault detection of PV modules and stated that infrared
thermography is a well-established and competitive fault
diagnostic method. But it is observed that most of the
research involving thermography also uses flash test to
extract the VI characteristics of the PV module. This process
has multiple drawbacks like, observing the thermal images
with human eyes for fault identification, and performing
flash tests to observe the VI curves to confirm the fault. This
involves a lot of equipment and time delay. Apart from
these, there are many methods like IV curve monitoring,
Dark current measurement, and resistance extraction
methods. The common property of the above-mentioned
measurement methods is that, they can be commonly used
on-site with little adjustments but lack in efficiency and
detection time. Remote monitoring is also considered as a
major issue in most of the methods. In order to over come
these drawbacks, new classification technique needs to be
developed.
On a parallel track, the field of image processing and
analysis has greatly advanced in the recent past. Primary
works mainly addressed the enhancement, denoising, and
compression of grayscale images in satellite and medical
applications. However, as sensor technology progressed,
color images with mega-pixel resolutions required not only
more sophisticated algorithms, but also new solutions for the
detection, classification, and comparison of objects in
multiple images. Information retrieval methods became
relevant as datasets surpassed the billionth image boundary.
In fact, research in image analysis spans a wide variety of
disciplines including feature extraction, pattern recognition,
classification, and artificial intelligence.
In this paper thermography, image processing and
machine learning techniques are adapted together for
developing a new fault classification technique as proposed
in figure 1.
Thermography and
Grey image
Data Preparation
Grey Level Contrast Homogenity
Feature Dataset
Co-occurrence
Matrix Entropy Correlation
Texture Features
extraction Energy
Classifier
Figure.1 Block representation of proposed classification algorithm
 The developed technique can overcome the drawbacks
of conventional techniques by providing advantages with
remote monitoring, least detection time and high
classification efficiency. Further sections of the paper
provide a detailed overview of every aspect of the proposed
technique. Section 2 provides an overview on thermography
and advantages with thermal images. Section 3 describes the
feature extraction method for thermal images using gray
level Co-occurrence matrix. Section 4 details the machine
learning techniques and classification requirements for
developing the algorithm. The experimental analysis, results
and conclusion were given in section 5 and section 6
respectively.
II. THERMOGRAPHY
Thermal imager registers thermal energy naturally
radiated from objects and produce images called
thermograms. These instruments detect thermal radiation
without involving direct contact with the targeted object or
scene. In fact, this is an advantageous feature of infrared
thermography, not only for the convenience of reading
precise temperature measurements remotely, but most
importantly, for meeting security, quality, and non-
intervention requirements of many applications. A color-
coded thermal image provides a means for the visual
assessment of temperature distributions of a targeted scene
in real-time. This is a relevant qualitative feature.
Nonetheless, the pivotal aspect to be examined is the
quantitative use of thermography.
In general, the quantitative analysis of a thermal image
involves three main steps: (i) the analysis of temperature
data, (ii) the identification of temperature irregularities, and
(iii) the comparison to a standard or different state. The
analysis of temperature data relies on the extraction of
statistical values, namely the minimum, maximum, mean,
and standard deviation, from the selected region of interest
within an image. The selection of the region of interest is
generally performed manually by using simple geometric
shapes. Such manual process is time-consuming, error-
prone, and expert-dependent. By using simple-shaped
regions, except for simple-shaped objects, the statistical
values become inaccurate from either including background
or excluding marginal data into calculations. In infrared
thermography, the identification of temperature irregularities
is based on the assessment of hot/cold spots and isotherms.
Current practice reports the application of global methods
for the identification of salient spots and regions. In this
case, relevant localized anomalies may not be recognized.
Finally, the comparison to a standard or different state
requires the analysis of regions within particular
thermograms for change detection.
III. FEATURE EXTRACTION
Feature extraction is the primitive step in pattern
recognition process. This step extracts some features such as
color, shape, and texture. These features comprise the
relevant information of an image and can be used in multiple
applications. In this paper texture features of images are
extracted as they provide a higher-order description of the
image and includes information about the spatial distribution
of tonal variations or gray tones [12].
A. Texture description
One of the most widely used texture description
methods is statistical texture analysis. In this method, texture
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
features are computed from the statistical distribution of
observed combinations of intensities at specified positions
relative to each other in the image [12], usually in a square
local window of odd size. According to the number of
intensity points (pixels) in each combination, statistics are
classified into first-order, second order and higher-order
statistics. First order statistics are, for example, mean,
variance and higher order moments. They are easy to
compute and variance. However, they only take into account
global texture parameters, and not mutual intensity
distribution between pixels. Second order statistical texture
features are called grey level co-occurrence matrices
(GLCMs) and were first presented by Haralick et al. [12].
B. Grey level co-occurrence matrices
A GLCM is a matrix where the number of rows and
columns is equal to the number of grey levels, G, in the
image.
There are many features that can be extracted from the
GLCM. Usually they measure the homogeneity or the
contrast in the local window. Some of them were presented
in the original paper [12], while others have been invented
later [13]. Some features were discovered based on the
Mahalanobis distance between average GLCMs for two
classes. In this research standard GLCM features were
extracted by converting the acquired thermal image in to
grey image. Features such as Contrast, Correlation, Energy,
Homogeneity, and Entropy were extracted using equations
(1)-(7).
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Where is elements of GLCM with two different
instances i and j, is the mean of GLCM, and
contribute to the variance of instances.
Once the features are extracted, they are tabulated for
classification process. Corresponding classes for each
feature set were defined for efficient functionality of the
classifier.
IV. CLASSIFICATION PROCESS
Classifier or classification process is defined as the
collection of specific machine learning algorithms for
extracting patterns from data. These algorithms are primarily
focused on automatic learning to recognize complex patterns
and make intelligent decisions based on data [14].
Nowadays, there are multiple machine learning (ML)
algorithms, often inspired in nature around us. In this paper
pattern recognition and classification aspects of Artificial
Neural Network (ANN) technique is used. The adapted
Machine Learning technique is described in detail in further
sections. A general overview of other machine learning
techniques is described in detail in [15].
A. Artificial Neural Networks
 Artificial neural networks are inspired by biological
brains. They can be used to solve a variety of tasks, such as
computer vision, medical diagnosis or speech recognition.
Neural networks can improve their performance on certain
task by taking examples into consideration. Such process
can be simply called learning.
A neural network (NN) consists of artificial neurons.
An artificial neuron (simply neuron) is inspired by
biological neuron. Artificial neurons are connected together.
Each connection is used to transmit a signal to another
neuron. The receiving neuron can process it and signal other
neurons connected to it. Each neuron has set of weights. The
weights are edges, connected to each neuron in the previous
layer. Neuron has one output value, which is weighted sum
of input values plus a bias passed to an activation function.
The Output value of neuron is defined by the following
formula:
(8)
Where are input values, are weights, tends to
number of input variables, is bias, is activation
function and is output of the neuron.
The simplest type of neural network is called
perceptron. It consists of one neuron and it is able to decide
whether an input defined by a vector of numbers belongs to
some specific class or not. Single-layer perceptron’s are
capable of solving linearly separable problems [16]. The
neuron in single-layer perceptron uses an activation function
which maps the weighted sum of inputs plus bias to values 0
or 1. Typically, neurons are organized into groups called
layers. A neural network consists of at least two layers, that
is input and output layer. It may also contain hidden layers.
Multi-layer perceptron (MLP) consists of at least three
layers. The output and hidden layers use non-linear
activation function. In contrary to single-layer perceptron,
multi-layer perceptron is capable of learning data that is not
linearly separable. MLP uses backpropagation as a learning
algorithm which is used to calculate adjustments to the
weights during supervised learning. The goal of supervised
learning is to map set of inputs to their correct output. A
scaled conjugate backpropagation is used to train neural
network with multiple layers by tuning the weights
according to loss function which represents the difference
between the network’s output and the expected output.
While training with neural network, activation and loss
function play a major role in classification process. An
activation function defines the output of a neuron for any
given input. In general, there are large number of activation
functions like binary step, sigmoid function, hyperbolic
tangent and rectified linear unit. For the purpose of pattern
recognition and classification sigmoid activation function is
adapted. A sigmoid function maps the resulting values in
between 0 and 1and is used in the output layer for tasks
where the output is probability.
Loss function or cost function is used to represent the
error of the neural network. It can be used to compare the
performance of multiple models. The loss function is chosen
based on the type of task being solved. The cross entropy
and mean square error are two frequently used loss
functions. Cross entropy is used to quantify the difference
between two probability distributions while solving
multiclass classification tasks. Whereas, mean squared error
(MSE) is used to measure the error of a neural network and
it is defined by:
(9)
Where, is the number of outputs, is the actual value and
is the predicted value. MSE punishes large mistakes much
more than small mistakes and its output is always positive.
In general, MSE measures the average squared error of our
predictions. For each point, it calculates square difference
between the predictions and the target and then average
those values. This metrics is highly useful while dealing
with unexpected values that needs to be taken care. With
that said, it is commonly used as a cost function for
regression tasks where the goal is to predict n real values.
Considering all the above-mentioned aspects, the
feature data is subjected for classification process. The
simulation and experimental analysis are explained in the
further sections.
V. EXPERIMENTAL ANALYSIS
The Experimental analysis is carried out on grid
connected roof top solar PV system of capacity 169.92kW
(320W X 531 modules arranges in 27 strings) at Faculty of
Engineering and Technology, Jamia Millia Islamia, New
Delhi, India. After careful Inspection, thermal images of
normal and potential faulty PV modules are captured in
multiple irradiance and temperature conditions to develop
the algorithm. Fluke TiS45 thermal imager is used to capture
the thermal images as shown in figure 2.
Figure. 2 Thermal image captured using Fluke TiS45.
A total of 7 faulty modules and 1 normal operation
module are considered for experimental analysis with the
proposed approach. The size of each image captured is
280pixels by 500 pixels. In order to increase the sample size
for each class, the original image is divided into 14 X 14
subregions as shown in figure 3.
Figure. 3 Process of capturing Thermal Image and splitting into sub-
windows
Each sub-window has 1/14 size of total surface and is
20 pixels by 35 pixels. This resolution provides all the
useful features required for developing the classification
algorithm. Any division above the 14 X 14 sub regions with
each region corresponding to 20 X 35 pixels will tamper
with the features available, and any division less than it will

Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
increase the processing and detection time. A total of 196
sub-windows are obtained for each thermal image. The
texture analysis methods considered in the previous sections
is tested on the thermal image set of PV module surface.
Each sub-window region is transformed to a gray scale
image and the red, green, blue channels are extracted. To
obtain the texture features the grey level co-occurrence
matrix is calculated as explained in section in 2. In this
research the normalization of co-occurrence matrix is
ignored as the features are used for comparison purposes
where normalization is not vital. The contrast, correlation,
homogeneity, energy and entropy of all the sub-windows are
extracted using equations and tabulated along with
corresponding classes for providing input to the classifier.
Once the feature data is obtained, neural network
classifier is adapted for developing the classification
algorithm. All the 1568 samples are subjected to their Figure.5 Training state of neural network
corresponding classes in order to prepare input and target Figure 5 depicts the training state of the neural network
data. Random division is applied to divide the data for by depicting the gradient and validation fail iterations. The
training, validation and testing purposes. With 8 classes and developed algorithm adapted scaled conjugate gradient back
5 features, the neural network is structured as 5-12-8 with 5 propagation algorithm whose value is observed to be
input layers, 12 hidden layers and 8 output layers. Scaled 0.0025671 in logarithmic scale for each epoch. This value
conjugate gradient back propagation is used to adjust the depicts the least position of local minimum for the goal
weights and bias of the network. Cross entropy and error function. The validation fails are iteration that represent the
percentage are used to quantify the performance of the increase in mean square error of validation data. Multiple
classifier. The classifier is trained for 203 epochs to obtain validation fails indicate that the data is overfitting with
the best results with Cross Entropy corresponding to training data.
4.09945e-0 for training, 11.5387e-0 for validation and
11.50579e-0 for testing. The error percentage corresponds to
6.6% for training, 8.7 % for validation and 8.3 % for testing.
The performance results are depicted in figure 4 to 8.

Figure.4 Performance Plot for Neural Network Training

Figure 4 depicts the performance plot of the training
record for training testing and validation performance. It can
be observed that as the number of epochs increase, the error
keeps on reducing. But at epoch 197 the error on the
validation dataset increases as the neural network overfits
the training data. In order to avoid this, the classifier is
adjusted to stop the training process after six consecutive
validation check fails. Hence the best performance can be
depicted by observing the epoch at least validation check
fail.
Figure.6 Error histogram for Neural Network training
Figure 6 depicts the error histogram for the training
process of the classifier. There are 20 bins (vertical bars)
ranging from -0.8994 to 0.9513 which depicts the total error
range of neural network. As there were 20 bins, each bin
corresponds to a width of 0.092535. These bins represent the
number of samples from the validation data set which have
error.

Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
 Validation & Validation & Validation &
Testing Testing Testing
No. of samples 1568 1568 1568
Network 5 Input, 12 5 Predictors, 5 Predictors,
Structures Hidden, 8 8 Responses. 8 Responses.
Output layers
Epochs 203 -- --
Performance Cross Entropy Cross Cross
Criteria and mean square Validation Validation
error
Training Scaled Quadratic Euclidean
algorithm conjugate Kernel Kernel
gradient
Classification 92.8% 80.3% 56.8%
Accuracy

From the table it can be learnt that the classification

Figure.7 All confusion plot for Neural network classifier.
Figure 7 depicts the confusion matrix for the
classification process of neural network. The green diagonal
represents the truly classified samples and all the samples on
the left and right side of the diagonal are the falsely
classified samples. The overall classification accuracy is
observed to be 92.8%. The training efficiency is 93.4%,
validation efficiency is 91.3% and the testing efficiency is
91.7%.
Figure.8 Receiver Operating Characteristics
Figure 8 visualizes the performance of the neural
network by plotting the receiver operating characteristics
between true positive rate and false positive rate. It is
observed that most of the data is classified towards true
positive rate depicting the effectiveness of the classifier.
From the results its observed that the developed
classification process is very effective in classifying the
module faults. A mere comparison with other classification
techniques is given in table 1.
Table 1. Comparison between classification accuracy of
different classifiers
Parameters ANN SVM KNN
(Artificial (Support (K-Nearest
Neural Vector Neighbour)
Network) Machine)
Training, Training, Training,
accuracy of the developed algorithm using NN techniques is
much better when compared to conventional classification
techniques like Support Vector Machine, and K- Nearest
Neighbor. Also, the method has been found to perform the
classification in the testing phase within fractions of a
second. From these results, the approach confirms
significant improvements on the prior works in the field.
VI. CONCLUSION
Thermography based fault classification algorithm is
developed in this paper. Thermal Images of 8 different
panels (7 faulty and 1 normal) were captured from
169.92kW PV system installed at Faculty of Engineering
and Technology, Jamia Millia Islamia, New Delhi, India, for
experimental purpose. The thermal images were split into
sub-windows to improve the efficiency of the developed
algorithm. The sub-windows are subjected for GLCM
feature extraction to observe the texture features of the
image. The extracted texture features are tabulated for
providing input to the neural network classifier and the
corresponding classes are tabulated for providing target data.
Scaled conjugate gradient back propagation algorithm is
adapted to adjust the weights and bias of the classifier
during training process. The final results of the classifier
depicted 93.4% training and 91.7% testing accuracy which
is better when compared to other classification methods.
These results validate the efficiency and reliability of the
developed algorithm.
REFERENCES
[1] A. Hooshyar, E. F. El-Saadany, and M. Sanaye-Pasand, “Fault
Type Classification in Microgrids Including Photovoltaic DGs,”
IEEE Trans. Smart Grid, vol. 7, no. 5, pp. 2218–2229, 2016.
[2] G. M. Lonkar, P. Klinkhachorn, U. B. Halabe, and H. V. S.
Gangarao, “Automatic Detection of Subsurface Defects Using
Infrared Thermography,” Evaluation, vol. 24, pp. 1469–1477.
[3] S. Deitsch et al., “Automatic Classification of Defective
Photovoltaic Module Cells in Electroluminescence Images,”
2018.
[4] B. V. Dobaria, V. Sharma, and A. Adeshara, “Investigation of
failure and degradation types of solar PV plants in a composite
climate: Abstract after 4–6, years of field operation,” Lect. Notes
Electr. Eng., vol. 435, pp. 227–235, 2018.
[5] S. Meyer et al., “Snail trails: Root cause analysis and test
procedures,” Energy Procedia, vol. 38, pp. 498–505, 2013.
[6] S. G. We, M. One, P. I. D. E. Pid, and P. I. Degradation, “The
five most common problems with solar panels,” pp. 1–2, 2017.
[7] S. Kajari-Schröder, I. Kunze, and M. Köntges, “Criticality of
cracks in PV modules,” Energy Procedia, vol. 27, pp. 658–663,
2012.
[8] A. Bouraiou et al., “Experimental investigation of observed
defects in crystalline silicon PV modules under outdoor hot dry

Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
 climatic conditions in Algeria,” Sol. Energy, vol. 159, no.
December 2017, pp. 475–487, 2018.
[9] M. H. Ali, A. Rabhi, A. El Hajjaji, and G. M. Tina, “Real Time
Fault Detection in Photovoltaic Systems,” Energy Procedia, vol.
111, no. September 2016, pp. 914–923, 2017.
[10] R. Bhoopathy, O. Kunz, M. Juhl, T. Trupke, and Z. Hameiri,
“Outdoor photoluminescence imaging of photovoltaic modules
with sunlight excitation,” Prog. Photovoltaics Res. Appl., vol. 26,
no. 1, pp. 69–73, 2018.
[11] S. Spataru, D. Sera, T. Kerekes, and R. Teodorescu, “Diagnostic
method for photovoltaic systems based on light I-V
measurements,” Sol. Energy, vol. 119, pp. 29–44, 2015.
Authorized licensed use limited to: UNIVERSIDADE DE SAO PAULO. Downloaded on October 04,2021 at 21:38:12 UTC from IEEE Xplore. Restrictions apply.
[12] R. M. Haralick and K. Shanmugam, “Textural Features for Image
Classification,” 1973.
[13] C. Calvin, Gotlieb, and Herbert., “Texture Descriptors Based on
Co-occurrence Matrices CALVIN,” vol. 86, p. 92741R, 2014.
[14] C. Bishop and N. Nasrabadi, “Pattern recognition and machine
learning,” Pattern Recognit., vol. 4, no. 4, p. 738, 2006.
[15] R. Malhotra, “A systematic review of machine learning
techniques for software fault prediction,” Appl. Soft Comput. J.,
vol. 27, pp. 504–518, 2015.
[16] J. J. Shynk, “Performance Surfaces of a Single-Layer
Perceptron,” IEEE Trans. Neural Networks, vol. 1, no. 3, pp.
268–274, 1990.