Electrical Power and Energy Systems 142 (2022) 108380

Contents lists available at ScienceDirect

International Journal of Electrical Power and Energy Systems

journal homepage: www.elsevier.com/locate/ijepes

Reliability evaluation of composite generation and transmission systems via

binary logistic regression and parallel processing
Fabrício S.V. Campos a , Fernando A. Assis a ,∗, Armando M. Leite da Silva b , Alex J.C. Coelho a ,
Rodolfo A.R. Moura a , Marco Aurélio O. Schroeder a
a
Federal University of São João del-Rei - UFSJ, São João del-Rei, MG, Brazil
b Pontifical Catholic University of Rio de Janeiro - PUC-Rio, Rio de Janeiro, RJ, Brazil

ARTICLE INFO ABSTRACT

Keywords: In power system planning studies that involve the search for optimal investments with low risks for the
Binary Logistic Regression power grid, probabilistic reliability assessments provide very useful tools and indices. A challenge faced in
Composite reliability the application of these tools is related to the computational effort demanded by the evaluation process,
Machine learning
especially when the combined effects of failure of generation and transmission equipment are considered.
Non-Sequential Monte Carlo simulation
In this context, the present work proposes a new method for efficient estimation of the main composite
Parallel computational processing
reliability indices by combining Binary Logistic Regression (BLR) technique, a machine learning tool used
for binary data classification, with the non-Sequential Monte Carlo simulation (NS-MCS) method. In addition,
a computational parallelization strategy is incorporated to the proposed method to improve even more the
efficiency of the composite reliability assessment. The performance of the proposed approach is analyzed
by evaluating composite reliability indices for the IEEE-RTS considering two different generation and load
scenarios, in addition to a real large-scale power system. The results obtained are compared with those using
the NS-MCS method in its conventional version.

1. Introduction process is defined as a composite reliability assessment and the static

The reliability evaluation of an electric power grid consists of
probabilistically calculating the risk related to its operational safety
under static conditions. The grid’s ability to satisfy the system load
requirements is known as static adequacy or just adequacy [1]. An
adequacy diagnosis can be performed based on the estimation of relia-
bility indices that measure, for instance, the probability of occurrence
of loss of load, the expected amount of unsupplied energy and the
frequency and duration of these load curtailments in a given period
of analysis [2]. Costs associated to unforeseen load interruptions can
also be estimated. All these indices are able to help engineers and
planners in most technical and economical decisions to be taken during
the expansion and operation planning stages of the power system.
Different probabilistic methods can be used to obtain the reliability
indices. In the evaluation of large electrical systems, emphasis has been
given to the use of Monte Carlo Simulation (MCS) techniques [1,2]. As
using these methods, the performance of a large number of operating
states in meeting the corresponding demand must be known. Clearly,
these performances are defined from the equipment availability, op-
erating strategies and system load level. When both unavailability of
generation and transmission equipment are considered together, the
evaluation of system states involves the use of power flow analysis
tools, coupled with remedial actions algorithms, which ensure the non-
violation of equipment operational limits by minimizing possible load
curtailments. Remedial actions evaluations are, in general, carried out
by optimal power flow (OPF) tools. The computational cost necessary
to estimate the reliability indices depends not only on the dimensions of
the system, but mainly on how rare the failure states are. Failure states
are those in which load curtailments are verified, whether caused by
insufficient generation capacity or by constraints imposed by the trans-
mission capacity of the system network. Therefore, in many situations,
the probabilistic reliability assessment can become very costly from the
computational point of view.
Many efforts have been made to reduce the computational time
required for assessing reliability indices for large power systems. In
these studies, which include recently published works, a wide range
of techniques and methods have been employed, such as: variance
reduction techniques [3–5]; intelligent population-based optimization
methods [6–10]; pattern recognition techniques [11–19]; and parallel
and distributed processing strategies [20–24]. It is worth mentioning

∗ Corresponding author.
E-mail address: fernandoassis@ufsj.edu.br (F.A. Assis).

https://doi.org/10.1016/j.ijepes.2022.108380
Received 29 November 2021; Received in revised form 3 May 2022; Accepted 28 May 2022
Available online 11 June 2022
0142-0615/© 2022 Elsevier Ltd. All rights reserved.
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

the second, the MLKNN (Multi Label K-Nearest Neighbor) is used. A

Nomenclature
method that combines the NS-MCS with the LS-SVM (Least Squares
𝛼𝐶𝑃 Cutoff point value Support Vector Machine) technique is proposed in [12], where a cost
𝛽𝐸[𝐹
̃ ] Estimator coefficient of variation function is responsible for reducing the computational effort existing in
𝛥𝑃 𝑘 State 𝑥𝑘 load shedding (MW) the training stage. In [16], the CNN technique (Convolutional Neural
𝛤𝑖 𝑖th coefficient of the BLR model Network) is used to reduce the computational effort required by the
NS-MCS. This approach also uses the technique known as Transfer
𝜆 Failure rate (occ./yr)
Learning, which consists of a machine learning method responsible
𝜇 Repair rate (occ./yr)
for speeding up the training stage of a neural network. Basically, the
̃ ]
𝐸[𝐹 Estimated expected value
technique makes use of the knowledge obtained during the solution
𝐴𝑑𝑗𝐵𝐿𝑅 Boolean variable related to BLR model fit
of similar problems, making sensitivity studies efficient, for example,
𝐹𝐸[𝐹
̃ ] ̃ ]
Test function of estimator 𝐸[𝐹 which demand the change of parameters and the retraining of the
𝐹𝑁 False negative network. The Logistic Regression technique, which is also part of the
𝐹 𝑂𝑅 Forced Outage Rate method proposed in this work, is used in [18] to pre-classify as ‘‘failure’’
𝐹𝑃 False positive or ‘‘success’’ the operating states of the system sampled via MCS. The
𝐿 Maximum likelihood function adjustment of the classification model is performed at an early stage
𝑁𝑆 Counter of sampled states using states generated by the MCS method when a relaxed tolerance for
𝑁𝑆𝑀𝐴𝑋 Maximum limit for 𝑁𝑆 convergence is considered. Significant reductions in terms of computa-
𝑛𝑎𝑑𝑗 Minimum number of samples to adjust the tional effort and high accuracy in the estimation of reliability indices
BLR model are verified in these works.
𝑁𝐹 𝐴𝐼𝐿 Number of failure states evaluated In addition to the application of machine learning techniques and
𝑁𝑃 𝑆 Number of states for parallel analysis tools, another important strategy that has also been used in a few
𝑁𝑆𝐴𝑀𝑃 Total number of sampled states works, with the aim of speeding up the evaluation of composite re-
𝑁𝑆𝑈 𝐶𝐶 Number of success states evaluated liability indices, concerns the application of parallel and distributed
computing. Among the strategies used, emphasis can be given to the
𝑇 Analysis period (hour)
parallelism performed through the CPU of a single computer [22,24], or
𝑇𝐸𝑉 𝐴𝐿 Total time of the reliability assessment
of a cluster of computers [20,21,23], as well as through the GPU [21].
process
In the present work, a parallelization strategy is proposed for the
𝑇𝑁 True negative
reliability evaluation process that presents good reduction levels of
𝑡𝑜𝑙 Tolerance for NS-MCS method convergence
the computational effort and that, at the same time, can be easily
𝑇𝑃 True positive implemented and does not depend on the programming language.
𝑢𝑖 𝑖th explanatory variable of the BLR model In order to efficiently estimate the main composite reliability indices
̃ ])
𝑉 (𝐸[𝐹 Estimator sample variance of power systems, the present work proposes a simple methodology
𝑥𝑘 𝑘th operating state sampled for the system that employs the Binary Logistic Regression technique (BLR) [26,27]
𝑌𝑘 𝑘th dichotomous response of the BLR model to assist the NS-MCS in classifying and evaluating system operating
BLR Binary Logistic Regression states. The BLR consists of a supervised machine learning tool that,
CC Critical Circuit after being adjusted, allows the modeling of the relationship between
EENS Expected Energy not Supplied (MWh/yr) a binary response variable and one or more explanatory variables
EPNS Expected Power not Supplied (MW/yr) (independent variables), based on a set of categorized data. In addition
LOLE Loss of Load Expectation (h/yr) to the information regarding system generation and load, as considered
LOLP Loss of Load Probability in [18], knowledge on the unavailability of critical equipment in the
MCS Monte Carlo Simulation transmission network is also handled by the proposed BLR model. This
information, which brings great completeness to the state classifica-
NS-MCS Non Sequential Monte Carlo Simulation
tion process, is translated through a new explanatory variable and
allows capturing system failure situations caused by insufficiency of the
transmission network [17], which are not properly treated in [18].
that only a small portion of the published work carries out the evalu- In the evaluation of the proposed composite reliability framework,
ation of its methods for real or large systems (e.g., [4,10,11,15,17,19, after a preliminary step of adjustment of the regression parameters, the
20]). BLR is able to classify the operating states of the system as ‘‘success’’
Emphasis can be given to studies that use pattern recognition tech- and ‘‘failure’’. Only states classified as failure will have their adequacy
niques that, in general, exploit machine learning tools [25], supervised in relation to meeting the load evaluated. Thus, a considerably reduced
or not, combined with the MCS to assist the process of analyzing the number of evaluations of this type is carried out, and, consequently,
static adequacy of the system states. Among the works that employ the computational effort required in the process is very much reduced.
supervised machine learning techniques in composite reliability assess- Furthermore, a computational parallelization strategy for evaluating
ment, it can be mentioned [11–13,16,18,19]. A small but significant sampled states is included into the proposed method, in order to make
group of work uses unsupervised learning techniques [14,15,17], that the composite reliability assessment even more efficient.
is, techniques that seek to cluster input data, not categorized, according The great advantage of the method proposed in this work is related
to their similarities. to the possibility of adjusting the confidence level regarding the clas-
In [11,13], for example, artificial intelligence techniques are applied sification performed by the BLR technique. Since the categorization
in conjunction with the Non-Sequential Monte Carlo Simulation (NS- performed by this technique is based on an output probability, the
MCS) method to classify the states into success or failure, after a threshold that defines the response can be changed by the planner.
training step, avoiding the power flow and OPF evaluations for the The performance of the BLR model is verified in this sense, with an
analysis of part of these states. In the first work, the GMDH (Group analysis of the estimation accuracy of the reliability indices and of the
Method Data Handling) polynomial network is implemented and in computational performance for different threshold values.

2
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

The following contributions can therefore be highlighted: According to Algorithm 1, after sampling of a new operating state
𝑥𝑘 , the test function 𝐹𝐸[𝐹
̃ ] must be calculated for each reliability index
• Proposition of a simple methodology for assessing the reliability to be estimated. In the present work, two indices are estimated: Loss
of composite generation and transmission systems, based on a su- of Load Probability (LOLP) and Expected Power not Supplied (EPNS).
pervised machine learning tool and on the use of a computational The test functions of these indices are given, respectively, by:
parallelization strategy, which can be easily implemented; {
• Consideration of the knowledge related to the unavailability of 1, if 𝑥𝑘 is a failure state
𝐹𝐿𝑂𝐿𝑃 (𝑥𝑘 ) = (1)
critical transmission network equipment, which makes the pro- 0, otherwise
{ 𝑘
posed method more complete and robust, ready to assess the 𝛥𝑃 , if 𝑥𝑘 is a failure state
𝐹𝐸𝑃 𝑁𝑆 (𝑥𝑘 ) = (2)
reliability of practical systems with different topological charac- 0, otherwise
teristics; where a state is called failure when there is a need to shed the load
• Sensitivity analysis in relation to the main parameters that define 𝛥𝑃 𝑘 in the system, either due to insufficient generation or restrictions
the method, which allows a detailed understanding regarding its imposed by the transmission network.
application; The estimated expected value 𝐸[𝐹̃ ] for each index is updated as
• Analysis of the performance of the proposed method in evaluating follows:
the composite reliability of a real and large-scale system. 𝑁𝑆
∑
̃ ]= 1
𝐸[𝐹 𝐹 ̃ (𝑥𝑘 ) (3)
In addition to this introductory section, this paper is made up of 𝑁𝑆 𝑘=1 𝐸[𝐹 ]
four more sections. In Section 2, the main concepts related to the
composite system reliability evaluation are presented. Details regarding It is worth noting that the adequacy analysis of the sampled state 𝑥𝑘 ,
the BLR technique and the method proposed in this work are presented which defines the need or not to carry out load shedding, is responsible
in Section 3. In Section 4, the performance of the proposed method for almost all the computational effort required in the run of the NS-
is extensively analyzed by evaluating the composite reliability indices MCS-based tool. In this step of the tool, a power flow coupled with
for the IEEE-RTS [28], considering two different generation and load an algorithm of remedial actions (i.e., an OPF), for redispatching of
scenarios. A large real system, which corresponds to the southern generation and/or load shedding, capable of assessing the new system
region of Brazil, is also analyzed in this section. The quality of the operating point, must be applied.
BLR adjustments in relation to the accuracy, specificity and sensitivity The NS-MCS convergence can be considered by reaching a maxi-
criteria are also evaluated. Finally, the main conclusions observed in mum number of sampled states (𝑁𝑆𝑀𝐴𝑋 ) or by comparing the coeffi-
the studies are presented in Section 5. ̃ ] given by Eq. (4) with tolerance value 𝑡𝑜𝑙 defined
cient of variation 𝛽𝐸[𝐹
as:
√
2. Composite system reliability evaluation ̃ ])
𝑉 (𝐸[𝐹
𝛽𝐸[𝐹
̃ ] = (4)
̃ ]
𝐸[𝐹
The composite reliability assessment consists of probing, in terms of
̃ ]) = 𝑉 (𝐹 )∕𝑁𝑆 represents the sampling variance of the
where 𝑉 (𝐸[𝐹
static adequacy, the capacity of the electrical system to meet the load,
with its generation and transmission equipment (e.g., transmission lines estimator.
and transformers) subject to the occurrence of failures during opera- Finally, after the convergence of Algorithm 1, two other reliability
tion. In the probabilistic reliability assessment, the diagnosis of system indices, Loss of Load Expectation (LOLE) and Expected Energy not
adequacy can be expressed through reliability indices [1]. Among Supplied (EENS), can be easily obtained for the analysis period 𝑇 using,
the reliability assessment techniques, the NS-MCS [2], described by respectively:
Algorithm 1, is one of the most used. LOLE = 𝑇 × LOLP (5)

EENS = 𝑇 × EPNS (6)

Algorithm 1 : NS-MCS Frequency and duration indices (i.e., loss of load frequency — LOLF,
1: Set the 𝑁𝑆𝑀𝐴𝑋 and 𝑡𝑜𝑙 parameters and loss of load duration — LOLD) can be easily estimated using
̃ ] = 1
2: Set 𝑁𝑆 = 0 and 𝛽𝐸[𝐹 suitable test functions [11], but they are not going to be calculated in
3: while 𝑁𝑆 < 𝑁𝑆𝑀𝐴𝑋 and 𝛽𝐸[𝐹 the present work.
̃ ] < 𝑡𝑜𝑙 do
4: Make 𝑁𝑆 = 𝑁𝑆 + 1
5: Generate a sample state 𝑥𝑘 3. Composite reliability via Binary Logistic Regression
6: Calculate 𝐹𝐸[𝐹
̃ ]
̃ ] In order to reduce the computational effort required in the process
7: Calculate the unbiased estimator 𝐸[𝐹
of evaluating the composite reliability of power systems, this paper
8: Estimate the coefficients of variation 𝛽𝐸[𝐹
̃ ]
proposes an efficient method that combines NS-MCS techniques [1,
9: end while
2] and Binary Logistic Regression (BLR) [26,27] and incorporates a
computational parallelization strategy for the process of evaluating the
operating states of the system. Details regarding the BLR technique and
In the evaluation process with the NS-MCS method, a system operat- the proposed methodology are presented as follows.
ing state 𝑥𝑘 is defined by the combination of the operating states of each
of its components and by a given load level. Each equipment can be 3.1. Binary Logistic Regression
represented by a two-state Markov model, available (Up) or unavailable
(Down), whose transitions are defined by equipment failure rate, 𝜆, and A Logistic Regression basically consists of a generalized linear model
repair rate, 𝜇, usually in occurrences per year. between a response variable and one or more explanatory variables (in-
The load can be considered through a multi-state Markov model dependent variables) based on a set of categorized data, that is, samples
[29] to represent its variation over the period 𝑇 of analysis, usually with a known value of the response variable [26]. This categorized
a year, i.e., 8736 or 8760 h. Each state in this model represents a load data is used to fit a regression model, which can then be employed
level and the transition rates among these states allow modeling the to make predictions. In comparison with other unsupervised machine
intensity with which the load migrates among the different levels. learning classification techniques, it can be said that Logistic Regression

3
F.S.V. Campos et al.
Fig. 1. Logistic function with an explanatory variable.
presents a high degree of flexibility in its assumptions, which makes its
applicability wide.
Logistic Regression models can be classified according to its re-
sponse variable into two basic categories: (i) Binary Logistic Regression
(BLR), when the output is defined by only two classes; (ii) Multinomial
Logistic Regression, when more than two classes are defined in the out-
put. In the present work, the proposed method for assessing composite
reliability indices is based on the BLR technique, where each sample
corresponds to an operating state 𝑥𝑘 of the system and the response
variable is categorized as ‘‘success’’ (𝑌 𝑘 = 1) or ‘‘failure’’ (𝑌 𝑘 = 0).
Therefore, the BLR indicates in its output, based on the observation of 𝑚
explanatory input variables considered (𝑢𝑘1 , … , 𝑢𝑘𝑗 , … , 𝑢𝑘𝑚 ), the probabil-
ity 𝑃 𝑘 of a given sample 𝑥𝑘 belongs to one of the two previously defined
classes. The logistic function employed [26], which allows defining the
probability of the output being classified as ‘‘success’’ (i.e., 𝑌 𝑘 = 1) for
a given state 𝑥𝑘 , is given by:
1 to use some iterative numerical method to identify the coefficients that
𝑃 𝑘 (𝑌 𝑘 = 1) =
−(𝛤0 +𝛤1 𝑢𝑘1 +⋯+𝛤𝑗 𝑢𝑘𝑗 +⋯+𝛤𝑚 𝑢𝑘𝑚 )
1+𝑒
where (𝛤0 , 𝛤1 , … , 𝛤𝑗 , … , 𝛤𝑚 ) are the coefficients associated with the
explanatory variables of the model. A graphical representation of the lo-
gistic function given by Eq. (7), when only one continuous explanatory
variable is considered (i.e., 𝑚 = 1), can be seen in Fig. 1. In this figure,
the blue and red dots stand respectively for ‘‘success’’ and ‘‘failure’’
states.
As, by definition, the BLR response variable is dichotomous, it is
necessary to determine a threshold for the probability that will indicate
the output class. This value is known as the cutoff point (𝛼𝐶𝑃 ), or
threshold, and it is responsible for indicating whether a given sample
𝑥𝑘 is classified as ‘‘success’’ (𝑌 𝑘 = 1) or ‘‘failure’’ (𝑌 𝑘 = 0), as described
by Eq. (8).
In the proposed method, the confidence level in relation to the
classification performed by the adjusted BLR model is controlled by
defining the value of 𝛼𝐶𝑃 . The lower the value set for 𝛼𝐶𝑃 , the higher
the confidence level in the classification is.
{ process. CCs are defined as those circuits whose eventual unavailabil-
1, if 𝑃 𝑘 ≥ 𝛼𝐶𝑃
𝑌𝑘 =
0, otherwise
For the tuning of the BLR, i.e., adjustment of the 𝑚 + 1 coefficients
𝛤𝑗 associated with the explanatory variables in Eq. (7), a relation given
by the logarithm of the maximum likelihood function is used, according
to Eq. (9). Basically, the model coefficients are those that maximize the
International Journal of Electrical Power and Energy Systems 142 (2022) 108380
value of 𝐿.
𝑛𝑎𝑑𝑗 [ ( 𝛤0 +𝛤1 𝑢𝑖 +⋯+𝛤𝑗 𝑢𝑖 +⋯+𝛤𝑚 𝑢𝑖 )
∑ 𝑒 1 𝑗 𝑚
𝐿= (𝑌 𝑖 )𝑙𝑛
𝛤0 +𝛤1 𝑢𝑖1 +⋯+𝛤𝑗 𝑢𝑖𝑗 +⋯+𝛤𝑚 𝑢𝑖𝑚
𝑖=1 1+𝑒
( )]
𝑖 1
+ (1 − 𝑌 )𝑙𝑛 (9)
𝛤 +𝛤 𝑢𝑖 +⋯+𝛤𝑗 𝑢𝑖𝑗 +⋯+𝛤𝑚 𝑢𝑖𝑚
1+𝑒 0 1 1
where 𝑌 𝑖 and (𝑢𝑖1 , … , 𝑢𝑖𝑗 , … , 𝑢𝑖𝑚 ) represent, respectively, the value of
the response variable, which follows a Bernoulli distribution, and the
values of the 𝑚 explanatory variables of the 𝑖th sample of the 𝑛𝑎𝑑𝑗
independent samples used in the adjustment process.
The set of samples used to fit the BLR model is called 𝛺𝑎𝑑𝑗 and must
contain a minimum number of 𝑛𝑎𝑑𝑗 operating states. Notice that this
set must be representative in terms of information diversity, therefore,
a ratio between the diagnoses of ‘‘success’’ and ‘‘failure’’ can be used to
select the states, e.g.: two success states for one failure state.
Since Eq. (9) is characterized as transcendent, it becomes necessary
(7) maximize the function. In the present work, the descending gradient
method is used; one of the most used algorithms to estimate parameters
in machine learning problems [25,30].
As input to the model, explanatory variables can be quantita-
tive (e.g., discrete or continuous) and/or categorical. In the proposed
method for assessing composite reliability indices, the following ex-
planatory variables are considered:
• Unavailable generation (𝑢1 ) - continuous variable given by the
amount of generation capacities unavailable in the operating state
𝑥𝑘 ;
• Load level (𝑢2 ) - continuous variable that corresponds to the
operating state load level 𝑥𝑘 ;
• Critical circuit unavailability (𝑢3 ) - dichotomous variable that in-
forms whether the state 𝑥𝑘 has or not (1 or 0, respectively) at
least one critical circuit (CC) unavailable.
The classification of the circuits of a system as critical must be
carried out in advance, at an initial step of the reliability assessment
(8) ity weakens the systemic operating adequacy. The unavailability of
circuits of this type represents an important piece of information for
the machine learning tool applied to composite reliability evaluations.
Classification of a circuit as critical or not can be performed in advance
by planners, based on their experiences in operating the power grid.
Additionally, deterministic criteria can be used. In this work, it is
4
F.S.V. Campos et al.
Fig. 2. Flowchart of the proposed methodology: NS-MCS combined with the BLR technique with parallelization strategy.
proposed to consider as critical the system circuits that present higher
levels of loading under intact network condition (i.e., no contingencies)
in order to meet the peak load. In this work, it is proposed to consider
as critical the system circuits that present higher levels of loading under
intact network condition (i.e., no contingencies) in order to meet the
peak load. In this case, the loading of each circuit is given by the
ratio between the observed power flow and the corresponding circuit
capacity. For that, the previous evaluation of the OPF is required when
using the method. The run of the OPF for this scenario only helps in
the performance of the power that flows through the circuits and in
the determination of which of them should be classified as ‘‘critical’’.
To perform this classification, the circuits must be ordered according to
their loading observed in the OPF run and an associated capacity limit
(e.g., 80% or 90%) must be applied. In addition, circuits that have at
least one of their terminal buses interconnected to the system by only
two circuits in series (the circuit itself and one more) or just by the
circuit itself are also considered critical. In case of unavailability of this
type of circuit, the bus becomes terminal or isolated from the rest of
the grid and, thus, weakens the system operation. Other criteria could
still be used to classify circuits as critical. For instance, the combined
effect between circuit loading and its failure rate can be considered;
equipment failure rates are input data already available for composite
system reliability assessment.
The definition of the explanatory variables (𝑢1 , 𝑢2 , 𝑢3 ) for the use
of the BLR technique is based on previously performed tests, which
sought to use significant information for the model that did not present
multicollinearity; one of the requirements for the technique to be
properly applied [26].
3.2. Proposed method algorithm
A simplified flowchart representing the steps of the proposed
method is shown in Fig. 2. According to this flowchart, to start running
the proposed method, the CCs of the system must be determined,
according to the definition presented in Section 3.1. As previously
stated, one of the explanatory variables used by the BLR corresponds
exactly to the existence or not of unavailable CCs in the operating
state under analysis. This information is very useful when evaluating
composite reliability power systems with the aid of machine learning
5
International Journal of Electrical Power and Energy Systems 142 (2022) 108380
techniques, given that the prediction of a system failure (i.e., load
shedding) caused by restrictions imposed by the transmission capacity
is much more complex than the one caused by a generation outage.
After defining the CCs, some general parameters must also be set:
maximum limit of states sampled in the evaluation process (𝑁𝑆𝑀𝐴𝑋 ),
tolerance for verifying the convergence in relation to the coefficient
of variation of the estimated index (𝑡𝑜𝑙), number of states to be sam-
pled at each iteration of the algorithm (𝑁𝑃 𝑆 ), and cutoff point value
for classification by the adjusted BLR method (𝛼𝐶𝑃 ). In addition, the
sampled state count variable (𝑁𝑆 = 0), the Boolean variable indicating
that the BLR model has not yet been adjusted (𝐴𝑑𝑗𝐵𝐿𝑅 = ‘‘𝑓 𝑎𝑙𝑠𝑒’’), and
the coefficient of variation of the estimate of the indices greater than
𝑡𝑜𝑙 (𝛽𝐸[𝐹
̃ ] = 1) must be initialized.
Following the previous steps, the iterative section of the proposed
method is started. According to the flowchart in Fig. 2, this section
basically consists of sampling and evaluating new operating states 𝑥𝑘
of the system, updating the estimate of the reliability indices that
one wishes to calculate. Until the BLR model is adjusted (i.e., while
𝐴𝑑𝑗𝐵𝐿𝑅 = ‘‘𝑓 𝑎𝑙𝑠𝑒’’), all new sampled states are evaluated in terms of
static adequacy through power flow with optimization of corrective
measures, as in a conventional NS-MCS-based tool. After the BLR ad-
justment, which occurs when the set 𝛺𝑎𝑑𝑗 has at least 𝑛𝑎𝑑𝑗 samples, only
the states not classified by the BLR model as ‘‘success’’ (i.e., 𝑃 𝑘 (𝑌 𝑘 =
1) < 𝛼𝐶𝑃 ) are evaluated by the power flow and OPF. At this point,
it becomes clear the benefits presented by the use of the BLR model
combined with the NS-MCS method compared to the conventional
NS-MCS presented in Section 2. In the proposed method, all states
classified as ‘‘success’’ by the BLR model, which correspond to most
of the sampled states, are not evaluated in terms of static adequacy,
i.e., through power flow and OPF tools.
It can be observed that the number of states classified as ‘‘success’’
by the BLR model, which is related to the computational speed-up
presented by the method, is directly linked to the value of 𝛼𝐶𝑃 , i.e., the
level of confidence placed in the classification made by the BLR model.
The smaller the value of 𝛼𝐶𝑃 , the greater the confidence given to the
classification of the model and, as a result, the smaller the number of
operating states evaluated by the power flow with OPF-based corrective
measures. A sensitivity study regarding the variation of this parameter
is presented and analyzed in Section 4.
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

Finally, according to the flowchart depicted in Fig. 2, the steps Table 1

presented in the two dashed regions and highlighted in green are Results — Original IEEE-RTS.

performed in a parallelized computational processing environment, Serial NS-MCS Parallel Serial NS-MCS Parallel NS-MCS
(Reference) NS-MCS with BLR with BLR
that is, the 𝑁𝑃 𝑆 states sampled at each main iteration of the method
are evaluated simultaneously, which further speeds up this process. LOLE (h/yr) 9.82 9.82 9.81 9.81
– – (0.17%) (0.17%)
The definition of the 𝑁𝑃 𝑆 parameter must therefore be related to
EENS (MWh/yr) 1154.68 1154.68 1154.92 1154.91
the available computational resource, such as the number of CPU – – (0.02%) (0.02%)
processing cores. In this case, each processor core is responsible for
𝑁𝑆𝐴𝑀𝑃 676,794 676,794 677,077 677,082
evaluating, in terms of adequacy or by BLR model classification, each 𝑁𝑆𝑈 𝐶𝐶 676,033 676,033 185,647 186,619
of the 𝑁𝑃 𝑆 sampled states. 𝑁𝐹 𝐴𝐼𝐿 761 761 760 760
𝑇𝐸𝑉 𝐴𝐿 (s) 185.28 44.76 60.55 23.15
4. Numeric results Speed-up – 4.14 3.06 8.01

4.1. Initial remarks

The performance of the method proposed in Section 3 is analyzed 𝑇𝑃

through composite reliability assessment studies with the IEEE-RTS
system [28], considering two different generation and load scenarios:
original scenario (Original IEEE-RTS) and scenario with double load
and generation amounts (Modified IEEE-RTS or IEEE-MRTS). In the
latter case, system failure states caused by the unavailability of trans-
mission equipment are more recurrent and more difficult to be captured
using pattern recognition techniques [11]. In addition, an equivalent
system from southern Brazil [31], which corresponds to a fairly large
real network, is also evaluated in the study. The Julia programming
language [32], version 1.5.0, is used to implement the algorithms,
with application of the Threads.@threads macro to fulfill the proposed
parallelization strategy. The adequacy analyses through a linearized
DC power flow, coupled with the corrective measures optimization
algorithm, are performed from a library implemented in FORTRAN. An
Intel Core i7-9750H computer, with 6 cores, 8 GB of RAM memory, and
Windows 10–64 bits operating system, is used to carry out all studies.
The definition of CCs is performed following the criteria presented
in Section 3.1. Basically, circuits with loading greater than or equal
to 80% are considered critical in meeting the peak load under intact
network condition, in addition to circuits whose unavailability makes
one of its terminal buses isolated or islanded from the rest of the
network. From these definitions, 6 CCs are checked for the Original
IEEE-RTS. For the IEEE-MRTS, whose transmission network is more
stressed, 13 circuits are considered critical. Finally, for the Brazilian
equivalent system, which has larger dimensions, a total of 104 circuits
are classified as critical.
The performance of the proposed method, both in terms of accuracy
in estimating reliability indices and in terms of computational effort,
is verified in comparison with the conventional NS-MCS method (Al-
gorithm 1, Section 2). A version of the proposed method without the
parallelization strategy (i.e., with 𝑁𝑃 𝑆 = 1) and a parallelized version
of the conventional NS-MCS method are also considered in the study.
Therefore, four different versions of composite reliability assessments
are analyzed in the present studies and they are summarized as follows:
• Serial NS-MCS: Conventional NS-MCS method implemented
without parallelization strategy;
• Parallel NS-MCS: Conventional NS-MCS method with paralleliza-
tion strategy;
• Serial NS-MCS with BLR: Proposed method without paralleliza-
tion strategy;
• Parallel NS-MCS with BLR: Proposed method with paralleliza-
tion strategy.
The performance of the adjusted BLR models in carrying out the
studies is evaluated through the measures of Accuracy, Sensitivity, and
Specificity, given, respectively, by Eqs. (10) to (12) [33]. These mea-
sures are calculated based on model evaluations, which can be of the
type: true positive (TP); false positive (FP); true negative (TN ); or false
negative (FN ).
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (10)
𝑇𝑃 + 𝐹𝑃 + 𝑇𝑁 + 𝐹𝑁
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 = (11)
𝑇𝑃 + 𝐹𝑁
𝑇𝑁
𝑆𝑝𝑒𝑐𝑖𝑓 𝑖𝑐𝑖𝑡𝑦 = (12)
𝐹𝑃 + 𝑇𝑁
Evaluations of type TP and TN correspond to correct classifications
performed by the BLR model for operating states of success and failure,
respectively. When an operating state of failure is classified by the
model as successful, the evaluation is called FP. Finally, an evaluation
is called FN when a successful state is classified by the model as failure.
Basically, the Accuracy represents the proportion of hits of the adjusted
model, compared to the total number of predictions. The Sensitivity
represents the probability of the adjusted model to match its prediction,
given that the state is successful. Finally, the Specificity indicates the
probability of the adjusted model to match its prediction, given that the
state is of failure. Therefore, the closer the values of these performance
measures to unity, the better the fit of the BLR model.
4.2. Original IEEE-RTS
In carrying out the initial composite reliability evaluation studies
for the IEEE-RTS system, with the original scenario of generation and
load [28], the parameters are set as: 𝑁𝑆𝑀𝐴𝑋 = 106 ; 𝑡𝑜𝑙 = 0.05; 𝑁𝑃 𝑆 =
1 and 𝑁𝑃 𝑆 = 6, respectively, for versions with serial and parallel
processing; 𝛼𝐶𝑃 = 0.5; and 𝑛𝑎𝑑𝑗 = 600. The original hourly load curve is
used and the period of analysis is 𝑇 = 1 year (yr). The results obtained
are presented in Table 1, where the Serial NS-MCS version is considered
as reference in the analysis. In addition to the estimated values for
the LOLE and EENS indices, with the respective percentage deviations
calculated from the reference, the following values are also presented
in Table 1 for each version analyzed: the total number of states sampled
in the evaluation process (𝑁𝑆𝐴𝑀𝑃 ); the numbers of failure and success
states evaluated in terms of system adequacy (𝑁𝑆𝑈 𝐶𝐶 and 𝑁𝐹 𝐴𝐼𝐿 ,
respectively); the total time required in the process (𝑇𝐸𝑉 𝐴𝐿 ); and the
speed-up evaluation against the reference version. It is possible to
observe that the value of 𝑁𝑆𝐴𝑀𝑃 is equal to the sum of the values of
𝑁𝑆𝑈 𝐶𝐶 and 𝑁𝐹 𝐴𝐼𝐿 only for the Serial and Parallel NS-MCS versions, in
which the system is evaluated in terms of static adequacy for all states
sampled. For the Serial and Parallel NS-MCS with BLR versions, the
difference between the 𝑁𝑆𝐴𝑀𝑃 value and the sum of the 𝑁𝑆𝑈 𝐶𝐶 and
𝑁𝐹 𝐴𝐼𝐿 values corresponds to the number of states in which the system
is not evaluated, these being the states classified as successful by the
BLR model.
From the results in Table 1, it can be seen that the LOLE and
EENS indices obtained using the BLR proposed approach (both in the
serial and in the parallelized case) present a very small difference in
relation to the conventional Serial NS-MCS, being 0.17%, for the LOLE
index, the highest percentage deviation verified. For the EENS index the
percentage deviation is close to zero, only 0.02%, showing that failure
states not captured by the BLR model (i.e., states with FP evaluations)
have lower load curtailments.
In terms of computational performance, on average, the use of
the BLR technique allows a reduction of 72.47% in the number of

6
F.S.V. Campos et al.
Fig. 3. Percentage deviations of LOLE and EENS indices with variation of the parameter 𝑛𝑎𝑑𝑗 - Original IEEE-RTS.
Fig. 4. BLR model performance measures and speed-ups with variation of the parameter 𝑛𝑎𝑑𝑗 - Original IEEE-RTS.
adequacy assessments for successful states (i.e., reduction of 𝑁𝑆𝑈 𝐶𝐶 ),
which certainly justifies the benefits of its application. Furthermore, the
proposed method is able to estimate the reliability indices up to 8.01
times faster than the conventional version of the NS-MCS. Highlight
for the Parallel NS-MCS version, which presents a speed-up greater
than the version without parallelization of the proposed method (Serial
NS-MCS with BLR), 4.14 against 3.06.
In order to analyze the robustness of the proposed method in
relation to parameter 𝑛𝑎𝑑𝑗 , which defines the number of minimum states
to fit the BLR model, new tests are performed with the Parallel NS-MCS
with BLR version. In addition to the value 600, four other values of
𝑛𝑎𝑑𝑗 are considered in these tests: 300, 450, 750, and 900. In Fig. 3,
the calculated percentage deviations are presented for the LOLE and
EENS indices in relation to the results of the conventional NS-MCS
method. Reduced deviation values are checked in all cases. Fig. 4 shows
the performance measures of the BLR model (Accuracy, Sensitivity, and
Specificity) and the speed-up obtained in each test.
The main impact observed with the variation of the parameter 𝑛𝑎𝑑𝑗
concerns the computational time gain. As can be seen, the smaller the
number of states selected to perform the adjustment of the BLR model,
the greater the speed-up values, reaching 9.96 for 𝑛𝑎𝑑𝑗 = 300. Regarding
the adjustments of the BLR model, excellent results can be verified in
all tests. These results show the good quality of the input information
(model explanatory variables) used and the robustness of the proposed
method. It is verified that the smallest proportion of correct predictions
7
International Journal of Electrical Power and Energy Systems 142 (2022) 108380
(i.e., Accuracy) and the smallest probability of correctness in identifying
a successful state (i.e., Sensitivity) are equal to 98.94%, when 𝑛𝑎𝑑𝑗 = 600.
It is also possible to observe that, in all cases, Accuracy and Sensitivity
exhibit values very close to each other. This happens because the
number of success states sampled in the system evaluation is much
higher than the number of failure states. Thus, the total number of
predictions (i.e., number of states sampled after the adjustment of the
BLR model) is very close to the number of successful states. Finally,
emphasis can also be given to the probability of correct predictions by
the model when a state is of failure (i.e., measure of Specificity), with
an average of 99.58% in all cases.
4.3. IEEE-MRTS
Initial tests with the same parameter settings considered for the
Original IEEE-RTS are also carried out for the IEEE-MRTS system.
Table 2 presents the results obtained for the four reliability evaluation
versions under study. In comparison with the results of Table 1, higher
values for LOLE and EENS indices can be verified. This is due to
the definition of this new analysis scenario, in which the increase in
the amounts of generation and load, keeping the same transmission
network capacity, makes the event of failure less rare. This fact also
justifies the smaller gains in terms of computational performance veri-
fied for the IEEE-MRTS system. However, even so, a speed-up of 3.72
is obtained using the method proposed in this work.
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

Fig. 5. Percentage deviations of LOLE and EENS indices with variation of the parameter 𝜎𝐶𝑃 - IEEE-MRTS.

Table 2 decrease in the confidence level in relation to the model classification,

Results - IEEE-MRTS.
greater probabilities of making a correct prediction for failure states
Serial NS-MCS Parallel Serial NS-MCS Parallel NS-MCS (i.e., higher values of Specificity) are checked. This result is expected,
(Reference) NS-MCS with BLR with BLR
since fewer failure states are mistakenly classified as successful. How-
LOLE (h/yr) 37.31 37.31 36.19 36.19
ever, it can be observed that, with higher values of 𝜎𝐶𝑃 , a greater
– – (3.00%) (3.00%)
EENS (MWh/yr) 5458.80 5458.80 5413.99 5413.99
number of states needs to be evaluated in terms of static adequacy,
– – (0.82%) (0.82%) which increases the number of evaluated successful states (𝑁𝑆𝑈 𝐶𝐶 )
𝑁𝑆𝐴𝑀𝑃 257,322 257,322 261,178 261,180 and, consequently, decreases the Accuracy and Sensitivity values. In
𝑁𝑆𝑈 𝐶𝐶 256,223 256,223 51,197 53,634 addition, there is a reduction in speed-ups verified with an increase
𝑁𝐹 𝐴𝐼𝐿 1099 1099 1082 1082 in this parameter. However, even when 𝜎𝐶𝑃 = 0.8, a speed-up of 3.44
𝑇𝐸𝑉 𝐴𝐿 (s) 76.39 27.70 29.82 20.55 is achieved using the proposed method.
Speed-up – 2.76 2.56 3.72
Finally, in order to demonstrate the importance of the dichotomous
explanatory variable ‘‘critical circuit unavailability’’ (𝑢3 ) for the BLR
model, a new test is performed with the IEEE-MRTS system. It is
It is important to note that the computational performance gains worth noting that operational failure states caused by the unavailability
resulting from the use of the parallelization strategy in relation to the of transmission network elements are more recurrent in this system
non-parallelized versions are different for the IEEE-RTS Original and and, therefore, the evaluation of its reliability through general pattern
IEEE-MRTS systems. Regarding the Serial NS-MCS with BLR version, recognition techniques becomes more complex. In the new test, the
the Parallel NS-MCS with BLR version presents, for example, speed-up same parameter adjustments of the methodology used to obtain the
gains equal to 8.01/3.06 ≅ 2.62 and 3.72/2.56 ≅ 1.45 for the IEEE-RTS results presented in Table 2 are used, with only explanatory variables
Original and IEEE-MRTS systems, respectively. The difference between 𝑢1 and 𝑢2 considered for the BRL model.
these gains is due to the greater proportion of failure states in the The results presented by the new test are shown in Table 3. For
second case, which results in longer evaluation times for each set of the two evaluation versions, it is possible to verify an increase in
𝑁𝑃 𝑆 states sampled in the iterations of the proposed method, as shown the percentage deviations for the reliability indices compared to the
in the flowchart in Fig. 2. reference version (Serial NS-MCS), with emphasis on the LOLE index.
In addition, higher percentage deviations for LOLE and EENS indices According to Eqs. (1) and (5), the calculation of the LOLE index is
are verified with the new system scenario, with 4.27% being the highest directly related to the proportion between the number of failure states
value. However, as verified for the Original IEEE-RTS system, the per- and the total number of states sampled. In Table 3, compared to
centage deviations presented by the EENS index with the IEEE-MRTS Table 2, there is an increase in the number of states sampled (𝑁𝑆𝐴𝑀𝑃 )
system are smaller than those verified for the LOLE index. Furthermore, and a decrease in the number of states defined as failure (𝑁𝐹 𝐴𝐼𝐿 ).
significant reductions in the evaluations of successful states (𝑁𝑆𝑈 𝐶𝐶 ) In other words, without the use of the explanatory variable 𝑢3 , the
are also verified in this case, on average, this reduction is 79.54% for BLR model presents a higher number of false positives (FP), which
the methods that utilize the BLR. corresponds to the wrong classification of fault operating states. This
A sensitivity study of the proposed approach (Parallel NS-MCS with observation is confirmed by the Specificity measure, which also presents
BLR) in relation to the parameter 𝜎𝐶𝑃 is carried out for the IEEE-MRTS lower values equal to 0.83 and 0.84, respectively, for the Serial and
system. Values below and above 0.5 are considered in this study. In Parallel NS-MCS with BLR versions.
Fig. 5 are shown the percentage deviations presented by the LOLE and In short, it is confirmed that the use of information related to
EENS indices in relation to the reference method (Serial NS-MCS) for unavailability in the transmission network, such as the explanatory
different values of 𝜎𝐶𝑃 . The performance measures of the BLR model variable 𝑢3 in this work, contributes significantly to the effectiveness of
and the verified Speed-ups are shown in Fig. 6. the process of evaluating the composite reliability of electrical systems
According to these new results, the value of 0.5 for the parameter through of BLR techniques.
𝜎𝐶𝑃 presents the best cost–benefit relation, when observing holistically
the performance of the proposed method. Values below this limit cause 4.4. Brazilian system
an increase in the percentage deviations presented by the LOLE and
EENS indices, which represents a worsening in the estimation accuracy. In order to verify the scalability of the proposed method in re-
On the other hand, values of 𝜎𝐶𝑃 greater than 0.5, which represents a lation to the dimensions of the systems under study, the composite

8
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

Fig. 6. BLR model performance measures and speed-ups with variation of the parameter 𝜎𝐶𝑃 - IEEE-MRTS.

Table 3 the academic systems. However, it is important to mention that these

New results (without using the explanatory variable 𝑢3 ) - IEEE-MRTS.
deviations are still considered acceptable, since an overall assessment
Serial NS-MCS with BLR Parallel NS-MCS with BLR of the system is performed with a single BLR model adjusted to capture
LOLE (h/yr) 31.71 32.07 the pattern of the entire network. Better levels of accuracy in estimating
(15.01%) (14.04%)
the indices can be obtained when specific regions of the network
EENS (MWh/yr) 5364.20 5357.32
(1.73%) (1.86%) (e.g., areas, zones or single buses) are evaluated. In these cases, the BLR
model is tuned to a specific region, which has a much better defined
𝑁𝑆𝐴𝑀𝑃 275,249 271,338
𝑁𝑆𝑈 𝐶𝐶 58,226 62,531 operating pattern. Reliability studies carried out for specific subsystems
𝑁𝐹 𝐴𝐼𝐿 999 996 provide more adequate information for risk assessment and decision
𝑇𝐸𝑉 𝐴𝐿 (s) 30.39 20.97 making about the network [4,36].
Speed-up 2.51 3.64
Regarding the computational performance, it is possible to observe
that the reliability indices are estimated by the proposed method with
Table 4 considerably reduced computation times in relation to the reference
Results — Brazilian system. method. In the case of the Parallel NS-MCS with BLR version, the rates
Serial NS-MCS Parallel Serial NS-MCS Parallel NS-MCS are estimated with a saving of 1.98 h (speed-up of 9.74), even though
(Reference) NS-MCS with BLR with BLR
it is necessary to sample 50.046 more states for convergence (𝑁𝑆𝐴𝑀𝑃 ).
LOLE (h/yr) 5.97 5.97 5.62 5.62 In terms of goodness-of-fit of the BLR model, for the Serial and
– – (5.82%) (5.82%)
Parallel versions, the performance measures Accuracy, Sensitivity, and
EENS (MWh/yr) 2517.47 2517.47 2435.98 2435.97
– – (3.24%) (3.24%) Specificity presented mean values equal to, respectively, 0.96, 0.96,
𝑁𝑆𝐴𝑀𝑃 1,237,968 1,237,968 1,288,010 1,288,014
and 0.93. Therefore, good expectations regarding the classifications
𝑁𝑆𝑈 𝐶𝐶 1,237,122 1,237,122 296,583 291,798 performed by the model are projected, which is proven by the accuracy
𝑁𝐹 𝐴𝐼𝐿 846 846 829 829 of the obtained reliability indices. In short, it is possible to verify
𝑇𝐸𝑉 𝐴𝐿 (h) 2.21 0.44 0.61 0.23 that the good performance of the proposed method, both in terms of
Speed-up – 4.99 3.66 9.74
quality in the estimation of reliability indices and in relation to the
computational gains and adjustments of the BLR model, are maintained
for the real system, whose evaluation is indeed more complex.
reliability evaluation is also carried out for a real and reasonably large
system with voltage levels of 230 kV and 525 kV, which corresponds 5. Conclusions
to a network equivalent of the southern region of Brazil, whose full
transmission grid is represented in Fig. 7. This system is composed This work proposes an efficient method to evaluate reliability in-
of 242 buses connected by 489 circuits, among transmission lines dices of electrical power systems, when the combined failures of gen-
and transformers [31]. Deterministic and stochastic system data can eration and transmission equipment are represented. In this method,
be found in [34]. The results obtained from the four evaluation ver- probabilistic reliability indices are estimated from the combination
sions considered are presented in Table 4. The same initial parameter of Binary Logistic Regression (BLR) techniques and non-Sequential
adjustments used in the previous subsections are applied. Monte Carlo simulation (NS-MCS). Basically, a BLR model is fitted and
For this new system, longer times are required in carrying out the employed to classify states during the assessment process, reducing the
static adequacy analysis (i.e., standard and optimal power flow with number of adequacy analyzes based on standard and optimal power
remedial measures), which is due to its larger dimension. Therefore, flows with corrective measures. In addition, the method includes a
there are higher values of 𝑇𝐸𝑉 𝐴𝐿 , now given in hours, compared to the simple and effective strategy for parallelizing the process of evaluating
Original IEEE-RTS and IEEE-MRTS systems. Furthermore, for conver- the system operating states, which allows for an even greater upgrading
gence of the evaluation process according to the tolerance defined for in its performance in terms of computational effort.
the coefficient of variation 𝛽𝐸[𝐹
̃ ] (i.e., 𝑡𝑜𝑙 = 0.05), a number largest of Results obtained in the evaluation of the reliability of different
states (𝑁𝑆𝐴𝑀𝑃 ) needs to be sampled. power systems, with different generation and load scenarios and dimen-
The percentage deviations verified in Table 4 for the LOLE and sions, show the benefits of the proposed method. Composite reliability
EENS indices estimated by the proposed method in comparison with indices are estimated with accuracy within reduced computational
the reference method are, on average, greater than those verified for efforts compared to conventional evaluation techniques. Furthermore,

9
F.S.V. Campos et al. International Journal of Electrical Power and Energy Systems 142 (2022) 108380

Fig. 7. Brazilian electrical transmission system [35].

performance measures of the adjusted BLR model confirm the quality Conceptualization, Methodology, Software. Rodolfo A.R. Moura: Con-
of the input information (explanatory variables) used. It should be ceptualization, Software, Validation. Marco Aurélio O. Schroeder:
noted that the proposed method is robust to the variation of the main Conceptualization, Software, Validation.
adjustment parameters, since the greatest impacts verified from the
sensitivity studies presented are related to the computational gain, with Declaration of competing interest
good levels of accuracy being maintained in the estimations of reliabil-
ity indices. In addition, it is verified that in the composite reliability The authors declare that they have no known competing finan-
assessment of power systems through pattern recognition tools, such cial interests or personal relationships that could have appeared to
as the proposed BLR method, the use of information related to trans- influence the work reported in this paper.
mission network outages is very relevant. The lack of this information
can cause significant losses in the accuracy of the results, particularly
Acknowledgments
with more ‘‘stressed’’ transmission networks. The application of this
information, built on the basis of network operational experiences,
This study was partially supported by the Coordination for the
can contribute more extensively to the proper classification of states
Improvement of Higher Education, CAPES, Brazil. It was also supported
through techniques such as the BLR.
by INERGE (National Institute of Electric Energy), CNPq (National
Finally, the benefits of applying the method proposed in this work
Council for Scientific and Technological Development), and FAPEMIG
are significantly intensified when the evaluation of composite reliability
(Foundation for Research of the State of Minas Gerais). The authors
assessment is inserted in the planning scope of power grids, where it
would like to thank Dr. José Filho C. Castro, today with the Electrical
is necessary to carry out a large number of evaluations required by
Engineering Department, Federal University of Pernambuco, for his
decision making processes.
valuable contributions in providing data for the construction of the
Brazilian equivalent power grid as a previous member of the Energy
CRediT authorship contribution statement
Research Office (EPE — Empresa de Pesquisa Energética) in Brazil.

Fabrício S.V. Campos: Conceptualization, Methodology, Software,

References
Validation, Formal analysis, Writing – original draft. Fernando A.
Assis: Conceptualization, Methodology, Software, Validation, Formal
[1] Billinton R, Li W. Reliability assessment of electrical power systems using Monte
analysis, Writing – original draft, Writing – review & editing. Armando Carlo methods. New York, NY: Springer US; 1994.
M. Leite da Silva: Conceptualization, Methodology, Formal analysis, [2] Pereira MV, Balu NJ. Composite generation/transmission reliability evaluation.
Writing – original draft, Writing – review & editing. Alex J.C. Coelho: Proc IEEE 1992;80(4):470–91.

10
F.S.V. Campos et al.
[3] Zhaohong B, Xifan W. Studies on variance reduction technique of Monte Carlo
simulation in composite system reliability evaluation. Electr Power Syst Res
2002;63(1):59–64.
[4] González-Fernández RA, Leite da Silva AM, Resende LC, Schilling MT. Composite
systems reliability evaluation based on Monte Carlo simulation and cross-entropy
methods. IEEE Trans Power Syst 2013;28(4):4598–606.
[5] Shu Z, Jirutitijaroen P, Leite da Silva AM, Singh C. Accelerated state evalua-
tion and latin hypercube sequential sampling for composite system reliability
assessment. IEEE Trans Power Syst 2014;29(4):1692–700.
[6] Earla R, Mitra J, Patra S. A particle swarm based method for composite system
reliability analysis. In: North American power symposium. 2004.
[7] Miranda V, Carvalho LM, Rosa MA, Leite da Silva AM, Singh C. Improving power
system reliability calculation efficiency with EPSO variants. IEEE Trans Power
Syst 2009;24(4):1772–9.
[8] Green II RC, Wang L, Alam M, Singh C. Intelligent state space pruning for
Monte Carlo simulation with applications in composite power system reliability.
Eng Appl Artif Intell 2013;26(7):1707–24.
[9] Ashok Bakkiyaraj R, Kumarappan N. Optimal reliability planning for a composite
electric power system based on Monte Carlo simulation using particle swarm
optimization. Int J Electr Power Energy Syst 2013;47:109–16.
[10] Benidris M, Elsaiah S, Mitra J. Power system reliability evaluation using a state
space classification technique and particle swarm optimisation search method.
IET Gener Transm Distrb 2015;9(14):1865–73.
[11] Leite da Silva AM, Resende LC, Manso LAF, Miranda V. Composite reliability
assessment based on Monte Carlo simulation and artificial neural networks. IEEE
Trans Power Syst 2007;22(3):1202–9.
[12] Pindoriya NM, Jirutitijaroen P, Srinivasan D, Singh C. Composite reliability
evaluation using Monte Carlo simulation and least squares support vector
classifier. IEEE Trans Power Syst 2011;26(4):2483–90.
[13] Urgun D, Singh C. A hybrid monte carlo simulation and multi label classification
method for composite system reliability evaluation. IEEE Trans Power Syst
2018;34(2):908–17.
[14] Luo X, Singh C, Patton A. Power system reliability evaluation using self
organizing map. In: 2000 IEEE power engineering society winter meeting. 2,
IEEE; 2000, p. 1103–8.
[15] Song Y, Bu G, Zhang R. A fast method for probabilistic reliability assessment of
bulk power system using FSOM neural network as system states filters. In: 2005
IEEE/PES transmission & distribution conference & exposition: Asia and Pacific.
IEEE; 2005, p. 1–6.
[16] Urgun D, Singh C. Composite system reliability analysis using deep learning
enhanced by transfer learning. In: International conference on probabilistic
methods applied to power systems. IEEE; 2020, p. 1–6.
[17] Assis FA, Coelho AJC, Rezende LD, Leite da Silva AM, Resende LC. Unsupervised
machine learning techniques applied to composite reliability assessment of power
systems. Int Trans Electr Energy Syst 2021;31(11):e13109.
11
International Journal of Electrical Power and Energy Systems 142 (2022) 108380
[18] Poudel S, Raj Karki N. Composite system adequacy assessment using Monte Carlo
simulation and logistic regression classifier. In: 2021 1st Odisha international
conference on electrical power engineering, communication and computing
technology(ODICON). Bhubaneswar, Odisha, India; 2021.
[19] Kamruzzaman M, Bhusal N, Benidris M. A convolutional neural network-based
approach to composite power system reliability evaluation. Int J Electr Power
Energy Syst 2022;135:107468.
[20] Borges CL, Falcao DM, Taranto GN. Cluster based power system analysis
applications. In: Proceedings IEEE international conference on cluster computing.
IEEE; 2000, p. 193–200.
[21] Borges CL, Falcao DM, Mello JCO, Melo ACG. Composite reliability evaluation
by sequential Monte Carlo simulation on parallel and distributed processing
environments. IEEE Trans Power Syst 2001;16(2):203–9.
[22] Green RC, Wang L, Alam M. Composite power system reliability evaluation using
support vector machines on a multicore platform. In: The 2011 international joint
conference on neural networks. IEEE; 2011, p. 2586–92.
[23] Ge H, Asgarpoor S. Parallel Monte Carlo simulation for reliability and cost
evaluation of equipment and systems. Electr Power Syst Res 2011;81(2):347–56.
[24] Chen F, Li F, Feng W, Wei Z, Cui H, Liu H. Reliability assessment method of
composite power system with wind farms and its application in capacity credit
evaluation of wind farms. Electr Power Syst Res 2019;166:73–82.
[25] Alpaydin E. Introduction to machine learning. 2014.
[26] James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning,
Vol. 112. Springer; 2013.
[27] Hosmer Jr DW, Lemeshow S, Sturdivant RX. Applied logistic regression, Vol.
398. John Wiley & Sons; 2000.
[28] Subcommittee PM. IEEE reliability test system. IEEE Trans Power Appar Syst
1979;PAS-98(6):2047–54.
[29] Leite da Silva AM, Manso LAF, Mello JCO, Billinton R. Pseudo-chronological
simulation for composite reliability analysis with time varying loads. IEEE Trans
Power Syst 2000;15(1):73–80.
[30] Russell S, Norvig P. Artificial intelligence: A modern approach. 3ed ed.. Prentice
Hall; 2010.
[31] Assis FA, Leite da Silva AM, Manso LAF, Castro JFC. Transmission expansion
planning of large power networks via constructive metaheuristics with secu-
rity constraints and load uncertainty analysis. Int Trans Electr Energy Syst
2021;31:e12168.
[32] Julia. The julia programming language. 2021, URL https://julialang.org/
(Accessed on: 01 sep. 2021).
[33] Provost F, Kohavi R. Glossary of terms. J Mach Learn 1998;30(2–3):271–4.
[34] Brazilian system reliability data. 2021, URL https://github.com/
ProfFernandoAssis/ReliabilityData (Accessed on: 23 nov. 2021).
[35] Brazilian interconnected power system maps. 2021, URL http://www.ons.org.br/
paginas/sobre-o-sin/mapas (Accessed on: 23 nov. 2021).
[36] Anders G, Hamoud G, Leite da Silva AM, Manso LAF. Optimal outage scheduling
- example of application to a large power system. Int J Electr Power Energy Syst
2003;25(8):607–14.