See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/344243595

Artificial intelligence techniques for stability analysis and control in smart

grids: Methodologies, applications, challenges and future directions

Article in Applied Energy · September 2020

DOI: 10.1016/j.apenergy.2020.115733

CITATIONS READS

0 194

8 authors, including:

Wei Yao
Huazhong University of Science and Technology
148 PUBLICATIONS 1,823 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Modeling and Analysis of Large-Scale Photovoltaic Plant Equipped with Panel Level DC Optimizers View project

All content following this page was uploaded by Wei Yao on 14 September 2020.

The user has requested enhancement of the downloaded file.

 Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

Highlights

Artificial intelligence techniques for stability analysis and control Applied Energy xxx (xxxx) xxx
in smart grids: Methodologies, applications, challenges and future
directions
Zhongtuo Shi, Wei Yao∗, Zhouping Li, Lingkang Zeng, Yifan Zhao, Runfeng Zhang, Yong Tang, Jinyu Wen

• An overview of state-of-the-art artificial intelligence (AI) is provided.

• A review of AI’s applications to smart grid stability and control is undertaken.
• This paper identifies some major challenges these applications face in practice.
• Suggestions and potentially important future research directions are put forward.

Graphical abstract and Research highlights will be displayed in online search result lists, the online contents
list and the online article, but will not appear in the article PDF file or print unless it is mentioned in the
journal specific style requirement. They are displayed in the proof pdf for review purpose only.
 Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

Artificial intelligence techniques for stability analysis and control in smart

grids: Methodologies, applications, challenges and future directions
Zhongtuo Shi a , Wei Yao a ,∗, Zhouping Li a , Lingkang Zeng a , Yifan Zhao a , Runfeng Zhang a ,
Yong Tang b , Jinyu Wen a
a
State Key Laboratory of Advanced Electromagnetic Engineering and Technology, School of Electrical and Electronic Engineering, Huazhong University of Science
and Technology, Wuhan, 430074, China
b China Electric Power Research Institute, Beijing, 100192, China

ARTICLE INFO ABSTRACT

Keywords: Smart grid is the new trend for clean, sustainable, efficient and reliable energy generation, delivery and
Smart grid use. To ensure stable and secure operation is essential for the smart grid, which needs effective stability
Artificial intelligence analysis and control. As the smart grid has evolved through a growing scale of interconnection, increasing
Stability analysis
integration of renewable energy, widespread operation of direct current power transmission systems, and
Stability control
liberalization of electricity markets, the stability characteristics of it are much more complex than the past.
Machine learning
Due to these changes, conventional stability analysis and control approaches have a series of drawbacks in
terms of speed, effectiveness and economy. On the contrary, the emerging artificial intelligence (AI) techniques
provide powerful and promising tools for stability analysis and control in smart grids and have attracted
growing attention. This paper aims to give a comprehensive and clear picture of recent advances in this
research area. First, we present a general overview of AI, including its definitions, history and state-of-
the-art methodologies. And then, this paper gives a comprehensive review of its applications to security
assessment, stability assessment, fault diagnosis, and stability control in smart grids. These applications have
achieved impressive results. Nevertheless, we also identify some major challenges these applications face in
practice: high requirements on data, imbalanced learning, interpretability of AI, difficulties in transfer learning,
the robustness of AI to communication quality, and the robustness against attack or adversarial examples.
Furthermore, we provide suggestions for potential important future investigation directions to overcome these
challenges and bridge the gap between research and practice.

1 1. Introduction more effective, efficient, economical, and sustainable way, has come 16
into being [4,5]. 17
2 Electricity has been one of the most important and the most widely The smart grid is defined as a stable, secure, reliable, resilient, 18
3 used forms of energy since the 19th century. And the electric power sustainable and efficient electric energy system that uses informa- 19
4 grid, which deliveries electric power from generation to consuming tion, two-way, cyber-secure communication technologies, and compu- 20
5 devices, was cited as the supreme engineering achievement of the 20th tational intelligence in an integrated fashion across electricity genera- 21
6 century [1] and became fundamental to modern society. Despite that, tion, transmission, substations, distribution and consumption [4,6]. The 22
7 the power grid is still developing. In recent years, the concerns of fossil smart grid allows for high penetration of stochastic and intermittent 23
8 fuel depletion, climate change and global warming, combined with the
renewable energy for reduced pollution [7] and brings many additional 24
9 gradual decline in costs, are driving the increasing integration of renew-
benefits [8,9]. Bidirectional energy flows are possible in smart grids, 25
10 able energy, such as wind and solar energy, to power grids. However,
where people can not only consume but also sell energy. And micro- 26
11 the variable and uncertain nature of renewable energy has brought
grids, electric cars and energy storage devices can charge from/to smart 27
12 significant challenges to power grids [2] since it is not ‘‘dispatchable’’—
13 the output power of renewable energy cannot be controlled. For this grids. Market enabling is another important characteristic of smart 28
14 reason, smart grid, an innovative electric energy system that facilitates grids for efficient energy use and improved economic welfare. These 29
15 large amounts of renewable energy integration [3] and operates in a new features of smart grids are illustrated in Fig. 1. Among them, secure 30

∗ Corresponding author.
E-mail address: w.yao@hust.edu.cn (W. Yao).

https://doi.org/10.1016/j.apenergy.2020.115733
Received 25 May 2020; Received in revised form 19 July 2020; Accepted 18 August 2020
 Z. Shi et al.

List of abbreviations SAC Soft actor critic

SCADA Supervisory control and data acquisition
A2C Advantage actor critic
SDAE Stacked denoising autoencoder
A3C Asynchronous advantage actor critic
SFS Sequential forward search
AdaBoost Adaptive boosting
SGC Smart generation control
ADASYN Adaptive synthetic
SMOTE Synthetic minority oversampling technique
ADP Adaptive dynamic programming
SSA Static security assessment
AE Autoencoder
SVC Static Var compensator
AGC Automatic generation control
SVG Static Var generator
AI Artificial intelligence
SVM Support vector machine
ANN Artificial neural network
TD3 Twin delayed deep deterministic policy
AVC Automatic voltage control
gradients
Bagging Bootstrap aggregating
TDS Time-domain simulation
CCT Critical clearing time
TEF Transient energy function
CNN Convolutional neural network
TLBO Teaching–learning based optimization
DAE Deep autoencoder
TRPO Trust region policy optimization
DBN Deep belief network
TSA Transient stability assessment
DC Direct current
TVSI Transient voltage collapse index
DDPG Deep deterministic policy gradient
VAE Variational autoencoder
DL Deep learning
VSA Voltage stability assessment
DNN Deep neural network
WADC Wide-area damping controller
DQN Deep Q network
WAMS Wide-area measurement system
DRL Deep reinforcement learning
WT Wavelet transform
DSA Dynamic security assessment
XGBoost Extreme gradient boosting
DT Decision tree
EEAC Extend equal area criterion
ELM Extreme learning machine
EMS Energy management system of the smart grid different and much more complex than that of con- 3
ENN Edited nearest neighbors ventional power grids. Under these conditions, conventional stability 4
FSA Frequency stability assessment analysis and control approaches became ineffective. Alternatively, with 5
GAN Generative adversarial network its recent advances, artificial intelligence (AI) has the potential to cope 6
GBDT Gradient boosting decision tree with these changes and provides promising tools for meeting the high 7
GCN Graph convolutional network requirement of security and stability of the smart grid [10]. 8
GNN Graph neural network Since its birth, AI has been studied to make computers capable of 9
GrHDP Goal representation heuristic dynamic pro- thinking and acting like human beings [11]. After several booms and 10
gramming winters, AI has experienced a resurgence in the 21st century follow- 11
HMM Hidden Markov model ing the advances in computation power, data volume and theoretical 12
HVDC High voltage direct current research. And the new generation AI has been an essential part of 13
industry, economy and life [12], helping to solve many challenging 14
IoT Internet of Things
problems in natural language processing [13,14], computer vision [15, 15
LLI Local linear interpreter
16], autonomous driving [17,18], go [19–21], et cetera. 16
LSTM Long short-term memory
The smart grid is also benefited from the innovations of AI [22– 17
MCD Multi-criteria decision making
24], and its high requirements for stability, security and reliability are 18
MDP Markov decision process
supported by AI techniques. It is known that ensuring them requires 19
ML Machine learning timely and effective stability analysis and automatic control. In the past 20
MLP Multi-layer perceptron decades, a large amount of research has been carried out on the appli- 21
MORL Multiobjective reinforcement learning cations of AI techniques such as machine learning, deep learning and 22
PCA Principal component analysis reinforcement learning to stability analysis and control. These applica- 23
PGM Probabilistic graphical model tions include security assessment, stability assessment, fault diagnosis, 24
PID Proportional integral derivative stability control, et cetera. For the reason that AI improves accuracy, 25
PMU Phasor measurement unit speed, effectiveness, and reduces human workload, these studies indeed 26
PPO Proximal policy optimization achieve some impressive results. This motivates us to examine them 27
PSS Power system stabilizer and compile their methods, results, and discoveries to create a readable 28
RF Random forest synthesis of the resources available in the literature under this topic. 29
RL Reinforcement learning But this paper is not only an exhaustive review of existing research. 30
RNN Recurrent neural network Instead, it aims to discuss open challenges that these applications face 31
in practice, such as high requirement on data, imbalanced learning, 32
interpretability of AI, difficulties in transfer learning, the robustness 33
of AI to communication quality (e.g., noise, data loss and time-delay), 34
1 and stable operation is one of the most critical requirements for smart and the robustness against attack or adversarial examples. Furthermore, 35
this paper provides some suggestions for future directions to overcome 36
2 grids. However, the new features have made the stability characteristics these challenges. The main contributions of this review are three-fold: 37
 Z. Shi et al.

Fig. 1. Major features of smart grids.

1 - We provide a general overview of the definitions, history and 2.2. History of AI 38

2 state-of-the-art techniques of AI and analyze its applicability to
3 smart grids. In 1956, John McCarthy put forward the term ‘‘Artificial Intelli- 39
4 - We give a comprehensive review of the applications of AI to smart gence’’ for the first time in the famous Dartmouth Conference [28]. 40
5 grid stability analysis and control–security assessment, stability After that, AI entered the first golden age, in which the main research 41
6 assessment, fault diagnosis, and stability control. direction of AI was reasoning as search and natural language process- 42
7 - We discuss the challenges that the existing applications face ing [25,30]. But in the 1970s, AI entered its first winter because of the 43
8 in practice and identify future directions, in order to provide performance constraints of computers and financial pressures caused 44
9 insightful guidance for bridging the research-practice gap. by stalled progress [25,28]. In the 1980s, the rise of expert systems 45
brought AI to another prosperous period, and the expert systems were 46
10 In the following sections, we will cover the four aspects of re-
also applied to solve the scheduling problem of power systems [31]. 47
11 cent advances in stability analysis and control with AI techniques:
However, after several years, due to the sudden drop in demand for 48
12 methodologies, applications, challenges and future directions. Section 2
AI hardware, the high maintenance cost of the expert systems and the 49
13 gives a general overview of AI, including its definition, history and
decline of investment enthusiasm, AI entered the next winter [25,30]. 50
14 typical methodologies. Section 3 analyzes the application potential
In the 1990s, as the new paradigm of ‘‘intelligent agent’’ was gradually 51
15 of AI in smart grid stability analysis and control. Section 4 provides
accepted [27], AI returned to its normal development, in which Deep 52
16 a comprehensive review of the current studies. Section 5 presents
Blue system defeated Garry Kasparov, the world chess champion. After 53
17 our discussions on the challenges and future directions. Finally, we
entering the 21st century, new application fields such as autonomous 54
18 conclude this review in Section 6. The main structure of this paper is
driving systems have also promoted the development of AI. Since 55
19 shown in Fig. 2 for clarity.
2011, with the rise of deep learning and big data, AI has entered an 56
20 2. A general overview of AI unprecedented hot phase. The brief development history of AI is shown 57
in Fig. 3. 58
21 2.1. Definitions of AI
2.3. Key techniques of AI 59
22 What is AI? Even though AI has developed for decades, there is
23 still no unified definition of AI in academia. John McCarthy (1927– 2.3.1. Machine learning 60
24 2011), one of the founders of the discipline of AI and the Turing Award Machine learning (ML), an important branch of AI, expects to find 61
25 winner, defined AI as the science and engineering of building intelligent patterns contained in existing data. By analyzing features of input data, 62
26 machines in 1955 [25]. Nils John Nilsson (1933–2019) pointed out models for specific tasks such as classification, prediction and clustering 63
27 that AI enables intelligent agents to acquire advanced human skills like can be built using ML algorithms. During the training of ML models, ex- 64
28 perception, communication, reasoning and learning in complex envi- periences will be learned from statistical characteristics of data. So far, 65
29 ronments [26]. David Poole specifically described intelligent agents as a lot of ML algorithms have been put forward, and some of them have 66
30 devices that can perceive its environment and take action to maximize widespread applications. In the 1990s, Vapnik and Cortes proposed the 67
31 the possibility of achieving goals, suggesting that the purpose of AI is support vector machine (SVM) [32], which has good performance for 68
32 to construct such a superior intelligent agent [27]. In recent years, AI classification and regression problems. SVM aims to find the optimal 69
33 is generally defined as the research and design of intelligent agents, hyperplane called decision boundary to divide the input data points 70
34 and the ultimate goal of AI is to make computers capable of thinking into different categories. The introduction of the kernel trick, which 71
35 and acting [28]. In simple terms, the challenge of AI is to solve tasks transforms existing data into higher dimensional data, enables SVM to 72
36 that are difficult to formalize, but relatively easy for humans with process nonlinear division. Decision Tree (DT) [33] is another famous 73
37 intuitions [29]. and interpretable ML algorithm that features a tree-like structure for 74
 Z. Shi et al.

Fig. 2. The structure of this paper.

Fig. 3. The brief development history of AI. AI was born in the 1950s. After several booms and winters, AI has experienced a resurgence in the 21st century.

1 decision. It is based on a series of nested rules to make predictions. of nonlinear activation function enables ANN to fit nonlinear relation- 8
2 Optimal features and threshold for each decision node need to be ships [35]. To speed up the training of ANN, the extreme learning 9
3 figured out to get a purer division while training DT. Also, the depth machine (ELM) [36] was proposed by Guang-Bin Huang in 2004. ELM 10
has a single hidden layer connected with the input layer by random 11
4 of the tree related to the overfitting problem should be considered.
weights and biases. The optimal weights that connect the hidden layer 12
5 Artificial neural network (ANN) [34], inspired by neuroscience study, and the output layer can be calculated without iteration, which is 13
6 has a strong capability to represent the relationship between inputs more efficient than gradient-based training methods. The probabilistic 14
7 and outputs through multiple layers of neurons. And the introduction graphical model (PGM) is a probabilistic model expressing relationships 15
 Z. Shi et al.

1 between variables based on graph representation. Bayesian network relationships. However, without proper initial weights, it is not feasible 66
2 and hidden Markov model (HMM) [37] are typical examples of directed to train a DNN using the back-propagation algorithm [47]. It was not 67
3 and undirected PGMs, respectively. PGM transforms learning tasks until 2006 that Hinton et al. proposed a greedy layer-wise pretraining 68
4 into calculating the probability distribution of variables represented by method using deep belief networks (DBN) [48] to solve this problem, 69
5 nodes in graphs. which was a milestone for the development of DL. Successful appli- 70
6 Despite the powerful learning abilities of various ML models, there cations of the convolutional neural network (CNN) in the computer 71
7 still remain some practical problems to be solved. Therefore, several vision field [49–51] have proved the power of DL. With the process 72
8 paradigms were proposed to overcome them. To improve the per- of multiple pairs of convolution and pooling layers which function as 73
9 formance and generalization abilities of ML algorithms in practical filters, structural features of different resolution in input images can be 74
10 applications, ensemble learning is proposed, which builds and com- extracted for image recognition. However, the overfitting problem be- 75
11 bines a group of models to achieve learning tasks. The bootstrap comes remarkable when training a CNN with multiple layers. Dropout 76
12 aggregating (Bagging) [39] is proposed by Leo Breiman in 1994. A technique [52,53] has been proven to be an effective regularization 77
13 group of diverse models will be trained by several datasets using the method that helps avoid overfitting. In the natural language processing 78
14 bootstrap sampling method. The random forest (RF) [44] algorithm field, the recurrent neural network (RNN) is usually preferred [54]. 79
15 is a typical implementation of Bagging with extra random feature
Long short-term memory (LSTM) [55], a commonly-used RNN, has 80
16 selection. Unlike Bagging, which can be conducted in parallel, the
memory cells for storage of history updated according to the gating 81
17 boosting algorithms attempt to improve the performance by learning
mechanism. The memory cell attempts to extract the long-range depen- 82
18 from the samples wrongly predicted by the previously trained model.
dency from data, which makes LSTM perform better than vanilla RNN 83
19 The gradient boosting decision tree (GBDT) [40] and extreme gradi-
while processing long time series. Attention mechanism [56] inspired 84
20 ent boosting (XGBoost) [41] are typical implementations of boosting
by that of humans is often useful when dealing with a very long time 85
21 algorithms.
series. Attention will mainly be paid to the critical information closely 86
22 In most ML application scenarios, unlabeled data is sufficient, but
related to the output, which helps improve the performance. 87
23 obtaining labels is a long, expensive, and laborious process. Active
24 learning, a form of semi-supervised learning where a learning algorithm Different from the discriminative models mentioned above, genera- 88
25 can actively query the user/teacher (usually called ‘‘oracle’’) for labels, tive models are used for generating new samples by learning existing 89
26 aims to achieve similar (or greater) performance to using a fully super- ones. Generative adversarial network (GAN), invented by Goodfel- 90
27 vised dataset with less but carefully selected labeled data. Membership low [57], is composed of a generator and a discriminator. The generator 91
28 query synthesis, stream-based selective sampling, and pool-based sam- tries to generate fake images that look as real from random noise to 92
29 pling are three typical settings of active leaning, of which pool-based deceive the discriminator while the discriminator tries to distinguish 93
30 sampling is the most popular. In it, instances are drawn from a pool between the synthesized images produced by the generator and real 94
31 that stores unlabeled data and assigned an informative measure. And images sampled from datasets. They contest with each other until they 95
32 then, the system selects the most informative instance(s) to query the reach the Nash equilibrium during the training. 96
33 oracle for the labels. The informative measure, or query strategy, is Autoencoder (AE) [58], a type of unsupervised method, is com- 97
34 the core of active learning. Uncertainty sampling, diversity sampling, posed of two neural networks serving as an encoder and a decoder, 98
35 expected model change, query by committee, and et al. are commonly respectively. The encoder compresses the input into code in a lower 99
36 used query strategies. Interested readers can refer to [42,45] for more dimension, and the decoder needs to recover the input using the code. 100
37 information. AE is trained by minimizing the difference between inputs and outputs 101
38 With the same problem of insufficient labeled samples, transfer to learn effective compressed representation rules. As a variant of 102
39 learning (TL) aims to train models for new tasks using knowledge AE, variational autoencoder (VAE) [59] introduces a regularization 103
40 learned from another related problem. In contrast, conventional ML so the inputs will be encoded as a distribution roughly subject to a 104
41 models need to be retrained once the task or data distribution changes, multivariate normal distribution over the latent space. This can be 105
42 which is inconvenient in practical applications. TL algorithms can be regarded as a constraint on VAE to ensure the regularity of latent space 106
43 roughly divided into instance-based methods, feature-based methods, so that VAE can generate more reasonable and meaningful outputs 107
44 relation-based methods and model-based methods according to what during the generative process. 108
45 is transferred [43]. Among these methods, feature-based methods are Most DL models operate on the Euclidean data structure and have 109
46 the most promising research directions of TL. Its goal is to minimize
achieved great achievements. However, many scenarios are represented 110
47 the gaps of features between the source domain and target domain
by the non-Euclidean data structure in reality, such as social network 111
48 after transformation in the feature space. The basic idea of instance-
models. The comparison between the Euclidean data structure and the 112
49 based methods is to adjust the weights of samples according to the
non-Euclidean data structure is shown at the bottom of Fig. 5. The non- 113
50 similarity between the source domain and the target domain. Model-
Euclidean graph can be represented by vertices, edges, and weights 114
51 based methods are also commonly used methods that transfer model
corresponding to each edge, which cannot be processed by general 115
52 structure and relevant parameters with the assumption that features
neural networks such as CNN. Therefore, graph neural networks (GNN) 116
53 extracted by the pre-trained models will be useful for related tasks.
emerges at the right moment. The input features of each vertex in a 117
54 Relation-based methods are potential directions of TL that still remain
55 further investigations. Overall, TL is worthy of exploration when used graph are fed into GNN based on the graph topology. The features of 118
56 in practical applications. each vertex can pass information to other vertices by edges through the 119
57 The schematic diagrams of these typical algorithms in ML are shown message propagation mechanism, and the information of each vertex 120
58 in Fig. 4. finally tends to be stable after repeated iterations. In recent years, 121
there have come into being five types of GNN: graph convolutional 122
59 2.3.2. Deep learning networks (GCN), graph generative networks, graph attention networks, 123
60 Deep learning (DL), a class of ML, aims to process high dimensional graph spatial–temporal networks and graph autoencoders [61]. Al- 124
61 input data directly and build end-to-end models for tasks. It does not though GNN has achieved great success in social network analysis [62], 125
62 rely on domain expertise or feature engineering techniques to extract image classification [63], traffic flow prediction [64,65] and so on, its 126
63 useful features for data preprocessing [46]. Compared with traditional application in smart grids is still worth further exploration. 127
64 ANN of shallow ML, deep neural network (DNN) has more hidden The schematic diagrams of these typical algorithms in DL are shown 128
65 layers that give DL a more powerful ability to represent complex in Fig. 5. 129
 Z. Shi et al.

Fig. 4. Schematic diagrams of some typical algorithms in ML. (see Refs. [32–34,36–43]).

1 2.3.3. Reinforcement learning 3. Application potential for stability analysis and control in smart 35
2 Unlike supervised learning requiring the labeled training samples, grids 36
3 reinforcement learning (RL) can train an agent based on Bellman’s
4 optimal principle, through the information interaction between the 3.1. Application potential for stability analysis in smart grids 37
5 agent and the environment. RL problems are usually modeled using the
6 Markov Decision Process (MDP), of which the elements include state 3.1.1. Necessity 38
7 𝑠𝑡 , action 𝑎𝑡 , policy 𝑝, and reward 𝑅. Generally, RL algorithms can be The purpose of stability analysis is to investigate whether the power 39
8 divided into two categories: model-based and model-free, as shown in grid can maintain stability under disturbances and to give the type 40
9 Fig. 6. The difference is whether the agent needs to know or learn the of instability. Appropriate measures for emergency control can be 41
10 model of the environment. taken in time only if the accurate type of instability can be provided 42
11 As shown in Fig. 7, the typical model-based RL algorithms include quickly. However, with the expansion of interconnected power grids, 43
12 MBMF [66], Alpha Zero [21], and World Models [67], etc. However, integration of renewable energy, and employment of HVDC, it be- 44
13 there exist errors between the real environment and the learned model.
comes more difficult for traditional methods to deal with due to the 45
14 This error may cause the agent trained with the learned model not
complex transient characteristics of the system. Currently, the time- 46
15 to achieve the expected results in the real environment. Instead of re-
domain simulation (TDS) method and transient energy function (TEF) 47
16 quiring an environment model, the model-free RL algorithms optimize
method are commonly used. Although with high precision, TDS is not 48
17 agent parameters through the direct information interaction with the
computationally efficient to meet the requirements of online stability 49
18 real environment, which is easier to implement and more popular in
analysis. Besides, complicated models with accurate parameters should 50
19 applications.
be built to simulate real power grids. TEF method does not require time- 51
20 The model-free RL algorithms are divided into two categories ac-
consuming computations, but energy function is hard to be derived 52
21 cording to different optimization objects: policy optimization and value
for large-scale power grids. And the mainly adopted second-order 53
22 optimization. The policy optimization-based RL optimizes parameter-
classic models in TEF, which ignore the complex transient process, may 54
23 ized policy functions with good stability and credibility, of which
24 the typical algorithms include A2C/A3C [69], PPO [70], TRPO [71], influence the accuracy of stability analysis. 55
25 etc. Whereas, the value optimization-based RL optimizes the value
26 corresponding to the state–action pair (𝑠𝑡 , 𝑎𝑡 ). The typical algorithms 3.1.2. Feasibility 56
27 include Q-Learning, DQN [75], C51 [76], etc. These RL algorithms Overall, a new stability analysis method suitable for the current 57
28 have a high efficiency of data utilization, but the training process development of smart grids is needed. The recent rapid development 58
29 is sometimes unstable. In order to balance the advantages of policy of AI technology has made it a promising solution. Stability analysis 59
30 optimization and value optimization, some RL algorithms are further can be seen as a typical classification problem solved by supervised 60
31 proposed, like DDPG [72], TD3 [73], and SAC [74], etc. With the learning algorithms. With the deployment of phasor measurement units 61
32 continuous development and improvement of RL theory, some of them (PMUs) and the wide-area measurement system (WAMS) in smart grids, 62
33 have been successfully applied to electricity trading [81,82], economic abundant real-time electrical data collected with high sampling rates 63
34 dispatch [83], and stability control [84] of smart grids. and a unified time frame can be served as the database for AI methods. 64
 Z. Shi et al.

Fig. 5. Schematic diagrams of some typical algorithms in DL. (see Refs. [48,49,54,57–60]).

Fig. 6. The MDP model for RL: model-free RL and model-based RL. The difference is whether the agent needs to know or learn the model of the environment.

1 Data-driven methods represented by ML, DL can extract features related 3.2. Application potential for stability control in smart grids 5

2 to the stability from the input data using powerful learning abilities. In
3.2.1. Necessity 6
3 addition, complex modeling and domain expertise of power grids are On the one hand, modern smart grids are becoming more and more 7
complex. A large number of power electronic devices in power gener- 8
4 not required for AI methods. ation, load, and DC transmission systems make the system difficult to 9
 Z. Shi et al.

Fig. 7. Algorithm classification of modern RL [68]. (see Refs. [21,66,67,69–80]).

1 model accurately. Under these conditions, the design of conventional

2 controllers based on power system models becomes more difficult. At
3 the same time, the large-scale integration of renewable energy increases
4 the uncertainty of the power grid, which makes it difficult to set the
5 parameters of traditional controllers.
6 On the other hand, the conventional security and stability control
7 measures are usually formulated by the technicians, which are heavily
8 dependent on their professional background knowledge and grid op-
9 eration experience. Then the effectiveness of the control measures is
10 verified with the help of power grid simulations. However, with the
11 increase in the scale of smart grids, the operating condition of the grid
12 is more variable, and the controllable devices are more complicated. As
13 a result, the traditional manual method of control measures formulation
14 appears inefficient in this situation.

15 3.2.2. Feasibility
16 RL does not require the precise model of the power grid and can
17 learn the control strategies by interacting with actual power grids
18 or simulation models. With the powerful feature extraction and state
19 characterization capabilities of DL, AI-based control agents will be Fig. 8. Applications of AI to stability analysis and control in the smart grid.
20 able to cope with the complex and changing operating conditions of
21 smart grids. In addition, combined with the concept of knowledge
22 graphs, professional knowledge and expert experience can be used to 4.1.1. Static security assessment 40
23 build a knowledge base. Then the decision-making efficiency would be Static security assessment (SSA) involves the analysis of the ability 41
24 improved through the technology of knowledge fusion and knowledge of power system to withstand a set of contingencies without violating 42
25 reasoning. system constraints, such as the limits of equipment ratings, bus voltage 43
and line power flows. SSA focuses on steady state, but neglects the 44
transient behaviors of system variables. Conventional SSA calculates 45
26 4. Applications of AI to stability analysis and control in smart power flow for any 𝑁 − 1 contingency [86]. It gives detailed and 46
27 grids accurate voltage profiles and power flows in a power grid, but its 47
shortcoming is also apparent: A large number of contingencies need to 48
28 AI has been applied to many aspects of the smart grid. This paper be evaluated, and it is too time-consuming to be applied online [87]. 49
29 mainly focuses on stability analysis and control as they are essential to AI provides another promising approach to tackle this problem. The 50
30 ensure the reliability of smart grids. The specific applications are shown most common practice is training a classifier to learn the mapping 51
31 in Fig. 8. relationship between the features that represent the operating condi- 52
tion and the secure/insecure result. The general framework of SSA 53
is shown in Fig. 9. For AI approaches, the main bulk of the work is 54
32 4.1. Security assessment done offline. Thus, in real time, system static security can be assessed 55
in a timely manner using the built AI model. For example, in [87], 56
sequential forward search (SFS), a greedy strategy method, was used to 57
33 Security is defined as the degree of risk in a power system’s ability
select features from system variables. And then, an SVM classifier was 58
34 to survive imminent disturbance (contingencies) without interruption
constructed on these features, whose performance was proven to give 59
35 to customer service [85]. And ensuring security is always a key concern higher accuracy and less misclassification rate than other equivalent 60
36 in the smart grid operation. Security analysis consists of two important algorithms. Ref. [88] presented an AdaBoost ensemble method that is 61
37 components: static security assessment and dynamic security assess- composed of many SVM classifiers. This approach can not only assess 62
38 ment, whose difference lies in whether the post-disturbance dynamic static security but also predict the violation types, that is, either voltage 63
39 transition process is considered. violation/line overload or both for insecure status. 64
 Z. Shi et al.

optimization problem to find the optimal trade-off between the ear- 39

liness and accuracy. And due to the fact that the performance of AI 40
approaches relies heavily on the quantity and quality of training data, 41
an R-vine copula-based distribution modeling and sampling framework 42
were provided in [92] to enrich the dataset. Although the application of 43
AI drastically reduces the online assessment time, the offline simulation 44
that is needed for labeling samples still takes a lot of time. To overcome 45
this, a semi-supervised tri-training algorithm was employed in [93] to 46
label unlabeled samples. And to overcome the missing data problem, 47
the popular generative model GAN was used in [94] to impute the 48
missing data. For most pre-contingency DSA studies, each DSA model 49
is used for only a specific fault. But based on transfer learning, the DSA 50
model in [95] can be used for unknown different but related faults. 51
AI is also applied in post-contingency DSA. Ref. [96] proposed an 52
adaptive boosting (AdaBoost) algorithm composed of SVMs. In addition 53
to security status, this approach can also determine the generator 54
coherency and individual generator synchronism state. 55
These applications to security assessment are summarized in 56
Table 1. 57

4.2. Stability assessment or prediction 58

In recent years, due to continuous growing load demand, increasing 59

Fig. 9. A general framework of security or stability assessment using AI techniques. It
integration of intermittent renewable energy, use of power electronics 60
can be divided into two stages: offline training and online application. And the main
bulk of the work is done offline. equipment, and the liberalization of electricity market in smart grids, 61
stability is becoming a more serious problem. Stability assessment, 62
that is, monitoring and early detection instability, plays an important 63
1 4.1.2. Dynamic security assessment role in secure and stable operation of smart grids. According to the 64
IEEE/CIGRE Joint Task Force [86], the stability problem can be cat- 65
2 Dynamic security assessment (DSA) considers both transient and
egorized into three types: rotor angle stability, voltage stability and 66
3 steady-state time frames. A dynamic secure system must meet all
frequency stability. AI has been applied to all of these areas for its 67
4 the security criteria [85], including the thermal overloading, volt-
advantages in rapidity, accuracy and adaptability. And the application 68
5 age and frequency variations (both transient and steady state), and
framework is similar to Fig. 9. 69
6 all forms of stability (transient stability, small-disturbance stability,
7 voltage stability, and frequency stability) [86].
4.2.1. Rotor angle stability assessment 70
8 DSA can be further divided into two categories, or rather two
Rotor angle stability refers to the ability of synchronous machines of 71
9 phases: pre-contingency DSA and post-contingency DSA. Pre-
an interconnected power system to remain in synchronism after being 72
10 contingency DSA evaluates whether the current operating condition
subjected to a disturbance. For large-disturbance rotor angle stability 73
11 is secure with respect to an anticipated but not yet occurred con-
(i.e., transient stability), conventional assessment methods (TSA) have 74
12 tingency [89]. Pre-contingency DSA triggers preventive controls if the
their drawbacks. Time-domain simulation is compute-intensive [97]. 75
13 result is potentially insecure. In contrast to pre-contingency DSA,
Transient energy functions method is conservative and is difficult to 76
14 post-contingency DSA predicts future system security following an on-
be applied online because some state variables are not available [98]. 77
15 going contingency with dynamic measurements such as rotor angle and
And extend equal area criterion (EEAC) can only be suitable for the 78
16 voltage trajectories. Post-contingency DSA corresponds to emergency classical generator model. 79
17 controls to impede the propagation of insecurity in the interconnected The main structure of TSA using AI (more specifically, ML) is also 80
18 power grid. depicted in Fig. 9. ML mines the mapping between the power grid 81
19 Conventional simulation methods for DSA include power flow calcu- variables and the stability status. For online application, once the 82
20 lation for voltage constraints or element ratings violation, time-domain required measurements are fed into the ML model, the TSA result can 83
21 simulation for transient security and voltage stability, and eigenvalue be obtained immediately. In recent years lots of ML approaches, both 84
22 analysis for small-disturbance stability assessment, which is rather shallow ML and DL, have been used for it, such as SVM [98], ELM [99], 85
23 time-consuming and unsuitable for online application. Nowadays, with DBN [100], DAE [101], LSTM [102,103] and CNN [97,104,105]. 86
24 the development of WAMS technology and the wide deployment of The common practice of shallow ML is to first design features 87
25 PMUs, AI is applied to DSA since it is a typical classification problem. manually or by algorithms, and then build a classifier on these features. 88
26 The application framework is shown in Fig. 9. For example, binary Jaya (BinJaya) was proposed in [99] to select 89
27 For pre-contingency DSA, Ref. [89] presents an intelligent system effective features from original measurements. And then, an ensemble 90
28 based on an ensemble of ELMs. Each ELM gives a secure, insecure of OS-ELMs (Online Sequential ELM) was built on them to predict the 91
29 or incredible result. If the number of individual ELMs that give in- stability status. Generally speaking, the performance of ML depends to 92
30 credible results exceeds a certain threshold, then the final result is a large extent on the internal features, and designing effective features 93
31 also considered incredible, or else the final result would be secure or requires considerable domain expertise and careful engineering. 94
32 insecure depending on the number of the other two kinds of results. As mentioned before, the newly emerging end-to-end DL does not 95
33 This makes the intelligent system reliable since potentially incredible rely on domain expertise or feature engineering techniques to extract 96
34 results can be eliminated. ELM was also used in [90]. Its framework useful features. Instead, the raw measurement data can be directly 97
35 consists of a DSA engine, a wind power and load demand forecasting fed into DL model. In [103], an LSTM network was built on the raw 98
36 engine, a database generation engine, and a model updating engine, all synchrophasor measurements to form an intelligent time-adaptive TSA 99
37 of which is with the purpose of dealing with the wind power impact. system. Ref. [97] transformed the generator voltage phasors to RGB 100
38 Also based on ELMs, Ref. [91] further constructed a multi-objective images, which is then fed into CNN to construct an online stability 101
 Z. Shi et al.

Table 1
Summary of AI’s applications to security assessment.
Paper Year Task Method Descriptions/Remarks
[87] 2012 SSA SVM • SFS wrapper was identified as a suitable method for feature
extraction.
[88] 2018 SSA AdaBoost ensemble of SVMs • The performance was further improved by fuzzy clustering
thresholding technique.
• The method can predict the type of violations (voltage
violation/line overload or both).
[89] 2012 Pre-contingency DSA Ensemble of ELMs • The method can estimate the credibility of DSA results and
discard incredible results.
[90] 2012 Pre-contingency DSA Ensemble of ELMs • Wind power impacts are concerned.
[91] 2017 Pre-contingency DSA Ensemble of ELMs • A multi-objective optimization problem was constructed to find
the optimal trade-off between accuracy and earliness.
[92] 2018 Pre-contingency DSA Deep autoencoder, R-vine copula • R-vine copula is used to generate a large number of anticipated
system states for training, and deep autoencoder extracts effective
features.
[93] 2019 Pre-contingency DSA Tri-training, data editing, ELM • Unlabeled samples are labeled by tri-training, and data editing
reduces the noise that comes with incorrectly labeled samples.
• KNN may be not sufficient to measure the similarity of samples.
[94] 2019 Pre-contingency DSA GAN, ELM, RVFL • GAN was used to complete the missing data.
• The mechanism of how GAN identifies the current operating
condition that the missing data corresponds to is not clear.
[95] 2020 Pre-contingency DSA Transfer learning, ELM, RVFL • Feature-based transfer leaning makes one DSA model can be
transferred to unknown different but related faults.
[96] 2019 Post-contingency DSA SVM, AdaBoost • The approach also determines the generator coherent state and
the synchronism status of each generating unit.

1 monitoring system. Ref. [105] proposed a TSA model based on an level, Ref. [111,112] applied greater weight on the misclassification of 43
2 ensemble of CNNs, in which active learning and fine-tuning techniques unstable samples in the cost function to guide the model to care more 44
3 were used for the online model update. In addition to the aperiodic about the minority class. And the problem of imbalanced learning will 45
4 instability of general concern, transient instability may also occur as be discussed in detail in Section 5.2. 46
5 oscillatory instability due to insufficient damping torque. Ref. [106] In most studies, PMUs are assumed to sample power grid variables 47
6 presented a TSA and instability mode prediction method based on CNN accurately. But in practice, PMU measurements may be affected by 48
7 to distinguish the two instability modes, in order to guide the selection noise, data loss and time-delay. Ref. [102] investigated the influence 49
8 of control actions. As for offline TSA, Ref. [107] developed an efficient of delays and proposed an LSTM-based method to overcome it. In 50
9 batch framework to reduce the average simulation time. the method, a large LSTM can make early preliminary assessment 51
10 Accuracy and rapidity are two trade-off goals of online TSA. Ref. despite delays – although the result may not be accurate – and then 52
11 [108] developed a self-adaptive TSA system composed of a series of an ensemble of small LSTM blocks will correct the predictions when 53
12 ELMs along the timeline after fault clearance. If the classification at more measurements are available. As for noise interference, denoising 54
13 time 𝑇 is evaluated credible, the system will deliver the TSA result at 𝑇 ; autoencoder was used in [101] to discover more robust features. And 55
14 otherwise, the classification will continue at time 𝑇 + 1 since the result the robustness of AI to communication quality will be discussed in 56
15 will be more credible with more available measurements. This adaptive Section 5.5. 57
16 pattern is shown in Fig. 10. In this way, the system can progressively In addition, TSA can also be achieved by predicting critical clearing 58
17 adjust the response time to balance the trade-off between response time time (CCT). Ref. [113] and [114] devoted to this task with an adaptive 59
18 and accuracy. Ref. [103] also presented a time-adaptive TSA system but neuro-fuzzy inference system and ELM, respectively. 60
19 based on one unique LSTM-based neural network. This implementation
20 can reduce the model complexity and the needed training time on the 4.2.2. Voltage stability assessment 61
21 one hand, and learn the temporal data dependencies thanks to the Voltage stability refers to the ability of a power grid to maintain 62
22 recurrent nature of LSTM on the other hand. Furthermore, the multi- steady voltages at all buses following a disturbance from a given 63
23 criteria decision making (MCD) theory was used in [109] to determine initial operating condition [86]. With an ever-increasing load demand, 64
24 the optimal prediction. Unlike these time-adaptive manners, a CNN- power grids are pushed to operate near the limits of their transmission 65
25 based continuous rotor angle stability monitoring system was proposed capabilities, which threatens voltage stability to some extent [115]. 66
26 in [97]. The system abandoned the concept of decision moment but And in industrial or urban areas, the characteristics of dynamic loads, 67
27 generated the stability prediction continuously, which can alert earlier such as motors and air conditioners, also tend to make the voltage 68
28 for unstable prediction. stability problem more serious. 69
29 In practice, AI for TSA may face data imbalances. In fact, modern Conventional methods for voltage stability assessment (VSA), such 70
30 smart grids can remain stable after being subjected to most disturbances as energy function [116] and bifurcation analysis [117], have difficul- 71
31 due to its robustness. Hence, unstable samples are rather rare, which ties when applied to a practical large-scale power grid or with com- 72
32 may significantly deteriorate the performance of AI. To address this plicated controllers. And time-domain simulation for large-disturbance 73
33 problem, some scholars have made some improvements to conventional VSA is compute-intensive [86]. This is also hard to be applied online 74
34 methods to adapt it to imbalanced learning. At the level of data or for quick assessment or prediction. 75
35 samples, Ref. [110] presented a data segmentation-based ensemble The emerging AI has been identified to be a powerful approach to 76
36 method. In this work, stable samples are divided into multiple subsets, VSA. Learned from simulation or measured data, the nonlinear rela- 77
37 and then the stable subset is merged with the unstable samples into a tionship between power system variables and corresponding voltage 78
38 new training subset. And in each training subset, stable samples are not stability status can be discovered. So at the online application stage, 79
39 more than unstable ones, on which an AdaBoost classifier is built. Sam- the stability can be assessed as long as the input is fed into the AI 80
40 pling is another method in the data level. Ref. [111] utilized ADASYN model. Ref. [115] presented an online short-term VSA scheme based on 81
41 sampling algorithm to synthesize unstable samples in the hidden space the time series shapelet classification method. With the shapelet tech- 82
42 in an autoencoder to balance the imbalanced dataset. At the model niques, the complex time series dataset was transformed into a simple 83
 Z. Shi et al.

Fig. 10. Structure of the self-adaptive AI models for TSA.

1 one, and then decision trees are able to build on it to achieve both can be easily improved online with only a few incremented samples be- 46
2 accuracy and interpretation. Besides, semi-supervised cluster learning is cause the information of historical samples has already been embedded 47
3 employed in it to overcome the unreliability of practical criteria for de- into historical models. 48
4 termining short-term voltage stability. On this basis, Ref. [118] further Since the purpose of FSA is to activate and coordinate emergency 49
5 considers the class-imbalanced learning problem. Forecasting-based control such as load shedding, some studies directly unified the two 50
6 nonlinear synthetic minority oversampling technique (FN-SMOTE) and modules. Ref. [126] proposed an event-driven voltage and frequency 51
7 cost-sensitive learning are utilized in it. In addition to binary clas- stability assessment and load shedding schema. SVM was used to 52
8 sification, Ref. [119] proposed a TVSI prediction subproblem that predict stability. And for unstable events, a new load shedding method 53
9 quantifies the unacceptable dynamic voltage deviation. Furthermore, was activated to restore the stability of the power system. Besides, 54
10 a self-adaptive method is proposed in [120] to minimize the overall to combine the advantages of data-driven methods and model-driven 55
11 assessment time consumption without impairing accuracy. methods, in [127], a serial integration of model-based system frequency 56
response (SFR) and AI-based ELM was proposed for FSA and control. 57
12 In addition to alarming for voltage instability, Ref. [121] also
SFR model is composed of transfer functions, and it can preserve basic 58
13 investigated fault-induced delayed voltage recovery phenomenon and
system frequency response characteristics. ELM was used to correct 59
14 further classified the stable state into three sub-states: emergency, alert,
the error caused by the simplification of SFR and fit the relationship 60
15 and normal, to activate control actions to prevent the occurrence of
between load shedding amount and system operation state. 61
16 delayed voltage recovery.
These applications to stability assessment are summarized in 62
17 The aforementioned applications are for the large-disturbance VSA, Table 2. 63
18 and AI is also used for small-disturbance voltage stability problems.
19 Based on local regression, Ref. [99] proposed an online voltage stability 4.3. Fault diagnosis 64
20 margin monitoring approach with reactive power reserve as the predic-
21 tor. An enhanced-online-random-forest model was developed in [122] Fault diagnosis can be divided into three parts: fault detection, 65
22 for small-disturbance voltage assessment. At the online application fault classification, and fault location. It plays an important role in the 66
23 stage, a significant increase in the uncertainty of the classifier will maintenance of system stability and quick service restoration when a 67
24 trigger the online update. failure occurs in a power system [128]. Many of the AI-based methods 68
have been proposed, which can be unified in a general fault diagnosis 69
25 4.2.3. Frequency stability assessment framework illustrated in Fig. 11. 70
26 Frequency stability refers to the ability of a power grid to main-
27 tain steady frequency after being subjected to a severe system upset 4.3.1. Fault information source and type 71
28 resulting in a significant imbalance between generation and load. In When a fault occurs in the power grid, the data collected from 72
29 modern smart grids, integration of stochastic and volatile renewable supervisory control and data acquisition (SCADA) is the primary in- 73
30 energy impacts the equilibrium between generation and load, and the formation source for fault diagnosis conventionally. With the rapid 74
31 use of power electronic devices tends to reduce system inertial. As development and promotion of the wide-area measurement system 75
(WAMS), wide-area electrical data can be acquired with a unified 76
32 a result of this, frequency stability becomes a prominent and serious
time base, which provides a significant database for data-driven fault 77
33 problem. Under such circumstances, frequency stability assessment
diagnosis techniques. Many different types of data have been adopted in 78
34 (FSA) becomes an effective tool to ensure and stable system operation.
fault diagnosis, including voltage, current, zero-sequence current, relay 79
35 AI has also been used in FSA for its high speed and accuracy.
protection information, et cetera. 80
36 Ref. [123] proposed a real-time FSA approach based on ELMs. The
37 predictor trained offline is then applied online to predict the fre- 4.3.2. Signal processing and feature extraction 81
38 quency stability following credible contingencies. Instead of binary The necessary features for fault diagnosis have to be extracted 82
39 classification, the predictor outputs a continuous frequency stability effectively from input data using signal processing techniques. Wavelet 83
40 margin index to measure the stability degree. Ref. [124] proposed an transform (WT) is commonly used to analyze faulty signals in the 84
41 ensemble algorithm to predict post-disturbance deviations of power time–frequency domain [129]. It can decompose the original signal 85
42 system frequency. And the trade-off between the prediction and speed into several coefficients in different resolutions as features. Other tech- 86
43 was well addressed with the aid of cross-entropy ensemble combination niques such as Hilbert–Huang transform (HHT) [130], principal com- 87
44 strategy. And in [125], an FSA method considering model inheritance ponent analysis (PCA) [131], S transform (ST) [132] also have decent 88
45 is proposed, where inheritance means that the prediction performance performances in practice. 89
 Z. Shi et al.

Table 2
Summary of AI’s applications to stability assessment.
Paper Year Task Method Descriptions/Remarks
[98] 2011 TSA SVM • The method was tested under various conditions.
[99] 2017 TSA BinJaya, OS-EMSs • BinJaya was proposed to select effective features from original
measurements.
[100] 2018 TSA DBN • Power grid structure is considered in the learning process.
[101] 2018 TSA Multi-branch stacked denoising AE • MSDAE benefits from its denoising strategy when confronted with
PMU noise.
[102] 2017 TSA LSTM • LSTM ensemble-based intelligent system was proposed to handle
the communication delay and data missing problem.
[103] 2018 TSA LSTM • A time-adaptive TSA process balances the trade-off between
response time and assessment accuracy.
• Easy to implement
[97] 2019 Stability Monitoring CNN • The approach continuously monitors the stability and does not
require information about fault clearance.
• The RGB heatmap representation of the measurements is
interesting.
[104] 2019 TSP CNN • A unique graphical transient characterization technique
transforms trajectories into transient images, and stability margin is
also predicted.
[105] 2019 TSA Ensemble of CNNs, active learning • Active learning and fine-tuning techniques were used for online
model update.
[106] 2020 TSA CNN • The method can determine both the stability status and
instability mode.
• SGDR training algorithm was used for better generalization.
[108] 2015 TSA Ensemble of ELM • The system can progressively adjust the response time to balance
the trade-off between speed and accuracy.
[109] 2019 TSP SVM • Multi-criteria decision making was used to determine the best
time for the stability prediction.
[110] 2019 TSA Data segmentation, AdaBoost • Data segmentation and AdaBoost were employed to cope with
the problem of data imbalance.
• May lead to over-fitting since the unstable samples in every
subset are the same.
[111] 2019 TSA ADASYN, SDAE • Synthesized unstable samples were generated in hidden space to
balance the dataset.
[112] 2016 TSA ELM, cost-sensitive learning • Cost-sensitive ELM was used for handling the problem of
imbalance.
[113], 2015, 2016 CCT prediction Fuzzy inference system, ELM • Directly predict the critical clearing time
[114]
[115] 2016 VSA DT, time series shapelet classification • Shapelets transform the complex time series dataset into a simple
and traditional one.
[118] 2017 VSA Time series shapelet, FN-SMOTE • FN-SMOTE is used for the imbalanced learning problem.
[119] 2016 VSA NNRW • Besides binary classification, TVSI is proposed to quantifies the
unacceptable dynamic voltage deviation.
[120] 2019 VSA Ensemble of ELM • A self-adaptive method is proposed to minimize the overall
assessment time consumption without impairing the accuracy.
[121] 2019 VSA Random forest • Stability cases are further classified into three types: emergency,
alert and normal state.
[122] 2018 VSA Random forest • The method enables to online update via tree growth and/or tree
replacement.
[123] 2013 FSA ELM • The predictor outputs a continuous frequency stability margin
index to measure the stability degree.
[124] 2017 FSA DT, MLR, ANN, LSSVM, ELM • The trade-off between prediction accuracy and rapidity was well
addressed with the aid of a cross-entropy ensemble combination
strategy.
[125] 2019 FSA ELM • The performance can be easily improved online with model
inheritance strategy.
[126] 2019 FSA SVM • FSA and load shedding schema are combined.
[127] 2019 FSA ELM • Data-driven and model-driven methods are integrated.

1 4.3.3. Fault detection, classification and location Ref. [137] provided a fault diagnosis solution using HMM, matching 16
2 Conventional fault diagnosis algorithms, such as the traveling wave- pursuit decomposition and other ML techniques. The voltage and fre- 17
3 based and impedance-based methods, cannot fully adapt to the penetra- quency signals were utilized to implement fault detection and location, 18
4 tion of distributed renewable generations [133] and high voltage direct respectively. A fault contour map that clusters the buses into several 19
5 current (HVDC) [134,135]. However, AI-based methods can avoid com- levels according to the degree of impacts was also offered, which is 20
6 plicated modeling and fault mechanism analysis. Many AI algorithms, useful for fault analysis. 21
7 such as ANN, SVM [131], CNN [130], Bayesian network [136] and Although satisfactory performance on fault diagnosis can be 22
8 Petri net, have been widely applied in practice. Ref. [130] proposed achieved by AI-based methods, there are still several problems that 23
9 a fault classification method based on a CNN fed with features ex- remained to be solved. In the future, information collected from dif- 24
10 tracted by HHT in power distribution systems. With the help of CNN’s ferent sources should be integrated and taken advantage of to improve 25
11 powerful feature learning ability, this method has considerable fault the accuracy of fault diagnosis. Efficient parallel algorithms should be 26
12 classification performances. Ref. [128] proposed a GCN-based fault designed for complex power grids. How to combine different methods 27
13 location method in power distribution systems. Different from general to exploit the advantages and avoid the disadvantages should be 28
14 ML-based methods, it preserves the spatial information of buses in considered. 29
15 the structure of GCN, which helps improve fault diagnosis accuracy. These applications to fault diagnosis are summarized in Table 3. 30
 Z. Shi et al.

Fig. 11. A general fault diagnosis framework.

Table 3
Summary of AI’s applications to fault diagnosis.
Paper Year Task Method Performance Descriptions/Remarks
[128] 2019 Fault location in GCN Fault diagnosis accuracy: • Preserved spatial information
distribution networks 99.26%
• Data augmentation to improve the robustness to noise
• Voltage and current of three phases are needed.
[129] 2017 Fault detection in CWT, CNN Fault detection accuracy: • Avoided the problems of selecting reasonable features and
distribution networks 99% classifiers
• Considered faults in resonant grounding distribution
systems
• Only the single-phase-to-ground faults were considered.
[131] 2019 Fault diagnosis SSAE, PCA, Fault diagnosis accuracy: • Improved the neural network structure
SVM 92.5%
• Solved the problems of local minimum, gradient diffusion
and state space explosion
• PCA is not effective for feature compression to improve
accuracy.
[132] 2013 Fault location in S-Transform Location average error: • Considered the influences of fault inception angle, fault on
multi-terminal lines 0.24 km tap points, fault on different line sections
• High sampling frequency is needed.
[133] 2019 Fault diagnosis for Energy SR-CNN Fault diagnosis accuracy: • Implemented fault diagnosis in Energy Internet
Internet 98.67%
• Accurate fault diagnosis eliminates the impact of
distributed generations, direction of power flow and fault
impedance.
[134] 2019 Fault location in HVDC CNN, HHT Fault location error: • Accurate location of high-resistance grounding faults
0.2 km
• Does not require double-end time synchronization
[135] 2019 Fault classification and CNN Fault location error: • Considered fault diagnosis in MMC-HVDC
location in AC transmission 1.044%; Fault
lines with MMC-HVDC Classification accuracy:
99.90%.
• No dependence on prior knowledge and human efforts in
feature design
[137] 2014 Fault diagnosis MPD, HMM, Fault detection accuracy: • Effective feature extraction using MPD takes advantage of
kNN, 100%; Fault identification: frequency and voltage signal.
k-means 97.5%
• Fault contour map locates the fault and indicates fault
impact levels.

1 4.4. Stability control margins are insufficient. Preventive control measures are needed to 16
restore the system to its normal state. 17
2 The smart grid confronts changing power demand and various types The smart grid may turn to the emergency state from the alert state 18
3 of disturbances during operation, both of which have impacts on the if the preventive control actions have not been undertaken or have not 19
4 security of the smart grid. According to its security level, the operating been successful [139] or directly from the normal state following a 20
5 states of a smart grid are classified [138,139] as in Fig. 12. In every
sufficiently severe contingency such as multiple faults. The system is 21
6 state, corresponding control actions are needed to ensure the stability
still intact, but some inequality constraints are violated. In this state, 22
7 and security of the smart grid.
emergency control actions are needed to restore the system to the alert 23
8 In the normal state, all constraints are met. But since there are
state and then to the normal state. 24
9 always small continuous changes in load demand in this state, damping
10 control, voltage control, and generation control and load frequency control The system state may be further deteriorated and transpose to in- 25
11 are necessary to ensure or improve the rotor angle stability, voltage extremis state if no emergency control actions have been undertaken. 26
12 stability, and frequency stability, respectively. The constraints of both equality and inequality are violated in this state, 27
13 The alert state arises when a plausible contingency occurs, causing and the system is not intact. In this state, effective emergency measures 28
14 the system to reach a state where some network equipment would be such as load shedding, generator tripping or controlled system splitting 29
15 overloaded. In this state, all the constraints are met, but the reserve are necessary to restore the system to restorative state [138]. 30
 Z. Shi et al.

Fig. 12. Smart grid operating state transition maps [138]. Different controls are needed
in different operating states.
Fig. 14. A general framework of voltage control/regulation using RL.

4.4.2. Voltage control 23

Automatic voltage control (AVC) refers to the optimization of the 24
voltage and reactive power of the power grid based on the grid opera- 25
tion data collected by the control system in real time. The controllable 26
objects of AVC include generator terminal voltage, on-load voltage- 27
regulating transformer taps, switchable reactive power compensation 28
devices, SVC, SVG and other voltage reactive power adjustment equip- 29
ment. AVC ensures the system voltage stability and quality and reduces 30
transmission losses through reactive power coordination control. 31
RL is also applied to AVC. The general application framework is 32
shown in Fig. 14. Shamik et al. [148] proposed the teaching–learning- 33
based optimization (TLBO) algorithm to optimize the parameters of a 34
PID controller for automatic voltage regulator. The simulation results 35
Fig. 13. A general framework of AI’s applications for stability control. demonstrate that the proposed approach guarantees robust perfor- 36
mance against parametric uncertainties. Duan et al. [149] proposed the 37
‘‘Grid mind’’, and for the first time, a DRL algorithm was applied to the 38
1 In the restorative state, restorative control actions are undertaken to AVC of the smart grid. DQN was used to adjust the on-load tap-changer 39
2 restart and reconnect all the equipment and to restore all system load. taps and shunt capacitors, while DDPG was used to adjust the generator 40
3 And then, the system would return to the normal state or the alert state. terminal voltage. The simulation results under a practical 200-bus 41
power grid verified that the proposed DQN/DDPG-based AVC can 42
4 In summary, different controls are needed in different operating
quickly generate a voltage adjustment scheme for different operating 43
5 states. And in the following subsections, we will give a comprehensive conditions. It shows that the application of DRL to AVC in smart grids 44
6 review of AI’s applications to these controls. is feasible. Further research would go to the coordination of different 45
voltage regulation methods (including discrete control variables and 46
continuous control variables) and the guarantee of the security and 47
7 4.4.1. Generation control robustness of control measures made by AI agents. 48
8 In the face of the poor performance of traditional PID control for
4.4.3. Damping control 49
9 auto generation control (AGC) in large-scale interconnected power
Local PSS and wide-area damping controller (WADC) based on 50
10 systems, smart generation control (SGC) has been a trend in recent the lead–lag link relies too much on the accurate parameters of the 51
11 years. The general framework of AI applications in generation con- system and have little adaptivity to the variations of working condi- 52
12 trol is shown in Fig. 13. Ref. [140] firstly applied Q-learning in the tions. Consequently, AI algorithms are introduced to break through 53
13 designing of AGC. Then Q-learning was used to realize anti-jamming these limitations. The application framework is similar to Fig. 13. 54
14 and plug-and-play of AGC units in [141,142], respectively. In terms Some work attempted to reform the existing controllers by AI algo- 55
15 of the curse of dimensionality and the slow convergence rate of Q- rithms. Refs. [150] and [151] applied RL and fuzzy neural network 56
16 learning, a deep Q learning (DQN) framework for AGC is developed to adjust the parameters of PSS in real-time. More researchers choose 57
to design a totally AI-based controller to evade the hard-acquired 58
17 in [143]. Refs. [144,145] extended multi-agent RL to handle inter-
parameters of the system. Among them, many controllers based on 59
18 actions among different agents in large-scale power systems. Besides,
Q-learning [152,153], tree-based batch mode RL [154], or the com- 60
19 lazy learning is integrated with RL to realize more functions like multi- bination of RL and ANN [155] are proposed and demonstrated to 61
20 input-multi-output in [146]. Apart from the safety of systems, economic be effective. Furthermore, communications issues like time-delay and 62
21 cost, transient performance, and emission are also considered through communication failure in damping control are also addressed with 63
22 multiobjective RL (MORL) proposed in [147]. Fuzzy-GrHDP in [156] and [157]. 64
 Z. Shi et al.

Fig. 15. The neural network architecture of the CNN ensemble classifier [105].

1 4.4.4. Frequency control braking and low-voltage load shedding control measures. Simulation 49
2 Similar to damping control, the deficiency of fixed-parameter and results of the four-machine-two-zone system and the New England 50
3 dynamic model-based controllers [158] still cannot be ignored in fre- system verified that the proposed intelligent emergency control mea- 51
4 quency control. Heuristic algorithms like human brain emotional learn- sures could effectively maintain system rotor angle stability and voltage 52
5 ing [159], particle swarm optimization [160], genetic algorithms [161], stability. And Zhang et al. [170] proposed DQN-based under-voltage 53
6 fuzzy logic in [162] caught much attention in the early stage, which load shedding for the short-term voltage stability, which uses convo- 54
7 were applied to tune the PID controller parameters or optimize the lutional LSTM to extract the spatial–temporal information, including 55
8 relationship between input and output of the controller. Nowadays, the topology information. The simulation results in South China Grid 56
9 RL are widely used. The application framework is similar to Fig. 13.
verified the effectiveness of the DRL-based load shedding controller. 57
10 Ref. [158] designed a supplementary load frequency controller based
It shows that the DRL algorithm is feasible for emergency control of 58
11 on approximate dynamic programming (ADP) algorithm. Robust ADP
the smart grid. However, different types of transient instability also 59
12 was adopted in [163] using only partial input information. Ref. [164]
correspond to different types of emergency control measures. How 60
13 aimed at controlling multiple machines with integral RL. Besides,
to coordinate different types of control measures and how to ensure 61
14 how to provide more accurate input reference for control is explored
15 with stacked denoising auto-encoders (SDAE) [165], long short-term that emergency control measures formulated by agents can not only 62
16 memory (LSTM) [166] and ELM [167]. effectively maintain the transient stability of the power grid, but also 63
avoid excessive control would be the valuable topics for subsequent 64
17 4.4.5. Preventive control intelligent emergency control. 65
18 Preventive control is adopted to restore the power grid to a more
19 secure and stable operating condition when security and stability analy-
4.4.7. Restorative control 66
20 sis shows that it is not secure enough under the expected contingencies.
Restorative control is a series of orderly controls and operations that 67
21 The conventional preventive controls include switching loads, changing
22 system structure, adjusting generator output and power flow, allocating are taken in order to restore the power grid to a normal state after the 68
23 reserve output, arranging decommissioning points, et cetera. Nikita emergency control. It includes restart, input of loads, juxtaposition of 69
24 et al. [168] proposed a centralized deep-grid preventive safety control power plants in the region, parallel connection of regional power grids, 70
25 based on hybrid DL, which uses a multi-layer perceptron (MLP) to reclosing of tie lines, and so on. 71
26 generate voltage/reactive power adjustment measures, and uses LSTM Ye et al. [171] proposed a hybrid multi-agent framework based on 72
27 to predict the voltage stability of the grid under this control measure. Q-learning for the restoration of power systems after a catastrophic 73
28 The simulation results obtained on the IEEE 118-node benchmark and disturbance like cascading failures. The hybrid restoration framework 74
29 the practical 33-bus Bodaibo grid show that the proposed preventive combines the benefits of centralized and decentralized approaches 75
30 control can effectively maintain the voltage stability with the help and requires less communication information and time consumption. 76
31 of the decentralized voltage control device. It indicates that the AI Also, Ghorbani et al. [172] developed a hierarchical distributed multi- 77
32 algorithms represented by DL are feasible for smart grid preventive agent system for power distribution systems with Q-learning to achieve 78
33 control. However, there are many types of power grid preventive the fault location, isolation, and restoration. The hierarchical system 79
34 control measures. The coordination of these measures is for further consists of zone agent, feeder agent and substation agent, of which 80
35 investigation.
the effectiveness is verified in a practical distribution system. Wu 81
et al. [173] proposed an effective sequential topology recovery method 82
36 4.4.6. Emergency control
using Q-learning for a complex power system with the consideration 83
37 Emergency controls are taken in an emergency state of the power
of potential cascading failure during the restoration. All the above 84
38 grid to make it return to a normal state or temporarily enter an alert
39 state, or taken in an in-extremis state to return it to its restorative successful applications show that the RL algorithm is functional for the 85
40 state. Emergency control measures include generator tripping, load restoration of the power system. 86
41 shedding, dynamic brake control, DC transmission power modulation, These applications to control are summarized in Table 4. 87
42 and system decommissioning. Different system operating conditions
43 and fault severities require different emergency control measures. Liu
4.5. Several typical application cases of AI for smart grid 88
44 et al. [168] proposed DDQN-based generator tripping emergency con-
45 trol measures, which used the random matrices and entropy to calculate
46 the power grid transient fluctuations as the reward to train the agent. In this subsection, we will provide an overview of several typical 89
47 The simulation results of the New England system verified the ef- application cases of AI for smart grids in a more comprehensive fashion. 90
48 fectiveness. Also, Huang et al. [169] proposed DQN-based dynamic The overview highlights the implementation and applied value of AI. 91
Z. Shi et al.

Table 4
Summary of AI’s applications to control.
Paper Year Task Method Descriptions / Remarks
[140] 2011 Generation control Q-learning • The NERC’s CPS-based AGC problem is modeled as an RL problem.
• Use semisupervisory group prelearning method.
• A large amount of remote information is required.
[141] 2015 Generation control Q-learning, WoLF-PHC • Anti-jamming problem of secondary users is formulated as Stackelberg
equilibrium problem.
[142], 2016, Generation control Wolf pack • Different types of renewable energy are regarded as decentralized agents.
[144], 2015, hunting/PDWoLF-
[145] 2018 PHC(𝜆)/Q-learning
• Multi-agent algorithms are used.
[143] 2018 Generation control DQN • Deep neural network is adopted by the action selector of Q learning to
speed up the convergence rate and mitigate the curse of dimensionality.
• May cause a systemic frequency deviation
[146] 2020 Generation control Lazy learning/RL • A lazy operator is integrated with RL to realize multiple-input
multiple-output.
[147] 2019 Generation control Multiobjective • Optimized the trade-off among economic cost, transient performance
reinforcement learning and emission with MORL
(MORL)
[148] 2016 AVR Teaching–learning, Sugeno • Robustness analysis is also performed to validate the performance of the
fuzzy logic designed TLBO-based PID controller.
[149] 2020 AVC DQN/DDPG • DDPG/DQN is adopted to control different objects according to
continuous/discrete action space.
[174] 2018 Damping control Fuzzy neural network • Adjust parameters of PSS with FNN
(FNN)
• The details of the model of the power systems are needed.
[150] 2018 Damping control Q-learning • Adjust the gain of PSS with RL and design a Q-learning controller
• An exact power system model is needed to tune parameters.
[152], 2013, Damping control Q-learning • A network of WD-PSS is designed based on an RL method.
[153] 2004
• The coordination between different PSSs might be difficult.
[154] 2014 Damping control Tree-based batch mode RL • Perform supplementary damping controller for multiple generators with
tree-based batch mode RL.
[156] 2016 Damping control Fuzzy-based GrHDP • Fuzzy logic controller as the action network of GrHDP.
• Only time delay is considered.
[157] 2019 Damping control GrHDP • Fuzzy logic module is used to adjust the learning rate of GrHDP.
• Other communication uncertainties are not considered.
[159], 2016, Frequency control Human brain emotional • Adjust the parameters of PI controller to self-tuning
[160], 2012, learning/particle swarm
[161], 2012, optimization/genetic
[162] 2014 algorithms/ fuzzy logic
• The methods are all model-based.
[158] 2016 Frequency control ADP • Introduced an action-dependent cost function to make training
model-free
• Employ a policy iteration algorithm to speed training speed
[175] 2018 Frequency control RADP • Use only partial states and input information
• Decentralized controller for every generator.
• Interactions between generators might be complex.
[164] 2019 Frequency control Internal RL • Develop a continuous-time mathematical framework that used the
integral forms of Bellman equation
• Use interactive online distributed learning system to adapt to multi-area
power system
[165] 2019 Frequency control DRL, SDAE • Policy network based DRL controller is used to keep local frequency as
constant.
• SDAE extracts input features as references for DRL controller.
• A complex nonlinear function needs to be maximized at each training
step.
[166] 2019 Frequency control LSTM • Identify active power fluctuations in real-time by LSTM for more
accuracy references of controller
• The uncertainties on the demand side are oversimplified.
[168] 2019 Preventive control MLP+LSTM • MLP is used to recognize reactive power injections for voltage control
and optimization, while LSTM network is used to predict system state.
[176] 2018 Emergency control CNN+DQN • The coordination of different emergency control is for further study.
[169] 2020 Emergency control DQN • It developed the first open-source platform for developing and
benchmarking DRL algorithms for power system control.
[170] 2018 Emergency control DQN • It used convolutional LSTM to extract the spatial–temporal information
including the topology information.
[171] 2011 Restoration control Q-Learning • Factors and constraints like power balance, dynamic characteristics are
not considered.
[172] 2016 Restoration control Q-Learning • Despite the neighbor’s message, each agent only transmits its own
message.
[173] 2019 Restoration control Q-Learning • Q-Learning is too simple for large-scale power grids with complex
operating.
 Z. Shi et al.

Fig. 16. The flow chart of the implementation of the transient stability prediction approach [105].

Fig. 17. The SR-CNN architecture for fault diagnosis [133].

1 4.5.1. Case 1: A CNN ensemble approach for transient stability prediction In summary, the application of AI to transient stability prediction 27
2 considering operational variability in smart grids can early detect the coming instability, thus remaining 28
3 Ref. [105] developed a transient stability prediction system with more time for emergency actions to prevent a possible collapse and 29
4 online updates using ensemble learning, cost-sensitive learning, active improving the security level of smart grids. And as the operating con- 30
5 learning, and fine-tuning techniques. As shown in Fig. 15, the approach ditions may change in practice, active learning and fine-tune strategies 31
6 is an ensemble of CNNs with different convolutional kernel sizes in are adopted for fast online updating. So the time costs of simulation and 32
7 the first layer. The input of the classifier is data cubes composed of model training are decreased, thus alleviating the practical obstacles of 33
8 normalized transient generators variables along the time axis such as
AI. 34
9 relative rotor angle, rotor speed, rotor acceleration, kinetic energy and
10 electromagnetic power.
11 The implementation of this approach is divided into three stages: 4.5.2. Case 2: Fault diagnosis based on SR-CNN (Spearman rank 35
12 offline training, period update and application. At offline training stage, correlation-based CNNs) 36
13 𝑁𝑜𝑐 × 𝑁𝑐 samples need to be generated to train the neural network for Ref. [133] developed a fault diagnosis system using SR-CNN. The 37
14 accurate stability prediction results, where 𝑁𝑜𝑐 and 𝑁𝑐 are the number system can detect faults and determine the fault type in Energy Internet 38
15 of possible operating conditions and contingencies, respectively. Since with a large amount of renewable energy generation and various new 39
16 power grids might operate under various conditions and be subjected loads. As is shown in Fig. 17, the neural network is composed of an 40
17 to different contingencies, online updates should be conducted to adapt
SR-image layer, several convolutional layers, pooling layers and fully 41
18 the model to new operating environments. The specific online update
connected layers. 42
19 method is shown in Fig. 16. With the active learning strategy, only
Specifically, CNN has superior performance in the computer vision 43
20 the incremented instances with high uncertainty (i.e., low confidence)
21 are considered important for the updating and should be annotated via domain, such as image recognition and object detection. But CNN fits 44
22 longer-term simulation. Then, the newly labeled instances are used to not well with fault diagnosis problem since the input is multivariate 45
23 fine-tune the CNN classifier. Since only a small number of instances time series rather than images. Simply putting the series side by side 46
24 require labeling and training, the time consumption can be reduced to to form a 2-D image may lead to weak correlations inside the local 47
25 a large degree. Then at the application stage, the CNN classifier can receptive fields of CNN, thus affecting its performance. To address 48
26 give transient stability rapidly. this problem, an SR-image layer is added before CNN to preprocess 49
 Z. Shi et al.

Fig. 18. SR-image built by SR-image layer [133].

Fig. 20. The flowchart of using the platform to train and test RL for power grid
control [169].
Fig. 19. The platform architecture of RL for power grid control [169].

5. Challenges and future directions of AI applications in smart 32

1 the input signal. For signal 𝑖 ∈ R𝑛×1 and 𝑗 ∈ R𝑛×1 , spearman rank grid stability analysis and control 33
2 correlation of them is calculated:
( )
cov rg𝑖 , rg𝑗 AI has the potential to resolve many challenging problems that have 34
3 𝜌𝑖,𝑗 = (1)
𝜎rg𝑖 𝜎rg𝑗 resisted the best attempts of conventional mechanism-based approaches 35
with favorable performance. Its applications to smart grids have been 36
4 where rg𝑖 and rg𝑗 are ranks of 𝑖 and 𝑗, respectively, and cov(⋅, ⋅) and a focus of intense research and do have achieved impressive results. 37
5 𝜎(⋅) are the covariance and standard deviations of the rank variables. However, these applications face some challenges in practice, such as 38
6 By normalizing the SR matrix, which is composed of 𝜌𝑖,𝑗 , the SR image high requirements on data, learning from imbalanced data, difficulties 39
7 is generated (Fig. 18). With the SR-image layer, the correlation of pixels in interpretation, difficulties in transfer learning, robustness to commu- 40
8 within the receptive field is enhanced, thus fitting CNN better. nication quality and robustness against attack or adversarial examples. 41
9 In summary, the introduction of AI accelerates the fault diagnosis In this section, we will discuss these challenges in terms of how these 42
10 with favorable accuracy in smart grids. Fault diagnosis makes the challenges arise, what impacts they have on the performance of AI, 43
11 self-healing feasible, which is an attractive feature of smart grids. and where the current corresponding research is heading. Furthermore, 44
we provide suggestions for potentially important future investigation 45
directions to overcome the challenges and bridge the gap between 46
12 4.5.3. Case 3: DRL for emergency control
research and practice. 47
13 Ref. [169] developed an open-source platform for the applications
14 of DRL for emergency control. Fig. 19 is the framework of the platform.
15 It contains two modules: the power grid simulation module and the DRL 5.1. High requirements on data 48
16 module. The former is the ‘environment,’ and it conducts simulation
17 with control actions to gain the state and reward of RL. The latter is
Many AI applications, especially ML and DL, are data-driven and 49
18 the ‘agent,’ and it leans an optimal action given the current state. The
depend heavily on the quantity and quality of data. This is an obvious 50
19 two modules interact with Py4J. The flexible and extensible platform
challenge of AI when applied to smart grids. 51
20 helps the development, validation and evaluation of RL for smart grid
21 control.
22 The flowchart of using the platform to train and test RL for power 5.1.1. High requirements on the quantity of data 52
23 grid control is shown in Fig. 20. Through interacting with the power Many AI algorithms require a large amount of data before they 53
24 grid and exploration and exploitation, the agent can learn optimal begin to give right and useful results. In the existing smart grid ap- 54
25 action policy, that is, when and where the control actions like generator plications, thousands of instances are usually needed. And the more 55
26 tripping and load shedding should be applied at the emergency state. complex the AI agent, the more instances are needed. Although the 56
27 These control actions aim to stop the propagation of insecurity in lack of data can be supplemented by simulation, simulation is usually 57
28 the interconnected power grid and restore it to a more secure state. time-consuming. And there exists a question of whether the simulation 58
29 And with RL approaches, emergency control is more effective but less can exhibit real-world operating conditions, which also requires serious 59
30 dependent on the expertise and human work, from which smart grids consideration. If AI agents face the real-world data that is significantly 60
31 benefit a lot. different from what they are trained on, their accuracy is likely to fall. 61
 Z. Shi et al.

1 5.1.2. High requirements on the quality of data In the AI community, imbalanced learning has been a focus of 62
2 In addition to quantity, favorable applications of AI to smart grids intense research. The emerging approaches can be divided into three 63
3 rely heavily on the data of high quality. How to efficiently collect or types: data-level methods, algorithm-level methods and hybrid meth- 64
4 generate datasets is also a challenge. Some pioneering studies have ods [180]. 65
5 achieved results in this direction. Ref. [177] proposed an efficient
- Data-level methods focus on modifying the training set to make 66
6 database generation method for power system security assessment. The
it suitable for standard AI approaches. Data-level methods in- 67
7 efficient sampling strategy in it maximizes the database information
clude undersampling (removing examples from majority class, 68
8 content while minimizing computing requirements. Ref. [178] further
e.g., random undersampling, NearMiss [181], ENN [182], Tomek- 69
9 developed a modular and highly scalable algorithm using convex re-
link [183]) and oversampling (replicating or synthesizing some 70
10 laxation and complex network theory to generate required operating
objects from the minority class, e.g., SMOTE [184], ADASYN 71
11 condition datasets for DSA. These studies provide efficient solutions
[185]). These methods concentrate on rebalancing the dataset. 72
12 to the challenge of the lack of good data as they can sample the
But one may ask: Should the classes be fully balanced? Or should 73
13 most influential operating conditions. However, they focused on the
the majority class stay the most represented? In fact, if the data- 74
14 generation of operating conditions rather than contingencies that are
level method is used, the classifier is learned on the non-real 75
15 required for many AI applications like TSA, VSA, et cetera. In fact,
dataset. In other words, the wrong proportions of the two or more 76
16 various element faults or loss may occur around the power grid; hence
classes are presented to the classifier during the training. So the 77
17 it is intractable to iterate through all the possibilities. So, how to effi-
classifier trained on the modified training dataset may have lower 78
18 ciently generate a contingency set is also an issue to AI’s applications.
accuracy when tested with real data than the classifier trained on 79
19 And the existing dataset generation methods are independent of the
the original dataset. 80
20 targeted AI approaches. So an interesting investigation is to use AI
Moreover, whether or not the minority class examples generated 81
21 methods to guide the dataset generation, transform sample distribution
via SMOTE, ADASYN or other advanced algorithms are reason- 82
22 and improve the balance, diversity, and reality of the dataset.
able is also a question. We believe that power grid data is more 83
complicated but less intuitive in some ways than images, to which 84
23 5.1.3. Ignorance of expertise in smart grids
these synthesis algorithms are frequently applied. Obviously, we 85
24 To date, most AI applications in smart stability analysis and control can easily judge whether the generated images are realistic, but 86
25 are data-driven. That is to say, little or no domain expertise is used. it is even harder for power grid data. So gaining insight into the 87
26 In fact, the smart grid is different from other application domains, quality of synthesized examples is still a challenge. 88
27 such as computation vision and speech recognition, after all. We have Furthermore, whether directly replicating or synthesizing exam- 89
28 accumulated a large amount of experience and knowledge from the ples for the minority class, the low diversity in minority class 90
29 operation of power systems in the past decades. It would obviously is always a concern since we cannot create additional informa- 91
30 be a waste to discard all of them. And the expertise helps to alleviate tion from the original dataset. In other words, in spite of the 92
31 AI’s dependence on data and improves its performance. So a potentially increase in the number of minority class samples, the diversity of 93
32 feasible attempt is to utilize the expertise, and combine data-driven them may increase very little, which may cause the over-fitting 94
33 methods with symbolic AI or knowledge engineering. problem. 95
34 To sum up, in regards to the high requirement on the quantity and Therefore, we should be careful when modifying the dataset 96
35 quality of data, we think that the following questions are worthy of with resampling methods since these methods tend to change the 97
36 in-depth investigation: reality. 98
37 - How can simulation exhibit practical operating conditions since - Algorithm-level methods focus on modifying existing learners to 99
38 the required data can be supplemented by simulations? guide them to care more about the minority class and alleviate 100
39 - How can we generate an efficient contingency set instead of their bias towards the majority class. The unfavorable perfor- 101
40 iterating through all the possibilities? mance on an imbalanced dataset can be partially attributed to 102
41 - Can AI, in turn, guide the generation of the samples it needs, and the fact that the objective functions of the learners are not well 103
42 improve the balance, diversity, and reality of the dataset? defined. Hence, one of the most popular algorithm-level methods 104
43 - Can power grid expertise be incorporated into data-driven AI is cost-sensitive learning [180]. For example, in TSA we assign 105
44 approaches to alleviate its dependence on data of high volume a high cost or penalty to misclassification of unstable samples 106
45 and quality? to boost their importance during the training process [111,112]. 107
Some research has shown that the cost-sensitive approach is 108
superior to data-level sampling approaches in some application 109
46 5.2. Learning from imbalanced data
domains [186–188]. But in practice, it is usually challenging to 110
set the actual value in the cost matrix, which requires consider- 111
47 AI may face the imbalanced learning problem when applied to smart
able domain expertise. Hence, in the future, how to gain a deep 112
48 grids. Taking TSA as an example, modern smart grids can remain stable
insight into the modified AI approaches and how to reasonably 113
49 after being subjected to most disturbances due to its robustness. Hence,
set the hyperparameters in cost-sensitive learning should be fully 114
50 in a TSA dataset, the number of stable samples (majority class) usually
understood. 115
51 exceeds the number of unstable samples (minority class) greatly.
- Hybrid methods combine the advantages of the above two meth- 116
52 However, most standard AI approaches assume or expect balanced
ods. For example, combining data-level sampling methods with 117
53 class distributions or equal misclassification costs [179]. That is to
classifier ensembles to avoid over-fitting, is a very popular ap- 118
54 say, the goal of most standard approaches is to optimize the overall
proach to cope with imbalances. 119
55 performance across all the classes. Therefore, when fed with highly
56 imbalanced datasets, these methods fail to discover the underlying Furthermore, apart from real-world data, most AI applications in 120
57 distribution characteristics of the data, thereby providing unfavorable smart grids are based on simulation data. Since simulation is under 121
58 performance. Since stability is a safety-critical issue, judging unstable our control to some extent, how to reasonably and efficiently generate 122
59 samples as stable ones costs much more than the reverse. We require balanced datasets is also an interesting investigation direction. 123
60 an approach that provides high accuracy for the minority class without To sum up, in regards to imbalanced learning, we believe that the 124
61 severely sacrificing the accuracy of the majority class. following questions need to be addressed: 125
 Z. Shi et al.

1 - For sampling methods, to what extent should the original dataset 5.4. Difficulties of transfer learning applied in smart grids 60
2 be balanced? Is it possible that the modified distribution charac-
3 teristics have a negative impact on classification performance? Up to now, many kinds of AI models have been discussed in smart 61
4 - For sampling methods, how can we reasonably evaluate the qual- grid applications with satisfactory performances. However, there are 62
5 ity of synthesized examples in the power grid? few studies about applications of TL in smart grids in the literature, 63
6 - For cost-sensitive learning, how can we better set the actual value which remains several problems to be studied. For a specific power 64
7 in the cost matrix instead of trial and error? grid, some tasks are closely related. For example, fault detection, clas- 65
8 - How can we control the simulation condition and process to sification and location models all analyze the data or features acquired 66
9 generate a more balanced and efficient dataset? from the same grid. Despite differences between tasks, it is reasonable 67
to believe that knowledge of one task may help the training of the 68
others related. 69
10 5.3. Interpretability of AI approaches applied to smart grids Another expectation of TL is the transference of tasks across dif- 70
ferent grids. Models trained on different power grids are expected to 71
11 Although AI has been applied to many domains in smart grids, extract some features in common for the same task using TL. These 72
12 model interpretability, the process of giving explanations to humans, features should be related to the mechanisms of the task that can be 73
13 has always been its limiting factor. Our need for interpretability arises transferred. The main difficulty is how to design algorithms with the 74
14 from the fact help of relevant knowledge of power systems to filter the information 75
related to the specific power grid while training models. But it is 76
15 that we want to know what the AI model has learned from the
not easy to achieve since the two parts of information are highly 77
16 data – expressed in a way that humans can understand – to produce
dimensional and coupled in a large-scale power grid. Feature-based 78
17 the final decision, and whether it can help us discover some potential
TL algorithms such as transfer component analysis [194], subspace 79
18 associations. For example, for the model of TSA, in addition to the
alignment [195] that can discover common features between source 80
19 final determination result, we also want to know what characteristics
and target domain can be regarded as a feasible solution to the two 81
20 of the grid make the model output an unstable judgment. Taking problems. 82
21 control measures against these characteristics will be more conducive In addition, due to the complicated transient characteristics of 83
22 to system stability than blind control. Moreover, no interpretability power systems, the labels of samples still rely on the efforts of experts, 84
23 means danger, especially in application scenarios with high security which is hard to be substituted by machines. It means that we should 85
24 requirements such as the smart grid. A classic case is to add carefully make full use of existing labeled samples to accelerate the training 86
25 crafted noise to the panda image to create a new image that people process with a better initial point and improve the generalization 87
26 would see as identical, but which a CNN sees as a gibbon, and we do not abilities of models for another grid. 88
27 know what is going wrong [189]. Conversely, for interpretable models, To sum up, in regards to transfer learning in smart grids, we think 89
28 it is possible to track and locate the cause of the abnormality when that the following questions should be answered: 90
29 facing these problems, thus discovering a lot about why it fails.
- How can we transfer the knowledge from one task to another 91
30 Interpretability since the beginning has been an important area of
related task in the same grid? 92
31 AI research. But in the smart grid, this is a little-studied aspect. In
- How can we make full use of existing samples and the trained 93
32 general, the performance and the interpretability of AI approaches are
model of the source domain grid for the target domain grid? 94
33 two trade-off factors [190–192]. Some approaches achieve superior
- How can we extract features which are useful for the training in 95
34 performance at the cost of high abstraction or vice versa (Fig. 21). And
target domain grids? 96
35 DT strikes a fairly good balance between the two aspects. Based on DT,
- How can we use knowledge of power systems to help design 97
36 Ref. [193] leveraged disjunctive programming to formulate novel train- efficient TL algorithms? 98
37 ing methods to construct high-quality DTs with good interpretability for
38 security assessment. Ref. [191] proposed a local linear interpreter (LLI)
5.5. Robustness to communication quality 99
39 model to give a reasonable interpretation of the DBN output result. LLI
40 fits DBN with a simple linear model and indicates the significant factors
Communication uncertainties like noise, data loss and time-delay 100
41 relative to system instability, which can be used to guide emergency
are unavoided in practical systems, whose effect on stability and control 101
42 control. has been studied in many previous studies [94,157]. However, these 102
43 Existing applications rarely consider interpretability, and the re- issues would still exist and are even more critical in the application of 103
44 search of only a few references is also at a preliminary stage. In fact, the AI since its characteristic of data-driven means data-sensitive. The cor- 104
45 black-box property hinders AI’s application to smart grids since most AI rupted samples would not only mislead the models to learn irrelevant 105
46 agents cannot reach 100% accuracy, and we have no idea what is going information from uncertainties, affecting the fitting to the real system 106
47 wrong when AI fails. As power grid stability and control is a safety- but also undermine the accuracy of the application. Furthermore, data 107
48 critical issue, it is significant for operators to understand the decisions transmission through communication networks would be more frequent 108
49 made by AI agents and locate the cause of the abnormality. Only then and necessary with the development of Internet of Things (IoT) [196], 109
50 can AI be trusted by operators and be widely applied to smart grid aggravating the issues. Thus, a higher communication quality must be 110
51 stability analysis and control. ensured, and robust AI structures should be developed to handle these 111
52 To sum up, in regards to the interpretability, we believe that the uncertainties. 112
53 following questions are worthy of in-depth investigation: In order to improve the robustness to communication quality, fea- 113
ture extraction methods like principal component analysis (PCA), SVM, 114
54 - How can we balance the trade-off between AI’s performance and DT or predicting methods like LSTM might contribute to estimating 115
55 interpretability? relatively accurate signals from raw measurements, which will be an 116
56 - How can we interpret the decisions made by AI agents? interesting research direction. Besides, the future work can also focus 117
57 - How do we measure the explanation quality? on designing a robust AI structure. 118
58 - How can we locate the cause of the abnormality of AI when it To sum up, we think that the following questions should be ad- 119
59 fails? dressed: 120
 Z. Shi et al.

Fig. 21. The trade-off relationship between model accuracy and interpretability [192]. DT strikes a fairly good balance between the two aspects.

1 - Is it possible to adopt state-of-the-art communication technologies smart grid have made its stability characteristics much more complex 39
2 to ensure a high communication quality? than the past. Under these conditions, conventional stability analysis 40
3 - How can we filter out the information of noise or loss before the and control approaches are ineffective in meeting its high requirements 41
4 measurements enter the models? of security and stability. With the recent advances in AI and the accu- 42
5 - How should we do to design a robust AI structure against com- mulation of data in smart grids, AI techniques provide powerful tools 43
6 munication uncertainties? for stability analysis and control in smart grids. In this paper, we first 44
presented a general overview of AI, including its definitions, history 45
7 5.6. Robustness against attack or adversarial examples and state-of-the-art methodologies. And then, we analyzed AI’s appli- 46
cability to smart grids and provided a comprehensive and systematic 47
8 Apart from communication uncertainties, the malicious cyber attack review of its applications to security assessment, stability assessment, 48
9 is another severe problem facing the IoT-integrated smart grids. The fault diagnosis and stability control in smart grids. 49
10 Ukraine outage in 2016 has confirmed the serious consequence of cyber In summary, AI resolves many challenging problems that have 50
11 attacks on power systems [197]. Although both communication un- resisted the best attempts of conventional mechanism-based approaches 51
12 certainties and attacks degrade AI performance by injecting irrelevant with favorable performance. It improves the efficacy and efficiency of 52
13 or bad data into the measurements, the cyber attack is more stealthy the stability analysis and control, improves the security of smart grids, 53
14 and hard to bear because it is well-designed by hackers. Moreover, and frees itself from reliance on expertise and human work. However, 54
15 the attack strategies will always be changing through the efforts of AI’s applications still face a series of challenges in practice. They are: 55
16 hackers. Thus, it will be a challenging and endless problem to detect
17 and mitigate the attacks. For example, recent studies indicate that a - High requirements on data. In practice, data are usually scarce. 56
18 specific type of attack, adversarial examples [189], could fool the AI to Future research could be directed towards improving the effi- 57
19 show good accuracy or effect, but they are actually harmful. ciency of datasets and incorporating power grid expertise into 58
20 Similar to communication quality, feature exaction or prediction data-driven AI approaches to alleviate its dependence on data. 59
21 methods can play an important role in realizing the robustness against - Learning from imbalanced data. Most smart grid datasets are 60
22 attack or adversarial examples as well. A difference is that attack is imbalanced due to its robustness. Future research could be im- 61
23 stealthy, so the first step is to manage to detect the attack. The Bayesian proving existing imbalanced learning approaches and controlling 62
24 network or GAN can be explored. simulation conditions to generate a more balanced and efficient 63
25 To sum up, we believe that the following questions are expected to dataset. 64
26 be addressed. - Interpretability. The black-box property may hinder AI’s appli- 65
cation to smart grids. Future research could be balancing the 66
27 - How can we detect the existence of attacks? And what should we trade-off between performance and interpretability and develop- 67
28 do to mitigate their impacts on stability and control?
ing ways to locate the abnormality of AI when it fails. 68
29 - In terms of different types of attacks like adversarial examples,
- Difficulties of transfer learning. Transfer learning in smart grids 69
30 what strategies should we formulate?
is difficult because the general knowledge and specific knowl- 70
31 - What improvements do we make to enhance the robustness of AI edge related to one power grid are highly dimensional and cou- 71
32 models?
pled. Future directions could be leveraging expert knowledge and 72
33 For clarity, the aforementioned challenges and future directions are developing more effective transfer learning approaches. 73
34 shown in Fig. 22. We believe that these questions are important for - Robustness to communication quality. Noise, data loss and time- 74
35 bridging the gap between research and practice. delay may undermine the performance of AI. Future directions 75
could be estimating accurate signals from raw measurements, 76
36 6. Conclusion designing a robust AI structure or adopting state-of-the-art com- 77
munication technologies to ensure a high communication quality. 78
37 Smart grids provide a platform for clean, sustainable, efficient and - Robustness against attack or adversarial examples. Malicious cy- 79
38 reliable energy generation, delivery and use. The new features of the ber attacks or adversarial attacks are very harmful to AI agents. 80
 Z. Shi et al.

Fig. 22. Summary of the challenges and future directions.

1 Future research could be detecting the attacks and enhancing the References 17
2 robustness of AI agents.
[1] Arnold GW. Challenges and opportunities in smart grid: A position article. Proc 18
3 We have particularly discussed these challenges and put forward IEEE 2011;99:922–7. 19
4 suggestions to future investigation directions to overcome the chal- [2] Kroposki B, Johnson B, Zhang Y, Gevorgian V, Denholm P, Hodge BM, et 20
5 lenges, aiming for bridging the gap between research and practice. We al. Achieving a 100% renewable grid: Operating electric power systems with 21
6 believe that AI will play an increasingly important role in ensuring extremely high levels of variable renewable energy. IEEE Power Energy Mag 22
2017;15:61–73. 23
7 secure and stable operation of the smart grids. [3] Hossain MS, Madlool NA, Rahim NA, Selvaraj J, Pandey AK, Khan AF. Role 24
of smart grid in renewable energy: An overview. Renew Sustain Energy Rev 25
8 Declaration of competing interest 2016;60:1168–84. 26
[4] Gharavi H, Ghafurian R. Smart grid: The electric energy system of the future. 27
Proc IEEE 2011;99:917–21. 28
9 The authors declare that they have no known competing finan- [5] Yu X, Xue Y. Smart grids: A cyber-physical systems perspective. Proc IEEE 29
10 cial interests or personal relationships that could have appeared to 2016;104:1058–70. 30
11 influence the work reported in this paper. [6] Bose BK. Power electronics, smart grid, and renewable energy systems. Proc 31
IEEE 2017;105:2011–8. 32
[7] Wang Q, Yao W, Fang J, Ai X, Wen J, Yang X, et al. Dynamic modeling and 33
12 Uncited references small signal stability analysis of distributed photovoltaic grid-connected system 34
with large scale of panel level DC optimizers. Appl Energy 2020;259:114132. 35
[8] Sun K, Yao W, Fang J, Ai X, Wen J, Cheng S. Impedance modeling and stability 36
13 [198]
analysis of grid-connected DFIG-based wind farm with a VSC-HVDC. IEEE J 37
Emerg Sel Top Power Electron 2020;8:1375–90. 38
14 Acknowledgment [9] Peng X, Yao W, Yan C, Wen J, Cheng S. Two-stage variable proportion 39
coefficient based frequency support of grid-connected DFIG-WTs. IEEE Trans 40
Power Syst 2020;35:962–74. 41
15 This work was supported by National Natural Science Foundation [10] Bose BK. Artificial intelligence techniques in smart grid and renewable energy 42
16 of China (U1866602). systems—Some example applications. Proc IEEE 2017;105:2262–73. 43
 Z. Shi et al.

1 [11] Li F, Du Y. From alphaGo to power system AI : What engineers can learn from [46] LeCun Y, Bengio Y, Hinton GE. Deep learning. Nature 2015;521:436–44. 81
2 solving the most complex board game. IEEE Power Energy Mag 2018;16:76–84. [47] Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural 82
3 [12] Clark J. Bloomberg: Why was a breakthrough year in artificial intelligence. networks. Science 2006;313:504–7. 83
4 Bloom Technol 2015;18(2015):2, [Online Journal]. [48] Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. 84
5 [13] Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Neural Comput. 2006;18:1527–54. 85
6 Natural language processing (almost) from scratch. J Mach Learn Res [49] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep 86
7 2011;12:2493–537. convolutional neural networks. In: Advances in Neural Information Processing 87
8 [14] Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C, et al. Systems. 2012, p. 1097–105. 88
9 Deep Speech 2 : End-to-end speech recognition in english and mandarin. In: [50] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale 89
10 Proc 33rd int conf mach learn. 2016. p. 173–82. image recognition. In: 3rd Int Conf Learn Represent. 2015. 90
11 [15] Taigman Y, Yang M, Ranzato M, Wolf L. DeepFace: Closing the gap to human- [51] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper 91
12 level performance in face verification. In: Proc IEEE Comput Soc Conf Comput with convolutions. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 92
13 Vis Pattern Recognit. 2014, p. 1701–8. http://dx.doi.org/10.1109/CVPR.2014. 2015, p. 1–9. http://dx.doi.org/10.1109/CVPR.2015.7298594. 93
14 220. [52] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improv- 94
15 [16] Kim J, Lee JK, Lee KM. Accurate image super-resolution using very deep ing neural networks by preventing co-adaptation of feature detectors. 2012, 95
16 convolutional networks. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Available from: arXiveprint:1207.0580. 96
17 Recognit. 2016, p. 1646–54. http://dx.doi.org/10.1109/CVPR.2016.182. [53] Srivastava N, Hinton G, Krizhevsky A, Salakhutdinov R. Dropout: A sim- 97
18 [17] Levinson J, Askeland J, Becker J, Dolson J, Held D, Kammel S, et al. Towards ple way to prevent neural networks from overfitting. J Mach Learn Res 98
19 fully autonomous driving: Systems and algorithms. In: IEEE Intell Veh Symp 2014;15:1929–58. 99
20 Proc.. 2011, p. 163–8. http://dx.doi.org/10.1109/IVS.2011.5940562. [54] Elman JL. Finding structure in time. Cogn Sci 1990;14:179–211. 100
21 [18] Yurtsever E, Lambert J, Carballo A, Takeda K. A survey of autonomous driving: [55] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput 101
22 Common practices and emerging technologies. IEEE Access 2020;8:58443–69. 1997;9:1735–80. 102
23 [19] Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, et al. [56] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning 103
24 Mastering the game of Go with deep neural networks and tree search. Nature to align and translate. 2014, Available from: arXiveprint:1409.0473. 104
25 2016;529:484. [57] Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. 105
26 [20] Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, et al. Generative adversarial nets. In: Adv Neural Inf Process Syst. 2014, p. 2672–80. 106
27 Mastering the game of Go without human knowledge. Nature 2017;550:354–9.
[58] Ballard DH. Modular Learning in Neural Networks. In: AAAI. 1987. p. 279–84. 107
28 [21] Silver D, Hubert T, Schrittwieser J, Antonoglou I, Lai M, Guez A, et al. A general
[59] Kingma DP, Welling M. Auto-encoding variational bayes. 2013, Available from: 108
29 reinforcement learning algorithm that masters chess, shogi, and go through
arXiveprint:1312.6114. 109
30 self-play. Science 2018;362:1140–4.
[60] Kipf TN, Welling M. Semi-supervised classification with graph convolutional 110
31 [22] Marugán AP, Márquez FPG, Perez JMP, Ruiz-Hernández D. A survey of artificial
networks. In: International Conference on Learning Representations. 2017. 111
32 neural network in wind energy systems. Appl Energy 2018;228:1822–36.
[61] Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS. A comprehensive survey on 112
33 [23] Vázquez-Canteli JR, Nagy Z. Reinforcement learning for demand response: A
graph neural networks. IEEE Trans Neural Networks Learn Syst 2020;1–21. 113
34 review of algorithms and modeling techniques. Appl Energy 2019;235:1072–89.
[62] Kipf TN, Welling M. Semi-supervised classification with graph convolutional 114
35 [24] O’Dwyer E, Pan I, Acha S, Shah N. Smart energy systems for sustainable
networks. In: 5th Int Conf Learn Represent. 2017. 115
36 smart cities: Current developments, trends and future directions. Appl Energy
37 2019;237:581–97.
[63] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on 116
38 [25] Crevier D. AI: The Tumultuous History of the Search for Artificial Intelligence.
graphs with fast localized spectral filtering. In: Adv. Neural Inf Process Syst. 117
39 Basic Books Inc.; 1993.
2016, p. 3844–52. 118
40 [26] Nilsson NJ, Nilsson NJ. Artificial Intelligence: A New Synthesis. Morgan
[64] Zhao L, Song Y, Zhang C, Liu Y, Wang P, Lin T, et al. T-GCN: A temporal 119
41 Kaufmann; 1998.
graph convolutional network for traffic prediction. IEEE Trans Intell Transp 120
42 [27] Poole D, Mackworth A, Goebel R. Computational Intelligence: A Logical
Syst 2019;1–11. 121
43 Approach. Oxford University Press; 1998.
[65] Ge L, Li H, Liu J, Zhou A. Temporal graph convolutional networks for traffic 122
44 [28] Russell S, Norvig P. Artificial Intelligence: A Modern Approach. Prentice Hall;
speed prediction considering external factors. In: Proc - IEEE Int Conf Mob Data 123
45 2002.
Manag. 2019, p. 234–42. http://dx.doi.org/10.1109/MDM.2019.00--52. 124
46 [29] Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016.
[66] Nagabandi A, Kahn G, Fearing RS, Levine S. Neural network dynamics for 125
47 [30] McCorduck P, Cfe C. Machines Who Think: A Personal Inquiry Into the History
model-based deep reinforcement learning with model-free fine-tuning. In: Proc 126
48 and Prospects of Artificial Intelligence. CRC Press; 2004.
- IEEE Int Conf Robot Autom. 2018, p. 7579–86. http://dx.doi.org/10.1109/ 127
49 [31] Li S, Shahidehpour SM, Wang C. Promoting the application of expert systems
ICRA.2018.8463189. 128
50 in short-term unit commitment. IEEE Trans Power Syst 1993;8:286–92. [67] Ha D, J. Schmidhuber. World models. 2018, http://dx.doi.org/10.5281/zenodo. 129
51 [32] Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:273–97. 1207631, Available from: arXiveprint:1803.10122. 130
52 [33] Safavian SR, Landgrebe D. A survey of decision tree classifier methodology. [68] OpenAI. Welcome to spinning up in deep RL!. 2018, https://spinningup.openai. 131
53 IEEE Trans Syst Man Cybern 1991;21:660–74. com/en/latest/, [Accessed 30 April 2020]. 132
54 [34] Rosenblatt F. The perceptron: A probabilistic model for information storage and [69] Mnih V, Badia AP, Mirza M, Graves A, Lillicrap TP, Harley T, et al. Asyn- 133
55 organization in the brain. Psychol Rev 1958;65:386–408. chronous methods for deep reinforcement learning. In: 33rd int conf mach learn. 134
56 [35] Leshno M, Lin VY, Pinkus A, Schocken S. Multilayer feedforward networks 2016. p. 2850–69. 135
57 with a nonpolynomial activation function can approximate any function. Neural [70] Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O. Proximal policy 136
58 Netw 1993;6:861–7. optimization algorithms. 2017, Available from: arXiveprint:1707.06347. 137
59 [36] Bin HuangG, Zhu QY, Siew CK. Extreme learning machine: A new learning [71] Schulman J, Levine S, Moritz P, Jordan M, Abbeel P. Trust region policy 138
60 scheme of feedforward neural networks. In: IEEE Int Conf Neural Networks. optimization. In: 32nd Int Conf Mach Learn. 2015. p. 1889–97. 139
61 2004, p. 985–90. http://dx.doi.org/10.1109/IJCNN.2004.1380068. [72] Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, et al. Continuous 140
62 [37] Stratonovich RL. Conditional Markov processes. Theory Probab Appl control with deep reinforcement learning. In: 4th int conf learn represent. 2016. 141
63 1960;5:156–78. [73] Fujimoto S, Van Hoof H, Meger D. Addressing function approximation error in 142
64 [38] Rish I. An empirical study of the naive Bayes classifier. In: IJCAI. 2001. p. actor-critic methods. In: 35th int conf mach learn. 2018. p. 2587–601. 143
65 41–6. [74] Haarnoja T, Zhou A, Abbeel P, Levine S. Soft actor-critic: Off-policy maximum 144
66 [39] Breiman L. Bagging predictors. Mach Learn 1996;24:123–40. entropy deep reinforcement learning with a stochastic actor. In: 35th int conf 145
67 [40] Friedman JH. Greedy function approximation: A gradient boosting machine. mach learn. 2018. p. 2976–89. 146
68 Ann Stat 2001;29:1189–232. [75] Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, et 147
69 [41] Chen T, Guestrin C. XGBoost: A scalable tree boosting system. In: Proc ACM al. Playing atari with deep reinforcement learning. 2013, Available from: 148
70 SIGKDD Int Conf Knowl Discov Data Min. 2016, p. 785–94. http://dx.doi.org/ arXiveprint:1312.5602. 149
71 10.1145/2939672.2939785. [76] Bellemare MG, Dabney W, Munos R. A distributional perspective on 150
72 [42] Settles B. Active Learning Literature Survey. Computer Sciences Technical reinforcement learning. In: 34th int conf mach learn. 2017. p. 693–711. 151
73 Report 1648, University of Wisconsin–Madison; 2009. [77] Dabney W, Rowland M, Bellemare MG, Munos R. Distributional reinforcement 152
74 [43] Pan SJ, Yang Q. A survey on transfer learning. IEEE Trans Knowl Data Eng learning with quantile regression. In: 32nd AAAI conf artif intell. 2018. 153
75 2010;22:1345–59. p.2892–901. 154
76 [44] Ho TK. Random decision forests. In: Proc Int Conf Doc Anal Recognition. 1995, [78] Andrychowicz M, Wolski F, Ray A, Schneider J, Fong R, Welinder P, et al. 155
77 p. 278–82. http://dx.doi.org/10.1109/ICDAR.1995.598994. Hindsight Experience Replay. In: Adv Neural Inf Process Syst. 2017. p. 5049–59. 156
78 [45] Robert N, Steve H. ICML 2019 tutorial: Active learning: From theory to [79] Weber T, Racanière S, Reichert DP, Buesing L, Guez A, Rezende DJ, et al. 157
79 practice. 2019, https://icml.cc/Conferences/2019/ScheduleMultitrack?event= Imagination-augmented agents for deep reinforcement learning. In: adv neural 158
80 4341, [Accessed 14 July 2020]. inf process syst. 2017. p. 5691–702. 159
 Z. Shi et al.

1 [80] Feinberg V, Wan A, Stoica I, Jordan MI, Gonzalez JE, Levine S. Model- [108] Zhang R, Xu Y, Dong ZY, Wong KP. Post-disturbance transient stability 80
2 based value estimation for efficient model-free reinforcement learning. 2018, assessment of power systems by a self-adaptive intelligent system. IET Gener 81
3 Available from: arXiveprint:1803.00101. Transm Distrib 2015;9:296–305. 82
4 [81] Ye Y, Qiu D, Sun M, Papadaskalopoulos D, Strbac G. Deep reinforcement [109] Hosseini H, Naderi S, Afsharnia S. New approach to transient stability prediction 83
5 learning for strategic bidding in electricity markets. IEEE Trans Smart Grid of power systems in wide area measurement systems based on multiple-criteria 84
6 2020;11:1343–55. decision making theory. IET Gener Transm Distrib 2019;13:4960–7. 85
7 [82] Xu H, Sun H, Nikovski D, Kitamura S, Mori K, Hashimoto H. Deep reinforcement [110] Chen Z, Han X, Fan C, He Z, Su X, Mei S. A data segmentation-based ensemble 86
8 learning for joint bidding and pricing of load serving entity. IEEE Trans Smart classification method for power system transient stability status prediction with 87
9 Grid 2019;10:6366–75. imbalanced data. Appl Sci 2019;9:4216. 88
10 [83] Hua H, Qin Y, Hao C, Cao J. Optimal energy management strategies [111] Tan B, Yang J, Tang Y, Jiang S, Xie P, Yuan W. A deep imbalanced learning 89
11 for energy internet via deep reinforcement learning approach. Appl Energy framework for transient stability assessment of power system. IEEE Access 90
12 2019;239:598–609. 2019;7:81759–69. 91
13 [84] Duan J, Xu H, Liu W. Q-learning-based damping control of wide-area power [112] Chen Z, Xiao X, Li C, Zhang Y, Hu Q. Real-time transient stability status 92
14 systems under cyber uncertainties. IEEE Trans Smart Grid 2018;9:6408–18. prediction using cost-sensitive extreme learning machine. Neural Comput Appl 93
15 [85] Morison K, Wang L, Kundur P. Power system security assessment. IEEE Power 2016;27:321–31. 94
16 Energy Mag 2004;2:30–9. [113] Phootrakornchai W, Jiriwibhakorn S. Online critical clearing time estimation 95
17 [86] Kundur P, Paserba J, Ajjarapu V, Andersson G, Bose A, Canizares C, et al. using an adaptive neuro-fuzzy inference system (ANFIS). Int J Electr Power 96
18 Definition and Classification of Power System Stability, Vol. 19. 2004, p. Energy Syst 2015;73:170–81. 97
19 1387–401. [114] Sulistiawati IB, Priyadi A, Qudsi OA, Soeprijanto A, Yorino N. Critical clearing 98
20 [87] Kalyani S, Swarup KS. Design of pattern recognition system for static security time prediction within various loads for transient stability assessment by means 99
21 assessment and classification. PAA Pattern Anal Appl 2012;15:299–311. of the extreme learning machine method. Int J Electr Power Energy Syst 100
22 [88] Thirugnanasambandam V, Jain T. AdaBoost classifiers for phasor measurements- 2016;77:345–52. 101
23 based security assessment of power systems. IET Gener Transm Distrib [115] Zhu L, Lu C, Sun Y. Time series shapelet classification based online short–term 102
24 2018;12:1747–55. voltage stability assessment. IEEE Trans Power Syst 2016;31:1430–9. 103
25 [89] Xu Y, Dong ZY, Zhao JH, Zhang P, Wong KP. A reliable intelligent system for [116] Praprost KL, Loparo KA. An energy function method for determining voltage 104
26 real-time dynamic security assessment of power systems. IEEE Trans Power Syst collapse during a power system transient. IEEE Trans Circuits Syst I Fundam 105
27 2012;27:1253–63. Theory Appl 1994;41:635–51. 106
28 [90] Xu Y, Dong ZY, Xu Z, Meng K, Wong KP. An intelligent dynamic security
[117] Huang GM, Zhao L, Song X. A new bifurcation analysis for power system 107
29 assessment framework for power systems with wind power. IEEE Trans Ind
dynamic voltage stability studies. In: Proc IEEE Power Eng Soc Transm Distrib 108
30 Informatics 2012;8:995–1003.
Conf. 2002, p. 882–7. http://dx.doi.org/10.1109/pesw.2002.985133. 109
31 [91] Zhang Y, Xu Y, Dong ZY, Xu Z, Wong KP. Intelligent early warning of power
[118] Zhu L, Lu C, Dong ZY, Hong C. Imbalance learning machine-based power 110
32 system dynamic insecurity risk: Toward optimal accuracy-earliness tradeoff.
system short-term voltage stability assessment. IEEE Trans Ind Inform 111
33 IEEE Trans Ind Informatics 2017;13:2544–54.
2017;13:2533–43. 112
34 [92] Sun M, Konstantelos I, Strbac G. A deep learning-based feature extrac-
[119] Xu Y, Zhang R, Zhao J, Dong ZY, Wang D, Yang H, et al. Assessing short-term 113
35 tion framework for system security assessment. IEEE Trans Smart Grid
voltage stability of electric power systems by a hierarchical intelligent system. 114
36 2018;10:5007–20.
IEEE Trans Neural Networks Learn Syst 2016;27:1686–96. 115
[120] Zhang Y, Xu Y, Dong ZY, Zhang R. A hierarchical self-aadaptive data-analytics 116
37 [93] Liu R, Verbi? G, Ma J. A new dynamic security assessment framework
method for real-time power system short-term voltage stability assessment. IEEE 117
38 based on semi-supervised learning and data editing. Electr Power Syst Res
Trans Ind Informatics 2019;15:74–84. 118
39 2019;172:221–9.
[121] Pinzón JD, Colomé DG. Real-time multi-state classification of short-term voltage 119
40 [94] Ren C, Xu Y. A fully data-driven method based on generative adversarial
stability based on multivariate time series machine learning. Int J Electr Power 120
41 networks for power system dynamic security assessment with missing data.
Energy Syst 2019;108:402–14. 121
42 IEEE Trans Power Syst 2019;34:5044–52.
[122] Su HY, Liu TY. Enhanced-online-random-forest model for static voltage sta- 122
43 [95] Ren C, Xu Y. Transfer learning-based power system online dynamic security
bility assessment using wide area measurements. IEEE Trans Power Syst 123
44 assessment: using one model to assess many unlearned faults. IEEE Trans Power
2018;33:6696–704. 124
45 Syst 2020;35:821–4.
[123] Xu Y, Dai Y, Dong ZY, Zhang R, Meng K. Extreme learning machine-based 125
46 [96] Venkatesh T, Jain T. Synchronised measurements based transient security
predictor for real-time frequency stability assessment of electric power systems. 126
47 assessment of power systems using AdaBoost classifiers. IET Gener Transm
Neural Comput Appl 2013;22:501–8. 127
48 Distrib 2019;13:3980–7.
[124] Tang Y, Cui H, Wang Q. Prediction model of the power system frequency using 128
49 [97] Gupta A, Gurrala G, Sastry PS. An Online power system stability moni- a cross-entropy ensemble algorithm. Entropy 2017;19:1–17. 129
50 toring system using convolutional neural networks. IEEE Trans Power Syst
[125] Wang Q, Zhang C, Ying L, Yu Z, Tang Y. Data inheritance-based updating 130
51 2019;34:864–72.
method and its application in transient frequency prediction for a power system. 131
52 [98] Gomez FR, Rajapakse AD, Annakkage UD, Fernando IT. Support vector Int Trans Electr Energy Syst 2019;29:1–16. 132
53 machine-based algorithm for post-fault transient stability status prediction using [126] Singh AK, Fozdar M. Event-driven frequency and voltage stability pre- 133
54 synchronized measurements. IEEE Trans Power Syst 2011;26:1474–83. dictive assessment and unified load shedding. IET Gener Transm Distrib 134
55 [99] Li Y, Yang Z. Application of EOS-ELM with binary Jaya-based feature selec- 2019;13:4410–20. 135
56 tion to real-time transient stability assessment using PMU data. IEEE Access [127] Wang Q, Li F, Tang Y, Xu Y. Integrating model-driven and data-driven methods 136
57 2017;5:23092–101. for power system frequency stability assessment and control. IEEE Trans Power 137
58 [100] Zheng L, Hu W, Zhou Y, Min Y, Xu X, Wang C, et al. Deep belief network Syst 2019;34:4557–68. 138
59 based nonlinear representation learning for transient stability assessment. In: [128] Chen K, Hu J, Zhang Y, Yu Z, He J. Fault location in power distribution 139
60 IEEE Power Energy Soc Gen Meet. 2018, p. 1–5. http://dx.doi.org/10.1109/ systems via deep graph convolutional networks. IEEE J Sel Areas Commun 140
61 PESGM.2017.8274126. 2020;38:119–31. 141
62 [101] Zhu Q, Chen J, Zhu L, Shi D, Bai X, Duan X, et al. A deep End-to-End model for [129] Guo MF, Zeng XD, Chen DY, Yang NC. Deep-learning-based earth fault detection 142
63 transient stability assessment with PMU data. IEEE Access 2018;6:65474–87. using continuous wavelet transform and convolutional neural network in 143
64 [102] Yu JJQ, Lam AYS, Hill DJ, Li VOK. Delay aware intelligent transient stability resonant grounding distribution systems. IEEE Sens J 2018;18:1291–300. 144
65 assessment system. IEEE Access 2017;5:17230–9. [130] Guo MF, Yang NC, Chen WF. Deep-learning-based fault classification us- 145
66 [103] Yu JJQ, Hill DJ, Lam AYS, Gu J, Li VOK. Intelligent time-adaptive transient ing Hilbert–Huang transform and convolutional neural network in power 146
67 stability assessment system. IEEE Trans Power Syst 2018;33:1049–58. distribution systems. IEEE Sens J 2019;19:6905–13. 147
68 [104] Zhu L, Hill D, Lu C. Hierarchical deep learning machine for power system [131] Wang Y, Liu M, Bao Z, Zhang S. Stacked sparse autoencoder with PCA and 148
69 online transient stability prediction. IEEE Trans Power Syst 2019;35:2399–411. SVM for data-based line trip fault diagnosis in power systems. Neural Comput 149
70 http://dx.doi.org/10.1109/tpwrs.2019.2957377. Appl 2019;31:6719–31. 150
71 [105] Zhou Y, Guo Q, Sun H, Yu Z, Wu J, Hao L. A novel data-driven approach [132] Ahmadimanesh A, Shahrtash SM. Transient-based fault-location method 151
72 for transient stability prediction of power systems considering the operational for multiterminal lines employing S-transform. IEEE Trans Power Deliv 152
73 variability. Int J Electr Power Energy Syst 2019;107:379–94. 2013;28:1373–80. 153
74 [106] Shi Z, Yao W, Zeng L, Wen J, Fang J, Ai X, et al. Convolutional neural [133] Yang D, Pang Y, Zhou B, Li K. Fault diagnosis for energy internet using 154
75 network-based power system transient stability assessment and instability mode correlation processing-based convolutional neural networks. IEEE Trans Syst 155
76 prediction. Appl Energy 2020;263:114586. Man, Cybern Syst 2019;49:1739–48. 156
77 [107] Yan R, Geng G, Jiang Q, Li Y. Fast transient stability batch assess- [134] Lan S, Chen MJ, Chen DY. A novel HVDC double-terminal non-synchronous 157
78 ment using cascaded convolutional neural networks. IEEE Trans Power Syst fault location method based on convolutional neural network. IEEE Trans Power 158
79 2019;34:2802–13. Deliv 2019;34:848–57. 159
 Z. Shi et al.

1 [135] Zhu B, Wang H, Shi S, Dong X. Fault location in AC transmission lines with [164] Abouhea M, Gueaieb W, Sharaf A. Load frequency regulation for multi-area 81
2 back-to-back MMC-HVDC using ConvNets. J Eng 2019;2019:2430–4. power system using integral reinforcement learning. IET Gener Transm Distrib 82
3 [136] Lin S, Chen X, Wang Q. Fault diagnosis model based on Bayesian network 2019;13:4311–23. 83
4 considering information uncertainty and its application in traction power supply [165] Yan Z, Xu Y. Data–driven load frequency control for stochastic power systems: A 84
5 system. IEEJ Trans Electr Electron Eng 2018;13:671–80. deep reinforcement learning method with continuous action search. IEEE Trans 85
6 [137] Jiang H, Zhang JJ, Gao W, Wu Z. Fault detection identification and location Power Syst 2019;34:1653–6. 86
7 in smart grid based on data-driven computational methods. IEEE Trans Smart [166] Wen S, Wang Y, Tang Y, Xu Y, Li P, Zhao T. Real-time identification of power 87
8 Grid 2014;5:2947–56. fluctuations based on lstm recurrent neural network: A case study on singapore 88
9 [138] CIGRE working group C4.601. Review of on-line dynamic security assessment power system. IEEE Trans Ind Informatics 2019;15:5266–75. 89
10 tools and techniques. 2007. [167] Wen S, Wang Y, Tang Y, Xu Y, Li P. Proactive frequency control based on 90
11 [139] Machowski J, Bialek JW, Bumby JR. Power System Dynamics: Stability and ultra-short-term power fluctuation forecasting for high renewables penetrated 91
12 Control. second ed.. John Wiley & Sons; 2008. power systems. IET Renew Power Gener 2019;13:2166–73. 92
13 [140] Yu T, Zhou B, Chan KW, Lu E. Stochastic optimal CPS relaxed control [168] Tomin NV, Reutsky ISKurbatskyVG. Hybrid intelligent technique for volt- 93
14 methodology for interconnected power systems using Q-learning method. J age/VAR control in power systems. IET Gener Transm Distrib 2019;13:4724–32. 94
15 Energy Eng 2011;137:116–29. [169] Huang Q, Huang R, Hao W, Tan J, Fan R, Huang Z. Adaptive power system 95
16 [141] Xiao L, Li Y, Liu J, Zhao Y. Power control with reinforcement learning emergency control using deep reinforcement learning. IEEE Trans Smart Grid 96
17 in cooperative cognitive radio networks against jamming. J Supercomput 2020;11:1171–82. 97
18 2015;71:3237–57. [170] Zhang J, Lu C, Si J, Song J, Su Y. Deep reinforcement leaming for short-term 98
19 [142] Xi L, Yu T, Yang B, Zhang X, Qiu X. A wolf pack hunting strategy based virtual voltage control by dynamic load shedding in China southem power grid. In: 99
20 tribes control for automatic generation control of smart grid. Appl Energy Proc int jt conf neural networks. 2018. p. 1–8. 100
21 2016;178:198–211. [171] Ye D, Zhang M, Sutanto D. A hybrid multiagent framework with q-learning for 101
22 [143] Yin L, Yu T, Zhou L. Design of a novel smart generation controller based on power grid systems restoration. IEEE Trans Power Syst 2011;26:2434–41. 102
23 deep Q learning for large-scale interconnected power system. J Energy Eng [172] Ghorbani MJ, Choudhry MA, Feliachi A. A multiagent design for power 103
24 2018;144:04018033. distribution systems automation. IEEE Trans Smart Grid 2016;7:329–39. 104
25 [144] Yu T, Xi L, Yang B, Xu Z, Jiang L. Multiagent stochastic dynamic game for [173] Wu J, Fang B, Fang J, Chen X, Tse CK. Sequential topology recovery of 105
26 smart generation control. J Energy Eng 2016;142:04015012. complex power systems based on reinforcement learning. Phys Stat Mech Appl 106
27 [145] Xi L, Chen J, Huang Y, Xu Y, Liu L, Zhou Y, et al. Smart generation control 2019;535:122487. 107
28 based on multi-agent reinforcement learning with the idea of the time tunnel. [174] Jhang SS, Lee HJ, Kim CN, Song CH, Yu WK. Control for damping low-frequency 108
29 Energy 2018;153:977–87. oscillation using deep learning. Australas. In: Univ Power Eng Conf. 2018, p. 109
30 [146] Yin L, Li S, Liu H. Lazy reinforcement learning for real-time generation 1–4. http://dx.doi.org/10.1109/AUPEC.2018.8757923. 110
31 control of parallel cyber-physical-social energy systems. Eng Appl Artif Intell [175] Shi Z, Wang Z, Luo Y, Ye D. Supplementary frequency control for multi-machine 111
32 2020;88:103380. power system based on adaptive dynamic programming. In: Int symp neural 112
33 [147] Wang H, Lei Z, Zhang X, Peng J, Jiang H. Multiobjective reinforcement learning- networks. 2018. p. 677–85. 113
34 based intelligent approach for optimization of activation rules in automatic
[176] Liu W, Zhang D, Wang X, Hou J, Liu L. A decision making strategy for 114
35 generation control. IEEE Access 2019;7:17480–92.
generating unit tripping under emergency circumstances based on deep rein- 115
36 [148] Chatterjee S, Mukherjee V. PID controller for automatic voltage regulator using
forcement learning. Proc CSEE 2018;38:109–19. http://dx.doi.org/10.13334/j. 116
37 teaching-learning based optimization technique. Int J Electr Power Energy Syst
0258--8013.pcsee.171747. 117
38 2016;77:418–29.
[177] Krishnan V, McCalley JD, Henry S, Issad S. Efficient database generation for 118
39 [149] Duan J, Shi D, Diao R, Li H, Wang Z, Zhang B, et al. Deep-reinforcement-
decision tree based power system security assessment. IEEE Trans Power Syst 119
40 learning-based autonomous voltage control for power grid operations. IEEE
2011;26:2319–27. 120
41 Trans Power Syst 2020;35:814–7.
[178] Thams F, Venzke A, Eriksson R, Chatzivasileiadis S. Efficient database gener- 121
42 [150] Tavakoli AR, Seifi AR, Arefi MM. Fuzzy-PSS and fuzzy neural network non-
ation for data-driven security assessment of power systems. IEEE Trans Power 122
43 linear PI controller-based SSSC for damping inter-area oscillations. Trans Inst
Syst 2020;35:30–41. 123
44 Meas Control 2018;40:733–45.
[179] He H, Garcia EA. Learning from imbalanced data. IEEE Trans Knowl Data Eng 124
45 [151] Yu T, Zhen WG. A reinforcement learning approach to power system stabilizer.
2009;21:1263–84. 125
46 In: IEEE Power Energy Soc Gen Meet. 2009. p. 1–5.
[180] Krawczyk B. Learning from imbalanced data: Open challenges and future 126
47 [152] Hadidi R, Jeyasurya B. Reinforcement learning based real-time wide-area
directions. Prog Artif Intell 2016;5:221–32. 127
48 stabilizing control agents to enhance power system stability. IEEE Trans Smart
49 Grid 2013;4:489–97.
[181] Zhang J, Mani I. Knn approach to unbalanced data distributions. In: Work Learn 128
50 [153] Ernst D, Glavic M, Wehenkel L. Power systems stability control: Reinforcement
from Imbalanced Datasets. 2003. 129
51 learning framework. IEEE Trans Power Syst 2004;19:427–35.
[182] Wilson DL. Asymptotic properties of nearest neighbor rules using edited data. 130
52 [154] Wang D, Glavic M, Wehenkel L. Trajectory-based supplementary damping
IEEE Trans Syst Man Cybern 1972;2:408–21. 131
53 control for power system electromechanical oscillations. IEEE Trans Power Syst
[183] Tomek I. Two modifications of CNN. IEEE Trans Syst Man Cybern 1976;769–72, 132
54 2014;29:2835–45.
SMC–6. 133
55 [155] Yousefian R, Sahami A, Kamalasadan S. Hybrid transient energy function- [184] Chawla NV, Bowyer KW, Hall LO, W.P. Kegelmeyer. SMOTE: Synthetic minority 134
56 based real-time optimal wide-area damping controller. IEEE Trans Ind Appl over-sampling technique. J Artif Intell Res 2002;16:321–57. 135
57 2017;53:1506–16. [185] He H, Bai Y, Garcia EA, Li S. ADASYN: Adaptive synthetic sampling approach 136
58 [156] Tang Y, Mu C, He H. SMES-based damping controller design using fuzzy-grhdp for imbalanced learning. In: Proc int jt conf neural networks. 2008. p. 1322–8. 137
59 considering transmission delay. IEEE Trans Appl Supercond 2016;26:1–6. [186] Zhou ZH, Liu XY. Training cost-sensitive neural networks with meth- 138
60 [157] Shen Y, Yao W, Wen J, He H, Jiang L. Resilient wide-area damping control ods addressing the class imbalance problem. IEEE Trans Knowl Data Eng 139
61 using GrHDP to tolerate communication failures. IEEE Trans Smart Grid 2006;18:63–77. 140
62 2019;10:2547–57. [187] McCarthy K, Zabar B, Weiss G. Does cost-sensitive learning beat sampling for 141
63 [158] Guo W, Liu F, Si J, He D, Harley R, Mei S. Online supplementary ADP classifying rare classes? In: Proc 1st Int Work Util Data Mining. 2005. p. 69–77. 142
64 learning controller design and application to power system frequency control [188] Liu XY, Zhou ZH. The influence of class imbalance on cost-sensitive learning: 143
65 with large–scale wind energy integration. IEEE Trans Neural Networks Learn An empirical study. In: Proc - IEEE Int Conf Data Mining. 2006, p. 970–4. 144
66 Syst 2016;27:1748–61. http://dx.doi.org/10.1109/ICDM.2006.158. 145
67 [159] Khalghani MR, Khooban MH, Mahboubi-Moghaddam E, Vafamand N, [189] Heaven D. Why deep-learning AIs are so easy to fool. Nature 2019;574:163–6. 146
68 Goodarzi M. A self-tuning load frequency control strategy for microgrids: [190] Ruppert D. The Elements of Statistical Learning: Data Mining, Inference, and 147
69 Human brain emotional learning. Int J Electr Power Energy Syst 2016;75:311–9. Prediction. Springer Science & Business Media; 2004. 148
70 [160] Bevrani H, Habibi F, Babahajyani P, Watanabe M, Mitani Y. Intelligent [191] Wu S, Zheng L, Hu W, Yu R, Liu B. Improved deep belief network and model 149
71 frequency control in an AC microgrid: Online PSO-based fuzzy tuning approach. interpretation method for power system transient stability assessment. J Mod 150
72 IEEE Trans Smart Grid 2012;3:1935–44. Power Syst Clean Energy 2020;8:27–37. 151
73 [161] Daneshfar F, Bevrani H. Multiobjective design of load frequency control using [192] Rane S. The balance: Accuracy vs. interpretability. 2018, https: 152
74 genetic algorithms. Int J Electr Power Energy Syst 2012;42:257–63. //towardsdatascience.com/the-balance-accuracy-vs-interpretability- 153
75 [162] Prakash S, Sinha SK. Simulation based neuro-fuzzy hybrid intelligent PI control 1b3861408062/, [Accessed 30 April 2020]. 154
76 approach in four-area load frequency control of interconnected power system. [193] Cremer JL, Konstantelos I, Strbac G. From optimization-based machine learn- 155
77 Appl Soft Comput J 2014;23:152–64. ing to interpretable security rules for operation. IEEE Trans Power Syst 156
78 [163] Shi Z, Wang Z, Luo Y, Ye D. Supplementary frequency control for multi-machine 2019;34:3826–36. 157
79 power system based on adaptive dynamic programming. In: International [194] Pan SJ, Tsang IW, Kwok JT, Yang Q. Domain adaptation via transfer component 158
80 symposium on neural networks. 2018. p. 677–85. analysis. IEEE Trans Neural Netw 2011;22:199–210. 159
 Z. Shi et al.

1 [195] Fernando B, Habrard A, Sebban M, Tuytelaars T. Unsupervised visual domain [197] Whitehead DE, Owens K, Gammel D, Smith J. Ukraine cyber-induced power 7
2 adaptation using subspace alignment. In: IEEE Int Conf Comput Vis. 2013, p. outage: Analysis and practical mitigation strategies. In: 70th Annu Conf Prot 8
3 2960–7. http://dx.doi.org/10.1109/ICCV.2013.368. Relay Eng. 2017. 9
4 [196] Hossain E, Khan I, Un-Noor F, Sikander SS, Sunny MSH. Application of big [198] Kundur P, Balu NJ, Lauby MG. Power System Stability and Control. New York: 10
5 data and machine learning in smart grid, and associated security concerns: A McGraw-hill; 1994. 11
6 review. IEEE Access 2019;7:13960–8.

View publication stats