Annals of Nuclear Energy 153 (2021) 108077

Contents lists available at ScienceDirect

Annals of Nuclear Energy

journal homepage: www.elsevier.com/locate/anucene

Development of a diagnostic algorithm for abnormal situations using

long short-term memory and variational autoencoder
Hyojin Kim, Awwal Mohammed Arigi, Jonghyun Kim ⇑
Department of Nuclear Engineering, Chosun University, 309 Pilmun-daero, Dong-gu, Gwangju 501-709, Republic of Korea

a r t i c l e i n f o a b s t r a c t

Article history: It is recognized that an abnormal situation diagnosis is a challenging task for nuclear power plant (NPP)
Received 1 June 2020 operators because of the excessive information and high workload in such situations. To help operators,
Received in revised form 3 December 2020 several studies have proposed operator support systems using artificial intelligence techniques. However,
Accepted 7 December 2020
those methods could neither assess anonymous cases as an unknown situation nor confirm whether its
outputs are reliable or not. In this study, an algorithm that can confirm the diagnosis results and deter-
mine unknown situations using long short-term memory (LSTM) and variational autoencoder (VAE) is
Keywords:
proposed. LSTM was adopted as the primary network for diagnosing abnormal situations. Meanwhile,
Event diagnosis algorithm
Abnormal situation
VAE-based assistance networks were added to the algorithm to ensure that the credibility of the diagno-
Long short-term memory sis is estimated via the anomaly score-based negative log-likelihood. The algorithm was tested and
Variational autoencoder implemented using the compact nuclear simulator for the Westinghouse 900 MWe NPP. Furthermore,
the simulation also considered noise-added data.
Ó 2020 Elsevier Ltd. All rights reserved.

1. Introduction To address these issues, several researchers have suggested

Diagnosing nuclear power plants (NPPs) in abnormal situations
is considered one of the most difficult tasks of operators. First,
there is excess information to consider in the decision-making pro-
cess of operators. Moreover, NPPs not only have approximately
4000 alarms and monitoring devices in the main control room,
but they also have more than 100 operating procedures that should
be followed during abnormal situations (Bae et al., 2006). Using
this large body of information, operators diagnose abnormal situa-
tions and execute specific actions in accordance with the relevant
operating procedures (Park et al., 2017). However, this excessive
information could confuse operators and increase the likelihood
of error caused due to excess mental workload (Park et al., 2019).
Additionally, some abnormal situations require speedy diagnosis
and response to prevent the reactor from being tripped.
Abbreviations: NPP, Nuclear power plant; LSTM, Long short-term memory; VAE,
Variational autoencoder; SVM, Support vector machines; AI, Artificial intelligence;
ANN, Artificial neural network; AE, Autoencoder; CNS, Compact nuclear simulator;
RNN, Recurrent neural network; AUC, Area under the curve; CCW, Component
coolant water; ROC, Receiver operating characteristic; AE, Autoencoder.
⇑ Corresponding author.
E-mail addresses: kim05140@chosun.kr (H. Kim), awwal@chosun.kr (A.M. Arigi),
jonghyun.kim@chosun.ac.kr (J. Kim).
operator support systems and algorithms to reduce the burden
on operators. For instance, (Hsieh et al., 2012) used an abnormal
symptom matrix, while (Kim and Jung, 2013) used a flowchart of
the AOPs generated through the C 5.0 package to implement oper-
ator support systems. Other works using support vector machines
(SVM) (Na et al., 2008), expert systems (Wang et al., 2016), and
artificial neural networks (ANNs) (Ohga and Seki, 1993) have
demonstrated the applicability of artificial intelligence (AI) tech-
niques. However, the SVM algorithms and expert systems have
their downsides. SVM algorithms are not suitable for large data-
sets, while expert systems are usually developed for specific
domains and acquiring the necessary knowledge is time consum-
ing. Meanwhile, ANNs are regarded as one of the most relevant
approaches to handle pattern recognition and large nonlinear data
(e.g., handwriting recognition, translation, financial forecasting).
Therefore, some papers have proposed algorithms using ANNs for
the diagnosis of abnormal situations in NPPs (Fantoni and
Mazzola, 1996; S ß eker et al., 2003; Embrechts and Benedek, 2004;
Uhrig and Hines, 2005; Yang and Kim, 2018).
Several diagnostic algorithms using ANNs have performed well
in trained cases, but some drawbacks are observed. The first one is
that it cannot correctly assess anonymous cases as an unknown sit-
uation if an unknown abnormal situation is provided. As some
abnormal events are not known and are unpredictable in actual

https://doi.org/10.1016/j.anucene.2020.108077
0306-4549/Ó 2020 Elsevier Ltd. All rights reserved.
H. Kim, Awwal Mohammed Arigi and J. Kim
NPPs, defining all abnormal events is also difficult. This limitation
could harm the safety of NPPs caused by the wrong diagnostic
results from an algorithm.
To determine unknown events, (Yang and Kim, 2020) suggested
a diagnostic algorithm utilizing long short-term memory (LSTM)
networks (i.e., a type of recurrent neural network) and autoencoder
(AE). To identify unknown events, the authors proposed a detector
using AE based on reconstruction error (i.e., differential between
the original and the reconstructed data). The idea behind this
detection is that an AE cannot adequately reconstruct unpredicted
patterns of anomalous data compared to predicted non-anomalous
data. Further, it has shown a good diagnostic performance and
capability of handling non-trained situations.
However, there are several disadvantages to identify unknown
events using AE. In this study, variational autoencoders (VAEs)
have been applied to compensate for the disadvantages in AE. First,
the AE is short of the ability to address variability. The latent vari-
ables of AE are defined by deterministic mappings. Even though
known events and unknown events might share the same mean
value, the variability can differ. Also, presumably unknown events
will have greater variance. Since deterministic mappings of AE can
be regarded as a mapping to the mean value of a Dirac delta distri-
bution, the AE cannot address variability. On the other hand, since
VAE uses the probabilistic encoder for modeling the distribution of
the latent variables rather than the latent variable itself, the vari-
ability of the latent space can be considered to account from the
data (Murphy, 2012; Kingma et al., 2014).
Second, the reconstruction errors are difficult to calculate if
the input variables are heterogeneous. AE-based unknown event
detection uses reconstruction errors as anomaly scores, which
are difficult to calculate when there are multiple input variables.
Fundamentally, a weighted sum is required to utilize multiple
data. The problem is that a universal objective method to decide
the appropriate weight is not available since the weights will be
different depending on the data. Furthermore, even after the
weights are decided, deciding the threshold for reconstruction
error is complicated. However, a VAE can calculate the recon-
struction log-likelihood of input by modeling the original proba-
bility distribution data (Park et al., 2018). This VAE-based
detector can detect an anomaly when the reconstruction log-
likelihood, which is based on the expected distribution of current
observation, is lower than the threshold. Since the probability dis-
tribution of each variable allows them to be separately calculated
by its variability, the calculation of the reconstruction log-
likelihood does not require weighting of the reconstruction error
of heterogeneous data and does not vary depending on the char-
acteristics of the heterogeneous data. Thus, deciding the threshold
of the reconstruction log-likelihood is much more objective, rea-
sonable, and easy to understand than that of the reconstruction
error (An and Cho, 2015).
In addition, the above diagnostic algorithm (Yang and Kim,
2020) needs to potential improvements. An improvement with this
diagnostic algorithm is that the algorithm produces the diagnosis
result with probabilities and the operators need to make the final
decision based on the probability result. In some situations, this
diagnostic algorithm may provide multiple and competing diagno-
sis results with competing probability values. For example, this
diagnostic algorithm produces an output such that Event-A has a
probability of 0.6 and Event-B has a probability of 0.4. In this case,
operators might make errors due to confusion and uncertainty
when the probabilities of diagnosis results are competing, espe-
cially during the initial period of abnormal situations when the
probabilities of those events may not be clearly distinguished. In
some abnormal situations, fast decision making is important.
Hence, confirming the diagnostic result of the algorithm during
the initial period of an abnormal situation is necessary. Moreover,
2
Annals of Nuclear Energy 153 (2021) 108077
the noise from the signal was not considered in previous studies,
which exists in actual NPPs.
Therefore, this study not only aims to propose a diagnostic algo-
rithm for abnormal situations by identifying unknown events, con-
firming the diagnostic results, and conducting the final diagnosis
through the algorithm but also increases the reliability of diagnos-
tic results. This algorithm combines LSTM and VAE for identifying
unknown situations and confirms the diagnosis of the LSTM net-
work. First, the utilized methodologies are briefly introduced in
Section 2. Second, this study develops a functional architecture of
the diagnostic algorithm that consists of several functions. Then,
it defines the function and interaction between the functions, in
Section 3. Finally, the implementation, training, and testing of
the proposed diagnostic algorithm are described in detail. The
training and testing data include a Gaussian noise to reflect the
actual situations in NPPs. This algorithm was trained and imple-
mented using a compact nuclear simulator (CNS). The reference
plant is based on the three-loop Westinghouse 930 MWe pressur-
ized water reactor.
2. Methods
This section briefly explains the LSTM and VAE networks that
were mainly used for developing the diagnosis algorithm. LSTM
was applied as the primary network for diagnosing abnormal
events. Meanwhile, VAE-based assistance networks were adopted
to ensure the credibility of the diagnosis result from the LSTM
network.
2.1. Long short-term memory
LSTM is a neural network architecture based on recurrent neu-
ral network (RNN) for processing long temporal sequences of data.
It has been suggested for long temporal sequence learning to han-
dle the vanishing gradient problem. Sequential data can also be
handled with other sequence models, such as Markov models, con-
ditional random fields, and Kalman filters. However, unlike LSTM,
these models exhibit difficulty in handling long-range dependen-
cies. Although LSTM has the same structure as RNNs, it uses a dif-
ferent equation to calculate the hidden state. Specifically, LSTM
uses a memory cell instead of an RNN neuron. It combines fast
training with efficient learning on tasks that require sequential
short-term memory storage for many time steps during a trial.
LSTM can learn to bridge minimal time lags over 1000 discrete
time steps by enforcing special units called memory cells. It deter-
mines whether the previous memory value should be altered and
then calculates the value to be stored in the current memory based
on the current state and input value of the memory cell. This struc-
ture is highly effective in storing long sequences. Further, alterna-
tive models (i.e., Markov models, conditional random fields, and
Kalman filters) require domain knowledge or feature engineering,
and they offer less chance for unexpected discovery, whereas LSTM
can learn representations and discover unpredicted structures
(Fantoni and Mazzola, 1996; Embrechts and Benedek, 2004;
Monner and Reggia, 2012; Lipton et al., 2016; Lee et al., 2020).
Fig. 1 shows the architecture of the LSTM cell applied in this
study. The input sample x passes through every gate as in a con-
veyor belt system. With other LSTM models, each LSTM cell adjusts
the output value using the input gate, forget gate, and output gate
while maintaining the cell. The information in the cell state is
unchanged and can be added or deleted through each gate. Fur-
thermore, as the operation of each gate is composed of an addi-
tional operation attached to the cell state, it can avoid the
vanishing gradient problem.
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Fig. 1. Architecture of LSTM.

The input gate determines the capacity of the input value, the
forget gate determines the degree to which the previous cell state
is forgotten, and the output gate determines the value of the out-
put. Eqs. (1)–(3) represent the input gate, forget gate, and output
gate denoted by i, f, and o, respectively; Here, r and t represent a
sigmoid function and time state, respectively. The cell state is
determined as shown in Eq. (4), where C represents the cell state.
Finally, LSTM provides the output using Eq. (5), where h represents
the output of the LSTM network.


f t ¼ r W f  C t1 ; ht1 ; xt  þ bf ð1Þ

it ¼ rðW i  ½C t1 ; ht1 ; xt  þ bi Þ ð2Þ

Fig. 2. Archtecture of AE.

ot ¼ rðW o  ½C t ; ht1 ; xt  þ bo Þ ð3Þ

b ¼ f ðW d hð X Þ þ bd Þ
X ð7Þ
C t ¼ f t  C t1 þ it  tanhðW c  ½ht1 ; xt  þ bc Þ ð4Þ
 
ht ¼ ot  tanhðC t Þ ð5Þ b ¼ kX  X
L X; X b k2 ð8Þ

The encoder in Eq. (6) maps an input X to a hidden layer repre-

2.2. Autoencoder
AE is a neural network that is trained by unsupervised learning,
which is trained to learn reconstructions that approximate the
input value to derive the output value (Zhou and Paffenroth,
2017). Fig. 2 shows the architecture of AE. AE comprises an encoder
that encodes the input to the hidden layer and a decoder that
decodes the encoded hidden unit and outputs the same size as
the input. As shown in Eq. (6), X 2 RD and hð X Þ 2 RM are the input
data and the hidden data representation, respectively. f ðzÞ is the
nonlinear transformation function. Generally, a logistic sigmoid
function f ðzÞ ¼ 1þexp
1
ðzÞ
is applied. W e 2 RMD and be 2 RM are the
weight matrix and bias vector, respectively. The hidden represen-
b 2 RD are shown
tation of the network output and reconstruction X
in Eq. (7), where W d 2 RDM and bd 2 RD are the weight matrix and
bias vector, respectively.
hð X Þ ¼ f ð W e X þ be Þ ð6Þ
3
sentation hð X Þ by a related mapping following a nonlinearity. The
decoder in Eq. (7) maps the hidden representation hð X Þ back to
the original input space as reconstruction by the same transforma-
tion as the encoder. The square of the error between the input and
output is called the reconstruction error as in Eq. (8). AE learns to
minimize this reconstruction error.
2.3. Variational autoencoder
VAE (Kingma et al., 2014) is an unsupervised deep learning gen-
erative model that can model the distribution of training data. If
input data are similar to the training data, the output appears to
be similar to the input. Otherwise, a probabilistic measure that
considers the variability of the distribution variables decreases
(Chen et al., 2019). Several studies have suggested a fault detection
algorithm using the reconstruction log-likelihood of VAE and
showed the compatibility of VAE with LSTM (An and Cho, 2015;
Park et al., 2018; Xu et al., 2018).
H. Kim, Awwal Mohammed Arigi and J. Kim
VAE provides a flexible formulation for interpreting and encod-
ing z as a potential variable in probabilistic generation models. As
shown in Fig. 3, the input sample x passes through the encoder to
obtain parameters of latent space distribution. The latent variable z
was obtained from sampling in the current distribution, and then z
was used to generate a reconstructed sample through the decoder
(Chen et al., 2019). VAE comprises of a probabilistic encoder
(q/ ðzjxÞ) and decoder (ph ðxjzÞ). As the posterior distribution
(ph ðzjxÞ) is intractable, VAE approximates ph ðzjxÞ using the encoder
q/ ðzjxÞ, which is assumed to be Gaussian and is parameterized by
£. This enables the encoder to learn and predict latent variables
z, which makes it is possible to draw samples from this
distribution.
To decode a sample z drawn from q/ ðzjxÞ to the input x, the
reconstruction loss (Eq. (9)) must be minimized. The first term of
Eq. (9) is the Kullback-Leibler (KL) divergence between the approx-
imate posterior and prior latent variable z. This term makes the
posterior distribution to be similar to the prior distribution by
working as a regularization term. The second term of Eq. (9) can
be understood in terms of reconstruction of x through the posterior
distribution q/ ðzjxÞ and likelihood ph ðxjzÞ.
 
Lðh; /; xÞ ¼ DKL q/ ðzjxÞjjph ðzÞ þ Eq/ ðzjxÞ ½logph ðxjzÞ 
The choice of distribution types is important because VAE mod-
els the approximated posterior distribution q/ ðzjxÞ from a prior
ph ðxÞ and likelihood ph ðxjzÞ. A typical choice for the posterior is
Gaussian distribution, where the standard normal distribution
N ð0; 1Þ is used for the prior ph ðxÞ.
2.4. Softmax
The softmax function is used for the post-processing of the
LSTM output. The softmax function is an activation function com-
monly used in the output layer of deep learning models that can
categorize more than three classes of output (Bishop, 2006). Soft-
max significantly deviations between the values and then normal-
izes the outputs. Eq. (10) represents the softmax function. For
y 2 Rk (the input vector to the softmax function), k is the number
of classes of output. For Sð yÞ1 . . . Sð yÞk , the normalized output val-
ues were between 0 and 1, and the sum of output values was
always 1.
e yi
Sð yÞi ¼ Pk ðfor i ¼ 1; . . . ; kÞ ð10Þ
yj
j¼1 e
Fig. 3. Architecture of VAE.
Annals of Nuclear Energy 153 (2021) 108077
3. Diagnostic algorithm for abnormal situations using LSTM and
VAE
This section describes the overall structure of the diagnostic
algorithm for abnormal situations using ANN. Fig. 4 illustrates
the functional architecture of the diagnostic algorithm design,
which consists of four functions: 1) input pre-processing function,
2) unknown event identification function, 3) event diagnosis func-
tion, and 4) confirmation of diagnosis result function. Table 1 pre-
sents a summary of descriptions, inputs, and outputs of each
function.
3.1. Input Pre-processing function
The first function of the algorithm is to process the plant param-
eters and make it suitable as network inputs. The inputs for the
networks are selected from operating procedures based on their
importance and ability to affect the state of the plant or system
availability. These inputs should have a range of values from 0 to
1. However, plant parameters have a different range of values or
states (e.g., pressurizer pressure: 158 kg/cm2, alarm: on or off).
Generally, variables with higher values will have a larger impact
ð9Þ
on the network results. However, higher values are not necessarily
more important for prediction. This problem produces local min-
ima. Therefore, the input pre-processing obtains the regular plant
parameters as input and then outputs the normalized plant param-
eters that will be utilized by the networks.
Min-max normalization is used to prevent local minima and
increase the learning speed. Thus, the input of the networks is cal-
culated by using Eq. (11). here, X is the current value of plant
parameters, and X min and X max are the minimum and maximum
values of collected data, respectively. In this equation, X input has a
range of 0–1.
ðX  X min Þ
X input ¼ ð11Þ
ðX max  X min Þ
3.2. Unknown event identification function
This function is used to identify the unknown event via a com-
bination of the VAE and LSTM networks. The VAE network was
combined with LSTM (Park et al., 2018) and it was used not only
to support the sequence of input data but also to capture the com-
plex temporal dependence in time series. This function receives
4
H. Kim, Awwal Mohammed Arigi and J. Kim
Fig. 4. Functional architecture of the diagnostic algorithm design.
normalized NPP parameters from the input pre-processing func-
tion and identifies the unknown event in real time. The anomaly
scores that indicate discrepancies between the actual and trained
data were used for this function. If the anomaly score is below
the threshold, the event is identified as a known event for which
the diagnosis network in the next function has been trained. If
the anomaly score is above the threshold, the event is unknown,
and the message ‘‘Unknown event occurrence” is provided to the
operators as the output.
The process of unknown event identification function is shown
in Fig. 5. To consider the temporal dependency of time-series data
in a VAE, a VAE is combined with LSTMs by replacing the feed-
forward network in a VAE with the LSTMs similar to conventional
5
Annals of Nuclear Energy 153 (2021) 108077
temporal AEs. Given the multi-parameters input xt 2 RD (where D
is the number of input parameters), made up of x1;t , . . . xD;t , at time
twhich is normalized plant parameters from input pre-processing
function, the encoder approximates the posterior pðzt jxt Þ by the
LSTM of the encoder to estimate the mean lzt 2 RM and variance
rzt 2 RM of the latent variable zt 2 RM . Then, the randomly sampled
zt from the posterior pðzt jxt Þ is fed into the LSTM of the decoder.
The final outputs are the reconstruction mean lxt 2 RD
 
(lx1;t ;    lxD;t Þ and variance rxt 2 RD rx1;t ;    rxD;t .
To identify the unknown event, the unknown event identifica-
tion function detects an anomalous execution when the current
anomaly score is above the threshold a in Eq. (12). The term
0
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Table 1
Summary of diagnostic algorithm functions.

No. Function Input Output

1. Input pre-processing function Plant parameters (from the NPP) Normalized plant parameters (to networks)
2. Identification of unknown event Normalized plant parameters (from input pre-processing Identification as a known event (to event diagnosis function)
function function) Identification as an unknown event (to operators)
3. Event diagnosis function Identification of known event (from identification of Eventual highest probability event (to confirmation of
unknown event function) diagnosis result function)
Normalized plant parameters (from input pre-processing
function)
4. Confirmation of diagnosis result Highest probability event (from event diagnosis function) Identified event (to operators)
function Normalized plant parameters (from Input pre-processing Incorrect diagnosis results (to event diagnosis function)
function)

Fig. 5. Various processes in the unknown event identification function.

a ¼ Smean þ 3Sstd
0
f s ðxt ; £; hÞ is an anomaly score calculator. The anomaly score is ð14Þ
defined as the negative log-likelihood of xt , represented in Eq.
The threshold a was determined using three-sigma limits
0

(13), with respect to the reconstructed distribution of xt from an

LSTM-VAE network. Here, lxt and rxt are the mean and variance
of the reconstructed distribution from an LSTM-VAE network with
parameters £ and h, respectively. Fig. 6 shows an example of the
anomaly score calculation. Note that xt 2 R3 (for x1;t ; x2;t ; x3;t ) repre-
sents the normalized input parameters, lxt 2 R3 (for lx1;t ; lx2;t ; lx3t )
represents the reconstruction mean, and rxt 2 R3 (for
rx1;t ; rx2;t ; rx3t Þ represents the reconstruction variance. Each ele-
ment goes through the Eq. (13) then the anomaly score is calcu-
lated. Notice that in this example, the number of input
parameters (D) is three. Therefore, the anomaly score calculated
in this example is 0.927 as shown in Fig. 6. A high anomaly score
means that the input has not been adequately reconstructed by
the LSTM-VAE network.
(
Unknownev ent; iff s ðxt ; £; hÞ > a
0
(Shewhart, 1931) after considering the anomaly score distribution
of the training data. The fact that the anomaly scores are achieving
a smaller value indicates that the output data (i.e., reconstructed
data from LSTM-VAE) is similar to the training data and the thresh-
old considers the upper control limit. Smean and Sstd are the mean
and standard deviation of the anomaly scores of the training data,
respectively (Eq. (14)). This not only sets a range for the process
parameter at 0.27% control limits (corresponding to the three-
sigma in normal distribution) but also minimizes the cost associ-
ated with preventing the error of classifying a known event as
unknown. Section 4.2.2 discusses the method to determine the
threshold and hyperparameters.
3.3. Event diagnosis function

ð12Þ This function produces diagnosis results of the plant situation

Knownev ent; otherwise; using LSTM. Fig. 7 shows the processes in the event diagnosis func-
tion. The LSTM network receives normalized plant parameters
1X D   from the input pre-processing function and produces identified
f s ðxt ; £; hÞ ¼  logp xi;t ; lxi;t ; rxi;t ð13Þ abnormal events with their probabilities. The output is post-
D i¼1
processed using the softmax function. The probability represents

Fig. 6. Example of anomaly score calculation.

6
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Fig. 7. Processes in the event diagnosis function.

Fig. 8. Processes in the confirmation of diagnosis results function.

Fig. 9. Plant overview interface of CNS.

7
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

the confidence level of the identified event. Then, this function nostic result. The procedure to determine hyperparameters, such
selects the event with the highest probability among the diagnostic as number of layers and nodes, is described in Section 4.2.3.
results for the confirmation function. In addition, multiple events
can be identified with different probabilities in the previous func-
3.4. Confirmation of diagnosis result function
tion. If the confirmation function returns the information stating
that the current situation is not consistent with the diagnostic
This function is used to confirm whether the current abnormal
result, then this function will select the next event with the highest
situation is identical to the event selected in the event diagnosis
probability until the current situation is consistent with the diag-
function. This function has a library that consists of LSTM-VAE net-

Table 2
Abnormal scenarios and the number of simulations.

No. Scenarios Training cases Verification cases Total cases

1 Failure of pressurizer pressure channel (High) 14 4 18
2 Failure of pressurizer pressure channel (Low) 20 6 26
3 Failure of pressurizer water level channel (High) – 6 6
4 Failure of pressurizer water level channel (Low) 11 4 15
5 Failure of steam generator water level channel (Low) 32 8 40
6 Failure of steam generator water level channel (High) 35 6 41
7 Control rod drop 38 10 48
8 Continuous insertion of control rod 7 1 8
9 Continuous withdrawal of control rod 6 2 8
10 Opening of pressurizer power-operated relief valve 42 10 52
11 Failure of pressurizer safety valve 35 8 43
12 Opening of pressurizer spray valve 41 9 50
13 Stopping of charging pump – 1 1
14 Stopping of two main feedwater pumps – 3 3
15 Main steam line isolation – 3 3
16 Rupture at the inlet of the regenerative heat exchanger 40 10 50
17 Leakage from chemical volume and control system to component coolant water (CCW) 40 10 50
18 Leakage at the outlet of charging control flow valve 24 6 30
19 Leakage into the CCW system from the reactor coolant system 24 6 30
20 Leakage from steam generator tube – 36 36
Total 409 149 558

Fig. 10. Examples of pressurizer temperature data ((a): original CNS data and (b): data with ± 5% Gaussian noise).

Fig. 11. Example of normalized pressurizer temperature.

8
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Fig. 12. Identification of unknown event function using the LSTM-VAE network.

Table 3
Accuracy comparison results of various configured networks.

No. Time step Batch size Layers Accuracy

1 5 32 2 0.9668
2 5 32 3 0.9638
3 5 64 2 0.9634
4 5 64 3 0.9650
5 10 32 2 0.9768
6 10 32 3 0.9746
7 10 64 2 0.9764
8 10 64 3 0.9741
9 15 32 2 0.9767
10 15 32 3 0.9762
11 15 64 2 0.9764
12 15 64 3 0.9766

Fig. 13. Result of ROC and AUC of the identification of unknown event function.
works for trained events. Further, the LSTM-VAE network for the
selected event is used to confirm that the selected event is identical
to the trained event.
Fig. 8 shows the confirmation process of the selected event.
First, this function selects the LSTM-VAE network from the library
that corresponds to the event identified in the previous function.
Next, it verifies whether the current situation is identical to the
selected event by using the LSTM-VAE network. To estimate the
anomaly score, this function uses negative log-likelihood (i.e., sim-
ilar to ‘‘the unknown event identification function”). If the negative
log-likelihood is below the threshold, then the algorithm declares
that the diagnosis result from the event diagnosis function is cor-
rect and confirmed. If the negative log-likelihood is beyond the
threshold, then it returns to the previous function to select another

9
H. Kim, Awwal Mohammed Arigi and J. Kim
event. The thresholds of LSTM-VAE networks are determined in a
similar manner as that described in Section 3.2.
4. Implementation
The suggested algorithm was implemented by using a CNS that
simulates a Westinghouse 900 MWe, three loops, pressurized
water reactor. Fig. 9 shows a plant overview interface of the CNS.
For implementation, a desktop computer with NVIDIA GeForce
GTX 1080 11 GB GPU, Intel 4.00 GHz CPU, Samsung 850 PRO
512 GB MZ-7KE512B SSD, and 24 GB RAM was used.
Python 3.7.3 was used as the coding language, and several python
libraries, including Keras and Pandas, were used to model the
algorithm.
4.1. Data collection
To implement, train, and validate the algorithm, CNS was used
as a real-time testbed. A total of 20 abnormal situations and 558
cases were simulated to collect data. Table 2 shows the abnormal
scenarios and the numbers of simulations for each event. The sce-
narios included representative abnormal situations in actual NPPs,
such as instrument failures (Nos. 1–6), component failures (Nos. 7–
16), and leakages (Nos. 17–20).
Based on the abnormal operating procedures of the reference
plant, 139 parameters were selected for input, including plant vari-
ables (e.g., temperature or pressure) and component states (e.g.,
pump or valve status). These parameters were collected every sec-
ond during the simulations.
Among the 20 scenarios collected, 15 scenarios containing 409
cases were used for training, and 5 scenarios were used for validat-
ing untrained events. A total of 149 cases were used for validating
the algorithm, including the five untrained events (i.e., 3, 13, 14,
15, and 20). Among them, 115 cases were used for determining
the thresholds of identification of unknown event function and
confirmation of diagnosis result function.
Fig. 14. Event diagnosis function using LSTM and softmax.
10
Annals of Nuclear Energy 153 (2021) 108077
The collected data were added with ± 5% Gaussian noise to
reflect actual signals from NPPs. The CNS produces data without
noises, as shown in Fig. 10(a). The noise was added to the CNS data
intentionally, as shown in Fig. 10(b).
4.2. Function implementation
4.2.1. Input preprocessing function
To implement the min–max normalization in the input prepro-
cessing function, we determined the maximum and minimum val-
ues of parameters based on all the collected data (i.e., 558 cases).
Moreover, as ± 5% Gaussian noise was added to the simulator data
(making it similar to actual NPP data), some data can be larger than
the maximum values or lower than the minimum values. There-
fore, the data will not fall between 0 and 1 when normalized. To
prevent this problem, we added a 10% margin to the maximum
and 10% to the minimum values for each parameter. Fig. 11
shows an example of a normalized pressurizer temperature.
4.2.2. Identification of unknown event function
The unknown event function was identified using the LSTM-
VAE network. In this network, a dataset has 10 s sequence and
139 input values. As mentioned in Section 4.1, 409 scenarios (i.e.,
192,637 datasets for 139 parameters) were trained. Fig. 12 illus-
trates how the unknown event function is identified by using the
LSTM-VAE network. VAE does not necessarily tune the hyperpa-
rameters as it provides a variational information bottleneck that
prevents overfitting. Consequently, it has the effect of making the
optimal bottleneck size in a given hyperparameter (Tishby et al.,
2015; Alemi et al., 2019; Ruder, 2016). Additionally, this LSTM-
VAE network used Adam (Kingma and Ba, 2015) optimizer with a
learning rate of 0.0001 and ran for 100 epochs.
This study used the receiver operating characteristic (ROC)
curve to evaluate the performance of the network. The ROC curve
is created by plotting the true positive rate (i.e., the ratio of cor-
rectly predicted positive observations to all observations in the
actual class) against the false positive rate (i.e., the ratio of the
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Fig. 15. Illustration of the confirmation of diagnosis results function.

Table 4
Results of training data, performance, and thresholds of LSTM-VAE networks.

No. Name Trained data AUC Threshold

1 VAE Ab 01 3459 datasets (Ab 01) 0.995 0.923
2 VAE Ab 02 3785 datasets (Ab 02) 0.934 0.923
3 VAE Ab 04 5658 datasets (Ab 04) 0.993 0.922
4 VAE Ab 05 5730 datasets (Ab 05) 0.998 0.921
5 VAE Ab 06 8312 datasets (Ab 06) 0.974 0.923
6 VAE Ab 07 34,735 datasets (Ab 07) 0.999 0.920
7 VAE Ab 08 2831 datasets (Ab 08) 0.958 0.924
8 VAE Ab 09 2070 datasets (Ab 09) 0.955 0.927
9 VAE Ab 10 8394 datasets (Ab 10) 0.985 0.921
10 VAE Ab 11 8004 datasets (Ab 11) 0.938 0.921
11 VAE Ab 12 23,885 datasets (Ab 12) 0.996 0.920
12 VAE Ab 16 24,600 datasets (Ab 16) 0.965 0.920
13 VAE Ab 17 30,904 datasets (Ab 17) 0.953 0.934
14 VAE Ab 18 14,805 datasets (Ab 18) 0.984 0.922
15 VAE Ab 19 1546 datasets (Ab 19) 0.967 0.920
16 VAE Normal 7602 datasets (Normal state) 0.995 0.923

incorrectly predicted negative observations to all observations in
the actual class) is a useful method of interpreting the performance
of a binary classifier. The area under the ROC curve (AUC) is an
effective measure to summarize the overall diagnostic accuracy
11
of the test and is interpreted as the average value of sensitivity
for all possible values of specificity. It can also take on any value
between 0 and 1, where a value of 0 indicates a perfectly inaccu-
rate test. The closer AUC is to 1, the better the overall diagnostic
H. Kim, Awwal Mohammed Arigi and J. Kim
performance. In general, an AUC of 0.5 suggests no discrimination,
0.7–0.8 is considered acceptable, 0.8 to 0.9 is considered excellent,
and more than 0.9 is considered outstanding (Bergstra and Bengio,
2012; Mandrekar, 2010; Park et al., 2004). The threshold was
determined as 0.923 using Eq. (14). Fig. 13 shows the result of
ROC and AUC of LSTM-VAE.
4.2.3. Event diagnosis function
The event diagnosis function was developed using LSTM. A total
of 409 scenarios (i.e., 192,637 datasets for 139 parameters) were
trained. To optimize the LSTM network in the event diagnosis func-
tion, a manual search method was used while individually adjust-
ing the hyperparameters (e.g., input sequence length, batch size,
and number of layers). In general, there is no golden rule for hyper-
parameter determination to optimize the network (Ruder, 2016;
Bergstra and Bengio, 2012; Snoek et al., 2012). Table 3 shows the
accuracy comparison results of different configured networks. The
accuracy is defined as the ratio of correctly predicted data to total
verification data. Consequently, an optimal LSTM network with
10 s time steps, 32 batch sizes, and 2 layers was selected. This net-
work used the Adam optimizer (Kingma and Ba, 2015) with a learn-
ing rate of 0.0001 and ran for 100 epochs. Fig. 14 shows the
illustration of the event diagnosis function using LSTM and softmax.
4.2.4. Confirmation of diagnosis result function
To implement this function, an LSTM-VAE library was devel-
oped. This library comprises 16 LSTM-VAE networks that are
trained for each known event (i.e., 15 abnormal events and 1 nor-
mal state). Fig. 15 shows the illustration of the confirmation of diag-
nosis result function (i.e., when Ab 01 event is diagnosed). Each
LSTM-VAE network and threshold were comprised and determined
Fig. 16. Process of diagnosing an untrained event.
12
Annals of Nuclear Energy 153 (2021) 108077
in a manner similar to the identification of an unknown event.
Table 4 shows the training dataset, AUC, and threshold of each
LSTM-VAE network in the confirmation of diagnosis result function.
4.3. Verification
Verifications were performed for 149 scenarios (i.e., 57,309
datasets). Among them, 100 cases and 47,987 datasets were used
for the trained events, while 49 cases and 9322 datasets were used
for untrained events. As a result of the verification, the accuracy of
the network of unknown event identification (i.e., predicting a
trained event as a known event and an untrained event as an
unknown event) is 96.72%. In addition, the accuracy of the net-
work for confirmation of diagnosis results is 98.44%. The verifica-
tion demonstrated that the algorithm could successfully
diagnose the trained events and identify the untrained events.
Fig. 16 shows an example for the diagnosis of an untrained event
(i.e., Ab 20), which is a leakage from the steam generator tubes.
In this case, the identification of unknown event function identifies
the event as an untrained event because the construction error
goes beyond the threshold. Thus, the message ‘‘Unknown Event”
is provided.
Fig. 17 illustrates how the algorithm diagnoses a trained event
(i.e., event Ab 08), which is the continuous insertion of control
rod. The input pre-processing function normalizes plant parame-
ters. Subsequently, the identification of unknown event function
identifies the current situation as a trained event. The event diag-
nosis function examines the event as Ab 08, and diagnosis result
function confirms that the diagnosis result is correct. Finally, ‘‘Ab
08: continuous insertion of control rod” is presented as the diagno-
sis result for the current situation.
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Fig. 17. Process of diagnosing a trained event.

13
H. Kim, Awwal Mohammed Arigi and J. Kim Annals of Nuclear Energy 153 (2021) 108077

Table 5 5. Result and discussion

Accuracy comparison between LSTM networks.

Data type Accuracy Many other AI-based diagnostic algorithms applied data with-
LSTM networks Original CNS data without noise 0.9735 out noise, whereas, in our algorithm, ±5% Gaussian noise was
CNS data with noise 0.9768 added to the simulator data, as the actual signal in NPPs generally
contains some noise. In the previous section, the proposed algo-
rithm showed a good performance for the noise-added data. We
Table 6
also compared the performance of the algorithm between the
AUC comparison between VAE networks. noise-added and the noise-free data. Tables 5 and 6 show the accu-
racy and AUC comparisons of VAE networks for two different data,
No. Name AUC
respectively. The comparison indicates that the algorithm pro-
Original CNS data CNS data with noise posed in this study shows similar performances for noise-added
1 VAE Unknown 0.973 0.981 and noise-free data.
2 VAE Ab 01 0.987 0.995 In this study, the VAEs have been applied to compensate for the
3 VAE Ab 02 0.931 0.934
disadvantages of AE. We compared the performance of the algo-
4 VAE Ab 04 0.993 0.993
5 VAE Ab 05 0.998 0.998 rithm between the AE network and the VAE network for the iden-
6 VAE Ab 06 0.974 0.974 tification of unknown event functions. Confusion matrix and
7 VAE Ab 07 0.998 0.999 accuracy of verification results are used to evaluate the accuracy
8 VAE Ab 08 0.959 0.958
and validity of the algorithms. The confusion matrix is adopted
9 VAE Ab 09 0.956 0.955
10 VAE Ab 10 0.977 0.985
to show the diagnostic algorithm accuracy for the trained data
11 VAE Ab 11 0.937 0.938 and untrained data. Figs. 18 and 19 show the confusion matrixes
12 VAE Ab 12 0.995 0.996 of the AE network and VAE network for the identification of
13 VAE Ab 16 0.954 0.965 unknown event functions, respectively. The comparison indicates
14 VAE Ab 17 0.952 0.953
that the AE network and VAE network show similar accuracy for
15 VAE Ab 18 0.982 0.984
16 VAE Ab 19 0.969 0.967 trained data. However, for the untrained data, the VAE network
17 VAE Normal 0.995 0.995 shows a 20% higher accuracy than the AE network (i.e., AE:
70.01% and VAE: 92.37%).

Fig. 18. Confusion matrix of the AE network for identification of unknown event function.

14
H. Kim, Awwal Mohammed Arigi and J. Kim
Fig. 19. Confusion matrix of the VAE network for identification of unknown event function.
6. Conclusion
In this study, an algorithm that uses LSTM and VAE networks to
diagnose abnormal situations in NPPs was proposed. The proposed
algorithm has the capability of finding unknown situations, diag-
nosing known situations, and confirming the results. For a more
realistic evaluation, noise-added signals were also considered.
The validation demonstrated that the proposed algorithm could
provide correct diagnostic results as intended. This algorithm can
be applied to an operator support system to improve the operator’s
situation awareness during abnormal situations in NPPs.
However, there are some further works for this approach to be
implemented as an operator support system in actual NPPs. First,
an operator support system needs to provide the information
about how the system draws a diagnostic result as well as the
event identified. One of solutions to this ‘‘black box” problem is
applying explainable AI technologies. Second, the capability to
detect the failure of sensors or signals is necessary. Since the cor-
rectness of support systems largely relies on the integrity and qual-
ity of signals from the NPP, wrong signals may directly lead to
wrong information to operators. Therefore, the algorithm to iden-
tify signal failures needs to be studied.
CRediT authorship contribution statement
Hyojin Kim: Conceptualization, Software, Visualization,
Writing-Original draft, Data curation, Investigation, Formal analy-
sis. Awwal Mohammed Arigi: Writing-Review & Editing, Valida-
15
Annals of Nuclear Energy 153 (2021) 108077
tion. Jonghyun Kim: Supervision, Project administration,
Validation.
Declaration of Competing Interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Acknowledgments
This work was supported by the National Research Foundation
of Korea (NRF) grant funded by the Korean government (Ministry
of Science and ICT) (2018M2B2B1065651) and by the Basic Science
Research Program through the NRF funded by the Ministry of
Science, ICT & Future Planning (N01190021-06).
References
Alemi, A.A., Fischer, I., Dillon, J. V., Murphy, K., 2019. Deep variational information
bottleneck. 5th Int. Conf. Learn. Represent. ICLR 2017 - Conf. Track Proc. 1–19.
An, J., Cho, S., 2015. Variational autoencoder based anomaly detection using
reconstruction probability. Special Lecture on IE 2 (1).
Bae, H., Chun, S.P., Kim, S., 2006. Predictive fault detection and diagnosis of nuclear
power plant using the two-step neural network models. Lect. Notes Comput.
Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) 3973
LNCS, 420–425. https://doi.org/10.1007/11760191_62
Bergstra, J., Bengio, Y., 2012. Random search for hyper-parameter optimization. J.
Mach. Learn. Res. 13, 281–305.
Bishop, C.M., 2006. Pattern recognition and machine learning. Springer.
Chen, R.-Q., Shi, G.-H., Zhao, W.-L., Liang, C.-H., 2019. Sequential VAE-LSTM for
Anomaly Detection on Time Series 1–7.
H. Kim, Awwal Mohammed Arigi and J. Kim
Embrechts, M.J., Benedek, S., 2004. Hybrid identification of nuclear power plant
transients with artificial neural networks. IEEE Trans. Ind. Electron. 51, 686–
693. https://doi.org/10.1109/TIE.2004.824874.
Fantoni, P.F., Mazzola, A., 1996. A pattern recognition-artificial neural networks
based model for signal validation in nuclear power plants. Ann. Nucl. Energy 23,
1069–1076. https://doi.org/10.1016/0306-4549(96)84661-5.
Hsieh, M.H., Hwang, S.L., Liu, K.H., Liang, S.F.M., Chuang, C.F., 2012. A decision
support system for identifying abnormal operating procedures in a nuclear
power plant. Nucl. Eng. Des. 249, 413–418. https://doi.org/10.1016/j.
nucengdes.2012.04.009.
Kim, Y., Jung, W., 2013. A Diagnosis Support System for Abnormal Situations of
Hanbit Units. Transactions of the Korean Nuclear Society Autumn Meeting.
Kingma, D.P., Ba, J.L., 2015. Adam: A method for stochastic optimization. 3rd Int.
Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc. 1–15.
Kingma, D. P., Mohamed, S., Rezende, D. J., & Welling, M., 2014. Semi-supervised
learning with deep generative models. In Advances in neural information
processing systems (pp. 3581-3589).
Lee, D., Arigi, A.M., Kim, J., 2020. Algorithm for autonomous power-increase
operation using deep reinforcement learning and a rule-based system. IEEE
Access 1–1. https://doi.org/10.1109/access.2020.3034218.
Lipton, Z.C., Kale, D.C., Elkan, C., Wetzel, R., 2016. Learning to diagnose with LSTM
recurrent neural networks. 4th Int. Conf. Learn. Represent. ICLR 2016 - Conf.
Track Proc. 1–18.
Mandrekar, J, 2010. Receiver operating characteristic curve in diagnostic test
assessment. Journal of Thoracic Oncology 5 (9), 1315–1316. https://doi.org/
10.1097/JTO.0b013e3181ec173d.
Monner, D., Reggia, J.A., 2012. A generalized LSTM-like training algorithm for
second-order recurrent neural networks. Neural Networks 25, 70–83. https://
doi.org/10.1016/j.neunet.2011.07.003.
Murphy, K.P., 2012. Machine Learning – A Probabilistic Perspective – Table-of-
Contents. MIT Press 1049. https://doi.org/10.1038/217994a0.
Na, M.G., Park, W.S., Lim, D.H., 2008. Detection and diagnostics of loss of coolant
accidents using support vector machines. IEEE Trans. Nucl. Sci. 55, 628–636.
https://doi.org/10.1109/TNS.2007.911136.
Ohga, Y., Seki, H., 1993. Abnormal event identification in nuclear power plants using
a neural network and knowledge processing. Nucl. Technol. 101, 159–167
https://doi.org/10.13182/NT93-A34777.
Park, S.H., Goo, J.M., Jo, C.H., 2004. Receiver operating characteristic (ROC) curve:
Practical review for radiologists. Korean J. Radiol. 5, 11–18. https://doi.org/
10.3348/kjr.2004.5.1.11.
16
Annals of Nuclear Energy 153 (2021) 108077
Park, D., Hoshi, Y., Kemp, C.C., 2018. A multimodal anomaly detector for robot-
assisted feeding using an LSTM-based variational autoencoder. IEEE Robot.
Autom. Lett. 3, 1544–1551. https://doi.org/10.1109/LRA.2018.2801475.
Park, J., Lee, D., Jung, W., Kim, J., 2017. An experimental investigation on relationship
between PSFs and operator performances in the digital main control room. Ann.
Nucl. Energy 101, 58–68. https://doi.org/10.1016/j.anucene.2016.10.020.
Park, J., Arigi, A.M., Kim, J., 2019. A comparison of the quantification aspects of
human reliability analysis methods in nuclear power plants. Ann. Nucl. Energy
133, 297–312. https://doi.org/10.1016/j.anucene.2019.05.031.
Ruder, S., 2016. An overview of gradient descent optimization algorithms 1–14.
Sßeker, S., Ayaz, E., Türkcan, E., 2003. Elman’s recurrent neural network applications
to condition monitoring in nuclear power plant and rotating machinery. Eng.
Appl. Artif. Intell. 16, 647–656. https://doi.org/10.1016/j.engappai.2003.10.004.
Shewhart, W.A., 1931. Economic control of quality of manufactured product.
Macmillan And Co Ltd, London.
Snoek, J., Larochelle, H., Adams, R.P., 2012. Practical bayesian optimization of
machine learning algorithms. Adv. neural Inf. Process. Syst., 2951–2959
Tishby, N., Zaslavsky, N., 2015. Deep learning and the information bottleneck
principle. 2015 IEEE Inf. Theory Work. ITW 2015. https://doi.org/10.1109/
ITW.2015.7133169.
Uhrig, R.E., Hines, J., 2005. Computational intelligence in nuclear engineering. Nucl.
Eng. Technol. 37 (2), 127–138.
Wang, W., Yang, M., Seong, P.H., 2016. Development of a rule-based diagnostic
platform on an object-oriented expert system shell. Ann. Nucl. Energy 88, 252–
264. https://doi.org/10.1016/j.anucene.2015.11.008.
Xu, H., Chen, W., Zhao, N., Li, Zeyan, Bu, J., Li, Zhihan, Liu, Y., Zhao, Y., Pei, D., Feng, Y.,
Chen, J., Wang, Z., Qiao, H., 2018. Unsupervised Anomaly Detection via
Variational Auto-Encoder for Seasonal KPIs in Web Applications. Web Conf.
2018 - Proc. World Wide Web Conf. WWW 2018 2, 187–196. https://doi.org/
10.1145/3178876.3185996.
Yang, J., Kim, J., 2018. An accident diagnosis algorithm using long short-term
memory. Nucl. Eng. Technol. 50, 582–588.
Yang, J., Kim, J., 2020. Accident diagnosis algorithm with untrained accident
identification during power-increasing operation. Reliab. Eng. Syst. Saf. 202.
https://doi.org/10.1016/j.ress.2020.107032.
Zhou, C., Paffenroth, R.C., 2017. Anomaly detection with robust deep autoencoders.
Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. Part F129685, 665–674.
https://doi.org/10.1145/3097983.3098052.