Structural Contr HLTH - 2019 - Li - Dam Monitoring Data Analysis Methods A Literature Review

Received: 11 July 2019 Revised: 28 October 2019 Accepted: 7 December 2019
DOI: 10.1002/stc.2501
REVIEW
Dam monitoring data analysis methods: A literature review
Bin Li1,2 | Jie Yang1,2 | Dexiu Hu1,2
1
State Key Laboratory of Eco-hydraulics
in Northwest Arid Region of China, Xi'an Summary
University of Technology, Xi'an, China Dam monitoring usually involves environmental variables (e.g., the water level
2
Institute of Water Resources and Hydro- and temperature) and effect variables (deformation, cracking, seepage, etc.).
electric Engineering, Xi'an University of
The associated monitoring data can reflect the trends in these variables over
Technology, Xi'an, China
time and are important information for managers to understand the opera-
Correspondence tional status of a dam. Therefore, research on monitoring data analysis
Jie Yang, State Key Laboratory of Eco-
hydraulics in Northwest Arid Region of
methods is very important for monitoring dam safety. Dam monitoring data
China, Xi'an University of Technology, analysis methods can be divided into monitoring model, monitoring index,
Xi'an 710048, China. and abnormal value detection methods. A monitoring model takes environ-
Email: lgdajs@163.com
ment variables as independent variables and effect variables as dependent vari-
Funding information ables. By studying the interactions among variables, the trends of effect
Water Science Plan Project of Shaanxi variables can be learned for monitoring and prediction. A monitoring index is
Province, Grant/Award Number:
2018SLKJ-5; The Key Projects of Natural established to denote warning or extreme value considering the previous
Science Basic Research Program of changes in effect variables to determine whether future changes are safe.
Shaanxi Province, Grant/Award Number:
Abnormal value detection is also an important method of finding abnormal
2018JZ5010; National Natural Science
Foundation of China, Grant/Award changes in the dam state. This paper summarizes the principles, research pro-
Numbers: 51579207, 51809212 gress, deficiencies, and development trends of these three types of monitoring
data analysis methods. This review promotes research in the field of dam
safety monitoring.
KEYWORDS
abnormal value detection, dam, monitoring data, monitoring index, monitoring model
1 | INTRODUCTION
Dam monitoring data are collected by GPS or the instruments installed on or inside a dam.1 The monitoring data can
show the variations in environmental variables (e.g., water level and temperature) over time and the effect variables
(deformation, cracking, seepage, etc.) that reflect the comprehensive state of the dam.2 When a dam is damaged or sub-
jected to an unconventional load, these effect variables will exhibit abnormal variations that are important to identify
the corresponding potential risk. The purpose of dam safety monitoring is to analyse these trends and variations and
determine whether they are normal or abnormal.
With the development of dam safety monitoring, dam monitoring has gradually been implemented automati-
cally and in real time, leading to an increasing number of monitoring variables and more collected data;
moreover, in intelligent monitoring, data analysis has gradually changed from offline and static to online and
dynamic, and to develop a universal strategy for dam safety management, it is necessary to establish a quantitative
analysis method for all managers.3,4 Therefore, research on the methods of dam monitoring data analysis is
important for real-time monitoring and intelligent monitoring and is of great significance for the healthy operation
of dams.5
Struct Control Health Monit. 2020;27:e2501. wileyonlinelibrary.com/journal/stc © 2019 John Wiley & Sons, Ltd. 1 of 14
https://doi.org/10.1002/stc.2501
15452263, 2020, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/stc.2501 by Queen'S University, Wiley Online Library on [30/05/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2 of 14 LI ET AL.
Predicting the variations of effect variables is often done by establishing a monitoring model. According to different
theories and methods, monitoring models can be divided into statistical models, machine learning (ML)-based models,
time series models, deterministic models, mixed models, and so forth.6-8 Monitoring models usually take environmental
variables as independent variables and effect variables as dependent variables to establish the interactive relationships
among variables. Statistical models are based on mature theory and have been widely used in practical engineering.
ML-based models have considerable potential in real-time monitoring and intelligent monitoring, and these models are
being rapidly developed. Therefore, this paper mainly focuses on statistical models and ML-based models.
Monitoring indices are usually established to denote warning values or extreme values. The purpose of these values
is to determine whether the effect variables are safe according to their previous trends. Indices play an important role
in dam safety management. Currently, the dam monitoring indices based on monitoring data mainly include the confi-
dence interval index and small probability index.
Anomaly detection is performed using various algorithms to find abnormal values or processes in monitoring data
sets. The abnormal values in dam monitoring data can be divided into jump values and spike values,9 as shown in Fig-
ure 1. These values are usually caused by three types of reasons: (a) a reading error, sensor error, or monitoring facility
anomaly; (b) unexpected loads, such as earthquakes, explosions, and landslides; and (c) internal damage to the dam.
The cause of an abnormal process is generally complicated and must be analysed according to the actual engineering
situation. Therefore, this paper only introduces abnormal value detection methods, which include statistical probability,
data mining, multiscale decomposition, and process line methods.
This paper is divided into three parts: a monitoring model, a monitoring index, and abnormal value detection. In
each part, the principles, research progress, shortcomings, and development trends of the relevant methods are intro-
duced. The content structure diagram is shown in Figure 2.
2 | M O N I T O R IN G M O D E L S
Monitoring models are generally established based on the relevant mathematical and mechanical principles, intelligent
algorithms, and so forth. The studies of monitoring models have generally focused on the following three points:7,10(a)
the mathematical relationships among environmental variables and effect variables, (b) the reasonable selection and
multicollinearity of environmental variables, and (c) improvements in model accuracy, robustness, and generalization
ability.
FIGURE 1 Jump values and spike values
FIGURE 2 The content structure diagram

of this paper
LI ET AL. 3 of 14
2.1 | Statistical model
Statistical models take environmental variables (the water level, temperature, and time) as independent variables and
effect variables (e.g., deformation, cracking, or seepage) as dependent variables to establish a regression model. A gen-
eral regression model can be expressed as follows:
y = δH + δT + δθ , ð1Þ
where y is the effect variable, δH is the water pressure component, δT is the temperature component, and δθ is a time-
dependent component. To improve the rationality of the physical interpretation of each component and the accuracy of
the model, the main research topics include the mathematical modelling of each component, regression modelling, the
selection of environmental variables, and multicollinearity analysis, among others.
Tonini11 first used statistical analysis to express both the water pressure component and temperature component as
a cubic polynomial. On this basis, combining dam theory, statistical theory, and mechanics, Several studies6,12 con-
ducted an in-depth study of the selection of environmental variables and the corresponding mechanisms. In addition,
mathematical models of the two components were developed, analysed, and discussed. According to creep theory,
Chen13 used a nonlinear parameter estimation method to analyse the time-dependent component of a concrete dam
and proposed an exponential function model, a hyperbolic function model, and a logarithmic function model of the
component. Additionally, many scholars have studied the selection and mathematical modelling of environmental vari-
ables. For example, (a) by considering less monitoring data during the reservoir storage period than during other
periods, Shen et al14 added a new “high water level factor” into the water pressure component to explain the variation
in the horizontal displacement of the dam. (b) Léger and Leclerc15 presented frequency domain solution algorithms of
the one-dimensional transient heat transfer equation that describes temperature variations in arch dam cross sections.
This solved the problem where the temperature variations are specified at the upstream and downstream faces. Then,
He and Shi16 used the average temperature Tm and the linear temperature difference Td to reflect and predict the
changes in the arch dam temperature field. Due to the uncertainties caused by a certain number of hypotheses in statis-
tical model, Tatin et al17 introduced the air temperature and the water temperature in the model so as to the mean and
the gradient of the temperature inside the structure could be taken into account properly. Then, a 2D finite element
model has been developed to study water temperature and temperature gradient in the assessment of the thermal dis-
placements. Finally, a hybrid physico-statistical model has been developed, and cases shows that the improved assess-
ment of thermal effects on reversible phenomena leads to a reduced uncertainty on residuals. (c) Considering the weak
shear resistance and large creep deformation of construction joints, Gu et al18 developed a Burgers' constitutive model
of a joint and improved the exponential model of the time-dependent component after finite element analysis; due to
the particularity of the environment in which a dam is located, there have been many similar studies.19-21
Multiple linear regression and stepwise regression are commonly used in statistical models. Multiple linear regres-
sion models have a long history and many applications.22 In multiple linear regression, the output is computed as a lin-
ear combination of the inputs.23 Hence, it is multilineal regression models, so the least square method is used to
estimate model parameters, and the model is directly established based on the environmental variables and effect vari-
ables.20,24 However, there are multiple environmental variables in statistical model. If there are more environmental
variables in the model, the environmental variables that have little effect on the effect variables are directly introduced
into the model. The coefficients of the environmental variables in the model may degenerate or become ill, which will
affect the accuracy of the equation, or even unable to be solved. For this problem, a stepwise regression model was
developed. In this model, environmental variables are added to the model one by one, and the significance of environ-
mental variables to the model is sequentially assessed to obtain the optimal variable set.25,26 With the development of
statistical regression, threshold regression,27 logistic regression,28 random forest regression,29 and so forth, many regres-
sion models have been gradually introduced into dam safety monitoring.
The key issues that affect the quality of a statistical model are the multicollinearity among environmental variables
and the effective optimization of the model. To address the problem of multicollinearity, Yang et al30 studied the corre-
lation among environmental variables and uncertainty of variables and used partial least square regression to effectively
eliminate multicollinearity. Xu and Deng31 used the truncated singular value decomposition method to establish a regu-
larized regression model that used a generalized crossover verification function and L-curve criterion to solve the
multicollinearity problem. Shao et al32 introduced the theory of panel data to dam deformation analysis, analysed moni-
toring data from both spatial and temporal dimensions, and built the potential relationship between different
4 of 14 LI ET AL.
measuring points based on the random-coefficient model of panel data, which is able to solve the multicollinearity
problem of traditional regression method. In the optimization of a model, the artificial immune algorithm,33 genetic
algorithm (GA),34 fruit fly optimization algorithm (FOA),35 and other optimization algorithms have been introduced to
optimize coefficients or independent variable sets and improve the quality of the model. The related research on statisti-
cal models is shown in Table 1.
The research on statistical models began early and has developed mature theories over time. The accuracy of a sta-
tistical model meet engineering requirements so that effect variables of a dam can be reasonably analysed. Thus, these
methods have been widely used in engineering. However, statistical models still have some shortcomings: (a) it is diffi-
cult to reflect the relationship between the dam and the foundation; (b) the monitoring data are often incomplete due
to instrument failure or damage, and as a result, modelling requirements may not be met; (c) the water pressure compo-
nent, temperature component, and time-dependent component are linearly superposed in the model, which does not
conform with the actual nonlinear relationship; and (d) the accuracy, robustness, and generalization of the model must
be continuously improved.
2.2 | ML-based model
ML is a discipline that involves training computers to simulate human learning activities. A variety of algorithms, such
as the back propagation neural network (BPNN), extreme learning machine (ELM), and support vector machine (SVM)
TABLE 1 Related research on statistical models
Research
content Author Theory or method Research result
The components Tonini D. Statistical analysis Expressed the water pressure component and temperature component as
a cubic polynomial
Wu Z. Dam theory; Statistics; Analysed and deduced mathematical models of the water pressure and
Mechanics temperature components
Chen J. Creep theory Proposed many mathematical models for time-dependent component
Shen D. Dam theory Added a new “high water level factor”
Léger P. Discrete Fourier series presented frequency domain solution algorithms of the one-dimensional
transient heat transfer equation
He J. Dam theory Tm and Td were used as temperature factors
Tatin M. Dam theory Air temperature and the water temperature were taken into account
Gu C. Creep theory Improved the mathematical model of the time-dependent component
Modelling Li M.; Mata Multiple regression Simple operation and reliable precision
methods J.
Hu D.; Stepwise regression The significance among variables was considered
Shen W.
He X. Threshold regression The threshold factor and threshold were introduced
Lan S. Logistic regression Introduced the logistic regression curve
Dai B. Random forest The environmental variables were extracted according to importance
regression
Multicollinearity Yang J. PLSR PLSR model
Xu C. TSVD Regularized regression model
Shao CF theory of panel data Built the relationship between different measuring points based on the
random-coefficient model
Model Xi G. AIA Optimization of model parameters or variable sets
optimization Zhang Y. FOA
Stojanovica GA
B.
Abbreviations: AIA, artificial immune algorithm; FOA, fruit fly optimization algorithm; GA, genetic algorithm; PLSR, partial least square regression; TSVD,
truncated singular value decomposition.
LI ET AL. 5 of 14
algorithms are used for ML. This approach is effective for dam safety monitoring.7,36,37 The main studies of ML-based
models focused on the accuracy and convergence speed of models, as well as obtaining the optimal solution.
A BPNN consists of an input layer, an implicit layer, and an output layer, as shown in Figure 3. After setting the
network structure and initializing the weights and thresholds of the neurons, an output value can be obtained when a
set of data is input. If the error between the output value and the actual value does not meet the requirements, the error
will be back propagated. Then, the parameters are adjusted by the gradient descent method, and the sum of the squared
errors of the output layer is minimized.
The key to building a good neural network (NN) model is to choose a reasonable network structure and training
algorithm. In choosing a reasonable network structure, Su et al38 used the floating point matrix for coding and added
some restriction conditions to the calculation process of a GA. This method can effectively find an appropriate network
structure and parameters. More environmental variables lead to more input variables of NN, which will affect the com-
putational efficiency and accuracy of the network. Therefore, Wang and Xu39 used the linear mapping transformation
of PCA to compress the 12 related input variables that affect the dam displacement into two independent input vari-
ables, which not only eliminated the multicollinearity of the 12 input variables, but also simplified the input layer of
the network and optimised the network structure. In the selection of the training algorithm and optimization of model
parameters, Li et al40 improved the accuracy and convergence speed of a BPNN by applying unscented Kalman filter
theory and introducing an adaptive factor into the BPNN. Rankovic et al41 used an improved resilient propagation algo-
rithm to train a feedforward NN. In contrast to the standard backpropagation algorithms, this algorithm takes the sign
of the derivative to indicate the direction of the weight update for calculation of the weight changes. Based on the classi-
cal metrics of monitoring model, a parametric study was performed in the form of a trial-and-error process and sensitiv-
ity analysis, and De Granrut et al42 selected the best parameters of artificial NN by observing the simulation graphical
results to prevent model overfitting. In addition, some models have been designed to improve the convergence speed
and find the optimal solution of a BPNN; these methods, such as the particle swarm optimization algorithm43 and
GA,44 optimize the weights and thresholds of the network structure. There have been many studies of NNs, such as the
hierarchical diagonal NN,45 autoregressive integrated moving average-BPNN,46 evolving artificial NN,47 and others.
This type of method has played an important role in the development of intelligent monitoring due to the advantages of
self-learning and self-adaptation.
An ELM is a type of feedforward NN with a single hidden layer. ELMs improve on the generalization performance,
convergence speed, and accuracy of BPNNs.48 Cheng and Xiong49 studied the reasonableness of the input variables of
an ELM and took the average values of the predicted results of a grey model (1,1) and regression analysis model (RAM)
as the input; the measured values were set as the output, and this approach improved the prediction accuracy of the
ELM. The accuracy of the ELM model is affected by the input weights and the hidden layer thresholds. Therefore,
Zhang and Fan50 used the improved particle swarm optimization algorithm to optimize the parameters of ELM and
achieve global optimization. This model also improved the generalization ability of the algorithm.
Compared with BPNNs, SVMs are based on statistical theory and have a simple structure. SVMs have many advan-
tages in solving small-sample, nonlinear, and high-dimensional learning problems.51 Song 52 introduced the least
squares method in SVM and combined it with the harmony search algorithm to establish the harmony search least
squares-SVM model, which greatly improved upon the convergence speed of the traditional SVM. Rankovic et al53 stud-
ied the parameter selection of support vector regression through a trial and error method. The results showed that the
performance of the support vector regression model was highly influenced by the parameter values. Considering the
FIGURE 3 Schematic diagram of the neural network structure

6 of 14 LI ET AL.
advantages of SVMs, Qian et al54 used an SVM to fit the trend of sample data and then fit the residual sequence with a
radial basis function NN. Finally, the SVM and radial basis function NN were linearly superimposed, and this model
was more accurate than the traditional SVM model. Salazar10 used sensitivity analysis to study the effect of training set
size on the accuracy of SVM. The results show that SVM is susceptible to the size of the training set, given that it has
more hyper-parameters and is more sensitive to the presence of correlated or uninformative inputs. Therefore, it is
advisable to select carefully the most appropriate training set size. These ML-based models and studies are listed in
Table 2.
ML-based models are characterised by effective self-organization, self-learning, self-adaptation, and nonlinear map-
ping. In terms of dam safety monitoring, although scholars have studied the network structure, computational speed
and accuracy of ML-based models, the models are still not widely used in practice due to the following shortcomings:
(a) the complexity of the network structure and the samples have a great influence on the models, which leads to a low
generalization ability and “overlearning”23; (b) the choice of the model parameters directly affects the learning perfor-
mance, and it is easy to converge to local minima, which results in poor stability; (c) the training data must be
standardised before modelling, and the future measured values may exceed the extreme values in the training data,
which makes it difficult to reasonably compare the predicted and measured values; and (d) the models have a black box
structure, so the water pressure component, temperature component, and time-dependent component cannot be
extracted and clarified.
3 | M O N I T O R IN G I N D I C E S
A monitoring index is generally established to denote warning value or extreme value set. The purpose of a monitoring
index is to determine whether the effect variables are within a safe range. Such indices are important for evaluating the
operational state of a dam and play an important role in ensuring dam safety.55Currently, the methods of establishing
monitoring indices based on monitoring data include the confidence interval method and small-probability method.
These two methods are mainly for univariate data analysis. The reason is that univariate data can most directly reflect
TABLE 2 Machine learning-based models and the corresponding research
Model Author Theory or method Modelling improvement

BPNN Su H. GA Model structure and accuracy
Wang Y. PCA Accuracy
Yan B. PSO Convergence speed and optimal solution
Zhang F. GA
Zhao E. HDNN Convergence speed and accuracy
Li L. UKF and adaptive factor Accuracy
Zhao E. HDNN Convergence speed and accuracy
Feng L. ARIMA Accuracy
Stojanovic B. EANN Accuracy
Hochreiter S. RNN
Rico J. RNN
ELM Cheng J. GM(1,1) and RAM Accuracy
Zhang H. IPSO Accuracy and generalization
SVM Song Z. LS and HSA Convergence speed
Rankovic V. Trial and error Parameter selection
Qian C. RBFNN Accuracy
Salazar F. SA Selection of training set
Abbreviations: ARIMA, autoregressive integrated moving average; BPNN, back propagation neural network; EANN, evolving artificial neural network; ELM,
extreme learning machine; GA, genetic algorithm; GM, Grey model; RAM, regression analysis model; HDNN, hierarchical diagonal neural network; HSA,
harmony search algorithm; IPSO, improved particle swarm optimization; LS, least square; PSO, particle swarm optimization; RBFNN, radial basis function
neutral network; RNN, recurrent neural network; SA, sensitivity analysis; SVM, support vector machine; UKF, unscented Kalman filter.
LI ET AL. 7 of 14
TABLE 3 Statistical probability methods for abnormal value detection involving dam monitoring data
Discriminant Applicable sample

Criterion criterion range Shortcoming
Parita 3σ 185 < n 1. It is difficult to guarantee the accuracy of long-term prediction
Chauvenet wnσ 25 ≤ n < 185 2. It ignores the information associated with similar measured points
0
Grubbs G and G 3 ≤ n < 185 3. It assumes a normal distribution is different than the actual distribution
Dixon Qi 3 ≤ n < 25 4. It is not possible to determine whether the monitoring data are safe or not
the real information of the measurement point, and the univariate analysis is also the basis of multivariate analysis. Just
like the linear discriminant model presented by Mata et al,56 the model can combine the multiple physical quantities
measured by the automated monitoring system, appropriately weighted, into a new single index allowing the classifica-
tion of the observations into one of the two classes (normal behaviour and development of a failure scenario). This is
still ultimately a univariate analysis.
3.1 | Confidence interval method
The basic principles of the confidence interval method are mathematical and statistical theories. The applicable premise
is that the sample data need to satisfy a certain probability distribution.57 However, the water level usually has a period-
icity according to the regulation of the reservoir. The temperature often displays annual periodicity. Therefore, the dam
monitoring data usually have a certain regularity and do not satisfy a probability distribution.
Therefore, establishing a monitoring index with the confidence interval method requires the use of a monitoring
model. The process is shown in Figure 4. The monitoring model can fit the relationship between environmental vari-
ables and effect variables. The residuals between the fitting value and the measured value should be a stationary
sequence that satisfies a certain probability distribution. The residual sequence Δyi is taken as a sample and expressed
as follows:
FIGURE 4 Flow chart of the monitoring index of the confidence

interval method
8 of 14 LI ET AL.
Δyi fΔy1 , Δy2 , …, Δyn g,
where Δyi is the residuals value of ith moment, (i = 1, 2, … ,n).

and σ Δy, respectively.
The mean value and standard deviation of the sample data Δy are expressed as Δy
Xn
=1
Δy Δy , ð2Þ
n i=1 i
rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1 Xn Þ2
σ Δy = ðΔyi − Δy ð3Þ
n−1 i=1
where yi is the measured value of ith moment, (i = 1, 2, … ,n).

The monitoring index E is given as follows:
E = y0i wσ Δy , ð4Þ
where y0i is the predicted value of ith moment (i = n + 1, n + 2, … ,n + m) and w is 2 or 3 based on the significance level
α (usually 1% to 5%).
If the measured value falls within the range of the monitoring index, it is a safe value. Otherwise, according
to the change rule of the previous data, the value is an unsafe value. However, the unsafe value may also be a
safe value if the dam is under a new action. Therefore, monitoring indices need to be re-established when the
dam is in a new action. The flow chart of establishing a monitoring index based on the confidence interval
method is shown in Figure 4.
3.2 | Small-probability method
A small-probability event is an event that has a probability of occurrence less than 5%. The premise of the small-proba-
bility method is that structural conditions, material properties, foundation properties, and working conditions are
unchanged. Thus, the effect variables are considered as random variables. The effect variables under an unfavourable
load are taken as a sample that is expressed as follows:
y : fy1 , y2 , …, yn g:
The mean value y and standard deviation σ y of the sample data y are calculated according to Equations (2) and (3).
Then, a statistical test method (such as the Kolmogorov–Smirnov method or Anderson–Darling method) be selected to
test the probability density function f(y) and the distribution function F(y) of the sample y. Then, according to the
abnormal probability α, the monitoring index E is calculated based on the following formula58:
ð∞
P ðy > E Þ = f ðyÞdy, ð5Þ
E

E = F − 1 y, σ y , α : ð6Þ
If the effect variables exceed the index E, a small-probability event occurs, and the dam state may be abnormal or
dangerous; otherwise, compared with the previous data, the value is an unsafe value. However, the unsafe value may
also be a safe value if the dam is under a new action. Therefore, it is necessary to re-establish a monitoring index E for
the new action. The flow chart of establishing a monitoring index based on the small-probability method is shown in
Figure 5.
LI ET AL. 9 of 14
F I G U R E 5 Flow chart of a monitoring index based on the small-

probability method
If the effect variables exceed the index, a small-probability event occurs, and the dam state may be abnormal or dan-
gerous. Otherwise, according to the previous state of the dam under the unfavourable load, the value is an unsafe value.
However, the unsafe value may also be a safe value if the dam is under a new action. Therefore, monitoring indices also
need to be re-established when the dam is in a new action. The flow chart of establishing a monitoring index based on
the small-probability method is shown in Figure 5.
The confidence interval method and small-probability method are based on statistical probability theory. These two
methods are simple and easy to understand and have been applied in practical engineering. The confidence interval
method uses the residual sequence between the fitted value and the measured value as a sample. The monitoring model
has a limited prediction length, so it needs to be updated regularly. The choice of a confidence range depends on the
sample size and expert experience. If the confidence range is too wide, unsafe values may not be detected. In contrast, a
large number of safe values may be deemed unsafe if the range is too narrow.
The small-probability method considers the effect of an unfavourable load on the strength and stability of a dam.
However, to obtain a set of suitable sample data, long-term data are needed, and the data should be measured under
unfavourable loads. It is difficult to obtain practical data in this situation. Additionally, the effect variables have a cer-
tain regularity, which is inconsistent with the assumptions of the small-probability method.
In addition, the confidence interval method and small-probability method are based on statistical concepts that do
not reflect the physical concepts and importance of dams, and there is no information that reflects the relevant spatial
correlations.
4 | ABNORMAL VALU E DETEC T I O N
Abnormal value detection is the most important task in dam monitoring. Identifying abnormal values in an appropriate
time is a key step in dam health diagnosis. This section introduces statistical probability methods, data mining, multi-
scale decomposition, and process line methods, all of which provide effective detection results and excellent application
prospects.
4.1 | Statistical probability method
Statistical probability methods are based on statistics and probability theory. This type of method uses a criterion to
detect abnormal values in monitoring data. In dam monitoring data, the Parita criterion, Chauvenet criterion, Grubbs
10 of 14 LI ET AL.
criterion, and Dikon criterion are generally used. These methods apply to different data size; in addition, the periodicity
and representativeness of the data is also a fundamental to obtain a good performance of each method. The duration of
the time period and the range of the main loads are also fundamental to obtain a good performance.
The Parita criterion requires the calculation of the mean value u and standard deviation σ of the data.59 If the differ-
ence between the measured value and u is greater than 2σ or 3σ, the measured value is an abnormal value.60 Thus, the
criterion is a type of confidence interval method. This criterion applies to sample sizes larger than 185. However, when
an abnormal value is detected, u and σ need to be calculated for the remaining data. The Chauvenet criterion is similar
to the Parita criterion, but the Chauvenet criterion is wnσ, not 3σ. wn is a variable related to the data set size n. This cri-
terion applies to sample sizes between 25 and 185.
Both the Grubbs criterion and Dixon criterion involve the ranking of data from small values to large. The sorted
data are expressed as follows:
n o
yð1Þ ,yð2Þ , …, yðnÞ :
After calculating mean value u and standard deviation σ, the Grubbs criterion also need to calculate the statistics G
0
and G as follows61:
yð1Þ −y 0 yðnÞ −y

G= G = ð7Þ
σ σ
The Dixon criterion need to calculate the statistic Qi as follows62:
X 2 y
ðnÞ −yn − i
Qi = : ð8Þ
y − yð1 + jÞ
i = 1 ðnÞ
These two methods must find the critical values G(α, n) and Q(α, n) from a table according to the test level α (gener-
0
ally 0.05 or 0.01) and the sample size n. Finally, if G or G exceeds the G(α, n), or Qi exceeds the Q(α, n), it indicates that
there is an abnormal value in the sample data. Otherwise, the original data are normal. The Grubbs criterion applies to
sample sizes between 3 and 185, and it has maximum accuracy when there is only one abnormal value in the data set.
The Dixon criterion applies to sample sizes between 3 and 25. These criteria are summarised in Table 1.
Statistical probability methods are supported by complete statistical theory. Different criteria apply to different
sample sizes. This type of method has been applied to detect the abnormal values in dam monitoring data sets,
but there are also some shortcomings to such methods. (a) These methods require stationary sequences as sam-
ples, so it is necessary to apply a monitoring model and use the residual sequence for detection. However, the
prediction length of the monitoring model is generally limited, and the accuracy of long-term detection cannot be
guaranteed. (b) It is usually assumed that the residual sequence obeys a normal distribution, which is often differ-
ent from the actual distribution. (c) These methods only detect anomaly values of a single point and ignore the
spatial information associated with measurement points. (d) Although these methods can detect anomaly values, it
is unknown whether these values are safe.
4.2 | Data mining
Data mining is the process of searching for information hidden in a large amount of data through algorithms. Outlier
detection is an important research direction in data mining.63 The purpose of this approach is to detect outliers in large
data sets that influence models.64 Such methods have been widely used in various fields.65
The local outlier factor (LOF) method is a density-based algorithm for outlier detection. This method determines
whether a value is an outlier based on the density of the value within a certain range. If the density is small, the value is
considerably different from most of the data. Conversely, a large density suggests that the value is similar to most of the
data. Considering the correlation between the water pressure, temperature, and dam deformation, Zheng66 used the
Eros distance to measure the similarity between different data sequences and then used the LOF algorithm to detect
LI ET AL. 11 of 14
outliers in deformation monitoring data. Hu et al67 used a global time constraint to calculate the dynamic time warping
distance and measure the similarity between different data sequences; then, the multivariate LOF algorithm was
applied.
There are many types of dam monitoring data sets with various sizes, relationships, and regularity. Data mining
technology has advantages in obtaining implicit information from dam monitoring data sets,68 such as through outlier
detection, trend detection, and correlation and similarity detection. There are many data mining algorithms, such as
deep learning-based algorithms, cluster-based algorithms, probability-based algorithms, and classification-based algo-
rithms.69,70 However, there have been few studies or applications of these methods in dam safety monitoring.
4.3 | Multiscale decomposition
Dam monitoring data can be regarded as being composed of digital signals with different frequencies. Therefore,
monitoring data can be decomposed on multiple scales according to the frequency. The sequences after decompo-
sition are studied to determine the abnormal values in the monitoring data. Wavelet analysis is an effective
method of decomposing data at multiple scales. A data sequence with outliers is characterised by modulus max-
ima in wavelet analysis. Specifically, the abnormal value is detected based on the modulus maxima. Research of
Xu et al71 and Liu and Lian72 indicated that abnormal values can be obviously detected in high-frequency
sequences after wavelet decomposition. Compared with a monitoring model, wavelet analysis can effectively detect
abnormal values.
Multiscale decomposition is suitable for detecting single or multiple abnormal values, and it has advantages in
causal analysis and monitoring models.73,74 However, this method cannot reflect the reason for the abnormal values,
which requires further analysis.
4.4 | Process line method
Usually, there is more than one measurement point for effect variables of the same type. In the same environment, the
monitoring data at different points have a certain spatiotemporal correlation.75 In the process line method, a process
line of the monitoring data collected at different measurement points is constructed. Then, it is determined whether
there are abnormal values or abnormal procedures in the data according to expert experience.22 This method is simple,
convenient, and is suitable for small monitoring data sets or cases in which the monitoring model has not yet been
established. Although this method is time-consuming and labour intensive in practice and relies too much on the level
of expert experience, it is still a common method of detecting and analysing abnormal values in data sets related to the
practical engineering of dams.
5 | C ON C L U S I ON S
Monitoring data analyses of dams are mainly used to detect abnormal values, analyse the cause of these abnormalities,
and monitor dam safety in real time. This paper reviewed the relevant data analysis methods, including monitoring
models, monitoring indices, and anomaly value detection methods.
1. A monitoring model can fit the trend of effect variables. Statistical models are based on mature theory and have
clear physical meaning, and they have been widely applied in practical engineering. However, the selection and
expression of environmental variables and the problem of multicollinearity among variables require further study.
ML-based models are characterised by effective self-organization, self-learning, self-adaptation, and nonlinear map-
ping; thus, they are important methods in intelligent monitoring. Although these models have high prediction accu-
racy, their stability and generalization are insufficient. Thus, ML-based models must be continually improved to
achieve intelligent monitoring.
2. Monitoring indices are very important for evaluating and monitoring dam safety. The confidence interval method
and small-probability method have great significance in statistical probability theory, but the physical cause of dam
12 of 14 LI ET AL.
safety issues is not considered. Establishing a monitoring index is a very complicated problem that involve many fac-
tors, such as the strength, stiffness, and damage properties of the concrete and foundation. It is also necessary to
combine many theoretical methods, such as those based on elastic mechanics, shape mechanics, and elastoplastic
mechanics, to effectively establish a monitoring index.
3. Abnormal value detection is an important monitoring task to determine whether the dam state is abnormal
according to the monitoring data. The process line method is widely used in dam monitoring analysis. However,
with the development of data analysis methods, more anomaly detection and data mining algorithms will be
increasingly developed and introduced to promote real-time and intelligent dam monitoring.
This paper summarises the main analysis methods for dam monitoring data. These methods usually require com-
plete and reliable data. However, the measured data usually contain gross error, random error, and systematic error, as
well as missing values. Currently, there are few studies of data preprocessing. To effectively use the relevant algorithms
and models to perform intelligent monitoring based on traditional analysis methods, it is necessary to continuously
combine new theories, methods, and techniques and perform data preprocessing.
ACK NO WLE DGE MEN TS

This paper is supported by National Natural Science Foundation of China, Grant/Award 51809212, 51579207; project
funded by the Key Projects of Natural Science Basic Research Program of Shaanxi Province, Grant/Award 2018JZ5010
and the Water Science Plan Project of Shaanxi Province, Grant/Award 2018SLKJ-5.
ORCID
Bin Li https://orcid.org/0000-0002-4661-0168
R EF E RE N C E S
1. Taşçi L. Dam deformation measurements with GPS. Geodesy and Cartography. 2008;34(4):116-121. https://doi.org/10.3846/1392-1541.
2008.34.116-121
2. Ma M, Shen ZZ, Tu XX. Study on deformation early warning index of concrete gravity dam. 11th ASCE Aerospace Division International
Conference (Earth and Space). 2008;1-10. https://doi.org/10.1061/40988(323)33
3. Li QB, Lin P. Demonstration on intelligent dam. Journal of Hydroelectric Engineering. 2014;33(1):139-146.
4. Zhong DH, Shi MN, Cui B, Wang JJ, Guan T. Research progress of the intelligent construction of dams. Journal of Hydraulic Engineer-
ing. 2019;50(1):38-52.
5. Pereira S, Magalh~aes F, Gomes GP, Cunha A, Lemos JV. Dynamic monitoring of a concrete arch dam during the first filling of the reser-
voir. Eng Struct. 2018;174:548-560.
6. Wu ZR. Safety Monitoring Theory & Its Application of Hydraulic Structures. 1st ed. Beijing: Higher Education Press; 2003.
7. Salazar F, Toledo MA, Oñate E, Morán R. An empirical comparison of machine learning techniques for dam behaviour modelling. Struct
Saf. 2015;56:9-17.
8. Cheng X, Li QQ, Zhou ZW, Luo ZX, Liu M, Liu L. Research on a seepage monitoring model of a high core rockfill dam based on
machine learning. Sensors. 2018;18:1-14.
9. Chen B, Hu TY, Huang ZS, Fang CH. A spatio-temporal clustering and diagnosis method for concrete arch dams using deformation
monitoring data. Struct Health Monit. 2019;18:1355–1371.
10. Salazar F. A machine learning based methodology for anomaly detection in dam behaviour. Catalunya: Universitat Politècnica de Catalu-
nya; 2017.
11. Tonini D. Observed behavior of several Italian arch dams. Journal Power Division. 1956;82:1-26.
12. Wu ZR, Shen CS, Ruan HX. Factor selection of statistical model for concrete dam deformation. Journal of Hohai University. 1988;16(6):
1-9.
13. Chen JY. The nonlinear parameter estimation of aging deformation of concrete dam. Dam Observation and Geotechnical Testing. 1980;
3-13.
14. Shen DB, Huang HB, Wang D, Hu MX. Research on new statistical model of horizontal displacement monitoring in gravity dam. Water
Resources and Power. 2012;30(12):81-84.
15. Léger P, Leclerc M. Hydrostatic, temperature, time-displacement model for concrete dams. J Eng Mech. 2007;133(3):267-277.
16. He JP, Shi YQ. Study on TMTD statistical model of arch dam deformation monitoring. Procedia Engineering. 2011;15:2139-2144.
17. Tatin M, Briffaut M, Dufour F, Simon A, Fabre JP. Thermal displacements of concrete dams: accounting for water temperature in statis-
tical models. Eng Struct. 2015;91:26-39.
18. Gu CS, Li Y, Song JX. Study on safety monitoring model for deformation of RCCD. Chinese Journal of Computational Mechanics. 2010;
27(2):286-290.
LI ET AL. 13 of 14
19. Gu CS, Li B, Xu BS, Hu J. Safety monitoring model of roller-compacted concrete dam seepage. Journal of Hohai University (Natural Sci-
ences). 2008;36(5):579-584.
20. Mata J, Tavares de Castro A, Sá da Costa J. Constructing statistical models for arch dam deformation. Struct Control Health Monit. 2013;
21:423-437.
21. Tatin M, Briffaut M, Dufour F, Simon A, Fabre JP. Statistical modelling of thermal displacements for concrete dams: influence of water
temperature profile and dam thickness profile. Eng Struct. 2018;165:63-75.
22. Rico J, Barateiro J, Mata J, Antunes A, Cardoso E. Applying advanced data analytics and machine learning to enhance the safety control
of dams. Machine Learning Paradigms. 2019;315-350.
23. Salazar F, Morán R, Toledo MA, Oñate E. Data-based models for the prediction of dam behaviour: a review and some methodological
considerations. Arch Comput Method Eng. 2015;24(1):1-21.
24. Li MJ, Wang JX. An empirical comparison of multiple linear regression and artificial neural network for concrete dam deformation
modelling. Math Probl Eng. 2019;1-13.
25. Hu DX, Zhou ZQ, Li Y, Wu XL. Dam safety analysis based on stepwise regression model. Advanced Materials Research. 2011;204:2158-
2161.
26. Shen WW, Ren JM. Multiple stepwise regression analysis crack open degree data in gravity dam. Applied Mechanics and Materials. 2014;
477:888-891.
27. He XF, Leng YB, Zhao SG. Threshold regression forecast model for dam safety monitoring and its application. Applied Mechanics and
Materials. 2010;36:182-186.
28. Lan SY, Fang C. Application of least square method based logistic regression model to settlement prediction of earth-rock fill dam. Water
Resources and Hydropower Engineering. 2012;43:16-18.
29. Dai B, Gu CS, Zhao EF, Qin XN. Statistical model optimized random forest regression model for concrete dam deformation monitoring.
Struct Control Health Monit. 2018;25:1-15.
30. Yang J, Hu DX, Wu ZR. Research on factor correlation and uncertainty of dam safety monitoring model. J Hydraul Eng. 2004;12:99-105.
31. Xu C, Deng CF. Solving multicollinearity in dam regression model using TSVD. Geo-spatial Information Science. 2011;14:230-234.
32. Shao CF, Gu CS, Yang M, Xu YX, Su HZ. A novel model of dam displacement based on panel data. Struct Control Health Monit. 2018;
e2037.
33. Xi GY, Yue JP, Zhou BX, Tang P. Application of an artificial immune algorithm on a statistical model of dam displacement. Comput
Math Appl. 2011;62:3980-3986.
34. Stojanovica B, Milivojevic M, Ivanovic M, Milivojevic N, Divac D. Adaptive system for dam behavior modeling based on linear regres-
sion. Adv Eng Softw. 2013;65:182-190.
35. Zhang YJ, Zheng DJ, Zheng ZT, Huang CS. Concrete dam deformation prediction model based on optimized fruit fly algorithm and sta-
tistical model and its application. Hydropower Energy Science. 2019;37(2):91-94.
36. Mata J. Interpretation of concrete dam behaviour with artificial neural network and multiple linear regression models. Eng Struct. 2011;
33:903-910.
37. Santillán D, Fraile-Ardanuy J, Toledo MA. Dam seepage analysis based on artificial neural networks: the hysteresis phenomenon. The
2013 International Joint Conference on Neural Networks (IJCNN). IEEE, 2013: 1-8.
38. Su HZ, Wu ZR, Wen ZP, Shen ZZ. Application of genetic algorithm in the establishment of neural network prediction model for dam
safety monitoring. J Hydraul Eng. 2001;44-48.
39. Wang YH, Xu C. Using genetic artificial neural network to model dam monitoring data. Second International Conference on Computer
Modeling and Simulation. 2010;2:3-7.
40. Li LH, Peng SJ, Jiang ZX. Prediction model of concrete dam deformation based on adaptive unscented Kalman Filter and BP neural net-
work. Applied Mechanics and Materials. 2014;513:4076-4079.
41. Rankovic V, Novakovic A, Grujovic N, Divac D, Milivojevic N. Predicting piezometric water level in dams via artificial neural networks.
Neural Computing and Applications. 2013;24(5):1115-1121.
42. De Granrut M, Simon A, Dias D. Artificial neural networks for the interpretation of piezometric levels at the rock-concrete interface of
arch dams. Eng Struct. 2019;178:616-634.
43. Yan B. Study on intelligent neural network model for dam safety monitoring and evaluation. Dalian: Dalian University of Technology;
2007.
44. Zhang F. Research on dam safety monitoring model based on neural network. Nanjing: Southeast University; 2016.
45. Zhao EF, Jin YQ. Dam deformation monitoring model and forecast based on hierarchical diagonal neural network. In: 4th International
Conference on Wireless Communications, Networking and Mobile Computing; 2008:1-4.
46. Feng LL, Li X. Dam safety monitoring model based on neural network and time series. Applied Mechanics and Materials. 2014;631:
543-547.
47. Stojanovic B, Milivojevic M, Milivojevic N, Antonijevic D. A self-tuning system for dam behavior modeling based on evolving artificial
neural networks. Adv Eng Softw. 2016;97:85-95.
48. Kang F, Liu J, Li JJ, Li SJ. Concrete dam deformation prediction model for health monitoring based on extreme learning machine. Struct
Control Health Monit. 2017;24:1-11.
49. Cheng JT, Xiong Y. Application of extreme learning machine combination model for dam displacement prediction. Procedia Computer
Science. 2017;107:373-378.
14 of 14 LI ET AL.
50. Zhang HL, Fan ZD. Nonlinear monitoring model for concrete dam deformation based on improved pso-elm algorithm. Hydropower
Energy Science. 2018;36(1):82-85.
51. Tabari MMR, Sanayei HRZ. Prediction of the intermediate block displacement of the dam crest using artificial neural network and sup-
port vector regression models. Soft Comput. 2019;23:9629-9645.
52. Song ZY. Study on dam safety monitoring method based on intelligent calculation. Dalian: Dalian University of Technology; 2007.
53. Rankovic´ V, Grujovic´ N, Divac D. Development of support vector regression identification model for prediction of dam structural
behaviour. Struct Saf. 2014;48:33-39.
54. Qian C, Li LJ, Zhou ZD. Application of SVM-RBFNN model in deformation monitoring of concrete double curvature arch dam. Water
Resources and Power. 2015;33(12):93-96.
55. Yang J, Wu ZR. Present conditions and development of dam safety monitoring and control researches home and abroad. Journal of Xi'an
University of Technology. 2002;18:26-30.
56. Mata J, Schclar Leit~ao N, Tavares de Castro A, Sá da Costa J. Construction of decision rules for early detection of a developing concrete
arch dam failure scenario. A discriminant approach. Comput Struct. 2014;142:45-53.
57. Li SY, Zhang W, Yao Y, Gu YN. Safety monitoring index determination of foundation seepage uplift of Fengman Dam. People's Yellow
River. 2011;33(4):124-126.
58. Zhong L, Zheng DJ, Xiang Y. Determination of horizontal displacement monitoring index for shuikou dam crest. Water Resources and
Power. 2004;22:13-15.
59. Xu BB, Cui CF. Research of outlier detection on automatic monitoring data of dam. Journal Geomatics. 2015;40(2):59-61.
60. Salazar F, Toledo MA, González JM, Oñate E. Early detection of anomalies in dam performance: a methodology based on boosted
regression trees. Struct Health Monit. 2017;24(11):1-16.
61. Tao JX, Xiong HY, Hu B. Method for judging abnormal values of dam safety monitoring data. J of Three Gorges Univ (Natural Sciences).
2016;38(6):15-18.
62. Lach S. The application of selected statistical tests in the detection and removal of outliers in water engineering data based on the exam-
ple of piezometric measurements at the Dobczyce dam over the period 2012-2016. E3S Web of Conferences. 2018;45:00045.
63. Shang WQ. Artificial Intelligence. 1st ed. Beijing: Tsinghua University Press; 2017.
64. Hawkins D. Identification of Outliers. 1st ed. London: Chapman and Hall; 1980.
65. Domingues R, Filippone M, Michiardi P, Zouaoui J. A comparative evaluation of outlier detection algorithms: experiments and analyses.
Pattern Recognit. 2018;74:406-421.
66. Zheng F. Abnormal deformation analysis of concrete dam based on local outlier factor. Water Resources and Power. 2016;34(6):103-105.
67. Hu J, Ma FH, Wu SH. Anomaly identification of foundation uplift pressures of gravity dams based on DTW and LOF. Struct Control
Health Monit. 2018;25:1-17.
68. Su HZ, Gu CS, Wu ZR. Application of artificial intelligence theory to dam safety monitoring. Dam Observation and Geotechnical Test.
2000;24(3):7-9.
69. Huang HY, Lin JX, Chen CC, Fan MH. Review of outlier detection. Application Research of Computers. 2006;8-13.
70. Agrawal S, Agrawal J. Survey on Anomaly Detection using data mining techniques. Procedia Computer Science. 2015;60:708-713.
71. Xu HZ, Wu ZR, Li XH, Shi B. Outliers detecting of dam observation data based on wavelet analysis. International Journal Hydroelectric
Energy. 2002;20(4):20-22.
72. Liu J, Lian JJ. Outliers detection of dam displacement monitoring data based on wavelet transform. Applied Mechanics and Materials.
2011;71:4590-4595.
73. Su HZ, Wu ZR. Identification model for dam behavior based on wavelet network. Computer-Aided Civil Infrastruct Eng. 2007;22:438-448.
74. Zheng KH, Wang ZQ. Application of systematical optimization GM (1,1) model based on lifting wavelet in dam displacement forecast.
Applied Mechanics and Materials. 2014;488:759-764.
75. Wang J, Wu Y, Zheng DJ. Analysis of dam safety monitoring data based on multi-sensor data fusion theory. Engineering Journal of
Wuhan University. 2004;37(1):32-35.
How to cite this article: Li B, Yang J, Hu D. Dam monitoring data analysis methods: A literature review. Struct
Control Health Monit. 2020;27:e2501. https://doi.org/10.1002/stc.2501

Structural Contr HLTH - 2019 - Li - Dam Monitoring Data Analysis Methods A Literature Review

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Structural Contr HLTH - 2019 - Li - Dam Monitoring Data Analysis Methods A Literature Review

Uploaded by

Copyright:

Available Formats

Received: 11 July 2019 Revised: 28 October 2019 Accepted: 7 December 2019

Dam monitoring data analysis methods: A literature review

Bin Li1,2 | Jie Yang1,2 | Dexiu Hu1,2

FIGURE 1 Jump values and spike values

FIGURE 2 The content structure diagram

2.1 | Statistical model

2.2 | ML-based model

FIGURE 3 Schematic diagram of the neural network structure

TABLE 2 Machine learning-based models and the corresponding research

Model Author Theory or method Modelling improvement

Discriminant Applicable sample

3.1 | Confidence interval method

FIGURE 4 Flow chart of the monitoring index of the confidence

Δyi fΔy1 , Δy2 , …, Δyn g,

where Δyi is the residuals value of ith moment, (i = 1, 2, … ,n).

where yi is the measured value of ith moment, (i = 1, 2, … ,n).

3.2 | Small-probability method

F I G U R E 5 Flow chart of a monitoring index based on the small-

4 | ABNORMAL VALU E DETEC T I O N

4.1 | Statistical probability method

yð1Þ −y 0 yðnÞ −y

The Dixon criterion need to calculate the statistic Qi as follows62:

4.2 | Data mining

4.3 | Multiscale decomposition

4.4 | Process line method

ACK NO WLE DGE MEN TS

You might also like