Z. Xiang, et al.

Reliability Engineering and System Safety 199 (2020) 106901

Table 2
Example 2, result comparison of the DRL-based sampling method with direct Monte Carlo simulation (MCS), subset simulation (SS), AK-MCS, and Latin hypercube
sampling (LHS) methods.
Methods Ncall TN FN TP FP Nmis Pf (C . O. V P^ ) ɛPf(%)
f

Direct MCS 5 × 105 - - - - - 2.316 × 10−3(2.9%) -

SS 4.4 × 104 - - - - - 2.46 × 10−3 6.11
DRL 2706 1158 1 498841 0 1 2.318 × 10−3 0.09
AK-MCS 519 591 0 498842 567 567 1.182 × 10−3 −48.96
LHS 2706 1147 195 498647 11 206 2.684 × 10−3 15.89

subset simulation methods, the results show that the DRL-based sam-
pling method can achieve higher prediction accuracy of failure prob-
ability when there is high nonlinearity of the limit state function.
In this paper, the problems of two random variables are used to
show the DRL-based sampling method. However, the potential applic-
ability of the DRL-based sampling method is not limited to 2-dimen-
sional problems. The dimension reduction techniques may be used to
transform the input state into 2-dimensional space. Subsequently, the
proposed method can be used to sampling and construct a surrogate
model for high-dimensional problems by increasing the dimensions of
the output layer (μ and σ) of the actor network. However, further study
is required to improve the reward function for high-dimensional pro-
blems.
CRediT authorship contribution statement
Zhengliang Xiang: Conceptualization, Methodology, Software,
Validation, Investigation, Writing - original draft. Yuequan Bao:
Conceptualization, Resources, Writing - review & editing, Project ad-
ministration, Funding acquisition. Zhiyi Tang: Conceptualization,
Methodology, Software. Hui Li: Conceptualization, Supervision,
Funding acquisition.
Declaration of Competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influ-
ence the work reported in this paper.
Acknowledgements
This research was supported by grants from the National Key R&D
Program of China (Grant 2017YFC1500603) and the National Natural
Science Foundation of China (Grant 51678203,U1711265,51638007
and 51978216) .
Supplementary materials
Supplementary material associated with this article can be found, in
the online version, at doi:10.1016/j.ress.2020.106901.
References
[1] Ang A.S., Tang W.Probability concepts in engineering planning and design[M].
1975.
[2] Melchers RE. Structural reliability analysis and prediction. 2nd ed. West Sussex:
England: John Wiley & Sons; 1999.
[3] O. Ditlevsen and H.O. Madsen. Structural reliability method[M]. 2007.
[4] Ang HS, Cornell CA. Reliability bases of structural safety and design[J]. J Struct Div
1974;100(9):1755–69.
[5] Hasofer AM, Lind NC. Exact and invariant second-moment code format[J]. J Eng
Mech Div 1974;100(1):111–21.
[6] Zhao YG, Ono T. A general procedure for first/second-order reliability method
(FORM/SORM)[J]. Struct Saf 1999;21(2):95–112.
[7] Zhao YG, Ono T. New approximations for SORM: part I[J]. J Eng Mech
1999;125(1):86–93.
[8] Rubinstein RY. Simulation and the Monte Carlo Method[M]// Simulation and the
Monte Carlo method. John Wiley & Sons; 2008. p. 167–8.
[9] Ang GL, Ang Alfredo H. H.edo H. H.on and the Monte Carlo Method[M]//
SimEstimator[J]. J Eng Mech 1992;118(6):1146–63.
[10] Au SK, Beck JL. Estimation of small failure probabilities in high dimensions by
subset simulation[J]. Probab Eng Mech 2001;16(4):263–77.
[11] Wang Z, Broccardo M, Song J. Hamiltonian Monte Carlo methods for subset si-
mulation in reliability analysis[J]. Struct Saf 2019;76:51–67.
[12] Ditlevsen† O. General multi-dimensional probability integration by directional si-
mulation[J]. Comput Struct 1990;36(2):355–68.
[13] Nie J, Ellingwood BR. Directional methods for structural reliability analysis[J].
Struct Saf 2000;22(3):233–49.
[14] Pradlwarter HJ, Schuëller GI, Koutsourelakis PS, et al. Application of line sampling
simulation method to reliability benchmark problems[J]. Struct Saf
2007;29(3):208–21.
[15] Yin C, Kareem A. Computation of failure probability via hierarchical clustering[J].
Struct Saf 2016;61:67–77.
[16] Shi X, Teixeira AP, Zhang J, et al. Structural reliability analysis based on prob-
abilistic response modelling using the Maximum Entropy method[J]. Eng Struct
2014;70:106–16.
[17] Chen JB, Li J. Development-process-of-nonlinearity-based reliability evaluation of
structures[J]. Probab Eng Mech 2007;22(3):267–75.
[18] Bucher CG. Bourgund. A fast and efficient response surface approach for structural
reliability problems[J]. Struct Saf 1990;7(1):57–66.
[19] Sobol IM. Theorems and examples on high dimensional model representation[J].
Reliab Eng Syst Saf 2003;79(2):187–93.
[20] Sachdeva SK, Nair PB, Keane AJ. Hybridization of stochastic reduced basis methods
with polynomial chaos expansions[J]. Probab Eng Mech 2006;21(2):182–92.
[21] Kaymaz I. Application of Kriging method to structural reliability problems[J].
Struct Saf 2005;27(2):133–51.
[22] Rocco CM, Moreno JA. Fast Monte Carlo reliability evaluation using support vector
machine[J]. Reliab Eng Syst Saf 2002;76(3):237–43.
[23] Chojaczyk AA, Teixeira AP, Neves LC, et al. Review and application of Artificial
Neural Networks models in reliability analysis of steel structures[J]. Struct Saf
2015;52(3):78–89.
[24] Elhewy AH, Mesbahi E, Pu Y. Reliability analysis of structure using Neural Network
method[J]. Probab Eng Mech 2006;21(1):44–53.
[25] Fang KT, Lin D KJ, Winker P, et al. Uniform design: theory and application[J].
Technometrics 2000;42(3):12.
[26] Liang XB. Orthogonal designs with maximal rates[J]. IEEE Trans Inf Theory
2003;49(10):2468–503.
[27] Helton JC, Davis FJ. Latin hypercube sampling and the propagation of uncertainty
in analyses of complex systems[J]. Reliab Eng Syst Saf 2003;81(1):23–69.
[28] Xiao NC, Zuo MJ, Zhou C. A new adaptive sequential sampling method to construct
surrogate models for efficient reliability analysis[J]. Reliab Eng Syst Saf 2017:S0.
[29] Fajraoui N, Marelli S, Sudret B. Sequential design of experiment for sparse poly-
nomial chaos expansions[J]. SIAM/ASA J Uncertain Quantif 2017;5(1):1061–85.
951832016305312.
[30] Li X, Gong C, Gu L, et al. A sequential surrogate method for reliability analysis
based on radial basis function[J]. Struct Saf 2018;73:42–53.
[31] Marelli S, Sudret B. An active-learning algorithm that combines sparse polynomial
chaos expansions and bootstrap for structural reliability analysis[J]. Struct Saf
2018;75:67–74.
[32] Zhou T, Peng Y, Li J. An efficient reliability method combining adaptive global
metamodel and probability density evolution method[J]. Mech Syst Sig Process
2019;131:592–616.
[33] Jones DR, Schonlau M, Welch WJ. Efficient global optimization of expensive black-
box functions[J]. J Global Optim 1998;13(4):455–92.
[34] Bichon BJ, Eldred MS, Swiler LP, et al. Efficient global reliability analysis for
nonlinear implicit performance functions[J]. AIAA J 2008;46(10):2459–68.
[35] Echard B, Gayton N, Lemaire M. AK-MCS: an active learning reliability method
combining Kriging and Monte Carlo simulation[J]. Struct Saf 2011;33(2):145–54.
[36] Zhang J, Xiao M, Gao L. An active learning reliability method combining Kriging
constructed with exploration and exploitation of failure region and subset simula-
tion[J]. Reliab Eng Syst Saf 2019;188:90–102.
[37] Yuan K, Xiao NC, Wang Z, et al. System reliability analysis by combining structure
function and active learning Kriging model[J]. Reliab Eng Syst Saf 2019:106734.
[38] Crombecq K, De Tommasi L, Gorissen D, et al. A novel sequential design strategy for
global surrogate modelling[C]//. IEEE 2009:731–42.
[39] Silver D, Huang A, Maddison CJ, et al. Mastering the game of Go with deep Neural
Networks and tree search[J]. Nature 2016;529(7587):484–9.
[40] Mnih V., Badia, Adrià Puigdomènech, Mirza M., et al. Asynchronous methods for
deep reinforcement learning[J]. 2016.