Reliability Engineering and System Safety 231 (2023) 109014

Contents lists available at ScienceDirect

Reliability Engineering and System Safety

journal homepage: www.elsevier.com/locate/ress

Reliability analysis with cross-entropy based adaptive Markov chain

importance sampling and control variates
Moien Barkhori Mehni a, *, Mohammad Barkhori Mehni b
a
Department of Civil Engineering, University of Jiroft, Jiroft, Iran
b
Department of Civil Engineering, University of Waterloo, Waterloo, Ontario, Canada

A R T I C L E I N F O A B S T R A C T

Keywords: In reliability analysis, high dimensional problems pose challenges to many existing sampling methods. Cross-
Importance sampling entropy based Gaussian mixture importance sampling has recently gained attention. However, it only per­
Markov chain forms well in problems with low to moderate dimensionality. Several efforts have been made to improve this
Kullback-leibler divergence
method. This paper, suggests a method of improving the performance of cross-entropy based Gaussian mixture
Control variates
Monte Carlo simulation
importance sampling, and compares its performance with the recent advancements. To enhance the efficiency for
Cross entropy high dimensional problems, the paper proposes to employ Markov Chain Monte Carlo (MCMC) sampling. In this
new approach, Markov chain samples gradually populate the failure domain in accordance with an optimal
density function. In this process, a seed generation scheme ensures that the Markov chain truly covers the whole
failure domain. Then, the parameters of the Gaussian mixture model are derived by modified closed-form for­
mulas. The incorporation of MCMC and modification of the parameter updating rule make the method more
robust against the dimensionality. Also, a control variates scheme further improves the performance. The per­
formance of the proposed approach is compared with recently developed importance sampling algorithms. The
results support the efficiency, robustness and accuracy of the proposed method.

1. Introduction
The main objective of reliability analysis is to compute the failure
probability of an engineering system by taking into account the uncer­
tainty of input variables. Suppose a random vector, X, contains all un­
certain variables involved in the problem. The state of system
performance is reflected in the limit state function, g(x), returning
negative values for unsatisfactory performance and positive values
otherwise. The failure probability Pf is then computed by a multidi­
mensional integral (1) as:
∫ state functions. Besides this, it is impossible to quantify the approxi­
Pf = I(x)P(x)dx (1)
Where P(x) is the joint probability density function of random variables,
and I(x) is the indicator function defined by:
I(x)= {
1 if g(x) ≤ 0
(2)
0 otherwise
In low-dimensional problems, various of numerical methods perform
well in computing the integral (1). Nevertheless, these methods often
become computationally inefficient in real engineering problems where
the number of random variables is relatively large. Besides this, in
practice, evaluating the limit state function g(x) demands running costly
numerical models. Consequently, a method yielding accurate enough
results with minimum limit state function evaluations is preferable [1].
In this regard, first-order (FORM) and second-order reliability methods
(SORM) approximate the limit state function with a relatively low
computational burden [2,3]. Despite their computational benefits, their
performance degrades with multimodal problems and nonlinear limit
mation inherently involved in these methods. On the contrary, sto­
chastic simulation methods like Monte Carlo simulation (MCS) [4–8]
and its variants like line sampling [9–12], directional simulation
[13–17], importance sampling [7,18–21] and variants of subset simu­
lation [6,22–24] etc. are asymptotically exact and more robust against
complex shapes of limit state function. Moreover, the quality of their
result can be easily assessed. Despite these benefits, MCS suffers from
inefficiency in rare event simulations dominant in practical engineering
problems. For problems with very small failure probabilities, the

* Corresponding author.
E-mail address: moien.barkhori@gmail.com (M.B. Mehni).

https://doi.org/10.1016/j.ress.2022.109014
Received 3 April 2022; Received in revised form 25 November 2022; Accepted 26 November 2022
Available online 28 November 2022
0951-8320/© 2022 Elsevier Ltd. All rights reserved.
M.B. Mehni and M.B. Mehni
number of random samples required for an acceptable accuracy pro­
hibitively increases, as the variance of MCS estimation is inversely
proportional to the failure probability. To reduce the variance of MCS
and the number of required samples, some advanced simulation
methods have been developed.
Importance sampling (IS) is a well-known variant of MCS. It im­
proves the performance of MCS by introducing an alternative sampling
density h(x). Therefore, Eq. (1) can be rewritten as (3)
∫ come from the algorithms which generate the training data and learning
I(x)P(x)
Pf = h(x)dx (3)
h(x)
Given that the support of h(x) involves the failure domain, the esti­
mation of IS can be obtained by Eq. (4) as:
∑N
̂f = 1
P
I(xi )P(xi )
(4)
N i=1 h(xi )
where the independent samples xi are drawn from h(x). The choice of
sampling function h(x) dramatically influences the variance of impor­
tance sampling estimation. It could be shown by variational calculus
that the optimal sampling function h∗ (x) would yield a zero-variance
estimate [25].
I(x)P(x)
h∗ (x) = ∫ (5)
I(x)P(x)dx
Obviously, we cannot use h∗ (x) directly because the denominator is
the sought-for failure probability. The more the sampling function re­
sembles h∗ (x), the better the estimation becomes [7]. Parametric and
non-parametric importance sampling and limit state approximation
methods are the main categories for generating sampling functions. In
the parametric adaptive importance sampling scheme, a parametric
finite mixture of a family of distributions is chosen, and the parameters
are optimized to minimize the discrepancy of distribution with h∗ (x). For
this, a distance measure is implemented. Usually, cross-entropy methods
are suitable choices [26,27]. The major obstacle to parametric impor­
tance sampling is the curse of dimensionality. By increasing the number
of uncertain variables, the number of free parameters rapidly increases.
This demands a much larger number of training samples to truly obtain
the optimal parameters. Some efforts exploit the specific characteristics
of high- dimensional random spaces. For instance, [5] proposes to use
Mises-Fisher mixture as the sampling density to generate samples on the
surface of a hypersphere. This reduces the number of mixture model
parameters to the order of O(d) compared to the Gaussian mixture
model with an order of O(d2), where d is the dimensionality. Alterna­
tively, the dimensionality reduction method can find the
low-dimensional structure of failure space and then the sampling func­
tion is developed in this projected low-dimensional subspace [20,
28–32]. On the other hand, in nonparametric importance sampling
methods, a kernel-based density function is defined as Eq. (6),
∫ Bayesian inference to find low-dimensional structure of high dimen­
1 ∑M
[( )/ ]
h(x) = d
Kq x − x(m) θ : Kq (x) ≥ 0, Kq (x)dx = 1 (6)
Mθ m=1
where Kq (x) is the non-negative kernel density, which is commonly a
Gaussian function. θ > 0 is a model parameter called bandwidth. The
samples x(m) are drawn from optimal density function h∗ (x). Finding
optimal density function amounts to finding the optimal bandwidth
parameter θ∗ . The non-parametric method also suffers from the curse of
dimensionality. One procedure is to find most influential random vari­
ables and develop an importance sampling function for those variables
[33]. Another alternative is implementing efficient sampling methods to
generate samples from the optimal density function h∗ (x). Markov chain
Monte Carlo is used in [34] to draw the training samples. Advanced
acceptance-rejection sampling methods could also be used for this
purpose as used in [35]. Advanced MCMC methods like Metropolis
adjusted Langevin diffusion algorithm [36,37] and Hamiltonian Monte
2
Reliability Engineering and System Safety 231 (2023) 109014
Carlo can accelerate the exploration of the failure domain [38,39]. In the
limit state approximation methods, the limit state function g(x) is
replaced by a substitute model ̂ g (x) with a low computational burden.
Then a sampling function is defined as Eq. (7).
/∫
h(x) = Îg(x)<0 (x)P(x) Îg(x)<0 (x)P(x)dx (7)
The main distinctions between limit state approximation methods
algorithms. There is a rich list of surrogate models like variants of
response surface methods [40], neural networks [41], support vector
machine regression [42], Gaussian processes [43,44] and polynomial
chaos expansion [45]. The limit state approximation methods also suffer
from the curse of dimensionality. The primary remedy for this problem
is the application of dimensional reduction techniques to reduce the
number of random variables and to decrease the computational cost
[46–48]. The interested reader can find a comprehensive review of
importance sampling and recent developments in [49]. The main
objective of this paper is to explore solutions to enhance the perfor­
mance of recently developed parametric cross-entropy-based impor­
tance sampling using the Gaussian mixture density function. The
method presented here aims to improve the performance of importance
sampling, especially in high dimensions.
The cross-entropy method finds the optimal parameters by mini­
mizing a discrepancy measure like Kullback-Leibler (KL) divergence.
Recently, Gaussian mixture density has gained attention for sampling in
multimodal problems [18,19,50,51]. Three methods, namely cross en­
tropy with Gaussian mixture (CE-GM), CE with single Gaussian (CE-SG)
and sequential importance sampling with Gaussian mixture (SIS-GM)
are compared in [18]. The study concludes that CE-SG outperforms
CE-GM and both perform poorly in high-dimensional problems; SIS-GM
is more robust in finding all failure domains and in dealing with high
dimensions but demands a relatively large number of samples to yield
accurate enough estimations. To enhance the efficiency for
high-dimensional problems, this paper presents a new method called
cross-entropy-based Markov chain importance sampling with Gaussian
mixture (CE-MCIS). This new method merges the Markov chain simu­
lation into CE-GM and modifies Gaussian mixture parameter updating
rules. The control variate method is also implemented into the method
to reduce the estimation variance. The performance of the CE-MCIS
method is compared with SIS-GM, CE-SG and CE-GM through several
system and component reliability problems. Especially, This comparison
is made with several state of the art high-dimensional importance
sampling contributions which have been recently put forward [5,20,32].
In [5], by leveraging the properties of high-dimensional standard
normal space, the authors use von-mises Fisher density mixture as the
sampling function on the unit hyper-sphere. By building a connection
between importance sampling and Bayesian inverse problems, the au­
thors in [20] implemented a dimension reduction technique from
sional rare event simulation problem to construct a low dimensional
sampling distribution. The algorithm proposed in [32] updates the
covariance matrix of Gaussian sampling function in sample mean vector
direction. The results show that the proposed method significantly
outperforms CE-SG, CE-MG and SIS-GM methods in robustness,
computational efficiency and accuracy. Compared with recently devel­
oped high dimensional importance sampling methods presented in [5,
20,32], the proposed method shows promising performance comparable
with these methods.
The rest of the paper is structured as follows. An outline of cross-
entropy-based importance sampling is provided in Section 2. Section 3
discusses the proposed CE-MCIS method in detail and presents its al­
gorithm. In section 4, the performance of proposed method is examined
via numerical examples regarding robustness, efficiency and accuracy,
and the results are compared with SIS-GM method. Finally, the
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

conclusions are drawn Section 5. 3. Cross entropy-based Markov chain importance sampling

2. Cross entropy-based importance sampling The basic idea of the proposed method will be outlined in the sequel.
The algorithm consists of three main steps. At the first step, which is
In Cross entropy-based importance sampling, the pivotal idea is to called pre-adaptation stage, some samples are produced in the failure
find the optimal parameters of a chosen family of density functions, domain in accordance with the optimal density function, h∗ (x). A pro­
which minimizes the estimation variation. The cross-entropy provides a cedure is developed to generate these samples and to cover the entire
suitable measure to tackle this problem. It is aimed that the sampling failure domain. At the next stage, which is called the adaptation step,
parameters are chosen such that the cross-entropy divergence of the more samples are produced in the failure domain using the samples
parametric sampling function and the optimal sampling density is generated in the previous stage and Gaussian mixture parameters are
minimized. Kullbak-Leibler divergence is a well-known cross-entropy adapted to the samples. By introducing Markov chain sampling and
measure. It reads as: modification of Eqs. (12)–(14) the degeneracy issue in high dimensional
∫ ( ∗ ) problems is resolved, and the algorithm becomes more stable and robust
h (x)
D(h∗ (x), h(x, v)) = h∗ (x)ln dx against dimensionality in comparison to SIS-GM, CE-GM and CE-SG.
h(x, v)
Finally, the estimation of failure probability is calculated in the evalu­
∫ ∫ ation stage. A control variates scheme is implemented in this step to
= h∗ (x)ln(h∗ (x))dx − h∗ (x)ln(h(x, v))dx (8) reduce the variance of estimation further. The proposed modifications to
the cross-entropy method involve the initialization, covariance matrix
where h(x, v) is the parametric density function with the parameter control, and remedy of the weight degeneration in high dimensional
vector, v. The optimization problem is expressed as: problems, which are explained afterwards.
∫
argminv D(h∗ (x), h(x, v)) = argmaxv h∗ (x)ln(h(x, v))dx (9) 3.1. Pre-adaptation stage

Differentiating both sides of (9) with respect to the parameter vector 3.1.1. Populating the failure domain
v yields: For the first step, the random space is transformed to standard
∫ ∫ normal space by methods like NATAF [53] or Rosenblatt [54] trans­
∇v h(x, v)
∇v h∗ (x)ln(h(x, v))dx = h∗ (x) dx = 0 (10) formations. To draw samples from the optimal sampling function h∗ (x),
h(x, v)
using Markov chain simulation, some seeds in the failure domain should
As suggested by the original paper [19], by introducing the Gaussian be generated. To do this, the subset simulation [6] procedure is imple­
mixture (GM) (11) as the parametric sampling density, the random mented. In this study, this process is called the pre-adaptation stage. At
vectors xi are drawn from h(x, u) in an iterative process. the first iteration, Np samples are drawn from the original distribution
which is standard normal here. At every iteration k of the pre-adaptation
∑
K
h(x, v) = πk N(x, μk , Σk ) (11) stage, Np samples are generated in the intermediate failure region
k=1 defined by rk = {x|g(x) < tk }. The threshold tk+1 is defined as the lower ρ
quantile of the limit state evaluation for the samples generated in iter­
In the Gaussian mixture (11), the parameters μk Σk and πk are,
ation k. So the thresholds tk produce a descending sequence. The samples
respectively, the mean vectors, the covariance matrices, and the weights
drawn in region rk+1 in iteration k are used as seeds of Markov chains to
of the normal distributions. These parameters are given in the following
populate region rk+1 at iteration k+1. The process goes on till tk+1 < 0 .
closed-form relations (12) to (14) for updating the sampling parameters
Finally, the samples falling in the failure domain would be selected as
of the density function [19].
seeds of Markov chains to draw samples from optimal density function.
∑N
I(xi )w(xi , u)γj (xi )xi The component-wise Markov chain [6] or conditional M-H sampling
μj = ∑N i=1 (12) discussed in [8] could be used in this process to generate the samples in
i=1 I(x i )f (xi )w(xi , u)γ j (xi )
each intermediate region. The latter method is utilized here for its
∑N ( )( )T promising performance in high dimensions. The number of samples in
i=1 I(xi )w(xi , u)γ j (xi ) xi − μ xi − μj
(13) each iteration, Np, and the ρ ratio are adopted in the tradeoff between
j
Σj = ∑N
i=1 I(x i )f (x i )w(xi , u)γ j i)
(x the computational cost and the accuracy in capturing all the failure
∑N modes.
I(xi )w(xi , u)γj (xi )
π j = ∑i=1
N (14)
i=1 I(xi )f (xi )w(xi , u) 3.2. Adaptation stage
In Eqs. (12) to (14), the ratios γ j (x) and w(x, u) are defined as follows:
( ) 3.2.1. Weight degeneration control
π j N x, μj , Σj As previously mentioned, the weight ratio w(x, u) defined by Eq. (16)
γ j (x) = ∑K (15)
k=1 π k N(x, μk , Σk )
becomes degenerate in the case of high dimensional random space. By
drawing samples from the optimal distribution, h∗ (x), the weights would
f (x) be equal to a constant.
w(x, u) = (16)
h(x, u) ∫ ∫
f (x) I(x)f (x)
w(x, u) = = I(x)f (x)dx (17)
Here, the random vectors xi are drawn from h(x, u) with parameters u I(x)f (x)
from the previous iteration. This could be problematic, especially in high Cancelling out the constant weight in the numerator and the de­
dimensional problems, where the ratio w(x, u) becomes degenerate [5, nominator of the Eqs. (12) to (14) would simplify them to the following
52]. Moreover, The algorithm presented in [19] suffers from sensitivity forms:
to the choice of the initial parameters. To remedy these deficiencies, it is ∑N
proposed to couple the Markov chain sampling procedure into the I(xi )γ j (xi )xi
μj = ∑N i=1 (18)
importance sampling scheme. i=1 I(xi )f (xi )γ j (xi )

3
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

∑N ( )( )T variables X and Y are as follows:

i=1 I(xi )γ j (xi ) xi − μ xi − μj
(19)
j
Σj = ∑N
i=1 I(xi )f (xi )γ j (xi ) I(x)f (x)
X= (24)
h(x)
∑N
I(xi )γj (xi )
π j = ∑i=1
N (20) J(x)f (x)
i=1 I(xi )f (xi ) Y= (25)
h(x)
Here, the degeneracy issue is eliminated by Markov chain, which draws
samples following the optimal density, however at the cost of correlation where J(x) is the indicator function, which returns 1 for a negative
between the sampled vectors. The important note in the application of realization of the linear function tangent to the limit state, L(x) ≤ 0, and
Eqs. (18) to (20) is that the older parameters μj , Σj and πj from the 0 otherwise. Accordingly, the new estimator of the failure probability
previous iteration are used to obtain the new ones in each iteration. would be:
∑N
(I(xi ) − αJ(xi ))f (xi )
3.2.2. Initiation and covariance matrix control ̂f = 1
P + αΦ(− β) (26)
N i=1 h(xi )
At the adaptation stage, the sampling parameters are calculated by
Eqs. (18) to (20). The number of Gaussians has a substantial role in the
where Φ(.) is the cumulative standard normal function, and β is the
performance of the method. With fewer Gaussians, some important
design point distance to the origin of standard normal space. The design
failure modes may be lost; in contrast, high number of Gaussians in­
point can be found by any constrained optimization technique like
creases the number of samples to get a satisfactory accuracy. There are
Hasofer-Lind method. The samples xi are drawn from h(x) which is the
various methods to obtain the optimal number of densities, one of which
adapted Gaussian mixture density function. The optimal α factor is
is k-means algorithm [55]. This method can find the clusters and the
estimated as:
centroids of samples generated in the last iteration of the pre-adaptation
∑N ∑N
phase. To check the quality of clustering, the silhouette value can be I(xi )J(xi )f 2 (xi )
− Φ(− β) I(xi )f (xi )

(27)
i=1 h2 (xi ) i=1 h(xi )
used. The weights are initialized by πj = 1/K, where K is the number of α= ∑N (J(xi )f (xi ) )2
Gaussians. The initial covariance matrices are set to unitary matrix Σj = i=1 h(xi )
− Φ(− β)
I. For the initial location vectors μj , K centroids of the initial samples in
For more clarification, the outline of the proposed algorithm is pre­
the failure domain are captured by the k-means algorithm. This pro­
sented as follows.
cedure makes the algorithm robust and ensures capturing all the failure
modes. It should be noted that by increasing the number of dimensions
3.4. Cross entropy-based Markov chain importance sampling algorithm
d, the number of parameters to be evaluated rapidly increases. This
means a lot more samples need to be drawn to yield accurate enough
Before starting the algorithm, the random space should be trans­
sampling parameters. Furthermore, the correlation of samples drawn
formed to the standard normal space by an appropriate transformation.
from MCMC worsens the situation. Here, in the case of high dimensional
The steps of CE-MCIS algorithm are summarized as follows.
problems, where d>15, it is proposed to fix the covariance matrices as
the unitary matrix I. This modification is supported numerically and
1- Pre-adaptation phase: At this stage, some sample seeds are generated
theoretically, as the optimal covariance matrix approaches to the uni­
in the failure region by the procedure explained in Section 3.1.1.
tary matrix in the standard normal space [52].
2- Initialization: As articulated in Section 3.2.2, the initial values are
At every iteration of the adaptation the phase, Na samples are
assigned to the sampling parameters.
generated from the optimal density function by MCMC. At the first
3- Adaptation: At this phase, the Gaussian mixture (GM) is adapted to
iteration, samples are generated in the failure domain by the Ns seeds
the samples generated in the failure domain, obeying the optimal
from the last iteration of the pre-adaptation stage. The k-means algo­
density function. In each iteration of this phase, Na samples are
rithm could be employed to generate these initial seeds. Afterwards, the
drawn from the optimal density function by Markov chain simula­
chains for each iteration would be the continuation of the chains from
tion. The final samples of each chain would be the seeds for the next
the previous iteration.
iteration. To generate Markov samples, the conditional metropolis-
Hastings algorithm presented in [8] is utilized for its acceptable
3.3. Evaluation stage behavior in high dimensions. The sampling parameters are updated
in each iteration by Eqs. (18) to (20) . The iterations repeat till the
3.3.1. Control variates implementation stopping criterion is met. The coefficient of variation (COV) for the
In this section, the variance reduction method of control variates is failure probability estimation for a prefixed number of samples (Nf )
introduced and implemented into the importance sampling algorithm. is proposed as the adequacy criterion. The COV estimate (̂ δ) is
Suppose the objective is to evaluate the mean of a random variable X. calculated by small number of m samples generated from the adapted
Another random variable Y with known mean μY and correlated with X Gaussian mixture as:
is introduced. With this basis, a new random variable Z is defined as √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
√ ⎡ ⎤
√
Z = X − αY + αμY (21) √ ⎢ 1 ∑m I(xi )f (xi )2
√ ⎢ ⎥
√1 i=1 h(xi )2 ⎥
(28)
m
̂
δ=√ ⎢ ⎥
Obviously, μX = μZ , but the variance of Z would be √Nf ⎢ ( ∑ )2
− 1 ⎥
√ ⎣ 1 m
m
I(x i )f
i=1
(x i )
h(xi )
⎦
σ = σ 2X + α2 σ2Y − 2ασ XY
2
Z (22)

By inserting the optimal coefficient α = σσXY2 into (22), the minimal

Y

variance of Z would be

(σXY )2 For stopping the algorithm, it suffices ̂

δ ≤ δtarget .
σ 2Z = σ 2X − (23)
σ2Y
According to this equation, a reduction in variance can be achieved
with an increase in the correlation of X and Y. In our case, the correlated

4
M.B. Mehni and M.B. Mehni
1- Failure probability estimation: After drawing N samples from the
final adapted Gaussian mixture, the failure probability is estimated
by Eq. (26).
The flow chart of MCIS algorithm is also depicted in Fig. 1 in which
colors differentiate the phases of the algorithm.
4. Numerical examples
Through several numerical examples, the performance of proposed
CE-MCIS method is investigated in comparison with the sequential
importance sampling (SIS-GM) method, cross entropy with Gaussian
mixture (CE-GM) and cross entropy with single Gaussian (CE-SG) arti­
culated in [18] and high dimensional methods presented in [5,20,32].
As the performance measure, the total number of limit state function
calls is chosen for being the main computational burden in structural
reliability analyses. Here, eight examples have been considered, all of
which occur in the standard normal space. The first example is a
two-dimensional concave parabolic limit state function with two distinct
modes. The second example consists of a series system with two stan­
dard normal random variables, challenging the methods with four fail­
ure domains. The effect of dimensionality on the performance of
methods is explored in the third example with a linear limit state
function and a fixed failure probability. The forth example is the
Fig. 1. the flowchart of MCIS algorithm.
5
Reliability Engineering and System Safety 231 (2023) 109014
combination of a linear and a convex limit state function on the opposite
sides of the origin. The results reported for SIS-GM, CE-GM and CE-SG
are extracted from the original paper [18]. The fifth to eighth exam­
ples include high-dimensional non-linear examples through which the
proposed method is compared with the high-dimensional methods pre­
sented in [5,20,32].
4.1. Parabolic concave limit state function
The first example involves a parabolic limit state with two failure
modes presented by Eq. (29). The limit state function and optimal
sampling distribution contour lines are displayed in Fig. 2. The random
space consists of two independent standard normal random variables.
The reference failure probability obtained by crude Monte Carlo with 5
× 106 samples is 3.00 × 10− 3.
g(x) = 5 − x2 − 0.5(x1 − 0.1)2 (29)
For the pre-adaptation phase of CE-MCIS method, Np =1000 samples
are generated in each level, and ρ = 0.05 is chosen as the quantile ratio.
This means there is 50 chains with the length of 20 samples. From the
last iteration of pre-adaptation stage, 50 seeds are extracted by k-means
algorithm for the chains of adaptation stage. Each chain would generate
20 samples, which means there would be Na=1000 samples in each
iteration of the adaptation stage; the final estimation is derived by N =
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Fig. 2. The parabolic concave limit state function and optimal sampling density contours.

1000 samples drawn from adapted Gaussian mixture. For each level of pre-adaptation stage and Na=1000 samples per iteration in the adap­
SIS-GM, 4000 samples are generated. By generating N = 4000 samples tation stage. For the final results, N = 1000 samples are generated in the
per level, both CE-GM and CE-SG methods become stable and unbiased. estimation step. In the case of SIS-GM, CE-GM and CE-SG methods, 8000
The average results obtained from fifty independent runs of four samples per level are generated to get unbiased results. Both CE-GM and
methods are summarized in Table 1. CE-SG methods need three steps to converge, while SIS-GM method
All methods could reach unbiased results and detect failure regions. converges in four steps. The averaged results of fifty independent runs
Regarding the COV of estimations, CE-MCIS yields more accurate results are presented in Table 2. The reference value for failure probability is
with a much smaller number of calls on the limit state function. CE-GM 2.24 × 10− 3.
and CE-SG methods need about three steps, while SIS-GM needs one This problem with four failure regions is challenging for all methods
more step, which yields to more computational effort. to cover all failure region and yield unbiased result. Also, Fig. 4 shows
that CE-MCIS is able to detect the failure modes of the problem and
4.2. Series system with four modes populate the important regions of the failure space. The results in
Table 2 show that CE-MCIS achieves the same level of accuracy of the
This example consists of the union of four failure regions defined by other methods with much fewer samples. The results also show that all
four limit state functions as Eq. (30) and shown in Fig. 3. The two methods can detect all failure regions and yield unbiased results.
random variables involved are standard normal and independent.
x1 + x2 4.3. The linear limit state function with varying number of variables
0.1(x1 − x2 )2 − √̅̅̅ + 3
2
In this part, the performance of both methods is investigated in
x1 + x2
2
0.1(x1 − x2 ) + √̅̅̅ + 3 dealing with varying dimensions. The linear limit state function is
2
g(x)= min{ (30) expressed in Eq. (31). The exact failure probability is Φ( − β), irre­
7
x1 − x2 + √̅̅̅ spective of the number of random variables [56]. By choosing the reli­
2 ability index β = 3.5, the failure probability would be 2.33 × 10− 4.
7
x2 − x1 + √̅̅̅ 1 ∑ n
2 g(x) = β − √̅̅̅ xi (31)
n i=1
The CE-MCIS is performed with Np =1000 samples per level in the
To have comparable results for different number of random vari­
ables, in the case of CE-MCIS method, the number of samples used in
Table 1
The results for the concave limit state function. each level of pre-adaptation phase and each iteration of the adaptation
phase are fixed. The pre-adaptation phase is performed with Np = 1000
E( P δ̂ Ntot
samples per level, while in the adaptation phase, Na=2000 samples are
̂f ) Pf

CE-MCIS 3.01 × 10− 3

0.029 5.2 × 103 generated per iteration. For the target cov (δtarget ) a value of 0.10 is
SIS-GM 3.00 × 10− 3
0.05 16 × 103 chosen for Nf = 4000. Furthermore, it must be noted that for greater
3
CE-GM 3.02 × 10− 0.04 12.23 × 103 than 10 number of random variables, the covariance matrix is fixed to
3
CE-SG 3.01 × 10− 0.05 12.00 × 103
the unitary matrix. The control variates part of CE-MCIS is also

6
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Fig. 3. Contour lines of the optimal density function and the limit state function for the second example.

Table 2 x1 + x2
0.1(x1 − x2 )2 − √̅̅̅ + 2.5
Comparison of methods for series system problem. 2
g(x)= min{ (32)
E( P
̂f ) δ̂ Ntot x1 + x2
Pf 3.2 + √̅̅̅
3
2
CE-MCIS 2.24 × 10− 0.04 5.6 × 103
SIS-GM 2.20 × 10− 3
0.05 32 × 103 The limit state function and the contour lines of the optimal density
3
CE-GM 2.21 × 10− 0.06 24 × 103 function are illustrated in Fig. 7. About 85% of the probability mass is
3
CE-SG 2.22 × 10− 0.06 24 × 103
located in the convex region, and the reference value of failure proba­
bility is 4.90 × 10− 4. The CE-MCIS is performed with Np=1000 sample
neglected to get comparable results with other methods. The other three per each level of the pre-adaptation stage and Na =1000 samples per
methods are performed with ns=8000 samples per level. The CE-SG, CE- each iteration of the adaptation stage. The final estimation is obtained
GM and SIS-GM methods converge in 5.67, 5.58 and 5 steps, respec­ by N = 1000 samples generated from the adapted Gaussian mixture
tively. To obtain the reported results, CE-MCIS needs 104 total limit state density. The value of 5% is chosen as the target COV of the algorithm. On
function calls on average, which is one-fourth of what SIS-GM demands. the other hand, SIS-GM, CE-GM and CE-SG algorithms are performed
The bias and COV increase with increasing the dimensionality. Still for with ns=8000 samples per level. The averaged results of fifty runs of all
CE-SG and CE-GM this problem is such dramatic that the results are no methods are compared in Table 3.
longer reliable for dimensionality larger than n = 20. As Table 3 shows, all methods deliver unbiased results; however,
As it is clear from Fig. 5, the COV of estimation increases for all with the same level of accuracy, CE-MCIS demands much less compu­
methods with the increase in dimensionality. The COV mildly increases tational burden than the others. The SIS-GM method estimates the result
for both MCIS and SIS-GM, while it dramatically increases for CE-GM by multiplying the probability estimations calculated at each level.
and CE-SG. It is also clear from Fig. 6 that with the increase of the Consequently, the error of each level is propagated to the final result.
number of random variables, the results of SIS-GM become negatively Moreover, its probability estimations are obtained by the correlated
biased compared to those of CE-MCIS, which are nearly unbiased. The samples generated by Markov chain. Meanwhile, CE-MCIS utilizes the
bias for both CE-GM and CE-SG is so significant that they become un­ Markov samples to adapt the Gaussian mixture to the optimal density
reliable for dimensionality larger than 20. function, and its outcome is gained by iid samples drawn from the
adapted mixture.
4.4. Union of convex and linear limit state functions All methods successfully could detect the two failure regions, as
shown in Fig. 8 for CE-MCIS method.
This example evaluates the performance of methods in dealing with a
series system limit state function, consisting of convex and linear func­ 4.5. Large portfolio losses
tions in two-dimensional standard normal space. The problem has two
distinct failure regions on the opposite sides of the origin. The analytical This example is presented in [32]. The limit state function is defined
limit state function is presented in Eq. (32) as: in Eq. (33) as:

7
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Fig. 4. The samples generated in the evaluation phase of CE-MCIS for series system problem.

Fig. 5. Coefficient of variation of failure probability estimation vs dimensionality.

∑
n All the random variables are independent standard normal. Here
g(x) = bn − I{ψ (U,μ,ηj )≥0.5√̅̅n } x1 = U, x2 = μ, x2+j = ηj , and the function FΓ (.) is cumulative distri­
j=1
bution for Γ(6, 6). The proposed method is compared with importance
( ) ( / ( )1/2 )[ − 1 ]− sampling algorithm presented in [32] for n ϵ {30, 100, 250}. The pa­
(33)
1/2
ψ U, μ, ηj = U 4 + 3 1 − q2 FΓ (Φ(μ))
rameters of CE-MCIS are presented in Table 5. The final results of
CE-MCIS are obtained by averaging over thirty independent runs of the

8
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Fig. 6. Averaged estimation of failure probability vs dimensionality.

Fig. 7. Contour lines of the optimal density function and the limit state function for the fourth example.

algorithm. The results are also compared in Table 4. but at higher dimensions CE-MCIS outperforms the two.
It is clear from Table 5 that the accuracy of proposed CE-MCIS
method is comparable to iCE-m* and CE-m*. CE-MCIS demands more
computational cost than iCE-m* and CE-m* to reach the same accuracy,

9
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Table 3 hyper-planes in the standard normal space. The limit state function is
Comparison of methods for the forth example. defined in Eq. (35) as:
E( P
̂f ) δ̂
Pf
Ntot g(x) = min(g1 (x), g2 (x))
− 3 3
CE-MCIS 4.90 × 10 0.037 5 × 10
3
SIS-GM 4.87 × 10− 0.030 32 × 103
3
CE-GM 4.80 × 10− 0.050 32 × 103
3
CE-SG 4.89 × 10− 0.040 24 × 103

Table 4
4.6. Modified Ackley function Final results for large portfolio loss example.
CE-MCIS iCE-m* CE-m*
This example is also adapted from [32]. It involves a highly nonlinear 3 3 3
n = 30 E( ̂
Pf ) 4.18 × 10− 4.29 × 10− 4.24 × 10−
high-dimensional limit state function presented in Eq. (34). Pf=4.29 × 10− 3
δ̂ 0.06 0.10 0.07
Pf
⎛ √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ ⎞
√∑ Relative bias -2.5% -0.06% -1.3%
√ n ( )2
g(x) = cn − 20exp⎝ − 0.2√ aj xj − 3 ⎠ Ntot 10030 7900 8100
3 3 3
j=1 n = 100 E( P
̂f ) 1.79 × 10− 1.90 × 10− 1.8 × 10−
3
( ) Pf=1.8 × 10− δ̂ 0.081 0.085 0.084
Pf
1∑ n
( ( ))
− exp cos 2π aj xj − 3 (34) Relative bias -0.55% 3% 1.3%
n j=1 Ntot 10400 8000 8100
5 5 5
n = 250 E( P
̂f ) 1.05 × 10− 1.01 × 10− 0.94 × 10−
5
2(j− 1) Pf=1.0 × 10− 0.11 1.10 0.60
where aj = n is a fixed vector. The constant cn is chosen according to δ̂
Pf

dimension number n ϵ {30, 100, 200} as {12.98, 11.968, 11.5565} to Relative bias 5% 1% -5.9%
Ntot 17700 8000 8100
adjust the probability. The proposed method is run thirty times inde­
pendently, and its results are compared with algorithms proposed by
[32] in Table 6. The parameters of CE-MCIS are also presented in
Table 7. Table 5
The results indicate that CE-MCIS can reach as accurate estimations Algorithm parameters for portfolio loss example.
as iCE-m* and CE-m*. Concerning computational efficiency, CE-MCIS Preadaptation adaptation estimation
outperforms CE-m*. iCE-m* shows more efficiency, but the results are n ρ Samples Seed Samples Target Samples for
per level number per CoV estimation
comparable to the proposed method.
(Np) (Ns) iteration (δtarget ) (Nf)
(Na)
4.7. Parallel hyperplanes 30 0.05 1000 30 3000 0.08 4000
100 0.05 1000 30 3000 0.08 4000
This problem is presented in [5]. The series limit state is defined as 250 0.05 1000 50 5000 0.10 4000
the minimum of two linear functions which define two parallel

Fig. 8. Samples generated in the estimation stage of CE-MCIS for the fourth example.

10
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Table 6 Table 9
Final results for Modified Ackley function example. Algorithm parameters for parallel hyperplanes example.
CE-MCIS iCE-m* CE-m* Preadaptation Adaptation Estimation
− 3 − 3 − 3 n ρ Samples Seed Samples Target Samples for
n = 30 E( P
̂f ) 1.63 × 10 1.62 × 10 1.62 × 10
3 per level number per CoV estimation
Pf=1.64 × 10− 0.063 0.076 0.13
δ̂
Pf (Np) (Ns) iteration (δtarget ) (Nf)
Relative bias -0.61% -1.1% -1.2% (Na)
Ntot 10600 8000 8000 100 0.05 1000 50 5000 0.05 8000
3 3 3
n = 100 E( P
̂f ) 1.20 × 10− 1. 17 × 10− 1.22 × 10− 300 0.1 1000 100 10000 0.06 8000
3
Pf=1.18 × 10− δ̂ 0.077 0.13 0.294 600 0.1 1500 150 15000 0.06 20000
Pf
Relative bias 1.7% -1.2% 3.4%
Ntot 11400 8100 8000
3 3 3
n = 200 E( P
̂f ) 1.70 × 10− 1.69 × 10− 1.72 × 10−
Pf=1.72 × 10− 3 κ 1 ∑ n
δ̂
Pf
0.11 0.116 0.269 g(x) = β + (x1 − x2 )2 − √̅̅̅ xi (36)
4 n i=1
Relative bias -1.16% -1.5% -0.05%
Ntot 14200 8000 8100
where β = 4 and constant κ ϵ {5, 10} determines the curvature, which
corresponds to failure probabilities {6.62 × 10− 6 , 4.73 × 10− 6 },
respectively. The number of random variables n is chosen from n ϵ {2,
Table 7
334, 1000}. The averaged results from twenty independent runs of CE-
Algorithm parameters for Modified Ackley function example.
MCIS are compared with those of the algorithm proposed by (iCEred)
Preadaptation Adaptation Estimation
[20] in Table 10. It is worth noting that [20] proposes a method which
n ρ Samples Seed Samples Target Samples for
per level number per CoV estimation
amounts to the computation of limit state function gradient. This by
(Np) (Ns) iteration (δtarget ) (Nf) itself can be very time demanding, since the gradient is a vector. The
(Na) authors also pointed out that their method is applicable where there is
30 0.05 1000 30 3000 0.08 4000 an efficient way of gradient calculation. Therefore, a direct comparison
100 0.05 1000 30 3000 0.09 4000 with CE-MCIS is impossible. To have a fair comparison, limit state calls
200 0.05 1000 50 5000 0.10 4000 plus gradient calls times dimensionality could be taken as the compu­
tational cost of iCEred. The number of LSF calls plus gradient calls is
reported for iCEred. The probability estimations are not reported in the
√̅̅̅ ∑
n
√̅̅̅ ∑
n
g1 (x) = β1 n − xi , g2 (x) = β2 n + xi (35) reference [20] so they are not also reported here. The parameters of
i=1 i=1 CE-MCIS are also presented in Table 11.
The theoretic failure probability is Φ( − β1 ) + Φ( − β2 ). Here β1 = β2 As expected, the iCEred method shows no sensitivity to dimension­
= 3.5 . The average results drawn from twenty independent runs of CE- ality, since this method is built on capturing the low dimensional
MCIS are compared with the proposed algorithms by [5] in Table 8. The structure of failure space. The main drawback of this method is its
parameters of CE-MCIS are also presented in Table 9. dependence on the limit state function gradient, which is very compu­
As it is vivid from Table 8, CE-MCIS can yield estimations with the tationally demanding. The gradient evaluation computational demand
same level of accuracy as CE-AIS-vMFM2 with significantly less grows proportional to the dimensionality. If we take the limit state
computational cost. function calls and gradient function calls times dimensionality as the
computational expense of iCEred, it is clear that CE-MCIS outperforms it
4.8. Quadratic high dimensional limit state function in high-dimensional problems.

This limit state is adopted from [20]. It is expressed in the standard 5. Conclusion
normal space as reads in Eq. (36),
This paper proposes a method called CE-MCIS, which mingles two

Table 8 Table 10
Final results for parallel hyperplanes example. Final results for quadratic limit state function.
CE-MCIS CE-AIS- CE-AIS- CE-MCIS CE-MCIS iCEred iCEred
vMFM1 vMFM2 κ =5 κ = 10 κ =5 κ = 10
4 4
n = 100 E( P
̂f ) 4.59 × 3.45 × 10− 4.56 × 10− n=2 E( P
̂f ) 6.60 × 4.71 × - -
Pf=4.65 × 10− 4 10− 6 10− 6
10− 4 δ̂ 0.06 0.05 0.05 δ̂ 0.045 0.025 0.06 0.06
Pf Pf
Relative -1.29% -25.8% -1.93% Relative -0.30% -0.42% - -
bias bias
Ntot 1.73 × 104 2.03 × 104 2.22 × 104 Ntot 8.10 × 103 8.10 × 103 518+2720 530+2560
n = 300 E( P
̂f ) 4.61 × 3.87 × 10− 4 4.55 × 10− 4 n = 334 E( P
̂f ) 6.51 × 4.65 × - -
Pf=4.65 × 10− 4 10− 6 10− 6
10− 4 δ̂ 0.06 0.05 0.05 δ̂ 0.061 0.089 0.04 0.05
Pf Pf
Relative -0.86% -16.7% -2.2% Relative -1.66% -1.69% - -
bias bias
4 4 4 4 4
Ntot 4.10 × 10 4.88 × 10 4.97 × 10 Ntot 4.52 × 10 4.53 × 10 507+2120 526+5370
4 4
n = 600 E( P
̂f ) 4.55 × 4.52 × 10− 4.58 × 10− n= E( P
̂f ) 6.75 × 4.25 × - -
Pf=4.65 × 10− 3 1000 10− 6 10− 6
10− 4 δ̂ 0.06 0.05 0.05 δ̂ 0.14 0.13 0.05 0.05
Pf Pf
Relative -2.15% -2.79% -1.50% Relative 1.96% -10% - -
bias bias
Ntot 5.60 × 104 8.72 × 104 8.83 × 104 Ntot 8.54 × 104 8.64 × 104 510+2470 540+2370

11
M.B. Mehni and M.B. Mehni Reliability Engineering and System Safety 231 (2023) 109014

Table 11 References
Algorithm parameters for the quadratic limit state function.
[1] Lemaire M. Structural reliability. John Wiley & Sons; 2013.
Preadaptation Adaptation Estimation
[2] Rackwitz R. Reliability analysis—a review and some perspectives. Struct Saf 2001;
n ρ Samples Seed Samples Target Samples for 23:365–95. https://doi.org/10.1016/S0167-4730(02)00009-7.
per level number per CoV estimation [3] Hohenbichler M, Gollwitzer S, Kruse W, Rackwitz R. New light on first-and second-
(Np) (Ns) iteration (δtarget ) (Nf) order reliability methods. Struct Saf 1987;4:267–84. https://doi.org/10.1016/
(Na) 0167-4730(87)90002-6.
[4] Rubinstein RY, Kroese DP. Simulation and the Monte Carlo method. John Wiley &
2 0.05 1000 20 2000 0.05 8000
Sons; 2016.
334 0.1 1000 100 10000 0.06 20000 [5] Wang Z, Song J. Cross-entropy-based adaptive importance sampling using von
1000 0.1 1500 150 15000 0.09 20000 Mises-Fisher mixture for high dimensional reliability analysis. Struct Saf 2016;59:
42–52. https://doi.org/10.1016/j.strusafe.2015.11.002.
[6] Au SK, Beck JL. Estimation of small failure probabilities in high dimensions by
well-known simulation methods of importance sampling and Markov subset simulation. Probab Eng Mech 2001;16:263–77. https://doi.org/10.1016/
chain simulation via cross-entropy concept. A procedure is proposed to S0266-8920(01)00019-4.
[7] Papaioannou I, Geyer S, Straub D. Improved cross entropy-based importance
generate samples in the failure domain complying with the optimal sampling with a flexible mixture model. Reliab Eng Syst Saf 2019;191:106564.
density function, and then Gaussian mixture parameters are updated by https://doi.org/10.1016/j.ress.2019.106564.
modified closed-form equations. Finally, the failure probability estima­ [8] Papaioannou I, Papadimitriou C, Straub D. Sequential importance sampling for
structural reliability analysis. Struct Saf 2016;62:66–75. https://doi.org/10.1016/
tion is obtained by drawing samples from adapted Gaussian mixture j.strusafe.2016.06.002.
distribution. Also, a control variates scheme further reduces the variance [9] Papaioannou I, Straub D. Combination line sampling for structural reliability
of the results. The proposed CE-MCIS method is compared with a analysis. Struct Saf 2021;88:102025. https://doi.org/10.1016/j.
strusafe.2020.102025.
promising sequential importance sampling method using Gaussian [10] Shayanfar MA, Barkhordari MA, Barkhori M, Rakhshanimehr M. An adaptive line
mixture (SIS-GM), which also uses Markov chain simulation and two sampling method for reliability analysis. Iran J Sci Technol Trans Civ Eng 2017;41:
cross entropy methods called CE-GM and CE-SG methods. The examples 275–82. https://doi.org/10.1007/s40996-017-0070-3.
[11] Song J, Valdebenito M, Wei P, Beer M, Lu Z. Non-intrusive imprecise stochastic
include both component and system reliability analysis and high- simulation by line sampling. Struct Saf 2020;84:101936. https://doi.org/10.1016/
dimensional problems. The findings show that CE-MCIS successfully j.strusafe.2020.101936.
detects all failure modes of multimodal problems and is robust against [12] Valdebenito MA, de Angelis M, Patelli E. Line sampling simulation: Recent
advancements and applications. Reliability-based analysis and design of structures
high dimensionality. The proposed method also proves to be far more
and infrastructure. CRC Press; 2021. p. 215–26.
computationally efficient than other methods in estimating the proba­ [13] Shayanfar MA, Barkhordari MA, Barkhori M, Barkhori M. An adaptive directional
bility of failure with the same level of accuracy. Regarding the biased­ importance sampling method for structural reliability analysis. Struct Saf 2018;70:
ness, CE-MCIS shows negligible bias compared to other methods 14–20. https://doi.org/10.1016/j.strusafe.2017.07.006.
[14] Guo Q, Liu Y, Chen B, Zhao Y. An active learning Kriging model combined with
method, especially in high-dimensional problems. To further investigate directional importance sampling method for efficient reliability analysis. Probab
the performance in high dimensions, four nonlinear high dimensional Eng Mech 2020;60:103054. https://doi.org/10.1016/j.
limit states are selected from recent references proposing importance probengmech.2020.103054.
[15] Ditlevsen O, Melchers RE, Gluver H. General multi-dimensional probability
sampling algorithms for high-dimensional problems. As expected, the integration by directional simulation. Comput Struct 1990;36:355–68. https://doi.
computational cost significantly increases by dimensionality. The pro­ org/10.1016/0045-7949(90)90134-N.
posed CE-MCIS could reach accurate estimations by computational costs [16] Nie J, Ellingwood BR. A new directional simulation method for system reliability.
Part II: application of neural networks. Probab Eng Mech 2004;19:437–47. https://
comparable with or lower than the recent high dimensional methods. doi.org/10.1016/j.probengmech.2004.03.005.
[17] Zhang X, Lu Z, Cheng K. Cross-entropy-based directional importance sampling with
Author statement von Mises-Fisher mixture model for reliability analysis. Reliab Eng Syst Saf 2022;
220:108306. https://doi.org/10.1016/j.ress.2021.108306.
[18] Geyer S, Papaioannou I, Straub D. Cross entropy-based importance sampling using
All persons who meet authorship criteria are listed as authors, and all Gaussian densities revisited. Struct Saf 2019;76:15–27. https://doi.org/10.1016/j.
authors certify that they have participated sufficiently in the work to strusafe.2018.07.001.
[19] Kurtz N, Song J. Cross-entropy-based adaptive importance sampling using
take public responsibility for the content, including participation in the
Gaussian mixture. Struct Saf 2013;42:35–44. https://doi.org/10.1016/j.
concept, design, analysis, writing, or revision of the manuscript. strusafe.2013.01.006.
Furthermore, each author certifies that this material or similar material [20] Uribe F, Papaioannou I, Marzouk YM, Straub D. Cross-entropy-based importance
has not been and will not be submitted to or published in any other sampling with failure-informed dimension reduction for rare event simulation.
SIAM ASA J Uncertain Quantif 2021;9:818–47. https://doi.org/10.1137/
publication. 20M1344585.
[21] Jia G, Gardoni P. Simulation-based approach for estimation of stochastic
CRediT authorship contribution statement performances of deteriorating engineering systems. Probab Eng Mech 2018;52:
28–39. https://doi.org/10.1016/j.probengmech.2018.03.001.
[22] Jia G, Tabandeh A, Gardoni P. A density extrapolation approach to estimate failure
Moien Barkhori Mehni: Writing – original draft, Visualization, probabilities. Struct Saf 2021;93:102128. https://doi.org/10.1016/j.
Validation, Software, Project administration, Methodology, Investiga­ strusafe.2021.102128.
[23] Giovanis D, Shields M. Imprecise subset simulation. Probab Eng Mech 2022;69.
tion, Data curation, Conceptualization. Mohammad Barkhori Mehni: https://doi.org/10.1016/j.probengmech.2022.103293.
Writing – review & editing, Investigation. [24] Chan J, Papaioannou I, Straub D. An adaptive subset simulation algorithm for
system reliability analysis with discontinuous limit states. Reliab Eng Syst Saf
2022:108607. https://doi.org/10.1016/j.ress.2022.108607.
Declaration of Competing Interest [25] Biondini G. An introduction to rare event simulation and importance sampling.
Handbook of statistics. Elsevier; 2015. p. 29–68.
The authors declare that they have no known competing financial [26] De Boer PT, Kroese DP, Mannor S, Rubinstein RY. A tutorial on the cross-entropy
method. Ann Oper Res 2005;134:19–67. https://doi.org/10.1007/s10479-005-
interests or personal relationships that could have appeared to influence
5724-z.
the work reported in this paper. [27] Cao QD, Choe Y. Cross-entropy based importance sampling for stochastic
simulation models. Reliab Eng Syst Saf 2019;191:106526. https://doi.org/
Data availability 10.1016/j.ress.2019.106526.
[28] Bach FR, Jordan MI. Kernel independent component analysis. J Mach Learn Res
2002;3:1–48.
Data will be made available on request. [29] Abdi H, Williams LJ. Principal component analysis. Wiley Interdiscip Rev Comput
Stat 2010;2:433–59. https://doi.org/10.1002/wics.101.
[30] Coifman R.R., Lafon S. Diffusion maps. Applied and computational harmonic
analysis. 2006;21:5–30. 10.1016/j.acha.2006.04.006.

12
M.B. Mehni and M.B. Mehni
[31] Soize C, Ghanem R. Data-driven probability concentration and sampling on
manifold. J Comput Phys 2016;321:242–58. https://doi.org/10.1016/j.
jcp.2016.05.044.
[32] El Masri M, Morio J, Simatos F. Improvement of the cross-entropy method in high
dimension for failure probability estimation through a one-dimensional projection
without gradient estimation. Reliab Eng Syst Saf 2021;216:107991. https://doi.
org/10.1016/j.ress.2021.107991.
[33] Jia G, Taflanidis AA. Sample-based evaluation of global probabilistic sensitivity
measures. Comput Struct 2014;144:103–18. https://doi.org/10.1016/j.
compstruc.2014.07.019.
[34] Au SK, Beck JL. A new adaptive importance sampling scheme for reliability
calculations. Struct Saf 1999;21:135–58. https://doi.org/10.1016/S0167-4730
(99)00014-4.
[35] Jia G, Taflanidis AA, Beck JL. A new adaptive rejection sampling method using
kernel density approximations and its application to subset simulation. ASCE
ASME J Risk Uncertain Eng Syst Part A Civ Eng 2017;3:D4015001. https://doi.org/
10.1061/AJRUA6.0000841.
[36] Mou W, Ma YA, Wainwright MJ, Bartlett PL, Jordan MI. High-order langevin
diffusion yields an accelerated MCMC algorithm. J Mach Learn Res 2021;22(42):
1–41.
[37] Ma YA, Chen T, Fox E. A complete recipe for stochastic gradient MCMC. Adv
Neural Inf Process Syst 2015;28.
[38] Brooks S, Gelman A, Jones G, Meng X-L. Handbook of Markov chain monte carlo.
CRC press; 2011.
[39] Chen Y, Dwivedi R, Wainwright MJ, Yu B. Fast mixing of Metropolized
Hamiltonian Monte Carlo: benefits of multi-step gradients. J Mach Learn Res 2020;
21(92):1–63.
[40] Guimarães H, Matos JC, Henriques AA. An innovative adaptive sparse response
surface method for structural reliability analysis. Struct Saf 2018;73:12–28.
https://doi.org/10.1016/j.strusafe.2018.02.001.
[41] Xiao NC, Zuo MJ, Zhou C. A new adaptive sequential sampling method to construct
surrogate models for efficient reliability analysis. Reliab Eng Syst Saf 2018;169:
330–8. https://doi.org/10.1016/j.ress.2017.09.008.
[42] Roy A, Manna R, Chakraborty S. Support vector regression based metamodeling for
structural reliability analysis. Probab Eng Mech 2019;55:78–89. https://doi.org/
10.1016/j.probengmech.2018.11.001.
[43] Zhao H, Yue Z, Liu Y, Gao Z, Zhang Y. An efficient reliability method combining
adaptive importance sampling and Kriging metamodel. Appl Math Model 2015;39:
1853–66. https://doi.org/10.1016/j.apm.2014.10.015.
13
Reliability Engineering and System Safety 231 (2023) 109014
[44] Chen Z, Li G, He J, Yang Z, Wang J. A new parallel adaptive structural reliability
analysis method based on importance sampling and K-medoids clustering. Reliab
Eng Syst Saf 2022;218:108124. https://doi.org/10.1016/j.ress.2021.108124.
[45] Marelli S, Sudret B. An active-learning algorithm that combines sparse polynomial
chaos expansions and bootstrap for structural reliability analysis. Struct Saf 2018;
75:67–74. https://doi.org/10.1016/j.strusafe.2018.06.003.
[46] Fukumizu K, Bach FR, Jordan MI. Kernel dimension reduction in regression. Ann
Stat 2009;37:1871–905. https://doi.org/10.1214/08-AOS637.
[47] Fukumizu K, Leng C. Gradient-based kernel dimension reduction for regression.
J Am Stat Assoc 2014;109:359–70. https://doi.org/10.1080/
01621459.2013.838167.
[48] Zuniga MM, Murangira A, Perdrizet T. Structural reliability assessment through
surrogate based importance sampling with dimension reduction. Reliab Eng Syst
Saf 2021;207:107289. https://doi.org/10.1016/j.ress.2020.107289.
[49] Tabandeh A, Jia G, Gardoni P. A review and assessment of importance sampling
methods for reliability analysis. Struct Saf 2022;97:102216. https://doi.org/
10.1016/j.strusafe.2022.102216.
[50] Yang DY, Teng J, Frangopol DM. Cross-entropy-based adaptive importance
sampling for time-dependent reliability analysis of deteriorating structures. Struct
Saf 2017;66:38–50. https://doi.org/10.1016/j.strusafe.2016.12.006.
[51] Barkhori M, Shayanfar MA, Barkhordari MA, Bakhshpoori T. Kriging-aided cross-
entropy-based adaptive importance sampling using Gaussian mixture. Iran J Sci
Technol Trans Civ Eng 2019;43:81–8. https://doi.org/10.1007/s40996-018-0143-
y.
[52] Au SK, Beck J. Important sampling in high dimensions. Struct Saf 2003;25:139–63.
https://doi.org/10.1016/S0167-4730(02)00047-4.
[53] Liu PL, Der Kiureghian A. Multivariate distribution models with prescribed
marginals and covariances. Probab Eng Mech 1986;1:105–12. https://doi.org/
10.1016/0266-8920(86)90033-0.
[54] Rosenblatt M. Remarks on a multivariate transformation. Ann Math Stat 1952;23:
470–2.
[55] MacQueen J. Some methods for classification and analysis of multivariate
observations. Proceedings of the fifth Berkeley symposium on mathematical
statistics and probability: Oakland, CA, USA; 1967. p. 281–97.
[56] Engelund S, Rackwitz R. A benchmark study on importance sampling techniques in
structural reliability. Struct Saf 1993;12:255–76. https://doi.org/10.1016/0167-
4730(93)90056-7.