Structural and Multidisciplinary Optimization (2023) 66:119

https://doi.org/10.1007/s00158-023-03571-3

RESEARCH PAPER

Reliability‑based structural optimization using adaptive neural

network multisphere importance sampling
John Thedy1 · Kuo‑Wei Liao1

Received: 20 July 2022 / Revised: 14 March 2023 / Accepted: 16 April 2023 / Published online: 2 May 2023
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2023

Abstract
An innovative adaptive neural network multisphere importance sampling (ANNM-IS) is proposed and integrated with
symbiotic organism search (SOS) to form a framework for finding an engineering optimal design. Building a single sphere
in IS to enhance the computational efficiency has been used for decades, ANNM-IS provides a pioneering idea, in which
multi-spheres are built. “Adaptive point”, found by neural network (NN), is proposed to help for generating multiple spheres.
ANNM-IS is further integrated with SOS to update NN for next iteration. As optimization iterations increase, adaptive
NN provides more accurate reliability estimates. A two-step SOS, considering exploration and exploitation, is designed to
enhance the search performance. Four reliability problem are first solved to confirm the correctness and effectiveness of
ANNM-IS, then another four structural optimization problem including a building controller design and a 25-bar truss design
are solved. Results shown that the proposed method drastically reduces the amount of function evaluation and computation
time without sacrificing accuracy in reliability compared to those of other sampling methods. The developed framework can
solve a complex structural optimization problem of accurate reliability with affordable price. The supporting source codes
are available for download at https://​github.​com/​johnt​hedy/​RBDO-​using-​MIS-​NN-​SOS.

Keywords Multisphere Importance Sampling · Artificial intelligence · Uncertainty · Reliability-based design optimization

1 Introduction idea for reliability analysis that is further implemented with

Realization of structural performance and cost are two of the
main concerns in engineering. In addition, the importance
of performance uncertainties has been long recognized.
Reliability-based design optimization (RBDO) is consid-
ered as one of the most suitable approaches to find the bal-
ance between cost and performance with consideration of
uncertainty. RBDO integrates reliability analysis and opti-
mization algorithm to find the optimal solution/cost and at
the same time, to satisfy the designated performance that is
Symbiotic Organism Search (SOS) to conduct the RBDO
task. The efficiency is significantly improved and the accu-
racy is ensured due to several techniques proposed in our
algorithm, such as multiple spheres in Importance Sampling
(IS), Neural Network aided RBDO and the two-step SOS. In
RBDO, probabilistic constraints are often described in terms
of failure probability that can be formulated as failure region
area in multivariate probability density function (PDF) as
indicated in Eq. (1).

�G(𝜽)≤0
described by probabilistic constraints. As known, efficiency Pf = f𝛉 (𝜽)d𝜽 (1)
is one of the major concerns for an optimization task. When
combined with reliability analysis, the issue of efficiency
where fθ(θ) is a joint PDF in Θ-space that indicates the
needs more attention. This study proposes an innovative
object performance, θ = X1, X2, ⋯, Xnrv is the vector of ran-
dom variables, G(θ) is the performance or limit state func-
Responsible Editor: Byeng D Youn tion. A straightforward procedure in solving Eq. (1) is to
discretize the integration followed by simulating the limit
* Kuo‑Wei Liao state function (G(θ)) numerous times so that the failure prob-
kliao@ntu.edu.tw
ability (Pf) could be quantified as expressed in Eq. (2).
1
Department of Bioenvironmental Systems Engineering,
National Taiwan University, Taipei, Taiwan

13
Vol.:(0123456789)
119 Page 2 of 25
I G 𝜽i ≤ 0
nsample
1 ∑ [ ( ) ]
Pf = (2)
nsample i=1
in which, nsample is the number of simulation. I[.] is defined
as indicator function resulting in 1 when G(θ) is less than
zero and 0 otherwise. Equation (2) is known as Monte
Carlo Simulation (MCS) (Rubinstein 1981), MCS provides
an accurate estimate of failure probability as the sampling
number nsample increases. The unbiased error estimator of
MCS is defined in Eq. (2).
1 − Pf
nsample = (3)
ECOVP2 Pf
f
where ECOVPf is the expected coefficient of variation of
the failure probability. One could estimate required sam-
ple size based on ECOVPf and Pf using Eq. (3). However,
failure probability (Pf) is often an unknown value before
simulation. In addition, MCS is also known for its lack of
efficiency because a large sample size is needed when failure
probability is extremely small that is usually encountered in
an engineering case. Various methods have been proposed
to increase the efficiency of MCS. In general, reliability
methods could be categorized into five different categories
which are the most probable point (MPP)-based methods,
surrogate-based methods, simulation-based methods, expan-
sion-based methods and numerical integration-based meth-
ods (Youn et al. 2008; Yang et al. 2010; Yang et al. 2020).
MPP-based approach mostly derives a formulation of failure
probability (Pf) using a simplified limit state function with
moments of random variables. Most common used methods
are First Order Second Moment (FOSM) (Dolinski 1982),
First Order Reliability Method (FORM) (Zhao and Ono
1999) and Second Order Reliability Method (SORM) (Der
Kiureghian et al. 1987). All methods utilize the first order
Taylor expansion to generate a simplified limit state. Unlike
the FOSM and FORM, SORM utilizes the second order
Taylor’s expansion in finding the failure probability. Both
FORM and SORM deeply depend on MPP that makes them
fail to accurately estimate failure probability for engineer-
ing cases with large coefficient of variation (COV) (Ching
and Phoon 2013). Instead of basing on MPP, COV-based
safety factor is proposed. Similar to FORM, the first order
Taylor expansion is adopted to generate a linear limit state
but with the quantile-based design point. The aforemen-
tioned MPP-based approaches attempt to mimic limit state
function with a simplified function that gives tremendous
advantages in many cases but high error is expected when
problem with highly nonlinear or multiple limit state func-
tions is encountered.
Surrogate model method (SMM) is one of the most effi-
cient reliability analysis methods in terms of the number of
13
J. Thedy, K.-W. Liao
function evaluation. The main idea is to replace complex
limit state function using an explicit or implicit surrogate
model. One of the major challenges for SMM is selecting
appropriate training data for constructing surrogate model.
Bucher and Bourgund (1990) adopted the adaptive sampling
method for selecting training data and used quadratic poly-
nomial function as surrogate model. Surrogate model con-
structed by Support Vector Machine (SVM) also has been
investigated (Li et al. 2006; Alibrandi et al. 2015). Recently,
Kriging was employed as implicit surrogate model for reli-
ability analysis (Echard et al. 2011; Balesdent et al. 2013;
Cadini et al. 2014; Yun et al. 2020; Xiao et al. 2020). Krig-
ing enables users to evaluate sample variance and select
the best sample as training data. However, as the sample
pool size increase, evaluation for selection training data
becomes one of the major computational burdens. Alter-
native methods that combine sampling-based approach
and various surrogate models could be found in references
(Grooteman 2011; Youn and Choi 2003; Asghar et al. 2019).
The expansion-based method, including Taylor expansion,
perturbation method and Neumann expansion method, uses
a small perturbation to simulate uncertainty and then the
statistical moments of system responses are estimated (Youn
et al. 2008). The numerical integration-based methods are
considered as a direct approach to estimate the PDF (Youn
et al. 2008). For example, a single variable PDF such as
Pearson or Three Points Lognormal could be constructed
through point estimate method (Zhao and Lu 2007; Liao
and Biton 2019). Such approach gives immense advantage
in calculating failure probability, especially, when one has a
small number of random variables.
Among various reliability approaches, simulation-based
method is the most prominent one in terms of accuracy.
Various simulation methods can be considered as deriva-
tive products of MCS. Some of them modify the sampling
technique or introduce an intermediate process. In any case,
reducing sampling size is a common goal. Latin Hypercube
Sampling (LHS) is a stratified sampling method based on the
Cumulative Density Function (CDF) of random variables
(Olsson et al. 2003). IS is another popular sampling-based
variant. IS shifts sample center from mean point to MPP (or
other points) with the proposed PDF. In consequence, each
sample is penalized with ratio between the proposed and the
original PDFs as described in Eq. (4).
nsample { ( )}
I G 𝜽i ≤ 0 ( )
∑ [ ( ) ] f 𝜽i
Pf =
1
(4)
nsample i=1 h 𝜽i
where h(θi) and f(θi) are the proposed and the original
PDFs, θi are samples generated using the proposed PDF
(Melchers 1989). For a system problem with multiple limit
states, Bucher (1988) suggested to pre-perform a sensitiv-
ity analysis in determining the mean values and variances
Reliability‑based structural optimization using adaptive neural network multisphere…
for θi. Numerous efforts have focused on finding the opti-
mal h(θi), including the PDF, mean values and variances. In
order to reduce variance of evaluated Pf, Ang George (1992)
introduced a kernel density function as the optimal PDF of
the importance sampling density that is determined by the
samples obtained from an initial Monte Carlo Simulation.
Papaioannou et al. (2016) developed a sequential importance
sampling approach. A sequence of distributions is generated
and gradually approach the optimal importance sampling
density, in which two Markov Chain Monte Carlo (MCMC)
algorithms for sampling the intermediate distributions are
proposed. Other sequential sampling variants with similar
concept could be found in other studies (Ching and Chen
2007; Martino et al. 2017; Katafygiotis and Zuev 2007).
Another IS variant is Subset Simulation (SS), it implements
conditional probability theory in evaluating the failure prob-
ability. SS divides sample space into several failure regions
and utilizes MCMC to generate samples from one region to
another. This method requires a priori knowledge to classify
failure threshold of each failure region (Au and Beck 2001).
In addition to generating samples using importance sampling
density, Radial-based Importance Sampling (RBIS) gener-
ates origin centered sphere in the safety domain to reduce
the sampling size. That is, samples inside the sphere are not
used for function evaluation. This method is very efficiency
when failure probability is extremely small. RBIS requires
MPP information to obtain the distance toward origin as the
sphere radius (Harbitz 1986).
The goal of this study is to develop an accurate RBDO
in an efficient manner. RBDO inherently conducts two dif-
ferent tasks with respect to design and random variables for
the outer and inner loops, respectively. One of the popular
RBDO approaches is a single loop procedure, in which the
inner loop process maybe simplified by adopting FORM
technique (Lopez et al. 2011; Du and Chen 2004; Ahn and
Kwon 2004, 2006). Alternatively, Lopez et al. (2011) pro-
posed a single loop RBDO approach by replacing reliability
analysis with Karush–Kuhn–Tucker (KKT) optimality con-
dition. Ahn and Kwon (2004) and Ahn and Kwon (2006)
take advantage of single loop RBDO by making the proba-
bilistic constraint (i.e., P[g(x) ≥ 0] ≤ Φ(− βt)) equivalent to
the deterministic constraint (i.e., g(x) ≤ 0). Ahn et al. (Ahn
and Kwon 2006) introduce Bi-level integrated system syn-
thesis (BLISS), in which multi-disciplinary equation con-
straints and trust region method was proposed, to handle
nonlinear behavior of the objective function. In addition to
probabilistic constraints adopted in RBDO, non-probabilistic
RBDO also proposed by Wen et al. (Wang et al. 2019; Yao
et al. 2013), in which the probabilistic constraints are con-
sidered using Unified Uncertainty Analysis (UUA). Another
notable RBDO approaches is Sequential Optimization and
Reliability Assessment (SORA) (Ahn and Kwon 2004).
SORA utilizes inverse MPP obtained from the previous loop
Page 3 of 25 119
to decouple the optimization and reliability analysis. That
is, the random variables stay as constant when optimization
is conducted and the design variables remain constant when
reliability analysis is performed.
In our proposed RBDO framework, reliability is evalu-
ated via simulation-based algorithm with surrogate model
and optimization is conducted via metaheuristic algorithm.
The relative studies are reviewed below. For the simulation-
based RBDO with surrogate model, Lee and Lee (2021) pro-
posed a probabilistic framework that considers the magni-
tude of failure differently via a weighted RBDO (WRBDO),
in which weighted probabilistic constraints are constructed
to reflect the weighted failure. In WRBDO, AK-MCS is
adopted to evaluate the weighted probabilistic constraints.
A new learning is proposed for updating Kriging model due
to the need of identification of failure magnitudes. Assum-
ing an accurate surrogate model is given, Lee et al. (2011)
calculated the stochastic sensitivities via MCS with the help
of score function. Unlike finite difference method, only one
MCS is required at a given design for obtaining the prob-
ability of failure and its sensitivity simultaneously. Yang
et al. (2021) conducted an RBDO task through single loop
approach (SLA). Kriging-based surrogate model is used to
evaluate the reliability constraints and the Kriging is updated
at each loop using MPP. To save cost, Karush–Kuhn–Tucker
(KKT) optimality condition is utilized to determine whether
an accurate MPP is needed in updating the Kriging. Yang
et al. (2022) used a single-loop strategy with local adaptive
Kriging approximation to perform an RBDO task, in which
both the objective and constraint functions are replaced by
the Kriging models. Kriging model of performance function
is updated using inverse most probable point (IMPP) if it is
active. Kriging model of objective function is sequentially
refined through points selected by their proposed learning
function.
For the metaheuristic-based RBDO, Meng et al. (2021)
conducted a comprehensive comparison work of metaheuris-
tics-based RBDO. They concluded that compared to gradi-
ent-based RBDO, metaheuristics-based RBDO is superior
in terms of global convergence, robustness and accuracy.
However, metaheuristics-based RBDO does not have a good
performance in computational efficiency. Azad (2021) pro-
posed a metaheuristic algorithm to solve a structural prob-
lem with discrete design variables and large size of random
variables. Truong and Kim (2017) proposed a double loop
RBDO, in which outer loop was handled by metaheuristic
harmony search and inner loop was solved by Importance
Sampling (IS). Panagant et al. (2019) performed a multiple
objective metaheuristic optimization using their novel whale
optimization while FOSM was adopted for probabilistic
analysis in the inner loop. Datta et al. (2020) also conducted
a metaheuristic RBDO with surrogate model as inner loop.
Different from single loop approach, metaheuristic-based
13
119 Page 4 of 25
RBDO often does not has strong connection between its
optimizer and reliability method. Taking a single loop
approach (i.e., SORA) for example, The SORA method
employs a serial of cycles of deterministic optimization and
reliability assessment. The reliability information obtained
in the previous cycle is used to shift the boundaries of vio-
lated constraints in deterministic optimization as proposed
in Du and Chen (2004).
This study proposes an innovative method for solving
an RBDO problem, in which reliability is evaluated via
simulation-based algorithm with surrogate model and opti-
mization is conducted via metaheuristic algorithm. Fur-
ther, an advance simulation technique (M-IS) is integrated
with surrogate model (NN) to find the adaptive points
and save the cost. To lessen the computational burden in
metaheuristics-based RBDO, a two-step SOS algorithm,
including exploration and exploitation, is developed in
our proposed algorithm. That is, metaheuristic optimiza-
tion and adaptive neural network multisphere importance
sampling (ANNM-IS) are integrated as a platform to solve
a RBDO problem. The major concept of the framework is
illustrated in Fig. 1. It is seen that the Symbiotic Organ-
ism Search (SOS) metaheuristic algorithm is utilized as the
optimizer in the outer loop, while ANNM-IS deals with the
reliability analysis in the inner loop. As mentioned, opti-
mizer and reliability approach often do not intensively and
collaboratively operate with each other in a metaheuristic-
based RBDO. Nevertheless, certain degree of collabora-
tion between optimizer and reliability analysis can be still
found in our proposed approach. As shown, in the inner
loop, M-IS reliability method (Thedy and Liao 2021) and
Neural Network (NN) (Rosenblatt 1958; Werbos 1991) are
coupled as the ANNM-IS, in which samples evaluated in
M-IS are treated as the training data for the NN and NN
serves as the surrogate model for reliability analysis. M-IS
is a sampling-based reliability method that employs multiple
spheres with various centers and radii to exclude safety sam-
ples inside the deployed multi-spheres and to enhance the
efficiency. That is, the proposed ANNM-IS is a sampling and
Fig. 1  Conceptual illustration of
Outer Loop : Metaheuristic
the proposed method
Optimization
Inner Loop : Multisphere
Importance Sampling (M-IS)
Neural Network
13
J. Thedy, K.-W. Liao
surrogate-based reliability approach. It is known that MPP-
based RBIS does not guarantee the built sphere always stays
in the safety region. To overcome this drawback, ANNM-IS
proposes an adaptive framework in finding the sphere loca-
tions and radii. During the adaptive process, a great num-
ber of function evaluation may be needed for each deployed
sphere. To reduce the number of function evaluation, NN is
imbedded in the M-IS process in finding each sphere. The
interaction between M-IS and NN is conceptually illustrated
in Fig. 1. As shown, NN is built at the end of each optimiza-
tion iteration and is used for the next optimization iteration.
Instead of using all samples, only samples in the M-IS pro-
cess of the previous optimization iteration is used to train the
NN. The more the iteration, the more converged the design
variable. Please note that samples generated for function
evaluation during adaptive process will distribute along
the failure surface when ANNM-IS is adopted. With this,
ANNM-IS provides an accurate reliability estimate with a
limit number of training samples. The proposed RBDO algo-
rithm integrating metaheuristic optimization, M-IS and NN
that can boost the RBDO efficiency in terms of computation
time and function evaluation. Details of implementing M-IS
with NN is rigorously described in Sect. 2. The step-by-step
procedures of the proposed RBDO is provided in Sect. 3.
Several engineering optimization problems are presented in
Sect. 4 to demonstrate the efficiency of proposed method.
2 Adaptive neural network multisphere
importance sampling (ANNM‑IS)
This section briefly describes the main idea of the proposed
ANNM-IS. ANNM-IS is an enhanced version of M-IS and
M-IS is a derivative reliability assessment method of RBIS.
ANNM-IS utilizes multiple spheres to identify more safety
samples. The general framework of ANNM-IS is shown in
Fig. 2. Since multiple spheres are constructed in ANNM-IS,
hereafter, two types of generated spheres termed as “origin
sphere” and “non-origin sphere” are introduced. The “origin
Function Evaluated
Samples from M-IS used
as training data for NN
NN help M-IS determine
samples are fail or safe
Reliability‑based structural optimization using adaptive neural network multisphere… Page 5 of 25 119

Fig. 2  General procedure of i=i+1

M-IS and ANNM-IS
Step 2 : Generate i=2
Step 1 : Deploy Step 3 : Deploy Non- Remove Sample inside
Additional Samples
Origin Sphere of β1 Origin Sphere of βi Sphere of βi
Outside Origin Sphere

Perform Function
Yes No
Evaluate Pf Evaluation for the i=nsphere
Remaining Samples

sphere” refers to the first built sphere, in which it is cen-
tered at zero point in the reduced space, while other subse-
quent spheres with centers located other than zero point are
named as “non-origin sphere”. Difference between RBIS
and M-IS or ANNM-IS could be depicted in Fig. 3a and b,
respectively. As shown in Fig. 3b, ANNM-IS uses multiple
spheres to recognize more safety samples. In ANNM-IS,
only a small number of samples (e.g., the orange dots) are
calculated by the performance function. Construction of
the “origin” and “non-origin” spheres will be thoroughly
explained in the subSects. 2.1 and 2.2, respectively.
2.1 Origin sphere
According to the general procedure of the proposed ANNM-
IS as shown in Fig. 2, the “origin sphere” needs to be first
determined. In this step, the only required parameter is the
radius. The flowchart is displayed in Fig. 4. Figure 5 displays
a step-by-step graphic illustration. At first, a ring region is
formed using βin and βout as shown in Fig. 5a. βout is an
arbitrary radius as long as the failure region is included in
a sphere with radius of βout, while βin is directly taken as
βout—0.05. Five samples are then randomly generated in the
ring region using Chi-square distribution. Function evalua-
tion are performed for these five samples to recognize their
locations (i.e., in the failure or safety area) and all informa-
tion related to these five samples are stored in matrix M1,
which are used as training data for the NN. In addition to
these five samples, additional random samples are continu-
ally generated in the ring region until the number of failure
samples reaches nap (i.e., 10) as shown in Fig. 5b. Note that
to save cost, locations of these additional samples are deter-
mined by NN and function evaluation is only conducted for
samples that are classified as failure samples as shown in
Fig. 4. That is, function evaluation is not performed if NN
indicates that sample is located in the safety region. These
identified failure samples in the ring region are termed as
“adaptive point”. The values of nap and βin are predetermined
based on numerous benchmark problems that can be found
in reference [40]. Again, results of function evaluation will
be stored in M1 matrix as NN training data. The number
of five samples in the initial step as shown in Fig. 5a is
to ensure one can have both samples (failure and safety)
for training NN. MPP candidates (MPPCs) are the junction

Fig. 3  Difference between a RBIS (Harbitz 1986) and b M-IS/ANNM-IS (Thedy and Liao 2021)

13
119 Page 6 of 25 J. Thedy, K.-W. Liao

Fig. 4  Flowchart of adaptive

Determine βout, βin, ntrial = 0, natt, and
scheme for determining the
radius of the “origin sphere” nap=10
ite = 1
Use Chi-Square to Generate and Evaluate
Stored to M1 Matrix
Single Samples between βout and βin
ntrial = ntrial +1
Yes
ntrial > natt
No
Function Evaluation
Using NN

No
Fail?
Yes
Function Evaluation

No
G(θ)<0
Yes
No
Number of Failure Samples = nap
Yes
nap Times of Line Search
End
β1=MPP
Radius
Determine MPP among MPPCs
Yes
No
MPP Radius < βin No nstop=3
nstop = nstop+1
ite = ite+1 Yes

Update
βout = MPP Radius , βin = MPP Radius – 0.05,
nap=3, natt, nstop = 0, and ntrial = 0

points of the performance function and the straight line con-
necting the origin and a failure point. Therefore, using infor-
mation of sphere center, limit state function, and adaptive
points, several MPPCs can be found by using any line search
technique as shown in Fig. 5c. This study uses bisection line
search method to locate MPPCs. Apparently, several func-
tion evaluations are expected in line search and any results
will be again, stored to M1 matrix. Among MPPCs, MPP is
determined by selecting the one with shortest distance to the
center. βout for the next iteration is equal to this minimum
distance as illustrated in Fig. 5d and e. Based on Fig. 4, the
stop criteria are: (1) the updated MPP radius is greater than
the βin and the iteration number is greater than 3, that is,
no more failure point can be found in the ring area within
three iterations (nstop = 3); (2) the sample size in the ring
area is large enough (ntrial > natt), where ntrial is the number
of sample generated and natt is the adequate sample size as
shown in Eq. (5). Equation (5) assumes that the variation in
the estimated failure probability has the same level of RBIS
with radius of βout.
( [ ( )])
1 − 1 − Φ 𝛽out ( ( ) ( ))
natt = [ ( )] 𝜒dof 2 2
𝛽out − 𝜒dof 𝛽in (5)
ECOVPf 1 − Φ 𝛽out
in which, Φ is the CDF of standard normal distribution and
2
𝜒dof is the Chi-Square distribution with degree of freedom
of dof. Extra samples outside origin sphere are created using
chi-square distribution and are stored in matrix Nsample (Step
2 of the Fig. 2). As illustrated in Fig. 5, during constructing
the “origin sphere”, several samples have been generated
either they are located in the failure or safety region. For
reliability analysis, additional samples are needed and are
added based on the radius of the “original sphere” (i.e., β1).
These additional samples are added to the existing samples
until the total amount of samples, which are outside the “ori-
gin sphere”, is equal to n sample as indicated in Eq. (6).

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 7 of 25 119

(a) (b)

(c) (d)

(e)
Fig. 5  Step-by-step graphic illustration for determining the radius of the “origin sphere”

13
119 Page 8 of 25 J. Thedy, K.-W. Liao

Equation (6) has similar concept as that of Eq. (5), in which of “Non-origin sphere” is to remove more samples and
only the reliability index is replaced by β1. reduces the number of function calculation. Figure 8a
( [ ( )]) illustrates the candidate centers for the “non-origin
1 − 1 − Φ 𝛽1 ( ( )) spheres”. As shown, 8 candidate centers, generated based
nsample = [ ( )] 1 − 𝜒n2 𝛽12 (6)
ECOVPf 1 − Φ 𝛽1 on 3-level full factorial design, are considered (i.e., 3nrv-1,
nrv is the number of random variables). Intuitively, more
samples will be eliminated from function evaluation when
a candidate point has a larger distance to the adaptive
2.2 Non‑origin sphere points. A list of candidate centers ordering by distance
is then generated and the top 1.5nrv are used as the cent-
This section provides information of generating the “non- ers for the “non-origin sphere”. Naturally, these centers
origin spheres”. Figure 6 illustrates the adaptive proce- should be in the safety area, if not, they are replaced by
dure for the “non-origin sphere”. Similarly, a step-by-step the subsequent candidates until 1.5nrv centers are chosen.
graphic illustration is provided in Fig. 7. The purpose

Fig. 6  flowchart of the proposed

adaptive scheme for the “non-
origin sphere” Determine Initial βout , βin, Or

nite = 1
End No Any Nsample Point between
βi = βout
βout and βin ?
(i ≠ 1)
Yes
Randomly Select a Single
Stored to M1 Matrix
Sample

Function Evaluation Using

NN

No
Fail?
Yes

Function Evaluation

No
g(θ)<0
Yes
Line Search of Failure
Sample to Obtain MPP
nite = nite+1
Update
βout = min [ βout , MPP Radius ],
βin = min [ βout , MPP Radius ] – 0.2

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 9 of 25 119

(a) (b)

(c) (d)

Fig. 7  Step-by-step graphic illustration of the adaptive scheme for the “non-origin sphere”

Once centers of the “non-origin spheres” are deter-
mined, the corresponding radii are readily determined.
Procedures of searching the radii for the “non-origin
sphere” are slightly different with that of “origin sphere”.
The initial β out is taken as the shortest distance among
adaptive points toward center of sphere as illustrated
in Fig. 8b, as shown, d 2 is selected as the initial radius.
While β in is taken as β out—0.2 to form a ring region as
shown in Fig. 7a. Samples inside N sample are randomly
picked until a single failure point is found as shown in
Fig. 7b. Then, line search is performed for that single
failure point to find MPP as illustrated in Fig. 7c. The
obtained MPP is used for the subsequent β out for the next
iteration as shown in Fig. 7d and this procedure is repeat-
edly performed until no single failure point can be found
in the ring region.n sphere “non-origin sphere” is needed
in the proposed framework, where nsphere is the rounding
value of 1.5n rv. To fulfill this requirement, the above-
mentioned process (i.e., Step 3 in Fig. 2) is successively
performed. When “non-origin spheres” are built, samples
inside the “non-origin spheres” are removed. The remain-
ing samples in Nsample (Fig. 2) are then used to calculate
the failure probability as indicated in Eq. (7), where NN
is used for function evaluation.

13
119 Page 10 of 25 J. Thedy, K.-W. Liao

(a) (b)

Fig. 8  a Candidate centers and di for a candidate center located at right bottom Corner, b Determination of the Initial βout for “non-origin sphere”

� �
⎡ � � n ⎤ ⎛� ⎞ To enhance the efficiency of RBDO, ANNM-IS is per-
= P⎢G 𝜽 = X1 , X2 , ..., Xn ≤ 0� Xj ≥ 𝛽 ⎥P⎜� Xj2 ≥ 𝛽1 ⎟
� �� ��n
�
Pf
⎢
2
⎥ ⎜ ⎟ formed using ECOVPf of 0.2 at the beginning of analysis.
⎣ j=1 ⎦ ⎝ j=1 ⎠ The Higher the ECOVPf, the smaller the sample size. How-
�
� n
⎡ � ⎤ ever, if 70% of the organisms have approached target reli-
= P⎢G 𝜽 = X1 , X2 , ..., Xn ≤ 0�� Xj2 ≥ 𝛽 ⎥ 1 − 𝜒nrv
� �� � � 2 ��
2
𝛽1
⎢ ⎥ ability index (βt) with 3% tolerance, ECOVPf will be shifted
⎣ j=1 ⎦
(7) to 0.05 as shown in Fig. 9. The change of ECOVPf can
also be regarded as the consideration of “exploration” and
in which, Pf is the failure probability, P denotes probability, “exploitation”. At early stage of RBDO, it is more impor-
X1, X2…, Xn is a set of nrv dimensional random variables, tant to ensure that one have examined enough regions in the
G(θ) are limit state functions, β is the reliability index, 𝜒nrv
2
feasible domain to lower the chance of being trapped in a
is the Chi-Square distribution with nrv degree of freedom. local minimum. When ECOVPf is 0.2, the search process is
more like “exploration” and the optimization goal is to find
the minimum difference between organisms’/swarms’ reli-
ability and target reliability (βt). This objective is termed
3 The proposed reliability based design as “Objective 1”. Once 70% of the entire organisms have
optimization reliability index near βt with certain tolerance, indicating
that a promising region has been identified. To thoroughly
The double-loop optimization technique, as shown in Fig. 9, search this recognized region to locate the local minimum,
is adopted here, in which SOS and ANNM-IS are used “exploitation” is performed and the objective is changed to
for the outer and inner loop analysis, respectively. SOS is “Objective 2”, in which the goal is to minimize the cost. In
selected due to its superiority compared to other optimiza- the proposed RBDO, feasible region is a set of dimensions
tion algorithms (Cheng and Prayogo 2014). SOS is a nature- that could give reliability equal to or more than 1-Φ(βt). A
inspired algorithm, consisting of three phases: mutualism, better organism in “Objective 2” is defined as below. (1) If
commensalism and parasitism. Figure describes the flow- a feasible candidate is compared to an infeasible candidate,
chart of the proposed RBDO. Because target reliability index the feasible candidate wins. (2) If two feasible candidates are
(βt) is a priori, the initial radius (βout) of the “origin sphere” compared, the one with lower cost wins. (3) If two infeasi-
is formulated as shown in Eq. (8). ble candidates are compared, the one with reliability index
√ ( ( )) closer to βt wins.
𝛽out = 𝜒dof 2
Φ 𝛽t (8) As mentioned, the proposed RBDO framework utilizes
a classification type of NN in determining an organism
is in a failure or safety region during ANNM-IS process.

13
Reliability‑based structural optimization using adaptive neural network multisphere…
Fig. 9  The proposed RBDO flowchart. *BF is benefit vector
NN is initially constructed with only 10 samples as shown
in Fig. 9. As shown, NN is used in reliability analysis
for every organism. If fails, function evaluation is per-
formed for the failed sample and recorded for training
NN in the next iteration. Note that a sample refers to a
vector of random variables used in the reliability analy-
sis and an organism is a vector of design variables used
in the optimization process. It is seen that in addition to
the initial 10 function evaluations, more evaluations are
obtained during optimization. In fact, as shown in Figs. 4
and 6, function evaluations may be conducted in determin-
ing the radius of the “Origin Sphere” and in finding the
MPPC for the “Non-Origin Sphere”. With these informa-
tion, the NN model is updated at each iteration. Because
sampling-based reliability assessment possesses an inherit
Page 11 of 25 119
randomness depending on ECOVPf, to calibrate reliability
assessment for every organism, all organisms are evaluated
using the most recent NN every four iteration.
A two-layer feed-forward network with a sigmoid
transfer function in the hidden layer and a softmax trans-
fer function in the output layer, adopted from MATLAB
machine learning toolbox, is used here to build the NN
model as shown in Fig. 10. Number of hidden layer is dif-
ferent for each case study depending on its needs. Weights
(w) and bias value (b) of hidden and output layer are deter-
mined using gradient search algorithm, in which the cross
entropy function is taken as the performance function as
shown in Eq. (9), where y and y are actual and predicted
output data, respectively.
13
119 Page 12 of 25 J. Thedy, K.-W. Liao

Input w w may be slightly different from that of ANNM-IS. One can

+ + Output further examine the accuracy of ANNM-IS in the second
b b
part, which demonstrates the applicability of the proposed
ANNM-IS-SOS. Several engineering cases are carried out
Sigmoid Softmax
Function Function to investigate the efficiency and accuracy of the integrated
ANNM-IS and SOS framework. Results of each numerical
Fig. 10  Neural network structure used in the current study
case are discussed in the following.

4.1 Part I: reliability analysis

( ) ∑ ( )
P y, y = y log y
4 Numerical demonstrations
This section contains two parts. The first part provides
efficiency and accuracy comparisons of the proposed reli-
ability method with other importance sampling approaches.
Ideally, ANNM-IS should be used in such comparison pro-
cess. However, ANNM-IS is fully integrated with the opti-
mization procedure and cannot be separately executed. For
example, in addition to the initial sampling (i.e., 10 sam-
ples), other training samples of ANNM-IS mainly rely on
feasibility evaluation outcome of each organism in optimi-
zation. Thus, instead of ANNM-IS, M-IS is used in the first
part of comparisons. Note that the efficiency of ANNM-IS
should be better than that of M-IS and the accuracy of M-IS
(9)
As mentioned, to demonstrate the efficiency and accuracy
of the proposed ANNM-IS, M-IS is compared with some
recently developed sampling methods such as line sampling
and sequential importance sampling via evaluating four
engineering reliability problems, as indicated in Table 1
(Papaioannou et al. 2018). The accuracy and efficiency
comparisons are provided in Tables 2 and 3, respectively.
For the accuracy aspect, as seen, compared with line sam-
pling, importance sampling (IS) and sequential importance
sampling (SIS), M-IS is able to provide a failure probabil-
ity with similar accuracy. In addition, the randomness of
failure probability obtained from M-IS is often better than
those of literature studies. Note that M-IS is repeatedly per-
formed 50 times to have the statistics in Tables 2 and 3, and
ECOVPf of 10% is used for each M-IS calculation. Results
of line sampling, importance sampling (IS) and sequential
importance sampling are directly acquired from the literature
(Papaioannou et al. 2018). Regarding to the efficiency, it is
seen that M-IS outperforms line sampling and SIS. In case

Table 1  Literature Problems used for Comparison in Reliability Analysis

Case Limit state Function Random variable Description
( )
A g=y− 600
Y + 600
X X = N(500, 100) Cantilevered beam subjected to biaxial loadings of X and Y with
wt2 w2 t
Y = N(1000, 100) consideration of stress constraint
( )
y = N 4 × 104 , 2 × 103
√( ) ( )2
B 4L3
2
Y X
X = N(500, 100) Cantilevered beam subjected to biaxial loadings of X and Y with
g = D0 − Ewt t2
+ w2 Y = N(1000, 100) consideration of displacement constraint
( )
E = N 29 × 106 , 1.45 × 106
( ) ( )
C g = 0.03 − ql2 3.81
+ 1.13 q = N 2 × 104 , 1400 Roof truss subjected to vertical loadings with consideration of
2 Ac Ec As E s serviceability constraint
l = N(12, 0.12)
( )
As = N 9.82 × 10−4 , 5.89 × 10−5
( )
Ac = N 0.04, 4.8 × 10−3
( )
Ec = N 1 × 1011 , 6 × 109
( )
Ec = N 2 × 1010 , 12 × 108
D g1 = 2M1 + 2M3 − 4.5S M1 = LN(200, 30) Elastoplastic frame structure subjected to a horizontal load with
g2 = 2M1 + M2 + M3 − 4.5S M2 = LN(200, 30) consideration of collapse mechanism
g3 = M1 + M2 + 2M3 − 4.5S M3 = LN(200, 30)
g4 = M1 + 2M2 + M3 − 4.5S S = LN(50, 20)
( )
g = min g1 , g2 , g3 , g4

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 13 of 25 119

Table 2  Accuracy comparison in terms of failure probability

Case Failure Probability
Exact Solution Line Sampling IS SIS M-IS (50 trials)
1 1 1
Pf E[Pf] COV E[Pf] COV E[Pf] COV E[Pf] COV1

A 3.02E-03 3.03E-03 7.00E-02 – – 3.01E-03 1.02E-01 2.98E-03 9.44E-02

B 2.57E-04 2.52E-04 1.40E-01 – – 2.45E-04 1.28E-01 2.54E-04 9.64E-02
C 9.38E-03 – – 9.39E-03 5.60E-02 9.24E-03 9.60E-02 9.15E-03 8.75E-02
D 5.33E-04 – – 5.19E-04 1.15E-01 5.27E-04 8.60E-02 5.26E-04 9.55E-02

Table 3  Efficiency comparison in terms of sample size

F
Case Sampling Size
Line Sam- IS SIS M-IS (50 trials)
pling
μ2 δ

1 1.60E + 03 – 4.00E + 03 6.35E + 02 1.23E-01 M2

2 2.64E + 03 – 5.00E + 03 9.91E + 02 1.55E-01

3 – 1.00E + ­031 3.00E + 03 1.51E + 03 2.15E-01 A A
4 – 2.00E + ­031 4.00E + 03 1.93E + 03 1.80E-01
h M1
1
cost of finding β is not included
2
mean value b
Section A-A

C, IS is more efficient than M-IS. However, IS assumes that

the information of reliability index (β) is given, that is, the
sample size for IS shown in Table 3 does not consider the
cost of finding β, so that this is not an “apples-to-apples”
comparison.
COV coefficient of variation.

4.2 Part II: RBDO, case 1 (non‑normal—short
column design)
In Case 1, a short column problem with a nonlinear limit
state function with six random variables (including four ran-
dom design parameters and two random design variables)
and two design variables (b and h) is considered. Figure 11
illustrates the problem, as shown, the column is subjected
to axial and torsional loadings from both directions. The
mathematical formulation of the RBDO problem is given
in Eq. (10).
Min.A = 𝜇b 𝜇h
Fig. 11  Illustration of short column including the external loading
where b is the width, h is the depth, 𝜇b is the mean value of
b, 𝜇h is the mean value of h, G(d, X) is the limit state func-
tion, d is the vector of design variables, X is the vector of the
random design parameters, PTf is the target failure probabil-
ity (i.e., 1.35 ×10−3), fy is the yield strength, M1 and M2 are
the bi-axial moments, F is the axial force. Random variables
in this RBDO are listed in Table 4. As shown, b and h are
random design variables, rest of them are random design
parameters. Parameters used in SOS are: 30 iterations, 10
organisms and NN of 10 layers.

s.t. Pr ob.[G(d, X) ≤ 0] ≤ PTF

It is of interest to investigate the accuracy of the pro-
posed NN. In light of this, although it is not required in
0.5 ≤ b ≤ 2
𝜇 the proposed RBDO framework, samples predicted using
𝜇h NN are also evaluated by the limit state function. Fig-
G(d, X) = 1 − (4M1 )∕(bh2 fy ) − (4M2 )∕(b2 hfy ) − F 2 ∕(bhfy )2 ure 12 displays the accuracy of NN at each iteration, as
(10) shown, the training data size is also included. Alterna-
tively, Fig. 13 shows the required training computation

13
119 Page 14 of 25 J. Thedy, K.-W. Liao

Table 4  Random design parameters (Top Four) and random design the accuracy of NN is always greater than 98%. From
variables (bottom two) in case 1 (non-normal) Figs. 12 and 13, one can find that the computation time
Description Symbol Mean COV1 Distribution Unit is linearly, not exponentially, increase with the training
data size, which is a very important efficiency issue when
Moment 1 M1 250 0.3 Gumbel kN-m
surrogate model is adopted for RBDO [29]. RBDO results
Moment 2 M2 125 0.3 Gumbel kN-m
are summarized in Table 5, while the convergence history
Axial Load F 2500 0.2 Gumbel kN
of fitness and reliability index are shown in Fig. 14. As
Yield Strength fy 40 0.1 Normal MPa
mentioned earlier, reliability of sampling-based method
Width b – 0.1 Lognormal m
possesses inherent randomness and therefore, fluctuation
Depth h – 0.1 Lognormal m
is found in Fig. 14. Nevertheless, the obtained reliability
1
COV: coefficient of variation of the final organism is compared to that of MCS, indi-
cating that the proposed method gives a very low error

Fig. 12  Training data size and

NN accuracy at each iteration
for Case 1

Fig. 13  NN training time at

each iteration for case 1

Table 5  RBDO comparisons for case 1

Description Proposed ANNM-IS- ANN-SMM-SOS 4MP-PSO/SOS 4MP-PSO/SOS
SOS (Bivariate) (Trivariate)
Best Organism (in) b = 0.68 h = 0.34 b = 0.64 h = 0.35 b = 0.67 h = 0.35 b = 0.63 h = 0.33 b = 0.60 h = 0.36

Fitness ­(in2) 0.23 0.22 0.23 0.21 0.22

βBest Fitness 3.00 3.00 3.00 3.00 3.00
βMCS 3.01 2.84 3.10 2.73 2.99
Error percentage (%) 0.33 5.33 3.33 9.00 0.33
Average FE per RA 159.07 175 250 778 1758

time at each iteration. In Fig. 12, although only data in

percentage, as shown in Table 5. Figure 15 shows required
the previous iteration is used for building NN, a promising
number of function evaluation per reliability analysis (i.e.,
prediction accuracy is observed. After the 1­ 5th iteration,

13
Reliability‑based structural optimization using adaptive neural network multisphere…
Fig. 14  Convergence history of
fitness and reliability index for
case 1 using ANNM-IS
Fig. 15  Required function evaluation for ANNM-IS in each iteration
in case 1
FE per RA) at every iteration. In average, the proposed
ANNM-IS needs 159 function evaluations.
For fair comparison in reliability-based design opti-
mization (i.e., RBDO), two newly metaheuristic-based
RBDO are used for comparison. The first one is the Fourth
Moment Pearson system (4MP) with Particle Swarm Opti-
mization (PSO) or Symbiotic Organism Search (SOS)
(Liao and Biton 2019), in which 4MP is for reliability
analysis and PSO/SOS is the optimizer (hereafter collec-
tively referred to as the “4MP-PSO/SOS”). The second
one is the artificial neural network-based surrogate model
method (ANN-based SMM) with Symbiotic Organism
Search (SOS) (Kang et al. 2015; Gholizadeh and Moham-
madi 2017; Lehký et al. 2022), in which ANN-based SMM
is for reliability analysis and SOS is the optimizer (here-
after collectively referred to as the “ANN-SMM-SOS”).
For the 4MP-PSO/SOS, results are directly acquired from
the literature. In contrast, results of the ANN-SMM-SOS
are reproduced by the current study based on the frame-
work outlined in the reference (Gholizadeh and Moham-
madi 2017). Note that Gholizadeh and Mohammadi (2017)
adopted PSO as their optimizer, instead of using PSO, this
study integrated ANN-SMM with SOS for fair compari-
son. Further, the number of iteration and organism size are
Page 15 of 25 119
Fig. 16  Illustration of the ten bar truss problem (Case 2)
identical for different approaches (e.g., ANN-SMM-SOS
and ANNM-IS-SOS).
As indicated in Table 5, the proposed ANNM-IS-SOS
and the 4MP-PSO/SOS with trivariate give the small-
est error percentage. For ANN-SMM-SOS, lower error
percentage can be obtained when a larger size of train-
ing data is used. As shown, the error percentage reduced
from 5.33% to 3.33% when function evaluation number for
each reliability analysis increased from 175 to 250. For the
aspects of efficiency, it is seen that the proposed ANNM-
IS-SOS has the minimum number of function evaluation
for each reliability analysis due to the training data for NN
are carefully selected.
4.3 Part II: RBDO, case 2 (ten bar truss)
In this case, problem with an implicit limit state function
with a higher number of random variables is investigated
using the proposed RBDO framework. Ten-bar truss is
a common engineering RBDO benchmark problem. Fig-
ure 16 illustrates the ten-bar truss with two external load-
ings. The objective is to minimize the weight of the truss
structure and the mathematical formulation is given in
Eq. (11).
13
119 Page 16 of 25 J. Thedy, K.-W. Liao

Table 6  Random Design Variables (Top One) and Random Design random variables which all follows the normal distribution.
Parameters (P and E) in the Ten Bar Truss Case The area, A1-10, are random design variables and the rest
Description Symbol Mean COV1 Distribution Unit are random design parameters. The external load, P, has a
mean of 1­ 05 lb and the modulus of elasticity, E, has a mean
Area of truss member A1…10 – 0.05 Normal in2
of ­107 psi. The COVs of the 12 random variables are all
Loading P 1 × ­105 0.05 Normal lb
equal to 0.05.
Elastic modulus E 1 × ­107 0.05 Normal psi
70 iterations and 20 organisms are used for the proposed
Height a 720 – – inch
ANNM-IS-based RBDO task. Figure 17 shows the NN accu-
Width b 360 – – inch
racy and trained data size. A greater number of trained data
1
COV: coefficient of variation is expected as the organism number increases. NN accuracy
is stable with a pretty high accuracy, in which the accuracy
is consistently above 95% after the 4­ 0th iteration. Figure 18
∑
10 displays the required training time, as shown, computation is
Min. w = 𝜌 Ai Li ,i = 1, 2, ⋯ , 10 increased due to a significant number of random and design
variables as presented in Table 6. Table 7 summarizes the
s.t. Pr ob[G(d, x) ≤ 0] ≤ Φ(−𝛽 T )
(11)
i=1

RBDO results. The optimal design (organism) has reliabil-

0.10 ≤ Ai ≤ 35.0
where ρ is the material density that is equal to 0.1 lb/
in3, Ai is the cross area of each truss member, Li is the
length of each truss member, G is the limit state function,
x is a vector of random design parameters as shown in
Table 6, d = [A1 , A2 , ⋯ , A10 ]T , G(d, x) = u2 − 2(inch), u2 is
the vertical displacement at point 2 as shown in Fig. 16, βT
is the target reliability index (i.e., 3.0). Parameters of a and b
in Fig. 16 are 720 and 360 (inch), respectively. As indicated
in Eq. (11) and Fig. 16, this ten-bar truss problem has 12
ity index that has 2% error percentage compared to that of
MCS of 1­ 06 sample, which is an acceptable design. The
convergence history and FE at each iteration are depicted in
Figs. 19 and 20. Comparison results are displayed in Table 7,
indicating that the proposed ANNM-IS-SOS achieves the
best balance between the error percentage and the number of
function evaluation. It is seen that although 4MP-PSO/SOS
has the smallest error percentage; its cost is almost ten times
of the proposed ANNM-IS-SOS. The cost of ANN-SMM-
SOS is similar to the proposed ANNM-IS-SOS. However,
its accuracy is quite different from our proposed method.

Fig. 17  Training data size and

NN accuracy at each iteration
for case 2

Fig. 18  NN training time at

each iteration for case 2

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 17 of 25 119

Table 7  RBDO comparisons for case 2

Description Proposed ANNM-IS-SOS ANN-SMM-SOS 4MP-PSO/SOS (Bivariate)

Best Organism (­ in2) A1-10 = [34.13, 0.11, 23.42, 16.17, 0.10, A1-10 = [35, 0.34, 23.63, 15.67, 0.15, A1-10 = [35, 0.10, 27.83, 20.17,
0.12, 3.26, 34.96, 33.31, 0.10] 0.17, 6.08, 35.00, 35.00, 0.10] 0.10, 0.10, 3.37, 29.01, 28.43,
0.10]
Fitness (lb) 18306.55 19073.34 17688.03
βBest Fitness 3.00 3.00 3.00
βMCS 3.07 3.32 2.96
Error Percentage (%) 2.33 10.52 1.26
Average FE per RA 305.6 357.14 3319

Fig. 19  Converged history of

fitness and reliability index for
case 2

m6

k6
m5

k5
m4

k4
m3

k3
Fig. 20  Required function evaluation for ANNM-IS in each iteration m2
in Case 2
k2
m1
4.4 Part II: RBDO, case 3 (LQR structural control)
k1
Figure 21 displays a 6-story structure with active controller at
the base, adopted from Ramallo et al. (2002), is used here to
demonstrate the proposed RBDO framework. The mass, stiff-
ness, and damping ratio of the 6-story structure are displayed Fig. 21  A 6-story structure with active controller at the base
in Fig. 21 and are slightly modified to form the RBDO prob-
lem in this study. Figure 22 provides a general scheme of struc-
tural active control. It is seen that to calculate the actuator force For each time increment, acceleration data are collected at
(F(t)), the accelerations at each story, the Kalman filter, the each floor and used as input for the Kalman Filter to predict
state matrix and linear quadratic regulator (LQR) are needed. structure state (i.e., displacement and velocity). The obtained

13
119 Page 18 of 25 J. Thedy, K.-W. Liao

Note :
mi = Mass at Story i m6
ki = Stiffness at Story i k6
Collect Accelerations from
Each Story at Interval Time
m5 Estimate State Using
Multi Degree of Kalman Filter
k5
Freedom Structure Noise
m4
Accelerometer at k4
Each Story State Matrix
m3

k3
Calculate F = K x State matrix
m2

k2 Optimize Control Parameters in K Matrix Using

m1 the Proposed ANNM-IS based RBDO
Send Command to Actuator to
k1
Generate Force Equivalent to F

Earthquake

Fig. 22  General scheme of a structural active control

�
state at each time increment will be multiplied by LQR param- �
� T2
eters, yielding an actuator force acting on structure to reduce �
=�
T2 − T1 �
1
Min.FRMS � [F(t)]2 dt
earthquake effect. This actuator force is consistently generated
at each time increment depending on recorded accelerations
T1
�
and LQR parameters (K). This K matrix is constructed based ⎡ � ⎤
� T2

u6 (t) dt < 0.012m⎥ ≤ Φ −𝛽t

on structural properties (e.g., mass, damping and stiffness) and
weighting coefficients such as Q and R as shown in Eq. (12).
Q is a matrix of 2n × 2n, in which n is the number of story,
qai and qvi (i = 1,2,…,6; j = 1,2,…,6) are the weighting coef-
ficients to penalize/control state variable, r is weighting coef-
ficient to penalize controller’ force. Detailed theory of LQR
active control regarding to calculation of the K matrix using
structural information and weighting matrices (Q and R) can
be found in reference (Liao and Thedy 2021). The objective
of this case study is to minimize the generated controller force
(F(t)), in which Root Mean Square (RMS) is adopted as the
indicator. The probabilistic constraint is the top story displace-
ment should be less than 0.012 m with reliability of 99.867%.
The mathematical formulation is given in Eq. (13).
(12)
⎢ � � �2 ⎥ � �
=�
�
s.t.Prob⎢u6RMS �
1
⎢ T2 − T1 ⎥
⎣ T1 ⎦
(13)
where (T2-T1) is the time interval considered, F(t) is the
actuator force, u6 is the top story displacement, Φ () is cumu-
lative probability density function of Gaussian distribution
and β is target reliability index. There are 14 random vari-
ables considered here and their statistics are described in
Table 8, in which noise amplitude is random design param-
eter and rest of them are random design variables. As seen,
all random variables are assumed to follow lognormal dis-
tribution with COV of 0.1. El Centro earthquake with Peak
Ground Acceleration (PGA) of 1 g is adopted as the external
excitation, as presented in Fig. 23. The state space formula-
tion, performed by MATLAB SIMULINK, is used to model
the structure, in which Rayleigh damping matrix is adopted
(Liu and Gorman 1995). The computer used here is i7 with
3.2 GHz CPU. More detailed explanations of the adopted
state space modelling can be found in reference (Liao and
Thedy 2021).
30 iterations and 20 organisms are used for the proposed
ANNM-IS-based RBDO task. The upper and lower bounds
of the qai (as shown in Eq. (12)) are 1­ 03 and 0, respectively.
The upper and lower bounds of the qvi (as shown in Eq. (12))
are ­102 and 0, respectively. The upper and lower bounds of r
are ­10–7 and 1­ 0–10, respectively. In total, there are 13 random
design variables and 1 random design parameters, as indi-
cated in Table 8. That is, the NN model needs the capability

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 19 of 25 119

Table 8  Statistics of random design variables (top seven) and random top story displacements. In this case, 4MP-PSO/SOS is
design parameter (bottom one) considered in this study not performed and compared because it requires a quite
Random Variable Mean COV1 Unit Dist. Type large number of sample size. To be specific, for a problem
with 14 random variables, 4MP-PSO/SOS requires 4558
m1, ­m2, ­m3, ­m4, ­m5, ­m6 5897 0.1 kg Lognormal
samples for a single reliability analysis. Therefore, only
k1, ­k2 33732 kN/m
ANN-SMM-SOS is conducted for comparison. As shown
k3 29093 kN/m
in Table 9, it is seen that under the similar cost (i.e., simi-
k4 28621 kN/m
lar sample size for a single reliability analysis), the pro-
k5 24954 kN/m
posed ANNM-IS-SOS is capable of delivering a promis-
k6 19059 kN/m
ing design. In contrast, the ANN-SMM-SOS fails to give a
ζ 2 %
satisfied design, in which the reliability of optimal design
Noise amplitude 1 m/s2
is far from that of MCS.
1
COV: coefficient of variation
4.5 Part II: RBDO, case 4 (25‑bar truss)

A 25-bar transmission tower (Camp and Farshchin 2014) is

Fig. 23  Earthquake time histories used in the numerical example
of handling 27 different input data. Figure 24 shows the opti-
mization convergence history. It is seen that objective/fitness
has reached a pretty stable status after 25 iterations, while
the reliability index still fluctuates slightly. The RMS of
the actuator force is converged 109.51kN with reliability of
3.03, as shown in Table 9. Table 9 also displays the optimal
LQR parameters obtained from the proposed ANNM-IS-
based RBDO. To verify the accuracy, MCS of ­106 samples
is used to calculate the reliability index for the final organ-
ism. As indicated in Table 9, the proposed ANNM-IS can
deliver a promising estimation of reliability index, in which
the error percentage is around 1.67%. Table 9 also indicates
that, in average, each reliability analysis takes 398 samples
that is much less than ­106 samples.
Figure 25a shows the top story displacements before and
after control.
Figure 25b depicts the generated actuator force at each
time step. As expected, the actuator designed by the pro-
posed ANNM-IS-based RBDO significantly reduces the
adopted to investigate the suitability of the proposed method
in a highly dimensional problem. Figure 26 shows the 25-bar
truss configuration. It consists of 10 nodes with four pinned
supports with coordinates shown in Table 10. Nodes 1, 2, 3,
and 6 sustain an external concentrated load in three direc-
tions as shown in Table 11. The objective, as indicated in
Eq. 14, is to minimize total steel weight of the bar structure.
Two types of constraints formulated in Eq. 14 are applied, in
which G1 and G2 are for stress and displacement constraints,
respectively. Each bar is required to have stress level that
is less than 35 Ksi with reliability index of 3.0. Similarly,
each node is required to have displacement in each direc-
tion that is less than 0.35 with reliability index of 3.0. That
is, 31 limit states are considered. As shown in Table 12, 25
random design variables and 1 random design parameter are
considered. All random variables follow normal distribution
with COV of 0.05. Same as previous cases, trained data size
and NN accuracy at each iteration is recorded in Fig. 27 and
the training time is recorded Fig. 28. Although this case con-
sists of 26 random variables, results show that training time
is still in a reasonable range. The convergence history and
number of function evaluation at each iteration are depicted
in Figs. 29 and 30.
To obtain a satisfied accuracy, the number of required
function evaluation is investigated. As shown in Table 13,
with 280 samples for each reliability analysis, the proposed
ANNM-IS-SOS is able to provide an acceptable accuracy
result with 3.25% error compared to that of MCS. In con-
trast, the ANN-SMM-SOS fails to give a satisfied design
under the similar cost. Even increasing to 4000 samples
for each reliability analysis, the ANN-SMM-SOS still has
much higher error percentage than that of the proposed
method. This indicates that although ANNM-IS-SOS
requires a larger sample size as number of random variable
increases, but the increasing rate is not as fast as that of the
ANN-SMM-SOS.

13
119 Page 20 of 25 J. Thedy, K.-W. Liao

Fig. 24  a Convergence history

of fitness and reliability index
for case 3. b required FE in each
iteration for case 3

(a)

(b)

Table 9  RBDO comparisons Description Proposed ANNM-IS-SOS ANN-SMM-SOS

for case 3
Best Organism qa1-qa6 = [171.3, 378.3, 785.8, 653.3, qa1-qa6 = [253.17, 54.06,
984.5, 789.2] 923.55, 931.78, 955.54,
qv1-qv6 = [0, 1.4, 0.4, 84.41, 72.5, 0] 540.88]
R = 1.073 × ­10–10 qv1-qv6 = [0, 97.55, 28.85,
97.11, 96.35, 32.62]
R = 1.00 × ­10–10
Fitness (kN) 109.51 127.65
βBest Fitness 3.03 3.00
βMCS 2.98 -Inf
Error Percentage (%) 1.67 –
Average FE per RA 398.7 416.67

∑
25 5 Conclusion
Min.w = 𝜌 Ai Li

s.t. P G1 (𝐝, 𝐱) ≤ 0 ≤ Φ −𝛽 T
[
i=1
] ( ) This study introduces an innovative RBDO framework for
(14) engineering problems. For example, the controller force in
P G2 (𝐝, 𝐱) ≤ 0 ≤ Φ −𝛽 T
[ ] ( )
a 6-story structure is minimized considering uncertainties
0.10 ≤ Ai ≤ 4.50 in mass, stiffness, damping ratio and earthquakes. In the
proposed RBDO, ANNM-IS is fully integrated with the
heuristic M-IS reliability analysis and SOS optimization
algorithm. The two-step SOS metaheuristic algorithm,
including exploration and exploitation, is designed here to

13
Reliability‑based structural optimization using adaptive neural network multisphere… Page 21 of 25 119

Fig. 25  a Top displacements before and after applying active controller b Controlling Force

Table 11  External loadings for nodes 1, 2, 3 and 6

Node Px (kips) Px (kips) Px (kips)

1 1.0 10.0 − 5.0

2 0.0 10.0 − 5.0
3 5.0 0.0 0.0
6 5.0 0.0 0.0

Table 12  random design variables (top one) and random design

parameters (bottom one) considered in case 4
Description Symbol Mean COV1 Distribution Unit

Area of truss member A1…25 – 0.05 Normal in2

Elastic modulus E 1 × ­107 0.05 Normal psi

enhance the search efficiency. The first step of SOS aims to

find the minimum difference between organisms’/swarms’
Fig. 26  25-Bar truss configuration reliability and target reliability (βt) using ECOVPf of 0.2.
The second step of SOS is to find the minimum cost when
a promising region has been identified, that is, 70% of the
organisms have reliability indices near βt. The proposed
Table 10  Coordinates for 25-bar truss ANNM-IS embeds NN into the uncertainty assessment
Node x (in) y (in) z (in) process to find the adaptive points and then reduce the
computational cost of reliability analysis. M-IS supplies
1 − 37.5 0.0 200.0
quality data as training input for NN, while NN helps M-IS
2 37.5 0.0 200.0
to reduce the cost of function evaluation. To further reduce
3 − 37.5 37.5 100.0
the computational burden, ANNM-IS is integrated with
4 37.5 37.5 100.0
the optimization process and utilizes samples in the pre-
5 37.5 − 37.5 100.0
vious iteration to update NN for next iteration. As itera-
6 − 37.5 − 37.5 0.0
tion proceeds, the search area is progressively moved to
7 − 100.0 100.0 0.0
the optimal area and the search size is gradually reduced
8 100.0 100.0 0.0
that makes the updated NN become more accurate. As a
9 100.0 − 100.0 0.0
result, efficiency in reliability analysis is greatly enhanced
10 − 100.0 − 100.0 0.0
in terms of function evaluation.

13
119 Page 22 of 25 J. Thedy, K.-W. Liao

Fig. 27  Training data size and

NN accuracy at each iteration
for case 4

Fig. 28  NN training time at

each iteration for case 4

Fig. 29  Converged history of

fitness and reliability index for
case 4

Fig. 30  Required function

evaluation for ANNM-IS in
each iteration in case 4

13
Reliability‑based structural optimization using adaptive neural network multisphere…
Table 13  RBDO Comparisons for Case 4
Description Proposed ANNM-IS-SOS
Best organism (­ in2) A1-25 = [0.07, 1.65, 0.01, 0.55, 0.01,
4.50, 4.20, 4.05, 3.80, 1.65, 3.17, 2.72, 4.49, 3.06, 4.50, 2.87, 0.01, 0.02, 0.07, 2.45, 2.21, 4.25, 4.06, 0.72, 2.57,
0.80, 1.13, 0.07, 0.46, 2.89, 0.48, 0.59, 0.01, 1.32, 0.02, 1.28, 0.03, 0.67, 1.60, 0.23, 0.98, 1.44, 0.15, 1.02, 0.18,
1.41, 0.19, 4.47, 3.92, 4.48, 3.89]
Fitness (lb) 623.44
βBest Fitness 3.00
βMCS 3.10
Error Percentage (%) 3.25
Average FE per RA 277.7
To demonstrate the proposed ANNM-IS-SOS, four
reliability problems are first used to confirm the accu-
racy and efficiency of the proposed ANNM-IS, then four
engineering cases are solved using the proposed ANNM-
IS-SOS. Results indicates that the proposed ANNM-IS-
SOS is shown to be capable of solving a broad spectrum
of RBDO problem such as non-normal random variable
(case 1), nonlinear (case 1) and implicit limit state func-
tions (case 2), seismic optimization and structural active
controller design (case 3), and high dimension problem
(case 4). Case 1 is a short column under axial and moment
loadings with 6 random variables but nonlinear limit state
and non-normal situation are considered. Case 2 inves-
tigates a greater size of RBDO problem with 12 random
variables and 10 design variables. More complex struc-
tural problem, a structural controller problem, is inves-
tigated in case 3. Case 4 is aimed to investigate the suit-
ability of the proposed method on a highly dimensional
problem. The applicability of the proposed method on
multiple failure domains is investigated in two aspects.
The first aspect is with respect to the reliability analy-
sis, as indicated in the section of 4.1, in which 4 limit
states are considered in case D. The second aspect is with
respect to the RBDO problem, as indicated in the section
of 4.5, in which 31 limit states are considered in case 4.
Both cases reveal that the proposed method is suitable for
problem with multiple failure domains. It is believed that
this study has investigated many aspects of a real-world
complicated structure.
As shown, the proposed ANNM-IS-SOS delivers a
very promising design for four engineering cases. For
example, the number of function evaluation needed for
each reliability analysis are approximately 160, 300, 400
and 300, respectively. The reliability error at each optimal
design are 0.03%, 2.2%, 1.67% and 3.25%, respectively.
The proposed ANNM-IS-SOS is capable of providing an
accurate optimal design with affordable price.
Page 23 of 25 119
ANN-SMM-SOS
A1-25 = [0.01, 2.66, 1.09, 0.24, 1.22, A1-25 = [0.61, 0.62, 2.92, 2.44, 4.31,
1.62, 0.01, 4.27, 4.50, 4.48, 3.17] 1.89, 2.38, 1.14, 0.32, 4.38, 3.58,
4.18, 3.21]
565.75 671.1317
3.00 3.00
-Inf 2.46
– 18.00
285.71 4167.77
Funding This study was supported by the Ministry of Science and
Technology of Taiwan under grant number MOST 108-2621-M-002
-020 -MY3. The support is gratefully acknowledged.
Declarations
Conflict of interest The authors declare that they have no conflict of
interest.
Replication of results The supporting source codes are available for
download at https://g​ ithub.c​ om/j​ ohnth​ edy/R
​ BDO-u​ sing-M
​ IS-N
​ N-S
​ OS.
Ethical approval This article does not contain any studies with human
participants or animals performed by any of the authors.
References
Ahn J, Kwon JH (2004) Sequential approach to reliability analysis
of multidisciplinary analysis systems. Struct Multidisc Optim
28(6):397–406
Ahn J, Kwon JH (2006) An efficient strategy for reliability-based mul-
tidisciplinary design optimization using BLISS. Struct Multidisc
Optim 31(5):363–372
Alibrandi U, Alani AM, Ricciardi G (2015) A new sampling strategy
for SVM-based response surface for structural reliability analysis.
Probab Eng Mech 41:1–12
Ang George L, Ang Alfredo HS, Tang Wilson H (1992) Optimal impor-
tance-sampling density estimator. J Eng Mech 118(6):1146–1163
Asghar MZ, Abbas M, Zeeshan K, Kotilainen P, Hämäläinen T (2019)
Assessment of deep learning methodology for self-organizing 5G
networks. Appl Sci 9(15):2975
Au SK, Beck JL (2001) Estimation of small failure probabilities
in high dimensions by subset simulation. Probab Eng Mech
16(4):263–277
Azad SK (2021) Design optimization of real-size steel frames
using monitored convergence curve. Struct Multidisc Optim
63(1):267–288
Balesdent M, Morio J, Marzat J (2013) Kriging-based adaptive impor-
tance sampling algorithms for rare event estimation. Struct Saf
44:1–10
Bucher CG (1988) Adaptive sampling — an iterative fast Monte Carlo
procedure. Struct Saf 5(2):119–126
13
119 Page 24 of 25
Bucher CG, Bourgund U (1990) A fast and efficient response surface
approach for structural reliability problems. Struct Saf 7(1):57–66
Cadini F, Santos F, Zio E (2014) An improved adaptive kriging-based
importance technique for sampling multiple failure regions of low
probability. Reliab Eng Syst Saf 131:109–117
Camp CV, Farshchin M (2014) Design of space trusses using modified
teaching–learning based optimization. Eng Struct 62–63:87–97
Cheng MY, Prayogo D (2014) Symbiotic organisms search: a new
metaheuristic optimization algorithm. Comput Struct 139:98–112
Ching J, Chen YC (2007) Transitional Markov chain monte Carlo
method for Bayesian model updating, model class selection, and
model averaging. J Eng Mech 133(7):816–832
Ching J, Phoon KK (2013) Quantile value method versus design value
method for calibration of reliability-based geotechnical codes.
Struct Saf 44:47–58
Datta G, Bhattacharjya S, Chakraborty S (2020) Efficient reliability-
based robust design optimization of structures under extreme
wind in dual response surface framework. Struct Multidisc Optim
62(5):2711–2730
Der Kiureghian A, Lin HZ, Hwang SJ (1987) Second-order reliability
approximations. J Eng Mech 113(8):1208–1225
Dolinski K (1982) First-order second-moment approximation in
reliability of structural systems: critical review and alternative
approach. Struct Saf 1(3):211–231
Du X, Chen W (2004) Sequential optimization and reliability assess-
ment method for efficient probabilistic design. J Mech Des
126(2):225–233
Echard B, Gayton N, Lemaire M (2011) AK-MCS: AN active learning
reliability method combining Kriging and Monte Carlo Simula-
tion. Struct Saf 33(2):145–154
Gholizadeh S, Mohammadi M (2017) Reliability-based seismic opti-
mization of steel frames by metaheuristics and neural networks.
ASCE-ASME J Risk Uncertainty Eng Syst Part a: Civ Eng
3(1):04016013
Grooteman F (2011) An adaptive directional importance sampling
method for structural reliability. Probab Eng Mech 26(2):134–141
Harbitz A (1986) An efficient sampling method for probability of fail-
ure calculation. Struct Saf 3(2):109–115
Kang F, Han S, Salgado R, Li J (2015) System probabilistic stability
analysis of soil slopes using Gaussian process regression with
Latin hypercube sampling. Comput Geotech 63:13–25
Katafygiotis LS, Zuev KM (2007) Estimation of small failure prob-
abilities in high dimensions by adaptive linked importance sam-
pling. Proc. Compdyn-2007.
Lee I, Choi KK, Noh Y, Zhao L, Gorsich D (2011) Sampling-based
stochastic sensitivity analysis using score functions for RBDO
problems with correlated random variables. J Mech Des
10(1115/1):4003186
Lee U, Lee I (2021) Sampling-based weighted reliability-based design
optimization. Struct Multidisc Optim 65(1):20
Lehký D, Šomodíková M, Lipowczan M (2022) A utilization of the
inverse response surface method for the reliability-based design
of structures. Neural Comput Appl 34(15):12845–12859
Li HS, Lü ZZ, Yue ZF (2006) Support vector machine for structural
reliability analysis. Appl Math Mech 27(10):1295–1303
Liao KW, Biton NIDR (2019) A heuristic optimization considering
probabilistic constraints via an equivalent single variable Pearson
distribution system. Appl Soft Comput 78:670–684
Liao KW, Thedy J (2021) Probabilistic optimal control parameters with
incomplete information on design variables using a heuristic algo-
rithm. Appl Soft Comput 110:107586
Liu M, Gorman DG (1995) Formulation of rayleigh damping and its
extensions. Comput Struct 57:277–285
Lopez RH, Lemosse D, Cursi ES, Rojas JE, El-Hami A (2011) An
approach for the reliability based design optimization of lami-
nated composite plates. Eng Optim 43(10):1079–1094
13
J. Thedy, K.-W. Liao
Martino L, Elvira V, Luengo D, Corander J (2017) Layered adaptive
importance sampling. Stat Comput 27(3):599–623
Melchers RE (1989) Importance sampling in structural systems.
Struct Saf 6:3–10
Meng Z, Li G, Wang X, Sait SM, Yıldız AR (2021) A Compara-
tive study of metaheuristic algorithms for reliability-based
design optimization problems. Arch Comput Methods Eng
28(3):1853–1869
Olsson A, Sandberg G, Dahlblom O (2003) On Latin hypercube sam-
pling for structural reliability analysis. Struct Saf 25(1):47–68
Panagant N, Bureerat S, Tai K (2019) A novel self-adaptive hybrid
multi-objective meta-heuristic for reliability design of trusses with
simultaneous topology, shape and sizing optimisation design vari-
ables. Struct Multidisc Optim 60(5):1937–1955
Papaioannou I, Papadimitriou C, Straub D (2016) Sequential impor-
tance sampling for structural reliability analysis. Struct Saf
62:66–75
Papaioannou I, Breitung K, Straub D (2018) Reliability sensitivity esti-
mation with sequential importance sampling. Struct Saf 75:24–34
Ramallo JC, Johnson EA, Spencer BF (2002) Smart base isolation
systems. J Eng Mech 128(10):1088–1099
Rosenblatt F (1958) The perceptron: a probabilistic model for
information storage and organization in the brain. Psycol Rev
65(6):386–408
Rubinstein RY (1981) Simulation and the Monte Carlo Method. John
Wiley, New York
Thedy J, Liao KW (2021) Multisphere-based importance sampling for
structural reliability. Struct Saf 91:102099
Truong VH, Kim SE (2017) An efficient method for reliability-based
design optimization of nonlinear inelastic steel space frames.
Struct Multidisc Optim 56(2):331–351
Wang L, XiongC WX, Liu G, Shi Q (2019) Sequential optimization
and fuzzy reliability analysis for multidisciplinary systems. Struct
Multidisc Optim 60(3):1079–1095
Werbos PJ (1991) An overview of neural networks for control. IEEE
Control Syst Mag 11(1):40–41
Xiao NC, Yuan K, Zhou C (2020) Adaptive kriging-based efficient
reliability method for structural systems with multiple failure
modes and mixed variables. Comput Methods Appl Mech Eng
359:112649
Yang D (2010) Chaos control for numerical instability of first order
reliability method. Commun Nonlinear Sci Numer Simul
15(10):3131–3141
Yang M, Zhang D, Han X (2020) New efficient and robust method
for structural reliability analysis and its application in reliability-
based design optimization. Comput Methods Appl Mech Eng
366:113018
Yang M, Zhang D, Jiang C, Han X, Li Q (2021) A hybrid adaptive
Kriging-based single loop approach for complex reliability-based
design optimization problems. Reliab Eng Syst Saf 215:107736
Yang M, Zhang D, Wang F, Han X (2022) Efficient local adaptive
Kriging approximation method with single-loop strategy for reli-
ability-based design optimization. Comput Methods Appl Mech
Eng 390:114462
Yao W, Chen X, Ouyang Q, Tooren MV (2013) A reliability-based
multidisciplinary design optimization procedure based on com-
bined probability and evidence theory. Struct Multidiscip Optim
48(2):339–354
Youn BD, Choi KK (2003) An investigation of nonlinearity of
reliability-based design optimization approaches. J Mech Des
126(3):403–411
Youn BD, Xi Z, Wang P (2008) Eigenvector dimension reduction
(EDR) method for sensitivity-free probability analysis. Struct
Multidiscip Optim 37(1):13–28
Reliability‑based structural optimization using adaptive neural network multisphere…
Yun W, Lu Z, Jiang X, Zhang L, He P (2020) AK-ARBIS: An improved
AK-MCS based on the adaptive radial-based importance sampling
for small failure probability. Struct Saf 82:101891
Zhao YG, Lu ZH (2007) Applicable range of the fourth-moment
method for structural reliability. J Asian Archit Build Eng
6(1):151–158
Zhao YG, Ono T (1999) A general procedure for first/second-order
reliability method (FORM/SORM). Struct Saf 21(2):95–112
Page 25 of 25 119
Publisher's Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Springer Nature or its licensor (e.g. a society or other partner) holds
exclusive rights to this article under a publishing agreement with the
author(s) or other rightsholder(s); author self-archiving of the accepted
manuscript version of this article is solely governed by the terms of
such publishing agreement and applicable law.
13