ARTICLE IN PRESS

Computers and Geotechnics xxx (2007) xxx–xxx

www.elsevier.com/locate/compgeo

Performance studies on six heuristic global optimization methods

in the location of critical slip surface
a,* a,b b
Y.M. Cheng , L. Li , S.C. Chi
a
Department of Civil and Structural Engineering, Hong Kong Polytechnic University, Hong Kong
b
School of Civil and Hydraulic Engineering, Da Lian University of Technology, China

Received 10 October 2006; received in revised form 14 December 2006

Abstract

Recently, many heuristic global optimization algorithms have evolved with success for treating various types of problems. Majority of
these algorithms have not been applied to slope stability problem for which the presence of soft band and convergence problem (discon-
tinuity of the objective function) may create difficulties in the minimization process. In this paper, six heuristic optimization algorithms
are applied to some simple and complicated slopes. The effectiveness and efficiency of these algorithms under different cases are evalu-
ated, and it is found that no single method can outperform all the other methods under all cases, as different method has different behav-
ior in different types of problems. For normal cases, the particle swarm method appears to be effective and efficient over various
conditions, and this method is recommended to be used. For special cases where the objective function is highly discontinuous, the sim-
ulated annealing method appears to be a more stable solution.
 2007 Elsevier Ltd. All rights reserved.

Keywords: Heuristic; Global optimization; Critical; Factor of safety; Control variable

1. Introduction 1. The objective function of the safety factor f is usually

Slope stability problem is critical in many developed cit-
ies, as slopes are formed for highways and buildings con-
struction. For example, billion of dollars (US) are spent
annually in Hong Kong for the slope stabilization works.
There is a strong need among the engineers to perform
accurate and efficient slope analyses with reduced efforts,
and there are many active research works in slope stability
analysis and stabilization works.
For general non-circular failure surface, the number of
control variables is much more than 3. To locate the critical
failure surface, the geometric method similar to that for cir-
cular failure surface will be impractical in application.
Some of the difficulties in the location of the general critical
non-circular failure surfaces are:
*
Corresponding author. Tel.: +852 27666042; fax: +852 23346389.
E-mail address: ceymchen@polyu.edu.hk (Y.M. Cheng).
non-smooth, non-convex and discontinuous over the
solution domain. ‘Failure to converge’ (discontinuity
for function f) may happen for complicated ground con-
ditions with external loads, which is equivalent to f
being discontinuous at discrete solution sub-domains.
Gradient type optimization methods or even some opti-
mization methods may break down under this situation,
which will be demonstrated by an interesting case from
Beijing in the latter section.
2. Chen and Shao [10] have found that multiple minima
will exist in general, and many classical optimization
methods can be trapped easily by the existence of the
local minima. The capability of an optimization algo-
rithm to escape from the local minimum close to the ini-
tial trial is crucial for the success in locating the critical
failure surface. The case studies in the present paper will
illustrate the importance of the local minima in the glo-
bal minimization analysis.

0266-352X/$ - see front matter  2007 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compgeo.2007.01.004

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
2 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
3. A good initial trial failure surface for the global minimi-
zation analysis is usually difficult to be estimated for
general conditions with arbitrary loadings and
geometry.
Optimization search for non-circular failure surface has
been considered by various classical methods: variation
method by Baker and Gaber [3], simplex method by Ngu-
yen [28], alternating variable method by Celestino and
Duncan [8], conjugate-gradient method by Arai and Tagyo
[2], dynamic programming by Yamagami and Jiang [32],
simplex method by Chen and Shao [10]. These classical
methods are applicable mainly to continuous functions
and are limited by the presence of the local minimum, as
the local minimum close to the initial trial will be obtained
by these methods. There is also a possibility that the global
minimum within the solution domain is not given by the
condition that the gradient of the objective function
$f 0 = 0, and a good example has been illustrated by Cheng
[12]. The presence of the other local minima or the global
minimum will not be obtained unless a good initial trial
is adopted, but a good initial trial is difficult to be estab-
lished for a general problem.
In view of the limitations of the classical optimization
methods, the current approach to locate the critical failure
surface is the adoption of heuristic global optimization
methods. The term heuristic is used for algorithms which
find solutions among all the possible ones, but they do
not guarantee that the best will be found, therefore, they
may be considered as approximate and not accurate algo-
rithms. These algorithms usually find a solution close to
the best one, and they find it fast and easily. Greco [23]
and Malkawi et al. [27] have adopted the Monte Carlo
technique for locating the critical slip surface with success
for some cases, but there is no precision control on the
accuracy of the global minimum. Zolfaghari et al. [34]
adopted the genetic algorithm while Bolton et al. [6] used
the leap-frog optimization technique to evaluate the mini-
mum factor of safety. All of the above methods are based
on the use of static bounds to the control variables, which
means that the solution domain for each control variable is
fixed and is pre-determined by engineering experience.
Cheng [12] has developed a procedure which transformed
the various constraints and the requirement of a kinemati-
cally acceptable failure mechanism to the evaluation of
upper and lower bounds of the control variables, and the
simulated annealing algorithm is used to determine the crit-
ical slip surface. The control variables are defined with
dynamic domains which are changing during the solution,
and the bounds are controlled by the requirement of a kine-
matically acceptable failure mechanism. Through such
approach, there is no need to define the pre-determined sta-
tic solution domain to each control variable by engineering
experience.
Global optimization problems are typically difficult to
be solved, and in the context of combinatorial problems,
they are often N-P hard type. Some of the special require-
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
ments for a robust and versatile global optimization algo-
rithm in slope stability which are not present in other
type of problems include: (1) discontinuity of the objective
function in some solution sub-domains; (2) presence of
multiple local minima which has been demonstrated by
Chen and Shao [10]; (3) soil parameters, ground condition,
external loads can vary over a wide range so that the choice
of the optimization parameters are difficult to be estab-
lished; (4) the number of control variables can be very large
and the solution time can be long if a refined analysis is
required or the size of the solution domain is large.
With the development of the computer software and
hardware, many artificial intelligent based algorithms based
on natural selection and the mechanisms of population
genetics are developed. These algorithms are commonly
applied in pattern recognition, electronic, production/con-
trol engineering or signal processing system. These new heu-
ristic optimization algorithms have been successfully
applied to many different disciplines for both continuous
and discrete optimization problems, but there are only lim-
ited uses of these methods in slope stability problems.
In the present study, six different heuristic optimization
methods are applied to determine the critical slip surfaces.
It is found that no single method can outperform all the
other methods under all cases, as different method has dif-
ferent behavior in different types of problems. In general,
the particle swarm method appears to be more stable while
the simulated annealing method is found to be more stable
for problems where the objective functions are highly
discontinuous.
2. Generation of trial slip surfaces
To locate the critical non-circular failure surface, a trial
slip surface generation algorithm is required. Consider the
Cartesian system of reference Oxy as shown in Fig. 1.
Function y = y1(x) describes the topographic profile of
the slope, while the water table is represented by
y = w(x). The bed rock surface is represented by the func-
tion y = R(x) and function y = li(x) can be introduced to
represent boundary between different soils. The slip surface
is described using the function y = s(x).
Fig. 1. A slip surface and the cross-section of a slope.
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 3

y xU
xL y6max xn+1
y5max
y4max
y=y1(x)
y3max
xu xn,yn
xl x1
y2max i+1
i
x2,y2
y=R(x)
y3min y4min y5min y6min
y2min
O x
Fig. 2. The control variables for a slip surface.

The method of slices requires the failure soil mass to be which is proposed by Cheng [12]. As shown in Fig. 3, if the
divided into n vertical slices, and the slip surface is repre- values of x1 to xn+1 are defined, y2min, y2max are then deter-
sented by n + 1 vertices [V1, V2, . . ., Vn+1] with coordinates mined by the geometry and the bed rock line. y2 is ran-
(x1, y1), (x2, y2), . . ., (xn+1, yn+1). Each slice can be identified domly generated within the range [y2min, y2max]. The line
by two adjacent vertices. Generally speaking, the potential between Vn+1 and V2 intersect with the line x = x3 at point
slip surfaces are concave upward (kinematically acceptable G with the y-ordinates denoted as yG, and the line passing
requirement) with only few exceptions. The concave through V1 and V2 is extended to intersect with the line
upward requirement can be formulated as follows: x = x3 at point H with the y-ordinates yH determined.
a1 6 a2 6    6 an ð1Þ y3min, y3max can then be determined using the following
equation:
where ai represents the inclination of slice i as shown in
y 3min ¼ maxfy H ; Rðx3 Þg y 3max ¼ minfy G ; y 1 ðx3 Þg ð4Þ
Fig. 1. Every slip surface can be identified by the control
variable vector X: y4min, y4max, . . ., ynmin, ynmax will be obtained in a way simi-
T lar to the above procedures, and the generated slip surfaces
X ¼ ½x1 ; y 1 ; x2 ; y 2 ; . . . ; xn ; y n ; xnþ1 ; y nþ1  ð2Þ
will always be acceptable from this slip surface generation
The variables y1 and yn+1 can be determined from the slope method. The slip surface is represented mathematically by
geometry and the variables x1 and xn+1, hence y1 and yn+1 the vector X 0 = [x1, xn+1, y2, . . ., yn]T. For the sake of easy
are not independent control variables in the analysis. For a implementation of the optimization algorithms, the lower
slope with n slices, there are altogether 2n independent con- and upper bounds to the control variables are usually
trol variables. pre-specified and remain unchanged during the optimiza-
For the failure surface as shown in Fig. 2, the horizontal tion process. Because the lower and upper bounds to vari-
distance between x1 and xn+1 can be simply subdivided into ables y2 to yn are correlated to the values of x1 and xn+1, the
n equal segments using the following equation: lower and upper bounds are set to 0.0 and 1.0, respectively,
xnþ1  x1 after the values of y2 min, y2max, . . ., ynmin, ynmax are deter-
xi ¼ x1 þ  ði  1Þ; i ¼ 2; . . . ; n ð3Þ mined, the value of yi can then be mapped to the dynamic
n
bounds with the following equation:
The control variables will then be x1, y2, . . ., yn, xn+1 if equal
division of slices in x direction is adopted. The lower y i ¼ ðy imax  y imin Þ  ri ; i ¼ 2; . . . ; n ð5Þ
bounds and upper bounds to y2, . . ., yn are denoted by
y2min, y2max, . . ., ynmin, ynmax, and the bounds can be ob-
tained by utilizing the geometry of the slope and the bed- x6
V6
rock line y = R(x) by the relation R(xi) < yi < y1(x), y=y1(x)
i = 2, . . ., n (see Fig. 2). The solution domains for the two y5,max
y4,max
variables x1 and xn+1 can be defined easily by engineering y2,max y3,max V5
experience, or sufficiently wide domains can be specified x1 G
V1
by the engineers. x=x5
V4 y5,min
During the search for the critical slip surface, the trial V2 V3
x=x4 y=R(x)
slip surface may be inadmissible as shown by the dotted x=x2 x=x3 y4,min
y2,min y3,min
lines in Fig. 2, if the bounds to the control variables are sta-
H
tic. To ensure that every generated trial surface is admissi-
ble, the bounds to each control variable should be dynamic Fig. 3. Procedures in generating admissible slip surface.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
4 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
where ri(0 6 ri 6 1) means the imaginary variable corre-
sponding to variable yi. The vector V = (x1, xn+1, r2, . . ., rn)
will then be used to represent a slip surface during the
optimization process. The optimization problem to locate
the critical slip surface is now formulated as follows:
min f ðX VÞ
s:t: xl 6 x1 6 xu xL 6 xnþ1 6 xU
0:0 < ri < 1:0 i ¼ 2; . . . ; n
3. Brief review about six major heuristic optimization
algorithms
Since most of the heuristic algorithms which are artifi-
cial intelligence based methods are relatively new and are
not familiar to the geotechnical discipline, a brief review
about each method will be given in this section.
3.1. Simulated annealing algorithm (SA)
The simulated annealing algorithm (Kirkpatrick, 1983)
is a combinatorial optimization technique based on the
simulation of very slow cooling process of heated metal,
called annealing. The concept of this algorithm is similar
to heating a solid to a high temperature, and the molten
material is cooled slowly in a controlled manner until it
crystallizes, which is the minimum energy level of the sys-
tem. The solution starts with a high temperature t0, and a
sequence of trial vectors are generated until inner thermal
equilibrium is reached. Once the thermal equilibrium is
reached at a particular temperature, the temperature is
reduced by using the coefficient k and a new sequence
of moves will start. This process is continued until a suf-
ficiently low temperature te is reached, at which no fur-
ther improvement in the objective function can be
achieved.
The flowchart of the SA is shown in Fig. 4, where t0,
te, k are the initial temperature, the stopping tempera-
ture and the cooling temperature coefficient, respec-
tively. Usually, the higher the value of t0, the lower
the value of te and the smaller the value of k, the more
trials will be required in the optimization analysis. The
parameter N identifies the number of iterations for a
given temperature to reach its inner thermal equilib-
rium, and the array ft(neps) restore the objective func-
tion values obtained at the consecutive neps inner
thermal equilibriums and to terminate the optimization
algorithm. Vg, fg are the best solution found so far and
its associated objective function value. Nit is the number
of iterations for the current temperature. rs is a random
number in the range [0, 1], after N iterations are per-
formed. If the termination criterion is not satisfied,
Vg, fg are given to V0, f0, and this approach is different
from the classical SA in that it uses the best solution
found so far instead of the randomly adjusted solution
to generate the next solution.
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
3.2. Genetic algorithms (GA)
The genetic algorithm is developed by Holland (1975). It
is an optimization approach based on the concepts of
genetics and natural reproduction and the evolution of
the living creatures, in which an optimum solution evolves
through a series of generations. Each generation consists of
ð6Þ a number of possible solutions (individuals) to the prob-
lem, defined by an encoding. The fitness of an each individ-
ual within the generation is evaluated, and it influences the
reproduction of the next generation. The algorithm starts
with an initial population of M individuals. An individual
is composed of real coordinates associated to the variables
of the objective function. The current generation is called
parent generation, by which offspring generations are cre-
ated using operators such as crossover and mutation.
Another M individuals are rechosen from the parent and
offspring generations according to their fitness value. The
flowchart for the genetic algorithm is given in Fig. 5, where
qc and qm are the probability of the crossover and mutation
of the algorithm. Usually, the value of qc varies from 0.8 to
0.9 whilst qm falls in the range of 0.001–0.1. N1 represents
the number of iterations in the first stage, while N2 repre-
sents the time interval by which the termination criterion
is defined. If the best individual with the fitness value fg
remains unchanged after N2 iterations, the algorithm will
stop. Niter is the variable restoring the total iterations per-
formed by the algorithm. j1 and j2 are used to perform the
non-uniform mutation operator. The crossover operator is
given by the following equation:
8
< v0jþ1;l ¼ vft;l  rc þ ð1:0  rc Þ  vmi;l
>
v0jþ2;l ¼ vmi;l  rc þ ð1:0  rc Þ  vft;l ð7Þ
>
:
l ¼ 1; 2; . . . ; n þ 1
where v0j+1,l and v0j+2,l mean the lth element of the vector
V0j+1 and V0j+2, respectively, given by Eq. (8). Similarly,
vmi,l and vfi,l, represent the lth element of the mother parent
and father parent vector Vmi. and Vfi, and n + 1 is the num-
ber of control variable in this study:
8
>
> v0jþ1;l ¼ v0jþ1;l  ðv0jþ1;l  vl min Þ  ð1:0  j1 =j2 Þ2  rm
>
<
rnd 6 0:5
> v
> 0jþ1;l ¼ v
2
0jþ1;l þ ðvl max  v0jþ1;l Þ  ð1:0  j1 =j2 Þ  rm
>
:
rnd > 0:5
ð8Þ
rm and rnd are random numbers in the range of 0–1. vlmin
and vlmax are the lower and upper bounds to the lth vari-
able in V = (x1, xn+1, r2, . . ., rn). e is the tolerance of the
solution and is set to 0.0001 in this study.
3.3. Particle swarm optimization algorithm (PSO)
The PSO is an algorithm developed by Kennedy and
Eberhart (1995). This method has received wide applica-
tions in continuous and discrete optimization problem,
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 5

Fig. 4. Flowchart for the simulated annealing algorithm.

and an improved version for slope stability analysis has
been developed by Cheng et al. [15]. Yin (2004) has pro-
posed a hybrid version of PSO for optimal polygonal
approximation of the digital curves, while Salman (2002)
and Ourique (2002) have adopted PSO for the task assign-
ment problem and dynamical analysis in chemical pro-
cesses, respectively. PSO is based on the simulation of
simplified social models, such as bird flocking, fish school-
ing, and the swarming theory. It is related to the evolution-
ary computation procedures, and has strong ties with the
genetic algorithms.
This method is developed on a very simple theoretical
framework, and it can be implemented easily with only
primitive mathematical operators. Besides that, it is com-
putationally inexpensive in terms of both the computer
memory requirements and the speed of the computation.
In the PSO, a group of particles (generally double
the number of the control variables, M) referred as the

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

6 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Fig. 5. Flowchart for the genetic algorithm.

candidates or the potential solutions [as V described above]
flown in the problem search space to determine their opti-
mum positions. This optimum position is usually charac-
terized by the optimum of a fitness function (e.g. factor
of safety for the present problem). Each ‘‘particle’’ is repre-
sented by a vector in the multidimensional space to charac-
terize its position ðV kl Þ, and another vector to characterize
its velocity ðW kl Þ at the current time step k. The algorithm
assumes that particle i is able to carry out simple space and
time computations in order to respond to the quality

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
environment factors. That is, a group of birds can deter-
mine the average direction and speed of flight during the
search for food, based on the amount of the food found
in certain regions of the space. The results obtained at
the current time step k can be used to update the positions
of next time step. It is also assumed that the group of par-
ticles is able to respond to the environmental changes. In
other words, after finding a good source of food in a cer-
tain region of the space, the group of the particles will take
this new piece of information into the consideration to for-
mulate the ‘flight plan’. Therefore, the best results obtained
throughout the current time step are considered to generate
the new set of positions of the whole group.
In order to pursue the optimum of the fitness function,
the velocity Wki , and hence the position V ki of each particle
are adjusted in each time step. The updated velocity Wkþ1i is
a function of the three major components:
1. The old velocity of the same particle ðWki Þ.
2. Difference of the ith particle’s best position found so far
(called Pi) and the current position of the ith particle Vki .
3. Difference of the best position of any particle within the
context of the topological neighborhood of ith particle
found so far (called Pg, its objective function value
called fg) and current position of the ith particle V ki .
Fig. 6. Flowchart for the particle swarm optimization method.
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
7
Each of the components 2 and 3 as mentioned above are
stochastically weighted and added to the component 1 to
update the velocity of each particle, with enough oscilla-
tions that should empower each particle to search for a bet-
ter pattern within the problem space. In brief, each particle
employs Eq. (9) to update its position:
   
W kþ1
i ¼ xW ki þ c1 r1 Pi  V ki þ c2 r2 Pg  V ki
V kþ1 ¼ V 0ki þ W kþ1 ð9Þ
i i
i ¼ 1; 2; . . . ; 2n
where c1 and c2 are responsible for introducing the stochas-
tic weighting to components 2 and 3, respectively. These
parameters are commonly chosen as 2 which will also be
used in this study. r1 and r2 are two random numbers in
the range [0, 1], and x is the inertia weight coefficient. A lar-
ger value for x will enable the algorithm to explore the
search space, while a smaller value of x will lead the algo-
rithm to exploit the refinement of the results. Chatterjee
and Siarry [9] have introduced a nonlinear inertia weight
variation for the dynamic adaptation in PSO. The flow-
chart for PSO in searching for the critical slip surface is
shown in Fig. 6.
The termination criterion for the PSO is not stated
explicitly by Kennedy and Eberhart (1995) (same for other
 ARTICLE IN PRESS

8 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

modern global optimization methods). Usually a fixed Table 1

number of trials are carried out with the minimum value The structure of the HM
from all the trials taken as the global minimum, and this HM Control variables Objective function
is limitation of the original PSO or other global optimiza- x1 x2 x3
tion algorithms. Based on the termination proposal by hm1 1.0 1.5 0.5 4.50
Cheng et al. [15], if Pg remains unchanged after N2 itera- hm2 1.5 2.0 1.8 4.29
tions are performed, the algorithm will terminate as given hm3 0.5 1.5 1.0 3.50
by the following equation: hm4 1.8 2.5 0.9 8.10
hm5 0.9 2.2 1.2 5.49
j fsf  fg j6 e ð10Þ hm6 1.1 1.9 1.5 3.87

where Vsf, fsf mean the best solution found so far and its
related objective function value. e is the tolerance of termi-
nation, which is chosen as 0.0001 in the present study. The
following parameters are adopted in the present PSO
study: c1 = c2 = 2.0, w = 0.5, M = 4n, N1 = 500,
N2 = 200. All of the global optimization methods require
some parameters which are difficult to be established for
general problems. Based on the authors’ extensive tests, it
is found that PSO is not sensitive to these default parame-
ters in most problems, which is an important consideration
for recommending this method to be used for slope stabil-
ity analysis.
3.4. Simple harmony search algorithm (SHM)
Geem et al. [19] and Lee and Geem [24] developed a har-
mony search meta-heuristic algorithm that was conceptual-
ized using the musical process of searching for a perfect
state of harmony. Musical performances seek to find pleas-
ing harmony (a perfect state) as determined by an aesthetic
standard, just as the optimization process seeks to find a
global solution determined by an objective function. The
harmony in music is analogous to the optimization solu-
tion vector, and the musician’s improvisations are analo-
gous to local and global search schemes in the
optimization process. The SHM uses a stochastic random
search that is based on the harmony memory considering
rate HR and the pitch adjusting rate PR, and it is a popu-
lation based search method. A harmony memory HM of
size M is used to generate a new harmony, which is prob-
ably better than the optimum in the current harmony mem-
ory. The harmony memory consists of M harmonies (slip
surfaces), and M harmonies are usually generated ran-
domly. Consider HM = {hm1 hm2, . . ., hmM}
hmi ¼ ðvi1 ; vi2 ; . . . ; vim Þ
where each element of hmi corresponds to that in vector V
described above. Consider the following function optimiza-
tion problem, where M = 6, m = 3. Suppose HR = 0.9 and
PR = 0.1:
min
2
f ðx1 ; x2 ; x3 Þ ¼ ðx1  1Þ þ x22 þ ðx3  2:0Þ
2 (14) is used to calculate the new value of vni:
s:t: 0 6 x1 6 2 1 6 x2 6 3 0 6 x3 6 2
Six randomly generated harmonies comprise the HM as
shown in Table 1. The new harmony can be obtained
by the harmony search algorithm with the following pro-
cedures. A random number in the range [0, 1] is gener-
ated, e.g., 0.6 (<HR), and one of values from
{1.0, 1.5, 0.5, 1.8, 0.9, 1.1} should be chosen as the value
of x1 in the new harmony. Take 1.0 as the value of x1;
then another random number of 0.95 (>HR) is obtained.
A random value in the range [1, 3] for x2 is generated (say
1.2), and similarly 0.5 is chosen from the HM as the va-
lue of x3, thus a coarse new harmony hm0n ¼ ð1:0; 1:2; 0:5Þ
is generated. The improved new harmony is obtained by
adjusting the coarse new harmony according to the
parameter PR. Suppose three random values in the range
[0, 1] (say 0.7, 0.05, 0.8) are generated. Since the former
value 0.7 is greater than PR, so the value of x1 in hm0n re-
mains unchanged. The second value 0.05 is lower than
PR, so the value of 1.2 should be adjusted (say 1.10).
The above procedures proceed until the final new har-
mony hmn = (1.0, 1.10, 0.5) is obtained. The objective
function of the new harmony is calculated as 3.46. The
objective function value of 3.46 is better than that of
the worst harmony hm4, thereby, hm4 is excluded from
the current HM, while hmn is included in the HM. Up
to this stage, one iteration step has finished. The algo-
rithm will continue until the termination criterion is
achieved.
The iterative steps of harmony search algorithm in the
optimization of Eq. (6) as given in Fig. 7 are as follows:
Step 1: Initialize the algorithm parameters: HR, PR, M
and randomly generate M harmonies (slip sur-
faces) and evaluate the harmonies.
Step 2: Generate a new harmony (as shown in Fig. 7) and
evaluate it.
Step 3: Update the HM. If the new harmony is better
than the worst harmony in the HM, the worst
ð11Þ harmony is replaced with the new harmony. Take
the ith value of the coarse harmony h0n ; v0ni for ref-
erence. Its lower bound and upper bounds are
named as vimin, vimax. A random number r0 in
the range [0, 1] is generated. If r0 > 0.5, then v0ni
is adjusted to vni. using Eq. (13), otherwise, Eq.
ð12Þ  
vni ¼ v0ni þ vimax  v0ni  rand r0 > 0:5 ð13Þ
 
vni ¼ v0ni  v0ni  vimin  rand r0 6 0:5 ð14Þ
where rand means a random number in the range
[0, 1].

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
Fig. 7. Flowchart for generating a new harmony.
Step 4: Repeat steps 2 and 3 until the termination criterion
is achieved. In the original harmony search by
Geem et al. [19] and Lee and Geem [24], an explicit
termination criterion is not given. Cheng et al. [16]
have proposed a termination criterion for the opti-
mization process. Suppose M · N1 iterations are
first performed, and the best solution found so
far is called Vsf, with the objective function value
equal to fsf. Another M · N2 iterations are then
preformed, and the best harmony in the current
HM is called Vg, with the objective function value
equal to fg. The optimization process can terminate
if the following equation is satisfied:
j fsf  fg j6 e ð15Þ
In the present study, the global optimization parameters
are chosen as: M = 2m; HR = 0.98, Ptf = 0.1.
3.5. Modified harmony search algorithm (MHM)
Based on many trials by the authors, it is found that the
SHM works extremely fast and gives good solutions for
simple problems with less than 25 control variables. For
more complicate problems with a large number of control
variables, the original harmony search algorithm becomes
inefficient and can be trapped by the local minima easily.
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
9
Cheng et al. [16] have developed improved harmony
search algorithms MHM to overcome the limitations of
the SHM, which differs from the SHM in the following
two aspects.
(1) Instead of the use of uniform probability in the origi-
nal harmony search method, the better the objective
function value of one harmony, the more probable
will it be chosen for the generation of a new harmony.
A parameter d(0 < d 6 1) is introduced and all the
harmonies in HM are sorted by ascending order,
and a probability is assigned to each of them. For
instance, pr(i) means the probability to choose the
ith harmony which is given as
i1
where prðiÞ ¼ d  ð1  dÞ ð16Þ
for i = 1, 2, . . ., M. From Eq. (16), it can be seen that
the larger the value of d, the more probable will the
first harmony being chosen. An array ST(i),
i = 0, 1, 2, . . ., M as given by Eq. (17) should be used
to implement the above procedures for choosing the
harmony:
X
i
ST ðiÞ ¼ prðjÞ ð17Þ
j¼1
 ARTICLE IN PRESS

10 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

where ST(i) represents the accumulating probability
for ith harmony. ST(0) equals 0.0 for the sake of imple-
mentation. A random number rc is generated from the
range [0,ST(M)], and the kth harmony in HM is to be
chosen if the following criterion is satisfied:
ST ðk  1Þ < rc 6 ST ðkÞ; k ¼ 1; 2; . . . ; M ð18Þ
(2) Instead of one new harmony, a certain number of
new harmonies (Nhm) are generated during each iter-
ation step in the modified harmony search algorithm.
The utilization of the HM is intuitively more exhaus-
tive by generating several new harmonies than by
generating one new harmony during one iteration.
In order to retain the structure of the HM
unchanged, the M harmonies with lower objective
functions (for the minimization optimization prob-
lem) from M + Nhm harmonies are included in the
HM again, and Nhm harmonies of the higher objec-
tive function values are rejected.
Based on extensive numerical tests by Cheng et al. [16],
it is found that the modified harmony search algorithms are
more effective in overcoming the local minima as compared
with the original harmony search method for complicated
problems. The global optimization parameters used in the
present study are given as: M = 2m; HR = 0.98,
PR = 0.1; Nhm = Mx0.1; N1 = 500 and N2 = 200. If Nhm
is lower than 10, then Nhm = 10 (Fig. 8).
3.6. Tabu search algorithm
Tabu search ([20,21]) is not exactly an optimization
algorithm, but a collection of guidelines to develop optimi-
zation algorithms. The basic idea of the Tabu search is to
explore the trial solutions for the problem, moving from
a point to another in its neighborhood which is composed
of solutions which have little differences from this one.
Reverse moves and cycles are avoided by the use of a ‘‘tabu
list’’, where the moves previously done are memorized. In

Fig. 8. Flowchart for the modified harmony search algorithm.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
Fig. 9. Flowchart for the Tabu search.
order to implement the Tabu search, the first step is the dis-
cretization of the problem space. Each dimension is divided
into d elements, and altogether dm hyper-cubes are
obtained. If a solution is tabu, it means that the super-cube
in which the solution locates is also tabu. It is very difficult
to directly generate a solution within the super-cubes which
are not tabu, and a trial procedure is proposed by the
authors in this study. The procedure by which the new har-
monies are obtained in harmony search algorithm is used
to obtain the trial solutions. If the super-cube of the new
trial solution is tabu, another trial solution will be tested
until a trial solution which does not belong to the tabu
super-cubes is found. The flowchart for the Tabu search
algorithm is shown in Fig. 9.
In Fig. 9, fg is the objective function value of the best
harmony in the HM, and the parameters of N1, N2, N3
are used to terminate the algorithm. e is the tolerance for
the termination of the search.
4. Ant-colony algorithm
The ant-colony algorithm is a meta-heuristic method
using natural metaphor to solve the complex combinatorial
optimization problems, which is originated by Dorigo [17].
It is inspired by the natural optimization mechanism con-
ducted by real ants. Basically, a problem under the study
is transformed into a weighted graph. Then, the ant-colony
algorithm iteratively distributes a set of artificial ants onto
the graph to construct tours corresponding to the potential
optimal solutions. The optimization mechanism of the ant-
colony algorithm is based on two important features: (1)
the probabilistic state transition rule that is applied when
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
11
an ant is choosing the next vertex to visit; and (2)
pheromone updating rule that dynamically changes the
preference degree for the edges that have been traveled
through.
The continuous optimization problem should be firstly
transformed into a weighted graph. In the case of locating
the critical slip surface, each dimension is equally divided
into d subdivisions and m dimension (5 and 3, respectively,
in Fig. 10) in the optimization problem. The solid circles
located in adjacent columns are connected between each
other.
An ant is first located at the initial point. Based on the
probabilistic transition rule, one solid circle in the ‘first var-
iable’ column is chosen and thus the value of the first var-
iable is determined, and the procedures proceed to other
variables. When an ant finishes determining the value of
the end variable, it will go back to the initial point through
the end point for the next iteration. Fig. 11 shows the flow-
chart for the ant-colony algorithm. In Fig. 11, Na means
the number of total ants. The probabilistic transition rule
Fig. 10. The weighted graph transformed for continuous optimization
problem.
 ARTICLE IN PRESS

12 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Fig. 11. Flowchart for the ant-colony algorithm.

8 Na
and pheromone updating rule are described briefly as < P Q=fs condition 1
follows: l
Dsij ¼ ð21Þ
: l¼1
0 condition 2
(1) Probabilistic transition rule
where fsl represents the objective function value of the
skij solution found by the 1-th ant. conditionl means that
qij ¼ Pd k
; j ¼ 1; 2; . . . ; m ð19Þ
i¼1 sij the 1-th ant has chosen the ith solid circle of the jth
variable. Correspondingly, condition2 means that no
where skij represents the pheromone deposited on ith ant has chosen the ith solid circle of the jth variable
solid circle of jth variable within kth iteration step, within the kth iteration step.
qij means the probability of ith solid circle of jth var-
iable to be chosen.
(2) Pheromone updating rules 5. Numerical studies of the efficiency and effectiveness of
skþ1 ¼ ð1:0  lÞ  skij þ Dsij ð20Þ various optimization algorithms
ij

where l 2 [0, 1] is a parameter which simulates the To examine the efficiency and effectiveness of the heuris-
evaporation rate of the pheromone intensity. Dsij is tic algorithms given in the previous sections, for which a
obtained using the following equation: comparison never has been carried out for the slope stabil-

(failure surfaces by SA,MHM,SHM,PSO,GA are virtually the same, failure surfaces

by TABU and Zolfaghari are virtually the same)
Fig. 12. Comparison of critical slip surfaces for example 1.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 13

Table 2
Minimum factor of safety for example 1 (Spencer method)
Optimization methods Minimum factors of safety NOFs
Total Critical
Simple genetic algorithm by Zolfaghari et al. [34] 1.75 Unknown
SA Slices 40 1.7267 103,532 102,590
GA Slices 40 1.7297 49,476 49,476
PSO Slices 40 1.7282 61,600 60,682
SHM Slices 40 1.7264 107,181 98,607
MHM Slices 40 1.7279 28,827 28,827
Tabu Slices 40 1.7415 58,388 988
Ant-colony Slices 40 1.7647 83,500 16,488

ity problems, six published numerical problems are consid-
ered in this section. Example 1 as shown in Fig. 12 is a sim-
ple slope taken out from the study by Zolfaghari et al. [34].
The soil parameters are: unit weight 19.0 kN/m3, cohesion
15.0 kPa, and effective friction angle 20. Zolfagahri [34]
used a simple genetic algorithm and Morgenstern and Price
method (f(x) = 1.0) to obtain a minimum factor of safety
of 1.75 for non-circular failure surface.
From Table 2, it is clear that all the methods under con-
sideration are effective in the optimization analysis for this
simple problem. SA and SHM give the lowest factors of
safety, which are only slightly smaller than those by the
other methods, but the NOFs are up to about 100,000
which are much greater than those for the other methods.
MHM finds a minimum of 1.7279 which is slightly larger
than those by SA and SHM, but only 28,827 trials are
required which is much more efficient in the analysis.
PSO and GA give slightly larger factor of safeties which
are 1.7297 and 1.7282, but the numbers of evaluations
are modest when compared with those required by SA
and SHM.
Example 2 as shown in Fig. 13 is taken out from the
ACADS study (1989) example 1, in which four different
loading conditions are considered. The present study

Fig. 13. Comparison of critical slip surfaces under two loading cases.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

14 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Table 3 Table 5
Geotechnical parameters for example 2 Results for example 3 (Spencer method)
Layers c (kN/m3) c (kPa) / () Optimization methods Minimum NOFs
factors of safety
1 19.5 0.0 38.0 Total Critical
2 19.5 5.3 23.0
Leap-frog (Bolton et al. [6]) 1.305 Unknown
3 19.5 7.2 20.0
SA Slices 20 1.2411 51,770 51,745
Slices 30 1.2689 77,096 75,314
considers only the third and fourth loading conditions in Slices 40 1.3238 190,664 190,648
the original study, which are denoted as case 1 and case GA Slices 20 1.2819 28,808 28,808
2, respectively, in this study. The soil parameters for exam- Slices 30 1.2749 39,088 39,088
ple 2 are given in Table 3. Case 2 is similar to case 1, with a Slices 40 1.2855 115,266 115,202
pseudo-static earthquake coefficient equal to 0.15. Goh [22] PSO Slices 20 1.2659 42,000 33,012
used the GA for this example, and a minimum factor of Slices 30 1.2662 64,800 55,810
safety 1.387 is obtained. The recommended solution by Slices 40 1.2600 94,400 94,400
the experts in the ACADS study is 1.39 (1989), while Bol- SHM Slices 20 1.3414 29,942 29,760
ton et al. [6] adopted the leap-frog optimization algorithm Slices 30 1.2784 118,505 97,055
and obtained a minimum factor of safety of 1.359. Slices 40 1.2521 123,581 106,210
It can be seen from Table 4 that all the six optimization MHM Slices 20 1.2813 34,668 34,648
algorithms are basically effective for cases 1 and 2. The Slices 30 1.2720 26,891 26,891
Slices 40 1.2670 38,827 38,817
PSO, the MHM and the Tabu search are more efficient
for this example, and the critical slip surfaces from different Tabu Slices 20 1.5381 30,548 1148
methods of optimizations for case 1 and case 2 are basically Slices 30 1.5354 44,168 768
Slices 40 1.5341 58,188 788
similar, which are shown in Fig. 13.
Example 3 is taken out from the works by Bolton et al. Ant-colony Slices 20 1.4897 43,500 4721
[6]. There is a weak soil layer sandwiched between two Slices 30 1.5665 63,500 7726
Slices 40 1.5815 83,500 1501
strong layers. Unlike the previous two examples, the mini-

Table 4
Summary of results for example 2 (Spencer method)
Different procedures Minimum factors of safety NOFs
Case 1 Case 2 Total Critical
Case 1 Case 2 Case 1 Case 2
SA Slices 20 1.3585 0.9815 52,310 52,040 51,955 50,375
Slices 30 1.3569 0.9769 74,120 77,366 72,656 75,897
Slices 40 1.3676 0.9857 119,546 108,278 119,525 107,952
GA Slices 20 1.3591 0.9799 49,012 47,098 49,012 47,098
Slices 30 1.3582 0.9820 39,088 57,882 39,088 57,826
Slices 40 1.3573 0.9849 62,594 63,552 62,594 63,552
PSO Slices 20 1.3591 0.9797 33,600 50,400 32,760 49,728
Slices 30 1.3591 0.9785 60,000 47,600 60,000 46,546
Slices 40 1.3579 0.9770 61,600 78,000 60,682 78,000
SHM Slices 20 1.3609 0.9787 55,235 46,606 45,920 46,397
Slices 30 1.3596 0.9926 106,237 43,941 105,032 541
Slices 40 1.3613 0.9976 90,978 58,053 88,135 653
MHM Slices 20 1.3600 0.9800 30,761 28,532 30,761 28,522
Slices 30 1.3587 0.9797 41,059 30,582 41,059 30,522
Slices 40 1.3570 0.9783 31,024 30,659 31,024 30,659
Tabu Slices 20 1.3767 0.9956 30,748 30,148 1348 748
Slices 30 1.3762 1.0025 44,168 44,168 768 768
Slices 40 1.3742 1.0016 58,188 58,388 788 988
Ant-colony Slices 20 1.3912 1.0021 52,200 52,200 3507 3450
Slices 30 1.3931 0.9999 76,200 76,200 13,989 1801
Slices 40 1.3930 1.0030 100,200 100,200 1801 1801
Bolton et al. [6] Leap-frog 1.359 Unknown
Goh [22] GA 1.387 Unknown

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 15

slope geometry
leapfrog-Spencer
45 leapfro g-Janbu
SA -slice40
40
GA -slice40
MH M- slice40
35
TA B U-slice40

30 ANTCOLONY-slice40

25

20
20 30 40 50 60 70 80
(failure surfaces by GA,PSO and SHM are virtually the same)
Fig. 14. Critical slip surfaces for example 3.

slope geometry
50
Zolfaghari-noncircular
48 GA-slice40
SHM-slice40
46 TABU-slice40
ANTCOLONY-slice40
44

42

40
10 15 20 25 30 35 40
(failure surfaces by GA,PSO, MHM are virtually the same, failure surfaces
by SA and TABU are virtually the same)
Fig. 15. Comparison of critical slip surfaces for example 4.

mum factor of safety will be very sensitive to the precise and the GA are the most effective methods while the Tabu
location of the critical failure surface. The soil parameters search and the ant-colony methods are not satisfactory.
for soil layers 1–3, respectively, are friction angle 20, 10 The critical failure surfaces from these optimization meth-
and 20; cohesive strength 28.73 kPa, 0.0 kPa, and ods are shown in Fig. 15 (Tables 7–9).
28.73 kPa; and unit weight is 18.84 kN/m3 for all the three Example 5 is the test problem 4 in the ACADS study
soil layers. The results of analysis are shown in Table 5. (1989), where there is a 600 mm thick soft band of soil
For this problem, all the methods are basically satisfac- and external vertical loading together with water pressure.
tory except for the Tabu search and the ant-colony meth- For this problem, the PSO, the SHM and the MHM are
ods. The performance of the Tabu search and the ant- the most effective methods while the Tabu search and
colony method are poor for this example, which indicate the ant-colony methods are poor in the performance.
that these two methods are trapped by the presence of SA and GA are inefficient and are not are giving good
the local minima in the analysis. Overall speaking, the results for example 5. Compared with the referee value
PSO is the most effective method for this problem, while
the MHM ranks the second with the fewest trials. The crit-
ical failure surfaces from different methods of optimization Table 6
are shown in Fig. 14. Geotechnical parameters for example 4
Example 4 as shown in Fig. 15 is a slope with four soils Layers c (kN/m3) c (kPa) / ()
considered by Zolfaghari et al. [34], and the soil parameters 1 19.0 15.0 20.0
are given in Table 6. Soil layer 3 is a poor soil so that the 2 19.0 17.0 21.0
factor of safety is very sensitive to the location of the crit- 3 19.0 5.00 10.0
4 19.0 35.0 28.0
ical failure surface. For this problem, the PSO, the MHM

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

16 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Table 7 Table 9
Results for example 4 (Spencer method) Results for example 5 (Spencer method)
Optimization methods Minimum NOFs Optimization methods Minimum factors NOFs
factors of safety of safety
Total Critical Total Critical
Simple genetic algorithm by 1.24 Unknown Referee answer 0.78 Unknown
Zolfaghari [34] from ACADS
SA Slices 20 1.1789 54,326 53,104 SA Slices 20 0.8122 183,350 183,341
Slices 30 1.2564 80,342 80,115 Slices 30 0.9369 86,024 85,937
Slices 40 1.2813 107,738 107,678 Slices 40 1.1657 119,816 119,800
GA Slices 20 1.1495 33,616 33,584 GA Slices 20 0.8209 87,544 87,544
Slices 30 1.1117 67,816 67,816 Slices 30 0.8048 68,796 68,796
Slices 40 1.1440 75,800 75,800 Slices 40 0.8081 88,968 88,968
PSO Slices 20 1.1080 33,600 18,270 PSO Slices 20 0.7262 84,000 84,000
Slices 30 1.1341 47,600 28,256 Slices 30 0.7559 47,600 28,566
Slices 40 1.1095 78,000 58,320 Slices 40 0.7626 177,600 177,600
SHM Slices 20 1.2512 29,692 12,815 SHM Slices 20 0.7448 55,099 54,469
Slices 30 1.2405 93,223 84,676 Slices 30 0.7405 118,147 96,104
Slices 40 1.2068 90,443 68,022 Slices 40 0.7403 238,212 224,944
MHM Slices 20 1.1509 32,418 32,388 MHM Slices 20 0.7602 32,625 32,565
Slices 30 1.1315 40,409 40,379 Slices 30 0.7742 37,630 37,630
Slices 40 1.1385 42,489 42,489 Slices 40 0.7397 68,679 68,469
Tabu Slices 20 1.4714 30,418 748 Tabu Slices 20 1.1045 30,148 748
Slices 30 1.4661 44,168 768 Slices 30 1.2156 44,168 768
Slices 40 1.4650 58,188 788 Slices 40 1.1010 58,188 788
Ant-colony Slices 20 1.4249 51,900 5913 Ant-colony Slices 20 1.1471 52,200 9479
Slices 30 1.5299 76,200 2426 Slices 30 1.1375 76,200 1801
Slices 40 1.5817 100,200 8070 Slices 40 1.0044 100,200 8215

Table 8 slices are 40, 41, 44 and 45 for case 1 to case 4. For this
Geotechnical parameters of example 5
example, the Tabu search and the ant-colony methods
Layers c (kN/m3) c (kPa) / () are not good while all the other methods are basically
1 19.0 40 17.0 satisfactory. The PSO, the GA and the MHM are the
2 19.0 30 10.0 most effective and efficient methods for this example
(Table 11).

of 0.78, safety factor of 1.16 is obtained by SA for a num- 6. Sensitivity analysis

ber of slice equals to 40, and its associated critical slip sur-
face is also different from those by PSO, SHM and MHM.
Under the assumption of slices 40, the SHM finds a factor
of safety of 0.7403 with 238,212 trials, which is very time-
consuming in the analysis. The MHM gives 0.7397 with
only 70,000 trials while the PSO provides a factor of
safety of 0.7626 after 177,600 iterations are performed
(Fig. 16).
Example 6 is a case considered by Zolfaghari et al.
[34], where there is a natural slope with 4 soil layers as
shown in Fig. 17. Zolfaghari et al. [34] have adopted
the GA and the Morgenstern and Price’s method for this
example. The geotechnical parameters for this example
are shown in Table 10. Four loading cases are considered
by Zolfaghari et al. [34]: no water pressure and no
earthquake loadings (case 1); water pressure and no
earthquake loading (case 2); earthquake loading (coeffi-
cient = 0.1) and no water pressure (case 3); water pres-
sure and earthquake loading; (case 4). The numbers of
In all of the heuristic global optimization methods, there
is no simple rule to determine the parameters used in the
analysis. In general, these parameters are established by
experience and numerical tests. It is surprising to find that
the sensitivity of different global optimization methods
with respect to different parameters is seldom considered
in the past, and the sensitivity of the parameters in slope
stability analysis has not been reported. The authors con-
sider this issue to be important for geotechnical engineering
problems as there are different topography, subsoil condi-
tions, ground water condition, soil parameters, soil nails
and external loads controlling the problem. It appears that
many researchers have not appreciated the importance of
the parameters used for the global optimization. The sensi-
tivity of each parameter can be obtained through the nine
numerical tests by the statistical orthogonal tests given in
Tables 12–18 for examples 1 and 3. If the F value (Factorial
Analysis of Variance after [18]) of one parameter is lager

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 17

50 slope profile
phreatic surface

45 external loadings
SA-slice40
GA-slice40
40 PSO-slice40
MHM-slice40
SHM-slice40
35
TABU-slice40
ANTCOLONY-slice40
30

25

20
20 30 40 50 60 70 80

Fig. 16. Summary of different critical slip surfaces of example 5.

51

49 Layer 1

47
Table 11
Layer 2 Example 6 with four loading cases for example 6 (Spencer method)
45 Layer 3 Optimization methods Minimum factors NOFs
Layer 4
43 of safety
Total Critical
41 Case 1 GA by Zolfaghari et al. [34] 1.48 Unknown
10 15 20 25 30 35 40 Case 2 GA by Zolfaghari et al. [34] 1.36 Unknown
Case 3 GA by Zolfaghari et al. [34] 1.37 Unknown
Fig. 17. Geotechnical features of example 6.
Case 4 GA by Zolfaghari et al. [34] 0.98 Unknown
SA Case 1 1.3961 135,560 135,069
than the critical value F0.05 and is smaller than F0.01, it Case 2 1.2837 106,742 106,662
implies that the calculated result is sensitive to this param- Case 3 1.1334 108,542 106,669
Case 4 1.0081 111,386 109,667
eter, otherwise if the F value is smaller than F0.05, the result
is insensitive to this parameter. If the F value is larger than GA Case 1 1.3733 63,562 63,496
F0.01, the result is hypersensitive to this parameter. Case 2 1.2324 77,178 77,114
Case 3 1.0675 98,332 98,332
For simple problem given by example 1, every method Case 4 0.9631 84,272 84,272
can work with satisfaction for different optimization
PSO Case 1 1.3372 62,800 33,116
parameters. For example 3 which is a difficult problem with Case 2 1.2100 83,400 83,400
a soft band (similar to a Dirac function), the Tabu search Case 3 1.0474 69,600 69,600
and the ant-colony methods are poor in the performance Case 4 0.9451 68,600 24,440
while all the other optimization methods are basically SHM Case 1 1.3729 172,464 149,173
acceptable with the F values less than F0.05. The efficiency Case 2 1.2326 126,445 100,529
of different methods is, however, strongly related to the Case 3 1.0733 99,831 98,070
choice of the parameters, unless the transformation tech- Case 4 0.9570 212,160 186,632

nique by Cheng [13] which is equivalent to the use of a ran- MHM Case 1 1.3501 32,510 32,500
dom number with more weighting in the soft band region is Case 2 1.2247 40,697 40,687
Case 3 1.0578 40,476 40,440
adopted. Case 4 0.9411 33,236 33,236

Table 10 Tabu Case 1 1.4802 58,588 1188

Geotechnical parameters of example 6 Case 2 1.3426 59,790 990
Case 3 1.1858 63,796 796
Layers c (kN/m3) c (kPa) / () Case 4 1.0848 65,398 998
1 19.0 15.0 20.0 Ant-colony Case 1 1.5749 100,200 13,400
2 19.0 17.0 21.0 Case 2 1.4488 102,600 1801
3 19.0 5.00 10.0 Case 3 1.3028 109,800 5689
4 19.0 35.0 28.0 Case 4 1.1372 112,200 18,436

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

18 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Table 12
The effects of parameters of SA on the analysis for examples 1 and 3
k t0 N Results NOFs
1–0.5 1–10.0 1–100 Example 1 Example 3 Example 1 Example 3
2–0.8 2–20.0 2–300
1 1 1 1 1.7256 1.3232 176,562 140,522
2 2 2 2 1.7241 1.2990 915,902 956,523
3 1 2 2 1.7235 1.2514 339,602 408,482
4 2 1 1 1.7264 1.2745 423,522 349,422
5 2 1 2 1.7258 1.2846 852,602 986,534
6 1 2 1 1.7239 1.3193 183,422 135,262
7 2 2 1 1.7262 1.3213 463,782 360,242
8 1 1 2 1.7268 1.2582 492,122 252,302
Fk = 0.14, F t0 ¼ 0:47, FN = 3.86, F0.05 = 7.7, and F0.01 = 21.2.

Table 13
The effects of parameters of GA on the analysis for examples 1 and 3
qc qm Results NOFs
1–0.85 1–0.001 Example 1 Example 3 Example 1 Example 3
2–0.95 2–0.1
1 1 1 1.7273 1.2849 80,384 94,418
2 1 2 1.7266 1.2794 52,544 40,626
3 2 1 1.7272 1.2767 89,806 104,116
4 2 2 1.7266 1.2998 58,612 45,224
F qc ¼ 0:18, F qm ¼ 0:38, F0.05 = 161.4, and F0.001 = 4052.

Table 14
The effects of parameters of PSO on the analysis for examples 1 and 3
c1 c2 x Results NOFs
1–1.0 1–1.0 1–0.3 Example 1 Example 3 Example 1 Example 3
2–3.0 2–3.0 2–0.8
1 1 1 1 1.7287 1.4430 59,200 45,200
2 2 2 2 1.7401 1.2671 59,200 45,200
3 1 2 2 1.7353 1.2692 59,200 94,800
4 2 1 1 1.7226 1.2368 108,400 231,800
5 2 1 2 1.7309 1.2545 59,800 46,400
6 1 2 1 1.7269 1.2405 75,600 57,600
7 2 2 1 1.7376 1.2747 59,200 45,200
8 1 1 2 1.7266 1.2479 59,200 58,200
F c1 ¼ 0:60, F c2 ¼ 0:37, F- = 0.52, F0.05 = 7.7, and F0.01 = 21.2.

Table 15
The effects of parameters of SHM on the analysis for examples 1 and 3
HR PR Results NOFs
1–0.80 1–0.05 Example 1 Example 3 Example 1 Example 3
2–0.95 2–0.10
1 1 1 1.7330 1.2947 57,717 180,221
2 1 2 1.7438 1.3748 57,763 81,194
3 2 1 1.7231 1.2799 107,191 68,529
4 2 2 1.7259 1.2824 57,931 118,340
FHR = 1.91, FPR = 1.13, F0.05 = 161.4, and F0.01 = 4052.

7. Discussion and conclusion because of the special features of the objective functions
which have been discussed before. Since there are many lim-
Minimization of the factor of safety for a general slope itations in using the classical optimization methods in slope
stability problem is a difficult N-P hard type problem stability problem, the current trend is the adoption of the

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 19

Table 16
The effects of parameters of MHM on the analysis for examples 1 and 3
HR PR Nhm d Results NOFs
1–0.80 1-0.05 1–0.1 1–0.3 Example 1 Example 3 Example 1 Example 3
2–0.95 2–0.10 2–0.3 2–0.8
1 1 1 1 1 1.7348 1.2838 7654 13,547
2 1 1 1 2 1.7323 1.3523 13,654 9545
3 1 2 2 1 1.7295 1.3102 31,446 34,436
4 1 2 2 2 1.7347 1.3025 31,446 45,235
5 2 1 2 1 1.7270 1.2976 17,159 27,053
6 2 1 2 2 1.7271 1.2874 16,986 26,591
7 2 2 1 1 1.7273 1.2989 9640 15,219
8 2 2 1 2 1.7271 1.2878 19,621 13,128
FHR = 0.97, FPR = 0.07, FNhm = 0.10, Fd = 0.26, F0.05 = 10.1, and F0.01 = 34.1.

Table 17
The effects of parameters of Tabu on the analysis for examples 1 and 3
d Nt HR PR Results NOFs
1–2 1–30 1–0.80 1–0.05 Example 1 Example 3 Example 1 Example 3
2–5 2–50 2–0.95 2–0.10
1 1 1 1 1 1.7411 1.5714 58,388 44,168
2 1 1 1 2 1.7413 1.5391 58,188 44,168
3 1 2 2 1 1.7424 1.5661 58,188 44,168
4 1 2 2 2 1.7413 1.5661 58,188 44,168
5 2 1 2 1 1.7429 1.5661 58,588 44,168
6 2 1 2 2 1.7415 1.5427 58,188 44,168
7 2 2 1 1 1.7415 1.5470 58,188 44,368
8 2 2 1 2 1.7354 1.5561 59,188 44,168
Fd = 0.63, F N t ¼ 0:17, FHR = 0.49, FPR = 1.43, F0.05 = 10.1, and F0.01 = 34.1.

modern global optimization methods in this type of prob-
lem. The GA, the SA, the HM and the PSO have recently
been used by the authors and other researchers, while the
Tabu search and the ant-colony method are new to the slope
stability discipline. All these six types of heuristic algorithms
can function well for normal problems which are demon-
strated in examples 1 and 2 and some other internal studies
by the authors. For simple problems and the number of con-
trol variable is less than 25, it appears that the SHM and
MHM are the most efficient optimization methods. The
SHM, the Tabu search and the ant-colony method can per-
form well in many other applications, but they have been
demonstrated to be less satisfactory for slope stability prob-
lems. Since the ant-colony method aims at continuous opti-
mization problems, it is not surprising that it is less
satisfactory for slope stability problem where discontinuity
of the objective function is generated by divergence of the
factor of safety. Similarly, when there are great differences
in the soil properties between different soils, SHM will be
less satisfactory due to the use of uniform probability to
individual harmony. The present study has illustrated the
special feature of slope stability analysis during the optimi-
zation analysis, which is not found in other applications as
production, system control or other similar disciplines.
On the other hand, for large scale optimization prob-
lems or problems similar to examples 3–6 with the presence
of a think layer of soft band which will create difficulties in
the optimization analysis, the effectiveness and the effi-
ciency of the different heuristic optimization methods vary
significantly between different problems. The Tabu search
and the ant-colony methods have demonstrated very poor
results in some of the problems, while the PSO method
appears to be the most stable and efficient and is recom-
mended for use in such case. The presence of a thin soft
band is difficult in the analysis, as a random number with
equal opportunity in every solution domain is used in the
generation of the trial failure surface. If the domain trans-
formation technique as suggested by Cheng [13] is adopted,
all the six methods can work effectively and efficiently for
problems with soft band, with the MHM and PSO being
the best solution algorithms in terms of efficiency.
Heuristic algorithms are approximate and not accurate
algorithms. These algorithms usually find a solution close
to the best one, and they usually find it fast and easily.
Every heuristic algorithm relies on the use of some param-
eters for analysis, but there is no rigorous method in deter-
mining these kinds of parameters. The success and
efficiency of a global optimization algorithm may rely on
the use of these parameters, and the authors adopt a uni-
form set of parameters for every optimization method for
all the problems. In normal most cases, it is found that
these global optimization algorithms are relatively insensi-

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

20 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx

Table 18
The effects of parameters of Ant-colony on the analysis for examples 1 and 3
l Q d Results NOFs
1–0.3 1–10.0 1–10 Example 1 Example 3 Example 1 Example 3
2–0.8 2–50.0 2–20
1 1 1 1 1.7447 1.5332 83,500 76,200
2 2 2 2 1.7404 1.9239 66,800 50,800
3 1 2 2 1.7636 1.8787 66,800 50,800
4 2 1 1 1.7717 1.7420 83,500 76,200
5 2 1 2 1.7377 1.9239 66,800 50,800
6 1 2 1 1.7538 1.5049 83,500 76,200
7 2 2 1 1.7569 1.7420 83,500 76,200
8 1 1 2 1.7591 1.8435 66,800 50,800
Fl = 11.8 FQ = 0.002 Fd = 39.7F0.05 = 7.7 F0.01 = 21.2.

50 slope geometry
Phreatic surface
Zolfaghari
SA
48 GA
PSO
MHM
SHM
46 TABU
ANTCOLONY

44

42
10 15 20 25 30

Fig. 18. Critical slip surfaces for case 1 of example 6.

50 slope geometry
Phreatic surface
Zolfaghari
48
SA
GA
46 PSO
ANTCOLONY

44

42

40 10 15 20 25 30 35

(failure surface by GA,MHM,SHM and TABU are virtually the same)

Fig. 19. Critical slip surfaces for case 2 of example 6.

tive to the use of the optimization parameters, but the
authors have also encountered special cases which are
worth further discussion (Figs. 18–21).
For Fig. 22 where the water table is above the ground
surface at the left-hand side of the slope, the SHM and
the MHM cannot locate the global minimum using Spencer
method even when the optimization parameters are varied,
unless the initial trial failure surface is close to the critical
solution. The authors have noted that many of the trial
failure surfaces (20%) fail to converge in the optimization

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 21

50 slope geometry
Phreatic surface

48 Zolfaghari
SA
GA
46
TABU
ANTCOLONY
44

42

40 10 15 20 25 30 35

(failure surfaces by GA,PSO,MHM and SHM are virtually the same)

Fig. 20. Critical slip surfaces for case 3 of example 6.

Fig. 21. The critical slip surfaces for case 4 of example 6.

analysis, which is equivalent to the presence of discontinu- breakdown) except for the SA. This case is particularly
ity in the objective function. The SHM and the MHM are interesting in that all the optimization methods (except
trapped by the local minima under such case if the initial for the SA) lose the direction of search and fail to find even
trial is not close to the critical solution and there are major one converged result before termination, unless the
discontinuities in the objective function. The GA, the Tabu
search and the ant-colony methods all suffer from this lim-
itation. On the other hand, the PSO and the SA can locate 6
the critical solution effectively.
5
Another interesting case is a steep slope in Beijing,
where there is a thin layer of soft material, a tension crack, 4
two soil nails, an external surcharge and water pressure at
3
the tension crack (Fig. 23). Only SA method can work
properly with the Spencer method for this case while all 2
the methods fail to work properly unless a good initial is
used. For this problem, the authors noticed that the initial 1

400–500 trials all fail to converge with the Spencer method.

0
Such a major discontinuity in the objective function creates
a great difficult in determining the directions of search for -1
2 4 6 8 10 12 14 16
all the global optimization methods, and no solution is
obtained from all the optimization methods (complete Fig. 22. Slope with pond water.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
 ARTICLE IN PRESS
22 Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx
Fig. 23. Steep slope with tension crack and soil nail.
optimization parameters and the initial trial are specially
tuned. This special example has also illustrated the diffi-
culty in locating the critical failure surface for some special
problems, where convergence is a critical issue. SA is less
sensitive to the discontinuity of the objective function
because it is based on the Markov chain with a double
looping search technique.
Every global optimization method can be tuned to work
well if suitable optimization parameters or initial trial are
adopted. Since the suitable optimization parameters or the
initial trial are difficult to be established for a general prob-
lem, the performance of a good optimization method should
be relatively insensitive to these factors. Based on the six
numerical examples and the two special cases as shown in
Figs. 22 and 23 and some other internal studies by the
authors, the general comments on the different heuristic arti-
ficial intelligence based global optimization methods are:
(1) For normal and simple problems, practically every
method can work well. The harmony method and
the genetic algorithm are the most efficient method
when the number of control variable is less than 20.
The Tabu search and the ant-colony method are
sometimes extremely efficient in the optimization pro-
cess, but the efficiency of these two methods fluctuate
significantly between different problems and are not
recommended to be used.
(2) For normal and simple problems where the number
of control variables exceeds 20, the MHM and the
PSO are the recommended solution as they are more
efficient in the solution, and the solution time will not
vary significantly between different problems.
(3) For more complicated problems or when the number of
control variables is great, the effectiveness and efficiency
of the PSO is nearly the best in all of the examples.
(4) Thin soft bandcreate great difficulty in the global
optimization analysis, and the PSO will be the best
method in this case. However, using the domain
transformation strategy by Cheng [13], all the global
optimization methods can work well for this case.
Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004
(5) For problems similar to that as shown in Fig. 23,
where an appreciable amount of trial failure surfaces
will fail to converge, the simulated annealing method
and the PSO are the recommended solutions.
In views of the differences in the performance between
different global optimization algorithms, a more satisfac-
tory solution is the combined used of two different algo-
rithms. For example, the PSO or the MHM can be
adopted for normal problems, while the SA can be adopted
when the ‘‘failure to convergence’’ counter is high. Further
improvement can be achieved by using the optimized
results from a particular optimization method as a good
initial trial, and a second optimization method adopts the
initial trial from the first optimization algorithm for the
second stage of optimization with a reduced solution
domain for each control variable. The authors have imple-
mented both options and have successfully overcome many
difficult problems in several countries.
References
[2] Arai K, Tagyo K. Determination of noncircular slip surfaces giving
the minimum factor of safety in slope stability analysis. Soils and
Foundations 1985;21:43–51.
[3] Baker R, Garber M. Theoretical analysis of the stability of slopes.
Geotechnique 1978;28:341–95.
[6] Bolton HPJ, Heymann G, Groenwold A. Global search for
critical failure surface in slope stability analysis. Eng Optimiz
2003;35:51–65.
[8] Celestino TB, Duncan JM. Simplified search for non-circular slip
surface. In: Proceeding 10th international conference on soil mechan-
ics and foundation engineering, Stockholm, Sweden; 1981. p. 391–4.
[9] Chatterjee A, Siarry P. Nonlinear inertia weight variation for
dynamic adaption in particle swarm optimization. Comput Oper
Res 2006;33:859–71.
[10] Chen Z, Shao C. Evaluation of minimum factor of safety in slope
stability analysis. Can Geotech J 1983;25(4):735–48.
[12] Cheng YM. Locations of critical failure surface and some
further studies on slope stability analysis. Comput Geotech
2003;30:255–67.
[13] Cheng YM. Global optimization analysis of slope stability by
simulated annealing method with dynamic bounds and ‘‘Dirac
function’’. Eng Optimiz 2007;39(1):17–32.
[15] Cheng YM, Li L, Li SC. Particle swarm optimization algorithm for
location of critical non-circular failure surface in two dimensional
slope stability analysis. Comput Geotech 2007;34(2):92–103.
[16] Cheng YM, Li L, Li SC. Location of the critical slip surface by new
harmony search algorithms. Comput Geotech 2006 [in preparation].
[17] Dorigo M. Optimization, learning and natural algorithms, Ph.D.
Thesis, Italy: Department of Electronics, Politecnico di Milano;
1992.
[18] Fisher RA. J Roy Stat Soc A 1963;126:159–70.
[19] Geem ZW, Kim JH, Loganathan GV. A new heuristic optimization
algorithm: harmony search. Simulation 2001;76:60–8.
[20] Glover F. Tabu search – part I. ORSA J Comput 1989;1:190–206.
[21] Glover F. Tabu search – part II. ORSA J Comput 1990;2:4–32.
[22] Goh ATC. Genetic algorithm search for critical slip surface in
multiple-wedge stability analysis. Can Geotech J 1999;36:359–82.
[23] Greco VR. Efficient Monte Carlo technique for locating critical slip
surface. J Geotech Eng ASCE 1996;122:517–25.
[24] Lee KS, Geem ZW. A new meta-heuristic algorithm for continuous
engineering optimization: harmony search theory and practice.
Comput Meth Appl Mech Eng 2005;194:3902–33.
 ARTICLE IN PRESS

Y.M. Cheng et al. / Computers and Geotechnics xxx (2007) xxx–xxx 23

[27] Malkawi AIH, Hassan WF, Sarma SK. Global search method for
locating general slip surface using Monte Carlo techniques. J Geotech
Geoenviron Eng 2001;127:688–98.
[28] Nguyen VU. Determination of critical slope failure surfaces. J
Geotech Eng ASCE 1985;111:238–50.
[32] Yamagami T, Jiang JC. A search for the critical slip surface in three-
dimensional slope stability analysis. Soils Foundations 1997;37(3):1–6.
[34] Zolfaghari AR, Heath AC, McCombie PF. Simple genetic algorithm
search for critical non-circular failure surface in slope stability
analysis. Comput Geotech 2005;32:139–52.

Please cite this article in press as: Cheng YM et al., Performance studies on six heuristic global optimization methods ..., Comput
Geotech (2007), doi:10.1016/j.compgeo.2007.01.004