Neri-Tirronen2010 Article RecentAdvancesInDifferentialEv

Artif Intell Rev (2010) 33:61–106
DOI 10.1007/s10462-009-9137-2
Recent advances in differential evolution:

a survey and experimental analysis
Ferrante Neri · Ville Tirronen
Published online: 27 October 2009

© Springer Science+Business Media B.V. 2009
Abstract Differential Evolution (DE) is a simple and efficient optimizer, especially for
continuous optimization. For these reasons DE has often been employed for solving vari-
ous engineering problems. On the other hand, the DE structure has some limitations in the
search logic, since it contains too narrow a set of exploration moves. This fact has inspired
many computer scientists to improve upon DE by proposing modifications to the original
algorithm. This paper presents a survey on DE and its recent advances. A classification, into
two macro-groups, of the DE modifications is proposed here: (1) algorithms which integrate
additional components within the DE structure, (2) algorithms which employ a modified DE
structure. For each macro-group, four algorithms representative of the state-of-the-art in DE,
have been selected for an in depth description of their working principles. In order to compare
their performance, these eight algorithm have been tested on a set of benchmark problems.
Experiments have been repeated for a (relatively) low dimensional case and a (relatively)
high dimensional case. The working principles, differences and similarities of these recently
proposed DE-based algorithms have also been highlighted throughout the paper. Although
within both macro-groups, it is unclear whether there is a superiority of one algorithm with
respect to the others, some conclusions can be drawn. At first, in order to improve upon
the DE performance a modification which includes some additional and alternative search
moves integrating those contained in a standard DE is necessary. These extra moves should
assist the DE framework in detecting new promising search directions to be used by DE.
Thus, a limited employment of these alternative moves appears to be the best option in suc-
cessfully assisting DE. The successful extra moves are obtained in two ways: an increase in
the exploitative pressure and the introduction of some randomization. This randomization
should not be excessive though, since it would jeopardize the search. A proper increase in
the randomization is crucial for obtaining significant improvements in the DE functioning.
F. Neri (B) · V. Tirronen

Department of Mathematical Information Technology, University of Jyväskylä, P.O. Box 35, Agora,
40014 Jyväskylä, Finland
e-mail: ferrante.neri@jyu.fi
V. Tirronen
e-mail: ville.tirronen@jyu.fi
123
62 F. Neri, V. Tirronen
Numerical results show that, among the algorithms considered in this study, the most efficient
additional components in a DE framework appear to be the population size reduction and the
scale factor local search. Regarding the modified DE structures, the global and local neigh-
borhood search and self-adaptive control parameter scheme, recently proposed in literature,
seem to be the most promising modifications.
Keywords Differential Evolution · Survey · Comparative Analysis · Self-Adaptation ·

Continuous Optimization
1 Introduction
Differential Evolution (DE, see Storn and Price 1995, Storn 1999, Price et al. 2005, and
Chakraborty 2008) is a reliable and versatile function optimizer. DE, like most popular Evo-
lutionary Algorithms (EAs), is a population-based tool. DE, unlike other EAs, generates
offspring by perturbing the solutions with a scaled difference of two randomly selected
population vectors, instead of recombining the solutions under conditions imposed by a
probabilistic scheme. In addition, DE employs a one-to-one spawning logic which allows
replacement of an individual only if the offspring outperforms its corresponding parent.
Thanks to, on one hand, its simplicity and ease of implementation, and on the other hand,
reliability and high performance, DE became very popular among computer scientists and
practitioners almost immediately after its original definition. The former having been appre-
ciative of and investigating the DE structure, see Storn (1996b), Storn and Price (1997),
Tvrdík and Krivý (1999), and Lampinen and Zelinka (2000) while the latter applied this
simple and powerful tool to the most various set of engineering problems, see e.g. Storn
(1996a), Masters and Land (1997), Thomas and Vernon (1997), Chang and Chang (1998),
Plagianakos (1998), and Lampinen and Zelinka (1999).
Although DE has great potential, it was clear to the scientific community that some
modifications to the original structure were necessary in order to significantly improve its
performance. This paper deals with modern modifications of the DE schemes in order to
analyze them and draw some conclusions on future trends. In this work we subdivide the
modified versions of DE into two classes:
1. DE integrating an extra component. This class includes those algorithms which use DE
as an evolutionary framework which is assisted by additional algorithmic components,
e.g. local searchers (see Tirronen et al. 2007, 2008, and Chen et al. 2008), and surrogate
assisted models (Zhang and Sanderson 2007). The algorithms belonging to this class can
be clearly decomposed as a DE framework and additional components.
2. Modified structures of DE. This class includes those algorithm which make a substantial
modification within the DE structure, in the search logic, the selection etc. Obviously
this modification is supposed to enhance the performance of the original DE.
This paper conducts a survey on DE in Sects. 2 and 3 and analyzes in depth, in Sects. 4
and 5, some algorithmic families which are in our opinion especially representative in liter-
ature and thus cornerstones in DE development. The algorithmic families taken into account
have been selected by considering those algorithms which are, according to our judgement,
the most promising and successful algorithmic solutions and have already been published on
relevant forums.
More specifically, Sect. 4 gives a description of the following algorithmic families:
123
Recent advances in differential evolution 63
– DE with Trigonometric Mutation, see Fan and Lampinen (2002, 2003b), Hu et al. (2005)
Angira and Santosha (2007), and Angira and Santosh (2008)
– DE with Simplex Crossover Local Search, see Noman and Iba (2005, 2008)
– DE with Population Size Reduction, see Brest and Maučec (2008) and Brest et al. (2008)
– DE with Scale Factor Local Search, see Neri and Tirronen (2009), Tirronen et al. (2009),
and Neri et al. (2009)
Section 5 describes the following algorithmic families:
– Self Adaptive Control Parameters, see Brest et al. (2006a,b, 2007), Zamuda et al. (2007),
and Brest et al. (2008)
– Opposition Based DE, see Rahnamayan et al. 2006a,b, 2007, 2008a, Rahnamayan and
Wang (2008), and Rahnamayan et al. (2008b)
– Global-Local Search DE, see Chakraborty et al. (2006) and Das et al. (2009)
– Self Adaptive Coordination of Multiple Mutation Rules, see Qin and Suganthan (2005),
Yang et al. (2008b), Qin et al. (2009)
In order to analyze the benefits and drawbacks of each algorithmic family listed above, a
rapresentative algorithm from each group has been implemented and tested on a broad set
of various test problems. Numerical results are reported in Sect. 6. Finally, Sect. 7 gives the
conclusion of this work.
2 Standard differential evolution
In order to clarify the notation used throughout this chapter we refer to the minimization
problem of an objective function f (x), where x is a vector of n design variables in a decision
space D.
According to its original definition given in Storn and Price (1995), DE consists of the
following steps. An initial sampling of Spop individuals is performed pseudo-randomly with
a uniform distribution function within the decision space D. At each generation, for each
individual xi of the Spop , three mutually distinct individuals xr , xs and xt are pseudo-ran-
domly extracted from the population. According to DE logic, a provisional offspring x off is
generated by mutation as:

xoff = xt + F(xr − xs ) (1)
where F ∈ [0, 1+[ is a scale factor which controls the length of the exploration vector
(xr − xs ) and thus determines how far from point xi the offspring should be generated. With
F ∈ [0, 1+[, it is meant here that the scale factor should be a positive value which cannot
be much greater than 1, see Price et al. (2005). While there is no theoretical upper limit for
F, effective values are rarely greater than 1.0. The mutation scheme shown in Eq. (1) is also
known as DE/rand/1. Other variants of the mutation rule have been subsequently proposed
in literature, see Qin and Suganthan (2005):
= x
– DE/best/1: xoff best + F (x s − x t )
= x + F (x
– DE/cur-to-best/1: xoff i best − x i ) + F (x s − x t )
= x
– DE/best/2: xoff best + F (x s − x t ) + F (x u − x v )
– DE/rand/2: xoff = x + F (x − x ) + F (x − x )
r s t u v
= x + F (x
– DE/rand-to-best/2: xoff t best − x i ) + F (xr − x s ) + F (x u − x v )
where xbest is the solution with the best performance among individuals of the population, x u
and xv are two additional pseudo-randomly selected individuals. It is worthwhile to mention
the rotation invariant mutation shown in Lampinen (1999):
123
– DE/current-to-rand/1 xoff = xi + K (xt − xi ) + F (xr − xs )

where K is is the combination coefficient, which as suggested in Lampinen (1999) should
be chosen with a uniform random distribution from [0, 1] and F = K · F. For this special
mutation the mutated solution does not undergo the crossover operation (since it already
contains the crossover), described below.
Recently, in Price et al. (2005), a new mutation strategy has been defined. This strategy,
namely DE/rand/1/either-or, consists of the following:

xt + F (xr − xs ) if rand (0, 1) < p F
x off = (2)
xt + K (xr + xs − 2xt ) otherwise
where for a given value of F, the parameter K is set equal to 0.5 (F + 1).
When the provisional offspring has been generated by mutation, each gene of the individ-
is exchanged with the corresponding gene of x with a uniform probability and the
ual xoff i
final offspring xoff is generated:

xi, j if rand (0, 1) < C R
xoff,j = (3)
xoff,j otherwise
where rand (0, 1) is a random number between 0 and 1; j is the index of the gene under exam-
ination. This crossover strategy is well-known as binary crossover and indicated as “bin”.
For the sake of completeness, we mention that there exist a few other crossover strategies,
for example the exponential strategy see Price et al. (2005). However in this paper we focus
on the bin strategy since it is the most commonly used and often the most promising.
The resulting offspring xoff is evaluated and, according to a one-to-one spawning strategy,
it replaces xi if and only if f (xoff ) ≤ f (xi ); otherwise no replacement occurs. For sake of
clarity, the pseudo-code highlighting the working principles of DE is shown in Fig. 1.
3 Differential Evolution: a survey
As shown in Sect. 2, DE is based on a very simple idea, i.e. a search by means of adding
vectors and a one-to-one spawning for the survivor selection. Thus, DE is very easily imple-
mented/coded and contains a limited amount of parameters to be tuned (only Spop , F, and
C R). In addition, the fact that DE is rather robust and versatile has encouraged engineers and
practitioners to employ it in various applications, see applications presented in Price et al.
(2005). For example, in Joshi and Sanderson (1999) a DE application to the multisensor fusion
problem is given. An application of a DE-based hybrid algorithm to aerodynamic design is
given in Rogalsky and Derksen (2000). In Chang and Chang (2000), Su and Lee (2003),
and Chiou et al. (2004) DE applications to power electronics are presented. In Wang and
Jang (2000) an application of DE to chemical engineering is proposed. In Liu and Lampinen
(2005) a DE variant is employed for training a radial basis function network. In Storn (2005),
Karaboga and Cetinkaya (2004, 2006) a filter design is carried out by DE. In Tirronen et
al. (2007, 2008), a DE-based algorithm is implemented to design a digital filter for paper
industry applications. An application to highway network capacity optimization is given in
Koh (2009). A review of DE applications is presented in Plagianakos (2008).
From an algorithmic viewpoint, the reasons for the success of DE have been highlighted
in Feoktistov (2006): the success of DE is due to an implicit self-adaptation contained within
the algorithmic structure. More specifically, since, for each candidate solution, the search rule
depends on other solutions belonging to the population (e.g. x t , xr , and xs ), the capability of
123
Fig. 1 DE/rand/1/bin pseudo-code
detecting new promising offspring solutions depends on the current distribution of solutions
within the decision space. During the early stages of the optimization process, the solutions
tend to be spread out within the decision space. For a given scale factor value, this implies
that mutation appears to generate new solutions by exploring the space by means of a large
step size (if xr and xs are distant solutions, F (xr − xs ) is a vector characterized by a large
modulus). During the optimization process, solutions of the population tend to concentrate
in specific parts of the decision space. Therefore, step size in the mutation is progressively
reduced and the search is performed in the neighborhood of the solutions. In other words,
due to its structure, a DE scheme is highly explorative at the beginning of the evolution and
subsequently becomes more exploitative during the optimization.
Although this mechanism seems, at first glance very efficient, it hides a limitation. If for
some reason the algorithm does not succeed at generating offspring solutions which outper-
form the corresponding parent, the search is repeated again with similar step size values and
will likely fail by falling into an undesired stagnation condition (see Lampinen and Zelinka
2000). Stagnation is that undesired effect which occurs when a population-based algorithm
does not converge to a solution (even suboptimal) and the population diversity is still high. In
the case of DE, stagnation occurs when the algorithm does not manage to improve upon any
123
solution of its population for a prolonged amount of generations. In other words, the main
drawback of DE is that the scheme has, for each stage of the optimization process, a limited
amount of exploratory moves and if these moves are not enough to generate new promising
solutions, the search can be heavily compromised. A theoretical analysis of the convergence
properties of DE is given in Zaharie (2002). This study allowed the definition of a critical
value of the scale factor, i.e. for given values of Spop and C R, the scale factor F should not
be smaller than:

1 − C2R
Fcrit = . (4)
Spop
It is clear that successful functioning of a DE depends on the parameter setting of the

three control parameters mentioned above. The population size Spop is related to the amount
of possible moving vectors. Over all the possible moves given by a population, some moves
are beneficial in the search for the optimum while some others are ineffective and result
in a waste of computational effort. Therefore, too small a population size can contain too
limited an amount of moves, while too large a population size may contain a high number of
ineffective moves which can likely mislead the search. To some extent the population sizing
of a DE is analogous to the other Evolutionary Algorithms (EAs), if too small it could cause
premature convergence and if too large it could cause stagnation (see Eiben and Smith 2003).
A good value can be found by considering the dimensionality of the problem similar to what
is commonly performed for the other EAs. A guideline is given in Storn and Price (1997)
where a setting of Spop equal to ten times the dimensionality of the problem is proposed.
However, this indication is not confirmed by a recent study in Neri and Tirronen (2008) where
it is shown that a population size lower than the dimensionality of the problem can be optimal
in many cases.
Regarding the scale factor F and the crossover rate C R, these settings may be a difficult
task. The setting of these two parameters is neither an intuitive nor a straightforward task but
is unfortunately crucial for guaranteeing the algorithmic functioning. Several studies have
thus been proposed in literature. The study reported in Lampinen and Zelinka (2000) arrives
at the conclusion, after an empirical analysis, that usage of F = 1 is not recommended,
since according to a conjecture of the authors it leads to a significant decrease in explorative
power. Analogously, the setting C R = 1 is also discouraged since it would dramatically
decrease the amount of possible offspring solutions. In Storn and Price (1997) and Liu and
Lampinen (2002a) the settings F ∈ [0.5, 1] and C R ∈ [0.8, 1] are recommended. In Liu and
Lampinen (2002b) and Rönkkönen et al. (2005) the setting F = C R = 0.9 is chosen on the
basis of discussion in Price and Storn (1997). The empirical analysis reported in Zielinski et
al. (2006) shows that in many cases the setting of F ≥ 0.6 and C R ≥ 0.6 leads to results
having better performance.
Several studies, e.g., Gämperle (2002), Liu and Lampinen (2002a), and Mallipeddi and
Suganthan (2008), highlight that an efficient parameter setting is very dependent on problems
(e.g., F = 0.2 could be a very efficient setting for a certain fitness landscape and completely
inadequate for another problem). This result can be seen as a confirmation of the validity of
the No Free Lunch Theorem Wolpert and Macready (1997) with reference to the DE schemes.
The problem of the parameter setting is emphasized when DE is employed for handling
difficulties of real-world applications such as high dimensionality and noisy optimization
problems. Clearly, the risk of the DE stagnation is higher for larger decision spaces and
worsens as the number of dimensions of the problem increases (see Zamuda et al. 2008,
Olorunda and Engelbrecht 2007, Yang et al. 2007, 2008a, Noman and Iba 2005, Gao and
123
Wang 2007, and Brest et al. 2008). A large decision space (in terms of dimensions) requires a
wide range of possible moves to enhance its capability of detecting new promising solutions.
Since, as mentioned before, an enlargement in population size causes an increase in the set
of potential ineffective moves, a proper choice of F and C R becomes a crucial aspect in the
success of DE.
High dimensionality is not the only curse of DE. As highlighted in Krink et al. (2004), DE
seems to be inefficient for noisy optimization problems as well and some specific counter-
measures must be applied, see Das et al. (2005), Das and Konar (2005), Liu et al. (2008) and
Caponio and Neri (2009). Poor performance of DE in noisy environment can be explained
by considering the fact (mentioned above) that an efficient parameter setting is very prone to
problems. A noisy optimization problem can be seen as a problem where the fitness landscape
varies with time. Thus, for a noisy problem, a deterministic choice of the scale factor can be
inadequate and a standard DE can easily fail at handling a noisy fitness function, as experi-
mentally shown in Krink et al. (2004). Looking at the problem from a different perspective,
DE employs too much a deterministic search logic for a noisy environment and therefore
tends to stagnate.
During recent years, it has become clear to the scientific community that although DE
is undoubtedly a good algorithm, there are considerable margins of improvement for the
algorithmic structure, especially in solving difficult problems. On the basis of these consid-
erations, many modifications of DE structures have recently been proposed in literature.
The majority of modified DE schemes present in literature propose an adaptation or a
self-adaptation in order to enhance the robustness properties of the algorithm. A simple mod-
ification has been proposed to handle noisy problems in Das and Konar (2005) and Das et al.
(2005). A randomized scale factor according to the following formula has been used:
F = 0.5 (1 + rand (0, 1)) (5)
where rand (0, 1) is a uniformly distributed random number between 0 and 1, see also dither
and jitter in Price et al. (2005).
The employment of the scale factor randomization turned out to be beneficial not only
for noisy problem but also for stationary problems, as shown in Das et al. (2005). Although
this topic is not yet clear to computer scientists, a randomization in the scale factor seems to
compensate the excessively deterministic search structure of a standard DE and offers new
potential search moves. In Das et al. (2005) a time varying scale factor is also proposed.
According to the latter proposal, a decrease in the scale factor value during the optimization
process can lead to high quality final results. The explanation for success of this mecha-
nism can be seen as a progressive narrowing of the search during the evolution and thus
an exploitative behavior at the end of the optimization with the likelihood of preventing
stagnation.
By following a similar idea, in Rönkkönen and Lampinen (2003), Omran et al. (2005) and
Salman et al. (2007), a normal distribution is employed in order to perform a self-adaptation
on the parameters F and C R. A Cauchy distribution in the self-adaptive scheme is proposed
in Soliman et al. (2007) and Soliman and Bui (2008). An alternative kind of self-adaptation
which employs the so called chaos mutation is proposed in Zhenyu et al. (2006). In Ali and
Fatti (2006), on the basis of a theoretical study attempting at determining a probability law
describing the DE mutation mechanism, a probabilistic approach for the offspring generation
is proposed. In paper Nearchou and Omirou (2006), a stochastic operator is integrated within
a DE framework for empowering the DE performance in combinatorial optimization.
123
On the other hand, the employment of randomization in DE is not always considered to be

beneficial. For example, paper Qing (2008) suggests, after a comparative analysis, to avoid
the use of random base vectors.
In Ali and Törn (2004), instead of applying a randomization, a fitness-based adaptation
has been proposed. A system with two evolving populations has been implemented. The
crossover rate C R has been set equal to 0.5 after an empirical study. Unlike C R, the value
of F is adaptively updated at each generation by means of the following scheme:
⎧

⎨ max lmin , 1 − fmax fmax
fmin
if f min < 1
F= (6)
⎩ max lmin , 1 − min
f
otherwise
f max
where lmin = 0.4 is the lower bound of F, f min and f max are the minimum and maximum
fitness values over the individuals of the populations.
In Yang et al. (2008b,b) the Neighborhood Search Differential Evolution (NSDE) is pro-
posed. In the NSDE, the scale factor F is adjusted by sampling values from probability
distributions. The mutation in the NSDE follows a logic inspired by Evolutionary Program-
ming:

(xr − xs ) N (0.5, 0.5) if rand (0, 1) < 0.5
xoff = xt + (7)
(xr − xs ) δ otherwise
where N (0.5, 0.5) is a value sampled from the normal distribution characterized by 0.5
mean value and 0.5 standard deviation; δ is a value sampled from a Cauchy distribution
characterized by a scale parameter equal to 1.
Papers Liu and Lampinen (2002b,c), and Liu and Lampinen (2005) propose the integration
of a fuzzy controller for the adaptive setting of the DE parameters.
Regarding adaptive population size, a first attempt has been proposed in Teo (2005, 2006).
The adaptive population size approach has recently been improved in two different imple-
mentations reported in Teng et al. (2009). Three DE self-adaptive variants in the same fashion
are proposed in Sing et al. (2007). In Tirronen and Neri (2009) a fitness diversity adaptation
for the population size and the other parameters has been proposed (for the fitness diversity
adaptation see Caponio 2007, Neri et al. 2007a,b, Tirronen et al. 2008 and Neri et al. 2007).
A comparative analysis of some adaptive schemes employed in DE is presented in Zielinski
et al. (2008).
Some papers modify the DE schemes by hybridizing DE with other algorithmic struc-
tures, for example, Chiou and Wang (1998, 1999). Papers Lin et al. (1999, 2001), and Su and
Lee (2003) propose hybrid DE algorithms for problems characterized by mixed integer/real
representation. In Hendtlass (2001) and Xu et al. (2008) DE is hybridized with a Particle
Swarm Optimizer. In Chiou et al. (2004) and Zhang et al. (2008) hybridizations of DE with
an Ant Colony Optimizer is proposed. Paper Kaelo and Ali (2007) proposes a DE scheme
with a hybrid mutation rule. Paper He and Han (2007) proposes a hybrid DE based on arti-
ficial immune systems. In Das et al. (2007) and Hu et al. (2008) the hybridization proposed
is with Simulated Annealing. In Caponio et al. (2009) the concept of super-fit adaptation
is introduced and employed for coordination of the various algorithmic components. This
adaptive scheme controls the performance of the best individual with respect to the average
performance of other individuals of the population. In Tirronen et al. (2008) a DE-based Me-
metic Algorithm employing three local search algorithms coordinated by means of fitness
diversity adaptation and a probabilistic scheme has been proposed.
In Zaharie (2003) a DE with a multi-population approach, a parameter adaptation strategy
and population diversity control has been proposed. Following the same idea, in Zaharie
123
and Petcu (2004) an adaptive Pareto DE algorithm for multi-objective optimization has been
designed along with its parallel implementation.
Although multi-objective optimization is beyond the scope of this paper, it is worthwhile
considering some DE versions representative of this class of problems. In Abbass (2001)
and Abbass and Sarker (2002) the concept of Pareto dominance is integrated within a DE
for extending the algorithmic functioning to the multi-objective case. In Abbass (2002) an
improvement of the algorithm in Abbass (2001) is proposed by including self-adaptation
within variation operators. In Madavan (2002), a DE incorporating the elitist sorting defined in
Deb et al. (2002) is presented. In Babu and Jehan (2003) two mechanisms are integrated within
a DE for solving bi-objective problems: (1) to incorporate one objective within the constraints,
(2) to use an aggregation function. In Li and Zhang (2006) the weighted Tchebycheff approach
is proposed for decomposition of the multi-objective problem into a set of single objective
subproblems. In Robič and Filipič (2005) a DE for multi-objective optimization integrating
Pareto-based ranking and crowding distance sorting is presented. For an extensive survey on
DE for multi-objective problems, see Mezura-Montes et al. (2008) and reference therein.
In addition to the DE-based algorithms described in this section, eight recently algorithms
which are, according to our judgment, very relevant deserve an in depth analysis. These
algorithms have been chosen because they offer a high performance in various problems,
contain a ground breaking algorithmic structure and therefore have been object of interest
of high quality journals. The following sections describe these eight algorithms, propose
a classification based on their algorithmic structure and show the results of an extensive
comparison.
4 Additional components in differential evolution
This section gives a description of the four additional components considered in this study and
attempts to justify the algorithmic philosophy which suggests these additions to a standard
DE framework.
4.1 Differential Evolution with Trigonometric Mutation
In Trigonometric Differential Evolution (TDE) the mutation operation in Eq. (1) is replaced,
with a prefixed probability Mt , by an alternative expression, namely trigonometric mutation
The trigonometric mutation, following the description given in Fan and Lampinen (2003b), is:
(xr + xs + xt )
xoff = + ( ps − pr ) (xr − xs ) + ( pt − ps ) (xs − xt ) + ( pr − pt ) (xt − xr )
3
(8)
where for k = r, s, t,
| f (xk )|
pk = . (9)
| f (xr )| + | f (xs )| + | f (xt )|
Thus, the trigonometric mutation is a greedy operator that for three given points generates an
offspring by exploiting the most promising search directions. The employment of this oper-
ator within TDE is supposed to offer an exploitative alternative to the standard exploration
rule of DE. The trigonometric mutation thus has the role of promoting the generation of the
offspring along (locally) optimal directions. In this sense this special mutation can be seen
as a single step local search (see Hart et al. 2004).
123
It is important to remark that, by combining the standard mutation in Eq. (1) and the
trigonometric mutation in Eq. (8), the same authors defined in Fan and Lampinen (2003a)
directed mutation. The formula of directed mutation is given by:
1 − f (xt ) 1 − f (xt )
xoff = xt + (xt − xr ) + (xt − xs ) (10)
f (xr ) f (xs )
where f (xt ) ≤ f (xr ) and f (xt ) ≤ f (xs ).
4.2 Differential Evolution with Adaptive Crossover Local Search
In order to enhance performance of DE, in Noman and Iba (2008) a memetic approach,
called Differential Evolution with Adaptive Hill Climbing Simplex Crossover (DEahcSPX),
has been proposed. The main idea is that a proper balance of the exploration abilities of DE
and the exploitation abilities of a Local Searcher (LS) can lead to an algorithm with high
performance. The proposed algorithm hybridizes DE described in Sect. 2 as an evolutionary
framework and a LS is deterministically applied to the individual of the DE population with
the best performance (in terms of fitness value).
The LS proposed for this hybridization is Simplex Crossover (SPX) Tsutsui et al. (1999).
More specifically, at each generation, that individual having the best fitness value, indicated
here with xb , is extracted and the LS described in Fig. 2 is applied. If the SPX succeeds in
improving upon the starting solution, a replacement occurs according to a meta-Lamarckian
logic Ong and Keane (2004).
It should be remarked that in Fig. 2 is a control parameter of the SPX which has been set
equal to 1 in Noman and Iba (2008). Finally, the DE framework employed is DE/rand/1/bin
described in Fig. 1.
Fig. 2 SPX pseudo-code
123
4.3 Differential Evolution with Population Size Reduction
The Differential Evolution with Population Size Reduction (DEPSR) employs, within a DE
framework, a variable population size which is progressively reduced during the optimization
process, see Brest and Maučec (2008). This population size reduction requires that initial
1 , total budget T , in terms of fitness evaluations, and number of stages N
population size Spop b s
(i.e. the number of population sizes employed during the algorithm’s run) are prearranged.
Thus, the total budget of the algorithm is divided into Ns periods, each period being char-
acterized by a population size value Spop k (for k = 1 we obtain the initial population size).
k
Each period is composed of N g generations which are calculated in the following way:

Tb
N gk = k
+ rk (11)
Ns Spop
where rk is a constant non-negative value which takes a positive value when Tb is not divisible
by Ns . In this case rk extra generations are performed. The population reduction is simply
carried out by halving the population size at the beginning of the new stage, see Brest and
Sk
Maučec (2008). In other words, for k = 1, 2, . . . , Ns − 1, Spop
k+1 = pop .
2
In this way, population size is progressively reduced during the optimization process until
the final budget is reached. The concept behind this strategy is that of focusing the search in
progressively smaller search spaces in order to inhibit the DE stagnation in an environment
with high dimensionality. During the early stages of the optimization process, the search
requires a highly explorative search rule, i.e. a large population size, in order to explore a
large portion of the decision space. During optimization, the search space is progressively
narrowed by decreasing the population size and thus exploiting the promising search direc-
tions previously detected. Although the number of stages and the population size values
remain arbitrary issues defined by the algorithmic designer, the idea seems to lead to a fairly
robust algorithmic behavior for the setting proposed in Zamuda et al. (2008) and seems to be
very promising for large scale problems, as highlighted in Brest and Maučec (2008).
The last topic to be clarified is the selection rule employed every time a population reduc-
tion occurs. At the end of each stage, i.e. at each N gk generation for k = 2, 3, . . . , Ns , the
population is divided into two sub-populations on the basis of the position index i of the
Sk
individuals. Each sub-population encounters pop 2 individuals. Thus, the first sub-population
is composed of the candidate solutions x1 , x2 , . . . , x Spop k while the second sub-population
2
is composed of the candidate solutions x Spop k +1 , x Spop k +2 , . . . , x Spop
k . The selection occurs
2 2
by applying one-to-one spawning, typical of the DE logic, to the two sub-populations anal-
ogous to the selection among parent and offspring individuals in a standard DE scheme. In
k
Spop
other words, the individuals xi and x Spop k +i are pairwise compared, for i = 1, 2, .. 2 , and
2
those individuals having the most promising fitness values are retained for the subsequent
generation.
Finally, it should be remarked that in order to guarantee a proper functioning of the pop-
ulation reduction mechanism, populations should never undergo sorting of any kind.
4.4 Differential Evolution with Scale Factor Local Search
Differential Evolution with Scale Factor Local Search (DESFLS) has been introduced in
Tirronen et al. (2009) and extended in Neri et al. (2009) for self-adaptive DE schemes. An
123
Fig. 3 Fitness function, f (F),

pseudo-code
improved version that we are referring to here, has been proposed in Neri and Tirronen
(2009). An application of this algorithmic strategy is also available in Leskinen et al. (2009).
The main idea in these algorithms is that a local search is applied, with a certain probability,
to the scale factor during the generation of an offspring individual. In other words, update of
the scale factor and thus generation of the offspring is, with a certain probability, controlled
in order to guarantee a high quality solution which can take on a key role in subsequent
generations, see also Lozano et al. (2004).
As shown in Tirronen et al. (2009), local search in the scale factor space can be seen as the
minimization over the variable F of fitness function f in the direction given by xr and xs and
modified by the crossover. More specifically, at first the scale factor local search determines
those genes which are undergoing crossover by means of the standard criterion explained in
Eq. (3), then it attempts to find the scale factor value which guarantees an offspring with the
best performance. Thus, for given values of xt , xr , xs , and the set of design variables to be
swapped during the crossover operation, the scale factor local search attempts to solve the
following minimization problem:
min f (F) in [−1.2, 1.2] . (12)
F
For sake of clarity, the procedure describing the fitness function is shown in Fig. 3.
It must be remarked that each scale factor (during the local search) corresponds to an off-
spring solution xoff , thus the proposed local search can be seen as an instrument for detecting
solutions with high quality over the directions suggested by a DE scheme. Negative values
of F are admitted up to −1.2, see Neri and Tirronen (2009). The meaning of the negative
scale factor is obviously, in this context, inversion of the search direction.
In order to perform this minimization, two local search algorithms have been considered
and compared in Tirronen et al. (2009) and Neri et al. (2009). Experimental results in Neri
and Tirronen (2009) showed that cooperative employment of both the algorithms leads to the
best performance. The two local search algorithms are described in the following.
4.4.1 Scale Factor Golden Section Search
Golden Section Search is a classical local search algorithm for non-differentiable fitness func-
tions introduced in Kiefer (1953). The Scale Factor Golden Section Search (SFGSS) applies
the Golden Section Search to the scale factor in order to generate a high quality offspring. The
SFGSS processes the interval [a = −1.2, b = 1.2] and generates two intermediate points:
b−a
F1 = b −
φ
b−a
F =a+
2
(13)
φ
123
Fig. 4 SFGSS pseudo-code
√
where φ = 1+2 5 is the golden ratio. The values of f F 1 and f F 2 are then computed and
1 2
compared. If f F < f F then F 2 replaces b and the procedure is repeated in the new

smaller [a, b] interval. If f F 1 ≥ f F 2 , F 1 replaces a and the procedure is repeated
in the new interval. The SFGSS is interrupted when a computational budget is exceeded.
Figure 4 shows the SFGSS pseudo-code.
In the context of the DE multidimensional search, the SFGSS performs control of the off-
spring generation by at first individuating a rough estimation of a promising scale factor value
(i.e. detects whether a high or low scale factor value is preferable) and then progressively
refines the search.
4.4.2 Scale Factor Hill-Climb
The uni-dimensional hill-climb local search is one of the simplest and most popular opti-
mization algorithms present in any optimization book e.g. Russell and Norvig (2003). The
algorithm uses the current value of F as a starting point and is composed of an exploratory
move and a decisional move. The exploratory move samples F − h and F + h where h is a
step size. The decisional move computes the min{ f (F − h) , f (F) , f (F + h)} and selects
the corresponding point as the center of the next exploratory move. If the center of the new
exploratory move is still F, step size h is halved. The local search is stopped when a budget
condition is exceeded. For the sake of providing a complete picture, the pseudo-code of the
Scale Factor Hill-Climb (SFHC) is shown in Fig. 5.
It should be remarked that the SFHC is a local search algorithm characterized by a steepest
descent pivot rule, see Hart et al. (2004), i.e. an algorithm which explores the whole neigh-
borhood of the candidate solution before making a decision on the search direction. This
property makes, in general, the local search accurate and thus relatively computationally
expensive. The computational cost in one dimension cannot, in any case, be dramatically
high.
It is interesting to visualize the functioning of this local searcher in terms of generation of
an offspring within a DE for a multi-dimensional problem. Since the scale factor is related
to the modulus of a moving vector (xr − xs ) in the generation of the preliminary offspring,
the SFHC in operation can be seen as a pulsing vector in a multi-dimensional space which
tunes to the best offspring and then generates this offspring.
123
Fig. 5 SFHC pseudo-code
The local search algorithms are applied with a certain probability to the individual with the
best performance. Thus, at each generation, the generation of one individual, occurs either
by means of SFGSS or SFHC or according to the standard DE mutation and crossover. More
specifically, when the best individual is taken into account for the offspring generation, with
a probability pG generation of the offspring is performed by means of the SFGSS, with a
probability p H generation of the offspring is performed by means of the SFHC, and with
a probability 1 − ( pG + p H ) the offspring generation occurs according to the standard DE
rules.
4.5 Discussion about additional components
Four algorithmic components which are supposed to improve upon the algorithmic perfor-
mance of a DE have been presented in this section. These components are clearly very diverse
in terms of algorithmic logic, structure and computational cost.
The trigonometric mutation is an alternative mutation which does not require extra fitness
evaluations. The simplex crossover and scale factor local search are local search algorithms,
the former operating on the chromosome, the latter on the scale factor which require extra
fitness evaluations over the average single DE generation. The population reduction does not
explicitly add an extra algorithmic structure but adds a rule within the DE scheme. For the
last component, there also is no increase in terms of computational overhead.
Although these components are very diverse, they have something in common in terms
of their algorithmic role and functioning principle. A general observation can be that these
additional components offer an alternative perspective to the DE search and thus, with ref-
erence to the explanation given in Sect. 3, allow extra moves that can enrich the original
set of DE moves. Furthermore, these moves tend to increase the exploitative pressure within
the explorative DE structure. It is clear that the presence of local search algorithms, such as
the simplex crossover and the scale factor local search, allow an increase of the exploitative
pressure and therefore a higher exploitation of promising search directions. The trigonomet-
ric mutation, as highlighted above, can be seen as a single step local search since it proposes
a guided offspring generation based on a gradient estimation. The population size reduction
corresponds to the restriction in the set of the search moves with a preference towards those
that are most promising. Although this component does not explicitly offer alternative search
moves, it progressively narrows the space in which the search is performed by eliminating
individuals characterized by a poor performance. This makes the algorithm more exploit-
123
ative and thus reduces the risk of stagnation. In other words, this component does not help to
detect the global optimum, but is fundamental in order to quickly improve upon the obtained
results after completing the exploratory procedure. To give an analogy, progressive reduction
in the population size is similar to a progressive increase in the selection pressure in Genetic
Algorithms. Following a different analogy, this mechanism is similar to a cascade algorithm
composed of as many algorithms as the amount of stages Ns , see Eq. (11). The search by
each algorithm is progressively focused into a smaller space after these promising search
directions are detected.
5 Modified structures of Differential Evolution
This section describes DE-based algorithms which involve a modification in the DE struc-
tures.
5.1 Self-Adapting Parameter Setting in Differential Evolution (jDE)
In order to avoid the manual parameter setting of F and C R a simple and effective strat-
egy has been proposed in Brest et al. (2006a). This strategy is called Self-Adapting Control
Parameters in Differential Evolution. The DE algorithm employing this strategy, namely jDE,
consists of the following.
With reference to Fig. 1, when the initial population is generated, two extra values between
0 and 1 are also generated per each individual. These values represent F and C R related to
the individual under analysis. Each individual is thus composed (in a self-adaptive logic) of
its genotype and its control parameters:

xi = xi,1 , xi,2 , . . . , xi, j , . . . xi,n , Fi , C Ri .
In accordance with a self-adaptive logic, see e.g., Rechemberg (1973) and Ohkura et al.
(2001), the variation operations are preceded by the parameter update. More specifically
when, at each generation, the ith individual xi is taken into account and three other individ-
uals are extracted pseudo-randomly, its parameters Fi and C Ri are updated according to the
following scheme:

Fl + Fu rand1 , if rand2 < τ1
Fi = (14)
Fi , otherwise

rand3 , if rand4 < τ2
C Ri = (15)
C Ri , otherwise
where rand j , j ∈ {1, 2, 3, 4}, are uniform pseudo-random values between 0 and 1; τ1 and τ2
are constant values which represent the probabilities that parameters are updated, Fl and Fu
are constant values which represent the minimum value that F could take and the maximum
variable contribution to F, respectively. The newly calculated values of Fi and C Ri are then
used for generating the offspring. The variation operators and selection scheme are identical
to that of a standard DE (see Sect. 2).
This self-adaptation seems to greatly enhance the robustness properties of a standard
DE and has also shown a good performance for large scale problems Brest et al. (2008),
constrained optimization Brest et al. (2006b) and multi-objective problems Zamuda et al.
(2007). A modified version of jDE has been presented in Brest et al. (2007). This kind of
self-adaptation is extensively discussed in Brest et al. (2008).
123
Fig. 6 jDE pseudo-code
For sake of clarity, the pseudo-code highlighting the working principle of jDE is given in
Fig. 6.
5.2 Opposition Based Differential Evolution
The Opposition Based Differential Evolution (OBDE), proposed in Rahnamayan et al.

(2006a) and Rahnamayan et al. (2008a), employs logic of the opposition points in order
to enhance search properties of DE and test a wide portion of the decision space.
For a given point xi = xi,1 , xi,2 , . . . , xi, j , . . . , xi,n belonging to a set D = [a1 , b1 ] ×
[a2 , b2 ] × · · · × [a j , b j ] × · · · × [an , bn ] its opposition point is defined as: x̃i =
a1 + b1 − xi,1 , a2 + b2 − xi,2 , . . . , a j + b j − xi, j , . . . , an + bn − xi,n .
The OBDE algorithm consists of a DE framework and two opposition based components:
the first after the initial sampling and the second after the survivor selection scheme. After
the initial sampling of Spop individuals, for each point xi = xi,1 , xi,2 , . . . , xi, j , . . . , xi,n
belonging to a set D = [a1 , b1 ] × [a2 , b2 ] × . . . × [a j , b j ] × . . . × [an , bn ] its opposition
point is calculated as:

x̃i = a1 + b1 − xi,1 , a2 + b2 − xi,2 , . . . , a j + b j − xi, j , . . . , an + bn − xi,n . (16)
The fitness values of both groups of points (original and opposition) are calculated. Then,
the Spop individuals having the best performance are selected for the first generation.
123
At each subsequent generation, with a probability jr (jump rate), the opposition based
again. In this case, instead of using the predefined boundaries [ai , bi ],
component is activated
for each point xi = xi,1 , xi,2 , . . . , xi, j , . . . , xi,n its opposition point is calculated as:

x̃i = , min xi,1 + max xi,1 − xi,1 , . . . , min xi, j + max xi, j − xi, j , . . . , min xi,n
i i i i i

+ max xi,n − xi,n (17)
i
where min xi, j and max xi, j are respectively the minimum and maximum values, over the
i i
coordinate j, taken by individuals of the population at the present generation. In other words,
at each generation, the range of variability of the individuals is taken as a reference (bounding
box) for the calculation of the opposition points. The two sets of points (original and oppo-
sition) are merged and those Spop points which have the best performance are then selected
for the subsequent generation.
This modification of the DE structure demonstrated a great potential for solving some of
the more difficult problems, especially for noisy problems Rahnamayan et al. (2006b) and
large scale problems Rahnamayan and Wang (2008). A related algorithm, namely Quasi-
Oppositional Differential Evolution has been presented in Rahnamayan et al. (2007). The
topic of OBDE is extensively discussed in Rahnamayan et al. (2008b).
For sake of clarity the pseudo-code describing the functioning of OBDE is shown in Fig. 7.
5.3 Differential Evolution with Global and Local Neighborhoods
The Differential Evolution with Global and Local Neighborhoods (DEGL) modifies the muta-
tion operation in DE, explained in Sect. 2, by defining a neighborhood as a portion of the
population identified by a radius k, see Chakraborty et al. (2006) and Das et al. (2009). More
specifically, individuals of the population are pseudo-randomly sorted and each individual
is characterized by a position index i. The neighborhood of the ith individual xi is given by
those individuals xi−k , . . . , xi , . . . , xi+k .
The concept of neighborhood is used during the mutation operation since the provisional
is acquired through a combination of two contributions, the first contribution
offspring xoff
is given by the neighborhood individuals and the second by the entire population. Thus, in
order to perform the mutation, for an individual xi , the local contribution is calculated as:

L i = xi + α (xn−best − xi ) + β x p − xq (18)
where xn−best is the individual having best performance in the neighborhood; x p and xq are
two individuals pseudo-randomly selected from the neighborhood. Values α and β are two
constants which have a similar role to that of the scale factor F, see Eq. (1). The global
contribution is given by:

G i = xi + α x p−best − xi + β (xr − xs ) (19)
where x p−best is that individual having the best performance out of the entire population, xr
and xs are two individuals pseudo-randomly selected from the population. The two contri-
butions are then combined by means of:
xoff = wG i + (1 − w) L i (20)
where wi is a weight factor to be set between 0 and 1.
123
Fig. 7 OBDE pseudo-code
Regarding the parameter setting, as suggested in Chakraborty et al. (2006), it has been set
α = β equal to a constant value, indicated here (for analogy with the scale factor) with F.
Also the neighborhood radius k has been set as a constant.
On the contrary, the weight factor w varies during the optimization process. According to
the original definition of DEGL, the weight factor varies in the following way:
g
w = wmin + (wmax − wmin ) (21)
gmax
123
where wmin and wmax are the lower and upper bounds of the weight factor, respectively. The
indexes g and gmax denote the current generation index and the maximum amount of gener-
ations, respectively. In other words, at the beginning of the optimization process (g = 0) the
weight factor is set equal to wmin and subsequently linearly varies over time. At the end of
the optimization process, the weight factor takes the value wmax .
In Das et al. (2009), four weight factor update schemes have been presented and compared.
The first one is simply the linear increase shown in Eq. (21) from 0 to 1:
g
w= . (22)
gmax
The second weight factor update is an increase exponential rule based on the generation
number:

g
w = exp ln (2) − 1. (23)
gmax
The third scheme is a simple randomization (uniform distribution) of the weight factor:
w = rand (0, 1).
The fourth scheme, which in Das et al. (2009) proved to be the most efficient and is con-
sidered in this study for comparison, is a self-adaptive update of the weight factor. In this
scheme, each candidate solution has its own weight factor:

xi = xi,1 , xi,2 , . . . , xi, j , . . . xi,n , wi .
During the initial sampling, each scale factor is initialized between 0 and 1. At each genera-
tion, before each moving operation described in Eq. (20), the weight factor wi associated to
solution xi is updated according to the following formula:
wi = wi + F (wbest − wi ) + F (wr − ws ) (24)
where wbest is the weight factor associated to that individual of the population having the
best performance and the values wr and ws are the weight factors associated to xr and xs in
Eq. (19). As suggested in Das et al. (2009), each weight factor is restricted within the interval
[0.05, 0.95] by saturating to the closest bound those values which are generated outside the
interval.
The crossover and replacement occur as with a plain DE, see Sect. 2. It should be noted
that the weight factor does not undergo crossover operation.
For sake of clarity the pseudo-code describing the functioning of DEGL is shown in Fig. 8.
5.4 Self-Adaptive Differential Evolution
The main feature of the algorithms belonging to this class is the employment of multiple
mutation strategy. The first version of Self-Adaptive Differential Evolution (SADE) has
been proposed in Qin and Suganthan (2005) and a more sophisticated and efficient imple-
mentation, of which we refer to in this Subsection, has recently been proposed in Qin et al.
(2009).
The main idea is that within the genotype of each individual some probabilities related
to the mutation strategy for the subsequent generation are encoded. More specifically, for a
given candidate solution xi , the individual looks like:

xi = xi,1 , xi,2 , . . . , xi, j , . . . xi,n , Fi , C Ri1 , C Ri2 , C Ri3 , C Ri4 , pi1 , pi2 , pi3 , pi4
123
Fig. 8 DEGL pseudo-code
where pik for k = 1, 2, 3, 4 is the probability that the mutation strategy 1, 2, 3, or 4, respec-
tively, isemployed in the moving operation related to individual xi . It should be remarked
that ∀i, 4k=1 pik = 1.
The four mutation strategies considered in Qin et al. (2009) are: DE/rand/1, DE/rand-to-
best/2, DE/rand/2, and DE/current-to-rand/1, see Sect. 2. Clearly, for the offspring generation,
the crossover is applied in the case of DE/rand/1, DE/rand-to-best/2, and DE/rand/2, but not
in the case of DE/current-to-rand/1.
When the initial sampling is performed for each solution, each probability is set equal
to 0.25. During the subsequent L P generations, (L P stands for Learning Period), for each
individual the number of successful generations n ks related to a certain mutation strategy, i.e.
the number of offspring generated by the k th strategy outperforming the generating parent,
is saved. In an analogous way, the number of failures n kf is also saved.
At the end of the learning period (after LP generations), the probabilities are updated for
each individual xi , at each generation G, according to the formula:
Sk
pik = 4 i k
(25)
k=1 Si
where
G−1
g=G−L P n ks
Sik = G−1 G−1 +ε (26)
g=G−L P n ks + g=G−L P n kf
123
where g is a generation index and ε is a small constant value equal to 0.01 whose role is sim-
ply to ensure a numerical stability of the algorithm if Sik = 0 for all the strategies. Thus, Sik
represents the success rate of the offspring solutions generated by means of the k th strategy.
The division in Eq. (25) is performed in order to guarantee a normalization of the probabili-
ties (the sum of probabilities is 1). At the end the self-adaptive strategy in the SADE follows
a trial and error approach to determine the probabilities for choosing the search structure
in DE. Given the four probabilities, the mutation strategy is chosen by means of Stochastic
Universal Sampling (SUS), see Baker (1987).
As shown above, within each candidate solution, Fi and C Ri are also encoded. The scale
factor Fi is generated, for each individual, by sampling a value from a normal distribution
having mean value in 0.5 and standard deviation equal to 0.3. Regarding the K factor in
DE/current-to-rand/1, a randomization is performed by sampling this value from a uniform
distribution between 0 and 1.
The setting of C Ri is also self-adaptively performed. When the initial sampling is exe-
cuted, C Rik is set equal to 0.5 for all the individuals and all the strategies. During the first
L P generations, each C Rik is updated by sampling a value from a normal distribution having
its center in C Rik and standard deviation equal to 0.1. During the learning period, for each
mutation strategy, the set of crossover rates which led to a successful offspring generation
C Rsk are saved. After the learning period, C Rik is updated by the median of C Rsk and the new
C Rsk is calculated by sampling a value from a normal distribution having its center in C Rik
and standard deviation equal to 0.1.
For sake of clarity the pseudo-code describing the functioning of the SADE is shown in
Fig. 9.
In literature, some algorithms which hybridize the features of two modified schemes have
recently been presented. For example in the paper Brest et al. (2007), the enhanced version
of jDE employs some multiple mutation strategies following a logic inspired by the SADE
model. In Yang et al. (2008b) an algorithm combining the structure of the NSDE proposed
in Yang et al. (2008b) and the self-adaptation of the SADE has been proposed.
5.5 Discussion about Modified Differential Evolution Schemes
In this section four modified schemes of DE have been presented. It is clear that all the
modifications proposed tend to include extra moves within a standard DE and all of them
attempt to find a logic for proposing the correct extra moves. The jDE functioning is based
on the idea that fixed values of F and C R can cause stagnation in a DE scheme. On the other
hand, a full randomization of the control parameters would lead to an algorithm which is
very similar to random search. For this reason, the restricted randomization proposed in jDE
seems to be an efficient compromise between the necessity of generating extra moves and
the willingness to keep some exploitative features of the optimization algorithm.
The OBDE algorithm proposes an attempt to check, under certain probability conditions,
unexplored areas of the decision space regardless the stage of evolution of the population.
The generation of extra individuals by means of projection with respect to a focus (opposition
points) relies on the fact that the extra points test unexplored areas and ”refresh” DE logic.
The DEGL is based on a slightly different logic. The additional moves to DE are given by
the combination of two contributions during offspring generation. According to our interpre-
tation, the action of the local component within its neighborhood is similar to the population
reduction scheme described above. More specifically, the concept of neighborhood allows a
mapping of the population into smaller populations (even if these small populations overlap)
and thus the local component can be seen as an exploitative move within a restricted portion
123
Fig. 9 SADE pseudo-code
of the population. This move is balanced by the explorative features of the global contribu-
tion. The combination of these two complementary actions make possible, in our opinion,
the success of DEGL.
The SADE has a rather straightforward interpretation: the employment of a multiple
mutation strategy allows an increase in the DE moves. On the other hand, the trial and error
self-adaptation increases exploitative features of the algorithm since successful moves tend
to be repeated and constitute a preferential choice during subsequent generations.
In our opinion, a crucial aspect in the DE enhancement is the increase of randomization
with respect to the standard DE. It should be remarked that all the modified schemes employ
a randomization. The stagnation problem and the limited amount of moves in a DE can be
seen as a consequence of the fact that the standard DE is too deterministic an algorithm.
This explains why in many enhanced versions of DE, illustrated in Sect. 3 a randomization
within the generation of the offspring is proposed. Obviously, as stated before, an excessive
randomization could lead to a random search which can be harmful in terms of computational
overhead. On the contrary, a proper balance in randomization can be extremely beneficial and
a crucial element for the algorithmic success. The search for this balance is, in our opinion, the
future of the DE scheme development. As further justification of this statement, it is worth-
while considering that the enhanced version of the SADE in Qin et al. (2009), with respect
123
Table 1 Test problems
Test problem Function Decision space

Ackley −20 + e + 20 exp − n 0.2 n 2 [−1, 1]n
i=1 xi

− exp n1 i=1 n cos(2π · x )x
i i
n
Alpine i=1 |x i sin x i + 0.1x i | [−10, 10]n
DeJong ||x||2 [−5.12, 5.12]n
√
1+cos 12 ||x||2
DropWave − 1 [−5.12, 5.12]n
2 ||x|| +2
2
||x||2 n xi
Griewangk 4000 −
√ +1
i=0 cos [−600, 600]n
i 2 20

n i·xi
Michalewicz − i=1 sin xi sin π [0, π ]n
⎛ ⎞
n−1 sin2 100xi2 +xi+12 −0.5
Pathological ⎝0.5 + 2 ⎠ [−100, 100]n
i=1
1+0.001× xi2 −2xi xi+1 +xi+1
2
n 2
Rastrigin 10n + i=0 xi − 10 cos(2π xi ) [−5.12, 5.12]n
2
n−1 2
Rosenbrock valley i=1 100 xi+1 − xi + (1 − xi )2 [−2.048, 2.048]n
n √
Schwefel i=1 xi sin |xi | [−500, 500]n
n i+1
Sum of Powers |x
i=1 i | [−1, 1]n

2
Tirronen 3 exp − ||x||10n − 10 exp −8||x||2 [−10, 5]n
n
+ 2.5
n i=1 cos (5xi (1 + i mod 2)) cos (||x||)

2 n i xi 2 n i xi 4
Zakharov ||x|| + i=1 2 + i=1 2 [−5, 10]n
to the original algorithm in Qin and Suganthan (2005), employs an extra randomization in
the choice of F and C R.
6 Numerical results
The eight algorithms described above have been tested and compared. Experiments related
to a standard DE have also been included as a reference. The thirteen test problems listed in
Table 1 have been considered in this study.
The Tirronen function in Table 1 has recently been defined, see Tirronen and Neri (2009)
for details. For some of the test problems listed in Table 1, its rotated version has been
included into the benchmark set. These rotated problems have been generated by means of
multiplication of the vector of variables to a randomly generated orthogonal rotation matrix.
Thus, a total of twenty-four test problems have been taken into account in this paper.
The DE has been run with F = 0.7 and C R = 0.3 in accordance with the suggestions
given in Zielinski et al. (2006) and Zielinski and Laur (2008).
Regarding the DE schemes with additional components the following parameter setting
has been performed.
– TDE has been run with F = 0.7, C R = 0.3 and with the trigonometric mutation proba-
bility Mt = 0.05, as suggested in Fan and Lampinen (2003b).
123
– DEahcSPX has been run with F = 0.7 and C R = 0.3. With reference to Fig. 2, n p has
been set equal to 10 and the simplex crossover is run with a maximum budget of 500
fitness evaluations.
– DEPSR has been run with F = 0.7 and C R = 0.3 and a Ns = 4 number of periods, as
suggested in Brest and Maučec (2008).
– DESFLS has been run with F = 0.7 and C R = 0.3. The SFGSS is run with a maximum
budget of 8 fitness evaluations and is activated with a probability pG = 0.03. The SFHC
is run with initial step size h = 0.1, a maximum budget of 30 fitness evaluations and is
activated with a probability p H = 0.04.
Regarding the modified DE structures the following parameter setting has been performed.
– jDE has been run with Fl = 0.1, Fu = 0.9, and τ1 = τ2 = 0.1, as suggested in Brest et
al. (2006a).
– OBDE has been run with F = 0.7, C R = 0.3 and the jump rate jr = 0.3, as suggested
in Rahnamayan et al. (2008a).
– DEGL has been run with α = β = F = 0.7 and C R = 0.3.
– SADE, apart from the Spop , which will be discussed later, is parameterless.
In order to estimate the dependence of the algorithmic performance on the dimensionality,

the eight algorithms have been run in order to perform minimization of the twenty-four test
problems mentioned above in 50 and 100 dimensions.
In accordance with the results reported in Neri and Tirronen (2008), the population size
Spop has been set equal to 50 for the 50-dimensional problems and 100 for the 100-dimen-
sional problems, for seven of the eight algorithms under analysis. For DEPSR, the initial
population size Spop1 has been set equal to 50 and 100 for 50 and 100 dimensions, respec-
tively. For DEGL, the neighborhood size k has been set equal to 20 and 40 for 50 and 100
dimensions, respectively.
In order to have statistically significant results, each algorithm has been run 50 times
for each test problem. Each single run has been performed with a computational budget of
100000 fitness evaluations.
This results section is divided into four subsections. Section 6.1 gives results for DE with
additional components in 50 dimensions. Section 6.2 gives results for the modified DE struc-
tures in 50 dimensions. Section 6.3 gives results for DE with additional components in 100
dimensions. Section 6.4 gives results for the modified DE structures in 100 dimensions. The
results related to DE/rand/bin (simply indicated as DE) are reported twice, for both additional
component and modified structure sections, in order to have a reference with the original DE.
6.1 Numerical Results: Additional Components in 50 Dimensions
For DE with additional components, Table 2 shows the final average detected results by
each algorithm ± the standard deviations for the 50 dimension case. The best results are
highlighted in bold face.
In order to prove the statistical significance of the results in Table 2, a two tail unequal
variance t-test has been applied according to the description given in Ruxton (2006) (see
also NIST/SEMATECH 2003) for a confidence level of 0.95. Table 3 shows the results of
the test. More specifically, Table 3 shows a ranking of the algorithms based on the statisti-
cal significance of the results. If one algorithm significantly outperforms another algorithm,
they appear in separate cells (over the same row). If, on the contrary, there is no statistical
significance, the algorithms have the same ranking (they appear in the same cell).
123
Table 2 Average final fitness values ± standard deviations for DE with additional components in 50 dimensions
DE TDE DEahcSPX DEPSR DESFLS
Ackley 8.37e−6 ± 1.06e−5 6.48e−2 ± 1.99e−1 6.30e−7 ± 9.03e−7 6.84e−3 ± 5.24e−2 6.91e−15 ± 1.30e−14
Alpine 6.53e−2 ± 7.43e−1 9.96e−2 ± 6.10e−1 3.12e−2 ± 2.54e−2 5.41e−3 ± 6.90e−2 5.88e−7 ± 1.99e−5
deJong 7.58e−9 ± 2.62e−8 5.05e−1 ± 2.92e0 3.71e−11 ± 1.19e−10 2.90e−2 ± 3.70e−1 2.16e−39 ± 4.56e−38
Dropwave −5.51e−1 ± 4.55e−1 −6.11e−1 ± 7.06e−1 −8.06e−1 ± 3.11e−1 −6.67e−1 ± 6.08e−1 −8.76e−1 ± 5.70e−1
Griewangk 4.66e−6 ± 2.24e−5 2.50e0 ± 1.06e1 2.18e−8 ± 7.75e−8 5.52e−1 ± 5.75e0 0.00e0 ± 0.00e0
Michalewicz −4.41e1 ± 4.48e0 −4.42e1 ± 7.08e0 −4.41e1 ± 3.76e0 −4.22e1 ± 8.77e0 −4.40e1 ± 5.75e0
Pathological 2.45e1 ± 1.35e−4 2.45e1 ± 1.58e−5 2.45e1 ± 1.40e−4 2.45e1 ± 2.20e−5 2.45e1 ± 1.52e−5
Recent advances in differential evolution
Rastrigin 6.91e1 ± 5.06e1 2.54e1 ± 4.60e1 6.77e1 ± 4.54e1 5.84e1 ± 9.20e1 5.99e−1 ± 1.35e1
Rosenbrock valley 7.33e−5 ± 1.95e−4 3.66e0 ± 8.61e0 1.97e−6 ± 5.56e−6 6.48e−1 ± 3.27e0 6.52e−1 ± 3.56e0
Schwefel −1.94e4 ± 3.71e3 −1.86e4 ± 2.95e3 −1.97e4 ± 3.22e3 −1.74e4 ± 3.86e3 −1.89e4 ± 3.01e3
Sum of Powers 2.42e−31 ± 3.43e−30 6.19e−9 ± 1.50e−7 1.24e−37 ± 1.58e−36 6.41e−9 ± 2.31e−7 1.07e−110 ± 5.28e−109
Tirronen −1.86e0 ± 3.56e−1 −1.78e0 ± 6.55e−1 −1.86e0 ± 4.22e−1 −2.15e0 ± 5.70e−1 −1.79e0 ± 5.85e−1
Zakharov 5.57e2 ± 3.97e2 3.87e1 ± 1.05e2 6.00e1 ± 6.82e1 7.18e1 ± 1.47e2 4.85e1 ± 1.87e2
Rotated Ackley 4.39e−5 ± 6.88e−5 9.53e−2 ± 4.13e−1 2.79e−6 ± 5.21e−6 1.70e−2 ± 1.54e−1 7.12e−15 ± 1.19e−14
Rotated Alpine 6.42e1 ± 4.21e1 1.53e0 ± 6.88e0 2.80e1 ± 2.25e1 1.34e−1 ± 1.11e0 1.46e−2 ± 1.05e−1
Rotated Griewangk 9.08e−2 ± 2.50e−1 2.96e0 ± 1.27e1 5.55e−3 ± 2.10e−2 5.52e−1 ± 4.73e0 4.07e−5 ± 2.01e−3
Rotated Michalewicz −9.94e0 ± 4.20e0 −1.17e1 ± 6.79e0 −1.01e1 ± 4.36e0 −1.25e1 ± 8.35e0 −1.17e1 ± 6.06e0
Rotated Pathological 2.45e1 ± 7.48e−4 2.45e1 ± 6.65e−4 2.45e1 ± 6.85e−4 2.45e1 ± 1.81e−4 2.45e1 ± 6.42e−4
Rotated Rastrigin 4.44e2 ± 1.10e2 3.14e2 ± 1.26e2 3.63e2 ± 1.18e2 2.86e2 ± 1.89e2 3.31e2 ± 4.90e2
Rotated Rosenbrockvalley 2.08e1 ± 1.70e−4 2.16e1 ± 6.78e0 2.08e1 ± 4.05e−6 2.09e1 ± 9.43e−1 2.09e1 ± 1.16e0
Rotated Schwefel −7.77e3 ± 2.78e3 −7.38e3 ± 3.05e3 −7.78e3 ± 2.58e3 −9.24e3 ± 7.79e3 −7.42e3 ± 3.36e3
Rotated Sum of Powers 3.73e−4 ± 1.36e−3 1.91e−6 ± 8.54e−6 3.42e−7 ± 1.59e−6 6.33e−8 ± 5.30e−7 2.36e−6 ± 1.11e−5
Rotated Tirronen −1.02e0 ± 4.61e−1 −9.99e−1 ± 4.03e−1 −1.03e0 ± 5.27e−1 −1.02e0 ± 5.78e−1 −1.00e0 ± 3.30e−1
Rotated Zakharov 6.03e2 ± 4.79e2 2.17e1 ± 6.81e1 7.49e1 ± 1.01e2 8.38e1 ± 1.88e2 2.65e1 ± 1.05e2
123
85
86
Table 3 Results of the unequal variance t-test for DE with additional components in 50 dimensions
1st 2nd 3rd 4th 5th
123
Ackley DESFLS DEahcSPX DE DEPSR TDE
Alpine DESFLS DEPSR DEahcSPX DE TDE
deJong DESFLS DEahcSPX DE DEPSR TDE
Dropwave DESFLS DEahcSPX DEPSR TDE DE
Griewangk DESFLS DEahcSPX DE DEPSR TDE
Michalewicz TDE DE/DEahcSPX – DESFLS DEPSR
Pathological DEPSR DESFLS TDE DEahcSPX DE
Rastrigin DESFLS TDE DEPSR DEahcSPX DE
Rosenbrock valley DEahcSPX DE DEPSR/DESFLS – TDE
Schwefel DEahcSPX DE DESFLS TDE DEPSR
Sum of Powers DESFLS DEahcSPX DE TDE/DEPSR –
Tirronen DEPSR DEahcSPX/DE – DESFLS TDE
Zakharov TDE DESFLS DEahcSPX DEPSR DE
Rotated Ackley DESFLS DEahcSPX DE DEPSR TDE
Rotated Alpine DESFLS DEPSR TDE DEahcSPX DE
Rotated Griewangk DESFLS DEahcSPX DE DEPSR TDE
Rotated Michalewicz DEPSR TDE/DESFLS – DEahcSPX DE
Rotated Pathological DEPSR DESFLS TDE DEahcSPX DE
Rotated Rastrigin DEPSR TDE DESFLS DEahcSPX DE
Rotated Rosenbrockvalley DEahcSPX DE DEPSR/DESFLS – TDE
Rotated Schwefel DEPSR DEahcSPX/DE – DESFLS/TDE –
Rotated Sum of Powers DEPSR DEahcSPX TDE DESFLS DE
Rotated Tirronen DEahcSPX/DEPSR/DE – – DESFLS/TDE –
Rotated Zakharov TDE DESFLS DEahcSPX DEPSR DE
F. Neri, V. Tirronen
Table 4 Results of the Q-test for DE with additional components in 50 dimensions
Ackley 4.21e0 8.75e−1 2.64e0 1.01e0 1.01e0

Alpine 1.07e1 7.97e−1 7.70e0 1.36e0 1.39e0
deJong 2.20e0 3.56e−1 7.45e−1 5.21e−1 5.41e−1
Dropwave ∞ ∞ 1.94e2 ∞ 1.82e1
Griewangk 2.22e0 3.58e−1 7.39e−1 5.20e−1 5.37e−1
Michalewicz 6.90e1 4.50e1 8.57e1 3.34e2 7.48e1
Pathological 1.66e−1 2.49e−1 1.36e−1 2.63e−1 2.28e−1
Rastrigin ∞ 1.16e1 ∞ 6.24e1 8.68e0
Rosenbrock valley 3.36e0 8.69e−1 2.19e0 7.82e−1 8.03e−1
Schwefel 5.34e1 ∞ 3.07e1 ∞ 6.63e2
Sum of Powers 7.59e−1 1.31e−1 1.19e−1 1.64e−1 1.76e−1
Tirronen ∞ ∞ ∞ 1.29e2 ∞
Zakharov 2.08e−2 1.96e−2 1.96e−2 1.98e−2 2.20e−2
Rotated Ackley 6.95e0 1.22e0 4.65e0 1.18e0 1.18e0
Rotated Alpine ∞ 1.25e0 ∞ 2.16e0 2.15e0
Rotated Griewangk 2.18e0 3.59e−1 7.37e−1 5.18e−1 5.38e−1
Rotated Michalewicz ∞ ∞ ∞ 9.07e2 ∞
Rotated Pathological 3.01e−1 3.85e−1 3.41e−1 4.51e−1 4.95e−1
Rotated Rastrigin ∞ ∞ ∞ ∞ 2.90e2
Rotated Rosenbrockvalley 2.55e0 3.38e−1 8.98e−1 4.91e−1 4.98e−1
Rotated Schwefel ∞ ∞ ∞ 9.47e2 ∞
Rotated Sum of Powers 4.86e−1 5.00e−2 9.42e−2 5.44e−2 5.82e−2
Rotated Tirronen 8.46e2 ∞ 4.55e2 7.20e1 ∞
Rotated Zakharov 1.06e−2 1.08e−2 1.00e−2 1.04e−2 1.08e−2
In order to carry out a numerical comparison of the convergence speed performance,

for each test problem, the average final fitness value returned by the best performing algo-
rithm G has been considered. Subsequently, the average fitness value at the beginning of
the optimization process J has also been computed by calculating the average fitness of
the initial populations. It must be remarked that in order to perform a fair comparison, the
various algorithms have been run with the same initial populations. The threshold value
THR = J − 0.95(J − G) has then been calculated. The value THR represents 95% of the
decay in the fitness value of the algorithm with the best performance. If an algorithm suc-
ceeds during a certain run to reach the value THR, the run is said to be successful. For each
test problem, the average amount of fitness evaluations ne ¯ required, for each algorithm, to
reach THR has been computed. Subsequently, the Q-test (Q stands for Quality) described in
Feoktistov (2006) has been applied. For each test problem and each algorithm, the Q measure
is computed as:
¯
ne
Q= (27)
R
123
(a)
(b)
(c)
(d)
Fig. 10 Performance trends of DE with additional components in 50 dimensions
123
Table 5 Average final fitness values ± standard deviations for the modified DE structure in 50 dimensions
DE jDE OBDE DEGL SADE
Alpine 6.53e−2 ± 7.43e−1 5.69e−14 ± 2.30e−12 2.67e−1 ± 4.63e0 9.29e−5 ± 2.45e−3 6.95e−9 ± 3.26e−7
deJong 7.58e−9 ± 2.62e−8 2.84e−20 ± 9.78e−19 1.31e−17 ± 8.75e−17 4.23e−35 ± 1.84e−33 2.17e−26 ± 2.50e−25
Griewangk 4.66e−6 ± 2.24e−5 3.89e−2 ± 7.46e−1 5.47e−13 ± 1.52e−11 1.23e−2 ± 1.70e−1 1.29e−2 ± 1.59e−1
Rastrigin 6.91e1 ± 5.06e1 2.12e1 ± 4.33e1 8.43e1 ± 6.64e1 9.03e1 ± 3.63e2 1.11e0 ± 7.66e0
Rosenbrock valley 7.33e−5 ± 1.95e−4 5.18e−5 ± 1.42e−3 6.27e−10 ± 1.79e−8 8.07e−13 ± 3.94e−11 4.81e−9 ± 8.13e−8
Rotated Ackley 4.39e−5 ± 6.88e−5 1.73e−11 ± 5.13e−10 4.84e−10 ± 1.73e−9 1.47e−14 ± 1.42e−14 4.17e−14 ± 1.03e−13
Rotated Alpine 6.42e1 ± 4.21e1 4.97e−1 ± 2.97e0 4.66e0 ± 4.84e1 7.61e−2 ± 9.85e−1 2.88e−1 ± 2.00e0
Rotated Griewangk 9.08e−2 ± 2.50e−1 3.45e−3 ± 3.77e−2 2.19e−3 ± 9.36e−2 4.24e−3 ± 4.92e−2 4.19e−3 ± 4.90e−2
Rotated Rosenbrockvalley 2.08e1 ± 1.70e−4 2.08e1 ± 4.79e−7 2.08e1 ± 1.79e−8 2.08e1 ± 1.09e−10 2.08e1 ± 2.66e−8
Rotated Sum of Powers 3.73e−4 ± 1.36e−3 6.95e−10 ± 4.68e−9 4.61e−5 ± 2.84e−4 6.69e−7 ± 3.45e−6 4.58e−9 ± 3.23e−8
Rotated Tirronen −1.02e0 ± 4.61e−1 −1.34e0 ± 2.04e0 −1.00e0 ± 4.29e−1 −9.90e−1 ± 5.98e−1 −7.92e−1 ± 5.84e−1
123
89
Table 6 Results of the unequal variance t-test for the modified DE structures in 50 dimensions
1st 2nd 3rd 4th 5th
Ackley DEGL SADE jDE OBDE DE

Alpine jDE SADE DEGL DE OBDE
deJong DEGL SADE jDE OBDE DE
Dropwave OBDE DEGL DE jDE SADE
Griewangk OBDE DE DEGL/SADE – jDE
Michalewicz jDE SADE DE OBDE DEGL
Pathological jDE OBDE DEGL SADE DE
Rastrigin SADE jDE DE OBDE DEGL
Rosenbrock valley DEGL OBDE SADE jDE DE
Schwefel SADE jDE DE OBDE DEGL
Sum of Powers DEGL SADE DE OBDE jDE
Tirronen jDE OBDE DE DEGL SADE
Zakharov jDE SADE DEGL DE OBDE
Rotated Ackley DEGL SADE jDE OBDE DE
Rotated Alpine DEGL SADE jDE OBDE DE
Rotated Griewangk OBDE jDE SADE/DEGL – DE
Rotated Michalewicz jDE DEGL SADE DE OBDE
Rotated Pathological jDE OBDE DEGL DE SADE
Rotated Rastrigin jDE SADE DEGL OBDE DE
Rotated Rosenbrockvalley DEGL SADE OBDE jDE DE
Rotated Schwefel jDE DEGL SADE DE OBDE
Rotated Sum of Powers jDE SADE DEGL OBDE DE
Rotated Tirronen jDE DE OBDE DEGL SADE
Rotated Zakharov jDE SADE DEGL DE OBDE
where the robustness R is the percentage of successful runs. It is clear that, for each test
problem, the smallest value equals the best performance in terms of convergence speed. The
value “∞” means that R = 0, i.e. the algorithm never reached the T H R.
Table 4 shows the Q values in 100 dimensions. The best results are highlighted in bold
face.
Results in Tables 2, 3 and 4 show that the standard DE is outperformed on a regular
basis by the other algorithms and thus confirm the effectiveness of the additional compo-
nents considered in this paper. In 50 dimensions, the components integrated within TDE
and DEahcSPX clearly improve upon the standard DE but they have a mediocre perfor-
mance compared with the other two algorithms under analysis. It can be seen from Ta-
ble 3 that TDE and DEahcSPX obtain the best performance in a limited amount of cases
(three for TDE and four for DEahcSPX). On the contrary, the scale factor local search
in DESFLS and the population size reduction in DEPSR appear to be the most effi-
cient components. It can be observed that DESFLS obtains the best performance in ten
cases out of the twenty-four considered while DEPSR does so in eight cases. Employ-
ment of the population size reduction seems especially promising for highly multimodal
and complex/rotated fitness landscapes. Regarding the convergence speed performance,
the Q-measures shown in Table 4 show that TDE, although in several cases is not
123
Table 7 Results of the Q-test for the modified DE structures in 50 dimensions
Ackley 4.21e0 1.12e0 3.14e0 7.21e − 1 1.08e0

Alpine 1.07e1 1.45e0 1.14e1 1.30e0 1.09e0
deJong 2.20e0 5.83e−1 1.14e0 3.48e−1 3.97e−1
Dropwave ∞ ∞ 2.05e1 1.00e2 ∞
Griewangk 2.24e0 6.06e−1 1.16e0 3.62e−1 4.22e−1
Michalewicz ∞ 6.85e0 ∞ ∞ 9.65e2
Pathological 1.21e−1 1.17e−1 8.68e−2 1.14e−1 1.60e0
Rastrigin ∞ 2.76e0 ∞ 1.04e2 9.11e0
Rosenbrock valley 3.40e0 8.80e−1 2.06e0 6.17e−1 1.20e0
Schwefel 1.39e2 1.06e1 ∞ ∞ 1.24e1
Sum of Powers 7.81e−1 1.77e−1 2.11e−1 1.10e−1 7.72e−2
Tirronen ∞ 4.62e1 ∞ ∞ ∞
Zakharov 1.88e−2 1.82e−2 2.64e−2 1.94e−2 1.40e−1
Rotated Ackley 6.95e0 1.39e0 2.62e0 8.32e−1 1.15e0
Rotated Alpine ∞ 2.66e0 1.52e1 3.71e0 1.37e0
Rotated Griewangk 2.18e0 5.94e−1 1.13e0 3.57e−1 4.20e−1
Rotated Michalewicz ∞ 2.28e2 ∞ ∞ ∞
Rotated Pathological 1.92e−1 2.04e−1 1.68e−1 3.07e−1 4.87e0
Rotated Rastrigin ∞ 2.95e1 ∞ ∞ 1.95e2
Rotated Rosenbrockvalley 2.55e0 5.41e−1 1.22e0 3.27e−1 4.73e−1
Rotated Schwefel ∞ 8.11e1 ∞ ∞ ∞
Rotated Tirronen ∞ 4.89e2 ∞ ∞ ∞
competitive with other algorithms (there are seven ∞ symbols), often has a very good
convergence speed performance. In addition, DEPSR has best performance in terms of
robustness (the least amount of ∞), DESFLS also has a very good performance in terms of
robustness.
Figure 10 shows some examples of performance trends in 50 dimensions.
6.2 Numerical Results: Modified DE Structures in 50 Dimensions
Numerical results related to modified DE structures in 50 dimensions are shown in Tables 5,

6, and 7, respectively.
Tables 5, 6, and 7 show that all these modified schemes significantly outperform the
standard DE. More specifically, jDE appears, in 50 dimensions, to be the most promising
modified DE structure. As shown in Table 6, jDE obtains the best performance for twelve test
problems out of the twenty-four considered. The second best performing algorithm seems
to be DEGL which obtains the best performance in seven cases. The OBDE and SADE
algorithms obtain the best performance in three and two cases respectively. On the other
hand, it should be remarked that the performance of the SADE is in most cases quite respect-
able. This algorithm seems fairly robust, as the fact that is ranked 2nd very often proves.
123
(a)
(b)
(c)
Fig. 11 Performance trends of modified DE structures in 50 dimensions
Nevertheless, among the algorithms analyzed jDE proves to have an excellent performance
in terms of robustness (only one ∞ in Table 7) and is thus the most efficient DE modification
in 50 dimensions.
6.3 Numerical Results: Additional Components in 100 Dimensions
Numerical results related to DE with additional components in 100 dimensions are shown in
Tables 8, 9, and 10, respectively.
123
Table 8 Average final fitness values ± standard deviations for DE with additional components in 100 dimensions
Ackley 5.24e-1 ± 3.39e-1 2.56e-2 ± 1.04e-1 1.00e-1 ± 8.02e-2 3.20e-3 ± 1.37e−2 8.03e−7 ± 1.85e−6
Alpine 8.01e1 ± 2.90e1 7.84e−2 ± 6.42e−1 3.90e1 ± 2.83e1 2.78e−2 ± 2.12e−1 3.16e−2 ± 1.21e−1
deJong 1.22e1 ± 8.68e0 1.98e−1 ± 1.78e0 9.82e−1 ± 1.09e0 3.93e−3 ± 4.06e−2 1.42e−10 ± 1.01e−9
Griewangk 4.27e1 ± 3.31e1 1.69e0 ± 8.20e0 4.31e0 ± 3.12e0 3.60e−1 ± 1.68e0 2.91e−8 ± 1.39e−7
Rosenbrock valley 3.40e1 ± 1.60e1 1.33e0 ± 3.46e0 1.48e1 ± 8.19e0 4.34e−1 ± 2.09e0 1.38e−2 ± 2.03e−1
Rotated Ackley 1.94e0 ± 8.45e−1 3.18e−2 ± 1.25e−1 4.41e−1 ± 4.06e−1 4.80e−3 ± 2.10e−2 1.15e−6 ± 2.53e−6
Rotated Alpine 1.94e2 ± 5.49e1 2.11e0 ± 1.15e1 7.27e1 ± 2.27e1 6.91e−1 ± 4.16e0 1.66e−1 ± 3.02e−1
Rotated Griewangk 4.26e1 ± 3.30e1 1.71e0 ± 5.61e0 4.30e0 ± 3.43e0 4.08e−1 ± 2.54e0 8.96e−4 ± 8.88e−3
Rotated Rosenbrockvalley 7.01e1 ± 1.27e1 5.30e1 ± 4.55e0 5.56e1 ± 2.56e0 5.27e1 ± 1.97e−1 5.27e1 ± 1.13e0
Rotated Sum of powers 2.09e0 ± 1.05e1 2.49e−5 ± 1.05e−4 8.24e−4 ± 4.43e−3 2.12e−6 ± 9.93e−6 3.60e−5 ± 1.36e−4
Rotated Tirronen −6.91e−1 ± 2.39e−1 −6.79e−1 ± 3.41e−1 −6.97e−1 ± 3.19e−1 −6.79e−1 ± 2.94e−1 −6.74e−1 ± 3.03e−1
123
93
Table 9 Results of the unequal variance t-test for DE with additional components in 100 dimensions
1st 2nd 3rd 4th 5th
Ackley DESFLS DEPSR TDE DEahcSPX DE

Alpine DEPSR DESFLS TDE DEahcSPX DE
deJong DESFLS DEPSR TDE DEahcSPX DE
Dropwave DESFLS TDE DEPSR DEahcSPX DE
Griewangk DESFLS DEPSR TDE DEahcSPX DE
Michalewicz DEPSR TDE DESFLS DEahcSPX DE
Pathological DEPSR DE/DEahcSPX – DESFLS/TDE –
Rastrigin DEPSR DESFLS TDE DEahcSPX DE
Rosenbrock valley DESFLS DEPSR TDE DEahcSPX DE
Schwefel DEPSR DE DEahcSPX DESFLS TDE
Sum of Powers DESFLS DEPSR TDE DEahcSPX DE
Tirronen DEPSR DE/DEahcSPX – TDE/DESFLS –
Zakharov DEahcSPX DEPSR TDE DESFLS DE
Rotated Ackley DESFLS DEPSR TDE DEahcSPX DE
Rotated Alpine DESFLS DEPSR TDE DEahcSPX DE
Rotated Griewangk DESFLS DEPSR TDE DEahcSPX DE
Rotated Michalewicz DEPSR TDE DESFLS DE DEahcSPX
Rotated Pathological DEPSR DE DEahcSPX TDE/DESFLS –
Rotated Rastrigin TDE DEPSR DESFLS DEahcSPX DE
Rotated Rosenbrockvalley DESFLS/DEPSR – TDE DEahcSPX DE
Rotated Schwefel DEahcSPX DE DEPSR TDE/DESFLS –
Rotated Sum of Powers DEPSR TDE DESFLS DEahcSPX DE
Rotated Tirronen DEahcSPX DE TDE/DEPSR – DESFLS
Rotated Zakharov DEPSR DEahcSPX TDE DESFLS DE
In 100 dimensions, the standard DE clearly suffers from the curse of dimensionality and
displays a poor performance with respect to the other more sophisticated algorithmic struc-
tures. Although in isolated cases the simplex crossover offers good performance, the scale
factor local search and population size reduction appear to be the most promising algorithmic
components, as well as for the high dimensional case, see Table 9. This fact was somehow
expected since DESFLS employs a dimension invariant local search which proved in Neri
and Tirronen (2009) to be promising for large scale problems. Regarding DEPSR, popula-
tion size reduction allows a progressive narrowing of the search space and thus a progressive
exploitation of the search directions detected during early stages of the evolution. This com-
ponent has already been proposed in Brest et al. (2008) as a useful mechanism for handling
large scale problems.
Results confirm that in high dimensions TDE has a good convergence speed performance
in some cases. However, TDE tends to converge to solutions characterized by an inferior
performance compared to other algorithms considered in this study. Finally, as shown in
Table 10, although DESFLS is promising in many cases, DEPSR seems to be the most robust
algorithm among those considered in this benchmark.
123
Table 10 Results of the Q-test for DE with additional components in 100 dimensions
Ackley ∞ 2.43e0 1.58e1 3.49e0 3.49e0

Alpine ∞ 1.95e0 ∞ 4.76e0 4.67e0
deJong 1.40e1 8.60e−1 2.49e0 1.82e0 1.81e0
Dropwave ∞ ∞ ∞ ∞ 7.66e2
Griewangk 1.39e1 8.44e−1 2.67e0 1.80e0 1.81e0
Michalewicz ∞ ∞ ∞ 3.71e1 ∞
Pathological 1.60e0 4.81e0 1.61e0 3.47e0 3.61e0
Rastrigin ∞ ∞ ∞ 3.47e2 8.52e1
Rosenbrock valley ∞ 2.90e0 1.24e1 2.70e0 2.74e0
Schwefel ∞ ∞ ∞ 4.81e1 ∞
Sum of Powers 4.45e0 3.91e−1 1.30e−1 7.24e−1 7.25e−1
Tirronen ∞ ∞ ∞ 1.60e2 ∞
Zakharov 8.48e−2 7.00e−2 8.32e−2 7.96e−2 7.61e−2
Rotated Ackley ∞ 2.74e0 ∞ 3.87e0 3.94e0
Rotated Alpine ∞ 2.96e0 ∞ 5.78e0 6.72e0
Rotated Griewangk 1.39e1 8.41e−1 2.73e0 1.79e0 1.81e0
Rotated Michalewicz ∞ 6.08e2 ∞ 3.67e2 7.78e2
Rotated Pathological 4.67e0 1.07e1 5.56e0 7.27e0 8.61e0
Rotated Rastrigin ∞ ∞ ∞ 3.16e2 ∞
Rotated Rosenbrockvalley 1.56e1 7.73e−1 2.66e0 1.59e0 1.61e0
Rotated Schwefel ∞ ∞ 3.07e2 8.81e2 ∞
Rotated Tirronen 8.43e2 3.07e2 1.27e2 5.43e2 9.83e2
6.4 Numerical Results: Modified DE Structures in 100 Dimensions
Numerical results related to modified DE structures in 100 dimensions are shown in Tables 11,
12, and 13, respectively.
Numerical results in 100 dimensions show that DEGL and jDE most often obtain the best
performance, see Table 12. The DEGL appears to be very promising for high dimensional
problems. On the other hand, when the DEGL obtains the best performance, it is sometimes
only marginally better than SADE and jDE. The SADE algorithm in some cases is very
competitive. The jDE algorithm obtains the best performance in many cases (especially for
complex landscapes like Rotated Michalewicz) and most importantly, as shown in Table 13,
is extremely promising in terms of robustness with respect to the other algorithms belonging
to the benchmark. Although DEGL can, in some cases, produce a very good performance,
results show that jDE is very competitive for almost all the test problems considered. In this
sense, for the comparative analysis carried out in this paper and the chosen parameter setting,
we can consider jDE a very promising and reliable modified structure among those recently
presented in literature.
123
(a)
(b)
(c)
Fig. 12 Performance trends of DE with additional components in 100 dimensions
7 Conclusion
This paper presents a survey on DE and analyzes the modifications recently proposed
in literature in order to enhance the DE performance. Eight modified DE schemes
have been analyzed in depth. The selected algorithms are, according to our judgement,
among the most efficient and representative of the state of the art of advanced DE
schemes.
Enhanced DEs have been classified into two macro-groups. To the first group belong those
algorithms which employ, within the standard DE structure, additional components which
are supposed to assist the external framework during the optimization process. The second
group is composed of those algorithms which are based on a modified DE structure.
123
Table 11 Average final fitness values ± standard deviations for the modified DE structure in 100 dimensions
Alpine 8.01e1 ± 2.90e1 3.66e−2 ± 3.58e−1 5.39e1 ± 2.93e1 6.37e−4 ± 3.12e−2 4.39e−4 ± 6.06e−3
deJong 1.22e1 ± 8.68e0 1.71e−6 ± 3.51e−5 3.25e−3 ± 1.22e−2 8.69e−11 ± 2.73e−9 6.43e−8 ± 2.60e−7
Griewangk 4.27e1 ± 3.31e1 1.95e−2 ± 3.63e−1 6.77e−1 ± 1.41e0 8.00e−3 ± 1.14e−1 1.07e−2 ± 1.23e−1
Rosenbrock valley 3.40e1 ± 1.60e1 2.14e−2 ± 1.14e−1 4.06e−1 ± 1.24e0 9.77e−4 ± 1.54e−2 5.70e−2 ± 1.43e−1
Tirronen −1.44e0 ± 2.87e−1 −1.60e0 ± 4.85e−1 −1.70e0 ± 3.37e−1 −1.23e0 ± 4.58e−1 −5.62e−1 ± 5.14e−1
Rotated Ackley 1.94e0 ± 8.45e−1 5.90e−5 ± 6.36e−4 5.30e−3 ± 9.86e−3 4.00e−7 ± 7.55e−6 2.02e−5 ± 3.70e−5
Rotated Alpine 1.94e2 ± 5.49e1 3.64e0 ± 1.33e1 1.62e1 ± 8.12e1 3.78e−1 ± 2.71e0 5.92e−1 ± 3.84e0
Rotated Griewangk 4.26e1 ± 3.30e1 6.02e−2 ± 6.67e−1 7.20e−1 ± 7.41e−1 4.03e−3 ± 4.68e−2 1.35e−2 ± 5.24e−2
Rotated Rosenbrockvalley 7.01e1 ± 1.27e1 5.27e1 ± 2.79e−2 5.30e1 ± 1.11e0 5.27e1 ± 5.52e−5 5.27e1 ± 2.18e−3
Rotated Sum of Powers 2.09e0 ± 1.05e1 6.44e−10 ± 6.01e−9 8.88e−5 ± 5.87e−4 7.83e−6 ± 8.79e−5 3.15e−10 ± 3.85e−9
Rotated Tirronen −6.91e−1 ± 2.39e−1 −7.15e−1 ± 3.72e−1 −6.94e−1 ± 3.95e−1 −6.36e−1 ± 2.81e−1 −4.26e−1 ± 4.13e−1
123
97
Table 12 Results of the unequal variance t-test for the modified DE structures in 100 dimensions
1st 2nd 3rd 4th 5th
Ackley DEGL SADE jDE OBDE DE

Alpine SADE/DEGL – jDE OBDE DE
deJong DEGL SADE jDE OBDE DE
Dropwave DEGL jDE/SADE – OBDE DE
Griewangk DEGL SADE jDE OBDE DE
Michalewicz jDE SADE OBDE DE DEGL
Pathological OBDE jDE DEGL SADE DE
Rastrigin jDE SADE DEGL OBDE DE
Rosenbrock valley DEGL jDE SADE OBDE DE
Schwefel jDE SADE DE OBDE DEGL
Sum of Powers OBDE SADE DEGL jDE DE
Tirronen OBDE jDE DE DEGL SADE
Zakharov jDE/SADE – DEGL DE OBDE
Rotated Ackley DEGL SADE jDE OBDE DE
Rotated Alpine DEGL SADE jDE OBDE DE
Rotated Griewangk DEGL SADE jDE OBDE DE
Rotated Michalewicz jDE DEGL DE SADE OBDE
Rotated Pathological OBDE jDE DE DEGL SADE
Rotated Rastrigin jDE SADE DEGL OBDE DE
Rotated Rosenbrockvalley DEGL SADE jDE OBDE DE
Rotated Schwefel jDE DE SADE/OBDE – DEGL
Rotated Sum of Powers SADE jDE DEGL OBDE DE
Rotated Tirronen jDE OBDE/DE – DEGL SADE
Rotated Zakharov jDE SADE DEGL OBDE/DE –
The reason for attempting to enhance the DE is due to the nature of the optimizer. The DE
can be very slow and fail in the detection of solutions with a high performance. On the other
hand, as shown in this paper, what would seem minor modifications can greatly enhance the
DE performance.
It is important to remark that all the DE schemes with additional components described
in this paper make a limited use of the additional components in order to achieve a great
improvement in performance. The TDE algorithm alternates the trigonometric mutation to
the standard mutation with a low probability. The DEahcSPX and DESFLS algorithms acti-
vate the local search only to that individual with the best performance while all others undergo
a normal DE scheme. The DESFLS algorithm further limits the local search activation by
imposing a low probability to the occurrence of this event. The population size reduction
is a slightly different additional component since it does not explicitly offer an alternative
search logic. On the other hand, as explained above, this component implicitly modifies the
set of possible moves by increasing the exploitation of the DE. Thus, DEPSR employs the
same variation rules of a standard DE but in fact varies the set of potential moves. In order
to observe the effectiveness of the population size reduction, it is necessary to allow sev-
eral generations. Moreover, size reduction is an irreversible change in the evolution. An a
123
Table 13 Results of the Q-test for the modified DE structures in 100 dimensions
Ackley ∞ 3.39e0 1.48e1 2.04e0 2.93e0

Alpine ∞ 4.18e0 ∞ 3.34e0 2.48e0
deJong 1.40e1 1.81e0 4.10e0 9.95e−1 9.42e−1
Dropwave ∞ ∞ ∞ 1.97e2 ∞
Griewangk 1.41e1 1.79e0 4.19e0 9.92e−1 9.40e−1
Michalewicz ∞ 8.46e1 ∞ ∞ ∞
Pathological 1.74e0 1.34e0 7.30e−1 1.21e0 8.63e0
Rastrigin ∞ 6.78e0 ∞ ∞ ∞
Rosenbrock valley ∞ 2.65e0 7.65e0 1.63e0 3.72e0
Schwefel ∞ 1.28e1 ∞ ∞ ∞
Sum of Powers 4.34e0 6.27e−1 5.49e−1 4.02e−1 1.52e−1
Tirronen ∞ 2.10e2 3.14e1 ∞ ∞
Zakharov 8.64e−2 7.96e−2 9.84e−2 6.12e−2 9.71e-1
Rotated Ackley ∞ 3.99e0 9.14e0 2.33e0 3.01e0
Rotated Alpine ∞ 7.45e0 4.12e1 8.68e0 3.13e0
Rotated Griewangk 1.41e1 1.87e0 4.17e0 9.69e−1 9.21e−1
Rotated Michalewicz ∞ 1.71e2 ∞ ∞ ∞
Rotated Pathological 3.42e0 2.97e0 1.27e0 9.87e0 ∞
Rotated Rastrigin ∞ 2.34e2 ∞ ∞ 6.86e2
Rotated Rosenbrockvalley 1.56e1 1.59e0 4.33e0 8.73e−1 1.10e0
Rotated Schwefel ∞ 8.20e2 ∞ ∞ ∞
Rotated Tirronen ∞ 5.96e2 6.02e2 ∞ ∞
posteriori re-introduction of individuals with mediocre performance would very likely cause
a stagnation condition. In summary, the population size reduction is a component which
has a weak impact on the DE search logic (but not on the performance) and thus requires a
prolonged application to be effective.
The algorithms which modify the DE structure also employ relatively mild variations to
the original DE. The OBDE algorithm generates, with a certain probability, extra candidate
solutions. The DEGL algorithm imposes an extra vector component during the offspring gen-
eration. The SADE algorithm diversifies the search moves by employing multiple mutation
strategies. Finally, jDE simply introduces a randomization in the choice of control parame-
ters. There is a low probability of a control parameter update occurring. Although changes in
jDE appear rather marginal on the DE structure, their impact on the algorithmic performance
is definitely major, as results show. An important fact that should be highlighted is that all the
proposed enhanced DE schemes include new stochastic elements within the DE. A certain
degree of randomization within a DE structure seems to have a highly relevant influence on
the algorithmic success.
The experimental analysis carried out allows us to conclude that the population size reduc-
tion and the scale factor local search are effective algorithmic components which can greatly
123
(a)
(b)
(c)
(d)
Fig. 13 Performance trends of modified DE structures in 100 dimensions
123
improve the DE performance. Nevertheless, the simplex crossover local search appears to
be efficient for specific problems. The OBDE algorithm also appears to be efficient in spe-
cific cases. The SADE algorithm proved to have a rather good performance but is outper-
formed, on a regular basis, either by DEGL or by jDE. Finally, it is worthwhile to consider
that DEGL has, for some problems, an excellent capability at detecting high quality solu-
tions while jDE is superior in terms of robustness and versatility over a diverse benchmark
set.
Acknowledgments This work is supported by Academy of Finland, Akatemiatutkija 00853, Algorithmic

Design Issues in Memetic Computing.
References
Abbass HA (2002) The self-adaptive pareto differential evolution algorithm. In: Proceedings of the IEEE
congress on evolutionary computation, vol 1, pp 831–836
Abbass HA, Sarker RA (2002) The pareto differential evolution algorithm. Int J Artif Intell Tools 11(4):531–
552
Abbass HA, Sarker R, Newton C (2001) Pde: A pareto-frontier differential evolution approach for multiob-
jective optimization problems. In: Proceedings of the IEEE congress on evolutionary computation, vol 2,
pp 971–978
Ali MM, Fatti LP (2006) A differential free point generation scheme in the differential evolution algorithm.
J Glob Optim 35(4):551–572
Ali MM, Törn A (2004) Population set based global optimization algorithms: some modifications and numer-
ical studies. Computers and operations research, vol 31. Elsevier, Amsterdam, 1703–1725
Angira R, Santosha A (2007) Optimization of dynamic systems: a trigonometric differential evolution
approach. Comput Chem Eng 31(9):1055–1063
Angira R, Santosh A (2008) A modified trigonometric differential evolution algorithm for optimization of
dynamic systems. In: Proceedings of the IEEE congress on evolutionary computation, pp 1463–1468
Babu B, Jehan M (2003) Differential evolution for multi-objective optimization. In: Proceedings of the IEEE
congress on evolutionary computation, vol 4, pp 2696–2703
Baker JE (1987) Reducing bias and inefficiency in the selection algorithm. In: Proceedings of the international
conference on genetic algorithms. Lawrence Erlbaum, Mahwah, pp 14–21
Brest J, Maučec MS (2008) Population size reduction for the differential evolution algorithm. Appl Intell
29(3):228–247
Brest J, Žumer V, Maucec M (2006) Self-adaptive differential evolution algorithm in constrained real-param-
eter optimization. In: Proceedings of the IEEE congress on evolutionary computation, pp 215–222
Brest J, Zamuda BBA, Žumer V (2008) An analysis of the control parameters’adaptation in DE. In:
Chakraborty UK (ed) Advances in differential evolution, vol 143 of studies in computational intelli-
gence. Springer, Berlin, pp 89–110
Brest J, Bošković B, Greiner S, Žumer V, Maučec MS (2007) Performance comparison of self-adaptive and
adaptive differential evolution algorithms. Soft Comput 11(7):617–629
Brest J, Greiner S, Bošković B, Mernik M, Žumer V (2006) Self-adapting control parameters in differential
evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–
657
Brest J, Zamuda A, Bošković B, Maucec MS, Žumer V (2008) High-dimensional real-parameter optimization
using self-adaptive differential evolution algorithm with population size reduction. In: Proceedings of
the IEEE world congress on computational intelligence, pp 2032–2039
Caponio A, Neri F (2009) Differential evolution with noise analysis. In: Applications of evolutionary com-
puting, vol 5484 of lecture notes in computer science. Springer, Berlin, pp 715–724
Caponio A, Neri F, Tirronen V (2009) Super-fit control adaptation in memetic differential evolution frame-
works. Soft Comput Fusion Found Methodol Appl 13(8):811–831
Caponio A, Cascella GL, Neri F, Salvatore N, Sumner M (2007) A fast adaptive memetic algorithm for on-line
and off-line control design of pmsm drives. IEEE Trans Syst Man Cybern B (special issue on Memetic
Algorithms) 37(1):28–41
Chakraborty UK (ed) (2008) Advances in differential evolution, vol 143 of studies in computational intelli-
gence. Springer, Berlin
123
Chakraborty UK, Das S, Konar A (2006) Differential evolution with local neighborhood. In: Proceedings of
the IEEE congress on evolutionary computation, pp 2042–2049
Chang TT, Chang HC (1998) Application of differential evolution to passive shunt harmonic filter planning.
In: Proceedings of the 8th international conference on harmonics and quality of power, vol 1, pp 149–153
Chang T-T, Chang H-C (2000) An efficient approach for reducing harmonic voltage distortion in distribution
systems with active power line conditioners. IEEE Trans Power Deliv 15(3):990–995
Chen W, Shi JY, Teng Hf (2008) An improved differential evolution with local search for constrained layout
optimization of satellite module. In: Advanced intelligent computing theories and applications. With
aspects of artificial intelligence, vol 5227 of lecture notes in computer science, Springer, Berlin, pp 742–
749
Chiou J-P, Wang F-S (1998) A hybrid method of differential evolution with application to optimal control prob-
lems of a bioprocess system. In: The 1998 IEEE international conference on evolutionary computation
proceedings, pp 627–632
Chiou J-P, Wang F-S (1999) Hybrid method of evolutionary algorithms for static and dynamic optimization
problems with application to a fed-batch fermentation process. Computers and Chemical Engineering,
vol 23. Elsevier, pp 1277–1291
Chiou J-P, Chang C-F, Su C-T (2004) Ant direction hybrid differential evolution for solving large capacitor
placement problems. IEEE Trans Power Syst 19:1794–1800
Das S, Konar A (2005) An improved differential evolution scheme for noisy optimization problems. In: Pattern
recognition and machine intelligence, vol 3776 of lecture notes in computer science. Springer, Berlin,
pp 417–421
Das S, Konar A, Chakraborty U (2005) Improved differential evolution algorithms for handling noisy optimi-
zation problems. In: Proceedings of the IEEE congress on evolutionary computation, vol 2, pp 1691–1698
Das S, Konar A, Chakraborty UK (2005) Two improved differential evolution schemes for faster global
search. In: Proceedings of the 2005 conference on genetic and evolutionary computation. ACM, New
York, pp 991–998
Das S, Konar A, Chakraborty UK (2007) Annealed differential evolution. In: Proceedings of the IEEE congress
on evolutionary computation, pp 1926–1933
Das S, Abraham A, Chakraborty UK, Konar A (2009) Differential evolution with a neighborhood-based muta-
tion operator. IEEE Trans Evol Comput 13(3):526–553
Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II.
IEEE Trans Evol Comput 6(2):182–197
Eiben AE, Smith JE (2003) Introduction to evolutionary computation. Springer, Berlin
Fan H-Y, Lampinen J (2002) A trigonometric mutation approach to differential evolution. In: Giannakoglou
KC, Tsahalis DT, Papailiou JPKD, Fogarty T (eds) Evolutionary methods for design, optimization and
control. CIMNE, Barcelona, pp 65–70
Fan H-Y, Lampinen J (2003a) A directed mutation operation for the differential evolution algorithm. Int J Ind
Eng 10(1):6–15
Fan H-Y, Lampinen J (2003b) A trigonometric mutation operation to differential evolution. J Glob Optim
27(1):105–129
Feoktistov V (2006) Differential evolution: in search of solutions. In: Optimization and its applications,
vol 5. Springer, New York, USA
Gao Y, Wang Y-J (2007) A memetic differential evolutionary algorithm for high dimensional functions’ opti-
mization. In: Proceesings of the 3rd international conference on natural computation, pp 188–192
Gämperle R, Müller SD, Koumoutsakos P (2002) A parameter study for differential evolution. In: Proceedings
of the conference in neural networks and applications (NNA), fuzzy sets and fuzzy systems (FSFS) and
evolutionary computation (EC), WSEAS, pp 293–298
Hart WE, Krasnogor N, Smith JE (2004) Memetic evolutionary algorithms. In: Hart WE, Krasnogor N, Smith
JE (eds) Recent advances in memetic algorithms. Springer, Berlin, pp 3–27
Hendtlass T (2001) A combined swarm differential evolution algorithm for optimization problems. In: Lecture
notes in computer science, vol 2070. Springer, Berlin, pp 11–18
He X, Han L (2007) A novel binary differential evolution algorithm based on artificial immune system. In:
Proceedings of the IEEE congress on evolutionary computation, pp 2267–2272
Hu S, Huang H, Czarkowski D (2005) Hybrid trigonometric differential evolution for optimizing harmonic
distribution. In: Proceedings of the IEEE international symposium on circuits and systems, vol 2, pp 1306–
1309
Hu Z-B, Su Q-H, Xiong S-W, Hu F-G (2008) Self-adaptive hybrid differential evolution with simulated
annealing algorithm for numerical optimization. In: Proceedings of the IEEE congress on evolutionary
computation, pp 1189–1194
123
Joshi R, Sanderson AC (1999) Minimal representation multisensor fusion using differential evolution. IEEE
Trans Syst Man Cybern A 29(1):63–76
Kaelo P, Ali MM (2007) Differential evolution algorithms using hybrid mutation. Comput Optim Appl
37:231–246
Karaboga N, Cetinkaya B (2004) Performance comparison of genetic and differential evolution algorithms
for digital FIR filter design. In: Advances in information systems, vol 3261 of lecture notes in computer
science. Springer, Berlin, pp 482–488
Karaboga N, Cetinkaya B (2006) Design of digital FIR filters using differential evolution algorithm. Circuits
Syst Signal Process 25:649–660
Kiefer J (1953) Sequential minimax search for a maximum. Proc Am Math Soc 4:502–506
Koh A (2009) An adaptive differential evolution algorithm applied to highway network capacity optimization.
vol 52 of Advances in Soft Computing. Springer, Berlin, 211–220
Krink T, Filipič B, Fogel GB (2004) Noisy optimization problems—a particular challenge for differential
evolution? In: Proceedings of the IEEE congress on evolutionary computation, pp 332–339
Lampinen J (1999) Mechanical engineering design optimization by differential evolution. In: Corne D, Dorigo
M, Glover F (eds) New ideas in optimization. McGraw-Hill, New York, pp 293–298
Lampinen J, Zelinka I (1999) An introduction to differential evolution. In: Corne D, Dorigo M, Glover F
(eds) New ideas in optimization. McGraw-Hill, New York, pp 127–146
Lampinen J, Zelinka I (2000) On stagnation of the differential evolution algorithm In: Oŝmera P (ed)
Proceedings of 6th international mendel conference on soft computing, pp 76–83
Leskinen J, Neri F, Neittaanmäki P (2009) Memetic variation local search vs life-time learning in electrical
impedance tomography. In: Applications of evolutionary computing, lecture notes in computer science.
Springer, Berlin, pp 615–624
Li H, Zhang Q (2006) A multiobjective differential evolution based on decomposition for multiobjective opti-
mization with variable linkages. In: Parallel problem solving from nature-PPSN IX, vol 4193 of lecture
notes in computer science. Springer, Berlin, pp 583–592
Lin Y-C, Wang F-S, Hwang K-S (1999) A hybrid method of evolutionary algorithms for mixed-integer non-
linear optimization problems. In: Proceedings of the IEEE congress on evolutionary computation, vol 3,
pp 2159–2166
Lin Y-C, Hwang K-S, Wang F-S (2001) Co-evolutionary hybrid differential evolution for mixed-integer opti-
mization problems. Eng Optim 33(6):663–682
Liu J, Lampinen J (2002b) A fuzzy adaptive differential evolution algorithm. In: Proceedings of the 17th IEEE
region 10th international conference on computer, communications, control and power engineering, vol I,
pp 606–611
Liu J, Lampinen J (2002c) Adaptive parameter control of differential evolution. In: Proceedings of the 8th
international Mendel conference on soft computing, pp 19–26
Liu J, Lampinen J (2002a) On setting the control parameter of the differential evolution algorithm. In:
Proceedings of the 8th international Mendel conference on soft computing, pp 11–18
Liu J, Lampinen J (2005) A fuzzy adaptive differential evolution algorithm. In: Soft Comput Fusion Found
Methodol Appl, vol 9. Springer, Berlin, pp 448–462
Liu B, Zhang X, Ma H (2008) Hybrid differential evolution for noisy optimization. In: Proceedings of the
IEEE congress on evolutionary computation, pp 587–592
Lozano M, Herrera F, Krasnogor N, Molina D (2004) Real-coded memetic algorithms with crossover hill-
climbing. Evol Comput Special Issue Memet Algorithms 12(3):273–302
Madavan NK (2002) Multiobjective optimization using a pareto differential evolution approach. In: Proceed-
ings of the IEEE congress on evolutionary computation, vol 2, pp 1145–1150
Mallipeddi R, Suganthan PN (2008) Empirical study on the effect of population size on differential evolution
algorithm. In: Proceedings of the IEEE congress on evolutionary computation, pp 3663–3670
Masters T, Land W (1997) A new training algorithm for the general regression neural network. In: Proceedings
of the IEEE international conference on systems, man, and cybernetics, computational cybernetics and
simulation, vol 3, pp 1990–1994
Mezura-Montes E, Reyes-Sierra M, Coello Coello CA (2008) Multi-objective optimization using differential
evolution: A survey of the state-of-the-art. In: Chakraborty UK (ed) Advances in differential evolution
, vol 143 of studies in computational intelligence. Springer, Berlin, pp 173–196
Nearchou AC, Omirou SL (2006) Differential evolution for sequencing and scheduling optimization. J Heu-
ristics 12(6):395–411
Neri F, Toivanen J, Cascella GL, Ong YS (2007) An adaptive multimeme algorithm for designing HIV mul-
tidrug therapies. IEEE/ACM Trans Comput Biol Bioinform 4(2):264–278
Neri F, Tirronen V, Kärkkäinen T, Rossi T (2007a) Fitness diversity based adaptation in multimeme algorithms:
A comparative study. In: Proceedings of the IEEE congress on evolutionary computation, pp 2374–2381
123
Neri F, Toivanen J, Mäkinen RAE (2007b) An adaptive evolutionary algorithm with intelligent mutation local
searchers for designing multidrug therapies for HIV. Appl Intell 27:219–235
Neri F, Tirronen V (2008) On memetic differential evolution frameworks: a study of advantages and lim-
itations in hybridization. In: Proceedings of the IEEE world congress on computational intelligence,
pp 2135–2142
Neri F, Tirronen V (2009) Scale factor local search in differential evolution. Memet Comput J 1(2):153–171
Neri F, Tirronen V, Kärkkäinen T (2009) Enhancing differential evolution frameworks by scale factor local
search—part II. In: Proceedings of the IEEE congress on evolutionary computation, pp 118–125
NIST/SEMATECH (2003) e-handbook of statistical methods, http://www.itl.nist.gov/div898/handbook/
Noman N, Iba H (2005) Enhancing differential evolution performance with local search for high dimensional
function optimization. In: Proceedings of the 2005 conference on genetic and evolutionary computation
ACM, New York, pp 967–974
Noman N, Iba H (2008) Accelerating differential evolution using an adaptive local search. IEEE Trans Evol
Comput 12(1):107–125
Ohkura K, Matsumura Y, Ueda K (2001) Robust evolution strategies. Appl Intell 15(3):153–169
Olorunda O, Engelbrecht A (2007) Differential evolution in high-dimensional search spaces. In: Proceedings
of the IEEE congress on evolutionary computation, pp 1934–1941
Omran MG, Salman A, Engelbrecht AP (2005) Self-adaptive differential evolution. In: Computational intel-
ligence and security, vol 3801 of lecture notes in computer science. Springer, Berlin, pp 192–199
Ong YS, Keane AJ (2004) Meta-lamarkian learning in memetic algorithms. IEEE Trans Evol Comput 8(2):
99–110
Plagianakos VP, Sotiropoulos DG, Vrahatis MN (1998) Integer weight training by differential evolution algo-
rithms. In: Mastorakis NE (ed) Recent advances in circuits and systems. World Scientific, Singapore,
pp 327–331
Plagianakos VP, Tasoulis DK, Vrahatis MN (2008) A review of major application areas of differential evolu-
tion. In: Chakraborty UK (ed) Advances in differential evolution, vol 143 of studies in computational
intelligence. Springer, Berlin, pp 197–238
Price K, Storn R (1997) Differential evolution: a simple evolution strategy for fast optimization. Dr Dobbs J
Softw Tools 22(4):18–24
Price KV, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization.
Springer, Berlin
Qin AK, Suganthan PN (2005) Self-adaptive differential evolution algorithm for numerical optimization.
In: Proceedings of the IEEE congress on evolutionary computation, vol 2, pp 1785–1791
Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global
numerical optimization. IEEE Trans Evol Comput 13(2):398–417
Qing A (2008) A study on base vector for differential evolution. In: Proceedings of the IEEE congress on
evolutionary computation, pp 550–556
Rahnamayan S, Wang GG (2008) Solving large scale optimization problems by opposition-based differential
evolution (ode). WSEAS Trans Comput 7(10):1792–1804
Rahnamayan S, Tizhoosh HR, Salama MM (2006a) Opposition-based differential evolution algorithms,
pp 2010–2017
Rahnamayan S, Tizhoosh H, Salama MMA (2006b) Opposition-based differential evolution for optimization
of noisy problems. In: Proceedings of the IEEE congress on evolutionary computation, pp 1865–1872
Rahnamayan S, Tizhoosh HR, Salama MMA (2007) Quasi-oppositional differential evolution. In: Proceedings
of the IEEE congress on evolutionary computation, pp 2229–2236
Rahnamayan S, Tizhoosh HR, Salama MM (2008) Opposition-based differential evolution. IEEE Trans Evol
Comput 12(1):64–79
Rahnamayan S, Tizhoosh H, Salama M (2008) Opposition-based differential evolution. In: Chakraborty UK
(ed) Advances in differential evolution, vol 143 of studies in computational intelligence. Springer, Berlin,
pp 155–171
Rechemberg I (1973) Evolutionstrategie: optimierung technisher systeme nach prinzipien des biologishen
evolution. Fromman-Hozlboog Verlag, Stuttgart, Germany
Robič T, Filipič B (2005) DEMO: Differential evolution for multiobjective optimization. In: Coello Coello
CA, Aguirre AH, Zitzler E (eds) Evolutionary Multi-Criterion Optimization , vol. 3410 of lecture notes
in computer science. Springer, Berlin, pp 520–533
Rogalsky T, Derksen RW (2000) Hybridization of differential evolution for aerodynamic design. In: Proceed-
ings of the 8th annual conference of the computational fluid dynamics society of Canada, pp 729–736
Russell SJ, Norvig P (2003) Artificial intelligence: a modern approach (2nd ed). Prentice Hall, Englewood
Cliffs, NJ, USA, 111–114
123
Ruxton GD (2006) The unequal variance t-test is an underused alternative to student’s t-test and the Mann–
Whitneyu test. Behav Ecol 17(4):688–690
Rönkkönen J, Lampinen J (2003) On using normally distributed mutation step length for the differential evo-
lution algorithm. In: Matousek R, Osmera P (eds) Proceedings of 9th international mendel conference
on soft computing, pp 11–18
Rönkkönen J, Kukkonen S, Price KV (2005) Real-parameter optimization with differential evolution. In:
Proceedings of IEEE international conference on evolutionary computation, vol 1, pp 506–513
Salman A, Engelbrecht AP, Omran MG (2007) Empirical analysis of self-adaptive differential evolution. Eur
J Oper Res 183(2):785–804
Sing TN, Teo J, Hijazi MHA (2007) Empirical testing on 3-parents differential evolution (3PDE) for uncon-
strained function optimization. In: Proceedings of the IEEE congress on evolutionary computation,
pp 2259–2266
Soliman OS, Bui LT (2008) A self-adaptive strategy for controlling parameters in differential evolution.
In: Proceedings of the IEEE congress on evolutionary computation, pp 2837–2842
Soliman OS, Bui LT, Abbass HA (2007) The effect of a stochastic step length on the performance of the
differential evolution algorithm In: Proceedings of the IEEE congress on evolutionary computation,
pp 2850–2857
Storn R (1996a) Differential evolution design of an IIR-filter. In: Proceedings of IEEE international conference
on evolutionary computation, pp 268–273
Storn R (1996b) On the usage of differential evolution for function optimization. In: Proceedings of the IEEE
biennial conference of the North American fuzzy information processing society, pp 519–523
Storn R (1999) System design by constraint adaptation and differential evolution. IEEE Trans Evol Comput
3(1):22–34
Storn R (2005) Designing nonstandard filters with differential evolution. IEEE Signal Process Mag 22(1):
103–106
Storn R, Price K (1995) Differential evolution—a simple and efficient adaptive scheme for global optimization
over continuous spaces. Technical Report TR-95-012, ICSI
Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over
continuous spaces. J Glob Optim 11:341–359
Su C-T, Lee C-S (2003) Network reconfiguration of distribution systems using improved mixed-integer hybrid
differential evolution. IEEE Trans Power Deliv 18:1022–1027
Teng NS, Teo J, Hijazi MHA (2009) Self-adaptive population sizing for a tune-free differential evolution. Soft
Comput Fusion Found Methodol Appl 13(7):709–724
Teo J (2005) Differential evolution with self-adaptive populations. In: Knowledge-based intelligent information
and engineering systems, vol 3681 of lecture notes in computer science. Springer, Berlin, pp 1284–1290
Teo J (2006) Exploring dynamic self-adaptive populations in differential evolution. Soft Comput Fusion Found
Methodol Appl 10(8):673–686
Thomas P, Vernon D (1997) Image registration by differential evolution. In: Proceedings of the 1st Irish
machine vision and image processing conference, pp 221–225
Tirronen V, Neri F (2009) Differential evolution with fitness diversity self-adaptation. In: Chiong R (ed)
Nature-inspired algorithms for optimisation, vol 193 of studies in computational intelligence. Springer,
Berlin, pp 199–234
Tirronen V, Neri F, Rossi T (2009) Enhancing differential evolution frameworks by scale factor local search—
part I. In: Proceedings of the IEEE congress on evolutionary computation, pp 94–101
Tirronen V, Neri F, Kärkkäinen T, Majava K, Rossi T (2007) A memetic differential evolution in filter design
for defect detection in paper production. In: Applications of evolutionary computing, vol 4448. Springer,
Berlin, pp 320–329
Tirronen V, Neri F, Kärkkäinen T, Majava K, Rossi T (2008) An enhanced memetic differential evolution in
filter design for defect detection in paper production. Evol Comput 16:529–555
Tirronen V, Neri F, Majava K, Kärkkäinen T (2008) The natura non facit saltus principle in memetic computing.
In: IEEE congress on evolutionary computation, pp 3881–3888
Tsutsui S, Yamamura M, Higuchi T (1999) Multi-parent recombination with simplex crossover in real coded
genetic algorithms. In: Proceedings of the Genetic Evol Comput Conf (GECCO), pp 657–664
Tvrdík J, Krivý I (1999) Simple evolutionary heuristics for global optimization. Comput Stat Data Anal
30(3):345–352
Wang F-S, Jang H-J (2000) Parameter estimation of a bioreaction model by hybrid differential evolution.
In: Proceedings of the IEEE congress on evolutionary computation, vol 1, pp 410–417
Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):
67–82
123
Xu X, Li Y, Fang S, Wu Y, Wang F (2008) A novel differential evolution scheme combined with particle
swarm intelligence. In: Proceedings of the IEEE congress on evolutionary computation, pp 1057–1062
Yang Z, He J, Yao X (2008) Making a difference to differential evolution. In: Michalewicz Z, Siarry P (eds)
Advances in metaheuristics for hard optimization, pp 397–414
Yang Z, Tang K, Yao X (2007) Differential evolution for high-dimensional function optimization. In: Pro-
ceedings of the IEEE congress on evolutionary computation, pp 3523–3530
Yang Z, Tang K, Yao X (2008a) Large scale evolutionary optimization using cooperative coevolution. Inf Sci
178(15):2985–2999
Yang Z, Tang K, Yao X (2008b) Self-adaptive differential evolution with neighborhood search. In: Proceedings
of the world congress on computational intelligence, pp 1110–1116
Zaharie D (2002) Critical values for control parameters of differential evolution algorithm. In: Matuŝek R,
Oŝmera P (eds) Proceedings of 8th international mendel conference on soft computing, pp 62–67
Zaharie D (2003) Control of population diversity and adaptation in differential evolution algorithms. In:
Matousek D, Osmera P (eds) Proceedings of MENDEL international conference on soft computing, pp
41–46
Zaharie D, Petcu D (2004) Adaptive pareto differential evolution and its parallelization. In: Parallel processing
and applied mathematics, vol 3019 of lecture notes in computer science, pp 261–268
Zamuda A, Brest J, Bošković B, Žumer V (2007) Differential evolution for multiobjective optimization with
self adaptation. In: Proceedings of the IEEE congress on evolutionary computation, pp 3617–3624
Zamuda A, Brest J, Bošković B, Žumer V (2008) Large scale global optimization using differential evolu-
tion with self-adaptation and cooperative co-evolution. In: Proceedings of the IEEE world congress on
computational intelligence, pp 3719–3726
Zhang J, Sanderson A (2007) DE-AEC: a differential evolution algorithm based on adaptive evolution control.
In: Proceedings of IEEE international conference on evolutionary computation, pp 3824–3830
Zhang X, Duan H, Jin J (2008) DEACO: Hybrid ant colony optimization with differential evolution.
In: Proceedings of the IEEE congress on evolutionary computation, pp 921–927
Zhenyu G, Bo C, Min Y, Binggang C (2006) Self-adaptive chaos differential evolution. In: Advances in natural
computation, vol 4221 of lecture notes in computer science. Springer, Berlin, pp 972–975
Zielinski K, Laur R (2008) Stopping criteria for differential evolution in constrained single-objective optimi-
zation. In: Chakraborty UK (ed) Advances in differential evolution, vol 143 of studies in computational
intelligence. Springer, Berlin, pp 111–138
Zielinski K, Weitkemper P, Laur R, Kammeyer K-D (2006) Parameter study for differential evolution using a
power allocation problem including interference cancellation. In: Proceedings of the IEEE congress on
evolutionary computation, pp 1857–1864
Zielinski K, Wang X, Laur R (2008) Comparison of adaptive approaches for differential evolution. In: Parallel
problem solving from nature—PPSN X, vol 5199 of lecture notes in computer science. Springer, Berlin,
pp 641–650
123

Neri-Tirronen2010 Article RecentAdvancesInDifferentialEv

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Neri-Tirronen2010 Article RecentAdvancesInDifferentialEv

Uploaded by

Copyright:

Available Formats

Artif Intell Rev (2010) 33:61–106

Recent advances in differential evolution:

Ferrante Neri · Ville Tirronen

Published online: 27 October 2009

F. Neri (B) · V. Tirronen

Keywords Differential Evolution · Survey · Comparative Analysis · Self-Adaptation ·

2 Standard differential evolution

generated by mutation as:

– DE/current-to-rand/1 xoff = xi + K (xt − xi ) + F (xr − xs )

3 Differential Evolution: a survey

Fig. 1 DE/rand/1/bin pseudo-code

It is clear that successful functioning of a DE depends on the parameter setting of the

F = 0.5 (1 + rand (0, 1)) (5)

On the other hand, the employment of randomization in DE is not always considered to be

4 Additional components in differential evolution

4.1 Differential Evolution with Trigonometric Mutation

where f (xt ) ≤ f (xr ) and f (xt ) ≤ f (xs ).

4.2 Differential Evolution with Adaptive Crossover Local Search

Fig. 2 SPX pseudo-code

4.3 Differential Evolution with Population Size Reduction

4.4 Differential Evolution with Scale Factor Local Search

Fig. 3 Fitness function, f (F),

4.4.1 Scale Factor Golden Section Search

Fig. 4 SFGSS pseudo-code

4.4.2 Scale Factor Hill-Climb

Fig. 5 SFHC pseudo-code

4.5 Discussion about additional components

5 Modified structures of Differential Evolution

5.1 Self-Adapting Parameter Setting in Differential Evolution (jDE)

Fig. 6 jDE pseudo-code

5.2 Opposition Based Differential Evolution

The Opposition Based Differential Evolution (OBDE), proposed in Rahnamayan et al.

5.3 Differential Evolution with Global and Local Neighborhoods

where wi is a weight factor to be set between 0 and 1.

Fig. 7 OBDE pseudo-code

wi = wi + F (wbest − wi ) + F (wr − ws ) (24)

5.4 Self-Adaptive Differential Evolution

Fig. 8 DEGL pseudo-code

5.5 Discussion about Modified Differential Evolution Schemes

Fig. 9 SADE pseudo-code

Table 1 Test problems

Test problem Function Decision space

In order to estimate the dependence of the algorithmic performance on the dimensionality,

6.1 Numerical Results: Additional Components in 50 Dimensions

DE TDE DEahcSPX DEPSR DESFLS

1st 2nd 3rd 4th 5th

Table 4 Results of the Q-test for DE with additional components in 50 dimensions

DE TDE DEahcSPX DEPSR DESFLS

Ackley 4.21e0 8.75e−1 2.64e0 1.01e0 1.01e0

In order to carry out a numerical comparison of the convergence speed performance,

DE jDE OBDE DEGL SADE

1st 2nd 3rd 4th 5th

Ackley DEGL SADE jDE OBDE DE

Table 7 Results of the Q-test for the modified DE structures in 50 dimensions

DE jDE OBDE DEGL SADE

Ackley 4.21e0 1.12e0 3.14e0 7.21e − 1 1.08e0

6.2 Numerical Results: Modified DE Structures in 50 Dimensions

Numerical results related to modified DE structures in 50 dimensions are shown in Tables 5,

6.3 Numerical Results: Additional Components in 100 Dimensions

DE TDE DEahcSPX DEPSR DESFLS

1st 2nd 3rd 4th 5th

Ackley DESFLS DEPSR TDE DEahcSPX DE

DE TDE DEahcSPX DEPSR DESFLS