om

s.c
r es
dp
Differential Evolution and Variants

or
.w
ar
m
ku
Rajesh Kumar, PhD, PDF (NUS, Singapore)
sh
SMIEEE (USA), FIET (UK) FIETE, FIE (I), SMIACSIT, LMISTE, MIAENG
je
Professor , Department of Electrical Engineering
rra

Malaviya National Institute of Technology, Jaipur, India,

Mobile: (91) 9549654481
/d

Email: rkumar.ee@mnit.ac,in, rkumar.ee@gmail.com

s:/

Web: http://drrajeshkumar.wordpress.com/
tp
ht
 What is Differential Evolution(DE)?

om
s.c
es
• DE is a population-based optimization

r
dp
algorithm.

or
• The algorithm was introduced by

.w
Storn and Price in 1996.

ar
m
• Developed to optimize real parameter,

ku
real valued functions.

sh
je
rra
/d
s:/
tp
ht

2
 Differential Evolution

om
s.c
es
• DE is a stochastic, population-based optimization algorithm for solving nonlinear

r
optimization problem.

dp
or
• The population is composed of Np individuals

.w
ar
• Every individual in population represents a possible solution

m
ku
• DE operates in three consecutive steps in every iteration

– Mutation
sh
je
– Crossover
rra

– Selection
/d
s:/
tp
ht

3
 ht
tp
s:/
/d
rra
je
Main stages of DE algorithm

sh
ku
m
ar
.w
or
dp
res
s.c
om
4
 ht
tp
s:/
/d
Initialize the population

rra
je
sh
ku
m
ar
.w
or
dp
res
s.c
om
5
Mutation

om
s.c
res
dp
or
.w
ar
• A parent vector from the current generation

m
is called target vector.

ku
• A mutant vector obtained through the
sh
differential mutation operation is known as
donor vector.
je
rra

• Finally an offspring formed by recombining

the donor with the target vector is called
/d

trial vector.
s:/
tp
ht

6
 ht
tp
s:/
/d
rra
je
sh
Different Mutation Strategies

ku
m
ar
.w
or
dp
res
s.c
om
7
Crossover

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra

• The donor vector exchanges

/d

its components with the

target vector X iG to form the
s:/

trial vector.
tp

• CR-Crossover Rate
ht

8
 Selection

om
s.c
res
dp
or
.w
• The fitness of target and trail

ar
vector is evaluated and the one

m
with lowest function value is
selected for next generation.

ku
sh
je
rra
/d
s:/
tp
ht

9
 Pseudo Code

ht
tp
s:/
/d
rra
je
sh
ku
m
ar
.w
or
dp
res
s.c
om
10
 DE Working

ht
tp
s:/
/d
rra
je
sh
ku
m
ar
.w
or
dp
res
s.c
om
11
Scope to Create Variants of DE

om
s.c
es
Deterministic

r
dp
Parameter Control

or
.w
ar
Three parameters

m
to control

ku
exploitation and
exploration i.e.
sh
je
F, Cr, Np
rra
/d

Self-adaptive Adaptive parameter

s:/

parameter control Control

tp
ht

12
 Deterministic Parameter Control

om
s.c
r es
dp
or
Parameters are altered using a deterministic rule regardless of the feedback from the

.w
evolutionary search.

ar
m
i.In first approach, F was created for each individual within the range (0.4, 1) and the

ku
interval (0.5, 0.7) was selected for Cr

ii.CoDE - Composite DE algorithm

sh
je
rra
/d
s:/
tp
ht

13
 CoDE

om
s.c
es
• A trial vector is selected from a set of groups.

r
• Main objective – To merge many trial vector strategies with different parameters at each iteration

dp
to construct new trial vectors.

or
• The selected strategies are:

.w
1. DE/rand/1/bin
2. DE/rand/2/bin

ar
3. DE/current-to- rand/1

m
ku
• The three pair choices for the control parameter settings:
sh
je
rra

1. (F = 1.0, Cr = 0.1)
2. (F = 1.0; Cr = 0.9)
/d

3. (F = 0.8; Cr = 0.2)
s:/
tp
ht

14
Illustration Of Combining Trial Vector Generation Strategies
With Control Parameter Settings

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

15
 Algo

ht
tp
s:/
/d
rra
je
sh
ku
m
ar
.w
or
dp
res
s.c
om
16
 Adaptive Parameter Control

om
s.c
es
Applies an evaluation from feedback of the F relay on an additional parameter (γ) that it is necessary to be

r
adjusted.

dp
or
i.Differential Evolution with Self-Adapting Populations (DESAP)

.w
ii.Fuzzy Adaptive Differential Evolution (FADE)

ar
iii.Self-Adaptive Differential Evolution (SaDE)

m
iv.Self-Adaptive NSDE (SaNSDE)

ku
v.Self-Adapting Parameter Setting in Differential Evolution (jDE)
vi.Adaptive DE Algorithm (ADE) sh
je
vii.Modified DE (MDE)
rra

viii.Modified DE with P-Best Crossover (MDE_pBX)

/d

ix.DE with Self-Adaptive Mutation and Crossover (DESAMC)

s:/

x.Adaptive Differential Evolution with Optional External Archive (JADE)

tp

xi.Differential Covariance Matrix Adaptation Evolutionary Algorithm (CMA-ES)

ht

17
 Differential Evolution With Self-adapting Populations (DESAP)

om
s.c
r es
dp
• DESAP dynamically adjusts the crossover and mutation parameters δ, η and the population size π.

or
.w
• DESAP performed better than DE in one of De Jong’s five exam problems, whereas the other

ar
solutions are almost identical.

m
ku
• The mutation factor F is retained as static, and η denotes the probability of implementing an
sh
extra mutation operation by using normally distributed.
je
rra
/d
s:/
tp
ht

18
Fuzzy Adaptive Differential Evolution (FADE)

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

19
 FADE

om
s.c
es
• The purpose of using fuzzy logic is to
make the DE’S control parameters

r
adaptive in the minimization process.

dp
• To minimize the error(e) between the

or
global optimum and the actual function

.w
value in case of knowing about global
optimum value.

ar
m
• Minimize the change in e of function
values between successive generation in

ku
case of knowing nothing about global
optimum function value.
sh
je
rra
/d
s:/
tp
ht

20
Self-adaptive differential evolution (SaDE)

om
s.c
es
• Self-adaptive differential evolution (SaDE) is simultaneously applied to a pair of mutation

r
dp
techniques “DE/rand/1” and “DE/current-to-best/2”

or
• In SADE, a mutation strategy is selected probabilistically.

.w
ar
• The 2 mutation startegies used are:

m
ku
• "DE/rand/1" : vi = xR1 + F(xR2 - xR3)
sh
• "DE/current-to-best/1" : vi = xR1 + F(xbest - xi ) + F(xR1 - xR2 )
je
• Probability of one is p1 and another is p2=(1-p1).
rra
/d
s:/
tp
ht

21
 SaDE

om
s.c
es
For population size = NP, randomly generate a vector of size NP
with uniform distribution in the range [0, 1] for each element.

r
dp
or
If jth element of vector <= p1 then DE/rand/1 will be applied to
the jth individual of population else DE/current-to-best/1 is

.w
applied.

ar
m
All trial vectors are evaluated. ns1, ns2 are successful trail

ku
vectors for both startegies respectively. ns1, ns2 are successful
trail vectors for both startegies respectively.
sh
je
rra
/d
s:/

Fi = NR(0.5, 0.3) and Cri = NR(Crm, 0.1)

tp

Cr and F are created based on normal distributions for each generation.

ht

22
 ht
tp
s:/
/d
rra
je
sh
Self-adaptive NSDE (SaNSDE)

ku
m
ar
.w
or
dp
res
s.c
om
23
Self-Adapting Parameter Setting in Differential Evolution (jDE)

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

24
 ht
tp
s:/
/d
rra
je
sh
Adaptive DE Algorithm (ADE)

ku
m
ar
.w
or
dp
res
s.c
om
25
 Modified DE (MDE)

om
s.c
• MDE uses only one array.

• Array is updated when a better solution is found.

es
• F – adjusted by Laplace distribution.

r
dp
• Improved the convergence speed and fewer evaluation procedures than basic DE.

or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

26
Modified DE with p-best crossover (MDE_pBX)

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

27
DE with self-adaptive mutation and crossover (DESAMC)

om
s.c
es
r
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

28
Adaptive differential evolution with optional external archive (JADE)

om
s.c
res
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

29
 Differential Evolution with Multiple Strategies

om
• Four mutation strategies are selected to form a pool.

s.c
• One crossover operator is used.

es
• For each individual mutation strategy is selected randomly from the pool.

r
dp
or
.w
ar
m
ku
sh
je
rra
/d
s:/
tp
ht

30
 DE-PSO

om
:

s.c
• Process of DE-PSO is as follows

r es
dp
or
.w
ar
m
ku
sh
je
rra

Else
PSO Activated (New particle is found using PSO)
/d

Again Selection is done as Basic DE

s:/
tp
ht

31
Other Hybrid Approaches

om
Methods hybridized Authors Optimization problems applied to

s.c
DE + Artificial Bee Colony Tran et al. Multi-objective optimization for optimal time-cost-quality

es
(ABC) Algorithm trade-off for planning of construction projects

r
dp
DE + Ant Colony Optimization Chang et al. Single-objective real parameter engineering optimization
(ACO)

or
.w
DE + Bacterial Foraging based Biswal et al. Single-objective continuous parameter clustering problem
Optimization (BFO)

ar
m
DE + Gravitational Search Chakraborti et al. Binary optimization problem involving highly

ku
Algorithm (GSA) discriminative feature subset selection

DE + Invasive Weed
sh
Basak et al. Single-objective, bound constrained function optimization
je
Optimization (IWO) problems
rra

DE + Firefly Algorithm(FFA) Abdullah et al. Single-objective nonlinear optimization problem involving

/d

estimation of biological model parameters

s:/
tp

DE + Fireworks Algorithm Zheng et al. Single-objective, bound constrained function optimization

(FWA) problems
ht

32
ht
tp
s:/
/d
rra
je
sh
ku
m
ar
.w
or
dp
res
s.c
om