This action might not be possible to undo. Are you sure you want to continue?

Welcome to Scribd! Start your free trial and access books, documents and more.Find out more

Evolutionary Algorithms

Progress report by

Peter Dueholm Justesen

Department of Computer Science

University of Aarhus

Denmark

January 13, 2009

Supervisors: Christian N. S. Pedersen and Rasmus K. Ursem

Abstract

This is a progress report describing my research during the last one and a half year,

performed during part A of my Ph.D. study. The research ﬁeld is multi-objective

optimization using evolutionary algorithms, and the reseach has taken place in a

collaboration with Aarhus Univerity, Grundfos and the Alexandra Institute.

My research so far has been focused on two main areas, i) multi-objective evo-

lutionary algorithms (MOEAs) with diﬀerent variation operators, and ii) decreasing

the cardinality of the resulting population of MOEAs. The outcome of the former

is a comparative analysis of MOEA versions using diﬀerent variation operators on a

suite of test problems. The latter area has given rise to both a new branch of multi-

objective optimization (MOO) called MODCO (Multi-Objective Distinct Candidates

Optimization) and a new MOEA which makes it possible for the user to directly set

the cardinality of the resulting set of solutions. To motivate and cover my previous

and future work, the progress report is divided into three main parts:

1. Introduction to the research area

2. The contributions made by this author

3. Future work

Peter Dueholm Justesen

Aarhus University

i

Table of contents

1 Introduction 1

1.1 Main concepts of multi-objective optimization . . . . . . . . . . . . . . 1

1.2 Multi-objective optimization using evolutionary algorithms . . . . . . . 3

1.2.1 The history of multi-objective evolutionary algorithms . . . . . 4

1.2.2 Goals of multi-objective evolutionary algorithms . . . . . . . . . 5

1.2.3 Basic operators of multi-objective evolutionary algorithms . . . 6

2 Contributions 8

2.1 Genetic algorithms versus Diﬀerential Evolution . . . . . . . . . . . . . 8

2.1.1 Content and ideas . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.3 Evaluation and discussion . . . . . . . . . . . . . . . . . . . . . 13

2.1.4 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2 Multiobjective Distinct Candidate Optimization . . . . . . . . . . . . . 14

2.2.1 Introduction, arguments and goals . . . . . . . . . . . . . . . . . 14

2.2.2 Evaluation and discussion . . . . . . . . . . . . . . . . . . . . . 17

2.2.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3 Cluster-Forming Diﬀerential Evolution . . . . . . . . . . . . . . . . . . 17

2.3.1 Content and ideas . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.3.3 Evaluation and discussion . . . . . . . . . . . . . . . . . . . . . 25

3 Future work 26

3.1 Testing MODCO algorithms on Grundfos problems . . . . . . . . . . . 26

3.1.1 Constrained problems . . . . . . . . . . . . . . . . . . . . . . . 27

3.1.2 Many-objective optimization . . . . . . . . . . . . . . . . . . . . 27

3.1.3 The eﬀect of changing MODCO parameters . . . . . . . . . . . 28

3.1.4 A more eﬃcient search? . . . . . . . . . . . . . . . . . . . . . . 28

3.2 Inventing performance indices . . . . . . . . . . . . . . . . . . . . . . . 29

3.3 Inventing alternate MODCO algorithms . . . . . . . . . . . . . . . . . 30

3.4 Relevant conferences and journals . . . . . . . . . . . . . . . . . . . . . 31

List of ﬁgures 32

List of algorithms 32

References 32

ii

1 Introduction

This section will introduce the main concepts of multi-objective optimization, as well

as motivate the use of evolutionary algorithms for this.

Multi-objective problems are problems with two or more, usually conﬂicting, objec-

tives. The main diﬀerence from single-objective optimization is that a multi-objective

problem does not have one single optimal solution, but instead has a set of optimal

solutions, where each represents a trade-oﬀ between objectives.

At Grundfos, a danish company manufacturing pumps, an example of a multi-

objective problem is that of designing centrifugal pumps. The designs of centrifugal

pumps are complex, and each design lead to a diﬀerent eﬃciency rate and produc-

tion price. Grundfos is naturally interested in making their pumps have maximum

eﬃciency, which may be measured as the throughput of water per second. However,

Grundfos do not want the pumps to cost too much to manufacture and sell, so at

the same time they want to minimize the production cost. Typically, increasing the

eﬃciency of a pump will also increase the production cost. This way optimal pump

designs will have trade-oﬀ between cost and eﬃciency, ranging from maximum eﬃ-

ciency at maximum cost to minimum eﬃciency at minimum cost. However, ﬁnding

the optimal designs is not easy, and the task calls for optimization procedures.

A way to avoid the complexities of multi-objective optimization is to convert the

multi-objective problem into a single-objective problem by assigning weights to the

diﬀerent objectives, calculating a single ﬁtness value. The major problem with this

weighted sum approach, is that it is subjective, as it ultimately leaves it to a deci-

sion maker (DM) to assign weights according to the subjective importance of the

corresponding objectives. The approach further assumes that the DM has a priori

knowledge of the importance of the diﬀerent objectives, which is often hard or impos-

sible to come by. The objective approach uses Pareto compliant ranking of solutions,

as explained in the following section. This approach favours solutions which are better

in a true, multi-objective sense. Only the latter approach has been investigated in my

work, as this makes no assumptions and does not rely on higher level knowledge.

1.1 Main concepts of multi-objective optimization

The primary concept of multi-objective optimization, is the multi-objective problem

having several functions to be optimized (maximized or minimized) by the solution

x, along with diﬀerent constraints to satisfy, as seen in Equation 1.

x is a vector of decision variables: x = (x

1

, x

2

, ..., x

n

)

T

, where each decision variable

x

i

∈ 1 is bounded by the lower bound x

L

i

and the upper bound x

U

i

. These bounds

constitute the decision variable space or simply the decision space T, and the M

objective functions f

m

(x) deﬁne a mapping from T to the objective space Z. The

surjective mapping is between the n-dimensional solution vectors x ∈ T and the m-

dimensional objective vectors f

m

(x) ∈ Z, such that each x ∈ T corresponds to one

point y ∈ Z, as illustrated in ﬁgure 1.

1

The problem may be constrained, so Equation 1 also shows J inequality and K

equality constraints. Solutions satisfying these constraints are feasible, and belong to

the feasible part of the decision space; o

D

⊂ T which the constraint functions map to

the feasible part of objective space o

Z

⊂ Z.

Minimize/Maximize f

m

(x), m = 1, 2, ..., M;

Subject to g

j

(x) ≥ 0, j = 1, 2, ..., J;

h

k

(x) = 0, k = 1, 2, ..., K;

x

L

i

≤ x

i

≤ x

U

i

, i = 1, 2, ..., N;

(1)

The transition from single-objective to multi-objective optimization problems intro-

duce a challenge in comparison of solutions, since performance is then a vector of ob-

jective values instead of a single scalar. The concept of Pareto dominance addresses

this issue, enabling comparison of solutions. We say that a solution x dominates

solution y, written x ≺ y, if and only if the following two conditions hold: 1) The

solution x is no worse than solution y in all objectives. 2) The solution x is better

than y on at least one objective. Formally, assuming minimization on all objectives:

f

m

(x) ≤ f

m

(y)∀m ∧ ∃i : f

i

(x) < f

i

(y) (2)

This deﬁnition entails that the dominating solution is, in a true multi-objective

sense, the better choice. The dominance relation allows for Pareto based ranking,

ranking solutions according to how dominating they are wrt. a domination based

comparison to other solutions. Other performance indicators should assign the best

value to the most dominating solutions; such indicators are called Pareto compliant,

in contrast to the weighted sum approach described above.

The binary dominance relation presented above is transitive, asymmetric and non-

reﬂexive, in short meaning that if x ≺ y and y ≺ z, then x ≺ z, if x ≺ y then y

⊀ x, and that x ⊀ x. However, several relations between solutions exist. A list of

the most commonly used relations between solutions, the corresponding notation and

formal interpretation is presented in Table 1, listed according to strictness imposed.

relation notation interpretation

strictly dominates x ≺≺ y f

m

(x) < f

m

(y)∀m

dominates x ≺ y f

m

(x) ≤ f

m

(y)∀m

∧∃i : f

i

(x) < f

i

(y)

weakly dominates x _ y f

m

(x) ≤ f

m

(y)∀m

incomparable x | y (x _ y) ∧ (y _ x)

indiﬀerent x ∼ y f

m

(x) = f

m

(y) ∀m

Table 1: Solution relations - assuming minimization on all objectives

2

From the deﬁnition of dominance, several other important deﬁnitions can be de-

rived. When optimizing, we are interested in locating the non-dominated set of solu-

tions. Among a set of solutions P, the non-dominated set of solutions P

are those

that are not dominated by any other member of the set P.

Correspondingly, we deﬁne the globally Pareto-optimal set as the non-dominated

set of the entire feasible search space o ∈ T. This is also referred to simply as the

Pareto-optimal set, and our goal using multi-objective optimizers is to approximate

this set. In objective space, the mapping from the Pareto-optimal set is denoted the

true Pareto-optimal front or simply the true Pareto front. See ﬁgure 1 for illustration.

Figure 1: Mapping from decision space to objective space - assuming maximization

1.2 Multi-objective optimization using evolutionary algorithms

One way to perform multi-objective optimization is by using an evolutionary algorithm

(EA). Evolutionary algorithms are optimizers inspired by Darwenian evolution, and

with this the concept of survival of the ﬁttest. In an EA, solutions to a given problem

is considered individuals of a population, where the ﬁtness of individuals are given by

how good they solve the problem at hand. In the population individuals may mate

to create oﬀspring, which makes parents and oﬀspring compete for inclusion in the

next generation. As only the most ﬁt will survive this ﬁght, the full population is

improving iteratively in each passing generation.

More formally, the strength of EAs comes from their use of a set of solutions,

not only improving on a single solution. This makes it possible to combine several

(good) solutions, when creating a new one. An EA is actually a stochastic meta-

heuristic, i.e. a general optimization method, basing itself on probabilistic operators.

Thus, contrary to deterministic algorithms, EAs may produce diﬀerent results from

diﬀerent runs.

3

The greatest diﬀerence between single-objective EAs and multi-objective EAs

(MOEAs), is that for single-objective optimization, it is simple to return the most

optimal solution in a population, as scalar based evaluation automatically implies a

total order on solutions. For MOEAs, the situation is very diﬀerent. Due to the higher

dimensionality of the objective space, all resulting individuals of the population may

be incomparable to each other, each representing an optimal trade-oﬀ between ob-

jectives. That is, the result of running a MOEA is typically a set of non-dominated

solutions. From this result set, it is up to the decision maker to ﬁnd out which

solution(s) to realise. The full process is illustrated in ﬁgure 2.

Minimize F2

...

Minimize Fn

Subject to constraints

Multi−objective optimization problem

Minimize F1

Chosen trade−off solution

Step 2: Decision making

Step 1: Optimization

Multiple candidate solutions

on true Pareto front

Multi−objective optimizer

IDEAL

Higher−level information

on all candidate solutions

Figure 2: Multi-objective optimization process

1.2.1 The history of multi-objective evolutionary algorithms

Here, we present a brief historic view of some of the algorithms that has been used in

the work of this author. Where single-objective EAs has been extensively researched

for many years now, the ﬁeld of MOEAs is relatively new. Here, we consider only

algorithms, which in some way incorporates the elitism concept, which ensures that

the number of non-dominating individuals in the population can only increase.

The ﬁrst, very popular elitist genetic algorithm for multi-objective optimization

was the Non-dominated Sorting Genetic Algorithm; NSGA-II, created by Deb et al

and published in 2000 [5]. This was the ﬁrst MOEA, which further incorporated a

diversity preserving mechanism, to ensure population diversity. Another elitist genetic

algorithm, also incorporating a density measure, was the Strength Pareto Evolutionary

Algorithm; SPEA2, created by Zitzler et al and published in 2001 [7]. Both of these

algorithms were shown to solve 2D problems very eﬃciently [5; 6; 7].

4

Later, an old idea made its way to the ﬁeld of MOEAs. Diﬀerential Evolution is

an alternative way of varying EA individuals based on vector diﬀerence, which was

published in 1997 by Price and Storn [8]. The approach was adopted in a number

of MOEAs, where we will consider only the most relevant to this research. Recently,

Robiˇc and Filipiˇc combined NSGA-II and SPEA-2 selection with the Diﬀerential Evo-

lution (DE) scheme for solution reproduction to create DEMO (DEMO: Diﬀerential

Evolution for Multi-objective Optimization). The DE-based MOEAs were named

DEMO

NSII

and DEMO

SP2

, and these have been shown to outperform both NSGA-II

and SPEA2 [9; 10].

As MOEAs have become increasingly popular over the last decade, it has been

noted that the cardinality of the resulting populations of MOEAs is often too high for

decision makers to make their ﬁnal choice of candidate solutions. One way of dealing

with this problem is by applying clustering, such that solutions within the same area

of the objective space are reported as a single solution. Very recently, Knowles and

Handl have suggested clustering by k-means (MOCK) [16], where solutions are being

assigned clusters after an optimization run. Alternatively, this author and fellow

researcher have suggested an approach for optimizing distinct candidates (MODCO)

[2] during the run. The main diﬀerence in these approaches besides when applied is

that in MODCO, the user directly sets the number of returned solutions, whereas the

number of clusters are automatically determined using MOCK.

In general, decreasing the result cardinality introduce some interesting options wrt.

decreasing the population size along with the number of generations performed. Fur-

thermore, this drastically decreases the amount of post-processing of results needed,

as will be discussed later in this report.

1.2.2 Goals of multi-objective evolutionary algorithms

As stated above, the goal of a multi-objective evolutionary algorithm, is to approxi-

mate the Pareto-optimal set of solutions. However, this goal is often subdivided into

the following three goals for the resulting population of a MOEA:

1. Closeness towards the true Pareto-front

2. An even distribution among solutions

3. A high spread of solutions

First, we want all of our solutions to be as optimal as possible by making them

get as close to the true Pareto front as possible, where closeness is measured as

Euclidian distance in objective space. As most problems solved by MOEAs are NP-

complete combinatorial problems, there is no way of “guessing” the decision vector

mapping to some good point in objective space. MOEAs select the most dominating

(or most non-dominated) individuals for survival to drive the population towards the

5

true Pareto front, since these must be closest to it. Optimally, all returned solutions

from a MOEA lie on the true Pareto front.

The second goal of MOEAs is to cover as much of the Pareto front as possible, and

this goal is very speciﬁc to multi-objective optimization. Having an even distribution

of solutions on the Pareto front ensures a diverse set of trade-oﬀs between objectives.

Having a set of solutions all equidistant to their nearest neighbour provides the DM

with an overview of the Pareto front, while also making the ﬁnal selection possible,

based on the range of trade-oﬀs between objectives represented by the population, as

illustrated in ﬁgure 2.

The third goal is very connected to the second. Having a high spread means to

have a high distance between the extreme solutions in objective space, and as before,

this is to ensure coverage of the Pareto front. Ensuring population diversity is most

often done by applying a density or crowding measure which penalise individuals, that

are close to each other in objective space. Such measure also ensures a high spread,

since it will force the full population to spread as far as possible.

From an application point of view, the ﬁrst goal is by far the most important,

since it directly determines how optimal the returned solutions are. The second goal

is important, but not crucial, since most often only a few solutions from the ﬁnal

population will be chosen for further investigation. Finally, the third goal is practically

irrelevant, since extreme solutions are rarely, if ever, implemented in reality.

To pursue the three goals, traditional MOEAs employ two mechanisms directly

meant to promote Pareto-front convergence and a good solution distribution. The

ﬁrst mechanism is elitism, which ensures that solutions closest to the true Pareto-

front will never be eliminated from the population under evolution, i.e. the number

of non-dominated solutions in the population can only increase. The second is a

measure of crowding or density among solutions, a secondary ﬁtness which is often

incorporated into the Pareto-rank to form a single, ﬁnal ﬁtness.

1.2.3 Basic operators of multi-objective evolutionary algorithms

Here, we brieﬂy introduce the MOEA operators, such that we have an overall idea

of their purpose before introducing more concrete mechanisms. The operators are

applied iteratively until some pre-determined termination criterion is met, usually

depending on the number of function evaluations performed, as this is where the

main computational eﬀort is spend. This is however depending on the population size

N, the dimensionality of the problem M, and the number of generations performed;

T. It is usual to create one new oﬀspring per parent, such that M N T function

evaluations are performed in T generations. The basic operators used by MOEAs are:

• Evaluation

• Selection

• Variation

6

Evaluation is typically based on the dominance relation, as described above. It

is intended to give an indication of the level of dominance for each individual by as-

signing a Pareto-rank, which bases itself on the number of other individuals in the

population that are dominated by the individual in question. This ranking must be

Pareto-compliant. Depending on which method used for Pareto ranking, it can be

more or less graded. Traditional MOEAs further incorporate a second ﬁtness crite-

rion when assigning the ﬁnal ﬁtness, in order to induce a total order on the quality

of individuals before selection. Evaluation is performed in objective space, and is

depending on the objective functions, which again are problem dependent.

Selection comes in two variants, and often base itself on the ﬁtness assigned dur-

ing evaluation. The ﬁrst selection to be applied, is called mating selection or sexual

selection, where it is decided which individuals of the current generation get to mate

and spawn oﬀspring. This selection is often random or may include all individuals,

in order to give all current generation individuals an equal chance to mate. Envi-

ronmental selection is applied after variation is performed, by applying the famous

rule of survival of the ﬁttest, such that only the best of the combined set of parents

and oﬀspring survive to the next generation. This typically truncates the expanded

population to its original size. Both kinds of selection are performed in objective space.

Variation is applied after mating selection to the individuals, who were selected

for mating. These individuals then get to create oﬀspring, which are a variation of

its parents. Here, two variation operators are typically applied; mutation and recom-

bination. Mutation is intended to cause only small changes in the oﬀspring, whereas

the recombination operator makes it possible to retain good parts of several parents

in the oﬀspring. In general, we say that mutation enhances exploitation, whereas

recombination enhances exploration of the search space. Common for all variation

operators, is that they are applied in decision space.

Ev a l ua t i on

Ma t i ng

Sel ect i on

Var i at i on

Env i r onme nt a l

Sel ect i on

Figure 3: Multi-Objective Evolutionary Algorithm main loop

7

2 Contributions

This section is devoted to the contributions made by this author and aﬃliates. To a

large part, my research so far has been focused on two main areas:

• Genetic algorithms versus Diﬀerential Evolution

• Multi-objective Distinct Candidate Optimization (MODCO)

We will go through these areas, ﬁrst introducing the relevant articles submitted

for conferences in the MOEA society. We then go through the contributions with the

following taxonomy:

1. Content and ideas

2. Results

3. Evaluation and discussion

4. Future work

The content and idea section is intended to present the content and general ideas

expressed in the contribution, while the results section will present results, both in the

form of data and in the form of implemented algorithms or operators. Evaluation and

discussion is concerned with evaluating and discussing the contribution wrt. related

research, and ﬁnally the section on future work intends to sketch future investigations.

2.1 Genetic algorithms versus Diﬀerential Evolution

The article “Introducing the Strength Pareto Diﬀerential Evolution 2 (SPDE2) Algo-

rithm - A Novel DE Based Approach for Multiobjective optimization” [1] was submit-

ted for the Parallel Problem Solving from Nature (PPSN) conference in may, 2008,

but was rejected. The article was an investigation of the diﬀerence in performance

between genetic algorithms and Diﬀerential Evolution, and introduces the idea of us-

ing SPEA2 ranking and truncation with Diﬀerential Evolution as the variation and

selection operator in a MOEA.

2.1.1 Content and ideas

EAs using variation operators that most often causes only small changes from parents

to oﬀspring are called genetic algorithms, for historical reasons. Another branch of

EAs are diﬀerential evolution algorithms. The diﬀerence in the two kinds of algorithms

lies in their way of applying mating selection and variation, along with an enhanced

elitism concept in DE. The main idea of the contribution was to compare two popular

multiobjective genetic algorithms (MOGAs); NSGA-II and SPEA2 with their DE-

enhanced counterparts. Here, we go through the main algorithmic diﬀerences, before

introducing the concrete algorithms implemented and tested; NSGA-II [5; 6], SPEA2

[7] and the DEMO versions [9; 10].

8

Genetic algorithms Using genetic algorithms, parents are stochastically selected

and copied to an oﬀspring vector, and are then subject to recombination and mutation

operators in order to form oﬀspring. After this, parents and oﬀspring are joined in one

population, subject to truncation by environmental selection based on ﬁtness. This

is illustrated in ﬁgure 4, where it may be noted that the population size always varies

from 2N to N during truncation.

Var i at i on

Mat i ng sel ect i on

Ev a l ua t i on

Env i r onme nt a l s e l e c t i on

Figure 4: Genetic algorithm

Diﬀerential Evolution Using Diﬀerential Evolution [8], we use vector diﬀerences

to create oﬀspring, using several individuals to create a new candidate oﬀspring to

compete against its parent. All individuals of the population are tried out as parents,

generating one oﬀspring each. The DE oﬀspring creation algorithm is depicted in

Algorithm 1.

Algorithm 1 Diﬀerential Evolution

Require: Parent P

i

, crossover factor CF, scaling factor F.

Ensure: Oﬀspring C.

1: Randomly select three individuals P

i

1

, P

i

2

, P

i

3

from population P, where i, i

1

, i

2

and i

3

are pairwise diﬀerent.

2: Calculate oﬀspring C as C = P

i

1

+ F (P

i

2

−P

i

3

).

3: Modify oﬀspring C by binary crossover with the parent P

i

using crossover factor

CF.

Another mayor diﬀerence using Diﬀerential Evolution, is that DE enhances elitism

by applying further rules after having created the candidate oﬀspring. These rules

makes the population size vary from N up to 2N before truncation, as can be seen in

ﬁgure 5.

9

Given parent x spawning oﬀspring y, the elitist rules are:

• If y ≺ x, x is replaced with y.

• If y | x, y is added to the population, as it is then possibly globally non-

dominated. This is revealed during the competition with the parent part during

environmental selection.

• If x ≺ y, y is discarded.

I nc ompa r a bl e of f s pr i ng Di f f er ent i al evol ut i on

Ev a l ua t i on

Env i r onme nt a l s e l e c t i on

Re pl a c e d by domi na nt of f s pr i ng

Figure 5: Diﬀerential Evolution

NSGA-II The elitism mechanism in NSGA-II [5; 6] is based on non-dominated

sorting of the current population P

t

. For each individual ∈ P

t

, non-dominated sorting

assigns a non-dominated rank equal to the non-dominated front label, which is used to

group the individuals into fronts. Here, the ﬁrst front

1

consists of the populations non-

dominated solutions, the next front consists of solutions dominated only by the ﬁrst

front and so on. Diﬀerent ways of performing non-dominated sorting are described in

[6].

The density measure in NSGA-II is called crowding-sort, and gives an indication

of the degree of any solutions distance to its nearest neighbours wrt. the M diﬀerent

objectives. The crowding measure assigned to individual i is the average side length of

the cuboid given by its nearest neighbours in M dimensions, normalized with respect

to the maximal and minimal function value of the respective objectives.

1

Where all individuals are assigned rank 1.

10

A full NSGA-II run is to perform non-dominated sorting on the combined pop-

ulation and oﬀspring population. From here, as many fronts as possible are accom-

modated in the next population. The last front to be inserted is sorted using the

crowding-sort procedure, and only the best individuals wrt. crowding distance is cho-

sen for inclusion. This truncation mechanism favors rank ﬁrst, and good crowding

distance next, just as the binary crowded tournament selection operator which NSGA-

II applies to the parent part of the population to ﬁll the oﬀspring vector. Finally all

individuals in the oﬀspring vector are subject to crossover and mutation operators.

SPEA2 SPEA2 [7] diﬀers from NSGA-II by maintaining an archive A

t

of size N of

the best solutions found so far, and from this archive, a population P

t

of size M is

continuously generated to compete against it in generation t. As in NSGA-II, SPEA2

uses the dominance concept to promote elitism, but here a Pareto-strength value is

assigned to each individual i in both population and archive according to how many

solutions in both archive and population i dominates:

S(i) = [¦j[j ∈ P

t

⊕A

t

∧ i ≺ j¦[, (3)

where [[ denote cardinality of a set, ⊕ is multiset union and i ≺ j means that i

dominates j. After all individuals i ∈ A

t

have been assigned a Pareto-strength, a

raw ﬁtness is assigned to individual i ∈ A

t

equal to the sum of the strength of its

dominators:

R(i) =

j∈P

t

⊕A

t

∧j≺i

S(j). (4)

The density estimation in SPEA2 is based on the k-th nearest neighbour method,

using Euclidean distance in objective space. In short, density for an individual i is

calculated as an inverse to its distance to its k-th nearest neighbour σ

k

i

, where k = 1

corresponds to the closest neighbour:

D(i) =

1

σ

k

i

+ 2

. (5)

Finally, each individual ∈ P

t

⊕ A

t

is assigned a ﬁtness value equal to the sum of its

raw ﬁtness and its density value; F(i) = R(i) +D(i) to provide a single value to judge

by.

After ﬁtness assignment, non-dominated solutions ∈ P

t

⊕ A

t

with F(i) < 1 are

ﬁrst archived, i.e. placed in A

t+1

. In case the archive is then too small, N − [A

t+1

[

dominated individuals ∈ P

t

⊕A

t

are archived, where individuals are chosen according

to their assigned ﬁtness value. Otherwise, if the archive is too large, it is truncated

by recursively removing the individual with the worst density from archive, i.e. the

individual who is closest to its k-th nearest (non-deleted) neighbour is repeatedly

chosen for deletion until the archive size is reached. After this, mating selection is

applied using binary tournament on the archive, in order to ﬁll the oﬀspring vector,

which is then subject to recombination and crossover operators.

11

DEMO versions The Diﬀerential Evolution versions of the above genetic algo-

rithms are called DEMO

NSII

and DEMO

SP2

[9; 10]. In the article [1], DEMO

SP2

is

denoted SPDE2 (Strength Pareto Diﬀerential Evolution 2), as we thought this was a

novel approach. However, the alternate DEMO version had already been investigated

under the name DEMO

SP2

, and so we will use this.

The DE versions of NSGA-II and SPEA2 are using the corresponding Pareto-

ranking, secondary ﬁtness assignment and truncation mechanism as their origins.

The diﬀerence lies in applying the elitist rules, and in using several individuals for

oﬀspring creation, as seen in Algorithm 1 and in ﬁgure 5. Using DE, the elitist rules

ensure the spread of good decision variables, while using several individuals increase

the chance of retaining good parts of the decision variables and decrease the risk of

premature stagnation by introducing more diversity than genetic variation operators.

2.1.2 Results

Here, we summarize the results of our contribution. Overall, the results from this

contribution have been:

1. Implementation of NSGA-II and SPEA2 with genetic operators.

2. Implementation of DEMO

NSII

and DEMO

SP2

.

3. Implementation of 5 ZDT (Zitzler, Deb, Thiele) test problems.

4. Implementation of 5 diﬀerent performance indices (PIs)

5. Data from runs using all algorithms on all test problems with all PIs.

Implemented algorithms All algorithms described in the contents section have

been implemented in a C++ framework supplied by Rasmus Ursem working at Grund-

fos R & T. The framework features classes for numerical individuals along with an

abstract class for multi-objective EAs, enabling implementation of diﬀerent concrete

versions. All algorithms have been implemented enabling several options regarding

variation operators, populations size, number of generations performed, etc.

As a part of implementing the genetic MOEAs, several genetic variation operators

were implemented. The simulated binary crossover (SBX) operator was implemented,

along with arithmetic crossover operators, and a Gaussian annealing mutation oper-

ator. For the DEMO versions, DE was implemented as in Algorithm 1 with the

enhanced elitism rules, along with a few other DE variants, which showed no im-

provement over the rand/1/bin scheme presented.

Test problems To test the diﬀerent algorithms, 5 2-dimensional ZDT problems

were implemented. These problems have diﬀerent true Pareto-front characteristica,

which attempts to make MOEAs prematurely converge in various ways. These prob-

lems are described further in [6].

12

Performance indices As the true Pareto-fronts of the ZDT problems are known, it

was possible to implement performance indices directly minded on giving an indication

of the MOEA performance wrt. the 3 goals of MOEAs described earlier. 2 PIs on

closeness, 2 PIs on distribution and 1 on spread were implemented.

Data To gather data, each algorithm was run 10 times on each test problem. From

the resulting populations and the known true Pareto-fronts, the PIs were calculated,

averaged, analyzed and discussed. It was clear, that the DEMO versions of NSGA-

II and SPEA2 performed better than their original versions on all indices. Further,

DEMO

SP2

was observed to perform slightly better than DEMO

NSII

, possibly due to

a more ﬁne-grained Pareto-ranking and a recursive density assignment procedure.

I further made a few investigations into the diﬀerences in distribution achieved by

the genetic and the DEMO algorithms, and found a discrepancy between DE and the

crowding/density measure of NSGA-II and SPEA2, as DE itself does not promote a

good distribution. Thus, DE - based algorithms seem to be more prone to generate

oﬀspring, which are removed due to crowding and thus does not contribute to the

steady qualitative improvement in the population.

2.1.3 Evaluation and discussion

As mentioned above, the article was submitted to the PPSN conference in may 2008,

but was rejected. Some of the points of criticism were:

1. Low news value

2. Test problems were outdated

The article was developed on background on the ﬁrst DEMO-article by Robiˇc

and Filipiˇc [9], but as the PPSN committee pointed out, a newer article had been

released meanwhile. In their second DEMO article [10], Robiˇc and Filipiˇc made an

investigation very similar to the one performed by this author. Especially they also

incorporated SPEA2 selection and truncation in DEMO. However, Robiˇc and Filipiˇc

tested on higher - dimensional problems, and with more general performance indices,

which made their article superior to my work and further decreased the news value

of my article. Unfortunately, the article by Robiˇc and Filipiˇc was not available when

I started working on my article, which is the reason of the overlap.

The problem with my chosen test suite for the article, was that MOEAs need

to be tested on problems with more than two objectives. It has been shown that

MOEAs working well on two - dimensional problems do not necessarily work well on

higher-dimensional problems. Especially there seems to be a blowup in complexity

when moving from two dimensions to three.

On the more positive side, my article and the article by Robiˇc and Filipiˇc [10]

agreed on the main point, that DE based MOEAs seem to outperform genetic algo-

rithms, especially with respect to closeness towards the true Pareto-front. Thus, even

though a part of my research was not up to date, it makes an interesting, valid point.

13

2.1.4 Future work

No future work is intended to follow up on this contribution. The implemented algo-

rithms all have been tested to work well on a range of problems, especially the DE

based ones. However, there seems to be little to gain in making further research in

this direction, as much is already thoroughly covered in [10]. Still, the algorithms have

been a basis for experimenting with new algorithms, partly based on the techniques

of NSGA-II, SPEA2 and the DEMO versions. Especially the pareto ranking methods

and the elite preserving truncation mechanism have been used in newly developed

algorithms, along with Diﬀerential Evolution.

2.2 Multiobjective Distinct Candidate Optimization

We decided early that we wanted to research in which extent it was possible to decrease

the cardinality of the resulting population of traditional MOEAs. After some discus-

sion, we named the research area Multiobjective Distinct Candidate Optimization

(MODCO), and submitted two papers on the area to the Evolutionary Multiobjective

Optimization (EMO) 2009 conference.

The ﬁrst paper “Multiobjective Candidates Optimization (MODCO) - A New

Branch of Multiobjective Optimization Research” [2] was mainly developed by co-

researcher Rasmus Kjær Ursem as an argumentation and quantiﬁcation of the MODCO

area, based on his experience in real world optimization at Grundfos. This paper was

rejected, mostly due to its non-technical content.

The second paper “Multiobjective Candidates Optimization (MODCO): A Cluster

Forming Diﬀerential Evolution Algorithm” [3] was mainly developed by this author

as a ﬁrst attempt to create a MOEA which complied with the MODCO goals. This

paper was accepted, mostly due to its novel approach to decrease result set cardinality.

Here, we want to brieﬂy present the main arguments and goals described in the

MODCO article before evaluating and discussing, as these are the basis of the second

article on Cluster-Forming Diﬀerential Evolution (CFDE). That is, the concrete al-

gorithm suggested in the CFDE paper is evaluated wrt. the goals of this paper. The

CFDE article is presented in Section 2.3. Note that the article on MODCO holds no

algorithms or results, and thus does not fully comply with the taxonomy from the

introduction to section 2.

2.2.1 Introduction, arguments and goals

The concept of MODCO is the optimization of a user-deﬁned low number of dis-

tinct candidates, with a user-deﬁned degree of distinctiveness. This is in contrast to

the traditional MOEA goal of covering as much as possible of the true Pareto-front.

However, having a full population of alternatives wrt. the trade-oﬀs between objec-

tives, just introduce more choices to be made by the DM. Basically, a standard EA

population is simply of too high cardinality to be directly usable.

14

MODCO basics MODCO tries to circumvent this problem by incorporating gen-

eral, higher level information to the search, replacing step 2 in ﬁgure 2. This info

includes relevant practical information such as:

1. How much time and money can be spend on post-processing the solution set?

2. How many solutions is it feasible to inspect and compare?

3. How distinct must solutions be in order to be distinguishable in tests?

This information is relevant, and these questions must be answered, for most real-

life applications. The goal of MODCO is to use this information as a guide towards

a small, ﬁnal set of distinct candidates, replacing step 2 in ﬁgure 2, such that this

ﬁnal selection is not only the responsibility of the DM, but is incorporated in the

optimization process.

Arguments for the soundness of MODCO To argue for the soundness of

MODCO, four main categories were identiﬁed and argued about in detail in [2]. Here,

we brieﬂy review these categories of arguments:

1. Post-processing of many Pareto-optimal solutions.

2. Physical realization of a solution.

3. Decision making among large sets of solutions.

4. Algorithmic and theoretical perspectives.

Post-processing many Pareto-optimal solutions is essential in most real world

applications, where it is necessary to further investigate the most promising solutions

from an optimization process, in order to to ﬁgure out which one(s) to physically

implement. This process is expensive, time consuming, and it may only address a

small part of the full ﬁnal design. Therefore it would be preferable to only have a

small set of solutions to perform post-processing on.

Post-processing is expensive, since this typically involves prototyping and testing,

where both may be very costly. Especially prototyping can be expensive, due to ma-

terials. Post-processing also includes more detailed simulations, which is normally

very time consuming. For example, a conducting a full computational ﬂuid dynam-

ics (CFD) calculation may take days, making it infeasible to simulate hundreds of

solutions, which is not an unusual cardinality of MOEA results. Finally, when the

optimization process is only concerned with a part of a design, it is not feasible to have

hundreds of alternatives, since the impact on the full design is expensive to calculate.

The physical realization of solution sets returned from a traditional MOEA also

suﬀers some disadvantages due to the high cardinality. This is due to the problem of

having a 100 % accurate simulation, which is often far too costly. Normally, a few

15

percent of tolerance is acceptable in simulation. However, if the simulator inaccuracy

is greater than the diﬀerence between neighbor solutions from the MOEA resulting

population, a lot of these will not be distinguishable from each other. Again, a smaller,

more distinct set of candidate solutions would be preferable.

Choosing among large sets of solutions implies problems for the DM, wrt.

human factors. The DM may not have the technical background, or the knowledge

on optimization to be able to select among 100-500 diﬀerent Pareto-optimal solutions

to a problem with high dimensionality. The amount of solutions makes it infeasible

to inspect and evaluate the diﬀerent trade-oﬀs, and further it is often not possible to

state explicit preference rules in order to guide this ﬁnal selection. The only general

rule is, that solutions in knee regions in the objective space are preferred. Knee regions

are areas in the objective space, where a small improvement in one objective leads to

a high deterioration in another, or put another way, a convex part of the Pareto front.

Hence, a small set of solutions in knee regions would be preferable from a DM point

of view.

Algorithmic perspectives on MODCO reveal a few points where the search

for only small set of solutions should have advantages over traditional MOEA result

sets. First of all, allowing a decrease in the local population diversity may result in

a more focused search, in some ways similar to local search, which results in a better

convergence toward the true Pareto front. Secondly, the approach makes it possible

to use a smaller population size, since we are now no longer interested in covering the

full Pareto front, but only in locating a few distinct candidates. This naturally implies

fewer computational steps, especially function evaluations, but without compromising

the quality of the found distinct candidates. Finally, if we are able to locate knee

regions, these are indeed the interesting areas, so no other solutions need be reported.

The search for knees may further improve convergence towards the true Pareto front,

as will be argued later.

MODCO goals The goals of MODCO are related to, but diﬀerent from the goals

of traditional MOEAs, and are inspired by the desire of ﬁnding a small set of optimal,

distinct candidates, preferably in knee sections. The features of the ideal MODCO

algorithm are described in more detail in [2], but in general, the goals of a MODCO

algorithm are:

1. Closeness towards the true Pareto front. Ideally, solutions are placed on the

true Pareto front, but otherwise they should be as close as possible.

2. Global distinctiveness in the returned solutions, i.e. that solutions are dis-

tinguishable wrt. performance or design. Ideally, this feature is parametrized,

in order for the user to set how distinct returned solutions should be.

3. Local multiobjective optimality in returned solutions, i.e. it is preferred to

return solutions in knee regions.

16

The ﬁrst goal corresponds to the ﬁrst goal of traditional MOEAs, as we are still

interested in optimizing our results. The two last goals are diﬀerent, and based on the

idea of a decreased cardinality in the result set. That is, if we want to decrease the

result set cardinality, we need other ways of controlling the global diversity. Further,

as argued earlier, results in knee regions are more interesting for a DM, and so these

should be reported.

2.2.2 Evaluation and discussion

The paper on MODCO was rejected for the EMO 2009 conference, mostly due to its

non-technical nature. Much of the argumentation is based on experience, and thus

deals with many non-technical issues related to decision making and the quantiﬁcation

of DM preferences. The connection to the more concrete paper on the CFDE algorithm

is strong and is pointed out several places, but this was not enough to be accepted.

On afterthought, the MODCO article could have been compressed, and maybe

inserted in the beginning of the CFDE paper, simply giving the goals and a short

argumentation. This could have increased the correlation between the theory and

the practical appliance of MODCO. However, we wanted to separate the theory from

practical experiments in order to be able to argue for the soundness and use of both.

2.2.3 Future work

The future work for the MODCO article includes a rewrite, incorporating some more

technical material from the technical report [4] also related to MODCO. This is due

to the criticism of the articles non-technical nature, and in order to make correlations

to CFDE more clear. As the MODCO paper gives much of the argumentation for

why this class of algorithms should even be considered, it is further important for the

CFDE article. Thus we need the article to be publicly available, and as such it must

be well founded.

Further, we are currently discussing the MODCO branch of MOO with a few well

known writers within the MOEA community, in order to establish the usability and

related issues of the MODCO branch of MOO. A few answers have so far been positive

towards the idea and have suggested further reading.

2.3 Cluster-Forming Diﬀerential Evolution

The article “Multiobjective Candidates Optimization (MODCO): A Cluster Forming

Diﬀerential Evolution Algorithm” [3] was mainly developed by this author as an em-

pirical investigation on the ﬁrst concrete algorithm which conforms to the MODCO

goals. The article was accepted at the EMO (Evolutionary Multiobjective Optimiza-

tion) 2009 conference. Due to this fact and the correlation to the MODCO paper [2],

future work of this paper is presented in the section devoted to this; Section 3.

17

2.3.1 Content and ideas

The article [3] demonstrates the ﬁrst instance of a MODCO algorithm; the Cluster-

Forming Diﬀerential Evolution (CFDE) algorithm. It is intended as a sequel to the

article introducing MODCO as described above, implementing and evaluating a con-

crete instance of the suggested algorithm class. The algorithm follows the goals of

MODCO, resulting in the following features:

1. User deﬁned result set cardinality, parametrized as K

NC

.

2. User deﬁned performance distinctiveness, parametrized as K

PD

.

3. The ability of converging towards knee regions.

These features corresponds to the MODCO goals, also used for evaluating the

CFDE algorithm performance. For reference, the CFDE algorithm is depicted in

Algorithm 2. As usual we have a population P of size N, but the primary data struc-

ture is now a vector of subpopulations P

i

∈ P of size N/K

NC

each, assuming WLOG

that N mod K

NC

= 0, as seen in ﬁgure 6. Further, a vector holding subpopulation

centroids, and a temporary oﬀspring vector are used. minDist(C

i

) is the function re-

turning the minimum distance from centroid C

i

to the nearest other centroid, whereas

the calculation of σ is depending on both K

PD

and the current problem.

Algorithm 2 Cluster-Forming Diﬀerential Evolution

Require: Population size N, K

NC

, K

PD

Ensure: K

NC

diﬀerent non-dominated individuals.

1: Initialize K

NC

subpopulations with N/K

NC

random individuals in each

2: while Halting criterion has not been met do

3: Perform global DE-based mating - store incomparable oﬀspring

4: Calculate subpopulation centroids C

i

5: Migrate incomparable oﬀspring to nearest subpopulation wrt. centroid

6: for All P

i

∈ P do

7: if minDist(C

i

) < σ then

8: Assign nearest other centroid distance to each individual x

i,j

∈ P

i

9: else

10: Assign knee utility function value to each individual x

i,j

∈ P

i

11: end if

12: end for

13: Assign ﬁnal ﬁtness wrt. global pareto rank, then secondary ﬁtness

14: Truncate subpopulations wrt. ﬁnal ﬁtness

15: end while

16: Return K

NC

solutions, by returning the non-dominated solution closest to the

subpopulation centroid from each subpopulation.

18

N/

Of f spr i ng par t

Par ent par t

K

N C

K

N C

N/ K

N C

Figure 6: Population after migration

Main algorithm To initialize, the CFDE algorithm creates N random individuals,

and insert these in K

NC

subpopulation with N/K

NC

individuals in each, which is

depicted as the parent part in ﬁgure 6. For problems with two objectives, an ini-

tial sorting is performed before insertion into subpopulations, in order to enhance

clustering.

Then the CFDE algorithm proceeds by performing global mating with replace-

ment as in usual DE, and as seen in Algorithm 1 and in ﬁgure 5. However, it stores

the incomparable oﬀspring in a temporary oﬀspring vector, until it can be determined,

which subpopulation they should belong to. From the parent part of the subpopula-

tions, a centroid for each is then calculated. Following this, the incomparable oﬀspring

are migrated to the subpopulations with the nearest centroid.

At this point, the CFDE algorithm determines which of the two secondary ﬁt-

ness measures to use for each subpopulation. The secondary ﬁtness measure is each

individual’s distance to the nearest other centroid, if the subpopulations centroid is

too close to its nearest neighboring centroid

2

. This makes subpopulations reject each

other by favoring the individuals furthest away from other subpopulation centroids.

Further, this most likely penalizes solutions created far from the subpopulation cen-

troid, as these are most likely to be close to other subpopulation centroids, eﬀectively

enhancing clustering. In case the centroid of the subpopulation is suﬃciently far away

from its neighbours, the secondary ﬁtness measure is Branke et al.’s utility function,

favoring individuals in knee regions [12]. Thus, a subpopulation will search for knees,

if not too close to another subpopulation centroid.

2

Wrt. σ which depends on K

PD

and the problem, see Section 2.3.2 for an example calculation.

19

Still, CFDE maintain focus on convergence towards the true pareto-front, so it

then assigns to each individual a global pareto rank using the NSGA-II non-dominated

sorting. This is used for assigning each individual a ﬁnal ﬁtness, such that the ﬁnal

ﬁtness incorporates both rank and secondary ﬁtness measure in a total order.

Finally each subpopulation is truncated to the original size of N/K

NC

using the

truncation mechanism of NSGA-II. Here, the main diﬀerences are, that truncation

is done locally in subpopulations, and that the subpopulations are truncated using

one of the two secondary ﬁtness measures, which is incorporated in the ﬁnal ﬁtness.

Hence, some subpopulations may be truncated using distance, and others using the

knee utility function. This way subpopulations may be attracted to diﬀerent knee

regions, while forming clusters during the evolutionary process. As we return only

one solution from each subpopulation, we get the wanted number of distinct solutions

returned.

Novel contributions In the article [3], a more thorough walktrough of all the

mechanisms of the CFDE algorithm is found, but here we will only go through the

novel contributions and their compliance to the MODCO goals. This is due to many

mechanisms used in the CFDE algorithm having already been explained in this report.

The novel contributions in the CFDE algorithm are:

1. Flexible subpopulation based Diﬀerential Evolution.

2. Centroid calculation for each subpopulation allowing for migration.

3. Subpopulation centroid distance based secondary ﬁtness.

4. Alternating secondary ﬁtness assignment to subpopulations.

The idea of subpopulations is not new in itself, and has also been used in single-

objective optimization, e.g. in multi-modal optimization, where decision spaces con-

tains several global optima to be discovered, not unlike multi-objective optimization.

However, to this authors knowledge, this is the ﬁrst algorithm with a parametrized

amount of subpopulations subject to Diﬀerential Evolution. The subpopulation ap-

proach is naturally crucial for the full algorithm, in that it allows for reporting back

the wanted number of distinct candidates.

The centroid calculation for each subpopulation is simple and based on the place-

ment of the parent part of each subpopulation in objective space. It gives the average

placement of the elite (parent) part of each subpopulation, and is used for both migra-

tion and the centroid distance calculation used in the secondary ﬁtness assignment.

Thus, calculating centroids is essential for the clustering ability of CFDE. For each

subpopulation P

i

, we calculate the centroid C

i

= [C

i,1

, C

i,2

...C

i,M

] as the average point

of the elite in objective space:

C

i,m

=

N/K

NC

j=1

f

m

(x

i,j

)

N/K

NC

, m = 1..M. (6)

20

The centroid distance calculation enhances clustering by penalizing the individuals

of any subpopulation with the distance to the nearest other subpopulation centroid.

As this is to be maximized, solutions created far from the subpopulation centroid are

most prone for penalty, as they are most likely to be close to the centroid of another

subpopulation. This further makes subpopulations reject each other, as solutions gen-

erated the furthest away from other centroids are then favored. Let dist(x, y) denote

the distance in objective space between point x and point y, each of dimension M.

Further, let min(S) denote the minimal element of the set S. For the subpopulation

P

i

, we then assign to each individual x ∈ P

i

a secondary ﬁtness SF as:

SF(x) = min(¦dist(f(x), C

j

), j = 1..K

NC

, j = i¦) (7)

The subpopulation approach of CFDE also makes it possible to assign alternating

secondary ﬁtnesses, according to case. To this authors knowledge, this is also a

new idea, facilitating the use of an arbitrary amount of secondary ﬁtness functions.

However, this calls for a priority among secondary ﬁtnesses. In the case of CFDE, we

are only interested in distinct candidates, and thus the two secondary ﬁtness measures

conﬂicts only if knee regions are located too close wrt. σ, i.e. we may not report back

solutions in knee regions located too close too each other, as they are not considered

distinct. Otherwise, knee search will not deteriorate clustering, as knee regions are

typically small, and likewise, the centroid distance assignment will not prevent knees

from being found when distinctiveness is achieved for any subpopulation.

The alternate secondary ﬁtness measure, the utility function proposed in [12], is

intended to discover knee regions by calculating an average ﬁtness value for a large

number of randomly sampled weight vectors. If this average ﬁtness is good, the

individual is more likely to reside in a knee region. Knee regions are characterized by

the fact that a small improvement in one objective, will result in large deterioration in

another objective. The utility function takes only one argument, precision, denoting

the number of sample weight vectors to apply. Let λ denote the weight vector of

dimension M, with

m

λ

m

= 1. Then we calculate the secondary ﬁtness SF with

precision precision of each individual x ∈ P

i

as:

SF(x) =

precision

p=1

λ

p

f(x)

precision

(8)

Lastly, let us see how CFDE comply with the goals of MODCO, and how this is

supported by the novel contributions. Goal 1 of convergence is ensured by elitism

within subpopulations, i.e. the number of globally non-dominated solutions within a

subpopulation can only increase. Further this is enhanced by the elitism rules using

Diﬀerential Evolution. Goal 2 of distinctiveness is achieved using the subpopulation

approach, which together with the centroid calculation, migration and centroid dis-

tance penalty forms clusters from the initial random subpopulations, each reporting

back a distinct solution. Further, the subpopulation approach is easily parametrized

to enable user setting. Goal 3 of detecting knees is achieved using Brancke’s knee

utility function when distinctiveness is reached.

21

2.3.2 Results

To provide results, the CFDE algorithm was demonstrated on 3 kinds of problems

from MOEA literature; the 2D ZDT problems [6; 5], the knee problems of Branke et

al. [12], and all non-constrained 3D DTLZ problems [11], all with problem settings

as suggested in the respective papers. The experiments were performed with respect

to the MODCO goals:

• Convergence performance (MODCO goal 1)

• Global distinctiveness (MODCO goal 2)

• User-deﬁned performance distinctiveness (MODCO goal 2)

• Local multiobjective optimality (MODCO goal 3)

First, we check CFDE convergence against the DEMO versions, which have demon-

strated good performance on many problems [9; 10]. Next, we want to demonstrate

convergence to K

NC

clusters, how we may change solution diversity by setting K

PD

,

and ﬁnally that CFDE are able to locate knees.

We demonstrate the use of K

PD

only on knee problems, as this is only relevant

for such problems. For ZDT and DTLZ problems, we therefore always set σ = ∞,

eﬀectively disabling knee search. For the 3D DTLZ problems, we have used a higher

K

NC

to ensure that we ﬁnd both extreme and intermediate trade-oﬀs. All other

settings of the algorithms investigated can be found in [3], Section 3, Table 1 and 2.

Convergence performance To deal with the diﬀerent cardinality of more standard

MOEAs and the CFDE algorithm, we use the universal notion of dominance. Here, we

compare the CFDE algorithm against DEMO

NSII

and DEMO

SP2

. One algorithmic

argument for MODCO is that the low number of returned solutions allows for a more

focused search because MODCO does not aim at an even distribution. Consequently,

a MODCO algorithm should be able to return solutions closer to the true pareto front.

So we wish to investigate in which extent the returned solutions from the CFDE

algorithm dominate the most similar solutions from the returned population of the

competing MOEAs, where similarity is measured as distance in objective space. This

way we see if the CFDE approach is competitive to simply picking K

NC

solutions

from the resulting populations of the DEMO versions.

For all results presented in [3], these are generated using the NSGA-II version of

global ranking and truncation in the CFDE algorithm. We have used 20 runs for both

the DEMO versions and for CFDE on each problem. For each generated population

of CFDE, we have compared each resulting individual to its most similar counterpart

from each of the DEMO populations. This indicator gives a percentage of the amount

of dominating, dominated and incomparable individuals CFDE was able to produce

and is independent of K

NC

and K

PD

.

22

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

F

2

F1

20 runs - 5 clusters

Figure 7: ZDT1 plot - 20 runs - 5 clusters

-1

-0.5

0

0.5

1

0 0.2 0.4 0.6 0.8 1

F

2

F1

20 runs - 10 clusters

20 runs - 5 clusters

Figure 8: ZDT3 plot - 20 runs

The results on each of the test problems can be found in Table 3 and 4 in [3] along

with a more thorough walkthrough, but here we will only note that the CFDE results

are almost never dominated by results from the DEMO versions, in fact only on a

single test problem with a high number of local Pareto fronts near the global one,

the DEMO versions seems to outperform CFDE. On the rest of the 15 test problems,

CFDE demonstrates equal or superior convergence, with up to 87 % dominating

solutions produced compared to the most similar DEMO counterparts. Overall, the

CFDE algorithm appears to be superior to the DEMO versions wrt. convergence.

Global distinctiveness Global distinctiveness is achieved by the CFDE algorithm

using the centroid distance to repel subpopulations. Figure 7 and 8 displays the

returned results of 20 runs of the CFDE algorithm on ZDT1 and ZDT3, using K

NC

= 5

and K

NC

= 10. As mentioned above, we here set σ = ∞. Similar robust convergence

is seen for the other test problems, i.e., CFDE found roughly the same set of distinct

candidates in repeated runs.

As can be seen in ﬁgure 7, all of the 20 runs of CFDE returned similar distinct

solutions. In ﬁgure 8, we see that using K

NC

= 5 ensures a result returned from

each of the 5 patches of the true pareto-front and again we see only a small variation.

However, using K

NC

= 10 makes the returned results be much more spread in the

20 runs, since there are now more clusters to be formed than there are discontinuous

patches. As can be observed from the density, solutions will here most often seek the

most outer part of the patches making the returned solutions as distinct as possible.

User-deﬁned performance distinctiveness The MODCO parameter K

PD

∈

[0, 1] allows the DM to set how distinct the returned solutions should be. A low value

corresponds to a low distinctiveness and a high value to a high distinctiveness, but

as mentioned, K

PD

is problem dependent. So to demonstrate the eﬀects of changing

K

PD

, we have chosen DEB3DK as an illustrative example, as it allows us to visualize

the eﬀect of altering the balance between knee search and subpopulation repelling.

We ﬁrst demonstrate the calculation of σ used in the CFDE algorithm. First, we

will assume settings K

PD

= 1 and K

NC

= 5. For DEB3DK, we may use reference

23

points z

∗∗

= (0, 0, 0) and z

I

= (8, 8, 8), which span the interesting part of the objective

space. Then we can calculate:

σ = K

PD

/K

NC

[[z

∗∗

−z

I

[[ = 1/5

√

192 ≈ 2.77 (9)

In this setting, subpopulations will repel each other if they get within a distance

of 2.77 of each others centroids. Setting K

PD

= 1 should ensure maximum global

distinctiveness, such that we get clusters uniformly spread across the objective space

spanned by the reference points. Contrary, setting K

PD

close to zero enables clusters

to get closer to each other while searching for knees.

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

0

1

2

3

4

5

6

7

8

F3

True front

K

P

D=1.0 - 20 runs

K

P

D=0.5 - 20 runs

K

P

D=0.2 - 20 runs

F1

F2

F3

Figure 9: DEB3DK plot - investigating user deﬁned performance distinctiveness

Figure 9 illustrates the results of setting K

PD

to 0.2, 0.5 and 1.0. For K

PD

= 1,

the 5 clusters are equidistant around the single knee, where one cluster is placed. The

four clusters not in the knee are repelled from each other as they reach a distance of

2.77 between centroids, as was demonstrated in the example calculation above. The

four clusters not in the knee are located in the partial knees on the lines forming a

cross. For K

PD

= 0.5, we always hit the knee with one cluster. Further, it can be

seen that some solutions has found other knee regions, crawling towards the one in

the middle, but not being allowed to get too close. Setting K

PD

= 0.2 results in all

clusters getting very close to the single knee region. Overall, it is clear that increasing

K

PD

indeed makes clusters repel each other more.

Local multiobjective optimality Figure 10 and 11 illustrates the knee problems

DO2DK and DEB2DK, with the resulting CFDE individuals of 20 runs. In the

DO2DK problem, we set K = 4 and s = 1.0, such that we have exactly the same

24

0

1

2

3

4

5

6

7

8

0 1 2 3 4 5

F

2

F1

true front

20 runs - 5 clusters

Figure 10: DO2DK plot - 20 runs - 5 clus-

ters - 4 knees

0

1

2

3

4

5

6

7

8

0 1 2 3 4 5 6 7 8

F

2

F1

true front

20 runs

Figure 11: DEB2DK plot - 20 runs - 4 clus-

ters - 4 knees

settings as has been used for creating the results illustrated in ﬁgure 4 in [12]. For

the 20 runs depicted in ﬁgure 10, it may be noticed, that the density of solutions near

knee regions is very high. When using K

NC

= 5, CFDE ﬁnds the 4 knee regions very

precisely, while one cluster typically hits an outer solution, or is caught in-between

knee regions. For DO2DK, we have used K

PD

= 0.75 corresponding to σ = 1.5. This

way we keep clusters separated, while still allowing for knees to be found.

For the DEB2DK problem, we have used K = 4 to replicate the results illustrated

in ﬁgure 5 in [12]. In ﬁgure 11, we again see that for the 20 runs the density of

solutions near knees are very high. Here, we have to set K

NC

to be equal to the

number of knees, and it is clear that all knees are discovered in all runs. Here, we

have used K

PD

= 0.2 corresponding to σ = 0.5. This is low, so the centroid distance

assignment is rarely used. Hence, subpopulations converge to knees, and as long as

σ > 0, the clusters formed will not overlap.

From the ﬁgures 10 and 11, and further ﬁgure 9, it is clear, that CFDE is indeed

able to locate knee regions. Further, it has been demonstrated how to balance the

search using K

PD

resulting in diﬀerent σ values.

2.3.3 Evaluation and discussion

The paper on the CFDE algorithm [3] was accepted at EMO 2009, and this author

is to present it in Nantes, France, in April 2009. The referees were impressed with

the novel approach, the presentation and the experiments, even though there was

some complaints about submitting both an abstract MOEA class and a concrete

implementation in two diﬀerent papers.

However, many usable comments were given in the reviews, especially considering

that decreasing the result set cardinality is a quite novel approach in MOEA research,

and some have given rise to future work. Given that we invented the binary per-

formance indicators for MOEAs with diﬀerent result set cardinality, the experiments

were still said to be illustrative and usable. This, along with a somewhat simple and

easily understandable parametrization are said to provide a potentially very useful

functionality, very competitive to pruning the result sets of standard MOEAs.

25

3 Future work

This section is devoted to my future work, which mainly will be concerned with the

appliance of MODCO algorithms in relation to Grundfos problems, with the CFDE

algorithm being the ﬁrst. The major challenge will be applying the newly developed

algorithms to the real world problems supplied by Grundfos, which are typically much

more complex than test problems, along with being constrained.

Apart from working on Grundfos problems, new performance indices, possibly

facilitating comparison between MOEAs of diﬀerent result set cardinality are to be

invented, along with new versions of MODCO algorithms. Lastly, we will investigate

which conferences and journals could be relevant to submit to, and how this could

coincide with my research.

In general, the hope is a synergy between the new knowledge acquired from testing

on real world problems, and the general appliance of MOEAs. Grundfos have both

problems and decision makers, which are both required in order to investigate the full

multiobjective optimization process, which is of general interest to computer scientists

in the MOO area. On the other hand, the resulting designs of these investigations

should be of interest to Grundfos, as they are presumably both distinct and optimal,

wrt. user settings and the precision used in simulations.

3.1 Testing MODCO algorithms on Grundfos problems

Testing algorithms on real world problems are usually more interesting, in that the

results have a concrete, physical interpretation. Grundfos has a general interest in

MOO, since they have parametrized design spaces of their products, including simu-

lators to give the performance of designs. That is, the objective functions are actually

simulations of varying degree of precision. This is one reason for applying MODCO

algorithms, as discussed in Section 2.2.

Considering MOEAs, the decision space of test problems holds no meaning, and

neither does the objectives. Test problems are ﬁne for their purpose; revealing if

and to which extent algorithms may solve them, in spite built-in traps and pitfalls

- e.g. many local Pareto fronts in objective space. Test problems further facilitate

performance comparison of algorithms, which is of general interest to the community.

Real world problems are contrary prone to more advanced investigations, in that

we may derive meaning and sense from e.g. the design space of centrifugal pumps.

Likewise, the objectives are measurable in a physical sense, and are as such much more

interesting. Working with specialists should enable new knowledge to be extracted

from designs proposed by e.g. the CFDE algorithm, which are of natural interest

to Grundfos. Likewise, we are more likely to uncover new knowledge on algorithmic

behavior, when both design space and results are comprehendable.

From testing on Grundfos problems, we hope to investigate several interesting

facets related to MODCO research:

26

• An approach for constrained problems.

• Whether MODCO is well suited for many-objective optimization.

• How changing MODCO parameters will aﬀect the result.

• Whether the subpopulation approach could result in less computations.

3.1.1 Constrained problems

Most real world problems are constrained, i.e. solutions belonging to a part of the

decision space are not feasible, since they violate constraints. The constraint space

is deﬁned by the constraint functions as seen in Equation 1, which maps the feasible

part of decision space o

D

to the feasible part of objective space, o

Z

independently

of the objective functions. This part is of varying shape and size, depending on the

problem at hand. However, for real world problems, the constrained space is often a

very large part of the objective space, and as such feasible solutions are much harder

to generate than unfeasible ones.

So the ﬁrst challenge connected to the appliance of MODCO algorithms to Grund-

fos problems, is to ﬁnd an approach for constrained problems. One idea so far is to

use the approach of Generalized Diﬀerential Evolution (GDE3) [14], where additional

rules are incorporated in the comparison of solutions. A feasible solution will always

be preferred over an unfeasible one, but if two solutions are both infeasible, compar-

ison then takes place in constraint space, such that the most dominant solution wrt.

the constraint functions are favored. That is, solution x is considered dominant to

solution y if x have the same or better constraint violation on all constraint functions,

and has at least one constraint function with less constraint violation than y. This

facilitates a search steadily progressing towards the feasible, unconstrained part of

objective space.

3.1.2 Many-objective optimization

Many-objective optimization is when our problem has more than 3 objectives, which

is also the case in many Grundfos related problems. The more precise we wish to

make our model, the more objectives and constraint functions may be included, and

a centrifugal pump design problem may easily contain 5 - 10 objectives.

The challenge of many-objective optimization lies in the dimensionality of the

objective space. The more dimensions, the more trade-oﬀs between the objectives can

be found. This may be illustrated by a simple counting argument; the more dimensions

of the objective space, the more edges and corners of the induced hypercube will exist,

representing possible non-dominated solutions. Thus, covering a high-dimensional

Pareto front can be very hard, and will require a much larger population than for

problems with 2 or 3 objectives.

27

MODCO algorithms, and with this the CFDE algorithm, are potentially well suited

for many-objective problems with high-dimensional Pareto fronts, since the more fo-

cused search does not need to put any eﬀort into covering the full pareto-front, which

can be very diﬃcult in high dimensional spaces, as argued above. As higher dimen-

sional problems also calls for larger population sizes, the problem with choosing from

the result set again becomes apparent for standard MOEAs, whereas this is circum-

vented using MODCO algorithms, e.g. CFDE.

3.1.3 The eﬀect of changing MODCO parameters

Another hope from applying MODCO algorithms to Grundfos problems, is to make

an investigation into diﬀerent parameters settings of the MODCO algorithms. For

test problems, the number of knees and the interesting part of the objective space

is given, but this is not the case when handling real world problems. Here, any a

priori knowledge of the problem and its objective space is hard to come by due to

the complexity of the objective functions, and as discussed, should not be considered

available in general.

An investigation into the newly introduced MODCO parameters could reveal how

these are used by a DM, and to which extent parameter changes aﬀect the results.

The experiments in [3] demonstrated a robust behavior of the CFDE algorithm, in

that solutions converged to the same areas of the objective space under changing

parameter settings. This behavior needs to be veriﬁed for real world problems, espe-

cially for constrained problems, and for problems with an unknown number of knee

regions, arbitrarily placed in the objective space. This includes an investigation of

the correlation between the parameters N, K

NC

and K

PD

to discover any synergies

or discrepancies between these.

Further, a study on the DE parameters F and CF could also be interesting,

especially in relation to the subpopulation approach. DE is intended to cause smaller

and smaller changes during a run, given individuals converge towards the same area

of the decision space, e.g. mapping to a knee region or the global Pareto front. Here,

we want to ﬁnd out if the subpopulation approach enhances this behavior, which

resembles local search.

3.1.4 A more eﬃcient search?

Lastly, we intent to make some investigations as to ﬁnd out if the subpopulation ap-

proach leads to a more eﬃcient search. Overall, we intend to discover if all MODCO

mechanisms are truly enhancing convergence, or if there are any discrepancies among

them. This includes making experiments regarding the number of computations per-

formed by MODCO algorithms vs. standard MOEAs.

It is likely that we may decrease the population size of e.g. the CFDE algorithm

without aﬀecting the quality of the reported solutions, since K

NC

<< N, and the K

NC

clusters each represent only a single solution. As the number of function evaluations

28

is depending on the population size, we may decrease the number of computations

correspondingly. The question is, how much we can decrease population size without

having the quality of results deteriorating. Further, the secondary ﬁtness assignment

of CFDE requires less computations than the density measures of NSGA-II or SPEA2,

but we still need some research on the eﬃciency of the alternating measures, wrt.

newly developed performance indices, as discussed below.

3.2 Inventing performance indices

One related challenge when designing new algorithms, is how to judge their per-

formance. As mentioned, the MODCO goals diﬀer from the ones of more standard

MOEAs, and hence new performance indices must be invented to facilitate comparison

- towards both standard MOEAs and other MODCO algorithms.

Several performance indices are suggested in [4], but due to the diﬀerent cardinality

of MODCO results, these need to be adapted and possibly changed in order to apply.

So due to time and space limitations, these were not used in [3], where a more general

approach was used in tests, based on the universal notion of dominance and objective

space distance. However, they are a starting point for the future development of PIs,

which are to be incorporated in the new version of [2], as discussed in Section 2.2.

The ﬁrst goal of MODCO is closeness towards the true Pareto front, which is the

same goal as of standard MOEAs. That is why we may adopt several metrics from

MOO literature, but with respect to diﬀerent cardinality of compared result sets. For

example, it may be possible to use the hypervolumen indicator [13], which calculates

the dominated space of solutions, given a reference point. This has the advantage of

giving an indication of to which extent a solutions set is better than another, which

is much more graded than the dominance relations used for statistics in [3]. However,

the hypervolumen indicator will favor solutions sets which covers as much objective

space as possible, and thus the lower cardinality of MODCO solutions set will be

penalized. This may be circumvented by only comparing the most similar solutions

in case of diﬀerent cardinality of results, as we did in [3], but more investigation is

needed in order to see if such a comparison is really fair. Same approach could be

used for the epsilon indicator [13], which gives the distance a population must be

moved towards an ideal point in order to dominate another population. Both of these

indicators are Pareto compliant, but will be aﬀected by the low number of results in

MODCO solution sets.

The second goal of MODCO is global distinctiveness, and we may measure this

in decision space, objective space or by using category functions [2; 4]. Again we

may be inspired by standard metrics such as the ´

∗

2

indicator giving the average

distance between solutions which are far enough away from each other in objective

space, wrt. some niching radius corresponding to σ in CFDE, as noticed in [4]. A

similar indicator ´

2

exist for design distinctiveness. Further, indicators for topology

based distinctiveness are suggested in [4], such as the one developed by Rasmus Ursem,

to reveal topological distinctions such as hills or valleys, based on testing a solution

29

position wrt. neighboring solutions. Using these diﬀerent indicators, we may see to

which extent the intended distinctiveness is achieved, based on statistical analysis.

The third goal of MODCO is to discover knee regions, and so we need indicators

to reveal this, as knee regions are not in general known a priori, as was the case with

the knee problems used for illustration in [3]. One such indicator is suggested in [4],

based on knowledge of an ideal corner of the hypercube spanned by the objective

functions. However, the utility function of Brance used in the CFDE algorithm will

give a similar indication of knee regions, and this is very simple to both understand,

adjust and use.

3.3 Inventing alternate MODCO algorithms

In order to investigate the performance of MODCO algorithms, it would be nice

to have some alternatives to compete against each other. So a part of my future

research will be devoted to inventing alternatives to the CFDE algorithm, to facilitate

comparison.

The ﬁrst step along this path is to create CFDE

SP2

, the CFDE algorithm using

SPEA2 ranking and truncation, as described in Section 2.1. The SPEA2 pareto

strength ranking is more graded than the non-dominated sorting of NSGA-II when

not all individuals of the population are non-dominated, which may be an advantage

for complex, constrained problems. Thus we get both CFDE

NSII

and CFDE

SP2

to

compare against each other, just as the DEMO versions. These should incorporate the

GDE3 approach described in Section 3.1.1, in order to apply to constrained problems.

Second step will be incorporating other forms of distinctiveness into the CFDE

algorithms, such that we may measure this in decision space instead of objective space,

as well as using category functions, as mentioned in [2] and described further in [4]. In

short, category functions map solutions in decision space to easily identiﬁed categories,

e.g. motors of diﬀerent standard sizes or electronics components of various standards.

These a priori known categories are interesting distinctions wrt. solutions to the

DM. However, note that the category mapping is not explicit preference functions.

This way we investigate how the alternate deﬁnitions on distinctiveness will aﬀect

the search and the results, to which extent they make sense to the DM, and how the

MODCO parameters are used in the diﬀerent cases.

Final step will be inventing MODCO algorithms not based on the CFDE algo-

rithm, but with the same parameters to adjust. This should lead to other approaches

than the one based on a ﬁxed user deﬁned number of subpopulations, which at that

point should be thoroughly researched. One may imagine MODCO algorithms, where

only upper and lower bounds on K

NC

are to be set, where after K

NC

and K

PD

will be

adjusted dynamically during the run, while automatically optimizing distinctiveness

and searching for knee regions. This could entail some mechanisms for dynamically

creating subpopulations from one overall population, when knee regions or other at-

tractive areas of the objective space are found, e.g. wrt. the optimization of global

distinctiveness.

30

3.4 Relevant conferences and journals

Here, we will provide an overview of the conferences and journals, to which we may

submit future publications. We will ﬁrst list conferences and then journals, as well

as give an early indication of what work may be done by these deadlines. The list

further shows approximate dates for the conferences.

• Evolutionary Multiobjective Optimization (EMO), April, 2009.

• IEEE Congress on Evolutionary Computation (CEC), May 2010.

• Genetic and Evolutionary Computations Conference (GECCO), July 2010.

These conferences are all focused on evolutionary computation, and are all well

suited for papers on new algorithms or new applications. EMO is more specialized, in

that it is more focused on multiobjective optimization, and thus is very relevant to this

author. So the acceptance of [3] for this conference is a good opportunity to discuss my

work here. However, as CEC and GECCO are much further away in the future (2010),

relevant submission will here include reports on testing new MODCO algorithms on

Grundfos problems including new performance indices, as will be developed by the

end of 2009, the approximate deadline for submissions to these conferences.

Relevant journals include:

• Applied Soft Computing - www.softcomputing.org/

• IEEE Transactions on Evolutionary Computations -

http://ieee-cis.org/pubs/tec/

Applied Soft Computing is a journal in connection to online conferences held by the

world federation of soft computing (WFSC), and are focused on industrial applications

of soft computing, such as evolutionary algorithms. The journal is connected to the

major publisher Elsevier, which hosts many diﬀerent forms of scientiﬁc papers and

journals. The IEEE transactions on Evolutionary Computations is also interested in

both methods and applications. For these journals, a thorough application of the

initial CFDE algorithm versions (CFDE

NS−II

and CFDE

SP2

) on Grundfos problems

could be very relevant, in that it is both a novel approach and further it is tested on

an industrial engineering application, which is more interesting than test problems.

This project would include a thorough rewrite of [2] and [4], to produce a single,

introductory paper on the abstract class of MODCO algorithms and how to measure

performance. This work is to be done during the spring and summer of this year,

2009.

31

List of Figures

1 Mapping from decision space to objective space - assuming maximization 3

2 Multi-objective optimization process . . . . . . . . . . . . . . . . . . . 4

3 Multi-Objective Evolutionary Algorithm main loop . . . . . . . . . . . 7

4 Genetic algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5 Diﬀerential Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

6 Population after migration . . . . . . . . . . . . . . . . . . . . . . . . . 19

7 ZDT1 plot - 20 runs - 5 clusters . . . . . . . . . . . . . . . . . . . . . . 23

8 ZDT3 plot - 20 runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

9 DEB3DK plot - investigating user deﬁned performance distinctiveness . 24

10 DO2DK plot - 20 runs - 5 clusters - 4 knees . . . . . . . . . . . . . . . 25

11 DEB2DK plot - 20 runs - 4 clusters - 4 knees . . . . . . . . . . . . . . . 25

List of Algorithms

1 Diﬀerential Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Cluster-Forming Diﬀerential Evolution . . . . . . . . . . . . . . . . . . 18

References

[1] Justesen, P.D. and Ursem, R.K.: Multiobjective Distinct Candidates Optimization

(MODCO): A Cluster-Forming Diﬀerential Evolution Algorithm. In: Accepted at

the Evolutionary Multiobjective Optimization (EMO 2009) conference. Prelimi-

nary download:

www.daimi.au.dk/

∼

ursem/publications/Justesen EMO2009 CFDE.pdf

[2] Ursem, R.K., Justesen, P.D.: Multiobjective Distinct Candidates Optimization

(MODCO) - A new Branch of Multiobjective Optimization Research. In: Rejected

at the Evolutionary Multiobjective Optimization (EMO 2009) conference. Down-

load: www.daimi.au.dk/

∼

ursem/publications/Ursem EMO2009 MODCO.pdf

[3] Justesen, P.D. and Ursem, R.K.: Introducing SPDE2: A Novel DE Based Ap-

proach for Multiobjective Optimization. Rejected at the Parallel Problem Solving

from Nature X (PPSN 2008) conference. Download:

www.daimi.au.dk/

∼

juste/MOEA/article.pdf

[4] R. K. Ursem and P. D. Justesen: Performance Metrics for Multiobjective Distinct

Candidates Optimization (MODCO) Algorithms. In: Technical Report no 2008-

01, Dept. of Computer Science, University of Aarhus, 2008. Preliminary download:

www.daimi.au.dk/

∼

ursem/publications/Ursem TR 2008-01.pdf

32

[5] Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective

genetic algorithm: NSGA-II. In: IEEE transactions on Evolutionary Computation

6, 2002, pp. 182 - 197.

[6] Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. Wiley,

2002.

[7] Zitzler, E., Laumanns, M. Thiele, L.: SPEA2: Improving the strength pareto

evolutionary algorithm. In: Technical Report 103, Computer Engineering and

Networks Laboratory (TIK), Swiss Federal Institute of Technology (ETH) Zurich,

Gloriastrasse 35, CH-8092 Zurich, Switzerland (2001).

[8] Price, K. V., Storn, R.: Diﬀerential Evolution - a simple evolution strategy for

fast optimization. In: Dr. Dobb’s journal 22, 1997, pp. 18 - 24.

[9] Robiˇc, T., Filipiˇc, B.: DEMO: Diﬀerential Evolution for Multiobjective Optimiza-

tion. In: LNCS 3410/2005, pp. 520 - 533.

[10] Robiˇc, T., Filipiˇc, B.: Diﬀerential Evolution Versus Genetic Algorithms in Mul-

tiobjective Optimization. In: LNCS volume 4403/2007, pp. 257 - 271.

[11] Deb, K., Thiele, L., Laumanns M., Zitzler, E.: Scalable Test Problems for Evo-

lutionary Multi-Objective Optimization. In: Proceedings of the 2002 Congress on

Evolutionary Computation, CEC ’02, pp. 825 - 830.

[12] Branke, J., Deb, K., Dierolf H, Osswald, M.: Finding Knees in Multi-objective

Optimization. In: LNCS volume 3242/2004, pp. 722-731.

[13] J. Knowles, L. Thiele, and E. Zitzler. A Tutorial on the Performance Assessment

of Stochastic Multiobjective Optimizers. In: TIK Report 214, Computer Engineer-

ing and Networks Laboratory (TIK), ETH Zurich, February 2006. Download:

www.tik.ee.ethz.ch/sop/publications/.

[14] Kukkonen S., Lampinen, J.: GDE3: The third Evolution Step of Generalized

diﬀerential evolution. In: Proceedings of the 2005 Congress on Evolutionary Com-

putation, CEC ’05, pp. 443 - 450.

[15] Karthis, S., Deb, K., Miettinen, A.: A Local Search Based Evolutionary Multi-

objective Optimization for Fast and Accurate Convergence. In: LNCS volume

5199/2008, pp. 815 - 824.

[16] Handl, J., Knowles, J.: An Evolutionary Approach to Multiobjective Clustering.

In: IEEE Transactions on Evolutionary Computation, volume 11, issue 1, Feb.

2007, pp. 56-76.

33

- Alleged Plots Involving Foreign Leaders, U.S. Senate, Select Committee to Study Governmental Operations with Respect to Intelligence Activities, S. Rep. No. 755, 94th Cong., 2d sess
- How does office temperature affect productivity?
- Pli cacheté de WolfgaSome comments on the Pli cacheté n° 11.668, by W. Doeblin : Sur l'équation de Kolmogoroff B. Bru, M. Yor ng Doeblin
- Who Writes Linux in 2012?
- lhomoparentalite_faitsetcroyances
- m Turk Methods
- 2006 - An Empirical Comparison of Supervised Learning Algorithms
- Interdisciplinary Research
- 1989 - On Genetic Crossover Operators for Relative Order Preservation
- KB DNS Comparison Guide
- Dragon Speech Recognition Feature Matrix
- ULDs Overview[1]
- Roadmap OVH 20120620
- Emotiv - Research Edition SDK[1]
- LATEX
- Contrat_HUBIC_2012-03-19
- Lancement du double diplôme École Polytechnique/HEC Paris
- Ferrucci-Watson2010 - Build Watson - An Overview of DeepQA Project
- Who Writes Linux (2012)
- 2009 - A Structure-free Aggregation Framework forVehicular Ad Hoc Networks
- 2012 - Ron Was Wrong, Whit is Right
- 1996 - Expression et Reconnaissance des Intentions Conceptuelles Apports de la Multimodalité
- 2006 - Utilitarisme pragmatique et reconnaissance d’intention dans les actes de langage indirects
- 2007 - Intention Recognition Using a Graph Representation
- 2010 - State of the Art - Logic-Based Approaches to Intention Recognition

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue listening from where you left off, or restart the preview.

scribd