Attribution Non-Commercial (BY-NC)

942 views

Attribution Non-Commercial (BY-NC)

- Face Recognition using PCA
- Deb 2001 Multi-Objective Optimization Using Evolutionary Algorithms
- PPT- NSGA-II
- Optimization For Engineers By Kalyanmoy Deb
- MULTI-OBJECTIVE
- A generic evolutionary algorithm for single
- Face Recognition
- Face Recognition
- Evolutionary Algorithms
- DLC Evolutionary Algorithm
- 2003 Face Recognition Using LDA-Based Algorithms
- 092014TranHouHanNguenChau
- 1-s2.0-S1568494610000773-main RUTEO.pdf
- PARETO FRONT APPROXIMATION WITH ADAPTIVE WEIGHTED SUM METHOD IN MULTIOBJECTIVE SIMULATION OPTIMIZATION
- b. Linnhoff and s. Ahmad
- Design Optimization and Automation
- Multiperiod Optimization Model for Oilfield Production Planning
- 2017-Binary Grey Wolf Optimizer for Large Scale Unit Commitment Problem
- f4[1]
- Busqueda

You are on page 1of 55

Kalyanmoy Deb a

Kanpur Genetic Algorithm Laboratory (KanGAL)

Indian Institute of Technology Kanpur

Kanpur, Pin 208016 INDIA

deb@iitk.ac.in

http://www.iitk.ac.in/kangal/deb.html

a Currently visiting TIK, ETH Zürich

1

Overview of the Tutorial

• Multi-objective optimization

• Classical methods

• History of multi-objective evolutionary algorithms (MOEAs)

• Non-elitst MOEAs

• Elitist MOEAs

• Constrained MOEAs

• Applications of MOEAs

• Salient research issues

2

Multi-Objective Optimization

2

90% C

B

Comfort

1

40%

10k 100k

Cost

3

More Examples

flight flight

4

Which Solutions are Optimal?

f2 (minimize)

Domination: 6

2

3

objectives 1 5

1 3

2. x(1) is strictly better than x(2)

in at least one objective 2 6 10 14 18

f1 (maximize)

5

Pareto-Optimal Solutions

a set of solutions P , the non- f2 (minimize)

dominated set of solutions P 0

2

are those that are not dominated 5 Non−dominated

front

4

by any member of the set P . 3

1 5

3

optimal set

A number of solutions are optimal

6

Pareto-Optimal Fronts

f2 Min−−Min f2 Min−−Max

f1 f1

f2 Max−−Min f2 Max−−Max

f1 f1

7

Preference-Based Approach

Multi−objective

Single−objective

optimization problem Estimate a

optimization problem

Minimize f1 Higher−level relative

importance F = w1 f1 + w 2 f2 +...+ w M fM

Minimize f2 information

...... vector or

Minimize fM (w1 w2.. wM) a composite function

subject to constraints

Single−objective

optimizer

One optimum

solution

8

Classical Approaches

• Posteriori methods (generating solutions)

• A priori methods (one preferred solution)

• Interactive methods (involving a decision-maker)

9

Weighted Sum Method

• Construct a weighted sum of f2

objectives and optimize

a

M

X

F (x) = wm fm (x).

m=1 A b

w1

w2

c

• User supplies weight vector w d

f1

Pareto−optimal front

10

Difficulties with Weighted Sum Method

f2

• Need to know w A

B

• Non-uniformity in Pareto-

optimal solutions

• Inability to find some

C

Pareto-optimal solutions a

b D

Pareto−optimal front

f1

11

-Constraint Method

constrain all other

B

Minimize fµ (x),

C

subject to fm (x) ≤ m , m 6= µ;

• User supplies a vector D

ε1d f

ε1 ε1

a b ε1 c

1

• Non-uniformity in Pareto-optimal solutions

12

Difficulties with Most Classical Methods

objective optimizer many

A

1

times 0.9

0.8

• Expect a lot of problem 0.7

B

knowledge 0.6

0.5

f2

• Even then, good distribu- 0.4

0.3 Pareto−optimal

front

tion is not guaranteed 0.2

0.1 C

• Multi-objective optimiza- 0 D

−0.1

tion as an application of 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

f1

1

single-objective optimiza-

tion

13

Ideal Multi-Objective Optimization

Multi−objective

optimization problem

Minimize f1

Minimize f2

......

Minimize f M

Step 1

subject to constraints

IDEAL

Multi−objective

optimizer

solutions found solution

Higher−level

information

Step 2

Step 2 Choose one from the set

14

Advantages of Ideal Multi-Objective Optimization

erate case of multi-objective optimiza-

tion

– Step 1 finds a single solution

– No need for Step 2

• Multi-modal optimization is a special

case of multi-objective optimization

15

Two Goals in Ideal Multi-Objective Optimization

f2

optimal front

2. Maintain as diverse a distri-

bution as possible

f1

16

Why Evolutionary?

• Niche-preservation methods can be exploited to find diverse

solutions

1 1

0.8 0.8

0.6

0.6

f2

f2

0.4

0.4

0.2

0.2

0

0

0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

f1 f1

17

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

History of Multi-Objective Evolutionary

Year

Algorithms (MOEAs)

and before

140

120

100

80

60

40

20

0

Number of Studies

18

suggestion

ap-

NPGA

• Early penalty-based

• MOGA, NSGA,

• VEGA (1984)

• Goldberg’s

(1993-95)

proaches

(1989)

19

What to Change in a Simple GA?

Begin

Initialize Population

t=0

No

Cond?

Yes Evaluation

t=t+1 Reproduction

Crossover

Mutation

20

Identifying the Non-dominated Set

Step 2 For a solution j ∈ P (but j 6= i), check if solution j

dominates solution i. If yes, go to Step 4.

Step 3 If more solutions are left in P , increment j by one and go

to Step 2; otherwise, set P 0 = P 0 ∪ {i}.

Step 4 Increment i by one. If i ≤ N , go to Step 2; otherwise stop

and declare P 0 as the non-dominated set.

O(M N 2 ) computational complexity

21

An Efficient Approach

Step 1 Sort the population in descend-

ing order of importance of f1

Step 2, Front(P ) If |P | = 1,

T1

return P as the output

of Front(P ). Otherwise,

T = Front(P (1) −−P (|P |/2) ) and T2

a merged set M = T ∪ {i}. Return

M as the output of Front(P ).

O N (log N )M −2 for M ≥ 4 and O(N log N ) for M = 2 and 3

22

A Simple Non-dominated Sorting Algorithm

• Discard them from population

• Identify the next-best non-dominated set

• Continue till all solutions are classified

• We discuss a O(M N 2 ) algorithm later

23

Non-Elitist MOEAs

• Vector optimized EA (VOES) (Kursawe, 1990)

• Weight based GA (WBGA) (Hajela and Lin, 1993)

• Multiple objective GA (MOGA) (Fonseca and Fleming, 1993)

• Non-dominated sorting GA (NSGA) (Srinivas and Deb, 1994)

• Niched Pareto GA (NPGA) (Horn et al., 1994)

• Predator-prey ES (Laumanns et al., 1998)

• Other methods: Distributed sharing GA, neighborhood

constrained GA, Nash GA etc.

24

Non-Dominated Sorting GA (NSGA)

• First front: Fitness F = N to all

• Niching among all solutions in first front

• Note worst fitness (say Fw1 )

• Second front: Fitness Fw1 − 1 to all

• Niching among all solutions in second front

• Continue till all fronts are assigned a fitness

25

Non-Dominated Sorting GA (NSGA)

30

25 6

f1 f2 Fitness

x Front before after 20

−1.50 2.25 12.25 2 3.00 3.00

f2 15

0.70 0.49 1.69 1 6.00 6.00

4.20 17.64 4.84 2 3.00 3.00 1

10

2.00 4.00 0.00 1 6.00 3.43

1.75 3.06 0.06 1 6.00 3.43 5 3

2

−3.00 9.00 25.00 3 2.00 2.00 54

0

0 5 10 15 20 25 30

f1

• Non-dominated solutions are emphasized

• Diversity among them is maintained

26

Vector-Evaluated GA (VEGA)

• Each block is reproduced with one objective function

• Complete population participates in crossover and mutation

• Bias towards to individual best objective solutions

• A non-dominated selection: Non-dominated solutions are

assigned more copies

• Mate selection: Two distant (in parameter space) solutions are

mated

• Both necessary aspects missing in one algorithm

27

Multi-Objective GA (MOGA)

F Asgn. Fit.

nated solutions (say n)

1 2 3 2.5

• Fitness: F = n + 1

2 1 6 5.0

• A fitness ranking adjust-

3 2 2 2.5

ment

4 1 5 5.0

• Niching in fitness space

5 1 4 5.0

• Rest all are similar to

6 3 1 1.0

NSGA

28

Niched Pareto GA (NPGA)

respect to a small subpopulation (tdom )

• If one dominated and other non-dominated, select second

• If both non-dominated or both dominated, choose the one with

smaller niche count in the subpopulation

• Algorithm depends on tdom

• Nevertheless, it has both necessary components

29

NPGA (cont.)

X Check for

Y domination

t_dom

Parameter Space

Population

X

Y

30

Shortcoming of Non-Elitist MOEAs

• Elite-preservation is missing

• Elite-preservation is important for proper convergence in

SOEAs

• Same is true in MOEAs

• Three tasks

– Elite preservation

– Progress towards the Pareto-optimal front

– Maintain diversity among solutions

31

Elitist MOEAs

EA Elite

Elite-preservation:

• Maintain an archive of non-dominated solu-

tions f2 (minimize)

2

Progress towards Pareto-optimal front: 5

4

Non−dominated

front

3 5

1

3

2 6 10 14 18

f1 (maximize)

Maintaining spread of solutions:

f2

tion for a place in the archive

clusters

f1

32

Elitist MOEAs (cont.)

1995)

• Thermodynamical GA (TDGA) (Kita et al., 1996)

• Strength Pareto EA (SPEA) (Zitzler and Thiele, 1998)

• Non-dominated sorting GA-II (NSGA-II) (Deb et al., 1999)

• Pareto-archived ES (PAES) (Knowles and Corne, 1999)

• Multi-objective Messy GA (MOMGA) (Veldhuizen and

Lamont, 1999)

• Other methods: Pareto-converging GA, multi-objective

micro-GA, elitist MOGA with coevolutionary sharing

33

Elitist Non-dominated Sorting Genetic Algorithm

(NSGA-II)

f2 6

• Calculate (ni , Si ) for each 1 10 (5, Nil)

solution i 7 11

2 8

• ni : Number of solutions

9

dominating i 3

4

• Si : Set of solutions domi- 5

(0, {9,11})

nated by i f1

34

NSGA-II (cont.)

Non−dominated Crowding

sorting distance Pt+1

sorting

F1

Pt F2

F3

Qt

Rejected

Rt

35

NSGA-II (cont.)

f2 0

Cuboid

i-1

i

l

i+1

f1

36

NSGA-II Simulation Results

1.1 2

1

0 NSGA−II

0.9

0.8 −2

0.7

0.6 −4

f2 f2

0.5 −6

0.4

0.3 −8

0.1

0 −12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 −20 −19 −18 −17 −16 −15 −14

f1 f1

37

Strength Pareto EA (SPEA)

• Pareto-dominance to assign fitness

– External members: Assign number of dominated solutions

in population (smaller, better)

– Population members: Assign sum of fitness of external

dominating members (smaller, better)

• Tournament selection and recombination applied to combined

current and elite populations

• A clustering technique to maintain diversity in updated

external population, when size increases a limit

38

SPEA (cont.)

Fitness Assignment

Population Ext_pop

x

x x

x

x x

x

x

x

Function Space

Function Space

39

Pareto Archived ES (PAES)

• An (1+1)-ES

• Parent pt and child ct are compared with an external archive At

• If ct is dominated by At , pt+1 = pt

• If ct dominates a member of At , delete it from At and include

ct in At and pt+1 = ct

• If |At | < N , include ct and pt+1 = winner(pt , ct )

• If |At | = N and ct does not lie in highest count hypercube H,

replace ct with a random solution from H and

pt+1 = winner(pt , ct ).

The winner is based on least number of solutions in the hypercube

40

Niching in PAES-(1+1)

f2

1 Offspring

3 Parent

2

Pareto−optimal f1

front

f2

Offspring

Parent

Pareto−optimal f1

front

41

Constrained Handling

Fm = fm + Rm Ω(~g ).

– Jimenez’s approach

– Ray-Tang-Seow’s approach

• Modified definition of domination

– Fonseca and Fleming’s approach

– Deb et al.’s approach

42

Constrain-Domination Principle

A solution i constrained-

dominates a solution j, if any is

true: 10

3

lution j is not. 1 2

6 5 3

2. Solutions i and j are both in- f2 4

6

4 5

feasible, but solution i has a Front 2

Front 1

smaller overall constraint vi- 2

olation. 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

lution j.

43

1.2 1.4

1

Constrained NSGA-II Simulation Results

f1

0

1.4

1.2

0.8

0.6

0.4

0.2

0

f2

44

1

0.9

0.8

0.7

0.6

f1

0.5

0.4

0.3

0.2

0.1

10

0

f2

Applications of MOEAs

• Engineering component design

• Microwave absorber design

• Ground-water monitoring

• Extruder screw design

• Airline scheduling

• VLSI circuit design

• Other applications (refer Deb, 2001 and EMO-01 proceedings)

45

Spacecraft Trajectory Optimization

• Three objectives for inter-planetary trajectory design

– Minimize time of flight

– Maximize payload delivered at destination

– Maximize heliocentric revolutions around the Sun

• NSGA invoked with SEPTOP software for evaluation

46

Earth–Mars Rendezvous

1000

22

36

900 73 72

Earth Earth

Mass Delivered to Target (kg.)

Mars

10.16.06 08.25.08

44

700

600

Individual 44 Individual 72

500

400

300

Mars

09.22.07

200

100

0

1 1.5 2 2.5 3 3.5 4 Earth Earth

Transfer Time (yrs.) 09.01.05 09.01.05

Mars

02.04.09

Individual 73 Individual 36

47

Salient Research Tasks

• Mathematically convergent algorithms with guaranteed spread

of solutions

• Test problem design

• Performance metrics and comparative studies

• Controlled elitism

• Developing practical MOEAs – Hybridization, parallelization

• Application case studies

49

Hybrid MOEAs

– Better convergence

– Faster approach

• Two hybrid approaches

– Local search to update each solution in an EA population

(Ishubuchi and Murata, 1998; Jaskiewicz, 1998)

– First EA and then apply a local search

97

Posteriori Approach in an MOEA

Multiple

MOEA local searches

Problem

Non−domination

Clustering check

98

Proposed Local Search Method

X

F = wi ∗ f i

i

• fi is scaled

• Weight wi chosen based on location of i in the obtained front

(fjmax − fj (x))/(fjmax − fjmin )

w̄j = PM

max − fk (x))/(fkmax − fkmin )

k=1 (fk

X

wi = 1

i

99

Fixed Weight Strategy

f2

• Extreme solutions are as-

max

signed extreme weights f2 a

A b

• Linear relation between MOEA solution set

min

• Many solution can converge f2

min max

search f1 f1

set after

local search

100

Design of a Cantilever Plate

18 18

16 16

NSGA−II Local search

Scaled deflection

Scaled deflection

14 14

12 12

10 10

8 8

6 6

4 4

2 2

20 25 30 35 40 45 50 55 60 20 25 30 35 40 45 50 55 60

Weight Weight

60 mm

P

18 18

16

100 mm

Scaled deflection

Scaled deflection

14 14

12 12

Base plate 10 10

8 8

6 6

4 4

2 2

20 25 30 35 40 45 50 55 60 20 25 30 35 40 45 50 55 60

Weight Weight

102

Trade-off Solutions

103

Conclusions

• Evolutionary algorithms are ideal candidates

• Many efficient algorithms exist, more efficient ones are needed

• With some salient research studies, MOEAs will revolutionize

the act of optimization

• EAs have a definite edge in multi-objective optimization and

should become more useful in practice in coming years

106

- Face Recognition using PCAUploaded byamit15094775
- Deb 2001 Multi-Objective Optimization Using Evolutionary AlgorithmsUploaded byAmir Moradi
- PPT- NSGA-IIUploaded byhema_iitbbs
- Optimization For Engineers By Kalyanmoy DebUploaded bylankyrck
- MULTI-OBJECTIVEUploaded bymrsingh_79
- A generic evolutionary algorithm for singleUploaded byMohammad Alauddin
- Face RecognitionUploaded byPalash Debnath
- Face RecognitionUploaded bychaithra580
- Evolutionary AlgorithmsUploaded byaksagar22
- DLC Evolutionary AlgorithmUploaded byAkash Raghu
- 2003 Face Recognition Using LDA-Based AlgorithmsUploaded byluongxuandan
- 092014TranHouHanNguenChauUploaded byNhan
- 1-s2.0-S1568494610000773-main RUTEO.pdfUploaded byCESARPINEDA
- PARETO FRONT APPROXIMATION WITH ADAPTIVE WEIGHTED SUM METHOD IN MULTIOBJECTIVE SIMULATION OPTIMIZATIONUploaded byAlberipa
- b. Linnhoff and s. AhmadUploaded byFarhan Hafeez
- Design Optimization and AutomationUploaded bybafulcher
- Multiperiod Optimization Model for Oilfield Production PlanningUploaded byJadiier Ariistiizabal
- 2017-Binary Grey Wolf Optimizer for Large Scale Unit Commitment ProblemUploaded byShahid Younas
- f4[1]Uploaded bydrsh81
- BusquedaUploaded byJose Fernandez
- Assembly Line BalancingUploaded byjabir68
- A23 Bhsakar_Rev_ChemE 2000.pdfUploaded bysagarsrinivas
- Multiobjective OptimizationUploaded byAmir Mosavi
- Decision Support for Build-To-OrderUploaded bydokk
- Stockpile and Cut-Off Optimization ModuleUploaded byBrandy Gonzalez
- 1. Formulation- LPPUploaded byriya pradhan
- FEM Beam OptimizationUploaded byZia Ullah
- 5772-1Uploaded bymps141066
- senouci2008.pdfUploaded byMatheus Souza
- Crushing Analysis and Numerical Optimization of Angle Element Structures Under Axial Impact LoadingUploaded byNhan

- Pli cacheté de WolfgaSome comments on the Pli cacheté n° 11.668, by W. Doeblin : Sur l'équation de Kolmogoroff B. Bru, M. Yor ng DoeblinUploaded byFranck Dernoncourt
- Alleged Plots Involving Foreign Leaders, U.S. Senate, Select Committee to Study Governmental Operations with Respect to Intelligence Activities, S. Rep. No. 755, 94th Cong., 2d sessUploaded byFranck Dernoncourt
- How does office temperature affect productivity?Uploaded byFranck Dernoncourt
- Who Writes Linux (2012)Uploaded byFranck Dernoncourt
- Interdisciplinary Research: Trend or TransitionUploaded byFranck Dernoncourt
- LATEX: Hyphens in Math ModeUploaded byFranck Dernoncourt
- m Turk MethodsUploaded byFranck Dernoncourt
- 2006 - An Empirical Comparison of Supervised Learning AlgorithmsUploaded byFranck Dernoncourt
- Emotiv - Research Edition SDK[1]Uploaded byFranck Dernoncourt
- Ferrucci-Watson2010 - Build Watson - An Overview of DeepQA ProjectUploaded byFranck Dernoncourt
- 1989 - On Genetic Crossover Operators for Relative Order PreservationUploaded byFranck Dernoncourt
- KB DNS Comparison GuideUploaded byFranck Dernoncourt
- 2000 - Content Planning and Generation in Continuous-speech Spoken Dialog SystemsUploaded byFranck Dernoncourt
- 2008 - T-Bot and Q-Bot - A Couple of AIML-Based Bots for Tutoring Courses and Evaluating StudentsUploaded byFranck Dernoncourt
- Dragon Speech Recognition Feature MatrixUploaded byFranck Dernoncourt
- ULDs Overview[1]Uploaded byFranck Dernoncourt
- 2010 - State of the Art - Logic-Based Approaches to Intention RecognitionUploaded byFranck Dernoncourt
- 2007 - Shawar - Are Chatbots Useful in EducationUploaded byFranck Dernoncourt
- 2009 - A Structure-free Aggregation Framework forVehicular Ad Hoc NetworksUploaded byFranck Dernoncourt
- 2012 - Ron Was Wrong, Whit is RightUploaded byFranck Dernoncourt
- Infografía de Google sobre la ley PIPAUploaded bycdperiodismo
- 2005 - Using Bigrams to Identify Relationships Between Student Certainness States and Tutor Responses in a Spoken Dialogue CorpusUploaded byFranck Dernoncourt
- 2007 - Intention Recognition Using a Graph RepresentationUploaded byFranck Dernoncourt
- 2007 - Intent Recognition for Human-Robot InteractionUploaded byFranck Dernoncourt
- 1990 - Intentions in Communication - Book ReviewUploaded byFranck Dernoncourt
- 2001 - Harnessing Models of Users' Goals to Mediate Clarification Dialog in Spoken Language SystemsUploaded byFranck Dernoncourt
- 2009 - Hybrid Approach to User Intention Modeling for Dialog SimulationUploaded byFranck Dernoncourt
- 2009 - TQ-Bot - An AIML-Based Tutor and Evaluator BotUploaded byFranck Dernoncourt
- 2009 - Zhang09 - E-Drama Facilitating Online Role-Play Using an AI Actor and Emotionally Expressive CharactersUploaded byFranck Dernoncourt

- Unregulated emissions and health risk potential from biodiesel (KB5, KB20) and methanol blend (M5) fuelled transportation diesel enginesUploaded byPuneet Verma
- Portable Pocket BookUploaded byedgar
- Failure of Soil Nail WallsUploaded byAndy Acoustic
- Design and Implementation of UPS - ProjectUploaded bysetsindia3735
- chouUploaded byyounghorse
- HAL Sample Paper (1)Uploaded byArun Vikraman
- CRG Logics Simplicity BlenderUploaded byTony Cresicbene
- TrgeorgisUploaded byloxodromen
- Attribute-Based Hierarchy Level StorageUploaded byIJARTET
- FC102 PID.pdfUploaded byKwameOpare
- 407.pdfUploaded byRA-04177
- OmniAccess-IAP92-93-105-134-135Uploaded byScribdUser250
- SlopUploaded byTerry Choi
- CondyleBookChapter3.pdfUploaded byShkelzen Komoni
- (de Gruyter Studies in Mathematics )Rene Schilling-Bernstein Functions Theory and Applications-Gruyter(2010)Uploaded byOlivera Taneska
- MATH_KELAS V_IIUploaded byRZm SayLvu
- PHASES of a FIRE, Classes of FireUploaded byhutsonianp
- Introduction to Computational Fluid DynamicsUploaded byJonyzhitop Tenorio
- IIT Kharagpur LecturesUploaded byBhavanarayana Chakka
- Allied Telesis VoIP Phone System PoEUploaded byKoudisseAmi
- Coupling of Particle Blast Method (PBM) With Discrete Element Method for Buried Mine Blast SimulationUploaded byAmir Iskandar
- Grignard ReactionUploaded bySulaiman Al Shidhani
- Us Dod Hemp TestingUploaded byprasobh
- Russ ElectricUploaded byTaner Ertürk
- IC-Lec-14.pptUploaded bygauravme1983
- Catia InterviewUploaded byashokpraveen939
- 35_shock Assembly FinalUploaded byadmam jones
- Consumer Driven Ecommerce - Literature Review, Design Framework and Research Agenda on Last Mile LosgisticsUploaded byAngela Simoes
- A Review of the Java.util PackageUploaded bylipika008
- Automatic Creation of PO in SAPUploaded byVishnu Kumar S