You are on page 1of 11

1

Multi-objective optimization and decision making:


Principles, Algorithms, and Case Studies
Michael T.M. Emmerich Assistant Professor LIACS, The Netherlands

Multiobjective Optimization, Autumn 2005: Michael Emmerich

LIACS Algorithms and Natural Computing Group


Theory and Practice of Algorithms for complex search and machine learning problems Theory research focusses on general problem classes, mathematical tools, and understanding of problems related to algorithmic solution of problems Applications of artificial intelligence in various fields of medicine/technology:
Leiden University Medical Department Leiden/Amsterdam Center for Drug Research Chemical Engineering Department Dortmund RWTH Aachen, Energy Systems Engineering NTU Athens, Aerospace Engineering AMOLF Amsterdam (Laser Physics) TU Eindhoven (Building Design) Automotive Industry (BMW, etc.) and many more

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Structure of the course


Part 1: Problems, Principles, Logic Lectures, Exercises Assignment 1 Part 2: Optimization Algorithms: Lectures, Exercises Assignment 2 Part 3: Case-studies: Presentations based on research paper in application field Seminar Example exercises (similar to the assignment) will be discussed in the class Assignments can be done in groups of two students Final grade:
(Grade Presentation + Grade Assignments) /2

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Multicriteria Optimization and Decision Making: An Interdisciplinary Research Field


1: STRUCTURAL SCIENCES (MATHEMATICS, LOGIC, ALGORITHMICS)

Mathematical/Structural Questions: Decision modelling: How to formalize decision problems? Order theory: Can we give general statements about the nature of decision problems and the geometry of solution sets? Exploration: How can we achieve an intuition 3: PEOPLE-CENTRIC QUESTIONS about the problem spaces and conflict types What are our goals? Are there by means of measures/visualizations? priorities and weights,uncertainties? Optimization: How can we efficiently obtain Interesting solutions with search algorithms? What modes of human-computer 2: APPLICATION FIELD interaction support goal Evaluation: How can we obtain goal formulation, decision processes measurements? appropriately? Uncertainty: What kind of uncertainties occur? How can groups make decisions? Constraints: What are physical/technological Constraints In a problem domain? History: What is the history of MCO methods in the field and what can we learn from it? What are the psychological aspects of decision making? How to achieve satisfactory results?

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Literature

Slides of the lecture:


www.liacs.nl/~emmerich Skript/Textbook: Work in progress

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Introduction
Views of the field, Examples, Fundamental Concepts, Overview

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Optimization ?
Optimization in a system theoretic framework Input System Output

! ! ?

? ! !
7

! ? !

Modeling, (Identification, Learning)

Simulation, (Prediction, Classification) Optimization, (Inverse Design, Calibration)

Multiobjective Optimization, Autumn 2005: Michael Emmerich

General optimization task

!
8

Multiobjective Optimization, Autumn 2005: Michael Emmerich

Constraints

!
9

Multiobjective Optimization, Autumn 2005: Michael Emmerich

10

Example: Tin problem


Minimize the area of surface A for a cylinder that contains V = 330ml sparkling juice!

x1

x2

Problem sketch

Multiobjective Optimization, Autumn 2005: Michael Emmerich

10

11

Example: Knapsack problem

Multiobjective Optimization, Autumn 2005: Michael Emmerich

11

12

Classification of optimisation problems

Multiobjective Optimization, Autumn 2005: Michael Emmerich

12

13

Hard-NLP

NLP problems can be hard:


1. Multimodal functions (many local optima) 2. Plateaus and discontinuities 3. High dimensional decision spaces 4. Complex restriction functions (disconnected feasible subspaces)

local optima

Multiobjective Optimization, Autumn 2005: Michael Emmerich

13

14

Difficulty of combinatorial optimization


1. The evaluation of the objective function is often quite efficient. 2. The search-space complexity can grow exponentially with number of inputs (e.g. conjectured for NP complete problems). 3. Heuristics might help to detect reasonable good or improved solution, but they often cannot tell You if there is a better solution in the huge search space.

Multiobjective Optimization, Autumn 2005: Michael Emmerich

14

15

Single objective optimization methods


A large number of optimization methods have been developed.

We will introduce some of them in the second part, Before generalizing them To the multiobjective case

Siarry, Collette, 2002

Multiobjective Optimization, Autumn 2005: Michael Emmerich

15

16

Summary: Single-Objective Optimization

Multiobjective Optimization, Autumn 2005: Michael Emmerich

16

17

Multi-objective optimization task

?
Multiobjective Optimization, Autumn 2005: Michael Emmerich

!
17

18

Decision conflicts: Car example


Your objectives: Cost min, Speed max

Cost

Speed Length Add constraint: Only red cars !


18

Multiobjective Optimization, Autumn 2005: Michael Emmerich

19

Example: Decision Conflicts in Elections


Mr. Motorhead, a passionate sports-car driver, wants to decide which party to vote for government. He is only interested in cars and money (that he needs to buy fast cars). His (sole) objectives: Gasoline Prices min, Tax min, Tempo-limit max, On all other topics, he is indifferent. Here are his decision alternatives:

Gasoline prices Black Party Green Party Red Party Exotic Party No Vote low high low high -

Taxes low low low high -

Tempo-limit high low low low -

Multiobjective Optimization, Autumn 2005: Michael Emmerich

19

20

Example: Power Plant Design


A team of engineers has to design a new power plant A finite number of configurations is possible The operating point (Pressure (p), Temperature (T)) has to be adjusted in an continuous range The energy consumption and the energy cost need to be minimized
Energy consumption min p Config2 Config1

P (Pressure) Config1

Config3 T(Temperature) Config4 Config2

Investment cost min


Multiobjective Optimization, Autumn 2005: Michael Emmerich 20

21

Example: Airfoil Shape Design

Multiobjective Optimization, Autumn 2005: Michael Emmerich

21

22

Example: Airfoil Shape Design


Lift

Drag

Multiobjective Optimization, Autumn 2005: Michael Emmerich

22

23

Example: Airfoil Shape Design


x2 Search space f2 Set of efficient solutions Image under f

x3

x1 Search space (decision space)

f1 Objective space (solution space)

Minimize deviation* to target profile 1. During the start f1(x) 2. During cruise f2(x) Decision variables are Bezier points of airfoil shape e.g. x1, x2, x3
Multiobjective Optimization, Autumn 2005: Michael Emmerich 23

*measured by some integral

24

Example: Laptop Choice


-2 1000 thinkfrog -3 dill 800 -3.8 -4 snellius 700 900 3.2 3.0 2.6 medium 2.5 sunny2.4 2.2 ubook low siemens high toshibar shark

-GHz

min

Price min

Weigh mint
24

Noise

min

Multiobjective Optimization, Autumn 2005: Michael Emmerich

25

Example: Multiobjective Knapsack

Multiobjective Optimization, Autumn 2005: Michael Emmerich

25

26

Fundamental Concepts: Domination


f2( min)
Indifference Zone Dominated Subspace

Reference Solution y

Dominating Subspace

Indifference Zone

f1 ( min)
Multiobjective Optimization, Autumn 2005: Michael Emmerich 26

27

Fundamental Concepts: Two spaces


map perfomed by f
x2 f2 Pareto front = Set of interesting solutions Image under f

x3 Search space (decision space)

x1 Objective space (solution space)

Multiobjective Optimization, Autumn 2005: Michael Emmerich

27

Fundamental Structure of MOO Problems: Sets, Metrics, Orders, Landscapes


Decision Space S
OPTIONAL equipped with Objective function(s) define

28

Objective Space F
REQUIRED equipped with

Neighborhood Structure
Measure of closeness between alternatives in the decision space (e.g. a metric)
EXPLOIT LOCALITY FOR SEARCH!?

Partial Order
Binary relation representing domination of elements by other elements based on preferences
MINIMAL ELEMENTS ?

Multiobjective Optimization, Autumn 2005: Michael Emmerich

28

29

Fundamental Concepts: Pareto optimality


f2( min)

Pareto front = Non-dominated subset

Non-dominated solutions
Multiobjective Optimization, Autumn 2005: Michael Emmerich 29

f1 ( min)

30

Thinking in terms of trade-off


e.g. f2 energy consumption Moving from x2 to x1 is a unbalanced tradeoff
Pareto front f2 Image under f

x1

x1 and x4 are ideal solutions


Trade-off curve (Pareto front)

Region of good compromise Solutions (knee points) x2 Moving from x1 to x2 is a fair trade-off. x3 x4
f1

e.g. price If objectives are equally important (scaled), unbalanced trade-offs are unfavorable moves, because while gaining a little in one objective you lose a lot in the other objective.
Multiobjective Optimization, Autumn 2005: Michael Emmerich 30

10

31

Interdisciplinary Approach

Algorithm Engineering (Computer Science)

Order and Geometry (Mathematics)

Various Application Domains (Economy, Engineering, Medicine etc)

Multiobjective Optimization, Autumn 2005: Michael Emmerich

31

32

Summary: Introduction

Multiobjective Optimization, Autumn 2005: Michael Emmerich

32

33 Multiobjective Optimization and Decision Making: Overview of the lectures

Introduction/Overview
Systems-Analysis Single-objective optimization Multiobjective optimization Domination and order Structure of Pareto Fronts and Efficient Sets Landscape Topologies in Pareto Optimization A-priori: Scalarization techniques Progressive: Interactive methods A-posteriori: Pareto optimization with metaheuristics Emprical vs. Experimental Analysis

Problems/Principles

Algorithms

Algorithm engineering

Decision Making and Decision Aid Application-oriented MOO


Introduction Case Studies

Multiobjective Optimization, Autumn 2005: Michael Emmerich

33

11