Dept.

of Mathematical Information Technology

Introduction to Multiobjective
Optimization
Jussi Hakanen
jussi.hakanen@jyu.fi
 Contents

Dept. of Mathematical Information Technology

Multiple Criteria Decision Making (MCDM)
Formulation of a multiobjective problem
On solving multiobjective problems
Basic concepts and definitions
Optimality in multiobjective optimization
What means multiobjective?

Dept. of Mathematical Information Technology

Consider several criteria simultaneously
Criteria are conflicting (e.g. usually good
quality is not chea) → all the criteria can not be
optimized simultaneously
Need for considering compromises between
the criteria
Compromise can be better than optimal
solution in practice (cf. optimize only
costs/profit)
http://en.wikipedia.org/wiki/Multiobjective_opti
mization
 Multiobjective decision making
process

Dept. of Mathematical Information Technology

Need for
optimization

Optimization problem
formulation
Modelling of the
problem (numerical
simulation)

Optimization &
decision making

Implementation &
testing of the best
solution found
Optimization problem formulation
By optimizing only one criterion, the rest

Dept. of Mathematical Information Technology

are not considered
Objective vs. constraint
Summation of the objectives
– adding apples and oranges
Converting the objectives (e.g. as costs)
– not easy, includes uncertaintes
Multiobjective formulation reveals
interdependences between the objectives
Example 1: Continuous casting
of steel

Dept. of Mathematical Information Technology

Optimal control of the secondary cooling of
continuous casting of steel
Long history of research in the Dept. of
Mathematical Information Technology,
Univ. of Jyväskylä
– modelling (1988)
– single objective optimization (1988-1994)
– multiobjective optimization (1994-1998)
Continuous casting of steel
Liquid steel enters

Dept. of Mathematical Information Technology

(tundish)
Initial cooling by a
water cooled mold →
thin crust
Movement supported
by rollers
Secondary cooling by
water sprays
Rest of the cooling by
radiation
 Continuous casting of steel
Measuring temperature in casting difficult →

Dept. of Mathematical Information Technology

numerical temperature distribution
Process modelled as a multiphase heat
equation (PDEs, solid & liquid phase) →
temperature distribution
Numerical model by using the finite element
method (FEM)
Dynamic process
 Continuous casting of steel
Secondary cooling significant: intensity of sprays

Dept. of Mathematical Information Technology

(easy to control) affects significantly to the
solidification of steel
Goal: minimize the amount of defects in steel
Quality depends on e.g. the temperature
distribution at the surface of steel
– too slow cooling → too long liquid part
– too fast cooling → defects appear
Objective function: keep the surface temperature
as close to a given profile as possible
Constraints e.g. for the change of temperature and
for the temperature in critical spots
 Continuous casting of steel
Analysis of single objective optimization

Dept. of Mathematical Information Technology

problem:
– constraints tight (based on metallurgy)
– no feasible solutions
– which constraints to relax?
Convert constraints as objective functions
(5 in total)
– enables simultaneous relaxation of different
constraints
– information on satisfaction of different
constraints and their interdependences
 Dept. of Mathematical Information Technology
Example 2: Water allocation
 Water allocation

Dept. of Mathematical Information Technology

Papermaking process consumes lots of water
(nowadays about 5-10 m3/ton of paper)
Water can be circulated and reused in different
parts of the process as long as it remains fresh
enough
– dissolved organic material accumulates
Fresh water costs
Process was modelles with the Balas process
simulator (http://balas.vtt.fi/)
How to formulate the optimization problem?
 Water allocation

Dept. of Mathematical Information Technology

Goal is to minimize the amount of fresh water
required for the process
Objective function: minimize the amount of
fresh water
Constraints
– the amount of dissolved organic material in
the white water of the papermachine
– the amount of dissolved organic material in
the pulp entering bleaching
Variables: 5 splitters ja 3 valves
 Water allocation
In practice

Dept. of Mathematical Information Technology

– set upper bounds for the amounts of
organic material
– minimize the amount of fresh water used
(one objective function)
How to set the upper bounds?
– based on engineering knowledge and
current technology
– what if the bounds could be relaxed a bit?
Multiobjective formulation where the
constraints would also be objectives (3)
 Example 3: Chemical separation
process
Consider a chemical separation process based

Dept. of Mathematical Information Technology

on chromatography
Applied to many important separationsin sugar,
petrochemical,and pharmaceutical industries
Utilizes the difference
in the migration speeds
of different chemical
components in liquid

* http://www.pharmaceutical-technology.com
 Adapted from Y. Kawajiri, Carnegie Mellon University

Chromatography (Single Column)

Dept. of Mathematical Information Technology

Desorbent Feed (Mixture of
two components)

2.
4. Elution
3.
1. Feed
Recover st product
1nd
Initial state
5. 2 product
Column is filled with desorbent

Pump
Chromatographic Column
(Vessel packed with adsorbent particles)
 Adapted from Y. Kawajiri, Carnegie Mellon University

Process simulation
Step

Dept. of Mathematical Information Technology

Cycle
Feed Desorbent Feed 108712
11
9616
13
14
1554321
Desorbent Feed Desorbent
Feed Desorbent Feed Desorbent
Feed Desorbent Feed Desorbent
Feed Desorbent

Liquid Flow
LiquidFlow
Liquid Flow

Extract
Extract Extract
Raffinate
Raffinate Extract
Raffinate ExtractRaffinate
Raffinate
Raffinate
RaffinateExtract Raffinate
Raffinate
Extract Extract
November 11, 2009 Bergische Universität Wuppertal
 Chemical separation process

Dept. of Mathematical Information Technology

Two inlet and two
outlet streams are
switched in the
direction of the
liquid flow at a
regular interval
(steptime)
Operating variables
– Switching interval
(Step Time)
– Liquid velocities
Chemical separation process

Dept. of Mathematical Information Technology

Typically a profit function is optimized
Formulation of a profit is not easy
Multiobjective formulation
– max throughput
– min desorbent consumption
– max purity of the product
– max recovery of the product
Enables more flexible consideration and
reveals how different objectives affect the
solution
 Multiobjective optimization
(MOO) problem

Dept. of Mathematical Information Technology

Multiple objective functions, number denoted
by k ( k > 1)
– special case: two objectives
– Objective vectors can be visualized when
k = 2, 3
Variables: values change the solution
Constraints: same as in single objective
problems
Feasible region S: consists of all the points
satisfying the constraints
 Mathematical formulation

Dept. of Mathematical Information Technology

• Vector valued objective function

• Objective vector

• Image of the feasible region

 Optimality: objective space

Dept. of Mathematical Information Technology

f2
min
Z = f(S)

Best values are located down and left f1

min
 Optimality for multiple
objectives

Dept. of Mathematical Information Technology

When objectives to be optimized are
conflicting → no single optimal solution
– cf. single objective optimization
Compromise
There are potentially infinitely many optimal
solutions
 Optimality

Dept. of Mathematical Information Technology

Which solutions can be ’’optimal’’?
How to find such solutions?
 Optimality: objective space

Dept. of Mathematical Information Technology

f2
min
Z = f(S)

Best values are located down and left f1

min
 Pareto optimality (PO)

Dept. of Mathematical Information Technology

Mathematical definition:

In other words: a vector is PO if no objective can

be improved without impairing some other one
Note: PO solutions can not be compared
mathematically without some additional
information
 Pareto optimal set
Also known as the Pareto front

Dept. of Mathematical Information Technology

Consists of all the PO solutions
Usually presented in the objective space
– e.g. k=2 → PO set is a subset of two
dimensional space
f2 f2 f2

discrete f1 linear f1 nonlinear f1

What means solving a problem?

Dept. of Mathematical Information Technology

Find all PO solutions
– theoretical approach, not feasible in practice
Find an approximation for PO front
– approximation with good diversity
(representatives in all parts of the PO set)
and good spread (no similar solutions)
– can also be an approximation of some
specific part of the front
Find a best compromise (PO solution)
– requires preferences from a DM
 How to choose ’’a best’’
compromise?

Dept. of Mathematical Information Technology

PO solutions can not be compared
mathematically without some additional
information
– cf. ordering vectors in a plane
There typically exist infinitely many PO
solutions for continuous problems
Additional information related to the
problem considered is needed
 Decision maker (DM)

Dept. of Mathematical Information Technology

Person(s) who is an expert in the
application area
Is able to express preferences related to
objectives
– e.g. is able to compare Pareto optimal
solutions
No need for expertize in optimization
Helps in finding a best compromise
 Weak Pareto optimality

Dept. of Mathematical Information Technology

Some objective can be f2
improved without worsening
others
PO solution is also weakly PO
PO solutions are better but more Weakly PO solutions

difficult to compute than weakly f1

PO ones
 Ranges for the PO set

Dept. of Mathematical Information Technology

Ranges for the objective function values in the
PO set provide information about achievable
solutions
Ideal objective vector (best values)
– how good values can be obtained
Nadir objective vector (worst values)
– how bad values possibly have to be
accepted
Usefull information in decision making
Are utilized also in some MOO methods
 Ideal objective vector

Dept. of Mathematical Information Technology

Consists of best values for each objective
when optimized independently
– smallest values for minimization
– k objective functions → optimization of k
single objective problems
Ideal objective vector is not feasible!
 Ideal objective vector

Dept. of Mathematical Information Technology

f2
min
Z = f(S)

z*
z2*

z1* f1
min
 Nadir objective vector

Dept. of Mathematical Information Technology

Consists of worst values for each objective
in the PO set
– largest values for minimization
Generally difficult to compute, need for
approximation
– Note: when k=2 → easier to compute
– e.g. by using a pay-off table
 Pay-off table

Dept. of Mathematical Information Technology

Is obtained by evaluating all the objective
functions in the points where ideal values were
obtained
– i:th row the values of objectives in a point
where fi has it’s optimal value
– diagonal has z*
– zinad = the worst value of the i:th column
Can give either optimistic or pessimistic
approximation for the Nadir objective vector
(depends on the problem considered)
 Nadir objective vector

Dept. of Mathematical Information Technology

f2
min
Z = f(S)
z2nad znad

z2*

z1* z1nad f1
min
 Reference point

Dept. of Mathematical Information Technology

Reference point = a vector in the objective
space that contains desirable values for the
objectives
The components of a reference point are
called aspiration levels
One way for the DM to express preferences
(intuitive)
Utilized also in some MOO methods
 Special case: 2 objectives

Dept. of Mathematical Information Technology

PO set can be visualized (if available)
If one of the objectives is
improved, the other one
f2
will impair min

A desired solution can

be chosen from the
”curve”
Very common in
practical applications
f1
min
 Scalarizing the problem
Often the idea of MOO methods is to some

Dept. of Mathematical Information Technology

way convert the problem into single
objective one
– methods of single objective optimization
can be utilized
This is called scalarization
Can be done in a good way or in a bad
way
– examples of scalarization will come in
later lectures
 Properties of a good MOO
method

Dept. of Mathematical Information Technology

Methods based on scalarization produce
usually one solution at a time
Good method should have the following
properties
– produce (weakly) PO solutions
– is able to find any (weakly) PO solution
(by using suitable parameters of the
method)
 Approaches
Plenty of methods developed for MOO

Dept. of Mathematical Information Technology

MOO methods can de categorized based
on the role of the DM
– No-preference methods (no DM)
– Aposteriori methods
– Apriori methods
– Interactive methods
The approaches are discussed more in
the coming lectures – just an appetizer!
 Examples of MOO literature
V. Changkong & Y. Haimes, Multiobjective

Dept. of Mathematical Information Technology

Decision Making: Theory and Methodology,
1983
Y. Sawaragi, H. Nakayama & T. Tanino,
Theory of Multiobjective Optimization, 1985
R.E. Steuer, Multiple Criteria Optimization:
Theory, Computation and Applications, 1986
K. Miettinen, Nonlinear Multiobjective
Optimization, 1999
K. Deb, Multi-Objective Optimization Using
Evolutionary Algorithms, 2001
 Examples of MOO literature

Dept. of Mathematical Information Technology

M. Ehrgott, Multicriteria Optimization, 2005
J. Branke, K. Deb, K. Miettinen & R.
Slowinski (eds): Multiobjective Optimization:
Interactive and Evolutionary Approaches,
2008
G.P. Rangaiah (editor), Multi-Objective
Optimization: Techniques and Applications
in Chemical Engineering, 2009
E. Talbi, Metaheuristics: from Design to
Implementation, 2009
 Dept. of Mathematical Information Technology
http://www.mcdmsociety.org/
Newsletter
22nd International Conference on Multiple
Criteria Decision Making
17-21 June 2013, Málaga (Spain)
11th MCDA/M Summer School 2013
Helmut-Schmidt-Universität, Hamburg,
Germany, July 22nd– August 2nd, 2013
Membership does not cost you anything!

January 23-27, 2012 Dagstuhl Seminar on Learning in

Multiobjective Optimization