You are on page 1of 39

Chapter 8 – Introduction to

Metaheuristic Optimization
Dr. Mohamad Hafis Izran Bin Ishak
Department of Control & Mechatronics (CMED)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia.

SKEM4173 – Artificial Intelligence


www.utm.my
Dr. Hafis innovative innovative• • global
SKEM4173 • entrepreneurial entrepreneurial • global

What are we going to learn?

• Purpose
– The course will give an overview of, and
introduction to, modern heuristic optimization
methods that are suitable for solving practical
optimization problems.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 2

1
Formulation of
Abstract Problem

Solve the
Abstract
Problem
Real-World Problem

Implement the Solution Interpret the Solution

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 3

Motivation: Operations
Research (OR)
• WW-II British military
• Scientists and engineers from several fields
• Analysis and Decision Support
– Placement of Radar Stations
– Guidance of convoys
– Bombing raids
– Anti-submarine campaigns
– Laying of mines
– Operational Analysis/Operations Research

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 4

2
Today: OR/Management
Science
• Scientific approach to decision making
regarding deciding on how a system is to be
designed and operated, usually under
conditions with limited resources
– (Winston: Operations Research Applications and
Algorithms)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 5

Operations Research
• Quantitative methods for decision support
• Several disciplines
– Mathematical modeling
– Optimization
– Probability theory
– Game theory
– Queuing theory
– Simulation
• Discrete optimization problems (DOP) are central

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 6

3
Example: Choice of Projects

• You are leading a big company


• Your employees has suggested a number of projects to
do, each with a known:
– profit
– cost
• You have a fixed budget
• Which projects should you select to maximize the profit?
• Strategic / tactical decision
– Optimization problem

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 7

Example:Vehicle Routing
• You are running a transportation company with your own
car
• You have a set of pickup and delivery jobs around town to
do tomorrow
• What sequence of stops should you choose to finish the
jobs as early as possible?

• Operative decision
– Optimization problem
– The Travelling Salesman Problem (TSP)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 8

4
Exampel Optimization Problem:
TSP – Travelling Salesman Problem
1 2 3 4 5 6 7
0 18 17 23 23 23 23
1
2 0 88 23 8 17 32
2 3 2
17 33 0 23 7 43 23
3
1 4 4 33 73 4 0 9 23 19
5 9 65 6 65 0 54 23
5
7 25 99 2 15 23 0 13
6
6 83 40 23 43 77 23 0
7
Feasible Solution: 1 2 7 3 4 5 6 1 with value: 184

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 9

Heuristic Algorithms
• Heuristic algorithms do not guarantee to
find the optimal solution, however:
• Heuristic algorithms do not even
necessarily have a bound on how bad they
can perform
– That is, they can return a solution that is
arbitrarily bad compared to the optimal
solution
• However, in practice, heuristic algorithms
(heuristics for short) have proven successful
• Most of this course will focus on this type
of heuristic solution method
Dr. Hafis SKEM4173 innovative • entrepreneurial • global 10

5
What is a heuristic?

• From greek heuriskein (meaning ”to find”)


• Wikipedia says:
– (…)A heuristic is a technique designed to solve a
problem that ignores whether the solution can be
proven to be correct, but which usually produces
a good solution (…).
– Heuristics are intended to gain computational
performance or conceptual simplicity, potentially
at the cost of accuracy or precision.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 11

Why don’t we always use exact


methods?
• If a heuristic does not guarantee a good solution, why
not use an (exact) algorithm that does?
• The running time of the algorithm
– For reasons explained soon, the running time of an
algorithm may render it useless on the problem you want to
solve
• The link between the real-world problem and the
formal problem is weak
– Sometimes you cannot properly formulate a COP/IP that
captures all aspects of the real-world problem
– If the problem you solve is not the right problem, it might
be just as useful to have one (or more) heuristic solutions,
rather than the optimal solution of the formal problem

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 12

6
Chapter Outline

8.1 Genetic Algorithm

8.2 Particle Swarm Optimization

8.3 Applications and case studies

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

8.1
Genetic Algorithm

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 14

7
The Genetic Algorithm
• Directed search algorithms based on the
mechanics of biological evolution
• Developed by John Holland, University of
Michigan (1970’s)
– To understand the adaptive processes of natural
systems
– To design artificial systems software that retains
the robustness of natural systems

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

The Genetic Algorithm (cont.)


• Provide efficient, effective techniques for
optimization and machine learning
applications
• Widely-used today in business, scientific and
engineering circles

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

8
Classes of Search Techniques

Search techniques

Calculus-based techniques Guided random search techniques Enumerative techniques

Direct methods Indirect methods Evolutionary algorithms Simulated annealing Dynamic programming

Finonacci Newton Evolutionary strategies Genetic algorithms

Parallel Sequential

Centralized Distributed Steady-state Generational

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Components of a GA

A problem to solve, and ...


• Encoding technique (gene, chromosome)
• Initialization procedure (creation)
• Evaluation function (environment)
• Selection of parents (reproduction)
• Genetic operators (mutation, recombination)
• Parameter settings (practice and art)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

9
Simple Genetic Algorithm

{
initialize population;
evaluate population;
while TerminationCriteriaNotSatisfied
{
select parents for reproduction;
perform recombination and mutation;
evaluate population;
}
}

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

The GA Cycle of Reproduction

children
reproduction modification
modified
parents children
population evaluation
evaluated children
deleted
members

discard

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

10
Population

population
Chromosomes could be:
– Bit strings (0101 ... 1100)
– Real numbers (43.2 -33.1 ... 0.0 89.2)
– Permutations of element (E11 E3 E7 ... E1 E15)
– Lists of rules (R1 R2 R3 ... R22 R23)
– Program elements (genetic programming)
– ... any data structure ...

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Reproduction
children
reproduction

parents

population

Parents are selected at random with selection chances biased in relation to


chromosome evaluations.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

11
Chromosome Modification

children
modification
modified children

• Modifications are stochastically triggered


• Operator types are:
– Mutation
– Crossover (recombination)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Mutation: Local Modification


Before: (1 0 1 1 0 1 1 0)
After: (0 1 1 0 0 1 1 0)

Before: (1.38 -69.4 326.44


0.1)
After: (1.38 -67.5 326.44
0.1)
• Causes movement in the search space
(local or global)
• Restores lost information to the population
Dr. Hafis SKEM4173 innovative • entrepreneurial • global

12
Crossover: Recombination

*
P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1

P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2

Crossover is a critical feature of genetic


algorithms:
– It greatly accelerates search early in evolution of a
population
– It leads to effective combination of schemata
(subsolutions on different chromosomes)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Evaluation

modified
evaluated children
children
evaluation

• The evaluator decodes a chromosome and assigns it


a fitness measure
• The evaluator is the only link between a classical GA
and the problem it is solving

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

13
Deletion

population
discarded members

discard

• Generational GA:
entire populations replaced with each iteration
• Steady-state GA:
a few members replaced each generation

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

An Abstract Example

Distribution of Individuals in Generation 0

Distribution of Individuals in Generation N

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

14
A Simple Example

“The Gene is by far the most sophisticated program around.”


- Bill Gates, Business Week, June 27, 1994

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

A Simple Example

The Traveling Salesman Problem:

Find a tour of a given set of cities so that


– each city is visited only once
– the total distance traveled is minimized

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

15
Representation

Representation is an ordered list of city


numbers known as an order-based GA.
1) London 3) Dunedin 5) Beijing 7) Tokyo
2) Venice 4) Singapore 6) Phoenix 8) Victoria

CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Crossover

Crossover combines inversion and


recombination:
* *
Parent1 (3 5 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4)

Child (5 8 7 2 1 6 3 4)

This operator is called the Order1 crossover.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

16
Mutation

Mutation involves reordering of the list:

* *
Before: (5 8 7 2 1 6 3 4)

After: (5 8 6 2 1 7 3 4)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

TSP Example: 30 Cities

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

17
Solution i (Distance = 941)

TSP30 (Performance = 941)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Solution j(Distance = 800)


44
62 TSP30 (Performance = 800)
69
67 120
78
64 100
62
54
80
42
50
40 y 60
40
38 40
21
35
67 20

60
60 0
40 0 10 20 30 40 50 60 70 80 90 100
42 x
50
99

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

18
Solution k(Distance = 652)

TSP30 (Performance = 652)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Best Solution (Distance = 420)


42
38 TSP30 Solution (Performance = 420)
35
120
26
21
35 100
32
7
80
38
46
44 y 60
58
60 40
69
76
20
78
71
69 0
67 0 10 20 30 40 50 60 70 80 90 100
62 x
84
94

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

19
Overview of Performance

T S P 3 0 - O ve r v i e w o f P e rfo rm a n c e

1800

1600

1400

1200
e
c 1000
n
ta
s
i 800
D
600

400

200

0 B es t
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
W o rs t
G e n e r a tio ns ( 10 0 0 )
A ve ra ge

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Considering the GA Technology


“Almost eight years ago ...
people at Microsoft wrote
a program [that] uses some
genetic things for finding
short code sequences.
Windows 2.0 and 3.2, NT,
and almost all Microsoft
applications products have
shipped with pieces of
code created by that
system.”
- Nathan Myhrvold, Microsoft Advanced
Technology Group, Wired, September 1995

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

20
Issues for GA Practitioners

• Choosing basic implementation issues:


– representation
– population size, mutation rate, ...
– selection, deletion policies
– crossover, mutation operators
• Termination Criteria
• Performance, scalability
• Solution is only as good as the evaluation function
(often hardest part)

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Benefits of Genetic Algorithms


• Concept is easy to understand
• Modular, separate from application
• Supports multi-objective optimization
• Good for “noisy” environments
• Always an answer; answer gets better with
time
• Inherently parallel; easily distributed

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

21
Benefits of Genetic Algorithms (cont.)
• Many ways to speed up and improve a GA-
based application as knowledge about
problem domain is gained
• Easy to exploit previous or alternate solutions
• Flexible building blocks for hybrid applications
• Substantial history and range of use

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

When to Use a GA
• Alternate solutions are too slow or overly complicated
• Need an exploratory tool to examine new approaches
• Problem is similar to one that has already been
successfully solved by using a GA
• Want to hybridize with an existing solution
• Benefits of the GA technology meet key problem
requirements

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

22
Some GA Application Types
Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboard


configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classification


algorithms, classifier systems
Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

Combinatorial set covering, travelling salesman, routing, bin packing,


graph colouring and partitioning
Optimization

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Conclusions

Question: ‘If GAs are so smart, why ain’t they rich?’


Answer: ‘Genetic algorithms are rich - rich in
application across a large and growing
number of disciplines.’
- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

23
8.2
Particle Swarm
Optimization

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 47

Introduction
• Concept first introduced by Kennedy and Eberhart (1995)
• Original idea was to develop a realistic visual simulation of
bird flock behaviour
• Simulation was then modified to include a point in the
environment that attracted the virtual bird agents
• Potential for optimisation applications then became apparent
• Combines self-experiences with social experiences

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

24
The natural metaphor
• A flock of birds (or school of fish, etc.) searching for food
• Their objective is to efficiently find the best source of food
• Nature-based theory underlying PSO:
The advantage of sharing information within a
group outweighs the disadvantage of having
to share the reward

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Introduction to the PSO: Origins

• Inspired from the nature social behavior and dynamic


movements with communications of insects, birds and fish

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

25
Introduction to the PSO: Origins
• In 1986, Craig Reynolds described this process in 3 simple
behaviors:

Separation Alignment Cohesion


avoid crowding local move towards the average move toward the average
flockmates heading of local position of local
flockmates flockmates

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Introduction to the PSO: Concept

 Uses a number of agents (particles)


that constitute a swarm moving
around in the search space looking for
the best solution

 Each particle in search space adjusts


its “flying” according to its own flying
experience as well as the flying
experience of other particles

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

26
Introduction to the PSO: Concept
• Collection of flying particles (swarm) - Changing solutions

• Search area - Possible solutions

• Movement towards a promising area to get the global


optimum

• Each particle keeps track:


– its best solution, personal best, pbest

– the best value of any particle, global best, gbest

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Introduction to the PSO: Concept


 Each particle modifies its
position according to:
• its current position
• its current velocity

• the distance between its


current position and pbest

• the distance between its


current position and gbest

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

27
Terminology

• The “particles” in PSO have no mass or volume (essentially


they are just points in space), but they do have acceleration
and velocity
• Behaviour of groups in the developed algorithm ended up
looking more like a swarm than a flock
• Hence the name Particle Swarm Optimisation

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Swarm intelligence
• Millonas’ five basic principles of swarm intelligence:
 Proximity: agents can perform basic space and time calculations
 Quality: agents can respond to environmental conditions
 Diverse response: population can exhibit a wide range of behaviour
 Stability: behaviour of population does not necessarily change every
time the environment does
 Adaptability: behaviour of population must be able to change when
necessary to adapt to environment

• A PSO swarm satisfies all of the above conditions

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

28
Cooperation example

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

The basic idea

• Each particle is searching for the optimum


• Each particle is moving and hence has a velocity.
• Each particle remembers the position it was in where it had its
best result so far (its personal best)
• But this would not be much good on its own; particles need
help in figuring out where to search.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

29
The basic idea II

• The particles in the swarm co-operate. They exchange


information about what they’ve discovered in the places
they have visited
• The co-operation is very simple. In basic PSO it is like this:
– A particle has a neighbourhood associated with it.
– A particle knows the fitnesses of those in its neighbourhood, and
uses the position of the one with best fitness.
– This position is simply used to adjust the particle’s velocity

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Initialization. Positions
and velocities

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

30
What a particle does

• In each timestep, a particle has to move to a new position. It


does this by adjusting its velocity.
– The adjustment is essentially this:
– The current velocity PLUS
– A weighted random portion in the direction of its personal
best PLUS
– A weighted random portion in the direction of the
neighbourhood best.
• Having worked out a new velocity, its position is simply its old
position plus the new velocity.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Neighbourhoods

geographical
social

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

31
Neighbourhoods

Global

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

The circular neighbourhood

1
Particle 1’s 3-
2
neighbourhood 8

7 3

Virtual circle 4
6
5

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

32
Particles adjust their positions according to a
‘Psychosocial compromise’ between what an
individual is comfortable with, and what society
reckons
My best
perf.
pi
Here I The best
am! x pg perf. of my
neighbours

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Pseudocode
http://www.swarmintelligence.org/tutorials.
php

Equation (a)
v[] = c0 *v[]
+ c1 * rand() * (pbest[] - present[])
+ c2 * rand() * (gbest[] - present[])
(in the original method, c0=1, but many
researchers now play with this parameter)

Equation (b)
present[] = present[] + v[]

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

33
Pseudocode
http://www.swarmintelligence.org/tutorials.
php
For each particle
Initialize particle
END

Do
For each particle
Calculate fitness value
If the fitness value is better than its peronal best
set current value as the new pBest
End

Choose the particle with the best fitness value of all as gBest
For each particle
Calculate particle velocity according equation (a)
Update particle position according equation (b)
End
While maximum iterations or minimum error criteria is not attained

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Pseudocode
http://www.swarmintelligence.org/tutorials.
php
Particles' velocities on each dimension are
clamped to a maximum velocity Vmax. If
the sum of accelerations would cause the
velocity on that dimension to exceed
Vmax, which is a parameter specified by
the user. Then the velocity on that
dimension is limited to Vmax.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

34
Parameters

• Number of particles (swarmsize)


• C1 (importance of personal best)
• C2 (importance of neighbourhood best)

• Vmax: limit on velocity

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

How to choose parameters

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

35
Parameters

• Number of particles
(10—50) are reported as usually sufficient.
• C1 (importance of personal best)
• C2 (importance of neighbourhood best)
• Usually C1+C2 = 4. No good reason other than
empiricism
• Vmax – too low, too slow; too high, too unstable.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Some functions often used for


testing real-valued
optimisation algorithms
Griewank Rastrigin

Rosenbrock

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

36
... and some typical results

Optimum=0, dimension=30

Best result after 40 000 evaluations


30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [±300] 0.003944 0.4033

Rastrigin [±5] 82.95618 46.4689

Rosenbrock [±10] 50.193877 1610.359

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

This is from Poli, R. (2008). "Analysis of the publications on the


applications of particle swarm optimisation". Journal of Artificial
Evolution and Applications 2008: 1–10.

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

37
Adaptive swarm size

There has been enough I try to kill myself


improvement
although I'm the worst

I try to generate a
I'm the best new particle
but there has been not enough
improvement

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Adaptive coefficients

rand(0…b)(p-x)
av
The better I The better is my best
am, the more I neighbour, the more
follow my own I tend to go towards
way him

Dr. Hafis SKEM4173 innovative • entrepreneurial • global

38
End of Syllabus
Good luck

Dr. Hafis SKEM4173 innovative • entrepreneurial • global 77

39