You are on page 1of 39

# Chapter 8 – Introduction to

Metaheuristic Optimization
Dr. Mohamad Hafis Izran Bin Ishak
Department of Control & Mechatronics (CMED)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia.

## SKEM4173 – Artificial Intelligence

www.utm.my
Dr. Hafis innovative innovative• • global
SKEM4173 • entrepreneurial entrepreneurial • global

## What are we going to learn?

• Purpose
– The course will give an overview of, and
introduction to, modern heuristic optimization
methods that are suitable for solving practical
optimization problems.

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 2

1
Formulation of
Abstract Problem

Solve the
Abstract
Problem
Real-World Problem

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 3

Motivation: Operations
Research (OR)
• WW-II British military
• Scientists and engineers from several fields
• Analysis and Decision Support
– Guidance of convoys
– Bombing raids
– Anti-submarine campaigns
– Laying of mines
– Operational Analysis/Operations Research

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 4

2
Today: OR/Management
Science
• Scientific approach to decision making
regarding deciding on how a system is to be
designed and operated, usually under
conditions with limited resources
– (Winston: Operations Research Applications and
Algorithms)

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 5

Operations Research
• Quantitative methods for decision support
• Several disciplines
– Mathematical modeling
– Optimization
– Probability theory
– Game theory
– Queuing theory
– Simulation
• Discrete optimization problems (DOP) are central

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 6

3
Example: Choice of Projects

## • You are leading a big company

• Your employees has suggested a number of projects to
do, each with a known:
– profit
– cost
• You have a fixed budget
• Which projects should you select to maximize the profit?
• Strategic / tactical decision
– Optimization problem

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 7

Example:Vehicle Routing
• You are running a transportation company with your own
car
• You have a set of pickup and delivery jobs around town to
do tomorrow
• What sequence of stops should you choose to finish the
jobs as early as possible?

• Operative decision
– Optimization problem
– The Travelling Salesman Problem (TSP)

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 8

4
Exampel Optimization Problem:
TSP – Travelling Salesman Problem
1 2 3 4 5 6 7
0 18 17 23 23 23 23
1
2 0 88 23 8 17 32
2 3 2
17 33 0 23 7 43 23
3
1 4 4 33 73 4 0 9 23 19
5 9 65 6 65 0 54 23
5
7 25 99 2 15 23 0 13
6
6 83 40 23 43 77 23 0
7
Feasible Solution: 1 2 7 3 4 5 6 1 with value: 184

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 9

Heuristic Algorithms
• Heuristic algorithms do not guarantee to
find the optimal solution, however:
• Heuristic algorithms do not even
necessarily have a bound on how bad they
can perform
– That is, they can return a solution that is
arbitrarily bad compared to the optimal
solution
• However, in practice, heuristic algorithms
(heuristics for short) have proven successful
• Most of this course will focus on this type
of heuristic solution method
Dr. Hafis SKEM4173 innovative • entrepreneurial • global 10

5
What is a heuristic?

## • From greek heuriskein (meaning ”to find”)

• Wikipedia says:
– (…)A heuristic is a technique designed to solve a
problem that ignores whether the solution can be
proven to be correct, but which usually produces
a good solution (…).
– Heuristics are intended to gain computational
performance or conceptual simplicity, potentially
at the cost of accuracy or precision.

## Why don’t we always use exact

methods?
• If a heuristic does not guarantee a good solution, why
not use an (exact) algorithm that does?
• The running time of the algorithm
– For reasons explained soon, the running time of an
algorithm may render it useless on the problem you want to
solve
• The link between the real-world problem and the
formal problem is weak
– Sometimes you cannot properly formulate a COP/IP that
captures all aspects of the real-world problem
– If the problem you solve is not the right problem, it might
be just as useful to have one (or more) heuristic solutions,
rather than the optimal solution of the formal problem

6
Chapter Outline

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

8.1
Genetic Algorithm

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 14

7
The Genetic Algorithm
• Directed search algorithms based on the
mechanics of biological evolution
• Developed by John Holland, University of
Michigan (1970’s)
– To understand the adaptive processes of natural
systems
– To design artificial systems software that retains
the robustness of natural systems

## The Genetic Algorithm (cont.)

• Provide efficient, effective techniques for
optimization and machine learning
applications
• Widely-used today in business, scientific and
engineering circles

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

8
Classes of Search Techniques

Search techniques

## Calculus-based techniques Guided random search techniques Enumerative techniques

Direct methods Indirect methods Evolutionary algorithms Simulated annealing Dynamic programming

## Finonacci Newton Evolutionary strategies Genetic algorithms

Parallel Sequential

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Components of a GA

## A problem to solve, and ...

• Encoding technique (gene, chromosome)
• Initialization procedure (creation)
• Evaluation function (environment)
• Selection of parents (reproduction)
• Genetic operators (mutation, recombination)
• Parameter settings (practice and art)

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

9
Simple Genetic Algorithm

{
initialize population;
evaluate population;
while TerminationCriteriaNotSatisfied
{
select parents for reproduction;
perform recombination and mutation;
evaluate population;
}
}

## The GA Cycle of Reproduction

children
reproduction modification
modified
parents children
population evaluation
evaluated children
deleted
members

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

10
Population

population
Chromosomes could be:
– Bit strings (0101 ... 1100)
– Real numbers (43.2 -33.1 ... 0.0 89.2)
– Permutations of element (E11 E3 E7 ... E1 E15)
– Lists of rules (R1 R2 R3 ... R22 R23)
– Program elements (genetic programming)
– ... any data structure ...

Reproduction
children
reproduction

parents

population

## Parents are selected at random with selection chances biased in relation to

chromosome evaluations.

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

11
Chromosome Modification

children
modification
modified children

## • Modifications are stochastically triggered

• Operator types are:
– Mutation
– Crossover (recombination)

## Mutation: Local Modification

Before: (1 0 1 1 0 1 1 0)
After: (0 1 1 0 0 1 1 0)

## Before: (1.38 -69.4 326.44

0.1)
After: (1.38 -67.5 326.44
0.1)
• Causes movement in the search space
(local or global)
• Restores lost information to the population
Dr. Hafis SKEM4173 innovative • entrepreneurial • global

12
Crossover: Recombination

*
P1 (0 1 1 0 1 0 0 0) (0 1 0 0 1 0 0 0) C1

P2 (1 1 0 1 1 0 1 0) (1 1 1 1 1 0 1 0) C2

## Crossover is a critical feature of genetic

algorithms:
– It greatly accelerates search early in evolution of a
population
– It leads to effective combination of schemata
(subsolutions on different chromosomes)

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Evaluation

modified
evaluated children
children
evaluation

## • The evaluator decodes a chromosome and assigns it

a fitness measure
• The evaluator is the only link between a classical GA
and the problem it is solving

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

13
Deletion

population

• Generational GA:
entire populations replaced with each iteration
a few members replaced each generation

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

An Abstract Example

14
A Simple Example

## “The Gene is by far the most sophisticated program around.”

- Bill Gates, Business Week, June 27, 1994

A Simple Example

## Find a tour of a given set of cities so that

– each city is visited only once
– the total distance traveled is minimized

15
Representation

## Representation is an ordered list of city

numbers known as an order-based GA.
1) London 3) Dunedin 5) Beijing 7) Tokyo
2) Venice 4) Singapore 6) Phoenix 8) Victoria

CityList1 (3 5 7 2 1 6 4 8)
CityList2 (2 5 7 6 8 1 3 4)

Crossover

## Crossover combines inversion and

recombination:
* *
Parent1 (3 5 7 2 1 6 4 8)
Parent2 (2 5 7 6 8 1 3 4)

Child (5 8 7 2 1 6 3 4)

16
Mutation

## Mutation involves reordering of the list:

* *
Before: (5 8 7 2 1 6 3 4)

After: (5 8 6 2 1 7 3 4)

## TSP Example: 30 Cities

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

17
Solution i (Distance = 941)

## TSP30 (Performance = 941)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

## Solution j(Distance = 800)

44
62 TSP30 (Performance = 800)
69
67 120
78
64 100
62
54
80
42
50
40 y 60
40
38 40
21
35
67 20

60
60 0
40 0 10 20 30 40 50 60 70 80 90 100
42 x
50
99

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

18
Solution k(Distance = 652)

## TSP30 (Performance = 652)

120

100

80

y 60

40

20

0
0 10 20 30 40 50 60 70 80 90 100
x

## Best Solution (Distance = 420)

42
38 TSP30 Solution (Performance = 420)
35
120
26
21
35 100
32
7
80
38
46
44 y 60
58
60 40
69
76
20
78
71
69 0
67 0 10 20 30 40 50 60 70 80 90 100
62 x
84
94

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

19
Overview of Performance

T S P 3 0 - O ve r v i e w o f P e rfo rm a n c e

1800

1600

1400

1200
e
c 1000
n
ta
s
i 800
D
600

400

200

0 B es t
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
W o rs t
G e n e r a tio ns ( 10 0 0 )
A ve ra ge

## Considering the GA Technology

“Almost eight years ago ...
people at Microsoft wrote
a program [that] uses some
genetic things for finding
short code sequences.
Windows 2.0 and 3.2, NT,
and almost all Microsoft
applications products have
shipped with pieces of
code created by that
system.”
Technology Group, Wired, September 1995

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

20
Issues for GA Practitioners

## • Choosing basic implementation issues:

– representation
– population size, mutation rate, ...
– selection, deletion policies
– crossover, mutation operators
• Termination Criteria
• Performance, scalability
• Solution is only as good as the evaluation function
(often hardest part)

## Benefits of Genetic Algorithms

• Concept is easy to understand
• Modular, separate from application
• Supports multi-objective optimization
• Good for “noisy” environments
time
• Inherently parallel; easily distributed

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

21
Benefits of Genetic Algorithms (cont.)
• Many ways to speed up and improve a GA-
problem domain is gained
• Easy to exploit previous or alternate solutions
• Flexible building blocks for hybrid applications
• Substantial history and range of use

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

When to Use a GA
• Alternate solutions are too slow or overly complicated
• Need an exploratory tool to examine new approaches
• Problem is similar to one that has already been
successfully solved by using a GA
• Want to hybridize with an existing solution
• Benefits of the GA technology meet key problem
requirements

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

22
Some GA Application Types
Domain Application Types
Control gas pipeline, pole balancing, missile evasion, pursuit

## Design semiconductor layout, aircraft design, keyboard

configuration, communication networks
Scheduling manufacturing, facility scheduling, resource allocation

## Machine Learning designing neural networks, improving classification

algorithms, classifier systems
Signal Processing filter design

## Combinatorial set covering, travelling salesman, routing, bin packing,

graph colouring and partitioning
Optimization

Conclusions

## Question: ‘If GAs are so smart, why ain’t they rich?’

Answer: ‘Genetic algorithms are rich - rich in
application across a large and growing
number of disciplines.’
- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

23
8.2
Particle Swarm
Optimization

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global 47

Introduction
• Concept first introduced by Kennedy and Eberhart (1995)
• Original idea was to develop a realistic visual simulation of
bird flock behaviour
• Simulation was then modified to include a point in the
environment that attracted the virtual bird agents
• Potential for optimisation applications then became apparent
• Combines self-experiences with social experiences

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

24
The natural metaphor
• A flock of birds (or school of fish, etc.) searching for food
• Their objective is to efficiently find the best source of food
• Nature-based theory underlying PSO:
The advantage of sharing information within a
group outweighs the disadvantage of having
to share the reward

## • Inspired from the nature social behavior and dynamic

movements with communications of insects, birds and fish

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

25
Introduction to the PSO: Origins
• In 1986, Craig Reynolds described this process in 3 simple
behaviors:

## Separation Alignment Cohesion

avoid crowding local move towards the average move toward the average
flockmates heading of local position of local
flockmates flockmates

##  Uses a number of agents (particles)

that constitute a swarm moving
around in the search space looking for
the best solution

##  Each particle in search space adjusts

its “flying” according to its own flying
experience as well as the flying
experience of other particles

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

26
Introduction to the PSO: Concept
• Collection of flying particles (swarm) - Changing solutions

optimum

## • Each particle keeps track:

– its best solution, personal best, pbest

## Introduction to the PSO: Concept

 Each particle modifies its
position according to:
• its current position
• its current velocity

## • the distance between its

current position and pbest

## • the distance between its

current position and gbest

27
Terminology

## • The “particles” in PSO have no mass or volume (essentially

they are just points in space), but they do have acceleration
and velocity
• Behaviour of groups in the developed algorithm ended up
looking more like a swarm than a flock
• Hence the name Particle Swarm Optimisation

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Swarm intelligence
• Millonas’ five basic principles of swarm intelligence:
 Proximity: agents can perform basic space and time calculations
 Quality: agents can respond to environmental conditions
 Diverse response: population can exhibit a wide range of behaviour
 Stability: behaviour of population does not necessarily change every
time the environment does
 Adaptability: behaviour of population must be able to change when

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

28
Cooperation example

## • Each particle is searching for the optimum

• Each particle is moving and hence has a velocity.
• Each particle remembers the position it was in where it had its
best result so far (its personal best)
• But this would not be much good on its own; particles need
help in figuring out where to search.

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

29
The basic idea II

## • The particles in the swarm co-operate. They exchange

information about what they’ve discovered in the places
they have visited
• The co-operation is very simple. In basic PSO it is like this:
– A particle has a neighbourhood associated with it.
– A particle knows the fitnesses of those in its neighbourhood, and
uses the position of the one with best fitness.
– This position is simply used to adjust the particle’s velocity

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Initialization. Positions
and velocities

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

30
What a particle does

## • In each timestep, a particle has to move to a new position. It

does this by adjusting its velocity.
– The adjustment is essentially this:
– The current velocity PLUS
– A weighted random portion in the direction of its personal
best PLUS
– A weighted random portion in the direction of the
neighbourhood best.
• Having worked out a new velocity, its position is simply its old
position plus the new velocity.

Neighbourhoods

geographical
social

31
Neighbourhoods

Global

1
Particle 1’s 3-
2
neighbourhood 8

7 3

Virtual circle 4
6
5

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

32
Particles adjust their positions according to a
‘Psychosocial compromise’ between what an
individual is comfortable with, and what society
reckons
My best
perf.
pi
Here I The best
am! x pg perf. of my
neighbours

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Pseudocode
http://www.swarmintelligence.org/tutorials.
php

Equation (a)
v[] = c0 *v[]
+ c1 * rand() * (pbest[] - present[])
+ c2 * rand() * (gbest[] - present[])
(in the original method, c0=1, but many
researchers now play with this parameter)

Equation (b)
present[] = present[] + v[]

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

33
Pseudocode
http://www.swarmintelligence.org/tutorials.
php
For each particle
Initialize particle
END

Do
For each particle
Calculate fitness value
If the fitness value is better than its peronal best
set current value as the new pBest
End

Choose the particle with the best fitness value of all as gBest
For each particle
Calculate particle velocity according equation (a)
Update particle position according equation (b)
End
While maximum iterations or minimum error criteria is not attained

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

Pseudocode
http://www.swarmintelligence.org/tutorials.
php
Particles' velocities on each dimension are
clamped to a maximum velocity Vmax. If
the sum of accelerations would cause the
velocity on that dimension to exceed
Vmax, which is a parameter specified by
the user. Then the velocity on that
dimension is limited to Vmax.

34
Parameters

## • Number of particles (swarmsize)

• C1 (importance of personal best)
• C2 (importance of neighbourhood best)

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

35
Parameters

• Number of particles
(10—50) are reported as usually sufficient.
• C1 (importance of personal best)
• C2 (importance of neighbourhood best)
• Usually C1+C2 = 4. No good reason other than
empiricism
• Vmax – too low, too slow; too high, too unstable.

## Some functions often used for

testing real-valued
optimisation algorithms
Griewank Rastrigin

Rosenbrock

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

36
... and some typical results

Optimum=0, dimension=30

## Best result after 40 000 evaluations

30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [±300] 0.003944 0.4033

## This is from Poli, R. (2008). "Analysis of the publications on the

applications of particle swarm optimisation". Journal of Artificial
Evolution and Applications 2008: 1–10.

37

## There has been enough I try to kill myself

improvement
although I'm the worst

I try to generate a
I'm the best new particle
but there has been not enough
improvement

## Dr. Hafis SKEM4173 innovative • entrepreneurial • global

rand(0…b)(p-x)
av
The better I The better is my best
am, the more I neighbour, the more
follow my own I tend to go towards
way him

38
End of Syllabus
Good luck

39