1415 OD Metaheuristics BC-2008-2

Simulated Annealing
Simulated Annealing
Enables search process to escape from local minima.

Instead of steepest ascent/mildest descent approach
as in tabu search, it tries to search for the tallest hill.
Early iterations take steps in random directions.
Let
Zc = objective function value for current trial solution.
Zn = objective function value for current candidate to
be next trial solution.
T = tendency to accept current candidate as next
solution if not better than current solution.
Move selection rule: among all immediate neighbors

of current trial solution, select one randomly. To
maximize, accept or reject candidate as follows:
Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho
466
Probability of accepting solutions
If Zn Zc, always accept candidate,

If Zn < Zc, accept candidate with following probability:
Z Zc
Prob{acceptance} = e x , where x = n
T
For minimization problems reverse Zn and Zc.
If candidate is rejected, repeat for another random
immediate neighbor.
If no immediate neighbors remain, terminate algorithm.
467
A simulated annealing algorithm
Zc
- TTTT
Zn
=
x
Larger T: probability of
Prob{acceptance} = ex
accepting is higher.
0.01
0.990
Simulated annealing starts
-0.25
0.779
with large T, enabling the
search to proceed in almost
-1
0.368
random directions, and
-3
0.050
gradually decreases it as the
-5
0.007
iterations proceed, in order to
emphasis on mostly climbing
upward.
A temperature schedule should be chosen.
Implementation of move selection rule: compare random
number between 0 and 1 to the probability of acceptance.
Initialization: start with a feasible initial trial solution.

Iteration: Use the move selection rule to select next
trial. If none of immediate neighbors of current
solution are accepted, the algorithm is terminated.
Check the temperature schedule: decrease T if a
certain number of iterations have been performed.
Stopping rule: stop after a predetermined number of
iterations have been performed at the smallest value
of T (or if there are no accepted solutions). Best trial
solution at any iteration is the final solution.
468
Questions in simulated annealing
Traveling salesman problem example

1. Initial trial solution: any feasible solution. Can be
chosen randomly or it can be 1-2-3-4-5-6-7-1.
2. Neighborhood structure: is one reached by a sub-tour
reversal, as described previously.
3. Random selection of an immediate neighbor:
beginning slot for sub-tour reversal cannot be first,
last, or next-to-last slots; ending slot cannot be last
slot.
4. Temperature schedule: five iterations for each of the
five values T1 = 0.2Zc (Zc is objective function for initial
solution), Ti+1 = 0.5Ti, i = 1, 2, 3, 4.
1. How should the initial solution be selected?

2. What is the neighborhood structure that specifies
which solutions are immediate neighbors?
3. What device should be used in the move selection
rule to randomly select one of the immediate
neighbors?
4. What is an appropriate temperature schedule?
Questions answered in the following examples.
469
470
471
Solving problem Iteration 1
Solving problem
Initial trial solution: 1-2-3-4-5-6-7-1, Zc= 69, T1= 13.8.

Sub-tour to reverse can begin between the second
slot (city 2) and the sixth slot (city 6).
Can have equal probabilities to start in any of the five slots.
After choosing beginning slot, choose end slot with

equal probability.
Suppose 3-4 was chosen to reverse, 1-2-4-3-5-6-7-1, Zn= 65.
Solutions are not always feasible (in that case choose new
pair of random numbers). If algorithm chooses to reverse
2-3-4-5, solution 1-5-4-3-2-6-7-1 is not feasible.
As Zn= 65 < Zc= 69, this is the next trial solution.
472
Suppose that Iteration 2 results in reversing 3-5-6, to

obtain 1-2-4-6-5-3-7-1, with Zn= 64.
Suppose now that Iteration 3 results in reversing 3-7,
to obtain 1-2-4-6-5-7-3-1, with Zn= 66. As Zn > Zc:
Prob{acceptance} = e
= e 2/ 13.8 = 0.865
candidate is accepted if next random number < 0.865.

One application of SA gave best solution 1-3-5-7-6-4-2-1
at iterations 13 and 15 (out of 25) with distance = 63.
Nonlinear programming application
473
Nonlinear programming application
Problems of the type:
Maximize
subject to
Zn Zc
T
3. Random selection of an immediate neighbor: Set
f ( x 1 , , x n )
j =
U j Lj
6
reset x j = x j + N(0, j ), for j = 1, , n
L j x j U j , for j = 1, , n
1. Initial trial solution: any feasible solution. Can be

xj = (Uj Lj)/2.
2. Neighborhood structure: any feasible solution, see
random selection of immediate neighbor.
where N(0,j) is a random observation of a normal

distribution with 0 mean and j standard deviation.
Repeat process until feasible solution is attained.
4. Temperature schedule: five iterations for each of
the five values T1 = 0.2Zc, Ti+1 = 0.5Ti, i = 1, 2, 3, 4.
474
Example
Genetic algorithms
Initial solution: x = 15.5, Zc = f(15.5) = 3 741 121, and

T1 = 0.2 Zc = 748 224. = (31 - 0)/6 = 5.167.
Iteration 1: x = 15.5 + N(0,5.167) = 15.5 7.5 = 8.
Zn = f(8) = 3 055 616. As
Motivation
What evolution brought us?

Zn Zc 3055616 3714121
=
= 0.916
T
748224
Prob{acceptance} = e 0.916 = 0.400
Vision
Hearing
Smelling
Taste
Touch
Learning and reasoning
Can we emulate the evolutionary process with

todays fast computers?
After 25 iterations, gives optimal value of x = 20.031

(optimal is x = 20).
475
476
477
Genetic algorithms
Basic elements
Randomized search algorithms based on mechanics

of natural selection and genetics.
Principle of natural selection through survival of the
fittest with randomized search.
Search efficiently in large spaces.
Robust with respect to the complexity of the search
problem.
Use a population of solutions instead of searching
only one solution at a time.
Candidate solution is encoded as a string of characters

in binary or real. Bit string is called a chromosome.
Solution represented by a chromosome is the
individual. A number of individuals forms a
population.
Population is updated iteratively; each iteration is
called a generation.
Objective function is called the fitness function.
Fitness value is maximized.
Multiple solutions are evaluated in parallel.
478
A basic genetic algorithm
479
Genetic algorithm
Initialization: Start an initial population of feasible

solutions, e.g. randomly. Evaluate the fitness for each
member of the population.
Iteration:
1. Select some members of population to become parents.
2. Cross genetic material of parents in a crossover operation.
Mutation can occur in some genes.
3. Take care of infeasible solutions, repeating birth process
until feasible solution is obtained.
4. Evaluate fitness of new members, including the clones.
Stopping rule: stop using fixed number of iterations,

fixed number of iterations without improvement, etc.
480
Encoding-crossover-mutation
10010110
01100010
10100100
10011001
01111101
...
...
...
...
Chromosome

1011 0110 1001

Gene
Crossover
10011110
10110010
10010010
10111110
Crossover point
Mutation
10011110
481
Genetic algorithm iteration
Binary encoding
(11, 6, 9)
Elitism
Selection
Crossover
Current
generation
Mutation bit
10011010
482
Mutation
10010110
01100010
10100100
10011101
01111001
...
...
...
...
Next
generation
483
Spaces in GA iteration
gene-space
generation N
01100
10001
11010
00111
11000
10110
Selection (reproduction)
fitness operators
problem-space
fitness-space
(de)coding
12
17
26
7
24
22
fitness function
34
48
23
15
41
50
Proportional selection:
p = 0.19
p = 0.23
p = 0.11
p = 0.07
p = 0.16
p = 0.24
genetic
operators
genes fitness
01100
34
10001
48
11010
23 reproduction
00111
15
11000
41
10110
50
genes
01100
10001
10001
11000
10110
10110
Sum = 211
generation N +1
01011
10111
11001
00011
11010
01010
Tournament selection:
Randomly select pairs
Fitter individual wins (deterministic or probabilistic)
484
Questions in genetic algorithms
485
Nonlinear programming example
1. What is the encoding scheme?

2. What should the population size be?
3. How should the individuals of the current
population be selected to become parents?
4. How should the features of the children be derived
from the features of the parents?
5. How should mutations occur in the features of the
children?
6. Which stopping rule should be used?
1. Encoding scheme: integers from 0 to 31. Five binary

genes are needed.
486
Nonlinear programming example
1. Encoding scheme: exactly as before (no encoding).
2. Population size: 10 (problem is simple).

3. Selection of parents: select randomly 4 from the 5
most fit, and 2 from the 5 least fit.
Select 3 pairs of parents, that will produce 6 children.
Elitism: four best solutions are clones in next generation.
6 generations were enough in this example (optimum

found on first iteration).
487
Example: 1-2-3-4-5-6-7-1.
Initial population: generated randomly, using possible
links between cities.
Parents are 0x01x. Children can be 01011 and 00010.
5. Mutation operator: mutation rate is 0.1 for each

gene.
6. Stopping criteria: stop after 5 consecutive
generations without improvement.
Fitness value is just the objective function value in
this example.
Select 3 pairs of parents, that will produce 6 children.

Elitism: four best solutions are clones in next generation.
4. Crossover operator: When a bit is difference in the

two parents, select 0 or 1 randomly (uniform dist.)
Example: x = 25 is 11001 in base 2.
2. Population size: 10 (problem is simple).

3. Selection of parents: select randomly 4 from the 5
most fit (according to fitness), and 2 from the 5 least
fit.
488
489

4. and 5. Operators: follows an algorithm:
1. Options for next link: links from current city not in
childrens tour that are used by parents.
2. Selection of next link: randomly with a uniform distribution.
3. Mutation: if a mutation occurs (RN<0.1) replace last link
with any other possible one, unless it is impossible.
4. Repetition: when there is still more than one link to include
in child, go to step 1.
5. Completion: add last city and complete tour if links exists. If
not, solution is infeasible (miscarriage) and all process
should be repeated.
5. Stopping criteria: 5 iterations without improvement.

490
Ant Colony Optimization
492
The foraging behaviour of ants
Note that problem is too simple, and solutions

(individuals) can be repeated in the population.
See this example in Hilliers book.
nest building, food storage

garbage collection, war
foraging (to wander in search of food)
There is no management in an ant colony

collective intelligence
They communicate using:

pheromones (chemical substances), sound, touch
Curiosities:
Ant colonies exist for more than 100 million years
Myrmercologists estimate that there are around 20 000 species of ants.
493
Food Source
Destination
nodes / arcs
environment is discrete
As the real ants, they:
choose paths based on pheromone

deposit pheromones on the travelled paths
The environment updates the pheromones
Artificial ants have more abilities:

d) At the end, all ants follow the shortest
path.
Photos: http://iridia.ulb.ac.be/~mdorigo/ACO/RealAnts.html
491
Ants can perform complex tasks:
Artificial ants move in graphs
b) - Ants go around the obstacle following one

of two different paths with equal
probability
c) - On the shorter path, more

pheromones are laid down
Child: From P1 can be 1-2 or 1-7. From P2 can be 1-2 or 1-3.

1-2 has 50% probability to be chosen. Why?
Suppose 1-2 was chosen. At next step links 2-3 (P1) and 2-4
(P2) can be chosen. Suppose it is 2-4.
Child is 1-2-4. And so on!
Artificial ants
How can almost blind animals manage to learn the shortest

route paths from their nests to the food source and back?
a) - Ants follow path between the

Nest and the Food Source
Example: P1: 1-2-3-4-5-6-7-1

P2: 1-2-4-6-5-7-3-1
What is special about ants?
Introduced by Marco Dorigo (1992), has been well

received by academic world and it is starting to be
used in industrial applications.
Ant Colony Optimization is the most used method of
the Artificial Life algorithms (Wasp, Bees, Swarm).
Applications: Travelling Salesman Problem, Vehicle
Routing, Quadratic Assignment Problem, Internet
Routing and Logistic Scheduling.
There are also some applications of ACO in clustering
and data mining problems.
Traveling salesman problem
494
Nest
Source
they can see (heuristic ) [former visibility]

they have memory (feasible neighbourhood N) [former tabu list ]
495
Notation for artificial ants in TSP
Mathematical framework of real ants
cij cost for transversal from city i to city j;

ij pheromone in edge (i,j);
ij amount of pheromone deposited in edge (i,j);
ij = 1/ cij local heuristic;
pij probability that ant k in city i visits city j;
N set of cities still to be visited by ant k in city i ;
evaporation coefficient;
t time is performance index;
, parameters that determine relative importance
of pheromone versus heuristic;
496
Mathematical framework
Choose node
ij ij
, if j
k
pij = ij ij
j
0, otherwise
Update Feasible Neighbourhood List
N =N\ j
Pheromone update
ij (l + 1) = ij (l ) (1 ) + ijk
1/ cij , if ant k travels from i to j
ijk =
0, otherwise
Initialization
Set ij = 0
For l =1: Nmax
Build a complete tour
For i = 1 to n
For k = 1 to m
Choose node
Update N
pijk = f ( ij )
Ant 1, t=0
12
13 +
j =2
Deposit pheromone
i =1
Ant 1, t=1
12
Environment (time)
updates pheromones
j =2
j =3
13
i =1
Ant 2, t=2
12
ij (t + 1) = ij (t ) (1 ) + k
ij
j =2
497
n cities (5)
Complexity: (n1)! / 2
2
2
Apply Local Heuristic
end
end
Analyze solutions
For k = 1 to m
Compute fk
end
Update pheromones
end
17
17
5
5
498
Solution using the nearest city heuristic
499
Solution using the nearest city heuristic
Step #2
2
j =3
13
i =1
Traveling Salesman Problem
Step #1
Choose trail
Step #5
2
The final solution is

obviously nonoptimal
17
2
2
Step #3
2
Step #4
This heuristic can give the

optimal solution if it is given
a proper initial node
2
2
2
17
2
2
5
2
2
500
17
5
501
ACO in Travelling Salesman Problem
Iteration l=1, ant m=1

All paths have the same
pheromone intensity 0=0.5
Pheromone trail and heuristic
information have the same weight
= 1, = 1, =0.1
An ant is randomly placed
The probability to choose is, in this
case, based only on heuristic
information
m ants
n cities
=1/d

vs
ACO balances the heuristic information with the
experience (pheromone) information
503

All paths have the same
pheromone intensity 0=0.5
information have the same
weight = 1, = 1,=0.1
The probability to choose is, in
this case, based only on heuristic
information
Step #3
32%
53%
22%
47%
46%
Step #5
Step #4
p12=31%
p13=16%
p14=22%
p15=31%
Ant m = 1 chooses node 5

502
Step #2
Step #1

100%
Step #1
p12=31%
p13=16%
p14=22%
p15=31%

f = 2 + 2 + 2 + 5 + 17 = 12.36
Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho 1
504
505
Iteration l=1, pheromone update (1)

Step #3
Step #2
The final solution of ant m=1

is d=12.36. The reinforcement
ij produced by this ant m=1
is 0,08.
39%
27%
34%
35%
65%
Step #4
Step #5

is d=10,47. The reinforcement
ij produced by ant m=2 is
0,095.
100%
f 2 = 2 + 5 + 5 + 2 + 2 = 10.47
506
507
Updating pheromone matrix (2)
The pheromone update can be done following different approaches:

Considering the pheromone dropped by every ants
0.5
0.5
(l + 1) = 0.5
0.5
0.5
0.5 0.5 0.5 0.5

0
0
0
0.08 0
0.095
0
0
0
0
0
0.5 0.5 0.5 0.5
0
0.08
0
0 0
0
0.095
0
0
0.5 0.5 0.5 0.5 (1 ) + 0.08

0
0
0
0 + 0
0
0
0.095
0
0.5 0.5 0.5 0.5

0.08
0
0
0 0
0
0
0
00.95
0
0
0.5 0.5 0.5 0.5
0
0
0.08
0 0.095
0
0
0
0
Considering the pheromone dropped by the best ant of the present iteration
0.5 0.5 0.5 0.5 0.5
0.5 0.5 0.5 0.5 0.5
0
0
0.5 0.5 0.5 0.5 0.5

0.5 0.5 0.5 0.5 0.5
0
0.095
(l + 1) = 0.5 0.5 0.5 0.5 0.5 (1 ) + 0
0.095
0
0
0.095
0
0
0.095
0
0
0
0
0
0
The pheromone trails have

different intensities
information have the same weight
= 1, = 1, =0.1
The probability to choose is

0
0
0
00.95
0
508

Step #2
Step #3
The pheromone trails have

different intensities
information have the same
weight = 1, = 1, =0.1
The probability to choose is
32%
22%
29%
Step #5
Step #4
509

71%
46%
p41=19%
p42=26%
p43=23%
p45=32%
Considering the pheromone dropped by the best ant in all iterations

(after iteration N=1, this is the same as the previous approach)
Step #1

100%
Step #1
p21=26%
p23=29%
p24=26%
p25=19%

f1 = 2 + 2 + 2 + 5 + 5 = 10.47
510

Step #2
511
Iteration l=2, pheromone update (1)

Step #3
24%
37%

and m=2 is d=10,47. The
reinforcement produced by
each ant is 0,095.
24%
52%
Step #3
63%
Step #4
100%
f 2 = 5 + 5 + 2 + 2 + 2 = 10.47
512
513
Updating pheromone matrix (2)

Considering the pheromone dropped by every ants
0.45 0.55 0.45 0.45 0.45
0.45 0.45 0.55 0.45 0.45
0
0
0.45 0.45 0.45 0.45 0.55

0.55 0.45 0.45 0.45 0.45
0
0.095
(l + 1) = 0.45 0.45 0.45 0.55 0.45 (1 ) + 0
0.095
0
0
0.095
0
0
0.095
0 0
0.095
0
0
0
0 0
0
0.095
0
0
0 + 0
0
0
0.095
0

00.95 0
0
0
0
00.95
0 0.095
0
0
0
0
Considering the pheromone dropped by the best ant of the present

iteration
0.45 0.55 0.45 0.45 0.45
0.45 0.45 0.55 0.45 0.45
0
0
0.45 0.45 0.45 0.45 0.55

0.55 0.45 0.45 0.45 0.45
0
0.095
(l + 1) = 0.45 0.45 0.45 0.55 0.45 (1 ) + 0
0.095
0
0
0
0
0.095
0
0
0.095
0
0
0
0
0
0
0
00.95
0
Considering the pheromone dropped by the best ant in all iterations

514

1415 OD Metaheuristics BC-2008-2

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1415 OD Metaheuristics BC-2008-2

Uploaded by

Copyright:

Available Formats

Simulated Annealing

Enables search process to escape from local minima.

Move selection rule: among all immediate neighbors

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Probability of accepting solutions

If Zn Zc, always accept candidate,

A simulated annealing algorithm

Initialization: start with a feasible initial trial solution.

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Questions in simulated annealing

Traveling salesman problem example

1. How should the initial solution be selected?

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Solving problem Iteration 1

Initial trial solution: 1-2-3-4-5-6-7-1, Zc= 69, T1= 13.8.

After choosing beginning slot, choose end slot with

Suppose that Iteration 2 results in reversing 3-5-6, to

candidate is accepted if next random number < 0.865.

Nonlinear programming application

Nonlinear programming application

Problems of the type:

3. Random selection of an immediate neighbor: Set

1. Initial trial solution: any feasible solution. Can be

where N(0,j) is a random observation of a normal

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Initial solution: x = 15.5, Zc = f(15.5) = 3 741 121, and

Prob{acceptance} = e 0.916 = 0.400

Can we emulate the evolutionary process with

After 25 iterations, gives optimal value of x = 20.031

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Randomized search algorithms based on mechanics

Candidate solution is encoded as a string of characters

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

A basic genetic algorithm

Initialization: Start an initial population of feasible

Stopping rule: stop using fixed number of iterations,

Genetic algorithm iteration

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Questions in genetic algorithms

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Nonlinear programming example

1. What is the encoding scheme?

1. Encoding scheme: integers from 0 to 31. Five binary

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Nonlinear programming example

1. Encoding scheme: exactly as before (no encoding).

2. Population size: 10 (problem is simple).

6 generations were enough in this example (optimum

Joo M. C. Sousa, U. Kaymak, C.A. Silva, A. Moutinho

Parents are 0x01x. Children can be 01011 and 00010.

5. Mutation operator: mutation rate is 0.1 for each

Select 3 pairs of parents, that will produce 6 children.

Traveling salesman problem example

4. Crossover operator: When a bit is difference in the

Example: x = 25 is 11001 in base 2.

2. Population size: 10 (problem is simple).