Ant Colony Optimization

Overview
Introduction
Biological Inspiration
Ant Colony Optimization
Ant System (AS) Algorithm
Simple ACO for TSP
Variants of ACO
Introduction
First proposed by M. Dorigo, 1992
Heuristic optimization method inspired
by biological systems
Multi-agent approach for solving difficult
combinatorial optimization problems
Traveling Salesman, vehicle routing,
sequential ordering, graph coloring, routing
in communications networks, etc.
Algorithm was inspired by observation
of real ant colonies
Biological Inspiration
Ants are social creatures behavior directed
to survival of colony
In many ant species the visual perspective
faculty is only very rudimentarily developed.
Most communication among individuals is
based on chemicals called pheromones.
Ants deposit pheromones on ground that
form a trail. The trail attracts other ants.
Biological Inspiration (Contd.)
Initially search for food is at random
Ants secrete pheromones while traveling
from the nest to food, and vice versa in
order to communicate with one another to
find the shortest path.
Pheromone mediated following
behavior induces the emergence of
shortest paths.
Probability of choosing a branch of a
path at a certain time depends on the
total amount of pheromone on the
branch.
The choice is proportional to the
number of ants that have used the
branches.
Biological Inspiration (Contd.)
Biological Inspiration (Contd.)
What is Ant Colony Optimization?
ACO is a population based
metaheuristic approach that applies
the ant - based shortest path
mechanism to any optimization
problem that can be mapped to a
graph problem such that the shortest
path in the graph represents the
optimal solution
General Working of ACO - Stages
Number of ants has to be initialized
Pheromone model
A pheromone model has to be defined for
the problem
Depends on problem nature and graph model
Effective pheromone model representation
determines the success of ACO to solve
problem
Pheromone between solution elements has
to be initialized with a value
General Working of ACO - Stages
Pheromone model
For Travelling salesman problem
Pheromone model is represented as
represents the pheromone value
between cities i and j
affinity of visiting city j from city i
j i,
t
General Working of ACO - Stages
Pheromone model
For Quadratic Assignment Problem
Pheromone model is represented as
represents the pheromone value
between location i and facility j
affinity of assigning facility j to location i
j i,
t
General Working of ACO - Stages
Solution Construction
Uses Probabilistic state transition rule for
step by-step construction of solution
Probabilistic rule is a function of pheromone
trail and a heuristic function
Heuristic function represents some local
information about the problem which the ant
uses for constructing the solution e.g. in
case of TSP : distance between cities
j i,
n
ij j i
d / 1
,
= n
j i
d
,
General Working of ACO - Stages
Memory of
visited nodes
i
j
k
r
t
ik
; n
ik
t
ij
; n
ij
t
ir
; n
ir
Stored in
pheromone table
A heuristic value of link (i,j)
which introduces
problem specific information

0
, ,
, ,
,
N c
j i j i
j i j i
j i
ij
p
F E
F E
L t
L t
General Working of ACO - Stages
Solution Construction
Probabilistic choice rule
E,F are scaling parameters defines the
relative importance and
Higher E relative to F - Exploration
Higher F relative to E - Exploitation

=
0
, ,
, ,
,
N c
j i j i
j i j i
j i
ij
p
F E
F E
n t
n t
otherwise
if j N
j i,
t
j i,
n
General Working of ACO - Stages
Pheromone Updating
Reinforcing the edges/paths that are followed
by each ant
Updating done so as to increase the pheromones
in the edges/path that are part of good
solutions
Ants will have more probability of selecting the
reinforced paths in the subsequent iteration
General Working of ACO - Stages
Pheromone Updating
Update pheromone using this increment, but
also evaporate some pheromone
V is the evaporation rate factor
L
k
is the length of the tour found by ant k

A + A
m
k
k
ij ij ij
1
) 1 ( t t t

=
0
1
k
k
ij
L
t
if ant k uses edge i,j
otherwise
General Working of ACO - Stages
A Simple ACO for
Traveling Salesman Problem
Simple ACO for TSP
Defined as:
We are given a set of cities and a symmetric distance matrix that
indicates the cost of travel from each city to every other city.
The goal is to find the shortest circular tour, visiting every city
exactly once, so as to minimize the total travel cost, which
includes the cost of traveling from the last city back to the first
city.
5
2
3
4
1
Simple ACO for TSP
Distance/Cost Matrix For TSP
1 2 3 4 5
1 0
2 3 0
3 2 3 0
4 4 10 1 0
5 9 1 3 8 0
Simple ACO for TSP
Input solution components
No. of cities, distance matrix
Solution construction and evaluation
Each ant generates tour and finds the tour length
Initialization
No of ant=5, Pheromone: t
ij
(t) =0.1
Termination
check
No of iterations
Pheromone updating
Updates the pheromone trail for all the ants
=0.1
Output module
Not Satisfied
Satisfied
Simple ACO for TSP
Initialization
The number of ants (k = 5) is defined
Pheromone trails used for constructing solutions are initialized
Pheromone trail intensity for route selection
) (t
ij
t
gives information about the desirability of selecting city j after city i
is initialized to between all pairs of
cities i and j
) 1 (
ij
t 1 . 0 ) 1 (
ij
t
1 . 0 ) 1 (
25
! t
1 . 0 ) 1 (
35
! t
1 . 0 ) 1 (
45
! t
1 . 0 ) 1 (
15
! t
1 . 0 ) 1 (
24
! t
1 . 0 ) 1 (
34
! t
1 . 0 ) 1 (
32
! t
1 . 0 ) 1 (
12
! t
1 . 0 ) 1 (
14
! t
Pheromone trail initialization between cities
2
3
4
1
5
1 . 0 ) 1 (
41
! t
1 . 0 ) 1 (
51
! t 1 . 0 ) 1 (
21
! t
1 . 0 ) 1 (
23
! t
1 . 0 ) 1 (
42
! t
1 . 0 ) 1 (
52
! t
1 . 0 ) 1 (
53
! t
1 . 0 ) 1 (
43
! t
1 . 0 ) 1 (
54
! t
Solution Construction and Evaluation
Each ant selects the first city randomly
e,.g. Ant -1 selects city-1 randomly
Each Ant selects the next city to move to
based on a probability given by
e.g Ant-1 selects to move from city-1 to other cities 2,3,4&5
by calculating the probabilities P
12
, P
13
, P
14
& P
15
E=1,F=2

|
!

0
, ,
, ,
,
N c
j i j i
j i j i
j i
ij
p
F E
F E
L t
L t
1 . 0 ) 1 (
13
= t
4 / 1
14
= n
2 / 1
13
= n
9 / 1
15
= n
3 / 1
12
= n
1 . 0 ) 1 (
15
= t
1 . 0 ) 1 (
12
= t
1 . 0 ) 1 (
14
= t
2
3
4
1
5
1
City 1 to
City No. 2 3 4 5

ij
(1) 0.1 0.1 0.1 0.1
n
ij
0.4 0.5 0.25 0.11
P
ij
0.33 0.515 0.129 0.026
Cumulative P
ij
0.33 0.845 0.974 1
rand() 0.218
Selected City
22
ITERATION 1
[]

|
!

0
, ,
, ,
,
N c
j i j i
j i j i
j i
ij
p
F E
F E
L t
L t
1 . 0 ) 1 (
25
! t
1 . 0 ) 1 (
24
! t
1 . 0 ) 1 (
23
! t
2
3
4
1
5
1
City 2 to
City No. 3 4 5

ij
(1) 0.1 0.1 0.1
n
ij
0.33 0.1 1
P
ij
0.158 0.008 0.834
Cum P
ij
0.158 0.166 1
rand() 0.389
Selected
City
55
3 / 1
23
= n
10 / 1
24
= n
1 / 1
25
= n
ITERATION 1
[1]

|
!

0
, ,
, ,
,
N c
j i j i
j i j i
j i
ij
p
F E
F E
L t
L t
1 . 0 ) 1 (
53
! t
1 . 0 ) 1 (
54
! t
2
3
4
1
5
City 5 to
City No. 3 4

ij
(1) 0.1 0.1
n
ij
0.33 0.125
P
ij
0.875 0.125
Cum P
ij
0.875 1
rand() 0.504
Selected City
33
8 / 1
54
! L
1
[1,2]
3 / 1
53
! L
ITERATION 1
2
3
4
1
5
ITERATION 1
Tour constructed by Ant-1
Tour Length L
1
=16
Solution Construction and Evaluation
At the end of iteration-1
1
4
3
2
5
1 2 5 4 3
2 1 3 4 5
5 3 1 2 4
4 1 5 3 2
3 4 2 5 1
16
15*
26
29
23
L
k
-Tour length
*Best Tour
Pheromone Updating

=
=
m
k
k
ij ij ij
1
) 1 ( t t V t

|
! A
0
1
k
k
ij
t
Pheromone after updation
1 2 3 4 5
1 0 0.1909 0.1567 0.09 0.1244
2 0.1567 0 0.09 0.1284 0.1958
3 0.1284 0.1525 0
0.2001
0.09
4 0.1244 0.09 0.1525 0 0.09
5 0.1334 0.09 0.1628 0.1525 0
) 2 (
ij
t
Simple ACO for TSP
Input solution components
No. of cities, distance matrix
Solution construction and evaluation
Each ant generates tour and finds the tour length
Initialization
No of ant=5, Pheromone: t
ij
(t) =0.1
Termination
check
No of iterations
Pheromone updating
Updates the pheromone trail for all the ants
=0.1
Output module
Not Satisfied
Satisfied
Parameters of ACO
Number of ants
Scaling parameters - E,F
Evaporation factor - V
Number of iterations
Visibility function heuristic information
Pheromone model
Performance of ACO depends on the values of
the above parameters A sensitivity analysis
need to done to select the best values for best
performance
Variants of ACO
Ant System
Ant Colony System
Max-Min Ant System
Rank-Based Ant System
Elitist Ant System (EAS)
First improvement on AS.
Provide strong additional reinforcement to
the arcs belonging to the best tour found
since the start of the algorithm.
Rank-Based Ant System (AS
rank
)
Another improvement over AS.
Each ant deposits an amount of pheromone
that decreases with its rank.
In each iteration, only the best (w-1) ranked
ants and the best-so-far ant are allowed to
deposit pheromone.
Min-Max Ant System (MMAS)
After all ants construct a solution,
pheromone values are updated.
(Evaporation is the same as in AS)
Lower and upper limits on pheromones limit
the probability of selecting a city.
Initial pheromone values are set to the upper
limit, resulting in initial exploration.
Occasionally pheromones are reinitialized.
Ant Colony System (ACS)
Uses ideas not included in the original
AS.
Differs from AS in three main points:
Exploits the accumulated search
experience more strongly than AS.
Pheromone evaporation and deposit take
place only on the best-so-far tour.
Each time an ant uses an arc, some
pheromone is removed from the arc.
Ant Colony System (ACS)
Pseudo-random proportional rule used to
decide which city to visit next.
Only best-so-far ant adds pheromone after
each iteration. Evaporation and deposit only
apply to best-so-far.
Ant Colony System (ACS)
The previous pheromone update was
global. Each ant in ACS also uses a local
update that is applied after crossing an arc.
Makes arc less desirable for following ants,
increasing exploration.
THANK YOU