A computational study & tool for the

Travelling Salesman Problem (TSP)

Mangesh Gharote, Dilys Thomas and Sachin Lodha

Tata Consultancy Service – Innovations Lab

Note: A general presentation for research and study purpose.

Original Article found on
http://dl.acm.org/citation.cfm?id=2694462
Cite as: Mangesh Gharote, Dilys Thomas, Sachin Lodha:
Excel Solvers for the Traveling Salesman Problem. COMAD 2012: 103-107
Talk outline
 Problem statement
 Applications
 Computational Complexity
 Top Awards
 Combinatorial algorithm
 nearest neighbour
 greedy
 Geometric repair heuristics
 intersection removal
 hinge and crest optimization
 Benchmark datasets
 cities, PCB layout etc.
 comparison of algorithms to state of art
2
 Problem Statement, Complexity
 Given
 n cities, An actual tour on the globe with
 Distances or costs between pair the lines indicating tour paths

or a metric on the space

 Find
 Permutation such that tour length is minimized
 Problem
 There are n! different permutations

 n=30 beyond reach of all computers on earth

combined
 100’s of mathematicians and computer scientist have
tried to solve this problem
3
Applications
 BPO Vehicle Routing
 Bus Scheduling
 Development of flight schedules
 Crew Scheduling
 Order-picking problem in warehouses
 Printing press scheduling problem

 Network cabling in a country

 Computer wiring
 Query workload ordering for optimization
 VLSI chip design connectivity layout
 Drilling of printed circuit boards

4
Applications
 Hot rolling scheduling problem in Iron & Steel
Industry
 Overhauling gas turbine engines (Ordering nozzle
guides)
 X-Ray crystallography (Ordering positions for
measurement)

 Global navigation satellite system

 Ordering test cases in regression suite to re-use
components

5
Popular Appeal
 TSP in 1960 achieved
national prominence in
the United States of
America

 Procter & Gamble

offered prizes up to
$10,000 for identifying
the most correct links in
a particular 33-city
problem.

6
Awards
 Proved to be NP complete in 1972 by Richard
Karp
 Richard Karp won the Turing Award 1985
 For contribution to the theory of algorithms and
NP completeness including this result
 He showed a reduction from a result on logic
satisfiability of conjunctive normal form clauses
that have 3 literals each (3SAT)

 The most recent 2010 Gödel Prize

 Awarded to Sanjeev Arora and Joseph S B
Mitchell for polynomial time approximation scheme
for Euclidian TSP
7
Objective of our work

 Develop our own algorithms on top of

reasonable in practice TSP algorithms
 Near optimal in practice
 Reduce run time for medium to large instances
 Scalable in memory for medium to large instances
 Ease of using the tool, handle different distance
metrics including longitude and latitude
 Easy to visualize tours

 Improve state of art TSP solvers available in

Excel

8
Different Approaches for Solving TSP
 Exact  Heuristics
 Enumeration  Simulated Annealing
 Linear Programming  Genetic Algorithm
- Miller–Tucker–Zemlin  Ant Colony
(1960) Optimization
 Dynamic Programming  Electromagnetism like
Algorithm

 Approximation
Algorithms  Lower Bound
- Nearest Neighbour  Held -Karp
- Greedy
- K-Opt, Kernighan Lin (70’s)
- Christofides (1965)

9
Approximation Algorithms
I) Nearest Neighbour
II) Greedy Algorithm

10
Nearest Neighbour (NN) Algorithm for TSP (Pseudo code)
1. Select an arbitrary vertex as
current vertex.

2. Find out the shortest edge

connecting current vertex and
an unvisited vertex V.

3. Set current vertex to V.

4. Mark V as visited.

5. If all the vertices in domain are

visited, then terminate;
otherwise, go to step 2. Time Complexity:
O(n2)

11
Greedy Algorithm for TSP (Pseudo code)
1. Sort all edges in increasing
order of weigths.
2. Select the shortest edge and
add it to our tour if it Sub
doesn’t violate any of the Cycle
below constraints:
a) if it is not yet in the tour
b) if adding it would not
create a degree-3 vertex Degree of Vertex <3
or
c) If does not create a
cycle of size less than N.
3. Do we have N edges in our
tour? If no, repeat step 2.  Time Complexity
 O( n2 log (n))

12
Numerical Example Using NN & Greedy

1 4 2
3
Input Graph 2 4
2
3 4
2

4 4
1 2 1 2
3
2 4
2
3 4 3 4
2 2
Starting Node
12 11

Nearest Neighbour Greedy

13
Nearest Neighbour with different starting vertex
16 Cities Benchmark Problem

Start

Difference from Difference from

Optimal 32 % Optimal 5 %

Start

14
16 Cities Benchmark Problem
Greedy

Difference from
Optimal 16 %

15
51 Cities Benchmark Problem

Nearest Neighbour Greedy

Difference from Optimal Difference from Optimal

18.7 % 13 %
16
Results on Benchmark Problems
Using Nearest Neighbour (NN) and Greedy

Sr. No Cities NN
Best Greedy
Greedy
1 14 15.6% 17.0%
2 16 4.5% 16.6%
4 48 13.0% 19.7%
5 51 19.2% 13.0%
6 67 7.2% 18.2%
7 96 11.8% 5.1%
8 101 17.0% 26.3%
9 280 21.4% 14.8%
10 505 20.7% 15.4%
11 783 25.0% 19.6%
12 1002 21.4% 19.2%
14 14051 21.3% Out of memory
Avg 16.1% 17%

17
Observation about NN and Greedy
 For ‘N’ node graph, theoretically can guarantee
only upto O(log N) multiplicative factor away
from optimal
 towards the end more expensive edges could be forced
 Start point makes a difference for nearest neighbour

 In practice gives 25% away from optimal for

moderately large sized instances

 They can be improved further with geometric

constructions
 2-opt, intersection removal, hinge-crest optimization

18
Types of Distances

 2D – Euclidean

 ND – Euclidean (L1 , L2, Linfty)

 Geographical (Longitude – Latitude)

 Symmetric - Asymmetric

 Triangular Inequality

19
Geographical Distance Code - Longitude & Latitude
PI = 3.141592
For computing the geographical distance;
RRR = 6378.388; /* Radius of earth in kms */

deg = (int) X[i];

min = X[i] – deg;
rad = PI * (deg + 5.0 * min/3.0)/180.0;

q1 = cos (longitude[i] – longitude[j]);

q2 = cos (latitude[i] – latitude[j]);
q3 = cos (latitude[i] + latitude[j]);

Dij = (int) (RRR * acos(0.5*((1.0 + q1)*q2 – (1.0 – q1)*q3))+1.0);

20
World Tour - 535 Airports NN – 2 Opt
19% Away from Optimal

21
World Tour - 535 Airports Greedy 2 Opt

Difference from Optimal

Alaska North America
Europe 10 %

Hawai
Asia Guam

Africa
South America Fiji

Australia

Santiago

Trans Pacific Flights 22

USA 48 Mainland Capitals

23
Africa and Islands
Optimal Tour

Sahara

Senegal

St. Helenas

Mauritius

Tristan de cunha
South Africa

24
PCB- Drilling Problem (Nodes-2103)
Optimal Solution

Actually our algorithm ensures

it is not a criss-cross

Referred as one of the Nasty TSP NN 2 Opt 6.5 %

25
State-of-the-art problem sizes

From: Book: Travelling Salesman Problem: A computational study

David L. Applegate, Robert E. Bixby, Vasek Chvatal, William J. Cook
26
State-of-the-art optimal solutions

From: Book: Travelling Salesman Problem: A computational study

David L. Applegate, Robert E. Bixby, Vasek Chvatal, William J. Cook 27
State-of-the-art Concorde Solver

From: Book: Travelling Salesman Problem: A computational study

David L. Applegate, Robert E. Bixby, Vasek Chvatal, William J. Cook

28
Largest TSP instances solved optimally till date

Ref: http://www.tsp.gatech.edu/optimal/index.html 29
Motivating Our Work: Need for Approximate Solutions

Year – 2001
 15,112 German towns solved using 110 processors
at Rice University and Princeton University ~ 22.6
years on 500MHz Alpha processor.

Year – 2005
 33,810 Circuit board using state-of-the-art
Concorde TSP Solver took 15.7 CPU - years

Year -2006
 85,900 points was solved using Concorde TSP
Solver took over 136 CPU-years.

30
Our Benchmark Results

Sr. No Distance Cities NN

Best NN 2 Opt* Greedy
Greedy Greedy 2 Opt*
1 GEO 14 15.6% 13.6% 17.0% 16.6%
2 GEO 16 4.5% 1.9% 16.6% 0.2%
4 P-EUC 48 13.0% 7.1% 19.7% 11.7%
5 EUC_2D 51 19.2% 8.5% 13.0% 11.0%
6 GEO 67 7.2% 1.2% 18.2% 1.0%
8 EUC_2D 101 17.0% 11.1% 26.3% 24.2%
9 EUD_2D 280 21.4% 12.5% 14.8% 8.1%
10 GEO 505 20.7% 19.3% 15.4% 10.1%
11 EUC_2D 783 25.0% 16.4% 19.6% 12.6%
12 EUC_2D 1002 21.4% 13.6% 19.2% 14.4%
13 EUC_2D 2103 9.4% 6.5%
14 EUC_2D 14051 21.3% 13.8% Out of memory
Avg 16.3% 10.5% 18% 11.0%

31
Well Known Approximation Algorithms
 Lin-Kernighan
 heuristic works well in practice

 Linear Programming Formulation for TSP

 basis for using CPEX, Gurobi, Xpress solvers

 Held-Karp Dynamic Programming (n2 2n)

 40 year open problem if there is an exact algorithm for
TSP with time (cn) for c < 2.

 Christofides 1.5 Approximation

 30 year open problem if there is an approximation
algorithm with factor < 1.5

32
Lin-Kernighan
 Costly to find 3-opt improvements: O(n3)
candidates

 k-opt: generalizes 3-opt O(nk)

 Tour modification:
 Collection of simple changes
 Some increase length
 Total set of changes decreases length

33
Linear Programming Formulation for TSP

Degree of Vertex is 2

Subtour elimination

34
Held-Karp Dynamic Programming
 O(n22n) algorithm for TSP

 Uses Dynamic programming

 Take some starting vertex s

 For set of vertices R (s  R), vertex w  R, let

 B(R,w) = minimum length of a path, that
Starts in s
Visits all vertices in R (and no other vertices)
Ends in w

A&N: TSP 35
TSP: Recursive Dynamic Programming formulation
 B({s},s) = 0

 If |S| > 1, then

 B(S,w) = minv  S – {w}B(S-{w}, v}) + weight(v,w)

 If we have all B(V,v) then we can solve TSP.

 Gives requested algorithm using DP-

techniques.

A&N: TSP 36
Christofides 1.5 Approximation
 Make a Minimum Spanning Tree T

 Set W = {v | v has odd degree in tree T}

 Compute a minimum weight matching M in the

graph G[W].

 Look at the graph T+M. (Note: Eulerian!)

 Compute an Euler tour C’ in T+M.

 Add shortcuts to C’ to get a TSP-tour

A&N: TSP 37
Work done by students (2010-11)

Metaheuristics:
 Simulated Annealing
 Ant Colony Optimization
 Electromagnetism-like Algorithm

Project Title:
“Excelling with Metaheuristics”
B.E. Project Done By:
Prem Nathan, Prashant Kumar and Sani Kumbhar

38
 Simulated Annealing (Kirkpatrick, 1985)
 SA is a variant of local Initial position
of the ball Simulated Annealing explores

(neighborhood) search
more. Chooses this move with a
small probability (Hill Climbing)

 Traditional local search always

moves in a direction of Greedy Algorithm

improvement gets stuck here!

Locally Optimum
Solution.

 SA allows non-improving moves to

avoid getting stuck at a local
optimum
Upon a large no. of iterations,
SA converges to this solution.

 Analogy is from Material Science

 Slowly cool down a heated solid, so that all particles arrange in
the ground energy state
 At each temperature wait until the solid reaches its thermal
equilibrium
 Probability of being in a state with energy ‘E’ given by Boltzmann
39
Ant Colony Optimization (Dorigo et al 1996)

 Ants secrete pheromone while traveling, in order to communicate with

one another to find the shortest path

The more ants follow a trail, the more attractive that trail becomes
for being followed.

 Even when the tracks are equal the behavior will encourage one over
the other—convergence
40
Electromagnetism Like Algorithm (Birbil & Fang, 2003)
Initialization Force Calculation Movement Neighborhood
Search

F
 Idea is based on the attraction repulsion mechanism of
electromagnetism theory (Coulomb’s law)
 force exerted on a point via other points is inversely
proportional to the distance between the points and directly
proportional to the product of their charges

 The charge of each particle relates to the value of objective

function
 the better the objective function value, the higher the
magnitude of attraction.
41
Results – Metaheuristics
Nodes EM (%) ANT(%) SA (%)
14 5.3 3.0 7.6
52 7.1 6.4 18.2
96 9.9 6.0 15.2
159 15.4 14.3 20.5
226 15.9 13.1 11.8
299 22.9 18.5 24.7
654 24.2 24.9 25.3
Time Time
Nodes EM SA
(min) (min)
1291 14.6% 3.1 18.0% 18.2
2103 6.3% 12.1 8.7% 34.3
3795 13.1% 20.1 19.0% 24.3
4461 20.0% 6.1 26.2% 31.1
5934 18.7% 6.1 23.8% 68.1
14051 21.4% 16.5 22.9% 44.6 42
Summary
 TSP - Geometric NP hard problem
 Exact solution expensive (time, space)
 Implemented combinatorial algorithms
 NN, Greedy, intersection removal, hinge-crest optimization

 Provide approximations in practice

 Can solve and visualize tours of medium to large size
 (e.g. 14,000 cities – two orders of magnitude larger than Elsevier paper)

 Multiple applications to PCB layout, X-ray crystallography, other

engineering and scientific domains
 Future Plans – To improve on solving even larger size problems
by optimizing algorithm implementations
 Privacy Preserving TSP
43
Demonstration – TSP Using Excel
 Demo 1
 48 US Mainland Capitals
 NN, NN 2-Opt, Greedy,
Greedy 2-Opt

 Demo 2
 Printed Circuit Board
Drilling Problem
 280 Locations on 2D plane
 NN and NN-2 Opt
 All intersections removed

44
Reference

Thank You !
Robert Bosch, February 2009
mona-lisa100K.tsp
$1000 Prize Offered
[Can convert pictures into tours
using Voronoi diagrams]

45