10/30/22

Optimization

CS314 Software
Engineering

Finding shorter tours for our users is an application of heuristic algorithms used
to solve the Traveling Salesman Problem.

• Traveling Salesman Problem • Construction phase

– Shortest Hamiltonian cycle in – Nearest Neighbor, …
a graph, O(n!). • Improvement phase
– Exact algorithms will not work – 2-opt, 3-opt, …
for large tours
– Heuristic algorithms gain
speed at cost of tour quality
– often performed in two
phases

1
 10/30/22

A trip to these places is 306,617 km using the best of the nearest neighbor
solutions starting from each location

The same trip is 271,618 km optimizing each of the nearest neighbor solutions
with 2-opt. It is not the 2-opt improvement of the best nearest-neighbor.

2
 10/30/22

The trip is 264,376 km optimizing each of the nearest neighbor solutions with 3-
opt.

Construction

3
 10/30/22

The Construction phase may be done using 1, some, or all the locations as the
starting points since they often yield different solutions.

Nearest Neighbor
– O(n^2) for a single starting locations
– O(n^3) to try all starting locations

nearestNeighbor(cities) {
for each starting city (as time permits) - O(n)
add the starting city to the tour and remove from the list of unvisited cities
while there are unvisited cities remaining - O(n)
add the nearest unvisited city from the last city to the tour - O(n)
remove from the list of unvisited cities
return the tour with the shortest distance
}

Here are some simple test cases to transform using nearest-neighbor on a plane.
What about on a sphere? Do we need to test tours with 1, 2, or 3 places?

4
 10/30/22

Improvement

10

The Improvement phase is applied to each solution constructed. The optimized

best constructed tour will usually not be the best optimized constructed tour.

2-opt and 3-opt

– O(n^2) and O(n^3) per constructed tour.
– O(n^3) and O(n^4) for all possible constructed tours.

nearestNeighborWithOptimization(cities) {
for each starting city (as time permits) O(n)
add the starting city to the tour and remove from the list of unvisited cities
while there are unvisited cities remaining - O(n)
add the nearest unvisited city from the last city to the tour - O(n)
improve the tour with 2-opt or 3-opt (if time permits) - O(n?)
return the tour with the shortest distance
}

https://web.tuke.sk/fei-cit/butka/hop/htsp.pdf

11

5
 10/30/22

2-opt

12

The 2-opt algorithm reverses a portion of the solution if it results in a shorter

overall distance. And does this until there are no more improvements.
route 23 15 3 9 7 … 21 11 5 4 23
index 0 1 2 3 4 … n-4 n-3 n-2 n-1 n
i i+1 k k+1
i k i+1 k+1
i i+1 k k+1
i k i+1 k+1
i i+1 - … - k k+1
i k - … - i+1 k+1
i i+1 - - - … - - - k k+1
i k - - - … - - - i+1 k+1

0 <= i < i+1 < k < k+1 <=n

13

6
 10/30/22

A 2-opt algorithm that determines the distance improvement without creating a

copy of the tour.
2opt(route) {
improvement = true
while improvement {
improvement = false
for (i = 0; i <= n-3; i++) {
for (k = i + 2; k <= n-1; k++) {
if 2optImproves(route, i, k) {
2optReverse(route, i+1, k)
improvement = true
}}}}

2optImproves(route, i, k) { // is new leg distance less than current leg distance

return legdis(route,i,k)+legdis(route,i+1,k+1) < legdis(route,i,i+1)+legdis(route,k,k+1) }

2optReverse(route, i1, k) { // reverse in place

while(i1 < k) {
temp = route[i1]
route[i1] = route[k]
route[k] = temp
i1++; k--
}
}

14

Here is the optimal solution and some test cases that can be transformed using
2-opt to eliminate the crossed lines.

Nearest-neighbor can also solve these test cases so you will need to test your 2-opt
algorithm independently of your nearest neighbor algorithm.

15

7
 10/30/22

3-opt

16

The 3-opt algorithm has seven cases that involve a combination three segment
reversals shown on the right.

(i1<k) (j1<k) (i1<j) #

original i i+1 --> j j+1 --> k k+1 0 0 0 0
2-opt i j <-- i+1 j+1 --> k k+1 0 0 1 1
2-opt i i+1 --> j k <-- j+1 k+1 0 1 0 2
2-opt i k <-- j+1 j <-- i+1 k+1 1 0 0 4
3-opt i j <-- i+1 k <-- j+1 k+1 0 1 1 3
3-opt i k <-- j+1 i+1 --> j k+1 1 0 1 5
3-opt i j+1 --> k j <-- i+1 k+1 1 1 0 6
3-opt i j+1 --> k i+1 --> j k+1 1 1 1 7

17

8
 10/30/22

The 3-opt algorithm is very similar to the 2-opt algorithm, operating on the
segments from i+1 to j, j+1 to k, and i+1 to k using the cases on previous slide.
3opt(route) {
improvement = true
while improvement {
improvement = false
for (i = 0; i <= n-3; i++) {
for (j = i+1, j < n-2; j++) {
for (k = j+1; k <= n-1; k++) {
reversals = 3optReversals(route, i, j, k)
if 3optReverseI1J(reversals) { 2optReverse(route, i+1, j) }
if 3optReverseJ1K(reversals) { 2optReverse(route, j+1, k) }
if 3optReverseI1K(reversals) { 2optReverse(route, i+1, k) }
if reversals > 0 { improvement = true }
}}}}}

3optReverseI1J(reversals) { return (reversals & 0b001) > 0 }

3optReverseJ1K(reversals) { return (reversals & 0b010) > 0 }
3optReverseI1K(reversals) { return (reversals & 0b100) > 0 }

3optReversals(route, i, j, k) { ... } // returns a number in range 0..7

18

Here are some 2-opt test cases (1 cross) and 3-opt test (2 or 3 cross) cases
where you eliminate the crossed lines in the optimization.

19

9
 10/30/22

Efficiency

20

Sachini Weerawardhana coined the term "ill-advised data structure use" while
helping students with their code in this course.

21

10
 10/30/22

This Slack discussion occurred a few days after this lecture.

22

The major challenge in optimization algorithms is space and time complexity.

Consider this 2-opt pseudocode from Wikipedia when number… is 1000.

https://en.wikipedia.org/wiki/2-opt

23

11
 10/30/22

Benchmarking algorithm operations to characterize performance before

implementation is often quite instructive.

N secs GCs Mbytes Benchmark

512 433 4,608 2,137,649 new ArrayList<Long>() create/add

512 384 3,261 1,513,080 new ArrayList<Long>(n) create/add

512 322 3,253 1,507,835 Long[n] create

512 716 2,658 1,232,746 new ArrayList<Long>(n) init/set

512 739 2,658 1,232,746 new ArrayList<Long>(n) init/clear/add

512 670 2,658 1,232,765 Long[n] init

512 177 1,188 552,991 long[n] create

512 150 - - long[n] init

24

Concurrency

25

12
 10/30/22

Let's flip a coin! A lot!

Monte Carlo methods rely on repeated

sampling to obtain numerical results,
using randomness to solve problems
having a probabilistic interpretation
such as the odds of a heads or tails on
a coin flip.

26

The package java.util.concurrent provides utility classes useful in concurrent

programming such as those highlighted here.

Set ExecutorService List

Coin newFixedThreadPool Future

implements
Callable
invokeAll

27

13
 10/30/22

The Coin class implements the Callable interface to flip a coin a specified
number of times. The call method has three variations.
class Coin implements Callable<Long> {
private final static int HEADS = 1;
private long flips, thread;

Coin(long f, long t) {
flips = f;
thread = t;
}

public Long call() {

// Random random = new Random(); //2
// ThreadLocalRandom random = ThreadLocalRandom.current(); //3
long heads = 0;
for (long i = 0; i < flips; i++)
//if ( random.nextInt(2) == HEADS) heads++; //2,3
heads += (int) (java.util.Math.random() * 2); //1
return heads;
}
}

28

The MonteCarlo class flips multiple coins concurrently by running a thread on

each available processor using a fixed thread pool.
class MonteCarlo {
private final static long flips = 3500000000L;
private final static long cores = Runtime.getRuntime().availableProcessors();
public static void main(String[] argv) {
long total = 0;
try {
Set<Callable<Long>> threads = new HashSet<>();
for (long i = 0; i < cores; i++) threads.add(new Coin(flips/cores, i));

ExecutorService executorService = Executors.newFixedThreadPool(cores);

List<Future<Long>> results = executorService.invokeAll(threads);
executorService.shutdown();

for (Future<Long> result : results) total += result.get();

} catch(Exception e) { }
System.out.printf("Heads: %d/%d\n", total, flips);
}
}

29

14
 10/30/22

Monte Carlo results for the three different random number generation methods
exhibit some interesting behavior with concurrency.
java.util.concurrent. java.util. java.lang.Math.
Threads
ThreadLocalRandom Random Random
1 6.51 39.65 78.823
2 3.36 20.12 271.619
3 1.83 13.81 402.885
4 1.69 10.35 564.534
5 1.35 8.29 919.358
6 1.13 6.90 1402.809
7 0.79 5.91 1908.080
8 0.69 5.18 ~

30

Random number generation method selection matters in a concurrent

environment (ThreadLocalRandom is 6X-7.5X JavaUtilRandom).
10
Relative Performance Improvement

9
8
7
6
5
4
3
java.util.concurrent.threadlocalrandom
2
java.util.random
1
threadlocalrandom / random
0
1 2 3 4 5 6 7 8
Threads on Mac M1 with 10 cores (8 performance)

31

15
 10/30/22

Random number generation method selection matters in a concurrent

environment (ThreadLocalRandom is 5-7X JavaUtilRandom).
30
ThreadLocalRandom
Relative Performance Improvement

25 JavaUtilRandom
Thread / Util
20

15

10

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40
Threads on a SuperMicro Xeon with 20 cores and hyper-threading

32

Comparing different versions of the Java on a Mac Mini i7 with 6 cores shows
the effect of hyperthreading.
12
11
10
9
8
7
6
5
4
3
2
1
0
1 2 3 4 5 6 7 8 9 10 11 12
cores j18i6thread j18i6 j15i6thread j15i6 j11i6thread j11i6 j8i6thread j8i6

33

16
 10/30/22

Comparing the Intel versus Apple Java 18 binaries available for the MacBook Pro
M1 Max (8 performance cores, 2 efficiency cores).
18
16
14
12
10
8
6
4
2
0
1 2 3 4 5 6 7 8 9 10
j18a10thread j18a10 Apple j18i10thread j18i10 Intel cores

34

17