0 Up votes0 Down votes

8 views43 pagesJul 20, 2017

© © All Rights Reserved

PDF, TXT or read online from Scribd

© All Rights Reserved

8 views

© All Rights Reserved

- EML5311_HW3_Ramin_Shamshiri.pdf
- Performance Evaluation of Route Cost for Wireless Sensor Networks with a Mobile Sink
- Sample chapter from Oxford A Level Mathematics for Edexcel Decision D2
- Comparing Simulated Annealing and Genetic Algorithm in Learning FCM
- MIT15_053S13_lec8.pdf
- 009 Informed Search 1_ Hill Climbing.pptx
- 15 - ALNS
- Function Diferential Matlab
- automatic2013zhou_tkde
- On Vt Design
- knap
- 04 - Relative Extrema.pdf
- Theory of Computation
- 55 Learning Objective Fort Lns
- Mat 575
- paper 03411740007001_Joana Maria Soares Sarmento_ kelas D..docx
- Kompendium
- -~arnabc-matalgop1.pdf
- CPM NetworkRules
- MIT15_053S13_lec18.pdf

You are on page 1of 43

and Heuristics

1

Quotes of the day

and we are born to it.

-- Thomas Harris

-- Donald Knuth

2

Heuristics

A heuristic is a technique designed for solving a

problem more quickly when classic methods are too

slow (from Wikipedia).

Todays lecture:

Heuristics illustrated on the traveling salesman problem.

Design principles for heuristics

Chances for practice

3

Traveling Salesperson Problem (TSP)

Find the shortest

distance tour

passing through

each node of the

network exactly

once.

cij = distance

from i to j.

http://www.math.uwaterloo.ca/tsp/

Courtesy of William Cook. Used with permission.

4

15,112 City Optimal Tour in Germany

(rotated)

5

Exercise: Try to find the best tour.

6

On solving hard problems

How did you select your tour?

it relied on visualization

perhaps you took a holistic view

solving hard problems?

7

The TSP is a hard problem

NP-hard (also NP-complete)

completeness before?

1. Yes.

2. No.

8

The TSP is a hard problem

Cannot bound the running time as less than nk for

any fixed integer k (say k = 15).

would be a polynomial time algorithm for every

NP-complete problem.

problem?

9

100 n15 vs. n!

Suppose that we could carry out 1 sextillion steps per

second (1021).

n = number of cities. (n-1)! tours.

compare time for 100 n15 steps vs. n! steps.

Remark: 100 n15 steps is NOT practically efficient.

n = 20, 3.27 seconds 2.4 milliseconds

n = 25, 1.5 minutes 4.2 hours

n = 30 24 minutes 8,400 years

n = 35 4 hours 326 billion years

10

Two types of Heuristics

scratch (starting with nothing).

Often called greedy heuristics. Each step

looks good, but it doesnt look ahead.

starts with a solution, and then tries to improve

the solution, usually by making small changes in

the current solution.

11

An easy construction heuristic:

Nearest unvisited neighbor

12

Animations from Stephen Mertens

13

Can we do better in construction?

Class exercise: try to develop a heuristic in

which we add one city at a time, but the next city

can be added anywhere in the tour (not just the

beginning or the end.)

Below is the beginning part of a tour

14

Courtesy of Stephan Mertens. Used with permission.

15

Which do you believe will give shorter

length tours:

16

Facility location problems.

Choose K facilities so as to minimize total

distance from customers to their closest facility.

example with three facilities

17

Exercise: try developing a good

solution where there are 2 facilities

18

Exercise: Develop a construction heuristic

for the facility location problem

Data: locations in a city.

cij = distance from i to j

19

Mental Break

20

Improvement heuristics

Improvement heuristics start with a feasible

solution and look for an improved solution that

can be found by making a very small number of

changes.

This will be made more formal

obtained from the other by deleting two edges

and adding two edges.

21

2-opt neighborhood search

A TSP tour T is called 2-optimal if there is no 2-

adjacent tour to T with lower cost than T.

lower cost than the current tour. If one is found,

then it replaces the current tour. This continues

until there is a 2-optimal tour.

22

An improvement heuristic: 2 exchanges.

Look for an improvement obtained by deleting

two edges and adding two edges.

3 10

T

2 1

7

8 3

4 2

4

5

6

1

9

10

9 8

6 7

5

23

After the two exchange

Deleting arcs (4,7) and (5, 1) flips the subpath from node 7 to node 5.

3 10

T1 T2

2 1 1

7

8 3 3

4 2 2

4 4

5 7

6 8

1

9 10

10 9

9 8 6

6 7 5

5

24

After the two exchange

Deleting arcs (1,3) and (2, 4) flips the subpath from 3 to 2.

3 10

T2 T3

2

7 1 1

8 3 2

4 2 3

4 4

7 7

1 8 8

10 10

9 9

9 6 6

6 5 5

5

25

After the final improving 2-exchange

Deleting arcs (7,8) and (10, 9) flips the subpath from 8 to 10.

3 10

T3 T4

2

7 1 1

8 2 2

4 3 3

4 4

7 7

1 8 10

10 8

9 9

9 6 6

6 5 5

5

26

2-exchange heuristic (also called 2-opt)

27

3-opt neighborhood

Two TSP tours are called 3-adjacent if one can be

obtained from the other by deleting three edges

and adding three edges.

adjacent tour to T with lower cost than T.

lower cost than the current tour. If one is found,

then it replaces the current tour. This continues

until there is a 3-optimal tour.

28

On Improvement Heuristics

Improvement heuristics are based on

searching a neighborhood . Let N(T) be

the neighborhood of tour T.

that can be obtained from T deleting two

arcs and inserting two arcs.

Improvement heuristic:

start with tour T

if there is a tour T N(T) with c(T) < c(T), then

replace T by T and repeat

otherwise, quit with a locally optimal solution.

29

How good are improvement heuristics?

Implementers

had to be very

clever to

achieve these

running times.

error bound

2-opt

3-opt

Facility location problems.

Class exercise. Suppose we want to solve a

facility location problem with 3 facilities. Design a

neighborhood search heuristic.

31

Using Randomization

An important idea in algorithm development:

randomization

way of using the same approach multiple times

and getting different answers. (Then choose the

best).

have an approach that is more likely to converge

to a good solution

32

On the use of randomization

Remark: 2-exchanges will behave differently

depending on the starting solution.

Start with a random tour

use the 2-exchange neighborhood until obtaining

a local optimum.

33

One difficulty: random tours are terrible.

34

2-opt heuristic starting from a random

solution.

35

Another use of randomization

Replace the nearest neighbor tour with the

following: at each iteration, visit either the

closest neighbor or the second or third closest

neighbors. Choose each with 1/3 probability.

good and may be a better starting point than a

totally random tour.

36

Other approaches to heuristics

The metaphor based approach to the design of

heuristics

simulated annealing

genetic algorithms

neural networks

ant swarming

That is, look for something that seems to work

well in nature, and then try to simplify it so that it

is practical and helps solve optimization

problems.

37

Simulated Annealing

A randomization heuristic based on neighborhood

search that permits moves that make a solution worse.

It is based on an analogy with physical annealing.

and have it reach a low

Photo removed due to energy state, one

copyright restrictions.

should cool it slowly.

A glass annealing oven.

www.carbolite.com

European Journal of Operational Research 46 (1990) 271-281. (PDF)

38

Simulated Annealing: a variation on local search.

and a starting temperature T = T0

( c( x)-c( y))/T

If c(y) > c(x), replace x by y with probability e

(e.g., T := .99T)

Yes

Is T still No

large enough Quit.

39

A typical acceptance rate

as a function of temp

Min Fang. All rights reserved. This content is excluded from our Creative Commonslicense. For more

information, see http://ocw.mit.edu/help/faq-fair-use/.

Source: Fang, M. Ch. 3 in "Layout Optimization for Point-to-Multi-Point Wireless Optical Networks Via

Simulated Annealing & Genetic Algorithm." Senior Project Report, University of Bridgeport, 2000.

40

A typical graph of Temp vs. cost.

Min Fang. All rights reserved. This content is excluded from our Creative Commonslicense. For more

information, see http://ocw.mit.edu/help/faq-fair-use/.

Source: Fang, M. Ch. 3 in "Layout Optimization for Point-to-Multi-Point Wireless Optical Networks Via

Simulated Annealing & Genetic Algorithm." Senior Project Report, University of Bridgeport, 2000.

41

Summary

Two types of heuristics

Use of randomization

Simulated annealing

hard problems.

42

MIT OpenCourseWare

http://ocw.mit.edu

Spring 2013

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

- EML5311_HW3_Ramin_Shamshiri.pdfUploaded byRaminShamshiri
- Performance Evaluation of Route Cost for Wireless Sensor Networks with a Mobile SinkUploaded byJournal of Computing
- Sample chapter from Oxford A Level Mathematics for Edexcel Decision D2Uploaded byOxford University Press Children and Schools
- Comparing Simulated Annealing and Genetic Algorithm in Learning FCMUploaded byFabián Seguel
- 009 Informed Search 1_ Hill Climbing.pptxUploaded byRashdeep Singh
- 15 - ALNSUploaded byAirs Xue
- Function Diferential MatlabUploaded byNadhilah
- automatic2013zhou_tkdeUploaded byDivya Hajeri
- On Vt DesignUploaded bykasim_1983
- knapUploaded byFahad Ahmed
- 04 - Relative Extrema.pdfUploaded byDanica Balmeo
- Theory of ComputationUploaded byRehan Shabbir
- 55 Learning Objective Fort LnsUploaded byzym1003
- Mat 575Uploaded byHf Hefney
- paper 03411740007001_Joana Maria Soares Sarmento_ kelas D..docxUploaded byMaria
- KompendiumUploaded bysamygamal
- -~arnabc-matalgop1.pdfUploaded byMaria Seltana
- CPM NetworkRulesUploaded bygkrishna118

- MIT15_053S13_lec8.pdfUploaded byShashank Singla
- MIT15_053S13_lec18.pdfUploaded byShashank Singla
- MIT15_053S13_lec19.pdfUploaded byShashank Singla
- MIT15_053S13_lec16.pdfUploaded byShashank Singla
- MIT15_053S13_lec9.pdfUploaded byShashank Singla
- MIT15_053S13_lec2Uploaded byShashank Singla
- MIT15_053S13_lec15.pdfUploaded byShashank Singla
- MIT15_053S13_lec14.pdfUploaded byShashank Singla
- New Business StrategyUploaded byShashank Singla
- MIT15_053S13_lec13.pdfUploaded byShashank Singla
- MIT15_053S13_lec7.pdfUploaded byShashank Singla
- MIT15_053S13_lec10.pdfUploaded byShashank Singla
- MIT15_053S13_lec12.pdfUploaded byShashank Singla
- MIT15_053S13_lec11.pdfUploaded byShashank Singla
- MIT15_053S13_lec6.pdfUploaded byShashank Singla
- MIT15_053S13_lec1.pdfUploaded byShashank Singla
- MIT15_053S13_lec3.pdfUploaded byShashank Singla
- MIT15_053S13_lec5.pdfUploaded byShashank Singla
- MIT15_053S13_lec4.pdfUploaded byShashank Singla
- Lscm IndiaUploaded byShashank Singla
- Corporate ValuationUploaded byShashank Singla

- 17150_FULLTEXTUploaded bykkkprot
- extra credit 2 ironyUploaded byapi-292792461
- We Exult in God Through Our Lord Jesus Christ (1)Uploaded byvasile_silion
- 6745473-MiracleofMiraj(1)Uploaded bydr_zafar
- James L. Meagher "How Christ Said the First Mass"Uploaded byGualterio Curzio
- 0deec5315ccd119e8e000000Uploaded bysofia
- Rebecca F Doherty Financial Disclosure Report for 2009Uploaded byJudicial Watch, Inc.
- Fortunato r. Baron v. Epe TransportUploaded byAlyssa Mateo
- Faculty Motivation to Do ResearchUploaded byAixa Dee Madialis
- Newcomer Psychological Contracts and SocialisationUploaded byCRNIPAS
- Report TextUploaded byduayuintan_5791
- Tranfer of LearningUploaded byDragan Ristovski
- Cahill-Physical Mathematics What Physicists and Engineers Need to KnowUploaded byrangvanrang
- bar q and a.docxUploaded byKaren Joyce Asuncion
- Tutorial Questions Spring 2014Uploaded bylalaran123
- Prepare Economy With Mrunal’s 7-PillarsTM ApproachUploaded byAjish Rajendran
- NPA - RBI CircularUploaded bySivasankar Rampa
- chapter 10-11 aotUploaded byapi-350127360
- megha-1Uploaded byMaria Khan
- Outline Commercial LawUploaded byKriska Herrero Tumamak
- Attachment ReportUploaded byAmiani 'Amio' David
- Philosophy After MarxUploaded byIvan Dias Martins
- IRFS630BUploaded byjmbernal7487886
- basit2002Uploaded byFerOrtiz
- FableUploaded byYudha Ikhsan
- 700DCP Customer Alignment Training Procedure Final1Uploaded byXcopy
- EOCQ_ans_6 biologyUploaded bySabQilah
- 4024_w11_ms_11.pdfUploaded byNaveed Akram
- Part 1 the Ruggedness Test in HPLCUploaded byNestor Armando Marin Solano
- 1. New Upload Function in MM17 and MASS Transaction _ SAP BlogsUploaded bynabigcs

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.