Professional Documents
Culture Documents
Tackling Difficult Combinatorial Problems
Tackling Difficult Combinatorial Problems
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-1
Exact Solution Strategies
exhaustive search (brute force)
• useful only for small instances
dynamic programming
• applicable to some problems (e.g., the knapsack problem)
backtracking
• eliminates some unnecessary cases from consideration
• yields solutions in reasonable time for many instances but worst case is
still exponential
branch-and-bound
• further refines the backtracking idea for optimization problems
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-2
Backtracking
Construct the state-space tree
• nodes: partial solutions
• edges: choices in extending partial solutions
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-3
Example: n-Queens Problem
1 2 3 4
1 queen 1
2 queen 2
3 queen 3
4 queen 4
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-4
State-Space Tree of the 4-Queens Problem
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-5
Example: Hamiltonian Circuit Problem
a b
c f
d e 0
with 3 w/o 3
3 0
with 5 w/o 5 with 5 w/o 5
8 3 5 0
with 6 w/o 6 with 6 w/o 6 with 6 w/o 6 X
0+13<15
14 8 9 3 11 5
X with 7 w/o 7 X X X X
14+7>15 9+7>15 3+7<15 11+7>14 5+7<15
15 8
solution X
8<15
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-6
Branch-and-Bound
An enhancement of backtracking
Example
Job 1 Job 2 Job 3 Job 4
Person a 9 2 7 8
Person b 6 4 3 7
Person c 5 8 1 8
Person d 7 6 9 4
Lower bound: Any solution to this problem will have total cost
at least: 2 + 3 + 1 + 4 (or 5 + 2 + 1 + 4)
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-8
Example: First two levels of the state-space tree
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-9
Example (cont.)
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-10
Example: Complete state-space tree
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-11
Example: Traveling Salesman Problem
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-12
Approximation Approach
Apply a fast (i.e., a polynomial-time) approximation algorithm
to get a solution that is not necessarily optimal but hopefully
close to it
Accuracy measures:
accuracy ratio of an approximate solution sa
r(sa) = f(sa) / f(s*) for minimization problems
r(sa) = f(s*) / f(sa) for maximization problems
where f(sa) and f(s*) are values of the objective function f for
the approximate solution sa and actual optimal solution s*
sa : A – B – C – D – A of length 10
6 2
3 3
s* : A – B – D – C – A of length 8
D C
1
Stage 2: Add next edge on the sorted list to the tour, skipping
those whose addition would’ve created a vertex of
degree 3 or a cycle of length less than n. Repeat
this step until a tour of length n is obtained
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-15
Twice-Around-the-Tree Algorithm
Stage 1: Construct a minimum spanning tree of the graph
(e.g., by Prim’s or Kruskal’s algorithm)
9 9
8 4 7 11
b d b d
8
6 10
c c
Walk: a – b – c – b – d – e – d – b – a Tour: a – b – c – d – e – a
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-17
Christofides Algorithm
Stage 1: Construct a minimum spanning tree of the graph
9 9
8 4 7 11
b d
8
6 10
c
a e a e
4 4 7 11
4 9 7 11
b d b
8 d
6 6
c c
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-19
Euclidean Instances
Theorem If P ≠ NP, there exists no approximation algorithm
for TSP with a finite performance ratio.
Definition An instance of TSP is called Euclidean, if its
distances satisfy two conditions:
1. symmetry d[i, j] = d[j, i] for any pair of cities i and j
2. triangle inequality d[i, j] ≤ d[i, k] + d[k, j] for any cities i, j, k
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-20
Local Search Heuristics for TSP
Start with some initial tour (e.g., nearest neighbor). On each
iteration, explore the current tour’s neighborhood by exchanging a
few edges in it. If the new tour is shorter, make it the current
tour; otherwise consider another edge change. If no change yields
a shorter tour, the current tour is returned as the output.
Example of a 2-change
C4 C3 C4 C3
C1 C2 C1 C2
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-21
Example of a 3-change
C5 C4
C6 C3
C1 C2
C5 C4 C5 C4
C6 C3 C6 C3
C1 C2 C1 C2
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-22
Empirical Data for Euclidean Instances
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-23
Greedy Algorithm for Knapsack Problem
Step 1: Order the items in decreasing order of relative values:
v1/w1… vn/wn
Step 2: Select the items in this order skipping those that don’t
fit into the knapsack
Accuracy
RA is unbounded (e.g., n = 2, C = m, w1=1, v1=2, w2=m, v2=m)
yields exact solutions for the continuous version
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-24
Approximation Scheme for Knapsack Problem
Accuracy
Number of extra bins never exceeds optimal by more than
70% (i.e., RA ≤ 1.7)
Empirical average-case behavior is much better. (In one
experiment with 128,000 bins, the relative error was found
to be no more than 2%.)
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-26
Bin Packing: First-Fit Decreasing Algorithm
Accuracy
Number of extra bins never exceeds optimal by more than
50% (i.e., RA ≤ 1.5)
Empirical average-case behavior is much better, too
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-27
Numerical Algorithms
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-28
Principal Accuracy Metrics
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-29
Two Types of Errors
truncation errors
• Taylor’s polynomial approximation
ex 1 + x + x2/2! + … + xn/n!
round-off errors
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-30
Solving Quadratic Equation
subtractive cancellation
use alternative formulas (see p. 411)
use double precision for D = b2 - 4ac
Nonlinear equation f(x) = 0 can have one, many, infinitely many, and
no roots at all
Useful:
• sketch graph of f(x)
• separate roots
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-32
Three Classic Methods
Three classic methods for solving nonlinear equation
f(x) = 0
in one unknown:
bisection method
Newton’s method
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-33
Bisection Method
Based on
Theorem: If f(x) is continuous on ax b and f(a) and f(b)
have opposite signs, then f(x) = 0 has a root on a < x < b
binary search idea
f(x)
x
a x1 b
.
f(x)
an
. .
xn
. bn
x
. f(xn)
. .
xn+1 x n
x
www.studentsfocus.com
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 12 12-38