P. 1
Computer Algorithms, Third Edition, Solutions to Selected Exercises

Computer Algorithms, Third Edition, Solutions to Selected Exercises

|Views: 608|Likes:
Published by David Antony
Sara Baase
Allen Van Gelder
Sara Baase
Allen Van Gelder

More info:

Categories:Types, Brochures
Published by: David Antony on Jul 12, 2013
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

04/26/2015

pdf

text

original

Computer Algorithms, Third Edition, Solutions to Selected Exercises

Sara Baase Allen Van Gelder

February 25, 2000

ii

I NTRODUCTION
This manual contains solutions for the selected exercises in Computer Algorithms: Introduction to Design and Analysis, third edition, by Sara Baase and Allen Van Gelder. Solutions manuals are intended primarily for instructors, but it is a fact that instructors sometimes put copies in campus libraries or on their web pages for use by students. For instructors who prefer to have students work on problems without access to solutions, we have chosen not to include all the exercises from the text in this manual. The included exercises are listed in the table of contents. Roughly every other exercise is solved. Some of the solutions were written specifically for this manual; others are adapted from solutions sets handed out to students in classes we taught (written by ourselves, teaching assistants, and students). Thus there is some inconsistency in the style and amount of detail in the solutions. Some may seem to be addressed to instructors and some to students. We decided not to change these inconsistencies, in part because the manual will be read by instructors and students. In some cases there is more detail, explanation, or justification than a student might be expected to supply on a homework assignment. Many of the solutions use the same pseudocode conventions used in the text, such as: 1. Block delimiters (“ ” and “ ”) are omitted. Block boundaries are indicated by indentation. 2. The keyword static is omitted from method (function and procedure) declarations. All methods declared in the solutions are static. 3. Class name qualifiers are omitted from method (function and procedure) calls. For example, x = cons(z, x) might be written when the Java syntax requires x = IntList.cons(z, x). 4. Keywords to control visibility, public, private, and protected, are omitted. 5. Mathematical relational operators “ ,” “ ,” and “ ” are usually written, instead of their keyboard versions. Relational operators are used on types where the meaning is clear, such as String, even though this would be invalid syntax in Java. We thank Chuck Sanders for writing most of the solutions for Chapter 2 and for contributing many solutions in Chapter 14. We thank Luo Hong, a graduate student at UC Santa Cruz, for assisting with several solutions in Chapters 9, 10, 11, and 13. In a few cases the solutions given in this manual are affected by corrections and clarifications to the text. These cases are indicated at the beginning of each affected solution. The up-to-date information on corrections and clarifications, along with other supplementary materials for students, can be found at these Internet sites: ftp://ftp.aw.com/cseng/authors/baase http://www-rohan.sdsu.edu/faculty/baase http://www.cse.ucsc.edu/personnel/faculty/avg.html

c Copyright 2000 Sara Baase and Allen Van Gelder. All rights reserved. ­ Permission is granted for college and university instructors to make a reasonable number of copies, free of charge, as needed to plan and administer their courses. Instructors are expected to exercise reasonable precautions against further, unauthorized copies, whether on paper, electronic, or other media. Permission is also granted for Addison-Wesley-Longman editorial, marketing, and sales staff to provide copies free of charge to instructors and prospective instructors, and to make copies for their own use. Other copies, whether paper, electronic, or other media, are prohibited without prior written consent of the authors.

List of Solved Exercises
1 Analyzing Algorithms and Problems: Principles and Examples 1.1 . 1.2 . 1.4 . 1.6 . 1.8 . 1.10 1.12 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 2 2 3 3 3 1.13 1.15 1.18 1.20 1.22 1.23 1.25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4 4 4 4 4 5 1.28 1.31 1.33 1.35 1.37 1.39 1.42 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 6 6 6 6 6 7 1.44 1.46 1.47 1.48 1.50 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 7 7 7 8

2

Data Abstraction and Basic Data Structures 2.2 . . . . . . . 2.4 . . . . . . . 2.6 . . . . . . . 9 9 9 2.8 . . . . . . . 2.10 . . . . . . 2.12 . . . . . . 9 11 11 2.14 . . . . . . 2.16 . . . . . . 2.18 . . . . . . 12 13 14

9

3

Recursion and Induction 3.2 . . . . . . . 3.4 . . . . . . . 17 17 3.6 . . . . . . . 3.8 . . . . . . . 17 18 3.10 . . . . . . 3.12 . . . . . . 18 18

17

4

Sorting 4.2 . 4.4 . 4.6 . 4.9 . 4.11 4.13 4.15 4.17 4.19 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 19 19 19 19 20 20 20 21 4.21 4.23 4.25 4.26 4.27 4.29 4.31 4.34 4.35 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 21 22 22 23 23 23 24 24 4.37 4.40 4.42 4.44 4.45 4.46 4.48 4.49 4.51 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 24 24 25 25 25 25 25 26 4.53 4.55 4.57 4.59 4.61 4.63 4.65 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19 26 27 27 28 28 29 29

5

Selection and Adversary Arguments 5.2 . . . . . . . 5.4 . . . . . . . 5.6 . . . . . . . 31 32 32 5.8 . . . . . . . 5.10 . . . . . . 5.12 . . . . . . 33 34 34 5.14 . . . . . . 5.16 . . . . . . 5.19 . . . . . . 34 34 35 5.21 . . . . . . 5.22 . . . . . . 5.24 . . . . . .

31 35 36 37

6

Dynamic Sets and Searching 6.1 . 6.2 . 6.4 . 6.6 . 6.8 . 6.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 39 40 40 41 41 6.12 6.14 6.16 6.18 6.20 6.22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 43 45 45 45 46 6.24 6.26 6.28 6.30 6.32 6.34 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 47 47 47 48 49 6.36 . . . . . . 6.37 . . . . . . 6.40 . . . . . .

39 49 49 50

.21 . . 12. . . . . . . . . . 93 93 94 94 96 96 13. .43 7. . . . . .18 13. . . . . . .10 8. . . . . . .6 . . . . .26 . . . . . . 12. . . .14 . . . . .30 13. . . . 12.16 8. .21 11. . . .1 . . . . . . 11. . . . . . . . . . . . . . . 9. . 51 51 51 51 51 52 52 7. 9. 84 84 84 84 11. . . . . . . . . . . . . . .6 . . . . . .24 .17 11. . . . .15 . . . . . . . . .4 . . . . . . . . . . . . . 65 65 65 67 8. . . . . .7 . . . . . . . . . . .8 .6 . . . . .14 13. . . . 73 74 75 75 10. . . . .3 . . . . . . .22 7. . . . . .2 11. . 7. 13. . . 69 72 10 Dynamic Programming 10.37 13. . . . . . . . . . . . . . . . . . 89 89 91 91 91 91 13. . . .12 . . . .28 13. . .20 7. . .47 7. . . . .41 7.21 13. . . . . . . .20 13. . .10 13. . . . .2 .7 . . . . . . 12. .1 8. . 53 53 53 54 54 54 57 7. . . . . . . . 9. . 71 71 71 9.14 .7 . .14 7. . .8 . . . . . 10.8 . . . . .45 7. . . . . . 13. . . . 81 86 12 Polynomials and Matrices 12. . .28 7. . . . . . . . . . . .5 8. .iv List of Solved Exercises 7 Graphs and Graph Traversals 7. . .8 . . . . . . . .34 7. .16 10. . 69 70 71 9. . . . . . . 10. . . . . All-Pairs Shortest Paths 9. . . . . . . . . . . .4 . . . 7. .12 8. . 89 96 96 98 99 99 99 . . .2 10. .4 11. . . 8. . . . .49 . .10 7. . . . . . .19 10.32 7. .24 7. .16 13. . . . .16 7. . . . . . . . . . . 9. . . . . . . . 57 57 57 57 58 59 59 7. .3 8. . . . . . . .16 . . . .22 . . . . . . . . . 12. . . . . . .32 13. . 7. .25 .9 . .37 7. . . . . . . . . .12 11. 64 64 64 64 8. . . .17 . . . .6 . 87 87 87 12. . . . . .1 11. . . . .4 . . . . . . . . .10 10. . . . . . . . 8.23 .4 10. . . .23 . . . . . . . . 88 88 88 87 13 N P -Complete Problems 13. . .30 7. . . . . . . 72 72 72 9. . .26 13. . . . . . . . . . . 63 67 67 67 9 Transitive Closure. . . . . . . .4 . . . . . .10 . . . . . . . . . . . . 87 87 88 12. . . . . . . . . .49 . . .19 11.12 8 . . .39 13.12 . . . 13. . . . . .39 . . . 8. . . . . 81 81 81 83 11. .18 . . . .47 13. . . .35 . 7. . . . . . 13. .27 . .20 8. . . . . .27 . . . . .5 10. . . .2 . . . . . . . . . . . . .8 . .2 . 63 63 63 64 8. 92 92 92 93 93 93 13.18 7. . . .14 . . . .14 . . 51 59 59 59 60 60 61 Graph Optimization Problems and Greedy Algorithms 8. . 12. . . . . . . .10 11. . . . . .23 . . . . . . . . .18 10. 73 73 73 73 10. .42 13. . . . . . . 7.16 .8 . . . . . . . . . .18 8. . .10 . . . . . .26 .34 13.35 7. 73 78 79 11 String Matching 11. . . . . 84 85 85 85 11. . . . . . . 75 76 77 78 10.12 . . . . .12 10. 9.6 . . . . . . . . . . . . . . .44 13. . 9. . . . .40 7.

4 14.32 . . . . 100 13. 103 106 107 107 108 108 .2 14. . . . . .30 14. . 100 13. . . . . . . 101 13. . . 100 13. . . . . . . . . . . . . . . . . . . . .29 14. . . . . . 100 13. . . . . .10 14.11 14. . .24 .List of Solved Exercises 13. . . . . . . . . . . . 103 103 103 104 104 14. .7 14.61 . . . .20 14. .14 14.13 14. . . . . . . . . . . . .55 . . . . . . . . . . . 105 106 106 106 106 14.19 14. . . . . . . . 99 13. .8 . . . . . .53 . .25 14. . .54 . .22 14. . .5 14. . .27 14. . 104 104 104 105 105 14. . . .59 . . . . . . . . .18 14. 101 v 14 Parallel Algorithms 14. . . . . .57 . . .51 .16 . . .

vi List of Solved Exercises .

middleName. String eMail.lastName = n.copy(p. p2. class Personal public static class Name String String String public firstName. public static PhoneNumber copy(PhoneNumber n) /* similar to Name.phone = PhoneNumber.firstName. . p2. static Name copy(Name n) Name n2. PhoneNumber phone.copy(p. n2. lastName. public static Personal copy(Personal p). int number.2 in the text) or after (as here). return n2.7 gives an alternative to spelling out all the instance fields in the copy methods (functions).address = Address. Address address.Chapter 1 Analyzing Algorithms and Problems: Principles and Examples Section 1. public static class Address String String String public street.copy() */ Name name. static Address copy(Address a) /* similar to Name.eMail. p2.address). Appendix A. n2. Personal p2.name = Name. return p2.middleName = n.firstName = n.middleName. city.eMail = p.name).lastName. p2. n2.1 It is correct for instance fields whose type is an inner class to be declared before that inner class (as in Figure 1. state.2: Java as an Algorithm Language 1.copy() */ public static class PhoneNumber int areaCode.copy(p.phone). int prefix.

) k 1 k 1.2 3.0 1. we have n 1 k n 1 k 1 ´n   1µ! k!´n   1   kµ! ´n   1µ!´n   kµ k!´n   kµ!   1µ!´kµ k!´n   kµ!   1 µ! ´k   1µ!´n   kµ! ´n ´n ´n Add them giving: For 0 n k we use the fact that a 0 whenever a b. twoToTheX *= 2. the whole set. 1.0 2.4 It suffices to show: logc x logb c Consider b raised to each side. while (twoToTheX < n+1) x += 1. The values computed by this procedure for small n and the approximate values of lg´n · 1µ are: n 0 1 2 3 4 5 6 7 8 9 x 0 1 2 2 3 3 3 3 4 4 lg´n · 1µ 0. twoToTheX = 1. The solution is based on the fact that 2x 1 x = 0.6 2.0 3.6 2.6 Let x right side   ¡   1 µ! ´n µ k!´n   kµ! n k logb x logb c logc x µ blogb c logc x b logb x ´b clogc x x x lg´n · 1µ . (There is no way to choose more elements than there are in b  n 1¡    1¡  n¡ 0 in all these cases.2 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples Section 1.3: Mathematical Background 1. n k. bleft side b So left side = right side. If n k. again confirming the equation. (We need the fact that 0! 1 when n k 1. n·1 2x .0 1.8 3. return x.3 .2 For 0 k n.3 2. confirming the equation. If n  n 1¡  n¡ and are both 1.) Thus k k 1 and k are both 0.

13 The entry in row i. so the probability is 0.12 We assume that the probability of each coin being chosen is 1/3. CH and CT cause a majority to be “heads”. . BH . 1. Define the elementary events. Call the coins A.Chapter 1 1. C is chosen and flipped and comes out “heads”. and C. B. so the probability is 2/3. so the probability is 1/3. b) No event causes a majority to be “heads”. as follows. C is chosen and flipped and comes out “tails”. A is chosen and flipped and comes out “tails”. By direct counting: C A D A C A D A B and D B and D B and D B and D Cµ Cµ Cµ Cµ Pr´A C and A B and D Pr´A B and D Cµ Cµ 3 24 6 24 3 24 6 24 1 24 6 24 1 6 5 24 6 24 3 6 3 6 5 6 1 2 1 2 Pr´A D C and A Bµ Pr´A B and D Cµ Pr´A B C and D Cµ Pr´A B and D Cµ Pr´A B D Cµ Pr´A B and D Cµ 1.8 Analyzing Algorithms and Problems: Principles and Examples 3 Pr´S T µ The second equation is similar. D3 beats D4 . B is chosen and flipped and comes out “heads”. AH AT BH BT CH CT A is chosen and flipped and comes out “heads”. a) BH and CH cause a majority to be “heads”. each having probability 1/6. B is chosen and flipped and comes out “tails”. Pr´S and T µ Pr´T µ Pr´SµPr´T µ Pr´T µ Pr´Sµ 1. that the probability that it shows “heads” after being flipped is 1/2 and that the probability that it shows “tails” after being flipped is 1/2. and D4 beats D1 .10 We know A B and D Pr´A Pr´A Pr´B Pr´B C. c) AH . column j is the probability that Di will beat D j . D2 beats D3 . ¼   12 36 18 36 22 36   12 36 20 36 22 36   12 36 18 36 22 36   12 36 16 36 22 36 ½ Note that D1 beats D2 .

1. Also. f ¼ ´yµ ´ f ´zµ 0.4 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples 2n3 ·3n2 ·n 6 1. “is logically equivalent to”. Since f ´xµ is differentiable. Eq. We need to show that f ´wµ f ´zµ. The base case is n 0. it is continuous. c) c.20 Let abbreviate the phrase. i 1 ∑ i2 n n 1 i 1 2n3 ∑ i2 · n2 2´n   1µ3 · 3´n   1µ2 · n   1 2 ·n 6 6 2n3   3n2 · n 6n2 · 6 6   6n2 · 6n   2 · 3n2   6n · 3 · n   1 · n2 2n3 · 3n2 · n 6 1. assume the formula holds for n   1. the upper limit of the sum. Then ∑0 i 1i So the equation holds for the base case. they are: Comparisons involving K : Comparisons involving index: Additions: Assignments to index: 1. 1. Therefore. n n·1 n n·1 b. By this theorem there is some point y. 1. for which f ¼ ´y µ By the hypothesis of the lemma. else median = b.22 The total number of operations in the worst case is 4n · 2. that is. f ´zµ   f ´wµ 0. 0.18 Consider any two reals w z. f ´zµ   f ´wµ 0.21) (by DeMorgan’s law. 1. x´A´xµ µ B´xµµµ x x ´A´xµ ´ µ B´xµµ B´xµµ A´xµ B´xµµ (by Eq.15 2 The proof is by induction on n. ´z   wµ   f ´wµµ ´z   wµ 0.23 a) if (a < b) if (b < c) median = else if (a < median = else median = else if (a < c) median = a.23) x ´A ´x µ Section 1. We call upon the Mean Value Theorem (sometimes called the Theorem of the Mean). For n 0.4: Analyzing Algorithms and Problems 1. which can be found in any college calculus text. else if (b < c) median = c. a.24) (by Eq. We use the identity ´ A A as needed. . such that w y z. and 0.

Analysis of Solution 2 requires material introduced in Chapter 3. Then find the maximum among the larger elements using Algorithm 1. /** Precondition: n > 0. Section 1. More careful analysis verifies the result 3n 2   2 for all n. average = 2 2 3. break the set in halves. Pair up the entries and find the larger of each pair.25 Solution 1. one element is not examined ´ n 2 comparisons). i <= n-3. and compare the smallest keys from each half to find the smallest overall.3.) If there are at most two entries in the set. is W ´nµ W ´nµ 1 for n 2 for n 2 2W ´n 2µ · 2 The recursion tree can be evaluated directly. if (odd(n)) min = E[n-1]. This is the largest entry in the set. = E[i+1]. else max = E[n-2]. 1. max = E[n-1].3. This is the smallest entry in the set. Whether n is odd or even.5: Classifying Functions by Their Asymptotic Growth Rates 1.Chapter 1 b) D is the set of permutations of three items. c) Worst case = 3. The recurrence equation for this procedure. and recursively find the smallest and largest in each half. This leads to the total of 3n 2   2 for the special case that n is a power of 2. i = i+2) if (E[i] < E[i+1]) if (E[i] < min) min = if (E[i+1] > max) max else if (E[i] > max) max = if (E[i+1] < min) min E[i]. if n is odd. (But most of them cannot show formally that it does roughly 3n 2 comparisons. When we assign this problem after covering Divide and Conquer sorting algorithms in Chapter 4. the total is 3 2 ´n   1µ . Otherwise. The result can also be proven by induction. = E[i+1].28 p´nµ nk ak 1 n a k  2 n2 a1 n k  1 a0 nk n ∞ lim n ∞ lim ak · · · · · ak 0 . Analyzing Algorithms and Problems: Principles and Examples 5 d) Three comparisons are needed in the worst case because knowing the median of three numbers requires knowing the complete ordering of the numbers. Solution 2. compare them to find the smaller and larger. Then find the minimum among the smaller elements using the appropriate modification of Algorithm 1. Then compare the largest keys from each half to find the largest overall. E[i]. It is important that the nonrecursive costs in the n 2 leaves of this tree are 1 each. The nonrecursive costs in the n 2   1 internal nodes are 2 each. again including the unexamined element if n is odd ( ´n · 1µ 2   1 comparisons). */ for (i = 0. including the unexamined element if n is odd ( ´n · 1µ 2   1 comparisons). assuming n is a power of 2. min = E[n-1]. max = E[n-1]. many students give the following Divide and Conquer solution. else if (E[n-2] < E[n-1]) min = E[n-2]. The following algorithm interleaves the three steps.

33 Let f ´nµ n. Since for any f .35 Property 1: Suppose f ¾ O´gµ. let h´nµ be constant on the interval kk n ´´k · 1µk·1   1µ and let h´nµ kk on this interval. Therefore. h´nµ f ´nµ kk ´´k · 1µk·1   1µ. The other direction is proved similarly. as well as examples using monotonic functions. For simplicity we show a counter-example in which a nonmonotonic function is used.9 of the text gives transitivity. h´nµ ¾ O´ f ´nµµ   Θ´ f ´nµµ.37 n en ln 2 . we find that the derivative of 2 is c2 . using the chain rule. we have 0 and Property 4: We show O´ f · gµ O´max´ f gµµ. Thus when n kk . Then for n n0 . Let c = ln2 0. h´nµ 2c max´ f gµ´nµ. Property 2 gives symmetry. But h´nµ ¾ Ω´ f ´nµµ. using L’Hˆ repeatedly. . so g ¾ Θ´ f µ. By Property 1. n lim nk ∞ 2n n lim knk 1 ∞ c2 n n lim k´k   1µnk 2 ∞ c2 2 n ¡¡¡ n ∞ ck 2 n lim k! 0 since k is constant. Property 2: f n 0 . so h´nµ ¾ Θ´ f ´nµµ. f reflexivity. g ¾ Ω´ f µ O´ f µ. ¾ Θ´gµ means f ¾ O´gµ Ω´gµ. There are c n0 such that for n n0 . There are c 0 and n0 such that for n g´nµ ´1 cµ f ´nµ. We will use L’Hˆ opital’s Rule. h´nµ c´ f · gµ´nµ.31 The solution here combines parts (a) and (b). so we need to differentiate 2n . f ´n µ cg´nµ. which tends to 0 as n gets large. The functions on the same line are of the same asymptotic order.7. Consider the function h´nµ: h´nµ n 1 for odd n for even n Clearly h´nµ ¾ O´ f ´nµµ. but when n ´k · 1µk·1   1. 1. Observe that 2n ´eln 2 µ n n n n opital’s Rule The derivative of e is e .6 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples 1. It remains to show that h´nµ ¾ o´ f ´nµµ. But this follows by the fact that h´nµ f ´nµ 1 for odd integers. The other direction is similar. lg lg n lg n ln n ´lg nµ2 Ôn n n lg n n 1 ·ε n2 n2 · lg n n3 n   n3 · 7n5 2 n  1 2 n en n! 1.39 1 n for odd n for even n n 1 for odd n for even n f ´nµ g ´n µ There are also examples using continuous functions on the reals. h´nµ f ´nµ 1. 1. For all k 1. Let h ¾ O´ f · gµ. Property 3: Lemma 1. With more difficulty h´nµ can be constructed to be monotonic. ¾ Θ´ f µ. Then for n n0 . Now. 1. so.

but this will give a good enough approximation for the average in general.4 (Binary Search) in the text. because mid might contain the key being searched for. which tests for exact equality when the range shrinks to a single entry. eliminating the terms for the gaps) gives an average of approximately lg n   1. write  x ln x as  1 . first. An extra base case is needed in lines 3–7.e. labeled with indexes 1 leaf is an output node for the case when x is not found. 10. E 4 . To keep the number of comparisons in O´log nµ. 12. 1. compare x to E 1 .6: Searching an Ordered Array 1. 8. The rightmost to the right. else index = -1. the search terminates. E 2k   1 (If x is found. 3. This procedure propagates that precondition into recursive calls. (The computation in Section 1.Chapter 1 Section 1. That is. if we can assume the precondition first last. (If E 1 x.) The probability that x is not in the array is 1   n m. else index = binarySearch(E. So the average for all cases is approximately n n n ´ lg n   1µ · ´1   µ lg n lg n   lg n m m m (Thus. else if (last == first) if (K == E[first]) index = first.3) with the assumption that x is in the array (i. 11.4 does not. 1. then lines 1–2 can be dispensed with. Compared to Algorithm 1. Analyzing Algorithms and Problems: Principles and Examples 7 int binarySearch(int[] E. Binary Search does roughly lg n comparisons.42 The revised procedure is: 1. 9.6. of course. 7.. k lg n µ. then do a binary search in the segment that must contain x if it is in the array at all. else int mid = (first + last) / 2. 6. .44 The sequential search algorithm considers only whether x is equal to or unequal to the element in the array being examined. under various assumptions. 1. the distance between elements examined in the first phase is doubled at each step. and the left subrange is increased from mid 1 to mid. The left child for a node labeled i is an output node labeled i. so we branch left in the decision tree for “equal” and branch right for “unequal.48 For x ln x 0. int first. lg n comparisons are done. K). Actually. int K) if (last < first) index = -1. this algorithm combines the tests of lines 5 and 7 into one test on line 10.3 assumes that n 2k   1 for some k. The tree will have a long path.) Then do the Binary Search in the range E 2k 1 · 1 using at most k   1 comparisons. examine E 0 .. Redoing the average analysis for Binary Search (Section 1. if (K E[mid]) index = binarySearch(E.” Each internal node contains the index of the element to which x is compared. We will find an element larger than x (perhaps maxint) after at most lg n · 1 probes (i. 4. mid.) 1. int last. down n. In this case (again assuming that n 2k   1µ.46 The probability that x is in the array is n m. E 2 . 5. E8 E 2k . last. 2.6. K). 13. with n internal nodes.) Thus the number of comparisons is at most 2k 2 lg n .47 We examine selected elements in the array in increasing order until an entry larger than x is found. in certain cases. return index.e. whereas the procedure of Algorithm 1. mid+1. By L’Hˆ opital’s rule the limit of this ratio as x ´ µ x 1 x 1 ´ 2µ x 0 is the same as the limit of x. 14.

1. 2. Round 1 Round 2 Round 3 Round 4 So four weighings suffice.50 The strategy is to divide the group of coins known to contain the fake coin into subgroups such that. after one more weighing. but we can do better by making three subgroups that are as equal as possible. 2 1. 1 (or 8. 8. no matter what the outcome of the weighing. the fake is in the third subgroup. 7) (or 3. 2) (or 1. two equal subgroups would work. If the two subgroups that are weighed have equal weight. 0) . 23. Then weigh two of those subgroups that have an equal number of coins. 8 3.8 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples 1. 3. 1. the subgroup known to contain the fake is as small as possible. 24 8. Obviously. 23. 8.

even if the implementation in Gorp. which can get somewhat complicated.3: Elementary ADTs — Lists and Trees 2. Trying to infer what the ADT operations do by looking at the implementation in Gorp. have a t most 2 lg n 1  1   1 nodes.root(aList).4 The proof is by induction on d . GorpTester. Solution 1. assume the formula holds for d   1. BinTree oldList) { return BinTree. but this is less natural. Solution 1.2. public static public static Object first(BinTree aList) { return BinTree. For d 0. } } . 2.2: ADT Specification and Design Techniques 2.nil. so all the objects are in the class BinTree. any tree with n nodes would have to have height at least lg´n · 1µ   1. } BinTree rest(BinTree aList) { return BinTree.) public class List { public static final BinTree nil = BinTree.2. Java is more strict. Solution 2. then the behavior of GorpTester. However.java would be preferable because the javadoc comments in it should contain the preconditions and postconditions for each operation. oldList). or almost full credit. For a given binary tree. as long as the ADT specifications do not change. (It is also possible to make all the right subtrees nil and use the left subtrees to contain the elements.6 A tree of height lg´n · 1µ 2 n d  1 2 ´2d  1 µ 2d   2 would.java changes. without worrying too much about details specific to Java. Solutions 2 and 3 have been compiled and tested in Java 1. We would give full credit. let nd denote the number of nodes at depth d . the depth of the node.8 The point of the exercise is to recognize that a binary tree in which all the left subtrees are nil is logically the same as a list. Section 2.java is not reliable. For d 0.buildTree(newElement.rightSubtree(aList). nd 2.2 Several answers are reasonable.java.java would be preferable because the javadoc utility (with an html browser) permits the ADT specifications and type declarations to be inspected without looking at the source code of Gorp. } public static BinTree cons(Object newElement. by Lemma 2. (In C.nil. to any solution that brings out that idea. Because each node at depth d   1 has at most 2 children.java will remain unchanged. Its virtue is simplicity. Therefore. the type discrepancy can be solved with a type cast. But 2lg n 1   1 n·1 1 n 2 lg n 1  1   1 ´ · µ ´ · µ ´ · µ Because that expression is less than n.Chapter 2 Data Abstraction and Basic Data Structures Section 2. so it is a reliable source of information. provided that they bring out that looking at the current implementation of the ADT is a bad idea.) We will give several solutions to demonstrate the range of possibilities. this solution doesn’t really meet the specifications of the List ADT. Gorp. BinTree. there is only 1 root node and 1 20 . This solution merely uses the BinTree functions and does not even return objects in the List class.

root(aList.listAsTree). but the fine points of Java are tangential to the purpose of the exercise.listAsTree). BinTree oldTree. newL. oldList)) == oldList. encapsulation. if (oldList == List. This solution creates objects in the List class. } public static List cons(Object newElement. BinTree. We assume that the BinTree class has been defined to permit subclasses with the appropriate protected nondefault constructor. return newL. so the following postcondition does not hold: rest( cons(newElement. public static final List nil = makeNil(). private static List makeNil() { List newL = new List(). in analogy with the nondefault IntList constructor in Fig.nil) oldTree = BinTree. } public static List rest(List aList) { List newL = new List(). which are covered in the appendix.listAsTree. } } The problem with this implementation is that rest creates a new object instead of simply returning information about an existing object. . However. } public static Object first(List aList) { return BinTree. return newL. This solution uses only Java features covered in the first two chapters of the text. This solution is included mainly to confirm the fact that it is possible to do sophisticated data abstraction with type safety. List oldList) { List newL = new List().nil.10 Chapter 2 Data Abstraction and Basic Data Structures Solution 2. oldTree). The technique follows appendix Section A. which is the binary tree that represents that list. but each List object has a single instance field. it has a subtle problem.nil. return newL.buildTree(newElement. public class List { BinTree listAsTree. The nondefault BinTree constructor is accessed by super in the code below.listAsTree = BinTree.6 in which the class IntList is extended to IntListA. else oldTree = oldList. There is no reasonable way to get around this problem without using subclasses. newL.rightSubtree(aList. This solution is the “most correct” for Java.listAsTree = BinTree. A.nil. Here we extend BinTree to List.listAsTree = BinTree. newL. and precise specifications in Java. Solution 3. which is pointed out after the code.11.

BinTree.Chapter 2 public class List extends BinTree { public static final List nil = (List)BinTree. TreeList. rsList). oldList). sList). TreeList. TreeList tuList = TreeList. TreeList. Tree v = Tree.nil). uList). public static Object first(List aList) { return BinTree.buildTree("p". TreeList qrsList = TreeList. TreeList. } protected List(Object newElement. Tree s = Tree. qrsList). Tree r = Tree.buildTree("q".nil).buildTree("u".buildTree("r".buildTree("t".nil).cons(s.cons(v. TreeList. } public static List rest(List aList) { return (List)BinTree.cons(u. List oldList) { List newList = new List(newElement. tuList).cons(t. . vList).buildTree("v". TreeList rsList = TreeList. TreeList.nil.nil). oldList).cons(r.root(aList).rightSubtree(aList). Tree q = Tree. TreeList. } public static List cons(Object newElement. } } Data Abstraction and Basic Data Structures 11 2. Tree p = Tree.cons(q. Tree u = Tree.10 Tree t = Tree. List oldList) { super(newElement.nil).nil. return newList.nil).nil).buildTree("s". TreeList uList = TreeList. TreeList vList = TreeList. TreeList sList = TreeList.

nil) { int ancestorData = InTreeNode. InTreeNode.nil) { int ancestorData = InTreeNode. public static Stack create() { Stack newStack = new Stack(). newStack.setNodeData(ancestor. p). void setSizedParent(InTreeNode v. when attaching to the new parent. InTreeNode makeSizedNode() { return InTreeNode. ancestorData += InTreeNode. while (ancestor != InTreeNode. Object e) . ancestorData). we must first remove it from its current parent’s tree. InTreeNode p) { InTreeNode ancestor = InTreeNode. } InTreeNode.nodeData(v).theList == List.parent(v).setNodeData(ancestor.nil.4: Stacks and Queues 2.nodeData(ancestor). and so on all the way up the tree.12 A newly created node has one node (itself) in its in-tree.12 Chapter 2 Data Abstraction and Basic Data Structures 2. Similarly.parent(v). } public static void push(Stack s. InTreeNode.nodeData(v).theList). while (ancestor != InTreeNode. return newStack.first(s.nil). ancestor = InTreeNode. our parent’s parent.14 public class Stack { List theList. } public static Object top(Stack s) { return List. } When setting the parent of a node. we must add to the sizes of the new parent and all of its ancestors. } } Section 2. so use 1 for the initial node data. } public static boolean isEmpty(Stack s) { return (s. ancestorData).nodeData(ancestor). This decreases the size of our parent.makeNode(1).theList = List.setParent(v. ancestorData -= InTreeNode.

size = n. public static Queue create(int n) { Queue newQueue = new Queue().back = newQueue. because frontIndex == backIndex both when the queue is empty and when it has n elements in it.theList = List.front = newQueue. frontIndex.theList = List.storage[frontIndex]. return newQueue. s. } public static void enqueue(Queue q. used.16 a) The precondition for enqueue would be that the total number of enqueues minus the total number of dequeues must be less than n. } Object front(Queue q) { return q. It could also be implemented without the count of used elements by making the array size n · 1 because frontIndex would no longer equal backIndex when the queue was full. newQueue. backIndex = (backIndex + 1) % size. } .Chapter 2 { s.theList). b) An array of n elements is used because there can never be more than n elements in the queue at a time.theList).cons(e. The indices for the front and back of the queue are incremented modulo n to circle back to the front of the array once the end is reached. backIndex. } } Data Abstraction and Basic Data Structures 13 Each Stack operation runs in O´1µ time because it uses at most 1 List operation plus a constant number of steps. 2. It is also necessary to keep a count of the number of elements used. } public static void pop(Stack s) { s. public class Queue { Object[] storage. newQueue.rest(s. Object e) { storage[backIndex] = e.used = 0. used++.storage = new Object [n].used == 0). newQueue. } public static boolean isEmpty(Queue q) { return (q. int size.

nodeDegrees[parentIndex]++. remaining). } Tree buildOutTree(InTreeNode[] inNode. breaks the task up into two subtasks: finding the node degrees and building the out-tree.18 The main procedure. int[] remaining) { Tree outputTree. Additional Problems 2. } void getNodeDegrees(InTreeNode[] inNode. ++i) { subtrees[i] = TreeList. // construct the out-tree bottom-up. convertTree. using remaining for bookkeeping return buildOutTree(inNode. i <= n. i <= n.create(). } } return nodeDegrees. storing used would be unnecessary. used--. int n) { int[] nodeDegrees = new int [n+1]. for (int i = 1. int n) { // initialize remaining to be the number of children for each node int[] remaining = getNodeDegrees(inNode. for (int i = 1. Tree convertTree(InTreeNode[] inNode. n.nodeData(parent). as isEmpty could just return (frontIndex == backIndex).isRoot(inNode[i])) { InTreeNode parent = InTreeNode. i <= n. } // nodes with no children are already "done" and // can be put into the sources stack Stack sources = Stack. ++i) { if (! InTreeNode. } // calculate nodeDegrees to be the number of children for each node for (int i = 1. ++i) { nodeDegrees[i] = 0. TreeList[] subtrees = new TreeList [n+1]. because they could never wrap around. ++i) { if (remaining[i] == 0) { . for (int i = 1. int n. Also.parent(inNode[i]).14 Chapter 2 Data Abstraction and Basic Data Structures public static void dequeue(Queue q) { frontIndex = (frontIndex + 1) % size.nil. i <= n. int parentIndex = InTreeNode. } } c) Incrementing frontIndex and backIndex modulo n would be unnecessary. n).

push(sources. for (int i = 1.isRoot(sourceNode)) { // If this source node was the root of the input tree. it is immediately processed. Tree buildOutTree(InTreeNode[] inNode. int parentIndex = InTreeNode.nil. } } Data Abstraction and Basic Data Structures 15 // Process elements from sources. while ( ! Stack. // mark as done InTreeNode sourceNode = inNode[currentNode]. parentIndex). When the stack is empty. subtrees[parentIndex]). int[] remaining) { Tree outputTree. outputTree = sourceTree. It uses a form of the sweep and mark technique that is introduced for graph traversals in Chapter 7. sources. ++i) { subtrees[i] = TreeList.Chapter 2 Stack. } // Sweep through remaining[] looking for sources. i <= n. if (InTreeNode. If processing a node // causes its parent to become a source. i).parent(sourceNode). subtrees[currentNode]). } . Tree sourceTree = Tree.isEmpty(sources) ) { int sourceIndex = Stack.isRoot(sourceNode)) { // If this source node was the root of the input tree. // Mark processed sources with -1 to prevent possible reprocessing // later in the sweep.top(sources). int n. i <= n. InTreeNode sourceNode = inNode[sourceIndex].cons(sourceTree. Stack.buildTree(sourceIndex. if (remaining[parentIndex] == 0) Stack.nodeData(parentNode). } else { InTreeNode parentNode = InTreeNode. } } return outputTree. for (int i = 1.buildTree(currentNode. all nodes will have // been processed and we’re done.push(sources. if (InTreeNode. } The buildOutTree procedure can also be implemented without the auxiliary stack. outputTree = sourceTree. subtrees[sourceIndex]). adding them to the TreeList for // their parent.pop(sources). TreeList[] subtrees = new TreeList [n+1]. thereby saving some temporary space. // it is also the root of our output tree. ++i) { int currentNode = i. When a source // is found. while (remaining[currentNode] == 0) { remaining[currentNode] = -1. subtrees[parentIndex] = TreeList. then process the parent // immediately (instead of pushing it on the stack). Tree sourceTree = Tree. // it is also the root of our output tree. remaining[parentIndex]--.

remaining[parentIndex]--. } . move up to process it currentNode = parentIndex. } } } } return outputTree. int parentIndex = InTreeNode. subtrees[parentIndex] = TreeList. if (remaining[parentIndex] == 0) { // if the parent is now a source.parent(sourceNode).cons(sourceTree. subtrees[parentIndex]).16 Chapter 2 Data Abstraction and Basic Data Structures else { InTreeNode parentNode = InTreeNode.nodeData(parentNode).

Chapter 3 Recursion and Induction
Section 3.2: Recursive Procedures 3.2 Combining parts 3 and 2 of Lemma 1.4, it suffices to show that the second derivative of x lg x is d2 ´x lg xµ dx2
´lg eµ

0 for x

0.

d2 ´x ln xµ dx2

´lg eµ

d ´ln x · 1µ dx

´lg eµ

1 x

0

3.4 Statement 2 is the correct one. The proof is by induction on n, the parameter of F . The base cases are n For n 1, F ´1µ 1 01
  3 ¡1
2

015. For n
  3 ¡k

2, F ´2µ

1

01

  3 ¡2
2

1 and n

2.

0225.

01 2 for all 1 k n. [Note that the range of k includes the base cases and is For n 2, assume that F ´kµ strictly less than the main induction variable; k is our auxiliary variable.] Then letting k n   1 in the first case below and k n   2 in the second case below [Notice that both these values are in the correct range for k, but would not be if the inductive cases began with n 2.], F ´n   1 µ F ´n   2 µ In this range of n we are given that F ´nµ F ´nµ 3 2 3 01 2 01 01 01 3 2 3 2
n 1 n 2

n 1

F ´n   1µ · F ´n   2µ, so
·

01

3 2

n  2

n

2 4 · 3 9

01

3 2

n

10 9

01

3 2

n

So the claim holds for all n

2 also.

Section 3.5: Proving Correctness of Procedures 3.6 We give the proof for part (a) with the added statements needed for part (b) enclosed in square brackets. We use the lexicographic order on pairs of natural numbers (nonnegative integers). That is, ´m nµ ´x yµ if m x or if m x and n y. The proof is by induction on ´m nµ in this order. The base cases are pairs of natural numbers of the form ´0 nµ, where n 0. (Recall that the precondition of gcd eliminates ´0 0µ.) In these cases gcd(0, n) returns n. By the definition of divisor, n is a divisor of both 0 and n, so n is a common divisor. [For part (b): No integer greater than n can be a divisor of n, so n is also the greatest common divisor.] For pairs of natural numbers ´m nµ with m 0, assume the gcd function meets its objectives for pairs ´0 0µ ´i jµ ´m nµ in the lexicographic order. There are two cases in the procedure: m n and m n. If m n, gcd(m, n) returns the same value as gcd(n, m). But ´0 0µ ´n mµ ´m nµ, so by the inductive hypothesis gcd(n, m) is a common divisor of m and n. [For part (b): the greatest common divisor.] But common divisors and the greatest common divisor don’t depend on order, so this value is correct for gcd(m, n). If m n, the function returns gcd(m, nLess), where nLess n   m 0. But ´0 0µ ´m nLessµ ´m nµ, so by the inductive hypothesis, gcd(m, nLess) returns a common divisor of m and nLess. [For part (b): the greatest common divisor.] Define d to be the value returned by gcd(m, nLess). Then n d nLess d · m d has no remainder, so d is a common divisor of m and n. This concludes the proof for part (a). [For part (b): Let h be any common divisor of m and n. It remains to prove that h d . But nLess h n h   m h also has no remainder, so h is a common divisor of m and nLess. Therefore, h d .]

18

Chapter 3

Recursion and Induction

Section 3.6: Recurrence Equations 3.8 Solution 1. Applying the Master Theorem (Theorem 3.17 of the text), we find that b cn ¾ Θ´n1 µ. Case 3 applies, so W ´nµ ¾ Θ´cnµ Θ´nµ.

1, c

2, E

0, and f ´nµ

Solution 2. Using the assumption that n is a power of 2, we can get an estimate for W ´nµ that will be of the same asymptotic order. To get the formula for W ´nµ, we look at the recursion tree or expand a few terms to see the pattern. W ´nµ cn · W ´n 2µ
lg n 1 i 0

cn · cn 2 · W ´n 4µ
1 ´ µlg n cn 2 1 1 2

cn cn So W ´nµ ¾ Θ´nµ.

1 1 n
1 2

1 2i

·1

·1

  2cn´1   1

 1 ·1

cn · cn 2 · cn 4 · W ´n 8µ (using Equation 1.9 of the text) 2 c ´n   1 µ · 1

nµ · 1

3.10 Notation is consistent with the Master Theorem (Theorem 3.17 of the text) and/or Exercise 3.9 and Eq. 3.14. b) Same as Exercise 3.8. T ´nµ ¾ Θ´nµ. c) b 2, c 2, E 1, f ´nµ T ´nµ ¾ Θ´n log nµ. d) b e) b 2, c 2, c 2, E 2, E 1, f ´nµ 1, f ´nµ a) b 1, c 2, E 0, f ´nµ c lg n ¾ Θ´log1 nµ. So use Eq. 3.14 with a 1. T ´nµ ¾ Θ´log2 nµ. 0),

cn ¾ Θ´n1 µ. By case 2 of the Master Theorem (or by Eq. 3.14 with a 1. T ´nµ ¾ Θ´n log2 nµ.

cn lg n ¾ Θ´n1 log1 nµ. So use Eq. 3.14 with a

cn2 ¾ Θ´n2 µ. By case 3 of the Master Theorem, T ´nµ ¾ Θ´n2 µ.

Additional Problems 3.12 The recurrence equation for the number of disk moves is: T ´nµ T ´0µ Use Eq. 3.12 as the solution of Eq. 3.11 with b T ´n µ 2T ´n   1µ · 1 0 2, c
n n

for n 1 for n
1 2

0 1, and f ´0µ 0:

1, f ´nµ 2
n

¼

1 2 ∑ h 2 h 1 2n   1

  2n1 1 1  1 2
·

½

Alternatively, we can see by inspection that the recursion tree is a complete binary tree of height n, that all the internal nodes have a nonrecursive cost of 1 and the leaves have a nonrecursive cost of 0. So the total nonrecursive cost over the whole tree is 2n   1.

Chapter 4 Sorting
Section 4.1: Introduction 4.2 a) A worst case occurs when the keys are in decreasing order; the for loop is executed for each value of numPairs from n   1 down to 1. The number of key comparisons is n´n   1µ 2.

b) If the keys are already sorted the for loop is executed once, doing n   1 comparisons and no interchanges. 4.4

a) Suppose the last interchange on some pass is done at the jth and j · 1st positions. Then the keys in positions j · 1 through n must be in order. If they were not, there would be a key that immediately preceded a smaller key, and those two would have been interchanged. We need to show that none of these keys belongs in earlier E j · 1 . (If there may be duplicate keys, positions in the array. Let x be the largest key from among E 0 consider the last instance of x in this range.) In the previous pass through the while loop, x would always have interchanged with the next position, so it must have reached the j · 1st position. Hence the smallest key among E n   1 is greater than or equal to all keys among E 0 E j · 1 , so E j · 1 E n   1 are E j·1 in their proper positions. b) The idea is to keep track of the position where the last interchange occurred and use that position to reset numPairs instead of simply decrementing numPairs by 1. We can eliminate the variable didSwitch.
last = n-1; while (last > 0) numPairs = last; last = -1; for (j = 0; j < numPairs; j++) if (E[j] > E[j+1]) Interchange E[j] and E[j+1]. last = j;

c) No, if the keys are in reverse order, the algorithm will still execute the for loop for each value of numPairs from n   1 down to 1. Section 4.2: Insertion Sort 4.6 1, n, n   1

2 and n, n   1

3, 1, 2.

4.9 The average will be lower. The while loop (in shiftVac) that finds the place for x stops if it finds a key equal to x.

4.11 This code uses insert1, which appears in Figure 2.6 of the text; the subroutine insert1() inserts one new element into a sorted linked list, using partial rebuilding.

with ranges of size n. sorted). 2´n   1µ element movements (E[first] is moved to pivotElement and then moved back each time partition is called). This is one more than if E[first] is simply chosen as the pivot. sorted = insert1(newE. that is. Since the larger subrange is reduced by only 2 at each depth of recursion in quickSort. . Section 4. we can arrange that the median-of-3 always comes out to be the second-smallest key in the range. E[(first+last)/2] = 99.20 Chapter 4 Sorting IntList listInsSort(IntList unsorted) IntList sorted. linear space is needed for the recursion stack in the average and worst cases. E[highVac-1] * whose key is pivot is moved to E[highVac] and * the index from which it was moved is returned. the number of garbage objects is proportional to the running time. (This assumes that first.. n   2. the total number of comparisons is about n2 4. In addition. Analysis: Assume unsorted has n elements.) The subroutine insert1 is called with k varying from 0 to n   1. Choosing the pivot requires 3 key comparisons. while (remUnsort nil) int newE = first(remUnsort). It also assumes that any garbage collection takes time proportional to the number of garbage (unreferenced) objects collected. 1. and is in Θ´n2 µ in the worst case. The space usage depends on how many garbage (unreferenced) objects are created during partial rebuilding. . . and k   2 elements are greater than the pivot. and average and worst-case times are Θ´n2 µ. Otherwise.. the best-case time is Θ´nµ.5 in the text. rest. using the pairing-up trick mentioned after Eq. it will require n 2 calls to partition. * If there is no such element. return sorted. An array that is initially in reverse order also requires Θ´n2 µ comparisons with the median-of-3 strategy for choosing the pivot. E[last] = 98. remUnsort = unsorted. With a “mutable” list ADT it would be possible to avoid the creation of garbage objects by changing their instance fields. Therefore..13 /** Postcondition for extendSmallRegion: * The leftmost element in E[low]. Then 99 becomes the pivot. remUnsort.4: Quicksort 4. An example of the worst case might be E[first] = 100.17 Consider a subrange of k elements to be partitioned. for the overall procedure. since insert1 might have a depth of recursion that is about k and k can be as large as n   1. the exercise specified that the list ADT given in the text be used. The remaining keys can be chosen so the the worst case repeats. With an automatic garbage collector. 98 is the only element less than the pivot. remUnsort = rest(remUnsort). However. */ 4. sorted = nil. and all other elements are over 100. So. 4. Note that space required for the output is not counted in this analysis.15 n´n   1µ 2 key comparisons. So k comparisons are done by the time partition is finished. then the call to insert1 has a best-case time of Θ´1µ and average and worst-case times of Θ´kµ. When sorted has k elements in it. it is reasonable to assume this is a constant. and cons are implemented in constant time. Now k   3 elements remain to be compared to the pivot (it is easy to arrange not to compare elements that were candidates for the pivot again by swapping them to the extremes of the subrange). n   4. highVac is returned.

the average number of moves goes down proportionally. 21 b) After the first call: 9. it is far from clear that it would save time in typical cases. 10 (9 was the pivot). This is only half the average number of comprobability of which is 1 2 parisons. 6. After the second call: 8. 4. No moves in partition. including its subroutines.5.  1 ´n   1µn moves. c) The only advantage of partitionL is its simplicity. if only half of the elements to be merged are copied. 7. 16 moves (2 ´n   2µ). 7. After the second call: the same (1 was the pivot). The body of Quicksort does two moves of the pivot element in connection with each call to partitionL or partition. and can Total: 2 ∑n i 1 i be avoided by doing a test. The “take home” message is that partition does a linear number of moves when the initial sequence is in reverse order. 3. the gion (but not both). 2. then the total is about 1 5 n lg n. almost all of the moves occur in partitionL or partition. Total: n 2 moves in partition calls. a) After the first call: 1. we shall see that these moves may be neglected in determining the leading term for the average number of moves done by the algorithm. two moves occur in partitionL. One move was done in partition.5 on page 175. 7. 4. the probability of which is 1 2 . plus 2´n   1µ moves for pivots in quickSort itself. The question addresses how many moves are done by Quicksort. 5. In most cases it does more element movement than partition. The relatively small number of element movements is the source of Quicksort’s speed in practice. 2. 5. 9. 5. is approximately 7n lg n plus a small linear term. for the complete run of Quicksort.21 The analysis assumes that all initial permutations are equally likely and all elements are distinct. Mergesort always does an element movement after each key comparison and so does about n lg n element movements on average. In both cases quickSort itself did an additional 20 moves. Although the test would probably save some time in this worst case. Parts (a) and (b) illustrated an extreme difference. So 2´k   1µ 2 moves occur on average in partitioning a range of k elements. that is.Chapter 4 Sorting 4. 1. 4. We also mention that if the average number of comparisons in Quicksort is lower. where partitionL did 90 moves while partition did only 5. 6.23 This algorithm is simple and elegant.3. 6. However. In extendSmallRegion an “unknown” element is moved if it is .5: Merging Sorted Sequences 4. b) For partition. quickSort itself does 2´n   1µ moves. Thus the total number of moves done by Quicksort on average.” done in Line 5 of partitionL. The probability of this event is 1 2 . This number is doubled if all elements to be merged are first copied into a work area. 8. using partition. as suggested in the text. In extendLargeRegion an “unknown” element is moved if it is the pivot. but we would give full credit (or almost) for any of the reasonable variants that demonstrated a correct understanding of what is happening. Since there are at most n   1 such calls in the entire run. before Algorithm 4. 1. 4. but it uses linear space in the frame stack (unless the compiler performs tailrecursion optimization). As before. This is equal to the average number of comparisons. as far as the leading term is concerned. using partitionL. (Half of these are moves “in place. Section 4. c) Quicksort with partition as given in the text does about 1 4 n lg n key comparisons but only about 0 7 n lg n element movements.27 is done.19 Note: Students might give several different answers depending on whether certain moves are counted. each “unknown” element is examined by extendSmallRegion or extendLargeRethe pivot. 2. 3. 3. whereas partitionL does a quadratic number of moves in this case. Thus using the median-of-3 method for picking the pivot element reduces both the average number of comparisons and the average number of moves. 9. So ´ k   1 µ 2 moves occur on average. 18 moves (2´n   1µ). assuming the optimization described in Exercise 4. as described in Section 4. 10 (10 was the pivot). Thus the total number of moves done by Quicksort on average. . 10 (10 was the pivot). is approximately 1 4n lg n plus a small linear term. a) Whenever the “unknown” element is less than the pivot. 8. whichever is used.

6: Mergesort 4. we have P´k mµ P´k   1 mµ · P´k m   1µ. If A 0 B 0 . merging the two halves will need only n 2 comparisons. listMerge(rest(in1). IntList in2) IntList merged. one of the two above subproblems remains. IntList remMerged. listMerge(in1. Some people prefer a less verbose style. So the recurrence relation is W ´nµ W´ n 2 µ ·W´ n 2 µ· n 2 and W ´1 µ 0 For simplicity. Let P´k mµ be the number of possible permutations of the merged sequences where the first sequence has k elements and the second has m. rest(in2))). else remMerged = listMerge(in1. it doesn’t matter how after moving B 0 to the output area has P´k m   1µ possible permutations. the keys from the second array must fill the remaining slots in order. P ´k m µ k Solution 2.2 in the text we see that n is a solution to this recurrence equation and satisfies the boundary conditions. Also. Section 4.) Therefore.e. By Exercise 1. if (e1 e2) remMerged = listMerge(rest(in1). rest(in2)). There are m · k slots in the output array. IntList in2) return in1 == nil ? in2 : in2 == nil ? in1 : first(in1) first(in2) ? cons(first(in1). remMerged). in2). . else if (in2 == nil) merged = in1. else int e1 = first(in1). m· k . for k 0 and m 0. Let n k · m. then W ´nµ n 2 i 1 ∑ 2 · 2 k W ´n k n 2k µ lg n. in2)) : cons(first(in2).22 Chapter 4 Sorting IntList listMerge(IntList in1. return merged.25 Solution 1 (follows hint in text). merged = cons(e1.. by expanding a few terms it is easy to see that W ´n µ Let k lg n. If A 0 B 0 . such as: IntList listMerge(IntList in1.26 If the keys are already sorted. the subproblem remaining after moving A 0 to the output area has P´k   1 mµ possible permutations. Thus the number of different output arrangements   k¡ is the number of ways to choose k slots from m · k. remMerged). assume that n is a power of 2. the subproblem remaining B 0 . 4. Once the slots for the k keys from the first array are determined. if k 0 or m 0 there is only 1 permutation. int e2 = first(in2). merged = cons(e2. Then. i. (If A 0 the tie is broken. if (in1 == nil) merged = in2.

2 ≥ 2:1 < ≥ < ≥ 1.0 0.length]. mergeSort(Element[] in. out.1.2.0. the second index is the index of the pivot element. mergeSort(E.31 The comparisons are done in the partition algorithm (Algorithm 4. Although there are three parameters (in. last). If B of these have no children. there are only two distinct arrays. W. work.2 . mid). out. mid+1. then ´2D 1   Bµ have two children each.3). 23 Now the idea is that the revised mergeSort. is about n lg n element movements. E and W.0. int last) Element[] W = new Element[E. shown below. anticipates that its output needs to be in out (its second parameter).4 in the text. Section 4. so the average for Mergesort. mergeSort(in. mid. In each internal node in the decision tree below.1 0.1 < 1. out). 4.Chapter 4 Sorting 4. compared to Algorithm 4. 2:0 < ≥ 1:0 1:0 < 1:2 ≥ 2. and so it arranges for the recursive calls to put their output in two subranges of work (its third parameter). so n 2´2D 1   Bµ · B 2D   B. first. Then mergeSort calls merge to merge from the two subranges of work into the desired subrange of out. Element[] out. last).21 showed that Quicksort does about 0 7 n lg n element movements on average. A wrapper might be used to allocate the work array: mergeSortWrap(Element[] E. work. last.2. Element[] work. actually in its subroutines. mergeSort(in. E. merge(work. b) Now merge does one move per comparison because its input and output areas are different. int first. first. Altogether there are n bases cases for Mergesort. int last) if (first == last) if (in out) out[first] = in[first]. So there are 2´2D 1   Bµ leaves (base cases for Mergesort) at depth D.0 2.1. Exercise 4. as optimized here. first. else mid = (first + last) / 2.29 There are 2D 1 nodes at depth D   1 in the recursion tree. The changes to merge. work). int first.7: Lower Bounds for Sorting by Comparison of Keys 4. are straightforward. out.27 a) Mergesort is revised to have a work array as an additional parameter. // else out[first] already = in[first].

36. the total is n   1. FixHeap does only one or two key comparisons each time it is called.34 Arranging the elements in a binary tree: 25 19 5 3 7 10 12 4 15 13 No. is: Y T X P O E M I C L 14 key comparisons are needed. 68.35 The array containing the heap.40 The statement to be added is Interchange E[1] and E[2].24 Chapter 4 Sorting Section 4. beginning at index 1. Compare 37. Compare K with parent of H[32]. because 5 is smaller than its child 7. Compare 99. vacant is at H[32]. K is smaller. so two key comparisons are done by FixHeap for each one. 4. 4. so so keep going with h 1. 4. Compare 85. b) Since the keys are in decreasing order. vacant is at H[2]. If n is even. . 84. then compare K to 37. “Pull out” 100 from H[1]. For each other internal node. In both cases. 98. so FixHeap does only one key comparison at that node. each of the ´n   1µ 2 internal nodes has two children. vacant is at H[16]. K is smaller. it’s not a heap. move 93 to H[4]. move 97 to H[2].37 a) 9. K is smaller.42 a) K H[100] 1. 96. As fixHeapFast begins. Compare 69. the last internal node has only one child. move 99 to H[1]. move 85 to H[8]. vacant is at H[4]. 92. so move 37 to H[32] and insert K in H[64]. vacant 1 and h 6. no keys move. but there is no decrease in the number of key comparisons. Compare 97. 4. If n is odd. vacant is at H[8]. which contains 69. c) Best case. two key comparisons are done.8: Heapsort 4. move 69 to H[16]. Compare 93. The overhead of an extra call to FixHeap is eliminated. so keep going with h 3. Compare K with parent of H[8]. which contains 93.

or a specialized hash function can be designed to compute the proper counter index given a publisher’s name. Scan the list of books and increment the appropriate counter. case (here we use the fact that h 0 as well as even). where k 1 2 h · 1µ · 1 1 2h lg´h · 1µ for all integers h 1 k. Assuming the number of bills and the number of checks are approximately equal. Again. then lg´ 2k 1   1 and the left side lg´ 1 2h ·1 µ ·1 k. one for each publisher. the number of steps is in Θ´n log nµ. This phase will cost about 1 2 Section 4.48 a) Sort both the bills and the checks by phone number. Otherwise.) The number of steps is in Θ´n log nµ. But lg´h · 1µ is not an exact integer in this lg´h · 1µ . so the last expression equals lg´h · 1µ . which is in Ω´n2 µ. then check through the piles for bills without corresponding checks. the right side 1 2 ´h · 2 µ µ 1 2 h · 1µ · 1 lg´2 If h is odd. Also. The first phase 2 2 .45 Suppose the largest of the 5 constants is k. it may be convenient to eliminate duplicates during the sort. then do binary search for each check. c) Enter the person’s name or identification number from each record into an array. and the total number of steps is in Θ´nµ. 2 1 h · 1. 25 c) fixHeap does 12 comparisons. The list of publishers can be alphabetized. Consider an initial sequence in the reverse of sorted order. as was to be shown. 1 is an integer.44 The goal is to show that lg´ If h 2k   1.10: Shellsort 4. b) Use an array of counters. and so on. and letting n be the approximate number. this method does Θ´n log nµ steps.11: Radix Sorting 4. Additional Problems 4. 2 ´h · 2µ If h is even the last expression in the above equation equals lg´h · 2µ . Sort this list. marking the bills that are paid. n sort elements 1 k · 1 2k · 1 . (Depending on the sorting algorithm used.Chapter 4 Sorting b) 4 · 3 · 2 9. . One sequential pass through the bills finds those that are unpaid. and an equal number to of Shellsort (with increment k) will do about 1 2 n k comparisons to sort elements 0 k 2k 2 k. the number of key comparisons is approximately 30 lg 30   1 4 £ 30 · n lg30 = 5n · 108. again as was to be shown.46 The radix (hence the number of buckets) is squared. so lg´h · 2µ Section 4. 4. If there are n books and the list of publishers is sorted and searched by binary search. Then scan the list ignoring duplicates and count distinct people. An alternative solution would be to sort just the bills. 2 at each level except the leaf.

which in general must have Θ´nµ slots to achieve linear-time sorting. Note: This exercise was updated in the second printing. which is now h-sorted. 1. so they do not increase the asymptotic order for the algorithm. for example. So fixHeap. it must be possible to access any element in this array in constant time. and this is true. and Shellsort can be adapted to linked lists within the same asymptotic order. Be sure that the errata in the first printing (available via Internet) are applied if you are using the first printing.53 There are several reasonable answers for this open-ended question. 2 and h1 1. 4. But there is no way to jump long distances in a linked list in constant time. Shellsort cannot be easily adapted to linked lists. Thus we solve 2 Raising both sides to the power 2k gives 22 Then taking logs twice gives 2k So T ´nµ cn lg lg n · n1 1 cn lg lg n · n1 1 2lglg n lg n k n1 2k n lg n and k lg lg n ¾ Θ´n log log nµ 4. Heapsort. Although Radix . rather than Ω´log nµ. Radix Sort also depends intimately on the buckets array. T ´n µ Ô Ô Ô Ô cn · n c n · n1 4T ´n1 4 µ cn · nT ´ nµ cn · cn · n1 2n1 4 T ´n1 cn · cn · n 1 2 ·1 4 4 µ ·n 1 8 cn 1 4 T ´n1 8 8 µ cn · cn · cn · n1 kcn · n 1 1 2k 2·1 4·1 8 1 2k T ´n1 µ T ´n µ The boundary condition is stated for T ´2µ. Shellsort. There is no easy way to jump by the appropriate increment in constant time to find the key to be compared. with h2 h) Radix Sort: Stable. Heapsort requires some implementation of a binary tree. g) Shellsort: Not stable: 2. it can overlap with the creation of the h sublists.) Then go through the sublists in rotation and assemble them into one list. With considerable trouble it might use the usual binary tree structure with edges directed away from the root. There is no way to use linked lists in a natural manner to implement Heapsort in Θ´n log nµ.51 a) Insertion Sort: Stable. d) Quicksort: Not stable: 2. e) Heapsort: Not stable: 1. but a sizable increase in the constant factor. However. using Insertion Sort for the sorting. What matters most is if the choices are supported by good reasons. where there’s a will there’s a way. we would need a list of the sublists. if the array were simulated by a linked list in a straightforward manner. That is. 1. These extra steps take only linear time per increment. f) Mergesort: Stable. 2. 1. 3.49 Expand the recurrence to see the pattern of the terms. one could make a pass through the linked list and create h sublists. (Actually. To sort subsequences separated by an increment h. One could argue that the binary tree can be implemented with linked lists.) Then sort each of the sublists. (For bookkeeping.26 Chapter 4 Sorting 4. and would degrade to a quadratic-time sort in any straightforward adaptation. 1. 3. but this goes against the spirit of the question. as with Heapsort. whereas it is possible with an array. and Radix Sort are all good answers. would require Ω´nµ time in the worst case. c) Bubblesort: Stable. 1. taking elements in rotation. b) Maxsort: Not stable: 1. 3. 2.

so there are n iterations.22).4 CEX 1. 27 With each iteration of the while loop either u is incremented or b is decremented. One idea to keep in mind is that sorted lists can be constructed from the largest element back to the smallest.28). A Discipline of Programming.2 Insert x4 Insert x5 CEX 3.5 CEX 3. Procedures can be found in Lisp texts.2.3 CEX 1. b --. Maxsort. else if (E[u] == white) u ++.2 . rather than the code. CEX 2.3. and Bubblesort can be adapted fairly easily for linked lists. if it were required to use a list in place of the buckets array. Insertion Sort (see Exercise 4. Dijkstra. while (u < b) if (E[u] == red) Interchange E[r+1] and E[u]. // index of last red int u.3 CEX 1. b = n+1. W. it would degrade to quadratic time. This is actually easier to do with lists than it is to do in place with arrays. enough CEX instructions are done to let each new key filter all the way to the beginning of the array if necessary.4 CEX 2. the third contains elements of unknown color.2 CEX 4. At an intermediate step the elements are divided into four We assume the elements are stored in the range 1 regions: the first contains only reds. While this is true for partition. the second contains only whites. 4. CEX 3. // index of first blue r = 0. u = 1. int r.57 a) Time step 1 2 3 Instructions CEX 1. There are three indexes.Chapter 4 Sorting Sort uses lists as well as the buckets array. n.3 CEX 1.2 Insert x3 CEX 2. Quicksort (see Exercise 4. and the fourth contains only blues.) 4. it is preferable to think of adapting the ideas. (From E. the general form should be clear.11). described in the comments below. r ++. when switching data structures.55 This is an extension of the strategy used by partitionL.3 b) The following algorithm is based on Insertion Sort.4 CEX 2. Some students who follow the algorithms too literally observe that there are difficulties in using linked lists to exactly imitate the way an algorithm works on arrays. Mergesort (see Exercise 4. Time step 1 2 3 4 5 6 7 Instructions Insert x2 CEX 1. Since there are no test-and-branch instructions. The algorithm is shown for n 5.4 CEX 2. else // (E[u] == blue) Interchange E[b-1] and E[u]. u ++. for example. Thus the central idea of partition is to partition the set into “small” and “large” subsets whose keys do not overlap. // index of first unknown int b.

59 This is a good problem to spend some time on in class because it has a variety of solutions that can be used to illustrate many points about developing good algorithms. . b) Suppose an algorithm does not examine some element. insertion of xn . Bentley describes a Divide and Conquer solution that runs in Θ´n log nµ time. What matters most is if the choices are supported by good reasons. Do this for each s. return maxSoFar. 4. s. Solution 2: Fix the starting point s. for (i = 0. hence do Ω´nµ steps. Parallel Sorting Algorithms for another interesting approach. e. so this solution takes time in Θ´n2 µ.28 Chapter 4 Sorting There are n   2 earlier time steps needed to get the other insertions started. i < n. The problem is discussed at length in Bentley. As e varies from s to n   1. each with an average of roughly n 2 terms to be summed. Thus each possible starting point for a subsequence does not have to be explicitly considered. Solution 3: Observe that if an element in a particular position is positive. The algorithm would give the same response. and each ending point. so this brute-force method takes time in Θ´n3 µ. Here’s the solution. n   s   1 additions are done. The last column. Both of the solutions above consider every subsequence explicitly. or if the sum of a subsequence ending at that position is positive. but when there is no longer anything to take away. The only places where a new subsequence must be started is immediately following a position where the sum has become negative. yet each of the solutions described below can be coded in fewer than ten lines. so Insertion Sort can be implemented in linear time using simultaneous CEX instructions. i ++) currentSum = currentSum + E[i]. the maximum subsequence can not begin at the next position because including the positive element or subsequence would increase the sum.” a) Solution 1: Consider each starting point. and compute the sum of each subsequence. The number of comparisons done to find the maximum sum is smaller than the number of additions. Programming Pearls (Addison-Wesley.61 There are many reasonable answers for this open-ended question. 4. Simply change E[i] to M · 1. some students argue that this is necessary and therefore any solution would require time in Ω´n2 µ. say E[i]. if (currentSum < 0) currentSum = 0. maxSoFar = 0. Thus any (correct) algorithm must examine each element. The number of comparisons of sums is roughly equal to the number of additions. and suppose its answer is M . There are Θ´n2 µ subsequences. always saving the maximum sum found so far. always saving the maximum sum found so far. 1986). hence shows the subtlety sometimes necessary in lower bounds arguments. It is more complicated than the solutions described here. but it is also interesting to present in class. takes n   1 time steps. The total is 2n   3. else if (currentSum > maxSoFar) maxSoFar = currentSum. The next solution is linear. currentSum = 0. so the total is in Θ´n2 µ. “A designer knows he has arrived at perfection not when there is no longer anything to add. See the odd-even transposition sort in Selim Akl. and it would be wrong. using only one addition. Many students turn in complicated solutions with a page and a half of code. I (Sara Baase) write the following quotation from Antoine de Saint-Exup´ ery on the board before I start discussing the problem. compute each new subsequence sum by adding the next element to the current sum. For each s. for a subsequence ´0 s e n   1µ. There are other solutions.

One is to throw everything together and sort once on a combined key of course and student name. So simply distribute into 26 main piles based on first letter. use the second table to distribute into 26 piles based on second letter. working right-to-left (or high-to-low).000 comparisons on average. a) Simply sort them with a comparison sort using three-way comparisons. A little thought reveals that if there are duplicates. Because sorting costs grow faster than linearly. c) Simply count how many have each key value. But suppose we frequently have data that is partially sorted. this is not a problem. Another is to sort each class individually by student name. it might be more expedient to use a simple method like Insertion Sort. Since the largest class will be 200 at most. If any three-way comparison is “equal” there are duplicates. The idea is similar to text section 4. Having decided to sort each class individually. even though its asymptotics are not favorable. as discussed above. as you just pile them up. using an array of 2n · 1 entries to store the counts.3. Insertion Sort does about 225 comparisons on average. The exercise implied that the input data was already grouped into classes.19(b). Readers who are familiar with the relative speeds of disk access and internal computations will realize that the time for these disk accesses will dwarf the sorting time. Each of these piles can probably be sorted ad hoc. and the sorting procedure is correct. Because of the small sizes the space overhead incurred by Mergesort is not a serious drawback. and an equal number of element movements. (This is often the case in real-world problems. b) The main problem with distributional sorts in a computer is the need to arrange space for buckets that will have an unknown number of items in them. There is no reason to expect the worst case for Insertion Sort.5. Merge them with the sorted elements. If a main pile is too large to sort ad hoc. which it would be if it was in the order in which students registered. 29 . In this case. then Radix Sort would probably be the best way to group it into classes so that each class can be sorted as a small set.Chapter 4 Sorting a) There are two basic approaches to this kind of sorting problem. Mergesort would use about 1600 comparisons on average. We might consider Quicksort also. the matching student record for each class record can be looked up after the class records are sorted. (We can quit as soon as any count becomes 2.63 One point to this problem is that you can’t do any better than the obvious method (if only comparisons are available). do a post-scan of the sorted array to check for equal adjacent elements. For manual sorting of exams. so the two might be quite close in time used for sets of 30. and given that most classes have about 30 students. b) The time is Θ´n log nµ using Mergesort or Heapsort. See Exercise 5. 4. so let’s look briefly at some average cases. Even allowing for Mergesort’s higher overhead per comparison. and Insertion Sort would run faster than average. If three-way comparisons are not convenient.) If such information is needed.65 To simplify the notation let m denote the number of elements in E2. so m is in Θ´log nµ. it is probably 4 to 5 times faster than Insertion Sort. if the data is randomly ordered. using the disk address in the class record.) 4. so the chosen sorting method is almost irrelevant in this case. The merge step requires n   1 comparisons at most. On a set of 30. Mergesort would stay about the same. If not. rather than random? This is a reasonable possibility when the data is extracted from another database. many small sorts will be cheaper than one large sort. But Mergesort probably has more overhead per comparison than Insertion Sort. the person can shuffle things around to make enough space somewhere. m is about lg k and n k · m. then there must be an “equal” comparison at some point. then Quicksort might perform slightly better than Mergesort on average because it does fewer moves and the data records are rather sizable. while Mergesort does about 150. However. For the first solution. A final issue to consider is the role of the student records. So the total number of comparisons is in Θ´nµ. Quicksort would tend to degrade toward its worst case. Trying to optimize the pattern of disk accesses goes beyond the scope of the text. Insertion Sort should use about 10. which are separate from the class records. The problem statement does not make it clear whether information is needed from the student records to compose the class lists. sort the unsorted elements in E2 by themselves with Θ´m log mµ Θ´log n log log nµ comparisons. doing 5000 separate sorts. but if one or two need a further breakdown. using the extra space at the end of E1 for the initial output area. for the occasional large classes.

(A related technique comes up in Chapter 14 where it is called cross-ranking. Once we know where every element of E2 belongs in the merged array. simply insert each element from E2 into E1. then store r in rankE1[j]. This means that when E1 and E2 are merged. it is sort of pointless to try to reduce the number of comparisons below Θ´nµ. and E1. For each j. But m lg k is in Θ´log2 nµ. E2[j] will go into E1[j+r]. Actually. in the manner of Insertion Sort.) After sorting E2 declare an array rankE1 with m entries. . For a third method. Since any method requires Ω´nµ element movements in the worst case. This step requires about m lg k comparisons. We work right-to-left through rankE1. for all i such that rankE1[j] i rankE1[j+1]. which is quite simple. we can perform the merge without doing any further comparisons. which is in Θ´n log nµ. This array will record where each element in the sorted E2 array belongs in the final merged sequence. 0 j lg k.30 Chapter 4 Sorting The second method will do o´nµ comparisons. However. With all this machinery. only n element movements are needed. If it is discovered that E2[j] should immediately precede E1[r]. This dominates the cost of sorting E2. this requires about ´k · 1 2 mµm comparisons and element movements in the worst case. the array rankE1 is not needed. E2. For example.8) reduces the number of comparisons to Θ´log2 nµ. but it simplifies the description of the algorithm. use binary search in E1 to find out where E2[j] belongs in sorted order. but it is possible. If done naively. But again. as its elements can be computed on the fly. the element initially in E1[i] must move to E1[i+j+1]. it does not reduce the number of element movements. binary search in E1 (see Exercise 4.

25 tells us that the total number of possible   ¡ n comparisons. but the meaning is clear here. This degree of analysis on the adversary argument gives us a lower bound with the same leading term (n) as that given in Theorem 4.4. Thus the adversary can force an algorithm to do at least lg n 2 requires some care to obtain a rigorous lower bound on this quantity. We wrote “¦O´log nµ” to remind ourselves that we do not know whether this term is positive or negative. Applying Stirling’s formula carefully gives the following bounds that are often useful in their own . We can carry one more term in our simplification of Stirling’s formula and get a more precise bound. It permutations is nn2 . There are n! possible permutations in all.18 cannot be used on the negative terms. 1. Solution 2.1: Introduction 5. This is the same result we obtained in Theorem 4. It always chooses a response to preserve at least half the remaining possibilities. n n 2 n n 2 ´n n! 2µ!´n 2µ! lg lg´n!µ   2 lg ´´n 2µ!µ The difficulty is that the simple lower bound given in Eq. n nÔ n Θ ´1 µ n! e lg´n!µ n´lg´nµ   lg´eµµ · 1 2 lg´nµ · Θ´1µ n lg ´´n 2µ!µ ´lg´nµ   1   lg´eµµ · 1 2 lg´nµ · Θ´1µ 2 n lg n  1 for n 2 and even.10. By inspection of Stirling’s formula (Eq. so the adversary makes any algorithm do at least lg n! comparisons in the worst case. 1. which is the best possible. the adversary considers how many permutations of the keys are consistent with both possible answers.4. Solution 3.2 a) For each comparison. 2 lg´nµ ¦ O´1µ n 2 Therefore the result given by this adversary argument is very close to the result of Theorem 4. but the second order term is not known. but is weaker by a log term.4 gave a lower bound of n   1 for this problem. Thus after each comparison. the number of remaining permutations is at least half as many as before the comparison. It gives the response consistent with the most permutations. b) The adversary keeps track of all remaining possible arrangements of the keys consistent with the responses it has given to the algorithm and the fact that the two sequences being merged are sorted. n 2 There is some abuse of notation to treat Θ´ µ and O´ µ as functions. also we do not know (from the above analysis) whether it is ¦Θ´log nµ because the leading terms might cancel. Theorem 4.Chapter 5 Selection and Adversary Arguments Section 5.19 in the text) we can write n n  Ô ¡ Θ n n! e lg´n!µ n´lg´nµ   lg´eµµ · Θ´log nµ n lg ´´n 2µ!µ ´lg´nµ   1   lg´eµµ · Θ´log nµ 2 n lg n ¦ O´log nµ for n 2 and even. Solution 1. Exercise 4.

One comparison is done before the loop. return secondLargest. The worst case occurs for keys in increasing order. Two comparisons are done if and only if E i is one of the two largest keys among the first i keys. by the induction hypothesis. Let’s assume the keys are distinct.11 was used to estimate 2 i 2 · i i n 1·4∑ i n n   3 · 2 ln n n n 1 1 · ∑1 2∑ 3 i i 3 i 3 i 1 · n   2 · 2∑ i n 1 3i i 3 ∑ i. We use an informal induction argument to show that after heapFindMax executes. finally: lg n n 2 n 1  1 2 lg´nµ for n 2 and even. So.4 a) By examination of the for loop we see that the array positions are filled in reverse order from n   1 down to 1 and that each position is assigned a value once and not changed. Thus. Then. It is filled when last 2k. while (i < n) if (E[2*i] == max) i = 2*i. the max of the original keys. At that time. if (E[i-1] > secondLargest) secondLargest = E[i-1]. The probability for this is 2 i.3: Finding the Second-Largest Key 5. c) max = E[1]. 1 . The probability that E i is one of the i   2 smaller keys is ´i   2µ i. The other child lost when that node was filled. if (E[i+1] > secondLargest) secondLargest = E[i+1]. the other child of each node we pass contains a key that lost to the max.6 a) Worst case number of comparisons = 1 (before the for loop) + 2´n   2µ (in the loop) = 2n   3. position k gets the maximum of its children. The base cases are the leaves. E[1] will have the maximum of all the leaves. i = 1. positions n through 2n   1. as we move down the tree from the root along the path where each node has a copy of max. Selection and Adversary Arguments Ö Ö 2 π 4 1  11n 2n Ô n n n 2 2 π 1· 1 11n 2 Ô n n Therefore the “¦O´1µ” term in Solution 2 is actually between   32 and  0 62. The claim is clearly true for them. so position k gets the maximum of the leaves in its subtree. Suppose the claim is true for all positions greater than k. each of which. Section 5.32 Chapter 5 right. depending on n. is the maximum of the leaves in its subtree. b) At each internal node. and consider node k. 5. so the average number of comparisons is A´nµ 1·∑ 2¡ i 3 n n   1 · 2´ln n   1µ Equation 1.. the key in that node is the same as one of its children. positions 2k and 2k · 1. b) Each time through the loop either one or two comparisons are done. secondLargest =  ∞. else i = 2*i+1. i.e. each position contains the maximum of all the leaves in its subtree.

whereas Quicksort must recursively sort both segments after partitioning the array. with the guessed solution substituted in. The same technique works for Solution 2. which is the call that would be used to find the median.Chapter 5 Section 5.key. In other words. int last. else if (k > splitPoint) ans = findKth(E.4: The Selection Problem 5. . first. To avoid a 2-parameter recurrence we might settle for an upper bound expression instead of an exact recurrence. pivot. ¡ 1   2 c´n · 2kn   2k2   3n · 2µ 2n 1 n´1 · 1 2 cµ · k´c   c´k nµµ · c´ 3 2 · 2 ´k nµ · 1 nµ   1 . */ Element findKth(Element[] E. The right-hand side of the recurrence. so the number of comparisons is in Ω´n2 µ. int k) Element ans.8 Selection and Adversary Arguments 33 a) The idea is that. For all of these calls the subrange that is partitioned has at least n 2 elements. Key pivot = pivotElement. Since k´c   c´k nµµ cn 4. k). This method of solution also works for the recurrence for T ´nµ. we have a worst case if each call to partition eliminates only one key. and the desired inequality holds. A recurrence for the average number of comparisons when there are n elements and rank k is to be selected is: T ´n k µ T ´1 0µ n 1· 0 1 n k 1 i 0 ∑ T ´n   1   i k   1   iµ · ∑ n 1 i k ·1 T ´i   1 k µ for n 1 0 k n The running time will be of the same asymptotic order as the number of comparisons. so its worst case is in Θ´n2 µ. we choose c 4. If the input is sorted. if (first == last) ans = E[first]. splitPoint . b) As with Quicksort. Also. /** Precondition: first k last. where c is a constant to be chosen. Then k T ´nµ n 1· 1 n k 1 i 0 ∑ T ´n   1   iµ · ∑ n 1 i k·1 T ´i   1µ d) Working with Solution 1 above. 2. findKth(E. last. will generate successive calls with first = 1. n 2 ). else ans = pivotElement. splitPoint + 1. n. 1. n 2 and last = n. c) Solution 1. return ans. findKth never does more comparisons than Quicksort. Solution 2. Let T ´nµ denote the average number of comparisons when there are n elements and k has its most unfavorable value. E[splitPoint] = pivotElement. and try to verify the guess. findKth need search only one segment. becomes: n 1· 1 n k 1 i 0 ∑ c´n   1   iµ · ∑ n 1 i k ·1 c´i   1µ n 1· n 1· ¡ 1  1 k´n   1 · n   kµc · 1 ´n   1   kµ´k · n   2µc 2 2 n We need to choose c so that this expression is at most cn when 0 k n. k).1. we will guess that the solution is T ´n kµ cn. last). int splitPoint = partition(E. T ´nµ max T ´n kµ. if (k < splitPoint) ans = findKth(E. else Element pivotElement = E[first]. int first. first.

The adversary maintains variables that are initialized and have meanings as follows.6: Designing Against an Adversary 5. For simplicity in presenting the solution. the adversary can force any algorithm to do at least n 2   1 non-crucial comparisons.5 // Now E[3] is the smallest of the three remaining keys. Hq i   1. The adversary follows the strategy summarized in Table 5. and each creates at most one S key. Thus the for loop does approximately ∑k i 1 2´lg n   iµ comparisons. it is the median. and the total time will still be linear in n. // We know E[3] E[4]. ´H · 1   Lµ. The search key is K and the array name is E . must do at least 3n 2   2 comparisons in the worst case.56.3 If an interchange occurs here. // E[1] is smaller than three other keys. answer K Hq 1 . Additional Problems 5. CEX 3. Lq E i . it may give status S to at most n 2   1 keys and status L to at most n 2.4. The modified version of Theorem 5. // We know E[3] E[4]. we use the CEX (compare-exchange) instructions described in Exercise 4.34 Chapter 5 Selection and Adversary Arguments 5. CEX 2.5: A Lower Bound for Finding the Median 5. L0 H0 M0 R0 0 n 1 ´n   1µ 2 n (low) (high) (midpoint) (range) smallest index whose key might equal K . by comparison of keys. If Lq 1 4. If i Lq 1 . ´L · H µ 2. Section 5. The subscript on the variable indicates how many questions have been answered. The sum is bounded by 2k lg n.4 CEX 1. answer K Hq 1 . largest index whose key might equal K . Lq E i . Update Hq Lq 1 and Hq Lq 1 and Hq i · 1. so k may be as large as n lg n. No updates (i. If i 2.12 At least n   1 crucial comparisons must be done whether n is even or odd. where the median is the n 2th smallest key. We’ll assume the algorithm does 3-way compares. M and R are recomputed whenever L or H is updated. . Section 5.. Lq 1 . it can be rejected. with any fractional part. answer K Hq 1 . When the algorithm compares K to E i (asks the q-th question). If Mq 1 Mq 1 . n   1. Update Lq E i . Since each comparison described in Table 5.5 CEX 2. also interchange E[2] and E[4]. answer K i i E i . the number of slots that might hold K . No updates (i. // E[2] is smaller than three other keys.2 CEX 3. CEX i. also interchange E[4] and E[5].10 Building the heap takes at worst 2n comparisons. CEX 1.4 creates at most one L key. . Hq 1 ).3 If an interchange occurs here. it can be rejected. DeleteMax uses FixHeap which does approximately 2 lg m comparisons if there are m keys in the heap.14 Any key that is found to be larger than three other keys or smaller than three other keys may be discarded. the adversary responds and updates L and H as follows: 1.16 The array indexes holding keys are 0. 3.3 is: Any algorithm to find the median of n keys for even n.j compares the keys with indexes i and j and interchanges them if necessary so that the smaller key is in the position with the smaller index. Lq Hq 1 ).e..e.

sorting needs at Ω´n log nµ three-way comparisons. If the algorithm knows the relative order of each pair xi and xi·1 . and copied into maxSoFar if it is larger. So q 5. For example. then it has enough information to sort the keys. The algorithm’s strategy is to save the ´n · 1µ 2 smallest keys it has seen so far. one pass through the sorted array can be used to detect duplicates in O´nµ time. we   1 ¡q lg´n · 1µ. After all keys have been read. Thus determining if the keys are distinct also needs Ω´n log nµ comparisons. The problem can’t be solved with fewer than two cells. the ”=” test in the comparison does not provide any extra information. The first key is read into maxSoFar. 5. in a sorted array. q lg´n · 1µ . Now assume the algorithm stops when Rq 0. because if there were only one. If sorting completes without any “equal” occurring. or the algorithm would be unsound.” We use shiftVac from Insertion Sort (Section 4. so the algorithm would give the wrong answer for that input.21 To establish the minimum number of cells needed requires both an algorithm and an argument that no other algorithm can solve the problem using fewer cells. For example. The problem can be solved with ´n · 1µ 2 · 1 cells. More specifically: . By the recurrence for Rq · 1 and the base case R0 · 1 n · 1. Let x1 x2 the algorithm sees them).19 a) Sort the keys using an Θ´n log nµ sort. the adversary can change xi to make it equal to xi·1 without affecting the results of any other comparisons. the algorithm must collect enough information to sort the keys. compared with maxSoFar. The three-way comparison is more powerful than the two-way comparisons for which we established the Ω´n log nµ lower bound for sorting. Then the algorithm does not have enough information to determine whether xi xi·1 since no comparisons of these keys with others in the list would provide that information. it is possible that K E Lq and it is possible that K is not in the array. so the total is Θ´n log nµ in the worst case. but stop and report that a key is duplicated if any comparison comes out “equal”. then there are still at least two possible outcomes of the search.2. If a standard sorting algorithm is used (without a test for ). to determine whether the keys are distinct.Chapter 5 The essential invariant is Rq · 1 Hq Lq ´Rq · 1µ i 1 ´Rq 1 · 1µ Selection and Adversary Arguments 35 2. in case 4. If the algorithm stops before Rq 0. Thus. Since q must be an integer. but if the keys are distinct. so that Rq 1. if Lq Hq . each remaining key is read into next.2) to insert the next key into the correct position if it is among the smallest keys. a) The algorithm uses two cells maxSoFar and next.) The total time is in Θ´n log nµ. b) For simplicity we assume n is odd. have Rq · 1 2 ´n · 1µ. no comparisons could be done. (Every pair of keys that are adjacent in the final sequence must have been compared directly. If the algorithm says the keys are all distinct. it is wrong. Suppose there are two keys xi and xi·1 whose relative order is not known by the algorithm. using the extra cell for “scratch work. xn be the keys in order. (not necessarily the way b) Consider a list where the keys are distinct. ´Hq 1 Mq 1   1   Lq 1µ 2   1 · Lq 1 2 ´R q  1 · 1 µ 2   2 · Lq 1 Lq 1 Hq   Lq · 2 ´Rq 1 · 1µ Case 3 is similar. then all keys are distinct. If the algorithm says there are two equal keys. so in the worst case. the largest key retained by the algorithm is the median.

Now we show that the problem can’t be solved with fewer than ´n · 1µ 2 · 1 cells. larger than the key chosen. so again. then reply consistently with earlier replies A key has won a comparison if and only its weight is zero.2. we show that an adversary can make it give a wrong answer. The discarded key cannot be the algorithm’s output. We will show that an adversary can make the destroyed key be the median. Sort these keys (in place). The adversary makes ´n   1µ 2   ´i   1µ of the unseen keys smaller than the discarded key and the other unseen keys larger. the adversary can make all the remaining keys smaller. Whenever i is the root of a tree. i < n. w´iµ is the number of nodes in that tree. If the algorithm compares E i and E j and w´iµ w´iµ prior ´w´iµ · w´ jµµ. 3. E[(n+1)/2 . 5. new w´iµ 0.22 a) Find the minimum of E k through E n with n   k comparisons.1]. and update new E j . w´ jµ. and the algorithm’s answer is wrong. If it chooses any but the largest of these. // the key that was in E[(n+1)/2 . If the algorithm chooses the largest of the keys it has seen. w´ jµ. 4. Suppose the first key that is overwritten by the algorithm is the ith smallest of the keys it has seen up to that point.1]) // Insert the new key in its proper place. Case 1: The algorithm gives its answer before overwriting a key in one of the cells. the algorithm has read no more than ´n · 1µ 2 keys. except that the replies are reversed. new w´ jµ 0. Then. w´iµ 1 for all keys initially. and update new E j . imagine that the adversary builds a tree to represent the ordering relations between the keys. except that the smallest key is at the root. 5. unseen by the algorithm. and update new 2. then reply that E i w´ jµ 0. The total number of keys smaller than the discarded keys is i   1 (seen by the algorithm) + ´n   1µ 2   ´i   1µ (not yet seen by the algorithm) = ´n   1µ 2. i ++) Read the next key into scratch. the winner of a comparison is the larger key. Specifically. Suppose. b) Follow the adversary strategy in the proof of Theorem 5. Then whatever output the algorithm makes. There are at least ´n   1µ 2 unseen keys and at least one key seen by the algorithm is larger than the key it chose. an adversary can make all the remaining keys. the adversary can demonstrate a consistent set of keys for which the output is wrong. and the loser is the new root. Two trees are combined only if neither root won before. at the time it determines its answer. As in the proof of Theorem 5. if (scratch < E[(n+1)/2 . discarding the key that was there. return E[(n+1)/2 .2 in the text. there is no key with weight n   k. . If the algorithm compares E i and E j and w´iµ w´ jµ prior ´w´iµ · w´ jµµ. the algorithm reads a key into a cell that was used earlier.) The sum of all weights is always n. If the algorithm compares E i and E j and w´iµ and do not update any w. for (i = (n+1)/2. If the algorithm compares E i and E j and w´iµ w´iµ prior ´w´iµ · w´ jµµ.36 Chapter 5 Selection and Adversary Arguments Read the first (n+1)/2 keys into E[0]. so there are too many larger keys. 1.1].1] is discarded. vacant = shiftVac(E. new w´ jµ 0. scratch). then reply that E i w´ jµ 0. (Recall. then reply that E i E j .1. so the discarded key was the median. when the algorithm stops. if the . E[vacant] = scratch. Case 2: Before choosing its answer. (n+1)/2 . There are at least ´n   1µ 2 keys it hasn’t seen (at that time). Suppose an algorithm used fewer cells. so in this problem we are looking for the champion loser. the one chosen by the algorithm is not the median.

the adversary can fill all slots with keys unequal to x. So our argument will show that any algorithm for this problem must examine at least 2n   1 matrix entries. 5. Since at least k keys are not in tree T . Observe that the known ordering of the rows and columns in the matrix implies nothing about the ordering of the keys on each diagonal. then x n   1. If the algorithm says x is not in M without examining all 2n   1 keys on the two diagonals. A tree of n   k · 1 nodes must have n   k edges. a correct algorithm can stop only when the weight of some key is at least n   k · 1. So the lower bound is n   k questions. this refutes the algorithm. the adversary puts x in an unexamined position on one of the diagonals. and on the diagonal just below this one ´M 1 n   1 to M n   1 1 µ. But each question creates at most one new edge in the set of trees that the adversary builds.24 The adversary will force an algorithm to examine each entry on the diagonal from the upper right corner ´M 0 n   1 µ to the lower left corner ´M n   1 0 µ. If the algorithm says x is in M . elements on the lower diagonal satisfy i · j n. There are 2n   1 keys on these diagonals. The first diagonal is characterized by the property that i · j (the sum of the row and column numbers of an element) is n   1. Whenever an algorithm compares x to an element M i j the adversary responds as follows until there is only one element on the two diagonals described above that has not been examined by the algorithm: if i · j if i · j n   1.Chapter 5 Selection and Adversary Arguments 37 output is m and E m is in tree T then make all keys not in tree T less than all keys that are in tree T . Therefore. then x Mi j Mi j The adversary’s answers are consistent with the given ordering of the rows and columns and with each other. so the algorithm would be wrong. .

38 Chapter 5 Selection and Adversary Arguments .

Note that the array size is at least n · 1 after the shrink and was at least 4n · 1 before being shrunk. push. reducing the number of elements to the original n. forcing an array double. Do 41 pushes. by the inductive hypothesis the cumulative accounting cost was at least ´2nµt before the shrink and at least ´nµt afterwards.1 The geometric series for array quadrupling is n. then 2 pushes.   ¡ . But the new array size 2n or 2n · 1 so the formula holds. Say the stack size is N0 and n N0   1. is straightforward. A proof by induction on i. the amount of space allocated might be almost four times what is necessary instead of only twice what is necessary. The cumulative accounting cost is ´3nh   n   N µt . array double from n to 2n: 3. Do 2 pushes. then 22 pops. leaving n elements and causing a shrink:  nt · 2t . and the cumulative accounting cost is negative. pop. compared to ´2nµt for array doubling. and sums to at most ´4n 3µt . push. so the formula still holds. The analysis is similar to part (b) except that before a shrink the cumulative accounting cost is nt . n 4. Note that an accounting cost of 1t on pop is not enough. pop. b) Use accounting costs as follows: 1. c) The accounting costs for pop are t and  nt · t . For pops. no array double: 2t . The cost of 4 operations is 4 · 2tN0   t 2n. shrink   1 ¡   1 2 a factor of E when n N E . for actual cost 2 · tN0 . If the i-th operation was a pop that caused a shrink with accounting cost  tn.Chapter 6 Dynamic Sets and Searching Section 6. which is less than 2N0 2. The new nh n. Section 6. Then the amortized cost is 1 · 2t for push and 1 · 2t for pop. and after it is 0. .  nt · 2t . 4. However. Now do 2 pops. So. the number of the operation. The cumulative accounting cost is always ´4nh 2n   N µt when the stack had nh elements at high water since the last size change. Suppose N0 10. The new size is 2N0 . 2. no shrink: 2t . The accounting cost for pop is E  1 t when there is no shrink and  nt · E  1 t when there is a shrink. n 16.2: Array Doubling 6.2 a) Worst case cost is Ω´nµ. So this forces a shrink. This sequence can be repeatedly indefinitely. by d) Expand the array by a factor E 2 and use accounting cost EE  1 t¡instead of 2t for push. at a cost of 2 · t ´N0   1µ.3: Amortized Time Analysis 6. then N 2 · 1 n 2   1 and the array size was 4n · 4 N 4n · 1. The new array size is n · 1 or n.

So this is 1 · 2´4h 1   1µ 1 2 ´4 µ   1. A D A e . . so part 1 holds. The principal subtrees of an ARBh tree are both RBh 1 trees. The base case is h 0. so parts 1-3 hold for this RB0 tree. Similarly. But 1 · 2´2h 1   1µ 2h   1.40 Chapter 6 Dynamic Sets and Searching Section 6. So this is 2´2h 1   1µ 2h   2. An RB0 tree consists of 1 external node. so parts 1-3 for ARB0 trees hold vacuously. any path from the root to a black node has at least as many black nodes as red nodes.. .4 ARB1 a = RB1 A = B = a C = a D = a a ARB2 e = A A f = A B g = A C h = A D i j k l Like above but B is left subtree m n o p Like above but C is left subtree In detail... and part 1 holds for ARBh trees. so the RBh tree has at most 1 h h 1 · 2´ 2 ´4 µ   1µ 4   1 internal nodes. for part 2. the black height of the RBn or ARBn tree. So black depth 1 2 depth.. An ARB0 does not exist. and part 2 holds for RBh trees. . . We showed above that each has at most 1 2 ´4 µ   1 nodes. t = q r s t Like above but D is left subtree RB2 A A . an ARBh tree has a maximum number of h nodes when both principal subtrees do. . An RBh tree has a maximum number of nodes when both h principal subtrees are ARBh trees. for RBh trees. An RBh tree has a minimum number of internal black nodes when both principal subtrees are RBh 1 trees. D and e. not counting the root. For h 0 assume the lemma holds for black heights 0 j h.6 The proof is by induction on h. For part 3. .. A t (20 right subtrees in all) 19 more rows like above but B. . t are the left subtrees Last tree in detail = 6..4: Red-Black Trees 6. An ARBh tree has a minimum number of internal black nodes when both principal subtrees do.

the case of repairRight is symmetrical. .leftSubtree. assume repairLeft was called.10 a) int colorOf(RBtree t) if (t == nil) return black. For definiteness. For definiteness. This status is passed to either repairLeft or repairRight. Then rbtIns is called on one of these these children. The base case is h 0. it is a red node with both children nil. t. where oldRBtree is either an RBh tree or an ARBh·1 tree. it is nil and case 3 of the lemma is true.color = black. assume recursion goes into the left subtree. For h 0. If oldRBtree is an ARB1 tree. so the recursive call returns status brb. then color flip higher in tree. Then case 4 of the lemma applies (case 5 after repairRight).8 Dynamic Sets and Searching 41 10 10 20 10 20 30 10 20 30 40 20 10 30 40 50 10 20 30 40 50 60 20 10 30 40 50 60 70 10 20 30 40 50 60 70 80 Inserting 80 needs color flip. If oldRBtree is an RB0 tree.color.12 The proof is by induction on h. and the inductive hypothesis applies to whichever principal subtree is passed as the parameter in the recursive call to rbtIns.color = red.rightSubtree. Then each principal subtree is either an RBh 1 tree or an ARBh tree. 6. t. else return t. The case of the right subtree is symmetrical. and case 3 applies. assume the lemma holds for 0 j h. then rebalance higher in tree. b) 6. Suppose oldRBtree is an RBh tree.color = black. There are 5 cases to consider for this recursive call. void colorFlip(RBtree t) t.Chapter 6 6. 20 10 30 40 50 60 70 80 90 10 20 30 50 40 60 70 80 90 100 Inserting 100 needs color flip.

It could have been assigned after rebalLeft was called.newTree as the new principal left subtree and returns status = ok. So all details of case 4 of the lemma are confirmed. Further. 5. In (recursive) case 1 of the lemma.newTree is RBh 1 or ARBh .status = brb and ansLeft.newTree as the left subtree of oldRBtree we have one of these configurations. The case for the right subtree is symmetrical.newTree.status = ok and ansLeft. so ans. so each of its principal subtrees is an RBh tree. no further repair is needed. The recursive call returns ansLeft. 3. now rooted at ans. This completes the proof for oldRBtree being an RBh tree. ansLeft. In (recursive) case 3 of the lemma. The recursive call returns ansLeft.rightSubtree.status = rrb and ans. either case 1 or case 5 of the lemma applies. and one of the cases described above applies to the value returned. while the right case is case 3.newTree.newTree is RBh 1 .status = brb The left case is again case 1 of the lemma. where white circles denote red nodes: ARBh RBh−1 After attaching ansLeft.rightSubtree is RBh .status = rbr and ansLeft. where the root is oldRBtree: RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 In the left case rebalLeft is called and in the right case colorFlip is called. So all returns from the recursive call of rbtIns lead to one of the cases stated in the lemma. Now suppose oldRBtree is an ARBh·1 tree. by arguments that are symmetrical to those above.newTree. are: RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 RBh−1 ans. 4. Notice that case 2 of the lemma never applied.leftSubtree = ansLeft. This is similar to case 1 above. This is the mirror image of case 4 above and leads to either case 1 or case 3 of the lemma. and case 1 of the lemma applies.status = rrb and ansLeft. .status = ok ans.status = brb and ansLeft. Notice that only cases 1 and 3 of the lemma occurred when the parameter of rbtIns was an RBh tree. which is case 1 of the lemma. Then repairLeft attaches ansLeft. Indeed. ans.42 Chapter 6 Dynamic Sets and Searching 1. Then a recursive call to rbtIns is made with one of these principal subtrees as a parameter. The recursive call returns ansLeft. which is ARBh·1 .newTree. rbr is never assigned as a status in any of the procedures as given. 2.newTree = oldRBtree and ans.newTree is ARBh .newTree is ARBh·1 . Now oldRBtree. This is similar to case 1 above. the same as oldRBtree. The recursive call returns ansLeft.color = red.newTree looks like this. Again we assume the recursion goes into the left subtree and consider the possible values the might be returned into ansLeft. The outcomes of each. For oldRBtree being an ARBh·1 tree and the recursion going to the right subtree.

The starting point for all parts is: 40 20 10 30 50 60 70 80 90 100 Circles with white backgrounds denote red nodes and gray circles are gray nodes. move 90 into 80’s node and delete 90’s prior node (see next). 6. rooted at 100. Delete 70: Fig. as the new right subtree of its parent. 6. which is nil. a) Delete 10: Fig.14 Note for instructor: The initial tree is not given explicitly in the text.17 (r is red) at 40 40 60 80 10 20 40 50 70 60 80 90 100 40 20 10 To delete 40. simply attach its right subtree. To delete 90. move 30 into 20’s node and delete 30’s prior node (see next). but is the result of Exercise 6. move 50 into 40’s node and delete 50’s prior node (see next). .17 (r is red) at 40 60 80 90 100 To delete 20. 6. To delete 100.16(b) at 20 40 20 30 50 60 70 80 90 100 20 30 50 40 60 80 20 30 40 50 70 Fig. as the new right subtree of its parent. 6. Recolor the right subtree to black. Delete 30: Fig. since its left subtree is nil. then (p is red) at 60 40 80 90 100 To delete 60.8.17 (p is red) at 80 40 60 50 80 90 100 50 40 60 80 90 100 To delete 80. Delete 50: 40 60 70 80 90 100 60 70 Fig. since its left subtree is nil. move 70 into 60’s node and delete 70’s prior node (see next). simply attach its right subtree. 6.17 (s is red) at 60. which is solved above.16(b) at 20 (mirror image) 60 50 70 80 90 100 10 20 50 Fig. 6.Chapter 6 Dynamic Sets and Searching 43 6.

17. 6. 6. giving the tree in the middle. which completes the operation. The fifth tree shows 30 being deleted. move 100 into its place and delete 100’s node. and leads to the middle tree. 6. terminating the operation. The rest is simple. 80 is deleted without difficulty in the right tree. 50 80 60 70 90 100 70 60 80 90 100 10 20 30 80 70 90 100 To delete 70 (now in the root). Deleting 30 is similar to deleting 70 in part (b). then apply the case “ p is red. shown in the third tree. Then deleting 40 is simple. This leads to the case “r is red” in Fig.44 Chapter 6 Dynamic Sets and Searching b) Delete 40 as in part (a).17 of the text. The fourth tree shows 20 being deleted. it is recolored to black.17. Although 80 is initially gray after the transformation. In this configuration it is necessary to apply the mirror image of the same transformation. Then deleting 20 is simple. Next. Deletion of the last two nodes is simple.” to the third tree. and leads to the right tree. The result is shown in the right tree. giving the tree on the left. delete 60 (now in the root) by moving 70 into its place and deleting 70’s prior node. 60 40 30 50 70 80 90 100 40 50 60 70 80 90 100 50 60 70 80 90 100 To delete 50. shown in the second tree. Then delete 50 (now in the root) by moving 60 into its place and deleting 60’s prior node. below. 80 20 10 30 90 100 10 20 30 80 90 100 10 20 30 90 100 To delete 90 (now in the root). Deleting 70 uses the case ““r is red” in Fig.” leading to the left tree below.16(b) in the text. Then apply the transformation for the mirror image of the case “p is red. leading to the second tree.16(b). Then deleting 60 is simple. and leads to the left tree below. shown in the fourth tree. 6. After applying the transformation of Fig. 80 60 70 90 100 70 80 90 100 80 90 100 90 100 . giving the result shown on the left below. move 80 into its place and delete 80’s prior node. since it is the root of the entire tree.17. 20 10 30 100 10 20 30 100 10 20 30 100 10 30 100 10 100 c) Deleting 10 is seen in part (a). the result is shown on the left below. leading to the middle tree below. apply the transformation for the case “s is red” in Fig. 6. 6. Deleting 80 uses Fig. Apply the transformation for the mirror image of the case “s is red” in Fig. with 90 now as the gray node.

r is red. // Copy this union into each row k such that k is in the union. however. are maintained. k n. The operations hashDelete and hashFind continue over an obsolete cell. ). it increments full and decrements empty. If hashInsert inserts into an empty cell. k ++) if (A[j][k] == 1) A[i][k] = 1. for (k = 1. break. just examine the corresponding matrix entry. The following code shows how obsolete is handled in hashDelete. whereas hashInsert continues over it without checking for a match. h   full   empty is the number of obsolete cells. Use an n ¢ n matrix A where A i j Implementation of an IS instruction is trivial. then reorganization into an array of the same size might be satisfactory. because all obsolete cells are discarded during reorganization. MAKE i j could be implemented as follows.16 Dynamic Sets and Searching 45 ¯ ¯ is red: rightRotate( . // Assign to row i the union of rows i and j. /** empty > 0 */ void hashDelete(Key K) i = hashCode(K). or s is red: leftRotate( p. but the order of the worst case time will still be high compared to the techniques developed in the text. while (true) if (H[i] == emptyCell) // K not there break. If empty gets too small. the efficiency of hashDelete and hashFind will degrade.key == K) H[i] = obsolete.18 The operations hashDelete and hashFind terminate in failure if they encounter an empty cell. As shown below. However. The combination is also called a right double rotation. s). k ++) if (A[i][k] == 1) Copy row i into row k. k n. then leftRotate( p. Section 6. if it inserts into an obsolete cell. and A i j 0 otherwise.Chapter 6 6.20 The point of this exercise is to show that a straightforward matrix implementation is not efficient.6: Dynamic Equivalence Relations and Union-Find Programs 6.5: Hashing 6. if full is 2 h. then reorganization into a larger array is indicated. it only increments full. full and empty. Code for the other operations follows the same pattern. Section 6. (We assume that hashDelete deletes just one occurrence if the key has multiple matches.) Two counters. hashDelete decrements full. The operations hashDelete and hashFind terminate with success at a cell whose key matches the search key. 1 if i and j are in the same equivalence class. if (H[i] obsolete) if (H[i]. The reader should be able to find ways to improve the algorithm below. full --. for (k = 1. s). assuming that duplicate keys are allowed. whereas hashInsert terminates with success at such a cell. with the differences mentioned in the paragraphs above. say 1 1 2 h. . i = ((i + 1) % h // Continue loop. p is red. whereas hashInsert terminates with success in that case. If full is also large.

3) Union(4. else parent[t] = u. u must have been the first argument of the hUnionµ.5) produce the two trees shown in the following figure. void hUnion(int t.7 completes the argument without modification.46 Chapter 6 Dynamic Sets and Searching The number of matrix entries operated on when implementing MAKE i j is proportional to n · pn where p is the number of elements in the new equivalence class (the union of i’s class and j’s class). a) Assume the trees are represented by arrays parent and height where height[i] is the height of the tree rooted at i if i is a root (and is irrelevant otherwise). if the next instruction is union(2.6 for hUnion. For the second case.2) Union(2. int u) if (height[t] > height[u]) parent[u] = t. then either h h1 (because h1 h2 µ or h1 h2 and h h1 · 1 h2 · 1 (in this case.5). b) The instructions Union(1.22 n´n   1µ · n2´n · 1µ 2   n ¾ Θ´n3 µ. if u was made a child of t . if (height[t] == height[u]) height[u] ++. In the proof of Lemma 6. c) The worst case is in Θ´n log nµ. The proof for the first case is the same as in the text.6 modified for hUnion. We just need to modify the proof of Lemma 6. MAKE 1 MAKE 3 MAKE 4 MAKE n 2 2 3 n 1 2 ´n · pnµ The number of steps is proportional to ∑n p 6. then wUnion will produce a tree with root 2. Consider the following program. Theorem 6. no matter whether the height or the number of nodes is used as the weight. h So h lg´2 min´k1 k2 µµ lg´k1 · k2 µ lg k h1 · 1 lg k1 ·1 and h h2 · 1 lg k2 ·1 . using the induction hypothesis. 2 1 3 5 4 However. and hUnion will produce a tree with root 5.

6 7 8 6. Thus the total amount of work done by the cFinds is bounded by a multiple of the number of distinct branches in T . Attaching v to r0 yields an S1 tree. Then these Si trees. then attaching ri to ri·1 for 0 i k   2 yields an Sk 1 tree. is the handle of Tk 2 as described. U can be decomposed into v T0 requirements of the decomposition. respectively.26 (see Fig. 6. Now let T be an Sk tree for k 1 and suppose that any Sk 1 tree has the described decomposition. and let r be the root of U . find(1): 2 parent accesses. Tk 1 with roots r0 rk 1 . and the handle v is the child of r0 . the parent of each of these roots is r in U ¼ . (Notice that v is not part of this Sk tree. No changes in tree. T was constructed by attaching the root of one Sk 1 tree U to the root of another V . Then T0 is the only tree in the decomposition and is an S0 tree. No change in tree. Let T ¼ be the tree that results from cFind(v). 6.27 (see Fig. satisfy the requirements of the decomposition described in Exercise 6. in T ¼ all nodes on the path from v to the root of T are now children of the root.28 Suppose that T is an Sk tree properly embedded in a tree U . .) 6. then cFind(w) follows only one link for the overlapping path segment since in T ¼ x is a child of the root. Since each wUnion introduces one new branch. the total amount of work done by the cFinds is in O´nµ. An S1 tree has two nodes. The handle of T . a worst-case program must have some cFinds. Conversely.30 in the text). then it is an Sk tree.24 Tree before any finds: find(1): 4 parent accesses. say at x. cFind( f ´vµ) in U causes all these roots to become children of r. We will use the name T to mean this tree as it appeared before any cFinds are done. find(6): 3 parent accesses. if a tree can be decomposed as described. (The sequence of attachments indexed by i is empty if k 2. Suppose that cFind(w) is done later. If the path in T from w to the root meets the path in T from v to the root. the path from T ’s handle v to the root of T consists of roots of Si trees for 0 i k   1. That is.Chapter 6 6.29 in the text). which is v. suppose that the tree is decomposed into v and trees T0 such that Ti is an Si tree. For k 1. The base case is k 1. By the decomposition described in Exercise 6. This can also be proved by a simple induction argument.26 First we show that an Sk tree can be decomposed as described. r0 . Let Tk 1 V . Let T be a tree constructed by the wUnions. so it consists of one node. attaching v to r0 . 6. The base case is k 1. Resulting tree: Dynamic Sets and Searching 47 9 4 2 1 3 7 6 8 find(4): 2 parent accesses. By the inductive hypothesis. The root r0 is an S0 tree. along with r.) It is given that Tk 1 is an Sk 1 tree. Resulting tree: 9 1 2 4 3 7 6 8 1 9 2 4 3 find(2): 2 parent accesses.30 Since wUnions take constant time. By the inductive hypothesis. So attaching rk 2 to rk 1 creates an Sk tree. The proof is by induction on k. it satisfies the U . so there is an Sk tree initially embedded in U ¼ .

This equivalence should be flagged as an error. vLdr). If the result is that vLdr becomes the leader of uLdr. then the new offset is inconsistent with the earlier offset. and can store information about them in arrays. It is necessary to update offsets during a find according to the rule . Which of these cases applies is determined as follows. vLdr). 2. it is its own leader and its offset is 0. to them when they are first encountered. At this point we execute makeSet on the new vertex.. d2 .v) offset(u) offset(v) newOffset(u. If the leaders are the same and newOffset(u. equivalence (d1 . offset(u) address(u)   address(uLdr) although the appropriate addresses will be determined later by a compiler. 1. .32 We will use a Dictionary ADT to keep track of variable and array names and to assign successive integers 1. When a declaration is first encountered.” we treat it as the sequence of binary equivalence declarations.v) Notice that we never need any actual addresses to compute newOffset(uLdr. . Similarly. No more updates are required for this equivalence.v) 3. If the result is that uLdr becomes the leader of vLdr.v) offset(u)   offset(v) offset(u)   offset(v) then the new offset is consistent with the earlier offset. the leader information that is stored for a vertex might be out of date until a find is executed for that vertex. indexed by the vertex number. d2 ) equivalence (d1 . The offset gives the relative memory location of the vertex from its leader. vLdr). then offset(uLdr) is updated to become newOffset(uLdr. 2. We initialize a Union-Find ADT with create(0). say “equivalence (d1 . If the leaders are the same and newOffset(u. dk ). our algorithm only works with differences between addresses. First we execute find(u) and find(v) to get their current leaders. Now we execute union(uLdr. For each vertex (i. Assume the underlying vertices are u and v. d3 ) equivalence (d1 .e. this offset is new information. 3. Conceptually. variable or array declaration). There are three possibilities for the new offset: 1.v). the offset will be out of date in these cases. The exact form of d1 and di determines a necessary offset between u and v. then offset(vLdr) is updated to become -newOffset(uLdr. We combine the following conceptual equations: address(u)   address(uLdr) address(v)   address(vLdr) address(u)   address(v) to get the new relationship: newOffset(uLdr. call them uLdr and vLdr. we keep track of its leader and its offset from its leader. As usual with the Union-Find ADT. 2. vLdr) address(uLdr)   address(vLdr) offset(u)   offset(v) · newOffset(u. When an equivalence declaration is encountered. So the rest of the program can treat them as vertices in a graph.48 Chapter 6 Dynamic Sets and Searching 6. vLdr). dk ) and we think of each binary equivalence as an edge between the two vertices whose names occur in the expressions di . The offset is consistent with an offset between u and v that was determined earlier. The offset is inconsistent with the earlier offset. which we’ll call newOffset(u. The offset is new information. If the leaders are different.

37 An obsolete node is no longer referenced by the xref array. Dynamic Sets and Searching 49 In cFind. The total number of comparisons is n   lg´nµ   2. a) To see that a lot of obsolete nodes can become roots of trees. Now vacant = 4. In this case. Now do a series of getMax and deleteMax until r is again the root of the only tree.forest). else return bypass(rest(remForest)).Chapter 6 offset[v] = offset[v] + offset[parent[v]]. set xref[7] = 0. just as it would be in the tournament method of Section 5. It uses n   1 comparisons.34 The point of this exercise is to show how xref is updated in tandem with the steps of the usual heap deletion procedure. Since vacant is a leaf now. depending on the outcomes of comparisons. Now isEmpty will require time in Θ´mµ.2). The code for isEmpty could be: boolean isEmpty(PairingForest pq) pq. It occurs somewhere in a list of trees. -1) . and reduce heapSize to 4. The second getMax performs lg´nµ   1 comparisons. ensuring that all of the keys become less than r. Now do deleteMax and the m obsolete nodes are roots. Section 6. Next.36 Yes. . TreeList bypass(TreeList remForest) if (remForest == nil) return (remForest) else if (root(first(remForest)). this statement would be placed right before the statement that updates parent[v] in Fig. 6.3. Now vacant = 2.5) is compared to 1(7.2. Do additional decreaseKeys on all the descendants of the vi . so we wait until we are traversing the list and come across it. If n is not an exact power of 2. making them less than r. The n inserts create n trees of one node each and use no comparisons. Since lg´nµ is an integer.forest = bypass(pq. return (pq. Execute decreaseKey on each vm creating m obsolete nodes in the children of r. so there are lg´nµ trees at this point.22 of the text. and the obsolete nodes are still children of r. vacant = 1 and xref[4] = 0 to delete element 4(3. but this was not required for the exercise.forest == nil). v1 . the total might be one less than the worst case.5) is compared to K and 3(4.4) and 3(4. First we save K 7´6 4µ. the first time we come across it we will be able to delete it cheaply.5) moves up to vacant. Elements are shown in the format id(priority) for clarity. This exercise fleshes out the details. The getMax reorganizes the elements into one tree in which the root has lg´nµ children.id return (remForest).7: Priority Queues with a Decrease Key Operation 6. This requires the update xref[5] = 2. The main idea is that it might be expensive to remove it from the list at the time it becomes obsolete. The deleteMax removes the root and leaves a forest in which the children of the former root are now roots. vm . Then 5(5.4) has no sibling to compare with. this agrees with Theorem 5. we store K in vacant and update xref[7] = 4. suppose we start with one tree whose root r has m children. This requires the update xref[3] = 1. Then 3(4. so 5(5. Luckily. it is always optimal for n 2k . 6.4) moves up to vacant. then the above argument still holds.2. replacing lg´nµ by lg´nµ .4) is compared to K and 5(5. 6.

Additional Problems 6. as well as its priority. The code. so this verification is straightforward. and the id is the index in xref. if (remForest == nil) newForest = cons(t1. Tree winner = pairTree(t1. newForest = cons(winner. To find the minimum element simply follow left subtrees to the internal node whose left subtree is nil. and rest all run in time O´1µ. Charge the bypass subroutine an accounting cost of –2 for each obsolete node that it encounters (and discards. return newForest. in addition to the actual cost of 2 for testing and discarding the node. newForest). Treat decreaseKey as a delete (locating the node through xref instead of by a BST search). There are only four places to check. newForest). Since bypass runs in amortized time O´1µ. Thus all operations have the same asymptotic order as they would for the binary-heap implementation of priority queues. remForest = bypass(oldForest). Thus decreaseKey runs in O´log nµ. could be: TreeList pairForest(TreeList oldForest) TreeList newForest. the xref array has to be kept up to date. in additional to the actual cost of 1. The cost is Θ´k · mµ but m can be arbitrarily large in relation to k. remForest. The usual method can be used to delete this node. isEmpty does also. remForest = bypass(rest(remForest)). while (remForest nil) Tree t1 = first(remForest). through xref. For a full priority queue use an xref array similar to the one described in the text except that xref[i] refers to the subtree whose root holds element i. t2). c) Charge decreaseKey an accounting cost of +2 for creating an obsolete node. as in rebalancing. pairForest runs in amortized time O´kµ because its loop executes about k 2 times and each pass runs in amortized time O´1µ. Therefore the cumulative accounting cost can never be negative. first. else Tree t2 = first(remForest). Create a forest with k genuine trees and m obsolete trees. Thus bypass runs in amortized time O´1µ. The usual method can be used to insert new elements and test for emptiness. cons. using bypass from part (a).40 It is straightforward for red-black trees to implement an elementary priority queue with each operation running in time O´log nµ. remForest = bypass(rest(remForest)). // Continue loop. Moreover. There is no problem doing getPriority in O´1µ. followed by an insert of the same id with the new priority. since it returns a list without that node). But each node contains the element’s id. . newForest = nil. Whenever the red-black tree undergoes any structural change.50 Chapter 6 Dynamic Sets and Searching b) The argument that pairForest does not run in O´kµ is similar to part (a). To verify that the obsolete nodes really are discarded we note that any list passed to bypass is never accessed again. Here we are using the facts that pairTree. so updating xref costs O´1µ per structural change. either because that variable is never accessed again or because the variable is overwritten with a new list from which the obsolete nodes seen by bypass have been removed.

For any vertex x in the tree. Lemma: Let v be the root of a breadth-first search tree for a graph G. Section 7. and the length of the shortest path is called the distance from v to x. so its distance from v in the tree is k.Chapter 7 Graphs and Graph Traversals Section 7.4: Depth-First Search on Directed Graphs 7.6 It is always true that height´TD µ height´TB µ. Let u be the next-to-last vertex on a shortest path from v to x in G.) Proof The proof is by induction on k. Then x cannot already be in the BFS tree at a depth of k   1 or less. the distance of a vertex x from v. The empty path is the shortest path from v to v and is the tree path from v to v.2: Definitions and Representations 7. x will be put ¾ on depth k of the tree. so by the inductive hypothesis.3 If v and w are any two distinct vertices of G and there is a path from v to w. (The length of a path from v to x is simply the number of edges in the path. then there is an edge vw in G. Suppose the distance of a vertex x from v in G is k 0. it puts v on w’s list in the new structure. Section 7. . Thus the height of a breadth-first search tree is the distance of the farthest vertex from the root.1 7. No spanning tree of G with the same root can have smaller height.3: Traversing Graphs 7. the path from v to x using tree edges is a shortest path from v to x in the graph. This follows from the following lemma. u is at depth k   1 in the breadth-first search tree. (a) 3/10 A 4/7 B 8/9 F C 5/6 D 2/11 G 1/14 E 12/13 8/9 F 7/12 A 10/11 B C 3/4 (b) D 6/13 G 1/14 E 2/5 7. Then the distance from v to u is k   1 0. The base case is k 0.4 The diagrams show discovery and finishing times. but which help to interpret the DFS tree.8 The algorithm goes through the adjacency lists of the given graph and constructs a new array of adjacency lists for the transpose. For each vertex w found on vertex v’s list. which were not asked. Since x is adjacent to u.

v n.12 Roots of the DFS trees are shaded.52 Chapter 7 a) Graphs and Graph Traversals IntList[] transpose(IntList[] adjVerts. v++) trAdvVerts[v] = nil. 7. v++) remAdj = adjVerts[v]. c for cross edge. dotted edges are nontree edges. trAdjVerts[w]). and d for descendant (forward) edge.10 Gray. Heavy solid edges are trees edges. int n) IntList[] trAdjVerts = new IntList[]. b) Assume the array of adjacency lists for G indexes vertices in alphabetical order. return trAdjVerts. v n. remAdj = rest(remAdj). trAdjVerts[w] = cons(v. while (remAdj nil) int w = first(remAdj). The order of the actual adjacency lists for G is irrelevant. (a) (b) A 1/22 b 2/21 B H 9/20 11/18 B A 10/19 b H 9/22 3/8 D b F 4/5 c G c E 10/19 12/17 D b c E 20/21 c C 11/18 b J 13/16 b K 14/15 F 13/14 c G 15/16 2/7 I b C 4/5 J 3/6 b K 1/8 6/7 12/17 I . their labels are b for back edge. for (v=1. The adjacency lists in the transpose graph will be in reverse alphabetical order because cons puts new nodes at the beginning of a list. for (v=1. The adjacency lists of GT are as follows: A: B: C: D: E: F: G:               F A F G G A E D E B D B A 7.

no node following c on path P1 appears on P2 at all.) Insertion at line 9 would also work. 7. Treat the tree as a directed graph. reversing the JK edge makes it a descendant edge. as follows. Node c exists because at least the root of the tree is on both paths. At “Exploratory processing” (line 7). and revises the times as follows: E(4/11). We would probably only assign one part of this exercise for homework and use another part for an exam. The active intervals of v and w are disjoint. K(18/19). and 13 and ans are not needed.18 There are six trees.Chapter 7 (c) (d) Graphs and Graph Traversals 53 A 1/22 b 2/21 B d 5/10 D b F 7/8 d G 6/9 12/17 I b C 11/18 b J 13/16 K 14/15 E 4/19 11/16 D b F 13/14 d G H 3/20 19/20 B c A 18/21 b H 9/22 E 10/17 c C 4/5 b J 3/6 b K 1/8 2/7 I 12/15 Notes for instructor: Note that only part (d) has all four edge types. reversing the EC edge makes it a cross edge. it does not appear on the part of P2 that follows c. By the definition of c. J(17/20). Small changes can cause all four edge types to appear in the other parts without having much effect on the solution. ¯ ¯ ¯ For part (a). there can’t be a tie because active intervals are either nested or disjoint—they can’t partially overlap. I(16/21). reversing the JC edge makes it a descendant edge. For part (b). The active intervals of the children of c are disjoint. Algorithm 7. (Lines 2. insert a statement to output the edge vw.16 Modify the directed DFS skeleton.14 Solution 1 (follows the hint). the first five have only one vertex each. H(3/12). one of them contains the active interval of v and some other one contains the active interval of w. and shuffle them around from term to term. and c is a common ancestor of v and w. 7.1 of the text. respectively. Let P1 and P2 be the unique simple paths from the root to v and w. so in particular. Therefore the part of P1 following c is disjoint from the part of P2 following c. . C(15/22). B(2/13). 7. and use the results about active intervals in Theorem 7. makes a second DFS tree. Let c be node with the smallest active interval that contains the active intervals of both v and w. A(1/14).3. Let c be the last vertex on path P1 that also appears in P2 . each time the course is taught. do a depth-first search. 11. Solution 2. 9. So the paths from c to v and c to w are disjoint except for c. and are contained in the active interval of the root. For part (c).

Let Si be the strong component of G that corresponds to the vertex si in the condensation (0 i k. so the original cycle assumed to be in the condensation of G cannot exist. and suppose s0 s1 because a condensation has no edges of the form vv. . which we will indicate by wi v1 w1 vk 1 wk 1 vk ´ v0 µ. D F G . vertex 9. there is an edge vi wi in G where vi is in Si and wi is in Si·1 . But then all the Thus we can form the following cycle in G: v0 w0 vertices in this cycle should have been in the same strong component. 7. containing the leaders for the components. We can assume k 2 Let G be a directed graph. Construction of the adjcency lists for the transpose digraph takes Θ´n · mµ time. The vertex v can be reached by following edges from each other vertex because there are no cycles and no other dead ends. The total is in Θ´n · mµ. (This may be easier to see by looking at the adjacency lists in Example 7.14. The DFS trees in the transpose graph each encompass one SCC. and C I J K (with the appropriate edges).5: Strongly Connected Components of a Directed Graph 7. Sk S0 ). go through the adjacency list array and determine if there is exactly one vertex. We show the algorithm’s output. In the diagrams the roots of the DFS trees are shaded. since there is an edge si si·1 in the condensation. If the edges are all reversed. of course. (See Exercise 7.8. under each diagram. neither can reach the other.24 The strong components are the same.) If there is more than one vertex with an empty adjacency list. c) There is exactly one vertex with an empty adjacency list. b) Checking the adjacency list arrays for empty lists takes Θ´nµ time.54 Chapter 7 Graphs and Graph Traversals 12/13 3 14/17 2 15/16 1 6 7/8 7 5/6 11/18 4 9/10 5 9 1/2 8 3/4 vertex topo 1 7 2 8 3 6 4 9 5 5 6 4 7 3 8 2 9 1 7.22 sk ´ s0 µ is a cycle in the condensation of G. from wi to vi·1 .) Thus the digraph in Example 7. If so.15 is a lattice. and nontree edges are dashed. that vertex can reach all others in the original digraph.) This takes Θ´n · mµ time. for each part of the problem. The finishStack at the end of Phase 1 is shown. For each i ´0 i k   1µ. the scc array. say v whose list is empty. Section 7. They are A B H . We continue only if we found a unique vertex v with an empty list. only vertex 4 will have an empty adjacency list. (Otherwise there would be a cycle in the digraph. E . Then check if there is exactly one vertex in the transpose graph with an empty adjacency list. tree edges are heavy solid. Now construct the adjacency lists for the transpose digraph. Since each Si is strongly connected there is a path in G vi·1 (where we let vk v0 ). so the DAG is not a lattice. So.20 a) There must be at least one vertex with an empty adjacency list.

Chapter 7 (a) Graphs and Graph Traversals 55 A 1/22 A 1/6 2/21 B H 9/20 A B H E C I J K D G F finishStack 3/4 B H 2/5 3/8 D E 10/19 17/22 D E 7/8 F 4/5 G 6/7 12/17 I C 11/18 J 13/16 K 14/15 F 18/21 G 19/20 11/14 I C 9/16 J 10/15 K 12/13 Original graph Transpose graph A B C D E F G H I J K scc: A A C D E D D A C C C (b) A 10/19 A 3/4 11/18 B H 9/22 H E A B D G F K I J C finishStack 2/5 B H 1/6 12/17 D E 20/21 9/14 D E 7/8 F 13/14 G 15/16 2/7 I C 4/5 J 3/6 K 1/8 F 10/13 G 11/12 17/20 I C 18/19 J 16/21 K 15/22 Original Graph Transpose Graph A B C D E F G H I J K scc: H H K D E D D H K K K .

56 Chapter 7 Graphs and Graph Traversals (c) A 1/22 A 1/6 2/21 B H 3/20 A B H E C I J K D G F finishStack 3/4 B H 2/5 5/10 D E 4/19 17/22 D E 7/8 F 7/8 G 6/9 12/17 I C 11/18 J 13/16 K 14/15 F 18/21 G 19/20 11/14 I C 9/16 J 10/15 K 12/13 Original Graph Transpose Graph A B C D E F G H I J K scc: A A C D E D D A C C C (d) A 18/21 A 3/4 19/20 B H 9/22 H A B E D G F K I J C 2/5 B H 1/6 11/16 D E 10/17 9/14 D E 7/8 F 13/14 G 12/15 2/7 I C 4/5 J 3/6 K 1/8 F 10/13 G 11/12 17/20 I C 18/19 J 16/21 K 15/22 Original Graph finishStack Transpose Graph A B C D E F G H I J K scc: H H K D E D D H K K K Note for instructor: Like Exercise 7. each time the course is taught.12. . and shuffle them around from term to term. we would probably only assign one part of this exercise for homework and use another part for an exam.

Without loss of generality. Algorithm 7. Case 2: The edge wv´ vwµ is first encountered while traversing the adjacency list of w. If it is known that G is connected. At “Exploratory processing” (line 7) and “Checking” (line 11) insert statements to output the edge vw.) Notice that outputting the edge vw as soon as it is obtained from the adjacency list (i.30 Use the connected component algorithm (Algorithm 7. it will be visited immediately as a child of v and vw will be a tree edge. this is not trivial. This shows the value of working with the skeleton and inserting code only in the indicated places. 7. The following figure illustrates the construction described in the next paragraph. (If m is not given as input. 9. Let C1 be a simple cycle containing e1 and e2 . e1 • • C1 w• • v e1 • • • v e2 • • • • e3 C2 w• e2 • • • • e3 . Let C2 be a cycle containing e2 and e3 .. the depth-first search from v must still be in progress.27 Modify the undirected DFS skeleton. by the recursive nature of the algorithm. At this point w could not yet have been visited because. and 13 and ans are not needed. I. so wv is a back edge. Suppose it is not. after line 5 of the skeleton) will not work for undirected graphs. a complete depth-first search from w would have been completed before returning to v and the edge wv´ vwµ would have been encountered first from w. A. then there is a simple cycle containing e1 and e3 . we are done. C is constructed by combining a piece of C1 with a piece of C2 . if w had been visited while traversing subtrees of v. otherwise vw would have already been encountered from v.Chapter 7 Section 7. 7.7: Biconnected Components of an Undirected Graph 7. count the edges. we assume that v is encountered before w in the depth-first search.28 Let vw be an edge.32 D. Since v was visited before w. The symmetric property is satisfied because if a cycle contains e1 and e2 . then it contains e2 and e1 . Since w has not been visited. 7. If e3 is in C1 also.2) modified to quit with a negative answer if any back edge is encountered or if any vertices remain unmarked after the first call to ccDFS.34 a) The reflexive property is satisfied because we are given that e1 Re2 if e1 e2 .8. Case 1: The edge vw is first encountered while traversing the adjacency list of v.6: Depth-First Search on Undirected Graphs Graphs and Graph Traversals 57 7. (Lines 2. Since we require a simple cycle. For the transitive property we must show that if there is a simple cycle containing e1 and e2 and a simple cycle containing e2 and e3 . although it would work for directed graphs. then G is a tree if and only if m n   1. at the time when wv is first encountered. Thus v is an ancestor of w.e. We will show that there is a simple cycle C that contains e1 and e3 .) Section 7. as follows.

then that it is a maximal biconnected subgraph (i.. there must be a simple path from v to y in G that does not use x. y. C and D (and the edge CDµ form one bicomponent. let C11 be the piece that contains e1 . the two triangles. but old2 C old2 D . We can append the edge xy to form a simple cycle in G. it would not be connected). Let v be any other vertex in H adjacent to x. b) The graph in the diagram in the text has two equivalence classes. so there is a path between x and y in H . D. of course). Because J is biconnected. x. A A B C D E C B A D E old1 D . C and D (and the edge CDµ form one bicomponent. let C23 be the piece that contains e3 . c) Let H be the subgraph based on such an equivalence class.35 Consider the following undirected graphs. but old1 C b) In the rightmost DFS tree. v and w are incident on edges in a cycle.) Now break C1 at v and w. so again xy should have been in H . Case 1: All vertices in J are also in H . Break C2 at v at w. which is connected. In the middle tree. a biconnected component). that C1 and C2 have in common.e. in H and the other. We will use the facts that H contains at least two vertices and every vertex in H is incident upon some edge in H . say v. there are two disjoint paths between v and w (disjoint other than the endpoints v and w. say w. but B and D are not in the same biconnected component. so assume that H has more than one edge for the rest of this paragraph. in which the solid lines are edges in a DFS tree rooted at A and dashed lines are back edges. Because the cycle is simple. Now suppose H is properly contained within J . Let xy be an edge in J that is not in H . it is biconnected by definition. . so the loss of any one other vertex leaves a path between v and w. J must include an edge that is not in H (since if it contained only additional vertices.58 Chapter 7 Graphs and Graph Traversals In C1 . old1 B component. i. J is connected. so let xy be an edge with one vertex. any two edges in H are in some simple cycle. (They must have at least two common vertices because e2 is on both cycles. so xy should have been in H . Continue around C1 to find the last vertex. It falls into two pieces. H is connected because for any two distinct vertices v and w in H . or H contains only one edge. We show first that H is biconnected. begin at e1 . The vertices x and y are in H . 7. Case 2: J has at least one vertex not in H . This path forms a simple cycle when the edges xv and xy are included. but B and D are not in the same biconnected A and old1 D A and old2 D D. Thus H is biconnected. in J but not H . old2 B A In the middle tree. C is a simple cycle because no vertex of C2 appears on C11 (except the endpoints v and wµ. A B C D E F a) In the leftmost depth-first search tree. Attach C11 and C23 at v and w to form the cycle C. a larger subgraph of G that is also biconnected. that is also in C2 . It also falls into two pieces.e. and continue around C1 until reaching the first vertex.. If H contains only one edge.

so removal of any one edge can not disconnect the graph. They both are linear in n. D. if ai j 0. Such a graph is biconnected. It will be popped later as part of the other bicomponent. b) Not true. but the graph is edge-biconnected. If there were. j. When the search backs up from D to C. the root is an articulation point. 7. CD. BC. Additional Problems 7. If a biconnected graph has more than one edge. there is a path between each pair of vertices (other than the root) that does not go through the root. the subtree rooted at that child contains all the other vertices.35. Let vs be a possible hypersink. when the search is ready to branch out again from C is the edge CE encountered and stacked. Let v C and w D. (The definition of hypersink does not specify anything about edges of the form ss. DE. and E . and since it is a tree.39 Consider the leftmost depth-first search tree shown in the solution of Exercise 7. and E . so the test suggested in the exercise would fail to detect the bicomponent consisting of C. D. and if ai j 1. then check whether the remaining vertex is actually a hypersink. The first solution is a little bit easier to follow and to write out. Thus. The shared vertex is an articulation point. then after the depth-first search backs up from E . A similar exercise is: Can a leaf in a depth-first search tree for a connected graph be an articulation point? 7.41 a) True except for a graph consisting of exactly one edge. every edge is part of a cycle. We move down column s as long as we see 1’s. then v j cannot be a Let A ´ai j µ be the adjacency matrix (1 i j n). and assume the vertices on each adjacency list are in alphabetical order. When a 0 is encountered. if there is more than one child. but removing the edge would leave two isolated vertices. So each probe in the matrix can eliminate one vertex as a possible hypersink.34(b) in the text). there can be no path from the first child to the second child that does not go through the root. it detects the bicomponent containing the vertices C.43 The general strategy of both solutions below is to first eliminate all but one vertex as possible hypersinks.) Solution 1. If the root has two (or more) children. Only then. BA. the second child of the root would have been encountered while doing the depth-first search from the first child and would have been put in the first subtree. So the root is not an articulation point. CB. 7. the second is a little faster. and it pops the top five entries from the stack. back = 1 (the “back” value for C). Proof.37 The root of a depth-first search tree for a graph is an articulation point if and only if the root has at least two children. . Consider a graph consisting of two triangles that share one vertex (the diagram in Exercise 7. EC. ED (top).Chapter 7 Graphs and Graph Traversals 59 7. edgeStack contains (bottom) AB. DC. If an edge is stacked on edgeStack each time it is encountered. then vi cannot be a hypersink. Observe that for i hypersink.35. At the time the algorithm backs up from D to C. CA. With minor changes ss can be considered to be required or prohibited. so the solution given allows ss to be present or not. If there is only one child. we jump right to the column of the next possible hypersink (the row number where we encountered the 0) and continue moving down the column.40 No. Consider the leftmost graph shown in the solution of Exercise 7.

giving G T . For example. if (i s && a[s][i] == 1) hsink = false. At “Checking” (line 11) insert statements to output the edge vw. // No vertex with index < i (including s) can be a hypersink. // Continue inner loop. Modify the undirected DFS skeleton. The relevant matrix entries do not have to be reexamined in the verification phase. If there is just one such vertex. then it (or any vertex in its SCC) can reach all vertices in the original graph. First find the SCCs of the directed graph G and create the condensation graph (denoted G in the text). In G T . i++) if (i s && a[i][s] == 0) hsink = false. s = i.60 Chapter 7 Graphs and Graph Traversals boolean hsink. return hsink. . if (hsink) // All vertices except s were eliminated. “An optimal algorithm for sink-finding. So s can reach every vertex in G .2 we saw that the overall winner of a tournament (in this problem. i n.” In Section 5. Now take the transpose (reverse the edges) of the condensation graph. The elimination phase of the algorithm examines n   1 matrix entries (one from each row other than the first row). So any vertex in the SCC of s can reach every vertex in G. hsink = true. The elimination phase can be conducted like a tournament (where examining an entry ai j is a “match” between vi and v j . for (i = 1.3) and the DFS starts at 1.47 This problem is a good example of the power of strongly connected component analysis. using Θ´n · mµ time. At “Backtrack processing” (line 9) insert a statement to output the edge wv. 7. (1. The verification phase examines 2n   2 entries. // possible hypersink s = 1. Solution 2.” Information Processing Letters. // Now check whether s is indeed a hypersink. May 1982. the one possible hypersink) will have competed directly against roughly lg n other vertices. hsink = true. // s is a candidate for the hypersink for (i = s+1. say s. if the graph has edges (1. while (s < n) int i. (For details see King and Smith-Thomas.8. break. int s. and the vertex that is not eliminated is the “winner. i++) if (a[i][s] == 0) hsink = false. as follows. So the problem can be solved by examining at most 3n   lg n   3 matrix entries.2).) 7. break. then the edge wv. the outputs would be 12 23 31 13 32 21. The total is 3n   3.3. The explanation is that every vertex can reach s in G T because the graph is acyclic and has no other dead end. // Continue outer loop. determine how many vertices have no outgoing edges. Algorithm 4. i n.45 For simplicity we will output the edges of the desired path in sequence.3) and (2. ¯ ¯ ¯ At “Exploratory processing” (line 7) insert a statement to output the edge vw.

49 Use depth-first search. another. each separate component has at most n 2 vertices. When the search backed up to w. unvisited. This can be done by initializing v’s descendant counter to 1 when v is first visited and adding the descendant counter for each of its children when the search backs up from the child to v. keep a count of how many descendants of v (including v itself) have been visited. the algorithm outputs v and halts. . Since we have already visited at least n 2 descendants of v. We will show that v satisfies the property. count w was found to be less than n 2 (otherwise the algorithm would have quit there and output wµ. when v is removed. When v is removed. Each time the search backs up to v test n 2. Thus. subtree of v could have more than n 2 vertices. so we cannot if count v conclude that v satisfies the required property. Let w be the root of a subtree of v that has been visited. If the test is satisfied. each subtree of v will be a separate component. and the ancestors of v along with their descendants (other than v and its subtrees) will be one other component.Chapter 7 Graphs and Graph Traversals 61 7. If the test fails. For each vertex v. and the component that contains ancestors of v has at most n 2 vertices. and the traversal continues. so the subtree rooted at w has fewer than n 2 vertices. each subtree of v that has not yet been visited has at most n 2 vertices.

62 Chapter 7 Graphs and Graph Traversals .

In fact.) There are three reasons why the text does not give one of these more sophisticated implementations for the “standard” version of Prim’s algorithm or Dijkstra’s algorithm: 1. the total is about 2Cn. These methods don’t check all of the fringe vertices. say zw. W ´xyµ. a) Let Gn be a simple cycle of n vertices.) If the average size of the fringe is in Ω´nµ. If the weights are floating point. so each of the n getMins requires at most one comparison of edge weights. not the average over some set of inputs.Chapter 8 Graph Optimization Problems and Greedy Algorithms Section 8. b) Let Gn be a simple chain of n vertices. More sophisticated implementations can make the bookkeeping time proportional to the number of weight comparisons plus n. which in turn are discussed in Section 6. that a comparison of an edge weight with ∞ does not count. let C be some constant. (We can’t omit “plus n” because of pathological examples which the number of weight comparisons is o´nµ. But zw is in T2 . (We mean the average over the course of the algorithm. (We adopt the convention given in part (b) of the exercise. as given in the text.2: Prim’s Minimum Spanning Tree Algorithm 8. For a more general class. 2. using a Pairing Forest or a Fibonacci Heap. for 1 n   1. because it addresses a constant factor. for 1 i n   1 and an edge between 1 and n. .) The updateFringe subroutine only does an edge-weight comparison for the last vertex to enter the fringe. Section 8.3 Add xy to T2 forming a cycle. as is done in this exercise and in several others in the chapter. There is no broad class of graphs for which we know that the average size of the fringe is in o´nµ. there is an edge between the ith and i · 1st vertices. So W ´xyµ W ´zwµ W ´uvµ. not the asymptotic order of the algorithm. i Remarks: Focusing on weight comparisons.2. requires some justification. Then there are at most two candidate edges to choose from at one time. so even if the number of weight comparisons is of a lower order. such as we saw in part (b) above. that is. weight comparisons are substantially more expensive than integer or boolean tests. At least one edge in 8. and consider the class of connected undirected graphs consisting of the union of at most C simple cycles. 3. that is. the bookkeeping work will dominate for large enough n. Be sure that the errata in the first printing (available via Internet) are applied if you are using the first printing. By the MST property all edges in the cycle have weights the cycle is not in T1 . But this is only a partial justification. and we can only hope for improving the constant factor.1 Start at s. We preferred to present the main ideas as simply as possible.5 Note: Part (a) of this exercise was updated in the second printing. Let the start vertex be 1. 1 s 8 u 1 1 v w 8 8.7. Then there are at most two fringe vertices per simple cycle at any one time and updateFringe does at most one edge-weight comparison per simple cycle. there is an edge between the ith and i · 1st vertices. then any implementation of getMin that checks all of the fringe vertices will put the overall cost in Θ´n2 µ. Prim’s algorithm does “bookkeeping” work that requires time in T heta´n2µ. If the graph has n vertices.2.8 outlines the best ways that are known to improve the performance of Prim’s and Dijkstra’s algorithms.

So the total is ´n   1µ´n   2µ. b) All of them. Limited empirical results indicate that the last technique would be profitable when the average fringe size is n 20 or less.status[w] == unseen) insert(pq. keeps track of how many fringe vertices there are at any moment. 8. Then findMin finds the vertex in inFringe with the minimum fringWgt and swaps it with the vertex in inFringe[fringeSize]. This accounts for ´n   2µ · ´n   3µ · ¡¡¡ · 1 weight comparisons. Finally. One alternative is to use a (mutable) linked list to hold the fringe vertices. else” as the “if” will always be false.64 Chapter 8 Graph Optimization Problems and Greedy Algorithms With that said. we can briefly mention a few ways to reduce the bookkeeping in getMin to be proportional to the size of the fringe. We believe that the overhead of lists and complications of implementing all the cases correctly make these alternatives less desirable than the alternative described below. The asymptotic order is unchanged. 8. and they are stored in the range 1. evaluating the time of the algorithms by considering only edge-weight comparisons (plus a linear term to cover pathological cases) would be fully justified. every candidate edge is involved in a weight comparison to find the candidate with minimum weight.numPQ = n. It is also possible to use the List ADT in the text. its edge to every vertex not yet in the tree is involved in a weight comparison in updateFringe. but we do not know what graphs tend to have this property. .7.7 All n   1 edges immediately become candidate edges. incrementing fringeSize. It would require extensive experimentation to determine under what circumstances these techniques produce improved performance. With one of these alternatives in place. fringeSize. 1 s 8 u 1 1 8 v w 8 8. deleteMin can simply subtract 1 from fringeSize. 4 s 8 v w −8 . Another alternative is to maintain the set of fringe vertices in an array. by incorporating the idea of obsolete nodes described in Section 6.1 delete “if (pq. A counter. insert adds new vertices at the end. Later. getMin does ´n   2µ · ´n   3µ · ¡¡¡ · 1 weight comparisons over the course of the algorithm. fringeSize of inFringe. In the create function. w.2.3: Single-Source Shortest Paths 8. 8. but each time a candidate edge is selected. Consider the following figure.8 a) Each time a vertex is taken into the tree (after the start vertex).12 The source vertex is s. In addition. which is immutable. initialize pq. or ´n   1µ´n   2µ 2 in all.10 In Algorithm 8. newWgt). Section 8. say inFringe.14 Dijkstra’s shortest path algorithm will not work correctly with negative weights. The total number of weight comparisons is ´n   2µ´n   1µ 2. status[v] = fringe and fringeWgt[v] = ∞ for 1 v n. v.

1. Therefore. assume the theorem holds for k   1. the index of this sequence. The current candidate edge to y is not replaced.) We will show that after k times through the main loop the algorithm has computed the correct shortest vk . but all shortest paths to x with xy appended on the end give paths to y as short as the best already known. Note that two paths are called distinct if they have any diffence at all.Chapter 8 Graph Optimization Problems and Greedy Algorithms 65 The algorithm. The status array is replaced by a boolean array inFringe. but each best path to x gives a best path to y. then the algorithm vk 1 (by the inductive terminates. At this point. then d ´s t µ · W ´tvk µ is the shortest path distance from s to vk in G. a) This solution uses an n ¢ n weight matrix W to represent the graph. Suppose that Dijkstra’s algorithm adds vertices to its tree vn 1 . so again count[y] = count[x]. so the algorithm has computed the shortest paths for v0 If there are no candidate edges remaining at the beginning of the k-th pass through the main loop. path distances for the vertices v0 The proof is by induction on k. Initialize count[s] = 1. otherwise there can be an infinite number of shortest paths.18 We use the notation of dijkstraSSSP and Section 8. 8. if the candidate edge that is taken in the k-th pass through the main and v0 loop is tvk . x’s adjacency list is traversed. so count[y] = count[x]. The shortest path from s to s is 0 and d ´s sµ 0 before the main loop.3. The edge xy replaces the current candidate to y. with s as the start vertex. 8.6 (the correctness argument for Dijkstra’s algorithm) to find where it uses the hypothesis that the weights are nonnegative. Use an array count such that for any vertex u. y was an unseen vertex. Let y be a vertex on the list.2. There were no paths to y known previously. y is a fringe vertex. There are three cases to handle. will pull w into the tree with the path of length 4. Then the hypotheses of Theorem 8.2. hypothesis) and they are ∞ for vk 8.16 We use the notation of dijkstraSSSP and Section 8. and d ´s xµ · W ´xyµ d ´s yµ. y is a fringe vertex. so the edge with negative weight.3. The base case is k 0. We assume there are no edges of weight 0. Another exercise: Review the proof of Theorem 8. they may partially overlap.6 hold with vk in the role of z vk 1 in the role of V ¼ . . 3. This is the value that the algorithm vk . shortest path distances have been computed correctly for v0 vn 1 . For k 0. When a vertex x is added to the tree. and d ´s xµ · W ´xyµ d ´s yµ. This is where the count is initialized for each vertex other than s. 2. and then pull v into the tree.20 We use the notation of dijkstraSSSP and Section 8.2. the paths that had been the best paths to y are discarded. count[u] is the number of distinct shortest paths from s to u known so far. stores for d ´s vk µ. Therefore count[y] is updated as follows: count[y] += count[x]. which gives a shorter path to w. Now each best path to x gives a best path to y. (If some vertices are unreachable from s they are added arbitrarily to the end of in the sequence v0 ´ sµ v1 the sequence. never becomes a candidate.3.

treeSize < n. there is a lot of wasted space. y ++) float newDist = dist[x] + W[x][y]. the current path to y (of length // dist[y]) cannot have higher weight than the path through x to y. more sophisticated implementations can make the bookkeeping time proportional to the number of weight comparisons. int[] parent. best = ∞. y. since all vertices are in the fringe at the beginning. b) The procedure in part (a) is clearly simpler than Algorithm 8.) x = y. float[] dist. int x) int y. float[] dist. inFringe[x] = false.2 will do substantially fewer weight comparisons on some graphs. treeSize. for (y = 1. parent[y] = -1. parent. dist. float best. y++) dist[y] = ∞. int n. then weight comparisons are substantially more expensive than integer or boolean tests. // Continue loop.5. However. Also. dist[y] = newDist. updateFringe(W. // Initialization for (y = 1. and only one is removed during each pass of the main loop. As discussed in the solution of Exercise 8. n. /** Replace appropriate candidate edges with edges from x. this algorithm will always do a 2 total of about 1 2 n weight comparisons when searching the fringe vertices to find the candidate edge with lowest weight. // Main loop dist[s] = 0. this one uses Θ´n2 µ space for the weight matrix. for (y = 1. . boolean[] inFringe) int x. */ void updateFringe(float[][] W. if (newDist < dist[y]) parent[y] = x. x = -1. y n. x). y n. Algorithm 8. // We did not have to explicitly test whether y is in the fringe or the // tree because if it is in the tree. If the actual graph is sparse. y. // Continue loop. Like any algorithm that treats the graph as a complete graph. inFringe). /** Scan vertices with inFringe[x] == true to find one for which dist[x] is minimum. int s. Focusing on weight comparisons requires some justification because the bookkeeping in both algorithms takes time in Θ´n2 µ. boolean[] inFringe = new boolean[n+1]. if the weights are floating point. y ++) if (inFringe[y] && dist[y] < best) best = dist[y]. return this vertex. */ int findMin(float[] dist. y n. int[] parent. return x. int n) int x.66 Chapter 8 Graph Optimization Problems and Greedy Algorithms void sssp(float[][] W. inFringe[y] = true. for (treeSize = 0.2. although we do not know any general class of graphs for which this is true. return. treeSize++) x = findMin(dist.

CJ .Chapter 8 Section 8. there is a path between them that does not use e (since e is not in F ). This is a spanning tree now so remaining edges can be ignored. then there must be a path from v to w in F . Thus this algorithm includes a Union-Find program with at most 2m · 3´n   1µ instructions. 8. count = 0. Take EK . The simplest solution is to use breadth-first search starting at s. ´AB EFK GHLM µ. m). There is no straightforward way to use depth-first search. ´AB EFK GHLµ. Adjoining e to the path gives a cycle. Take GL. On the other hand. where two vertices are in the same equivalence class if and only if they are in the same connected component of the graph consisting of the edges read in so far. i ++) Read (v. if (IS(v.27 Use a dynamic equivalence relation (Section 6. Drop FK . The worst-case running time is in where lg£ is the slowly-growing function defined in Definition 6. Take GM . Read (n. Take GH . ´ABCGHLMEFK DJ µ. ´ABCGHLM EFK µ.6). When w is encountered. HM .6 of the text). else connected = false. Each IS is implemented with two cFinds and each MAKE uses two cFinds and one wUnion. if v and w are in the same connected component of F .26 Dijkstra’s shortest path algorithm could be used with all edge weights assigned 1.4: Kruskal’s Minimum Spanning Tree Algorithm Graph Optimization Problems and Greedy Algorithms 67 8. Take BC. ´ABCGHLMEFKDJI µ. Take AB. Take JM . ´ABC EFK GHLM µ. if (count == n-1) connected = true. Take FG. Drop LM . Take EF . Additional Problems 8. Take CM . i m. ´ABµ. AH . 8. but there is a lot of unneeded overhead. ´AB EFK µ. ´AB EF µ. ´ABCGHLM DJ EFK µ.9 (Section 6. O ´n · mµ lg£ ´n · mµ   ¡ . w) == false) MAKE v w. Take DJ . so v and w are in the same connected component of F . for (i = 1. Drop HL. ´AB EFK GH µ. w).22 If e forms a cycle with edges in F . ´ABCGHLMEFKDJ µ. CD. the path in the breadth-first search tree is a shortest path from s to w.24 The equivalence classes other than singletons are shown after each union. Take AI . count ++.

68 Chapter 8 Graph Optimization Problems and Greedy Algorithms .

Solution 1.2: The Transitive Closure of a Binary Relation 9. pathList). R[u][w] = 1. Algorithm 7. IntList remAdj. but it accesses the vertices on the recursion stack (the vertices on the path from the root to the current vertex). finishTime[v] = time. All-Pairs Shortest Paths Section 9. except 1’s on the main diagonal (in time Θ´n2 µ). // Number v and fill matrix entries for its ancestors. named pathList to store these vertices. time ++.”) . Solution 2. At postorder time for vertex v (line 13 in the DFS skeleton) we compute row v of the matrix R as follows: In a loop through v’s adjacent list. // "pop" v. to be described in a moment. Initialize R to zeros. else if (discoverTime[w] < discoverTime[v]) // vw is a back edge or cross edge For all u in pathList down to w (or to the end): remAdj = rest(remAdj). but we use a linked list. pathList = cons(v. so we will not give a complete solution. the two given here should be taken with a grain of salt. The algorithm is written recursively for clarity.Chapter 9 Transitive Closure. if (color[w] == white) reachFrom(w). for every vertex w that is adjacent to v (via an out-edge). Since the two solutions given here in part (a) are shown not to be very good in parts (b) and (c). For all u in pathList: R[u][v] = 1. but part (c) shows that it also misses some paths. Except for the postorder processing. a) We would expect students to try the following procedure (to be called repeatedly to start new depth-first searches from vertices that have not yet been visited).3. discoverTime[v] = time. // "push" v. “or” row w of R into row v of R. some students may come up with different solutions. An alternative second solution is sketched briefly. Assume the usual initializations done in dfsSweep. Perform a depth-first search according to the DFS skeleton. pathList = rest(pathList). perhaps with the embellishment described after it. the cost for the whole graph of these “steps” is in Θ´nmµ. That is. while (remAdj nil) w = first(remAdj). this take time in Θ´n · mµ. time ++. if R w i 1 set R v i to 1 for 1 i n. some paths will be missed. // Consider vertices adjacent to v. R is the (global) reachability matrix being computed. color[v] = black. (We put “step” in quotes because it is a for loop within a while loop—quite a substantial “step. remAdj = adjacencyList[v]. This could be implemented by not using built-in recursion and manipulating the stack directly. color[v] = gray. We will see when we look at the example in part (c) that this is not enough. The list operates like a stack except that all elements can be inspected without modifying the list. and pathList = nil. void reachFrom(int v) int w. The cost of this “step” for v is n times the out-degree of v.2 The point of this exercise is to see that depth-first search does not give a particularly good solution to the problem.

the algorithm seems to find long paths very quickly. . For solution 2. Then do more runs for values where you hope to find a higher maximum than you found in your small sample. For purposes of making Algorithm 8.1 requires more than four passes through the while loop. (Solution 1 uses a variation of this idea. which is studied in this chapter. in this case. . the depth-first search tree will contain all five vertices. ´1 q · 1µ. noting that p lg n. but it is about the best solution known for sparse graphs and has the same asymptotic order on dense graphs (O´n3 µ) as Warshall’s algorithm. including the pass to detect that nothing changed. Regardless of how large n is. that at most four passes are needed.4 Note to instructor: It is surprisingly difficult. For a sequence of p beginning at 2 and going as high as your computer can stand it.3: Warshall’s Algorithm for Transitive Closure 9. . followed by zero or more vertices in descending order. including a last pass in which nothing changes: 3 The final R matrix is: ¾ 5 1 1 1 1 1 1 1 0 1 1 0 1 0 1 7 0 0 1 0 0 0 0 2 0 1 1 1 1 0 1 0 0 1 0 1 0 0 4 0 1 1 1 1 1 1 6 0 0 1 0 1 0 1 ¿ 1 As we mentioned. and the total is Θ´n´n · mµµ. It is not easy to predict the course of the algorithm “by eye. The simple way to overcome the bugs in these solutions is to start a depth-first search from each vertex. where n is the number of vertices and m is the number of edges. on various tries).1 on this graph and report how many passes through the while loop were used. q+2) ´q · 2 2q · 2µ. and so on. for arbitrarily large n. Do about 10 such runs for each p to get an idea of what is happening. For solution 2.” Unless students check their constructions by programming and running Algorithm 8. 2)towraparound then(2. but instead of reading in a graph. merely giving a permutation of 1 Solution 1.1 to make four passes through the while loop. Form a conjecture about the general case. they are likely to jump to the wrong conclusions (as both authors have. to find a graph for which Algorithm 8. the authors have not been able to find a graph (with any number of vertices) that requires five or more passes. there are never more than n vertices on the stack. (q+1.1? It can be shown. Run Algorithm 8. Such a graph is specified by n. ´´q   2µq · 2 ´q   1µq · 2µ. Warshall’s algorithm and other methods studied in this chapter introduce algorithmic themes that are useful in other situations.70 Chapter 9 Transitive Closure. set n 2 p and generate a random permutation of vertices 1 through n. This obviously will do some repeated work. (q-1)q+1) then((q1)q+1. Solution 2. Here are the edges for a directed graph with 7 vertices that forces Algorithm 8. so the worst case time is in O´nmµ. Solution 3. Plot the maximum number of passes observed for each p. from left to right.1 run for as many passes as possible through its while loop. if not impossible. 2q+1) . the times for various parts were given in part (a). and there are many other variations. then ´´q   1µq · 2 3µ to wrap around. ((q-2)q+1.1. c) For solution 1. have a procedure to generate one.) How many passes will the while loop take in Algorithm 8. no matter how long the path is. Let q be about n. The themes of this chapter are revisited in Chapters 12 and 14. it is sufficient to consider graphs whose directed edges form a single simple path through all n vertices. Ô . Section 9. Make directed edges as follows. if reachFrom is called with v1 as the root. followed by zero or more vertices in ascending order will be found in one pass of the while loop. Program Algorithm 8. Make n   1 directed edges between adjacent vertices in the random sequence. Note that a path beginning at any vertex. but the paths from v2 v3 and v4 to v5 will not be recorded in R. All-Pairs Shortest Paths b) For solution 1.1. any path that requires a back edge will be missed.

void routeTable(float[][] D. i++) for (j = 1. Algorithm 9. ¾ 0 2 4 3 3 0 7 3 5 7 0  3 ∞  1 4 0 ¿ ¾ 0 2 4 3 3 0 7 3 5 7 0  3 2  1 4 0 ¿ ¾ 0 2 4 1 3 0 7 3 5 7 0  3 2  1 4 0 ¿ ¾ 0 0 4 1 3 0 7 3  1  4 0  3 2  1 4 0 ¿ b) As long as there are no negative-weight cycles. 9. Initially. k++) for (i = 1. j n.Chapter 9 Section 9. 9. else go[i][j] = j. j++) if (D[i][j] == ∞) go[i][j] = -1. j n. i n. the first shows the matrix after all iterations of the inner for loops for k 1. j.8 a) The matrices below show the steps to calculate the distance matrix. the second after the inner loops for k 2. let E 12 24 43 . int[][] go. D contains the weight matrix for the graph. first step from i is to j. // Find shortest paths and record best first step in go. The last matrix is the distance matrix.4 is guaranteed to find the shortest simple path. It will incorrectly leave D 1 3 with the value ∞. i++) for (j = 1. there is always a simple shortest path between any pair of nodes. The algorithm.10 The following algorithm computes the routing table go as well as the distance matrix D. computes Initially D 2 3 D 1 3 before it computes D 2 3 and D 1 4 . regardless of whether weights are negative or not. int n) int i. and let the weight of each edge be 1.7 ¼ D 0 3 5 4 2 0 4 1 4 7 0 4 3 3 3 0 ½ 9. .4: All-Pairs Shortest Paths in Graphs 9. That is. 1 3 2 4 D1 3 D1 4 ∞. for (i = 1. i n. k n. for (k = 1. j++) if (D[i][j] > D[i][k] + D[k][j]) D[i][j] = D[i][k] + D[k][j]. // Initialize go: If edge ij exists.6 Let V Transitive Closure. All-Pairs Shortest Paths 71 1 2 3 4 . k. go[i][j] = go[i][k]. etc. with the modification described in the exercise.

72

Chapter 9

Transitive Closure, All-Pairs Shortest Paths

9.12 The shortest nonempty path from one vertex to itself is the shortest cycle. In Algorithm 9.4, if we initialize D i i ∞ for 1 i n (after copying W into D), then run this algorithm, the final value in D i i is the length of a shortest cycle containing i, as long as there are no negative-weight cycles. There is a negative-weight cycle if and only if some D i i 0. If the graph is undirected, the algorithm will incorrectly interpret an edge vw as a cycle v w v. Although the exercise did not explicitly require discussion of negative-weight cycles, it is worth knowing that they can be dealt with fairly straightforwardly. Should a negative-weight cycle be discovered when Algorithm 9.4 terminates, then other nodes might be in nonsimple negative-weight cycles that have not been discovered yet. One simple way to find all such nodes is to set D i i to  ∞ wherever D i i 0 and use this matrix as input for a second run of Algorithm 9.4. (This time, of course, do ∞ before starting. We might need to evaluate ∞ · ´ ∞µ; the appropriate value is ∞, denoting that the not set D i i path doesn’t exist.) Section 9.5: Computing Transitive Closure by Matrix Operations 9.14 vk ´ wµ be a path from v to w with k n. Since there are only n vertices in the graph, some Let P v0 ´ vµ v1 vi v j·1 vk is also a vertex must appear more than once on the path. Suppose vi v j where i j. Then v0 path from v to w, but its length is strictly smaller than k. If the length of the new path is still larger than n, the same shortening process (removing loops in the path) can be used until the length is less than n. Section 9.6: Multiplying Bit Matrices—Kronrod’s Algorithm 9.16 Element j ¾ Ci if and only if
n

aik bk j
k 1

1 if and only if there is a k such that aik
k¾Ai

1 and bk j

1 if and only if there

is a k ¾ Ai such that j ¾ Bk if and only if j ¾ Additional Problems

Bk .

9.18 If the graph is undirected, compute the Boolean matrix product A3 and see if there is a 1 on the main diagonal. If the graph is directed and we interpret a triangle as a simple cycle with three edges, we must avoid using selfloops. So copy the adjacency matrix to a new matrix T , replace all main-diagonal entries with zero, compute T 3 , and see if there is a 1 on the main diagonal. If straightforward matrix multiplication is used, the number of operations is in Θ´n3 µ.

Chapter 10 Dynamic Programming
Section 10.1: Introduction

10.2

if (n 1) return n; prev2 = 0; prev1 = 1; for (i = 2; i n; i ++) cur = prev1 + prev2; prev2 = prev1; prev1 = cur; return cur;

Section 10.3: Multiplying a Sequence of Matrices

10.4 Let n high   low be the problem size. When k low · 1, line 7 of the mmTry2 procedure in the text makes a recursive call with problem size n   1. When k high   1, line 6 makes another recursive call with problem size n   1. The loop beginning at line 5 executes n   1 times, and makes 2 recursive calls in each pass, for a total of 2n   2. For n 2, we have 2n   2 n. Therefore, letting T ´nµ denote the number of recursive calls made by mmTry2 on a problem of size n: T ´nµ For an easy¡lower bound, we claim T ´nµ 2n   2 for n   n 2 2  1   2 · n 2n · n   4 2n   2 for n 2. 2T ´n   1µ · n for n 2 T ´1µ 0 0 and 1. The proof is simple by induction, since 21   2

10.5
¾

cost

0 600 2800 1680 0 1200 1520 0 2400 0

¿

¾

 1

1  1

last

1 2  1

1 3 3  1

¿

multOrder

¢

2 31

£

10.7 Let the dimensions of A, B, and C be 100 ¢ 1, 1 ¢ 100, and 100 ¢ 1, respectively. Section 10.4: Constructing Optimal Binary Search Trees

10.9 a) In the matrix below, expressions p · q w mean that p p´i jµ, the weight of elements i through j, and q is the minimum weighted cost of two subtrees, for various choices of root from i through j, so the w is the optimum weighted cost for subproblem ´i jµ.

74

Chapter 10

Dynamic Programming
¾ ¿

0

0 20 0

0 44 · 0 20 0 60 · 0 36 0 88 · 0 80 0 92 · 0 88 1 00 · 1 08 0 64 0 96 1 68 1 80 2 08 0 24 0 0 40 · 0 16 0 68 · 0 52 0 72 · 0 60 0 80 · 0 72 0 56 1 20 1 32 1 52 0 16 0 0 44 · 0 16 0 48 · 0 20 0 56 · 0 32 0 60 0 68 0 88 0 28 0 0 32 · 0 04 0 40 · 0 16 0 36 0 56 0 04 0 0 12 · 0 04 0 16 0 08 0
¾ ¿

cost

 1
root

 1

1

2 2  1

2 2 3  1

2 3 4 4  1

2 3 4 4 5  1

2 4 4 4 6 6  1

b) A

ºB ²

ºD² C ºF
E

10.10 We build a binary search tree where each node has a key field Kk . The buildBST function is called with
T = buildBST(1, n).

Here’s the function buildBST. To simplify the notation, we let the matrix root be global.
/** buildBST constructs the tree in postorder using the global root matrix. The current subtree being constructed will contain the keys K f irst Klast . */ BinTree buildBST (int first, int last) BinTree ans; if (last < first); ans = nil; else int k = root[first][last]; // index of the root’s key BinTree leftSubtree = buildBST(first, k-1); BinTree rightSubtree = buildBST(k+1, last); ans = buildTree(Kk , leftSubtree, rightSubtree); return ans;

Since there is no loop within the procedure and each call generates one new node, the running time is in Θ´nµ.

Tree 9 costs 1. Tree 4 costs 1 + Tree 5 + Tree 6 = 31. Tree 11 costs 1. 1. k-1) + C(n-1. else ans = C(n-1. 10. Tree 6 costs 1 + Tree 7 + Tree 8 = 10. and there will be more than C´n kµ leaves in the recursive call tree. For Method 1. . Tree 5 costs 1 + Tree 6 + Tree 7 + Tree 8 = 20. the number of recursive calls to C (and the number of additions) will in general be exponential in n. Since the depth of the call tree will be n. Section 10. k). Tree 7 costs 1 + Tree 8 + Tree 9 + Tree 10 = 6. this is a very inefficient computation. we are given the precondition that n k. “Tree k” refers to the subproblem tree rooted at subproblem k and the cost is the number of nodes in the tree. This is true because C´n kµ ´n kµk . return ans. . c) In the cost calculations below for the non-DP recursive version. else if (n == 0) ans = 0.14 penalty 343 1 64 64 0 472 b) The subproblems that are evaluated are those beginning at words 11. The strategy is nonoptimal for the input A´ 41µ B´ 40µ C´ 20µ because it will make a stringy tree to the right with weighted cost 0 41 · 2 £ 0 40 · 3 £ 0 20 1 81. Tree 3 costs 1 + Tree 4 + Tree 5 = 52. the recursive function definition is: int C(int n. We use this fact in some of the solutions. Tree 1 costs 1 + Tree 2 + Tree 3 + Tree 4 = 168. The balanced BST with root B has weighted cost 2 £ 0 41 · 0 40 · 2 £ 0 20 1 62.5: Separating Words into Lines 10.16 It is not hard to see that if k n then C´n kµ 0. Tree 10 costs 1.Chapter 10 Dynamic Programming 75 10. including the main problem. 1. Θ´nµ stack space is used. Tree 2 costs 1 + Tree 3 + Tree 4 = 84. However. Then recursively build the left subtree for the range first through k   1 and build the right subtree for the range k · 1 through last (using the same greedy strategy). choose the most probable key in the range as the root of the BST. Each subproblem is evaluated once in the DP version. so the total is 11. so other cases are computed only at the algorithm’s convenience. a) words Those who cannot remember the past are condemned to repeat it X 7 1 4 4 0 As with the Fibonacci numbers. 9.12 For keys in the range first through last. Additional Problems 10. Suppose its index is k. 8. Each node represents one activation frame. int k) if (n 0 && k == 0) ans = 1. Tree 8 costs 1 + Tree 9 + Tree 10 = 3.

C k k are equal to 0 and are not needed in the computation. and all entries in the top row. i ++) numerator = numerator * i. i n. for (i = 2. computing numerator could cause integer overflow. All entries above the diagonal consisting of C 0 0 . 3. Method 4 could be implemented with simple loops in O´n · kµ time as follows: // Compute n! numerator = 1. However. hence k´n   k · 1µ additions. At most nk entries are computed. Floating point numbers even overflow their exponent parts for n! when n has fairly modest values. so the integer division j is precise. Thus only entries in a strip of n   k · 1 diagonals need be computed. Since the computation of each entry needs only entries in the previous row.1. 10. To compute an entry C i j only two neighbors in the previous row are needed. at most two rows need be stored. so at most nk additions are done. A little thought shows that this solution can be improved. Because k is made less than n 2. All entries on the diagonal are equal to 1. C n k are not needed in the these can be initialized. After j passes through the loop. even when the quantity C´n kµ is well within the range of values that can be represented on the machine. All such . the two partial rows the algorithm is using at any one time can be stored in 2´n   k · 1µ cells.76 Chapter 10 Dynamic Programming 2. Method 3 could be implemented with one loop in Θ´kµ time as follows: if (k > n/2) k = n-k. which is an integer. The critical range is Note the parenthesization. 4. i ++) C = (C * (n-i)) / (i+1). If floating point arithmetic is used there is the possibility that the answer will be inaccurate and might not be an integer. With careful indexing. i ++) denominator = denominator * i. For Method 2. So the table can easily be filled until C n k is computed. i k. . for (i = 2. i < k. intermediate values will be about k times as large as C and may cause an integer overflow that might have been avoided in the first two methods. // Compute k! * (n-k)! denominator = 1. C = 1. // Compute C C = numerator / denominator. for (i = 2. the brute force string matching algorithm). n . so this method used O´kµ space. The only drawback is that for a small range of inputs it will overflow when the first two methods do not. initialized to 0. i ++) denominator = denominator * i.18 Many students try to solve this problem by a brute force method using index variables to keep track of the current location in each string (more or less generalizing Algorithm 11. Entries below the diagonal consisting of C n   k 0 computation. for (i = 0. This method is clearly faster than the first two and uses less space (essentially none). That includes k´n   k · 1µ entries. This method is clearly superior to the first one. i n-k. we could use a table C 0 n 0 k with all entries in the lefthand column initialized to 1. if n and k are large. C maxint k n k maxint The moral is that mathematical transformations may pay bigger dividends than clever programming. other than the first. the value of C increases monotonically. However.

It can be implemented as a two-dimensional array lcsLength[n+1][n+1]. increment i and j. the longest common subsequence. len = lenB. not its contents. // point 2 return len. If the first elements of both sequences are equal. but it uses more space (the whole n ¢ m table).Chapter 10 Dynamic Programming 77 solutions that I have seen were wrong. if (lenA > lenB) action = ’A’. The significance of point 1 and point 2 is explained below. remove one or the other. or B. The values of lcsAction[i][j] will be one of the characters T. j). j) if (i > n) return 0. the remaining entries are computed as follows: Si j S i 1 j 1 0 ·1 if xi y j otherwise The table entries can be computed row-by-row. . This method also takes Θ´nmµ time. Each such string corresponds to a common substring.19 We can construct a common subsequence by a variant of the merge procedure of Section 4. adding their common value to the common subsequence. lenB = findLCS(i. we also want to record the decisions that led to the longest common subsequence. they missed some valid matches. remove both. A. The outline of a recursive solution is int findLCS(i. However. j+1). j+1). The answer to the problem is the largest entry. Actually. len = 1 + findLCS(i+1. Define problem ´i jµ to be the problem of finding the longest common subsequence of A i n and B j n . If they are unequal. the algorithm scans all diagonals (slanting downward to the right) for the longest string of contiguous 1’s. Discard the element at B j and increment j. Computing each table entry and updating the maximum if necessary clearly takes constant time so the total time in the worst case is in Θ´nmµ. To convert into DP. Our first solution (dynamic programming) uses a table S 0 n 0 m where S i j = the length of the longest common substring ending at xi and y j . Discard the element at A i and increment i. define a dictionary with the id ´i jµ. To find. we need to backtrack over both choices of which element to remove when the front elements are unequal. it is easy to keep track of the largest value computed so far while filling the table. An alternative method uses a table M 1 n 1 m where Mi j 1 if xi y j 0 otherwise After filling the table. only two rows need actually be stored. The top row and lefthand column are initialized to contain 0. else action = ’B’. meaning: T A B Take the element at both A i and B j . len = lenA. The worst case time for a correct brute force algorithm would probably be cubic. When we convert into the DP version we will also record the decisions that allow us to construct it. in the first version we will only find the length of the longest common subsequence. else lenA = findLCS(i+1. Then continue with the remaining subsequences. // point 1 if (A[i] == B[j]) action = ’T’. 10.5 in the text. so we define a second two-dimensional array lcsAction[n+1][n+1]. if (j > n) return 0.

The processing per “vertex” and per “edge” is in O´1µ. The second solution is similar to the solutions for the problems discussed in Section 10. Let the table be M 1 n 0 a . and subsets of s1 that do contain si . to determine if a sum j can be achieved using some subset si .21. the answer to the problem is “yes. computation of each entry uses only the previous row. Exercise 10. The dictionary requires space in O´n2 µ. c) Let the coin denominations be 25. In the terminology of Chapter 13. Solution 2. Computation of each table entry takes constant time. Initialize the entries of lcsLength to –1 to denote that the solution is unknown for that entry. If point 2 is reached. Each subproblem calls two other subproblems so the subproblem graph has about 2 n2 edges. and one quarter).S). and so on. one nickel. If any entry in the last column is found to be true.21 If i 1 ∑ si is odd. In general. Modify the recursive findLCS to check whether lcsLength[i][j] is nonnegative at point 1. We don’t know what the first coin should be. Suppose ∑ si is even. S may be very large relative to n.2. Let M j be the minimum number of coins needed to make change for j cents. Use a table P 1 si sums to j n n 0 H where Pi j true f alse if some subset of s1 otherwise All entries in column 0 are initialized to true (the empty set has sum 0). and three pennies. The answer will be in M n a . and one quarter. Let H i 1 n n 1 2 i 1 ∑ si . return its value immediately. not the number of data. Computing each entry takes constant time. Other algorithms whose running time varies with the magnitude of the data are considered in Section 13.” As we have seen with many dynamic programming problems. the algorithm does not have to store the whole table. and 1. use two half-dollars. store len and action in lcsLength[i][j] and lcsAction[i][j] before returning. For example. Then using any of the coins c1 Mi j min´M i   1 j 1 · M i j   ci µ where the second term is considered to be ∞ if j ci . After deciding on the first coin. There is a solution very similar to the solution for the Partition Problem. to make change for $1. one dime.78 Chapter 10 Dynamic Programming In the above procedure. So half-dollars would be used for all but at most 54 cents (the value of four pennies.43 (U. then as many of the next largest as possible. and try to make change for 30 cents. Remember that H is computed from the data.23 a) Use as many of the largest coin as possible. but not preserved after the procedure returned. b) An optimal way to make change must use at most four pennies (five could be replaced by one nickel). so the total time used in the worst case is in Θ´nH µ Θ´nSµ. where M i j is the minimum number of coins needed to make change for j cents ci . for which we can find the answer in P i   1 j . 10. continue as before. one nickel (two could be replaced by one dime). otherwise. it is easy to see that the greedy algorithm is optimal. So Pi j P i   1 j or P i   1 j   si where the second term is retrieved only if j si and is considered false otherwise. this is not a polynomial-bounded algorithm. action was set accordingly. it is approximately the magnitude of the data. one quarter. two dimes (three could be replaced by a quarter and a nickel). we consider subsets that do not contain si . two dimes. The rest of the first row is initialized with P 1 s1 = true and all other entries false. 10. so the worst case time is in Θ´naµ. By considering all the remaining cases. so we . d) Solution 1. so depth-first search of the subproblem graph requires time in O´n2 µ. If so. where the rest of the subset must sum to j   si . There are about n2 subproblems. one nickel.3.1). A wrapper initializes the dictionary and calls findLCS(1. 10.5. so we find the answer in P i   1 j   si . we would make change for the remaining amount in the optimal way. there is no solution.

i ++) T[i][i] = T[i-1][i-1] # j ). j ++) T[1][j] = min(T[1][j-1]. Thus the worst case time is in Θ´naµ. i To find the answer. 10. The initialization and search for the answer take Θ´nµ steps. . and only two columns need be stored. At most i songs can fit on the five 60-minute disks satisfying the constraints of the problem. Only the entries on and above the main diagonal are needed. but computing an entry may take as many as Θ´nµ steps. Computing each table entry takes constant time. j 500. In general.26 We use a table T 1 500 1 500 as described in the hint in the exercise. We define a summing operation # (to compute the amount of time a sequence of songs takes up on 60-minute disks) as follows: a#b a·b 60k · b ´a if ´a mod 60µ · b 60 mod 60µ · b 60 and a 60 k j. for (i = 2. The collection of i songs whose total time is represented in T i j may or may not include Ti j min´T i j   1 T i   1 j   1 # jµ so The entries can be computed column-by-column (only down as far as the main diagonal). i 500. search the last column from the bottom up for the first entry (the largest i) such that T i 500 300. The top row and the main diagonal can be initialized by T[1][1] = 1 .Chapter 10 try all possibilities. for (j = 2. let n be the number of songs. Note that only a onedimensional table is used here. M j min ´1 · M 1 i n j   ci µ Dynamic Programming 79 where M 0 0 and terms with index less than zero are ignored (or treated as ∞). so the total time is in Θ´n2 µ.

80 Chapter 10 Dynamic Programming .

// Match found break. k = P. // Slide pattern forward. . match = nil. j = rest(j). 11. IntList T) IntList match. i = j. Delete all other references to i. Since j and k are incremented together when characters from the pattern and text match. k. replace “match = i” with “match = j .1.1 In Algorithm 11. OpenQueue create(int n) precondition: n > 0. Because the invariant i j k·1 is satisfied in Algorithm 11. j   m must be where the match began in the text. and k become (references to) linked lists and “++” becomes the rest operation. // j begins at the current character in T. 11. maxLength(q) = n.2 The idea is that i.backup” has the same effect as “k = 1.1.2: A Straightforward Solution 11. Also. start over.” IntList listScan(IntList P.backup” has the same effect as “j = i” in that algorithm. // k begins at the current character in P.m” (on the line with the comment “Match found ”). k = rest(k). a) Looking ahead to part (c) we let elements of OpenQueue be type char instead of Object. return match. // Continue loop. j = T. 2. The pattern length m is not needed because the list will be empty when the pattern is exhausted. int maxLength(OpenQueue q) precondition: q was created by create. the statement “j = j . 3. q refers to a new object. j = i. j. while (j nil) if (k == nil) match = i. // i is the current guess at where P begins in T. when the algorithm terminates with k m · 1. else // Back up over matched characters. // value to return IntList i. postconditions: If q = create(n) 1.4 Answers can vary considerably for this open-ended question. if (first(j) == first(k)) j = rest(j). k = P. “k = k . depending on the software-engineering style. i = j.Chapter 11 String Matching Section 11. j. length(q) = 0.

void deq(OpenQueue q) int n = q.82 Chapter 11 String Matching int length(OpenQueue q) precondition: q was created by create. 1.count ++. For 03.elements. whose position is 0. All methods in Text are public static. // circular array of elements void create(int n) OpenQueue q = new OpenQueue(). // index of the first element in the queue int rear. void enq(OpenQueue q.length. q. p) = getElmt(/q/.1.rear] = e.elements[index]).front = (q. p).rear = n-1.elements = new char[n]. q. length(q) = k . getElmt(q.rear + 1) % n. q. precondition: length(q) > 0. q.elements. < k. q. getElmt(q. return (q. q. p). precondition: length(/q/) < maxLength(/q/). Although a method getPosition is provided for completeness. . // number of elements in the queue char[] elements. 0) = e. 2. 1. int maxLength(OpenQueue q) return q.position + n) % n. int index = (q. q.front = 0. q. 0 position < length(q). p getElmt(q. For 0 p < k-1. // index of the last element in the queue int count. length(q) = k + 1.count --.front + 1) % n. int length(OpenQueue q) return q. postconditions: Let k = length(/q/). c) Note that position counts backward (to the left) from the most recently read text character. q. postcondition: Let k = length(/q/).length. int position) int n = q. b) All methods in OpenQueue are public static. char e) Notation: /q/ denotes the queue before the operation.elements. none of the pattern-matching algorithms need it. p+1) = getElmt(/q/.count = 0.rear = (q. void enq(OpenQueue q.elements. class OpenQueue int front.length.elements[q. char e) int n = q. char getElmt(OpenQueue q. int position) preconditions: length(q) > 0.rear . char getElmt(Queue q.length. void deq(OpenQueue q) Notation: /q/ denotes the queue before the operation.count. 2.

T.fp or endText(T) is true.forward. T. int m) Text T = new Text().q). // c is either if -1 (c or == in -1) the range of the type char.q).fp = input. forward): * If forward getPosition(/T/). String Matching 83 /** endText(T) is true if an earlier advanceText(T) encountered end-of-file. * Postcondition for advanceText(T. /** Precondition: backup + getPosition(/T/) < length(T. if (length(T.position = newpos. enq(T. boolean endText(Text T) return T.Chapter 11 class Text OpenQueue q.read(). newpos = T. while (newpos < 0) int c = T. */ /** Postcondition: getPosition(T) = getPosition(/T/) + backup. then getPosition(T) = getPosition(/T/) . getPosition(T) = 0 and either (forward .create(m+1).q.eof = true. T.q)) // there’s no room. T. */ T. delete one first deq(T.position = 0. newpos = 0. int position.q) == maxLength(T.position .position += backup. * Otherwise.position)). Reader fp.eof = false. */ /** Notation: /T/ denotes T before the operation.3: The Knuth-Morris-Pratt Algorithm 11.6 a) b) c) A A A B 0 1 2 3 A A B A A C A A B A B A 0 1 2 1 2 3 1 2 3 4 5 1 A B R A C A D A B R A 0 1 1 1 2 1 2 1 2 3 4 . int backup) T. */ Text create(Reader input. char getTextChar(Text T) return getElmt(T. T. // add 1 for safety.forward.q = OpenQueue. /** m should be the pattern length. */ Section 11. T.fp. (char)c). void backupText(Text T. newpos ++. boolean eof.eof. int getPosition(Text T) return T.getPosition(/T/)) * new characters were read from T. break. */ void advanceText(Text T.int forward) int newpos. return T.q.position.

k m. j can increase at most n times. pattern character.4: The Boyer-Moore Algorithm 11. and never decreases. Then charJump t j . Since k is incremented the same number of times as j (note that in the second if statement. b) otherwise. Section 11. for (k = 2. suppose t j is the same as pm . in the last of the three if statements. and the loop terminates when j n (where n is the length of T ). b) 2n   ´m   1µ (for m c) Let Q consist of m   2 A’s followed by two B’s. where the character comparison is done. is executed at most 2n times. is incremented by exactly 1 whenever it is incremented. k can decrease at most n times. k is incremented at most n times. Then kmpSetup will do 2m   5 character comparisons (for m 2). Since j begins at 1. Thus the last if statement. 11. k is assigned 1). Since k starts at 1 and is never negative. To see that this can happen. So we count how many times j is incremented overall and how many times k is decremented. that should never happen. 11. 11. k ++) if ( pk == p f ail k ) fail[k] = fail[fail[k]]. Each time this if statement is executed. if k 0.15 a) charJump[A] charJump[B] charJump[C] charJump[D] charJump[R] charJump[x] charJump[A] charJump[C] charJump[R] charJump[S] charJump[T] charJump[x] = = = = = = = = = = = = 0 2 6 4 1 11 0 5 1 3 2 11 otherwise.12 Add the following loop at the end of the procedure kmpSetup (after the end of both the while loop and the for loop in the procedure). The pattern would shift left instead of right.8 At most one character comparison is done each time the body of the loop is executed.84 Chapter 11 d) String Matching A S T R A C A S T R A 0 1 1 1 1 2 1 2 3 4 5 11. the last 0. either j is incremented (along with k) or k is decremented.17 It is possible that charJump[t j ] is so small that j · charJump t j is smaller than the index of the text character that is currently lined up with the end of the pattern. 2).10 a) The fail links are A B A A A ¡ ¡ ¡ 0 1 2 ¡ ¡ ¡ m 2 m 1 The procedure kmpSetup does m   2 character comparisons to compute them.

to find only // non-overlapping copies. one can do so. The entry in the length table is updated as follows.Chapter 11 String Matching 85 The distance from t j to the text character that is currently lined up with the end of the pattern is m   k. the longest suffix and prefix that “match” are length 0). leave j unchanged. if (k > m) Output (j-m). D i   1 j   1 · 1 (if pi t j ) 3. the number of new characters of text read is matchJump k   ´m   k µ k   m · matchJump k This doesn’t count text characters already read while matching right-to-left from pm to the current pk . If one wants to bias the algorithm in favor of a particular kind of mismatch. . k for some j. length[i][j] = length [i][j-1] +1. k = 1. 2. . In the algorithm. It outputs a list (possibly empty) of indexes in T where a copy of P begins. Additional Problems 11.e. 5. // This will find overlapping copies. length[i][j] = length [i-1][j]. 3. j) == false) // New outer while loop Unchanged while loop from Alg.21 Use an additional 2-dimensional array length such that length[i][j] is the length of the text segment ending at t j that matches the pattern segment p1 ¡¡¡ pi . according to which case above gave the minimum value. If charJump[t j ] matchJump[k]. D i   1 j   1 (if pi t j ) 2.1 and 11. combined with cases in which pk·1 ¡¡¡ pm matches an earlier substring of P or some suffix of pk·1 ¡¡¡ pm matches a prefix of P or neither (i. the number of differences between the two strings at this point.19 Assuming that matchJump[k] slide k charJump[t j ]. m-k+1). D i j   1 · 1 (the case where t j is missing from P). 1.3). Initialization lines unchanged. then the number of new text characters needed is greater than the above amount. 4. Note that more than one of the cases may occur (more than one of the values considered when finding the minimum may have the minimum value). that is D i j . .1 or 5. where m is the pattern length and k is the maximum number of differences Now suppose D m j allowed. D i   1 j · 1 (the case where pi is missing from T )..23 The following modification works for both the straightforward algorithm and Knuth-Morris-Pratt (Algorithms 11.3. is computed by taking the minimum of the following values: 1. length[i][j] = length [i-1][j-1] +1. length[i][j] = length [i-1][j-1] +1. . 4. Thus there may be more than one correct value for length of the matched segment. Then length[m][j] is the length of the text segment that matches the pattern. Consider cases in which t j does or does not occur in P. j = j-m+1. 11.5: Approximate String Matching 11. Another exercise: Determine in exactly which cases charJump[t j ] matchJump[k] when a mismatch occurs. Do you need to split these cases still further? Section 11. // Continue main while loop. so the correct way to increment j to force the pattern to move at least one place to the right is j += max(charJump[t j ]. while (endText(T.

an outer while loop is added as above. 11. j = j+m+1. The new if statement that follows the original (now inner) while loop would be if (k == 0) Output (j+1).6).25 Use the Knuth-Morris-Pratt algorithm to determine if there is a copy of X in YY (that is.86 Chapter 11 String Matching For the Boyer-Moore algorithm (Algorithm 11. // j += 2*m for non-overlapping instances k = m. two copies of Y concatenated together). . Setting up the fail links takes Θ´nµ time and the scan takes Θ´2nµ = Θ´nµ time.

Since we were given the answer. Then subtract 1 and divide by x   1. x 1 k x2 . the simplest solution is to verify it by induction. x4 . 12.2 x n ·1   1 . that is.10 a) n. To get xn·1 . as required for k 1. so xn·1 can be computed by multiplying together the Now.1. 21 appropriate terms.6 Since n 2k   1.8 No */’s are required because each term ui vi in the dot product can be computed by adding ui copies of vi .4 In a straight-line program.Chapter 12 Polynomials and Matrices Section 12. start with t x and multiply t by itself k times. Start by expanding the recurrence to see the pattern of the terms. The last expression simplifies to 3 ¡ 2k 1   2. b) n · 1. For k 1.2: Evaluating Polynomial Functions 12. a) Observe that p´xµ xn · ¡¡¡ · 1 12. 3 ¡ 20   2 1. for a total of at most 2k · 3 2 lg´n · 1µ · 3 arithmetic operations on the data. we use the fact that A´1µ A´kµ for 1 j k 1. a step of the form s = q 0 would be illegal. Let k lg´n · 1µ .3: Vector and Matrix Multiplication 12. ∑ 2 i · 2 k  1 ¡ 1 2 k   2 · 2 k  1 3 ¡ 2k 1   2 Section 12. x. b) Observe that p´xµ ´x · 1µn . In the proof of Lemma 12. Solution 2. x8 2k . the latter by the inductive hypothesis. 0 is substituted for an wherever it appears. A´kµ 2A´k   1µ · 2 2 ¡ 3 ¡ 2k 2   2. we need to show that A´kµ 3 ¡ 2k 1   2. Let’s suppose we weren’t given the answer and want to discover it. saving the intermediate results. If there were any steps of the form s = q an this transformation would not produce a valid program. . This can be computed with at most by 2 lg n · 1 arithmetic operations on the data by the same technique used in part (a). x2 . n · 1 is the sum of powers of 2 among 20 . 12. using at most k more multiplications. A´kµ 2 · 2A´k   1µ 2 · 2 2 · 2A´k   2µ 2 · 4 · 4A´k   2µ 2 · 4 · 4 2 · 2A´k   3µ 2 · 4 · 8 · 8A´k   3µ 21 · 22 · ¡¡¡ · 2 j · 2 j A´k   jµ k 1 i 1 So let j k   1 to get a sum without any recursive A terms. Solution 1.

Here’s the algorithm. Suppose 1 ω2 ´ω2 µ2 k m are not distinct. An alternative argument for Property 2: In polar coordinates. but not the sum itself. so we need to compute ac   bd and ad · bc using only three multiplications. c. then ωc n.bd imaginaryPart = z . which is about 1 618n 1. Let ω and ω be two of these terms that are equal. and ω2 is a primitive ´n 2µ-th root of 1. ω ´1 π µ  1.12 a) If n n  1 c j ´ω µ j 0 1. For Property 2: The square roots of 1 are 1 and –1. but ωn 2 1 because ω is a primitive nth root of 1.) Additional Problems 12. f) The recurrence equation for each operation is A´kµ 2A´k   1µ · n 2 for k 1 A ´0 µ 0 This method does roughly the same number of arithmetic operations on the data as recursiveFFT. The corresponding element in G3 is ω2´i·n 2µ j ω2i j·n j ω2i j since ωn j 1. There is also bookkeeping proportional to lg´nµ to find the 1-bits in the binary form of n. But k and m are both less than n   1. The recursive computation of Fn requires about Fn 1 +’s. so ωn 2 ´1 n 2 n 2 ¡ 2π nµ 12. and (d). ω ωi ω2i j ωi´2 j·1µ G2 i j . b) G1 i j ωi´2 j·1µ .14 Let a. and ω2 is a primitive ´n 2µ-th root of 1. plus bookkeeping.4: The Fast Fourier Transform and Convolution 12.4. If n divides c. realPart = u . The corresponding element in G4 is ω´i·n 2µ´2 j·1µ ωi´2 j·1µ·n j·n c) G2 i j n j 1 and ωn 2  1.” . each involving eight multiplications and four additions. Thus the terms must be distinct.88 Chapter 12 Polynomials and Matrices Section 12. so the roots don’t sum to zero. b. and d be the given data. The formula for FnV follows from (a). so ωn 2 must be –1. contradicting the fact that ω is a primitive nth root of 1. The product of these complex numbers is ´ac   bd µ · ´ad · bcµi. So this matrix-power method with repeated squaring is “exponentially faster. so ∑ ´ω2 µn 2 1 b) For Property 1: ω2 is an ´n 2µ-th root of 1 because ´ω2 µn 2 ωn 1. This requires about 2 lg´nµ matrix multiplications of 2 ¢ 2 matrices.17 Use the technique of repeated squaring given in the solution for Exercise 12. z = (a+b) * (c+d). v = b * d. the only nth root of 1 is 1 itself. So the total is about 16 lg´nµ *’s and 8 lg´nµ +’s. ´ωn 2 µ2 1. ω2i j . d) The element in row i and column j of DG1 is ωi G1 i j 2  ωi 2 j ´ ·1µ .v. // z = ac + ad + bc + bd u = a * c. since e) The entries of G1 are ω2i j . ´1 2π nµ. representing the complex numbers a · bi and c · di. 1. etc. (See the recurrence that follows Algorithm 12. // realPart = ac . // imaginaryPart = ad + bc 12. The iterative computation of Fn requires about n +’s. (b).2(a) to compute An 1 . Permuting the columns of Fn and elements of V in the same way will affect the order of the terms in the summation.v.16  1 i j a) The ith component of FnV is ∑n j 0 ω v j . (c).u .

Note: The issue of arithmetic with rational numbers also arises in Exercises 13. 1. the time to add two of them digit by digit is in O´Lµ. which can be arbitrarily larger than n. If each si is represented as a terminating decimal. A simple marking scheme can be used to check that the indexes listed include n. comma. An intermediate sum will never exceed L digits. We will show that the amount of time needed to check a proposed solution is bounded by a polynomial in L. sn . In all cases. as ´ni di µ. The length of a valid solution is in O´n log nµ since writing each index may use up to Θ´log nµ digits. the sums can all integers in 1 be computed and checked in one pass over the proposed solution string. Multiplying two L-digit numbers takes time in O´L2 µ. or semi-colon as one n. or O´L3 µ. Proposed solution: a sequence of indexes with commas separating items to go in the same bin and semi-colons marking the end of the sequence of indexes for each bin. So multiplying them all together yields a product with at most L digits. where ni and di are positive integers. 7. or O´L3 µ. so adding si to an intermediate sum is also in O´Lµ.2: P and N P 13. . Counting the semicolons is easy.Chapter 13 N P -Complete Problems Section 13. spaces may appear for readability although they are not mentioned. also each integer is in the range 1 The indexes in the sequence include all of the integers 1 There are at most k semicolons (at most k bins). 6. which is in O´L4 µ. Also. Be sure that the current errata (available via Internet) are applied if this error is present in any of the texts being used. a) The input is an integer k and a sequence of rational numbers. The only remaining question is how long that pass takes. we cannot assume they can be added in one step. 5. The formats for the proposed solutions are certainly not unique. . For example.2 The graph has no edges. part of the checking is to check for correct syntax. symbol. these are some of several reasonable approaches. ∑ si i¾I n. We need to compute one common denominator and n new numerators. The properties to be checked to verify the solution are: ¯ ¯ ¯ ¯ The proposed solution contains at most 2n symbols. 3. any numerator needed to represent an si with the common denominator is at most L digits. An invalid solution may be arbitrarily long but the checking algorithm returns false as soon as it detects an integer greater than n or more than 2n symbols. and that can be done in O´n2 L2 µ. 13. The issue is less clear if si are represented in the “pure” form of rational numbers. For each set of indexes I denoting a set of elements to go in a single bin. Similarly. takes time in O´nL2 µ. where the product is at most L digits. Since the si are arbitrary precision. Let L be the length of the input. We would expect different levels of sophistication and completeness in solutions in undergraduate courses and graduate courses. Assuming that the sizes si are in an array. undergraduates would probably omit the discussion of the length of the proposed solution that is emphasized in parts (a) and (c) below. counting each integer. Example: 2. 4. so we do not state that each time. In this case the total time for checking sums is certainly in O´Ln2 µ. s1 . 1. The input to the bin packing problem has length L larger than n since each of the n sizes may be many digits long. Adding up the new numerators is in O´Ln2 µ. First note that the total number of digits in all di together is less than L.4 Note: Part (e) of this exercise was updated by a correction that was too late for the second printing. Now forming the sum involves computing a common denominator.8 and 13. Multiplying n L-digit numbers.59.

we do not want to assume we can use an array with n entries. m. The overall input length is L. The properties to be checked to verify the solution are: ¯ ¯ ¯ The proposed solution contains exactly n indexes (i. (An alternative is an integer n and an n ¢ n bit matrix. no.” SIAM Journal on Computing. Interpretation: These are the vertices in a vertex cover.e. b) The input is an integer n and a sequence of edges i j. e) Note: Due to an error. at most m integers). pp. x and y. If x j i and i does not appear in the proposed solution. If x j i and the i appears in the proposed solution. (Since n may be very large in comparison to L. but it would be a very advanced graduate-student exercise. Proposed solution: a sequence of integers in the range 1. Interpretation: This is the order in which to visit the Proposed solution: a sequence of indexes. but is clearly straightforward and can be done in O´n2 µ time.) n and m edges. The work done to check these properties depends on the representation of the graph. but the intended question is “Is n nonprime?”. . A clause consisting of the literals x j evaluates to true if at least one of the x j evaluates to true. A proposed solution that is extremely long or contains extremely large numbers can be rejected as soon as we detect an integer that is not incident upon any input edge or detect more than m integers. n. . i. There is an edge vi vi·1 for 1 i n   1 and an edge vn v1 . Proposed solution: a sequence of integers. 214–220. the problem is adding them in polynomial time. Interpretation: xy The properties to be checked to verify the solution are: n. since each literal in a clause can be checked in O´Lµ by scanning the proposed solution. but would be 369 digits if written out in decimal form. Then a very large integer can be written with just a few characters. Proposed solution: Two positive integers. The input formula can be checked clause by clause in time O´L2 µ. The input is an L-bit integer n. Interpretation: if i is present. and each of these For each edge ´i jµ in the graph. where i and j are between 1 and n.e. n.) vk . Is bin-packing still in N P ? There is no problem about multiplying such numbers. such as 2 ££100 ¢ 7 ££400. let’s use “2 ££100” to denote 2100 and use ¢ to denote multiplication. then x j evaluates to true. then the truth value to be assigned to variable i is true. c) The input is a CNF formula in which the propositional variables are represented by integers in the range 1. 4.. v1 v2 vertices in a Hamiltonian cycle. 1975. at most L. too hard even for two stars in the context of the text. . “Every prime has a succinct certificate. the first two printings of the text ask for a proposed solution to the question. That is.90 Chapter 13 N P -Complete Problems An advanced study question: Suppose the si are written as ´ni di µ but ni and di may be written in factored form. 3. which is 13 characters in this form. k No index in the proposed solution is repeated. “Is n prime?”. vol. d) The input is an integer k and a graph with vertices 1 vq . A proposed solution that is extremely long or contains extremely large numbers can be rejected as soon as we detect an integer that is not in the correct range or detect more than n integers. at least one of i and j is in the sequence.. otherwise i is true. The properties to be checked to verify the solution are: ¯ ¯ The proposed solution is no longer than the input. The first question is indeed also in N P . then x j evaluates to true. see Vaughan Pratt. Note that n might be much larger than L. The work to check these properties is clearly straightforward and can be done in O´m3 µ time. v1 The properties to be checked to verify the solution are: ¯ ¯ The proposed solution contains at most k integers (or if k integers is incident upon some edge in the input graph. n) and each index is in the range 1 n. Each clause ¨ © in the CNF expression evaluates to true with these truth values.

4. j ¾J 13. Then we can conclude that the size of y is at most cp´nµ. Since the integers are a subset of the rational numbers.3. so SSI P SSR . and dC (all of which are integers). UT takes input x for P and produces U ´T ´xµµ. such that the answer for R with this input is the correct answer for P with input x. cise 13. so the time for the transformation UT is at most p´nµ · q´cp´nµµ. that UT preserves correct answers and that it can be computed in polynomial time.3: N P -Complete Problems 13. For any subset J of 1 So the input for SSR has a yes answer if and only if the transformed input for SSI does. there is one in H . respectively. where the weight of an edge ´i jµ ¾ F is 1 if ´i jµ ¾ E and is 2 otherwise. UT ´xµ U ´T ´xµµ. Let c be the (constant) limit on the number of units of output (perhaps characters) an algorithm may produce in one step. The answer for Q with this input is the correct answer for P with input x. P P Q. The product xy can be computed by the manual long-multiplication technique in time proportional to L2 . vw and wv. such that the answer for Q with this input is the correct answer for R with input y. so they provide a Hamiltonian cycle for G. and C be an input for SSR. The input is transformed to ds1 dsn . an input for Q. ∑ s j C if and only if ∑ ds j dC. Students are likely to say: Since T and U are computed with input x. and SSR j ¾J P SSI . then H has a tour of length k by taking the edges of the Hamiltonian cycle. If H has a tour of length k. then there is one in G.12 Let the transformation T take an undirected graph G ´V E µ as input and produce H and k as output. Let n be the size of x. 13. (A similar argument appears in the proof of Theorem 13. The second point. let UT be the composite function that first applies T to its argument. There is a polynomial-time transformation U that takes input y for R and produces U ´yµ. the total time is ´ p · qµ´nµ.e. but there is a (small) difficulty in the proof that should not be overlooked. and Q. sn . i. and C. may seem clear too. then applies U to the result. it must consist entirely of edges of weight 1 because k cities need to be visited. Therefore the edges in the tour all occur in G. The value of k is V . Then if there is a Hamiltonian cycle in G. If G has a Hamiltonian cycle. If there is a Hamiltonian cycle in H . Section 13. Each integer is greater than 1. simply replace either edge vw or wv in the cycle in H by the corresponding undirected edge in G. H ´V F µ. 13.) Hence. xy n. The first is easy to see. the timing for UT . There are two things we must verify.Chapter 13 N P -Complete Problems 91 ¯ ¯ ¯ The proposed solution is a pair of integers of at most L bits each.6 Let the problems be P. any algorithm for SSR solves SSI . which may have size much larger than n. the cycle in H uses the edge from a vw and wv pair that corresponds to the direction the edge was traversed in G. let s1 sn . For each edge in the cycle in G. The hitch is that U is applied to y T ´xµ. where P P R and R P Q.10 Transform an undirected graph G into a directed graph H by replacing each edge vw of G by edges in each direction. . an input for P. which is polynomial. an input for Q. R. Let d be the product of the denominators of For the other direction. this can be done in polynomial time. where H is a complete weighted undirected graph and k is an integer. H and k are calculated from G as follows. As discussed in Exers1 n . There is a polynomial-time transformation T that takes input x for P and produces T ´xµ. Now. Let p and q be polynomial time bounds for algorithms to compute T and U .8 Let SSI be the subset sum problem for integers and let SSR be the subset sum problem for rational numbers. an input for R.. Clearly T ´Gµ can be computed in polynomial time.

So the total number of literals in the formula is 2mk · nk.) And for each edge ´i jµ. suppose a CNF formula E is satisfiable and fix some satisfying truth assignment for the variables. let a proposed solution be a sequence of vertices. the clause ´ xic Every vertex must have some color. We may immediately reject any input with k n. but that can’t happen here because all these literals are true. that each is actually a vertex in the graph. G has a Hamiltonian cycle. (If more than one literal in a clause is true. so we never need to encode such problems. for some 1 c k. so for every vertex 1 v n we create the clause ´xv1 xv2 ¡¡¡ xvk µ. we show that SAT P Clique. The “meaning” of xvc is that vertex v is colored c. hence O´n2 µ. So each pair of vertices in V ¼ is adjacent. so in the following discussion. A Hamiltonian cycle does not in general traverse every edge in the graph.18 To see that the clique problem is in N P . Also the formula is in 3-CNF in this case. Edges are omitted choose any one. edges to be checked. we will show that the graph constructed from E (as described in the exercise) has a k-clique. with two literals each. so for every edge ´i jµ ¾ G and every color 1 c k we create x jc µ. then we claim that the propositional variables that are true in a satisfying assignment define a k-coloring for G.14 T ´Gµ is bipartite. Thus a cycle in T ´Gµ that corresponds to a cycle in G will miss those new vertices in the edges that aren’t in G’s cycle. choose any one of them. For each v at least one of xvc is true. We will use this encoding as our transformation to prove that graph coloring is reducible to SAT. This amounts to n clauses of k literals each. so E is satisfiable. First.) The set of propositional variables is xvc 1 v n 1 c k . so i and j have different colors. we show that E is satisfiable. The problem is trivial if k n.” as given above. G is encoded as a CNF formula that is satisfiable if and only if G is k-colorable. Assign the value true to each literal li mi for 1 i k. let ti be an index of a true literal in that clause. we assume k n. G (Has a Hamiltonian cycle) T(G) (Has no Hamiltonian cycle) 13. There is an example in the figure below. for 1 i p.92 Chapter 13 N P -Complete Problems 13. . No two adjacent vertices may have the same color. It remains to show that this transformation preserves both yes and no answers for the graph colorability problem. Suppose a graph G has n vertices and m edges. and V ¼ is a k-clique. k was specified as 3. are true. we note that the transformation can be done in quadratic time. For each clause Ci . (There are Θ´k2 µ. (The exercise asked for k 3. ´i ti µ 1 i p . and k colors are available. but there are some graphs G that have Hamiltonian cycles where T ´Gµ does not. because there are no edges between vertices that correspond to literals in the same clause. One literal in each clause will be true. so the length is in O´n · mµ. Another exercise: Show that Independent Set is N P -complete by reducing SAT to Independent Set. If the formula is satisfiable. The vertices in the k-clique must differ in the first component.16 There is a well-known way to encode graph colorability instances as CNF-satisfiability instances. but the encoding is general. then x jc must be false.) Now suppose that the graph constructed for E has a k-clique. Thus we may conclude that the vertices in the k-clique are ´i mi µ. 13. Finally. and that they are mutually adjacent.) Now.) So li ti . (If more than one xvc is true for a given v. Therefore the formula’s length is in O´n3 µ and is easily computable in polynomial time. It is easy to check in polynomial time that there are k vertices in the sequence. but T ´Gµ does not. (Recall k p. For this exercise. This amounts to mk clauses. for 1 i k. Consider the vertices V ¼ from the graph only if li ti l j t j . There can be no conflicts in this assignment because li mi l jm j . If G is k-colorable. A related construction can be used. if xic is true. then the CNF formula is satisfiable by setting all the propositional variables according to their “meaning.

has the same vertices as G and has exactly those undirected edges that are not in G. Let the input for Clique be a graph G and an integer k. Construct the adjacency matrix representation of the graph in Θ´n2 µ time if it is not given in that form. There is no problem if G is presented using N adjacency lists or an N ¢ N adjacency matrix. For an undirected graph G. There are n 4 the existence (or nonexistence) of six edges must be checked. let it be the set of vertices C. Transform G to a directed graph H by replacing each edge vw of G by edges in each direction.28 Suppose we have a polynomial time Boolean subroutine canPack(S. the complement of G. Thus H may have T heta´N 2µ edges. We claim it is a vertex cover for G. Therefore it is not necessary for any vertex in K to be in a vertex cover of H . Vertex Cover. Let i and j be any two vertices not in C. (We explain this restriction later. and select any edge in the cycle.g. or a simple cycle. 13. In H .. We claim that the vertex cover for G is a feedback vertex set for H . suppose H has a feedback vertex set.Chapter 13 N P -Complete Problems 13.26  ¡ n´n   1µ´n   2µ´n   3µ 24 such sets. logarithmic) compared to N . This edge came from an edge vw in G. and Independent Set can be reduced to any of the others. G could be presented as the integer N . we use only vertices incident upon some edge of G. The input for the feedback vertex set problem will be H and k. specifying the number of vertices without enumerating them. n. If the number of edges is tiny (e. The transformation can be done in linear time. either has size in Ω´N µ. 13. let n be the number of such vertices. All the transformations have a similar flavor. If H has a vertex cover of size c. Consider any directed cycle in H . (The number of edges is in O´nµ. and there are k n   c of them. On the other hand. Thus at least one of the vertices v or w must be in the feedback vertex set. The transformation can be computed in time O´n3 µ. so at least one vertex is in the vertex cover. whether the size of H is bounded by a polynomial function of the size of the input graph G. a simple chain of vertices. the roughly N 2 edges of H cannot be constructed in polynomial time. so the algorithm’s time is in Θ´n4 µ. Then any edge can be checked in O´1µ. where G is the complement of G. For each set of four vertices.21 Let G (an undirected graph) and k be an input for the vertex cover problem.20 This solution uses the notion of the complement of a graph. then each connected component of the graph is either an isolated vertex. then a sequence of edges. denoted G. Then i j cannot be an edge in H . 93 . so it must be an edge in G. that is.) We transform G and k to the graph H G. vw and wv. 13. Therefore all the vertices not in C comprise a clique in G. “Does H have a vertex cover of size c (or less)?” If G has a clique of size k. Check each subset of four vertices. Suppose G has a vertex cover using at most k vertices. For purposes of the transformation. We use the function as follows. k) which returns true if and only if the n items whose sizes are in the array S can be packed in k bins. The question for the transformed input is.23 If each vertex has degree at most two. 13. However. Choose the remaining c n   k vertices (those not in K ) as the vertex cover of H . Suppose G actually has N vertices. let it be the set of vertices K . These conditions can be detected using a straightforward depth-first search. and the integer c n   k. Let vw be an edge of G.) The chromatic number of the graph will be three if there is a cycle of odd length. there are no edges between any pair of vertices in K . otherwise it will be two (if there are any edges at all) or one (if there are no edges). Why did we define n to exclude isolated vertices? The problem has to do with the size of H . Other variations: Show that any of Clique. In H there is a cycle consisting of only the two directed edges vw and wv.

The following construction works for k Group 1: 4. he or she would probably notice that one Group-1 object and one Group-2 object left a lot of waste space..e. 3. so if canPack runs in polynomial time. We’ll define ε´kµ size 1   1 k2 . so does minBins. Section 13. 2 1 · k k 1 1 k 1 · · ¯ Verify: No other objects fit with these two. Section 13. However. The idea is that the optimum packing fills k   1 bins exactly by using one object each from Groups 1. (True if k 1. Claim: The sizes are listed in nonincreasing order. it is “tricked” into distributing Group 2 among the first k   1 bins after it distributes Group 1 into those bins. (True if k 4. return k. n) for (k = 1. k   1 objects 2 k 1 Group 2: k   1 objects size ´k   1µ 1 ´k · 1 µ Group 3: k   1 objects size · ε´kµ k k2 1 ´k   1 µ Group 4: k   1 objects size   ε´kµ k k2 In an optimal packing one object from each of the groups 1. We give two schemes for this. 1   . We know there will be at most n iterations of the for loop. we will simply state the claim and the condition it requires on k. . 2µ Verify: An object from Group 1 and Group 2 do fit. (True if k 4. in the kth bin. k). i. and no others.) 1.5: Bin Packing 13.30 FS´F µ is the set of all subsets of F that are covers of A. Solution 1. if any. Claim: FFD puts all the Group 3 objects. n.4: Approximation Algorithms 13. k n. This is the usual technique for fooling a greedy algorithm: make sure the optimal solution requires more lookahead than the greedy strategy has. since FFD handles larger sizes first. val´F xµ is the number of sets in x.94 Chapter 13 N P -Complete Problems int minBins(S. so k   1 bins are filled with all the objects in these groups. For any such cover x. All the objects in Group 2 fit in one bin. creating some waste. 3. but one each of Groups 1. 1 k2 2.) k 1 1 k·1 ¯ Verify: k   . and 4. k ++) cando = canPack(S. so the optimal number of bins is k. If a person were trying to fill the bins. and puts all of Group 2 in one bin. 3.e. Nothing else will fit into the first k   1 bins. i. 1. So it is easy to make FFD use an extra bin. ¯ Verify: 1   2 k 1 ¯ ¯ Verify: Two objects from Group 1 won’t fit. if (cando) break.. The technical difficulty is forcing it to put k   1 objects into that bin. We establish the behavior of First Fit Decreasing on this input by verifying a series of claims. Claim: FFD puts one object from Group 1 and one object from Group 2 together in each of the first k   1 bins. and 4 fill a bin completely.32 The objects are specified in four groups by decreasing sizes. and 4 was a perfect fit. We won’t include all the details. 1   2 k k 1 k2 3.

D´kµ. Claim: FFD puts all the Group-3 objects and exactly three Group-4 objects in the kth bin. 2k2 · 13k · 19 D´kµ k2 · 7k · 12 size D ´k µ 2 k · 7k · 10 size D ´k µ k 2 · 6k · 9 size D´kµ size 1   ¯ ¯ ¯ ¯ ¯ ¯ Verify: D´kµ   ´2k2 · 13k · 19µ 2 ´k · 7k · 12µ. All the remaining objects fit in one bin.) 2 ´2k · 13k · 19µ. . (True if k 2 µ 2 k2 Thus FFD has k   1 remaining objects that it puts in extra bins. (True if 2 · 7k · 10µ · 4´k · 6k · 9µ Thus FFD has k   1 remaining Group-4 objects that it puts in an extra bin. ´k   1µ´k2 · 7k · 10µ · 3´k2 · 6k · 9µ 2 3. (True if k Verify: An object from Group 1 and Group 2 do fit. k2 ·1 k 1 ¯ Verify: No Group 4 object fits in this bin.. since 2 2 ´k   1µ´k · 7k · 12µ · 3´k · 6k · 9µ D´kµ so the optimal number of bins is k. The following construction works for k 2. (In fact. 3. Claim: The sizes are listed in nonincreasing order. Verify: No other objects fit with these two. to be D´kµ ´k · 2µ´k · 3µ´k · 4µ 95 ¯ Verify: All Group 3 objects fit in one bin. It is a refinement of the idea in solution 1. and 4 fill a bin completely.e. Claim: FFD puts one object from Group 1 and one object from Group 2 together in each of the first k   1 bins. ´k2 · 7k · 12µ · ´k2 · 6k · 9µ Verify: These objects fit. i.e. i.) Verify: Two objects from Group 1 won’t fit. 1. 2.. the common denominator..e.. ´k   1µ´k k 2µ D´kµ. 2. leaving Group 2 and three objects from Group 4..10. i. and so covers all cases of Lemma 13.e. We’ll define D´kµ.e. they will all go in one bin. Solution 2.) It is informative to work out an example with k 10. ´k   1µ k3 · 9k2 · 26k · 24 Group 1: Group 2: Group 3: Group 4: k   1 objects k   1 objects k   1 objects k · 2 objects In an optimal packing one object from each of the groups 1. (True if k 2. Verify: A fourth Group-4 object does not fit. The behavior of FFD is shown with claims similar to solution 1. ´k   1µ k k · 1.Chapter 13 N P -Complete Problems k·1 1. so k   1 bins are filled with the objects in these groups. i. i.

1. where object i has profit pi and has size si . 0.35 a) 0. j binsUsed. 0. so that all objects had equal profit density.96 Chapter 13 N P -Complete Problems 13. j. which can be chosen arbitrarily large. j ++) if (used[j] + si 1. are numbered. This is defined as pi si . // bin[i] will be the number of the bin to which si is assigned. binsUsed i   1. except that all bins are considered before choosing one.2. int i. 0. i ++) // Look for a bin in which s[i] fits best. but two is optimal. .2. if (bestJ > binsUsed) binsUsed = bestJ.6: The Knapsack and Subset Sum Problems 13. 13. for (j = 1. bin) float[] used = new float[n+1]. Lines that are new or changed. In the simplified version in the text. Sort S into descending (nonincreasing) order.2. // used[j] is the amount of space in bin j already used up.4. but FFD uses three.3. 0. at the beginning of the ith iteration of the outer for loop. int binsUsed.1) closely.1. Section 13. 13. although s2 would fill the knapsack completely. i n. n.6. As in Algorithm 13. 0. bin[i] = bestJ.34 Best Fit Decreasing follows the procedure of First Fit Decreasing (Algorithm 13.3. for (i = 1. 0. we assumed that pi si .39 The main departure from Algorithm 13. giving the sequence s1 s2 ¡¡¡ sn . Initialize all used entries to 0. so the inner for loop executes Θ´n2 µ times altogether. 0. binpackBFD(S.1.37 Consider the input where s1 1 and s2 C. 0.2.7.35. BFD uses two bins.25. 0. The output parameter take is in this class. bestJ = binsUsed + 1.0. // Indexes for the array bin correspond to indexes in the sorted sequence. used[bestJ] += si .0 && used[j] > used[bestJ]) bestJ = j. // Continue for (i) The worst case time occurs when n bins are used. In such cases. relative to Algorithm 13. b) 0.2. The greedy algorithm described in the exercise puts s1 in the knapsack and quits.2. binsUsed = 0. 0. 0. 0. The ratio of the optimal to the algorithm’s result is C. BFD uses three bins. bestJ.2 is that profit density is used. we assume the class IndexSet is available to represent subsets of N and the needed set operations are implemented efficiently in this class.

float maxProfit. adding those that fit.) Thus the optimal solution is indexed as follows. Let Ak be the sequence of algorithms given above for the general knapsack problem. 2 . RAk ´mµ and SAk ´nµ.. return maxProfit.” JACM. This subset will be considered explicitly next. pk·1 sk·1 ¡¡¡ pq 1 sq 1 pq sq ¡¡¡ pt st k largest profits Middle group Small group Observe that pq is not one of the k largest profits in the optimal solution. profit += p j . Theorem.13 becomes much more complicated. “Approximate algorithms for the 0/1 knapsack problem. Copy fields of T into take. Let pk·1 pt be indexed in decreasing (or nonincreasing) profit density order. so pq m ´k · 1µ (13.2) . 2. 4. Suppose t k. We will examine the action of Ak when it works with this particular T . 3. 1. the worst-case ratios of the optimal solution to the value found by Ak . Let p1 p2 as T by Ak . / . (If the algorithm’s output uses a different T . T = T j . then Ak gives an optimal solution. since for k 0. we will denote these objects by indexes 1. For k 0. Sort S and P into descending (nonincreasing) order of profit density. 7. and it will be useful to think of the objects divided into the three groups indicated. if (sum C) // Consider remaining objects. The proof of Theorem 13. // Continue with next subset of at most k indexes. sum. now it is in Θ´n log nµ. described pk be the k objects from T £ with the largest profits.12 must be modified to include the time for the sorting. Thus this indexing does not correspond exactly to the indexes used in the algorithm and does not imply that the optimal solution consists of the t objects with greatest profit density. Let T £ be the set of objects in an optimal solution. so proving the theorem for T £ will be sufficient. maxProfit = profit. S. are at most 1 · 1 k for all m and n. Jan. The worst-case time for A0 was linear before. Let q be the first index in the optimal solution that is not added to T by Ak .1) We will use this bound later. To keep the notation simple. p1 ¡¡¡ pk . If t k. The sum of the profits in the optimal solution can be described as: m kLargest · middles · smalls (13.Chapter 13 N P -Complete Problems knapsackk (C. t rather their actual indexes. giving the sequences s1 s2 sn and p1 p2 pn such that p1 s1 p2 s2 ¡¡¡ pn sn . int j. IndexSet T = new IndexSet. 0. maxProfit = 0. profit = ∑i¾T pi . that set must give at least as good a profit sum. the worst-case time for Ak is in Ω´n2 µ. Sahni. which has total profit m. Proof : (Based on S. profit. maxSum = 0. P. For each j not in T : if (sum + s j C) sum += s j . take) int maxSum. 6. // See if T fills the knapsack best so far. 1975). then T £ is explicitly considered by Ak so val´Ak I µµ m and rAk ´I µ 1. (If there is no such q. 5. We will assume that the objects in the optimal solution are indexed in a particular way. take = 0 For each subset T N with at most k elements: sum = ∑i¾T si .) Ak considers the remaining objects in profit density order. Fix k and a particular input I with opt ´I µ m. The only case where this affects the result is when k 0. if (maxProfit < profit) maxSum = sum. 97 Theorem 13.

Now we aim to get bounds on the profits of some of the groups that make up the optimal solution and the algorithm’s solution. so the maximum number of colors that can be rejected is the degree of v. (13. The color assigned to v is at most degree´vµ · 1. its profit is profit kLargest · middles · extras (13. until it finds one it can use. using the last result and Equations (13. beginning with color 1.4) ∑ pj ∑ pj sj sj ∑ pq sq sj pq sq extras where the j in the summations indexes the extra objects added by Ak that are not in the optimal solution. So. We will use group to mean the total size of the objects in a group.5). we can conclude that m kLargest · middles · smalls kLargest · middles · extras   ´kLargest · middles · extrasµ · pq sq pq sq extras ´C · pq sq ´C   kLargest   middles µ µ   kLargest   middles   extras because C   kLargest   middles   extras sq .7: Graph Coloring 13.1) and (13. the capacity of the knapsack: smalls Proof : smalls pq sq t ´C   kLargest   middles µ (13.2). and (13. we now have m ´kLargest · middles · extrasµ · pq profit · m k·1 m val´Ak I µ · k·1 and it is easy to rearrange terms to get m val´Ak I µ Thus RAk ´mµ 1· 1 and SAk ´nµ k 1 1· .42 For each vertex v. The next bound we need involves C. So now using Equations (13.5) i q ∑ pi pq sq ´C i q ∑ t pq sq si pq sq smalls µ   kLargest   middles The last inequality follows from the fact that the small group must fit in the knapsack along with the largest and middle groups. Sequential Coloring tries each color in turn.3) where the extras all have profit density at least pq sq (because they were considered before the qth object).98 Chapter 13 N P -Complete Problems At the point when Ak considers and rejects object q. .4). otherwise Ak would have put the qth object in the knapsack. k 1· 1 k ´kLargest · middles · extrasµ · pq Section 13.3). The first bound is: extras   Proof : extras pq sq extras 0 (13. Hence the maximum number of colors used is at most ∆´Gµ · 1. A color is rejected only if there is a vertex adjacent to v that has already been assigned that color.

Another exercise: Construct a graph for which nearest2ends does better than the nearest-neighbor strategy. else v = w.3). more chances to be greedy can produce even worse tours.49 Consider the graph at the right. starting at any vertex.8: The Traveling Salesperson Problem 13.4. then (4. note that any triangle must use three colors.4). 99 Section 13.47 In the graph at the right. W) Select an arbitrary vertex s to start the cycle C. // One end of the chain. 2). // The other end. If the selected edge is uw u = w.2.44 G1 G2 has n1 n2 vertices and n1 m2 · m1 n2 2 edges. 4. The nearest-neighbor strategy produces the cylce 1. 2. 1. E. The nearest neighbor algorithm chooses the tour 1. but the graph is not 3-colorable. and is forced to choose (1. then (3. However. Starting at 1. v = s.3) with weight 50. since being greedy doesn’t work for the TSP problem. which has weight 44. 2. The results are similar starting elsewhere. 1 for a cost of 82. Add the selected edge to C. 3). nearest2ends produces a worse tour than the ordinary nearest-neighbor strategy. the neighborhood of every vertex is 2-colorable.3. 4 for a cost of 105.1. return C. where w is not in C. the possibilities are quickly exhausted. 1). 3. 13. In the graph at the right. u = s. nearest2ends selects the edge (1. then (2. c) It might seem that having more choices at each step gives a better chance of finding a minimum tour. b) It finds the same tour. The shortest link algorithm chooses edge (2. 15 10 4 12 3 1 50 12 2 5 13. 3. 1 20 25 50 4 20 3 22 15 2 . Start at vertex 1. Add the edge vu to C.51 a) nearest2ends(V.Chapter 13 N P -Complete Problems 13. While there are vertices not yet in C: Select an edge uw or vw of minimum weight. To verify the latter. and finally it completes the cycle 4.

and R3 be the strings for vertices 1.e. so the identity transformation shows that 2-SAT P satisfiability. If P N P . 2. d1 11 ¡¡¡ d1 20 d2 1 ¡¡¡ d2 10 d2 11 ¡¡¡ d2 20 d3 1 ¡¡¡ d3 10 If R4 happens to be d1 16 ¡¡¡ d1 20 d2 1 ¡¡¡ d2 15 . if we take preorder time as being when they are visited. Any algorithm for satisfiability solves 2-SAT (2-satisfiability.57 A simple greedy algorithm could work as follows. However. Consider an optimum tour.. 13.55 a) True. What remains is a spanning tree. and the actual MST-based tour costs at most the same. Viewing DFS as a journey around the graph (i. we will present an intuitive argument. 2. i. it is easy to see that it extends to round-about routes of k 2 edges. The traveling salesperson problem is N P -complete. so it has a weight at least as great as the minimum spanning tree. vn . 2 and time requirements 7. 3. Always assign the next job to the processor that finishes soonest. b) False.100 Chapter 13 N P -Complete Problems 13. ti . but an outtree is more convenient for preorder traversal. All we know now is that if P there is no such approximation algorithm. v2 . The triangle inequality implies that a direct route between two cities is never more than a round-about route. so using a general satisfiability algorithm would be overkill. Let R1 . because any direct routes it uses are less in weight than the round-about route used by the pseudo-tour.18 showed that an in-tree can be converted to an out-tree in linear time. Additional Problems 13.53 a) Θ´n2 µ is the best possible because the graph has about n2 2 edges (so we can’t do better) and Prim’s algorithm runs in Θ´n2 µ. (In fact.54 At the beginning of Step 2c. R2 . For simplicity of numbering.e. but using each edge twice. 1. Section 13. Now think of the MST-based tour as using only edges in the MST.) d) Unknown. beginning at vstart and ending at vend . v1 with total weight T . then . So this pseudo-tour that stays in the MST edges costs twice the MST weight. once forward and once backward. 2. Exercise 2. add the next ti to a bin with the smallest current sum.9: Computing with DNA 13. b) Rather than give a formal proof. Sort the job time requirements. Choose any two consecutive edges in such a path. 2 shows N P .) This clearly can be implemented in O´n2 µ time. Then. into decreasing (nonincreasing) order. 6. P N P. we have single DNA strands representing paths of length n. in the order they are traversed by depth-first search. suppose they are (1. Prim’s algorithm produces an in-tree representation of the minimum spanning tree. The cost of this “pseudo-tour” is twice the weight of the MST. Although it only is explicit about round-about routes of two edges. (In terms of bins.. Remove the minimum-weight edge in this tour. the MST) is helpful here. and 3. c) True. So it suffices to show that the tour generated by the MST-based algorithm costs at most twice the weight of the MST. then R4 could attach to such a strand. But this does not imply that 2-SAT is N P -hard. The vertices are visited in the same order as in the MST-based tour. sometimes called 2-CNF-SAT). some strands include the sequence S1 2 S2 3 .2) and (2.3). then exact graph coloring would be in P . The example with p that this strategy does not always produce optimal solutions. 2-SAT is in P . . v1 .

b) Use the algorithm minTSP in part (a) to find T . Let n be the number of variables in E . // Continue loop. The idea is to choose a mula E is satisfiable.47. the definitions given imply that an empty clause cannot be made true by any assignment. decideTSP is called E times. As discussed in the solution of Exercise 13. k) be a Boolean function that decides the traveling salesperson decision problem for a complete weighted graph G ´V E W µ and tour bound k. which is polynomially bounded in the size of the input. Let Tmax Wmax n. Tmax . We consider an edge discarded if its final weight is greater than T . then multiplying all the weights by that common denominator. The idea of the algorithm is to use Binary Search on the integers 0 (This is similar to the solution of Exercise 1. assign. T) For each edge e in G: Change W(e) to T+1. Let Wmax be the largest edge weight in G. this can be done in polynomial time by putting all the rational numbers over a common denominator.59 a) The TSP is commonly defined with integer edge weights. findTour(G. substitute it into the formula. 13. Although the text does not go into detail on empty clauses. while (low + 1 high) mid = (low + high) / 2. If decideTSP runs in polynomial time. and then use polySatDecision to check whether there is a truth value assignment for the remaining variables that will make the simpler formula true. an edge with weight greater than T cannot be in a tour of total weight T . low = 0. is supplied to store the assignment. This blows up the input length by a quadratic factor at most.) If not. else low = mid + 1. (The substitution process may create empty clauses. choose the opposite value for the variable. then findTour does. . if (decideTSP(G. if (decideTSP(G. mid) == true) high = mid. If rational weights are allowed. we will assume that the weights have been scaled to integers as a preprocessing step. The next algorithm constructs an optimal tour by testing each edge in G and discarding those that are not required. Change W(e) back to its original value. (The method described in this solution does not work if the weights are floating point numbers. 101 The number of iterations of the loop is about lg Tmax lg´Wmax µ · lg´nµ.4. Clearly. then so does minTSP. Note that the values of Wmax and Tmax are not polynomially bounded in the size of G. The weight of an optimal tour is at most Tmax . and let the variables be x1 truth value for each variable in turn. Tmax = Wmax * n. the weight of an optimal tour for the graph G ´V E W µ. certain weights are increased in the for loop.Chapter 13 N P -Complete Problems 13. T) == false) // e is needed for a minimum tour. Output edges with W(e) T. so we don’t time time to check all the integers in the range 0 through Tmax to find the optimal tour weight.) Let decideTSP(G.61 Let polySatDecision(E) be a polynomial-time Boolean function that determines whether or not the CNF forxn . so if decideTSP runs in polynomial time.) int minTSP(G) Wmax maxe¾E W ´eµ. return high. A boolean array. and so such a clause causes the CNF formula to be unsatisfiable. high = Tmax .

then setAssign does also. for (i = 1. so the number of iterations of the loop is polynomial in the size. if polySatDecision runs in polynomial time. Etrue = the formula constructed from E by deleting all clauses containing xi and deleting the literal xi from all other clauses. i n. // Continue loop. Since there are n variables. i ++) // Try assigning true to xi . the size of the formula is at least n.102 Chapter 13 N P -Complete Problems satAssign(E. and the formula never gets larger. E = Efalse. else assign[i] = false. E = Etrue. if (polySatDecision(Etrue) == true) assign[i] = true. return. Thus. The modifications needed to produce the new formula can be made in one pass over the current version of the formula. Output truth assignment in assign array. . Efalse = the formula constructed from E by deleting all clauses containing xi and deleting the literal xi from all other clauses. assign) if (polySatDecision(E) == false) Output "not satisfiable".

3: Some Simple PRAM Algorithms 14. Section 14. parTransitiveClosure(A. 14. Write pid into idx[pid] and write n into idx[pid+n].1. for 0 k n. An element newR[i][j][k] in a three-dimensional work array is set to 1 in each iteration if and only if a path from i to j through k has been found. . M[n-1]. incr = 1. If ri j is already 1. Key key0. . Read idx[pid] into idx0 and read idx[pid+incr] into idx1. incr = 2 * incr. n) int incr. Write  ∞ (some very small value) into M[n]. else idxBig = idx0. The array idx[0]. the sum will be left in M[0]. if (key1 > key0) idxBig = idx1. using binary fan-in. incr = 2 * incr. key1. . n   1. R. while (incr < n) if (ri j == 0) Each Pi j k does: newR[i][j][k] = rik rk j . Then use binary fan-in to combine the results for each ´i jµ and update ri j . Read M[idx0] into key0 and read M[idx1] into key1. idx. idx[2*n-1] is used and the final parTournMaxIndex(M.4: Handling Write Conflicts 14. in the range 0 result is left in idx[0] instead of M[0]. while (incr < n) Read M[pid] into sum0 and read M[pid+incr] into sum1. incr = 2 * incr. newR[i][j][n-1]. parSum(M. sum1. Each processor carries out the algorithm using its own index for pid. idxBig. The integers to be added are in memory cells M[0]. Write (sum0 + sum1) into M[pid]. incr = 1. Write 0 into M[n+pid]. Each Pi j 0 does: if(newR[i][j][0] == 1) write 1 into ri j .2 Each processor’s index is pid. Pi j 0 copies ai j into ri j . Write idxBig into idx[pid]. M[1]. Pi i 0 (those on the diagonal) set rii to 1. sum0.4 Use n3 processors. leaving the result in newR[i][j][0]. Pi j k . indexed Pi j k for 0 i j k n. incr = 1. while (incr < n) int idx0.Chapter 14 Parallel Algorithms Section 14. do no more work. and no path from i to j was already known. to parallelize the nested for loops in the original algorithm. Each group of processors with k as their third index does: Compute the Boolean OR of newR[i][j][0]. n) int incr. idx1.5 This is a simple modification of Algorithm 14. n) int incr.

contradicting the lower bound for Boolean or.4 (see above). writing the result in ci j instead of M[0].3 could fail if each processor chooses the loser arbitrarily when the keys it compares are equal. indexed P0 . We will show that the matrix multiplication algorithm could be used to compute PRAM. Let x0 x1 the Boolean or of these bits in o´log nµ steps. so there will be no entry in the loser array that is zero. . parMultBooleanMatrix(A. it does each insertion (the work of the while loop) in parallel in a constant number of steps. B. A local variable pid is the processor’s index. Now the body of the while loop executes in Θ´1µ steps and there are Θ´log nµ iterations. Step 4: Pi j 0 . Section 14. it writes a 1 into ri j . they all write a 1. all equal.1 before earlier iterations are completed. one group of n processors Pi j0 to compute each entry ci j of C.8 The solution is similar to Exercise 14. . 14. n) Step 1: Each Pi j 0 writes 0 into ci j . and P12 might write 1 in loser[1]. The first (upper left corner) element in the matrix product is x0 x1 ¡¡¡ xn 1 . The solution given here is a little easier to follow. and let C be the product. and let B be an n ¢ n Boolean matrix whose first column contains all 1’s. . It uses n   1 processors. Let A be an n ¢ n Boolean matrix whose first row contains the xi .13 There are two natural solutions. so this algorithm works on Common-Write or stronger PRAM models.104 Chapter 14 Parallel Algorithms 14. Steps 2 and 3: Each Pi j k computes aik bk j and stores it in cTerm[i][j][k].11 Algorithm 14.2 more than one processor may write a 1 in the same cell at one time.5: Merging and Sorting 14. if multiple processors write into the same cell. 14. n 1 ci j k 0 aik bk j Pi j n 1 We use n3 processors.10 Suppose that there were an algorithm that can multiply two n ¢ n Boolean matrices in o´log nµ steps on a CREW xn 1 be n bits. 14. cTerm[i][j][n-1] as in Algorithm 14. One solution overlaps the insertion of keys. Suppose there are three keys. In Algorithm 14. it begins a new iteration of the outer loop (the for loop) in Algorithm 4.7 Let the matrices be A and B. P01 might write 1 in loser[0]. P02 might write 1 in loser[2]. that is.2. C. . Thus. except that newR is a local variable in each processor and if Pi j k ’s value of newR is 1. Pi j n 1 compute the Boolean OR of cTerm[i][j][0]. Pn 2 .

else key0 =  ∞. n) Key key0. Finally. A simple variant of Algorithm 14. i < n. Use n2 processors. 14. if Kx Ky then ry mx and rx my . // Only one processor writes newKey (the original M[i]). if (newKey < key1) write key1 into M[pid+1]. Each group of processors with j as their second index does: Sum column j of isBefore using binary fan-in. 105 Since there are no concurrent writes. So the output location of K1 is less than that of K2 . If (pid > 0) read M[pid-1] into key0. This index in row v is leader[v]. and Kx and Ky cannot be written to the same output location.16 The keys are in M[0]. It is also easy to show that two elements in X . this is a CREW PRAM algorithm.4 or 14. Section 14. The argument for two elements in Y is the same. M[i-1].’’ If (key0 newKey < key1) write newKey into M[pid]. parCountingSort(M. Each P0 j does: Read isBefore[0][j] into rankJ.8.1 can be used. where m1 m2 . So mx · rx my · ry . Use the isBefore matrix for work space. key1. perform Transitive Closure by Shortcuts by the method from Exercise 14. The exercise asks only about an X element and a Y element. So mx · rx my · ry . Then the algorithm sums each column of the matrix to find rank´ jµ. else Write 0 into isBefore[i][j]. newKey. i ++) // Insert the key in M[i] into its proper place among the sorted // keys in M[0]. Read M[pid] into key1. cannot be written to the same output location. the number of keys that should precede M[j] in the sorted order. Call the resulting matrix R. determine the minimum column index among elements in that row that contain a 1. and Kx and Ky cannot be written to the same output location. Suppose K1 is at position m1 and K2 is at position m2 . Pi j for 0 i j n. depending on the PRAM model being used. Since K1 K2 . The processors work in n groups of n . Next. K1 and K2 . and // it writes into the correct position. Each processor does: Read M[i] into newKey. leaving the result in isBefore[0][j]. the original M[j] is placed in M[rank´ jµ].14 Assume that there is an X element Kx at position mx and a Y element Ky at position my .6: Finding Connected Components 14. The algorithm computes elements of a matrix isBefore such that isBefore[i][j] is 1 if M[i] should be somewhere before M[j] in the sorted order and is 0 otherwise. // M[j] was read earlier. for (i = 1. The binary fan-in takes Θ´log nµ steps and the other parts take Θ´1µ steps. If Kx Ky . if (M[i] < M[j] || (M[i] == M[j] && i < j)) Write 1 into isBefore[i][j]. M[n-1]. for each row v of R.18 First. // This moves all keys larger than M[i] to the ‘‘right. Let the cross-rank of Kx be rx and the cross-rank of Ky be ry . then ry mx and rx my . . Write the saved value of M[j] into M[rankJ]. n) Each Pi j does: Read M[i] and M[j]. 14. Similarly. the cross-rank of K1 is less than or equal that of K2 . .Chapter 14 Parallel Algorithms parInsertionSort(M.

The leaders can be computed in Θ´log nµ steps with a variant of Algorithm 14. If node i is a singleton. except for isolated nodes. 14.4. but it won’t change the asymptotic order. 14.22 If there is any undirected edge i j in G such that i j then the conditional singleton hooking step of initParCC hooks i to something. Therefore.20 The result is the same tree in both cases because 7 is its own parent and 7 is 8’s parent. a variant of Algorithm 14. Then compute the sum of the entries in the root array in O´log nµ steps (using the solution to Exercise 14. so all nodes are in in-trees of two of more nodes at the end of this step. else root[i] = 0. Attaching the singleton i to the parent of j cannot produce a cycle. Use n processors. Now consider the unconditional singleton hooking step. no node in j’s tree is a singleton. singleton[parent[v]] = false. then any undirected edge i j in G must satisfy i j. In the conditional singleton hooking step of initParCC it is possible that every node i 2 is attached to node 1. The processors work in n groups of n2 for each row in parallel.19 The idea is to count the roots. node i must get attached to something. immediately making one star containing all . so no node in j’s tree is attached to any new node in this step. 9 10 7 11 12 8 14.24 Do in parallel for all vertices v: singleton[v] = true.3 can be used.8. (For the Common-Write model. the Transitive Closure by Shortcuts can be done in Θ´log nµ steps by the method from Exercise 14. so node i is not a root for the next step. The leaders can be computed in Θ´1µ steps with a variant of Algorithm 14. So the total is Θ´log nµ.5). So the total time is Θ´log2 nµ. Each Pi does: if (parent[i] == i) root[i] = 1. b) For the CREW model.) a) For the Common-Write model. In either case. so j cannot be a root at this point. 14.25 Let the undirected graph Gn consist of edges ´1 iµ and ´i   1 iµ for 2 i n. Let root be an n-element array in memory.3. if (parent[v] v) singleton[v] = false. 14. No cycles are formed by conditional singleton hooking because all directed edges (parent pointers) in the data structure point from a higher numbered vertex to a lower numbered one. the Transitive Closure by Shortcuts can be done in Θ´log2 nµ steps by the method from Exercise 14.106 Chapter 14 Parallel Algorithms for each row in parallel.1.

so Θ´log nµ iterations are needed. This works because only one processor. making an in-tree of height n   1. else write 0. M k   1 .Chapter 14 Parallel Algorithms the nodes.4 of the text so that processors first try to reserve the right to write in parent[v] by writing their pid in reserve[v]. if (winningPid == pid) Write parJ into parent[parI]. reads M 0 and writes a 0 if M 0 0 or writes a 1 if M 0 1. 107 Additional Problems 14. . only processor Pn k will succeed in writing k to M 0 . will write a result to M 0 and so there can be no write conflict. . Read parent[i] into parI. write n   i to M 0 . . where k 0. n   1. (Two processors may have tried to write the same value in the same memory location. i then write 1 into M i . if no processor writes. Because low numbered processors have priority. which is correct. Write pid into reserve[parI]. except that Pn 1 just assigns 0 to next. but M 0 already contains a 0 so the result is correct. if desired. then P0 decides whether to write a 0 (if M 0 0) or a 1 (if M 0 1 and M 1 0) or not write (both are 1’s). but for different hook operations.26. Processors Pn k . Read reserve[parI] into winningPid. In the second case. . i. different edges.. it is only necessary to record the edges i j that were associated with successful hooking operations. Solution 2. Each processor Pi does: If cur is 1 and next is 0. Write 1 into stc[i][j]. k   1. If it is a 1. .e. a) Each processor Pi reads k from M 0 . . As in part (b). Solution 2. 14. the depth of node n starts at n   1 and shrinks by a factor of approximately 2 in each iteration (see the proof of Theorem 14.29 Processors are indexed by 0. If there are no 1’s in M 0 . respectively. . then M 0 0. . Each processor Pi reads M i · 1 into next. so P0 still finishes in the second step. . the lowest-priority processor. the one that reads the final 1. j) Pi j does: Read parent[j] into parJ.27 Based on Exercise 14. Pn 1 will all be trying to write results k. . 1. This decision is made during the computation phase. Nothing is hooked in the unconditional singleton hooking step in either case. c) Each processor Pi reads M i into cur. But if the output is required to be in a different location. M n   1 . into M 0 . which precedes the write phase. However. then no processor will attempt to write into M 0 . if the problem were changed slightly to require the output in a different location. In the first case. Pi does: Read M[i] into bit.) We introduce a new array reserve and modify the hooking operation given in Definition 14. Suppose there are 1’s in M 0 . It is also possible that each node i 2 is attached to node i   1. hook(i. After a write operation is performed. else if (i == n-1) Write n into M[0]. it may be impossible to determine which processor actually succeeded in writing. If k The result may be written into a location different from M[0]. . . Pn k·1. Each processor Pi reads M n   1   i . then write i · 1 to M 0 .2). if (bit == 0) Write i into M[0]. b) Solution 1. the algorithm terminates after one iteration of the while loop because nothing changes. then Pn 1 .

6). in the statement of the exercise) if it is in the array at all. halt. The size of the interval being searched is roughly 1 ´ p · 1µ times the size of the interval at the preceding iteration. are or are not in N C .” SIAM Journal on Computing. K) P0 and Pp 1 do in parallel: Write 0 into first. Matrix multiplication (Section 14. the sum. Merging sorted sequences (Algorithm 14. the processors can determine which interval can contain K (x. (The wording of the exercise is a little loose. Finding connected components (Algorithm 14. if (M[lower] K < M[upper]) Write lower into first. and the Boolean or of n inputs (Algorithm 14.2). P0 does: if (K == M[first]) Output first. . From the results of these 2 p comparisons. its processor index.30 This solution is a simplified version of one given by Marc Snir. as well as pid. halt. Write last into last. The idea is to divide the keys into p · 1 roughly equal intervals and use the p processors to “sample” the endpoints of the first p intervals. upper = ((p . parSearch(M.13) does not run in poly-log time. not algorithms. Sorting (Algorithm 14. Each processor reads first and last. 14. The more precise question would be “Are there any algorithms in this chapter that do not run in poly-log time with a polynomially bounded number of processors?”) The O´nµ implementation of Insertion Sort (Exercise 14. We can think of M[n] as containing ∞. 1985. Almost all the algorithms described in or to be written for the exercises solve problems that are in N C . The search then proceeds in the same manner on an interval roughly 1 ´ p · 1µ times the size of the previously searched interval. A straightforward parallelization of some dynamic programming algorithms might not run in poly-log time. Each processor has local variables lower and upper that indicate which elements it is to examine. // Continue while loop. problems.4).1). n   1 of M . Read first and read last. Thus the number of steps is in Θ´log´n · 1µ log´ p · 1µµ. The work done in one iteration of the loop is constant.4). The variables first and The keys reside in indexes 0 last (the boundaries of the interval being searched) are in the shared memory. Finding the transitive closure of a relation (Exercise 14. 3.pid + 1) * first + (pid) * last) / (p+1). else Output K " is not in the array". p. if (K < M[0]) Output K " is not in the array".32 All the following problems are in N C : ¯ ¯ ¯ ¯ ¯ ¯ Finding the maximum.pid) * first + (pid + 1) * last) / (p+1). lower = ((p . no. write n into last.108 Chapter 14 Parallel Algorithms 14. while (first < last . n. else if (pid == n-1 && K M[upper]) Write upper into first. vol. if (K > M[n-1]) Output K " is not in the array". so the number of iterations is roughly lg´n · 1µ lg´ p · 1µ. Write upper into last.1) // Precondition: M[first] K < M[last].3.5). 14. “On Parallel Searching. Each processor executes the following loop. // Only one processor writes into first and last.

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->