Problems on Algorithms
Second Edition
Ian Parberry and William Gasarch
July 2002
Consisting of
Problems on Algorithms, First Edition, by Ian
Parberry, formerly published in 1994 by Prentice
Hall, Inc.
Errata to the First Edition, published in 1998.
Supplementary Problems, by William Gasarch.
© Ian Parberry and William Gasarch, 2002.
License Agreement
This work is copyright Ian Parberry and William Gasarch. All rights reserved. The authors offer
this work, retail value US$20, free of charge under the following conditions:
• No part of this work may be made available on a public forum (including, but not limited to a
web page, ftp site, bulletin board, or internet news group) without the written permission of
the authors.
• No part of this work may be rented, leased, or offered for sale commercially in any form or
by any means, either print, electronic, or otherwise, without written permission of the authors.
• If you wish to provide access to this work in either print or electronic form, you may do so by
providing a link to, and/or listing the URL for the online version of this license agreement:
http://hercule.csci.unt.edu/ian/books/free/license.html.
You may not link directly to the PDF file.
• All printed versions of any or all parts of this work must include this license agreement.
Receipt of a printed copy of this work implies acceptance of the terms of this license
agreement. If you have received a printed copy of this work and do not accept the terms of
this license agreement, please destroy your copy by having it recycled in the most appropriate
manner available to you.
• You may download a single copy of this work. You may make as many copies as you wish
for your own personal use. You may not give a copy to any other person unless that person
has read, understood, and agreed to the terms of this license agreement.
• You undertake to donate a reasonable amount of your time or money to the charity of your
choice as soon as your personal circumstances allow you to do so. The author requests that
you make a cash donation to The National Multiple Sclerosis Society in the following amount
for each work that you receive:
• $5 if you are a student,
• $10 if you are a faculty member of a college, university, or school,
• $20 if you are employed fulltime in the computer industry.
Faculty, if you wish to use this work in your classroom, you are requested to:
• encourage your students to make individual donations, or
• make a lumpsum donation on behalf of your class.
If you have a credit card, you may place your donation online at:
https://www.nationalmssociety.org/donate/donate.asp.
Otherwise, donations may be sent to:
National Multiple Sclerosis Society  Lone Star Chapter
8111 North Stadium Drive
Houston, Texas 77054
If you restrict your donation to the National MS Society's targeted research campaign, 100% of
your money will be directed to fund the latest research to find a cure for MS. For the story of Ian
Parberry's experience with Multiple Sclerosis, see http:// multiplesclerosissucks.com.
Problems on Algorithms
by
Ian Parberry
(ian@ponder.csci.unt.edu)
Dept. of Computer Science
University of North Texas
Denton, TX 76203
August, 1994
Contents
Preface ix
1 Introduction 1
1.1 How to Use This Book 1
1.2 Overview of Chapters 1
1.3 Pseudocode 3
1.4 Useful Texts 4
2 Mathematical Induction 8
2.1 Summations 8
2.2 Inequalities 10
2.3 Floors and Ceilings 10
2.4 Divisibility 11
2.5 Postage Stamps 11
2.6 Chessboard Problems 12
2.7 Fibonacci Numbers 14
2.8 Binomial Coeﬃcients 14
2.9 What is Wrong? 15
2.10 Graphs 16
2.11 Trees 17
2.12 Geometry 18
2.13 Miscellaneous 19
2.14 Hints 20
2.15 Solutions 21
2.16 Comments 23
iii
iv Contents
3 BigO and BigΩ 28
3.1 Rank the Functions 28
3.2 True or False? 30
3.3 Proving BigO 31
3.4 Manipulating BigO 32
3.5 Alternative Deﬁnitions 33
3.6 Find the Functions 34
3.7 Hints 35
3.8 Solutions 35
3.9 Comments 36
4 Recurrence Relations 37
4.1 Simple Recurrences 37
4.2 More Diﬃcult Recurrences 38
4.3 General Formulae 39
4.4 Recurrences with Full History 39
4.5 Recurrences with Floors and Ceilings 40
4.6 Hints 41
4.7 Solutions 41
4.8 Comments 44
5 Correctness Proofs 46
5.1 Iterative Algorithms 46
5.1.1 StraightLine Programs 47
5.1.2 Arithmetic 47
5.1.3 Arrays 49
5.1.4 Fibonacci Numbers 51
5.2 Recursive Algorithms 51
5.2.1 Arithmetic 52
5.2.2 Arrays 53
5.2.3 Fibonacci Numbers 54
5.3 Combined Iteration and Recursion 54
5.4 Hints 55
5.5 Solutions 56
5.6 Comments 58
6 Algorithm Analysis 59
6.1 Iterative Algorithms 59
Contents v
6.1.1 What is Returned? 59
6.1.2 Arithmetic 61
6.1.3 Arrays 61
6.1.4 Fibonacci Numbers 62
6.2 Recursive Algorithms 62
6.2.1 Arithmetic 63
6.2.2 Arrays 63
6.2.3 Fibonacci Numbers 64
6.3 Combined Iteration and Recursion 64
6.4 Hints 64
6.5 Solutions 65
6.6 Comments 66
7 DivideandConquer 67
7.1 Maximum and Minimum 67
7.2 Integer Multiplication 68
7.3 Strassen’s Algorithm 73
7.4 Binary Search 74
7.5 Quicksort 75
7.6 Towers of Hanoi 77
7.7 DepthFirst Search 78
7.8 Applications 79
7.9 Hints 81
7.10 Solutions 83
7.11 Comments 85
8 Dynamic Programming 87
8.1 Iterated Matrix Product 87
8.2 The Knapsack Problem 89
8.3 Optimal Binary Search Trees 90
8.4 Floyd’s Algorithm 91
8.5 Applications 92
8.6 Finding the Solutions 97
8.7 Hints 99
8.8 Solutions 99
8.9 Comments 100
vi Contents
9 Greedy Algorithms 101
9.1 Continuous Knapsack 101
9.2 Dijkstra’s Algorithm 102
9.3 MinCost Spanning Trees 103
9.4 Traveling Salesperson 109
9.5 Applications 109
9.6 Hints 111
9.7 Solutions 111
9.8 Comments 115
10 Exhaustive Search 116
10.1 Strings 116
10.2 Permutations 118
10.3 Combinations 120
10.4 Other Elementary Algorithms 121
10.5 Backtracking 122
10.6 Applications 123
10.7 Hints 130
10.8 Solutions 131
10.9 Comments 133
11 Data Structures 135
11.1 Heaps 135
11.2 AVL Trees 138
11.3 2–3 Trees 141
11.4 The UnionFind Problem 142
11.5 Applications 142
11.6 Hints 145
11.7 Solutions 145
11.8 Comments 146
12 ^{completeness 147
12.1 General 147
12.2 Cook Reductions 148
12.3 What is Wrong? 150
12.4 Circuits 152
12.5 OneinThree 3SAT 153
12.6 Factorization 154
Contents vii
12.7 Hints 154
12.8 Solutions 155
12.9 Comments 155
13 Miscellaneous 156
13.1 Sorting and Order Statistics 156
13.2 Lower Bounds 157
13.3 Graph Algorithms 158
13.4 Maximum Flow 160
13.5 Matrix Reductions 161
13.6 General 162
13.7 Hints 164
13.8 Solutions 166
13.9 Comments 166
Bibliography 168
Index 174
viii Contents
Preface
The ability to devise eﬀective and eﬃcient algorithms in new situations is a skill
that separates the master programmer from the merely adequate coder. The best
way to develop that skill is to solve problems. To be eﬀective problem solvers,
masterprogrammersintraining must do more than memorize a collection of stan
dard techniques and applications — they must in addition be able to internalize and
integrate what they have learned and apply it in new circumstances. This book is a
collection of problems on the design, analysis, and veriﬁcation of algorithms for use
by practicing programmers who wish to hone and expand their skills, as a supple
mentary text for students enrolled in an undergraduate or beginning graduate class
on algorithms, and as a selfstudy text for graduate students who are preparing for
the qualifying (often called “breadth” or “comprehensive”) examination on algo
rithms for a Ph.D. program in Computer Science or Computer Engineering. It is
intended to augment the problem sets found in any standard algorithms textbook.
Recognizing that a supplementary text must be costeﬀective if it is to be useful,
I have made two important and perhaps controversial decisions in order to keep its
length within reasonable bounds. The ﬁrst is to cover only what I consider to be
the most important areas of algorithm design and analysis. Although most instruc
tors throw in a “fun” advanced topic such as amortized analysis, computational
geometry, approximation algorithms, numbertheoretic algorithms, randomized al
gorithms, or parallel algorithms, I have chosen not to cover these areas. The second
decision is not to search for the origin of the problems that I have used. A lengthy
discussion of the provenance of each problem would help make this book more schol
arly, but would not make it more attractive for its intended audience — students
and practicing programmers.
To make this book suitable for selfinstruction, I have provided at the end of
each chapter a small collection of hints, solutions, and comments. The solutions are
necessarily few for reasons of brevity, and also to avoid hindering instructors in their
selection of homework problems. I have included various preambles that summarize
the background knowledge needed to solve the problems so that students who are
familiar with the notation and style of their textbook and instructor can become
more familiar with mine.
ix
x Preface
The organization of this book is a little unusual and requires a few words of
explanation. After a chapter of introduction, it begins with ﬁve chapters on back
ground material that most algorithms instructors would like their students to have
mastered before setting foot in an algorithms class. This will be the case at some
universities, but even then most students would proﬁt from brushing up on this
material by attempting a few of the problems. The introductory chapters include
mathematical induction, bigO notation, recurrence relations, correctness proofs,
and basic algorithm analysis methods. The correctness proof chapter goes beyond
what is normally taught, but I believe that students proﬁt from it once they over
come their initial aversion to mathematical formalism.
The next four chapters are organized by algorithm design technique: divideand
conquer, dynamic programming, greedy algorithms, and exhaustive search. This
organization is somewhat nonstandard, but I believe that the pedagogical beneﬁts
outweigh the drawbacks (the most signiﬁcant of which seems to be the scattering of
problems on graph algorithms over several chapters, mainly in Sections 2.10, 2.11,
7.7, 8.4, 9.2, 9.3, 9.4, 13.3, and 13.4). Standard textbooks usually devote little time
to exhaustive search because it usually requires exponential time, which is a pity
when it contains many rich and interesting opportunities to practice the application
of algorithm design, analysis, and veriﬁcation methods. The manuscript is rounded
out with chapters on advanced data structures and ^{completeness. The ﬁnal
chapter contains miscellaneous problems that do not necessarily ﬁt into the earlier
chapters, and those for which part of the problem is to determine the algorithmic
technique or techniques to be used.
The algorithms in this book are expressed in a Pascallike pseudocode. Two
problems (Problems 326 and 327) require the reader to examine some code written
in Pascal, but the details of the programming language are not a major factor in
the solutions. In any case, I believe that students of Computer Science should be
literate in many diﬀerent programming languages.
For those who are interested in technical details, this manuscript was produced
from cameraready copy provided by the author using L
A
T
E
X version 2.09 (which
used T
E
X C Version 3.14t3), using the Prentice Hall macros phstyle.sty writ
ten by J. Kenneth Shultis, and the twoside, epsf, and makeidx style ﬁles. The
Bibliography was created by BibT
E
X. The index was produced using the program
makeindex. The ﬁgures were prepared in encapsulated Postscript form using idraw,
and the graphs using xgraph. The dvi ﬁle produced by L
A
T
E
X was translated into
Postscript using dvips.
A list of errata for this book is available by anonymous ftp from ftp.unt.edu
(IP address 129.120.1.1), in the directory ian/poa. I will be grateful to receive
feedback, suggestions, errata, and in particular any new and interesting problems
(including those from areas not presently covered), which can be sent by electronic
mail to ian@ponder.csci.unt.edu.
Chapter 1
Introduction
Problems on Algorithms contains 668 problems on the design, veriﬁcation, and
analysis of algorithms. This chapter is the only background that you will get before
we start listing problems. It is divided into four sections. The ﬁrst section describes
the philosophy of this book and how to get the best out of it. The second section
contains an overview of the remaining chapters. The third section brieﬂy touches
on the pseudocode used to describe algorithms. The fourth section gives a quick
review of some of the textbooks available on algorithms and related subjects.
1.1 HOW TO USE THIS BOOK
Most chapters have sections with hints for, solutions to, and comments on selected
problems. It is recommended that readers ﬁrst attempt the problems on their own.
The hints should be consulted only when encountering serious diﬃculty in getting
started. The solutions can be used as examples of the type of answers that are
expected. Only a limited number of solutions are given so that students are not
tempted to peek, and to ensure that there remains a large number of problems that
instructors can assign as homework.
Each problem is labeled with a sequence of icons. The more mortarboards
a problem has, the more diﬃcult it is. Problems with one or two mortarboards are
suitable for a seniorlevel undergraduate class on algorithms. Problems with two or
three mortarboards are suitable for a graduate class on algorithms. The remaining
icons indicate the existence of additional material: The lightbulb indicates that
there is a hint, the scroll and feather indicates that there is a solution, and the
smiley face indicates that there is a comment. This is summarized for the
reader’s convenience in Table 1.1.
1.2 OVERVIEW OF CHAPTERS
Chapters 2–4 contain problems on some background material on discrete mathe
matics that students should already have met before entering an undergraduate
algorithms course; respectively mathematical induction, bigO notation, and recur
rence relations. If you are a student in an algorithms class, I strongly recommend
1
2 Chap. 1. Introduction
Symbol Meaning
easy
medium
diﬃcult
hint
solution
comment
Table 1.1. Problem symbols and their meaning.
that you review this material and attempt some of the problems from Chapters 2–4
before attending the ﬁrst class. Some kind instructors may spend a few lectures
“reminding” you of this material, but not all of them can aﬀord the time to do so.
In any case, you will be ahead of the game if you devote some time to look over
the material again. If you ignored the material on mathematical induction in your
discrete mathematics class, thinking that it is useless to a programmer, then think
again. If you can master induction, you have mastered recursion — they are two
sides of the same coin.
You may also meet the material from Chapters 5 and 6 in other classes. Chap
ter 5 contains problems on correctness proofs. Not all instructors will emphasize
correctness proofs in their algorithms class, but mastering them will teach you im
portant skills in formalization, expression, and design. The approach to correctness
proofs taken in Chapter 5 is closer to that taken by working researchers than the
formallogic approach expounded by others such as Dijkstra [21]. Chapter 6 con
tains some problems involving the analysis of some naive algorithms. You should
have met this material in earlier classes, but you probably don’t have the depth of
knowledge and facility that you can get from working the problems in this chapter.
Chapters 7–10 consist of problems on various algorithm design techniques: re
spectively, divideandconquer, dynamic programming, greedy algorithms, and ex
haustive search. The problems address some of the standard applications of these
techniques, and ask you to demonstrate your knowledge by applying the techniques
to new problems. Divideandconquer has already been foreshadowed in earlier
chapters, but the problems in Chapter 7 will test your knowledge further. The
treatment of exhaustive search in Chapter 10 goes beyond what is normally covered
in algorithms courses. The excuse for skimming over exhaustive search is usually a
natural distate for exponential time algorithms. However, extensive problems are
provided here because exhaustive search contains many rich and interesting oppor
tunities to practice the application of algorithm design, analysis, and veriﬁcation
methods.
Chapter 11 contains problems on advanced data structures, and Chapter 12
Sec. 1.3. Pseudocode 3
contains problems on ^{completeness, Finally, Chapter 13 contains miscellaneous
problems, deﬁned to be those that do not necessarily ﬁt into the earlier chapters,
and those for which part of the problem is to determine the algorithmic technique
to be used.
1.3 PSEUDOCODE
The algorithms in this text are described in a Pascallike pseudocode. Here is a
quick overview of the conventions used:
Data Types: Most variables are either integers or one or twodimensional arrays
of integers. The notation P[i..j] is shorthand for an array P, whose elements
are P[i], P[i +1], . . . , P[j]. Occasionally, variables will be other mathematical
objects such as sets or lists.
BlockStructuring: Indentation is used to indicate blockstructuring, in an at
tempt to save space.
Assignment Statements: The “:=” symbol is the assignment operator.
Sequencing: A sequence of commands will be on separate lines or separated by
semicolons.
Selection: The selection command uses Pascal’s ifthenelse syntax, which has
the form
if condition
then S
1
else S
2
If the condition is true at time of execution, then S
1
is executed; otherwise
S
2
is executed. The else clause will be omitted if it is empty.
Indeﬁnite Iteration: Indeﬁnite iteration uses Pascal’s pretested while loop or
posttested repeat loop. The while loop has the form
while condition do
S
If the condition is true, then S is executed. The condition is then reevaluated.
If it is false, the loop terminates. Otherwise S is executed again. The process
repeats until the condition is false. The repeat loop has the form
repeat
S
until condition
4 Chap. 1. Introduction
and is equivalent to
S
while not condition do
S
Deﬁnite Iteration: Deﬁnite (countcontrolled) iteration is achieved using Pascal’s
for loop. Some loops count up (using the keyword to), and some count
down (using the keyword downto). An upwardcounting forloop will be
done zero times if the start value is larger than the ﬁnish value (analogously
for downwardcounting forloops). The syntax is as follows, with the upward
counting loop on the left, and the downwardcounting loop on the right:
for i := s to f do for i := s downto f do
S S
which are equivalent to, respectively,
i := s i := s
while i ≤ f do while i ≥ f do
S S
i := i + 1 i := i −1
Subroutines and Parameters: Subroutines are expressed using Pascalstyle pro
cedures and functions. Functions return a value, which unlike Pascal may
be a complicated object. I have chosen to follow the lead of Aho, Hopcroft,
and Ullman [2] in using a Clike return statement to describe the value re
turned by a function. Procedures return values through their parameters.
Most parameters are value parameters, though some are variable parameters.
You’ll be able to tell the diﬀerence by context — those that return values are
obviously variable parameters! Comments in the algorithms should give you
an extra hint.
1.4 USEFUL TEXTS
There are many good algorithms texts available, and more get published every
year. Here is a quick thumbnail sketch of some of them, and some related texts
that you might ﬁnd useful. I recommend that the conscientious student examine
as many texts as possible in the library and choose one that suits their individual
taste. Remember, though, that diﬀerent texts have diﬀering areas of strength, and
appeal to diﬀerent types of reader. I also strongly recommend that the practicing
programmer include in their library a copy of Bentley [9, 10]; Cormen, Leiserson,
and Rivest [19]; Garey and Johnson [28]; Graham, Knuth, and Patashnik [30]; and
Knuth [45, 46, 47].
Sec. 1.4. Useful Texts 5
Aho, Hopcroft, and Ullman [1] This was the standard graduate text on algo
rithms and data structures for many years. It is written quite tersely, with
some sentences requiring a paragraph of explanation for some students. If you
want the plain facts with no singing or dancing, this is the book for you. It is
a little dated in presentation and material, but not particularly so.
Aho, Hopcroft, and Ullman [2] This is the undergraduate version of the pre
vious book, with more emphasis on programming. It is full of minor errors.
A large number of the Pascal program fragments don’t work, even when you
correctly implement their Clike return statement.
Aho and Ullman [3] This textbook is redeﬁning the undergraduate computer sci
ence curriculum at many leading institutions. It is a good place to go to brush
up on your discrete mathematics, data structures, and problem solving skills.
Baase [7] A good algorithms text at the upperdivision undergraduate level.
Bentley [9, 10] This delightful pair of books are a collection of Jon Bentley’s Pro
gramming Pearls column from Communications of the ACM. They should be
recommended reading for all undergraduate computer science majors. Bent
ley explores the problems and pitfalls of algorithm design and analysis, and
pays careful attention to implementation and experimentation. The subjects
chosen are too idiosyncratic and anecdotal to be a textbook, but nonetheless
a useful pair of books.
Brassard and Bratley [14] This is another good algorithms text with a strong
emphasis on design techniques. The title comes from the French word algo
rithmique, which is what they (perhaps more aptly) call Computer Science.
Cormen, Leiserson, and Rivest [19] This is an excellent text for those who can
handle it. In the spirit of Aho, Hopcroft, and Ullman [1], it does not mess
around. A massive tome, it contains enough material for both a graduate and
undergraduate course in algorithms. It is the deﬁnitive text for those who
want to get right down to business, but it is not for the faint of heart.
Even [26] This is the canonical book on graph algorithms covering most of the
graph algorithm material taught in the typical algorithms course, plus more.
It is a little dated, but still very useful.
Garey and Johnson [28] This is the canonical book on ^{completeness. It is
particularly useful for its large list of ^{complete problems. It is showing
its age a little, but David Johnson has expressed an intention to work on a
Second Edition. More modern subjects are covered by David Johnson in “The
NPCompleteness Column: An OnGoing Guide” published on a semiregular
basis in Journal of Algorithms (the latest was in September 1992, but David
Johnson recently assured me that more are planned).
6 Chap. 1. Introduction
Gibbons [29] A more modern and carefully presented text on graph algorithms,
superior in many ways to Even [26].
Graham, Knuth, and Patashnik [30] An excellent book on discrete mathemat
ics for computer science. However, many students ﬁnd it quite diﬃcult going.
It is noted for its marginal (and often obscure) graﬃti.
Greene and Knuth [33] A highly mathematical text for an advanced course on
algorithm analysis at Stanford. Recommended for the serious graduate stu
dent.
Harel [34] This is another good text along the lines of Brassard and Bratley. It
contains excellent highlevel descriptions of subjects that tend to confuse the
beginning student if they are presented in all their detailed glory. It is noted
for its biblical quotes.
Horowitz and Sahni [37] This is a reasonable text for many of the topics found
in the typical algorithms class, but it is weak on analysis and but its approach
to backtracking is somewhat idiosyncratic and hard to follow.
Kingston [44] A strong textbook for the elementary undergraduate algorithms
course, but the coverage is a little shallow for advanced undergraduates. The
chapter on correctness proofs is an unusual but welcome addition. Divide
andconquer and dynamic programming are covered very sketchily, and the
approach to greedy algorithms is idiosyncratic but likely to strike a chord with
some students.
Knuth [45, 46, 47] The Knuth threevolume series is the standard reference text
for researchers and students. The material in it has aged well, except for
Knuth’s penchant for an obscure ﬁctitious assemblylevel code instead of the
more popular and easy to understand pseudocode.
Kozen [50] A text for graduate students containing 40 lectures on algorithms. The
terse presentation is not for everybody. It contains enough material for a single
graduate course, as opposed to the normal text that allows the instructor
to pick and choose the material to be covered. The choice of material is
strong but not to everybody’s taste. The homework problems (all of which
have solutions) and miscellaneous exercises (some of which have solutions) are
particularly useful.
Lewis and Denenberg [52] A strong data structures text that also covers some
of the material in the typical algorithms course.
Manber [56] Some students ﬁnd the approach taken in this book, which is algo
rithm design by successive reﬁnement of intuitive but ﬂawed version of the
algorithm, very helpful. Others, who do not think this way, hate it. It is
certainly more realistic in its approach to algorithm design than more formal
Sec. 1.4. Useful Texts 7
texts, but perhaps it encourages the wrong habits. Some important topics are
not covered in very great detail.
Moret and Shapiro [58] This is planned as the ﬁrst volume of a twovolume set.
This volume covers tractable problems, and the second volume (unsighted
as yet) will cover intractable problems. It has a deﬁnite combinatorial opti
mization ﬂavor. Dynamic programming is a notable omission from the ﬁrst
volume.
Papadimitriou and Steiglitz [59] A reasonable text for an algorithms course,
but one which is very heavy with the jargon and mindset of the combinatorial
optimization and numerical analysis community, which makes it a diﬃcult
book to just dip into at random for the parts of it that are useful.
Purdom and Brown [64] A mathematically rigorous text that focuses on anal
ysis almost to the exclusion of design principles.
Rawlins [67] An entertaining text that nonetheless is technically rigorous and
detailed.
Sedgewick [72] This book studies a large array of standard algorithms, but it is
largely descriptive in nature and does not spend much time on veriﬁcation,
analysis, or design techniques. It is excellent for the elementary undergraduate
algorithms curriculum, but not suﬃciently rigorous for upperdivision courses.
Smith [74] This is another strong algorithms text with an awesome collection of
problems.
Solow [75] This is a good text for the student who has trouble with mathematical
formalism. The chapter on induction may be particularly useful for those who
have trouble getting started.
Weiss [82] A strong data structures text that also covers quite a bit of the material
in the typical algorithms course.
Wilf [83] A book that does a good job on analysis, but does not cover algorithm
design techniques. The selection of topics is limited to chapters on mathe
matical preliminaries, recursion, network ﬂow, numbertheoretic algorithms,
and ^{completeness.
Chapter 2
Mathematical Induction
At ﬁrst sight, this chapter may seem out of place because it doesn’t seem to be
about algorithms at all. Although the problems in this chapter are mainly mathe
matical in nature and have very little to do with algorithms, they are good practice
in a technique that is fundamental to algorithm design, veriﬁcation, and analysis.
Unfortunately, many students come to my algorithms class poorly versed in the use
of induction. Either it was badly taught or they discarded it as useless information
(or both). If you are one of these students, you should start by attempting at least
some of the following problems. In addition to mastering the technique of mathe
matical induction, you should pay particular attention to the results in Sections 2.1,
2.2, 2.3, 2.7, 2.8, 2.10, and 2.11. Some of them are quite useful in algorithm design
and analysis, as we shall see later.
2.1 SUMMATIONS
1. Prove by induction on n ≥ 0 that
¸
n
i=1
i = n(n + 1)/2.
2. Prove by induction on n ≥ 0 that
¸
n
i=1
i
2
= n(n + 1)(2n + 1)/6.
3. Prove by induction on n ≥ 0 that
¸
n
i=1
i
3
= n
2
(n + 1)
2
/4.
4. Prove by induction on n ≥ 0 that
n
¸
i=0
(2i + 1)
2
= (n + 1)(2n + 1)(2n + 3)/3.
5. Prove by induction on n ≥ 0 that
¸
n
i=1
i(i+1) = n(n+1)(n+2)/3.
6. Prove by induction on n ≥ 0 that
n
¸
i=1
i(i + 1)(i + 2) = n(n + 1)(n + 2)(n + 3)/4.
8
Sec. 2.1. Summations 9
7. Prove by induction on n ≥ 0 that
¸
n
i=1
i i! = (n + 1)! −1.
8. Prove by induction on n ≥ 1 that
¸
n
i=1
1/2
i
= 1 −1/2
n
.
9. Prove by induction on n ≥ 1 that for every a = 1,
n
¸
i=0
a
i
=
a
n+1
−1
a −1
.
10. Prove by induction on n ≥ 0 that for every a = 1, and all 0 ≤ j ≤ n,
n
¸
i=j
a
i
=
a
n+1
−a
j
a −1
.
11. Prove by induction on n ≥ 0 that
¸
n
i=0
2
i
= 2
n+1
−1.
12. Prove by induction on n ≥ 1 that
n
¸
i=1
1
i(i + 1)
=
n
n + 1
.
13. Prove by induction on n ≥ 0 that
¸
n
i=1
i2
i
= (n −1)2
n+1
+ 2.
14. Prove by induction on n ≥ 0 that for every a = 1,
n
¸
i=1
ia
i
=
na
n+2
−(n + 1)a
n+1
+a
(a −1)
2
.
15. Prove by induction on n ≥ 0 that
n
¸
i=1
i
2
2
i
= n
2
2
n+1
−n2
n+2
+ 3 2
n+1
−6.
16. Prove by induction on n ≥ 0 that
n
¸
i=1
i
2
2
n−i
= 2
n+3
−2
n+1
−n
2
−4n −6.
17. Prove by induction on n ≥ 0 that
n
¸
i=1
1
n +i
=
n
¸
i=1
1
2i −1
−
1
2i
.
10 Chap. 2. Mathematical Induction
2.2 INEQUALITIES
18. Prove by induction on n ≥ 1 that if x > −1, then (1 +x)
n
≥ 1 +nx.
19. Prove by induction on n ≥ 7 that 3
n
< n!.
20. Prove by induction on n ≥ 5 that 2
n
> n
2
.
21. Prove by induction on k ≥ 1 that
¸
n
i=1
i
k
≤ n
k
(n + 1)/2.
22. Prove by induction on n ≥ 0 that
n
¸
i=1
1
i
2
< 2 −
1
n
.
23. Prove by induction that if n ≥ 4 is even, and 2 ≤ i ≤ n/2, then
i
¸
k=1
k
¸
j=1
(n −2j + 1) ≤ 2
i
¸
j=1
(n −2j + 1).
2.3 FLOORS AND CEILINGS
Suppose x ∈ IR
+
. The ﬂoor of x, denoted x, is deﬁned to be the largest integer
that is no larger than x. The ceiling of x, denoted x, is deﬁned to be the smallest
integer that is no smaller than x.
24. Prove by induction on n ≥ 0 that
n
2
¸
=
n/2 if n is even
(n −1)/2 if n is odd.
25. Prove by induction on n ≥ 0 that
n
2
¸
=
n/2 if n is even
(n + 1)/2 if n is odd.
26. Prove by induction on n ≥ 1 that for all m ∈ IR
+
,
n
m
¸
=
¸
n +m−1
m
¸
.
Sec. 2.4. Divisibility 11
2.4 DIVISIBILITY
27. Prove by induction on n ≥ 0 that n
3
+ 2n is divisible by 3.
28. Prove by induction on n ≥ 0 that n
5
−n is divisible by 5.
29. Prove by induction on n ≥ 0 that 5
n+1
+ 2 3
n
+ 1 is divisible by 8.
30. Prove by induction on n ≥ 0 that 8
n+2
+ 9
2n+1
is divisible by 73.
31. Prove by induction that for all n ≥ 0, 11
n+2
+12
2n+1
is divisible by 133.
32. Deﬁne S ⊆ IN IN as follows. (0, 0) ∈ S. If (m, n) ∈ S, then
(m+2, n+3) ∈ S. Prove by induction on n ≥ 0 that for all (m, n) ∈ S, m+n
is divisible by 5.
33. Prove by induction that a decimal number is divisible by 3 iﬀ the sum
of its digits is divisible by 3.
34. Prove by induction that a decimal number is divisible by 9 iﬀ the sum
of its digits is divisible by 9.
35. Prove by induction that the sum of the cubes of three successive natural
numbers is divisible by 9.
36. Let S
n
= ¦1, 2, . . . , 2n¦ be the set of integers from 1 to 2n. Let
T ⊂ S
n
be any subset containing exactly n + 1 elements of S
n
. Prove by
induction on n that there exists x, y ∈ T, x = y, such that x divides evenly
into y with no remainder.
2.5 POSTAGE STAMPS
37. Show that any integer postage greater than 7 cents can be formed
by using only 3cent and 5cent stamps.
38. Show that any integer postage greater than 34 cents can be formed by
using only 5cent and 9cent stamps.
39. Show that any integer postage greater than 5 cents can be formed by
using only 2cent and 7cent stamps.
40. Show that any integer postage greater than 59 cents can be formed by
using only 7cent and 11cent stamps.
41. What is the smallest value of k such that any integer postage greater
than k cents can be formed by using only 4cent and 9cent stamps? Prove
your answer correct.
12 Chap. 2. Mathematical Induction
42. What is the smallest value of k such that any integer postage greater
than k cents can be formed by using only 6cent and 11cent stamps? Prove
your answer correct.
43. Show that for all n ≥ 1, any positive integer amount of postage that
is at least n(n−1) cents can be formed by using only ncent and (n+1)cent
stamps.
44. Show that for all m, n ≥ 1 such that gcd(m, n) = 1, there exists k ∈ IN
such that any positive integer amount of postage that is at least k cents can
be formed by using only mcent and ncent stamps.
2.6 CHESSBOARD PROBLEMS
45. Prove by induction that for all n ∈ IN and all even m ∈ IN, an n m
chessboard has exactly the same number of black squares and white squares.
46. Prove by induction that for all odd n, m ∈ IN, an n m chessboard has
all four corner squares colored the same.
47. Prove by induction that for all odd n, m ∈ IN, an nm chessboard
with the corner squares colored white has one more white square than black
square.
48. Recall that a knight can make one of eight legal moves de
picted in Figure 2.1. Figure 2.2 shows a closed knight’s tour on an 8 8
chessboard, that is, a circular tour of knight’s moves that visits every square
of the chessboard exactly once before returning to the ﬁrst square. Prove by
induction that a closed knight’s tour exists for any 2
k
2
k
chessboard for all
k ≥ 3.
A triomino is an “L”shaped tile formed by three adjacent squares of a chessboard,
shown in Figure 2.3. An arrangement of triominoes is a tiling of a chessboard if
it covers it exactly without overlap, except for one square which is missed. For
example, Figure 2.4 shows a chessboard with a missing square ﬁlled in black on the
left, and a tiling with triominos that covers every square of the board except this
one on the right.
49. Prove by induction on n ≥ 1 that any 2
n
2
n
chessboard that
is missing one square can be tiled with triominoes, regardless of where the
missing square is.
50. Prove or disprove that all chessboards of the following dimensions
can be tiled by triominoes: (a) 3 2
n
, (b) 6 2
n
, (c) 3
n
3
n
, (d) 6
n
6
n
.
Sec. 2.6. Chessboard Problems 13
1
2 3
4
5
6 7
8
Figure 2.1. The eight legal moves that a knight on the
center square can make.
Figure 2.2. A closed knight’s tour on an 8 8 chessboard.
Figure 2.3. A triomino.
Figure 2.4. A chessboard with a missing square ﬁlled in
black (left), and a tiling with triominos that covers every
square of the board except the missing one (right).
14 Chap. 2. Mathematical Induction
51. Prove that any 2
n
2
n
chessboards with one square missing
can be tiled by triominoes of only three colors so that no pair of triominoes
that share an edge have the same color.
2.7 FIBONACCI NUMBERS
The Fibonacci numbers F
n
for (n ≥ 0) are deﬁned recursively as follows: F
0
= 0,
F
1
= 1, and for n ≥ 2, F
n
= F
n−1
+F
n−2
.
52. Prove by induction on n that
¸
n
i=0
F
i
= F
n+2
−1.
53. Prove by induction that F
n+k
= F
k
F
n+1
+F
k−1
F
n
.
54. Prove by induction on n ≥ 1 that
¸
n
i=1
F
2
i
= F
n
F
n+1
.
55. Prove by induction on n ≥ 1 that F
n+1
F
n+2
= F
n
F
n+3
+ (−1)
n
.
56. Prove by induction on n ≥ 2 that F
n−1
F
n+1
= F
2
n
+ (−1)
n
.
2.8 BINOMIAL COEFFICIENTS
The binomial coeﬃcient
n
r
,
for n, r ∈ IN, r ≤ n, is deﬁned to be the number of ways of choosing r things from
n without replacement, that is,
n
r
=
n!
r!(n −r)!
.
57. Prove by induction on n that
n
¸
m=0
n
m
= 2
n
.
58. Prove by induction on n ≥ 1 that for all 1 ≤ m ≤ n,
n
m
≤ n
m
.
Sec. 2.9. What is Wrong? 15
59. Prove by induction on n that
(a +b)
n
=
n
¸
m=0
n
m
a
m
b
n−m
.
60. Prove by induction on n that for all even n, if k = n/2, then
n
n/2
>
n
k
.
61. Prove by induction on n that for all even n,
n
n/2
= Ω(2
n
/n).
62. Prove by induction that for all n ∈ IN,
n
¸
i=0
i
n
i
= n2
n−1
.
63. Prove by induction that for all n ∈ IN,
n
¸
i=0
2
−i
n
i
= (3/2)
n
.
2.9 WHAT IS WRONG?
64. What, if anything is wrong with the following reasoning? All horses
are the same color. The proof is by induction on the number of horses. The
base of the induction is easy: If there is one horse, then it is trivially the same
color as itself. Now suppose that there are n horses, numbered 1 through n.
By the induction hypothesis, horses 1 through n−1 have the same color (let’s
say black). In particular, horse 2 is black. Also by the induction hypothesis,
horses 2 through n have the same color. Since horse 2 is black, this means
that horses 2 through n must be black. Therefore, all of the horses have the
same color.
65. What is wrong with the following proof? We claim that 6n = 0 for all
n ∈ IN. The proof is by induction on n ≥ 0. Clearly, if n = 0, then 6n = 0.
Now, suppose that n > 0. Let n = a+b. By the induction hypothesis, 6a = 0
and 6b = 0. Therefore,
6n = 6(a +b) = 6a + 6b = 0 + 0 = 0.
16 Chap. 2. Mathematical Induction
66. What is wrong with the following reasoning? We show that for every
n ≥ 3, F
n
is even. The base of the induction is trivial, since F
3
= 2. Now
suppose that n ≥ 4 and that F
m
is even for all m < n. Then F
n
= F
n−1
+F
n−2
,
and by the induction hypothesis F
n−1
and F
n−2
are even. Thus, F
n
is the
sum of two even numbers, and must therefore be even.
2.10 GRAPHS
A graph is an ordered pair G = (V, E), where V is a ﬁnite set and E ⊆ V V . The
elements of V are called nodes or vertices, and the elements of E are called edges.
We will follow the normal conventions of using n for the number of nodes and e for
the number of edges. Where convenient we will take V = ¦v
2
, v
2
, . . . , v
n
¦.
67. Prove by induction that a graph with n vertices can have at most
n(n −1)/2 edges.
68. A tournament is a directed graph formed by taking the complete
undirected graph and assigning arbitrary directions on the edges. That is,
it is a graph G = (V, E) such that for all u, v ∈ V , either exactly one of
(u, v), (v, u) ∈ E. Show that every tournament has a Hamiltonian path, that
is, a path that visits every vertex exactly once.
69. An Eulerian cycle in a connected graph is a cycle in which
every edge appears exactly once. Prove by induction that every graph in
which each vertex has even degree (that is, each vertex has an even number
of edges incident with it) has an Eulerian cycle.
The hypercube of dimension n is a graph deﬁned as follows. A hypercube of di
mension 0 is a single vertex. To build a hypercube of dimension n, start with a
hypercube of dimension n −1. Take a second, exact copy of this hypercube. Draw
an edge from each vertex in the ﬁrst copy to the corresponding vertex of the second
copy. For example, Figure 2.5 shows the hypercubes of dimensions 0, 1, 2, 3, and 4.
70. Prove by induction that a hypercube of dimension n has 2
n
vertices.
71. Prove by induction that a hypercube of dimension n has n2
n−1
edges.
72. Prove by induction that every hypercube has a Hamiltonian cycle, that
is, a cycle that visits every vertex exactly once.
73. Prove by induction that the vertices of a hypercube can be colored using
two colors so that no pair of adjacent vertices have the same color.
Sec. 2.11. Trees 17
Dimension 0 Dimension 1 Dimension 2
Dimension 3 Dimension 4
Figure 2.5. Some hypercubes of various dimensions.
74. Prove by induction that the edges of an ndimensional hypercube can be
colored using n colors so that no pair of edges that share a common vertex
have the same color.
75. Prove by induction that an ndimensional hypercube has exactly
n
k
2
n−k
diﬀerent subcubes of dimension k.
2.11 TREES
A tree is a special kind of graph deﬁned as follows
1
. A single vertex v is a tree
with root v. Suppose T
i
= (V
i
, E
i
) are disjoint trees with roots r
i
respectively, for
1 ≤ i ≤ k. Suppose r ∈ V
1
, V
2
, . . . , V
k
. Then, T = (V, E) is a tree, where
V = V
1
∪ V
2
∪ ∪ V
k
∪ ¦r¦,
E = E
1
∪ E
2
∪ ∪ E
k
∪ ¦(r, r
1
), (r, r
2
), . . . , (r, r
k
)¦.
1
Technically, this is actually called a rooted tree, but the distinction is not of prime importance
here.
18 Chap. 2. Mathematical Induction
The root of a tree is said to be at level 1. For all i ≥ 1, a child of a level i node
is said to be at level i +1. The number of levels in a tree is the largest level number
of any node in the tree. A binary tree is a tree in which all nodes have at most two
children. A complete binary tree is a binary tree in which all leaves are at the same
level.
76. Prove by induction that a tree with n vertices has exactly n −1 edges.
77. Prove by induction that a complete binary tree with n levels has 2
n
−1
vertices.
78. Prove by induction that if there exists an nnode tree in which all the
nonleaf nodes have k children, then n mod k = 1.
79. Prove by induction that for every n such that n mod k = 1, there exists
an nnode tree in which all of the nonleaf nodes have k children.
80. Prove by induction that between any pair of vertices in a tree there is a
unique path.
81. A cycle in a graph G = (V, E) is a sequence of distinct vertices
v
1
, v
2
, . . . , v
k
such that (v
k
, v
1
) ∈ E, and for all 1 ≤ i < k, (v
i
, v
i+1
) ∈ E.
Prove by induction that a tree is a connected graph that has no cycles.
82. Prove by induction that a connected graph that has no cycles is a
tree.
83. A spanning tree of a graph G = (V, E) is a tree T = (V, F), where F ⊆ E.
Prove by induction that an nnode graph can have as many as (n−1)! spanning
trees.
2.12 GEOMETRY
84. Prove that any set of regions deﬁned by n lines in the plane can be
colored with two colors so that no two regions that share an edge have the
same color.
85. Prove by induction that n circles divide the plane into n
2
−n +2
regions if every pair of circles intersect in exactly two points, and no three
circles intersect in a common point. Does this hold for closed shapes other
than circles?
86. A polygon is convex if every pair of points in the polygon can be
joined by a straight line that lies in the polygon. Prove by induction on n ≥ 3
that the sum of the angles of an nvertex convex polygon is 180(n−2) degrees.
Sec. 2.13. Miscellaneous 19
87. Consider any arrangement of n ≥ 3 lines in general position in the plane
(that is, no pair of lines is parallel, and no three lines can have a common
point). Prove that at least one of the minimal connected regions that they
form is a triangle.
88. Consider any arrangement of n ≥ 3 lines in general position in the
plane. Prove that at least n−2 of the minimal connected regions in any such
arrangement are triangles. Is this bound tight?
89. Prove that n straight lines in the plane, all passing through a single
point, divide the plane into 2n regions.
90. Prove that n planes in space, all passing through a single point, no
three of which meet in a straight line, divide space into n(n −2) + 2 regions.
2.13 MISCELLANEOUS
91. Suppose M
i
is an r
i−1
r
i
matrix, for 1 ≤ i ≤ n. Prove by induction on
n ≥ 1 that the matrix product M
1
M
2
M
n
is an r
0
r
n
matrix.
92. Prove by induction on n ≥ 1 that the binary representation of n has
log n + 1 bits.
93. If x, y ∈ ¦true, false¦, let x ⊕ y denote the exclusiveor of x and y,
which is deﬁned to be true iﬀ exactly one of x and y is true. Note that
the exclusiveor operation is associative, that is, a ⊕ (b ⊕ c) = (a ⊕ b) ⊕ c.
Prove by induction on n that x
1
⊕x
2
⊕ ⊕x
n
is true iﬀ an odd number of
x
1
, x
2
, . . . , x
n
are true.
94. (The Pigeonhole Principle) Suppose n pigeons roost in m holes.
Prove by induction on n that there must be at least one hole containing at
least n/m pigeons.
95. Suppose A
n
= ¦a
1
, a
2
, . . . , a
n
¦ is a set of distinct coin types,
where each a
i
∈ IN, for 1 ≤ i ≤ n. Suppose also that a
1
≤ a
2
≤ ≤ a
n
. The
coinchanging problem is deﬁned as follows. Given C ∈ IN, ﬁnd the smallest
number of coins from A
n
that add up to C, given that an unlimited number of
coins of each type are available. Show that for all n ≥ 2, if a
1
= 1, then there
is always a solution to the coinchanging problem, and that solution uses less
than a
2
coins of type a
1
. What happens when a
1
= 1?
A Gray code is a list of the 2
n
nbit strings in which each string diﬀers from the
previous one in exactly one bit. Consider the following algorithm for listing the
20 Chap. 2. Mathematical Induction
nbit strings. If n = 1, the list is 0, 1. If n > 1, ﬁrst take the list of (n − 1)bit
strings, and place a 0 in front of each string. Then, take a second copy of the list of
(n−1)bit strings, place a 1 in front of each string, reverse the order of the strings,
and place it after the ﬁrst list. So, for example, for n = 2 the list is 00, 01, 11, 10,
and for n = 3 the list is 000, 001, 011, 010, 110, 111, 101, 100.
96. Show that every nbit string appears exactly once in the list generated
by the algorithm.
97. Show that the list has the Gray code property, that is, that each string
diﬀers from the previous one in exactly one bit.
2.14 HINTS
37. You will need to use strong induction. And your base case may not be what
you expect at ﬁrst,
48. Try combining four n/2n/2 tours into one. Your induction hypothesis must
be stronger than what you really need to prove — the tours you build must
have some extra structure to allow them to be joined at the corners.
49. Look carefully at the example given before the problem. It illustrates the
technique that you will need. Also, consider the hint to Problem 48.
53. You will need to use strong induction here.
57. You will need to use the identity
n
r
=
n −1
r
+
n −1
r −1
.
(Can you prove this identity?) Be careful how you apply this identity (it
doesn’t always hold), and take special care with the upper and lower limits
of the summation.
69. First, prove that any graph in which all vertices have even degree must have a
cycle C (try constructing one). Delete C, leaving a smaller graph in which all
vertices have even degree. Apply the induction hypothesis (careful: there is
something I am not telling you). Put together C and the results of applying
the induction hypothesis to give the Eulerian cycle.
85. The circles are a red herring.
86. The hardest part of this is the base case, proving that the sum of the three
angles of a triangle is 180 degrees. It can be proved as follows. First, prove
Sec. 2.15. Solutions 21
it for a rightangled triangle (using symmetry). This can be extended to any
triangle by judicious choice of diagonal. Once you have mastered the base
case, the inductive step should be obvious.
92. You’ll need to make two cases in your inductive step, according to whether n
is a power of 2 (or something like that) or not.
2.15 SOLUTIONS
1. We are required to prove that for all n ≥ 0,
¸
n
i=1
i = n(n + 1)/2. The claim
is certainly true for n = 0, in which case both sides of the equation are zero.
Suppose that n ≥ 0 and
¸
n
i=1
i = n(n + 1)/2. We are required to prove that
¸
n+1
i=1
i = (n + 1)(n + 2)/2. Now,
n+1
¸
i=1
i =
n
¸
i=1
i + (n + 1)
= n(n + 1)/2 + (n + 1) (by the induction hypothesis)
= (n + 1)(n + 2)/2,
as required.
8. We are required to prove that for all n ≥ 1,
¸
n
i=1
1/2
i
= 1 −1/2
n
. The claim
is true for n = 1, since in this case both sides of the equation are equal to
1/2. Now suppose that n ≥ 1, and
¸
n
i=1
1/2
i
= 1−1/2
n
. It remains to prove
that
¸
n+1
i=1
1/2
i
= 1 −1/2
n+1
.
n+1
¸
i=1
1/2
i
=
1
2
+
1
4
+
1
8
+ +
1
2
n+1
=
1
2
+
1
2
1
2
+
1
4
+
1
8
+ +
1
2
n
=
1
2
+
1
2
n
¸
i=1
1
2
i
−
1
2
+
1
2
(1 −
1
2
n
) (by the induction hypothesis)
= 1 −1/2
n+1
,
as required.
11. We are required to prove that for all n ≥ 0,
¸
n
i=0
2
i
= 2
n+1
− 1. The
claim is certainly true for n = 0, in which case
¸
0
i=0
2
i
= 1 = 2
1
− 1.
Suppose that n ≥ 0 and
¸
n
i=0
2
i
= 2
n+1
− 1. We are required to prove that
22 Chap. 2. Mathematical Induction
¸
n+1
i=0
2
i
= 2
n+2
−1. Now,
n+1
¸
i=0
2
i
=
n
¸
i=0
2
i
+ 2
n+1
= (2
n+1
−1) + 2
n+1
(by the induction hypothesis)
= 2
n+2
−1,
as required.
27. We are required to prove by induction on n ≥ 0 that n
3
+ 2n is divisible by
3. The claim is true for n = 0, since then n
3
+ 2n = 0, which is certainly
divisible by 3. Suppose that n ≥ 0, and n
3
+ 2n is divisible by 3. We are
required to prove that (n + 1)
3
+ 2(n + 1) is divisible by 3. Now,
(n + 1)
3
+ 2(n + 1) = (n
3
+ 3n
2
+ 3n + 1) + (2n + 2)
= n
3
+ 3n
2
+ 5n + 3
= (n
3
+ 2n) + (3n
2
+ 3n + 3).
Since the ﬁrst term is divisible by 3 (by the induction hypothesis) and the
second term is obviously divisible by 3, it follows that (n + 1)
3
+ 2(n + 1) is
divisible by 3, as required.
37. We are required to show that any amount of postage that is a positive integer
number of cents n > 7 can be formed by using only 3cent and 5cent stamps.
The claim is certainly true for n = 8 (one 3cent and one 5cent stamp), n = 9
(three 3cent stamps), and n = 10 (two 5cent stamps). Now suppose that
n ≥ 11 and all values up to n − 1 cents can be made with 3cent and 5cent
stamps. Since n ≥ 11, n − 3 ≥ 8, and hence by hypothesis, n − 3 cents can
be made with 3cent and 5cent stamps. Simply add a 3cent stamp to this
to make n cents. Notice that we can prove something stronger with just a
little more work. The required postage can always be made using at most
two 5cent stamps. Can you prove this?
52. We are required to prove that for all n ≥ 0,
¸
n
i=0
F
i
= F
n+2
− 1. The claim
is certainly true for n = 0, since the lefthand side of the equation is F
0
= 0,
and the righthand side is F
2
−1 = 1 −1 = 0. Now suppose that n ≥ 1, and
that
¸
n−1
i=0
F
i
= F
n+1
−1. Then,
n
¸
i=0
F
i
=
n−1
¸
i=0
F
i
+F
n
= (F
n+1
−1) +F
n
(by the induction hypothesis)
= F
n+2
−1,
as required.
Sec. 2.16. Comments 23
L
Figure 2.6. Shading the regions formed by all lines except
L (top), and the new shading obtained by ﬂipping the colors
on one side of L (bottom).
84. We are required to prove that any set of regions deﬁned by n lines in the
plane can be colored with only two colors so that no two regions that share
an edge have the same color. The hypothesis is true for n = 1 (color one side
light, the other side dark). Now suppose that the hypothesis is true for n
lines. Suppose we are given n +1 lines in the plane. Remove one of the lines
L, and color the remaining regions with two colors (which can be done, by
the induction hypothesis). Replace L. Reverse all of the colors on one side
of the line. Consider two regions that have a line in common. If that line is
not L, then by the induction hypothesis, the two regions have diﬀerent colors
(either the same as before or reversed). If that line is L, then the two regions
formed a single region before L was replaced. Since we reversed colors on one
side of L only, they now have diﬀerent colors.
2.16 COMMENTS
1. This identity was apparently discovered by Gauss at age 9 in 1786. There is
an apocryphal story attached to this discovery. Gauss’ teacher set the class
to add the numbers 1 +2 + +100. One account I have heard said that the
teacher was lazy, and wanted to occupy the students for a long time so that
he could read a book. Another said that it was punishment for the class being
unruly and disruptive. Gauss came to the teacher within a few minutes with
an answer of 5050. The teacher was dismayed, and started to add. Some time
later, after some fretting and fuming, he came up with the wrong answer. He
refused to believe that he was wrong until Gauss explained:
24 Chap. 2. Mathematical Induction
1 + 2 + + 50
+ 100 + 99 + + 51
101 + 101 + + 101 = 50 101 = 5050.
Can you prove this result without using induction? The case for even n can
easily be inferred from Gauss’ example. The case for odd n is similar.
3. I could go on asking for more identities like this. I can never remember the
exact form of the solution. Fortunately, there is not much need to. In general,
if you are asked to solve
n
¸
i=1
i
k
for some k ≥ 1, just remember that the solution is a polynomial in n with
largest exponent k +1 (see Problem 21). Simply hypothesize that the sum is
a
k+1
n
k+1
+a
k
n
k
+ +a
1
n +a
0
for some choice of a
i
∈ IR, 0 ≤ i ≤ k + 1, and try to prove it by induction.
The right values for the constants will drop right out of the proof. It gets a
bit tricky as k becomes larger, though. If you want to practice this technique,
start with k = 2. If you want a challenge to your ability to do arithmetic, try
k = 4.
4. Yes, I realize that you can solve this without induction by applying some
elementary algebra and the identities of Problems 1 and 2. What I want you
to do is practice your induction by proving this identity from ﬁrst principles.
5. See the comment to Problem 4.
6. See the comment to Problem 4.
9. There is an elegant noninductive proof of this that you should have met in
high school. Let S =
¸
n
i=0
a
i
. Then,
aS −S =
n+1
¸
i=1
a
i
−
n
¸
i=0
a
i
= a
n+1
−1.
Hence, S = (a
n+1
−1)/(a −1), as required.
10. This can also be solved by applying the identity of Problem 9 twice.
11. This is a special case of the identity in Problem 9. There is an easy noninduc
tive proof of this fact that appeals more to computer science students than
math students. The binary representation of
¸
n
i=0
2
i
is a string of n+1 ones,
that is,
11 1
. .. .
n+1
.
Sec. 2.16. Comments 25
n
n1
2
n
2
n1
2
n1
Figure 2.7. Using the method of areas to show that
¸
n
i=1
i2
i
= (2n −1)2
n
−
¸
n−1
i=1
2
i
.
If you add 1 to this, you get, in binary,
1 00 0
. .. .
n+1
,
that is, 2
n+1
. Therefore,
¸
n
i=0
2
i
+ 1 = 2
n+1
or, equivalently,
¸
n
i=0
2
i
=
2
n+1
−1.
13. There is an elegant noninductive proof of this using the method of areas. This
technique works well for sums of products: Simply make a rectangle with area
equal to each term of the sum, ﬁt them together to “nearly” make a rectangle,
and analyze the missing area.
n
¸
i=1
i2
i
= (2n −1)2
n
−
n−1
¸
i=1
2
i
(see Figure 2.7)
= (2n −1)2
n
−(2
n
−2) (by Problem 1)
= (n −1)2
n+1
+ 2.
15. This can also be solved using the method of areas sketched in the comment
to Problem 13.
16. This can also be solved using algebra and the identity from Problem 15.
19. And, therefore, 3
n
= O(n!). But we’re getting ahead of ourselves here. See
Chapter 3.
20. And, therefore, 2
n
= Ω(n
2
). But we’re getting ahead of ourselves here. See
Chapter 3.
48. There are actually closed knight’s tours on n n chessboards for all even
n ≥ 6 (see Problem 369). Note that Problem 47 implies that there can be no
such tour for odd n. The formal study of the knight’s tour problem is said
26 Chap. 2. Mathematical Induction
to have begun with Euler [25] in 1759, who considered the standard 8 8
chessboard. Rouse Ball and Coxeter [8] give an interesting bibliography of
the history of the problem from this point.
51. This problem was suggested to the author by Nainan Kovoor in 1990.
60. Don’t cheat by using Stirling’s approximation. You’re supposed to be prac
ticing your induction!
61. See the comment for Problem 60.
62. There is an elegant noninductive proof of the identity
n
¸
i=0
i
n
i
= n2
n−1
.
Write down every string of n bits. Since there are
n
i
strings with exactly
i ones, you must have written exactly
n
¸
i=0
i
n
i
ones. But since you have written exactly as many ones as zeros, and you have
written n2
n
bits in all, there must be n2
n−1
ones.
64. Of course it is obvious that the proof is wrong, since all horses are not of the
same color. What I want you to do is point out where in the proof is there a
false statement made?
67. This can be proved using Problem 1, but what I’m looking for is a straight
forward inductive proof.
69. An Eulerian cycle is not possible if the graph has at least one vertex of odd
degree. (Can you prove this?) Therefore, we conclude that a graph has a
Eulerian cycle iﬀ it is connected and all vertices have even degree. The Swiss
mathematician Leonhard Euler ﬁrst proved this in 1736 [24].
81. This problem together with Problem 82 show that “a connected graph with
no cycles” is an alternate deﬁnition of a tree. You’ll see this in the litera
ture as often as you’ll see the recursive deﬁnition given at the beginning of
Section 2.11.
82. See the comment for Problem 81.
Sec. 2.16. Comments 27
94. Yes, this is a little contrived. It’s easier to prove by contradiction. But it’s
good practice in using induction.
95. See also Problems 415 and 471.
Chapter 3
BigO and BigΩ
BigO notation is useful for the analysis of algorithms since it captures the asymp
totic growth pattern of functions and ignores the constant multiple (which is out
of our control anyway when algorithms are translated into programs). We will use
the following deﬁnitions (although alternatives exist; see Section 3.5). Suppose that
f, g: IN→IN.
• f(n) is O(g(n)) if there exists c, n
0
∈ IR
+
such that for all n ≥ n
0
, f(n) ≤
c g(n).
• f(n) is Ω(g(n)) if there exists c, n
0
∈ IR
+
such that for all n ≥ n
0
, f(n) ≥
c g(n).
• f(n) is Θ(g(n)) if f(n) is O(g(n)) and f(n) is Ω(g(n)).
• f(n) is o(g(n)) if lim
n→∞
f(n)/g(n) = 0.
• f(n) is ω(g(n)) if lim
n→∞
g(n)/f(n) = 0.
• f(n) ∼ g(n) if lim
n→∞
f(n)/g(n) = 1.
We will follow the normal convention of writing, for example, “f(n) = O(g(n))”
instead of “f(n) is O(g(n))”, even though the equality sign does not indicate true
equality. The proper deﬁnition of “O(g(n))” is as a set:
O(g(n)) = ¦f(n) [ there exists c, n
0
∈ IR
+
such that for all n ≥ n
0
, f(n) ≤ c g(n)¦.
Then, “f(n) = O(g(n))” can be interpreted as meaning “f(n) ∈ O(g(n)).”
3.1 RANK THE FUNCTIONS
98. Consider the following eighteen functions:
√
n n 2
n
nlog n n −n
3
+ 7n
5
n
2
+ log n
n
2
n
3
log n
n
1/3
+ log n (log n)
2
n!
lnn n/ log n log log n
(1/3)
n
(3/2)
n
6
28
Sec. 3.1. Rank the Functions 29
Group these functions so that f(n) and g(n) are in the same group if and
only if f(n) = O(g(n)) and g(n) = O(f(n)), and list the groups in increasing
order.
99. Draw a line from each of the ﬁve functions in the center to the best
bigΩ value on the left, and the best bigO value on the right.
Ω(1/n) O(1/n)
Ω(1) O(1)
Ω(log log n) O(log log n)
Ω(log n) O(log n)
Ω(log
2
n) O(log
2
n)
Ω(
3
√
n) O(
3
√
n)
Ω(n/ log n) 1/(log n) O(n/ log n)
Ω(n) 7n
5
−3n + 2 O(n)
Ω(n
1.00001
) (n
2
+n)/(log
2
n + log n) O(n
1.00001
)
Ω(n
2
/ log
2
n) 2
log
2
n
O(n
2
/ log
2
n)
Ω(n
2
/ log n) 3
n
O(n
2
/ log n)
Ω(n
2
) O(n
2
)
Ω(n
3/2
) O(n
3/2
)
Ω(2
n
) O(2
n
)
Ω(5
n
) O(5
n
)
Ω(n
n
) O(n
n
)
Ω(n
n
2
) O(n
n
2
)
For each of the following pairs of functions f(n) and g(n), either f(n) = O(g(n))
or g(n) = O(f(n)), but not both. Determine which is the case.
100. f(n) = (n
2
−n)/2, g(n) = 6n.
101. f(n) = n + 2
√
n, g(n) = n
2
.
102. f(n) = n + log n, g(n) = n
√
n.
103. f(n) = n
2
+ 3n + 4, g(n) = n
3
.
104. f(n) = nlog n, g(n) = n
√
n/2.
105. f(n) = n + log n, g(n) =
√
n.
106. f(n) = 2(log n)
2
, g(n) = log n + 1.
107. f(n) = 4nlog n +n, g(n) = (n
2
−n)/2.
30 Chap. 3. BigO and BigΩ
3.2 TRUE OR FALSE?
108. n
2
= O(n
3
).
109. n
3
= O(n
2
).
110. 2n
2
+ 1 = O(n
2
).
111. nlog n = O(n
√
n).
112.
√
n = O(log n).
113. log n = O(
√
n).
114. n
3
= O(n
2
(1 +n
2
)).
115. n
2
(1 +
√
n) = O(n
2
).
116. n
2
(1 +
√
n) = O(n
2
log n).
117. 3n
2
+
√
n = O(n
2
).
118. 3n
2
+
√
n = O(n +n
√
n +
√
n).
119. log n +
√
n = O(n).
120.
√
nlog n = O(n).
121. 1/n = O(log n).
122. log n = O(1/n).
123. log n = O(n
−1/2
).
124. n +
√
n = O(
√
nlog n).
125. If f(n) ∼ g(n), then f(n) = Θ(g(n)).
126. If f(n) = Θ(g(n)), then g(n) = Θ(f(n)).
For each of the following pairs of functions f(n) and g(n), state whether f(n) =
O(g(n)), f(n) = Ω(g(n)), f(n) = Θ(g(n)), or none of the above.
127. f(n) = n
2
+ 3n + 4, g(n) = 6n + 7.
128. f(n) =
√
n, g(n) = log(n + 3).
129. f(n) = n
√
n, g(n) = n
2
−n.
130. f(n) = n +n
√
n, g(n) = 4nlog(n
2
+ 1).
131. f(n) = (n
2
+ 2)/(1 + 2
−n
), g(n) = n + 3.
132. f(n) = 2
n
−n
2
, g(n) = n
4
+n
2
.
Sec. 3.3. Proving BigO 31
3.3 PROVING BIGO
133. Prove that (n + 1)
2
= O(n
2
).
134. Prove that 3n
2
−8n + 9 = O(n
2
).
135. Prove that for all k ≥ 1 and all a
k
, a
k−1
, . . . , a
1
, a
0
∈ IR,
a
k
n
k
+a
k−1
n
k−1
+ +a
1
n +a
0
= O(n
k
).
136. Prove that log n = O(n).
137. Prove that 3nlog n = O(n
2
).
138. Prove that n
2
−3n −18 = Ω(n).
139. Prove that n
3
−3n
2
−n + 1 = Θ(n
3
).
140. Prove that n = O(2
n
).
141. Prove that 2n + 1 = O(2
n
).
142. Prove that 9999n + 635 = O(2
n
).
143. Prove that cn +d = O(2
n
) for all c, d ∈ IR
+
.
144. Prove that n
2
= O(2
n
).
145. Prove that cn
2
+d = O(2
n
) for all c, d ∈ IR
+
.
146. Prove that cn
k
+d = O(2
n
) for all c, d, k ∈ IR
+
.
147. Prove that 2
n
= O(n!).
148. Prove that n! = Ω(2
n
).
149. Does n
log n
= O((log n)
n
)? Prove your answer.
150. Does n
log n
= Ω((log n)
n
)? Prove your answer.
151. Does n
log log log n
= O((log n)!)? Prove your answer.
152. Does n
log log log n
= Ω((log n)!)? Prove your answer.
153. Does (n!)! = O(((n −1)!)!(n −1)!
n!
)? Prove your answer.
154. Does (n!)! = Ω(((n −1)!)!(n −1)!
n!
)? Prove your answer.
155. Prove or disprove:
O
n
2
log log n
1/2
= O(
√
n).
32 Chap. 3. BigO and BigΩ
156. Prove or disprove: 2
(1+O(1/n))
2
= 2 +O(1/n).
Compare the following pairs of functions f, g. In each case, say whether f = o(g),
f = ω(g), or f = Θ(g), and prove your claim.
157. f(n) = 100n + log n, g(n) = n + (log n)
2
.
158. f(n) = log n, g(n) = log log(n
2
).
159. f(n) = n
2
/log n, g(n) = n(log n)
2
.
160. f(n) = (log n)
10
6
, g(n) = n
10
−6
.
161. f(n) = nlog n, g(n) = (log n)
log n
.
162. f(n) = n2
n
, g(n) = 3
n
.
For each of the following pairs of functions f(n) and g(n), ﬁnd c ∈ IR
+
such that
f(n) ≤ c g(n) for all n > 1.
163. f(n) = n
2
+n, g(n) = n
2
.
164. f(n) = 2
√
n + 1, g(n) = n +n
2
.
165. f(n) = n
2
+n + 1, g(n) = 2n
3
.
166. f(n) = n
√
n +n
2
, g(n) = n
2
.
167. f(n) = 12n + 3, g(n) = 2n −1.
168. f(n) = n
2
−n + 1, g(n) = n
2
/2.
169. f(n) = 5n + 1, g(n) = (n
2
−6n)/2.
170. f(n) = 5
√
n −1, g(n) = n −
√
n.
3.4 MANIPULATING BIGO
171. Prove that if f
1
(n) = O(g
1
(n)) and f
2
(n) = O(g
2
(n)), then
f
1
(n) +f
2
(n) = O(g
1
(n) +g
2
(n)).
172. Prove that if f
1
(n) = Ω(g
1
(n)) and f
2
(n) = Ω(g
2
(n)), then f
1
(n) +
f
2
(n) = Ω(g
1
(n) +g
2
(n)).
173. Prove that if f
1
(n) = O(g
1
(n)) and f
2
(n) = O(g
2
(n)), then f
1
(n) +
f
2
(n) = O(max¦g
1
(n), g
2
(n)¦).
Sec. 3.5. Alternative Deﬁnitions 33
174. Prove that if f
1
(n) = Ω(g
1
(n)) and f
2
(n) = Ω(g
2
(n)), then f
1
(n) +
f
2
(n) = Ω(min¦g
1
(n), g
2
(n)¦).
175. Suppose that f
1
(n) = Θ(g
1
(n)) and f
2
(n) = Θ(g
2
(n)). Is it true
that f
1
(n) + f
2
(n) = Θ(g
1
(n) + g
2
(n))? Is it true that f
1
(n) + f
2
(n) =
Θ(max¦g
1
(n), g
2
(n)¦)? Is it true that f
1
(n) +f
2
(n) = Θ(min¦g
1
(n), g
2
(n)¦)?
Justify your answer.
176. Prove that if f
1
(n) = O(g
1
(n)) and f
2
(n) = O(g
2
(n)), then f
1
(n)
f
2
(n) = O(g
1
(n) g
2
(n)).
177. Prove that if f
1
(n) = Ω(g
1
(n)) and f
2
(n) = Ω(g
2
(n)), then f
1
(n)f
2
(n) =
Ω(g
1
(n) g
2
(n)).
178. Prove or disprove: For all functions f(n) and g(n), either f(n) =
O(g(n)) or g(n) = O(f(n)).
179. Prove or disprove: If f(n) > 0 and g(n) > 0 for all n, then O(f(n) +
g(n)) = f(n) +O(g(n)).
180. Prove or disprove: O(f(n)
α
) = O(f(n))
α
for all α ∈ IR
+
.
181. Prove or disprove: O(x +y)
2
= O(x
2
) +O(y
2
).
182. Multiply log n + 6 +O(1/n) by n +O(
√
n) and simplify your answer as
much as possible.
183. Show that bigO is transitive. That is, if f(n) = O(g(n)) and g(n) =
O(h(n)), then f(n) = O(h(n)).
184. Prove that if f(n) = O(g(n)), then f(n)
k
= O(g(n)
k
).
185. Prove or disprove: If f(n) = O(g(n)), then 2
f(n)
= O(2
g(n)
).
186. Prove or disprove: If f(n) = O(g(n)), then log f(n) = O(log g(n)).
187. Suppose f(n) = Θ(g(n)). Prove that h(n) = O(f(n)) iﬀ h(n) = O(g(n)).
188. Prove or disprove: If f(n) = O(g(n)), then f(n)/h(n) = O(g(n)/h(n)).
3.5 ALTERNATIVE DEFINITIONS
Here is an alternative deﬁnition of O.
• f(n) is O
1
(g(n)) if there exists c ∈ IR such that lim
n→∞
f(n)/g(n) = c.
189. Prove that if f(n) = O(g(n)), then f(n) = O
1
(g(n)), or ﬁnd a counterex
ample to this claim.
34 Chap. 3. BigO and BigΩ
190. Prove that if f(n) = O
1
(g(n)), then f(n) = O(g(n)), or ﬁnd a counterex
ample to this claim.
Here are two alternative deﬁnitions of Ω.
• f(n) is Ω
1
(g(n)) if there exists c ∈ IR
+
such that for inﬁnitely many n, f(n) ≥
c g(n).
• f(n) is Ω
2
(g(n)) if there exists c ∈ IR
+
such that for all n
0
∈ IN, there exists
n ≥ n
0
such that f(n) ≥ c g(n).
191. Prove that if f(n) = Ω(g(n)), then f(n) = Ω
2
(g(n)), or ﬁnd a counterex
ample to this claim.
192. Prove that if f(n) = Ω
2
(g(n)), then f(n) = Ω(g(n)), or ﬁnd a coun
terexample to this claim.
193. Prove that if f(n) = Ω
1
(g(n)), then f(n) = Ω
2
(g(n)), or ﬁnd a coun
terexample to this claim.
194. Prove that if f(n) = Ω
2
(g(n)), then f(n) = Ω
1
(g(n)), or ﬁnd a coun
terexample to this claim.
195. Prove or disprove: If f(n) = O(g(n)), then f(n) = Ω(g(n)). If
f(n) = O(g(n)), then f(n) = Ω
2
(g(n)).
196. Deﬁne the relation ≡ by f(n) ≡ g(n) iﬀ f(n) = Ω(g(n)) and g(n) =
Ω(f(n)). Similarly, deﬁne the relation ≡
2
by f(n) ≡ g(n) iﬀ f(n) = Ω
2
(g(n))
and g(n) = Ω
2
(f(n)). Show that ≡ is an equivalence relation, but ≡
2
is not
an equivalence relation.
3.6 FIND THE FUNCTIONS
Find two functions f(n) and g(n) that satisfy the following relationships. If no such
f and g exist, write “None.”
197. f(n) = o(g(n)) and f(n) = Θ(g(n)).
198. f(n) = Θ(g(n)) and f(n) = o(g(n)).
199. f(n) = Θ(g(n)) and f(n) = O(g(n)).
200. f(n) = Ω(g(n)) and f(n) = O(g(n)).
201. f(n) = Ω(g(n)) and f(n) = o(g(n)).
Sec. 3.7. Hints 35
3.7 HINTS
98. There are a lot of groups.
99. Be careful with (n
2
+n)/(log
2
n + log n).
136. When solving problems that require you to prove that f(n) = O(g(n)), it is a
good idea to try induction ﬁrst. That is, pick a c and an n
0
, and try to prove
that for all n ≥ n
0
, f(n) ≤ c g(n). Try starting with n
0
= 1. A good way
of guessing a value for c is to look at f(n) and g(n) for n = n
0
, , n
0
+ 10.
If the ﬁrst function seems to grow faster than the second, it means that you
must adjust your n
0
higher — eventually (unless the thing you are trying to
prove is false), the ﬁrst function will grow more slowly than the second. The
ratio of f(n
0
)/g(n
0
) gives you your ﬁrst cut for c. Don’t be afraid to make c
higher than this initial guess to help you make it easier to prove the inductive
step. This happens quite often. You might even have to go back and adjust
n
0
higher to make things work out correctly.
171. Start by writing down the deﬁnitions for f
1
(n) = O(g
1
(n)) and f
2
(n) =
O(g
2
(n)).
3.8 SOLUTIONS
133. We are required to prove that (n + 1)
2
= O(n
2
). We need to ﬁnd a constant
c such that (n + 1)
2
≤ cn
2
. That is, n
2
+ 2n + 1 ≤ cn
2
or, equivalently,
(c −1)n
2
−2n −1 ≥ 0. Is this possible? It is if c > 1. Take c = 4. The roots
of 3n
2
− 2n − 1 are (2 ±
√
4 + 12)/2 = ¦−1/3, 1¦. The second root gives us
the correct value for n
0
. Therefore, for all n ≥ 1, (n + 1)
2
≤ 4n
2
, and so by
deﬁnition, (n + 1)
2
= O(n
2
).
136. We are required to prove that log n = O(n). By looking at log n for
small values of n, it appears that for all n ≥ 1, log n ≤ n. The proof is
by induction on n. The claim is certainly true for n = 1. Now suppose that
n > 1, and log(n −1) ≤ n −1. Then,
log n ≤ log(n −1) + 1
≤ (n −1) + 1 (by the induction hypothesis)
= n.
Hence, we can take c = 1 and n
0
= 1.
137. We are required to prove that 3nlog n = O(n
2
). By looking at 3nlog n for
small values of n, it appears that for all n ≥ 1, 3nlog n ≤ 3n
2
. The proof is
by induction on n. The claim is certainly true for n = 1. Now suppose that
36 Chap. 3. BigO and BigΩ
n > 1, and 3(n −1)log(n −1) ≤ 3(n −1)
2
. Then,
3nlog n
≤ 3n(log(n −1) + 1)
= 3(n −1)(log(n −1) + 1) + 3(log(n −1) + 1)
= 3(n −1)log(n −1) + 3(n −1) + 3(log(n −1) + 1)
≤ 3(n −1)
2
+ 3(n −1) + 3(log(n −1) + 1)
(by the induction hypothesis)
≤ 3(n −1)
2
+ 3(n −1) + 3n (see the solution to Problem 136)
= 3n
2
−6n + 3 + 3n −3 + 3n
= 3n
2
.
Hence, we can take c = 3 and n
0
= 1.
171. We are required to prove that if f
1
(n) = O(g
1
(n)) and f
2
(n) = O(g
2
(n)), then
f
1
(n)+f
2
(n) = O(g
1
(n)+g
2
(n)). Suppose for all n ≥ n
1
, f
1
(n) ≤ c
1
g
1
(n) and
for all n ≥ n
2
, f
2
(n) ≤ c
2
g
2
(n). Let n
0
= max¦n
1
, n
2
¦ and c
0
= max¦c
1
, c
2
¦.
Then for all n ≥ n
0
, f
1
(n) +f
2
(n) ≤ c
1
g
1
(n) +c
2
g
2
(n) ≤ c
0
(g
1
(n) +g
2
(n)).
3.9 COMMENTS
173. This is often called the sum rule for bigOs.
176. This is often called the product rule for bigOs.
Chapter 4
Recurrence Relations
Recurrence relations are a useful tool for the analysis of recursive algorithms, as we
will see later in Section 6.2. The problems in this chapter are intended to develop
skill in solving recurrence relations.
4.1 SIMPLE RECURRENCES
Solve the following recurrences exactly.
202. T(1) = 1, and for all n ≥ 2, T(n) = 3T(n −1) + 2.
203. T(1) = 8, and for all n ≥ 2, T(n) = 3T(n −1) −15.
204. T(1) = 2, and for all n ≥ 2, T(n) = T(n −1) +n −1.
205. T(1) = 3, and for all n ≥ 2, T(n) = T(n −1) + 2n −3.
206. T(1) = 1, and for all n ≥ 2, T(n) = 2T(n −1) +n −1.
207. T(1) = 5, and for all n ≥ 2, T(n) = 2T(n −1) + 3n + 1.
208. T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 2T(n/2) + 6n −1.
209. T(1) = 4, and for all n ≥ 2 a power of 2, T(n) = 2T(n/2) + 3n + 2.
210. T(1) = 1, and for all n ≥ 2 a power of 6, T(n) = 6T(n/6) + 2n + 3.
211. T(1) = 3, and for all n ≥ 2 a power of 6, T(n) = 6T(n/6) + 3n −1.
212. T(1) = 3, and for all n ≥ 2 a power of 3, T(n) = 4T(n/3) + 2n −1.
213. T(1) = 2, and for all n ≥ 2 a power of 3, T(n) = 4T(n/3) + 3n −5.
214. T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 3T(n/2) +n
2
−n.
215. T(1) = 4, and for all n ≥ 2 a power of 2, T(n) = 3T(n/2) +n
2
−2n +1.
216. T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 3T(n/2) +n −2.
37
38 Chap. 4. Recurrence Relations
217. T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 3T(n/2) + 5n −7.
218. T(1) = 1, and for all n ≥ 2 a power of 3, T(n) = 4T(n/3) +n
2
.
219. T(1) = 1, and for all n ≥ 2 a power of 3, T(n) = 4T(n/3) +n
2
−7n +5.
220. T(1) = 1, and for n ≥ 4 a power of 4, T(n) = T(n/4) +
√
n + 1.
4.2 MORE DIFFICULT RECURRENCES
221. Suppose 0 < α, β < 1, where α + β = 1. Let T(1) = 1, and for all
n ≥ 1, T(n) = T(αn) + T(βn) + cn, for some c ∈ IN. Prove that T(n) =
O(nlog n). You may make any necessary assumptions about n.
222. The Fibonacci numbers F
n
for n ≥ 0 are deﬁned recursively as
follows: F
0
= 0, F
1
= 1, and for n ≥ 2, F
n
= F
n−1
+F
n−2
. Prove by induction
on n that F
n
= (φ
n
−
ˆ
φ
n
)/
√
5, where φ = (1 +
√
5)/2, and
ˆ
φ = (1 −
√
5)/2.
Let X(n) be the number of diﬀerent ways of parenthesizing the product of n values.
For example, X(1) = X(2) = 1, X(3) = 2 (they are (xx)x and x(xx)), and X(4) = 5
(they are x((xx)x), x(x(xx)), (xx)(xx), ((xx)x)x, and (x(xx))x).
223. Prove that if n ≤ 2, then X(n) = 1; and otherwise
X(n) =
n−1
¸
k=1
X(k) X(n −k)
224. Show that for all n ≥ 1, X(n) ≥ 2
n−2
.
225. Show that
X(n) =
1
n
2n −2
n −1
.
Solve the following recurrences exactly.
226. T(1) = 1, T(2) = 6, and for all n ≥ 3, T(n) = T(n−2)+3n+4.
227. T(0) = c, T(1) = d, and for n > 1, T(n) = T(n −2) +n.
228. T(1) = 1, T(2) = 6, T(3) = 13, and for all n ≥ 4,
T(n) = T(n −3) + 5n −9.
229. T(1) = 1, and for all n ≥ 2, T(n) = 2T(n −1) +n
2
−2n + 1.
230. T(1) = 1, and for all n ≥ 2, T(n) = n T(n −1) +n.
Sec. 4.3. General Formulae 39
4.3 GENERAL FORMULAE
It is normal to teach the general solution to recurrences of the form T(n) =
aT(n/c) + bn (see, for example, Aho, Hopcroft, and Ullman [1, Theorem 1.1]).
The following are interesting variants of this recurrence.
231. State and prove a general formula for recurrences of the form
T(n) =
d if n ≤ 1
aT(n/c) +b otherwise.
232. State and prove a general formula for recurrences of the form
T(n) =
d if n ≤ 1
aT(n/c) +bn
2
otherwise.
233. State and prove a general formula for recurrences of the form
T(n) =
d if n ≤ 1
aT(n/c) +bn
k
otherwise.
234. Prove that the asymptotic solution of the recurrence relation
T(n) =
0 if 0 ≤ n < c
2T(n −c) +k otherwise.
where c, k ∈ IN, is T(n) = Θ(d
n
) for some constant d.
4.4 RECURRENCES WITH FULL HISTORY
Solve the following recurrences exactly.
235. T(1) = 1, and for all n ≥ 2,
T(n) =
n−1
¸
i=1
T(i) + 1.
236. T(1) = 1, and for all n ≥ 2,
T(n) =
n−1
¸
i=1
T(i) + 7.
40 Chap. 4. Recurrence Relations
237. T(1) = 1, and for all n ≥ 2,
T(n) =
n−1
¸
i=1
T(i) +n
2
.
238. T(1) = 1, and for all n ≥ 2,
T(n) = 2
n−1
¸
i=1
T(i) + 1.
239. T(1) = 1, and for all n ≥ 2,
T(n) =
n−1
¸
i=1
T(n −i) + 1.
240. T(1) = 1, and for all n ≥ 2,
T(n) =
n−1
¸
i=1
(T(i) +T(n −i)) + 1.
4.5 RECURRENCES WITH FLOORS AND CEILINGS
241. Suppose T(1) = 1, and for all n ≥ 2, T(n) = T(n/2) +n−1. Show
that 2n −log n −1 ≤ T(n) ≤ 2n −log n/2 −1.
242. Suppose T(1) = 1, and for all n ≥ 2, T(n) = T(n/2) +
T(n/2) +n −1. Find an exact solution for T(n).
243. Solve the following recurrence relation exactly. T(1) = 1 and for
all n ≥ 2, T(n) = T(n/2) + 1.
244. Solve the following recurrence relation exactly. T(1) = 1 and for all
n ≥ 2, T(n) = T(n/2) + 1.
245. Solve the following recurrence relation exactly. T(1) = 1, and for
n ≥ 2, T(n) = 2T(n/2) + 6n −1.
246. Solve the following recurrence relation exactly. T(1) = 2, and for all
n ≥ 2, T(n) = 4T(n/3) + 3n −5.
247. Solve the following recurrence: T(1) = 1, and for all n ≥ 2, T(n) =
T(
√
n) + 1.
248. Solve the following recurrence: T(1) = T(2) = 1, and for all
n ≥ 3, T(n) = T(n/ log n) + 1.
Sec. 4.6. Hints 41
4.6 HINTS
202. Try repeated substitution (see the comment to Problem 202 for a deﬁnition
of this term).
224. Use induction on n. The base case will be n ≤ 4.
226. If you think about it, this is really two independent recurrence relations, one
for odd n and one for even n. Therefore, you will need to make a special case
for even n and one for odd n. Once you realize this, the recurrences in this
group of problems are fairly easy.
242. The solution is T(n) = nlog n−2
log n
+1. This can be proved by induction.
Can you derive this answer from ﬁrst principles?
243. This problem is diﬃcult to solve directly, but it becomes easy when you
use the following fact from Graham, Knuth, and and Patashnik [30, Sec
tion 6.6]. Suppose f : IR→IR is continuous and monotonically increasing, and
has the property that if f(x) ∈ ZZ, then x ∈ ZZ. Then, f(x) = f(x) and
f(x) = f(x).
248. The answer is that for n ≥ 3, T(n) ≤ 1.76 log n/ log log n, but this may not
help you much.
4.7 SOLUTIONS
202. Suppose T(1) = 1, and for all n ≥ 2, T(n) = 3T(n − 1) + 2. If n is large
enough, then by repeated substitution,
T(n) = 3T(n −1) + 2 (after one substitution)
= 3(3T(n −2) + 2) + 2
= 9T(n −2) + 2 3 + 2 (after two substitutions)
= 9(3T(n −3) + 2) + 2 3 + 2
= 27T(n −3) + 2 9 + 2 3 + 2 (after three substitutions).
It looks like there’s a pattern developing. After i substitutions,
T(n) = 3
i
T(n −i) + 2
i−1
¸
j=0
3
j
. (4.1)
42 Chap. 4. Recurrence Relations
We can prove that identity (4.1) is true by induction on i. It is trivially true
for i = 1. Now suppose that i > 1 and that
T(n) = 3
i−1
T(n −i + 1) + 2
i−2
¸
j=0
3
j
.
Then,
T(n) = 3
i−1
T(n −i + 1) + 2
i−2
¸
j=0
3
j
= 3
i−1
(3T(n −i) + 2) + 2
i−2
¸
j=0
3
j
= 3
i
T(n −i) + 2
i−1
¸
j=0
3
j
,
as required. Now that we’ve established identity (4.1), we can continue with
solving the recurrence. Suppose we take i = n −1. Then, by identity (4.1),
T(n) = 3
n−1
T(1) + 2
n−2
¸
j=0
3
j
= 3
n−1
+ 3
n−1
−1 (by Problem 9)
= 2 3
n−1
−1.
208. Suppose T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 2T(n/2) + 6n −1.
If n is large enough, then by repeated substitution,
T(n) = 2T(n/2) + 6n −1 (after one substitution)
= 2(2T(n/4) + 6n/2 −1) + 6n −1
= 4T(n/4) + (6n −2) + (6n −1) (after two substitutions)
= 4(2T(n/8) + 6n/4 −1) + (6n −2) + (6n −1)
= 8T(n/8) + (6n −4) + (6n −2) + (6n −1)
(after three substitutions).
Therefore, after i substitutions,
T(n) = 2
i
T(n/2
i
) + 6in −
i−1
¸
j=0
2
j
.
Sec. 4.7. Solutions 43
This can be veriﬁed easily by induction. Hence, taking i = log n,
T(n) = nT(1) + 6nlog n −
log n−1
¸
j=0
2
j
= n + 6nlog n −(2
log n
−1) (by Problem 11)
= 6nlog n + 1.
214. Suppose T(1) = 1, and for all n ≥ 2 a power of 2, T(n) = 3T(n/2) +n
2
−n.
If n is large enough, then by repeated substitution,
T(n) = 3T(n/2) +n
2
−n (after one substitution)
= 3(3T(n/4) +n
2
/4 −n/2) +n
2
−n
= 9T(n/4) + 3(n
2
/4 −n/2) + (n
2
−n) (after two substitutions)
= 9(3T(n/8) +n
2
/16 −n/4) + 3(n
2
/4 −n/2) + (n
2
−n)
= 27T(n/8) + 9(n
2
/16 −n/4) + 3(n
2
/4 −n/2) + (n
2
−n)
(after three substitutions).
Therefore, after i substitutions,
T(n) = 3
i
T(n/2
i
) +n
2
i−1
¸
j=0
(3/4)
j
−n
i−1
¸
j=0
(3/2)
j
= 3
i
T(n/2
i
) −4n
2
(3/4)
i
+ 4n
2
−2n(3/2)
i
+ 2n (by Problem 11).
This can be veriﬁed easily by induction. Hence, taking i = log n,
T(n) = 3
log n
−4n
2
(3/4)
log n
+ 4n
2
−2n(3/2)
log n
+ 2n
= n
log 3
−4n
log 3
+ 4n
2
−2n
log 3
+ 2n
= 4n
2
−5n
log 3
+ 2n.
226. Suppose T(1) = 1, T(2) = 6, and for all n ≥ 3, T(n) = T(n −2) + 3n + 4. If
n is large enough, then by repeated substitution,
T(n) = T(n −2) + 3n + 4
= T(n −4) + (3(n −2) + 4) + (3n + 4)
= T(n −6) + (3(n −4) + 4) + (3(n −2) + 4) + (3n + 4).
Therefore, after i substitutions,
T(n) = T(n −2i) + 3
i−1
¸
j=0
(n −2j) + 4i
44 Chap. 4. Recurrence Relations
= T(n −2i) + 3in −6
i−1
¸
j=0
j + 4i
= T(n −2i) + 3in −3i(i −1) + 4i (by Problem 1)
= T(n −2i) + 3in −3i
2
+ 7i.
This can be veriﬁed easily by induction. Now, suppose that n is even. Take
i = n/2 −1. Then,
T(n) = T(n −2i) + 3in −3i
2
+ 7i
= T(2) + 3(n/2 −1)n −3(n/2 −1)
2
+ 7(n/2 −1)
= 6 + 3n
2
/2 −3n −3n
2
/4 + 3n −3 + 7n/2 −7
= 3n
2
/4 + 7n/2 −4.
Now, suppose that n is odd. Take i = (n −1)/2. Then,
T(n) = T(n −2i) + 3
i−1
¸
j=0
(n −2j) + 4i
= T(1) + 3n(n −1)/2 −3((n −1)/2)
2
+ 7(n −1)/2
= 1 + 3n
2
/2 −3n/2 −3n
2
/4 + 3n/2 −3/4 + 7n/2 −7/2
= 3n
2
/4 + 7n/2 −13/4.
Therefore, when n is even, T(n) = (3n
2
+ 14n − 16)/4, and when n is odd,
T(n) = (3n
2
+ 14n − 13)/4. Or, more succinctly, T(n) = (3n
2
+ 14n − 16 +
3(n mod 2))/4.
235. Suppose that T(1) = 1, and for all n ≥ 2, T(n) =
¸
n−1
i=1
T(i) + 1.
T(n) −T(n −1) = (
n−1
¸
i=1
T(i) + 1) −(
n−2
¸
i=1
T(i) + 1)
= T(n −1).
Therefore, T(n) = 2T(n − 1). Thus, we have reduced a recurrence with full
history to a simple recurrence that can be solved by repeated substitution
(I’ll leave that to you) to give the solution T(n) = 2
n−1
.
4.8 COMMENTS
202. The technique used to solve this problem in the previous section is called
repeated substitution. It works as follows. Repeatedly substitute until you
see a pattern developing. Write down a formula for T(n) in terms of n and
the number of substitutions (which you can call i). To be completely formal,
Sec. 4.8. Comments 45
verify this pattern by induction on i. (You should do this if you are asked to
prove that your answer is correct or if you want to be sure that the pattern
is right, but you can skip it is you are short of time.) Then choose i to be
whatever value it takes to make all of the T()s in the pattern turn into the
base case for the recurrence. You are usually left with some algebra to do,
such as a summation or two. Once you have used repeated substitution to
get an answer, it is prudent to check your answer by using induction. (It
is not normally necessary to hand in this extra work when you are asked to
solve the recurrence on a homework or an exam, but it’s a good idea to do it
for your own peace of mind.) Let’s do it for this problem. We are required
to prove that the solution to the recurrence T(1) = 1, and for all n ≥ 2,
T(n) = 3T(n − 1) + 2, is T(n) = 2 3
n−1
− 1. The proof is by induction
on n. The claim is certainly true for n = 1. Now suppose that n > 1 and
T(n −1) = 2 3
n−2
−1. Then,
T(n) = 3T(n −1) + 2
= 3(2 3
n−2
−1) + 2 (by the induction hypothesis)
= 2 3
n−1
−1,
as required.
222. Hence, F
n
= O(1.62
n
). For more information on the constant multiple in the
bigO bound, see Graham, Knuth, and Patashnik [30].
Chapter 5
Correctness Proofs
How do we know that a given algorithm works? The best method is to prove it
correct. Purely recursive algorithms can be proved correct directly by induction.
Purely iterative algorithms can be proved correct by devising a loop invariant for
every loop, proving them correct by induction, and using them to establish that the
algorithm terminates with the correct result. Here are some simple algorithms to
practice on. The algorithms use a simple pseudocode as described in Section 1.3.
5.1 ITERATIVE ALGORITHMS
To prove correctness of an iterative algorithm, you need to do the following:
• Write down a speciﬁcation for the output to be produced by the algorithm as
a function of its inputs (this will be easy for the simple algorithms considered
in this chapter).
• Verify the algorithm one loop at a time, starting at the inner loop in case of
nested loops.
• For each loop, devise a loop invariant that remains true each time through
the loop and captures the “progress” made by the loop.
• Prove that the loop invariants hold. This is usually done by induction on the
number of iterations. Start by listing the new values of the variables in terms
of the old values. Use this for your inductive step.
• Use the loop invariants to prove that the algorithm terminates.
• Use the loop invariants and the termination conditions to prove that the
algorithm computes the correct result (according to the speciﬁcation).
The loop invariants discussed in this section use the following convention: If
v is a variable used in a loop, then v
j
denotes the value stored in v immediately
after the jth iteration of the loop, for j ≥ 0. The value v
0
means the contents of v
immediately before the loop is entered.
46
Sec. 5.1. Iterative Algorithms 47
5.1.1 StraightLine Programs
249. Prove that the following algorithm for swapping two numbers is correct.
procedure swap(x, y)
comment Swap x and y.
1. x := x +y
2. y := x −y
3. x := x −y
5.1.2 Arithmetic
250. Prove that the following algorithm for the addition of natural
numbers is correct.
function add(y, z)
comment Return y +z, where y, z ∈ IN
1. x := 0; c := 0; d := 1;
2. while (y > 0) ∨ (z > 0) ∨ (c > 0) do
3. a := y mod 2;
b := z mod 2;
4. if a ⊕b ⊕c then x := x +d;
5. c := (a ∧ b) ∨ (b ∧ c) ∨ (a ∧ c);
6. d := 2d; y := y/2;
z := z/2;
7. return(x)
251. Prove that the following algorithm for incrementing a natural number is
correct.
function increment(y)
comment Return y + 1, where y ∈ IN
1. x := 0; c := 1; d := 1;
2. while (y > 0) ∨ (c > 0) do
3. a := y mod 2;
4. if a ⊕c then x := x +d;
5. c := a ∧ c;
6. d := 2d; y := y/2;
7. return(x)
252. Prove that the following algorithm for the multiplication of natural
numbers is correct.
48 Chap. 5. Correctness Proofs
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. if z mod 2 = 1 then x := x +y;
4. y := 2y; z := z/2;
5. return(x)
253. Prove that the following algorithm for the multiplication of
natural numbers is correct.
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. x := x +y (z mod 2);
4. y := 2y; z := z/2;
5. return(x)
254. Prove that the following algorithm for the multiplication of natural
numbers is correct, for all integer constants c ≥ 2.
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. x := x +y (z mod c);
4. y := c y; z := z/c;
5. return(x)
255. Prove that the following algorithm for division and remaindering of
natural numbers is correct.
function divide(y, z)
comment Return q, r ∈ IN such that y = qz +r
and r < z, where y, z ∈ IN
1. r := y; q := 0; w := z;
2. while w ≤ y do w := 2w;
3. while w > z do
4. q := 2q; w := w/2;
5. if w ≤ r then
6. r := r −w; q := q + 1
7. return(q, r)
Sec. 5.1. Iterative Algorithms 49
256. Prove that the following algorithm for exponentiation is correct.
function power(y, z)
comment Return y
z
, where y ∈ IR, z ∈ IN
1. x := 1;
2. while z > 0 do
3. x := x y
4. z := z −1
5. return(x)
257. Prove that the following algorithm for exponentiation is correct.
function power(y, z)
comment Return y
z
, where y ∈ IR, z ∈ IN
1. x := 1;
2. while z > 0 do
3. if z is odd then x := x y
4. z := z/2
5. y := y
2
6. return(x)
258. Prove that the following algorithm for computing factorials is correct.
function factorial(y)
comment Return y!, where y ∈ IN
1. x := 1
2. while y > 1 do
3. x := x y; y := y −1
4. return(x)
5.1.3 Arrays
259. Prove that the following algorithm that adds the values in an array
A[1..n] is correct.
function sum(A)
comment Return
¸
n
i=1
A[i]
1. s := 0;
2. for i := 1 to n do
3. s := s +A[i]
4. return(s)
50 Chap. 5. Correctness Proofs
260. Prove that the following algorithm for computing the maximum value
in an array A[1..n] is correct.
function max(A)
comment Return max A[1], . . . , A[n]
1. m := A[1]
2. for i := 2 to n do
3. if A[i] > m then m := A[i]
4. return(m)
261. Prove the correctness of the following bubblesort algorithm. The
values to be sorted are in an array A[1..n].
procedure bubblesort(A[1..n])
comment Sort A[1], A[2], . . . , A[n] into nondecreasing order
1. for i := 1 to n −1 do
2. for j := 1 to n −i do
3. if A[j] > A[j + 1] then
4. Swap A[j] with A[j + 1]
262. Prove the correctness of the following patternmatching algorithm.
The input consists of a string S[1..n], and a pattern P[0..m− 1], where 1 ≤
m ≤ n. The algorithm locates the ﬁrst contiguous occurrence of the pattern
P in the string S, that is, = p if S[p..p +m−1] = P, and = n −m+ 1 if
the pattern P does not occur at all in the string S.
function match(P, S, n, m)
comment Find the pattern P[0..m−1] in string S[1..n]
1. := 0; matched := false
2. while ( ≤ n −m) ∧ matched do
3. := + 1 ;
4. r := 0; matched := true
5. while (r < m) ∧ matched do
6. matched := matched ∧ (P[r] = S[ +r])
7. r := r + 1
8. return()
263. Prove that the following matrix multiplication algorithm is correct.
Sec. 5.2. Recursive Algorithms 51
procedure matmultiply(Y, Z, n);
comment multiplies n n matrices Y Z
1. for i := 1 to n do
2. for j := 1 to n do
3. X[i, j] := 0;
4. for k := 1 to n do
5. X[i, j] := X[i, j] +Y [i, k] Z[k, j];
6. return(X)
264. Prove that the following algorithm for evaluating the polynomial
a
n
x
n
+a
n−1
x
n−1
+ +a
1
x +a
0
is correct, where the coeﬃcients are stored
in an array A[0..n], with A[i] = a
i
for all 0 ≤ i ≤ n. The algorithm is named
after its inventor, William G. Horner, and is often called Horner’s rule.
function Horner(A, n)
comment Return
¸
n
i=0
A[i] x
i
1. v := 0
2. for i := n downto 0 do
3. v := A[i] +v x
4. return(v)
5.1.4 Fibonacci Numbers
265. Prove that the following algorithm for computing Fibonacci
numbers is correct.
function ﬁb(n)
1. comment Return F
n
, the nth Fibonacci number
2. if n = 0 then return(0) else
3. last:=0; current:=1
4. for i := 2 to n do
5. temp:=last+current; last:=current; current:=temp
6. return(current)
5.2 RECURSIVE ALGORITHMS
To prove correctness of a recursive algorithm, you need to do the following:
• You will prove correctness by induction on the “size” of the problem being
solved (for example, the size of array chunk, number of bits in an integer,
etc.). Your ﬁrst task is to identify what is to be used as the “size.”
52 Chap. 5. Correctness Proofs
• Then, you prove the base of the induction, which will usually involve only the
base of the recursion.
• Next, you need to prove that recursive calls are given subproblems to solve
(that is, there is no inﬁnite recursion). This is often trivial, but it can be
diﬃcult to prove and so should not be overlooked.
• Finally, you prove the inductive step: Assume that the recursive calls work
correctly, and use this assumption to prove that the current call works cor
rectly.
5.2.1 Arithmetic
266. Prove that the following recursive algorithm computes 3
n
− 2
n
for all
n ≥ 0.
function g(n)
1. if n ≤ 1 then return(n)
2. else return(5 g(n −1) −6 g(n −2))
267. Prove that the following recursive algorithm for incrementing a natural
number is correct.
function increment(y)
comment Return y + 1.
1. if y = 0 then return(1) else
2. if y mod 2 = 1 then
3. return(2 increment(y/2))
4. else return(y + 1)
268. Prove that the following recursive algorithm for the addition of natural
numbers is correct.
function add(y, z, c)
comment Return the sum y +z +c, where c ∈ ¦0, 1¦.
1. if y = z = 0 then return(c) else
2. a := y mod 2; b := z mod 2;
3. return(2 add(y/2, z/2, (a +b +c)/2) + (a ⊕b ⊕c))
269. Prove that the following recursive algorithm for the multiplication
of natural numbers is correct.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. if z is odd then return(multiply(2y, z/2)+y)
3. else return(multiply(2y, z/2))
Sec. 5.2. Recursive Algorithms 53
270. Prove that the following recursive algorithm for the multiplication of
natural numbers is correct.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. return(multiply(2y, z/2) +y (z mod 2))
271. Prove that the following recursive algorithm for the multiplication of
natural numbers is correct, for all integers constants c ≥ 2.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. return(multiply(cy, z/c) +y (z mod c))
272. Prove that the following recursive algorithm for exponentiation is correct.
function power(y, z)
comment Return y
z
, where y ∈ IR, z ∈ IN.
1. if z = 0 then return(1)
2. if z is odd then return(power(y
2
, z/2) y)
3. else return(power(y
2
, z/2))
273. Prove that the following recursive algorithm for computing factorials is
correct.
function factorial(n)
comment Return n!.
1. if n ≤ 1 then return(1)
2. else return(n factorial(n −1))
5.2.2 Arrays
274. Prove that the following recursive algorithm for computing the maximum
value in an array A[1..n] is correct.
function maximum(n)
comment Return max of A[1..n].
1. if n ≤ 1 then return(A[1])
2. else return(max(maximum(n −1),A[n]))
function max(x, y)
comment Return largest of x and y.
3. if x ≥ y then return(x) else return(y)
54 Chap. 5. Correctness Proofs
275. Prove that the following recursive algorithm that adds the values in an
array A[1..n] is correct.
function sum(n)
comment Return sum of A[1..n].
1. if n ≤ 1 then return(A[1])
2. else return(sum(n −1) +A[n])
5.2.3 Fibonacci Numbers
276. Prove that the following algorithm for computing Fibonacci num
bers is correct.
function ﬁb(n)
comment Return F
n
, the nth Fibonacci number.
1. if n ≤ 1 then return(n)
2. else return(ﬁb(n −1)+ﬁb(n −2))
277. Prove that the following algorithm for computing Fibonacci
numbers is correct.
function ﬁb(n)
comment Return (F
n−1
, F
n
)
1. if n is odd then
2. (a, b) := even(n −1)
3. return(b, a +b)
4. else return(even(n))
function even(n)
comment Return (F
n−1
, F
n
) when n is even
1. if n = 0 then return(1, 0)
2. else if n = 2 then return(1, 1)
3. else if n = 4 then return(2, 3)
4. (a, b) := ﬁb(n/2 −1)
5. c := a +b; d := b +c
6. return(b d +a c, c (d +b))
5.3 COMBINED ITERATION AND RECURSION
The following questions ask you to prove correct some recursive algorithms that
have loops in them. Naturally enough, you can solve them by applying both of the
techniques you have used in this chapter.
Sec. 5.4. Hints 55
278. Prove that the following recursive bubblesort algorithm is correct. The
values to be sorted are in an array A[1..n].
procedure bubblesort(n)
comment Sort A[1..n].
1. if n > 1 then
2. for j := 1 to n −1 do
3. if A[j] > A[j + 1] then
4. Swap A[j] with A[j + 1]
5. bubblesort(n −1)
279. Prove that the following variant of quicksort is correct. The values
to be sorted are in an array A[1..n].
1. procedure quicksort(, r)
2. comment sort S[..r]
3. i := ; j := r
a := some element from S[..r]
4. repeat
5. while S[i] < a do i := i + 1
6. while S[j] > a do j := j −1
7. if i ≤ j then
8. swap S[i] and S[j]
9. i := i + 1; j := j −1
10. until i > j
11. if < j then quicksort(, j)
12. if i < r then quicksort(i, r)
5.4 HINTS
In each of the following hints, if the algorithm uses a variable v, then v
i
denotes the
contents of v immediately after the ith iteration of the single loop in the algorithm
(v
0
denotes the contents of v immediately before the loop is entered for the ﬁrst
time).
250. The loop invariant is the following: (y
j
+z
j
+c
j
)d
j
+x
j
= y
0
+z
0
.
252. The loop invariant is the following: y
j
z
j
+x
j
= y
0
z
0
.
255. The loop invariant is the following: q
j
w
j
+r
j
= y
0
and r
j
< w
j
.
256. The loop invariant is the following: x
j
y
j
z
j
= y
z
0
0
.
56 Chap. 5. Correctness Proofs
257. The loop invariant is the following: x
j
y
j
z
j
= y
z
0
0
.
258. The loop invariant is the following: m
j
=
¸
n
k=n−j+1
k.
259. The loop invariant is the following: s
j
=
¸
j
i=1
A[i].
260. The loop invariant is the following: m
j
is the maximum of A[1], . . . , A[j +1].
261. The loop invariant for the inner loop is the following: after the jth iteration,
for all 1 ≤ i < j, A[i] ≤ A[j]. The loop invariant for the outer loop is the
following: after the jth iteration, for all n − j + 1 ≤ i ≤ n, for all k < i,
A[k] ≤ A[i].
277. The identity from Problem 53 might help.
5.5 SOLUTIONS
250. This correctness proof will make use of the fact that for all n ∈ IN,
2n/2 + (n mod 2) = n. (5.1)
(Can you prove this?) We claim that if y, z ∈ IN, then add(y, z) returns the
value y + z. That is, when line 8 is executed, x = y + z. For each of the
identiﬁers, use a subscript i to denote the value of the identiﬁer after the ith
iteration of the whileloop on lines 3–7, for i ≥ 0, with i = 0 meaning the
time immediately before the while loop is entered and immediately after the
statement on line 2 is executed. By inspection,
a
j+1
= y
j
mod 2
b
j+1
= z
j
mod 2
y
j+1
= y
j
/2
z
j+1
= z
j
/2
d
j+1
= 2d
j
.
From line 6 of the algorithm, c
j+1
is 1 iﬀ at least two of a
j+1
, b
j+1
, and c
j
are 1 (see Problem 93). Therefore,
c
j+1
= (a
j+1
+b
j+1
+c
j
)/2.
Note that d is added into x in line 5 of the algorithm only when an odd
number of a, b, and c are 1. Therefore,
x
j+1
= x
j
+d
j
((a
j+1
+b
j+1
+c
j
) mod 2).
We will now prove the following loop invariant: For all natural numbers j ≥ 0,
(y
j
+z
j
+c
j
)d
j
+x
j
= y
0
+z
0
.
Sec. 5.5. Solutions 57
The proof is by induction on j. The base j = 0 is trivial, since c
0
= 0, d
0
= 1,
and x
0
= 0. Now, suppose that
(y
j
+z
j
+c
j
)d
j
+x
j
= y
0
+z
0
.
We are required to prove that
(y
j+1
+z
j+1
+c
j+1
)d
j+1
+x
j+1
= y
0
+z
0
.
By the preceding,
(y
j+1
+z
j+1
+c
j+1
)d
j+1
+x
j+1
= (y
j
/2 +z
j
/2 +(y
j
mod 2 +z
j
mod 2 +c
j
)/2)2d
j
+x
j
+d
j
((y
j
mod 2 +z
j
mod 2 +c
j
) mod 2)
= (y
j
/2 +z
j
/2)2d
j
+x
j
+d
j
(y
j
mod 2 +z
j
mod 2 +c
j
)
(by Equation 5.1)
= (y
j
+z
j
+c
j
)d
j
+x
j
(by Equation 5.1 twice).
Therefore, by the induction hypothesis,
(y
j+1
+z
j+1
+c
j+1
)d
j+1
+x
j+1
= y
0
+z
0
.
This ends the proof of the loop invariant. It remains to use it to prove that
the algorithm is correct. We need to prove that the algorithm terminates
with x containing the sum of y and z. First, we will prove that it terminates.
By inspection, the values of y and z are halved (rounding down if they are
odd) on every iteration of the loop. Therefore, they will eventually both have
value zero, and stay that way. At the ﬁrst point at which y = z = 0, either
c will equal zero or c will be assigned zero on the next iteration of the loop.
Therefore, eventually y = z = c = 0, at which point the loop terminates.
Now we will prove that x has the correct value on termination. Suppose the
loop terminates after t iterations, for some t ≥ 0. By the loop invariant,
(y
t
+z
t
+c
t
)d
t
+x
t
= y
0
+z
0
.
Since y
t
= z
t
= c
t
= 0, we see that x
t
= y
0
+ z
0
. Therefore, the algorithm
terminates with x containing the sum of the initial values of y and z, as
required.
269. The correctness proof is by induction on z. We claim that multiply(y, z)
returns the product yz. The hypothesis is true for z = 0, since multiply(y, 0)
returns 0. Now, suppose that for z ≥ 0, multiply(y, z) returns yz. We now
must prove that multiply(y, z + 1) returns y(z + 1). There are two cases to
be considered, depending on whether z + 1 is odd or even. By inspection, if
58 Chap. 5. Correctness Proofs
z + 1 is odd, then multiply(y, z + 1) returns
multiply(2y, (z + 1)/2) +y
= 2y(z + 1)/2 +y (by the induction hypothesis)
= 2y(z/2) +y (since z is even)
= y(z + 1).
By inspection, if z + 1 is even, then multiply(y, z + 1) returns
multiply(2y, (z + 1)/2)
= 2y(z + 1)/2 (by the induction hypothesis)
= 2y(z + 1)/2 (since z is odd)
= y(z + 1).
5.6 COMMENTS
253. The hard part about this problem (and other similar problems in this section)
is that you will need to ﬁnd your own loop invariant.
265. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is the obvious
iterative algorithm.
276. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is the obvious
recursive algorithm.
277. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is a sneaky
recursive algorithm.
Chapter 6
Algorithm Analysis
Algorithm analysis usually means “give a bigO ﬁgure for the running time of an
algorithm.” (Of course, a bigΘ bound would be even better.) This can be done by
getting a bigO ﬁgure for parts of the algorithm and then combining these ﬁgures
using the sum and product rules for bigO (see Problems 173 and 176). As we will
see, recursive algorithms are more diﬃcult to analyze than nonrecursive ones.
Another useful technique is to pick an elementary operation, such as additions,
multiplications, or comparisons, and observe that the running time of the algorithm
is bigO of the number of elementary operations. (To ﬁnd the elementary operation,
just ignore the “book keeping” code and look for the point in the algorithm where
the real work is done.) Then, you can analyze the exact number of elementary
operations as a function of n in the worst case. This is often easier to deal with
because it is an exact function of n and you don’t have the messy bigO symbols to
carry through your analysis.
6.1 ITERATIVE ALGORITHMS
The analysis of iterative algorithms is fairly easy. Simply charge O(1) for code
without loops (assuming that your pseudocode isn’t hiding something that takes
longer), and use the sum and product rules for bigO (see Problems 173 and 176).
If you get tired of carrying around bigOs, use the “elementary operation” technique
described earlier.
6.1.1 What is Returned?
280. What is the value returned by the following function? Express your
answer as a function of n. Give, using bigO notation, the worstcase running
time.
59
60 Chap. 6. Algorithm Analysis
function mystery(n)
1. r := 0;
2. for i := 1 to n −1 do
3. for j := i + 1 to n do
4. for k := 1 to j do
5. r := r + 1
6. return(r)
281. What is the value returned by the following function? Express your
answer as a function of n. Give, using bigO notation, the worstcase running
time.
function pesky(n)
1. r := 0;
2. for i := 1 to n do
3. for j := 1 to i do
4. for k := j to i +j do
5. r := r + 1
6. return(r)
282. What is the value returned by the following function? Express your
answer as a function of n. Give, using bigO notation, the worstcase running
time.
function pestiferous(n)
1. r := 0;
2. for i := 1 to n do
3. for j := 1 to i do
4. for k := j to i +j do
5. for := 1 to i +j −k do
6. r := r + 1
7. return(r)
283. What is the value returned by the following function? Express
your answer as a function of n. Give, using bigO notation, the worstcase
running time.
function conundrum(n)
1. r := 0;
2. for i := 1 to n do
3. for j := i + 1 to n do
4. for k := i +j −1 to n do
5. r := r + 1
6. return(r)
Sec. 6.1. Iterative Algorithms 61
6.1.2 Arithmetic
284. Analyze the algorithm for the addition of natural numbers in Prob
lem 250. How many additions does it use (that is, how many times is line 4
executed) in the worst case?
285. Analyze the algorithm for incrementing a natural number in Problem 251.
How many additions does it use (that is, how many times is line 4 executed)
in the worst case?
286. Analyze the algorithm for the multiplication of natural numbers in Prob
lem 252. How many additions does it use (that is, how many times is line 3
executed) in the worst case?
287. Analyze the algorithm for the multiplication of natural numbers in Prob
lem 253. How many additions does it use (that is, how many times is line 3
executed) in the worst case?
288. Analyze the algorithm for the multiplication of natural numbers in Prob
lem 254. How many additions does it use (that is, how many times is line 3
executed) in the worst case? You may assume that c ≥ 2 is an integer.
289. Analyze the algorithm for division and remaindering of natural numbers
in Problem 255. How many subtractions does it use (that is, how many times
is line 6 executed) in the worst case?
290. Analyze the algorithm for exponentiation in Problem 256. How many
multiplications does it use (that is, how many times is line 3 executed) in the
worst case?
291. Analyze the algorithm for exponentiation in Problem 257. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
292. Analyze the algorithm for computing factorials in Problem 258. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
6.1.3 Arrays
293. Analyze the array addition algorithm in Problem 259. How many ad
ditions does it use (that is, how many times is line 3 executed) in the worst
case?
294. Analyze the array maximum algorithm in Problem 260. How many
comparisons does it use (that is, how many times is line 3 executed) in the
worst case?
62 Chap. 6. Algorithm Analysis
295. Analyze the bubblesort algorithm in Problem 261. How many compar
isons does it use (that is, how many times is line 3 executed) in the worst
case?
296. Analyze the patternmatching algorithm in Problem 262. How many
accesses of the array P does it use (that is, how many times is line 6 executed)
in the worst case?
297. Analyze the matrix multiplication algorithm in Problem 263. How many
multiplications does it use (that is, how many times is line 5 executed) in the
worst case?
298. Analyze the algorithm for evaluating a polynomial in Problem 264. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
6.1.4 Fibonacci Numbers
299. Analyze the algorithm for computing Fibonacci numbers in Problem 265.
How many additions does it use (that is, how many times is line 3 executed)
in the worst case?
6.2 RECURSIVE ALGORITHMS
The analysis of recursive algorithms is a little harder than that of nonrecursive ones.
First, you have to derive a recurrence relation for the running time, and then you
have to solve it. Techniques for the solution of recurrence relations are explained in
Chapter 4.
To derive a recurrence relation for the running time of an algorithm:
• Decide what “n”, the problem size, is.
• See what value of n is used as the base of the recursion. It will usually be a
single value (for example, n = 1), but may be multiple values. Suppose it is
n
0
.
• Figure out what T(n
0
) is. You can usually use “some constant c,” but some
times a speciﬁc number will be needed.
• The general T(n) is usually a sum of various choices of T(m) (for the recursive
calls), plus the sum of the other work done. Usually, the recursive calls will
be solving a subproblems of the same size f(n), giving a term “a T(f(n))”
in the recurrence relation.
The “elementary operation” technique described in the ﬁrst paragraph of this
chapter can be used to good eﬀect here, too.
Sec. 6.2. Recursive Algorithms 63
6.2.1 Arithmetic
300. Analyze the recursive algorithm from Problem 266. How
many additions does it use (that is, how many times is line 2 executed) in the
worst case?
301. Analyze the recursive algorithm for incrementing a natural number in
Problem 267. How many rightshifts does it use (that is, how many times is
line 3 executed) in the worst case?
302. Analyze the recursive algorithm for the addition of natural numbers in
Problem 268. How many times is line 3 executed in the worst case?
303. Analyze the recursive algorithm for the multiplication of natural
numbers in Problem 269. How many additions does it use (that is, how many
times is line 2 executed) in the worst case?
304. Analyze the recursive algorithm for the multiplication of natural numbers
in Problem 270. How many additions does it use (that is, how many times is
line 2 executed) in the worst case?
305. Analyze the recursive algorithm for the multiplication of natural numbers
in Problem 271. How many additions does it use (that is, how many times is
line 2 executed) in the worst case? You may assume that c ≥ 2 is an integer.
306. Analyze the recursive algorithm for exponentiation in Problem 272. How
many multiplications does it use (that is, how many times is line 2 executed)
in the worst case?
307. Analyze the recursive algorithm for computing factorials in Problem 273.
How many multiplications does it use (that is, how many times is line 2
executed) in the worst case?
6.2.2 Arrays
308. Analyze the recursive algorithm for computing the maximum value in an
array in Problem 274. How many comparisons does it use (that is, how many
times is line 2 executed) in the worst case?
309. Analyze the recursive algorithm that adds the values in an array in
Problem 275. How many additions does it use (that is, how many times is
line 2 executed) in the worst case?
64 Chap. 6. Algorithm Analysis
6.2.3 Fibonacci Numbers
310. Analyze the algorithm for computing Fibonacci numbers in Problem 276.
How many additions does it use (that is, how many times is line 2 executed)
in the worst case?
311. Analyze the algorithm for computing Fibonacci numbers in Prob
lem 277. How many additions does it use (that is, how many times is line 3
executed) in the worst case?
6.3 COMBINED ITERATION AND RECURSION
The following questions ask you to analyze some recursive algorithms that have loops
in them. Naturally enough, you can do this by applying both of the techniques you
have used in this chapter.
312. Analyze the recursive version of bubblesort in Problem 278. How many
comparisons does it use (that is, how many times is line 3 executed) in the
worst case?
313. Analyze the variant of quicksort in Problem 279. How many com
parisons does it use (that is, how many times are lines 5 and 6 executed) in
the worst case?
6.4 HINTS
283 If you use the technique from the solution of Problem 280 blindly, you’ll get
the answer n(n−1)/2. This is wrong. More subtlety is necessary. The correct
answer is n(n+2)(2n−1)/24 if n is even, and something diﬀerent if n is odd.
(You didn’t think I would give you the whole answer, did you?) You can
partially verify your solution in a few minutes by coding the algorithm as a
program and printing n and r for a few values of n.
284. I am looking for a bigO analysis of the running time, and an exact function
of n (with no bigO at the front) for the number of additions. You can do the
running time ﬁrst, or if you are clever, you can observe that additions are the
“elementary operation,” count them exactly, and then put a bigO around
the function you get as answer to the second part of the question (reducing
it to the simplest form, of course) to give the answer for the ﬁrst part of the
question.
Sec. 6.5. Solutions 65
300. Fibonacci numbers are involved (see Section 2.7 for deﬁnitions). Speciﬁcally,
you will need to apply the results from Problems 52 and 222.
6.5 SOLUTIONS
280. The forloop on lines 4–5 has the same eﬀect as r := r + j. Therefore, the
forloop on lines 3–5 has the following eﬀect:
3–5. for j := i + 1 to n do
r := r +j,
equivalently, r := r +
¸
n
j=i+1
j. Now,
n
¸
j=i+1
j =
n
¸
j=1
j −
i
¸
j=1
j = n(n + 1)/2 −i(i + 1)/2.
Therefore, the forloop on lines 2–5 has the following eﬀect:
2–5. for i := 1 to n −1 do
r := r +n(n + 1)/2 −i(i + 1)/2,
equivalently, r := r +
¸
n−1
i=1
(n(n + 1)/2 −i(i + 1)/2). Now,
n−1
¸
i=1
(n(n + 1)/2 −i(i + 1)/2)
=
(n −1)n(n + 1)
2
−
1
2
n−1
¸
i=1
i
2
−
1
2
n−1
¸
i=1
i
= (n −1)n(n + 1)/2 −n(n −1)(2n −1)/12 −n(n −1)/4
(by Problems 1 and 2)
= n(n
2
−1)/3.
Therefore, function mystery returns n(n
2
− 1)/3. A sloppy analysis goes as
follows. The forloop that begins on line 2 is executed for O(n) iterations. The
forloop that begins on line 3 is executed for O(n) iterations. The forloop
that begins on line 4 is executed for O(n) iterations. Lines 1 and 5 take O(1)
time. Therefore, function mystery runs in time O(n
3
). This analysis is tight
because the preceding paragraph showed that line 5 is executed n(n
2
−1)/3
times.
300. Let A(n) be the number of additions used in function g(n). Then A(0) =
A(1) = 0 and for all n > 1, A(n) = A(n −1) +A(n −2) +1. This recurrence
relation is a little tricky to solve, but repeated substitution will do it:
A(n) = A(n −1) +A(n −2) + 1
= (A(n −2) +A(n −3) + 1) +A(n −2) + 1
66 Chap. 6. Algorithm Analysis
= 2A(n −2) +A(n −3) + 1 + 1
= 2(A(n −3) +A(n −4) + 1) +A(n −3) + 1 + 1
= 3A(n −3) + 2A(n −4) + 1 + 1 + 2
= 3(A(n −4) +A(n −5) + 1) + 2A(n −4) + 1 + 1 + 2
= 5A(n −4) + 3A(n −5) + 1 + 1 + 2 + 3.
There is a pattern involving Fibonacci numbers (see Section 2.7) developing
here. It can be veriﬁed by induction on i ≥ 1 that after i substitutions,
A(n) = F
i+1
A(n −i) +F
i
A(n −i −1) +
i
¸
j=1
F
j
.
Hence, taking i = n −1,
A(n) = F
n
A(1) +F
n−1
A(0) +
n−1
¸
j=1
F
j
=
n−1
¸
j=1
F
j
= F
n+1
−1 (see Problem 52).
The running time of procedure g(n) is clearly bigO of the number of addi
tions, and is hence O(F
n+1
) = O(1.62
n
) (for the latter, see the comment to
Problem 222).
303. Let n be the number of bits in z, and A(n) be the number of additions
used in the multiplication of y by z. Then A(0) = 0 and for all n ≥ 1,
A(n) = A(n −1) + 1. The solution to this recurrence is A(n) = n. (Can you
verify this by induction on n? If I didn’t tell you the answer, could you have
derived it for yourself?) The running time of procedure multiply is clearly
bigO of the number of additions, and is hence O(n).
6.6 COMMENTS
291. Which algorithm would you use in practice, the one in this problem or the
one from Problem 290?
311. Which algorithm would you use in practice, the one in this problem, the one
from Problem 299, or the one from Problem 310?
313. The worstcase analysis of quicksort is often presented in class, but if not,
then it makes a good exercise.
Chapter 7
DivideandConquer
Divideandconquer is perhaps the most commonly used algorithm design technique
in computer science. Faced with a big problem P, divide it into smaller problems,
solve these subproblems, and combine their solutions into a solution for P. But how
do you solve the smaller problems? Simply divide each of the small problems into
smaller problems, and keep doing this until the problems become so small that it is
trivial to solve them. Sound like recursion? Not surprisingly, a recursive procedure
is usually the easiest way of implementing divideandconquer.
7.1 MAXIMUM AND MINIMUM
The following is a divideandconquer algorithm for ﬁnding the maximum value in an
array S[1..n]. The main body of the algorithm consists of a call to maximum(1, n).
function maximum(x, y)
comment return maximum in S[x..y]
1. if y −x ≤ 1 then return(max(S[x], S[y]))
2. else
3. max1:=maximum(x, (x +y)/2)
4. max2:=maximum((x +y)/2 + 1, y)
5. return(max(max1,max2))
314. Prove that the algorithm is correct. You may assume that n is a
power of 2.
315. Write down a recurrence relation for the worstcase number of com
parisons used by maximum(1, n). Solve this recurrence relation. You may
assume that n is a power of 2.
316. What is the running time of maximum(1, n)? Explain your answer.
Most textbooks introduce divideandconquer using the MAXMIN problem, the
problem of ﬁnding the largest and smallest values in an array of n values. It is
67
68 Chap. 7. DivideandConquer
usually described for n a power of 2. The following problems ask you to extend this
to arbitrary n.
317. Design and analyze a divideandconquer MAXMIN algorithm
that uses 3n/2 −2 comparisons for any n ∈ IN.
318. Consider the following MAXMIN algorithm. How many comparisons
does it use? Is it likely to be faster or slower than the divideandconquer
algorithm in practice?
procedure maxmin2(S)
comment computes maximum and minimum of S[1..n]
in max and min resp.
1. if n is odd then max:=S[n]; min:=S[n]
2. else max:=−∞; min:=∞
3. for i := 1 to n/2 do
4. if S[2i −1] ≤ S[2i]
5. then small:=S[2i −1]; large:=S[2i]
6. else small:=S[2i]; large:=S[2i −1]
7. if small < min then min:=small
8. if large > max then min:=small
319. Show that any comparisonbased MAXMIN algorithm must use at
least 3n/2 −2 comparisons in the worst case, for all n ∈ IN.
7.2 INTEGER MULTIPLICATION
Another popular example of divideandconquer is the O(n
1.59
) time integer mul
tiplication algorithm. The following questions will test your understanding of this
algorithm and its analysis.
320. What is the depth of recursion of the divideandconquer integer multi
plication algorithm? At what depth does it start to repeat the same subprob
lems?
321. Suppose you are to write an algorithm for multiplying two nbit
integers, and you are told that you have a computer with word size
√
n, and
hence that you can multiply two
√
nbit integers in O(1) time. Show that, by
using the divideandconquer multiplication and stopping the recursion early,
you can multiply two nbit integers in time O(n
1.2925
).
322. Devise an algorithm to multiply an nbit integer by an mbit integer,
where n ≥ m, in time O(nm
log 3−1
).
Sec. 7.2. Integer Multiplication 69
Complete the design of the following algorithm for performing integer multiplication
in time O(n
1.63
). (This is slower than the standard algorithm, but its veriﬁcation
and analysis will test your abilities.) We use a technique similar to the standard
divideandconquer algorithm. Instead of dividing the inputs y and z into two parts,
we divide them into three parts. Suppose y and z have n bits, where n is a power
of 3. Break y into three parts, a, b, c, each with n/3 bits. Break z into three parts,
d, e, f, each with n/3 bits. Then,
yz = ad2
4n/3
+ (ae +bd)2
n
+ (af +cd +be)2
2n/3
+ (bf +ce)2
n/3
+cf.
This can be computed as follows:
r
1
:= ad
r
2
:= (a +b)(d +e)
r
3
:= be
r
4
:= (a +c)(d +f)
r
5
:= cf
r
6
:= (b +c)(e +f)
x := r
1
2
4n/3
+ (r
2
−r
1
−r
3
)2
n
+ (r
3
+r
4
−r
1
−r
5
)2
2n/3
+ (r
6
−r
3
−r
5
)2
n/3
+r
5
.
323. Show that x = yz.
324. Show that this algorithm runs in O(n
1.63
) bitoperations, as claimed.
Joe D. Student is confused. He was told to implement the divideandconquer
algorithm for integer multiplication, and ﬁnd the value of n for which the divide
andconquer algorithm is faster than the naive algorithm (see Problem 252) on
nbit integers. It must be faster for large enough n since, as he saw in class, the
naive algorithm takes time O(n
2
) and the divideandconquer algorithm takes time
O(n
1.585
). Here is the Pascal data structure that he used:
const Precision=5000;
type Bit=0..1;
longinteger=packed array[1..Precision] of Bit;
He started by writing procedures to do addition, leftshifting, and so on. For
example, here is his addition procedure:
procedure LongAdd(var result,summand:longinteger);
{result:=result+summand}
var index,digitsum:integer;
70 Chap. 7. DivideandConquer
carry:Bit;
begin{LongAdd}
carry:=0;
for index:=1 to Precision do
begin{for}
digitsum:=result[index]+summand[index]+carry;
carry:=digitsum div 2;
result[index]:=digitsum mod 2;
end;{for}
if carry>0 then LongError(LongOverflow)
end;{LongAdd}
Joe carefully implemented the naive multiplication algorithm from Problem 252
as procedure LongMultiply, and the divideandconquer multiplication algorithm as
procedure FastLongMultiply. Both of these procedures worked perfectly. However,
he found that LongMultiply was faster than FastLongMultiply, even when he tried
multiplying 2
n
−1 by itself (which is guaranteed to make LongMultiply slow). Joe’s
data show that as n increases, FastLongMultiply becomes slower than LongMultiply
by more than a constant factor. Figure 7.1 shows his running times, and Figure 7.2
their ratio.
325. Show that the worst case for the naive integer multiplication algorithm
is, as claimed above, multiplying 2
n
−1 by itself.
326. What did Joe do wrong?
327. What can Joe do to ﬁx his program so that FastLongMultiply is faster
than LongMultiply for large enough n?
Complete the design of the following algorithm for performing integer multiplica
tion using O(n
1.585
/(log n)
0.585
) bitoperations. First, construct a table of all kbit
products. To multiply two nbit numbers, do the following. Perform the divide
andconquer algorithm with the base of recursion n ≤ k. That is, if n ≤ k, then
simply look up the result in the table. Otherwise, cut the numbers in half and
perform the recursion as normal.
328. Show that a table of all kbit products can be constructed using O(k4
k
)
bitoperations.
329. Devise and solve a recurrence relation for the divideandconquer
multiplication algorithm when the base of recursion is T(n) = O(1) for all
n ≤ k. Your solution must be a function of both n and k.
330. Show that the best value of k is Θ(log n), and that this gives an
algorithm that uses O(n
1.585
/(log n)
0.585
) bitoperations, as claimed.
Sec. 7.2. Integer Multiplication 71
Comparison of Multiplication Algorithms
FastLongMultiply
LongMultiply
Time (ms) x 10
6
3
n x 10
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
1.30
1.40
1.50
1.60
1.70
1.80
1.90
2.00
0.00 0.50 1.00 1.50 2.00 2.50
Figure 7.1. Running times for procedures LongMultiply
and FastLongMultiply observed by Joe D. Student.
72 Chap. 7. DivideandConquer
Ratio of Running Times
Ratio
3
n x 10
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
45.00
50.00
55.00
60.00
0.00 0.50 1.00 1.50 2.00 2.50
Figure 7.2. The ratio of the running times for procedures
LongMultiply and FastLongMultiply observed by Joe D. Stu
dent.
Sec. 7.3. Strassen’s Algorithm 73
7.3 STRASSEN’S ALGORITHM
Strassen’s algorithm is a divideandconquer algorithm for multiplying nn matrices
in time O(n
2.81
). The key ideas behind the algorithm are that 2 2 matrices
can be multiplied using only seven multiplications instead of the usual eight (see
Problems 263 and 297), and that this can be used as the base of recursion in a
divideandconquer algorithm.
331. The standard description of Strassen’s algorithm assumes that n is a
power of 2. Devise an algorithm that runs in time O(n
2.81
) when n is not
necessarily a power of 2.
332. Suppose we were to come up with a variant of Strassen’s algorithm based
on the fact that 3 3 matrices can be multiplied in only m multiplications
instead of the normal 27. How small would m have to be for this algorithm
to be faster than Strassen’s algorithm for large enough n?
333. The current record for matrix multiplication is O(n
2.376
), due to Cop
persmith and Winograd [18]. How small would m need to be in Problem 332
in order to improve on this bound?
Another version of Strassen’s algorithm uses the following identities. To compute
¸
A B
C D
=
¸
E F
G H
¸
I J
K L
,
ﬁrst compute the following values:
s
1
= G+H m
1
= s
2
s
6
t
1
= m
1
+m
2
s
2
= s
1
−E m
2
= EI t
2
= t
1
+m
4
s
3
= E −G m
3
= FK
s
4
= F −s
2
m
4
= s
3
s
7
s
5
= J −I m
5
= s
1
s
5
s
6
= L −s
5
m
6
= s
4
L
s
7
= L −J m
7
= Hs
8
s
8
= s
6
−K.
Then,
A = m
2
+m
3
B = t
1
+m
5
+m
6
C = t
2
−m
7
D = t
2
+m
5
.
334. Prove that this algorithm is correct.
74 Chap. 7. DivideandConquer
335. Analyze its running time. Is it likely to be faster or slower than the
standard version of Strassen’s algorithm in practice?
7.4 BINARY SEARCH
Binary search is a classic divideandconquer algorithm that is usually covered in
more elementary courses, but deserves to be dusted oﬀ again for a brief inspection
during the algorithms course. I usually teach it for n a power of 2, and leave the
extension to general n as an exercise (Problem 336).
336. Show that the number of comparisons used by the binary search
algorithm when n is not necessarily a power of 2 is at most log n.
337. What is wrong with the following binary search algorithm?
function search(A, x, , r)
comment ﬁnd x in A[..r]
if = r then return()
else
m := ( +r)/2
if x ≤ A[m]
then return(search(A, x, , m))
else return(search(A, x, m, r))
338. Is the following algorithm for binary search correct? If so, prove it. If
not, give an example on which it fails.
function search(A, x, , r)
comment ﬁnd x in A[..r]
if = r then return()
else
m := + (r − + 1)/2
if x ≤ A[m]
then return(search(A, x, , m))
else return(search(A, x, m+ 1, r))
339. Do an exact analysis for the average number of comparisons used
by binary search for successful searches, assuming that all elements in the
sequence are accessed with equal probability.
340. Use the binary search technique to devise an algorithm for the prob
lem of ﬁnding squareroots of natural numbers: Given an nbit natural number
N, compute
√
n using only O(n) additions and shifts.
Sec. 7.5. Quicksort 75
The following problems ask you to modify binary search in some way. For each of
them, write pseudocode for the algorithm, devise and solve a recurrence relation for
the number of comparisons used, and analyze the running time of your algorithm.
341. Modify the binary search algorithm so that it splits the input not into two
sets of almostequal sizes, but into two sets of sizes approximately onethird
and twothirds.
342. Modify the binary search algorithm to give a ternary search algorithm
that splits the input not into two sets of almostequal sizes, but into three sets
of sizes approximately onethird.
343. Let T[1..n] be a sorted array of distinct integers. Give a divideand
conquer algorithm that ﬁnds an index i such that T[i] = i (if one exists) and
runs in time O(log n).
7.5 QUICKSORT
The following is a highlevel version of Hoare’s quicksort algorithm. Suppose S is a
set of numbers.
function quicksort(S)
1. if [S[ ≤ 1
2. then return(S)
3. else
4. Choose an element a from S
5. Let S
1
, S
2
, S
3
be the elements of S that are respectively <, =, > a
6. return(quicksort(S
1
),S
2
,quicksort(S
3
))
The operation described in line 5 is known as pivoting on a.
344. The median of a set of n values is the n/2th smallest value. Suppose
quicksort were to always pivot on the median value. How many comparisons
would be made then in the worst case?
345. Suppose quicksort were to always pivot on the n/3th smallest value.
How many comparisons would be made then in the worst case?
An αpseudomedian of a list of n distinct values (where 0 < α < 1) is a value that
has at least n
α
list elements larger than it, and at least n
α
list elements smaller than
it. The following is a divideandconquer algorithm for computing a pseudomedian
(that is, an αpseudomedian for some value of α to be determined later). Assume
76 Chap. 7. DivideandConquer
n is a power of 3. If n = 3, then simply sort the 3 values and return the median.
Otherwise, divide the n items into n/3 groups of 3 values. Sort each group of
3, and pick out the n/3 medians. Now recursively apply the procedure to ﬁnd a
pseudomedian of these values.
346. Let T(n) be the number of comparisons used by the preceding algorithm
for computing a pseudomedian. Write a recurrence relation for T(n), and
solve it exactly. Hence, show that the algorithm runs in time O(n).
347. Let E(n) be the number of values that are smaller than the value
found by the preceding algorithm. Write a recurrence relation for E(n), and
hence prove that the algorithm does return a pseudomedian. What is the
value of α?
348. Any odd number can be used instead of 3. Is there any odd number
less than 13 which is better? You may use the following table, which gives
the minimum number of comparisons S(n) needed to sort n ≤ 11 numbers.
n 1 2 3 4 5 6 7 8 9 10 11
S(n) 0 1 3 5 7 10 13 16 19 22 26
349. Suppose the pseudomedian algorithm is used to ﬁnd the pivot
value in quicksort. Does it improve the worstcase number of comparisons
made?
The following is an implementation of Quicksort on an array S[1..n] due to Di
jkstra [21]. You were asked to prove it correct in Problem 279 and analyze it in
Problem 313.
1. procedure quicksort(, r)
2. comment sort S[..r]
3. i := ; j := r;
4. a := some element from S[..r];
5. repeat
6. while S[i] < a do i := i + 1;
7. while S[j] > a do j := j −1;
8. if i ≤ j then
9. swap S[i] and S[j];
10. i := i + 1; j := j −1;
11. until i > j;
12. if < j then quicksort(, j);
13. if i < r then quicksort(i, r);
350. How many comparisons does it use to sort n values in the worst case?
Sec. 7.6. Towers of Hanoi 77
351. How many comparisons does it use to sort n values on average ?
352. Suppose the preceding algorithm pivots on the middle value, that is,
line 4 is replaced by a := S[( +r)/2]. Give an input of 8 values for which
quicksort exhibits its worstcase behavior.
353. Suppose the above algorithm pivots on the middle value, that is,
line 4 is replaced by a := S[( +r)/2]. Give an input of n values for which
quicksort exhibits its worstcase behavior.
7.6 TOWERS OF HANOI
The Towers of Hanoi problem is often used as an example of recursion in introduc
tory programming courses. This problem can be resurrected later in an algorithms
course, usually to good eﬀect. You are given n disks of diﬀering diameter, arranged
in increasing order of diameter from top to bottom on the leftmost of three pegs.
You are allowed to move a single disk at a time from one peg to another, but you
must not place a larger disk on top of a smaller one. Your task is to move all
disks to the rightmost peg via a sequence of these moves. The earliest reference to
this problem known to the author is
´
Edouard Lucas [55] in Graham, Knuth, and
Patashnik [30].
354. How many moves does the divideandconquer algorithm use?
355. Show that the following algorithm solves the Towers of Hanoi
problem. Think of the pegs as being arranged in a circle, with clockwise moves
being from peg 1 to peg 2 to peg 3 to peg 1. If n is odd, then start by moving
the smallest disk one jump in the counterclockwise direction. Alternate be
tween moving the smallest disk in this manner and the only other legal move
available. If n is even, then start by moving the smallest disk one jump in
the clockwise direction. Alternate between moving the smallest disk in this
manner and the only other legal move available.
356. Prove that any algorithm that solves the Towers of Hanoi problem must
make at least 2
n
−1 moves.
Each of the following problems asks you to devise a divideandconquer algorithm
for a variant of the Towers of Hanoi problem. Write pseudocode for each algorithm,
prove that it is correct, and analyze the number of moves made.
357. Devise a divideandconquer algorithm for the Towers of Hanoi
problem when moves between peg 1 and peg 3 are not allowed (that is, all
moves must be either to or from peg 2).
78 Chap. 7. DivideandConquer
358. Devise an eﬃcient divideandconquer algorithm for the Towers of Hanoi
problem when there are 2n disks of n diﬀerent sizes, two of each size, and you
are not allowed to put a larger disk on top of a smaller one (but you can put
a disk on top of a samesize one).
359. Devise an eﬃcient divideandconquer algorithm for the Towers of
Hanoi problem when all moves must be in a clockwise direction (that is, from
peg 1 to peg 2, from peg 2 to peg 3, or from peg 3 to peg 1.)
360. Devise an eﬃcient divideandconquer algorithm for the Towers
of Hanoi problem that works when the disks are started in any legal position
(and still moves all disks to peg 3). How many moves does your algorithm
make?
361. Devise a divideandconquer algorithm for the Towers of Hanoi
problem when there are four pegs, using at most O(n2
√
2n
) moves.
362. Devise a divideandconquer algorithm for the Towers of Hanoi
problem when there are k ≥ 4 pegs, using at most O(
w
√
n
2
2
w
w
√
n/
√
2
) moves,
where w = k −2 is the number of workpegs.
363. Devise an eﬃcient divideandconquer algorithm for the Towers of
Hanoi problem when the disks are colored alternately red and blue, and we
add the extra rule that no disk may be placed on any other disk of the same
color.
7.7 DEPTHFIRST SEARCH
Depthﬁrst search is a search technique for directed graphs. Among its many ap
plications, ﬁnding connected components of a directed graph and biconnected com
ponents of an undirected graph are the ones most commonly covered in algorithms
courses. Depthﬁrst search is performed on a directed graph G = (V, E) by execut
ing the following procedure on some v ∈ V :
procedure dfs(v)
1. mark v used
2. for each w ∈ V such that (v, w) ∈ E do
3. if w is unused then
4. mark (v, w) used
5. dfs(w)
The set of marked edges form a depthﬁrst spanning tree of a connected component
of G.
Sec. 7.8. Applications 79
2
3 4
5
+
+
*
/
Figure 7.3. A DAG representing the arithmetic expression
2 + 3 ∗ 4 + 5/(3 ∗ 4).
364. Suppose an arithmetic expression is given as a DAG (directed acyclic
graph) with common subexpressions removed. For example, the expression
2+3∗4+5/(3∗4) would be given as the DAG shown in Figure 7.3. Devise an
algorithm for evaluating such a DAG with outdegree 2 (that is, all vertices
have at most 2 outgoing edges) in time O(n).
365. An undirected graph G = (V, E) is said to be kcolorable if all the
vertices of G can be colored using k diﬀerent colors such that no two adjacent
vertices have the same color. Design an algorithm that runs in time O(n +e)
to color a graph with two colors or determine that the graph is not 2colorable.
366. A triangle in an undirected graph G = (V, E) is a set of three pairwise
distinct vertices u, v, w ∈ V such that (u, v) ∈ E, (v, w) ∈ E, and (u, w) ∈ E.
Design an algorithm to test whether an undirected graph has a triangle. If G
has n vertices and e edges, then your algorithm should run in time O(n +e).
367. Design an algorithm that, given a directed graph G = (V, E) and a
distinguished vertex s ∈ V , determines for each v ∈ V the shortest path from
s to v. If G has n vertices and e edges, then your algorithm must run in time
O(n +e).
368. A forest is a graph composed of zero or more disconnected trees.
Design an algorithm that, given a graph G with n nodes, determines whether
G is a forest in time O(n).
7.8 APPLICATIONS
The following problems ask you to solve new problems using divideandconquer.
For each of your algorithms, describe it in prose or pseudocode, prove it correct,
and analyze its running time.
80 Chap. 7. DivideandConquer
369. Devise a divideandconquer algorithm for construct
ing a closed knight’s tour on an n n chessboard for all even n ≥ 6 (see
Problem 48 for deﬁnitions). Your algorithm must run in time O(n
2
).
370. The longest ascending subsequence problem is deﬁned as follows.
Given an array A[1..n] of natural numbers, ﬁnd the length of the longest
ascending subsequence of A. (A subsequence is a list A[i
1
], A[i
2
], . . . , A[i
m
]
for some 1 ≤ i
1
< i
2
< < i
m
≤ n. The value m is called the length of
the subsequence. Such a subsequence is called ascending if A[i
1
] ≤ A[i
2
] ≤
≤ A[i
m
].) Devise a divideandconquer algorithm for solving the longest
ascending subsequence problem in time O(n
2
).
371. The maximum subsequence sum problem is deﬁned as follows.
Given an array A[1..n] of natural numbers, ﬁnd values of i and j with 1 ≤ i ≤
j ≤ n such that
j
¸
k=i
A[k]
is maximized. Devise a divideandconquer algorithm for solving the maxi
mum subsequence sum problem in time O(nlog n).
372. Devise a divideandconquer algorithm for multiplying n complex num
bers using only 3(n −1) real multiplications.
373. Suppose we are given a set of n numbers k
1
, . . . , k
n
, where each
k
i
has an associated weight w
i
≥ 0 such that
¸
n
i=1
w
i
= 1. The weighted
median of the set ¦(k
i
, w
i
) [ 1 ≤ i ≤ n¦ is the number k
m
such that
¸
k
l
<k
m
w
l
<
1
2
,
¸
k
l
≤k
m
w
l
≥
1
2
.
For example, given
k
1
= 4.13, k
2
= 2.76, k
3
= 9.00, k
4
= 3.09, k
5
= 7.65,
w
1
= 0.30, w
2
= 0.15, w
3
= 0.25, w
4
= 0.10, w
5
= 0.20
the weighted median is k
1
= 4.13 because
¸
k
l
<4.13
w
l
= w
2
+w
4
= 0.15 + 0.10 = 0.25,
¸
k
l
≤4.13
w
l
= w
2
+w
4
+w
1
= 0.15 + 0.10 + 0.30 = 0.55.
Design an eﬃcient (that is, linear time) algorithm to ﬁnd the weighted me
dian. (Note that the k
i
s need not be given in sorted order.) Prove that your
algorithm performs as claimed.
Sec. 7.9. Hints 81
374. An induced subgraph of a graph G = (V, E) is a graph H = (U, F)
such that U ⊆ V , and F = E∩(UU). Given an undirected graph G = (V, E)
and an integer k, ﬁnd the maximum induced subgraph H of G such that each
vertex in H has degree at least k, or determine that it does not exist. The
algorithm should run in time O(n +e).
375. Find an algorithm that, given a connected graph in which all
vertices have even degree, constructs an Eulerian cycle in time O(n+e). (See
also Problem 69.)
376. Let G = (V, E) be a directed graph (not necessarily acyclic). Design
an eﬃcient algorithm to label the vertices of the graph with distinct labels
from 1 to [V [ such that the label of each vertex v is greater than the label of
at least one of v’s predecessors (if v has any), or to determine that no such
labeling is possible (w is a predecessor of v iﬀ (w, v) ∈ E). Your algorithm
should run in time O(n +e).
377. Given two sorted sequences with m, n elements, respectively, design
and analyze an eﬃcient divideandconquer algorithm to ﬁnd the kth ele
ment in the merge of the two sequences. The best algorithm runs in time
O(log(max(m, n))).
7.9 HINTS
317. Separate the n numbers into disjoint subsets (ﬁnding the right decomposition
is crucial) and ﬁrst run the known recursive algorithm on the subsets.
324. Observe that x is computed using only six multiplications of n/3bit integers,
plus some additions and shifts. Solve the resulting recurrence relation.
326. Start by deriving and solving a recurrence relation for the number of times
that Joe’s procedure LongAdd is called by procedures LongMultiply and Fast
LongMultiply.
355. Show that this algorithm makes exactly the same moves as the standard
divideandconquer algorithm.
357. No, it is not enough to use the standard algorithm and replace every move
from peg 1 to peg 3 (for example) with two moves to and from peg 2. That
may require you to place a large disk on top of a small one.
359. No, it is not enough to use the standard algorithm and replace every counter
clockwise move with two clockwise moves. That may require you to place a
large disk on top of a small one. It is possible to devise a divideandconquer
82 Chap. 7. DivideandConquer
algorithm for moving all of the disks one peg clockwise using (4
n
−1)/3 moves.
This can be used to solve the Towers of Hanoi problem (which is to move all of
the disks one peg counterclockwise) in 2(4
n
−1)/3 moves. The best algorithm
known to the author uses less than (1 +
√
3)
n
−1 = O(2.73
n
) moves.
361. An analysis of a kpeg algorithm by Lu [54] can be found in in Veerasamy
and Page [81]. We are interested in the case k = 4, which slightly simpliﬁes
the analysis. In addition, the analysis is much simpler if you prove it by
induction. If you try to pass oﬀ their analysis as your own, it is long enough
and idiosyncratic enough that your instructor will be sure to recognize it.
362. An algorithm by Lu [54] was shown to use at most n2
w
√
w!·n
moves by Veerasamy
and Page [81]. (If you read their paper, note that their k is the number of
work pegs, which is w in our notation.) The algorithm that I have in mind
has recurrence T(k, n) = T(k/2 + 1,
√
n)
2
.
363. Surprisingly, the standard algorithm solves this problem. You can prove it
by induction on n, but be careful about the bottom disk on each peg. The
induction hypothesis will have to be stronger than just the statement of the
required result. To get the idea, play with some examples.
365. Take note of which section this problem is in.
368. It’s easy to do this in time O(n + e), but that is not what is asked for. A
problem from Chapter 2 is relevant.
369. In order to construct closed knight’s tours on square chessboards, you will
need a method for constructing tours on rectangular boards. If you are smart
enough, you will only need n (n + 2) tours. You will also need some extra
structure in your tours to enable them to be joined at the corners (see Prob
lem 48). You will have to devise an exhaustive search algorithm for ﬁnding
small tours for the base of your divideandconquer (see Problem 513), or you
can make use of the tours shown in Figure 7.4.
374. Use the adjacency list representation. Store for each v ∈ V an additional
ﬁeld called its degree. Start by giving an algorithm that sets this ﬁeld to the
correct value. Then write a divideandconquer algorithm that, given a vertex
v of degree less than k, deletes that vertex from the graph, and all vertices of
degree less than k that are adjacent to it. Apply this algorithm to solve the
required problem.
375. See Problem 69.
Sec. 7.10. Solutions 83
r r r r r r
r r r r r r
r r r r r r
r r r r r r
r r r r r r
r r r r r r
✁
✁❆
❆ ❍
❍
✟
✟❍
❍✁
✁
❆
❆
✟
✟ ❍
❍
❆
❆
❍
❍✁
✁
❍
❍ ✟
✟
❍
❍❆
❆ ✁
✁
❆
❆
❍
❍✟
✟
❆
❆
✁
✁
✟
✟
❆
❆
❆
❆ ❍
❍ ✟
✟ ❆
❆
✁
✁
✟
✟❍
❍
✟
✟
❍
❍
✁
✁ ❍
❍ ✟
✟ r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
✟
✟
✟
✟
✁
✁ ❍
❍
❆
❆
✁
✁
❍
❍
❍
❍✟
✟
❍
❍
✁
✁
❆
❆
❍
❍❆
❆
✟
✟ ❆
❆
✁
✁
❍
❍
❆
❆
✁
✁ ❆
❆ ✁
✁✟
✟
✁
✁
❆
❆
✟
✟ ❍
❍ ✟
✟
❍
❍
❆
❆
❍
❍✟
✟
❆
❆
✁
✁
✟
✟ ❍
❍ ✟
✟
✁
✁
❆
❆✁
✁
✟
✟
❍
❍✁
✁
❍
❍
❆
❆
❍
❍
✁
✁
✁
✁ r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
r r r r r r r r
✁
✁❆
❆ ❍
❍
✁
✁
❆
❆
✁
✁❆
❆
✟
✟✁
✁
✟
✟❍
❍✟
✟
✁
✁
❆
❆ ❍
❍
✁
✁❆
❆
✁
✁
❆
❆ ❍
❍ ✟
✟ ❍
❍
❆
❆
✁
✁
❆
❆
❍
❍✟
✟
❆
❆✟
✟
❍
❍ ✟
✟
✁
✁
❆
❆✁
✁
✟
✟
✟
✟
❆
❆✟
✟
✁
✁
❆
❆
❆
❆
✟
✟
❆
❆
❆
❆
❍
❍❆
❆
❍
❍
❆
❆✁
✁
❆
❆
❍
❍
❆
❆ ❍
❍
✁
✁
✟
✟
❆
❆
✁
✁
❆
❆
✟
✟ ❆
❆
❍
❍
✟
✟ ❍
❍ ✟
✟
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
✁
✁
❆
❆
✁
✁
✟
✟❍
❍✟
✟❍
❍
✁
✁
❆
❆
✁
✁ ❆
❆
❍
❍
✟
✟ ❍
❍ ✟
✟ ❍
❍
❆
❆
✁
✁
❆
❆
❍
❍✟
✟❍
❍✟
✟
❆
❆ ❍
❍
✟
✟
✁
✁
❆
❆
✁
✁
✟
✟ ❍
❍
❆
❆
✟
✟
❆
❆ ❍
❍
✁
✁
❍
❍
✟
✟ ❍
❍
✁
✁ ❆
❆
✟
✟
✁
✁
✟
✟
❆
❆✟
✟
✟
✟
❆
❆
✟
✟
❆
❆✟
✟
❆
❆
❍
❍ ✟
✟ ❍
❍ ✟
✟
❍
❍
✁
✁
✁
✁
❍
❍✁
✁
❆
❆
❍
❍
✁
✁
❍
❍✟
✟
✁
✁
❍
❍
❍
❍❆
❆ ✁
✁
✟
✟
❆
❆✟
✟❍
❍❆
❆
✟
✟
✟
✟ ❍
❍ ✟
✟ r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
r r r r r r r r r r
✟
✟❍
❍✟
✟❍
❍✁
✁
✟
✟
❍
❍❆
❆ ✁
✁✟
✟
❆
❆
✁
✁
❆
❆
✁
✁
✟
✟
❍
❍❆
❆
✟
✟ ❍
❍ ✟
✟ ❍
❍
❆
❆
✁
✁
❆
❆
✁
✁
❍
❍✟
✟❍
❍✟
✟
✁
✁
✟
✟✁
✁
✁
✁
❆
❆ ✁
✁
✁
✁
✁
✁
✟
✟
❍
❍✁
✁
✁
✁
✟
✟
✟
✟
✟
✟ ❆
❆
✟
✟
❆
❆ ✁
✁✟
✟
❍
❍
✁
✁
❍
❍❆
❆
✟
✟
❍
❍✁
✁
❍
❍✟
✟❍
❍
✁
✁
❆
❆
✟
✟✁
✁
❍
❍
❍
❍
✁
✁
✟
✟ ❍
❍
✁
✁
❍
❍
✁
✁
✁
✁
❆
❆✁
✁❆
❆✟
✟
❆
❆
❆
❆
❍
❍❆
❆
❍
❍✁
✁
❆
❆
✟
✟
❆
❆
❍
❍❆
❆ ✁
✁
❆
❆
✟
✟ ❆
❆
❍
❍
❆
❆✟
✟
❆
❆
✟
✟
✁
✁
✁
✁
✟
✟
✁
✁ r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
r r r r r r r r r r r r
✁
✁❆
❆ ❍
❍
✁
✁
❆
❆
❍
❍
✟
✟✁
✁
✁
✁
✟
✟✁
✁
✟
✟❍
❍✟
✟❍
❍✟
✟
✁
✁
❆
❆
✁
✁
❆
❆
✟
✟ ❍
❍ ✟
✟ ❍
❍ ✟
✟ ❆
❆
✟
✟
❍
❍
✁
✁
❆
❆
✟
✟
✁
✁ ❆
❆
❍
❍
✟
✟✁
✁
❍
❍✟
✟❍
❍✟
✟❍
❍
✁
✁
❆
❆
✁
✁
❆
❆ ❍
❍ ✟
✟ ❍
❍ ✟
✟ ❍
❍
✟
✟
✁
✁
❍
❍
✟
✟✁
✁
✟
✟
❆
❆ ✁
✁✟
✟
✁
✁❆
❆
✁
✁
❆
❆ ❍
❍
✁
✁
❍
❍
✟
✟
✟
✟✁
✁
✟
✟
❆
❆
✟
✟ ❍
❍
✟
✟
✁
✁
✟
✟
❆
❆✟
✟
❍
❍
❍
❍
❆
❆
✁
✁ ❆
❆
❍
❍
❆
❆✁
✁
❍
❍
❆
❆ ❍
❍
✟
✟
✁
✁
❆
❆
✟
✟ ❍
❍ ✟
✟✁
✁
✁
✁❆
❆
✟
✟✁
✁
✟
✟
❆
❆✁
✁
❍
❍
❆
❆ ❍
❍
✁
✁❆
❆
✁
✁
❆
❆
✁
✁ ❆
❆
❍
❍
✁
✁ ❍
❍ ✟
✟ ❆
❆ ✁
✁ ❍
❍ ✟
✟
Figure 7.4. Knight’s tours for (in rowmajor order) 6 6,
6 8, 8 8, 8 10, 10 10, and 10 12 boards.
7.10 SOLUTIONS
314. Here is a full solution when n is not necessarily a power of 2. The size of
the problem is the number of entries in the array, y − x + 1. We will prove
by induction on n = y −x +1 that maximum(x, y) will return the maximum
value in S[x..y]. The algorithm is clearly correct when n ≤ 2. Now suppose
n > 2, and that maximum(x, y) will return the maximum value in S[x..y]
whenever y − x + 1 < n. In order to apply the induction hypothesis to the
ﬁrst recursive call, we must prove that (x + y)/2 − x + 1 < n. There are
two cases to consider. If y −x +1 is even, then y −x is odd, and hence y +x
is odd. Therefore,
¸
x +y
2
¸
−x + 1 =
x +y −1
2
−x + 1 =
y −x + 1
2
=
n
2
< n.
(The last inequality holds since n > 2.) If y −x+1 is odd, then y −x is even,
and hence y +x is even. Therefore,
¸
x +y
2
¸
−x + 1 =
x +y
2
−x + 1 =
y −x + 2
2
=
n + 1
2
< n.
(The last inequality holds since n > 1.) In order to apply the induction
hypothesis to the second recursive call, we must prove that y −((x+y)/2 +
84 Chap. 7. DivideandConquer
1) + 1 < n. There are two cases to consider. If y −x + 1 is even, then
y −
¸
x +y
2
¸
+ 1
+ 1 = y −
x +y −1
2
=
y −x + 1
2
=
n
2
< n.
If y −x + 1 is odd, then
y −
¸
x +y
2
¸
+ 1
+ 1 = y −
x +y
2
=
y −x + 1
2
−1/2 =
n
2
−
1
2
< n.
Procedure maximum divides the array into two parts. By the induction hy
pothesis, the recursive calls correctly ﬁnd the maxima in these parts. There
fore, since the procedure returns the maximum of the two maxima, it returns
the correct values.
315. Let T(n) be the number of comparisons used by maximum(x, y), where n =
y −x + 1 is the size of the array chunk processed. Then if n is a power of 2,
T(1) = 0, and for all n > 1 a power of 2, T(n) = 2T(n/2) + 1. Hence, using
the techniques of Chapter 4, T(n) = n −1.
316. The running time of procedure maximum is clearly a constant times the num
ber of comparisons. Hence, by the preceding, it runs in time O(n).
354. Let T(n) be the number of moves it takes to move n disks from peg i to peg
j. Clearly,
T(n) =
1 if n = 1
2T(n −1) + 1 otherwise.
Hence, using the techniques of Chapter 4, T(n) = 2
n
−1.
355. Let D be a direction, either clockwise or counterclockwise. Let D be the
opposite direction. To move n disks in direction D, we are told to alternate
between the following two moves:
• If n is odd, move the smallest disk in direction D. If n is even, move the
smallest disk in direction D.
• Make the only other legal move.
We claim that when the recursive algorithm is used to move n disks in di
rection D, it alternates between the preceding two moves. This is enough to
prove correctness of the new algorithm. The proof of the claim is by induction
on n. The claim is obviously true when n = 1. Now suppose that the claim is
true for n disks. Suppose we use the recursive algorithm to move n +1 disks
in direction D. It does the following:
• Move n disks in direction D.
• Move one disk in direction D.
• Move n disks in direction D.
Sec. 7.11. Comments 85
Let
• “D” denote moving the smallest disk in direction D,
• “D” denote moving the smallest disk in direction D, and
• “O” denote making the only other legal move.
Case 1. n + 1 is odd. Then n is even, and so by the induction hypothesis,
moving n disks in direction D uses moves:
DODO OD.
(Note that by Problem 354, the number of moves is odd, so the preceding
sequence of moves ends with D, not O.) Hence, moving n+1 disks in direction
D uses
DODO OD
. .. .
n disks
ODODO OD
. .. .
n disks
,
as required.
Case 2. n + 1 is even. Then n is odd, and so by the induction hypothesis,
moving n disks in direction D uses moves
DODO OD.
(Note that by Problem 354, the number of moves is odd, so the preceding
sequence of moves ends with D, not O.) Hence, moving n+1 disks in direction
D uses moves:
DODO OD
. .. .
n disks
ODODO OD
. .. .
n disks
,
as required. Hence, by induction, the claim holds for any number of disks.
369. A solution to this problem appears in Parberry [62].
7.11 COMMENTS
321. See also Problem 329.
329. See also Problem 321.
330. So where does this algorithm get its speed from? The secret is in Problem 320.
Although it is faster than the standard divideandconquer, there is a faster
algorithm than this one (see Sch¨ onhage and Strassen [70]).
336. Binary search is normally described and analyzed for n a power of 2. This
problem asks you to extend this to any value of n.
349. I don’t propose this as a viable alternative to quicksort. This is just an
exercise in the design, correctness proof, and analysis of divideandconquer
algorithms.
86 Chap. 7. DivideandConquer
354. According to legend, there is a set of 64 gold disks on 3 diamond needles being
solved by priests of Brahma. The Universe is supposed to end when the task is
complete. If done correctly, it will take T(64) = 2
64
−1 = 1.84 10
19
moves.
At one move per second, that’s 5.85 10
11
years. More realistically, at 1
minute per move (since the largest disks must be very heavy) during working
hours, it would take 1.54 10
14
years. The current age of the Universe is
estimated at ≈ 10
10
years. Perhaps the legend is correct.
360. There is an algorithm that will do this using at most 2
n
−1 moves. There is
even one that is provably optimal.
369. Alternative algorithms for constructing closed knight’s tours on rectangular
chessboards appear in Cull and DeCurtins [20] and Schwenk [71].
370. There is a faster algorithm for this problem. See Problem 409.
371. There is an O(n) time algorithm for this problem.
Chapter 8
Dynamic Programming
Dynamic programming is a fancy name for divideandconquer with a table. In
stead of solving subproblems recursively, solve them sequentially and store their
solutions in a table. The trick is to solve them in the right order so that whenever
the solution to a subproblem is needed, it is already available in the table. Dynamic
programming is particularly useful on problems for which divideandconquer ap
pears to yield an exponential number of subproblems, but there are really only a
small number of subproblems repeated exponentially often. In this case, it makes
sense to compute each solution the ﬁrst time and store it away in a table for later
use, instead of recomputing it recursively every time it is needed.
In general, I don’t approve of busywork problems, but I have found that under
graduates don’t really understand dynamic programming unless they ﬁll in a few
tables for themselves. Hence, the following problems include a few of this nature.
8.1 ITERATED MATRIX PRODUCT
The iterated matrix product problem is perhaps the most popular example of dy
namic programming used in algorithms texts. Given n matrices, M
1
, M
2
, . . . , M
n
,
where for 1 ≤ i ≤ n, M
i
is a r
i−1
r
i
matrix, parenthesize the product M
1
M
2
M
n
so as to minimize the total cost, assuming that the cost of multiplying an r
i−1
r
i
matrix by a r
i
r
i+1
matrix using the naive algorithm is r
i−1
r
i
r
i+1
. (See also
Problems 91 and 263.) Here is the dynamic programming algorithm for the matrix
product problem:
function matrix(n)
1. for i := 1 to n do m[i, i] := 0
2. for d := 1 to n −1 do
3. for i := 1 to n −d do
4. j := i +d
5. m[i, j] := min
i≤k<j
(m[i, k] +m[k + 1, j] +r
i−1
r
k
r
j
)
6. return(m[1, n])
378. Find three other orders in which the cost table m can be ﬁlled in
(ignoring the diagonal).
87
88 Chap. 8. Dynamic Programming
379. Prove that there are Ω(2
n
) diﬀerent orders in which the cost table m can
be ﬁlled in.
Fill in the cost table m in the dynamic programming algorithm for iterated matrix
products on the following inputs:
380. n = 4; r
0
= 2, r
1
= 5, r
2
= 4, r
3
= 1, r
4
= 10.
381. n = 4; r
0
= 3, r
1
= 7, r
2
= 2, r
3
= 5, r
4
= 10.
382. n = 5; r
0
= 12, r
1
= 2, r
2
= 15, r
3
= 16, r
4
= 8, r
5
= 3.
383. n = 5; r
0
= 8, r
1
= 3, r
2
= 2, r
3
= 19, r
4
= 18, r
5
= 7.
Find counterexamples to the following algorithms for the iterated matrix products
problem. That is, ﬁnd n, r
0
, r
1
, . . . , r
n
such that, when the product of the matrices
with these dimensions is evaluated in the order given, the cost is higher than optimal.
384. Suppose n > 2, and that r
i
is the smallest of r
1
, r
2
, . . . , r
n−1
. Break
the product after M
i
, and recursively apply this procedure to the products
M
1
M
2
M
i
and M
i+1
M
2
M
n
.
385. Multiply left to right.
386. Multiply right to left.
387. Start by cutting the product in half, and then repeat the same thing
recursively in each half.
388. Suppose r
i
is the largest of r
1
, r
2
, . . . , r
n−1
. Start by multiplying M
i
by
M
i+1
. Repeat this until the product has been evaluated.
389. Suppose r
i
is the smallest of r
1
, r
2
, . . . , r
n−1
. Start by multiplying M
i
by M
i+1
. Repeat this until the product has been evaluated.
390. Suppose r
i
is the largest of r
1
, r
2
, . . . , r
n−1
. Break the product after M
i
,
and recursively apply this procedure to the products M
1
M
2
M
i
and
M
i+1
M
2
M
n
.
Sec. 8.2. The Knapsack Problem 89
8.2 THE KNAPSACK PROBLEM
The knapsack problem (often called the zeroone knapsack problem) is as follows:
given n rods of length s
1
, s
2
, . . . , s
n
, and a natural number S, ﬁnd a subset of the
rods that has total length exactly S. The standard dynamic programming algorithm
for the knapsack problem is as follows. Let t[i, j] be true if there is a subset of the
ﬁrst i items that has total length exactly j.
function knapsack(s
1
, s
2
, . . . , s
n
, S)
1. t[0, 0] :=true
2. for j := 1 to S do t[0, j] :=false
3. for i := 1 to n do
4. for j := 0 to S do
5. t[i, j] := t[i −1, j]
6. if j −s
i
≥ 0 then t[i, j] := t[i, j] ∨ t[i −1, j −s
i
]
7. return(t[n, S])
391. Fill in the table t in the dynamic programming algorithm for the knap
sack problem on a knapsack of size 19 with rods of the following lengths:
15, 5, 16, 7, 1, 15, 6, 3.
Fill in the table t in the dynamic programming algorithm for the knapsack problem
on a knapsack of size 10 with rods of the following lengths:
392. 1, 2, 3, 4.
393. 5, 4, 3, 6.
394. 2, 2, 3, 3.
Find counterexamples to the following algorithms for the knapsack problem. That
is, ﬁnd S, n, s
1
, . . . , s
n
such that when the rods are selected using the algorithm
given, the knapsack is not completely full.
395. Put them in the knapsack in left to right order (the ﬁrstﬁt algorithm).
396. Put them in smallest ﬁrst (the bestﬁt algorithm).
397. Put them in largest ﬁrst (the worstﬁt algorithm).
90 Chap. 8. Dynamic Programming
8.3 OPTIMAL BINARY SEARCH TREES
Binary search trees are useful for storing a set S of ordered elements, with opera
tions:
search(x, S): return true iﬀ x ∈ S,
min(S): return the smallest value in S,
delete(x, S): delete x from S,
insert(x, S): insert x into S.
The optimal binary search tree problem is the following. Given the following
three sets of data,
1. S = ¦x
1
, . . . x
n
¦, where x
i
< x
i+1
for all 1 ≤ i < n,
2. the probability p
i
that we will be asked member(x
i
, S), for 1 ≤ i ≤ n,
3. the probability q
i
that we will be asked member(x, S) for some x
i
< x < x
i+1
(where x
0
= −∞, x
n+1
= ∞), for 0 ≤ i ≤ n,
construct a binary search tree (see also Section 11.2) that has the minimum number
of expected comparisons.
The standard dynamic programming algorithm for this problem is as follows. Let
T
i,j
be the mincost binary search tree for x
i+1
, . . . , x
j
, and c[i, j] be the expected
number of comparisons for T
i,j
.
function bst(p
1
, p
2
, . . . , p
n
, q
0
, q
1
, . . . , q
n
)
1. for i := 0 to n do
2. w[i, i] := q
i
; c[i, i] := 0
3. for := 1 to n do
4. for i := 0 to n − do
5. j := i +
6. w[i, j] := w[i, j −1] +p
j
+q
j
7. c[i, j] := min
i<k≤j
(c[i, k −1] +c[k, j] +w[i, j])
8. return(c[0, n])
Fill in the cost table c in the dynamic programming algorithm for the optimal
binary search tree problem on the following inputs.
398. n = 4, p
1
= 0.01, p
2
= 0.11, p
3
= 0.2, p
4
= 0.12; q
0
= 0.1, q
1
= 0.1,
q
2
= 0.28, q
3
= 0.03, q
4
= 0.05.
399. n = 4, p
1
= 0.15, p
2
= 0.32, p
3
= 0.02, p
4
= 0.07; q
0
= 0.01, q
1
= 0.16,
q
2
= 0.21, q
3
= 0.02, q
4
= 0.04.
400. n = 4, p
1
= 0.07, p
2
= 0.22, p
3
= 0.07, p
4
= 0.17; q
0
= 0.03, q
1
= 0.03,
q
2
= 0.21, q
3
= 0.09, q
4
= 0.12.
Sec. 8.4. Floyd’s Algorithm 91
1 2
5 3
4
10
30
20
65
4
6
23
70
2 1
3 5
4
100
30
20
65
4
6
9
23
25
4 3 1 2
5
8
3
21
4
3
12
1
9
(a) (b) (c) 50
44
Figure 8.1. Some graphs.
8.4 FLOYD’S ALGORITHM
The allpairs shortestpaths problem is as follows. Given a labeled directed graph
G = (V, E), ﬁnd for each pair of vertices v, w ∈ V the cost of the shortest (that
is, the leastcost) path from v to w in G. The standard dynamic programming
algorithm for the all pairs shortest paths problem (Floyd’s algorithm) is as follows.
Suppose V = ¦1, 2, . . . , n¦.
function ﬂoyd(C, n)
1. for i := 1 to n do
2. for j := 1 to n do
3. if (i, j) ∈ E
4. then A[i, j] := cost of edge (i, j)
5. else A[i, j] := ∞
6. A[i, i] := 0
7. for k := 1 to n do
8. for i := 1 to n do
9. for j := 1 to n do
10. if A[i, k] +A[k, j] < A[i, j] then A[i, j] := A[i, k] +A[k, j]
11. return(A)
401. Fill in the cost table A in Floyd’s algorithm on the graphs shown in
Figure 8.1.
402. Show that Floyd’s algorithm on an undirected graph chooses a mini
mum cost edge (v, w) for each vertex v. Can an allpairs shortestpath algo
rithm be designed simply by choosing such a cheapest edge for each vertex?
What if all edge costs are diﬀerent?
Modify Floyd’s algorithm to solve the following problems on directed graphs. An
alyze your algorithms.
92 Chap. 8. Dynamic Programming
403. Determine the number of paths (possibly inﬁnite) between each pair
of vertices.
404. Determine the number of evenlength paths and of oddlength paths
between each pair of vertices.
405. Determine the number of shortest paths between each pair of vertices.
406. Given an nvertex graph whose vertices are labeled with distinct in
tegers from 1 to n, the label of each path is obtained by taking the labels of
vertices on the path in order. Determine the label and length of the lexico
graphically ﬁrst shortest path between each pair of vertices.
8.5 APPLICATIONS
The problems in this section ask you to apply dynamic programming in new situa
tions. One way to proceed is to start by devising a divideandconquer algorithm,
replace the recursion with table lookups, and devise a series of loops that ﬁll in the
table in the correct order.
407. A contextfree grammar in Chomsky Normal Form consists of
• A set of nonterminal symbols N.
• A set of terminal symbols T.
• A special nonterminal symbol called the root.
• A set of productions of the form either A → BC, or A → a, where
A, B, C ∈ N, a ∈ T.
If A ∈ N, deﬁne L(A) as follows:
L(A) = ¦bc [ b ∈ L(B), c ∈ L(C), where A → BC¦ ∪ ¦a [ A → a¦.
The language generated by a grammar with root R is deﬁned to be L(R). The
CFL recognition problem is the following: For a ﬁxed contextfree grammar in
Chomsky Normal Form, on input a string of terminals x, determine whether
x is in the language generated by the grammar. Devise an algorithm for the
CFL recognition problem. Analyze your algorithm.
408. Fill in the tables in the dynamic programming algorithm for the CFL
recognition problem (see Problem 407) on the following inputs. In each case,
the root symbol is S.
(a) Grammar: S → SS, S → s. String: ssssss.
(b) Grammar: S → AR, S → AB, A → a, R → SB, B → b. String:
aaabbb.
Sec. 8.5. Applications 93
t h i s s a i s r t n o f c h i g a r s
Cost: 20 units
Cost: 17 units
Cost: 12 units
Total: 49 units
t h i s s a i s r t n o f c h i g a r s
t h i s s a i s r t n o f c h i g a r s
t h i s s a i s r t n o f c h i g a r s
Figure 8.2. The cost of making the breaks in lefttoright
order.
(c) Grammar: S → AX, X → SA, A → a, S → BY , Y → SB, B → b,
S → CZ, Z → SC, C → c. String: abacbbcaba.
409. Devise a dynamic programming algorithm that solves the longest
ascending subsequence (see Problem 370) in time O(nlog n).
410. A certain stringprocessing language allows the programmer to break
a string into two pieces. Since this involves copying the old string, it costs
n units of time to break a string of n characters into two pieces. Suppose a
programmer wants to break a string into many pieces. The order in which
the breaks are made can aﬀect the total amount of time used. For example,
suppose we wish to break a 20 character string after characters 3, 8, and 10
(numbering the characters in ascending order from the lefthand end, starting
from 1). If the breaks are made in lefttoright order, then the ﬁrst breaks
costs 20 units of time, the second breaks costs 17 units of time, and the third
breaks costs 12 units of time, a total of 49 units of time (see Figure 8.2). If
the breaks are made in righttoleft order, then the ﬁrst break costs 20 units
of time, the second break costs 10 units of time, and the third break costs 8
units of time, a total of 38 units of time (see Figure 8.3). Devise a dynamic
programming algorithm that, when given the numbers of the characters after
which to break, determines the cheapest cost of those breaks in time O(n
3
).
411. Find counterexamples to the following algorithms for the stringcutting prob
lem introduced in Problem 410. That is, ﬁnd the length of the string, and
several places to cut such that when cuts are made in the order given, the
cost in higher than optimal.
(a) Start by cutting the string as close to the middle as possible, and
then repeat the same thing recursively in each half.
94 Chap. 8. Dynamic Programming
Cost: 20 units
Cost: 10 units
Cost: 8 units
Total: 38 units
t h i s s a i s r t n o f c h i g a r s
t h i s s a i s r t n o f c h i g a r s
t h i s s a i s r t n o f c h i g a r s
t h i s s a i s r t n o f c h i g a r s
Figure 8.3. The cost of making the breaks in righttoleft
order.
(b) Start by making (at most) two cuts to separate the smallest sub
string. Repeat this until ﬁnished. Start by making (at most) two cuts
to separate the largest substring. Repeat this until ﬁnished.
412. There are two warehouses V and W from which widgets are to be
shipped to destinations D
i
, 1 ≤ i ≤ n. Let d
i
be the demand at D
i
, for 1 ≤ i ≤
n, and r
V
, r
W
be the number of widgets available at V and W, respectively.
Assume that there are enough widgets available to ﬁll the demand, that is,
that
r
V
+r
W
=
n
¸
i=1
d
i
. (8.1)
Let v
i
be the cost of shipping a widget from warehouse V to destination D
i
,
and w
i
be the cost of shipping a widget from warehouse W to destination D
i
,
for 1 ≤ i ≤ n. The warehouse problem is the problem of ﬁnding x
i
, y
i
∈ IN for
1 ≤ i ≤ n such that when x
i
widgets are sent from V to D
i
and y
i
widgets
are sent from W to D
i
:
• the demand at D
i
is ﬁlled, that is, x
i
+y
i
= d
i
,
• the inventory at V is suﬃcient, that is,
¸
n
i=1
x
i
= r
V
,
• the inventory at W is suﬃcient, that is,
¸
n
i=1
y
i
= r
W
,
and the total cost of shipping the widgets,
n
¸
i=1
(v
i
x
i
+w
i
y
i
)
is minimized.
(a) Let g
j
(x) be the cost incurred when V has an inventory of x widgets,
and supplies are sent to destinations D
i
for all 1 ≤ i ≤ j in the optimal
Sec. 8.5. Applications 95
Advance Move cursor one character to the right
Delete Delete the character under the cursor, and move the cursor
to the next character.
Replace Replace the character under the cursor with another. The
cursor remains stationary.
Insert Insert a new character before the one under the cursor.
The cursor remains stationary.
Kill Delete all characters from (and including) the one under
the cursor to the end of the line. This can only be the last
operation.
Table 8.1. Operations allowed on a smart terminal.
manner (note that W is not mentioned because knowledge of the inven
tory for V implies knowledge of the inventory for W, by (8.1).) Write a
recurrence relation for g
j
(x) in terms of g
j−1
.
(b) Use this recurrence to devise a dynamic programming algorithm that
ﬁnds the cost of the cheapest solution to the warehouse problem. Analyze
your algorithm.
413. Consider the problem of neatly printing a paragraph of text on
a printer with ﬁxedwidth fonts. The input text is a sequence of n words
containing
1
,
2
, . . . ,
n
, characters, respectively. Each line on the printer can
hold up to M characters. If a printed line contains words i through j, then
the number of blanks left at the end of the line (given that there is one blank
between each pair of words) is
M −j +i −
j
¸
k=i
k
.
We wish to minimize the sum, over all lines except the last, of the cubes of
the number of blanks at the end of each line (for example, if there are k lines
with b
1
, b
2
, . . . , b
k
blanks at the ends, respectively, then we wish to minimize
b
3
1
+b
3
2
+ +b
3
k−1
). Give a dynamic programming algorithm to compute this
value. Analyze your algorithm.
414. A “smart” terminal changes one string displayed on the screen into
another by a series of simple operations. The cursor is initially on the ﬁrst
character of the string. The operations are shown in Table 8.1. For example,
one way to transform the string algorithm to the string altruistic is shown
in Figure 8.4.
96 Chap. 8. Dynamic Programming
Operation String
Algorithm
advance aLgorithm
advance alGorithm
replace with t alTorithm
advance altOrithm
delete altRithm
advance altrIthm
insert u altruIthm
advance altruiThm
insert s altruisThm
advance altruistHm
insert i altruistiHm
insert c altruisticHm
kill altruistic
Figure 8.4. Transforming algorithm to altruistic using
the operations shown in Table 8.1. The cursor position is
indicated by a capital letter.
There are many sequences of operations that achieve the same result. Sup
pose that on a particular brand of terminal, the advance operation takes a
milliseconds, the delete operation takes d milliseconds, the replace operation
takes r milliseconds, the insert operation takes i milliseconds, and the kill
operation takes k milliseconds. So, for example, the sequence of operations to
transform algorithm into altruistic takes time 6a +d +r + 4i +k.
(a) Show that everything to the left of the cursor is a preﬁx of the new string,
and everything to the right of the cursor is a suﬃx of the old string.
(b) Devise a dynamic programming algorithm that, given two strings x[1..n]
and y[1..n], determines the fastest time needed to transform x to y.
Analyze your algorithm.
415. Suppose A
n
= ¦a
1
, a
2
, . . . , a
n
¦ is a set of distinct coin types,
where each a
i
∈ IN, for 1 ≤ i ≤ n. Suppose also that a
1
< a
2
< < a
n
. The
coinchanging problem is deﬁned as follows. Given C ∈ IN, ﬁnd the smallest
number of coins from A
n
that add up to C, given that an unlimited number
of coins of each type are available. Design a dynamic programming algorithm
that on inputs A
n
and C, outputs the minimum number of coins needed to
solve the coinchanging problem. Analyze your algorithm.
Sec. 8.6. Finding the Solutions 97
416. Arbitrage is the use of discrepancies in currencyexchange rates to
make a proﬁt. For example, there may be a small window of time during which
1 U.S. dollar buys 0.75 British pounds, 1 British pound buys 2 Australian
dollars, and 1 Australian dollar buys 0.70 U.S. dollars. Then, a smart trader
can trade one U.S. dollar and end up with 0.75 2 0.7 = 1.05 U.S. dollars,
a proﬁt of 5%. Suppose that there are n currencies c
1
, . . . , c
n
, and an n n
table R of exchange rates, such that one unit of currency c
i
buys R[i, j] units
of currency c
j
. Devise and analyze a dynamic programming algorithm to
determine the maximum value of
R[c
1
, c
i
1
] R[c
i
1
, c
i
2
] R[c
i
k−1
, c
i
k
] R[c
i
k
, c
1
].
417. You have $1 and want to invest it for n months. At the beginning of
each month, you must choose from the following three options:
(a) Purchase a savings certiﬁcate from the local bank. Your money will be
tied up for one month. If you buy it at time t, there will be a fee of C
S
(t)
and after a month, it will return S(t) for every dollar invested. That is,
if you have $k at time t, then you will have $(k − C
S
(t))S(t) at time
t + 1.
(b) Purchase a state treasury bond. Your money will be tied up for six
months. If you buy it at time t, there will be a fee of C
B
(t) and after
six months, it will return B(t) for every dollar invested. That is, if you
have $k at time t, then you will have $(k −C
B
(t))B(t) at time t + 6.
(c) Store the money in a sock under your mattress for a month. That is, if
you have $k at time t, then you will have $k at time t + 1.
Suppose you have predicted values for S, B, C
S
, C
B
for the next n months. De
vise a dynamic programming algorithm that computes the maximum amount
of money that you can make over the n months in time O(n).
8.6 FINDING THE SOLUTIONS
418. Show the contents of the array P for each of the parts of Problems 380–
383. Write down the cheapest parenthesization of each of the matrix products.
419. Modify the dynamic programming algorithm for the knapsack prob
lem so that the actual packing of the knapsack can be found. Write an al
gorithm for recovering the packing from the information generated by your
modiﬁed dynamic programming algorithm.
420. Show the contents of the array P for each of the graphs in Problem 401.
Write down the shortest path between each pair of vertices.
98 Chap. 8. Dynamic Programming
421. Modify your dynamic programming algorithm for the stringcutting
problem (Problem 410) so that the cuts can be recovered. Write an algorithm
for recovering the cuts from the information generated by your modiﬁed dy
namic programming algorithm.
422. Modify your dynamic programming algorithm for the warehouse prob
lem (Problem 412) so that the values x
i
and y
i
for 1 ≤ i ≤ n can be recovered.
Write an algorithm for recovering these values from the information generated
by your modiﬁed dynamic programming algorithm.
423. Modify your dynamic programming algorithm for the printing prob
lem (Problem 413) so that the number of words on each line can be recovered.
Write an algorithm for recovering these values from the information generated
by your modiﬁed dynamic programming algorithm.
424. Modify your dynamic programming algorithm for the fastest edit
sequence problem (Problem 414) so that the fastest edit sequence can be
recovered. Write an algorithm for recovering this sequence of operations from
the information generated by your modiﬁed dynamic programming algorithm.
425. Modify your dynamic programming algorithm for the coinchanging
problem (Problem 415) so that the number of each type of coin can be re
covered. Write an algorithm for recovering this data from the information
generated by your modiﬁed dynamic programming algorithm.
426. Modify your algorithm for Problem 416 to determine the sequence of
currencies i
1
, i
2
, . . . , i
k
that maximizes
R[c
1
, c
i
1
] R[c
i
1
, c
i
2
] R[i
k−1
, i
k
] R[i
k
, 1].
427. Modify your algorithm for Problem 417 so that it determines the
optimal investment strategy.
After graduation, you start working for a company that has a hierarchical supervisor
structure in the shape of a tree, rooted at the CEO. The personnel oﬃce has ranked
each employee with a conviviality rating between 0 and 10. Your ﬁrst assignment
is to design a dynamic programming algorithm to construct the guest list for the
oﬃce Christmas party. The algorithm has input consisting of the hierarchy tree,
and conviviality ratings for each employee. The output is to be a guest list that
maximizes the total conviviality, and ensures that for every employee invited to
the party, his or her immediate supervisor (that is, their parent in the tree) is not
invited.
428. Design such a dynamic programming algorithm, and analyze it.
Sec. 8.7. Hints 99
429. You are not sure if the CEO should get invited to the party, but you
suspect that you might get ﬁred if he is not. Can you modify your algorithm
to ensure that the CEO does get invited?
8.7 HINTS
378. Start by drawing a picture of the array entries on which m[i, j] depends. Make
sure that the array is ﬁlled in such way that the required entries are there
when they are needed.
409. Construct an array B[1..n] such that B[i] contains the length of the longest
ascending subsequence that starts with A[i], for 1 ≤ i ≤ n. Fill B from back
to front. From the information in B, you should be able to ﬁnd the length of
the longest ascending subsequence. You should also, with a little thought, be
able to output a longest ascending subsequence (there may be more than one
of them, obviously) in time O(n).
413. In addition to the table of costs for subproblems, you will need to keep track of
the number of blanks in the last line of text in the solution of each subproblem.
The base case is more than just ﬁlling in the diagonal entries.
415. An O(nC) time algorithm can be constructed by exploiting the similarity be
tween the coin changing problem and the knapsack problem (see Section 8.2).
It is easy to formulate an inferior algorithm that appears to run in time
O(nC
2
), but with a little thought it can be analyzed using Harmonic numbers
to give a running time of O(C
2
log n), which is superior for C = o(n/ log n).
8.8 SOLUTIONS
383. 1 2 3 4 5
1 0 48 352 1020 1096
2 0 114 792 978
3 0 684 936
4 0 2394
5 0
384. Suppose n = 3, and r
0
= 1, r
1
= 2, r
2
= 32, r
3
= 12. The suggested algorithm
parenthesizes the product as M
1
(M
2
M
3
), at a cost of 22312+1212 = 792.
The most eﬃcient way to parenthesize the product is (M
1
M
2
) M
3
, at a
cost of 1 2 32 + 1 32 12 = 448. For a larger example, take n = 7, and
dimensions 1, 2, 32, 12, 6, 5, 17, 12.
407. For each nonterminal symbol S ∈ N, keep a Boolean table t
S
[1..n, 1..n]. We
100 Chap. 8. Dynamic Programming
will ﬁll the tables so that for 1 ≤ i ≤ j ≤ n, t
S
[i, j] is true iﬀ the string
x
i
x
i+1
x
j
is in the language generated by S. t
S
[i, j] is true iﬀ either i = j
and S → x
i
, or there exists k such that i ≤ k < j and S → TU, where t
T
[i, k]
and t
U
[k + 1, j] are true.
function cﬂ(x
1
x
2
x
n
)
1. for each nonterminal S do
2. for i := 1 to n do
3. t
S
[i, i] :=true iﬀ S → x
i
4. for d := 1 to n −1 do
5. for i := 1 to n −d do
6. j := i +d
7. for each nonterminal S do
8. t
S
[i, j] :=false
9. for each production S → TU do
10. t
S
[i, j] := t
S
[i, j] ∨
j−1
k=i
(t
T
[i, k] ∧ t
U
[k + 1, j])
11. return(t
R
[1, n]) where R is the root
Since there are O(1) nonterminals:
• The forloop on lines 1–3 costs O(n).
• Line 10 costs O(n) (a single forloop).
• The forloop on lines 7–10 costs O(n).
• The forloop on lines 5–10 costs O(n
2
).
• The forloop on lines 4–10 costs O(n
3
).
Therefore, the algorithm runs in time O(n
3
).
8.9 COMMENTS
380. Although the easiest way to do this is to write a program to do it for you,
I recommend strongly that you do it by hand instead. It will help you to
understand the subtleties of the algorithm. However, there is nothing wrong
with using such a program to verify that your answer is correct.
415. In some cases, a greedy algorithm for this problem is superior to dynamic
programming (see Problem 471). See also Problem 95.
Chapter 9
Greedy Algorithms
A greedy algorithm starts with a solution to a very small subproblem and augments
it successively to a solution for the big problem. This augmentation is done in
a “greedy” fashion, that is, paying attention to shortterm or local gain, without
regard to whether it will lead to a good longterm or global solution. As in real life,
greedy algorithms sometimes lead to the best solution, sometimes lead to pretty
good solutions, and sometimes lead to lousy solutions. The trick is to determine
when to be greedy.
9.1 CONTINUOUS KNAPSACK
The continuous knapsack problem is similar to the knapsack problem studied in
Section 8.2. We are given n objects, A
1
, A
2
, . . . , A
n
, where for all 1 ≤ i ≤ n, object
A
i
has length s
i
and weight w
i
. We must ﬁll a knapsack of length S as full as possible
using fractional parts of the objects, so that the total weight is minimized, assuming
that the objects are uniformly dense (so that for all 0 ≤ x
i
≤ 1, an x
i
fraction of
A
i
has length x
i
s
i
and weight x
i
w
i
). That is, we must ﬁnd 0 < x
1
, x
2
, . . . , x
n
< 1
that minimizes
¸
n
i=1
x
i
w
i
subject to the constraint S =
¸
n
i=1
x
i
s
i
.
The greedy algorithm proceeds as follows. Deﬁne the density of object A
i
to
be w
i
/s
i
. Use as much of lowdensity objects as possible. That is, process each
in increasing order of density. If the whole thing ﬁts, use all of it. If not, ﬁll the
remaining space with a fraction of the current object, and discard the rest. That
is, we ﬁrst sort the objects in nondecreasing density, so that w
i
/s
i
≤ w
i+1
/s
i+1
for
1 ≤ i < n. Then, we perform the following pseudocode:
s := S; i := 1
while s
i
≤ s do
x
i
:= 1;s := s −s
i
; i := i + 1
x
i
:= s/s
i
for j := i + 1 to n do x
j
:= 0
430. Show that the greedy algorithm gives a solution of mini
mum weight.
101
102 Chap. 9. Greedy Algorithms
431. Solve the continuous knapsack problem on the following inputs:
(a) S = 20; s
1
= 9, s
2
= 5, s
3
= 7, s
4
= 12, s
5
= 3; w
1
= 4, w
2
= 4, w
3
= 8,
w
4
= 5, w
5
= 1.
(b) S = 18; s
1
= 9, s
2
= 5, s
3
= 7, s
4
= 12, s
5
= 3; w
1
= 18, w
2
= 15,
w
3
= 28, w
4
= 50, w
5
= 6.
(c) S = 25; s
1
= 12, s
2
= 6, s
3
= 17, s
4
= 32, s
5
= 23; w
1
= 18, w
2
= 15,
w
3
= 28, w
4
= 50, w
5
= 47.
9.2 DIJKSTRA’S ALGORITHM
The singlesource shortestpaths problem is deﬁned as follows. Given a labeled,
directed graph G = (V, E) and a distinguished vertex s ∈ V , ﬁnd for each vertex
w ∈ V , the shortest (that is, leastcost) path from s to w. (See also Problem 367 for
the unweighted version of this problem, that is, all edge labels are equal to unity.)
Vertex s is called the source.
Dijkstra’s algorithm for the singlesource shortestpaths problem is the following.
C[i, j] denotes the cost of the edge from vertex i to vertex j.
1. S := ¦s¦
2. for each v ∈ V do
3. D[v] := C[s, v]
4. if (s, v) ∈ E then P[v] := s else P[v] := 0
5. for i := 1 to n −1 do
6. choose w ∈ V −S with smallest D[w]
7. S := S ∪ ¦w¦
8. for each vertex v ∈ V do
9. if D[w] +C[w, v] < D[v] then
10. D[v] := D[w] +C[w, v]; P[v] := w
432. Show the contents of arrays D and P after each iteration of the main
forloop in Dijkstra’s algorithm executed on the graphs shown in Figure 9.1,
with source vertex 1. Use this information to list the shortest paths from
vertex 1 to each of the other vertices.
433. Show that an arbitrary collection of singlesource shortest paths in
an undirected graph does not necessarily form a spanning tree.
434. Show that the singlesource shortest paths constructed by Dijkstra’s
algorithm on a connected undirected graph form a spanning tree. (See also
Problem 433.)
435. Is the spanning tree formed by Dijkstra’s algorithm on a connected
undirected graph a mincost spanning tree? (See also Problem 434.)
Sec. 9.3. MinCost Spanning Trees 103
1 2
5 3
4
10
30
20
65
4
6
23
70
2 1
3 5
4
100
30
20
65
4
6
9
23
25
4 3 1 2
5
8
3
21
4
3
12
1
9
(a) (b) (c) 50
44
Figure 9.1. Some graphs.
436. Suppose we want to solve the singlesource longest path prob
lem. Can we modify Dijkstra’s algorithm to solve this problem by changing
minimum to maximum? If so, then prove your algorithm correct. If not, then
provide a counterexample.
437. Suppose you are the programmer for a travel agent trying to de
termine the minimum elapsed time from DFW airport to cities all over the
world. Since you got an A in my algorithms course, you immediately think
of using Dijkstra’s algorithm on the graph of cities, with an edge between
two cities labeled with the ﬂying time between them. But the shortest path
in this graph corresponds to shortest ﬂying time, which does not include the
time spent waiting at various airports for connecting ﬂights. Desperate, you
instead label each edge from city x to city y with a function f
xy
(t) that maps
from your arrival time at x to your arrival time at y in the following fashion:
If you are in x at time t, then you will be in y at time f
xy
(t) (including waiting
time and ﬂying time). Modify Dijkstra’s algorithm to work under this new
formulation, and explain why it works. State clearly any assumptions that
you make (for example, it is reasonable to assume that f
xy
(t) ≥ t).
9.3 MINCOST SPANNING TREES
A graph S = (V, T) is a spanning tree of an undirected graph G = (V, E) if S is
a tree (that is, S is connected and has no cycles), and T ⊆ E. It is a mincost
spanning tree of a labeled graph if it is a spanning tree of smallest cost.
Prim’s algorithm constructs a spanning tree using a priority queue Q of edges,
as follows:
104 Chap. 9. Greedy Algorithms
1. T := ∅; V
1
:= ¦1¦; Q := empty
2. for each w ∈ V such that (1, w) ∈ E do
3 insert(Q, (1, w))
4. while [V
1
[ < n do
5. e :=deletemin(Q)
6. Suppose e = (u, v), where u ∈ V
1
7. if v ∈ V
1
then
8. T := T ∪ ¦e¦
9. V
1
:= V
1
∪ ¦v¦
10. for each w ∈ V such that (v, w) ∈ E do
11. insert(Q, (v, w))
Kruskal’s algorithm constructs a spanning tree using the unionﬁnd abstract
data type (see Section 11.4). F is the set of vertexsets in the forest.
1. T := ∅; F := ∅
2. for each vertex v ∈ V do F := F ∪ ¦¦v¦¦
3. Sort edges in E in ascending order of cost
4. while [F[ > 1 do
5. (u, v) := the next edge in sorted order
6. if ﬁnd(u) = ﬁnd(v) then
7. union(u, v)
8. T := T ∪ ¦(u, v)¦
438. Draw the spanning forest after every iteration of the main whileloop in
Kruskal’s algorithm on each of the graphs in Figure 9.1, ignoring the directions
on the edges.
439. Draw the spanning forest after every iteration of the main whileloop in
Prim’s algorithm on each of the graphs in Figure 9.1, ignoring the directions
on the edges.
440. Draw the spanning forest after every iteration of the main whileloop in
Kruskal’s algorithm on the graph shown in Figure 9.2. Number the edges in
order of selection.
441. Draw the spanning forest after every iteration of the main whileloop
in Prim’s algorithm on the graph shown in Figure 9.2. Number the edges in
order of selection.
442. Would Prim’s algorithm start by making the choice of edges shown in
Figure 9.4 for the graph shown in Figure 9.3?
443. Would Kruskal’s algorithm start by making the choice of edges shown in
Figure 9.5 on the graph shown in Figure 9.3?
Sec. 9.3. MinCost Spanning Trees 105
3
1
2
5
4
7
8
6
10
11
12
9
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Figure 9.2. A graph.
6
2
3
5
7
4 1
2
3
4
5
6
7
8
9
10
11
1
12
Figure 9.3. Another graph.
6
2
3
5
7
4 1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4 1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4 1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4 1
2
3
4
5
6
7
8
9
10
11
1
12
Figure 9.4. A mincost spanning tree under construction
using Prim’s algorithm?
106 Chap. 9. Greedy Algorithms
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
Figure 9.5. A mincost spanning tree under construction
using Kruskal’s algorithm?
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
6
2
3
5
7
4
1
2
3
4
5
6
7
8
9
10
11
1
12
Figure 9.6. A mincost spanning tree under construction?
444. Could a greedy algorithm for mincost spanning trees start by making
the choice of edges shown in Figure 9.6 on the graph shown in Figure 9.3?
445. Could a greedy algorithm for mincost spanning trees start by making
the choice of edges shown in Figure 9.7 on the graph shown in Figure 9.3?
446. Show that Prim’s algorithm and Kruskal’s algorithm always con
struct the same mincost spanning tree on a connected undirected graph in
which the edge costs are distinct.
447. Show that Prim’s algorithm and Kruskal’s algorithm always con
struct the same mincost spanning tree on a connected undirected graph in
which no pair of edges that share a vertex have the same cost.
448. Is the path between a pair of vertices in a mincost spanning tree of
an undirected graph necessarily a mincost path? Is this true if the mincost
spanning tree is unique? If it is not unique?
Sec. 9.3. MinCost Spanning Trees 107
6
2
3
5
7
4
5
2
3
4
1
6
7
8
9
10
2
1
12
6
2
3
5
7
4
5
2
3
4
1
6
7
8
9
10
2
1
12
6
2
3
5
7
4
5
2
3
4
1
6
7
8
9
10
2
1
12
6
2
3
5
7
4
5
2
3
4
1
6
7
8
9
10
2
1
12
Figure 9.7. A mincost spanning tree under construction?
449. Show how to modify Prim’s algorithm so that it runs in time
O(nlog k) on a graph that has only k diﬀerent edge costs.
450. Here is a version of Prim’s algorithm that is diﬀerent from the one
given earlier in this section. Is it correct? If so, prove it correct. If not, explain
why it is wrong and give a graph on which it fails.
1. T := ∅; V
1
:= ¦1¦; Q := empty
2. for each w ∈ V such that (1, w) ∈ E do
3 insert(Q, (1, w))
4. while [V
1
[ < n do
5. e :=deletemin(Q)
6. Suppose e = (u, v), where u ∈ V
1
7. T := T ∪ ¦e¦
8. V
1
:= V
1
∪ ¦v¦
9. for each w ∈ V such that (v, w) ∈ E do
10. if w ∈ V
1
then insert(Q, (v, w))
In the vanilla version of Prim’s algorithm and Kruskal’s algorithm, it is not speciﬁed
what to do if a choice must be made between two edges of minimum cost.
451. Prove that for Prim’s algorithm, it doesn’t matter which of the two edges
is chosen.
452. Prove that for Kruskal’s algorithm, it doesn’t matter which of the two
edges is chosen.
453. Suppose that there are exactly two edges that have the same cost. Does
Prim’s algorithm construct the same mincost spanning tree regardless of
which edge is chosen?
108 Chap. 9. Greedy Algorithms
454. Suppose that there are exactly two edges that have the same cost. Does
Kruskal’s algorithm construct the same mincost spanning tree regardless of
which edge is chosen?
455. Suppose Prim’s algorithm and Kruskal’s algorithm choose the lexi
cographically ﬁrst edge ﬁrst. That is, if a choice must be made between two
distinct edges, e
1
= (u
1
, v
1
) and e
2
= (u
2
, v
2
) of the same cost, where u
1
< v
1
and u
2
< v
2
, then the following strategy is used. Suppose the vertices are
numbered 1, 2, . . . , n. Choose e
1
if u
1
< u
2
, or u
1
= u
2
and v
1
< v
2
. Other
wise, choose e
2
. Prove that under these conditions, both algorithms construct
the same mincost spanning tree.
456. Design an algorithm for the maxcost spanning tree.
A directed spanning tree of a directed graph is a spanning tree in which the directions
on the edges point from parent to child.
457. Suppose Prim’s algorithm is modiﬁed to take the cheapest edge di
rected out of the tree.
(a) Does it ﬁnd a directed spanning tree of a directed graph?
(b) If so, will it ﬁnd a directed spanning tree of minimum cost?
458. Suppose Kruskal’s algorithm is used to construct a directed spanning
tree.
(a) Does it ﬁnd a directed spanning tree of a directed graph?
(b) If so, will it ﬁnd a directed spanning tree of minimum cost?
459. Consider the following algorithm. Suppose Prim’s algorithm is modi
ﬁed as described in Problem 457, and then run n times, starting at each vertex
in turn. The output is the cheapest of the n directed spanning trees.
(a) Does this algorithm output a directed spanning tree of a directed graph?
(b) If so, will it ﬁnd a directed spanning tree of minimum cost?
Consider the following algorithm for ﬁnding a mincost spanning tree. Suppose the
graph has vertices numbered 1, 2, . . . , n. Start with a forest containing n trees, each
consisting of a single vertex. Repeat the following until a single tree remains. For
each vertex v = 1, 2, . . . , n in turn, consider the cheapest edge with at least one
endpoint in the tree that contains v. If it has both endpoints in the same tree, then
discard it. If not, then add it to the spanning forest.
460. Is this algorithm correct? Justify your answer.
461. Explain how to implement this algorithm in time O(e log n).
Sec. 9.4. Traveling Salesperson 109
9.4 TRAVELING SALESPERSON
The traveling salesperson problem is deﬁned as follows. Given a directed labeled
graph, ﬁnd a Hamiltonian cycle of minimum total cost. A “vanilla” greedy algorithm
for the traveling salesperson problem starts at node number 1, and moves along the
edge of minimum cost to the next node, and repeats this until it ﬁnds a node that
it has visited before, or can go no further.
462. Show that this greedy algorithm may fail to ﬁnd a cycle.
463. Show that even when the greedy algorithm ﬁnds a cycle, it may not be
a Hamiltonian cycle, even when one exists in the graph.
464. Suppose the greedy algorithm does ﬁnd a Hamiltonian cycle. Prove
that it is a Hamiltonian cycle of minimum cost.
465. Suppose the edge costs are distinct, and the greedy algorithm does
ﬁnd a Hamiltonian cycle. Show that it is the only mincost Hamiltonian cycle
in the graph.
466. Here is another greedy algorithm for the traveling salesperson prob
lem. Simply run the vanilla greedy algorithm described above n times, start
ing at each vertex in turn, and output the cheapest Hamiltonian cycle found.
Suppose this greedy algorithm does output a Hamiltonian cycle. Can it be a
diﬀerent cycle from the one found by the vanilla greedy algorithm? Justify
your answer.
Another greedy algorithm for the traveling salesperson problem proceeds as follows.
Start at node number 1, and move along the edge of minimum cost that leads to
an unvisited node, repeating this until the algorithm arrives at a vertex from which
all edges lead to visited nodes.
467. Show that this greedy algorithm may fail to ﬁnd a cycle.
468. Suppose the graph has a Hamiltonian cycle. Will this algorithm ﬁnd it?
469. Suppose this greedy algorithm ﬁnds a Hamiltonian cycle. Is it neces
sarily a Hamiltonian cycle of minimum cost? Justify your answer.
9.5 APPLICATIONS
The problems in this section ask you to apply the greedy method in new situations.
One thing you will notice about greedy algorithms is that they are usually easy to
design, easy to implement, easy to analyze, and they are very fast, but they are
almost always diﬃcult to prove correct.
110 Chap. 9. Greedy Algorithms
470. Given n ﬁles of length m
1
, m
2
, . . . , m
n
, the optimal tapestorage
problem is to ﬁnd which order is the best to store them on a tape, assuming
that each retrieval starts with the tape rewound, each retrieval takes time
equal to the length of the preceding ﬁles in the tape plus the length of the
retrieved ﬁle, and that ﬁles are to be retrieved in the reverse order. The greedy
algorithm puts the ﬁles on the tape in ascending order of size. Prove that this
is the best order.
471. Suppose A
n
= ¦a
1
, a
2
, . . . , a
n
¦ is a set of distinct coin types,
where each a
i
∈ IN, for 1 ≤ i ≤ n. Suppose also that a
1
< a
2
< < a
n
. The
coinchanging problem is deﬁned as follows. Given C ∈ IN, ﬁnd the smallest
number of coins from A
n
that add up to C, given that an unlimited number
of coins of each type are available.
(a) Suppose a
1
= 1. A greedy algorithm will make change by using the
larger coins ﬁrst, using as many coins of each type as it can before it
moves to the next lower denomination. Show that this algorithm does
not necessarily generate a solution with the minimum number of coins.
(b) Show that for all c ∈ IN, if A
n
= ¦1, c, c
2
, . . . , c
n−1
¦, c ≥ 2, then for
all C ∈ IN, the greedy algorithm always gives a minimal solution to the
coinchanging problem.
472. The vertex cover problem is deﬁned as follows. Let G = (V, E) be an
undirected graph. A vertex cover of G is a set U ⊆ V such that for each edge
(u, v) ∈ E, either u ∈ U or v ∈ U. A minimum vertex cover is a vertex cover
with the smallest number of vertices in it. The following is a greedy algorithm
for this problem:
function cover(V, E)
1. U := ∅
2. repeat
3. let v ∈ V be a vertex of maximum degree
4. U := U ∪ ¦v¦; V := V −¦v¦
5. E := E −¦(u, w) such that u = v or w = v¦
6. until E = ∅
7. return(U)
Does this algorithm always generate a minimum vertex cover? Justify your
answer.
473. The oneprocessor scheduling problem is deﬁned as follows. We are
given a set of n jobs. Each job i has a start time t
i
, and a deadline d
i
. A
feasible schedule is a permutation of the jobs such that when the jobs are
performed in that order, then every job is ﬁnished before the deadline. A
greedy algorithm for the oneprocessor scheduling problem processes the jobs
Sec. 9.6. Hints 111
in order of deadline (the early deadlines before the late ones). Show that if a
feasible schedule exists, then the schedule produced by this greedy algorithm
is feasible.
9.6 HINTS
430. Show that a greedy solution (x
1
, x
2
, . . . , x
n
) can be changed to an optimal
solution (y
1
, y
2
, . . . , y
n
) without aﬀecting its length by doing the following for
i = 1, 2, . . . , n. Change x
i
to y
i
and adjust the remaining x
i+1
, . . . , x
n
so that
the length remains the same.
446. This can be proved directly by induction. But a more insightful way to do it
is to prove that the mincost spanning tree is unique under these conditions.
447. See Problem 446.
449. This is really a data structures question.
471. For part (a), just play around until you get a counterexample. For part (b),
show that there is only one solution in which the number of coins of each type
is less than c, and that the mincost solution must have this form. These two
facts should enable you to easily prove that the greedy algorithm does the
right thing.
9.7 SOLUTIONS
430. Let X = (x
1
, x
2
, . . . , x
n
) be the solution generated by the greedy algorithm.
If all the x
i
are 1, then the solution is clearly optimal (it is the only solution).
Otherwise, let j be the smallest number such that x
j
= 1. From the algorithm,
x
i
= 1 for 1 ≤ i < j
0 ≤ x
j
< 1
x
i
= 0 for j < i ≤ n.
Therefore,
j
¸
i=1
x
i
s
i
= S. (9.1)
Let Y = (y
1
, y
2
, . . . , y
n
) be a solution of minimal weight. We will prove that
X must have the same weight as Y , and hence has minimal weight too. If
X = Y , we are done. Otherwise, let k be the least number such that x
k
= y
k
.
112 Chap. 9. Greedy Algorithms
It must be the case that y
k
< x
k
. To see this, consider the three possible
cases:
Case 1. k < j. Then x
k
= 1. Therefore, since x
k
= y
k
, y
k
must be smaller
than x
k
.
Case 2. k = j. By the deﬁnition of k, x
k
= y
k
. If y
k
> x
k
,
n
¸
i=1
y
i
s
i
=
k−1
¸
i=1
y
i
s
i
+y
k
s
k
+
n
¸
i=k+1
y
i
s
i
=
k−1
¸
i=1
x
i
s
i
+y
k
s
k
+
n
¸
i=k+1
y
i
s
i
= S + (y
k
−x
k
)s
k
+
n
¸
i=k+1
y
i
s
i
(by Equation 9.1, since k = j)
> S,
which contradicts the fact that Y is a solution. Therefore, y
k
< x
k
.
Case 3. k > j. Then x
k
= 0 and y
k
> 0, and so
n
¸
i=1
y
i
s
i
=
j
¸
i=1
y
i
s
i
+
n
¸
i=j+1
y
i
s
i
=
j
¸
i=1
x
i
s
i
+
n
¸
i=j+1
y
i
s
i
= S +
n
¸
i=j+1
y
i
s
i
(by Equation 9.1)
> S.
This is not possible, hence Case 3 can never happen. In the other two cases,
y
k
< x
k
as claimed. Now that we have established that y
k
< x
k
, we can
return to the proof. Suppose we increase y
k
to x
k
, and decrease as many of
y
k+1
, . . . , y
n
as necessary to make the total length remain at S. Call this new
solution Z = (z
1
, z
2
, . . . , z
n
). Then,
(z
k
−y
k
)s
k
> 0, (9.2)
n
¸
i=k+1
(z
i
−y
i
)s
i
< 0, (9.3)
(z
k
−y
k
)s
k
+
n
¸
i=k+1
(z
i
−y
i
)s
i
= 0. (9.4)
Sec. 9.7. Solutions 113
Therefore,
n
¸
i=1
z
i
w
i
=
k−1
¸
i=1
z
i
w
i
+z
k
w
k
+
n
¸
i=k+1
z
i
w
i
=
k−1
¸
i=1
y
i
w
i
+z
k
w
k
+
n
¸
i=k+1
z
i
w
i
=
n
¸
i=1
y
i
w
i
−y
k
w
k
−
n
¸
i=k+1
y
i
w
i
+z
k
w
k
+
n
¸
i=k+1
z
i
w
i
=
n
¸
i=1
y
i
w
i
+ (z
k
−y
k
)w
k
+
n
¸
i=k+1
(z
i
−y
i
)w
i
=
n
¸
i=1
y
i
w
i
+ (z
k
−y
k
)s
k
w
k
/s
k
+
n
¸
i=k+1
(z
i
−y
i
)s
i
w
i
/s
i
≤
n
¸
i=1
y
i
w
i
+ (z
k
−y
k
)s
k
w
k
/s
k
+
n
¸
i=k+1
(z
i
−y
i
)s
i
w
k
/s
k
(by Equation 9.4 and density)
=
n
¸
i=1
y
i
w
i
+
(z
k
−y
k
)s
k
+
n
¸
i=k+1
(z
i
−y
i
)s
i
w
k
/s
k
=
n
¸
i=1
y
i
w
i
(by Equation 9.4).
Now, since Y is a minimalweight solution,
n
¸
i=1
z
i
w
i
<
n
¸
i=1
y
i
w
i
.
Hence, Y and Z have the same weight. But Z looks “more like” X than Y
does, in the sense that the ﬁrst k entries of Z are the same as X, whereas
only the ﬁrst k − 1 entries of Y are the same as X. Repeating this proce
dure suﬃciently often transforms Y into X and maintains the same weight.
Therefore, X must have minimal weight.
470. Suppose we have ﬁles f
1
, f
2
, . . . , f
n
, of lengths m
1
, m
2
, . . . , m
n
, respectively.
Let i
1
, i
2
, . . . , i
n
be a permutation of 1, 2, . . . , n. Suppose we store the ﬁles in
order
f
i
1
, f
i
2
, . . . , f
i
n
.
114 Chap. 9. Greedy Algorithms
To retrieve the kth ﬁle on the tape, f
i
k
, costs
¸
k
j=1
m
i
j
. Therefore, the cost
of retrieving them all is
n
¸
k=1
k
¸
j=1
m
i
j
=
n
¸
k=1
(n −k + 1)m
i
k
. (9.5)
To see this, consider the following table of costs:
f
i
1
: m
i
1
f
i
2
: m
i
1
+m
i
2
f
i
3
: m
i
1
+m
i
2
+m
i
3
.
.
.
f
i
n−1
: m
i
1
+m
i
2
+m
i
3
+ +m
i
n−1
f
i
n
: m
i
1
+m
i
2
+m
i
3
+ +m
i
n−1
+m
i
n
The lefthand side of Equation 9.5 adds this table by rows. If the total is
computed by columns, we get instead
nm
i
1
+ (n −1)m
i
2
+ (n −2)m
i
3
+ + 2m
i
n−1
+m
i
n
=
n
¸
k=1
(n −k + 1)m
i
k
,
which is the righthand side of Equation (9.5). The greedy algorithm picks
ﬁles f
i
in nondecreasing order of their size m
i
. It remains to prove that this
is the minimum cost permutation.
Claim: Any permutation in nondecreasing order of m
i
’s has minimum cost.
Proof: Let Π = (i
1
, i
2
, . . . , i
n
) be a permutation of 1, 2, . . . , n that is not
in nondecreasing order of m
i
’s. We will prove that it cannot have minimum
cost. Since m
i
1
, m
i
2
, . . . , m
i
n
is not in nondecreasing order, there must exist
1 ≤ j < n such that m
i
j
> m
i
j+1
. Let Π
be permutation Π with i
j
and i
j+1
swapped. The cost of permutation Π is
C(Π) =
n
¸
k=1
(n −k + 1)m
i
k
=
j−1
¸
k=1
(n −k + 1)m
i
k
+ (n −j + 1)m
i
j
+ (n −j)m
i
j+1
+
n
¸
k=j+2
(n −k + 1)m
i
k
.
The cost of permutation Π
is
C(Π
) =
j−1
¸
k=1
(n−k+1)m
i
k
+(n−j+1)m
i
j+1
+(n−j)m
i
j
+
n
¸
k=j+2
(n−k+1)m
i
k
.
Sec. 9.8. Comments 115
Hence,
C(Π) −C(Π
) = (n −j + 1)(m
i
j
−m
i
j+1
) + (n −j)(m
i
j+1
−m
i
j
)
= m
i
j
−m
i
j+1
> 0 (by deﬁnition of j).
Therefore, C(Π
) < C(Π), and so Π cannot be a permutation of minimum
cost. This is true of any Π that is not in nondecreasing order of m
i
’s. There
fore, the minimumcost permutation must be in nondecreasing order of m
i
’s.
9.8 COMMENTS
436, 456. These problems were suggested to the author by Farhad Shahrokhi in
1993.
471. See also Problem 95. In general, a dynamic programming algorithm is the
best solution to this problem (see Problem 415). This question is designed
to illustrate the point that a greedy algorithm is better in some (but not all)
cases.
Chapter 10
Exhaustive Search
Often it appears that there is no better way to solve a problem than to try all pos
sible solutions. This approach, called exhaustive search, is almost always slow, but
sometimes it is better than nothing. It may actually be practical in real situations if
the problem size is small enough. The most commonly used algorithmic technique
used in exhaustive search is divideandconquer (see Chapter 7).
The approach taken to exhaustive search in this chapter is very diﬀerent from
that taken by most textbooks. Most exhaustive search problems can be reduced to
the problem of generating simple combinatorial objects, most often strings, permu
tations, and combinations. Sections 10.1–10.4 cover some simple algorithms that
form our basic tools. Section 10.6 covers the application of these tools to various
interesting exhaustive search problems. Section 10.5 compares exhaustive search
algorithms to backtracking. Sometimes backtracking is faster, sometimes not.
10.1 STRINGS
A bitstring is an array of n bits. Bitstrings are often used to represent sets
(the term usually used is the bitvector representation of a set). The following
is a recursive algorithm for generating bitstrings. We assume the existence of a
procedure process(C) that processes a bitstring stored in an array C. Our aim is
to process all bitstrings of a ﬁxed length.
procedure binary(m)
comment process all binary strings of length m
1. if m = 0 then process(C) else
2. C[m] := 0; binary(m−1)
3. C[m] := 1; binary(m−1)
474. Show that the algorithm is correct, that is, that a call to binary(n)
processes all binary strings of length n.
475. How many assignments to array C are made (that is, how often are
lines 2 and 3 executed)? Hence, deduce the running time of the procedure.
116
Sec. 10.1. Strings 117
The following algorithm generates all strings of n bits in an array C[1..n]. A sentinel
is used at both ends of C, so C is actually an array indexed from 0 to n + 1. The
operation “complement a bit” used in line 8 means change it from a 0 to a 1, or
viceversa. It works by treating C as a binary number and repeatedly incrementing
it through all 2
n
possible values.
1. for i := 0 to n + 1 do C[i] := 0
2. while C[n + 1] = 0 do
3. process(C)
4. increment(C)
procedure increment(C)
5. i := 1
6. while C[i −1] = 0 do
7. Complement C[i]
8. i := i + 1
476. Prove that the algorithm is correct, that is, that procedure process(C)
is called once with C[1..n] containing each string of n bits.
477. Prove that for all n ≥ 1, the number of times that a bit of C is comple
mented (that is, the number of times that line 7 is executed) is 2
n+1
−(n+2).
Hence, deduce the running time of the algorithm.
A minimalchange algorithm for generating bitstrings generates them in such an
order that each string diﬀers from its predecessor in exactly one place. The following
algorithm generates all of the bitstrings of length n in minimalchange order. The
array b is initially set to all zeros, and the bitstrings are generated with a call to
generate(n).
procedure generate(n)
1. if n = 0 then process(b)
2. else
3. generate(n −1)
4. Complement b[n]
5. generate(n −1)
478. Prove that this algorithm does generate the 2
n
bitstrings of length
n in minimalchange order.
479. Prove that for all n ≥ 1, the number of times that a bit of b is comple
mented (that is, the number of times that line 4 is executed) is 2
n
−1. Hence,
deduce the running time of the algorithm.
118 Chap. 10. Exhaustive Search
A kary string is a string of digits drawn from ¦0, 1, . . . , k − 1¦. For example, if
k = 2, then a kary string is simply a binary string.
480. Design an optimal exhaustive search algorithm for generating all kary
strings of length n in lexicographic order in time O(k
n
). For example, if k = 3
and n = 4, then your algorithm should generate the strings
0000, 0001, 0002, 0010, 0011, 0012, 0020, 0021, 0022, 0100, . . . , 2221, 2222.
481. Prove that your algorithm for Problem 480 is correct.
482. Analyze the running time of your algorithm for Problem 480.
A minimalchange algorithm for generating kary strings generates them in such an
order that each string diﬀers from its predecessor in exactly one place.
483. Design a minimalchange exhaustive search algorithm for generating
all kary strings of length n. (See also Problem 480.)
484. Prove that your algorithm for Problem 483 is correct.
485. Analyze the running time of your algorithm for Problem 483.
10.2 PERMUTATIONS
Consider the problem of exhaustively generating all permutations of n things. Sup
pose A[1..n] contains n distinct values. The following algorithm generates all permu
tations of A[1..n] in the following sense: A call to procedure permute(n) will result
in procedure process(A) being called once with A[1..n] containing each permutation
of its contents.
procedure permute(n)
1. if n = 1 then process(A) else
2. B := A
3. permute(n −1)
4. for i := 1 to n −1 do
5. swap A[n] with A[1]
6. permute(n −1)
7. A := B cyclically shifted one place right
486. Prove that this algorithm calls process(A) with A containing each
permutation of n entries exactly once.
Sec. 10.2. Permutations 119
487. What is the running time of this algorithm?
488. Show that the eﬀect of line 7 is the same as performing a cyclic
shift of A[2..n]. Use this observation to eliminate the need for the array B.
Does this aﬀect the running time of the algorithm?
Consider the following algorithm for generating all permutations. To generate all
permutations of any n distinct values in A[1..n], call permute(1).
procedure permute(i)
1. if i = n then process(A) else
4. permute(i + 1)
2. for j := i + 1 to n do
3. swap A[i] with A[j]
4. permute(i + 1)
5. swap A[i] with A[j]
489. Prove this algorithm correct.
490. How many swaps are made by this algorithm, as a function of n? What
is its running time?
The following is a minimalchange algorithm for generating permutations. That is,
each permutation generated diﬀers from the previous one by having just two values
swapped. It is known as Heap’s algorithm (Heap [35]).
procedure permute(n)
1. for i := 1 to n do A[i] := i
2. if n is odd
3. then oddpermute(n)
4. else evenpermute(n)
procedure oddpermute(n)
5. if n = 1 then process else
6. evenpermute(n −1)
7. for i := 1 to n −1 do
8. swap A[1] and A[n]
9. evenpermute(n −1)
procedure evenpermute(n)
10. if n = 1 then process else
11. oddpermute(n −1)
12. for i := 1 to n −1 do
13. swap A[i] and A[n]
14. oddpermute(n −1)
120 Chap. 10. Exhaustive Search
491. Show that the algorithm is correct, that is, it generates all
permutations in minimalchange order.
492. Show that Heap’s algorithm uses the minimum number of swaps,
(that is, n! −1 swaps). What is its running time?
10.3 COMBINATIONS
Consider the problem of generating all combinations of r things chosen from n, for
some 0 ≤ r ≤ n. We will, without loss of generality, assume that the n items to be
chosen from are the natural numbers 1, 2, . . . , n.
The following algorithm generates all combinations of r things drawn from n
in the following sense: Procedure process(A) is called once with A[1..r] containing
each combination. To use it, call procedure choose(1, r).
procedure choose(b, c)
comment choose c elements out of b..n
1. if c = 0 then process(A)
2. else for i := b to n −c + 1 do
3. A[c] := i
4. choose(i + 1, c −1)
493. Prove that this algorithm is correct.
494. Prove that the combinations generated by this algorithm are in descend
ing order, that is, when process(A) is called, A[i] > A[i +1] for all 1 ≤ i < n.
495. Let T(n, r) be the number of assignments to array A. Show that
T(n, r) ≤ r
n
r
. Hence, deduce that the algorithm runs in time O
r
n
r
.
496. Show that for all 1 ≤ r ≤ n/2, T(n, r) ≤ 2
n
r
−r. Hence, deduce
that the algorithm is optimal for r ≤ n/2.
A minimalchange algorithm for combinations generates them in such a manner
that each choice of r objects diﬀers from the previous one by having one object
removed and replaced by another. The following is a minimalchange algorithm for
generating combinations. It is used by calling up(1, r).
Sec. 10.4. Other Elementary Algorithms 121
procedure up(b, c)
1. if c = 0 then process(a)
2. else for i := b to n do
3. a[r −c + 1] := i
4. if i is odd then up(i + 1, c −1) else down(i + 1, c −1)
procedure down(b, c)
5. if c = 0 then process(a)
6. else for i := n downto b do
7. a[r −c + 1] := i
8. if i is odd then up(i + 1, c −1) else down(i + 1, c −1)
497. Prove that the algorithm is correct, that is, that procedure
process(a) is called with a containing a sequence of combinations in minimal
change order.
498. How many assignments to a does the algorithm make? Hence, deduce
its running time.
499. Prove that the combinations generated by this algorithm are in ascending
order, that is, when process(a) is called, a[i] < a[i + 1] for all 1 ≤ i < n.
10.4 OTHER ELEMENTARY ALGORITHMS
If n is even, a perfect matching of size n is an unordered sequence of unordered
pairs (v
1
, v
2
), (v
3
, v
4
), . . . , (v
n−1
, v
n
), where ¦v
1
, v
2
, . . . , v
n
¦ = ¦1, 2, . . . , n¦. Note
that two perfect matchings are deemed to be identical even if the pairs are listed
in a diﬀerent order, and the values within each pair are listed in a diﬀerent order.
The following is an algorithm for generating all perfect matchings of size n in an
array M[1..n], where n is even:
procedure match(n)
1. if n = 2 then process(M) else
2. match(n −2)
3. for i := n −2 downto 1 do
4. swap A[i] with A[n −1]
5. match(n −2)
6. circular shift A[1..n −1] one place right
500. Let M(n) be the number of perfect matchings of n values. Show that if
n is even, then
M(n) =
n/2
¸
i=1
(2i −1).
122 Chap. 10. Exhaustive Search
501. Prove that the algorithm is correct.
502. Show that the algorithm runs in time O(M(n)).
503. If n is odd, the deﬁnition of perfect matching is similar except that there
is a leftover value that is not paired with any other. What is M(n) when n is
odd? How can the above algorithm be used to generate all perfect matchings
when n is odd, in time O(M(n))?
Suppose you have m marbles and n jars, each of which can hold at most c marbles.
504. Use divideandconquer to design an algorithm that generates all of
the ways of distributing the marbles into the jars so that no jar holds more
than c marbles, where m ≤ cn. Your algorithm must print the number of
marbles in each jar for each of the diﬀerent ways of distributing the marbles.
So, for example, with m = 10, n = 3, and c = 5, the output of your algorithm
would be
0 5 5 3 2 5 4 3 3 5 3 2
1 4 5 3 3 4 4 4 2 5 4 1
1 5 4 3 4 3 4 5 1 5 5 0
2 3 5 3 5 2 5 0 5
2 4 4 4 1 5 5 1 4
2 5 3 4 2 4 5 2 3
505. Prove that your algorithm for Problem 504 is correct.
506. Analyze the running time of your algorithm for Problem 504.
10.5 BACKTRACKING
Backtracking is a type of exhaustive search in which the combinatorial object is
constructed recursively, and the recursion tree is pruned, that is, recursive calls are
not made when the part of the current object that has already been constructed
cannot lead to a valid or optimal solution. For example, the recursive algorithm for
generating bitstrings from Section 10.1 can be modiﬁed easily to give a backtracking
algorithm for generating all bitstrings of length n with k or fewer bits. To use the
algorithm, call binary(n, 0).
procedure binary(m, c)
comment process all binary strings of length m
1. if m = 0 then process(C) else
2. C[m] := 0; binary(m−1, c)
3. if c < k then C[m] := 1; binary(m−1, c + 1)
Sec. 10.6. Applications 123
Note that the pruning is done in line 3: If enough ones have been put into C
already, then the recursion tree is pruned to prevent more ones from being put in.
507. Show that the algorithm is correct, that is, that a call to binary(n, 0)
processes all binary strings of length n with k or fewer bits.
508. How many assignments to array C are made (that is, how often are
lines 2 and 3 executed)? Hence, deduce the running time of the procedure. Is
it asymptotically optimal?
The following is a backtracking algorithm for generating permutations. It ﬁlls an
arrayA[1..n] with nary strings, and prunes away nonpermutations by using an
additional array U[1..n]. U is a Boolean array, with U[i] set to true if i has not yet
been used in the permutation. Initially, all entries in U are assumed to be true.
The algorithm is used by calling permute(n).
procedure permute(m)
comment process all permutations of length m
1. if m = 0 then process(A) else
2. for j := 1 to n do
3. if U[j] then
4. U[j]:=false
5. A[m] := j; permute(m−1)
6. U[j]:=true
509. Show that the algorithm is correct, that is, that a call to permute(n)
processes all permutations of length n.
510. How many assignments to array A are made (that is, how often are
lines 3 and 4 executed)? Hence, deduce the running time of the procedure. Is
it asymptotically optimal?
10.6 APPLICATIONS
The problems in this section are to be solved by reduction to one of the standard
exhaustive search algorithms from Sections 10.1–10.4. Some of them require back
tracking, and some do not.
511. A clique is an induced complete subgraph. The clique problem is the
following: Given a graph G and a natural number r, does G contain a clique
on r vertices? Devise an algorithm for the clique problem that runs in time
O
r
2
n
r
. Brieﬂy describe how to improve the running time by a factor
of Ω(r).
124 Chap. 10. Exhaustive Search
512. Deﬁne the Ramsey number R(i, j) to be the smallest number
n such that every graph on n vertices has either a clique of size i or an
independent set of size j. Ramsey’s Theorem states that R(i, j) exists for all
i, j ∈ IN. Devise an algorithm that, given i, j, ﬁnds R(i, j). Analyze your
algorithm.
513. An open knight’s tour is a sequence of n
2
− 1 knight’s moves
starting from some square of an n n chessboard, such that every square
of the board is visited exactly once. Design an algorithm for ﬁnding open
knight’s tours that runs in time O(n
2
8
n
2
). A closed knight’s tour, as we have
seen already, is a cyclic version of an open knight’s tour (that is, the start
point is reachable from the ﬁnish point a single knight’s move). Design an
algorithm for for ﬁnding closed knight’s tours that runs in time O(8
n
2
).
514. Suppose you have n students in a class and wish to assign them into
teams of two (you may assume that n is even). You are given an integer array
P[1..n, 1..n] in which P[i, j] contains the preference of student i for working
with student j. The weight of a pairing of student i with student j is deﬁned to
be P[i, j] P[j, i]. The pairing problem is the problem of pairing each student
with another student in such a way that the sum of the weights of the pairings
is maximized. Design an algorithm for the pairing problem that runs in time
proportional to the number of pairings. Analyze your algorithm.
515. The 15puzzle, invented by Sam Loyd, has been a source of enter
tainment for over a century. The puzzle consists of 15 square tiles numbered
1 through 15, arranged in rowmajor order in a square grid with a onetile
gap. The aim is to randomize the puzzle and then attempt to return it to
the initial conﬁguration by repeatedly sliding an adjacent tile into the gap
(see, for example, Figure 10.1). The general form of this puzzle, based on an
nn grid, is called the (n
2
−1)puzzle. Design an algorithm that, given some
conﬁguration of the (n
2
−1)puzzle, ﬁnds a sequence of moves that return the
puzzle to the initial conﬁguration, if such a sequence of moves exists. Your
algorithm should run in time O(n
2
+ 4
k
), where k is the number of moves
required for solution of the puzzle.
516. The following puzzle is called HiQ. You are given a board in which 33
holes have been drilled in the pattern shown in Figure 10.2. A peg (indicated
by a square) is placed in every hole except the center one (indicated by a
circle). Pegs are moved as follows. A peg may jump over its neighbor in the
horizontal or vertical direction if its destination is unoccupied. The peg that
is jumped over is removed from the board. The aim is to produce a sequence
of jumps that removes all pegs from the board except for one. The generalized
HIQ puzzle has n(n + 4m) holes arranged in a symmetrical crossshape as
shown in Figure 10.3, for any m and any odd n. Design an algorithm that
Sec. 10.6. Applications 125
13
14 15 9
10 11
12
5
6
7
8 1
2
3
4
13
14 15 9
10 11
12
5
6
7
8 1
2
3
4
13
14 15 9
10 11
12
5
6
7
8
1 2
3
4
13
14 15 9
10 11
12
5
6
7
8
1 2
3
4
13
14 15 9
10 11
12
5 6
7
8
1 2
3
4
13
14 15 9
10 11
12
5 6
7
8
1 2
3
4
13
14 15 9
10 11
12
5 6
7
8
1 2 3 4
13
14 15 9
10 11
12
5 6 7 8
1 2 3 4
13
14 15 9
10 11
12
5 6 7 8
1 2 3 4
13
14 15
9 10 11
12
5 6 7 8
1 2 3 4
13
14 15
9 10 11
12
5 6 7 8
1 2 3 4
13 14 15
9 10 11
12
5 6 7 8
1 2 3 4
13 14 15
9 10 11
12
5 6 7 8
1 2 3 4
13 14 15
9 10 11
12
5 6 7 8
1 2 3 4
13 14 15
9 10 11
12
5 6 7 8
1 2 3 4
13 14 15
9 10 11 12
5 6 7 8
1 2 3 4
Figure 10.1. Solving the 15puzzle.
The initial configuration
After peg
jumps peg
Figure 10.2. The initial conﬁguration of the HiQ puzzle
(left), and the same conﬁguration after a single move (right).
Pegs are indicated by a square, and empty holes are indicated
by a circle.
solves the HiQ puzzle. Let p = n(n + 4m). Your algorithm should run in
time O((4p)
p−2
).
517. The nqueens problem (also known as the peaceful queens
problem) is the problem of determining the number of ways that n queens
can be placed on an n n chessboard so that no queen threatens any other
(for example, Figure 10.4 shows one way of placing eight peaceful queens on
an 8 8 chessboard). Devise an exhaustive search algorithm that solves the
nqueens problem.
126 Chap. 10. Exhaustive Search
m n m
Figure 10.3. The generalized HiQ puzzle.
Figure 10.4. One way of placing eight peaceful queens on
an 8 8 chessboard.
518. The IC Ice Cream Company plans to place ice cream carts
at strategic intersections in Dallas so that for every street intersection, either
there is an ice cream cart there, or an ice cream cart can be reached by
walking along only one street to the next intersection. You are given a map of
Dallas in the form of a graph with nodes numbered 1, 2, . . . , n representing the
intersections, and undirected edges representing streets. For each ice cream
cart at intersection i, the city will levy $c
i
in taxes. The IC Ice Cream
Company has k ice cream carts, and can make a proﬁt if it pays at most
$d in taxes. Write a backtracking algorithm that outputs all of the possible
locations for the k carts that require less than $d in taxes. Analyze your
algorithm.
Sec. 10.6. Applications 127
519. Design a backtracking algorithm that inputs a natural number C,
and outputs all of the ways that a group of ascending positive numbers can
be summed to give C. For example, if C = 6, the output should be
1+2+3
1+5
2+4
6
and if C = 10, the output should be
1+2+3+4
1+2+7
1+3+6
1+4+5
1+9
2+3+5
2+8
3+7
4+6
10
520. Devise an algorithm that, given a directed graph G, prints all
Hamiltonian cycles in G. If G has n vertices, your algorithm must run in time
O(n!). Analyze the running time of your algorithm.
521. Devise an algorithm that, given a directed graph G, prints all
Hamiltonian cycles in G. If G has n vertices, your algorithm must run in time
O((n −1)!). Analyze the running time of your algorithm.
522. The outdegree of a vertex v in a graph is the number of edges directed
out of v. The outdegree of a graph is the largest outdegree of its vertices.
Devise a backtracking algorithm that, given a directed graph G of outdegree
d, determines whether G has a Hamiltonian cycle in time O(d
n
). Be sure to
analyze the running time of your algorithm.
523. If you were given a graph and asked to ﬁnd all Hamiltonian cycles
in it, which algorithm would you use, the algorithm from Problem 520 or the
algorithm from Problem 522? Explain your answer.
A dominating set of size k in a graph G = (V, E) is a set U ⊆ V of k vertices such
that for all v ∈ V , either v ∈ U, or there exists an edge (u, v) ∈ E such that u ∈ U.
For example, the shaded vertices in the graph of Figure 10.5 form a dominating set.
128 Chap. 10. Exhaustive Search
Figure 10.5. A dominating set (shaded vertices).
524. Let D(n, k) be the number of candidates for a dominating set in an
exhaustive search algorithm. Write a formula for D(n, k) in terms of n and k.
525. Devise an algorithm that, given G and k, ﬁnds a dominating set of
size k, if one exists. Your algorithm should run in time O(n
2
D(n, k)). Analyze
the running time of your algorithm.
526. Describe how to improve the running time of your algorithm to
O(nD(n, k)). Justify brieﬂy the running time of your modiﬁed algorithm.
A weighing matrix is an n n matrix with entries from ¦−1, 0, 1¦ such that the
scalar product of any two rows is zero. That is, a matrix
M =
m
1,1
m
1,2
. . . m
1,n
m
2,1
m
2,2
. . . m
2,n
.
.
.
m
n,1
m
n,2
. . . m
n,n
¸
¸
¸
¸
¸
,
where for all 1 ≤ j ≤ n and all 1 ≤ i < j,
n
¸
k=1
m
i,k
m
j,k
= 0.
527. Design an algorithm that ﬁnds all nn weighing matrices in time
O(n
3
3
n
2
). Analyze your algorithm.
528. Describe how to reduce the running time of your algorithm to
O(n3
n
2
).
Sec. 10.6. Applications 129
1 2 3 4
1
2
3
4
2 3 1 1
1 1 4 3
1 2 3 4
2 3 2 1
P
1 2 3 4
1
2
3
4 2 3
1
2 1
1
4
3
1
2
3
4
3 3
1
N
4
1 2 3 4
1 2 3 4
Pilot:
Navigator:
P[1,2]N[2,1]+P[2,4]N[4,2]+P[3,1]N[1,3]+P[4,3]N[3,4]
= 3.4+3.2+1.3+2.4 = 12+6+3+8 = 29
Preference
Matrices:
Candidate
Matching:
Weight:
Figure 10.6. The preference matrices P and N (top), a
candidate matching (center), and the sum of the weights of
the pairs in this matching (bottom).
Suppose you are given n pilots and n navigators. You have an integer array
P[1..n, 1..n] in which P[i, j] is the preference of pilot i for navigator j, and an
integer array N[1..n, 1..n] in which N[i, j] is the preference of navigator i for pilot
j. The weight of a pairing of pilot i with navigator j is deﬁned to be P[i, j] N[j, i].
The ﬂightassignment problem is the problem of pairing each pilot with a navigator
in such a way that the sum of the weights of the pairings is maximized. Figure 10.6
shows an example of the preference arrays, a candidate matching, and the sum of
the weights of the pairs in this matching:
529. Design an algorithm for the ﬂightassignment problem that runs in time
O((n + 1)!). Analyze your algorithm.
530. Describe how to improve the running time of your algorithm from Problem 529
to O(n!).
The postage stamp problem is deﬁned as follows. The Fussytown Post Oﬃce has n
diﬀerent denominations of stamps, each a positive integer. Fussytown regulations
allow at most m stamps to be placed on any letter.
531. Design an algorithm that, given n diﬀerent postage stamp values in
an array P[1..n] and the value of m, computes the length of the consecutive
series of postage values that can legally be realized under these rules starting
with the value 1. For example, if n = 4, m = 5, and the stamps have face
value 1, 4, 12, and 21, then all postage values from 1 to 71 can be realized.
Your algorithm should run in time O(n
m
).
130 Chap. 10. Exhaustive Search
Figure 10.7. The 4ominos.
532. Design an algorithm that, given n and m, computes the m postage
stamp values that can realize the longest consecutive run of postage values
starting at 1. Analyze your algorithm.
A polyomino, or more precisely, an nomino is a plane ﬁgure made up of n unitsize
squares lined up on a grid. For example, Figure 10.7 shows all of the 4ominos.
533. Design an algorithm that runs in time O(3
n
) to generate all nominos,
including all rotations and reﬂections.
534. Modify your algorithm to remove all rotations and reﬂections.
10.7 HINTS
487. Be very careful how you choose your recurrence relation. The obvious choice
of recurrence is quite diﬃcult to solve. The right choice can make it easy.
489. Suppose that A[i] = i for 1 ≤ i ≤ n, and pretend that procedure process(A)
prints the contents of A. Trace the algorithm by hand to see what it generates
when n = 1, 2, 3, 4. This should give you some intuition as to how it works,
what your induction hypothesis should be, and how the basic structure of
your proof should look.
491. See the hint for Problem 489. Your induction hypothesis should look some
thing like the following:
When n is even, a call to evenpermute(n)
• processes all permutations of the values in A[1..n], and
• ﬁnishes with the entries in A permuted in some order in which, if re
peated n − 1 times, cycles A[1] through all possible values from 1 to n.
Determining and describing this permutation is the diﬃcult part.
When n is odd, a call to oddpermute(n)
• processes all permutations of the values in A[1..n], and
• ﬁnishes with the ﬁrst and last entries in A swapped.
Sec. 10.8. Solutions 131
502. You may need to use the identity from Problem 23.
510. The answer is at most 1.718(n + 1)!. You will need to use the fact that
¸
n
i=1
i! = e −1 ≈ 1.718 (where e is the base of the natural logarithm).
518. Think about dominating sets.
517. It is possible to solve this problem in time O(n!).
520. The running time gives away the technique to be used. Reduce the problem
to that of generating permutations.
525. Reduce the problem to that of generating combinations, and take advantage
of Section 10.3.
526. Use a minimalchange algorithm.
527. Use the algorithm from Problem 480.
528. See Problem 483.
10.8 SOLUTIONS
487. Let T(n) be the running time of permute(n). Line 6 can be done in time
O(n). Therefore, T(1) = c, and for n > 2, T(n) = nT(n − 1) + d(n − 1), for
some c, d ∈ IN. We claim that for all n ≥ 1, T(n) = (c +d)n! −d. The proof
is by induction on n. The claim is certainly true for n = 1, since T(1) = c
and (c +d)1! −d = c. Now suppose the hypothesis is true for n. Then,
T(n + 1) = (n + 1)T(n) +dn
= (n + 1)((c +d)n! −d) +dn
= (c +d)(n + 1)! −d.
Hence, the running time is O(n!), which is linear in the number of permuta
tions.
520. The following algorithm prints all of the Hamiltonian cycles of a graph G.
132 Chap. 10. Exhaustive Search
for each permutation P[0..n −1] do
process(P)
procedure process(P)
if hamiltonian(P) then print(P)
function hamiltonian(P):boolean
ok:=true
for i := 0 to n −1 do
ok:=ok ∧ (P[i], P[(i + 1) mod n]) ∈ E
return(ok)
We use an optimal permutation generation algorithm such as one of the al
gorithms from Section 10.2. Since function hamiltonian(P) takes time O(n),
the search takes time O(n n!). The running time can be reduced to O(n!)
by observing that all of the permutations can begin with vertex 1.
521. The solution is similar to the solution of Problem 520, but we use a minimal
change algorithm such as Heap’s algorithm (see Section 10.2) to generate
permutations.
We keep a Boolean array D[0..n−1], where D[i] is true iﬀ (P[i], P[(i +1) mod
n) ∈ E, for 0 ≤ i < n, and a count of missing edges, called “missing.”
We initialize the array and count when processing the ﬁrst permutation. A
minimal change to P requires a minimal change to the array and the count.
Procedure process is exactly as in the solution to Problem 520.
missing:=0
P:= ﬁrst permutation
process(P)
for each permutation P in minimalchange order do
Suppose P[x] and P[y] were just swapped
iprocess(P, x, y)
procedure edge(i, j)
D[i] := (i, j) ∈ E
if (i, j) ∈ E then missing:=missing+1
procedure iprocess(P, x, y)
if ihamiltonian(P, x, y) then print(P)
function ihamiltonian(P, x, y):boolean
iedge(P[(x −1) mod n], P[x]);
iedge(P[x], P[(x + 1) mod n]);
iedge(P[(y −1) mod n], P[y]);
iedge(P[y], P[(y + 1) mod n]);
return(missing = 0)
Sec. 10.9. Comments 133
procedure iedge(i, j)
if D[i] ∧ ((i, j) ∈ E) then
D[i]:=false; missing:=missing + 1
else if (not D[i]) ∧ ((i, j) ∈ E) then
D[i]:=true; missing:=missing −1
The time needed to process the ﬁrst permutation is O(n). The other n! − 1
permutations require O(1) time each to process. The overhead for generating
the permutations is O(n!) with a minimalchange algorithm such as Heap’s
algorithm. Therefore, the total running time is O(n +(n! −1) +n!) = O(n!).
As before, this running time can be reduced by a factor of Ω(n) by observing
that all of the permutations can begin with vertex 1.
10.9 COMMENTS
478. The algorithm generates the binary reﬂected Gray code (see also Problems 96
and 97).
497. The algorithm is adapted from Lam and Soicher [51]. You will ﬁnd a cor
rectness proof there, but there is an easier and clearer explanation. Another
interesting algorithm with a stronger minimalchange property is given by
Eades and McKay [23].
507. See also Problem 474.
508. See also Problem 475.
512. For more information on Ramsey numbers, consult Graham and Spencer [32],
and Graham, Rothschild, and Spencer [31].
513. See also Problems 48 and 369 (and associated comments), which deal with
closed knight’s tours. Backtracking for n n closed knight’s tours is only
practical for n = 6 (there are no tours for smaller n), in which case there
are 9, 862 closed knight’s tours. It is not known exactly how many 8 8
tours exist. Rouse Ball and Coxeter [8] describe an interesting random walk
algorithm that they attribute to Euler. A heuristic attributed to Warnsdorﬀ
in 1823 by Conrad, Hindrichs, Morsy, and Wegener [15, 16] appears to help
both the exhaustive search and random walk algorithms ﬁnd solutions faster:
When there is a choice of squares to move to, move to the one with the least
number of unused legal moves out of it. Takefuji and Lee [79] (reprinted in
Takefuji [78, Chapter 7]) describe a neural network for ﬁnding closed knight’s
tours, but empirical evidence indicates that their algorithm is impractical
(Parberry [62]).
517. Although an algorithm that runs in time O(n!) is considered very slow, it
134 Chap. 10. Exhaustive Search
n count time
6 4 < 0.001 seconds
7 40 < 0.001 seconds
8 92 < 0.001 seconds
9 352 0.034 seconds
10 724 0.133 seconds
11 2,680 0.6 seconds
12 14,200 3.3 seconds
13 73,712 18.0 seconds
14 365,596 1.8 minutes
15 2,279,184 11.6 minutes
16 14,772,512 1.3 hours
17 95,815,104 9.1 hours
18 666,090,624 2.8 days
Table 10.1. The number of solutions to the nqueens prob
lem for 6 ≤ n ≤ 18 and the running time required to ex
haustively search for them using a naive exhaustive search
algorithm. The algorithm was implemented in Berkeley Unix
Pascal on a Sun Sparc 2.
is still usable in practice for very small n. Table 10.1 shows the number
of solutions to the nqueens problem and the running time required to ﬁnd
them using Berkeley Unix Pascal on Sun Sparc 2. It is known that there is
at least one solution to the nqueens problem for all n ≥ 4 (see, for example,
Bernhardsson [11]). Heuristics for ﬁnding large numbers of solutions (but not
necessarily all of them) include Takefuji [78], and Susic and Gu [77].
518. This problem was described to me by Mike Fellows.
Chapter 11
Data Structures
Algorithms and data structures are obviously very closely related. Firstly, you must
understand algorithms in order to design data structures, since an important issue
in choosing a particular data structure is the availability of eﬃcient algorithms for
creating, updating, and accessing it. Secondly, you must understand data structures
in order to design algorithms, since the proper organization of data is often the
key to developing eﬃcient algorithms. Some schools separate algorithms and data
structures into separate classes, but I prefer to teach them together. In case your
algorithms class requires it, here are some problems on advanced data structures,
principally heaps, binary search trees, AVL trees, 2–3 trees, and the unionﬁnd data
structure.
11.1 HEAPS
A heap is a binary tree with the data stored in the nodes. It has two important
properties: balance and structure. Balance means that it is as much like a complete
binary tree as possible. Missing leaves, if any, are on the last level at the far right
(see, for example, Figure 11.1). Structure means that the value in each parent is no
greater than the values in its children (see, for example, Figure 11.2).
535. Prove that the value in each node is no greater than the values in its
descendants (that is, its children, its children’s children, etc.)
Figure 11.1. A balanced binary tree.
135
136 Chap. 11. Data Structures
20
3
5
24
12 40 30 15 21
11
9
10
Figure 11.2. A heap.
536. Prove by induction that a heap with n vertices has exactly n/2 leaves.
537. Suppose a heap is stored in an array H in the standard fashion.
That is, store the root in H[1], and for every node stored in H[k], store its
children in H[2k] and H[2k +1]. Prove that an nnode heap occupies the ﬁrst
n contiguous entries of H.
538. Suppose a heap is stored in an array H in the standard fashion (see
Problem 537). Prove that a list of the array elements H[1], H[2], . . . , H[n]
form a breadthﬁrst traversal of the heap.
539. Devise a divideandconquer algorithm (see Chapter 7) for building a
heap stored in an array H[1..n] in O(n) time.
540. Give the array representation heap shown in Figure 11.2, and redraw the
heap and its array representation after each of the following operations have
been applied in sequence:
(a) deletemin, followed by
(b) insert the value 0.
541. Construct a heap containing the items 10, 2, 9, 16, 8, 6, 1, 3, 12. You
are to draw the heap (as a binary tree) at each of the major steps in the
construction.
(a) Inserting the items in the order given into an empty heap.
(b) Building the heap from the bottom up, as in heapsort.
542. Write a procedure that, given parameters A, n, and i, where A is the
array representation of an nelement heap, and 1 ≤ i ≤ n, deletes the element
A[i] from the heap in O(log n) time.
543. Halving is the process of taking a set S of integers and separating it
into two halfsized sets S
1
, S
2
, where all of the values in S
1
are smaller than
all of the values in S
2
. Show that halving is equivalent to ﬁnding the median,
that is:
Sec. 11.1. Heaps 137
(a) If the median can be found using T(n) comparisons, then a set of size n
can be halved with T(n) +n −1 comparisons.
(b) If a set of size n can be halved using T(n) comparisons, then the median
can be found using T(n) +n/2 −1 comparisons.
544. Devise a data structure that supports the following operations on
an nelement set: insert in time O(log n), and deletemin and deletemax in
O(1) time.
The standard heapsort algorithm uses binary heaps, that is, heaps in which each
node has at most two children. In a kary heap, each node has at most k children.
Heapsort works not only with binary heaps, but with kary heaps for any k =
1, 2, 3, . . ..
545. Deﬁne kary heaps and describe a spaceeﬃcient implementation in
an array. In particular, describe in detail how to get from any heap element
to its parent and its k children.
546. Write pseudocode for kary heapsort.
547. What familiar sorting algorithm is kary heapsort for k = 1?
548. For k > 1, analyze the number of binary comparisons used by kary
heapsort in terms of n and k, determining the coeﬃcient (depending on k) of
the most signiﬁcant nterm. Which k minimizes this factor? Why is k = 2
probably best anyway?
A parallel priority queue is a priority queue in which the insert operation inserts
p values, and the deletemin operation deletes the p smallest values in the priority
queue and returns them in no particular order. A parallel heap is a data structure
with the same tree structure as a heap, but with p values per node. It also has the
property that all of the values in a node are smaller than the values in its children,
and all of the values in one sibling are smaller than the values in the other (it doesn’t
matter which is the “smaller” sibling). For example, Figure 11.3 shows a parallel
heap with p = 4.
Sequential algorithms for parallel heaps are fairly straightforward: An insert
operation is processed by ﬁrst placing the p new items in the next unused node d,
which we will call dirty. Combine the values in d and its parent r, and place the
smallest p values into r, and the largest p values into d. Combine the values in d
and its sibling (if it has one), and place the largest p values into the sibling that
had the largest value between them, and the smallest p values into the other sibling.
Node r then becomes the dirty node, which is processed in a similar fashion. For
example, Figure 11.4 shows an insertion into a 4node parallel heap with p = 4.
138 Chap. 11. Data Structures
6 1
2
5
8
9
15
19
20
25
30
40
16
18
22
25
26
27
32
45
Figure 11.3. A parallel heap with p = 4.
A deletemin operation is processed by removing the values from the root and
replacing them with the values from the last used node. Call the root dirty. Let
d be the dirty node. Combine the values in d with those of its smallest child s,
and place the smallest p values into d and the largest p values into s. Combine the
values in d and its sibling (if it has one), and place the largest p values into the
sibling that had the largest value between them, and the smallest p values into the
other sibling. Node s then becomes the dirty node, which is processed in a similar
fashion.
549. Show that the insert algorithm is correct, that is, it results in a
parallel heap.
550. Show that the deletemin algorithm, that is, it results in a parallel
heap.
551. Analyze the running time of the insert and deletemin algo
rithms.
11.2 AVL TREES
A binary search tree is a binary tree with the data stored in the nodes. The following
two properties hold:
• The value in each node is larger than the values in the nodes of its left subtree.
• The value in each node is smaller than the values in the nodes of its right
subtree.
552. Prove that if the nodes of a binary search tree are printed in inorder,
they come out sorted.
Sec. 11.2. AVL Trees 139
8
9
15
19
26
27
32
45
20
25
30
40
6 1
2
5
3
10
50
23
8
9
15
19
26
27
32
45
20
25
30
40
6 1
2
5
8
9
15
19
26
27
32
45
20
40
6 1
2
5
3
10
50
23
25
30
8
9
15
19
26 27
32
45
20
40
6 1
2
5
3
10
50
23
25
30
8
9
15
19
26 27
32
45
20
40
6
1
2
5
3
10
50
23
25
30
26 27
32
45
40
1
2
5
3
50
15
19
20
23
25
30
9
6
10
8
Insert (50,3,23,10) Create a new node
Combine with parent Combine with sibling
Combine with parent Combine with sibling
Figure 11.4. An insertion into a 4node parallel heap with
p = 4. Heavily outlined nodes have been combined. The
dirty node is shaded.
140 Chap. 11. Data Structures
100
50 150
30 60 120 160
10 40
Figure 11.5. An AVL tree.
100
50 150
30
20
200 70
60 80
Figure 11.6. A second AVL tree.
An AVL tree is a binary search tree with one extra structure condition: For every
node, the magnitude of the diﬀerence in height between its left and right subtrees is
at most one. (the height of an AVL tree is the number of edges in the longest path
from the root to a leaf). This diﬀerence in height is maintained during insertions and
deletions using operations called single rotations and double rotations. Operations
insert, delete, member, and findmin can be performed in time O(log n).
553. Insert 5 into the AVL tree shown in Figure 11.5.
554. Insert 55 into the AVL tree shown in Figure 11.6.
555. Delete 30 from the AVL tree shown in Figure 11.7.
556. Find an AVL tree for which the deletion of a node requires two single
rotations. Draw the tree, indicate the node to be deleted, and explain why
two rotations are necessary and suﬃcient.
Sec. 11.3. 2–3 Trees 141
100
50 150
30
210
200 120
Figure 11.7. A third AVL tree.
557. Find an AVL tree for which the deletion of a node requires two double
rotations. Draw the tree, indicate the node to be deleted, and explain why
two rotations are necessary and suﬃcient.
558. Show that for all n ∈ IN, there exists an AVL tree for which the
deletion of a node requires n (single or double) rotations. Show how such
a tree is constructed, indicate the node to be deleted, and explain why n
rotations are necessary and suﬃcient.
559. Prove that the height of an AVL tree with n nodes is at most
1.4404 log n.
11.3 2–3 TREES
A 2–3 tree in a tree in which every internal vertex has 2 or 3 children, and every
path from the root to a leaf has the same length. The data are stored in the leaves in
sorted order from left to right. The internal nodes carry information to allow search
ing, for example, the largest values in the left and center subtrees. Insertions and
deletions are performed using a divideandconquer algorithm. Operations insert,
delete, member and findmin can be performed in time O(log n).
560. Insert the following elements in sequence into an initially empty 2–3
tree: 5, 2, 7, 0, 3, 4, 6, 1, 8, 9. Show the 2–3 tree after every insertion. Be sure
to include all of the information that is kept in the internal nodes. Delete the
element 3 from the resulting 2–3 tree.
561. Give a sequence of insertions and deletions that constructs the 2–3 trees
shown in Figure 11.8 (starting from the empty 2–3 tree each time). Draw the
2–3 tree after every insertion and deletion operation.
142 Chap. 11. Data Structures
1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
1:2 4:5 6:7
2:5
1:2 3:4 6:7
2:4
1:2 3:4 5:6
(a) (b) (c)
3:5
Figure 11.8. Three 2–3 trees.
11.4 THE UNIONFIND PROBLEM
Given a set ¦1, 2, . . . , n¦ initially partitioned into n disjoint subsets, one member
per subset, we want to perform the following operations:
• find(x): return the name of the subset that x is in, and
• union(x, y): combine the two subsets that x and y are in.
There is a standard data structure using a tree. Use a tree:
• implemented with pointers and records;
• set elements are stored in the nodes;
• each child has a pointer to its parent;
• there is an array P[1..n] with P[i] pointing to the node containing i.
There is an algorithm for union and find operations that works in worstcase time
O(log n) and amortized time O(log
∗
n).
562. Design an algorithm that can perform a sequence m union and
find operations on a Universal set of n elements, consisting of a sequence of
unions followed by a sequence of finds, in time O(m+n).
Suppose we modify the unionﬁnd algorithm so that the root of the shallower tree
points to the root of the deeper tree (instead of the root of the smaller tree pointing
to the root of the deeper tree).
563. Does this aﬀect the running time of the algorithm in the worst case?
564. Does it aﬀect the amortized running time when path compression is
used?
11.5 APPLICATIONS
This section asks you to use your knowledge of data structures to design algorithms
for new problems.
Sec. 11.5. Applications 143
565. Design a data structure for a set in which insertions, deletions, and
membership queries can be processed in O(1) time in the worst case. You may
assume that the set elements are integers drawn from a ﬁnite set ¦1, 2, . . . , n¦.
566. Let S = ¦s
1
, s
2
, . . . , s
¦ and T = ¦t
1
, t
2
, . . . , t
m
¦ be two sets of inte
gers such that 1 ≤ s
i
, t
j
≤ n for all 1 ≤ i ≤ and 1 ≤ j ≤ m. Design an
algorithm for determining whether S = T in time O( +m).
567. Design an implementation of the following Abstract Data Type:
a set with the operations:
insert(x, T) Insert item x into the set T.
delete(k, T) Delete the kth smallest element from T.
member(x, T) Return true iﬀ x ∈ T.
All operations on an n item set are to take time O(log n).
568. Design an implementation of the following Abstract Data Type: a
set with the operations:
insert(x, T) Insert item x into the set T.
delete(x, T) Delete x from T.
member(x, T) Return true iﬀ x ∈ T.
next(x, T) Return the smallest element of T greater than x
union(S, T) Set S to the union of S and T.
The operations insert, delete, member, and next are to take time O(log n),
and union is to take time O(n).
569. Design an implementation of an Abstract Data Type consisting of a
set with the following operations:
insert(S, x) Insert x into the set S.
delete(S, x) Delete x from the set S.
member(S, x) Return true if x ∈ S, false otherwise.
position(S, x) Return the number of elements of S less than x.
concatenate(S, T) Set S to the union of S and T, assuming every
element in S is smaller than every element of T.
All operations involving nelement sets are to take time O(log n).
570. The bin packing problem is the following. You are given n metal ob
jects, each weighing between zero and one kilogram. You also have a collection
of large, but fragile bins. Your aim is to ﬁnd the smallest number of bins that
will hold the n objects, with no bin holding more than one kilogram.
144 Chap. 11. Data Structures
The ﬁrstﬁt heuristic for the bin packing problem is the following. Take each
of the objects in the order in which they are given. For each object, scan the
bins in increasing order of remaining capacity, and place it into the ﬁrst bin
in which it ﬁts. Design an algorithm that implements the ﬁrstﬁt heuristic
(taking as input the n weights w
1
, w
2
, . . . , w
n
and outputting the number of
bins needed when using ﬁrstﬁt) in O(nlog n) time.
571. Suppose (A, ) is a semigroup, where A = ¦a
1
, . . . , a
n
¦. (That is,
A is closed under “”, and “” is associative.) Devise a data structure that
allows you to compute a
i
a
i+1
a
j
for any i ≤ j in only O(log n) semigroup
operations.
572. Reduce the running time in Problem 571 to O(1) when (A, ) is a
group, and group inversion is allowed for free.
573. The Dallas Tollway uses a computerized toll collection system.
The highway is modeled as as an ordered list of toll booths t
1
, t
2
, . . . , t
n
in
which toll booth t
i
collects a toll of $a
i
, for 1 ≤ i ≤ n. Design a data structure
that can perform the following operations in O(log n) time.
insert(k, a) Insert a new toll booth after t
k
with toll $a.
delete(k) Delete toll booth t
k
.
toll(i, j) Return total toll for booths i through j, inclusive.
update(i, j, a) Add $a to the toll for booths i through j, inclusive.
574. Suppose we are to support a set of ordered pairs (p, k), where k is an
integer key and p is an integer priority. Devise a data structure in which the
following operations can be implemented in O(log n) time:
insert(p, k) Insert item with priority p and key k
member(k) Return item of smallest priority among those with key ≤ k.
delete(k) Delete all items with key k.
You are to design a memoryeﬃcient data structure that is constructed from a
sequence of n values x
1
, x
2
, . . . , x
n
, and can quickly answer queries of the form:
“Given i, j, ﬁnd the smallest value in x
i
, . . . , x
j
.”
575. Design a data structure that uses O(nlog n) space and answers queries
in O(log n) time.
576. Design a data structure that uses O(n) space and answers queries in
O(log n) time.
Sec. 11.6. Hints 145
577. Use your solution to Problem 576 to design a data structure to quickly
answer leastcommonancestor queries in a tree. That is, your data structure
will be constructed from a description of a tree, and using this data structure
you must answer queries of the form: “What is the leastcommonancestor of
tree vertices v and w?”
11.6 HINTS
535. Prove it by induction on level.
537. Prove it by induction on n.
544. Use two heaps A and B, each containing n/2 elements (if n is odd, store
one item separately). Make A a minheap, B a maxheap, and ensure that
for each 1 ≤ i ≤ n/2, A[i] ≤ B[i]. Then, the root of A will be the smallest
value and the root of B will be the largest value.
551. See Problem 543.
559. Let N(h) be the minimum number of nodes in an AVL tree of height h.
Devise a recurrence relation for a lower bound for N(h) (start with N(0) = 1,
N(1) = 2). Show that this recurrence has solution
N(h) >
1 +
√
5
2
h
.
562. It’s the standard unionﬁnd algorithm with path compression. The diﬃcult
part is the amortized analysis.
570 You will need a data structure that allows you to ﬁnd the “ﬁrst bin in which
it ﬁts” in time O(log n).
11.7 SOLUTIONS
545. A kary heap is a kary tree with the data stored in the nodes. It has two
important properties: balance and structure. Balance means that it is as
much like a complete kary tree as possible. Missing leaves, if any, are on the
last level at the far right. Structure means that the value in each parent is
no greater than the values in its children. A kary heap on n vertices can be
eﬃciently implemented in storage using contiguous locations of a linear array
A[1..n], such that the root of the heap is stored in A[1], the vertices are stored
in level order, and for any level, the vertices in it are stored in lefttoright
146 Chap. 11. Data Structures
order. It can be shown that the parent of A[m] is A[(m+k −2)/k] and its
children are A[km−k + 2], . . . , A[min(km+ 1, n)].
559. Let N(h) be the minimum number of nodes in an AVL tree of height h.
Clearly, N(0) = 1 and N(1) = 2, and for h ≥ 2, N(h) > N(h−1) +N(h−2).
We claim that N(h) ≥ c
h
for some choice of c. The proof is by induction on
h. The claim is true for h ≤ 2 provided c < 2. For h > 2, by the induction
hypothesis,
N(h) > N(h −1) +N(h −2) ≥ c
h−1
+c
h−2
.
So we need (assuming c > 0) c
2
−c −1 ≤ 0. The largest value of c for which
this is true is (1 +
√
5)/2. Therefore, by induction,
N(h) >
1 +
√
5
2
h
.
What is the height of an AVL tree with n nodes? We know that
n ≥ N(h) >
1 +
√
5
2
h
.
Therefore,
h <
log n
log((1 +
√
5)/2)
≈ 1.4404 log n.
567. We use a 2–3 tree augmented with extra ﬁelds in each internal node recording
how many leaves are descendants of the left, middle, and right children. The
insertion is done as normal, with extra code to increment the leafcount of the
nodes on the path from the root to the new leaf, and to take care of the new
ﬁelds during node splitting. The deletion code is modiﬁed in a similar way,
with the extra modiﬁcation that the location of the leaf to be deleted is done
using the new leafcount ﬁelds in the obvious way (recall that in a 2–3 tree,
the leaves are in ascending order from left to right). The member operations
are done as normal.
11.8 COMMENTS
537. You’ll see this result in all competent data structures textbooks, but you’ll
seldom see a proof of it.
570. This problem is ^{complete (see Chapter 12 if you don’t know what this
means), and so it is unlikely that there is a polynomialtime algorithm for
ﬁnding an exact solution.
Chapter 12
^{completeness
Deﬁne { to be the set of decision problems (that is, problems that have a yes/no
answer) that can be solved in polynomial time. If x is a bit string, let [x[ denote the
number of bits in x. Deﬁne ^{ to be the set of decision problems of the following
form, where R ∈ {, c ∈ IN: “Given x, does there exist y with [y[ ≤ [x[
c
such that
(x, y) ∈ R.” That is, ^{ is the set of existential questions that can be veriﬁed in
polynomial time. Clearly, { ⊆ ^{. It is not known whether { = ^{, although
the consensus of opinion is that probably { = ^{. But it is known that there are
problems in ^{ with the property that if they are members of {, then { = ^{.
That is, if anything in ^{ is outside of {, then they are, too. They are called
^{complete problems.
A problem A is reducible to B if an algorithm for B can be used to solve A.
More speciﬁcally, if there is an algorithm that maps every instance x of A into an
instance f(x) of B such that x ∈ A iﬀ f(x) ∈ B. A problem A is polynomial time
reducible to B, written A ≤
m
p
B, if there is a polynomial time algorithm that maps
every instance x of A into an instance f(x) of B such that x ∈ A iﬀ f(x) ∈ B. Note
that the size of f(x) can be no greater than a polynomial of the size of x.
Listing problems on ^{completeness is a little bit redundant. The truly dedi
cated student can practice by picking two random ^{complete problems from the
long list in the back of Garey and Johnson [28] and attempting to prove that one
is reducible to the other. But here are a few problems that I ﬁnd useful.
12.1 GENERAL
This section contains a few general questions about ^{completeness. Many of
them use variants of the satisﬁability problem. The standard terminology is used
here: A literal is a complemented or uncomplemented variable, a clause is a dis
junction of literals, and a Boolean formula is a conjunction of clauses. Then SAT
is deﬁned as follows:
SAT
Instance: A Boolean formula F.
147
148 Chap. 12. ^{completeness
Question: Is there a truth assignment to the variables of F that makes
F true?
578. Prove that “≤
m
p
” is transitive, that is, if A ≤
m
p
B and B ≤
m
p
C,
then A ≤
m
p
C.
579. 2SAT is the satisﬁability problem with at most two literals per clause.
Show that 2SAT ∈ {.
580. A monom is a conjunction of literals. A Boolean formula in disjunctive
normal form is a disjunction of monoms. Consider the following problem:
DNF SAT
Instance: A Boolean formula F in disjunctive normal form.
Question: Is there a truth assignment to the variables of F that
makes F true?
Prove that DNF SAT ∈ {.
581. Show that the following problem can be solved in polynomial time:
STRONG 3SAT
Instance: A Boolean formula F in conjunctive normal form with
at most three literals per clause.
Question: Is there a truth assignment to the variables of F that
makes at least two literals in each clause of F true?
582. Show that the following problem can be solved in polynomial time:
PARITY 3SAT
Instance: A Boolean formula F in conjunctive normal form with
at most three literals per clause.
Question: Is there a truth assignment to the variables of F that
makes an odd number of literals in each clause of F true?
12.2 COOK REDUCTIONS
The deﬁnition of “reduction” used in the preamble to this chapter is the one most
commonly used in algorithms classes. Technically, it is called a polynomial time
manyone reduction or a Karp reduction (after Karp [43]).
There is another deﬁnition of reducibility that you will meet in the literature if
you read more about ^{completeness. A problem A is polynomial time Turing
reducible to B, written A ≤
T
p
B, if there is an algorithm for B that calls the
Sec. 12.2. Cook Reductions 149
algorithm for A as a subroutine, and this algorithm makes polynomially many calls
to A and runs in polynomial time if the calls to A are not counted. This type of
reduction is sometimes called a Cook reduction after Cook [17].
583. Prove that if A ≤
T
p
B and B ∈ {, then A ∈ {.
584. Prove that if B ∈ ^{ and for all A ∈ ^{, A ≤
T
p
B, then B is ^{
complete.
585. Prove that “≤
T
p
” is transitive, that is, if A ≤
T
p
B and B ≤
T
p
C, then
A ≤
T
p
C.
586. Prove that if B ≤
T
p
C, C ∈ ^{, and B is ^{complete, then C is
^{complete.
Consider the following two versions of the satisﬁability problem.
SAT1
Instance: A Boolean formula F.
Question: Is there a truth assignment to the variables of F that makes
F true?
SAT2
Instance: A Boolean formula F.
Output: A truth assignment to the variables of F that makes F true,
if one exists.
The former is the decision problem for satisﬁability, and the second requires the
construction of a solution. Clearly the decision problem is Cook reducible to the
solution problem. The term selfreducibility, coined by Mike Paterson, is used to
describe problems for which the opposite is true.
587. Show that SAT2 ≤
T
p
SAT1.
Consider the following variants of the traveling salesperson problem:
TSP1
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Question: Is there a Hamiltonian cycle in G of cost at most B?
TSP2
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Output: A Hamiltonian cycle in G of cost at most B, if one exists.
150 Chap. 12. ^{completeness
TSP3
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Output: A Hamiltonian cycle in G of minimum cost, if one exists.
588. Prove that TSP2 ≤
T
p
TSP1.
589. Prove that TSP3 ≤
T
p
TSP2.
590. Prove directly that TSP3 ≤
T
p
TSP1, without using the fact that “≤
T
p
”
is transitive.
12.3 WHAT IS WRONG?
Scientists have been working since the early 1970s to either prove or disprove that
{ = ^{. This open problem is rapidly gaining popularity as one of the leading
mathematical open problems today (ranking with Fermat’s last theorem and the
Reimann hypothesis). There are several incorrect proofs that { = ^{ announced
every year. What is wrong with the following proofs that { = ^{?
591. It is known that the dominating set problem (see also Problem 523)
remains ^{complete even when the dominating set is required to be con
nected (see, for example, Garey and Johnson [28]). The interior nodes of a
breadthﬁrst search tree form a connected dominating set. For each vertex
v of a graph G = (V, E) with n vertices and e edges, ﬁnd the breadthﬁrst
search tree rooted at v in time O(n+e). Then, pick the tree with the smallest
number of interior vertices. These form a minimumsize connected dominating
set, found in time O(ne +n
2
). Hence, { = ^{.
592. It is known that 3SAT is ^{complete. Given a Boolean formula in
conjunctive normal form with at most three literals per clause, use the dis
tributive law to construct an equivalent formula in disjunctive normal form.
For example,
(x
1
∨ x
2
∨ x
3
) ∧ (x
1
∨ x
2
) = (x
1
∧ x
2
) ∨ (x
2
∧ x
1
) ∨ (x
3
∧ x
1
) ∨ (x
3
∧ x
2
).
Since DNFSAT ∈ { (see Problem 580), a satisfying assignment for the new
formula, and hence for the old formula, can be found in polynomial time. This
shows that 3SAT ∈ {, and hence that { = ^{.
593. The knapsack problem is deﬁned as follows:
KNAPSACK
Instance: Positive integers s
1
, s
2
, . . . , s
n
, S
Question: Does there exist X ⊆ ¦1, 2, . . . , n¦ such that
¸
i∈X
s
i
=
S?
Sec. 12.3. What is Wrong? 151
It is known that KNAPSACK is ^{complete. However, in Section 8.2 we
saw a dynamic programming algorithm for the knapsack problem that runs
in time O(nS). Therefore, KNAPSACK ∈ {, and hence { = ^{.
594. A split Turing machine is a variant of the Turing machine
in which the input and the computation are each split into two parts. (To
refresh your memory about standard Turing machines see, for example, Aho,
Hopcroft, and Ullman [1].) The ﬁrst part of the computation (which uses a
Turing machine) has access to the ﬁrst part of the input only, and the second
part of the computation (which uses Boolean formula evaluation) has access
both to the second part of the input and to the output of the ﬁrst part of the
computation. The ﬁrst part of the input is a string of bits that is presented
as input to a standard Turing machine in the usual manner, together with
the representation, in unary, of a natural number k. The Turing machine
computes in the normal manner, and produces an output of exactly k bits.
The second part of the input is the encoding of a Boolean formula using the
standard encoding scheme from Garey and Johnson [28].
A deterministic split Turing machine M is said to accept an input string
x#y, where x ∈ IB
n
, y ∈ IB
m
, iﬀ y is the encoding of a Boolean formula F
with k variables, and M on input x#1
k
outputs a satisfying assignment for
F. The time taken by a computation of a deterministic split Turing machine
M on input x#y is the number of steps taken by M on input x#1
k
. The time
complexity of M is deﬁned to be
T
M
(n, m) = max¦t [ there is an x#y where x ∈ IB
n
, y ∈ IB
m
, where y
encodes a Boolean formula on k variables, such that the
computation of M on input x#1
k
takes time t¦.
A nondeterministic split Turing machine M is said to accept an input string
x#y, where x ∈ IB
n
, y ∈ IB
m
, iﬀ y is the encoding of a Boolean formula F with
k variables, and some computation of M on input x#1
k
outputs a satisfying
assignment for F. The time taken by a computation of a deterministic split
Turing machine M on input x#y is the minimum over all accepting compu
tations of M of the number of steps taken by M on input x#1
k
. The time
complexity of M is deﬁned to be
T
M
(n, m) = max¦1¦ ∪ ¦t [ there is an x#y where x ∈ IB
n
, y ∈ IB
m
,
where y encodes a Boolean formula on k variables, such
that M takes time t to accept x#1
k
¦.
A split Turing machine M is said to run in polynomial time if there exists
a polynomial p such that for all m, n ≥ 1, T
M
(n, m) ≤ p(n, m). The class
152 Chap. 12. ^{completeness
split{ is then deﬁned to be the set of languages accepted by a deterministic
split Turing machine in polynomial time, and the class split^{ is deﬁned to
be the set of languages accepted by a nondeterministic split Turing machine
in polynomial time.
We claim that split{ = split^{. The proof is as follows. Consider the
language S that consists of the strings ε#y such that y is the encoding of
a satisﬁable Boolean formula F. Clearly, S ∈ split^{, since the Turing
machine on input ε#1
k
, where k is the number of variables in F, merely
need guess an output string of k bits, which it can do in time linear in k,
which is linear in the number of bits in y. However, S cannot be in split{,
since the deterministic part of the computation has access only to the number
of variables in F, and hence must give the same output for the twovariable
Boolean formulae x
1
∧x
2
and x
1
∧x
2
. Both of these formulae are satisﬁable, yet
since there is no truth assignment that satisﬁes them both, any deterministic
split Turing machine must make an error on one of them. Thus, S ∈ split^{,
and S ∈ split{, and so split{ = split^{.
Now, it is easy to see that { = split{. To see that { ⊆ split{, simply
assume that L ∈ { is encoded in binary, and append to every member of
L a ﬁxed string encoding the identity formula. Clearly, this is a member of
split{. To see that split{ ⊆ {, note that a deterministic Turing machine can
simulate a split Turing machine with polynomial overhead in time by simply
simulating the Turing machine part and then evaluating the Boolean formula
on the result obtained. This implies that { = split{, and it can be similarly
shown that ^{ = split^{. Hence, { = split{ = split^{ = ^{.
12.4 Circuits
An architecture is a ﬁnite combinational circuit (that is, a circuit constructed with
out feedback loops) with the gate functions left unspeciﬁed. A task for an archi
tecture is an inputoutput pair, and a task set is a ﬁnite set of tasks. The loading
problem is the following: Given an architecture and a task set, ﬁnd functions for
each of the gates that enable the circuit to give the correct output for each cor
responding input in the task set. The following questions deal with the decision
problem version of the loading problem: Given an architecture and a task set, de
cide whether there exist functions for each of the gates that enable the circuit to
give the correct output for each corresponding input in the task set. The problems
in this section are from Parberry [63].
595. Show that the loading problem is ^{complete.
596. Show that the loading problem for circuits of depth 2 is ^{
complete.
Sec. 12.5. OneinThree 3SAT 153
597. Show that the loading problem for circuits of fanin 3 is ^{
complete.
598. Show that the loading problem for circuits with nodefunction
set consisting of disjunctions and conjunctions of literals is ^{complete even
for circuits with only four nodes.
A cyclic circuit is a network of gates that may have cycles. Time is quantized,
and at each step, one or more of the gates reads its inputs, and computes a new
output. All connections are directed, and selfloops are allowed. The gatefunctions
are limited to are conjunction, disjunction, and complement. The circuit is said to
have converged to a stable conﬁguration if the output of all gates remains ﬁxed over
time. Consider the following problems:
The Stable Conﬁguration Problem (SNN)
Instance: A cyclic circuit M.
Question: Does M have a stable conﬁguration?
599. Show that SNN is ^{complete.
600. Show that SNN is ^{complete for cyclic circuits of fanin 2.
12.5 ONEINTHREE 3SAT
Consider the following variants of OneinThree 3SAT:
OneinThree 3SAT (O3SAT1)
Instance: A list of clauses C in which each clause contains at most
three literals.
Question: Is there a truth assignment for C in which each clause
contains exactly one true literal?
OneinThree 3SAT (O3SAT2)
Instance: A list of clauses C in which each clause contains exactly
three literals.
Question: Is there a truth assignment for C in which each clause
contains exactly one true literal?
OneinThree 3SAT (O3SAT3)
Instance: A list of clauses C in which each clause contains exactly
three variables, all uncomplemented.
Question: Is there a truth assignment for C in which each clause
contains exactly one true variable?
154 Chap. 12. ^{completeness
Balanced OneinThree 3SAT (B3SAT)
Instance: A list of clauses C in which each clause consists of three
variables and every variable appears in exactly three clauses.
Question: Is there a truth assignment for C in which each clause
contains exactly one true variable?
O3SAT1 is the original version of the problem studied by Schaefer [68], and O3SAT3
is the version listed by Garey and Johnson [28]. In the following problems, you may
assume that O3SAT1 is ^{complete.
601. Prove that O3SAT2 is ^{complete.
602. Prove that O3SAT3 is ^{complete.
603. Prove that B3SAT is ^{complete.
12.6 FACTORIZATION
Let p
i
denote the ith prime number, starting with p
1
= 2. The factorization problem
is deﬁned as follows:
FACTORIZATION
Instance: A natural number, N.
Output: A list of pairs (p
i
, a
i
) for 1 ≤ i ≤ k, such that N = p
a
1
1
p
a
2
2
p
a
k
k
.
604. Prove that if { = ^{, then FACTORIZATION can be solved in
polynomial time.
The Smarandache function, η: IN→IN, is deﬁned as follows: η(k) is the smallest
m ∈ IN such that k divides evenly into m!.
605. Prove that if FACTORIZATION can be solved in polynomial time,
then the Smarandache function can be computed in polynomial time.
606. Prove that if the Smarandache function can be computed in polyno
mial time, then FACTORIZATION can be solved in polynomial time.
12.7 HINTS
594. A lot of the details of this proof are missing. Most of them are correct. At
least one of them is wrong. It is up to you to discover which.
604. Prove a Turing reduction to the problem “is there a factor less than M.”
Sec. 12.8. Solutions 155
12.8 SOLUTIONS
595–597. A solution to these problems can be found in Judd [41].
598. A solution to this problem can be found in Parberry [61].
599–600. A solution to this problem can be found in Parberry [63].
603. A solution to this problem can be found in Parberry [60]. However, the
reduction is from O3SAT3, not O3SAT1 as requested; so technically you’ll
have to solve Problem 602 in order to complete this proof.
12.9 COMMENTS
578. Most textbooks (and classes) prove that if A ≤
m
p
B and B ∈ {, then A ∈ {,
and omit the proof of this result saying only that it is “similar.”
591. This problem came out of a conversation with B. Chitturi in 1994.
594. In a feverinduced dream in 1993 I hallucinated that I had succeeded in prov
ing that { = ^{ , and this was the proof. I awoke at 3 A.M. and instantly
knew that the proof was wrong (I was not that sick!).
595. For more information on the complexity of variants of the loading problem,
consult also Judd [39, 40, 42], Lin and Vitter [53], and Blum and Rivest [12,
13].
Chapter 13
Miscellaneous
Here are some miscellaneous problems, deﬁned to be those that do not necessarily
ﬁt into the earlier chapters, and those for which part of the problem is to determine
the algorithmic technique to be used. You’re on your own!
13.1 SORTING AND ORDER STATISTICS
This section contains questions on sorting and order statistics (the latter is a fancy
name for “ﬁnd the k smallest element”). These are mainly upper bounds. Problems
on lower bounds are mostly (but not exclusively) found in Section 13.2.
607. Show that n positive integers in the range 1 to k can be sorted in time
O(nlog k).
608. Devise an algorithm for ﬁnding the k smallest elements of an unsorted
set of n integers in time O(n +k log n).
609. Show that ﬁnding the median of a set S and ﬁnding the kth smallest
element are reducible to each other. That is, if [S[ = n, any algorithm for
ﬁnding the median of S in time T(n) can be used to design an O(T(n)) time
algorithm for ﬁnding the kth smallest, and viceversa, whenever the following
holds: T is monotone increasing and there exists a constant 0 < c < 1 such
that T(n/2) < c T(n) for n ∈ IN.
The nuts and bolts problem is deﬁned as follows. You are given a collection of n
bolts of diﬀerent widths, and n corresponding nuts. You are allowed to try a nut
and bolt together, from which you can determine whether the nut is too large, too
small, or an exact match for the bolt, but there is no way to compare two nuts
together, or two bolts together. You are to match each bolt to its nut.
610. Show that any algorithm for the nuts and bolts problem must
take Ω(nlog n) comparisons in the worst case.
156
Sec. 13.2. Lower Bounds 157
611. Devise an algorithm for the nuts and bolts problem that runs
in time O(nlog n) on average .
612. Suppose that instead of matching all of the nuts and bolts, you wish to
ﬁnd the smallest bolt and its corresponding nut. Show that this can be done
with only 2n −2 comparisons.
13.2 LOWER BOUNDS
The lower bounds considered in this section are for what is known as comparison
based algorithms. These are algorithms that store their input in memory, and the
only operations permitted on that data are comparisons and simple data movements
(that is, the input data are not used for computing an address or an index). The
two most common techniques used are the adversary argument and the decision
tree.
613. Show a matching lower bound for the sorting problem of Problem 607
in a comparisonbased model.
614. Prove that any comparisonbased algorithm that searches a sorted
array must make at least Ω(log n) comparisons.
615. Show that any comparisonbased algorithm for ﬁnding the median must
use at least n −1 comparisons.
616. Show that any comparisonbased algorithm for ﬁnding the secondsmallest
of n values can be extended to ﬁnd the smallest value also, without requiring
any more comparisons to be performed.
617. Show that at least n +log n −2 comparisons are necessary to ﬁnd
the smallest and second smallest of a list of n items.
618. Show that any comparisonbased algorithm for sorting can be modiﬁed
to remove all duplicates without requiring any more comparisons to be per
formed.
619. Show that any comparisonbased algorithm for removing duplicates
from a list of values must use Ω(nlog n) comparisons.
620. Use decision trees to ﬁnd a lower bound of
n + 0.5 log n −2.66
on the number of comparisons required to ﬁnd the median. You may use
Stirling’s approximation,
n! ∼
√
2πn(n/e)
n
.
158 Chap. 13. Miscellaneous
621. Show that if each of the n! permutations is equally likely, then
the average key is about n/3 places from its proper position in sorted order.
What does this say about the average case complexity of bubblesort?
13.3 GRAPH ALGORITHMS
This section contains general questions on graph algorithms. More graph algorithm
problems can be found in Sections 2.10, 2.11, 7.7, 8.4, 9.2, 9.3, 9.4, and 13.4, and a
few scattered in applications sections in various chapters.
622. An ancestor query for a tree is a request for information about the
ancestor relationship between two nodes in the tree. It can be viewed as
evaluating the function:
ancestor(v, w) =
−1 if v is a proper ancestor of w
1 if w is a proper ancestor of v
0 otherwise.
Design and analyze an algorithm that takes a tree as input and preprocesses
it so that ancestor queries can subsequently be answered in O(1) time. Pre
processing of an nnode tree should take O(n) time.
623. Let G = (V, E) be a directed graph with n vertices. A sink is a vertex
s ∈ V such that for all v ∈ V , (v, s) ∈ E. Devise an algorithm that, given the
adjacency matrix of G, determines whether or not G has a sink in time O(n).
624. Suppose a set of truck drivers work for a shipping company that deliv
ers commodities between n cities. The truck drivers generally ignore their
instructions and drive about at random. The probability that a given truck
driver in city i will choose to drive to city j is p
ij
. You may assume that for
all 1 ≤ i, j ≤ n, 0 ≤ p
ij
≤ 1, and that for all 1 ≤ i ≤ n,
¸
n
j=1
p
ij
= 1. Devise
an algorithm that determines in time O(n
3
) the most probable path between
cities i and j for all 1 ≤ i, j ≤ n.
625. A graph is triconnected if there is no pair of vertices whose removal
disconnects the graph. Design an algorithm that determines whether a graph
with n vertices and e edges is triconnected in time O(ne).
Let G = (V, E) be a directed labeled graph with n vertices and e edges. Suppose
the cost of edge (v, w) is (v, w). Suppose c = (v
0
, v
2
. . . , v
k−1
) is a cycle, that is,
(v
i
, v
(i+1)modk
) ∈ E for all 0 ≤ i < k. The mean cost of c is deﬁned by
µ(c) =
1
k
k−1
¸
i=0
w(v
i
, v
(i+1)modk
).
Sec. 13.3. Graph Algorithms 159
Let µ be the cost of the minimum meancost cycle in G, that is, the minimum value
of µ
c
over all cycles c in G.
Assume without loss of generality that every v ∈ V is reachable from a source
vertex s ∈ V . Let δ(v) be the cost of the cheapest path from s to v, and δ
k
(v) be
the cost of the cheapest path of length exactly k from s to v (if there is no path
from s to v of length exactly k, then δ
k
(v) = ∞).
626. Show that if µ = 0, then G contains no negativecost cycles and for
all v ∈ V ,
δ(v) = min
0≤k≤n−1
δ
k
(v).
627. Show that if µ = 0, then for all vertices v ∈ V ,
max
0≤k≤n−1
δ
n
(v) −δ
k
(v)
n −k
≥ 0.
628. Let c be a cycle of cost zero, and let u, v be any pair of vertices
on c. Suppose that the cost of the path from u to v along c is x. Prove that
δ(v) = δ(u) +x.
629. Show that if µ = 0, then there exists a vertex v on the minimum
meancost cycle such that
max
0≤k≤n−1
δ
n
(v) −δ
k
(v)
n −k
= 0.
630. Show that if µ = 0, then
min
v∈V
max
0≤k≤n−1
δ
n
(v) −δ
k
(v)
n −k
= 0.
631. Show that if we add a constant c to the cost of every edge of G, then
µ is increased by exactly c. Hence, show that
µ = min
v∈V
max
0≤k≤n−1
δ
n
(v) −δ
k
(v)
n −k
.
632. Give an O(ne) time algorithm to compute µ.
160 Chap. 13. Miscellaneous
13.4 MAXIMUM FLOW
Suppose we are given a directed graph G = (V, E) in which each edge (u, v) ∈ E
is labeled with a capacity c(u, v) ∈ IN. Let s, t ∈ V be distinguished vertices called
the source and sink, respectively. A ﬂow in G is a function f : E→IN such that the
following two properties hold:
• for all e ∈ E, 0 ≤ f(e) ≤ c(e), and
• for all v ∈ V , v = s, t,
¸
u∈V
f(u, v) =
¸
u∈V
f(v, u)
(if (u, v) ∈ E, we assume that c(u, v) = f(u, v) = 0).
The total ﬂow of f is deﬁned to be
¸
v∈V
f(v, s) −f(s, v).
The maximum ﬂow problem is to ﬁnd a ﬂow f that has maximum total ﬂow.
Algorithms for the maximum ﬂow problem include the FordFulkerson algo
rithm, the Dinic algorithm, and the EdmondsKarp algorithm.
633. Let G = (V, E) be a DAG. Devise and analyze an eﬃcient algo
rithm to ﬁnd the smallest number of directed vertexdisjoint paths that cover
all vertices (that is, every vertex is on exactly one path).
634. Suppose we are given the maximum ﬂow in a ﬂow network G =
(V, E) with source s, sink t, and integer capacities. Suppose the capacity of a
single edge is increased by one. Give an O(n +e) algorithm for updating the
maximum ﬂow, where G has n vertices and e edges.
635. Suppose we are given the maximum ﬂow in a ﬂow network G =
(V, E) with source s, sink t, and integer capacities. Suppose the capacity of a
single edge is decreased by one. Give an O(n +e) algorithm for updating the
maximum ﬂow, where G has n vertices and e edges.
Consider a ﬂow problem on a graph G = (V, E) with n vertices and e edges with all
capacities bounded above by a polynomial in n. For any given ﬂow, let the largest
edge ﬂow be
max
(u,v)∈E
f(u, v).
636. Show that two diﬀerent maximum ﬂows in G can have diﬀerent largest
edge ﬂows. Design an algorithm to ﬁnd the minimum possible
largest edge ﬂow over all maximum ﬂows in G in time O(ne
2
log n).
Sec. 13.5. Matrix Reductions 161
Suppose we are given the maximum ﬂow in a ﬂow network G = (V, E) with source
s, sink t, and integer capacities c(u, v) on each edge (u, v) ∈ E. Suppose G has n
vertices and e edges. The following modiﬁed version of the standard FordFulkerson
algorithm can be used to ﬁnd a maximum ﬂow in G.
function maxﬂowbyscaling(G, s, t)
1. c := max
(u,v)∈E
c(u, v)
2. initialize ﬂow f to zero
3. k := 2
log c
4. while k ≥ 1 do
5. while there exists an augmenting path p of capacity ≥ k do
6. augment ﬂow f along p
7. k := k/2
8. return(f)
637. Prove that maxﬂowbyscaling returns a maximum ﬂow.
638. Show that the residual capacity of a minimum cut of G is at most
2ke whenever line 4 is executed.
639. Prove that the inner while loop of lines 5–6 is executed O(e) times
for each value of k.
640. Conclude from the statements of Problems 638–639 that maxﬂow
byscaling can be implemented in time O(e
2
log c).
13.5 MATRIX REDUCTIONS
Consider two problems A and B that have time complexities T
A
(n) and T
B
(n),
respectively. Problems A and B are said to be equivalent if T
A
(n) = Θ(T
B
(n)).
Some textbooks show that matrix multiplication is equivalent (under some extra
conditions) to matrix inversion. Here are some more problems along this line:
641. Show that multiplying two n n matrices is equivalent to
squaring an n n matrix.
642. Show that multiplying two n n matrices is equivalent to cubing
an n n matrix.
643. Show that multiplying two n n matrices is equivalent to raising
an n n matrix to the fourth power.
644. Show that for all k ≥ 2, multiplying two nn matrices is equivalent
to raising an n n matrix to the kth power.
162 Chap. 13. Miscellaneous
645. Deﬁne the closure of an n n matrix A to be
¸
∞
i=0
A
i
, where A
0
is the n n identity matrix, and for k > 0, A
k
= A A
k−1
, where“” denotes
matrix multiplication. (Note that closure is not deﬁned for all matrices.) Show
that multiplying two nn matrices is equivalent to computing the closure of
an n n matrix.
646. Show that multiplying two nn matrices is equivalent to multiplying
two n n lower triangular matrices.
647. Show that multiplying two n n matrices is equivalent to squaring
an n n lower triangular matrix.
648. Show that multiplying two n n matrices is equivalent to cubing an
n n lower triangular matrix.
649. Show that multiplying two n n matrices is equivalent to raising an
n n lower triangular matrix to the fourth power.
650. Show that for all k ≥ 2, multiplying two n n matrices is equivalent
to raising an n n lower triangular matrix to the kth power.
651. Show that multiplying nn matrices is equivalent to computing the
closure of an nn lower triangular matrix. (See Problem 645 for the deﬁnition
of matrix closure.)
13.6 GENERAL
652. You are facing a high wall that stretches inﬁnitely in both directions.
There is a door in the wall, but you don’t know how far away or in which
direction. It is pitch dark, but you have a very dim lighted candle that will
enable you to see the door when you are right next to it. Show that there
is an algorithm that enables you to ﬁnd the door by walking at most O(n)
steps, where n is the number of steps that you would have taken if you knew
where the door is and walked directly to it. What is the constant multiple in
the bigO bound for your algorithm?
653. You are given an n 2 matrix of integers. You are allowed
to permute the rows of the matrix as monolithic items, and to perform com
parisons on the elements of the matrix. Your task is to ﬁnd an O(nlog n)
time algorithm that permutes the rows so that neither column of the ma
trix contains an increasing subsequence (not necessarily contiguous) of length
exceeding
√
n.
654. It’s Texas Chile Pepper day and you have a schedule of events that
looks something like this:
Sec. 13.6. General 163
8:30 – 9:15 Pepper Pancake Party
9:30 – 10:05 Pepper Eating Contest
9:50 – 11:20 Pepper Personality Proﬁle
9:00 – 11:00 Pepper Parade
.
.
.
The schedule lists the times at which each event begins and ends. The events
may overlap. Naturally, you want to attend as many events as you can. Design
an eﬃcient algorithm to ﬁnd the largest set of nonoverlapping events. Prove
that your algorithm is correct. How fast is it?
655. Let A be an n n matrix of zeros and ones. A submatrix S of
A is any group of contiguous entries that forms a square, more formally,
¦A[i, j] [ ≤ i ≤ +k −1, m ≤ j ≤ m+k −1¦ for some 1 ≤ , m ≤ n, 0 ≤
k ≤ min(n − + 1, n −m+ 1). S is said to have size k. Design an algorithm
that determines the size of the largest submatrix of ones in A in time O(n
2
).
656. We wish to compute the laziest way to dial an ndigit number on a
standard pushbutton telephone using two ﬁngers. We assume that the two
ﬁngers start out on the ∗ and # keys, and that the eﬀort required to move a
ﬁnger from one button to another is proportional to the Euclidean distance
between them. Design and analyze an algorithm that computes in time O(n)
the method of dialing that involves moving your ﬁngers the smallest amount
of total distance.
657. Suppose S = ¦1, 2, . . . , n¦ and f : S→S. If R ⊂ S, deﬁne f(R) =
¦f(x) [ x ∈ R¦. Devise an O(n) time algorithm for determining the largest
R ⊂ S such that f(R) = R.
658. Let E be a set of m linear equations of the form x
i
= x
j
+ c
ij
over
the variables x
1
, . . . , x
n
(c
i,j
∈ ZZ for all 1 ≤ i, j ≤ n). Devise an O(m) time
algorithm for determining whether the equations in E are consistent, that is,
whether an assignment of integers can be made to the variables so that all of
the equations in E are satisﬁed.
The following group of questions deals with the generalized version of the popular
Sam Loyd 15puzzle, called the (n
2
−1)puzzle. For deﬁnitions, see Problem 515.
659. Design an algorithm for the (n
2
−1)puzzle that runs in time
O(n
3
). Your algorithm must input a legal state of the puzzle and output a
series of O(n
3
) moves that solves it.
660. Show that the worst case number of moves needed to solve the
(n
2
−1)puzzle is at least n
3
−O(n
2
).
164 Chap. 13. Miscellaneous
661. Show that the average case number of moves needed to solve the
(n
2
−1)puzzle is at least 2n
3
/3 −O(n
2
).
662. Show that for some constants 0 < α, β < 1, a random legal con
ﬁguration of the (n
2
− 1)puzzle requires at least αn
3
moves to solve with
probability 1 −1/e
−βn
2
.
Suppose you are given a collection of gold coins that includes a clever counterfeit
(all the rest are genuine). The counterfeit coin is indistinguishable from the real
coins in all measurable characteristics except weight. You have a scale with which
you can compare the relative weight of coins. However, you do not know whether
the counterfeit coin is heavier or lighter than the real thing. Your problem is to ﬁnd
the counterfeit coin and determine whether it is lighter or heavier than the rest.
663. Show that if there are 12 coins, the counterfeit coin can be
found in three weighings.
664. Show that if there are 39 coins, the counterfeit coin can be found in
four weighings.
665. Show that if there are n ≥ 3 coins, the counterfeit coin can be
found in log
3
2n weighings.
666. Show that if there are n ≥ 3 coins, at least log
3
2n weighings are
needed to ﬁnd the counterfeit coin.
Your job is to seat n rambunctious children in a theater with n balconies. You are
given a list of m statements of the form “i hates j.” If i hates j, then you do not
want to seat i above or in the same balcony as j, otherwise i will throw popcorn at
j instead of watching the play.
667. Give an algorithm that assigns the balconies (or says that it is not
possible) in time O(m+n).
668. Design and analyze an algorithm for ﬁnding the minimum number of
balconies needed.
13.7 HINTS
614. Show that if the search for two diﬀerent elements x, y, where x < y, in the set
leads to the same leaf of the comparison tree, then the search for any element
a of the universe with x < a < y must also lead to that leaf, which contradicts
the correctness of the algorithm.
Sec. 13.7. Hints 165
627. Use both properties from Problem 626.
628. The cost of the path from v to u along c is −x.
629. Show that a mincost path to any vertex on the minimum meancost cycle
can be extended along the cycle to the next vertex on the cycle.
633. Suppose V = ¦v
1
, . . . , v
n
¦. Form a network G
= (V
, E
) as follows.
V
= ¦s, t¦ ∪ ¦x
1
, . . . , x
n
¦ ∪ ¦y
1
, . . . , y
n
¦,
E
= ¦(s, x
i
) [ 1 ≤ i ≤ n¦ ∪ ¦(y
i
, t) [ 1 ≤ i ≤ n¦ ∪ ¦(x
i
, y
i
) [ (v
i
, v
j
) ∈ E¦.
The capacity of all edges is one. Show that the minimum number of paths
that cover V in G is equal to n −f, where f is the maximum ﬂow in G
.
636. Consider the EdmondsKarp algorithm for maximum ﬂow.
641–645. Consider Problems 647–651. The lower triangular property can actually
make the problem easier, if you think about it for long enough. On the other
hand, there do exist nontriangular solutions, as the solution to Problem 641
given later demonstrates.
659. Use a combination of greedy and divideandconquer techniques. Suppose the
hole is immediately to the right of a square S. Let L, R, U, D denote moving
the hole one place left, right, up, and down, respectively. Then the sequence
of moves ULDLUR moves S one square diagonally up and to the right. Use
this sequence of moves, and others like it, to move each square to its correct
space, stopping when there is not enough space left to perform the maneuver.
Figure out how to ﬁnish it, and you’ve solved the problem! The best I can
do is 5n
3
−Ω(n
2
) moves. Make sure that the conﬁguration you come up with
is solvable. It can be shown that if the hole is in the lower right corner, the
solvable states of the puzzle are exactly those that can be made with an even
number of transpositions.
660. The Manhattan distance of a tile in square (i, k) that belongs in square (k, )
is deﬁned to be [i − k[ + [j − [. The Manhattan distance of a conﬁguration
of the puzzle is deﬁned to be the sum of the Manhattan distances of its tiles.
Show that the Manhattan distance is a lower bound on the number of moves
needed to solve the puzzle, and that there exists a conﬁguration that has
Manhattan distance at least n
3
−O(n
2
).
661. Find a lower bound for the average Manhattan distance.
662. Find a lower bound for the Manhattan distance of a random conﬁguration.
The best value that I can get for α is 16/243. You will need to use Chernoﬀ
bounds. Let B(m, N, p) be the probability of obtaining at least m successes
166 Chap. 13. Miscellaneous
out of N Bernoulli trials, each with probability p of success. The following
result is a wellknown consequence of the Chernoﬀ bounds (see, for example,
Angluin and Valiant [5] and Valiant and Brebner [80]): Let β = m/Np − 1.
If 0 ≤ β ≤ 1, then β(m, N, p) ≤ e
−β
2
Np/2
.
663. Start by dividing the coins into three piles of four coins. Pick two piles and
compare their weights.
665. Start by solving Problems 663 and 664. Then generalize your solution.
13.8 SOLUTIONS
641. The trick is to embed A and B into a larger matrix that, when squared, has
AB has a submatrix. The solution can then be read oﬀ easily in optimal (that
is, O(n
2
)) time. There are many embeddings that work, including
¸
0 A
B 0
2
=
¸
AB 0
0 AB
.
663. Actually, I’m not going to give you a solution. This problem is so popular
with mathematicians that you ought to be able to ﬁnd one in the literature
somewhere. It even appears in a science ﬁction novel by Piers Anthony [6,
Chapter 16].
13.9 COMMENTS
610. The nuts and bolts problem is from Rawlins [66].
611. This algorithm is mentioned in Alon et al. [4].
653. This problem was suggested to the author by Arny Rosenberg. Interestingly,
one can prove by a counting argument that one can so permute an nk matrix,
but an actual eﬃcient algorithm is known only for k = 1, 2. For k = 1, this
is a relative of the Erd¨osSzekeres Theorem , which states that there exists
in every sequence of n integers a monotonic subsequence (either ascending or
descending) of length
√
n (for more recent proofs of this theorem, see, for
example, Dijkstra [22], and Seidenberg [73]).
659. Horden [36] gives an interesting history of this puzzle and its variants. See
also Gardner [27]. Early work on the 15puzzle includes Johnson [38] and
Storey [76]. The minimum number of moves to solve each legal permutation
of the 8puzzle has been found by exhaustive search by Schoﬁeld [69]. The
minimum number of moves needed to solve the 15puzzle in the worst case
is unknown, but has been the subject of various papers in the AI literature,
Sec. 13.9. Comments 167
including Michie, Fleming and Oldﬁeld [57] and Korf [48]. Ratner and War
muth [65] have proved that the problem of determining the minimum length
number of moves for any given legal conﬁguration of the (n
2
− 1)puzzle is
^{complete, and they demonstrate an approximation algorithm that makes
no more than a (fairly large) constant factor number of moves than necessary
for any given legal conﬁguration. Kornhauser, Miller, and Spirakis [49] have
shown an algorithm for the (n
2
−1)puzzle and its generalizations that always
runs in time O(n
3
).
Bibliography
[1] A. V. Aho, J. E. Hopcroft, and J. D. Ullman. The Design and Analysis of
Computer Algorithms. AddisonWesley, 1974.
[2] A. V. Aho, J. E. Hopcroft, and J. D. Ullman. Data Structures and Algorithms.
AddisonWesley, 1983.
[3] A. V. Aho and J. D. Ullman. Foundations of Computer Science. Computer
Science Press, 1992.
[4] N. Alon, M. Blum, A. Fiat, S. Kannan, M. Naor, and R. Ostrovsky. Matching
nuts and bolts. In Proc. 5th Annual Symposium on Discrete Algorithms, pages
690–696, 1994.
[5] D. Angluin and L. Valiant. Fast probabilistic algorithms for Hamiltonian cir
cuits and matchings. In Proceedings of the Ninth Annual ACM Symposium on
Theory of Computing, pages 30–41. ACM Press, 1977.
[6] P. Anthony. With a Tangled Skein, volume 3 of Incarnations of Immortality.
Ballantine Books, 1985.
[7] S. Baase. Computer Algorithms: Introduction to Design and Analysis. Addison
Wesley, 1978.
[8] W. W. Rouse Ball and H. S. M. Coxeter. Mathematical Recreations and Essays.
University of Toronto Press, 12th edition, 1974.
[9] J. Bentley. Programming Pearls. AddisonWesley, 1986.
[10] J. Bentley. More Programming Pearls: Confessions of a Coder. Addison
Wesley, 1988.
[11] B. Bernhardsson. Explicit solutions to the nqueens problem for all n. SIGART
Bulletin, 2(2):7, 1991.
168
Bibliography 169
[12] A. Blum and R. L. Rivest. Training a 3node neural network is NPcomplete. In
Neural Information Processing Systems 1, pages 494–501. Morgan Kaufmann,
1989.
[13] A. Blum and R. L. Rivest. Training a 3node neural network is NPcomplete.
Neural Networks, 5(1):117–127, 1992.
[14] G. Brassard and P. Bratley. Algorithmics: Theory and Practice. Prentice Hall,
1988.
[15] A. Conrad, T. Hindrichs, H. Morsy, and I. Wegener. Wie es dem springer
gelang, schachbretter beliebiger groesse und zwischen beliebig vorgegebe
nen anfangs und endfeldern vollstaendig abzuschreiten. Spektrum der Wis
senschaft, pages 10–14, February 1992.
[16] A. Conrad, T. Hindrichs, H. Morsy, and I. Wegener. Solution of the knight’s
Hamiltonian path problem on chessboards. Discrete Applied Mathematics,
50:125–134, 1994.
[17] S. A. Cook. The complexity of theorem proving procedures. In Proceedings of
the Third Annual ACM Symposium on Theory of Computing, pages 151–158.
ACM Press, 1971.
[18] D. Coppersmith and S. Winograd. Matrix multiplication via arithmetic pro
gressions. Journal of Symbolic Computation, 9:251–280, 1990.
[19] T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms.
MIT Press, 1990.
[20] P. Cull and J. DeCurtins. Knight’s tour revisited. Fibonacci Quarterly, 16:276–
285, 1978.
[21] E. W. Dijkstra. A Discipline of Programming. Prentice Hall, 1976.
[22] E. W. Dijkstra. Some beautiful arguments using mathematical induction. Acta
Informatica, 13:1–8, 1980.
[23] P. Eades and B. McKay. An algorithm for generating subsets of ﬁxed size
with a strong minimal change property. Information Processing Letters, pages
131–133, 1984.
[24] L. Euler. Solution problematis ad geometriam situs pertinentis. Comentarii
Academiae Scientarum Petropolitanae, 8:128–140, 1736.
[25] L. Euler. Solution d’une question curieuse qui ne paroit soumise ` a aucune
analyse. Mem. Acad. Sci. Berlin, pages 310–337, 1759.
[26] S. Even. Graph Algorithms. Pitman, 1979.
170 Bibliography
[27] M. Gardner. The Mathematical Puzzles of Sam Loyd. Dover, 1959.
[28] M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the
Theory of NPCompleteness. W. H. Freeman, 1979.
[29] A. M. Gibbons. Algorithmic Graph Theory. Cambridge University Press, 1985.
[30] R. L. Graham, D. E. Knuth, and O. Patashnik. Concrete Mathematics: A
Foundation for Computer Science. AddisonWesley, 1989.
[31] R. L. Graham, B. L. Rothschild, and J. H. Spencer. Ramsey Theory. John
Wiley & Sons, 1990.
[32] R. L. Graham and J. H. Spencer. Ramsey theory. Scientiﬁc American, 263(1),
July 1990.
[33] D. H. Greene and D. E. Knuth. Mathematics for the Analysis of Algorithms.
Birkh¨auser, 1982.
[34] D. Harel. Algorithmics: The Spirit of Computing. AddisonWesley, 1987.
[35] B. R. Heap. Permutations by interchanges. Computer Journal, 6:293–294,
1963.
[36] L. E. Horden. Sliding Piece Puzzles. Oxford University Press, 1986.
[37] E. Horowitz and S. Sahni. Fundamentals of Computer Algorithms. Computer
Science Press, 1978.
[38] W. A. Johnson. Notes on the 15 puzzle 1. American Journal of Mathematics,
2(4):397–399, 1879.
[39] J. S. Judd. Learning in networks is hard. In Proc. of the First International
Conference on Neural Networks, pages 685–692. IEEE Computer Society Press,
1987.
[40] J. S. Judd. Neural Network Design and the Complexity of Learning. PhD
thesis, University of Massachusetts, Amherst, MA, 1988.
[41] J. S. Judd. On the complexity of loading shallow neural networks. Journal of
Complexity, 4:177–192, 1988.
[42] J. S. Judd. Neural Network Design and the Complexity of Learning. MIT Press,
1990.
[43] R. M. Karp. Reducibility among combinatorial problems. In R. E. Miller
and J. W. Thatcher, editors, Complexity of Computer Computations. Plenum
Press, New York, 1972.
Bibliography 171
[44] J. H. Kingston. Algorithms and Data Structures: Design, Correctness, Analy
sis. AddisonWesley, 1990.
[45] D. E. Knuth. Fundamental Algorithms, volume 1 of The Art of Computer
Programming. AddisonWesley, second edition, 1973.
[46] D. E. Knuth. Sorting and Searching, volume 3 of The Art of Computer Pro
gramming. AddisonWesley, 1973.
[47] D. E. Knuth. Seminumerical Algorithms, volume 2 of The Art of Computer
Programming. AddisonWesley, second edition, 1981.
[48] R. E. Korf. Depthﬁrst iterative deepening: An optimal admissible tree search.
Artiﬁcial Intelligence, 27(1):97–109, 1985.
[49] D. Kornhauser, G. Miller, and P. Spirakis. Coordinating pebble motion on
graphs, the diameter of permutation groups, and applications. In 25th An
nual Symposium on Foundations of Computer Science, pages 241–250. IEEE
Computer Society Press, 1984.
[50] D. C. Kozen. The Design and Analysis of Algorithms. SpringerVerlag, 1992.
[51] C. W. H. Lam and L. H. Soicher. Three new combination algorithms with the
minimal change property. Communications of the ACM, 25(8), 1982.
[52] Lewis and Denenberg. Data Structures and Their Algorithms. Harper Collins,
1991.
[53] J.H. Lin and J. S. Vitter. Complexity results on learning by neural nets.
Machine Learning, 6:211–230, 1991.
[54] X.M. Lu. Towers of Hanoi problem with arbitrary k ≥ 3 pegs. International
Journal of Computer Mathematics, 24:39–54, 1988.
[55] E. Lucas. R´ecr´eations Math´ematiques, volume 3. GauthierVillars, 1893.
[56] U. Manber. Introduction to Algorithms: A Creative Approach. AddisonWesley,
1989.
[57] D. Michie, J. G. Fleming, and J. V. Oldﬁeld. A comparison of heuristic, inter
active, and unaided methods of solving a shortestroute problem. In D. Michie,
editor, Machine Intelligence 3, pages 245–255. American Elsevier, 1968.
[58] B. M. E. Moret and H. D. Shapiro. Design and Eﬃciency, volume 1 of Algo
rithms from P to NP. Benjamin/Cummings, 1991.
[59] C. H. Papadimitriou and K. Steiglitz. Combinatorial Optimization: Algorithms
and Complexity. Prentice Hall, 1982.
172 Bibliography
[60] I. Parberry. On the computational complexity of optimal sorting network ver
iﬁcation. In Proceedings of The Conference on Parallel Architectures and Lan
guages Europe, in Series Lecture Notes in Computer Science, volume 506, pages
252–269. SpringerVerlag, 1991.
[61] I. Parberry. On the complexity of learning with a small number of nodes.
In Proc. 1992 International Joint Conference on Neural Networks, volume 3,
pages 893–898, 1992.
[62] I. Parberry. Algorithms for touring knights. Technical Report CRPDC947,
Center for Research in Parallel and Distributed Computing, Dept. of Computer
Sciences, University of North Texas, May 1994.
[63] I. Parberry. Circuit Complexity and Neural Networks. MIT Press, 1994.
[64] P. W. Purdom Jr. and C. A. Brown. The Analysis of Algorithms. Holt, Rine
hart, and Winston, 1985.
[65] D. Ratner and M. K. Warmuth. The (n
2
− 1)puzzle and related relocation
problems. Journal for Symbolic Computation, 10:11–137, 1990.
[66] G. Rawlins. Compared to What? An Introduction to the Analysis of Algorithms.
Computer Science Press, 1991.
[67] G. J. Rawlins. Compared to What?: An Introduction to the Analysis of Algo
rithms. Computer Science Press, 1992.
[68] T. J. Schaefer. The complexity of satisﬁability problems. In Proceedings of
the Tenth Annual ACM Symposium on Theory of Computing, pages 216–226.
ACM Press, 1978.
[69] P. D. A. Schoﬁeld. Complete solution of the eight puzzle. In N. L. Collins and
D. Michie, editors, Machine Intelligence 1, pages 125–133. American Elsevier,
1967.
[70] A. Sch¨ onhage and V. Strassen. Schnelle multiplikation grosser zahlen. Com
puting, 7:281–292, 1971.
[71] A. J. Schwenk. Which rectangular chessboards have a knight’s tour? Mathe
matics Magazine, 64(5):325–332, 1991.
[72] R. Sedgewick. Algorithms. AddisonWesley, 1983.
[73] A. Seidenberg. A simple proof of a theorem of Erd¨ os and Szekeres. Journal of
the London Mathematical Society, 34, 1959.
[74] J. D. Smith. Design and Analysis of Algorithms. PWSKent, 1989.
[75] D. Solow. How to Read and Do Proofs. Wiley, second edition edition, 1990.
Bibliography 173
[76] W. E. Storey. Notes on the 15 puzzle 2. American Journal of Mathematics,
2(4):399–404, 1879.
[77] R. Susic and J. Gu. A polynomial time algorithm for the nqueens problem.
SIGART Bulletin, 1(3):7–11, 1990.
[78] Y. Takefuji. Neural Network Parallel Computing. Kluwer Academic Publishers,
1992.
[79] Y. Takefuji and K. C. Lee. Neural network computing for knight’s tour prob
lems. Neurocomputing, 4(5):249–254, 1992.
[80] L. G. Valiant and G. J. Brebner. A scheme for fast parallel communication.
SIAM Journal on Computing, 11(2):350–361, 1982.
[81] J. Veerasamy and I. Page. On the towers of Hanoi problem with multiple spare
pegs. International Journal of Computer Mathematics, 52:17–22, 1994.
[82] M. A. Weiss. Data Structures and Algorithm Analysis. Benjamin/Cummings,
1992.
[83] H. S. Wilf. Algorithms and Complexity. Prentice Hall, 1986.
Index
8puzzle, 166
15puzzle, 124, 163, 166
2–3 tree, 135, 141, 146
2colorable, 79
2SAT, 148
3SAT, 150
balanced, see B3SAT
oneinthree, see O3SAT1, O3SAT2,
O3SAT3
parity, see parity 3SAT
strong, see strong 3SAT
accept, 151, 152
acyclic, 79, 81
addition, 47, 52, 59, 61–66, 69, 74, 81
adjacency
list, 82
matrix, 158
adversary, 157
airport, 103
allpairs shortestpath, 91
αpseudomedian, see pseudomedian
amortized
analysis, ix, 145
time, 142
ancestor, 145, 158
approximation algorithm, ix, 167
arbitrage, 97
architecture, 152
area, 25
areas, method of, see method of areas
arithmetic expression, 79
ascending, 80
assignment, 116, 120, 121, 123
operator, 3
satisfying, see satisfying assign
ment
truth, see truth assignment
average, 74, 77, 157, 158, 164, 165
AVL tree, 135, 140–141, 145, 146
B3SAT, 154
backtracking, 6, 116, 122–123, 126,
127, 133
balance, 135, 145
balanced oneinthree 3SAT, see B3SAT
balcony, 164
bank, 97
Bernoulli trial, 166
biconnected, 78
bin packing problem, 143–144
binary, 19, 24, 25, 152
comparison, 137
heap, 137
reﬂected Gray code, 133
search, 74–75, 85
tree, 90, 135, 138–140
string, 116, 118, 123
tree, 18, 135, 136, 138
binomial, 14
bolt, see nuts and bolts
Brahma, 86
breadthﬁrst
search tree, 150
traversal, 136
bubblesort, 50, 55, 62, 64, 158
174
Index 175
capacity, 144, 160, 161, 165
ceiling, 10
Chernoﬀ bound, 165, 166
chessboard, 12–14, 25, 26, 80, 82, 86,
124, 125
child, 18, 108, 135–138, 141, 142, 145,
146
Chomsky Normal Form, 92
circle, 18, 20
circuit, 152–153
cyclic, see cyclic circuit
city, 103, 126, 158
clause, 147, 148, 150, 153, 154
clique, 123, 124
problem, 123
closed knight’s tour, 12, 25, 80, 82, 86,
124, 133
closure, 162
coin, 19, 96, 98, 110, 111, 164, 166
coinchanging problem, 19, 96, 98, 110
color, 12, 14–18, 23, 26, 78, 79
colorable, 79
combination, 116, 120–121, 131
Communications of the ACM, 5
comparisonbased algorithm, 68, 157–
158
complete binary tree, 18, 135
connected, 16, 18, 19, 26, 81, 102, 103,
106, 150
component, 78
dominating set, 150
contextfree grammar, 92
continuous knapsack problem, 101–102
converge, 153
convex, 18
Cook reduction, 148–150
cost
max, see max cost
mean, see mean cost
min, see mincost
counterfeit, 164
cycle, 16, 18, 20, 26, 103, 109, 153,
158, 159, 165
Eulerian, see Eulerian cycle
Hamiltonian, see Hamiltonian cy
cle
cyclic, 124
circuit, 153
shift, 118, 119
DAG, 79, 160
decision
problem, 147, 149, 152
tree, 157
degree, 16, 20, 26, 79, 81, 82, 110
out, see outdegree
delete, 82, 90, 96, 136, 137, 140, 141,
143, 144, 146
deletemax, 137
deletemin, 104, 107, 136–138, 140
dense, 101
density, 101
depthﬁrst
search, 78–79
spanning tree, 78
descendant, 135, 146
dial, 163
Dijkstra’s algorithm, 102–103
dimension, 16, 17
Dinic algorithm, 160
directed spanning tree, 108
dirty, 137–139
disjunctive normal form, 148, 150
divideandconquer, x, 2, 6, 67–87, 92,
116, 122, 136, 165
division, 48, 61
DNF SAT, 148
dominating set, 127, 128, 131, 150
connected, see connected domi
nating set
double rotation, 140, 141
dynamic programming, x, 2, 6, 7, 87–
100, 115, 151
EdmondsKarp algorithm, 160, 165
elementary operation, 59, 62, 64
Erd¨ osSzekeres Theorem, 166
Euclidean distance, 163
176 Index
Euler, Leonhard, 26, 133
Eulerian cycle, 16, 20, 26, 81
exhaustive search, x, 2, 116–134, 166
existential, 147
exponential, 87
time, x, 2
exponentiation, 49, 53, 61, 63
factorial, 49, 53, 61, 63
factorization, 154
feedback loop, 152
Fermat’s last theorem, 150
Fibonacci numbers, 14, 38, 51, 54, 58,
62, 64–66
ﬁrstﬁt heuristic, 143–144
ﬂoor, 10
ﬂow, 7, 160
maximum, see maximum ﬂow
Floyd’s Algorithm, 91–92
FordFulkerson algorithm, 160, 161
forest, 79, 104, 108
spanning, see spanning forest
Gauss, 23, 24
geometry, ix, 18
grammar, see contextfree grammar
graph, x, 5, 6, 16–18, 20, 26, 78, 79,
81, 82, 91, 92, 97, 102–104,
106–110, 123, 124, 126, 127,
131, 149, 150, 158, 160
Gray code, 19, 20, 133
greedy algorithm, x, 2, 6, 100–115,
165
halving, 136, 137
Hamiltonian
cycle, 16, 109, 127, 131, 149, 150
path, 16
Hanoi, see towers of Hanoi
heap, 135–138, 145
Heap’s algorithm, 119, 120, 132, 133
heapsort, 136, 137
height, 140, 141, 145, 146
heuristic, 133
ﬁrstﬁt, see ﬁrstﬁt heuristic
HiQ, 124–125
Horner’s rule, 51
horse, 15, 26
hypercube, 16–17
ice cream cart, 126
increment, 47, 52, 61, 63, 117, 146
independent set, 124
induced subgraph, 81, 123
induction, x, 1, 2, 8–27, 35, 38, 41–46,
51, 52, 57, 66, 82–85, 111,
130, 131, 136, 145, 146
Journal of Algorithms, 5
Karp
reduction, 148
Richard M., 148, 160, 165
knapsack problem, 89, 97, 99, 150–
151
continuous, see continuous knap
sack problem
zeroone, see zeroone knapsack
problem
knight, 12
knight’s tour
closed, see closed knight’s tour
open, see open knight’s tour
Kruskal’s algorithm, 104, 106–108
language, 92, 100, 152
programming, see programming
language
string processing, see string pro
cessing language
level, 18, 135, 145
literal, 147, 148, 150, 153
loading, 152, 153, 155
longest ascending subsequence, 80, 93,
99
loop invariant, 46, 55–58
Loyd, Sam, 124, 163
Manhattan distance, 165
Index 177
marble, 122
matrix, 19, 87
adjacency, see adjacency matrix
inversion, 161
multiplication, 50, 62, 73–74, 161–
162
product, 19, 87, 88, 97
weighing, see weighing matrix
max cost spanning tree, 108
maximum
ﬂow, 160–161, 165
subsequence sum, 80
MAXMIN, 67, 68
mean cost, 158
median, 75, 76, 136, 137, 156, 157
weighted, see weighted median
merge, 81
method of areas, 25
mincost spanning tree, 102–108, 111
minimalchange, 117–121, 131–133
monom, 148
(n
2
−1)puzzle, 124, 163, 164, 167
nqueens problem, 125, 133–134
neural network, 133
nonterminal symbol, 92, 99, 100
^{, 147, 149–152, 154, 155
^{complete, x, 3, 5, 7, 146–155
nuts and bolts, 156–157, 166
O3SAT1, 153–155
O3SAT2, 153–154
O3SAT3, 153–155
oneinthree 3SAT, see O3SAT1, O3SAT2,
O3SAT3
oneprocessor scheduling problem, 110
open knight’s tour, 124
optimal, 86, 88, 93, 94, 98, 111, 118,
120, 122, 123, 132, 166
binary search tree, 90
tapestorage problem, 110
outdegree, 79, 127
{, 147–152, 154, 155
parallel
algorithm, ix
heap, 137, 138
lines, 19
priority queue, 137
parent, 98, 108, 135, 137, 142, 145,
146
parenthesize, 38, 87, 97, 99
parity 3SAT, 148
Pascal, x, 3–5, 69, 134
path, 16, 18, 79, 91, 92, 97, 102, 103,
106, 140, 141, 146, 158–160,
165
compression, 142, 145
Hamiltonian, see Hamiltonian path
patternmatching, 50, 62
peaceful queens problem, see nqueens
problem
perfect matching, 121, 122
permutation, 110, 113–116, 118–120,
123, 130–133, 158, 166
Pigeonhole Principle, 19
pivot, 75–77
polygon, 18
polynomial time, 147–152, 154
manyone reduction, 148
reducible, 147
Turing reduction, 148
polyomino, 130
popcorn, 164
postage stamp, 11–12, 22, 129
predecessor, 81
Prim’s algorithm, 103, 104, 106–108
prime, 154
priority, 144
queue, 103
parallel, see parallel priority queue
product rule, 36, 59
production, 92, 100
programming language, x
prune, 122, 123
pseudomedian, 75, 76
queen, 125
178 Index
quicksort, 55, 64, 66, 75–77, 85
Ramsey number, 124, 133
Ramsey’s Theorem, 124
recurrence relation, x, 1, 37–45, 62,
65–67, 70, 75, 76, 81, 95, 130,
145
recursion
tree, 122, 123
reﬂection, 130
Reimann hypothesis, 150
remaindering, 48, 61
repeated substitution, 41–45, 65
root, 17, 18, 35, 92, 98, 100, 136, 138,
140–142, 145, 146, 150
rooted tree, 17
rotation, 130
double, see double rotation
single, see single rotation
salesperson
traveling, see traveling salesper
son
SAT, 147, 149
DNF, see DNF SAT
satisﬁability problem, see SAT
satisfying assignment, 150, 151
savings certiﬁcate, 97
schedule, 110, 111, 162, 163
selfreducibility, 149–150
semigroup, 144
sentinel, 117
sibling, 137, 138
single rotation, 140, 141
singlesource
longestpaths, 103
shortestpaths, 102
sink, 158, 160, 161
Smarandache function, 154
sort, 50, 55, 62, 64, 66, 75–77, 80, 81,
85, 101, 104, 136–138, 141,
156–158
source, 159–161
spanning
forest, 104, 108
tree, 18, 102
depthﬁrst, see depthﬁrst span
ning tree
directed, see directed spanning
tree
maxcost, see maxcost span
ning tree
mincost, see mincost spanning
tree
squareroot, 74
stable conﬁguration, 153
stamp, see postage stamp
Stirling’s approximation, 26, 157
Strassen’s algorithm, 73–74
string processing language, 93
strong 3SAT, 148
sum rule, 36, 59
task, 152
telephone, 163
terminal symbol, 92
Texas Chile Pepper Day, 162
tile, 12, 14, 124, 165
toll, 144
tournament, 16
towers of Hanoi, 77–78
travel agent, 103
traveling salesperson, 109, 149–150
treasury bond, 97
tree, 17–18, 26, 79, 98, 137, 141, 142,
145, 150, 158, 164
23, see 23 tree
AVL, see AVL tree
binary, see binary tree
complete, see complete binary
tree
decision, see decision tree
recursion, see recursion tree
spanning, see spanning tree
triangle, 19–21, 79
triconnected, 158
triomino, 12–14
truck, 158
Index 179
truth assignment, 148, 149, 152–154
Turing machine, 151
unionﬁnd, 135, 142, 145
vertex cover, 110
warehouse, 94, 95, 98
weighing matrix, 128
weighted median, 80
zeroone knapsack problem, 89
Errata to “Problems on Algorithms”
∗
Ian Parberry
†
Department of Computer Sciences
University of North Texas
April 7, 1998
This note contains corrections to Parberry [5]. An uptodate copy of this document and
other useful information can be found on the WorldWide Web (Parberry [7]). I welcome
further errata and comments from readers — I prefer electronic mail, but if you must use
more primitive modes of communication, full contact information can be found in [6].
Then anyone who leaves behind him a written manual, and likewise anyone who
receives it, in the belief that such writing will be clear and certain, must be ex
ceedingly simpleminded.
— Plato, Phaedrus
General Errata
Apparently I couldn’t make up my mind whether the keyword “return” in my algorithms
should be in bold face or Roman type. It appears in bold face in Chapters 5, 7–9, and 13;
in Roman face in Chapters 5, 6, and 10. Out of 71 occurrences, 39 are in bold face.
A foolish consistency is the hobgoblin of little minds.
— Ralph Waldo Emerson
Speciﬁc Errata
Page 3, Line 1: The ﬁrst comma should be a period.
Page 6, Line 14: The second “but” should be deleted.
Page 7, Last Paragraph: Wilf’s textbook is now out of print, and is very diﬃcult to
obtain in its original form. Fortunately, the author has made it generally available in
postscript form from http://www.cis.upenn.edu/~wilf/index.html.
∗
Copyright c Ian Parberry. This Errata may be reproduced without fee provided that the copies are not
made for proﬁt, and the full text, including the author information and this notice, are included.
†
Author’s address: Department of Computer Sciences, University of North Texas, P.O. Box 13886, Den
ton, TX 762036886, U.S.A. Electronic mail: ian@cs.unt.edu.
1
Page 10, Problem 22: That should be induction on n ≥ 2. If n = 0, the right side of
inequality doesn’t make sense, and if n = 1 equality holds.
Page 12, Problem 47: This problem does not have a comment associated with it; the
should be deleted.
Page 16, Line 10: In the set V , the ﬁrst occurrence of v
2
should be v
1
.
Page 16, Problem 67: I intended the graph here to be undirected and with no selfloops,
but this did not make it into the deﬁnition. I need to add to the deﬁnition of a graph
at the start of Section 2.10 the extra property that V is an ordered set and for all
(u, v) ∈ E, u < v.
Page 20, Hint to Problem 37: The comma at the end of the second sentence should be
a period.
Page 21, Solution to Problem 8: The fourth line of the multiline equation should begin
with “=”, not “−”.
Page 22, Solution to Problem 37, Line 5: Change “all values up to n−1 cents” to “all
values from 8 to n − 1 cents”.
Page 25, Comment to Problem 13, Line 6: Change “by Problem 1” to “by Problem 11”.
Page 35, Solution to Problem 133: On line 4, the denominator of the fraction should
be 6, not 2.
Page 40, Problem 241: This problem should ask you to show that 2n − 3log n − 1 ≤
T(n) ≤ 2n − log n − 1.
Page 40, Problem 242: That should be T(1) = 0, and T(n) = T(n/2)+T(n/2)+n−1
(otherwise it is inconsistent with the answer given on p. 41); it is the number of
comparisons used by mergesort.
Page 45, Line 3: “skip it is” should read “skip it if”.
Page 52, Problem 266: In Line 1 of the algorithm, the Roman “n” in the return statement
should be an italic “n”.
Page 52, Problem 269: In Line 2 of the algorithm, the ﬁnal Roman “y” in the return
statement should be an italic “y”.
Page 56, Hint to Problem 258: The loop invariant is x
j
=
y
0
k=y
j−1
k, for all j ≥ 1.
Page 56, Solution to Problem 250:
Line 4: “line 8” should read “line 7”.
Line 6: “lines 3–7” should read “lines 3–6”.
Line 14: “line 6” should read “line 5”.
Line 15: Delete the parenthetical comment “(see Problem 93)”.
Line 17: “line 5” should read “line 4”.
Line 15: Insert “(see Problem 93)” immediately before the period.
Page 64, Hint to Problem 283: Insert a period after the problem number.
Page 62, Problem 299: “line 3” should read “line 5”.
Page 68, Problems 317–318: When I wrote Problem 317 I assumed that the standard
recursive algorithm described at the start of Section 7.1 achieved this bound. But it
doesn’t — it requires 5n/3 − 2 comparisons when n = 3 · 2
k
. To ﬁx it, simply ensure
that the sizes of the two recursive splits are not both odd. (Thanks to Mike Paterson
for pointing all this out). Problem 317 should be rewritten, or a hint added to help
2
make this clearer. The algorithm of Problem 318 almost meets the required bound.
It makes 2 more comparisons than necessary, and should be rewritten slightly to save
them (hint: consider the second and third comparisons).
Page 68, Problem 320: The second question is not worded very clearly. The algorithm
could be called upon to solve duplicate subproblems at the ﬁrst level of recursion, given
suitable inputs (can you ﬁnd one?), but I was looking for more. There comes a point
before the bottommost level of recursion at which the algorithm is obliged to solve
duplicate subproblems. This is also the key to the algorithm in Problems 328330.
Page 68, Problem 321: This problem has a comment, but is missing the .
Page 68, Problem 322: The solution to this problem is a little bit more subtle than I ﬁrst
thought. It begins: “Break the nbit number into n/m mbit numbers.” The remainder
of the analysis from that point is straightforward, but in that ﬁrst step lies a small
diﬃculty. It can be done by repeatedly shifting the large number to the right, removing
single bits and reconstructing the pieces in m registers. One needs to keep two counters,
one of which runs from 0 to m, and the other oﬀ which runs from 0 to n/m. Consider
the second counter. A naive analysis would show that this requires O(n/m) increment
operations, each of which takes O(log n−log m) bit operations. However, one needs to
notice that the total is O(n/m) amortized time (see Problems 476, and 477). I need
to increase the diﬃculty level of this problem and add a hint to this eﬀect.
Page 74, Problem 340: Change
√
n to
√
N.
Page 78, preamble to Section 7.7: Line 4 of the depthﬁrst search algorithm should read
“mark w used”.
Page 79, Problem 366: This problem should be deleted. There is no known way to ﬁnd
triangles in time O(n + e), using depthﬁrst search or otherwise. Of course, it can be
solved in time O(ne). This problem could be replaced with the much easier one of
ﬁnding a cycle (of any size) in a directed graph in time O(n).
Page 80, Problem 371: The array A should contain integers, not natural numbers. Oth
erwise the problem is trivial.
Page 82, Hint to Problem 361: Delete the second adjacent occurrence of the word “in”.
Page 85, Solution to Problem 369: Perhaps this should be called a comment, not a
solution.
Page 87, Preamble to Section 8.1: In Line 6 of the algorithm, the Roman “m” should
be replaced by an italic “m”.
Page 88, Problem 384, Line 3: The second occurrence of “M
2
” should be “M
i+2
”.
Page 88, Problem 390, Line 3: “M
2
” should be “M
i+2
”.
Page 90, Preamble to Section 8.3, Lines 11–12: The operation “member” should be
“search” (two occurrences).
Page 91, Preamble to Section 8.4, Line 4: “all pairs shortest paths” should be “all
pairs shortestpaths”.
Page 98, Problem 426: The function to be maximized should be:
R[c
1
, c
i
1
] · R[c
i
1
, c
i
2
] · · · R[c
i
k−1
, c
i
k
] · R[c
i
k
, c
1
].
Page 99, Hint to Problem 415: The last three lines are wrong, and should be deleted.
Page 106, Problem 447: The stated proposition is wrong! Here is a counterexample. The
3
problem should be changed to “prove or disprove”.
1
3
4
2
1
2
1
2
Page 107, Problem 449: The running time should be e log k.
Page 107, Problem 450: The third line of the putative variant of Prim’s algorithm should
have a period after the line number.
Page 117, Line 3: “line 8” should read “line 7”.
Page 119, Preamble to Problems 489, 490: The line numbering in procedure permute(i)
is totally wrong. The lines should of course be numbered consecutively.
Page 124, Problem 513, Line 4: The running time should be O(n8
n
2
) in line 4. One of
the two consecutive instances of the word “for” in the last line should be deleted.
Page 129, Problems 529, 530: Both of these problems should be ﬂagged . A
comment should also be added explaining that exhaustive search is unnecessary — the
ﬂightassignment problem can be solved in polynomial time by reducing the problem
to that of ﬁnding a maximumweight matching. For a polynomial time algorithm for
the latter, see, for example, Gabow [3], and Gabox and Tarjan [4].
Page 131, Hint to Problem 510: The series should read
∞
i=1
1/i! = e − 1 ≈ 1.718.
Page 131, Hints to Problems 517, 518: These hints are listed in the wrong order.
Page 131, Hint to Problem 518: A note should be added that the reader should read
ahead to p. 127 for the deﬁnition of dominating set. Either that, or Problem 518 should
be moved after Problem 526.
Page 131, Solution to Problem 487, Line 1: The sentence “Line 6 can be done in time
O(n).” should be replaced by the observation that the work in all of the other lines
can be done in time O(n). A comment should be made that the tail of “ + d(n − 1)”
in the recurrence for T(n) is used because it makes the solution very simple. If the
obvious “ +dn” is used, then the recurrence becomes very diﬃcult to solve.
Page 135, Preamble to Section 11.1: Heap operations have not been deﬁned. A heap
is a set S in which the following operations can be performed:
insert(x) insert x into S, that is, S := S ∪ {x}
findmin return the smallest value in S
deletemin return and delete the smallest value in S
Page 135, Problem 535: The should come after the .
Page 136–137, Problem 543: This problem should be moved to Section 11.5.
Page 137, Problem 544: This problem should ask for insert, deletemin, and deletemax
in O(log n) time, and findmin and findmax in O(1) time.
Page 138, Problem 550: Insert “is correct” after the word “algorithm”.
Page 140, Line 3: Capitalize the word “the” at the start of the parenthetical comment.
4
Page 141, Preamble to Section 11.3, Line 1: Change the ﬁrst occurrence of “in” to
“is”.
Page 142, Problem 562, Line 1: Insert “of” after the word “sequence”.
Page 143, Problem 570: This problem has a hint and a comment, but lacks the appro
priate icons .
Page 144, Problem 573: Change “as as” to “as” in line 2.
Page 145, Hint to Problem 544: This hint is misleading, and should be deleted. (Less
signiﬁcantly, n/2 should be n/2 on lines 1 and 3).
Page 145, Hint to Problem 570: Insert a period after the problem number.
Page 150, Preamble to Section 12.3: “Riemann” is misspelled ‘Reimann” (also in the
Index on p. 178). Fermat’s Last Theorem may have been proved recently.
Page 155, Solutions: These aren’t really solutions, just bibliographic notes. Perhaps they
should be moved to the Comments section.
Page 155, Solutions to Problems 599–600: Change “this problem” to “these problems”.
Page 157, Problem 620: This problem should be changed from ﬁnding the median to
that of halving, which is dividing n values into two sets X, Y , of size n/2 and n/2,
respectively, such that for all x ∈ X and y ∈ Y , x < y. A comment should be added
explaining that this problem is merely practice in using decision trees, since there is a
3n/2 lower bound using the adversary method due to Blum et al. [2].
Page 164, Problem 665 This problem is much harder than I thought. An upper bound of
log
3
n+1 weighings is easy. A little more thought will get you log
3
9n/4 weighings.
The best algorithm known to me uses log
3
2n + 2 weighings (see Aigner [1]).
Page 171, references [46, 47]: For some reason BibT
E
X chose to list Knuth Vol. 3 before
Vol. 2.
Page 171, reference [52]: The authors’ initials are missing — H. R. Lewis and L. Denen
berg.
Page 172, references [66, 67]: These appear to be the same book. The correct reference
is [67].
Page 172, reference [75]: The second occurrence of the word “Edition” should be deleted.
Acknowledgements
The author is very grateful to the following individuals who helpfully pointed out many of the
above errors in the manuscript: Timothy Dimacchia, Sally Goldman, Narciso MartiOliet,
G. Allen Morris III, Paul Newell, Mike Paterson, Steven Skiena, Mike Talbert, Steve Tate,
and HungLi Tseng.
References
[1] M. Aigner. Combinatorial Search. WileyTeubner, 1988.
[2] M. Blum, R. L. Floyd, V. Pratt, R. L. Rivest, and R. E. Tarjan. Time bounds for
selection. Journal of Computer and System Sciences, 7:448–461, 1973.
5
[3] H. N. Gabow. An eﬃcient implementation of Edmond’s algorithm for maximum matching
on graphs. Journal of the ACM, 23(2):221–234, 1976.
[4] H. N. Gabow and R. E. Tarjan. Faster scaling algorithms for general graphmatching
problems. Journal of the ACM, 38(4):815–853, 1991.
[5] I. Parberry. Problems on Algorithms. Prentice Hall, 1995.
[6] I. Parberry. Ian Parberry’s Home Page. A WorldWide Web Document with URL
http://hercule.csci.unt.edu/ian, Created 1994.
[7] I. Parberry. Problems on Algorithms. A WorldWide Web Document with URL
http://hercule.csci.unt.edu/ian/books/poa.html, Created 1994.
6
Supplementary Problems
By Bill Gasarch
Mathematical Induction
1 Games
1. Let A be any natural number. We deﬁne a sequence as a
n
as follows:
(1) a
10
= A. (2) To obtain a
n
, ﬁrst write a
n−1
in base n − 1. Then
let x be the number that is that string of symbols if it was in base n.
Let a
n
= x − 1. (EXAMPLE: if a
20
= 1005 then I write 1005 in base
20 as 2T5 = 2 20
2
+T 20 + 5 1 where T is the symbol for ten. So
x = 2 (21)
2
+ 10 21 + 5 1. So a
21
= x −1 = 1097 −1 = 1096.) For
which natural numbers A will the sequence go to inﬁnity?
2. Here is a game: there are n toothpicks on a table. There are two
players, called player I and player II. Player I will go ﬁrst. In each
move the player whose turn it is will remove 1,2, or 3 toothpicks from
the table. The player who removes the last toothpick wins. (Example
game with n = 10: player I removes 3 (so there are 7 left), player II
removes 2 (so there are 5 left), player I removes 2 (so there are 3 left),
player II removes 3 (so there are 0 left, and Player II wins).) Player I
has a FORCED WIN if there is a move he can make as his ﬁrst move
so that, NO MATTER WHAT PLAYER II does, player I can win
the game. This is called a WINNING FIRST MOVE. Player II has a
FORCED WIN if, whatever player I does for the ﬁrst move, there is
a move player II can make so that after that point, no matter what
player I does, player II can win. Player II’s move is called a WINNING
RESPONSE. (Example: if n = 4 then player II has a forced win: if (1)
player I removes 1, player II removes 3, (2) if player I removes 2 then
player II removes 2, (3) if player I removes 3 then player II removes
1. To specify a winning response you need to specify responses to all
moves.)
(a) Fill in the following sentence and justify: “Player I has a forced
win iﬀ X(n).”
(b) Modify the game so that you can remove 1, 2, 3, 4, 5, 6, or 7 sticks.
Fill in the following sentence and justify: “Player I has a forced
win iﬀ X(n).”
(c) Let k ∈ N. Modify the game so that you can remove 1, 2, . . . , k−
1, or k sticks. Fill in the following sentence and justify: “Player
I has a forced win iﬀ X(n).”
1
(d) Let k ∈ N. Modify the game so that you can remove 1, 2, or 4
sticks. Fill in the following sentence and justify: “Player I has a
forced win iﬀ X(n).”
(e) Let a
1
, . . . , a
k
∈ N. Modify the game so that you can remove
a
1
, . . . , a
k
sticks. Show that there exists A ∈ N and X ⊆ ¦0, . . . , A−
1¦ such that, for all but a ﬁnite number of n ∈ N, if n sticks are
in the pile then
Player I wins iﬀ (∃x ∈ X)[n ≡ x (mod A)].
3. (a) Consider the following game played between FILLER and EMP
TYIER There is initially a box with a ﬁnite (nonempty) number
of balls in it. Each ball has a natural number on it, called its
rank. (There may be many balls of the same rank. For ex
ample, a box could have 100 balls of rank 1, 20000 of rank 2,
and 4 of rank 1000.) In every round EMPTYIER makes the
ﬁrst move. EMPTYIER’s move consists of taking a ball from
the box. FILLER then counters be replacing the ball with a
ﬁnite number of balls of lower rank. (For example, he could re
place a ball of rank 1000 with 999999999999999 balls of rank 999,
888888888876234012 balls of rank 8, and 4 balls of rank 1.) EMP
TYIER wants the box to be empty eventually. FILLER wants
the box to never be empty. Show that no matter what the initial
box of balls has in it, EMPTYIER can win.
(b) Consider the same game as above except that the balls are labeled
with integers. Fill in the following sentence and prove it: “Player
XXX can always win.”
(c) Consider the same game as above except that the balls are labeled
with positive rationals. Fill in the following sentence and prove
it: “Player XXX can always win.”
(d) Consider the same game as above except that the balls are labeled
with ordered pairs of numbers, and we consider (a, b) to be of
lower rank than (c, d) if either a < c or a = c and b < d. Fill
in the following sentence and prove it: “Player XXX can always
win.”
(e) Fill in the following sentence and prove it: “If the game can
be played with a set A with ordering ≤ such that the ordering
has property XXX then the FILLER can win; however, if it has
property NOT(XXX) then the EMPTYIER can win.”
2
4. Consider the following game of “NIM”. There are two piles of stones
of diﬀerent sizes. Players alternate turns. At each turn a player may
take as many stones as he or she likes from one (and only one) pile,
but must take at least one stone. The goal is to take the last stone.
The following is a winning strategy for the ﬁrst player: Take stones
from the larger pile so as to make the two piles have equal size. Prove
that this really is a winning strategy using mathematical induction.
5. Consider the following 2player game. There are n sticks on the table.
Player I can remove 1,2 or 3 sticks. Player II can remove 2,4, or 6
sticks. The player who removes the last stick WINS. Show that:
if n ≡ 1 mod 2 then player I has a winning strategy.
2 Sequences, Summations, Products, and Recur
rences
1. Consider the following sequence:
T(n) ≤ T(n −1) + 3
T(n −2) + 2n, T(0) = 0, T(1) = 1 .
Show that T(n) ≤ 16n
2
.
2. Consider the following sequence:
T(n) ≤ 3T(n −1) + 10, T(0) = 0 .
Find a constant c such that T(n) ≤ c
n
.
3. Consider the following sequence:
T(n) ≤ T(n −2) + 5
√
n, T(0) = 0, T(1) = 1 .
Find a constant c such that T(n) ≤ cn
3/2
. (Hint:
√
n −2 <
√
n.)
4. Consider the following sequence:
A(n) ≤ A(n/3) +A(n/5) + 2n, T(0) = 0 .
Find a constant c such that A(n) ≤ cn.
5. Let P(n) be a statement about an INTEGER n. Assume that the
following two statements are known:
3
(a) P(10).
(b) (∀k)[P(k) → P(k −2)].
For which integers n do we know that P(n) is true?
6. Let a
n
be deﬁned as follows:
a
0
= 6
a
1
= 21
a
2
= 11
a
n
= 3a
n−1
+a
n−3
+ 10n + 2
7. Show that (∀n ∈ N)[a
n
≡ 1 mod 5].
8. Prove (∀n ∈ Z)[n
4
+ 1 ≡ 0 (mod 5) → n ≡ 0 (mod 5)].
9. Find a number a such that the following is true:
(∀n ∈ N, n ≥ a)[n
2
+ 5n + 6 < 2
n
].
Prove your claim by induction.
10. Let P(n) be some property of the natural number n (e.g. P(n) could
be ‘2
17n
≡ 10 (mod 8)’). Assume that P(10) and P(11) are known
to be true. Assume that
(∀n ≥ 5)[P(n −1) → P(n + 1)]
is known. For which n can we conclude P(n) is true? Brieﬂy explain.
11. Deﬁne the sequence a
n
by a
0
= 1 and a
n
=
n−1
i=0
a
i
. Find a function
f(n) such that (∀n)[a
n
≤ f(n)]. Prove your result.
12. Let a
n
be deﬁned as below. Show that (∀n ≥ 1)[a
n
≡ 0 (mod 3)].
a
0
= 3
a
1
= 6
(∀n ≥ 3)[a
n
= (a
n−1
+a
n−2
)
2
+ 6n]
13. Let P(n) be some property of the natural numbers. Assume that
the following are known (1) For all primes q, P(q) is true, and (2)
(∀n ∈ ^)[P(n) → P(n
2
)]. For which n ∈ ^ can we conclude P(n) is
true? Express your answer clearly. No explanation is needed.
4
14. Let P(z) be some property of the integers. Assume that the following
are known
(a) P(−23).
(b) (∀z ∈ Z)[P(z) → P(z + 3)].
For which z can we conclude P(z) is true? (Express your answer
clearly!) No justiﬁcation required.
15. Let a
0
= 9, a
1
= 4, a
2
= 100, a
3
= 64, and
(∀n ≥ 4)
a
n
= 25a
n−1
a
log n
].
Let SQ = ¦n ∈ N [ (∃x ∈ N)[n = x
2
]¦. (That is, SQ is the set of all
squares.) Show that (∀n ≥ 0)[a
n
∈ SQ].
16. A sequence b
0
, b
1
, . . . is INCREASING if (∀x)(∀y)[x < y → b
x
< b
y
].
Let a
n
be deﬁned as follows: a
0
= 10, a
1
= 12, a
2
= 13, a
3
= 14,
a
4
= 15, a
5
= 16, and
(∀n ≥ 2) [a
n
= a
n−1
+a
n−3
−a
n−5
] .
17. Describe the set of all a ∈ N such that the following is true:
(n
2
+ 3n +a ≡ 0 (mod 5)) → (n ≡ 0 (mod 5)).
(No justiﬁcation required.)
18. Let S(n) =
n
i=1
i(i + 1). Find a, b such that S(n) = an
3
+bn. Prove
that your answer is correct by induction.
19. Deﬁne a sequence via a
1
= 4 and
(∀n ≥ 2)[a
n
= a
2
n−1
].
Show that
(∀n ≥ 1)[a
n
≤ 4 3
(3
n
)
].
20. Let a
0
= 4, a
1
= 49, a
2
= 104, and
(∀n ≥ 3)
a
n
= (a
n
2
)
3
+a
√
n
+ 11
.
Show that
(∀n ≥ 0)[a
n
≡ 4 (mod 5)].
5
21. Let P(q) be some property of the rational q. Assume that the following
are known
(a) P(1).
(b) (∀q ∈ Q)[P(q) → P(
q
2
)].
Let A be the set of all q such that we can conclude P(q) is true.
Write down the set A clearly but DO NOT use the notation “. . .” No
justiﬁcation required.
22. Let a
0
= 10, a
1
= 90, and
(∀n ≥ 2)[a
n
= 5a
n−1
+ 20a
n−2
].
Find POSITIVE constants A, B, C ∈ N, such that
(∀n ≥ 0)[A (C −1)
n
≤ a
n
≤ B C
n
].
No justiﬁcation required (but show work for partial credit).
23. Let T(n) = 2T(n − 1) + T(
n
2
) + n. Prove what you can about this
recurrence.
24. Let T(n) ≤ T(
3
4
n) + T(
1
4
n) + 4n. Find a value of d such that for all
n, T(n) ≤ dnlog n.
25. Let T(n) ≤ T(an) + T(bn) + cn. For what values of a, b and c is it
possible to ﬁnd a d such that for all n, T(n) ≤ dnlog n.
26. Consider the following recurrence. Initial conditions T(0) = 0, T(1) =
5. T(n) = 7T(n−1) −12T(n−2). Solve the recurrence exactly. Then
express the answer more simply using Onotation.
27. Prove by induction that if T is a tree then T is 2colorable.
28. Let T(n) be deﬁned by T(1) = 10 and
T(n) = T(n −log n) +n.
Find a simple function f such that T(n) = Θ(f(n)). NO PROOF
REQUIRED.
29. Use mathematical induction to show that
6
(a)
n
i=1
i(i + 1) =
n(n + 1)(n + 2)
3
(b)
n
i=0
i2
i−1
= (n −1)2
n
−1
30. Assume every node of a tree with n nodes has either 0 or 3 children.
How many leaves does the tree have? Prove your answer.
31. Simplify the following summation:
n
i=1
n
j=i
n−j
k=1
1
32. Simplify the following summation:
n
i=1
n
j=n−i
2j
k=j
1
33. Consider the following code fragment.
for i = 1 to n do
for j = i to 2*i do
for k = nj+1 to n do
output ‘foobar’
a. Let T(n) denote the number of times ‘foobar’ is printed as a func
tion of n. Express T(n) as a summation (actually three nested
summations).
b. Give a closedform solution for T(n) (i.e. no summations) by sim
plifying your summation. Show your work.
34. a. Assume that Christmas has n days. Exactly how many presents
did my “true love” send me? [If you do not understand this
question, you should do some research.]
7
b. Using part (a), exactly how many presents did my “true love”
send during the twelve days of Christmas?
35. Evaluate the sum
∞
k=0
(3k −1)x
3k
for 0 < x < 1.
36. Evaluate
n
k=1
4 3
k
37. Consider the following code fragment.
for i=1 to n do
for j=i to 2*i do
output ‘foobar’
Let T(n) denote the number of times ‘foobar’ is printed as a function
of n.
a. Express T(n) as a summation (actually two nested summations).
b. Simplify the summation. Show your work.
38. Consider the following code fragment.
for i=1 to n step 2 do
for j=1 to i do
output ‘foobar’
Let T(n) denote the number of times ‘foobar’ is printed as a function of
n. Express T(n) as two nested summations. Be careful your “formula”
works for n both even and odd. Do not simplify the summation.
39. Consider the following code fragment.
for i=1 to n/2 do
for j=i to ni do
for k=1 to j do
output ‘foobar’
Assume n is even. Let T(n) denote the number of times ‘foobar’ is
printed as a function of n.
(a) Express T(n) as three nested summations.
(b) Simplify the summation. Show your work.
8
40. Given an m n array of (positive and negative) numbers, ﬁnd the
largest sum of values in a (contiguous) rectangle.
a. Write down an English description of the “brute force” algorithm
for the “maximum contiguous rectangle problem”. One or two
sentences should suﬃce.
b. Write down the “brute force” algorithm in pseudocode.
c. How many times is the inner loop executed? Write it using sum
mations.
d. Simplify your answer. Justify your work. [If you do this right,
the solution involves very little calculation.]
e. Find a better algorithm for the “maximum contiguous rectangle
problem”. How well can you do?
41. Approximate
n
k=1
k
3
using integrals. Make sure to get an upper and
lower bound.
42. Approximate
n
k=1
4
3k−2
43. Use integrals to give upper and lower bounds for
∞
k=1
1
k
3/2
.
44. Appoximate
n
j=0
√
j using an integral.
45. Consider the following code fragment.
for i=1 to n do
for j=1 to lg i do
print ‘foobar’
Let T(n) denote the number of times ‘foobar’ is printed as a function
of n.
(a) Express T(n) exactly as two nested summations.
(b) Simplify down to one summation.
(c) Give good upper and lower bounds for T(n).
46. Find the solution for the recurrence relation
T(n) = T(n −c) +k
where both c and k are integer constants, n is a natural number, and
T(x) is 0 if x is a negative integer.
9
47. Solve the recurrence
T(n) =
T(n −2) +n n > 1
0 otherwise
assuming n is even. Show your calculations.
48. Solve the recurrence T(n) = T(n − 1) + 2n − 1 (T(0) = 0) exactly.
Justify your answer.
49. Solve the recurrence T(n) = T(n − 1) + 3n − 2 (T(0) = 0) exactly.
Justify your answer.
50. Solve the recurrence
T(n) =
T(n/3) + 2 n > 1
0 otherwise
assuming n is a power of 3. Show your calculations.
51. Solve the recurrence T(n) = 4T(n/3) + 2n − 1 (T(0) = 0) exactly.
Assume n is a power of 3. Justify your answer.
52. Solve the recurrence T(n) = 3T(n/4) + 2n − 1 (T(0) = 0) exactly.
Assume n is a power of 4. Justify your answer.
53. Solve the recurrence
T(n) =
T(n/5) + 2 n > 1
3 otherwise
assuming n is a power of 5. Show your calculations.
54. Solve the recurrence T(n) = 5T(n/2) + 2n − 1 (T(1) = 7) exactly.
Assume n is a power of 2. Justify your answer.
55. Consider the following recurrence, deﬁned for n a power of 5:
T(n) =
19 if n = 1
3T(n/5) +n −4 otherwise
a. Solve the recurrence exactly using the iteration method. Simplify
as much as possible.
b. Use mathematical induction to verify your solution.
10
56. Consider the following recurrence, deﬁned for n a power of 5:
T(n) =
7 if n = 1
4T(n/5) + 2n −3 otherwise
a. Solve the recurrence exactly using the iteration method. Simplify
as much as possible.
b. Use mathematical induction to verify your solution.
57. Obtain exact solutions to the following two recurrences.
a. T(n) = 5T(n/3) + 2n (T(1) = 7). Assume n is a power of 3.
b. T(n) = 2T(n/6) + 5n (T(1) = 7). Assume n is a power of 6.
58. Obtain exact solutions to the following two recurrences.
a. T(n) = 4T(n/3) + 2n (T(0) = 0).
b. T(n) = 3T(n/4) + 2n (T(0) = 0).
59. Obtain exact solutions to the following two recurrences.
a. Let n be a power of 2.
T(n) =
4 if n = 1
5T(n/2) + 3n
2
otherwise
b. Let n be a power of 4.
T(n) =
3 if n = 1
2T(n/4) + 4n otherwise
60. Obtain exact solutions to the following two recurrences.
(a) Let n be a power of 2.
T(n) =
3 if n = 1
7T(n/2) + 4n
2
+ 1 otherwise
(b) Let n be a power of 4.
T(n) =
4 if n = 1
3T(n/4) + 5n −2 otherwise
11
61. Consider the following recurrence.
T(n) =
0 if n = 0
T(n/2) +T(n/4) + 3n otherwise
Find a constant c such that T(n) ≤ cn. Try to make c as small as
possible.
62. Consider the following recurrence.
T(n) =
0 if n = 1
T(2n/3) +T(n/6) + 5n otherwise
Use constructive induction to ﬁnd a constant c such that T(n) ≤ cn.
63. (a) Let r
1
, r
2
, . . . , r
k
be a list of positive fractions, i.e. 0 < r
i
< 1,
such that
k
i=1
r
i
< 1. Let a > 0 be a constant. Let
T(n) ≤
k
i=1
T(r
i
n) +an.
Show that T(n) is linear, i.e. T(n) ≤ cn for some constant c.
What can you say about c?
(b) What if
k
i=1
r
i
= 1? What if
k
i=1
r
i
> 1?
64. Let
T(n) =
2T(n/2) + 2 lg n if n > 1
0 if n = 1
a. Use constructive induction to show that T(n) = an + b lg n + c.
Find constants a, b, and c.
b. Solve the recurrence using the iteration method.
65. An S
0
tree is a single node. An S
i
tree, i > 0, is created by taking two
S
i−1
trees and making the root of one a child of the root of the other.
(These are the trees that create the largest height for as few nodes as
possible for the UNIONFIND algorithm with balancing.)
(a) Draw S
0
, S
1
, S
2
, and S
3
. You may also want to draw S
4
for
yourself.
12
(b) The nodes on level i are all nodes distance i from the root. Tree
S
k
will have nodes on levels 0, . . . , k (and thus have k +1 levels).
For S
3
, how many nodes are on levels 0, 1, 2, and 3.
(c) Make a conjecture for how many nodes are on level i of S
k
. (You
may also want to look at S
4
.)
(d) Prove your conjecture.
66. Consider an Ftree created as follows. An F
0
tree is a single node.
An F
n+1
tree is created by taking two F
n
trees and making the root
of one the two trees the child of the other tree. Here are the ﬁrst few
Ftrees.
a. How many nodes are there in an F
n
tree. Justify your answer.
b. Recall that the depth of a node is its distance from the root (so
the root has depth 0, its children have level 1, etc.). Write down a
recurrence for how many nodes there are at depth d in F
n
? Note
that there are two parameters.
c. Guess a solution to your recurrence (by looking a few trees).
d. Prove that your guess is correct using mathematical induction on
your recurrence.
13
Floors and Ceilings
1. Let r be a real number. Determine all real numbers such that the
following statement is true: r + = r.
2. For which positive reals x does x +
1
6
= x +
1
2
?
3. Find a number x such that (x)
2
< x
2
.
4. For which x is 4x = 4x?
5. For which real numbers x does the following hold: x +
1
3
= x +
1
2
.
6. For which real numbers x does x satisfy the following two conditions:
(a) 1 < x < 2, and (b) x
2
= x
2
. (HINT: Express x as 1 + where
0 < < 1.)
14
Combinatorics
1. For this problem we express numbers in base 10. The last digit of a
number is the rightmost digit of the number (e.g., the last digit of
39489 is 9). Find the largest x ∈ N such that the following statements
is true:
If A ⊆ N and A = 59 then there are at least x numbers in A that
have the same last digit.
2. Every Sunday Bill makes lunches for Carolyn. Lunch consists of a
sandwich, a fruit, and a desert. Sandwich is either ham and cheese,
peanut butter and jelly, egg salad, or tuna ﬁsh. Fruit is either an
apple, an orange, or a coconut. Desert is either pretzels, a brownie, or
an apple pie.
(a) How many diﬀerent lunchs can Bill make for Carolyn ?
(b) If Carolyn does not like to have egg salad with apple pie, or tuna
ﬁsh with an apple, then how many lunches can Bill make for
Carolyn.
(c) Why does Bill call Carolyn his ‘gnilrad’ ?
3. Can you place 24 rooks on an 8 by 8 chessboard so that if you remove
all but 4 then there will always be 2 that attack each other? How
about 25?
4. Let f be the function that takes a number x = d
n
d
n−1
· · · d
0
(these
are the digits of x base 10) to the sum of the 2001 powers of the digits
(e.g., f(327) = 3
2001
+ 2
2001
+ 7
2001
). Show that, for any x, the set
{f(x), f(f(x)), f(f(f(x))), . . .} is ﬁnite.
5. Assume that there are at most 200 countries in the world. Assume
that there are 900 people in a room. Find a number x such that the
statement a is TRUE and statement b is FALSE. Brieﬂy explain WHY
statement a is TRUE and WHY statement b is FALSE.
(a) There must exist x people in the room that are from the same
country.
(b) There must exist x+1 people in the room that are from the same
country.
15
Probability
1. Let r be a random number taking on the values 1, . . . , n, with proba
bility
P(r = i) =
1
n
1 ≤ i ≤
n
4
2
n
n
4
< i ≤
n
2
1
2n
n
2
< i ≤ n
For the algorithm below perform an average case analysis.
if (r ≤ n/4) then perform 10 operations
else
if (r > n/4 and r ≤ n/2) then perform 20 operations
else
if (r > n/2 and r ≤ 3n/4) then perform 30 operations
else perform n operations
Perform an average case cost analysis of this algorithm.
2. You are given a deck of ﬁfty two cards which have printed on them a
pair of symbols: an integer between 1 and 13, followed by one of the
letters “S,” “H,” “D,” or “C,” There are 4 × 13 = 52 such possible
combinations, and you may assume that you have one of each type,
handed to you face down.
(a) Suppose the cards are randomly distributed and you turn them
over one at a time. What is the average cost (i.e. number of
cards turned over) of ﬁnding the card [11,H]?
(b) Suppose you know that the ﬁrst thirteen cards have one of the
letters “H” or “D” on them, but otherwise all cards are randomly
distributed. Now what is the average cost of ﬁnding [11,H]?
3. Let P be a program that has the following behavior:
(a) The only inputs are {1, 2, . . . , n}, n is large, and n is divisible by
2.
(b) If the input i is one of {1, 2, . . . ,
n
2
} then P takes i
4
steps.
(c) If the input i is one of {
n
2
+ 1, . . . , n} then P takes 2
i
steps
16
Find a distribution on the inputs such that every input has a nonzero
probability of occurring and the expected running time is θ(n
17
).
4. Suppose we have one loaded die and one fair die, where the probabil
ities for the loaded die are
P(1) = P(2) = P(5) = P(6) = 1/6, P(3) = 1/7, P(4) = 4/21.
(a) What is the probability that a toss of these two dice produces a
sum of either seven or eleven?
(b) What is the expected value of the sum of the values of the dice?
(c) the product of the values of the dice?
5. Let a be a random number taking on the values 1, . . . , n, with proba
bility 1/2 that it is contained in (n/3, 2n/3], and probability 1/4 that
it is contained in either [1, n/3] or (2n/3, n]. Consider an algorithm
that has the following behavior:
if (a ≤ n/3) then perform i operations
else
if (a > n/3 and a ≤ 2n/3) then perform log
3
n operations
else
perform n
2
operations
Perform an average case cost analysis of this algorithm.
6. Given n numbers in a sorted list, we know that in general, the worst
case cost for linear search is n, and its average cost is is (n + 1)/2,
whereas the worst and average case costs for binary search are log
2
n
and (log
2
n)/2. Suppose instead that we have a special situation where
the probability that the item we are seeking is ﬁrst in the list is p, and
the probability that it in position i, 2 ≤ i ≤ n, is (1 −p)/(n −1). For
what values of p are we better oﬀ using linear search, in the sense that
the average cost for linear search is higher than that for binary search?
7. Let P be a program that has the following behavior:
(a) The only inputs are {1, 2, . . . , n}, n is large, and n is divisible by
3.
(b) If the input is one of {1, 2, . . . ,
n
3
} then P takes log n steps.
17
(c) If the input is one of {
n
3
+ 1, . . . ,
2n
3
} then P takes n steps
(d) If the input is one of {
2n
3
+ 1, . . . , n} then P takes 2
n
steps
Find a probability distribution on the inputs such that every input has
a nonzero probability of occurring, but the expected time is θ(n).
8. Let A[1..n] be a sorted list. We are pondering writing a program that
will, given a number x, see if it is on the list. Suppose that we have a
special situation where the probability that the item we are seeking is
ﬁrst in the list is p, and the probability that it is in the second position
in the list is p, and the probability that it in position i, 3 ≤ i ≤ n,
is (1 − 2p)/(n − 2). For what values of p will the average case for
linear search do better than log n? (Obtain an inequality involving p,
without summations, but do not simplify it.)
9. Consider the following experiments. For each of them ﬁnd the expected
value and the variance.
(a) Roll a 6sided die and a 4sided die. Add the two values together.
Both dice are fair.
(b) Roll a 2sided die and a 3sided die. Multiply the two values
together. Both dice are fair.
(c) Roll a 3 sided die and another 3sided die. Add the two values
together. The ﬁrst die has probability p
1
of being 1, p
2
of being
2, and p
3
of being 3. The second die is fair. (In this case E and
V will be in terms of p
1
, p
2
, and p
3
.)
10. Let a be a random number taking on the values 1, . . . , n, with proba
bility 1/2 that it is contained in (n/3, 2n/3], and probability 1/4 that
it is contained in [1, n/3], and probability 1/4 that it is in (2n/3, n].
Consider an algorithm that has the following behavior:
if (a ≤ n/3) then perform a log a operations
else if (a > n/3 and a ≤ 2n/3) then perform log
3
n operations
else perform n
2
operations
Perform an average case cost analysis of this algorithm. The ﬁnal
answer can be in Θ notation.
18
11. Given n (n > 10) numbers in a sorted list, we know that in general,
the worst case cost for linear search is n, and its average cost is (n +
1)/2, whereas the average case for binary search is (roughly) log n (its
actually (log n)−c for some constant c). Suppose instead that we have
a special situation where the probability that the ith item is sought is
p
i
, and (1) for i ≤ 10 we have p
i
= p, and (2) for i > 10 we have the
probability that it is in position i is p
i
=
1−10p
n−10
. What is the average
case cost of linear search with this probability distribution (the answer
will be in terms of p). For this problem do NOT use Θnotation, you
will need to look at exact results. Using a calculator ﬁnd values of
n > 10 and p < 1 such that linear search performs better than log n
average case.
12. We are going to pick a number from {1, . . . , n} There are constants
p
1
, p
2
, . . ., and a quantity A(n), such that the probability of picking i
is p
i
= A(n)i. What is A(n)?
13. Let n be a large number. Assume we pick numbers from the set
{1, . . . , n}, and after we get a number we square it. (Formally the
sample space is {1, . . . , n} and the random variable is X(a) = a
2
.)
Find a probability distribution such that E(X) = θ(log n).
14. Let P be a program that has the following behavior:
(a) The only inputs are {1, 2, . . . , n}, n is large.
(b) If the input i is ≤ log n then P takes 2
i
steps
(c) If the input i is such that log n < i ≤
n
2
then P takes n
2
steps
(d) If the input i is such that i >
n
2
then P takes n
3
steps.
Answer the following questions.
(a) What is the expected run time if the distribution is uniform? You
may use Θnotation.
(b) Find a distribution where the expected time is Θ(n
2
). (In this
distribution all elements must occur with nonzero probability.)
15. Let S = {1, 2, . . . , n}. We are going to pick a number i out of S
with the following probability distribution. Prob(i) =
A(n)
i
2
(where
A(n) is some function of n). Once we have this number i we compute
X(i) = i
3.2
. (a) Show A(n) = O(1). (b) Find E(X)? (Use Θ notation.)
19
16. Let n be a large number. Let f be a function. Assume we pick
numbers from the set {1, . . . , n}, and after we get a number we apply
f to the result. (Formally the sample space is {1, . . . , n} and the
random variable is X(i) = f(i).)
(a) What is the expected value if f(i) = i
2
and the distribution is
uniform? (Use Θ notation.)
(b) Let a, b be reals such that 1 < a << b (a is much less than b).
Assume that f(i) = i
b
. Show that there exists a distribution such
that E(X) = Θ(n
a
). (There are several valid solutions to this
problem, some of which do not use the assumption 1 < a << b
and some of which do.)
17. You play a game of throwing two six sided dice (with integer values
1, . . . , 6). You lose a dollar if the two dice have diﬀerent values. You
win d dollars if the two dice land on the same value d (i.e. you throw
double d’s).
a. What is the expected value of playing this game once.
b. What is the variance of playing this game once.
c. What is the standard deviation of playing this game once.
d. What is the expected value of playing this game n times.
e. What is the variance of playing this game once n times.
f. What is the standard deviation of playing this game once n times.
18. Consider an ssided die, where the sides are marked with the numbers
1, 2, . . . , s, and each side has equal probability (1/s) of being rolled.
(a) What is the expected value of one roll of this die.
(b) What is the expected value of the sum of n (independent) rolls
of this die.
(c) What is the variance and standard deviation of one roll of this
die.
(d) What is the variance and standard deviation of the sum of n
(independent) rolls of this die.
(e) What is the probability of two rolls of this die summing to an
integer t (2 ≤ t ≤ 2s)? (Give your answer as a function of s and
t.)
20
(f) What is the probability of two rolls of this die both being s given
that you know that at least one of the two rolls was an s. (Note:
The roll that was an s is NOT necessarily the ﬁrst roll.)
19. A combination lock works by turning to the right three times and
stopping a on particular integer from 1 to 60 (inclusive), turning to
the left two times and stopping on a particular integer from 1 to 60,
and then turning to the right and stopping on a ﬁnal integer from 1
to 60. How many possible combinations are there.
20. Assume a lock has C combinations. A thief tries a combination ran
domly. If it does not work he again tries a combination randomly,
which may (or may not) be the same as the ﬁrst. He keeps doing this
until he ﬁnally opens the lock. On average, how many times will he
try the lock?
21. Professor Pascal wakes up in the morning. He remembers that either
he has six blue socks and four red socks in the drawer or he has four
blue socks and six red socks in the drawer, but he cannot remember
which (assume each is equally likely). He pulls two socks randomly
out of the drawer. It turns out that they are both blue. What is the
probability that initially there were more blue socks.
22. A coin is tossed n times, each time with an independent probability p
of coming up heads and 1−p of coming up tails. Let H be the number
of heads occurring. What is
(a) E[H], the expected number of heads?
(b) V [H], the variance of H?
(c) the standard deviation of H?
(d) the probability that H > 2?
Show your calculations.
23. Consider a threeside die where a 1 has probability p of being rolled, a
2 has probability q of being rolled, and a 3 has probability r = 1−p−q
of being rolled.
(a) Assume you roll the dice once.
a. What is the expected value?
21
b. What is the variance?
c. What is the standard deviation?
(b) Assume n independent such dice are rolled and their values are
summed.
a. What is the expected value?
b. What is the variance?
c. What is the standard deviation?
Show your calculations.
24. Assume every child born has probability 1/2 of being a boy or a girl.
Assume that (even within a family) children’s sexes are independent.
(a) Mr. Jones has two children. The older one is a boy. What is the
probability that they are both boys? Why?
(b) Mr. Smith has two children. At least one is a boy. What is the
probability that they are both boys? Why?
(c) Mr. Johnson has three children. At least one is a boy. What is
the probability that at least one is a girl? Why?
22
Asympototics
1. For each of the following expressions f(n) ﬁnd a simple g(n) such that
f(n) = Θ(g(n)).
(a) f(n) =
n
i=1
1
i
.
(b) f(n) =
n
i=1
1
i
.
(c) f(n) =
n
i=1
log i.
(d) f(n) = log(n!).
2. Place the following functions into increasing asymptotic order.
f
1
(n) = n
2
log
2
n, f
2
(n) = n(log
2
n)
2
, f
3
(n) =
n
i=0
2
i
, f
4
(n) =
log
2
(
n
i=0
2
i
).
3. Let a, b be real numbers such that a ≥ 0 and Let 0 ≤ b ≤ 1.
(a) Let a ≥ 0. Find a function f
a
that does not involve summations
such that
n
i=0
a
i
= θ(f
a
(n)). (The function f
a
might involve sev
eral cases and might depend on a.)
(b) Let b ≥ 0. Find a function g
b
that does not involve summations
such that
n
i=1
i
b
= θ(g
b
(n)). (The function g
b
might involve sev
eral cases and might depend on b.)
(c) For what values of a and b does f
a
(n) = O(g
b
(n)). (Only consider
the cases where 0 ≤ a, b.)
4. Place the following functions into increasing asymptotic order. If two
or more of the functions are of the same asymptotic order then indicate
this.
f
1
(n) =
n
i=1
√
i, f
2
(n) = (
√
n) log n, f
3
(n) = n
√
log n, f
4
(n) =
12n
3
2
+ 4n,
5. Let k be an integer such that k ≥ 1. Find the constant A such that
the following holds.
n
i=1
i
k
= An
k+1
+ O(n
k
). Prove your result.
6. Let a, b, c be real numbers such that a, b, c ≥ 10. Consider the recur
rence T(n) = aT(
n
b
) +n
c
. You are given the option of either reducing
a, b, or c by 1. You want to make T(n) as small as possible. For
which values of a, b, c are you better oﬀ reducing a? For which values
of a, b, c are you better oﬀ reducing b? For which values of a, b, c are
you better oﬀ reducing c?
23
7. Let a, b be real numbers such that a, b > 0. Consider the recurrence
T(n) = T(
n
a
)+T(
n
b
)+n. Specify an inﬁnite number of values for (a, b)
such that T(n) = Θ(n).
8. Let a, b, c be numbers such that 0 ≤ a, b < 1, and c > 0. Let T(n) be
deﬁned by T(2) = 4 and T(n) = T(an) + T(bn) + cn.
(a) Show that if a+b < 1 then T(n) is bounded by a linear function.
(b) Show that if a + b = 1 then T(n) is bounded by O(nlog n).
(c) Does there exist a d such that, for all a, b, c as above, T(n) =
O(n
d
)? (Recall that Onotation just means that n
d
is an upper
bound on T(n).)
9. Let c > 0. Let T be the function deﬁned by
T(0) = 1
T(n) = 81T(
n
3
) + c
n
Find a function g(n) such that T(n) = Θ(g(n)). (The function g may
involve several cases depending on what c is. The cases should be
clearly and easily speciﬁed.)
10. (a) For what values of a, b does the following hold?
n
i=0
i
a
b
i
= O(1).
(b) For which constants a is it the case that 3
n
2
+ 2
n
= O(a
n
)?
11. (a) Find a function f such that
n
i=1
i log i = Θ(f(n)).
(b) Find a simple function f such that
n
i=1
√
i log i = Θ(f(n)). Ex
plain your answer.
12. For each of the following expressions f(n) ﬁnd a simple g(n) such that
f(n) = Θ(g(n)). (You should be able to prove your result by exhibiting
the relevant parameters, but this is not required for the hw.)
(a) f(n) =
n
i=1
3i
4
+ 2i
3
−19i + 20.
(b) f(n) =
n
i=1
3(4
i
) + 2(3
i
) −i
19
+ 20.
(c) f(n) =
n
i=1
5
i
+ 3
2i
.
13. Which of the following are true?
(a)
n
i=1
3
i
= Θ(3
n−1
).
24
(b)
n
i=1
3
i
= Θ(3
n
).
(c)
n
i=1
3
i
= Θ(3
n+1
).
14. Show that
∞
i=0
i
5
4
i
3
= O(1).
15. Let a, b be real numbers such that 0 ≤ a ≤ 1 and Let b ≥ 0.
(a) Find a function f
a
that does not involve summations such that
n
i=1
i
−a
= Θ(f
a
(n)). (The function f
a
might involve several
cases and might depend on a.)
(b) Find a function g
b
that does not involve summations such that
n
i=0
b
i
= Θ(g
b
(n)).
(c) For what values of a and b does f
a
(n) = O(g
b
(n))? (Only consider
the cases where 0 ≤ a ≤ 1, and b ≥ 0.)
(d) For what values of a and b does g
b
(n) = O(f
a
(n))? (Only consider
the cases where 0 ≤ a ≤ 1, and b ≥ 0.)
16. Let a, b be real numbers such a > 0 and b > 1. Consider the recurrence
T(n) = aT(
n
b
) + n. You are given the option of either reducing a or b
by 1. You want to make T(n) as small as possible. For which values
of a, b would reducing a by 1 reduce T(n)? For which values of a, b
would reducing b by 1 reduce T(n)?
17. Let a, b be real numbers such that a, b > 0. Consider the recurrence
T(1) = 10, T(n) = T(
n
a
) + T(
n
b
) + n. Specify an inﬁnite number of
values for (a, b) such that T(n) = Θ(n).
18. Let T(1) = 10 and T(n) = T(
√
n) + n. Show that T(n) = O(n).
19. Let a, b be real numbers such that −100 ≤ a ≤ 100 and b > 0.
(a) Find a function f
a
that does not involve summations such that
n
2
i=0
i
7a
= Θ(f
a
(n)). (The function f
a
might involve several
cases and might depend on a. Each case should be simple.)
(b) Find a function g
b
that does not involve summations such that
√
n
i=1
b
12i
= Θ(g
b
(n)). (The function g
b
might involve several
cases and might depend on b. Each case should be simple.)
20. For each of the following functions f ﬁnd a simple function g such that
f(n) = Θ(g(n)).
25
(a) f
1
(n) = (1000)2
n
+ 4
n
.
(b) f
2
(n) = n + nlog n +
√
n.
(c) f
3
(n) = log(n
20
) + (log n)
10
.
(d) f
4
(n) = (0.99)
n
+ n
100
.
21. Let c, d be real numbers such that c, d ≥ 0. Consider the recurrence
T(n) = 49T(
n
7
) + n
c
d
n
.
Find a function g
c,d
such that T(n) = Θ(g
c,d
). (The function g
c,d
might
involve several cases and might depend on c and d. Each case should
be simple.)
22. Let a, b be real numbers such that a > 0 and −100 ≤ b ≤ 100.
23. (a) Find a function f
a
that does not involve summations such that
n
2
i=1
a
7i
= Θ(f
a
(n)). (The function f
a
might involve several
cases and might depend on a. Each case should be simple.)
(b) Find a function g
b
that does not involve summations such that
n
i=0
i
5b
= Θ(g
b
(n)). (The function g
b
might involve several cases
and might depend on b. Each case should be simple.)
(c) For what values of a and b does f
a
(n) = O(g
b
(n))? (Only consider
the cases a > 0 and −100 ≤ b ≤ 100.)
(d) For what values of a and b does f
a
(n) = Ω(g
b
(n))? (Only consider
the cases a > 0 and −100 ≤ b ≤ 100.)
24. Let a, b be any real numbers Let T be the function deﬁned by
T(n) =
n
i=1
i
a
(log n)
b
.
Find a function g
a,b
(n) such that T(n) = Θ(g(n)). The function g
a,b
should be as simple as possible. (The function g
a,b
may involve several
cases depending on what a, b is. The cases should be clearly and easily
speciﬁed. NO PROOF REQUIRED.)
25. For each pair of expressions (A, B) below, indicate whether A is O,
o, Ω, ω, or Θ of B. Note that zero, one or more of these relations may
hold for a given pair; list all correct ones.
26
A B
(a) n
100
2
n
(b) (lg n)
12
√
n
(c)
√
n n
cos(πn/8)
(d) 10
n
100
n
(e) n
lg n
(lg n)
n
(f) lg (n!) nlg n
27
Graph Problems
1. Let K
n
denote the complete graph with n vertices (i.e. each pair
of vertices contains one edge between them), and let K
n
denote the
modiﬁcation of K
n
in which one edge is removed.
(a) When does K
n
have an Euler path?
(b) When does K
n
have a Hamiltonian cycle?
(c) What is the mininum number of colors needed for a proper col
oring of K
n
?
2. Prove that in any (undirected) graph, the number of vertices of odd
degree is even.
3. Find a number a, 0 < a < 1 such that the following statement is true
“If G is a planar graph on n vertices then at least an vertices have
degree ≤ 99.”
4. Let n
1
, n
2
be natural numbers such that n
1
≥ 3 and n
2
≥ 3. Let
C(n
1
, n
2
) be the graph that is formed by taking C
n
1
, the cycle graph
on n
1
vertices, and C
n
2
, the cycle graph on n
2
vertices, and connecting
ALL the vertices of of C
n
1
to ALL the vertices of C
n
2
.
(a) For which values of (n
1
, n
2
) does C(n
1
, n
2
) have an Eulerian cy
cle?
(b) For which values of (n
1
, n
2
) does C(n
1
, n
2
) have a Hamiltonian
path?
(c) For which values of (n
1
, n
2
) is C(n
1
, n
2
) 5colorable but not 4
colorable?
5. Let A be a class of graphs with the following properties: (1) If G ∈ A
and H is a subgraph of G, then H ∈ A. (2) If G ∈ A then G has a
vertex of degree ≤ 10. Show that every graph in A is 10colorable.
6. Prove rigorously that if a graph has three vertices of degree 1, then
that graph does not have a Hamiltonian path.
7. For each of the following combinations of properties either exhibit a
connected graph on 10 vertices that exhibits these properties, or show
that no such graph can exist.
28
(a) Eulerian but not Hamiltonian.
(b) Hamiltonian but not Eulerian.
(c) Hamiltonian and 2colorable.
8. Consider the following real world problem: Given n cities you want to
build a highway system such that, for any two cities x and y, there is
a way to go from x to y that only passes through at most one other
city. You want to minimize the number of roads you build. STATE
this problem rigorously as a problem in graph theory.
9. For every n, ﬁnd the maximal number of edges that a graph with n
vertices can have and still be unconnected.
10. Show that if G is a planar graph then at least half the vertices have
degree ≤ 10.
11. Give an example of a planar graph with an Eulerian path that has 2
vertices of degree 5.
12. A graph is almostplanar if there exists a vertex v it such that if you
remove that vertex then the graph is planar. Show that every almost
planar graph is 5colorable (you may assume that every planar graph
is 4colorable).
13. Let W
n
be the ‘wheel graph’ (pictured below). Formally it has vertices
{1, 2, . . . , n, n + 1} and edge set is the union of the sets
{(1, 2), (2, 3), . . . , (n −2, n −1), (n −1, n), (n, 1)}
and
{(1, n + 1), (2, n + 1), . . . , (n, n + 1)}.
We only consider n ≥ 3.
(a) For which values of n is W
n
2colorable?
(b) For which values of n is W
n
3colorable but not 2colorable?
(c) For which values of n is W
n
4colorable but not 3colorable?
(d) For which values of n is W
n
5colorable but not 4colorable?
14. Find a number c such that the following is true:
29
If G is a connected planar graph then at least at least
3
4
of the vertices
have to have degree less than c.
You must prove that your answer is correct. You may assume that the
number of vertices in G is divisible by 4.
15. For n, m ∈ N let K
n,m
be the complete bipartite graph on n, m vertices
(i.e., the left set has n vertices, the right set has m vertices, and for
every left vertex a and right vertex b there is an edge between a and
b).
(a) Draw K
1,3
and K
3,4
.
(b) For what values of n, m does K
n,m
have a cycle that includes
every vertex exactly once.
(c) For what values of n, m does K
n,m
have a path that includes
every vertex exactly once.
16. For any graphs G and H the graph G ⊕ H is formed by placing an
edge between EVERY vertex of G and EVERY vertex of H. Let
n ≥ 3. Let C
n
= (V, E) be the cycle graph (i.e., V = {1, . . . , n} and
E = {(1, 2), . . . , (n −1, n), (n, 1)}.)
(a) Find all values of n, m such that C
n
⊕ C
m
4colorable but not
3colorable.
(b) Find all values of n, m such that C
n
⊕ C
m
5colorable but not
4colorable.
(c) Find all values of n, m such that C
n
⊕ C
m
6colorable but not
5colorable.
17. Show that if a graph on n vertices has ≥ n
√
n edges then it must have
a vertex of degree ≥ log n. (You may assume that n is large.)
18. Let G be a bipartite graph with m nodes on the left side and n nodes
on the right side. What is the maximum number of edges G can
have? What is the maximum number of edges G can have and still be
disconnected?
19. Deﬁne a notion of a kpartite graph. Phrase questions about kpartite
graphs similar to those in the last questions.
30
20. The courses 150, 251, 451, and 650 must be taught. Bill can teach 150
or 650. Clyde can teach 150 or 451. Diane can teach 451 or 650. Edna
can teach 150 or 251.
(a) Model this scenario with a graph.
(b) Find all possible schedules that assign teachers to classes so that
all four classes are taught and every teacher teaches just one class.
21. Let T be a weighted tree. Let f be some function that is easily com
puted (it goes from Naturals to Naturals). An fminimal spanning tree
is a spanning tree where the weights are w
1
, ..., w
m
and
m
i=1
f(w
i
) is
minimal among all spanning trees. Give an eﬃcient algorithm to ﬁnd
the fminimal spanning tree of a weighted tree.
22. Let G = (V, E) be a directed graph.
(a) Assuming that Gis represented by an adjacency matrix A[1..n, 1..n],
give a Θ(n
2
)time algorithm to compute the adjacency list repre
sentation of G. (Represent the addition of an element v to a list
l using pseudocode by l ←l ∪ {v}.)
(b) Assuming that G is represented by an adjacency list Adj[1..n],
give a Θ(n
2
)time algorithm to compute the adjacency matrix of
G.
(c) Is there a reasonable assumption you can make to get the run
time of part b down to O(E)?
23. If we take a complete undirected graph (a graph that has n vertices,
and edges between all pairs of vertices) and direct all the edges in some
arbitrary manner, we obtain a graph called a tournament. (Think of
n players playing a game, in which every two players play against
each other, and the edge is directed towards the winner.) A directed
Hamilton path in G, is a directed path that includes every vertex of
G exactly once. Prove that every tournament has a Hamilton path.
(Hint: Try Induction.)
24. Prove that in a graph G = (V, E), the number of vertices that have
odd degree are even. (Hint: First show that
v∈V
deg(v) = 2m where
m is the number of edges in G.)
31
25. Describe concisely and accurately a lineartime algorithm for solving
the singlesource shortest paths problem in a weighted DAG. Give a
rigorous, concise proof that your algorithm is correct.
Hints: Topological sort; relaxation lemma; try your alg. on examples.
26. Using depth ﬁrst search, outline an algorithm that determines whether
the edges of a connected, undirected graph can be directed to produce
a strongly connected directed graph. If so, the algorithm outputs such
an orientation of the edges. (Hint: Show that this can be done if and
only if removing an edge leaves a connected graph.)
27. An Euler Circuit for an undirected graph is a path which starts and
ends at the same vertex and uses each edge exactly once.
(a) Prove that a connected, undirected graph has an Euler circuit if
and only if every vertex has even degree.
(a) Give an O(V  + E) algorithm to ﬁnd an Euler circuit if one
exists.
28. A bipartite graph is one whose vertex set can be partitioned into two
sets A and B, such that each edge in the graph goes between a vertex
in A and a vertex in B. (No edges between nodes in the same set are
allowed.)
Give an O(E +V ) algorithm that takes an input graph (represented
in adjacency list form) and decides if the graph is bipartite. If the
graph is bipartite, the algorithm should also produce the bipartition.
(Hint: use BFS.)
29. Consider the following problem: There is a set of movies M
1
, M
2
, . . . , M
k
.
There is also a set of customers, with each one indicating the two
movies they would like to see this weekend. (Assume that customer i
speciﬁes a subset S
i
of the two movies that he/she would like to see.
Movies are shown on saturday evening and sunday evening. Multiple
movies may be screened at the same time.)
You need to decide which movies should be televised on saturday, and
which movies should be televised on sunday, so that every customer
gets to see the two movies they desire. The question is: is there a
schedule so that each movie is shown at most once? Design an eﬃcient
algorithm to ﬁnd such a schedule if one exists?
32
(One naive solution would be to show all movies on both days, so that
each customer can see one desired movie on each day. We would need
k channels if k is the number of movies in this solution – hence if there
is a solution in which each movie is shown at most once, we would like
to ﬁnd it.)
30. Consider a rooted DAG (directed, acyclic graph with a vertex – the
root – that has a path to all other vertices). Give a linear time (O(V +
E)) algorithm to ﬁnd the length of the longest simple path from the
root to each of the other vertices. (The length of a path is the number
of edges on it.)
31. The diameter of a tree T = (V, E) is given by
max
u,v∈V
δ(u, v)
(where δ(u, v) is the number of edges on the path from u to v). De
scribe an eﬃcient algorithm to compute the diameter of a tree, and
show the correctness and analyze the running time of your algorithm.
Hint: Use BFS or DFS.
32. Show that the vertices of any nvertex binary tree can be partitioned
into two disconnected sets A and B by removing a single edge such
that A and B are at most 3n/4. Give a simple example where this
bound is met.
33. Let G = (V, E) be an undirected graph. Give an O(n + e)time algo
rithm to compute a path in G that traverses each edge in E exactly
once in each direction.
item Suppose you are given a set of tasks T, where task t takes time
D[t] from start to ﬁnish. Tasks may be performed simultaneously, with
the constraint that for each task t there is a set of tasks P[t] which
must be completed before task t is begun.
Let R =
t∈T
P[t]. Describe (give pseudocode for) an O(T +
R)time, DFSbased algorithm which, given D and P, computes the
minimum amount of time needed to complete all the tasks, or returns
∞ if it is not possible to meet the constraints.
Hint: work bottomup.
33
34. a Show that if (u, v) is the lightest edge on a cut, there exists a
MST containing (u, v).
b Show that if (u, v) is the heaviest edge on a cycle, there exists a
MST not containing (u, v).
35. A pseudotree in an undirected graph is a connected subgraph that
is either a tree or a tree plus a single edge (so that it contains ex
actly one simple cycle). A pseudoforest in is a collection of vertex
disjoint pseudotrees. Describe and analyze a fast algorithm for ﬁnding
a pseudoforest of maximum total edge weight in an undirected graph
with nonnegative edge weights.
FACT: There are algorithms for this problem that run a least as fast
as the fastest known spanning tree algorithms.
36. Let G = (V, E) be a directed graph. The reversal of G is a graph
G
R
= (V, E
R
) where the directions of the edges have been reversed
(i.e. E
R
= {(i, j)(j, i) ∈ E}).
(a) Assuming that Gis represented by an adjacency matrix A[1..n, 1..n],
give an O(n
2
)time algorithm to compute the adjacency matrix
A
R
for G
R
.
(b) Assuming that G is represented by an adjacency list Adj[1..n],
give an O(n + e)time algorithm to compute an adjacency list
representation of G
R
.
37. Show that the vertices of any nvertex binary tree can be partitioned
into two disconnected sets A and B by removing a single edge such
that A and B are at most 3n/4. Give a simple example where this
bound is met.
34
Data Structures
1. Let p
1
, . . . , p
n
be n points in the plane, with both coordinates in the
set {1, 2, . . . , N}. Devise an O(n
2
) data structure for them such that
the following type of query can be answered in O(1) steps: given a
rectangle, how many or p
1
, . . . , p
n
are in the rectangle.
2. Recall that a heap allows inserts and deletes in O(log n) steps, and
FINDMAX in O(1) steps. Devise a structure that allows inserts and
deletes in O(log n) steps and both FINDMAX and FINDMIN in O(1)
steps.
3. Devise an (nlog n) algorithm for the following problem: Given n points
in the plane, ﬁnd the pair with the shortest distance between them.
(Hint: Sort the points on the xcoordinate and then store in some
cleverly chosen data structure.)
4. Let S be a set of n orthogonal lines in the plane (lines parallel to either
the xaxis or yaxis). Devise a Data Structure for the following such
that the following query will be easy: Given an orthogonal line (not
in S), which of the n lines in S does it intersect?”
5. Let S be a set of N intervals I
1
, . . . , I
N
which have endpoints in
{1, 2, . . . n}. Assume every interval in S has a natural number associ
ated to it. Devise a data structure such that the following operation
are O(log n).
(a) Given k, what is
k∈I
j
∈S
(element associated to I
j
).
(b) Update by changing the value associated to an interval.
6. Let S = {p
1
, . . . , p
n
} be n points in the plane. Devise a Data Structure
for S such that the following operations can be carried out quickly
(a) a) which pair of points is closest together,
(b) b) insert a point,
(c) c) delete a point.
7. Let S = {p
1
, . . . , p
n
} be n points in the plane. Assume they all have x
coordinate in the set {1, 2, . . . , N}. Devise a Data Structure to answer
the following type of query: Given x
0
, x
1
∈ {1, 2, . . . , N} and y
0
a
natural number, ﬁnd {(x, y)  x
0
≤ x ≤ x
1
, y ≤ y
0
} ∩ S. The data
35
structure should take up no more than O(N+n) space, and each query
should take O(k + log log n).
8. Let S = {w
1
, . . . , w
k
} where the w
i
are (very long) strings. Devise a
data structure that will support the following operations.
(a) FIND:Σ
+
→ Σ
+
. FIND(w) returns the longest preﬁx of w that
is a subword of some w
i
.
(b) FREQ:Σ
+
→N. FREQ(w) returns how often w occurs as a sub
word of any w
i
.
(c) LOC:Σ
+
→ 2
N×N
. LOC(w) returns the places w occurs, e.g. if
the output is {(1, 3), (1, 7), (4, 5)} then there is a w in w
1
starting
in the 3
rd
position, a w in w
1
starting in the 7
th
position, and a
w in w
4
starting in the 5
th
position.
9. Let A be a set of n disjoint sets. Devise a data structure that will
accommodate the following operations in the time bounds listed
(a) UNION(X, Y ). This sets X to X ∪ Y , and destroys Y .
(b) FIND(x). This returns where x is (recall that the sets are all
disjoint and these operations will keep them that way).
(c) DEUNION: Undoes the last union
(d) BACKTRACK: Undoes all the unions going back to and includ
ing the largest one formed.
10. Picture an environment where updates and queries are coming in
FAST. You might not have ANY time to do an update to the struc
ture, so you put the update itself in a buﬀer. Devise a structure for a
set on integers where the queries are just to see if some integer is in
the structure. How well does it do.
11. A Deap is a data structure that is a tree such that (a) there is nothing
at the root, (b) the left subtree is a Maxheap, (c) the right subtree
is a Minheap, (d) every leaf in the Maxheap is bigger that its corre
sponding leaf in the Minheap.
Show how to implement FINDMIN, FINDMAX in O(1), INSERT,
and DELETEMIN and DELETEMAX in O(log n), and CREATE in
O(n). (My notes about the reference claim that INSERT can be done
in O(log log n) steps — but I am not sure I believe that.
36
12. Devise a Data Structure for a weighted Directed Graph on n nodes
that can do the following operations quickly.
(a) Merge two components,
(b) locate which component a vertex is in,
(c) retrieve minimum edge in a component.
13. Devise a Data Structure for a directed graph that can answer the
following three types of queries.
(a) Given two vertices x and y, determine if there is a path between
them. If so, then output the path. This should take time O(1) if
there is no path, and O(LENGTH OF PATH) if there is a path.
(b) Given edge, delete it.
(c) Given a nonedge, add it as an edge.
14. Consider the movetofront heuristic where you decide whether or not
to move the item up probabilistically: the more often the item has
been accessed, the more likely to move it up. Write pseudocode to
implement this heuristic.
15. Devise a data structure to store n horizontal lines so that you can
tell, given an inﬁnite vertical line, if it hits one of them. The data
structure should take O(n
2
) space and be able to answer the query in
O(1). Look into variants.
37
Analysis of Algorithms
1. For the following program ﬁnd a function f(n) such the runtime is
Θ(f(n)).
BEGIN
Input(n)
For i := 1 to n do [TAKES 5
i
STEPS]
i := n
While i ≥
n
2
do
begin
SOMETHING THAT TAKES i STEPS
i := i −1
END
2. Assume you could multiply two 3 × 3 matrices in 18 multiplications
and 54 additions. How fast can you multiply two n ×n matrices (you
may assume n is a power of 3). Is this better than the algorithm in
class for n × n matrices? Give conditions on a, b, c, and d such the
following statement holds when a, b, c, and d satisfy that condition:
“If one could multiply 2 a × a matrices using b multiplications and c
additions then one could multiply two n ×n matrices in a better way
than that shown in class.” (Try to make the conditions as general as
possible.)
3. The result of applying Gaussian elimination to a square matrix is an
upper triangular matrix, U. Suppose you wish to solve the equation
Ux = c
for the unknown vector x, where U has n rows and columns and x
and c are vectors of size n. How many multiplications are required?
Justify your answer.
38
Divide and Conquer
1. Assume we could multiply two 3 digit numbers with k multiplications.
a. How fast (just the order, i.e. do not worry about the constants)
is a new “fancy” multiplication algorithm based on this fact?
b. How small does k have to be for this algorithm to be asymptoti
cally faster than the “standard” (O(n
2
)) algorithm?
c. How small does k have to be for this algorithm to be asymptoti
cally faster than the O(n
lg 3
) algorithm?
2. Assume that additions and multiplications of numbers both take ex
actly one time unit.
(a) Exactly how many multiplications does the standard matrix mul
tiplication algorithm use to multiply two n×n matrices? Exactly
how many additions does it use? What is the exact time it uses
to multiply two n ×n matrices?
(b) What is the exact time to multiply two n × n matrices using
Strassen’s algorithm?
(c) For what values of n is Strassen’s algorithm better than the stan
dard algorithm? Justify your answer.
3. Devise an O(nlog n) algorithm for the following problem: Given n
points in the plane, ﬁnd the pair that is closed together.
39
Dynamic Programming
1. Let k, n ∈ N. The (k, n)egg problem is as follows: There is a building
with n ﬂoors. You are told that there is a ﬂoor f such that if an egg is
dropped oﬀ the fth ﬂoor then it will break, but if it is dropped oﬀ the
(f −1)st ﬂoor it will not. (It is possible that f = 1 so the egg always
breaks, or f = n + 1 in which case the egg never breaks.) If an egg
breaks when dropped from some ﬂoor then it also breaks if dropped
from a higher ﬂoor. If an egg does not break when dropped from some
ﬂoor then it also does not break if dropped from a lower ﬂoor. Your
goal is, given k eggs, to ﬁnd f. The only operation you can perform is
to drop an egg oﬀ some ﬂoor and see what happens. If an egg breaks
then it cannot be reused. You would like to drop eggs as few times
as possible. Let E(k, n) be the minimal number of eggdroppings that
will always suﬃce.
(a) Show that E(1, n) = n.
(b) Show that E(k, n) = Θ(n
1
k
).
(c) Find a recurrence for E(k, n). Write a dynamic program to ﬁnd
E(k, n). How fast does it run?
(d) Write a dynamic program that can be used to actually yield the
optimal strategy for ﬁnding the ﬂoor f. How fast does it run?
2. Let α and β be constants. Assume that in a tree it costs α to go left,
and β to go right. Devise an algorithm that will, given keys k
1
, . . . , k
n
which have probabilities of being searched p
1
, . . . , p
n
, builds a tree with
optimal worst case cost.
3. Consider the following greedy algorithm for solving the chained matrix
multiplication problem: Let the matrices A
1
, ..., A
n
, have dimensions
p
1
×q
1
, ..., p
n
× q
n
. Look for the smallest dimension, say q
i−1
and p
i
,
and parenthesize between matrices A
i−1
and A
i
, so that we multiply
A
1
to A
i−1
in the best possible way, A
i
to A
n
in the best possible
way, and ﬁnally multiply the two resulting matrices. In order the
determine how to multiply A
1
to A
i−1
and A
i
to A
n
we apply the
algorithm recursively.
Show that this greedy algorithm does not ﬁnd the optimal way to
multiply a chain of matrices.
40
4. Write an eﬃcient algorithm to determine an order of evaluating the
matrix product M
1
× M
2
× M
3
. . . ×M
n
so as to minimize the scalar
multiplications in the case where each M is of dimension 1 × 1, 1 ×
d, d ×1, d ×d for some ﬁxed d.
5. We are given a collection of n books, which must be placed in a library
book case. Let h[1..n] and w[1..n] be arrays, where h[i] denotes the
height of the ith book and w[i] denotes the width of the ith book.
Assume that these arrays are given by the books’ catalogue numbers
(and the books MUST be shelved in this order), and that books are
stacked in the usual upright manner (i.e., you CANNOT lay a book
on its side). All book cases have width W, but you may have any
height you like. The books are placed on shelves in the book case.
You may use as many shelves as you like, placed wherever you like (up
to the height of the book case), and you may put as many books on
each shelf as you like (up to the width of the book case). Assume for
simplicity that shelves take up no vertical height.
The bookshelver’s problem is, given h[1..n], w[1..n], and W, what is the
minimum height book case that can shelve all of these books. Below
shows an example of a shelving. The height and width of the 6th
book are shown to the right. (Notice that there is some unnecessarily
wasted height on the third shelf.)
h[6]
W
H
w[6]
(a) Write down a recurrence for book shelver’s problem.
(b) Use your recurrence to develop an eﬃcient dynamic programming
for solving the book shelver’s problem.
(c) Analyze the running time of your algorithm.
41
6. Consider the problem of not only ﬁnding the value of the maximum
contiguous subsequence in an array, but also determining the two end
points. Give a linear time algorithm for solving this problem. [What
happens if all entries are negative?]
7. Assume that there are n numbers (some possibly negative) in an array
, and we wish to ﬁnd the maximum contiguous sum. Give an eﬃcient
algorithm for solving this problem. What is its time?
8. Assume that there are n numbers (some possibly negative) in a circle,
and we wish to ﬁnd the maximum contiguous sum on the circle. Give
an eﬃcient algorithm for solving this problem. What is its time?
9. Given an array a[1, . . . , n] of integers, consider the problem of ﬁnding
the longest monotonically increasing subsequence of not necessarily
contiguous entries in your array. For example, if the entries are 10, 3,
9, 5, 8, 13, 11, 14, then a longest monotonically increasing subsequence
is 3, 5, 8, 11, 14. The goal of this problem is to ﬁnd an eﬃcient dynamic
programming algorithm for solving this problem.
HINT: Let L[i] be the length of the longest monotonically increasing
sequence that ends at and uses element a[i].
(a) Write down a recurrence for L[i].
(b) Based on your recurrence, give a recursive program for ﬁnding
the length of the longest monotonically increasing subsequence.
What can you say about its running time?
(c) Based on your recurrence, give a Θ(n
2
) dynamic programming
algorithm for ﬁnding the length of the longest monotonically in
creasing subsequence.
(d) Modify your algorithm to actually ﬁnd the longest increasing sub
sequence (not just its length).
(e) Improve your algorithm so that it ﬁnds the length of the longest
monotonically increasing subsequence in Θ(nlg n) time.
10. Consider the problem of computing the probability of which team will
win a game consisting of many points. Assume team A has probability
p of winning each point and team B has probability q = 1−p of winning
each point. Let P(i, j) be the probability that team A will win the
game given that it needs i points to win and team B needs j points.
42
(a) Write down a recurrence for P(i, j).
(b) Use your recurrence to develop an eﬃcient dynamic programming
algorithm for determining the probability that team A will win
the game given that it needs i points and team B needs j points.
(c) Analyze the running time of your algorithm.
11. The traditional world chess championship is a match of 24 games. The
current champion retains the title in case the match is a tie. Not only
are there wins and losses, but some games end in a draw (tie). Wins
count as 1, losses as 0, and draws as 1/2. The players take turns
playing white and black. White has an advantage, because he moves
ﬁrst. Assume the champion is white in the ﬁrst game, has probabilities
w
w
, w
d
, and w
l
of winning, drawing, and losing playing white, and
has probabilities b
w
, b
d
, and b
l
of winning, drawing, and losing playing
black.
(a) Write down a recurrence for the probability that the champion
retains the title. Assume that there are g games left to play in
the match and that the champion needs to win i games (which
may end in a 1/2).
(b) Based on your recurrence, give a dynamic programming to cal
culate the champion’s probability of retaining the title.
(c) Analyze its running time assuming that the match has n games.
12. You are given an n × n matrix A whose entries are either 0 or 1. We
want to decompose this matrix into rectangular submatrices such that
the entries in any submatrix are either all 0 or all 1. We limit the
decompositions to a special type called guillotine decompositions. A
guillotine decomposition of a subarray A[i
1
..i
2
, j
1
..j
2
] is deﬁned recur
sively, as follows. If i
1
= i
2
and j
1
= j
2
, then no further decomposition
is possible. If i
1
< i
2
then select any i, i
1
≤ i < i
2
and split the array
horizontally just after row i, creating two new subarrays A[i
1
..i, j
1
..j
2
]
and A[(i + 1)..i
2
, j
1
..j
2
]. If j
1
< j
2
then select any j, j
1
≤ j < j
2
and split the array vertically just after column j, creating two new
subarrays A[i
1
..i
2
, j
1
..j] and A[i
1
..i
2
, (j + 1)..j
2
]. If both i
1
< i
2
and
j
1
< j
2
then you are free to make either type of cut. Once the two
submatrices are formed, they may be further decomposed by a guil
lotine decomposition. An example of a guillotine decomposition into
submatrices of one value is shown below.
43
0
0
0
0
0 0
1 1
1 1
1
1
1
1
1
1
0 0
0 0
0 0
0
1 0 0 1
0
0 0
0 0
0 0
1
1
1
1
1
1
1 1
1 1
0 0
0
0
0
0
Write a dynamic programming algorithm to determine the MINIMUM
number of guillotine cuts needed to decompose the matrix A into sub
matrices of one value. Justify the correctness of your algorithm, and
derive its running time. (Note: You do not have to output the actual
cuts themselves, just the number of cuts.)
13. Consider the problem of neatly printing a paragraph on a printer. The
input text is a sequence of n words of lengths l
1
, l
2
, . . . , l
n
, measured
in characters. We want to print this paragraph neatly on a number
of lines that hold a maximum of M characters each. Our criterion for
“neatness” is as follows. There MUST be at least one space between
any two words on the same line. Except for the last line, if a given
line contains words i through j, word i starts in the leftmost position
of the line, and word j ends in the rightmost position of the line, and
space is inserted between the words as evenly as possible. The amount
of extra space between any two words on this line is
M −
j
k=i
l
k
j −i
−1,
and the total cost of the line is the cube of this value multiplied times
(j −i). For the last line, we simply put one space between each word,
starting at the leftmost position of the line. We wish to minimize the
sum of costs of all lines (except the last). Give a dynamic programming
algorithm to determine the neatest way to print the words. It suﬃces
to determine the minimum cost, not the actual printing method. You
may assume that every word has length less than M/2, so that there
are at least two words on each line. Analyze the running time and
space of your algorithm.
44
Greedy Algorithms
1. Write a greedy algorithm for the Traveling Salesperson Problem Find
an inﬁnite set of graphs for which it ﬁnds the optimal solution (no proof
required). (The Traveling Salesperson problems is, given a weighted
graph G, ﬁnd the minimum cost hamiltonian cycle.)
2. Give a GREEDY ALGORITHM that tries to ﬁnd a coloring of a graph
that uses the least number of colors. Give a class of graphs for which
your algorithm succeeds in ﬁnding the least number of colors needed.
Give a class of graphs for which your algorithm always uses more colors
then it has to.
3. Assume that you are given a set {x
1
, . . . , x
n
} of n points on the real
line and wish to cover them with unit length closed intervals.
(a) Describe an eﬃcient greedy algorithm that covers the points with
as few intervals as possible.
(b) Prove that your algorithm works correctly.
4. While walking on the beach one day you ﬁnd a treasure trove. Inside
there are n treasures with weights w
1
, . . . , w
n
and values v
1
, . . . , v
n
.
Unfortunately you have a knapsack that only holds a total weight M.
Fortunately there is a knife handy so that you can cut treasures if
necessary; a cut treasure retains its fractional value (so, for example,
a third of treasure i has weight w
i
/3 and value v
i
/3).
(a) Describe a Θ(nlg n) time greedy algorithm to solve this problem.
(b) Prove that your algorithm works correctly.
(c) Improve the running time of your algorithm to Θ(n).
5. Modify Kruskal’s algorithm as follows. Sort the edges of the graph
in DECREASING order, and then run the algorithm as before. True
or False: This modiﬁed algorithms produces a Maximum Cost Span
ning Tree. If True then prove your answer, and if False then give a
counterexample.
6. Modify Dijkstra’s algorithm as follows. (1) Initialize all d[v] values to
−∞, (2) rather than performing an ExtractMin, perform an Extract
Max, and (3) in relaxation, reverse all comparisons so that the shortest
45
path estimate d[v] increases rather than decreases. True or False: This
modiﬁed algorithm determines the cost of the longest simple path from
the source to each vertex. If True then prove your answer, and if False
then give a counterexample.
7. Suppose we are given a graph G (connected, undirected) with costs on
the edges (all costs > 0). Now we construct a new graph G
, which is
the same as G, except for the costs. The cost of an edge e is deﬁned
to be 1/c
e
where c
e
is the cost of e in G.
(a) Is the Minimum cost spanning tree in G, the Maximum cost span
ning tree in G
? Prove or disprove.
(b) Suppose that P is the shortest path from s to v in G. Is P the
longest simple path from s to v in G
? Prove, or disprove.
8. Assume that we have a network (a connected undirected graph) in
which each edge e
i
has an associated bandwidth b
i
. If we have a path
P, from s to v, then the capacity of the path is deﬁned to be the mini
mum bandwidth of all the edges that belong to the path P. We deﬁne
capacity(s, v) = max
P(s,v)
capacity(P). (Essentially, capacity(s, v) is
equal to the maximum capacity path from s to v.) Give an eﬃcient
algorithm to compute capacity(s, v), for each vertex v; where s is some
ﬁxed source vertex. Show that your algorithm is “correct”, and ana
lyze its running time.
(Design something that is no more than O(n
2
), and with the right
data structures takes O(mlg n) time.)
9. Suppose that your boss gives you a connected undirected graph G,
and you compute a minimum cost spanning tree T. Now he tells
you that he wants to change the cost of an edge. Your job is to
recompute the new minimum spanning tree (say T
). A good algorithm
should recompute the MST by considering only those edges that might
change.
For each of the following situations, indicate whether 1) the MST
cannot change 2) the MST can possibly change. If case 2) occurs then
describe what changes might occur in the MST. State WHICH edges
might possibly be deleted from the MST, and WHICH edges might
possibly be inserted into the MST, and HOW you would select the
edges to be inserted/deleted. (For extra credit: give a proof for your
scheme.)
46
(a) He decreases the cost of an edge e that belongs to T.
(b) He increases the cost of an edge e that belongs to T.
(c) He decreases the cost of an edge f that does not belong to T.
(d) He increases the cost of an edge f that does not belong to T.
You do not need to design an algorithm, but to simply state the edges
that need to be considered.
10. Prove that if a connected undirected graph has distinct edge weights,
then the minimum cost spanning tree is unique. (Hint: Recall the
proof of Kruskal’s algorithm.)
11. Prove the following lemma:
If an edge of an undirected weighted graph is the unique most expensive
edge on a cycle, then it is not in any minimumweight spanning tree.
12. Modify Dijkstra’s algorithm as follows. (1) Initialize all d[v] values to
−∞, (2) rather than performing an ExtractMin, perform an Extract
Max, and (3) in relaxation, reverse all comparisons so that the shortest
path estimate d[v] increases rather than decreases. True or False: This
modiﬁed algorithm determines the cost of the longest simple path from
the source to each vertex. If True then prove your answer, and if False
then give a counterexample.
13. A coloring of an undirected graph is assignment of colors to the nodes
so that no two neighboring nodes have the same color. In the graph
coloring problem, we desire to ﬁnd the minimum number of colors
needed to color an undirected graph.
One possible greedy algorithm for this problem is to start with an
arbitrary node and color it “1”. Then to consider every other node
one at a time and color it “1” if it does not neighbor a node already
colored “1”. Now do the same for all of the remaining nodes using the
color “2”. Continue in this fashion until all of the nodes in the graph
are colored.
(a) How would you implement this algorithm eﬃciently? Be brief but
clear. What is your running time as a function of the number of
vertices V , edges E, and colors C.
(b) Prove that for every graph, the algorithm might get lucky and
color it with as few colors as possible.
47
(c) Prove that the algorithm can be arbitrarily bad. In other words,
for every constant α > 0 the ratio between the number of colors
used by the greedy algorithm and by the optimal algorithm could
be at least α.
48
Sorting and Selection
1. Assume you have some kind of superhardware that, when given two
lists of length m that are sorted, merges them into one sorted list,
and takes only
√
m steps. Write down a recursive algorithm that uses
this hardware to sort lists of length n. Write down a recurrence to
describe the run time. (For your own enlightenment replace
√
m with
other functions and see what happens.)
2. Consider the following variation of mergesort. Given a set of n num
bers, divide it into three subsets of equal size, sort each of the three
subsets, merge the ﬁrst two subsets in sorted order, and then merge
the result with the third subset in sorted order. You may assume that
you have available a procedure merge that merges two sorted lists of
size n
1
and n
2
in time n
1
+n
2
. You may also assume that n is a power
of three.
(a) Write an informal program that could implement this algorithm.
(b) Analyze the cost of this algorithm.
3. Let a be a number such that 0 ≤ a ≤ 1. Assume that you have a
subroutine that ﬁnds the minimum of n numbers in n
a
steps. Using
this subroutine, write an algorithm that sorts a list of n numbers.
How fast does your algorithm run (you may use θnotation). For what
values of a is the algorithm better than θ(nlog n) (you may assume n
is large and reason asymptotically).
4. Assume that you have specialpurpose hardware that can take two
sorted lists of equal size n and merge them in
(n) steps. Devise
a sorting algorithm using this hardware that performs substantially
better than O(nlog n).
5. Let A[1..n] be an array such that the ﬁrst n−
√
n elements are already
sorted (though we know nothing about the remaining elements. Write
an algorithm that will sort A in substantially better than nlog n steps.
6. For what values of α, 0 ≤ α ≤ 1 is the following statement TRUE:
Given an array A[1..n] where the ﬁrst n − n
α
are sorted (though we
know nothing about the remaining elements), A can be sorted in linear
time (i.e., O(n) steps).
49
7. Let P be a program that has the following format.
PROCEDURE P
begin
Input(n) (this takes 0 steps)
if n = 1 then execute 4 operations and stop. (if n = 1 this takes 0
operations.)
Execute n operations
Call P on input
n
3

Execute n operations
Call P on input
n
4

end
Write a recurrence for T(n), the running time of this program. Find
a number c such that for all n ≥ 1, T(n) ≤ cn.
8. Assume you had a way of expressing the product of two numbers of n
bits long by using a products of numbers that are
n
b
long, along with
addition of c numbers. How fast can you multiply two numbers of n
bits long. For what values of a, b, c is this better than the algorithm
in class.
9. In American we have coins for 1 cent, 5 cents, 10 cents, and 25 cents.
Show that if you want x cents and the least amount of coins, the greedy
algorithm is optimal. Exhibit of set of coin denominations for which
greedy is not optimal, the includes a 1 cent coin.
10. Devise a structure that allows MAX and MIN in O(1) steps, INSERT
in O(log n) steps, and DELETE in O(n). (For your own enlighten
ment, and ONE extra point, come up with a structure that allows
MAX and MIN in O(1) steps, INSERT in O(log n), and DELETE
also in O(log n).)
11. You wish to store a set of n numbers in either a heap or a sorted array.
(Heap is MAXheap, has MAX element on top, etc.) For each of the
following desired properties state whether you are better oﬀ using a
heap or an ordered array, or that it does not matter. Justify your
answers.
50
(a) Want to ﬁnd the MAX quickly.
(b) Want to be able to delete an element quickly.
(c) Want to be able to form the structure quickly.
(d) Want to ﬁnd the MIN quickly.
12. Given n elements we are going to, in the course of time, (1) do an add
or delete a times, (2) try to ﬁnd MAX b times, (3) try to ﬁnd MIN
c times. We can assume a, b, c are much smaller than n. For what
values of a, b, c are we better oﬀ using a heap? For what values are we
better oﬀ using a sorted array. (You must also take into consideration
the time spent forming these structures.)
13. Assume that we are going to be sorting arrays A[1..n] but that we have
a MAGIC algorithm that sorts arrays of size ≤
n
3
in n steps. Rewrite
the pseudocode for QUICKSORT on page 154 so that it can use this
MAGIC algorithm. What is the worst case performance? What is the
best case performance?
14. For each of the following scenarios indicate if you would use QUICK
SORT or HEAPSORT and why.
(a) It is absolutely crucial that the array gets sorted quickly.
(b) The array is such that you have a good idea of where the median
is.
(c) Space is a crucial parameter you can’t use even a little bit of
extra space.
(d) Space is relevant, but using a little extra space is okay.
15. Assume that you have hardware that can sort log n elements in ONE
step. We are thinking of sorting a list of n elements using this hard
ware. What is a LOWER BOUND on how well we can do? Explain
your result. (You can assume that the only operations allowed are
comparisons, the operation above, and swaps. Hence a variation of
the decision tree model is appropriate.)
16. Let f(n) be a function of n. Assume that we are going to sort n
numbers from the set ¦1, . . . , f(n)¦. Assume that we use RADIX sort.
Find an inﬁnite number of functions f such that RADIX SORT per
forms substantially better than O(nlog n).
51
17. The algorithm for bucket sort is geared to take inputs in the range
[0, 1) (fractions allowed of course). Let p be a positive natural number.
Adjust the algorithm so it takes inputs that are natural numbers in
¦1, . . . , pn¦ and uses p buckets. Discuss informally how this algorithm
might behave if the inputs are chosen with the distribution Prob(x =
i) =
K
i
(Where K is the appropriate constant). How might you adjust
the algorithm so that it performs well for this distribution?
18. Consider the following two statements: (1) Sorting n elements RE
QUIRES Ω(nlog n) operations, (2) Radix sort, counting sort, and
bucket sort perform in time substantially better than O(nlog n). These
statements seem to be in contradiction. Explain carefully why they
are not.
19. Assume that you can ﬁnd the median of 7 elements in 12 comparisons.
Devise a lineartime MEDIAN FINDING algorithm that uses this.
Give an upper bound on the number of comparisons used (do NOT
use Onotation). Try to make the bound as tight as possible.
20. Let a, b be numbers. Assume that you can ﬁnd the median of a el
ements in b comparisons. Use this in the lineartime median ﬁnding
algorithm. Give an upper bound on the number of comparisons used
(do NOT use Onotation). Try to make the bound as tight as possible.
Find values of a, b such that you can ﬁnd medians in ≤ 4n comparisons.
21. For each of the following scenarios say whether you would use MERGE
SORT, QUICKSORT (or variants), RADIX SORT, or COUNTING
SORT. Brieﬂy explain why you chose the one you did.
(a) The elements to be sorted are numbers from the set ¦1, . . . , log n¦
but the only operations we are allowed to use are COMPARE and
SWAP.
(b) We have access to a VERY FAST median ﬁnding algorithm that
we can use as a subroutine.
(c) The elements to be sorted are numbers from the set ¦1, . . . , n¦
and we have access to the actual bits of the number.
(d) No bound is known on the elements to be sorted, but we need to
guarantee fast performance.
52
22. Assume that c ≥ 0. Assume you had some kind of superhardware
that, when given two lists of length m that are sorted, merges them
into one sorted list, and takes only m
c
steps. Write down a recursive
algorithm that uses this hardware to sort lists of length n Write down
a recurrence to describe the run time. For what values of c does this
algorithm perform substantially better than O(nlog n)?
23. Assume that n numbers are in a MAXheap.
(a) Show that the third largest element can be found in O(1) steps.
(b) Show that if i is any constant then the ith largest element can be
found in O(1) steps
24. Let d(n) be a function of n such that d(n) << n. Imagine that we
redesigned MAXheaps so that instead of every internal node hav
ing 2 children (except possibly the rightmost internal node on the
secondtolast level, which may have 1 or 2 children), they had d(n)
children (except possibly the rightmost internal node on the second
tolast level, which may have 1, . . . , d(n) children). How fast would the
following operations be? You may use Θnotation. (a) FIND MAX,
(b) INSERT, (c) DELETE MAX, (d) FIND (given a key, determine
whether it is in the structure).
25. Let d ≥ 0. We have special hardware labeled MAGIC that, given
an array B[1..m], places the smallest
m
2
elements into B[1..
m
2
] (not
necessarily in order), places the largest
m
2
into B[(
m
2
+ 1)..m] (not
necessarily in order), and returns the index of the largest element in the
ﬁrst partition (i.e., MAGIC does what PARTITION does in the usual
QUICKSORT algorithm). Assume that this hardware works in m
d
steps. Describe a sorting algorithm similar to QUICKSORT that uses
MAGIC to sort A[1..n]. For what d, will this sorting algorithm perform
(1) much better than Θ(nlog n), (2) much worse than Θ(nlog n), (3)
in Θ(nlog n)?
26. In answering a, b, and c no explanations are needed. Also, QUICK
SORT refers to the randomized version.
(a) Give a scenario where RADIXSORT is better than MERGE
SORT.
(b) Give a scenario where MERGESORT is better than QUICK
SORT.
53
(c) Give a scenario where QUICKSORT is better than RADIXSORT.
27. Let a, b ≥ 0. We are going to be sorting numbers between 1 and
n
a
(log n)
b
. Find all values of a, b such that COUNTING SORT will
perform substantially better than O(nlog n).
28. We are going to be sorting numbers in the interval [0, 1) using BUCK
ETSORT. (We use the version in the book where there are n buckets
and each individual one is sorted by INSERTION SORT.)
(a) Find an increasing function f such that the following holds: ‘If
when doing BUCKETSORT we are guaranteed that no bucket
has more than f(n) elements in it, then BUCKETSORT is guar
anteed to do substantially better than O(nlog n).’
(b) Find an inﬁnite number of such functions with diﬀerent orders of
magnitude.
29. Assume you have some kind of superhardware that, when given two
lists of length m that are sorted, merges them into one sorted list,
and takes only
√
m steps. Write down a recursive algorithm that uses
this hardware to sort lists of length n. Write down a recurrence to
describe the run time. (For your own enlightenment replace
√
m with
other functions and see what happens.)
30. Assume that we have a MAGIC ALGORITHM that can, given an
array of n elements, ﬁnds the
n
3
rd largest element in
√
n comparisons.
Use this to devise an algorithm that takes cnlog n comparisons. similar
to QUICKSORT. What is the lowest YOU can make c? (This last part
is for speculation, not serious grading.)
31. Assume that we have MAGIC HARDWARE that can sort k elements
in 1 step.
(a) Give an algorithm for sorting (n elements) that uses this hard
ware. What is its running time?
(b) Give a lower bound on how well any algorithm with this hardware
can do.
32. Assume that we have MAGIC HARDWARE that can ﬁnd the MAX
of three elements in 1 step.
54
(a) Give an algorithm for MAX (of n elements) that uses this hard
ware. What is its running time?
(b) Give a lower bound on how well any algorithm with this hardware
can do. It should match the upper bound.
33. Assume that the array A[1..n] only has numbers from ¦1, . . . , n
2
¦ but
that at most log log n of these numbers ever appear. Devise an algo
rithm that sorts A in substantially less than O(nlog n).
34. In some applications the lists one wants to sort are already ‘partially
sorted’ Come up with a formal way to measure ‘how sorted’ a list
is. Devise an algorithm that does very well on lists that are almost
sorted (according to your deﬁnition). (Note— there is no one ‘right
answer’ to this problem, however whatever answer you give must be
well thought out. Your deﬁnition must be rigorous.)
35. Let a, b be real numbers such that a, b ≥ 0. Assume you have some
kind of superhardware that, when given 12 sorted lists of length m,
merges them in a
m
m
b
steps.
(a) Write an algorithm that uses this hardware to, given an array
A[1..n], sort it.
(b) Let T(n) be the running time of your algorithm. Find a function
f
a,b
such that T(n) = θ(f
a,b
). (The function f
a,b
might involve
several cases.)
(c) For what values of a, b does your algorithm run substantially
faster than O(nlog n)?
36. Let a FEAP be just like a HEAP except that the internal nodes have
FIVE children instead of TWO (only the right most internal node
might have less than FIVE children).
(a) Give an algorithm that does the following: given a FEAP and a
new element, remove the root of the FEAP and insert the new
element. (When you are done you must have a FEAP again.)
You must describe your algorithm informally (i.e., NO CODE).
Give an example of your algorithm at work.
(b) How many comparisons does your algorithm take in the worst
case? How many comparisons does your algorithm take in the
55
best case? How many moves does it make in the worst case?
How many moves does it make in the best case? (Do not ignore
multiplicative constants.)
37. Assume that the array A[1..n] only has numbers from ¦1, . . . , n
1.2
¦ but
that all numbers that appear are multiples of
√
n. (You may assume
that
√
n and n
1.2
are integers.) Devise an O(n) algorithm to sort A.
38. Let a, b be real numbers such that a, b ≥ 0. Assume you have some
kind of superhardware that, when given an array A[1..m] returns
the following: the (
m
5
)
th
largest element, the (
2m
5
)
th
largest element,
the (
3m
5
)
th
largest element, and the (
4m
5
)
th
largest element. Assume
this hardware runs in time m
a
(log m)
b
. (Formally the numbers above
should be
m
5
, etc., but you may ignore this for the purposes of this
problem.)
(a) Write an algorithm that uses this hardware to sort lists of length
n.
(b) Let T(n) be the running time of your algorithm. Find a function
f
a,b
such that T(n) = θ(f
a,b
). (The function f
a,b
might involve
several cases and might depend on a and b.)
39. (a) Exhibit an O(log n) algorithm that will, given a heap on n ele
ments and a number m with m ≤ log log n, ﬁnd and delete the
mth largest element in the heap. (The end result is a heap that
has that one element deleted.)
(b) Assume m is a constant. Find functions g
B
(n) and g
W
(n) such
that your algorithm takes g
B
(n) + θ(1) comparisons in the Best
case, and g
W
(n) + θ(1) comparisons in the Worst case. (Note
that multiplicative constants are important.)
(c) Assume m is log log n. Find functions h
B
(n) and h
W
(n) such
that your algorithm takes h
B
(n) + θ(1) comparisons in the Best
case, and h
W
(n) + θ(1) comparisons in the Worst case. (Note
that multiplicative constants are important.)
40. Consider the following problem: given a list of n elements, output
some element larger than the median element. Derive both upper and
lower bounds for this problem in terms of the number of comparisons
needed. Also consider the model where you can ask k comparisons at
a time.
56
41. Assume that we have MAGIC HARDWARE that can ﬁnd the 3rd
largest of log n elements in 1 step. Give a lower bound on how well
any algorithm with this hardware can do.
42. Let A be an array of positive integers (> 0), where A(1) < A(2) <
. . . < A(n). Write an algorithm to ﬁnd an i such that A(i) = i (if one
exists). What is the running time of your algorithm ?
If the integers are allowed to be both positive and negative, then what
is the running time of the algorithm (Hint: redesign a new algorithm
for this part.) ?
43. Consider a rectangular array of distinct elements. Each row is given
in increasing sorted order. Now sort (in place) the elements in each
column into increasing order. Prove that the elements in each row
remain sorted.
44. The professor claims that the Ω(nlg n) lower bound for sorting n num
bers does not apply to his workstation, in which the control ﬂow of the
program can split three ways after a single comparison a
i
: a
j
, accord
ing to whether a
i
< a
j
, a
i
= a
j
, or a
i
> a
j
. Show that the professor is
wrong by proving that the number of three way comparisons required
to sort n elements is still Ω(nlg n).
45. Let hilo BE the problem of, given n numbers, ﬁnd the smallest and
largest elements on the list. You may assume that n is even.
(a) Give an algorithm to solve HILO with 2n − 3 comparisons.
(b) Give an algorithm that uses substantially less than 2n compar
isons. (HINT: Imagine you are running a tennis tournament and
wish to ﬁnd the best and worst players.)
(c) Show that the algorithm presented in the last part is optimal.
46. Write an algorithm Partition(A,p,r,s) to partition array A from p
to r based on element A[s], using exactly r −p comparisons.
47. Show how to implement quicksort in worst case O(nlg n) time. Your
algorithm should be high level and very brief. (HINT: We need to
choose the partition element carefully.) Supply the analysis, which
should also be very brief.
57
48. Consider the following quicksortlike sorting algorithm. Pick two el
ements of the list. Partition based on both of the elements. So the
elements smaller than both are to the left, the elements in between are
in the middle, and the elements larger than both are to the right.
a. Write high level code for this algorithm.
b. How many comparisons and exchanges does the partition algo
rithm use?
c. Assume the partition elements always partition exactly into thirds.
Write a recurrence for T(n), the number of comparisons. Find a
constant a such that, for all n, T(n) ≤ an.
d. Assume the partition elements always partition so exactly one
quarter are to the left, one half in the middle, and one quarter
to the right. Write a recurrence for T(n), the number of compar
isons. Find a constant a such that, for all n, T(n) ≤ an.
e. Can you reﬁne your analysis of this version of quicksort? For
example, can you ﬁnd its average case?
49. The usual deterministic linear time (worst case) SELECT algorithm
begins by breaking the set into groups of 5 elements each. Assume that
we changed the algorithm to break the set into groups of 7 elements
each.
a. Write down the recurrence for the new running time. (You can
let c be the number of comparisons needed to ﬁnd the median of
7 elements.)
b. Solve the recurrence.
50. A dary heap is like a binary heap, but instead of two children, nodes
have d children.
(a) How would you represent a dary heap in an array?
(b) What is the height of a dary heap of n elements in terms of n
and d.
(c) Explain loosely (but clearly) how to extract the maximum ele
ment from the dary heap (and restore the heap). How many
comparisons does it require?
(d) How many comparisons does it take to sort? Just get the high
order term exactly, but show your calculations.
58
(e) What value(s) of d are optimal? Justify your answer.
51. We are going to derive the average number of moves for quicksort using
a somewhat unusual partitioning algorithm. We partition on the ﬁrst
element. Take it out. Look for the right most element that is smaller
and place it in the ﬁrst position (which is the newly opened position
on the left side). Look for the left most element that is larger and
place it in the newly opened position on the right side. Starting from
there look for the right most element that is smaller and place it in
the the newly opened position on the left side. Starting from there
look for the left most element that is larger and place it in the newly
opened position on the right side. Continue in this fashion until the
pointers cross. Finally, put the partition element into the hole, which
is its ﬁnal position in the sorted array.
(a) Assume that the partition element ends up in position q. What
is the probability that an element originally to the left went to
the right?
(b) Assume that the partition element ends up in position q. What
is the expected number of elements originally to the left that go
to the right?
(c) Assume that the partition element ends up in position q. What
is the expected number of moves?
(d) Write a recurrence for the expected number of moves.
(e) Simplify the recurrence, but do not solve it.
(f) To keep the calculations simple, assume you got the following
recurrence:
T(n) =
2
n
[
n−1
q=1
T(q) +q −
q
2
n
]
Use constructive induction to show that T(n) ≤ anlnn. Try to
get a good value of a.
(g) Heapsort makes about nlg n moves. How does quicksort com
pare?
52. Devise an algorithm that ﬁnds the second largest of four elements.
(a) Brieﬂy describe the algorithm in English.
(b) Give a decision tree for it.
59
53. Let MAX
n
be the problem of ﬁnding the max of n numbers. Let
SEC
n
be the problem of ﬁnding the second largest of n numbers. We
assume the decision tree model, i.e., only comparisons are the main
operation.
(a) Show that in any Decision Tree for MAX
n
, every branch has
depth at least n −1. (HINT: if only n −2 comparisons are made
then show that the maximum cannot be determined.)
(b) Show that any Decision tree for MAX
n
has 2
n−1
leaves.
(c) Let T be a Decision tree for SEC
n
. Let i be such that 1 ≤ i ≤ n.
Let T
i
be formed as follows: take tree T and, whenever there is
a comparison between x
i
and x
j
, eliminate the comparison by
assuming that x
i
wins.
i. Argue that T
i
is essentially a decision tree to ﬁnd MAX on
¦x
1
, . . . , x
n
¦ − ¦x
i
¦.
ii. Show that, for all i, T
i
has 2
n−2
leaves.
iii. Show that, for all i and j, the leaves of T
i
and the leaves of
T
j
are disjoint.
iv. Find a lower bound on the number of leaves in T by using
the last two items.
v. Show that T has depth n + lg n −O(1).
54. Assume you had a special hardware component that could ﬁnd the
minimum of
√
n numbers in one step. Give upper and lower bounds
for sorting using this component.
55. Assume you had a special hardware component that could sort
√
n
numbers in one step. Give upper and lower bounds for sorting using
this component.
56. We are going to derive a lower bound for merging two lists of sizes
m, n. (Assume all of the elements are distinct.)
(a) Let S(m, n) be the number of arrangements of two lists X, Y
where each list keeps its original order. In this problem you will
ﬁnd S(m, n) two ways.
i. Show that
S(m,n)=m+n
m
using a combinatorial argument.
ii. In all of the arrangements, either the ﬁrst element of X is
the smallest or the ﬁrst element of Y is the smallest (but
60
not both). Assume it is the ﬁrst element of X. How many
arrangements of the remaining elements are there? Assume
it is the ﬁrst element of Y . How many arrangements of the
remaining elements are there? Based on this information,
write down a recurrence for S(m, n). Show that
m+n
m
is a
solution to your recurrence.
(b) Give a lower bound for the number of comparisons needed to
merge two lists. Simplify your answer using Stirling’s formula.
(c) What is your answer when m = n (write it as simply as possible).
How does it compare to the lower bound of 2n − 1.
57. Consider the problem of sorting a sequence of n 0’s and 1’s using
comparisons. For each comparison of two values x and y, the algorithm
learns which of x < y, x = y, or x > y holds.
(a) Give an algorithm to sort in n−1 comparisons in the worst case.
Show that your algorithm is optimal.
(b) Give an algorithm to sort in 2n/3 comparisons in the average case
(assuming each of the n inputs is 0 or 1 with equal probability).
Show that your algorithm is optimal.
58. Assume you have a list of n elements, where each element is within k
positions of its correct sorted position (on either side). (If k = 0 then
the deck is sorted.)
(a) Give a lower bound on the number of comparisons it takes to sort
the list. Make your bound as exact as possible. A little looseness
in your “proof” is acceptable here. Justify your answer.
(b) Give an eﬃcient comparisonbased algorithm for sorting the list.
How many comparisons does it use in the worst case.
(c) Compare your upper and lower bounds.
59. Consider the problem of partitioning n elements into k equal sized
groups, so that the ﬁrst group has the smallest n/k elements (not nec
essarily sorted), the second group has the next smallest n/k elements
(not necessarily sorted), etc. (You may assume that k divides n.)
a. Derive a lower bound for the number of comparisons needed for
solving this problem.
61
b. Give an eﬃcient algorithm for solving this problem.
c. Analyze its running time.
60. (a) Assume you are given a number X and two sorted lists of n
numbers each: A
1
≤ A
2
≤ ≤ A
n
and B
1
≤ B
2
≤ ≤
B
n
. Design a linear (i.e. O(n)) time algorithm to determine how
many pairs (i, j) there are such that A
i
+ B
j
≤ X. Justify the
correctness and running time of your algorithm.
(b) Show how to preprocess the lists A and B (in any way that you
like and producing any data structure that you like) so that given
X, the pair counting problem can be solved in o(n) time. How
much preprocessing time do you use? How much storage is used?
How much time is needed to answer one query (after preprocess
ing)?
61. Illustrate the operation of radix sort on the following list of English
words: RUT, TOP, SUP, POT, TON, OPT, TOR, ROT, TOO, OUT,
SUR, PUT
62. For this problem, assume that you have access to a procedure Merge(A,
B, C, na, nb), that merges two sorted arrays A[1..na] and B[1..nb]
into a single sorted array C[1..na+nb]. This procedure runs in O(n
a
+
n
b
) time. Assume we also have access to unlimited dynamic array al
location.
Suppose that you have a collection of n, 1dimensional arrays each of
length n, D[1..n][1..n], such that each array is sorted (i.e. D[i][j]
<= D[i][j+1]). We want to compute a single sorted array that con
tains all n
2
elements.
The ﬁrst strategy for accomplishing this is called cascading merging.
First merge D[1] with D[2] to form an array of length 2n, then merge
this array with D[3] to form an array of length 3n, etc. The ﬁnal
merge is between an array of length (n − 1)n with D[n] to form the
ﬁnal sorted array of length n
2
. See the ﬁgure below.
62
❍
❍
❍
❍
✟
✟
✟
✟
❍
❍
✟
✟
❍
❍
✟
✟
❍
❍
✟
✟
✟
✟
✟
✟
❍
❍
✟
✟
✟
✟
❍
❍
✟
✟
Balanced Merging Cascading Merging
D[1]
D[2]
D[3]
D[4] D[4]
D[3]
D[2]
D[1]
The second strategy is called balanced merging. Assume that n is a
power of 2. First merge pairs D[1] and D[2] together, D[3] and D[4]
together, and so on until merging D[n1] and D[n] together. The
result is n/2 sorted arrays of size 2n each. Next, repeat the balanced
merging on these arrays, resulting in n/4 arrays of size 4n each. This
is repeated until there is 1 array of size n
2
.
(a) What is the total running time of cascading merging? Justify
your answer.
(b) What is the total running time of balanced merging? Justify your
answer.
For both parts (a) and (b) count only the time spent in the merging
procedure.
63. Suppose you have a list of n numbers consisting of k ≤ n distinct
values. For 1 ≤ i ≤ k there are m
i
copies of the ith value.
(a) Show that
O
n + lg
n!
m
1
!m
2
! . . . m
k
!
comparisons are necessary to sort this list by a comparisonbased
sorting algorithm. (Note: You are required to show ONLY the
lower bound, you do not need to show that this is always possible.)
Note that any algorithm for this problem does NOT know in
advance what the values of m
1
, m
2
, . . . , m
k
are.
(b) Show that this many comparisons are suﬃcient to sort (i.e. give
an algorithm). Derive the running time of your algorithm.
64. Selection Sort can be thought of as a recursive algorithm as follows:
Find the largest element and put it at the end of the list (to be sorted).
Recursively sort the remaining elements.
63
a. Write down the recursive version of Selection Sort in pseudocode.
b. Derive a recurrence for the exact number of comparisons the al
gorithm uses.
c. Use the iteration method to solve the recurrence. Simplify as
much as possible.
d. Use mathematical induction to verify your solution.
65. Insertion sort can be expressed as a recursive procedure as follows. In
order to sort A[1 . . . n], recursively sort A[1 . . . n − 1] and then insert
A[n] into the sorted array A[1 . . . n − 1].
a. Write the pseudocode for this (recursive) algorithm.
b. Give a recurrence for the number of comparisons this algorithm
uses in the worstcase.
c. What is the solution to the recurrence. Prove it using mathemat
ical induction.
66. Consider the following algorithm that merges two lists: X = x
1
, x
2
, . . . , x
n
and Y = y
1
, y
2
, . . . , y
n
. Assume n > 1 is a power of two. (For n = 1,
just compare the two elements.)
1) Separate X and Y into even and odd indexed items, so X
=
x
1
, x
3
, . . . , x
n−1
and X
= x
2
, x
4
, . . . , x
n
, and Y
= y
1
, y
3
, . . . , y
n−1
and Y
= y
2
, y
4
, . . . , y
n
.
2) Merge X
and Y
into U.
3) Merge X
and Y
into V .
4) Interleave U and V into u
1
, v
1
, u
2
, v
2
, u
3
, v
3
, . . . , u
n
, v
n
.
5) Compare each pair of elements v
i
and u
i+1
(and exchange if v
i
>
u
i+1
).
Answer the following questions about the algorithm.
(a) Write a recurrence for how many comparisons this merging algo
rithm uses.
(b) Solve the recurrence using the iteration method.
(c) Prove that this algorithm merges correctly.
64
67. Consider an n n array A containing integer elements (positive, neg
ative, and zero). Assume that the elements in each row of A are in
strictly increasing order, and the elements if each column of A are in
strictly decreasing order. (Hence there cannot be two zeroes in the
same row or the same column.) Describe an eﬃcient algorithm that
counts the number of occurrences of the element 0 in A. Analyze its
running time.
68. Let f be a function deﬁned on the interval 0 to 1 (i.e. 0 ≤ x ≤ 1),
which achieves its maximum value at point x
M
, a point in the interval,
and is strictly increasing for 0 ≤ x < x
M
and strictly decreasing for
x
M
< x ≤ 1. (Note: x
M
may be 0 or 1.) The only operation you can
do with f is evaluate it at points in its interval.
(a) Write an algorithm to ﬁnd x
M
with error at most , i.e., if your
algorithm’s answer is p, then [p − x
M
[ < . Use instructions of
the form y ← f(x) to indicate evaluation of f at the point x. Try
to use as few evaluations as possible. Count your evaluations as
exactly as possible.
(b) How many evaluations does your algorithm do in the worst case?
65
NPCompleteness
1. If G = (V, E) is a graph then a vertex cover for G is a set V
⊆ V such
that for every edge in G one of the endpoints is in V
. Let
A
1
= {(G, k)  G has a vertex cover of size ≤ k}.
A
2
= {G  G has a vertex cover of size ≤ 12}.
Show that A
1
∈ NP. Show that A
1
is NPcomplete. Show that A
2
∈ P.
Show that A
2
can be solved in O(n
a
) where a < 10.
2. We deﬁne A ≤
ww
B if there are 12 functions f
1
, . . . , f
12
such that each
one can be computed in polynomial time and
x ∈ A iﬀ at least 6 elements of {f
1
(x), . . . , f
12
(x)} are in B.
Show that if B ∈ P and A ≤
ww
B then A ∈ P.
3. Assume f
1
takes time p
1
(n), f
2
takes time p
2
(n), . . ., f
12
takes time
p
12
(n), and the algorithm for B takes time q(n), where n is the length
of the input. Express the runtime for the algorithm for A in terms of
p
1
, p
2
, . . . , p
12
and q.
4. For each of the following sets state whether it is in NP or not (no proof
required). Then state whether or not it is known to be in P. If it is
known to be in P then give an algorithm for it that takes polynomial
time. If it is not known to be in P then nothing more is required.
(a) A
1
= {(a
1
, . . . , a
k
, N)  some subset of {a
1
, . . . , a
k
} adds up to a
number that is ≤ N}.
(b) A
2
= {(a
1
, . . . , a
k
, N)  some subset of {a
1
, . . . , a
k
} adds up to
exactly N}.
(c) A
3
= {(a
1
, . . . , a
k
, N)  some subset of {a
1
, . . . , a
k
} adds up to a
number that is ≥ N}
5. Let G be a graph where every edge is labelled with a diﬀerent Boolean
variable. Let v be a vertex of G. Let φ
G,v
be the exclusiveOR of
all the edges that have v as an endpoint. Let φ
G
be the AND of all
the φ
G,v
as v ranges over the vertices of G. Is the following problem
NPcomplete: Given a graph G that has edges labelled with diﬀerent
Boolean variables, determine if φ
G
is satisﬁable.
66
6. We deﬁne A ≤
w
B if there are 2 functions f
1
, f
2
such that each one
can be computed in polynomial time and
x ∈ A iﬀ f
1
(x) ∈ B and f
2
(x) / ∈ B.
Show that if B ∈ P and A ≤
w
B then A ∈ P. (Assume f
1
takes
time p
1
(n), f
2
takes time p
2
(n), and the algorithm for B takes time
q(n). With this, you should be able to express the running time for
the algorithm for A in terms of p
1
, p
2
, and q.)
7. Prove that P = NP, then P = coNP.
8. A stable set of vertices in a graph do not have any edges between them.
The optimization version of this problem is to ﬁnd a maximum stable
set in a graph.
a. Deﬁne a decision version of the stable set problem.
b. Show that the decision version is in NP.
c. Show that if you could solve the optimization version in polyno
mial time that you could also solve the decision version in poly
nomial time.
d. Show that if you could solve the decision version in polynomial
time that you could also solve the optimization version in poly
nomial time.
e. Show that it is NPcomplete. HINT: Use CLIQUE. (Or alterna
tively show that it is polynomially equivalent to the clique prob
lem.)
f. Show that you can solve the decision version for constant size
stable sets in polynomial time.
9. The Hamiltonian path problem (HP) is: given an undirected graph
G = (V, E), does there exist a simple path that passes through every
vertex of G? The Directed Hamiltonian path problem (DHP) is: given
a directed graph G = (V, E), does there exist a simple path that passes
through every vertex of G? The Hamiltonian cycle problem (HC) is:
given an undirected graph G = (V, E), does there exist a simple path
that passes through every vertex of G? The Directed Hamiltonian
cycle problem (DHC) is: given a directed graph G = (V, E), does
there exist a simple path that passes through every vertex of G?
67
Show that all four of the problems above are in NP. For all pairs of
problems P
1
, P
2
from {HP, DHP, HC, DHC} show that P
1
≤
p
P
2
.
(HINT on HP ≤
p
HC. You will need to change the graph. Replace
some vertex v ∈ V by two new vertices, x and y, that are adjacent to
the same vertices that v was. What else do you need to do? Try to
“force” any Hamiltonian path to visit these vertices ﬁrst and last.)
10. Consider the problem DENSE SUBGRAPH: Given G, does it contain
a subgraph H that has exactly K vertices and at least Y edges ?
Prove that this problem is NPcomplete. You could pick any problem
to reduce from.
11. Assume that the following problem is NPcomplete.
PARTITION: Given a ﬁnite set A and a “size” s(a) (a positive integer)
for each a ∈ A. Is there a subset A
⊆ A such that
a∈A
s(a) =
a∈A−A
s(a) ?
Now prove that the following SCHEDULING problem is NPcomplete:
Given a set X of “tasks”, and a “length” (x) for each task, three
processors, and a “deadline” D. Is there a way of assigning the tasks
to the three processors such that all the tasks are completed within
the deadline D? A task can be scheduled on any processor, and there
are no precedence constraints. (Hint: ﬁrst prove the NPcompleteness
of SCHEDULING with two processors.)
12. Prove that the following ZERO CYCLE problem is NPcomplete:
Given a simple directed graph G = (V, E), with positive and negative
weights w(e) on the edges e ∈ E. Is there a simple cycle of zero weight
in G? (Hint: Reduce PARTITION to ZERO CYCLE.)
13. Let A be an algorithm that accepts strings in a language L. Let p(n)
be a polynomial. Assume that A runs in p(n) time if a given string x is
in L, but in superpolynomial time if x is not in L. Give a polynomial
time algorithm for recognizing strings in L.
14. Prove that the following problem is NPcomplete.
SPANNING TREE WITH A FIXED NUMBER OF LEAVES: Given
a graph G(V, E), and an integer K, does G contain a spanning tree
with exactly K leaves ?
(a) First prove that the problem is in NP by describing a “guessing”
algorithm for it. (Write out the algorithm in detail.)
68
(b) Now prove that the problem is NPhard, by reducing UNDI
RECTED HAMILTONIAN PATH to it.
HAMILTONIAN PATH asks for a simple path that starts at some
vertex and goes to some other vertex, going through each remain
ing vertex exactly once.
15. Show that both of the following two problems are NPcomplete. For
each show that the problem is in NP, and show that a known NP
complete problem is reducible to this problem.
Subgraph Isomorphism: Given two undirected graphs G
1
= (V
1
, E
1
)
and G
2
= (V
2
, E
2
), is G
1
is a subgraph of G
2
? More formally is
there are 1–1 function π : V
1
→ V
2
such that for all {u, v} ∈ E
1
,
{π(u), π(v)} ∈ E
2
. (Hint: Use either CLIQUE or Hamiltonian
Cycle.)
Longest Simple Cycle: Given an undirected graph G and an inte
ger k, does G have a simple cycle of length greater than or equal
to k? (Hint: Use Hamiltonian Cycle.)
16. Assume that you are given access to function CLIQUE(G, K) that
returns “True” if G contains a clique of size at least K, and “False”
otherwise.
(a) Use the function CLIQUE to determine, in polynomial time, the
size of the largest clique by making. (Try and solve the problem
by asking the function CLIQUE as few questions as possible.)
(b) How can you use the function CLIQUE to actually compute a
clique that is largest in size ? (Try and solve the problem by
asking the function CLIQUE as few questions as possible.)
17. The cycle cover problem (CC) is: given a directed graph G = (V, E)
and an integer k is there a subset of vertices V
⊆ V of size k such
that every cycle of G passes through a vertex in V
? Prove that CC
is NPcomplete. (Hint: The proof that CC is in NP is tricky. Note
that a graph can have an exponential number of cycles, so the naive
veriﬁcation algorithm will not work. The reduction is from vertex
cover.)
18. Suppose we had a Boolean function SAT(F), which given a Boolean
formula F in conjunctive normal form, in polynomial time returns
69
True if the function is satisﬁable and False otherwise. (This is very
unlikely since it would imply P = NP!) This function tells you nothing
about how to set the variables to satisfy the formula. Show how to use
this function to determine the actual variable assignment that makes
the formula True.
19. If G = (V, E) is a graph then a Dominating Set is a set V
⊆ V such
that, for all vertices v, either v ∈ V
or v is adjacent to an element of
V
.
(a) Let DOM = {(G, k)  G has a dominating set of size k}. Show
that DOM is NPcomplete (Hint: Use the Vertex Cover prob
lem.)
(b) Let PLANAR−DOM = {(G, k)  G is planar and has a dominating set of size k}.
Show that PLANAR − DOM is NPcomplete (Hint: Use the
DOM problem.)
(c) Let IND−PLANAR−DOM = {(G, k)  G is planar and has a dominating set of size k tha
Show that IND−PLANAR−DOM is NPcomplete (Hint: Use
the PLANAR −DOM problem.)
(d) Fix k. Let IND−PLANAR−DOM
k
= {G  G is planar and has a dominating set of size k
Show that IND −PLANAR −DOM
k
can be solved in O(6
k
n)
time where the constant does not depend on k. (Hint: Use that
a planar graph always has a vertex of degree ≤ 5.)
20. Let q, k be a ﬁxed constant. Let qCNF be CNF where each clause has
≤ q literals. A formulas is MONOTONE if it has no negated variables.
(a) Let q − k − MONO − SAT be the set of all qCNF monotone
formulas that are satisﬁable with an assignment that has at most
k variables set to TRUE. Show that q −k −MONO −SAT can
be solved in O(2
k
n) where the constant does not depend on k.
(b) q −k −SAT be the set of all qCNF formulas that are satisﬁable
with an assignment that has at most k variables set to TRUE.
Show that q−k−SAT can be solved in O(2
k
n) where the constant
does not depend on k.
21. (a) Show that the problem of testing if a graph is kcolorable is NP
complete.
(b) Show that if G is a graph and v is a node of degree ≤ d then G
is d + 1colorable iﬀ G−{v} is d + 1colorable.
70
(c) Fix k. Show that determining if a graph is n − kcolorable can
be done in O(2
k
n) time.
22. If G = (V, E) is an undirected graph then let G = (V, E) be deﬁned
by
(x, y) ∈ E iﬀ (x, y) / ∈ E.
G is called the complement of G. Let HAMCGC be the set of graphs
G such that either G or G has a Hamiltonian cycle. Show that
HAMCGC is NPcomplete. (Let HAMP be the set of all (H, a, b)
such that H has a Hamiltonian Path that starts at a and ends at b.
You may use the fact that HAMP is NPcomplete.)
23. (a) Let
A = {G  either G or G has a Hamiltonian Cycle}.
Either show A ∈ P or that A is NPcomplete.
(b) Make up and solve similar problems to the one above where you
take some property of graphs and ask if the graph or its compli
ment has it.
71
Problems on Algorithms
Second Edition Ian Parberry and William Gasarch
July 2002
Consisting of
Problems on Algorithms, First Edition, by Ian Parberry, formerly published in 1994 by PrenticeHall, Inc. Errata to the First Edition, published in 1998. Supplementary Problems, by William Gasarch.
© Ian Parberry and William Gasarch, 2002.
License Agreement
This work is copyright Ian Parberry and William Gasarch. All rights reserved. The authors offer this work, retail value US$20, free of charge under the following conditions: • No part of this work may be made available on a public forum (including, but not limited to a web page, ftp site, bulletin board, or internet news group) without the written permission of the authors. • No part of this work may be rented, leased, or offered for sale commercially in any form or by any means, either print, electronic, or otherwise, without written permission of the authors. • If you wish to provide access to this work in either print or electronic form, you may do so by providing a link to, and/or listing the URL for the online version of this license agreement: http://hercule.csci.unt.edu/ian/books/free/license.html. • You may not link directly to the PDF file. All printed versions of any or all parts of this work must include this license agreement. Receipt of a printed copy of this work implies acceptance of the terms of this license agreement. If you have received a printed copy of this work and do not accept the terms of this license agreement, please destroy your copy by having it recycled in the most appropriate manner available to you. You may download a single copy of this work. You may make as many copies as you wish for your own personal use. You may not give a copy to any other person unless that person has read, understood, and agreed to the terms of this license agreement. You undertake to donate a reasonable amount of your time or money to the charity of your choice as soon as your personal circumstances allow you to do so. The author requests that you make a cash donation to The National Multiple Sclerosis Society in the following amount for each work that you receive: • $5 if you are a student, • $10 if you are a faculty member of a college, university, or school, • $20 if you are employed fulltime in the computer industry.
• •
Faculty, if you wish to use this work in your classroom, you are requested to: • encourage your students to make individual donations, or • make a lumpsum donation on behalf of your class. If you have a credit card, you may place your donation online at: https://www.nationalmssociety.org/donate/donate.asp. Otherwise, donations may be sent to: National Multiple Sclerosis Society  Lone Star Chapter 8111 North Stadium Drive Houston, Texas 77054 If you restrict your donation to the National MS Society's targeted research campaign, 100% of your money will be directed to fund the latest research to find a cure for MS. For the story of Ian Parberry's experience with Multiple Sclerosis, see http:// multiplesclerosissucks.com.
Problems on Algorithms
by
Ian Parberry
(ian@ponder.csci.unt.edu) Dept. of Computer Science University of North Texas Denton, TX 76203 August, 1994
Contents
Preface 1 Introduction 1.1 How to Use This Book 1.2 Overview of Chapters 1.3 Pseudocode 1.4 Useful Texts 2 Mathematical Induction 2.1 Summations 2.2 Inequalities 2.3 Floors and Ceilings 2.4 Divisibility 2.5 Postage Stamps 2.6 Chessboard Problems 2.7 Fibonacci Numbers 2.8 Binomial Coeﬃcients 2.9 What is Wrong? 2.10 Graphs 2.11 Trees 2.12 Geometry 2.13 Miscellaneous 2.14 Hints 2.15 Solutions 2.16 Comments
ix 1 1 1 3 4 8 8 10 10 11 11 12 14 14 15 16 17 18 19 20 21 23
iii
2 True or False? 3.5 Solutions 5.3 Combined Iteration and Recursion 5.1.3 Arrays 5.7 Solutions 4.1 Iterative Algorithms 5.1.4 Manipulating BigO 3.2 Arithmetic 5.2 Recursive Algorithms 5.9 Comments 4 Recurrence Relations 4.1 Rank the Functions 3.1 Simple Recurrences 4.iv 3 BigO and BigΩ 3.6 Find the Functions 3.1.1 StraightLine Programs 5.3 General Formulae 4.2.3 Proving BigO 3.2.2 More Diﬃcult Recurrences 4.8 Comments 5 Correctness Proofs 5.5 Alternative Deﬁnitions 3.6 Hints 4.2.1 Iterative Algorithms
Contents
28 28 30 31 32 33 34 35 35 36 37 37 38 39 39 40 41 41 44 46 46 47 47 49 51 51 52 53 54 54 55 56 58 59 59
.4 Recurrences with Full History 4.4 Fibonacci Numbers 5.5 Recurrences with Floors and Ceilings 4.8 Solutions 3.3 Fibonacci Numbers 5.1.2 Arrays 5.7 Hints 3.1 Arithmetic 5.6 Comments 6 Algorithm Analysis 6.4 Hints 5.
8 Solutions 8.1.4 Binary Search 7.Contents
v 6.6
7 DivideandConquer 7.11 Comments 8 Dynamic Programming 8.3 Strassen’s Algorithm 7.2 Arrays 6.6 Towers of Hanoi 7.2 Arithmetic 6.1.1 Arithmetic 6.7 Hints 8.1 What is Returned? 6.3 Arrays 6.4 Floyd’s Algorithm 8.4 6.8 Applications 7.1.1 Iterated Matrix Product 8.2.1.9 Hints 7.2.5 6.6 Finding the Solutions 8.5 Applications 8.10 Solutions 7.1 Maximum and Minimum 7.5 Quicksort 7.2 The Knapsack Problem 8.7 DepthFirst Search 7.4 Fibonacci Numbers Recursive Algorithms 6.9 Comments
.2 Integer Multiplication 7.2
6.2.3 Fibonacci Numbers Combined Iteration and Recursion Hints Solutions Comments 59 61 61 62 62 63 63 64 64 64 65 66 67 67 68 73 74 75 77 78 79 81 83 85 87 87 89 90 91 92 97 99 99 100
6.3 Optimal Binary Search Trees 8.3 6.
4 The UnionFind Problem 11.vi 9 Greedy Algorithms 9.8 Comments 10 Exhaustive Search 10.8 Solutions 10.5 Backtracking 10.2 AVL Trees 11.3 2–3 Trees 11.4 Other Elementary Algorithms 10.2 Permutations 10.7 Solutions 11.2 Dijkstra’s Algorithm 9.4 Traveling Salesperson 9.9 Comments 11 Data Structures 11.3 Combinations 10.6 Hints 9.1 General 12.6 Factorization
Contents
101 101 102 103 109 109 111 111 115 116 116 118 120 121 122 123 130 131 133 135 135 138 141 142 142 145 145 146 147 147 148 150 152 153 154
.8 Comments 12 N Pcompleteness 12.4 Circuits 12.7 Hints 10.5 Applications 11.6 Applications 10.7 Solutions 9.3 What is Wrong? 12.5 OneinThree 3SAT 12.1 Strings 10.2 Cook Reductions 12.1 Continuous Knapsack 9.5 Applications 9.1 Heaps 11.6 Hints 11.3 MinCost Spanning Trees 9.
6 General 13.8 Solutions 12.5 Matrix Reductions 13.1 Sorting and Order Statistics 13.4 Maximum Flow 13.3 Graph Algorithms 13.7 Hints 13.9 Comments Bibliography Index
.Contents
vii 154 155 155 156 156 157 158 160 161 162 164 166 166 168 174
12.8 Solutions 13.2 Lower Bounds 13.7 Hints 12.9 Comments 13 Miscellaneous 13.
viii
Contents
.
The solutions are necessarily few for reasons of brevity. numbertheoretic algorithms. I have included various preambles that summarize the background knowledge needed to solve the problems so that students who are familiar with the notation and style of their textbook and instructor can become more familiar with mine. It is intended to augment the problem sets found in any standard algorithms textbook. I have chosen not to cover these areas. but would not make it more attractive for its intended audience — students and practicing programmers. I have provided at the end of each chapter a small collection of hints. To make this book suitable for selfinstruction. I have made two important and perhaps controversial decisions in order to keep its length within reasonable bounds. program in Computer Science or Computer Engineering. To be eﬀective problem solvers.D. ix
. and comments. The second decision is not to search for the origin of the problems that I have used. randomized algorithms. analysis. computational geometry. The best way to develop that skill is to solve problems. masterprogrammersintraining must do more than memorize a collection of standard techniques and applications — they must in addition be able to internalize and integrate what they have learned and apply it in new circumstances. and also to avoid hindering instructors in their selection of homework problems. solutions. Although most instructors throw in a “fun” advanced topic such as amortized analysis. This book is a collection of problems on the design. and veriﬁcation of algorithms for use by practicing programmers who wish to hone and expand their skills.Preface
The ability to devise eﬀective and eﬃcient algorithms in new situations is a skill that separates the master programmer from the merely adequate coder. and as a selfstudy text for graduate students who are preparing for the qualifying (often called “breadth” or “comprehensive”) examination on algorithms for a Ph. as a supplementary text for students enrolled in an undergraduate or beginning graduate class on algorithms. or parallel algorithms. The ﬁrst is to cover only what I consider to be the most important areas of algorithm design and analysis. A lengthy discussion of the provenance of each problem would help make this book more scholarly. Recognizing that a supplementary text must be costeﬀective if it is to be useful. approximation algorithms.
but the details of the programming language are not a major factor in the solutions. correctness proofs. and in particular any new and interesting problems (including those from areas not presently covered). The Bibliography was created by BibTEX. 9. errata.11.x
Preface
The organization of this book is a little unusual and requires a few words of explanation.edu (IP address 129.3.sty written by J.
. Kenneth Shultis.14t3). recurrence relations. dynamic programming. and the twoside.unt. Standard textbooks usually devote little time to exhaustive search because it usually requires exponential time. greedy algorithms. using the Prentice Hall macros phstyle. 9.csci. mainly in Sections 2. This organization is somewhat nonstandard.7. and basic algorithm analysis methods. 8. 13.120. bigO notation. and 13. A list of errata for this book is available by anonymous ftp from ftp.4). A and the graphs using xgraph. it begins with ﬁve chapters on background material that most algorithms instructors would like their students to have mastered before setting foot in an algorithms class. The ﬁnal chapter contains miscellaneous problems that do not necessarily ﬁt into the earlier chapters. epsf. The next four chapters are organized by algorithm design technique: divideandconquer. analysis.3. 2. and those for which part of the problem is to determine the algorithmic technique or techniques to be used.edu. The algorithms in this book are expressed in a Pascallike pseudocode. but even then most students would proﬁt from brushing up on this material by attempting a few of the problems.unt. This will be the case at some universities. I will be grateful to receive feedback. The ﬁgures were prepared in encapsulated Postscript form using idraw. but I believe that the pedagogical beneﬁts outweigh the drawbacks (the most signiﬁcant of which seems to be the scattering of problems on graph algorithms over several chapters. but I believe that students proﬁt from it once they overcome their initial aversion to mathematical formalism. and exhaustive search. which is a pity when it contains many rich and interesting opportunities to practice the application of algorithm design. The introductory chapters include mathematical induction. The dvi ﬁle produced by L TEX was translated into Postscript using dvips.4. The manuscript is rounded out with chapters on advanced data structures and N Pcompleteness. in the directory ian/poa. Two problems (Problems 326 and 327) require the reader to examine some code written in Pascal.1). The index was produced using the program makeindex. 9. In any case.09 (which used TEX C Version 3. The correctness proof chapter goes beyond what is normally taught.10. and makeidx style ﬁles. and veriﬁcation methods. this manuscript was produced A from cameraready copy provided by the author using L TEX version 2.4.2. which can be sent by electronic mail to ian@ponder. 7. After a chapter of introduction.1. suggestions. For those who are interested in technical details. I believe that students of Computer Science should be literate in many diﬀerent programming languages.
Each problem is labeled with a sequence of icons. It is recommended that readers ﬁrst attempt the problems on their own. Only a limited number of solutions are given so that students are not tempted to peek. Problems with two or three mortarboards are suitable for a graduate class on algorithms. The remaining indicates that icons indicate the existence of additional material: The lightbulb there is a hint. The more mortarboards a problem has. This chapter is the only background that you will get before we start listing problems. The hints should be consulted only when encountering serious diﬃculty in getting started. the more diﬃcult it is. bigO notation. respectively mathematical induction. and the smiley face indicates that there is a comment. veriﬁcation.1
HOW TO USE THIS BOOK
Most chapters have sections with hints for. the scroll and feather indicates that there is a solution. The ﬁrst section describes the philosophy of this book and how to get the best out of it. and to ensure that there remains a large number of problems that instructors can assign as homework. I strongly recommend 1
.1.Chapter 1
Introduction
Problems on Algorithms contains 668 problems on the design.
1. The solutions can be used as examples of the type of answers that are expected. This is summarized for the reader’s convenience in Table 1. The second section contains an overview of the remaining chapters. and analysis of algorithms. and recurrence relations. and comments on selected problems.2
OVERVIEW OF CHAPTERS
Chapters 2–4 contain problems on some background material on discrete mathematics that students should already have met before entering an undergraduate algorithms course. The third section brieﬂy touches on the pseudocode used to describe algorithms. solutions to. The fourth section gives a quick review of some of the textbooks available on algorithms and related subjects. If you are a student in an algorithms class.
1. Problems with one or two mortarboards are suitable for a seniorlevel undergraduate class on algorithms. It is divided into four sections.
You may also meet the material from Chapters 5 and 6 in other classes. Problem symbols and their meaning. Chapter 11 contains problems on advanced data structures. but the problems in Chapter 7 will test your knowledge further. expression.
that you review this material and attempt some of the problems from Chapters 2–4 before attending the ﬁrst class. If you can master induction. The problems address some of the standard applications of these techniques. dynamic programming.1. greedy algorithms. and exhaustive search. and design. 1. Chapters 7–10 consist of problems on various algorithm design techniques: respectively. Introduction
Symbol
Meaning easy medium diﬃcult hint solution comment
Table 1. but mastering them will teach you important skills in formalization. but not all of them can aﬀord the time to do so. Divideandconquer has already been foreshadowed in earlier chapters. Some kind instructors may spend a few lectures “reminding” you of this material. The treatment of exhaustive search in Chapter 10 goes beyond what is normally covered in algorithms courses. and ask you to demonstrate your knowledge by applying the techniques to new problems. Not all instructors will emphasize correctness proofs in their algorithms class. However. If you ignored the material on mathematical induction in your discrete mathematics class. Chapter 5 contains problems on correctness proofs. extensive problems are provided here because exhaustive search contains many rich and interesting opportunities to practice the application of algorithm design.2
Chap. The approach to correctness proofs taken in Chapter 5 is closer to that taken by working researchers than the formallogic approach expounded by others such as Dijkstra [21]. The excuse for skimming over exhaustive search is usually a natural distate for exponential time algorithms. and veriﬁcation methods. then think again. In any case. you will be ahead of the game if you devote some time to look over the material again. Chapter 6 contains some problems involving the analysis of some naive algorithms. but you probably don’t have the depth of knowledge and facility that you can get from working the problems in this chapter. and Chapter 12
. thinking that it is useless to a programmer. analysis. You should have met this material in earlier classes. you have mastered recursion — they are two sides of the same coin. divideandconquer.
The process repeats until the condition is false. The condition is then reevaluated. If it is false. .
1. The notation P [i. then S is executed. The else clause will be omitted if it is empty.or twodimensional arrays of integers. and those for which part of the problem is to determine the algorithmic technique to be used. Otherwise S is executed again. Sequencing: A sequence of commands will be on separate lines or separated by semicolons. Pseudocode
3
contains problems on N Pcompleteness. . Occasionally.Sec. whose elements are P [i]. Selection: The selection command uses Pascal’s ifthenelse syntax. which has the form if condition then S1 else S2 If the condition is true at time of execution. P [i + 1]. then S1 is executed.3
PSEUDOCODE
The algorithms in this text are described in a Pascallike pseudocode. Chapter 13 contains miscellaneous problems. The while loop has the form while condition do S If the condition is true. deﬁned to be those that do not necessarily ﬁt into the earlier chapters. otherwise S2 is executed. Indeﬁnite Iteration: Indeﬁnite iteration uses Pascal’s pretested while loop or posttested repeat loop. 1. in an attempt to save space. BlockStructuring: Indentation is used to indicate blockstructuring.. The repeat loop has the form repeat S until condition
. variables will be other mathematical objects such as sets or lists. Finally. P [j]. the loop terminates. . Here is a quick overview of the conventions used: Data Types: Most variables are either integers or one.j] is shorthand for an array P .3. . Assignment Statements: The “:=” symbol is the assignment operator.
You’ll be able to tell the diﬀerence by context — those that return values are obviously variable parameters! Comments in the algorithms should give you an extra hint. and more get published every year. 47]. Most parameters are value parameters. and some count down (using the keyword downto). Here is a quick thumbnail sketch of some of them. and Ullman [2] in using a Clike return statement to describe the value returned by a function. and Patashnik [30]. Cormen. though some are variable parameters. Garey and Johnson [28]. Some loops count up (using the keyword to). Leiserson. and Rivest [19]. and the downwardcounting loop on the right: for i := s to f do S which are equivalent to. An upwardcounting forloop will be done zero times if the start value is larger than the ﬁnish value (analogously for downwardcounting forloops). with the upwardcounting loop on the left. and Knuth [45. I recommend that the conscientious student examine as many texts as possible in the library and choose one that suits their individual taste.4 and is equivalent to S while not condition do S
Chap. i := s while i ≤ f do S i := i + 1 i := s while i ≥ f do S i := i − 1 for i := s downto f do S
Subroutines and Parameters: Subroutines are expressed using Pascalstyle procedures and functions. which unlike Pascal may be a complicated object. Knuth. Hopcroft. Graham.4
USEFUL TEXTS
There are many good algorithms texts available. Remember. though. 46. and some related texts that you might ﬁnd useful. and appeal to diﬀerent types of reader. 1. Functions return a value.
1. I have chosen to follow the lead of Aho. 10]. respectively. Introduction
Deﬁnite Iteration: Deﬁnite (countcontrolled) iteration is achieved using Pascal’s for loop. Procedures return values through their parameters. I also strongly recommend that the practicing programmer include in their library a copy of Bentley [9. The syntax is as follows.
. that diﬀerent texts have diﬀering areas of strength.
it does not mess around.Sec. Brassard and Bratley [14] This is another good algorithms text with a strong emphasis on design techniques. They should be recommended reading for all undergraduate computer science majors. Useful Texts
5
Aho. It is particularly useful for its large list of N Pcomplete problems. Hopcroft. and Ullman [1] This was the standard graduate text on algorithms and data structures for many years. data structures. In the spirit of Aho. Aho. Garey and Johnson [28] This is the canonical book on N Pcompleteness. this is the book for you. but it is not for the faint of heart. The subjects chosen are too idiosyncratic and anecdotal to be a textbook. Baase [7] A good algorithms text at the upperdivision undergraduate level. with more emphasis on programming. even when you correctly implement their Clike return statement. Cormen. but David Johnson has expressed an intention to work on a Second Edition. The title comes from the French word algorithmique. but not particularly so. it contains enough material for both a graduate and undergraduate course in algorithms. Leiserson. A massive tome. If you want the plain facts with no singing or dancing. Even [26] This is the canonical book on graph algorithms covering most of the graph algorithm material taught in the typical algorithms course. and Rivest [19] This is an excellent text for those who can handle it. which is what they (perhaps more aptly) call Computer Science. A large number of the Pascal program fragments don’t work. It is a little dated in presentation and material. 1. It is showing its age a little.
. It is the deﬁnitive text for those who want to get right down to business. It is a little dated. More modern subjects are covered by David Johnson in “The NPCompleteness Column: An OnGoing Guide” published on a semiregular basis in Journal of Algorithms (the latest was in September 1992. plus more. with some sentences requiring a paragraph of explanation for some students. and pays careful attention to implementation and experimentation. Bentley [9. and problem solving skills. It is a good place to go to brush up on your discrete mathematics. but nonetheless a useful pair of books. and Ullman [1]. but still very useful. It is full of minor errors. It is written quite tersely. but David Johnson recently assured me that more are planned). and Ullman [2] This is the undergraduate version of the previous book. Bentley explores the problems and pitfalls of algorithm design and analysis. Hopcroft. 10] This delightful pair of books are a collection of Jon Bentley’s Programming Pearls column from Communications of the ACM. Hopcroft.4. Aho and Ullman [3] This textbook is redeﬁning the undergraduate computer science curriculum at many leading institutions.
The material in it has aged well. The choice of material is strong but not to everybody’s taste. very helpful. It is noted for its marginal (and often obscure) graﬃti. The chapter on correctness proofs is an unusual but welcome addition. 47] The Knuth threevolume series is the standard reference text for researchers and students. hate it. Graham. and Patashnik [30] An excellent book on discrete mathematics for computer science. but the coverage is a little shallow for advanced undergraduates.6
Chap. except for Knuth’s penchant for an obscure ﬁctitious assemblylevel code instead of the more popular and easy to understand pseudocode. Lewis and Denenberg [52] A strong data structures text that also covers some of the material in the typical algorithms course. Kozen [50] A text for graduate students containing 40 lectures on algorithms. Others. Introduction
Gibbons [29] A more modern and carefully presented text on graph algorithms. Kingston [44] A strong textbook for the elementary undergraduate algorithms course. It contains excellent highlevel descriptions of subjects that tend to confuse the beginning student if they are presented in all their detailed glory. Divideandconquer and dynamic programming are covered very sketchily. The homework problems (all of which have solutions) and miscellaneous exercises (some of which have solutions) are particularly useful. superior in many ways to Even [26]. but it is weak on analysis and but its approach to backtracking is somewhat idiosyncratic and hard to follow. It is certainly more realistic in its approach to algorithm design than more formal
. It is noted for its biblical quotes. and the approach to greedy algorithms is idiosyncratic but likely to strike a chord with some students. as opposed to the normal text that allows the instructor to pick and choose the material to be covered. It contains enough material for a single graduate course. Harel [34] This is another good text along the lines of Brassard and Bratley. Horowitz and Sahni [37] This is a reasonable text for many of the topics found in the typical algorithms class. 1. which is algorithm design by successive reﬁnement of intuitive but ﬂawed version of the algorithm. The terse presentation is not for everybody. who do not think this way. Knuth [45. Knuth. 46. Greene and Knuth [33] A highly mathematical text for an advanced course on algorithm analysis at Stanford. Manber [56] Some students ﬁnd the approach taken in this book. Recommended for the serious graduate student. However. many students ﬁnd it quite diﬃcult going.
This volume covers tractable problems. Papadimitriou and Steiglitz [59] A reasonable text for an algorithms course. 1. Moret and Shapiro [58] This is planned as the ﬁrst volume of a twovolume set. analysis. Solow [75] This is a good text for the student who has trouble with mathematical formalism.Sec. which makes it a diﬃcult book to just dip into at random for the parts of it that are useful. Rawlins [67] An entertaining text that nonetheless is technically rigorous and detailed. The selection of topics is limited to chapters on mathematical preliminaries. but one which is very heavy with the jargon and mindset of the combinatorial optimization and numerical analysis community. It is excellent for the elementary undergraduate algorithms curriculum. Purdom and Brown [64] A mathematically rigorous text that focuses on analysis almost to the exclusion of design principles. but not suﬃciently rigorous for upperdivision courses. network ﬂow. It has a deﬁnite combinatorial optimization ﬂavor. but perhaps it encourages the wrong habits. Dynamic programming is a notable omission from the ﬁrst volume. Wilf [83] A book that does a good job on analysis. Weiss [82] A strong data structures text that also covers quite a bit of the material in the typical algorithms course. recursion. The chapter on induction may be particularly useful for those who have trouble getting started. but does not cover algorithm design techniques.
.4. Some important topics are not covered in very great detail. Sedgewick [72] This book studies a large array of standard algorithms. but it is largely descriptive in nature and does not spend much time on veriﬁcation. and the second volume (unsighted as yet) will cover intractable problems. or design techniques. and N Pcompleteness. Smith [74] This is another strong algorithms text with an awesome collection of problems. numbertheoretic algorithms. Useful Texts
7
texts.
and analysis. 2.
Prove by induction on n ≥ 0 that Prove by induction on n ≥ 0 that
n
n i=1
i(i + 1) = n(n + 1)(n + 2)/3.
i=0
5. this chapter may seem out of place because it doesn’t seem to be about algorithms at all.
n 3 i=1 i
Prove by induction on n ≥ 0 that Prove by induction on n ≥ 0 that
n
= n2 (n + 1)2 /4. as we shall see later.
i(i + 1)(i + 2) = n(n + 1)(n + 2)(n + 3)/4.
(2i + 1)2 = (n + 1)(2n + 1)(2n + 3)/3. you should start by attempting at least some of the following problems. Unfortunately. Some of them are quite useful in algorithm design and analysis.
2.3. 2. veriﬁcation. 4.Chapter 2
Mathematical Induction
At ﬁrst sight. 6. Either it was badly taught or they discarded it as useless information (or both).
SUMMATIONS
Prove by induction on n ≥ 0 that Prove by induction on n ≥ 0 that
n 2 i=1 i n i=1
i = n(n + 1)/2. 2.1
1.
= n(n + 1)(2n + 1)/6.7.1. 2. and 2. you should pay particular attention to the results in Sections 2. 2. 2. many students come to my algorithms class poorly versed in the use of induction. If you are one of these students.8.
i=1
8
.10. they are good practice in a technique that is fundamental to algorithm design.11. In addition to mastering the technique of mathematical induction. 3.2. Although the problems in this chapter are mainly mathematical in nature and have very little to do with algorithms.
2.
n
ai =
i=0
an+1 − 1 . (a − 1)2
15.
Prove by induction on n ≥ 0 that for every a = 1.
Prove by induction on n ≥ 1 that for every a = 1.
i=1
17.
n
ai =
i=j
an+1 − aj .
Prove by induction on n ≥ 0 that
n
i2 2n−i = 2n+3 − 2n+1 − n2 − 4n − 6.
i=1
16. 8.
n 1 = . 12.Sec.
n
iai =
i=1
nan+2 − (n + 1)an+1 + a .
n i=1
Prove by induction on n ≥ 1 that
1/2i = 1 − 1/2n . Summations
9
n i=1
7. i(i + 1) n+1
n i=1
13.
.
Prove by induction on n ≥ 0 that
n i=1
1 = n+i
n i=1
1 1 − 2i − 1 2i
.1. 9. a−1
10.
Prove by induction on n ≥ 0 that
i · i! = (n + 1)! − 1.
Prove by induction on n ≥ 0 that for every a = 1. a−1
n i=0
11. 14.
Prove by induction on n ≥ 0 that
i2i = (n − 1)2n+1 + 2.
Prove by induction on n ≥ 0 that
n
i2 2i = n2 2n+1 − n2n+2 + 3 · 2n+1 − 6. and all 0 ≤ j ≤ n.
Prove by induction on n ≥ 0 that Prove by induction on n ≥ 1 that
n i=1
2i = 2n+1 − 1.
Prove by induction that if n ≥ 4 is even. n+m−1 n = . The ﬂoor of x. is deﬁned to be the smallest integer that is no smaller than x.3
FLOORS AND CEILINGS
Suppose x ∈ IR+ .
INEQUALITIES
Prove by induction on n ≥ 1 that if x > −1. 21.
Prove by induction on n ≥ 0 that n = 2 n/2 if n is even (n + 1)/2 if n is odd. 20. Prove by induction on n ≥ 5 that 2n > n2 .
1 1 <2− . Prove by induction on n ≥ 0 that n = 2 25. 19.10
Chap. Prove by induction on k ≥ 1 that Prove by induction on n ≥ 0 that
n i=1 n k i=1 i
≤ nk (n + 1)/2. is deﬁned to be the largest integer that is no larger than x. denoted x . 2. n/2 if n is even (n − 1)/2 if n is odd.
j=1
2. then (1 + x)n ≥ 1 + nx. The ceiling of x.2
18. i2 n
23. Prove by induction on n ≥ 7 that 3n < n!. 24. m m
. denoted x .
Prove by induction on n ≥ 1 that for all m ∈ IR+ . 22. Mathematical Induction
2. and 2 ≤ i ≤ n/2. then
i k i
(n − 2j + 1) ≤ 2
k=1 j=1
(n − 2j + 1).
26.
Prove by induction that for all n ≥ 0.
33. Prove by induction that the sum of the cubes of three successive natural numbers is divisible by 9. 2.5
37. then (m + 2. 0) ∈ S.4. . Prove by induction on n that there exists x. Deﬁne S ⊆ IN × IN as follows.4
27.
DIVISIBILITY
Prove by induction on n ≥ 0 that n3 + 2n is divisible by 3. 41. 32. 34. Prove by induction on n ≥ 0 that 8n+2 + 92n+1 is divisible by 73. 2. such that x divides evenly into y with no remainder. n) ∈ S.Sec. x = y. 28. What is the smallest value of k such that any integer postage greater than k cents can be formed by using only 4cent and 9cent stamps? Prove your answer correct. Let T ⊂ Sn be any subset containing exactly n + 1 elements of Sn . 29. 30. 11n+2 + 122n+1 is divisible by 133. Prove by induction that a decimal number is divisible by 9 iﬀ the sum of its digits is divisible by 9. 36. . 38. 35. y ∈ T . Prove by induction that a decimal number is divisible by 3 iﬀ the sum of its digits is divisible by 3. n) ∈ S. Prove by induction on n ≥ 0 that n5 − n is divisible by 5. 31. Prove by induction on n ≥ 0 that 5n+1 + 2 · 3n + 1 is divisible by 8. 39.
2. n + 3) ∈ S. .
. m + n is divisible by 5.
POSTAGE STAMPS
Show that any integer postage greater than 7 cents can be formed by using only 3cent and 5cent stamps. Show that any integer postage greater than 34 cents can be formed by using only 5cent and 9cent stamps. Show that any integer postage greater than 59 cents can be formed by using only 7cent and 11cent stamps. (0. Let Sn = {1. Show that any integer postage greater than 5 cents can be formed by using only 2cent and 7cent stamps. . 40. Divisibility
11
2. 2n} be the set of integers from 1 to 2n. Prove by induction on n ≥ 0 that for all (m. If (m.
m ∈ IN. n ≥ 1 such that gcd(m. regardless of where the missing square is. Prove by induction on n ≥ 1 that any 2n × 2n chessboard that is missing one square can be tiled with triominoes. 49. that is. an n × m chessboard has all four corner squares colored the same.
Chap.
CHESSBOARD PROBLEMS
Prove by induction that for all n ∈ IN and all even m ∈ IN. an n × m chessboard has exactly the same number of black squares and white squares.
A triomino is an “L”shaped tile formed by three adjacent squares of a chessboard.
43. For example. Mathematical Induction
What is the smallest value of k such that any integer postage greater than k cents can be formed by using only 6cent and 11cent stamps? Prove your answer correct.2 shows a closed knight’s tour on an 8 × 8 chessboard.12 42.
50. Figure 2.
44. 47. 2.
. and a tiling with triominos that covers every square of the board except this one on the right. Prove or disprove that all chessboards of the following dimensions can be tiled by triominoes: (a) 3 × 2n . any positive integer amount of postage that is at least n(n − 1) cents can be formed by using only ncent and (n + 1)cent stamps. 46. Show that for all n ≥ 1. an n × m chessboard with the corner squares colored white has one more white square than black square.6
45. Figure 2. m ∈ IN.1. there exists k ∈ IN such that any positive integer amount of postage that is at least k cents can be formed by using only mcent and ncent stamps. Recall that a knight can make one of eight legal moves depicted in Figure 2. Prove by induction that a closed knight’s tour exists for any 2k × 2k chessboard for all k ≥ 3. Prove by induction that for all odd n. except for one square which is missed. An arrangement of triominoes is a tiling of a chessboard if it covers it exactly without overlap.
48. Prove by induction that for all odd n.
2.3. Show that for all m. n) = 1. shown in Figure 2. (b) 6 × 2n . a circular tour of knight’s moves that visits every square of the chessboard exactly once before returning to the ﬁrst square. (d) 6n × 6n . (c) 3n × 3n .4 shows a chessboard with a missing square ﬁlled in black on the left.
2.
Figure 2. Chessboard Problems
13
2 1
3 4
8 7 6
5
Figure 2.4. and a tiling with triominos that covers every square of the board except the missing one (right).3.1. A chessboard with a missing square ﬁlled in black (left).6.
Figure 2.
Figure 2.
. A triomino.Sec. 2. The eight legal moves that a knight on the center square can make. A closed knight’s tour on an 8 × 8 chessboard.
n m ≤ nm .
.
58. Mathematical Induction
Prove that any 2n × 2n chessboards with one square missing can be tiled by triominoes of only three colors so that no pair of triominoes that share an edge have the same color. and for n ≥ 2.
2 Prove by induction on n ≥ 2 that Fn−1 Fn+1 = Fn + (−1)n .
Prove by induction on n ≥ 1 that Fn+1 Fn+2 = Fn Fn+3 + (−1)n .8
BINOMIAL COEFFICIENTS
n r . 55.
The binomial coeﬃcient
for n. Fn = Fn−1 + Fn−2 . r ≤ n. r!(n − r)!
Prove by induction on n that
n m=0
n m
= 2n .
2.7
FIBONACCI NUMBERS
The Fibonacci numbers Fn for (n ≥ 0) are deﬁned recursively as follows: F0 = 0. Prove by induction on n that
n i=0
Fi = Fn+2 − 1. r ∈ IN.
2. 56. F1 = 1. n r 57. 54.14 51. Prove by induction on n ≥ 1 that
n i=1
Fi2 = Fn Fn+1 .
Prove by induction on n ≥ 1 that for all 1 ≤ m ≤ n. 2. 52.
Prove by induction that Fn+k = Fk Fn+1 + Fk−1 Fn . = n! . is deﬁned to be the number of ways of choosing r things from n without replacement. that is.
Chap. 53.
then it is trivially the same color as itself. Now suppose that there are n horses. What is wrong with the following proof? We claim that 6n = 0 for all n ∈ IN. The base of the induction is easy: If there is one horse. then 6n = 0.
Prove by induction on n that for all even n.
Prove by induction on n that for all even n.
Prove by induction on n that
n
(a + b)n =
m=0
n m
am bn−m . 6a = 0 and 6b = 0. suppose that n > 0.
Prove by induction that for all n ∈ IN.9
64. numbered 1 through n. Let n = a + b. Since horse 2 is black. The proof is by induction on n ≥ 0. this means that horses 2 through n must be black. all of the horses have the same color. horse 2 is black. then n n/2 > n k .
n
2−i ·
i=0
n i
= (3/2)n . Therefore.
2. Also by the induction hypothesis.
.
n
i·
i=0
n i
= n2n−1 . horses 2 through n have the same color. horses 1 through n − 1 have the same color (let’s say black).
65.
WHAT IS WRONG?
What.
Prove by induction that for all n ∈ IN. By the induction hypothesis. By the induction hypothesis.
61. Now. 2. if k = n/2. In particular. if n = 0.
62. n n/2 = Ω(2n /n). Clearly.
63. Therefore.
60.Sec.9. The proof is by induction on the number of horses. What is Wrong?
15
59. 6n = 6(a + b) = 6a + 6b = 0 + 0 = 0. if anything is wrong with the following reasoning? All horses are the same color.
We will follow the normal conventions of using n for the number of nodes and e for the number of edges. 67. a cycle that visits every vertex exactly once. To build a hypercube of dimension n. that is. v2 . Prove by induction that a hypercube of dimension n has 2n vertices. a path that visits every vertex exactly once. Then Fn = Fn−1 +Fn−2 .
69. Mathematical Induction
What is wrong with the following reasoning? We show that for every n ≥ 3. Fn is the sum of two even numbers. . 2. Fn is even. Prove by induction that a graph with n vertices can have at most n(n − 1)/2 edges. An Eulerian cycle in a connected graph is a cycle in which every edge appears exactly once. . The base of the induction is trivial. vn }. 3.
2. u) ∈ E.10
GRAPHS
A graph is an ordered pair G = (V. Draw an edge from each vertex in the ﬁrst copy to the corresponding vertex of the second copy. v ∈ V . Show that every tournament has a Hamiltonian path. 68. each vertex has an even number of edges incident with it) has an Eulerian cycle. E) such that for all u. For example. Thus. and 4. Prove by induction that the vertices of a hypercube can be colored using two colors so that no pair of adjacent vertices have the same color. it is a graph G = (V. That is. A hypercube of dimension 0 is a single vertex.5 shows the hypercubes of dimensions 0. Take a second. either exactly one of (u. 70. and must therefore be even. Now suppose that n ≥ 4 and that Fm is even for all m < n. A tournament is a directed graph formed by taking the complete undirected graph and assigning arbitrary directions on the edges. exact copy of this hypercube. E). and by the induction hypothesis Fn−1 and Fn−2 are even. Where convenient we will take V = {v2 . . Prove by induction that every hypercube has a Hamiltonian cycle. since F3 = 2. that is. 2. . 1. 71. The elements of V are called nodes or vertices. Prove by induction that every graph in which each vertex has even degree (that is.
73. where V is a ﬁnite set and E ⊆ V × V . Figure 2. start with a hypercube of dimension n − 1. and the elements of E are called edges. Prove by induction that a hypercube of dimension n has n2n−1 edges.
. v).
The hypercube of dimension n is a graph deﬁned as follows. (v.
Chap.16 66. 72.
(r.
1 Technically. Suppose r ∈ V1 . (r. 2n−k
75.11
TREES
A tree is a special kind of graph deﬁned as follows1 .
. Then. this is actually called a rooted tree. Ei ) are disjoint trees with roots ri respectively. . . . r2 ).11. . where V E = V1 ∪ V2 ∪ · · · ∪ Vk ∪ {r}.
2. Prove by induction that an ndimensional hypercube has exactly n k diﬀerent subcubes of dimension k. . E) is a tree. Trees
17
Dimension 0
Dimension 1
Dimension 2
Dimension 3
Dimension 4
Figure 2. V2 . Vk . r1 ). Suppose Ti = (Vi .
Prove by induction that the edges of an ndimensional hypercube can be colored using n colors so that no pair of edges that share a common vertex have the same color. Some hypercubes of various dimensions. rk )}.
74. for 1 ≤ i ≤ k. . T = (V.Sec. = E1 ∪ E2 ∪ · · · ∪ Ek ∪ {(r. but the distinction is not of prime importance here. . 2. . A single vertex v is a tree with root v.5.
18
Chap. 81. . A binary tree is a tree in which all nodes have at most two children. and for all 1 ≤ i < k. . For all i ≥ 1. v2 . (vi .
86.
82. where F ⊆ E. vi+1 ) ∈ E.
GEOMETRY
Prove that any set of regions deﬁned by n lines in the plane can be colored with two colors so that no two regions that share an edge have the same color. 2. A complete binary tree is a binary tree in which all leaves are at the same level. . E) is a tree T = (V. Prove by induction that between any pair of vertices in a tree there is a unique path. Prove by induction on n ≥ 3 that the sum of the angles of an nvertex convex polygon is 180(n−2) degrees. A cycle in a graph G = (V. 76. Mathematical Induction
The root of a tree is said to be at level 1. then n mod k = 1. Prove by induction that an nnode graph can have as many as (n−1)! spanning trees. Prove by induction that for every n such that n mod k = 1. Prove by induction that a tree is a connected graph that has no cycles. 80.
. Prove by induction that n circles divide the plane into n2 − n + 2 regions if every pair of circles intersect in exactly two points.
2. 79. A spanning tree of a graph G = (V. . 83. Does this hold for closed shapes other than circles? A polygon is convex if every pair of points in the polygon can be joined by a straight line that lies in the polygon. 77. and no three circles intersect in a common point. Prove by induction that a connected graph that has no cycles is a tree. The number of levels in a tree is the largest level number of any node in the tree. E) is a sequence of distinct vertices v1 .
85. a child of a level i node is said to be at level i + 1.12
84. vk such that (vk . v1 ) ∈ E. Prove by induction that if there exists an nnode tree in which all the nonleaf nodes have k children. F ). Prove by induction that a tree with n vertices has exactly n − 1 edges. Prove by induction that a complete binary tree with n levels has 2n − 1 vertices. 78. there exists an nnode tree in which all of the nonleaf nodes have k children.
If x.13
91. let x ⊕ y denote the exclusiveor of x and y. Given C ∈ IN. if a1 = 1. divide the plane into 2n regions. and that solution uses less than a2 coins of type a1 . . Suppose An = {a1 .
95. where each ai ∈ IN. Suppose also that a1 ≤ a2 ≤ · · · ≤ an .
Consider any arrangement of n ≥ 3 lines in general position in the plane (that is. Prove that at least n − 2 of the minimal connected regions in any such arrangement are triangles.
MISCELLANEOUS
Suppose Mi is an ri−1 × ri matrix. no three of which meet in a straight line. . that is. 92. no pair of lines is parallel. 90. Prove by induction on n that there must be at least one hole containing at least n/m pigeons. x2 . divide space into n(n − 2) + 2 regions. all passing through a single point. Show that for all n ≥ 2. . 2. for 1 ≤ i ≤ n. . Prove that n planes in space. a ⊕ (b ⊕ c) = (a ⊕ b) ⊕ c. Is this bound tight? Prove that n straight lines in the plane.
89.
2. . (The Pigeonhole Principle) Suppose n pigeons roost in m holes. Note that the exclusiveor operation is associative. for 1 ≤ i ≤ n.Sec. 93. Consider the following algorithm for listing the
. . Miscellaneous
19
87. . given that an unlimited number of coins of each type are available. Prove that at least one of the minimal connected regions that they form is a triangle. ﬁnd the smallest number of coins from An that add up to C. xn are true. Prove by induction on n that x1 ⊕ x2 ⊕ · · · ⊕ xn is true iﬀ an odd number of x1 . which is deﬁned to be true iﬀ exactly one of x and y is true. What happens when a1 = 1?
94.
88. a2 . Prove by induction on n ≥ 1 that the binary representation of n has log n + 1 bits. and no three lines can have a common point). . false}. then there is always a solution to the coinchanging problem. The coinchanging problem is deﬁned as follows.13. Prove by induction on n ≥ 1 that the matrix product M1 · M2 · · · Mn is an r0 × rn matrix. all passing through a single point.
A Gray code is a list of the 2n nbit strings in which each string diﬀers from the previous one in exactly one bit. Consider any arrangement of n ≥ 3 lines in general position in the plane. y ∈ {true. an } is a set of distinct coin types.
111. prove that any graph in which all vertices have even degree must have a cycle C (try constructing one). 86. 97. The circles are a red herring. 96. If n > 1. reverse the order of the strings. 110. 11. and place a 0 in front of each string.
2. Try combining four n/2 × n/2 tours into one.
. You will need to use the identity n r = n−1 r + n−1 r−1 . 100.
(Can you prove this identity?) Be careful how you apply this identity (it doesn’t always hold). for n = 2 the list is 00. Your induction hypothesis must be stronger than what you really need to prove — the tours you build must have some extra structure to allow them to be joined at the corners. 01. ﬁrst take the list of (n − 1)bit strings. for example. 011. take a second copy of the list of (n − 1)bit strings. 69. It can be proved as follows. It illustrates the technique that you will need. So. Show that every nbit string appears exactly once in the list generated by the algorithm. 101. 53. 2. 57. that each string diﬀers from the previous one in exactly one bit. First. and take special care with the upper and lower limits of the summation. The hardest part of this is the base case. You will need to use strong induction here. Show that the list has the Gray code property.
49. prove
85. consider the hint to Problem 48. place a 1 in front of each string. leaving a smaller graph in which all vertices have even degree. Look carefully at the example given before the problem. 10. 1. 48. Also. Apply the induction hypothesis (careful: there is something I am not telling you). Then. and for n = 3 the list is 000.14
37. Mathematical Induction
nbit strings. proving that the sum of the three angles of a triangle is 180 degrees.20
Chap. First.
HINTS
You will need to use strong induction. 010. Put together C and the results of applying the induction hypothesis to give the Eulerian cycle. Delete C. that is. And your base case may not be what you expect at ﬁrst. and place it after the ﬁrst list. 001. If n = 1. the list is 0.
according to whether n is a power of 2 (or something like that) or not. The i=0 2 = 2 0 i 2 = 1 = 21 − 1.
n i n+1 We are required to prove that for all n ≥ 0. Once you have mastered the base case. and n 1/2i = 1 − 1/2n . This can be extended to any triangle by judicious choice of diagonal. in which case i=0 Suppose that n ≥ 0 and n 2i = 2n+1 − 1. as required. claim is certainly true for n = 0. as required. The claim is true for n = 1.Sec. Now.
2. Solutions
21
it for a rightangled triangle (using symmetry).
n+1 n n
i =
i=1 i=1
i + (n + 1) (by the induction hypothesis)
= n(n + 1)/2 + (n + 1) = (n + 1)(n + 2)/2. Now suppose that n ≥ 1. 92.
We are required to prove that for all n ≥ 1. i=1 i = n(n + 1)/2. 8. We are required to prove that i=0
. Suppose that n ≥ 0 and n i = n(n + 1)/2.15
1. You’ll need to make two cases in your inductive step. i=1
n+1
n
1/2i
i=1
= = =
1 1 1 1 + + + · · · + n+1 2 4 8 2 1 1 + 2 2 1 1 + 2 2 1 1 1 1 + + + ··· + n 2 4 8 2
n i=1
1 2i (by the induction hypothesis)
1 1 1 + · (1 − n ) − 2 2 2 = 1 − 1/2n+1 . It remains to prove i=1 that n+1 1/2i = 1 − 1/2n+1 . 2. We are required to prove that i=1 n+1 i=1 i = (n + 1)(n + 2)/2. − 1. i=1 1/2i = 1 − 1/2n . since in this case both sides of the equation are equal to 1/2.15. in which case both sides of the equation are zero. 11.
SOLUTIONS
We are required to prove that for all n ≥ 0. the inductive step should be obvious. The claim is certainly true for n = 0.
2. 27. it follows that (n + 1)3 + 2(n + 1) is divisible by 3. Can you prove this? We are required to prove that for all n ≥ 0. Mathematical Induction
2i = 2n+2 − 1. Simply add a 3cent stamp to this to make n cents. The claim is true for n = 0. Now. which is certainly divisible by 3.
n n−1 n
52. n = 9 (three 3cent stamps). i=0 Fi = Fn+2 − 1. n − 3 cents can be made with 3cent and 5cent stamps. Notice that we can prove something stronger with just a little more work. Now suppose that n ≥ 1. as required. as required. and n−1 that i=0 Fi = Fn+1 − 1. Now. (n + 1)3 + 2(n + 1) = = = (n3 + 3n2 + 3n + 1) + (2n + 2) n3 + 3n2 + 5n + 3 (n3 + 2n) + (3n2 + 3n + 3). and n = 10 (two 5cent stamps). as required.
Fi
i=0
=
i=0
Fi + Fn (by the induction hypothesis)
= (Fn+1 − 1) + Fn = Fn+2 − 1. Then. We are required to prove that (n + 1)3 + 2(n + 1) is divisible by 3. since then n3 + 2n = 0. and n3 + 2n is divisible by 3. Suppose that n ≥ 0. and the righthand side is F2 − 1 = 1 − 1 = 0. We are required to show that any amount of postage that is a positive integer number of cents n > 7 can be formed by using only 3cent and 5cent stamps. since the lefthand side of the equation is F0 = 0. Since n ≥ 11. 37.
. and hence by hypothesis.
Since the ﬁrst term is divisible by 3 (by the induction hypothesis) and the second term is obviously divisible by 3. The claim is certainly true for n = 8 (one 3cent and one 5cent stamp). The required postage can always be made using at most two 5cent stamps.22
n+1 i=0
Chap.
n+1 n
2i
i=0
=
i=0
2i + 2n+1 (by the induction hypothesis)
= (2n+1 − 1) + 2n+1 = 2n+2 − 1. The claim is certainly true for n = 0. n − 3 ≥ 8.
We are required to prove by induction on n ≥ 0 that n3 + 2n is divisible by 3. Now suppose that n ≥ 11 and all values up to n − 1 cents can be made with 3cent and 5cent stamps.
2. he came up with the wrong answer. then by the induction hypothesis. Reverse all of the colors on one side of the line.Sec. Some time later. then the two regions formed a single region before L was replaced. and started to add. 2. If that line is L. Gauss’ teacher set the class to add the numbers 1 + 2 + · · · + 100. Shading the regions formed by all lines except L (top).
84. Gauss came to the teacher within a few minutes with an answer of 5050. He refused to believe that he was wrong until Gauss explained:
. and wanted to occupy the students for a long time so that he could read a book.16
1. the other side dark). and the new shading obtained by ﬂipping the colors on one side of L (bottom). There is an apocryphal story attached to this discovery. Replace L. Comments
23
L
Figure 2.16. Suppose we are given n + 1 lines in the plane. Since we reversed colors on one side of L only. by the induction hypothesis). If that line is not L. they now have diﬀerent colors. after some fretting and fuming. the two regions have diﬀerent colors (either the same as before or reversed). and color the remaining regions with two colors (which can be done. Another said that it was punishment for the class being unruly and disruptive. Consider two regions that have a line in common. The teacher was dismayed.
COMMENTS
This identity was apparently discovered by Gauss at age 9 in 1786.
We are required to prove that any set of regions deﬁned by n lines in the plane can be colored with only two colors so that no two regions that share an edge have the same color.6. One account I have heard said that the teacher was lazy. The hypothesis is true for n = 1 (color one side light. Remove one of the lines L. Now suppose that the hypothesis is true for n lines.
There is an elegant noninductive proof of this that you should have met in high school. If you want to practice this technique. I realize that you can solve this without induction by applying some elementary algebra and the identities of Problems 1 and 2. start with k = 2. Can you prove this result without using induction? The case for even n can easily be inferred from Gauss’ example. Then. If you want a challenge to your ability to do arithmetic. It gets a bit tricky as k becomes larger. This is a special case of the identity in Problem 9. See the comment to Problem 4.
aS − S =
i=1
ai −
i=0
ai = an+1 − 1. 4. 11 · · · 1 . that is.24
Chap. i=0
n+1 n
5. 11. and try to prove it by induction. I can never remember the exact form of the solution. Let S = n ai . Fortunately. there is not much need to. In general. See the comment to Problem 4. if you are asked to solve
n
ik
i=1
for some k ≥ 1. 0 ≤ i ≤ k + 1. The case for odd n is similar. What I want you to do is practice your induction by proving this identity from ﬁrst principles. The right values for the constants will drop right out of the proof. 6. The binary representation of i=0 2i is a string of n + 1 ones. Simply hypothesize that the sum is ak+1 nk+1 + ak nk + · · · + a1 n + a0 for some choice of ai ∈ IR. This can also be solved by applying the identity of Problem 9 twice. + 3. as required. Mathematical Induction
1 + 2 + ··· + 50 100 + 99 + · · · + 51 101 + 101 + · · · + 101 = 50 × 101 = 5050. just remember that the solution is a polynomial in n with largest exponent k + 1 (see Problem 21). Yes. 9. try k = 4.
n+1
. I could go on asking for more identities like this. There is an easy noninductive proof of this fact that appeals more to computer science students than n math students. 2.
Hence. 10. though. S = (an+1 − 1)/(a − 1).
ﬁt them together to “nearly” make a rectangle. in binary. And. 2n = Ω(n2 ). The formal study of the knight’s tour problem is said
. Therefore. you get. See Chapter 3. There are actually closed knight’s tours on n × n chessboards for all even n ≥ 6 (see Problem 369). and analyze the missing area. Comments
25
2 n1 n1 n 2n 2 n1
Figure 2.
If you add 1 to this.
This can also be solved using the method of areas sketched in the comment to Problem 13. 2n+1 − 1. See Chapter 3. 16.
n+1
that is. 2. This can also be solved using algebra and the identity from Problem 15. therefore. 15.16. And. therefore.Sec.7) (by Problem 1)
= (2n − 1)2n − (2 − 2) = (n − 1)2n+1 + 2.
n i=0
2i =
There is an elegant noninductive proof of this using the method of areas. 19. This technique works well for sums of products: Simply make a rectangle with area equal to each term of the sum. But we’re getting ahead of ourselves here.
n i=0
2i + 1 = 2n+1 or.
n n−1
i2i
i=1
= (2n − 1)2n −
i=1 n
2i
(see Figure 2. But we’re getting ahead of ourselves here. 1 00 · · · 0. Using the method of areas to show that n n−1 i i n i=1 i2 = (2n − 1)2 − i=1 2 .7. 13. Note that Problem 47 implies that there can be no such tour for odd n. 3n = O(n!). equivalently. 48. 2n+1 . 20.
67. you must have written exactly
n
strings with exactly
i·
i=0
n i
ones. 60. See the comment for Problem 81. The Swiss mathematician Leonhard Euler ﬁrst proved this in 1736 [24]. What I want you to do is point out where in the proof is there a false statement made? This can be proved using Problem 1. 69. but what I’m looking for is a straightforward inductive proof. since all horses are not of the same color.
. You’re supposed to be practicing your induction! See the comment for Problem 60. This problem was suggested to the author by Nainan Kovoor in 1990. Since there are i ones.11. we conclude that a graph has a Eulerian cycle iﬀ it is connected and all vertices have even degree. 64. and you have written n2n bits in all. who considered the standard 8 × 8 chessboard. Rouse Ball and Coxeter [8] give an interesting bibliography of the history of the problem from this point. This problem together with Problem 82 show that “a connected graph with no cycles” is an alternate deﬁnition of a tree.
n i
Write down every string of n bits.26
Chap. An Eulerian cycle is not possible if the graph has at least one vertex of odd degree. 51.
82. 62.
81. There is an elegant noninductive proof of the identity
n
i·
i=0
n i
= n2n−1 . But since you have written exactly as many ones as zeros. Of course it is obvious that the proof is wrong. 61. You’ll see this in the literature as often as you’ll see the recursive deﬁnition given at the beginning of Section 2. Don’t cheat by using Stirling’s approximation. Mathematical Induction
to have begun with Euler [25] in 1759. there must be n2n−1 ones. 2. (Can you prove this?) Therefore.
It’s easier to prove by contradiction.Sec.
Yes. 95. But it’s good practice in using induction.16.
. this is a little contrived. See also Problems 415 and 471. 2. Comments
27
94.
Then. f (n) ≤ c · g(n). • f (n) is Ω(g(n)) if there exists c. The proper deﬁnition of “O(g(n))” is as a set: O(g(n)) = {f (n)  there exists c. • f (n) is Θ(g(n)) if f (n) is O(g(n)) and f (n) is Ω(g(n)).1
98. f (n) ≤ c · g(n)}. Suppose that f. We will follow the normal convention of writing. f (n) ≥ c · g(n). for example.
RANK THE FUNCTIONS
Consider the following eighteen functions: √ n n log n n2 n1/3 + log n ln n (1/3)n n n − n3 + 7n5 n3 (log n)2 n/ log n (3/2)n 2n n2 + log n log n n! log log n 6
28
.Chapter 3
BigO and BigΩ
BigO notation is useful for the analysis of algorithms since it captures the asymptotic growth pattern of functions and ignores the constant multiple (which is out of our control anyway when algorithms are translated into programs). • f (n) ∼ g(n) if limn→∞ f (n)/g(n) = 1. • f (n) is o(g(n)) if limn→∞ f (n)/g(n) = 0. n0 ∈ IR+ such that for all n ≥ n0 .”
3. n0 ∈ IR+ such that for all n ≥ n0 . n0 ∈ IR+ such that for all n ≥ n0 . “f (n) = O(g(n))” can be interpreted as meaning “f (n) ∈ O(g(n)). • f (n) is ω(g(n)) if limn→∞ g(n)/f (n) = 0. We will use the following deﬁnitions (although alternatives exist. even though the equality sign does not indicate true equality. • f (n) is O(g(n)) if there exists c. “f (n) = O(g(n))” instead of “f (n) is O(g(n))”. see Section 3. g : IN → IN.5).
g(n) = n n/2. √ f (n) = n + log n. 107. Rank the Functions
29
Group these functions so that f (n) and g(n) are in the same group if and only if f (n) = O(g(n)) and g(n) = O(f (n)). 100. f (n) = (n2 − n)/2. g(n) = n n. 106. 102. f (n) = 4n log n + n. 101.1. either f (n) = O(g(n)) or g(n) = O(f (n)).00001 ) O(n2 / log2 n) O(n2 / log n) O(n2 ) O(n3/2 ) O(2n ) O(5n ) O(nn ) 2 O(nn )
1/(log n) 7n5 − 3n + 2 (n2 + n)/(log2 n + log n) 2 2log n 3n
For each of the following pairs of functions f (n) and g(n). Determine which is the case. and list the groups in increasing order. 3. g(n) = (n2 − n)/2.00001 ) Ω(n2 / log2 n) Ω(n2 / log n) Ω(n2 ) Ω(n3/2 ) Ω(2n ) Ω(5n ) Ω(nn ) 2 Ω(nn ) O(1/n) O(1) O(log log n) O(log n) O(log2 n) √ O( 3 n) O(n/ log n) O(n) O(n1. 103. √ f (n) = n + 2 n. g(n) = n3 . but not both. f (n) = 2(log n)2 . √ f (n) = n + log n. g(n) = n. 105. √ f (n) = n log n. Draw a line from each of the ﬁve functions in the center to the best bigΩ value on the left. 104. g(n) = log n + 1. 99. Ω(1/n) Ω(1) Ω(log log n) Ω(log n) Ω(log2 n) √ Ω( 3 n) Ω(n/ log n) Ω(n) Ω(n1. g(n) = 6n. and the best bigO value on the right. f (n) = n2 + 3n + 4.Sec.
. g(n) = n2 .
g(n) = log(n + 3). 119. 3. 129. 124. 132. √ f (n) = n. 126. f (n) = 2n − n2 .30
Chap.
TRUE OR FALSE?
n2 = O(n3 ). 125. 123. 111.2
108. √ n = O(log n).
For each of the following pairs of functions f (n) and g(n). 122. n3 = O(n2 (1 + n2 )). 115. 1/n = O(log n). If f (n) ∼ g(n). 116. log n = O(n−1/2 ). If f (n) = Θ(g(n)). √ √ n + n = O( n log n). √ log n = O( n). 112. √ f (n) = n n. f (n) = n2 + 3n + 4. 109. 117. g(n) = 6n + 7. √ √ √ 3n2 + n = O(n + n n + n). 2n2 + 1 = O(n2 ). then g(n) = Θ(f (n)). 127. 120. 128. n3 = O(n2 ). 114. f (n) = (n2 + 2)/(1 + 2−n ). g(n) = n + 3. then f (n) = Θ(g(n)). √ n2 (1 + n) = O(n2 log n). 121. state whether f (n) = O(g(n)). 130. √ log n + n = O(n). log n = O(1/n). 131. √ 3n2 + n = O(n2 ). √ n log n = O(n). √ n2 (1 + n) = O(n2 ). or none of the above. g(n) = n4 + n2 . 118. f (n) = Θ(g(n)).
. √ f (n) = n + n n. g(n) = n2 − n. g(n) = 4n log(n2 + 1). f (n) = Ω(g(n)). 110. 113. √ n log n = O(n n). BigO and BigΩ
3.
Prove that cn2 + d = O(2n ) for all c. 3. Prove or disprove: O n2 log log n
1/2
= O(
√ n ).
Prove that log n = O(n). Does nlog log log n = Ω((log n)!)? Prove your answer.3
133. 153. Prove that n! = Ω(2n ). 142. . 146.3. 154. 135. . Prove that 3n2 − 8n + 9 = O(n2 ). 137. Does (n!)! = O(((n − 1)!)!(n − 1)!n! )? Prove your answer. 152. Does nlog n = Ω((log n)n )? Prove your answer. 144. Prove that n2 = O(2n ). ak nk + ak−1 nk−1 + · · · + a1 n + a0 = O(nk ). ak−1 . 148. d ∈ IR+ . d. . Prove that cn + d = O(2n ) for all c.
136. k ∈ IR+ . d ∈ IR+ .Sec. Prove that 2n + 1 = O(2n ). 140. Prove that 3n log n = O(n2 ). 134. Prove that for all k ≥ 1 and all ak . 147. . Proving BigO
31
3. 150. Does nlog n = O((log n)n )? Prove your answer. Prove that cnk + d = O(2n ) for all c. 143. 141.
. Prove that 2n = O(n!). 139. 155. Prove that n3 − 3n2 − n + 1 = Θ(n3 ). 138. Does nlog log log n = O((log n)!)? Prove your answer. Does (n!)! = Ω(((n − 1)!)!(n − 1)!n! )? Prove your answer. Prove that n = O(2n ). Prove that 9999n + 635 = O(2n ). 149. a1 . Prove that n2 − 3n − 18 = Ω(n).
PROVING BIGO
Prove that (n + 1)2 = O(n2 ). 145. a0 ∈ IR. 151.
f (n) = log n. then f1 (n) + f2 (n) = O(max{g1 (n). Prove that if f1 (n) = Ω(g1 (n)) and f2 (n) = Ω(g2 (n)). f (n) = n2 + n + 1. Prove or disprove: 2(1+O(1/n)) = 2 + O(1/n). g(n) = n10 . g(n) = n − n . 164. f (n) = n log n. ﬁnd c ∈ IR+ such that f (n) ≤ c · g(n) for all n > 1. 161. 3. g(n) = n(log n)2 . 172. In each case. 168. g(n) = n2 . 158.
2
Chap.
MANIPULATING BIGO
Prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)). g(n) = 2n − 1. g(n) = n + n2 . g(n) = n2 /2.
. g2 (n)}). 157. g(n) = n + (log n)2 . g(n) = (n2 − 6n)/2. √ f (n) = n n + n2 . 163. √ √ f (n) = 5 n − 1. g(n) = (log n)log n . f (n) = 12n + 3. f (n) = n2 /log n. 162. 160.
6 −6
For each of the following pairs of functions f (n) and g(n). f (n) = 5n + 1. or f = Θ(g). f = ω(g). g(n) = 3n . 166. 165. 167. f (n) = n2 − n + 1.
3. f (n) = 100n + log n. 173. and prove your claim.4
171. say whether f = o(g). g(n) = 2n3 . g(n) = log log(n2 ). g(n) = n2 . then f1 (n) + f2 (n) = O(g1 (n) + g2 (n)). BigO and BigΩ
Compare the following pairs of functions f . 159. 170. Prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)). 169. f (n) = n2n . f (n) = n2 + n. g.32 156. then f1 (n) + f2 (n) = Ω(g1 (n) + g2 (n)). √ f (n) = 2 n + 1. f (n) = (log n)10 .
183. 188. Prove or disprove: If f (n) = O(g(n)).
Prove that if f1 (n) = Ω(g1 (n)) and f2 (n) = Ω(g2 (n)). Show that bigO is transitive.5
ALTERNATIVE DEFINITIONS
Here is an alternative deﬁnition of O. then f1 (n)·f2 (n) = Ω(g1 (n) · g2 (n)). either f (n) = O(g(n)) or g(n) = O(f (n)). then f (n)/h(n) = O(g(n)/h(n)).5. g2 (n)}). then O(f (n) + g(n)) = f (n) + O(g(n)). Prove that if f (n) = O(g(n)). if f (n) = O(g(n)) and g(n) = O(h(n)). Is it true that f1 (n) + f2 (n) = Θ(g1 (n) + g2 (n))? Is it true that f1 (n) + f2 (n) = Θ(max{g1 (n). Alternative Deﬁnitions
33
174. Prove that h(n) = O(f (n)) iﬀ h(n) = O(g(n)). 181. That is. 180. g2 (n)})? Justify your answer. or ﬁnd a counterexample to this claim.
3. √ 182. 178. 184. Prove or disprove: If f (n) > 0 and g(n) > 0 for all n.
176. 179. Multiply log n + 6 + O(1/n) by n + O( n) and simplify your answer as much as possible.
Prove or disprove: O(x + y)2 = O(x2 ) + O(y 2 ). 187. then 2f (n) = O(2g(n) ).
. 177. then f (n) = O(h(n)). Prove that if f (n) = O(g(n)).Sec. Prove or disprove: If f (n) = O(g(n)). 186. Prove or disprove: For all functions f (n) and g(n). Prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)). then log f (n) = O(log g(n)). 189. Suppose that f1 (n) = Θ(g1 (n)) and f2 (n) = Θ(g2 (n)). 3. g2 (n)})? Is it true that f1 (n) + f2 (n) = Θ(min{g1 (n). then f (n) = O1 (g(n)). Prove or disprove: If f (n) = O(g(n)). then f1 (n) + f2 (n) = Ω(min{g1 (n). then f (n)k = O(g(n)k ). 175. Prove that if f1 (n) = Ω(g1 (n)) and f2 (n) = Ω(g2 (n)). Prove or disprove: O(f (n)α ) = O(f (n))α for all α ∈ IR+ . • f (n) is O1 (g(n)) if there exists c ∈ IR such that limn→∞ f (n)/g(n) = c. Suppose f (n) = Θ(g(n)). then f1 (n) · f2 (n) = O(g1 (n) · g2 (n)). 185.
34 190.
Chap. 3. BigO and BigΩ
Prove that if f (n) = O1 (g(n)), then f (n) = O(g(n)), or ﬁnd a counterexample to this claim.
Here are two alternative deﬁnitions of Ω. • f (n) is Ω1 (g(n)) if there exists c ∈ IR+ such that for inﬁnitely many n, f (n) ≥ c · g(n). • f (n) is Ω2 (g(n)) if there exists c ∈ IR+ such that for all n0 ∈ IN, there exists n ≥ n0 such that f (n) ≥ c · g(n). 191. 192. 193. 194. 195. 196. Prove that if f (n) = Ω(g(n)), then f (n) = Ω2 (g(n)), or ﬁnd a counterexample to this claim. Prove that if f (n) = Ω2 (g(n)), then f (n) = Ω(g(n)), or ﬁnd a counterexample to this claim. Prove that if f (n) = Ω1 (g(n)), then f (n) = Ω2 (g(n)), or ﬁnd a counterexample to this claim. Prove that if f (n) = Ω2 (g(n)), then f (n) = Ω1 (g(n)), or ﬁnd a counterexample to this claim. Prove or disprove: If f (n) = O(g(n)), then f (n) = Ω(g(n)). If f (n) = O(g(n)), then f (n) = Ω2 (g(n)). Deﬁne the relation ≡ by f (n) ≡ g(n) iﬀ f (n) = Ω(g(n)) and g(n) = Ω(f (n)). Similarly, deﬁne the relation ≡2 by f (n) ≡ g(n) iﬀ f (n) = Ω2 (g(n)) and g(n) = Ω2 (f (n)). Show that ≡ is an equivalence relation, but ≡2 is not an equivalence relation.
3.6
FIND THE FUNCTIONS
Find two functions f (n) and g(n) that satisfy the following relationships. If no such f and g exist, write “None.” 197. 198. 199. 200. 201. f (n) = o(g(n)) and f (n) = Θ(g(n)). f (n) = Θ(g(n)) and f (n) = o(g(n)). f (n) = Θ(g(n)) and f (n) = O(g(n)). f (n) = Ω(g(n)) and f (n) = O(g(n)). f (n) = Ω(g(n)) and f (n) = o(g(n)).
Sec. 3.7. Hints
35
3.7
98. 99.
HINTS
There are a lot of groups. Be careful with (n2 + n)/(log2 n + log n).
136. When solving problems that require you to prove that f (n) = O(g(n)), it is a good idea to try induction ﬁrst. That is, pick a c and an n0 , and try to prove that for all n ≥ n0 , f (n) ≤ c · g(n). Try starting with n0 = 1. A good way of guessing a value for c is to look at f (n) and g(n) for n = n0 , · · · , n0 + 10. If the ﬁrst function seems to grow faster than the second, it means that you must adjust your n0 higher — eventually (unless the thing you are trying to prove is false), the ﬁrst function will grow more slowly than the second. The ratio of f (n0 )/g(n0 ) gives you your ﬁrst cut for c. Don’t be afraid to make c higher than this initial guess to help you make it easier to prove the inductive step. This happens quite often. You might even have to go back and adjust n0 higher to make things work out correctly. 171. Start by writing down the deﬁnitions for f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)).
3.8
SOLUTIONS
133. We are required to prove that (n + 1)2 = O(n2 ). We need to ﬁnd a constant c such that (n + 1)2 ≤ cn2 . That is, n2 + 2n + 1 ≤ cn2 or, equivalently, this possible? It is if c > 1. Take c = 4. The roots (c − 1)n2 − 2n − 1 ≥ 0. Is√ of 3n2 − 2n − 1 are (2 ± 4 + 12)/2 = {−1/3, 1}. The second root gives us the correct value for n0 . Therefore, for all n ≥ 1, (n + 1)2 ≤ 4n2 , and so by deﬁnition, (n + 1)2 = O(n2 ). 136. We are required to prove that log n = O(n). By looking at log n for small values of n, it appears that for all n ≥ 1, log n ≤ n. The proof is by induction on n. The claim is certainly true for n = 1. Now suppose that n > 1, and log(n − 1) ≤ n − 1. Then, log n ≤ log(n − 1) + 1 ≤ (n − 1) + 1 (by the induction hypothesis) = n.
Hence, we can take c = 1 and n0 = 1. 137. We are required to prove that 3n log n = O(n2 ). By looking at 3n log n for small values of n, it appears that for all n ≥ 1, 3n log n ≤ 3n2 . The proof is by induction on n. The claim is certainly true for n = 1. Now suppose that
36 n > 1, and 3(n − 1) log(n − 1) ≤ 3(n − 1)2 . Then,
Chap. 3. BigO and BigΩ
3n log n 3n( log(n − 1) + 1) 3(n − 1)( log(n − 1) + 1) + 3( log(n − 1) + 1) 3(n − 1) log(n − 1) + 3(n − 1) + 3( log(n − 1) + 1) 3(n − 1)2 + 3(n − 1) + 3( log(n − 1) + 1) (by the induction hypothesis) ≤ 3(n − 1)2 + 3(n − 1) + 3n (see the solution to Problem 136) = 3n2 − 6n + 3 + 3n − 3 + 3n = 3n2 . ≤ = = ≤ Hence, we can take c = 3 and n0 = 1. 171. We are required to prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)), then f1 (n)+f2 (n) = O(g1 (n)+g2 (n)). Suppose for all n ≥ n1 , f1 (n) ≤ c1 ·g1 (n) and for all n ≥ n2 , f2 (n) ≤ c2 ·g2 (n). Let n0 = max{n1 , n2 } and c0 = max{c1 , c2 }. Then for all n ≥ n0 , f1 (n) + f2 (n) ≤ c1 · g1 (n) + c2 · g2 (n) ≤ c0 (g1 (n) + g2 (n)).
3.9
COMMENTS
173. This is often called the sum rule for bigOs. 176. This is often called the product rule for bigOs.
Chapter 4
Recurrence Relations
Recurrence relations are a useful tool for the analysis of recursive algorithms, as we will see later in Section 6.2. The problems in this chapter are intended to develop skill in solving recurrence relations.
4.1
SIMPLE RECURRENCES
T (1) = 1, and for all n ≥ 2, T (n) = 3T (n − 1) + 2. T (1) = 8, and for all n ≥ 2, T (n) = 3T (n − 1) − 15. T (1) = 2, and for all n ≥ 2, T (n) = T (n − 1) + n − 1. T (1) = 3, and for all n ≥ 2, T (n) = T (n − 1) + 2n − 3. T (1) = 1, and for all n ≥ 2, T (n) = 2T (n − 1) + n − 1. T (1) = 5, and for all n ≥ 2, T (n) = 2T (n − 1) + 3n + 1. T (1) = 1, and for all n ≥ 2 a power of 2, T (n) = 2T (n/2) + 6n − 1. T (1) = 4, and for all n ≥ 2 a power of 2, T (n) = 2T (n/2) + 3n + 2. T (1) = 1, and for all n ≥ 2 a power of 6, T (n) = 6T (n/6) + 2n + 3. T (1) = 3, and for all n ≥ 2 a power of 6, T (n) = 6T (n/6) + 3n − 1. T (1) = 3, and for all n ≥ 2 a power of 3, T (n) = 4T (n/3) + 2n − 1. T (1) = 2, and for all n ≥ 2 a power of 3, T (n) = 4T (n/3) + 3n − 5. T (1) = 1, and for all n ≥ 2 a power of 2, T (n) = 3T (n/2) + n2 − n. T (1) = 4, and for all n ≥ 2 a power of 2, T (n) = 3T (n/2) + n2 − 2n + 1. T (1) = 1, and for all n ≥ 2 a power of 2, T (n) = 3T (n/2) + n − 2. 37
Solve the following recurrences exactly. 202. 203. 204. 205. 206. 207. 208. 209. 210. 211. 212. 213. 214. 215. 216.
38 217. 218. 219. 220.
Chap. 4. Recurrence Relations
T (1) = 1, and for all n ≥ 2 a power of 2, T (n) = 3T (n/2) + 5n − 7. T (1) = 1, and for all n ≥ 2 a power of 3, T (n) = 4T (n/3) + n2 . T (1) = 1, and for all n ≥ 2 a power of 3, T (n) = 4T (n/3) + n2 − 7n + 5. √ T (1) = 1, and for n ≥ 4 a power of 4, T (n) = T (n/4) + n + 1.
4.2
221.
MORE DIFFICULT RECURRENCES
Suppose 0 < α, β < 1, where α + β = 1. Let T (1) = 1, and for all n ≥ 1, T (n) = T (αn) + T (βn) + cn, for some c ∈ IN. Prove that T (n) = O(n log n). You may make any necessary assumptions about n. The Fibonacci numbers Fn for n ≥ 0 are deﬁned recursively as follows: F0 = 0, F1 = 1, and for n ≥ 2, Fn = Fn−1 +Fn−2 . Prove by induction √ √ √ ˆ ˆ on n that Fn = (φn − φn )/ 5, where φ = (1 + 5)/2, and φ = (1 − 5)/2.
222.
Let X(n) be the number of diﬀerent ways of parenthesizing the product of n values. For example, X(1) = X(2) = 1, X(3) = 2 (they are (xx)x and x(xx)), and X(4) = 5 (they are x((xx)x), x(x(xx)), (xx)(xx), ((xx)x)x, and (x(xx))x). 223. Prove that if n ≤ 2, then X(n) = 1; and otherwise
n−1
X(n) =
k=1
X(k) · X(n − k)
224. 225.
Show that for all n ≥ 1, X(n) ≥ 2n−2 . Show that X(n) = 1 n 2n − 2 n−1 .
Solve the following recurrences exactly. 226. 227. 228. T (1) = 1, T (2) = 6, and for all n ≥ 3, T (n) = T (n−2)+3n+4. T (0) = c, T (1) = d, and for n > 1, T (n) = T (n − 2) + n. T (1) = 1, T (2) = 6, T (3) = 13, and for all n ≥ 4, T (n) = T (n − 3) + 5n − 9. 229. 230. T (1) = 1, and for all n ≥ 2, T (n) = 2T (n − 1) + n2 − 2n + 1. T (1) = 1, and for all n ≥ 2, T (n) = n · T (n − 1) + n.
4
RECURRENCES WITH FULL HISTORY
Solve the following recurrences exactly.1]). 235.
Prove that the asymptotic solution of the recurrence relation T (n) = 0 2T (n − c) + k if 0 ≤ n < c otherwise.3.
4.
234. Hopcroft.
State and prove a general formula for recurrences of the form T (n) = d aT (n/c) + bnk if n ≤ 1 otherwise. Theorem 1.
where c. and for all n ≥ 2. d if n ≤ 1 aT (n/c) + b otherwise.
. State and prove a general formula for recurrences of the form T (n) = 232.
n−1
T (n) =
i=1
T (i) + 7. 231.
State and prove a general formula for recurrences of the form T (n) = d aT (n/c) + bn2 if n ≤ 1 otherwise.3
GENERAL FORMULAE
It is normal to teach the general solution to recurrences of the form T (n) = aT (n/c) + bn (see.
T (1) = 1. T (1) = 1. and for all n ≥ 2. 4.
n−1
T (n) =
i=1
T (i) + 1. General Formulae
39
4.
236. for example. k ∈ IN.
233. Aho. and Ullman [1. is T (n) = Θ(dn ) for some constant d. The following are interesting variants of this recurrence.Sec.
5
241. 247. 4. T (n) = 2T ( n/2 ) + 6n − 1. T (1) = 2.
4.
240.
RECURRENCES WITH FLOORS AND CEILINGS
Suppose T (1) = 1. T (n) = T ( n/2 ) + n − 1. 243. T (n) = T ( n/2 ) + 1. T (n) = n ) + 1.
238. and for all n ≥ 2.
n−1
Chap. T (1) = 1. T (n) = T ( n/ log n ) + 1. 242. Suppose T (1) = 1. 246.
. T (1) = 1 and for all n ≥ 2. and for n ≥ 2. Solve the following recurrence relation exactly. T (n) = T ( n/2 ) + T ( n/2 ) + n − 1. 244. and for all n ≥ 3. and for all n ≥ 2. and for all n ≥ 2. 245.
T (1) = 1.
239.
Solve the following recurrence: T (1) = T (2) = 1. T (1) = 1. T (n) = T ( n/2 ) + 1. and for all n ≥ 2. Show that 2n − log n − 1 ≤ T (n) ≤ 2n − log n /2 − 1. T (n) = 4T ( n/3 ) + 3n − 5. T (1) = 1 and for all n ≥ 2. and for all n ≥ 2.40 237.
n−1
T (n) =
i=1
(T (i) + T (n − i)) + 1. Recurrence Relations
T (n) =
i=1
T (i) + n2 . Solve the following recurrence relation exactly. and for all n ≥ 2. T( √ Solve the following recurrence: T (1) = 1.
T (1) = 1.
T (1) = 1.
n−1
T (n) =
i=1
T (n − i) + 1. Solve the following recurrence relation exactly.
248. Find an exact solution for T (n). and for all n ≥ 2.
n−1
T (n) = 2
i=1
T (i) + 1. Solve the following recurrence relation exactly. and for all n ≥ 2.
f ( x ) = f (x) . The base case will be n ≤ 4.76 log n/ log log n.Sec. but this may not help you much. one for odd n and one for even n. T (n) ≤ 1. The answer is that for n ≥ 3. you will need to make a special case for even n and one for odd n. T (n) = 3T (n − 1) + 2.6
HINTS
202. and has the property that if f (x) ∈ Z then x ∈ Z Then. then by repeated substitution.1)
. Suppose T (1) = 1. This can be proved by induction. 242.
i−1
T (n) = 3i T (n − i) + 2
j=0
3j . The solution is T (n) = n log n −2 log n +1. Knuth. 224. Try repeated substitution (see the comment to Problem 202 for a deﬁnition of this term). If you think about it. and for all n ≥ 2. 226. Use induction on n. Z. but it becomes easy when you use the following fact from Graham. Hints
41
4. 248. 4. If n is large enough. This problem is diﬃcult to solve directly.
(4.6]. Section 6. T (n) = = = = = 3T (n − 1) + 2 (after one substitution) 3(3T (n − 2) + 2) + 2 9T (n − 2) + 2 · 3 + 2 (after two substitutions) 9(3T (n − 3) + 2) + 2 · 3 + 2 27T (n − 3) + 2 · 9 + 2 · 3 + 2 (after three substitutions). Therefore. the recurrences in this group of problems are fairly easy. Once you realize this. Can you derive this answer from ﬁrst principles? 243. After i substitutions.
4. and and Patashnik [30.6. Suppose f : IR → IR is continuous and monotonically increasing. this is really two independent recurrence relations. f ( x ) = f (x) and Z.7
SOLUTIONS
202.
It looks like there’s a pattern developing.
Suppose we take i = n − 1. after i substitutions. Recurrence Relations
We can prove that identity (4. T (n) = 2T (n/2) + 6n − 1.
. Now suppose that i > 1 and that
i−2
T (n) = 3i−1 T (n − i + 1) + 2
j=0
3j .
as required.
Therefore.
n−2
T (n) = 3
n−1
T (1) + 2
j=0
3j (by Problem 9)
= 3n−1 + 3n−1 − 1 = 2 · 3n−1 − 1.
208.42
Chap. and for all n ≥ 2 a power of 2. It is trivially true for i = 1. T (n) = = = = = 2T (n/2) + 6n − 1 (after one substitution) 2(2T (n/4) + 6n/2 − 1) + 6n − 1 4T (n/4) + (6n − 2) + (6n − 1) (after two substitutions) 4(2T (n/8) + 6n/4 − 1) + (6n − 2) + (6n − 1) 8T (n/8) + (6n − 4) + (6n − 2) + (6n − 1) (after three substitutions). then by repeated substitution. by identity (4. Suppose T (1) = 1.1) is true by induction on i. If n is large enough. Now that we’ve established identity (4. 4.
i−1
T (n) = 2i T (n/2i ) + 6in −
j=0
2j .
i−2
T (n) = 3i−1 T (n − i + 1) + 2
j=0
3j
i−2
= 3i−1 (3T (n − i) + 2) + 2
j=0 i−1
3j
= 3i T (n − i) + 2
j=0
3j .
Then.1). we can continue with solving the recurrence.1). Then.
Therefore. T (n) = 3log n − 4n2 (3/4)log n + 4n2 − 2n(3/2)log n + 2n = nlog 3 − 4nlog 3 + 4n2 − 2nlog 3 + 2n = 4n2 − 5nlog 3 + 2n. T (2) = 6.Sec. T (n) = = = = = 3T (n/2) + n2 − n (after one substitution) 3(3T (n/4) + n2 /4 − n/2) + n2 − n 9T (n/4) + 3(n2 /4 − n/2) + (n2 − n) (after two substitutions) 9(3T (n/8) + n2 /16 − n/4) + 3(n2 /4 − n/2) + (n2 − n) 27T (n/8) + 9(n2 /16 − n/4) + 3(n2 /4 − n/2) + (n2 − n) (after three substitutions). 226.
log n−1
T (n) = nT (1) + 6n log n −
j=0
2j (by Problem 11)
= n + 6n log n − (2log n − 1) = 6n log n + 1.7. after i substitutions. T (n) = T (n − 2) + 3n + 4 = T (n − 4) + (3(n − 2) + 4) + (3n + 4) = T (n − 6) + (3(n − 4) + 4) + (3(n − 2) + 4) + (3n + 4). T (n) = T (n − 2) + 3n + 4. If n is large enough. then by repeated substitution.
i−1 i−1
T (n) = 3i T (n/2i ) + n2
j=0 i i 2
(3/4)j − n
j=0 i 2
(3/2)j (by Problem 11). then by repeated substitution. Suppose T (1) = 1. Solutions
43
This can be veriﬁed easily by induction. T (n) = 3T (n/2) + n2 − n. and for all n ≥ 2 a power of 2.
= 3 T (n/2 ) − 4n (3/4) + 4n − 2n(3/2)i + 2n
This can be veriﬁed easily by induction.
i−1
T (n) =
T (n − 2i) + 3
j=0
(n − 2j) + 4i
. 4. If n is large enough. after i substitutions.
214. Hence. Hence. taking i = log n. and for all n ≥ 3. taking i = log n. Suppose T (1) = 1. Therefore.
Now. we have reduced a recurrence with full history to a simple recurrence that can be solved by repeated substitution (I’ll leave that to you) to give the solution T (n) = 2n−1 .
4. Take i = n/2 − 1.44
i−1
Chap.
T (n) − T (n − 1)
= (
i=1
T (i) + 1) − (
i=1
T (i) + 1)
= T (n − 1).
Therefore. T (n) =
n−1 n−1 i=1 n−2
T (i) + 1. Take i = (n − 1)/2. when n is even.8
COMMENTS
202. It works as follows. Therefore. suppose that n is odd. and when n is odd.
This can be veriﬁed easily by induction. Then.
i−1
T (n) = = = =
T (n − 2i) + 3
j=0
(n − 2j) + 4i
T (1) + 3n(n − 1)/2 − 3((n − 1)/2)2 + 7(n − 1)/2 1 + 3n2 /2 − 3n/2 − 3n2 /4 + 3n/2 − 3/4 + 7n/2 − 7/2 3n2 /4 + 7n/2 − 13/4. T (n) = (3n2 + 14n − 16 + 3(n mod 2))/4. Write down a formula for T (n) in terms of n and the number of substitutions (which you can call i). and for all n ≥ 2. The technique used to solve this problem in the previous section is called repeated substitution. T (n) = T (n − 2i) + 3in − 3i2 + 7i = T (2) + 3(n/2 − 1)n − 3(n/2 − 1)2 + 7(n/2 − 1) = 6 + 3n2 /2 − 3n − 3n2 /4 + 3n − 3 + 7n/2 − 7 = 3n2 /4 + 7n/2 − 4. Or. Then. Now. Repeatedly substitute until you see a pattern developing. 4. Suppose that T (1) = 1. T (n) = (3n2 + 14n − 16)/4. Recurrence Relations
= = =
T (n − 2i) + 3in − 6
j=0
j + 4i (by Problem 1)
T (n − 2i) + 3in − 3i(i − 1) + 4i T (n − 2i) + 3in − 3i2 + 7i. T (n) = (3n2 + 14n − 13)/4. Thus. suppose that n is even.
. T (n) = 2T (n − 1). 235. more succinctly. To be completely formal.
Sec. is T (n) = 2 · 3n−1 − 1. For more information on the constant multiple in the bigO bound. Knuth. but you can skip it is you are short of time. Then. Now suppose that n > 1 and T (n − 1) = 2 · 3n−2 − 1. (It is not normally necessary to hand in this extra work when you are asked to solve the recurrence on a homework or an exam. it is prudent to check your answer by using induction. The claim is certainly true for n = 1. Hence. but it’s a good idea to do it for your own peace of mind.62n ). 222. The proof is by induction on n. (You should do this if you are asked to prove that your answer is correct or if you want to be sure that the pattern is right. such as a summation or two.) Let’s do it for this problem. Once you have used repeated substitution to get an answer. T (n) = = = as required. 4. We are required to prove that the solution to the recurrence T (1) = 1. and Patashnik [30]. T (n) = 3T (n − 1) + 2. and for all n ≥ 2.
(by the induction hypothesis)
. Comments
45
verify this pattern by induction on i. see Graham. You are usually left with some algebra to do. Fn = O(1.8. 3T (n − 1) + 2 3(2 · 3n−2 − 1) + 2 2 · 3n−1 − 1.) Then choose i to be whatever value it takes to make all of the T (·)s in the pattern turn into the base case for the recurrence.
1
ITERATIVE ALGORITHMS
To prove correctness of an iterative algorithm. Start by listing the new values of the variables in terms of the old values. Here are some simple algorithms to practice on. The value v0 means the contents of v immediately before the loop is entered. Use this for your inductive step. • Verify the algorithm one loop at a time. 46
. Purely iterative algorithms can be proved correct by devising a loop invariant for every loop.
5. you need to do the following: • Write down a speciﬁcation for the output to be produced by the algorithm as a function of its inputs (this will be easy for the simple algorithms considered in this chapter). • Use the loop invariants to prove that the algorithm terminates. and using them to establish that the algorithm terminates with the correct result. devise a loop invariant that remains true each time through the loop and captures the “progress” made by the loop. then vj denotes the value stored in v immediately after the jth iteration of the loop. This is usually done by induction on the number of iterations. for j ≥ 0. • For each loop. starting at the inner loop in case of nested loops.3. Purely recursive algorithms can be proved correct directly by induction.Chapter 5
Correctness Proofs
How do we know that a given algorithm works? The best method is to prove it correct.
The loop invariants discussed in this section use the following convention: If v is a variable used in a loop. • Use the loop invariants and the termination conditions to prove that the algorithm computes the correct result (according to the speciﬁcation). proving them correct by induction. • Prove that the loop invariants hold. The algorithms use a simple pseudocode as described in Section 1.
5.
StraightLine Programs
Prove that the following algorithm for swapping two numbers is correct. d := 1. y := y/2 . 251.
Prove that the following algorithm for incrementing a natural number is correct. d := 2d. d := 2d. d := 1. 7. 2. 6.1. c := 1.
.2
250. 252. if a ⊕ b ⊕ c then x := x + d.1. while (y > 0) ∨ (c > 0) do a := y mod 2. where y. 7. function increment(y) comment Return y + 1. c := a ∧ c. c := (a ∧ b) ∨ (b ∧ c) ∨ (a ∧ c). y := y/2 . y) comment Swap x and y. 3. 6. while (y > 0) ∨ (z > 0) ∨ (c > 0) do a := y mod 2.
Arithmetic
Prove that the following algorithm for the addition of natural numbers is correct. z) comment Return y + z.Sec. 3. b := z mod 2. 2. 4. 5. 3. return(x)
1. where y ∈ IN x := 0.
5.
Prove that the following algorithm for the multiplication of natural numbers is correct. 4. x := x + y y := x − y x := x − y
1. procedure swap(x. 5. c := 0.1
249. if a ⊕ c then x := x + d. z := z/2 . function add(y. 2. z ∈ IN x := 0.1. return(x)
1. Iterative Algorithms
47
5.
w := z. 4. 255. z ∈ IN x := 0.
. 4. 5. where y. if w ≤ r then r := r − w. r ∈ IN such that y = qz + r and r < z. 4. 3. Correctness Proofs
1. function divide(y. return(x)
Prove that the following algorithm for the multiplication of natural numbers is correct.
function multiply(y. z) comment Return yz. 254. 2. 2. where y. function multiply(y. 3. return(x)
1. 5. where y. y := c · y. 5. 3. 6. q := q + 1 return(q. z ∈ IN r := y. q := 0. z) comment Return yz. where y. z) comment Return q. z) comment Return yz. z := z/2 . 5. 7. y := 2y. while w > z do q := 2q. 4.48
Chap. return(x)
1. while z > 0 do x := x + y · (z mod 2).
Prove that the following algorithm for the multiplication of natural numbers is correct. z ∈ IN x := 0. z := z/c . 2. w := w/2 . 2. 3. for all integer constants c ≥ 2. while z > 0 do x := x + y · (z mod c).
Prove that the following algorithm for division and remaindering of natural numbers is correct. while z > 0 do if z mod 2 = 1 then x := x + y. r)
1. while w ≤ y do w := 2w. z := z/2 . z ∈ IN x := 0. y := 2y. 5. 253. function multiply(y.
Sec. 4. 258. while z > 0 do if z is odd then x := x · y z := z/2 y := y 2 return(x)
1. function power(y.
Arrays
Prove that the following algorithm that adds the values in an array A[1.
. 4.1.3
259. z) comment Return y z .
Prove that the following algorithm for exponentiation is correct. 5. 2. 6. z) comment Return y z . 4.. where y ∈ IN x := 1 while y > 1 do x := x · y. 2. while z > 0 do x := x · y z := z − 1 return(x)
1. z ∈ IN x := 1. 2. function factorial(y) comment Return y!. function sum(A) comment Return s := 0. 3. 257. 3. 3. Iterative Algorithms
49
256. where y ∈ IR. y := y − 1 return(x)
1. 5. for i := 1 to n do s := s + A[i] return(s)
n i=1
A[i]
1. z ∈ IN x := 1. 3. 4.
Prove that the following algorithm for exponentiation is correct. 2.1. function power(y.n] is correct.
5.
Prove that the following algorithm for computing factorials is correct. 5. where y ∈ IR.
50 260.
Chap. 5. Correctness Proofs
Prove that the following algorithm for computing the maximum value in an array A[1..n] is correct.
1. 2. 3. 4.
function max(A) comment Return max A[1], . . . , A[n] m := A[1] for i := 2 to n do if A[i] > m then m := A[i] return(m)
261.
Prove the correctness of the following bubblesort algorithm. The values to be sorted are in an array A[1..n].
1. 2. 3. 4.
procedure bubblesort(A[1..n]) comment Sort A[1], A[2], . . . , A[n] into nondecreasing order for i := 1 to n − 1 do for j := 1 to n − i do if A[j] > A[j + 1] then Swap A[j] with A[j + 1]
262.
Prove the correctness of the following patternmatching algorithm. The input consists of a string S[1..n], and a pattern P [0..m − 1], where 1 ≤ m ≤ n. The algorithm locates the ﬁrst contiguous occurrence of the pattern P in the string S, that is, = p if S[p..p + m − 1] = P , and = n − m + 1 if the pattern P does not occur at all in the string S. function match(P, S, n, m) comment Find the pattern P [0..m − 1] in string S[1..n] := 0; matched := false while ( ≤ n − m) ∧ ¬matched do := + 1 ; r := 0; matched := true while (r < m) ∧ matched do matched := matched ∧ (P [r] = S[ + r]) r := r + 1 return( )
1. 2. 3. 4. 5. 6. 7. 8.
263.
Prove that the following matrix multiplication algorithm is correct.
Sec. 5.2. Recursive Algorithms
51
1. 2. 3. 4. 5. 6. 264.
procedure matmultiply(Y, Z, n); comment multiplies n × n matrices Y Z for i := 1 to n do for j := 1 to n do X[i, j] := 0; for k := 1 to n do X[i, j] := X[i, j] + Y [i, k] · Z[k, j]; return(X)
Prove that the following algorithm for evaluating the polynomial an xn + an−1 xn−1 + · · · + a1 x + a0 is correct, where the coeﬃcients are stored in an array A[0..n], with A[i] = ai for all 0 ≤ i ≤ n. The algorithm is named after its inventor, William G. Horner, and is often called Horner’s rule. function Horner(A, n) comment Return n A[i] · xi i=0 v := 0 for i := n downto 0 do v := A[i] + v · x return(v)
1. 2. 3. 4.
5.1.4
265.
Fibonacci Numbers
Prove that the following algorithm for computing Fibonacci numbers is correct. function ﬁb(n) comment Return Fn , the nth Fibonacci number if n = 0 then return(0) else last:=0; current:=1 for i := 2 to n do temp:=last+current; last:=current; current:=temp return(current)
1. 2. 3. 4. 5. 6.
5.2
RECURSIVE ALGORITHMS
To prove correctness of a recursive algorithm, you need to do the following: • You will prove correctness by induction on the “size” of the problem being solved (for example, the size of array chunk, number of bits in an integer, etc.). Your ﬁrst task is to identify what is to be used as the “size.”
52
Chap. 5. Correctness Proofs
• Then, you prove the base of the induction, which will usually involve only the base of the recursion. • Next, you need to prove that recursive calls are given subproblems to solve (that is, there is no inﬁnite recursion). This is often trivial, but it can be diﬃcult to prove and so should not be overlooked. • Finally, you prove the inductive step: Assume that the recursive calls work correctly, and use this assumption to prove that the current call works correctly.
5.2.1
266.
Arithmetic
Prove that the following recursive algorithm computes 3n − 2n for all n ≥ 0. 1. 2. function g(n) if n ≤ 1 then return(n) else return(5 · g(n − 1) − 6 · g(n − 2))
267.
Prove that the following recursive algorithm for incrementing a natural number is correct. function increment(y) comment Return y + 1. if y = 0 then return(1) else if y mod 2 = 1 then return(2 · increment( y/2 )) else return(y + 1)
1. 2. 3. 4. 268.
Prove that the following recursive algorithm for the addition of natural numbers is correct. function add(y, z, c) comment Return the sum y + z + c, where c ∈ {0, 1}. if y = z = 0 then return(c) else a := y mod 2; b := z mod 2; return(2 · add( y/2 , z/2 , (a + b + c)/2 ) + (a ⊕ b ⊕ c))
1. 2. 3. 269.
Prove that the following recursive algorithm for the multiplication of natural numbers is correct. function multiply(y, z) comment Return the product yz. if z = 0 then return(0) else if z is odd then return(multiply(2y, z/2 )+y) else return(multiply(2y, z/2 ))
1. 2. 3.
Sec. 5.2. Recursive Algorithms
53
270.
Prove that the following recursive algorithm for the multiplication of natural numbers is correct. function multiply(y, z) comment Return the product yz. if z = 0 then return(0) else return(multiply(2y, z/2 ) + y · (z mod 2))
1. 2. 271.
Prove that the following recursive algorithm for the multiplication of natural numbers is correct, for all integers constants c ≥ 2. function multiply(y, z) comment Return the product yz. if z = 0 then return(0) else return(multiply(cy, z/c ) + y · (z mod c))
1. 2. 272.
Prove that the following recursive algorithm for exponentiation is correct. function power(y, z) comment Return y z , where y ∈ IR, z ∈ IN. if z = 0 then return(1) if z is odd then return(power(y 2 , z/2 ) · y) else return(power(y 2 , z/2 ))
1. 2. 3. 273.
Prove that the following recursive algorithm for computing factorials is correct. function factorial(n) comment Return n!. if n ≤ 1 then return(1) else return(n · factorial(n − 1))
1. 2.
5.2.2
274.
Arrays
Prove that the following recursive algorithm for computing the maximum value in an array A[1..n] is correct. function maximum(n) comment Return max of A[1..n]. if n ≤ 1 then return(A[1]) else return(max(maximum(n − 1),A[n])) function max(x, y) comment Return largest of x and y. if x ≥ y then return(x) else return(y)
1. 2.
3.
54 275.
Chap. 5. Correctness Proofs
Prove that the following recursive algorithm that adds the values in an array A[1..n] is correct. function sum(n) comment Return sum of A[1..n]. if n ≤ 1 then return(A[1]) else return(sum(n − 1) + A[n])
1. 2.
5.2.3
276.
Fibonacci Numbers
Prove that the following algorithm for computing Fibonacci numbers is correct. function ﬁb(n) comment Return Fn , the nth Fibonacci number. if n ≤ 1 then return(n) else return(ﬁb(n − 1)+ﬁb(n − 2))
1. 2. 277.
Prove that the following algorithm for computing Fibonacci numbers is correct. function ﬁb(n) comment Return (Fn−1 , Fn ) if n is odd then (a, b) := even(n − 1) return(b, a + b) else return(even(n)) function even(n) comment Return (Fn−1 , Fn ) when n is even if n = 0 then return(1, 0) else if n = 2 then return(1, 1) else if n = 4 then return(2, 3) (a, b) := ﬁb(n/2 − 1) c := a + b; d := b + c return(b · d + a · c, c · (d + b))
1. 2. 3. 4.
1. 2. 3. 4. 5. 6.
5.3
COMBINED ITERATION AND RECURSION
The following questions ask you to prove correct some recursive algorithms that have loops in them. Naturally enough, you can solve them by applying both of the techniques you have used in this chapter.
Sec. 5.4. Hints
55
278.
Prove that the following recursive bubblesort algorithm is correct. The values to be sorted are in an array A[1..n]. procedure bubblesort(n) comment Sort A[1..n]. if n > 1 then for j := 1 to n − 1 do if A[j] > A[j + 1] then Swap A[j] with A[j + 1] bubblesort(n − 1)
1. 2. 3. 4. 5. 279.
Prove that the following variant of quicksort is correct. The values to be sorted are in an array A[1..n]. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. procedure quicksort( , r) comment sort S[ ..r] i := ; j := r a := some element from S[ ..r] repeat while S[i] < a do i := i + 1 while S[j] > a do j := j − 1 if i ≤ j then swap S[i] and S[j] i := i + 1; j := j − 1 until i > j if < j then quicksort( , j) if i < r then quicksort(i, r)
5.4
HINTS
In each of the following hints, if the algorithm uses a variable v, then vi denotes the contents of v immediately after the ith iteration of the single loop in the algorithm (v0 denotes the contents of v immediately before the loop is entered for the ﬁrst time). 250. The loop invariant is the following: (yj + zj + cj )dj + xj = y0 + z0 . 252. The loop invariant is the following: yj zj + xj = y0 z0 . 255. The loop invariant is the following: qj wj + rj = y0 and rj < wj .
z 256. The loop invariant is the following: xj yj zj = y00 .
56
z 257. The loop invariant is the following: xj yj zj = y00 .
Chap. 5. Correctness Proofs
258. The loop invariant is the following: mj = 259. The loop invariant is the following: sj =
n k=n−j+1 k. j i=1
A[i].
260. The loop invariant is the following: mj is the maximum of A[1], . . . , A[j + 1]. 261. The loop invariant for the inner loop is the following: after the jth iteration, for all 1 ≤ i < j, A[i] ≤ A[j]. The loop invariant for the outer loop is the following: after the jth iteration, for all n − j + 1 ≤ i ≤ n, for all k < i, A[k] ≤ A[i]. 277. The identity from Problem 53 might help.
5.5
SOLUTIONS
250. This correctness proof will make use of the fact that for all n ∈ IN, 2 n/2 + (n mod 2) = n. (5.1)
(Can you prove this?) We claim that if y, z ∈ IN, then add(y, z) returns the value y + z. That is, when line 8 is executed, x = y + z. For each of the identiﬁers, use a subscript i to denote the value of the identiﬁer after the ith iteration of the whileloop on lines 3–7, for i ≥ 0, with i = 0 meaning the time immediately before the while loop is entered and immediately after the statement on line 2 is executed. By inspection, aj+1 bj+1 yj+1 zj+1 dj+1 = yj mod 2 = zj mod 2 = yj /2 = zj /2 = 2dj .
From line 6 of the algorithm, cj+1 is 1 iﬀ at least two of aj+1 , bj+1 , and cj are 1 (see Problem 93). Therefore, cj+1 = (aj+1 + bj+1 + cj )/2 . Note that d is added into x in line 5 of the algorithm only when an odd number of a, b, and c are 1. Therefore, xj+1 = xj + dj ((aj+1 + bj+1 + cj ) mod 2). We will now prove the following loop invariant: For all natural numbers j ≥ 0, (yj + zj + cj )dj + xj = y0 + z0 .
Sec. 5.5. Solutions
57
The proof is by induction on j. The base j = 0 is trivial, since c0 = 0, d0 = 1, and x0 = 0. Now, suppose that (yj + zj + cj )dj + xj = y0 + z0 . We are required to prove that (yj+1 + zj+1 + cj+1 )dj+1 + xj+1 = y0 + z0 . By the preceding, (yj+1 + zj+1 + cj+1 )dj+1 + xj+1 = ( yj /2 + zj /2 + (yj mod 2 + zj mod 2 + cj )/2 )2dj + xj + dj ((yj mod 2 + zj mod 2 + cj ) mod 2) = ( yj /2 + zj /2 )2dj + xj + dj (yj mod 2 + zj mod 2 + cj ) (by Equation 5.1) = (yj + zj + cj )dj + xj (by Equation 5.1 twice). Therefore, by the induction hypothesis, (yj+1 + zj+1 + cj+1 )dj+1 + xj+1 = y0 + z0 . This ends the proof of the loop invariant. It remains to use it to prove that the algorithm is correct. We need to prove that the algorithm terminates with x containing the sum of y and z. First, we will prove that it terminates. By inspection, the values of y and z are halved (rounding down if they are odd) on every iteration of the loop. Therefore, they will eventually both have value zero, and stay that way. At the ﬁrst point at which y = z = 0, either c will equal zero or c will be assigned zero on the next iteration of the loop. Therefore, eventually y = z = c = 0, at which point the loop terminates. Now we will prove that x has the correct value on termination. Suppose the loop terminates after t iterations, for some t ≥ 0. By the loop invariant, (yt + zt + ct )dt + xt = y0 + z0 . Since yt = zt = ct = 0, we see that xt = y0 + z0 . Therefore, the algorithm terminates with x containing the sum of the initial values of y and z, as required. 269. The correctness proof is by induction on z. We claim that multiply(y, z) returns the product yz. The hypothesis is true for z = 0, since multiply(y, 0) returns 0. Now, suppose that for z ≥ 0, multiply(y, z) returns yz. We now must prove that multiply(y, z + 1) returns y(z + 1). There are two cases to be considered, depending on whether z + 1 is odd or even. By inspection, if
58 z + 1 is odd, then multiply(y, z + 1) returns
Chap. 5. Correctness Proofs
multiply(2y, (z + 1)/2 ) + y = 2y (z + 1)/2 + y (by the induction hypothesis) = 2y(z/2) + y (since z is even) = y(z + 1). By inspection, if z + 1 is even, then multiply(y, z + 1) returns multiply(2y, (z + 1)/2 ) 2y (z + 1)/2 (by the induction hypothesis) 2y(z + 1)/2 (since z is odd) y(z + 1).
= = =
5.6
COMMENTS
253. The hard part about this problem (and other similar problems in this section) is that you will need to ﬁnd your own loop invariant. 265. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is the obvious iterative algorithm. 276. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is the obvious recursive algorithm. 277. See Section 2.7 for the deﬁnition of Fibonacci numbers. This is a sneaky recursive algorithm.
such as additions. Another useful technique is to pick an elementary operation.
What is Returned?
What is the value returned by the following function? Express your answer as a function of n. or comparisons. Give.
6. the worstcase running time. Simply charge O(1) for code without loops (assuming that your pseudocode isn’t hiding something that takes longer). 59
. just ignore the “book keeping” code and look for the point in the algorithm where the real work is done.) Then.
6. This is often easier to deal with because it is an exact function of n and you don’t have the messy bigO symbols to carry through your analysis. recursive algorithms are more diﬃcult to analyze than nonrecursive ones. As we will see. a bigΘ bound would be even better.1. (To ﬁnd the elementary operation.) This can be done by getting a bigO ﬁgure for parts of the algorithm and then combining these ﬁgures using the sum and product rules for bigO (see Problems 173 and 176). multiplications.” (Of course. If you get tired of carrying around bigOs. use the “elementary operation” technique described earlier. and observe that the running time of the algorithm is bigO of the number of elementary operations. using bigO notation.Chapter 6
Algorithm Analysis
Algorithm analysis usually means “give a bigO ﬁgure for the running time of an algorithm. and use the sum and product rules for bigO (see Problems 173 and 176).1
ITERATIVE ALGORITHMS
The analysis of iterative algorithms is fairly easy. you can analyze the exact number of elementary operations as a function of n in the worst case.1
280.
5. 5. 6. for i := 1 to n − 1 do for j := i + 1 to n do for k := 1 to j do r := r + 1 return(r)
What is the value returned by the following function? Express your answer as a function of n. 3. 6. 3. Give. Give. function pesky(n) r := 0.
function mystery(n) r := 0. the worstcase running time. 2. 5. 6. function pestiferous(n) r := 0. using bigO notation. 4. Give. 2. the worstcase running time.
What is the value returned by the following function? Express your answer as a function of n. for i := 1 to n do for j := 1 to i do for k := j to i + j do r := r + 1 return(r)
282. 6. 1. 3. 4. 5. 1. 4. 3. Algorithm Analysis
1. the worstcase running time. for i := 1 to n do for j := 1 to i do for k := j to i + j do for := 1 to i + j − k do r := r + 1 return(r)
283.60
Chap. using bigO notation. for i := 1 to n do for j := i + 1 to n do for k := i + j − 1 to n do r := r + 1 return(r)
. 7. 2. function conundrum(n) r := 0. 1. 2. using bigO notation. 4. 281.
What is the value returned by the following function? Express your answer as a function of n. 6.
3
293. how many times is line 3 executed) in the worst case? Analyze the algorithm for exponentiation in Problem 257. How many comparisons does it use (that is.
Arrays
Analyze the array addition algorithm in Problem 259. 6.
290. How many additions does it use (that is.
6. Iterative Algorithms
61
6.1. How many additions does it use (that is. how many times is line 6 executed) in the worst case? Analyze the algorithm for exponentiation in Problem 256.
288.
291. How many multiplications does it use (that is.2
284.Sec.
292. Analyze the algorithm for division and remaindering of natural numbers in Problem 255. how many times is line 3 executed) in the worst case? Analyze the algorithm for computing factorials in Problem 258. how many times is line 3 executed) in the worst case? Analyze the array maximum algorithm in Problem 260. how many times is line 4 executed) in the worst case? Analyze the algorithm for incrementing a natural number in Problem 251.1. how many times is line 3 executed) in the worst case?
285. how many times is line 3 executed) in the worst case? Analyze the algorithm for the multiplication of natural numbers in Problem 254. how many times is line 4 executed) in the worst case? Analyze the algorithm for the multiplication of natural numbers in Problem 252. How many additions does it use (that is. How many multiplications does it use (that is.
Arithmetic
Analyze the algorithm for the addition of natural numbers in Problem 250. How many additions does it use (that is.
289. How many additions does it use (that is. how many times is line 3 executed) in the worst case?
294. how many times is line 3 executed) in the worst case? Analyze the algorithm for the multiplication of natural numbers in Problem 253.
287.
286.
. how many times is line 3 executed) in the worst case? You may assume that c ≥ 2 is an integer. How many additions does it use (that is. How many multiplications does it use (that is. How many subtractions does it use (that is.1.
6.
Chap. Suppose it is n0 . how many times is line 5 executed) in the worst case? Analyze the algorithm for evaluating a polynomial in Problem 264. giving a term “a · T (f (n))” in the recurrence relation. How many accesses of the array P does it use (that is. How many comparisons does it use (that is.2
RECURSIVE ALGORITHMS
The analysis of recursive algorithms is a little harder than that of nonrecursive ones. how many times is line 3 executed) in the worst case? Analyze the patternmatching algorithm in Problem 262.4
299. To derive a recurrence relation for the running time of an algorithm: • Decide what “n”. you have to derive a recurrence relation for the running time. • Figure out what T (n0 ) is. It will usually be a single value (for example.62 295.
Fibonacci Numbers
Analyze the algorithm for computing Fibonacci numbers in Problem 265. Algorithm Analysis
Analyze the bubblesort algorithm in Problem 261. • The general T (n) is usually a sum of various choices of T (m) (for the recursive calls). First. how many times is line 3 executed) in the worst case?
296. how many times is line 6 executed) in the worst case? Analyze the matrix multiplication algorithm in Problem 263.
298. How many multiplications does it use (that is.
297. n = 1). Usually. You can usually use “some constant c. How many additions does it use (that is. and then you have to solve it.
. • See what value of n is used as the base of the recursion. but may be multiple values.1. How many multiplications does it use (that is. The “elementary operation” technique described in the ﬁrst paragraph of this chapter can be used to good eﬀect here.” but sometimes a speciﬁc number will be needed. too. how many times is line 3 executed) in the worst case?
6. Techniques for the solution of recurrence relations are explained in Chapter 4. plus the sum of the other work done. the problem size. 6. is. the recursive calls will be solving a subproblems of the same size f (n).
How many additions does it use (that is.1
300. How many additions does it use (that is. How many additions does it use (that is. how many times is line 2 executed) in the worst case? Analyze the recursive algorithm for the multiplication of natural numbers in Problem 271.
303. how many times is line 2 executed) in the worst case?
301.2. How many additions does it use (that is.
.2. how many times is line 2 executed) in the worst case? Analyze the recursive algorithm that adds the values in an array in Problem 275. How many comparisons does it use (that is. 6. how many times is line 2 executed) in the worst case?
309.
6.
305. How many multiplications does it use (that is.
Arrays
Analyze the recursive algorithm for computing the maximum value in an array in Problem 274.2. how many times is line 3 executed) in the worst case? Analyze the recursive algorithm for the addition of natural numbers in Problem 268. Recursive Algorithms
63
6.2
308. How many rightshifts does it use (that is.
307.
302. how many times is line 2 executed) in the worst case? You may assume that c ≥ 2 is an integer. How many additions does it use (that is. How many times is line 3 executed in the worst case? Analyze the recursive algorithm for the multiplication of natural numbers in Problem 269. how many times is line 2 executed) in the worst case? Analyze the recursive algorithm for computing factorials in Problem 273.
304. how many times is line 2 executed) in the worst case? Analyze the recursive algorithm for the multiplication of natural numbers in Problem 270.Sec.
306. how many times is line 2 executed) in the worst case? Analyze the recursive algorithm for incrementing a natural number in Problem 267. Analyze the recursive algorithm for exponentiation in Problem 272.
Arithmetic
Analyze the recursive algorithm from Problem 266. How many multiplications does it use (that is.
Analyze the recursive version of bubblesort in Problem 278.4
HINTS
283 If you use the technique from the solution of Problem 280 blindly. you can do this by applying both of the techniques you have used in this chapter. (You didn’t think I would give you the whole answer. How many comparisons does it use (that is. How many comparisons does it use (that is.
Fibonacci Numbers
Analyze the algorithm for computing Fibonacci numbers in Problem 276.” count them exactly. 284.
6. Algorithm Analysis
6. and then put a bigO around the function you get as answer to the second part of the question (reducing it to the simplest form.3
COMBINED ITERATION AND RECURSION
The following questions ask you to analyze some recursive algorithms that have loops in them.
.64
Chap. How many additions does it use (that is. you can observe that additions are the “elementary operation. and something diﬀerent if n is odd. How many additions does it use (that is. You can do the running time ﬁrst.3
310. how many times is line 3 executed) in the worst case?
311. and an exact function of n (with no bigO at the front) for the number of additions. The correct answer is n(n + 2)(2n − 1)/24 if n is even. This is wrong. I am looking for a bigO analysis of the running time. 6. Naturally enough. 312. of course) to give the answer for the ﬁrst part of the question. you’ll get the answer n(n−1)/2. More subtlety is necessary.2. how many times is line 2 executed) in the worst case? Analyze the algorithm for computing Fibonacci numbers in Problem 277. how many times are lines 5 and 6 executed) in the worst case?
313. or if you are clever.
6. did you?) You can partially verify your solution in a few minutes by coding the algorithm as a program and printing n and r for a few values of n. how many times is line 3 executed) in the worst case? Analyze the variant of quicksort in Problem 279.
Therefore. n−1 equivalently. for j := i + 1 to n do r := r + j. This recurrence relation is a little tricky to solve. equivalently. function mystery returns n(n2 − 1)/3. r := r + i=1 (n(n + 1)/2 − i(i + 1)/2). The forloop that begins on line 3 is executed for O(n) iterations. function mystery runs in time O(n3 ). r := r + n(n + 1)/2 − i(i + 1)/2. the forloop on lines 3–5 has the following eﬀect: 3–5. but repeated substitution will do it: A(n) = = A(n − 1) + A(n − 2) + 1 (A(n − 2) + A(n − 3) + 1) + A(n − 2) + 1
. Solutions
65
300.5
SOLUTIONS
280. A sloppy analysis goes as follows. j. Now. This analysis is tight because the preceding paragraph showed that line 5 is executed n(n2 − 1)/3 times. The forloop that begins on line 4 is executed for O(n) iterations. the forloop on lines 2–5 has the following eﬀect: for i := 1 to n − 1 do 2–5. Lines 1 and 5 take O(1) time.7 for deﬁnitions). Speciﬁcally. Therefore. Fibonacci numbers are involved (see Section 2. you will need to apply the results from Problems 52 and 222.
6. Let A(n) be the number of additions used in function g(n). Therefore.
n−1
(n(n + 1)/2 − i(i + 1)/2)
i=1
=
(n − 1)n(n + 1) 1 − 2 2
n−1
i2 −
i=1
1 2
n−1
i
i=1
= (n − 1)n(n + 1)/2 − n(n − 1)(2n − 1)/12 − n(n − 1)/4 (by Problems 1 and 2) = n(n2 − 1)/3. A(n) = A(n − 1) + A(n − 2) + 1. Then A(0) = A(1) = 0 and for all n > 1. Now. r := r + n j=i+1
n n i
j=
j=i+1 j=1
j−
j=1
j = n(n + 1)/2 − i(i + 1)/2.5.Sec. The forloop on lines 4–5 has the same eﬀect as r := r + j. 6. The forloop that begins on line 2 is executed for O(n) iterations. 300.
Therefore.
The worstcase analysis of quicksort is often presented in class. could you have derived it for yourself?) The running time of procedure multiply is clearly bigO of the number of additions.7) developing here. Algorithm Analysis
2(A(n − 3) + A(n − 4) + 1) + A(n − 3) + 1 + 1 3A(n − 3) + 2A(n − 4) + 1 + 1 + 2 3(A(n − 4) + A(n − 5) + 1) + 2A(n − 4) + 1 + 1 + 2 5A(n − 4) + 3A(n − 5) + 1 + 1 + 2 + 3. Then A(0) = 0 and for all n ≥ 1. then it makes a good exercise.
n−1
A(n) = Fn A(1) + Fn−1 A(0) +
j=1 n−1
Fj
=
j=1
Fj (see Problem 52). Which algorithm would you use in practice.
. It can be veriﬁed by induction on i ≥ 1 that after i substitutions. 6.
There is a pattern involving Fibonacci numbers (see Section 2. and A(n) be the number of additions used in the multiplication of y by z. the one from Problem 299. see the comment to Problem 222). Which algorithm would you use in practice. but if not. the one in this problem. The solution to this recurrence is A(n) = n.66 = = = = = 2A(n − 2) + A(n − 3) + 1 + 1
Chap. Let n be the number of bits in z.
Hence. or the one from Problem 310? 313. (Can you verify this by induction on n? If I didn’t tell you the answer. A(n) = A(n − 1) + 1. and is hence O(Fn+1 ) = O(1. 303.
= Fn+1 − 1
The running time of procedure g(n) is clearly bigO of the number of additions.6
COMMENTS
291.
6. taking i = n − 1. and is hence O(n).62n ) (for the latter. the one in this problem or the one from Problem 290? 311.
i
A(n) = Fi+1 A(n − i) + Fi A(n − i − 1) +
j=1
Fj .
and keep doing this until the problems become so small that it is trivial to solve them. n).. y) comment return maximum in S[x. (x + y)/2 ) max2:=maximum( (x + y)/2 + 1.
316. 2. It is 67
. the problem of ﬁnding the largest and smallest values in an array of n values. 314.1
MAXIMUM AND MINIMUM
The following is a divideandconquer algorithm for ﬁnding the maximum value in an array S[1.max2))
1. n). 315. 3.
Most textbooks introduce divideandconquer using the MAXMIN problem.. and combine their solutions into a solution for P . Solve this recurrence relation. n)? Explain your answer.n]. Write down a recurrence relation for the worstcase number of comparisons used by maximum(1. S[y])) else max1:=maximum(x. 5.
7. What is the running time of maximum(1.y] if y − x ≤ 1 then return(max(S[x]. y) return(max(max1.Chapter 7
DivideandConquer
Divideandconquer is perhaps the most commonly used algorithm design technique in computer science. divide it into smaller problems. You may assume that n is a power of 2. The main body of the algorithm consists of a call to maximum(1. function maximum(x. solve these subproblems. But how do you solve the smaller problems? Simply divide each of the small problems into smaller problems. 4.
Prove that the algorithm is correct. You may assume that n is a power of 2. a recursive procedure is usually the easiest way of implementing divideandconquer. Faced with a big problem P . Sound like recursion? Not surprisingly.
The following questions will test your understanding of this algorithm and its analysis. min:=S[n] else max:=−∞. 6. 320..
322. in time O(nmlog 3−1 ).
321. 2. 7. 4. How many comparisons does it use? Is it likely to be faster or slower than the divideandconquer algorithm in practice? procedure maxmin2(S) comment computes maximum and minimum of S[1.n] in max and min resp. you can multiply two nbit integers in time O(n1. by using the divideandconquer multiplication and stopping the recursion early. DivideandConquer
usually described for n a power of 2.2925 ).
. Consider the following MAXMIN algorithm. Devise an algorithm to multiply an nbit integer by an mbit integer.
7. 319. 7. and √ hence that you can multiply two nbit integers in O(1) time. and you are told that you have a computer with word size n. if n is odd then max:=S[n]. large:=S[2i] else small:=S[2i].
Show that any comparisonbased MAXMIN algorithm must use at least 3n/2 − 2 comparisons in the worst case. 8. Design and analyze a divideandconquer MAXMIN algorithm that uses 3n/2 − 2 comparisons for any n ∈ IN. where n ≥ m. large:=S[2i − 1] if small < min then min:=small if large > max then min:=small
1. 3. min:=∞ for i := 1 to n/2 do if S[2i − 1] ≤ S[2i] then small:=S[2i − 1].2
INTEGER MULTIPLICATION
Another popular example of divideandconquer is the O(n1. 318. for all n ∈ IN. The following problems ask you to extend this to arbitrary n.59 ) time integer multiplication algorithm. 317. Show that. 5.68
Chap. What is the depth of recursion of the divideandconquer integer multiplication algorithm? At what depth does it start to repeat the same subproblems? Suppose you are to write an algorithm for multiplying two nbit √ integers.
{result:=result+summand} var index. the naive algorithm takes time O(n2 ) and the divideandconquer algorithm takes time O(n1. a. (This is slower than the standard algorithm.Sec. each with n/3 bits. yz = ad24n/3 + (ae + bd)2n + (af + cd + be)22n/3 + (bf + ce)2n/3 + cf. here is his addition procedure: procedure LongAdd(var result. Integer Multiplication
69
Complete the design of the following algorithm for performing integer multiplication in time O(n1. For example. Then. where n is a power of 3.585 ). Break z into three parts. He started by writing procedures to do addition.. and so on. type Bit=0. f . d. 324.2.
. Student is confused.. Break y into three parts. It must be faster for large enough n since.63 ).Precision] of Bit. Here is the Pascal data structure that he used: const Precision=5000. but its veriﬁcation and analysis will test your abilities. as claimed.
Joe D. b. longinteger=packed array[1. e. as he saw in class.
Show that x = yz. leftshifting. each with n/3 bits.63 ) bitoperations. c. He was told to implement the divideandconquer algorithm for integer multiplication.) We use a technique similar to the standard divideandconquer algorithm. we divide them into three parts. Instead of dividing the inputs y and z into two parts.summand:longinteger).1. Suppose y and z have n bits. Show that this algorithm runs in O(n1. 7. and ﬁnd the value of n for which the divideandconquer algorithm is faster than the naive algorithm (see Problem 252) on nbit integers. This can be computed as follows: r1 := r2 := r3 := r4 := r5 := r6 := x := ad (a + b)(d + e) be (a + c)(d + f ) cf (b + c)(e + f ) r1 24n/3 + (r2 − r1 − r3 )2n + (r3 + r4 − r1 − r5 )22n/3 + (r6 − r3 − r5 )2n/3 + r5 .
323.digitsum:integer.
That is. result[index]:=digitsum mod 2. and Figure 7. Otherwise. then simply look up the result in the table. even when he tried multiplying 2n − 1 by itself (which is guaranteed to make LongMultiply slow). Both of these procedures worked perfectly.
. he found that LongMultiply was faster than FastLongMultiply. carry:=digitsum div 2. What did Joe do wrong? What can Joe do to ﬁx his program so that FastLongMultiply is faster than LongMultiply for large enough n?
Complete the design of the following algorithm for performing integer multiplication using O(n1.585 /(log n)0. do the following. begin{LongAdd} carry:=0. To multiply two nbit numbers. FastLongMultiply becomes slower than LongMultiply by more than a constant factor. as claimed. Figure 7. Show that the worst case for the naive integer multiplication algorithm is. end.70
Chap.585 ) bitoperations. Joe’s data show that as n increases. Devise and solve a recurrence relation for the divideandconquer multiplication algorithm when the base of recursion is T (n) = O(1) for all n ≤ k. and the divideandconquer multiplication algorithm as procedure FastLongMultiply.{for} if carry>0 then LongError(LongOverflow) end. Show that a table of all kbit products can be constructed using O(k4k ) bitoperations. DivideandConquer
carry:Bit. 328. 329.{LongAdd} Joe carefully implemented the naive multiplication algorithm from Problem 252 as procedure LongMultiply. 7. if n ≤ k. Perform the divideandconquer algorithm with the base of recursion n ≤ k. 327. First. construct a table of all kbit products. cut the numbers in half and perform the recursion as normal. 325.585 ) bitoperations. 326. multiplying 2n − 1 by itself. and that this gives an algorithm that uses O(n1. However.585 /(log n)0. for index:=1 to Precision do begin{for} digitsum:=result[index]+summand[index]+carry. Show that the best value of k is Θ(log n).
330. as claimed above.2 their ratio.1 shows his running times. Your solution must be a function of both n and k.
80 1.60 1.50 1. Running times for procedures LongMultiply and FastLongMultiply observed by Joe D.20 1.10 1.90 1.30 1.50 0.Sec.50
Figure 7.00 1.50 2.10 0. Integer Multiplication
71
Time (ms) x 106 2.
.40 1.1.40 0.00 0.80 0.60 0.30 0.2.00 2.90 0.00 1.70 0. 7.00
Comparison of Multiplication Algorithms
FastLongMultiply LongMultiply
n x 103 0.70 1. Student.00 0.20 0.50 1.
7. DivideandConquer
Ratio of Running Times
60.00 20.50 2.00 1.00 15. Student.00 50.00 40.00 5.00 45.00 0.50 1.2.00 2.00 25.50 n x 103 Ratio
Figure 7.00 10.00 30.
. The ratio of the running times for procedures LongMultiply and FastLongMultiply observed by Joe D.00 35.72
Chap.00 55.00 0.00 0.
3. 331.81 ).81 ) when n is not necessarily a power of 2.376 ). To compute A C B D = E G F H · I K J L . The standard description of Strassen’s algorithm assumes that n is a power of 2. = = = = m2 + m3 t1 + m 5 + m 6 t2 − m 7 t2 + m 5 .
. How small would m have to be for this algorithm to be faster than Strassen’s algorithm for large enough n? The current record for matrix multiplication is O(n2.3
STRASSEN’S ALGORITHM
Strassen’s algorithm is a divideandconquer algorithm for multiplying n×n matrices in time O(n2. =G+H = s1 − E =E−G = F − s2 =J −I = L − s5 =L−J = s6 − K. and that this can be used as the base of recursion in a divideandconquer algorithm.
333. 7. A B C D 334.
ﬁrst compute the following values: s1 s2 s3 s4 s5 s6 s7 s8 Then. Strassen’s Algorithm
73
7. The key ideas behind the algorithm are that 2 × 2 matrices can be multiplied using only seven multiplications instead of the usual eight (see Problems 263 and 297). How small would m need to be in Problem 332 in order to improve on this bound?
332.
Another version of Strassen’s algorithm uses the following identities. due to Coppersmith and Winograd [18]. Suppose we were to come up with a variant of Strassen’s algorithm based on the fact that 3 × 3 matrices can be multiplied in only m multiplications instead of the normal 27. Devise an algorithm that runs in time O(n2.Sec. m1 m2 m3 m4 m5 m6 m7 = s2 s6 = EI = FK = s3 s7 = s1 s5 = s4 L = Hs8 t1 = m 1 + m 2 t2 = t 1 + m 4
Prove that this algorithm is correct.
m. and leave the extension to general n as an exercise (Problem 336). If not.74 335. Do an exact analysis for the average number of comparisons used by binary search for successful searches. prove it. function search(A. 7. m)) else return(search(A. x. What is wrong with the following binary search algorithm? function search(A. give an example on which it fails.. Is it likely to be faster or slower than the standard version of Strassen’s algorithm in practice?
7. I usually teach it for n a power of 2. r) comment ﬁnd x in A[ . Use the binary search technique to devise an algorithm for the problem of ﬁnding √ squareroots of natural numbers: Given an nbit natural number n using only O(n) additions and shifts. r) comment ﬁnd x in A[ . N . . assuming that all elements in the sequence are accessed with equal probability. x. Is the following algorithm for binary search correct? If so.
Chap.r] if = r then return( ) else m := ( + r)/2 if x ≤ A[m] then return(search(A.. m)) else return(search(A. x. . . but deserves to be dusted oﬀ again for a brief inspection during the algorithms course.r] if = r then return( ) else m := + (r − + 1)/2 if x ≤ A[m] then return(search(A. compute
340. 336. x. m + 1. x. x. Show that the number of comparisons used by the binary search algorithm when n is not necessarily a power of 2 is at most log n . .
. DivideandConquer
Analyze its running time. r)) 338. 337. r)) 339.4
BINARY SEARCH
Binary search is a classic divideandconquer algorithm that is usually covered in more elementary courses.
n] be a sorted array of distinct integers. and analyze the running time of your algorithm.
343. 2. devise and solve a recurrence relation for the number of comparisons used. For each of them.S2 . How many comparisons would be made then in the worst case? Suppose quicksort were to always pivot on the n/3 th smallest value.Sec. 3. 5. Assume
. but into three sets of sizes approximately onethird. 4. and at least nα list elements smaller than it. Suppose S is a set of numbers. Let T [1..
7. function quicksort(S) if S ≤ 1 then return(S) else Choose an element a from S Let S1 . 7. 341. S3 be the elements of S that are respectively <. Modify the binary search algorithm to give a ternary search algorithm that splits the input not into two sets of almostequal sizes.
342.quicksort(S3 ))
The operation described in line 5 is known as pivoting on a. The median of a set of n values is the n/2 th smallest value. Quicksort
75
The following problems ask you to modify binary search in some way. Give a divideandconquer algorithm that ﬁnds an index i such that T [i] = i (if one exists) and runs in time O(log n).5. Suppose quicksort were to always pivot on the median value. Modify the binary search algorithm so that it splits the input not into two sets of almostequal sizes. but into two sets of sizes approximately onethird and twothirds. 1. The following is a divideandconquer algorithm for computing a pseudomedian (that is. =.5
QUICKSORT
The following is a highlevel version of Hoare’s quicksort algorithm. S2 . write pseudocode for the algorithm. 6. 344. > a return(quicksort(S1 ).
An αpseudomedian of a list of n distinct values (where 0 < α < 1) is a value that has at least nα list elements larger than it. How many comparisons would be made then in the worst case?
345. an αpseudomedian for some value of α to be determined later).
r) comment sort S[ .r] i := . if < j then quicksort( . j). which gives the minimum number of comparisons S(n) needed to sort n ≤ 11 numbers. 10. Is there any odd number less than 13 which is better? You may use the following table. until i > j. 12. If n = 3. 6. and hence prove that the algorithm does return a pseudomedian. 346. j := r. Now recursively apply the procedure to ﬁnd a pseudomedian of these values. procedure quicksort( .. 1. if i < r then quicksort(i.. r).r]. Let T (n) be the number of comparisons used by the preceding algorithm for computing a pseudomedian. Write a recurrence relation for T (n). Does it improve the worstcase number of comparisons made?
The following is an implementation of Quicksort on an array S[1. divide the n items into n/3 groups of 3 values. 13. repeat while S[i] < a do i := i + 1. DivideandConquer
n is a power of 3. show that the algorithm runs in time O(n). 1 0 2 1 3 3 4 5 5 7 6 10 7 13 8 16 9 19 10 22 11 26
347.
How many comparisons does it use to sort n values in the worst case?
. Let E(n) be the number of values that are smaller than the value found by the preceding algorithm. 7. 2.76
Chap. What is the value of α? Any odd number can be used instead of 3. 4. a := some element from S[ . and pick out the n/3 medians. j := j − 1.
348. Write a recurrence relation for E(n). n S(n) 349. i := i + 1.. 7. if i ≤ j then swap S[i] and S[j]. and solve it exactly. 11.
Suppose the pseudomedian algorithm is used to ﬁnd the pivot value in quicksort. Otherwise. Sort each group of 3. You were asked to prove it correct in Problem 279 and analyze it in Problem 313. Hence. 350.n] due to Dijkstra [21]. 8. 9. 3. then simply sort the 3 values and return the median. 5. while S[j] > a do j := j − 1.
Alternate between moving the smallest disk in this manner and the only other legal move available. and Patashnik [30]. 354.Sec. with clockwise moves being from peg 1 to peg 2 to peg 3 to peg 1. Suppose the above algorithm pivots on the middle value. Devise a divideandconquer algorithm for the Towers of Hanoi problem when moves between peg 1 and peg 3 are not allowed (that is.
Each of the following problems asks you to devise a divideandconquer algorithm for a variant of the Towers of Hanoi problem. If n is odd. all moves must be either to or from peg 2).
7.
356.6
TOWERS OF HANOI
The Towers of Hanoi problem is often used as an example of recursion in introductory programming courses. Think of the pegs as being arranged in a circle. You are allowed to move a single disk at a time from one peg to another.6. Give an input of n values for which quicksort exhibits its worstcase behavior. Towers of Hanoi
77
351. Knuth. then start by moving the smallest disk one jump in the clockwise direction. Prove that any algorithm that solves the Towers of Hanoi problem must make at least 2n − 1 moves. Alternate between moving the smallest disk in this manner and the only other legal move available. usually to good eﬀect.
353. The earliest reference to ´ this problem known to the author is Edouard Lucas [55] in Graham. 355. prove that it is correct. line 4 is replaced by a := S[ ( + r)/2 ]. 357.
. 7. that is. This problem can be resurrected later in an algorithms course. but you must not place a larger disk on top of a smaller one. and analyze the number of moves made. Give an input of 8 values for which quicksort exhibits its worstcase behavior. then start by moving the smallest disk one jump in the counterclockwise direction. How many moves does the divideandconquer algorithm use? Show that the following algorithm solves the Towers of Hanoi problem. line 4 is replaced by a := S[ ( + r)/2 ]. 352. arranged in increasing order of diameter from top to bottom on the leftmost of three pegs. Your task is to move all disks to the rightmost peg via a sequence of these moves. If n is even. You are given n disks of diﬀering diameter.
How many comparisons does it use to sort n values on average ? Suppose the preceding algorithm pivots on the middle value. Write pseudocode for each algorithm. that is.
Devise an eﬃcient divideandconquer algorithm for the Towers of Hanoi problem when all moves must be in a clockwise direction (that is. and we add the extra rule that no disk may be placed on any other disk of the same color. E) by executing the following procedure on some v ∈ V : 1. Depthﬁrst search is performed on a directed graph G = (V. 5. w) ∈ E do if w is unused then mark (v.78 358. using at most O(n2 2n ) moves. 4. Devise an eﬃcient divideandconquer algorithm for the Towers of Hanoi problem when the disks are colored alternately red and blue.
362. DivideandConquer
Devise an eﬃcient divideandconquer algorithm for the Towers of Hanoi problem when there are 2n disks of n diﬀerent sizes. How many moves does your algorithm make? Devise a divideandconquer algorithm for√the Towers of Hanoi problem when there are four pegs.
Chap.
360. 2.
7. ﬁnding connected components of a directed graph and biconnected components of an undirected graph are the ones most commonly covered in algorithms courses. from peg 1 to peg 2.) Devise an eﬃcient divideandconquer algorithm for the Towers of Hanoi problem that works when the disks are started in any legal position (and still moves all disks to peg 3). Among its many applications.
. 3. where w = k − 2 is the number of workpegs. w) used dfs(w)
The set of marked edges form a depthﬁrst spanning tree of a connected component of G. two of each size. using at most O( n2 · 2w n/ 2 ) moves. 7. Devise a divideandconquer algorithm for the Towers of Hanoi √ √ √ w w problem when there are k ≥ 4 pegs. and you are not allowed to put a larger disk on top of a smaller one (but you can put a disk on top of a samesize one).
363. procedure dfs(v) mark v used for each w ∈ V such that (v.7
DEPTHFIRST SEARCH
Depthﬁrst search is a search technique for directed graphs. from peg 2 to peg 3. or from peg 3 to peg 1.
359.
361.
given a graph G with n nodes.
364.3.Sec.
Suppose an arithmetic expression is given as a DAG (directed acyclic graph) with common subexpressions removed. (v.8. then your algorithm should run in time O(n + e). prove it correct. then your algorithm must run in time O(n + e). An undirected graph G = (V.
366. If G has n vertices and e edges.
365. v. A forest is a graph composed of zero or more disconnected trees. determines for each v ∈ V the shortest path from s to v. If G has n vertices and e edges. and (u. Design an algorithm that runs in time O(n + e) to color a graph with two colors or determine that the graph is not 2colorable. Design an algorithm to test whether an undirected graph has a triangle. the expression 2 + 3 ∗ 4 + 5/(3 ∗ 4) would be given as the DAG shown in Figure 7. Design an algorithm that. E) is said to be kcolorable if all the vertices of G can be colored using k diﬀerent colors such that no two adjacent vertices have the same color. w) ∈ E. A triangle in an undirected graph G = (V.
7. v) ∈ E. describe it in prose or pseudocode.
368. 7. w ∈ V such that (u. Design an algorithm that. Devise an algorithm for evaluating such a DAG with outdegree 2 (that is.3.
367. For each of your algorithms. E) is a set of three pairwise distinct vertices u.
. determines whether G is a forest in time O(n). given a directed graph G = (V. A DAG representing the arithmetic expression 2 + 3 ∗ 4 + 5/(3 ∗ 4). E) and a distinguished vertex s ∈ V .8
APPLICATIONS
The following problems ask you to solve new problems using divideandconquer. Applications
79
+ + 2 3 * 4 / 5
Figure 7. and analyze its running time. all vertices have at most 2 outgoing edges) in time O(n). w) ∈ E. For example.
15 + 0. . DivideandConquer
Devise a divideandconquer algorithm for constructing a closed knight’s tour on an n × n chessboard for all even n ≥ 6 (see Problem 48 for deﬁnitions). The weighted i=1 median of the set {(ki .15.
kl <4. ﬁnd the length of the longest ascending subsequence of A. w4 = 0. The longest ascending subsequence problem is deﬁned as follows.25. linear time) algorithm to ﬁnd the weighted median. ﬁnd values of i and j with 1 ≤ i ≤ j ≤ n such that
j
370. k4 = 3. .55.13
wl = w2 + w4 + w1 = 0.80 369. Given an array A[1. . 2
For example. (A subsequence is a list A[i1 ]. w2 = 0.09. .n] of natural numbers.) Prove that your algorithm performs as claimed. wi )  1 ≤ i ≤ n} is the number km such that wl <
kl <km
1 .10 = 0. (Note that the ki s need not be given in sorted order. Such a subsequence is called ascending if A[i1 ] ≤ A[i2 ] ≤ · · · ≤ A[im ].) Devise a divideandconquer algorithm for solving the longest ascending subsequence problem in time O(n2 ).
.13
Design an eﬃcient (that is. Devise a divideandconquer algorithm for solving the maximum subsequence sum problem in time O(n log n). w5 = 0.13. w1 = 0.
kl ≤4. kn .13 because wl = w2 + w4 = 0. . The maximum subsequence sum problem is deﬁned as follows.25.10. Suppose we are given a set of n numbers k1 .10 + 0. w3 = 0. . given k1 = 4.65.
A[k]
k=i
is maximized. A[im ] for some 1 ≤ i1 < i2 < · · · < im ≤ n.. 372. A[i2 ].. 7. 373.76.
Chap.20 the weighted median is k1 = 4.n] of natural numbers. The value m is called the length of the subsequence.30.15 + 0. Given an array A[1. . Your algorithm must run in time O(n2 ).
371. where each ki has an associated weight wi ≥ 0 such that n wi = 1. Devise a divideandconquer algorithm for multiplying n complex numbers using only 3(n − 1) real multiplications. k2 = 2. k5 = 7. 2
wl ≥
kl ≤km
1 .30 = 0.00. . k3 = 9.
and F = E ∩(U ×U ). E) and an integer k. given a connected graph in which all vertices have even degree.Sec. Your algorithm should run in time O(n + e). Hints
81
374. Show that this algorithm makes exactly the same moves as the standard divideandconquer algorithm. That may require you to place a large disk on top of a small one. n elements.9. No. v) ∈ E). plus some additions and shifts. or to determine that no such labeling is possible (w is a predecessor of v iﬀ (w. design and analyze an eﬃcient divideandconquer algorithm to ﬁnd the kth element in the merge of the two sequences. ﬁnd the maximum induced subgraph H of G such that each vertex in H has degree at least k. Given two sorted sequences with m. Start by deriving and solving a recurrence relation for the number of times that Joe’s procedure LongAdd is called by procedures LongMultiply and FastLongMultiply.
377. That may require you to place a large disk on top of a small one. n))). Design an eﬃcient algorithm to label the vertices of the graph with distinct labels from 1 to V  such that the label of each vertex v is greater than the label of at least one of v’s predecessors (if v has any). 326. (See also Problem 69. The best algorithm runs in time O(log(max(m. F ) such that U ⊆ V .) Let G = (V. Given an undirected graph G = (V. it is not enough to use the standard algorithm and replace every counterclockwise move with two clockwise moves.
An induced subgraph of a graph G = (V. The algorithm should run in time O(n + e).
7. E) is a graph H = (U.
376. Separate the n numbers into disjoint subsets (ﬁnding the right decomposition is crucial) and ﬁrst run the known recursive algorithm on the subsets. 357. It is possible to devise a divideandconquer
. Solve the resulting recurrence relation.
375. Observe that x is computed using only six multiplications of n/3bit integers. No. 359. 355. 324. it is not enough to use the standard algorithm and replace every move from peg 1 to peg 3 (for example) with two moves to and from peg 2. 7. E) be a directed graph (not necessarily acyclic). Find an algorithm that. or determine that it does not exist. constructs an Eulerian cycle in time O(n + e). respectively.9
HINTS
317.
given a vertex v of degree less than k. 365. 375. 362. you will need a method for constructing tours on rectangular boards.73n ) moves.) The algorithm that I have in mind √ has recurrence T (k. the standard algorithm solves this problem. Apply this algorithm to solve the required problem. If you are smart enough. deletes that vertex from the graph. An analysis of a kpeg algorithm by Lu [54] can be found in in Veerasamy and Page [81]. Then write a divideandconquer algorithm that. You can prove it by induction on n. If you try to pass oﬀ their analysis as your own. DivideandConquer
algorithm for moving all of the disks one peg clockwise using (4n −1)/3 moves. You will have to devise an exhaustive search algorithm for ﬁnding small tours for the base of your divideandconquer (see Problem 513). 363. 361. but be careful about the bottom disk on each peg. You will also need some extra structure in your tours to enable them to be joined at the corners (see Problem 48). 374. The best algorithm √ known to the author uses less than (1 + 3)n − 1 = O(2. which slightly simpliﬁes the analysis. and all vertices of degree less than k that are adjacent to it. Take note of which section this problem is in. In addition. note that their k is the number of work pegs. but that is not what is asked for. We are interested in the case k = 4. It’s easy to do this in time O(n + e).4. 369. 368. The induction hypothesis will have to be stronger than just the statement of the required result. you will only need n × (n + 2) tours. In order to construct closed knight’s tours on square chessboards. To get the idea. n)2 . play with some examples. n) = T (k/2 + 1. A problem from Chapter 2 is relevant. which is w in our notation. Use the adjacency list representation. An algorithm by Lu [54] was shown to use at most n2 w!·n moves by Veerasamy and Page [81].
√ w
. This can be used to solve the Towers of Hanoi problem (which is to move all of the disks one peg counterclockwise) in 2(4n − 1)/3 moves. Start by giving an algorithm that sets this ﬁeld to the correct value. or you can make use of the tours shown in Figure 7. Surprisingly. (If you read their paper. it is long enough and idiosyncratic enough that your instructor will be sure to recognize it. the analysis is much simpler if you prove it by induction.82
Chap. 7. See Problem 69. Store for each v ∈ V an additional ﬁeld called its degree.
7. and hence y + x is odd.) If y − x + 1 is odd. Solutions
83 r r r r r r r r ✟ ❍r ✟❍❆ ❍✟ ✟❍✟✟✟✟✟ ✟ ❆ r ✁❆ r r r r r ✁r ❆ ✟❆ ✟❍✟❍ ❆ r ❆ ❆ ❍r ✁❆ ✁✁r ❆r ✁r ✁r ❆✟r ❆r ✁ ❆ ❍ ✁ r ✁✟r r r r r ❆r ❆❆r ✁ ✟❆ ✁ ❆✁ ✁ ✟ ❍r ❆r r r ❍❍ ❆ ✟ ❆ ✁✁ r ❆r r r ✁❍ ❆❆ ✁ ✟❆ ❍ r ✁r ❆r ❆✁✁r r r ❆r ❆ ❍r ❍ ✁ ✟ ❆ ❍ ❆ r ✁❆ r r ❆r r r ✁r ❆ ❍❍❍❆ ❆ ✟ ❍ ✁❆r ✟✁ ✟❍❆❆ ❍r ❆r r r ❆r ❆✁❆r r ❆✟❍✟✟✟r ✟✟ ✁ ❍ ❍ ✟❍ r r r r r r r r r r r r ❍✟✟❍r ✟✟❍❍✟r ❍ ✟❍❆ ❍❍❍ ✟ r ✁r ✁r r ✁❆ r r r ✁r r ✁❆✁r ❆ ❍❍✟❍✟❍✟✟❍✟ ✟✟ ✟ ✁ ✁❆ ❆ r ✁r r r ✁r ❆r ✁r ❆✁✁❆ r r ✁r ❆ ✁ ✁❆✟ ✟✟ ✁ ✟ ✟ ❍r r r r ❆r r ❆r ✁ ✟❍ ✟✟ ❍❆✁ ❍ r ❆r ✁r ✁r r ✁✁✟r r ✁ ✟ ❍✟✟❍❆ ✁ ❆ ✁ ✁ ❍ ✟✟✟❆❆ ✁ ❍ ❆ r ✁r r r r r r ❆r r ✁r r ❆r ✁ ✟✟❍✁ ✟ ❍ ❆ ✟ ❍r r ❆r r r r r ✁r ❆r r r ❍ r ✁❍ ❆ ❍✟ ✟✁ ❍❆✁ ❆ ✟ ✁ ❍ ❆ ✟ ❍❆ r ❆r r ✁r ✁r ✁r r ✁r r ✁r r ✁r ✟✟ ✁ ✟ ✟❍❍ ❆✟❍ ✁ ❆ ❍✁❆ ❍❍✟ ❍ ❆ ✁ ✟✁ ❍✁ r ✁r ❆r r r ✁r r r r ❆r r ✁r ✁ ✟ ✁ ✁ ✟ ✟✟❍❆ ❆ ✁ r ✁❆ r r ❆✁✁r r r r r ❆r ❆r ✁ r ✟ r ❍✟❆ ✟❍❆✁ ❆ ✟✟❍✟❆ ✟❍❍ ✟ ✟ ❍✟ ❍❍r r r r ❆r r ❆r r r ❆r ✟❍❍ r ✟ ❍✁ ✟❍ ❍✁ ✁❆r ❆❍✟❍✟❍✟❍✟❍
r r r r r r ❍❍✟✟❆ r ✁r r r r ✁r ❆ ✟❍❍✟ ✟❆ ✟❍❍ ❍✟ ✟✟ r ❆r ❆r r ✁r ❆r ✁✁❍ ❆ ✁ ❍ ✟ r r r r ❆r ✁r ❆ ✟❍❍❆ ✁ ❍✁ ❍ r ✁❆ r r r ❆r ✟r ❍✁ ❆❆ ❆ ❍r ❆r r r ✁r ✟ r ❆❍❍✟✟ ✁ ❍✟❍❍ ✟✟❍❍✁❆
r r r r r r r ❍❍❍❍✟✟❆✁r ❍❍❍ ✟✟r r ✁❍✟r r ❍ r ❆✁✟✟❍r ✁r ✟r ❍ ❍❍❍❍r r r ❆r ✁❆r ✁r r ❍✟✁ r✁ ✁ ✁ ✟ ❆ ❆ ❍❍❆ ❆ ✟❆❆ r ✟r ✟ r ✁ ✁ ❆ r r r r ✁r r ❆❍ ❆ ❆ ✟✟✟ ❆✟❆ r ✁ ❆✁r ✁r ✁r ✁r ✁r ✁❆r ❍❍✁❆r ✁ r ❆r ❆ r r ❍✁ ❍✟❍ r ❍✁ ✁❆✁ r ❍✟✟✟r ✟ r r r r r r r r r r ❍❍✟✟❍❍✟✟✟ ❍ ✟ r ✁r r ✁r r ✁r r r r ✁r ❆ ✟❆ ❍❍✟❆ ✟❆ ✟❍❍✟✟✟✟❍❍ ❍✁ ❍❍✁ ❆ r ✁r r ❆r r r r ❆r ✁r ❆r ❍ ❆ ✁❆ ❍ ✟ ✁ ❍r r r r ❆r ✁r ✁r r ✁r ❍❍❆ ✁ ❆❆ r ✁❍ ✟ ✁ ✁ ❍❍ ❆ ✁ ❍✁❆ ✁✁ ❍❍ r ✁r ✁r r r r r ✁r r ❆r ✁ ❆ ❍❍✁ ❍❍❆ ✟r r ✁r ❆r r r ✁r ✁r ❍❍❆❆❆ ❆ ✁ ✁❆ r r❆ ❆ ✁ ✟✟ ✁ ❆ ✟❍ ✟ ✟ r ❆r ❆✟r ❆r r ✁r ✁r ✁r r ✁ ✁ ❆ ✁ ✟ ✟ r ❆ ✟✟ ✟✁ r ✁r r r ❆r r r r r ✁r ❆ ✁ ✟ ✟✟ ✁ ✟ ❆ ✟✟✟✟✁ ✁ ✟❆ r ✁r ✁❆ r r ❆r ✁r r ✁r ❆r ✁❆✟r ✟ ✟❍✟❆✁ ❆ ❍✁ ❍❍✟❍✟❍✟✟❍ ✟❍ ✟ ❍✟ r ❆✁ r ❆✟r r r r r ❆r ✟✟❍r ❍✁ ✁ ✟ ❍✁ ✁ r
r r r r r r r r r r ❍ ✟ ❍✟✟❍❍✟✟❍❍ ✟✟❍❍✟✟❍✟✟ ❍ r ✁r r r r ✁r r r r ✁r ❆ ❍✟✟❍❍✟✟❆ ❍ ❍✟ ❆ r ✁r r r r r r r r ✁r ✁❆❍✟ ✁ ✟ ❍❍ ❍❍❍ ✁❆ ❍❍ r ✁r ✁r ✁r r ✁r r ✁r r ❆r ❆ ✁ ✟ ❍✟❍❆ ❆ ✁ r r r ✁r r r r ❆✟r ❆r ✁ ❆ ✟ ✟ ❍ ❆ ❆ ✁ ✟✟❆ ✁ r ❆❆ ✁ ✟❆ ✁ ✟❍❍ r ❆r ❆r ❆r r ❆r ❆r r r ❆r ❆✁ ✁ ❆ ❍❍ ❍ ✁ ✟ ✟✟ ✟ ✟ r ✁r r ✁r ❆r r r r r ✁r ✁ ❆❆❍✟✟✟❍✟❍❍ ✟✟✁ ❆ ❍❍✁ ❆ ❍✟❆ ✁ r ❆✟r r r ❆✟r r ❆r ❆r ✁ r ❍✟❍r ❍✟✟ ❍ ✟✟❍❍✟✟
Figure 7. then y − x is even.) In order to apply the induction hypothesis to the second recursive call. y) will return the maximum value in S[x. Here is a full solution when n is not necessarily a power of 2. y − x + 1. In order to apply the induction hypothesis to the ﬁrst recursive call.
7. y) will return the maximum value in S[x.y] whenever y − x + 1 < n. 6 × 8. and 10 × 12 boards. 10 × 10. Now suppose n > 2. Knight’s tours for (in rowmajor order) 6 × 6. −x+1= 2 2 2 2 (The last inequality holds since n > 2. We will prove by induction on n = y − x + 1 that maximum(x. The size of the problem is the number of entries in the array. then y − x is odd. If y − x + 1 is even.y]. x+y y−x+2 n+1 x+y −x+1= = < n.10
SOLUTIONS
314. we must prove that (x + y)/2 − x + 1 < n. There are two cases to consider. 8 × 8. 8 × 10.Sec. we must prove that y − ( (x + y)/2 +
. Therefore..4. y−x+1 n x+y x+y−1 −x+1= = < n. −x+1= 2 2 2 2 (The last inequality holds since n > 1. and that maximum(x. The algorithm is clearly correct when n ≤ 2.10.. and hence y + x is even. Therefore.
it alternates between the preceding two moves. and for all n > 1 a power of 2. T (n) = n − 1. Let T (n) be the number of moves it takes to move n disks from peg i to peg j.
. we are told to alternate between the following two moves: • If n is odd. Suppose we use the recursive algorithm to move n + 1 disks in direction D. It does the following: • Move n disks in direction D. Let D be the opposite direction. Hence. T (n) = 2T (n/2) + 1. Let D be a direction. either clockwise or counterclockwise. it returns the correct values. Let T (n) be the number of comparisons used by maximum(x. +1 +1=y− 2 2 2 2
If y − x + 1 is odd. The proof of the claim is by induction on n. Therefore. We claim that when the recursive algorithm is used to move n disks in direction D. using the techniques of Chapter 4. move the smallest disk in direction D. DivideandConquer
1) + 1 < n. 355. where n = y − x + 1 is the size of the array chunk processed. The claim is obviously true when n = 1. T (1) = 0. then y− x+y y−x+1 n x+y−1 = = < n. • Move one disk in direction D. y). 7.84
Chap. 354. T (n) = 2n − 1. If y − x + 1 is even. To move n disks in direction D. by the preceding. move the smallest disk in direction D. • Move n disks in direction D. Hence. Hence. Then if n is a power of 2. Now suppose that the claim is true for n disks. using the techniques of Chapter 4. 1 if n = 1 T (n) = 2T (n − 1) + 1 otherwise. +1 +1=y− 2 2 2 2 2
Procedure maximum divides the array into two parts. • Make the only other legal move. 316. then y− x+y y−x+1 n 1 x+y = − 1/2 = − < n. since the procedure returns the maximum of the two maxima. If n is even. This is enough to prove correctness of the new algorithm. the recursive calls correctly ﬁnd the maxima in these parts. By the induction hypothesis. Clearly. it runs in time O(n). The running time of procedure maximum is clearly a constant times the number of comparisons. 315. There are two cases to consider.
This problem asks you to extend this to any value of n. 349. See also Problem 321. the claim holds for any number of disks. n disks n disks as required. So where does this algorithm get its speed from? The secret is in Problem 320. • “D” denote moving the smallest disk in direction D.) Hence. Then n is odd. Then n is even. I don’t propose this as a viable alternative to quicksort. and • “O” denote making the only other legal move. moving n+1 disks in direction D uses moves: DODO · · · OD O DODO · · · OD . the number of moves is odd. moving n disks in direction D uses moves: DODO · · · OD.11. Hence. n + 1 is odd. Binary search is normally described and analyzed for n a power of 2. See also Problem 329. correctness proof. moving n disks in direction D uses moves DODO · · · OD. and so by the induction hypothesis. so the preceding sequence of moves ends with D. moving n+1 disks in direction D uses DODO · · · OD O DODO · · · OD . there is a faster algorithm than this one (see Sch¨nhage and Strassen [70]).Sec. the number of moves is odd. Case 1. 329. (Note that by Problem 354. Although it is faster than the standard divideandconquer. so the preceding sequence of moves ends with D. o 336. 330. not O. and analysis of divideandconquer algorithms.11
COMMENTS
321. n + 1 is even. and so by the induction hypothesis. A solution to this problem appears in Parberry [62]. (Note that by Problem 354.
7. by induction. This is just an exercise in the design. Case 2.
. 7. n disks n disks as required. Comments
85
Let • “D” denote moving the smallest disk in direction D.) Hence. 369. not O.
There is an algorithm that will do this using at most 2n − 1 moves. If done correctly. DivideandConquer
354. it would take 1.84 × 1019 moves. There is even one that is provably optimal. 369. More realistically.85 × 1011 years. there is a set of 64 gold disks on 3 diamond needles being solved by priests of Brahma. 360. Perhaps the legend is correct. 7.86
Chap. Alternative algorithms for constructing closed knight’s tours on rectangular chessboards appear in Cull and DeCurtins [20] and Schwenk [71].54 × 1014 years. The current age of the Universe is estimated at ≈ 1010 years. 371. There is a faster algorithm for this problem.
. See Problem 409. According to legend. There is an O(n) time algorithm for this problem. At one move per second. it will take T (64) = 264 − 1 = 1. 370. The Universe is supposed to end when the task is complete. that’s 5. at 1 minute per move (since the largest disks must be very heavy) during working hours.
3. I don’t approve of busywork problems.
8. Mn . (See also Problems 91 and 263. 2. . function matrix(n) for i := 1 to n do m[i. assuming that the cost of multiplying an ri−1 × ri matrix by a ri × ri+1 matrix using the naive algorithm is ri−1 ri ri+1 . 4. The trick is to solve them in the right order so that whenever the solution to a subproblem is needed. In general. j] + ri−1 rk rj ) return(m[1. Hence. the following problems include a few of this nature. M1 . where for 1 ≤ i ≤ n. 378. Instead of solving subproblems recursively. solve them sequentially and store their solutions in a table. it makes sense to compute each solution the ﬁrst time and store it away in a table for later use. k] + m[k + 1. it is already available in the table. . 5. . 87
.) Here is the dynamic programming algorithm for the matrix product problem: 1. but there are really only a small number of subproblems repeated exponentially often. j] := mini≤k<j (m[i. i] := 0 for d := 1 to n − 1 do for i := 1 to n − d do j := i + d m[i. Mi is a ri−1 ×ri matrix. parenthesize the product M1 ·M2 · · · Mn so as to minimize the total cost. Dynamic programming is particularly useful on problems for which divideandconquer appears to yield an exponential number of subproblems.1
ITERATED MATRIX PRODUCT
The iterated matrix product problem is perhaps the most popular example of dynamic programming used in algorithms texts. In this case.Chapter 8
Dynamic Programming
Dynamic programming is a fancy name for divideandconquer with a table. 6. instead of recomputing it recursively every time it is needed. but I have found that undergraduates don’t really understand dynamic programming unless they ﬁll in a few tables for themselves. M2 . Given n matrices. n])
Find three other orders in which the cost table m can be ﬁlled in (ignoring the diagonal). .
. r3 = 5. n = 4. . Start by multiplying Mi by Mi+1 . and that ri is the smallest of r1 . r3 = 16. r3 = 19. n = 5. when the product of the matrices with these dimensions is evaluated in the order given. .
390. . r0 .88 379. 386. ﬁnd n. . 384. r1 = 3. r2 = 2. r1 .
Find counterexamples to the following algorithms for the iterated matrix products problem. r2 . r2 = 2. and then repeat the same thing recursively in each half. r5 = 3. r1 = 7. r0 = 2. Start by multiplying Mi by Mi+1 . r0 = 8. rn−1 . and recursively apply this procedure to the products M1 × M2 × · · · × Mi and Mi+1 × M2 × · · · × Mn . r2 .
388. r2 . . Multiply left to right. . 383. 382. Suppose ri is the largest of r1 . . r4 = 18. That is. . . . . Suppose n > 2. rn−1 . r2 = 15. r4 = 10. 387. Dynamic Programming
Prove that there are Ω(2n ) diﬀerent orders in which the cost table m can be ﬁlled in. 381. . r5 = 7. Repeat this until the product has been evaluated. r4 = 8. and recursively apply this procedure to the products M1 × M2 × · · · × Mi and Mi+1 × M2 × · · · × Mn . r2 = 4. n = 5. . . r3 = 1. rn−1 . Start by cutting the product in half. rn such that. . . r1 = 5.
Fill in the cost table m in the dynamic programming algorithm for iterated matrix products on the following inputs: 380. . r1 = 2. Multiply right to left. Break the product after Mi . Suppose ri is the smallest of r1 . the cost is higher than optimal. r4 = 10.
. 8. . .
385. Break the product after Mi . Suppose ri is the largest of r1 .
Chap. rn−1 . r0 = 3. r2 .
389. r0 = 12. Repeat this until the product has been evaluated. n = 4.
s2 . . The standard dynamic programming algorithm for the knapsack problem is as follows. Put them in the knapsack in left to right order (the ﬁrstﬁt algorithm). Put them in largest ﬁrst (the worstﬁt algorithm). 396. 3. function knapsack(s1 .2
THE KNAPSACK PROBLEM
The knapsack problem (often called the zeroone knapsack problem) is as follows: given n rods of length s1 . and a natural number S. Put them in smallest ﬁrst (the bestﬁt algorithm). ﬁnd a subset of the rods that has total length exactly S. 3. That is. . 2.
Find counterexamples to the following algorithms for the knapsack problem. 395. 6. j] ∨ t[i − 1. sn such that when the rods are selected using the algorithm given. 7. 393.
Fill in the table t in the dynamic programming algorithm for the knapsack problem on a knapsack of size 10 with rods of the following lengths: 392. n. j] := t[i − 1.2. 2. s1 . 0] :=true for j := 1 to S do t[0. 1. j − si ] return(t[n. 4. 4.
. 1. . 394. 3. s2 . 7.
Fill in the table t in the dynamic programming algorithm for the knapsack problem on a knapsack of size 19 with rods of the following lengths: 15. . j] if j − si ≥ 0 then t[i. 3. . . 397. 5. S])
1. . 2. j] be true if there is a subset of the ﬁrst i items that has total length exactly j. . 3. the knapsack is not completely full. 6. 3. sn . The Knapsack Problem
89
8. sn . Let t[i. 6. j] :=false for i := 1 to n do for j := 0 to S do t[i. 15. 5. .Sec. 8. . 391. S) t[0. . ﬁnd S. 16. j] := t[i. 2. 5. 4. .
. . S): min(S): delete(x. for 1 ≤ i ≤ n. . q0 = 0. j]) return(c[0.12. . q0 .09.07. .j . q1 = 0. insert x into S. q3 = 0. i] := qi . p3 = 0. the probability qi that we will be asked member(x. j] + w[i.16.12. .03.1. p4 = 0. 3.21. S). 5. k − 1] + c[k. 2. pn . n = 4. construct a binary search tree (see also Section 11.02. with operations: search(x. . p1 = 0. p2 . 1. for 0 ≤ i ≤ n. Given the following three sets of data.01. S) for some xi < x < xi+1 (where x0 = −∞. n = 4. j − 1] + pj + qj c[i. where xi < xi+1 for all 1 ≤ i < n. Dynamic Programming
8. 3.22. the probability pi that we will be asked member(xi . 8.j be the mincost binary search tree for xi+1 . 6. q0 = 0. q1 = 0.07. p2 = 0. p1 = 0. 1. . j] be the expected number of comparisons for Ti. q1 = 0. q0 = 0. .21. S = {x1 . q4 = 0.3
OPTIMAL BINARY SEARCH TREES
Binary search trees are useful for storing a set S of ordered elements.
The optimal binary search tree problem is the following.04. .01. and c[i. p2 = 0. Let Ti. . p3 = 0. q2 = 0.07. j] := mini<k≤j (c[i. 4. xn+1 = ∞). xn }. p4 = 0. q2 = 0. .2. S): insert(x. 398. p2 = 0.90
Chap. .17. xj . p4 = 0. q3 = 0.03. j] := w[i. c[i. The standard dynamic programming algorithm for this problem is as follows. .02. 7. q3 = 0. q4 = 0. 8.28. n])
Fill in the cost table c in the dynamic programming algorithm for the optimal binary search tree problem on the following inputs.2) that has the minimum number of expected comparisons.03. . . 400.15.1. q2 = 0. S): return true iﬀ x ∈ S. p3 = 0.32. q1 . return the smallest value in S.11.05. 2. i] := 0 for := 1 to n do for i := 0 to n − do j := i + w[i. qn ) for i := 0 to n do w[i. p1 = 0. function bst(p1 . n = 4. q4 = 0. delete x from S. 399.
j] < A[i. 1.4
FLOYD’S ALGORITHM
The allpairs shortestpaths problem is as follows. Floyd’s Algorithm
91
(a) 1 10 4 6 5 2
50
(b) 2 100 4 6 1 30
(c) 12 4 9 21 3 3 3 5 4 1 8 1 2
30
65 70 3 23 4 20 44
65 25 3 5 9 20 23 4
Figure 8. w) for each vertex v.1. 2. Show that Floyd’s algorithm on an undirected graph chooses a minimum cost edge (v. Suppose V = {1. 11. j] := cost of edge (i. 10. 401. w ∈ V the cost of the shortest (that is. The standard dynamic programming algorithm for the all pairs shortest paths problem (Floyd’s algorithm) is as follows.
. ﬁnd for each pair of vertices v. 7. 6. j] := A[i. . . 8. i] := 0 for k := 1 to n do for i := 1 to n do for j := 1 to n do if A[i.4. j] then A[i. n) for i := 1 to n do for j := 1 to n do if (i. Given a labeled directed graph G = (V. Can an allpairs shortestpath algorithm be designed simply by choosing such a cheapest edge for each vertex? What if all edge costs are diﬀerent?
Modify Floyd’s algorithm to solve the following problems on directed graphs. 8. 2. j] := ∞ A[i.Sec. function ﬂoyd(C. the leastcost) path from v to w in G. j) else A[i. 4. 402. . E). 9. 5. j] return(A)
Fill in the cost table A in Floyd’s algorithm on the graphs shown in Figure 8. k] + A[k. j) ∈ E then A[i. . n}. Analyze your algorithms. 3. k] + A[k.
8.1. Some graphs.
S → AB. Determine the label and length of the lexicographically ﬁrst shortest path between each pair of vertices. B. A special nonterminal symbol called the root.5
APPLICATIONS
The problems in this section ask you to apply dynamic programming in new situations. c ∈ L(C). A → a.
Chap. 407. Determine the number of shortest paths between each pair of vertices. One way to proceed is to start by devising a divideandconquer algorithm. In each case. B → b. A set of terminal symbols T . 404.92 403. on input a string of terminals x. replace the recursion with table lookups. A set of productions of the form either A → BC. String: aaabbb. C ∈ N . The language generated by a grammar with root R is deﬁned to be L(R). the root symbol is S. 8. 406. 405.
8. Grammar: S → AR. and devise a series of loops that ﬁll in the table in the correct order. Dynamic Programming
Determine the number of paths (possibly inﬁnite) between each pair of vertices. Devise an algorithm for the CFL recognition problem. Fill in the tables in the dynamic programming algorithm for the CFL recognition problem (see Problem 407) on the following inputs.
. String: ssssss. Determine the number of evenlength paths and of oddlength paths between each pair of vertices. where A. deﬁne L(A) as follows: L(A) = {bc  b ∈ L(B). Given an nvertex graph whose vertices are labeled with distinct integers from 1 to n. The CFL recognition problem is the following: For a ﬁxed contextfree grammar in Chomsky Normal Form.
If A ∈ N . Analyze your algorithm. or A → a. S → s. R → SB. 408. a ∈ T . (a) (b) Grammar: S → SS. • • • • A contextfree grammar in Chomsky Normal Form consists of A set of nonterminal symbols N . determine whether x is in the language generated by the grammar. where A → BC} ∪ {a  A → a}. the label of each path is obtained by taking the labels of vertices on the path in order.
Sec. Applications
93
t h i s i s a s t r i ngo f c ha r s t h i s i s a s t r i ngo f c ha r s t h i s i s a s t r i ngo f c ha r s t hi s i sas t r i ngo f c ha r s Cost: 20 units Cost: 17 units Cost: 12 units Total: 49 units
Figure 8. when given the numbers of the characters after which to break. That is. (a) Start by cutting the string as close to the middle as possible.
411. String: abacbbcaba. determines the cheapest cost of those breaks in time O(n3 ). a total of 49 units of time (see Figure 8. and the third break costs 8 units of time. The order in which the breaks are made can aﬀect the total amount of time used. X → SA.
409. the cost in higher than optimal. Find counterexamples to the following algorithms for the stringcutting problem introduced in Problem 410. then the ﬁrst breaks costs 20 units of time. A certain stringprocessing language allows the programmer to break a string into two pieces.
Devise a dynamic programming algorithm that solves the longest ascending subsequence (see Problem 370) in time O(n log n). Suppose a programmer wants to break a string into many pieces. then the ﬁrst break costs 20 units of time. it costs n units of time to break a string of n characters into two pieces. 8. S → CZ.
(c)
Grammar: S → AX. For example. the second breaks costs 17 units of time. suppose we wish to break a 20 character string after characters 3. and the third breaks costs 12 units of time. Z → SC. Devise a dynamic programming algorithm that. B → b. The cost of making the breaks in lefttoright order. and then repeat the same thing recursively in each half.2.3). and 10 (numbering the characters in ascending order from the lefthand end. S → BY . C → c. If the breaks are made in lefttoright order. Since this involves copying the old string. A → a. ﬁnd the length of the string. and several places to cut such that when cuts are made in the order given.
410. a total of 38 units of time (see Figure 8.5. If the breaks are made in righttoleft order.
. starting from 1). 8.2). Y → SB. the second break costs 10 units of time.
and the total cost of shipping the widgets. that is. n xi = rV . Dynamic Programming
t h i s i s a s t r i ngo f c ha r s t his i sas t r t his i sas t r t hi s i sas t r i ngo f c ha r s i ngo f c ha r s i ngo f c ha r s Cost: 20 units Cost: 10 units Cost: 8 units Total: 38 units
Figure 8.
n
(vi xi + wi yi )
i=1
is minimized. xi + yi = di . i=1 n • the inventory at W is suﬃcient. that is.
(b)
Start by making (at most) two cuts to separate the smallest substring. and supplies are sent to destinations Di for all 1 ≤ i ≤ j in the optimal
. 8. respectively.
There are two warehouses V and W from which widgets are to be shipped to destinations Di .
412. Repeat this until ﬁnished.3. for 1 ≤ i ≤ n. Let di be the demand at Di .1)
Let vi be the cost of shipping a widget from warehouse V to destination Di . (a) Let gj (x) be the cost incurred when V has an inventory of x widgets. for 1 ≤ i ≤ n. The cost of making the breaks in righttoleft order. that is. • the inventory at V is suﬃcient. and wi be the cost of shipping a widget from warehouse W to destination Di .
(8. 1 ≤ i ≤ n. rW be the number of widgets available at V and W . Assume that there are enough widgets available to ﬁll the demand. that is. and rV . i=1 yi = rW . The warehouse problem is the problem of ﬁnding xi . yi ∈ IN for 1 ≤ i ≤ n such that when xi widgets are sent from V to Di and yi widgets are sent from W to Di : • the demand at Di is ﬁlled. Repeat this until ﬁnished. Start by making (at most) two cuts to separate the largest substring. that n rV + rW =
i=1
di .94
Chap.
The cursor remains stationary. . of the cubes of the number of blanks at the end of each line (for example. .1). by (8.
. The operations are shown in Table 8.
manner (note that W is not mentioned because knowledge of the inventory for V implies knowledge of the inventory for W . (b) Use this recurrence to devise a dynamic programming algorithm that ﬁnds the cost of the cheapest solution to the warehouse problem. Delete all characters from (and including) the one under the cursor to the end of the line. b2 . Analyze your algorithm. 414. Applications
95
Advance Delete Replace Insert Kill
Move cursor one character to the right Delete the character under the cursor. one way to transform the string algorithm to the string altruistic is shown in Figure 8. respectively.1. Give a dynamic programming algorithm to compute this 1 2 k−1 value. and move the cursor to the next character. respectively. This can only be the last operation. A “smart” terminal changes one string displayed on the screen into another by a series of simple operations.
Table 8. . .5. For example. characters. Consider the problem of neatly printing a paragraph of text on a printer with ﬁxedwidth fonts. . The input text is a sequence of n words containing 1 . if there are k lines with b1 . . 2 . Analyze your algorithm. 413. over all lines except the last.) Write a recurrence relation for gj (x) in terms of gj−1 . The cursor is initially on the ﬁrst character of the string. Replace the character under the cursor with another. The cursor remains stationary. If a printed line contains words i through j. bk blanks at the ends. then the number of blanks left at the end of the line (given that there is one blank between each pair of words) is
j
M −j+i−
k=i
k. 8.1.4. Each line on the printer can hold up to M characters. Operations allowed on a smart terminal. then we wish to minimize b3 + b3 + · · · + b3 ). . .Sec.
We wish to minimize the sum. n . Insert a new character before the one under the cursor.
Dynamic Programming
Operation advance advance replace with t advance delete advance insert u advance insert s advance insert i insert c kill
String Algorithm aLgorithm alGorithm alTorithm altOrithm altRithm altrIthm altruIthm altruiThm altruisThm altruistHm altruistiHm altruisticHm altruistic
Figure 8. So. the delete operation takes d milliseconds. The coinchanging problem is deﬁned as follows. a2 . for example. (a) Show that everything to the left of the cursor is a preﬁx of the new string. Given C ∈ IN. determines the fastest time needed to transform x to y. Transforming algorithm to altruistic using the operations shown in Table 8. The cursor position is indicated by a capital letter. 8. the insert operation takes i milliseconds.
. an } is a set of distinct coin types. 415. .4. Suppose that on a particular brand of terminal. the advance operation takes a milliseconds. for 1 ≤ i ≤ n. Analyze your algorithm. the replace operation takes r milliseconds. Suppose also that a1 < a2 < · · · < an . (b) Devise a dynamic programming algorithm that.
There are many sequences of operations that achieve the same result.96
Chap. and the kill operation takes k milliseconds. . Analyze your algorithm. and everything to the right of the cursor is a suﬃx of the old string.n] and y[1..1. the sequence of operations to transform algorithm into altruistic takes time 6a + d + r + 4i + k. .n]. given that an unlimited number of coins of each type are available. Design a dynamic programming algorithm that on inputs An and C. given two strings x[1. where each ai ∈ IN. Suppose An = {a1 .. . ﬁnd the smallest number of coins from An that add up to C. outputs the minimum number of coins needed to solve the coinchanging problem.
75 British pounds.S. dollar and end up with 0. .
417. That is. If you buy it at time t. That is.Sec. Devise a dynamic programming algorithm that computes the maximum amount of money that you can make over the n months in time O(n). Then. Write down the cheapest parenthesization of each of the matrix products. if you have $k at time t. it will return B(t) for every dollar invested. dollars. (c) Store the money in a sock under your mattress for a month. B.
8. Finding the Solutions
97
416. Devise and analyze a dynamic programming algorithm to determine the maximum value of R[c1 .05 U. Write down the shortest path between each pair of vertices. if you have $k at time t. there may be a small window of time during which 1 U.S. 419. ci2 ] · · · R[cik−1 . Show the contents of the array P for each of the graphs in Problem 401. it will return S(t) for every dollar invested. At the beginning of each month. 1 British pound buys 2 Australian dollars. then you will have $k at time t + 1. Suppose you have predicted values for S. Write an algorithm for recovering the packing from the information generated by your modiﬁed dynamic programming algorithm.
You have $1 and want to invest it for n months. If you buy it at time t.
Arbitrage is the use of discrepancies in currencyexchange rates to make a proﬁt. . and an n × n table R of exchange rates. you must choose from the following three options: (a) Purchase a savings certiﬁcate from the local bank. For example.
. CB for the next n months.6. Your money will be tied up for six months. then you will have $(k − CB (t))B(t) at time t + 6. c1 ]. . That is.7 = 1. CS . dollars. j] units of currency cj .S. there will be a fee of CS (t) and after a month.
FINDING THE SOLUTIONS
Show the contents of the array P for each of the parts of Problems 380– 383. (b) Purchase a state treasury bond. cik ] · R[cik .S. Suppose that there are n currencies c1 . a proﬁt of 5%. such that one unit of currency ci buys R[i. there will be a fee of CB (t) and after six months. Your money will be tied up for one month. cn . then you will have $(k − CS (t))S(t) at time t + 1. Modify the dynamic programming algorithm for the knapsack problem so that the actual packing of the knapsack can be found. if you have $k at time t. . a smart trader can trade one U. and 1 Australian dollar buys 0. dollar buys 0. ci1 ] · R[ci1 .70 U.75 × 2 × 0.
420.6
418. 8.
After graduation. The output is to be a guest list that maximizes the total conviviality. .
427. ci2 ] · · · R[ik−1 . . 8.
Chap.
Modify your algorithm for Problem 417 so that it determines the optimal investment strategy. rooted at the CEO. Modify your dynamic programming algorithm for the fastest editsequence problem (Problem 414) so that the fastest edit sequence can be recovered. The algorithm has input consisting of the hierarchy tree. i2 . you start working for a company that has a hierarchical supervisor structure in the shape of a tree. Design such a dynamic programming algorithm.
423.
. Write an algorithm for recovering these values from the information generated by your modiﬁed dynamic programming algorithm. Write an algorithm for recovering this data from the information generated by your modiﬁed dynamic programming algorithm. Dynamic Programming
Modify your dynamic programming algorithm for the stringcutting problem (Problem 410) so that the cuts can be recovered. their parent in the tree) is not invited.
426.
422. ik ] · R[ik . . and analyze it.
424. Modify your dynamic programming algorithm for the warehouse problem (Problem 412) so that the values xi and yi for 1 ≤ i ≤ n can be recovered. Write an algorithm for recovering this sequence of operations from the information generated by your modiﬁed dynamic programming algorithm. The personnel oﬃce has ranked each employee with a conviviality rating between 0 and 10. . ci1 ] · R[ci1 . Your ﬁrst assignment is to design a dynamic programming algorithm to construct the guest list for the oﬃce Christmas party. 428. and conviviality ratings for each employee. Modify your algorithm for Problem 416 to determine the sequence of currencies i1 . Modify your dynamic programming algorithm for the printing problem (Problem 413) so that the number of words on each line can be recovered. Modify your dynamic programming algorithm for the coinchanging problem (Problem 415) so that the number of each type of coin can be recovered.
425. Write an algorithm for recovering these values from the information generated by your modiﬁed dynamic programming algorithm.98 421. Write an algorithm for recovering the cuts from the information generated by your modiﬁed dynamic programming algorithm. ik that maximizes R[c1 . his or her immediate supervisor (that is. 1]. and ensures that for every employee invited to the party.
1. and r0 = 1. you should be able to ﬁnd the length of the longest ascending subsequence. From the information in B.
SOLUTIONS
1 1 2 3 4 5 0 2 48 0 3 352 114 0 4 1020 792 684 0 5 1096 978 936 2394 0
384. which is superior for C = o(n/ log n). 5. Make sure that the array is ﬁlled in such way that the required entries are there when they are needed. Construct an array B[1. j] depends. 8. Fill B from back to front. but with a little thought it can be analyzed using Harmonic numbers to give a running time of O(C 2 log n). Hints
99
429. 409. take n = 7. you will need to keep track of the number of blanks in the last line of text in the solution of each subproblem. For a larger example.8
383. An O(nC) time algorithm can be constructed by exploiting the similarity between the coin changing problem and the knapsack problem (see Section 8. be able to output a longest ascending subsequence (there may be more than one of them.Sec. r2 = 32. The most eﬃcient way to parenthesize the product is (M1 · M2 ) · M3 . r3 = 12.. 32.
8. The base case is more than just ﬁlling in the diagonal entries. 12. 6. 17. You should also. The suggested algorithm parenthesizes the product as M1 ·(M2 ·M3 ).
You are not sure if the CEO should get invited to the party.n] such that B[i] contains the length of the longest ascending subsequence that starts with A[i]. For each nonterminal symbol S ∈ N . for 1 ≤ i ≤ n. 415. and dimensions 1. 12. obviously) in time O(n). Start by drawing a picture of the array entries on which m[i. 2. Suppose n = 3.n. We
. with a little thought.7. keep a Boolean table tS [1.7
HINTS
378. but you suspect that you might get ﬁred if he is not.2). Can you modify your algorithm to ensure that the CEO does get invited?
8..n]. 407. It is easy to formulate an inferior algorithm that appears to run in time O(nC 2 ). In addition to the table of costs for subproblems. r1 = 2.. at a cost of 1 · 2 · 32 + 1 · 32 · 12 = 448. at a cost of 2·23·12+1·2·12 = 792. 413.
for each production S → T U do 10. The forloop on lines 7–10 costs O(n). I recommend strongly that you do it by hand instead. j] is true iﬀ either i = j and S → xi . j] is true iﬀ the string xi xi+1 · · · xj is in the language generated by S.
.9
COMMENTS
380. The forloop on lines 5–10 costs O(n2 ). However. In some cases. k] ∧ tU [k + 1. j] := tS [i. for each nonterminal S do 2.
Therefore. j] are true. k] and tU [k + 1. tS [i. tS [i. Although the easiest way to do this is to write a program to do it for you. j := i + d 7. 8. i] :=true iﬀ S → xi 4. or there exists k such that i ≤ k < j and S → T U . return(tR [1. tS [i. function cﬂ(x1 x2 · · · xn ) 1. 415. It will help you to understand the subtleties of the algorithm. the algorithm runs in time O(n3 ). n]) where R is the root Since there are O(1) nonterminals: • • • • • The forloop on lines 1–3 costs O(n). for each nonterminal S do 8. See also Problem 95. tS [i. a greedy algorithm for this problem is superior to dynamic programming (see Problem 471). Line 10 costs O(n) (a single forloop). for d := 1 to n − 1 do 5. Dynamic Programming
will ﬁll the tables so that for 1 ≤ i ≤ j ≤ n. j] :=false 9. j]) k=i 11. tS [i. j] ∨ j−1 (tT [i. for i := 1 to n do 3. where tT [i.100
Chap. there is nothing wrong with using such a program to verify that your answer is correct. for i := 1 to n − d do 6. The forloop on lines 4–10 costs O(n3 ).
8.
assuming that the objects are uniformly dense (so that for all 0 ≤ xi ≤ 1. . That is. paying attention to shortterm or local gain. . Then. we ﬁrst sort the objects in nondecreasing density. object Ai has length si and weight wi . Use as much of lowdensity objects as possible. If not. . 101 Show that the greedy algorithm gives a solution of mini
. without regard to whether it will lead to a good longterm or global solution. Deﬁne the density of object Ai to be wi /si . . ﬁll the remaining space with a fraction of the current object. x2 . . That is.1
CONTINUOUS KNAPSACK
The continuous knapsack problem is similar to the knapsack problem studied in Section 8. an xi fraction of Ai has length xi si and weight xi wi ). process each in increasing order of density. use all of it. and discard the rest. so that wi /si ≤ wi+1 /si+1 for 1 ≤ i < n. greedy algorithms sometimes lead to the best solution. and sometimes lead to lousy solutions. If the whole thing ﬁts. sometimes lead to pretty good solutions. we perform the following pseudocode: s := S. . xn < 1 n n that minimizes i=1 xi wi subject to the constraint S = i=1 xi si . We must ﬁll a knapsack of length S as full as possible using fractional parts of the objects. The trick is to determine when to be greedy. The greedy algorithm proceeds as follows. As in real life.2. . that is. A1 .
9. so that the total weight is minimized. A2 . i := i + 1 xi := s/si for j := i + 1 to n do xj := 0 430. An . we must ﬁnd 0 < x1 . That is. i := 1 while si ≤ s do xi := 1.s := s − si . where for all 1 ≤ i ≤ n. This augmentation is done in a “greedy” fashion. We are given n objects.Chapter 9
Greedy Algorithms
A greedy algorithm starts with a solution to a very small subproblem and augments it successively to a solution for the big problem. mum weight. .
w2 = 15. all edge labels are equal to unity. 10. w1 = 18. (See also Problem 433. 7. s5 = 3.
Chap.2
DIJKSTRA’S ALGORITHM
The singlesource shortestpaths problem is deﬁned as follows. s4 = 12. E) and a distinguished vertex s ∈ V .
435. w3 = 28. s4 = 32. w5 = 47. v) ∈ E then P [v] := s else P [v] := 0 for i := 1 to n − 1 do choose w ∈ V − S with smallest D[w] S := S ∪ {w} for each vertex v ∈ V do if D[w] + C[w. s3 = 7. P [v] := w
Show the contents of arrays D and P after each iteration of the main forloop in Dijkstra’s algorithm executed on the graphs shown in Figure 9.)
433. s2 = 6. Use this information to list the shortest paths from vertex 1 to each of the other vertices. Show that an arbitrary collection of singlesource shortest paths in an undirected graph does not necessarily form a spanning tree. 6. w4 = 5. w3 = 8. directed graph G = (V. 4.1. v] < D[v] then D[v] := D[w] + C[w. S := {s} for each v ∈ V do D[v] := C[s.
9.102 431. 9. 434. (See also Problem 367 for the unweighted version of this problem. s5 = 23. s1 = 12. s2 = 5. s4 = 12. s3 = 17.) Is the spanning tree formed by Dijkstra’s algorithm on a connected undirected graph a mincost spanning tree? (See also Problem 434. the shortest (that is. s5 = 3. v]. w5 = 1. w3 = 28. (b) S = 18.) Vertex s is called the source. s1 = 9. w1 = 4. Show that the singlesource shortest paths constructed by Dijkstra’s algorithm on a connected undirected graph form a spanning tree. w5 = 6. with source vertex 1. C[i. 432. 5. Greedy Algorithms
Solve the continuous knapsack problem on the following inputs: (a) S = 20. 3. s1 = 9. leastcost) path from s to w. (c) S = 25. 1. s3 = 7. 2. w4 = 50. v] if (s. w2 = 15. 8. Dijkstra’s algorithm for the singlesource shortestpaths problem is the following. Given a labeled. w2 = 4. 9.
. w4 = 50. s2 = 5. j] denotes the cost of the edge from vertex i to vertex j. that is. ﬁnd for each vertex w ∈ V . w1 = 18.
then provide a counterexample.Sec. Suppose you are the programmer for a travel agent trying to determine the minimum elapsed time from DFW airport to cities all over the world. and T ⊆ E. It is a mincost spanning tree of a labeled graph if it is a spanning tree of smallest cost. as follows:
. T ) is a spanning tree of an undirected graph G = (V. 9. then you will be in y at time fxy (t) (including waiting time and ﬂying time).3. you immediately think of using Dijkstra’s algorithm on the graph of cities.
436.
9. S is connected and has no cycles). Can we modify Dijkstra’s algorithm to solve this problem by changing minimum to maximum? If so. Desperate. If not. with an edge between two cities labeled with the ﬂying time between them. you instead label each edge from city x to city y with a function fxy (t) that maps from your arrival time at x to your arrival time at y in the following fashion: If you are in x at time t. E) if S is a tree (that is. and explain why it works. Prim’s algorithm constructs a spanning tree using a priority queue Q of edges. But the shortest path in this graph corresponds to shortest ﬂying time. then prove your algorithm correct. which does not include the time spent waiting at various airports for connecting ﬂights. it is reasonable to assume that fxy (t) ≥ t).
437.3
MINCOST SPANNING TREES
A graph S = (V. State clearly any assumptions that you make (for example. MinCost Spanning Trees
103
(a) 1 10 4 6 5 2
50
(b) 2 100 4 6 1 30
(c) 12 4 9 21 3 3 3 5 4 1 8 1 2
30
65 70 3 23 4 20 44
65 25 3 5 9 20 23 4
Figure 9.1.
Suppose we want to solve the singlesource longest path problem. Modify Dijkstra’s algorithm to work under this new formulation. Since you got an A in my algorithms course. Some graphs.
w))
Kruskal’s algorithm constructs a spanning tree using the unionﬁnd abstract data type (see Section 11. 11.2.
442. 1. Q := empty for each w ∈ V such that (1. F is the set of vertexsets in the forest. where u ∈ V1 if v ∈ V1 then T := T ∪ {e} V1 := V1 ∪ {v} for each w ∈ V such that (v. w) ∈ E do insert(Q. Number the edges in order of selection. 3.1. (1.4). 7.
441. 4. Draw the spanning forest after every iteration of the main whileloop in Kruskal’s algorithm on the graph shown in Figure 9. F := ∅ for each vertex v ∈ V do F := F ∪ {{v}} Sort edges in E in ascending order of cost while F  > 1 do (u. T := ∅. 9. v) T := T ∪ {(u. ignoring the directions on the edges. 438.5 on the graph shown in Figure 9.1. V1 := {1}. 5.3?
439. Number the edges in order of selection. 5. w)) while V1  < n do e :=deletemin(Q) Suppose e = (u. Draw the spanning forest after every iteration of the main whileloop in Prim’s algorithm on the graph shown in Figure 9. 10. 3 4. Draw the spanning forest after every iteration of the main whileloop in Prim’s algorithm on each of the graphs in Figure 9.
. 8. ignoring the directions on the edges.3? Would Kruskal’s algorithm start by making the choice of edges shown in Figure 9.4 for the graph shown in Figure 9.
Chap. 2. Greedy Algorithms
T := ∅. Would Prim’s algorithm start by making the choice of edges shown in Figure 9.
440. 9. v) := the next edge in sorted order if ﬁnd(u) = ﬁnd(v) then union(u. 8. v).104 1. 443. w) ∈ E do insert(Q. 2.2. (v. 7. 6. 6. v)}
Draw the spanning forest after every iteration of the main whileloop in Kruskal’s algorithm on each of the graphs in Figure 9.
2.4. A graph. A mincost spanning tree under construction using Prim’s algorithm?
.3. 9. Another graph.
3 5 1 1 7 2 8 6 12 6 9 3 7 2 4 10 4 5 11 1 1 7 2 8 6 12 6 9 3 5 2 4
3 5 4 5 10 7 11 1 1 7 2 8 6 12 6 9 3 2 4
3 5 4 5 10 7 11 1 1 7 2 8 6 12 6 9 3 2 4
3 4 5 10 7 11
Figure 9.3.
3 5 1 1 7 2 8 6 12 6 9 3 7 2 4 10 4 5 11
Figure 9. MinCost Spanning Trees
105
16 1 7 10 3
2
2 13 5
9
6 20
8 4
11 11 6 5 14 9 12
12 1 4 21 3 15 7 18 8 17
10 19
Figure 9.Sec.
Show that Prim’s algorithm and Kruskal’s algorithm always construct the same mincost spanning tree on a connected undirected graph in which no pair of edges that share a vertex have the same cost.6 on the graph shown in Figure 9. Is the path between a pair of vertices in a mincost spanning tree of an undirected graph necessarily a mincost path? Is this true if the mincost spanning tree is unique? If it is not unique?
445.
448.
Could a greedy algorithm for mincost spanning trees start by making the choice of edges shown in Figure 9. A mincost spanning tree under construction using Kruskal’s algorithm?
3 5 2 1 1 7 8 6 12 6 9 3 7 2 4 10 4 5 2 8 7 6 12 6 9 3 1 1 11 5 2 4
3 5 4 5 10 7 2 8 7 6 12 6 9 3 1 1 11 2 4
3 5 4 5 10 7 2 8 7 6 12 6 9 3 1 1 11 2 4
3 4 5 10 7 11
Figure 9. A mincost spanning tree under construction?
444.
446.7 on the graph shown in Figure 9. Greedy Algorithms
3 5 2 1 1 7 8 6 12 6 9 3 7 2 4 10 4 5 2 8 7 6 1 1 11 5 12 6 9 3 2 4
3 4 5 10 7 11 2 1 1 7 8 6 5 12 6 9 3 2 4
3 4 5 10 7 11 2 1 1 7 8 6 5 12 6 9 3 2 4
3 4 5 10 7 11
Figure 9. 9.5.
447.3? Could a greedy algorithm for mincost spanning trees start by making the choice of edges shown in Figure 9.3? Show that Prim’s algorithm and Kruskal’s algorithm always construct the same mincost spanning tree on a connected undirected graph in which the edge costs are distinct.6.106
Chap.
.
2. v).
Show how to modify Prim’s algorithm so that it runs in time O(n log k) on a graph that has only k diﬀerent edge costs. 453. Q := empty for each w ∈ V such that (1. T := ∅. 3 4. Suppose that there are exactly two edges that have the same cost. Here is a version of Prim’s algorithm that is diﬀerent from the one given earlier in this section.3. (1. If not. Is it correct? If so. 9. w))
In the vanilla version of Prim’s algorithm and Kruskal’s algorithm. 452. Prove that for Kruskal’s algorithm.7. V1 := {1}. 1. it is not speciﬁed what to do if a choice must be made between two edges of minimum cost. MinCost Spanning Trees
107
3 1 2 5 1 7 8 6 12 4 6 9 3 7 10 5 2 5 1 7 1 2 4 2 8 6 12 4 6 9 3 2
3 4 5 10 7 2 2 5 1 7 8 6 1 12 4 6 9 3 2
3 4 5 10 7 2 2 5 1 7 8 6 1 12 4 6 9 3 2
3 4 5 10 7 2
Figure 9. it doesn’t matter which of the two edges is chosen. 450. 451. Prove that for Prim’s algorithm.Sec. it doesn’t matter which of the two edges is chosen. 9. 6. explain why it is wrong and give a graph on which it fails. w) ∈ E do insert(Q. 8. Does Prim’s algorithm construct the same mincost spanning tree regardless of which edge is chosen?
. A mincost spanning tree under construction?
449. 7. w) ∈ E do if w ∈ V1 then insert(Q. prove it correct. where u ∈ V1 T := T ∪ {e} V1 := V1 ∪ {v} for each w ∈ V such that (v. w)) while V1  < n do e :=deletemin(Q) Suppose e = (u. 10. (v. 5.
if a choice must be made between two distinct edges. will it ﬁnd a directed spanning tree of minimum cost? Consider the following algorithm for ﬁnding a mincost spanning tree. will it ﬁnd a directed spanning tree of minimum cost? 458. 2. n in turn. or u1 = u2 and v1 < v2 . Does Kruskal’s algorithm construct the same mincost spanning tree regardless of which edge is chosen? Suppose Prim’s algorithm and Kruskal’s algorithm choose the lexicographically ﬁrst edge ﬁrst. 457. (a) Does it ﬁnd a directed spanning tree of a directed graph? (b) If so. e1 = (u1 . . n. . The output is the cheapest of the n directed spanning trees. n. starting at each vertex in turn. v2 ) of the same cost. and then run n times. Is this algorithm correct? Justify your answer. tree. . . choose e2 . then discard it. each consisting of a single vertex. If it has both endpoints in the same tree. . . . (a) Does it ﬁnd a directed spanning tree of a directed graph? (b) If so. (a) Does this algorithm output a directed spanning tree of a directed graph? (b) If so. . Suppose Kruskal’s algorithm is used to construct a directed spanning
. Choose e1 if u1 < u2 . Explain how to implement this algorithm in time O(e log n). then the following strategy is used. Greedy Algorithms
Suppose that there are exactly two edges that have the same cost.
A directed spanning tree of a directed graph is a spanning tree in which the directions on the edges point from parent to child. v1 ) and e2 = (u2 . Suppose the vertices are numbered 1. Consider the following algorithm. If not. For each vertex v = 1. 9. Design an algorithm for the maxcost spanning tree. . where u1 < v1 and u2 < v2 .
Chap. will it ﬁnd a directed spanning tree of minimum cost? 459. .
456. 2.108 454. then add it to the spanning forest. Otherwise. Suppose the graph has vertices numbered 1. consider the cheapest edge with at least one endpoint in the tree that contains v. both algorithms construct the same mincost spanning tree. . Start with a forest containing n trees. Suppose Prim’s algorithm is modiﬁed to take the cheapest edge directed out of the tree. 2. That is. 460.
455. Suppose Prim’s algorithm is modiﬁed as described in Problem 457. . 461. Prove that under these conditions. Repeat the following until a single tree remains.
Show that it is the only mincost Hamiltonian cycle in the graph. ﬁnd a Hamiltonian cycle of minimum total cost.
9. Is it necessarily a Hamiltonian cycle of minimum cost? Justify your answer. Suppose the greedy algorithm does ﬁnd a Hamiltonian cycle. and output the cheapest Hamiltonian cycle found. 465. Prove that it is a Hamiltonian cycle of minimum cost. Suppose this greedy algorithm does output a Hamiltonian cycle. and repeats this until it ﬁnds a node that it has visited before. and the greedy algorithm does ﬁnd a Hamiltonian cycle. Here is another greedy algorithm for the traveling salesperson problem. 464. Suppose the graph has a Hamiltonian cycle. and moves along the edge of minimum cost to the next node. Simply run the vanilla greedy algorithm described above n times. repeating this until the algorithm arrives at a vertex from which all edges lead to visited nodes.5
APPLICATIONS
The problems in this section ask you to apply the greedy method in new situations.
466. even when one exists in the graph. Show that this greedy algorithm may fail to ﬁnd a cycle. A “vanilla” greedy algorithm for the traveling salesperson problem starts at node number 1. Can it be a diﬀerent cycle from the one found by the vanilla greedy algorithm? Justify your answer.
Another greedy algorithm for the traveling salesperson problem proceeds as follows. Traveling Salesperson
109
9. easy to implement. Will this algorithm ﬁnd it? Suppose this greedy algorithm ﬁnds a Hamiltonian cycle. Start at node number 1. 462. it may not be a Hamiltonian cycle. but they are almost always diﬃcult to prove correct. 469. easy to analyze. starting at each vertex in turn. and they are very fast. 467. 9. and move along the edge of minimum cost that leads to an unvisited node. One thing you will notice about greedy algorithms is that they are usually easy to design. Given a directed labeled graph. 468.Sec.
. 463. Show that this greedy algorithm may fail to ﬁnd a cycle.4. Show that even when the greedy algorithm ﬁnds a cycle. Suppose the edge costs are distinct. or can go no further.4
TRAVELING SALESPERSON
The traveling salesperson problem is deﬁned as follows.
Chap. A greedy algorithm for the oneprocessor scheduling problem processes the jobs
. 7. Greedy Algorithms
Given n ﬁles of length m1 . The greedy algorithm puts the ﬁles on the tape in ascending order of size. c. . then for all C ∈ IN. Given C ∈ IN. mn . given that an unlimited number of coins of each type are available. (a) Suppose a1 = 1. .
472. c ≥ 2. assuming that each retrieval starts with the tape rewound. A minimum vertex cover is a vertex cover with the smallest number of vertices in it. if An = {1. 473.110 470. A feasible schedule is a permutation of the jobs such that when the jobs are performed in that order. A greedy algorithm will make change by using the larger coins ﬁrst. w) such that u = v or w = v} until E = ∅ return(U)
1. 5. m2 . ﬁnd the smallest number of coins from An that add up to C. . using as many coins of each type as it can before it moves to the next lower denomination. V := V − {v} E := E − {(u. . 9. 6. where each ai ∈ IN. 3. . then every job is ﬁnished before the deadline. and that ﬁles are to be retrieved in the reverse order. The following is a greedy algorithm for this problem: function cover(V. (b) Show that for all c ∈ IN. cn−1 }. .
The vertex cover problem is deﬁned as follows. A vertex cover of G is a set U ⊆ V such that for each edge (u. for 1 ≤ i ≤ n. We are given a set of n jobs. . . The oneprocessor scheduling problem is deﬁned as follows.
471. each retrieval takes time equal to the length of the preceding ﬁles in the tape plus the length of the retrieved ﬁle. Prove that this is the best order. a2 . . E) U := ∅ repeat let v ∈ V be a vertex of maximum degree U := U ∪ {v}. Let G = (V. c2 . the greedy algorithm always gives a minimal solution to the coinchanging problem. Suppose also that a1 < a2 < · · · < an . 4. Suppose An = {a1 .
Does this algorithm always generate a minimum vertex cover? Justify your answer. . the optimal tapestorage problem is to ﬁnd which order is the best to store them on a tape. . . The coinchanging problem is deﬁned as follows. 2. v) ∈ E. and a deadline di . E) be an undirected graph. either u ∈ U or v ∈ U . an } is a set of distinct coin types. Show that this algorithm does not necessarily generate a solution with the minimum number of coins. Each job i has a start time ti .
We will prove that X must have the same weight as Y . .6
HINTS
430. See Problem 446. x2 . .1)
Let Y = (y1 . y2 . . . .
9. xn ) be the solution generated by the greedy algorithm. . But a more insightful way to do it is to prove that the mincost spanning tree is unique under these conditions. then the schedule produced by this greedy algorithm is feasible. From the algorithm.
. . 471. . and hence has minimal weight too. let k be the least number such that xk = yk . If X = Y . Hints
111
in order of deadline (the early deadlines before the late ones). These two facts should enable you to easily prove that the greedy algorithm does the right thing. yn ) be a solution of minimal weight. If all the xi are 1. . 449.7
SOLUTIONS
430. .
i=1
(9. . . 447. xn so that the length remains the same. . .
j
xi si = S. just play around until you get a counterexample. . xn ) can be changed to an optimal solution (y1 . . Let X = (x1 . Therefore. . 2. n. Change xi to yi and adjust the remaining xi+1 . show that there is only one solution in which the number of coins of each type is less than c. and that the mincost solution must have this form. Otherwise. y2 . xi = 1 for 1 ≤ i < j 0 ≤ xj < 1 xi = 0 for j < i ≤ n. 9. . For part (a). . Show that a greedy solution (x1 . . let j be the smallest number such that xj = 1. Otherwise.
9.Sec. . we are done. . x2 . Show that if a feasible schedule exists. .6. . 446. then the solution is clearly optimal (it is the only solution). This is really a data structures question. yn ) without aﬀecting its length by doing the following for i = 1. This can be proved directly by induction. For part (b).
. Then xk = 0 and yk > 0.
which contradicts the fact that Y is a solution. xk = yk . Then xk = 1. In the other two cases. . Therefore. By the deﬁnition of k. (zk − yk )sk
n
> <
0. yn as necessary to make the total length remain at S. k > j. k < j. Case 2. If yk > xk . zn ).1. Call this new solution Z = (z1 . . k = j. Now that we have established that yk < xk .1)
= >
S+
i=j+1
S. consider the three possible cases: Case 1. . since k = j)
i=k+1
= S + (yk − xk )sk + > S. 9.
(9. Case 3.3)
(zi − yi )si
i=k+1 n
(zk − yk )sk +
(zi − yi )si
i=k+1
=
0. Greedy Algorithms
It must be the case that yk < xk . Suppose we increase yk to xk . Therefore. yk < xk . . .
(9. z2 .2) (9. and so
n j n
yi si
i=1
=
i=1 j
yi si +
i=j+1 n
yi si
=
i=1
xi s i +
i=j+1 n
yi si yi si (by Equation 9. hence Case 3 can never happen. yk < xk as claimed. . since xk = yk . .4)
.112
Chap. Then. yk must be smaller than xk . 0.
n k−1 n
yi si
i=1
=
i=1 k−1
yi si + yk sk +
i=k+1 n
yi si
=
i=1
xi si + yk sk +
i=k+1 n
yi si yi si (by Equation 9. To see this.
This is not possible. and decrease as many of yk+1 . we can return to the proof.
fn . . . . . f2 .
. Suppose we store the ﬁles in order fi1 . m2 . . respectively. .4 and density)
n n
(zi − yi )si wk /sk
i=k+1
=
i=1 n
yi wi +
(zk − yk )sk +
(zi − yi )si
i=k+1
wk /sk
=
i=1
yi wi (by Equation 9. . . whereas only the ﬁrst k − 1 entries of Y are the same as X.
Hence. Therefore. .
Now. . 470.4). fi2 . . But Z looks “more like” X than Y does. n. X must have minimal weight. since Y is a minimalweight solution.7. . . of lengths m1 .
n n
zi wi <
i=1 i=1
yi wi .Sec. . Y and Z have the same weight. mn . 9. Suppose we have ﬁles f1 . in the sense that the ﬁrst k entries of Z are the same as X. . Repeating this procedure suﬃciently often transforms Y into X and maintains the same weight. 2. fin . . Solutions
113
Therefore. .
n k−1 n
zi wi
i=1
=
i=1 k−1
zi wi + zk wk +
i=k+1 n
zi wi
=
i=1 n
yi wi + zk wk +
i=k+1 n
zi wi
n
=
i=1 n
yi wi − yk wk −
i=k+1
yi wi + zk wk +
i=k+1 n
zi wi
=
i=1 n
yi wi + (zk − yk )wk + yi wi + (zk − yk )sk wk /sk +
i=1 n
(zi − yi )wi
i=k+1 n
= ≤
i=1
(zi − yi )si wi /si
i=k+1 n
yi wi + (zk − yk )sk wk /sk + (by Equation 9. . in be a permutation of 1. i2 . Let i1 . . .
. The cost of permutation Π is
n
C(Π) =
j−1
(n − k + 1)mik
k=1
= +
(n − k + 1)mik + (n − j + 1)mij + (n − j)mij+1
k=1 n
(n − k + 1)mik . Greedy Algorithms
mij . 9. . Claim: Any permutation in nondecreasing order of mi ’s has minimum cost. We will prove that it cannot have minimum cost.114 To retrieve the kth ﬁle on the tape. we get instead
n
nmi1 + (n − 1)mi2 + (n − 2)mi3 + · · · + 2min−1 + min =
(n − k + 1)mik . in ) be a permutation of 1. 2. . . . . n that is not in nondecreasing order of mi ’s. .
k=1
which is the righthand side of Equation (9.
fin−1 : mi1 +mi2 +mi3 +· · ·+min−1 fin : mi1 +mi2 +mi3 +· · ·+min−1 +min The lefthand side of Equation 9. . Let Π be permutation Π with ij and ij+1 swapped. Since mi1 . . It remains to prove that this is the minimum cost permutation. mi2 . Therefore.
k=j+2
The cost of permutation Π is
j−1 n
C(Π ) =
(n−k+1)mik +(n−j +1)mij+1 +(n−j)mij +
k=1
(n−k+1)mik . If the total is computed by columns. .
k=j+2
. . min is not in nondecreasing order. . there must exist 1 ≤ j < n such that mij > mij+1 . i2 .
k=1
(9. consider the fi1 : fi2 : fi3 :
following table of costs: m i1 mi1 +mi2 mi1 +mi2 +mi3 . Proof: Let Π = (i1 .5). The greedy algorithm picks ﬁles fi in nondecreasing order of their size mi . fik .5)
To see this. . costs of retrieving them all is
n k n k j=1
Chap. the cost
m ij =
k=1 j=1
(n − k + 1)mik . .5 adds this table by rows.
Therefore. These problems were suggested to the author by Farhad Shahrokhi in 1993. Comments
115
Hence.
Therefore. 456. See also Problem 95.
.Sec. This is true of any Π that is not in nondecreasing order of mi ’s. In general.8. and so Π cannot be a permutation of minimum cost. C(Π) − C(Π ) = (n − j + 1)(mij − mij+1 ) + (n − j)(mij+1 − mij ) = mij − mij+1 > 0 (by deﬁnition of j). 9. 471.
9. This question is designed to illustrate the point that a greedy algorithm is better in some (but not all) cases. the minimumcost permutation must be in nondecreasing order of mi ’s. a dynamic programming algorithm is the best solution to this problem (see Problem 415). C(Π ) < C(Π).8
COMMENTS
436.
3. 474. and combinations. Our aim is to process all bitstrings of a ﬁxed length.
475. that is. Section 10. deduce the running time of the procedure. that a call to binary(n) processes all binary strings of length n.Chapter 10
Exhaustive Search
Often it appears that there is no better way to solve a problem than to try all possible solutions.1
STRINGS
A bitstring is an array of n bits.6 covers the application of these tools to various interesting exhaustive search problems. The following is a recursive algorithm for generating bitstrings. binary(m − 1)
1. Section 10. Bitstrings are often used to represent sets (the term usually used is the bitvector representation of a set).5 compares exhaustive search algorithms to backtracking. It may actually be practical in real situations if the problem size is small enough. sometimes not. procedure binary(m) comment process all binary strings of length m if m = 0 then process(C) else C[m] := 0. is almost always slow. How many assignments to array C are made (that is.
Show that the algorithm is correct. The approach taken to exhaustive search in this chapter is very diﬀerent from that taken by most textbooks. permutations.4 cover some simple algorithms that form our basic tools. Most exhaustive search problems can be reduced to the problem of generating simple combinatorial objects. binary(m − 1) C[m] := 1. 2. most often strings. We assume the existence of a procedure process(C) that processes a bitstring stored in an array C. but sometimes it is better than nothing. The most commonly used algorithmic technique used in exhaustive search is divideandconquer (see Chapter 7). This approach. how often are lines 2 and 3 executed)? Hence.
10. called exhaustive search. Sections 10.1–10. Sometimes backtracking is faster. 116
.
Strings
117
The following algorithm generates all strings of n bits in an array C[1. the number of times that line 4 is executed) is 2n − 1. 10.
. 2. Hence. 478. 477. or viceversa. for i := 0 to n + 1 do C[i] := 0 while C[n + 1] = 0 do process(C) increment(C) procedure increment(C) i := 1 while C[i − 1] = 0 do Complement C[i] i := i + 1
5.n] containing each string of n bits. so C is actually an array indexed from 0 to n + 1. the number of times that line 7 is executed) is 2n+1 − (n + 2). The following algorithm generates all of the bitstrings of length n in minimalchange order. Prove that for all n ≥ 1. 3. 4. that procedure process(C) is called once with C[1.1.
Prove that this algorithm does generate the 2n bitstrings of length n in minimalchange order. that is.
A minimalchange algorithm for generating bitstrings generates them in such an order that each string diﬀers from its predecessor in exactly one place. deduce the running time of the algorithm. The operation “complement a bit” used in line 8 means change it from a 0 to a 1. 1. 2. The array b is initially set to all zeros. 5. 479. the number of times that a bit of C is complemented (that is. 8. deduce the running time of the algorithm.. 476. and the bitstrings are generated with a call to generate(n). the number of times that a bit of b is complemented (that is.n].
Prove that the algorithm is correct. A sentinel is used at both ends of C. Prove that for all n ≥ 1. It works by treating C as a binary number and repeatedly incrementing it through all 2n possible values. Hence.. 3. procedure generate(n) if n = 0 then process(b) else generate(n − 1) Complement b[n] generate(n − 1)
1.Sec. 6. 4. 7.
) Prove that your algorithm for Problem 483 is correct. 6. . 482.. 0022. . 481. procedure permute(n) if n = 1 then process(A) else B := A permute(n − 1) for i := 1 to n − 1 do swap A[n] with A[1] permute(n − 1) A := B cyclically shifted one place right
Prove that this algorithm calls process(A) with A containing each permutation of n entries exactly once. Analyze the running time of your algorithm for Problem 483.n] contains n distinct values. 1. For example.
. 0021. . 2221. 4. then your algorithm should generate the strings 0000. Prove that your algorithm for Problem 480 is correct.
10. 0011. 7. 5. 2. . .2
PERMUTATIONS
Consider the problem of exhaustively generating all permutations of n things. 483.. if k = 2. 3. 0001. . k − 1}. 0012. Exhaustive Search
A kary string is a string of digits drawn from {0. (See also Problem 480. 1.. 484. Design an optimal exhaustive search algorithm for generating all kary strings of length n in lexicographic order in time O(kn ). Suppose A[1. 480. 0100. 0002. if k = 3 and n = 4. then a kary string is simply a binary string. . 2222. 485. Design a minimalchange exhaustive search algorithm for generating all kary strings of length n.
A minimalchange algorithm for generating kary strings generates them in such an order that each string diﬀers from its predecessor in exactly one place. .n] containing each permutation of its contents. The following algorithm generates all permutations of A[1.n] in the following sense: A call to procedure permute(n) will result in procedure process(A) being called once with A[1. 0020. Analyze the running time of your algorithm for Problem 480. 0010. For example. 10.118
Chap. 486.
490.n].. call permute(1).Sec. 4. To generate all permutations of any n distinct values in A[1. 1. 2. Use this observation to eliminate the need for the array B. How many swaps are made by this algorithm. 11. 4. 9..2. It is known as Heap’s algorithm (Heap [35]). 1. 2. 4. 14. 3. 12.
Show that the eﬀect of line 7 is the same as performing a cyclic shift of A[2. Permutations
119 What is the running time of this algorithm?
487.
. 488. 5. procedure permute(i) if i = n then process(A) else permute(i + 1) for j := i + 1 to n do swap A[i] with A[j] permute(i + 1) swap A[i] with A[j]
Prove this algorithm correct. 489. 13.
10. 3. 10. 8. 7.n]. each permutation generated diﬀers from the previous one by having just two values swapped. procedure permute(n) for i := 1 to n do A[i] := i if n is odd then oddpermute(n) else evenpermute(n) procedure oddpermute(n) if n = 1 then process else evenpermute(n − 1) for i := 1 to n − 1 do swap A[1] and A[n] evenpermute(n − 1) procedure evenpermute(n) if n = 1 then process else oddpermute(n − 1) for i := 1 to n − 1 do swap A[i] and A[n] oddpermute(n − 1)
5. as a function of n? What is its running time?
The following is a minimalchange algorithm for generating permutations. Does this aﬀect the running time of the algorithm?
Consider the following algorithm for generating all permutations. That is. 6.
it generates all permutations in minimalchange order. 493. To use it. assume that the n items to be chosen from are the natural numbers 1. Show that T (n.
− r. Hence. . 2. r). c) comment choose c elements out of b.
Chap. call procedure choose(1. r r Show that for all 1 ≤ r ≤ n/2. procedure choose(b. We will.
. A[i] > A[i + 1] for all 1 ≤ i < n.3
COMBINATIONS
Consider the problem of generating all combinations of r things chosen from n. for some 0 ≤ r ≤ n. when process(A) is called. 4. n! − 1 swaps).
10. 3.r] containing each combination. that is..
496. Show that Heap’s algorithm uses the minimum number of swaps. Let T (n. . n. that is. c − 1)
1. Prove that the combinations generated by this algorithm are in descending order.
Prove that this algorithm is correct. The following algorithm generates all combinations of r things drawn from n in the following sense: Procedure process(A) is called once with A[1.
n r
495. . What is its running time?
492. r) be the number of assignments to array A. (that is. The following is a minimalchange algorithm for generating combinations.n if c = 0 then process(A) else for i := b to n − c + 1 do A[c] := i choose(i + 1. It is used by calling up(1. deduce that the algorithm runs in time O r n . 494. T (n. . r).. 10. deduce
A minimalchange algorithm for combinations generates them in such a manner that each choice of r objects diﬀers from the previous one by having one object removed and replaced by another. r) ≤ 2 that the algorithm is optimal for r ≤ n/2. Hence. 2.120 491. without loss of generality. r) ≤ r n . Exhaustive Search
Show that the algorithm is correct.
that is. 4. c − 1) procedure down(b. . 3.4. where {v1 . n}. 7.n − 1] one place right
Let M (n) be the number of perfect matchings of n values. 6. 3. . . 2. (v3 . (vn−1 .
498. c − 1) else down(i + 1. deduce its running time. procedure match(n) if n = 2 then process(M ) else match(n − 2) for i := n − 2 downto 1 do swap A[i] with A[n − 1] match(n − 2) circular shift A[1.
procedure up(b.
10. . a[i] < a[i + 1] for all 1 ≤ i < n. How many assignments to a does the algorithm make? Hence.
Prove that the algorithm is correct.Sec. . c) if c = 0 then process(a) else for i := b to n do a[r − c + 1] := i if i is odd then up(i + 1. Prove that the combinations generated by this algorithm are in ascending order. vn ). 6. and the values within each pair are listed in a diﬀerent order. 500. that is. 10. 4.. where n is even: 1. The following is an algorithm for generating all perfect matchings of size n in an array M [1. . .4
OTHER ELEMENTARY ALGORITHMS
If n is even. Note that two perfect matchings are deemed to be identical even if the pairs are listed in a diﬀerent order. . v2 . then
n/2
M (n) =
i=1
(2i − 1). v4 ). a perfect matching of size n is an unordered sequence of unordered pairs (v1 . c) if c = 0 then process(a) else for i := n downto b do a[r − c + 1] := i if i is odd then up(i + 1. c − 1) else down(i + 1.
. 497. Other Elementary Algorithms
121
1. c − 1)
5. Show that if n is even. v2 ). . . that procedure process(a) is called with a containing a sequence of combinations in minimalchange order. 499.. vn } = {1.n]. . 5. when process(a) is called. 2. 8. . 2.
To use the algorithm. c) comment process all binary strings of length m if m = 0 then process(C) else C[m] := 0. What is M (n) when n is odd? How can the above algorithm be used to generate all perfect matchings when n is odd. So.
.
Chap.
10.1 can be modiﬁed easily to give a backtracking algorithm for generating all bitstrings of length n with k or fewer bits. If n is odd. each of which can hold at most c marbles. Prove that the algorithm is correct. Analyze the running time of your algorithm for Problem 504. for example.5
BACKTRACKING
Backtracking is a type of exhaustive search in which the combinatorial object is constructed recursively. c) if c < k then C[m] := 1. binary(m − 1. 10. For example. 502. c + 1)
1. 506. Use divideandconquer to design an algorithm that generates all of the ways of distributing the marbles into the jars so that no jar holds more than c marbles. 503. and c = 5. 2. 3. the deﬁnition of perfect matching is similar except that there is a leftover value that is not paired with any other. Exhaustive Search
Show that the algorithm runs in time O(M (n)). the recursive algorithm for generating bitstrings from Section 10. 0). procedure binary(m.122 501. with m = 10. and the recursion tree is pruned. binary(m − 1. call binary(n. where m ≤ cn. n = 3. Your algorithm must print the number of marbles in each jar for each of the diﬀerent ways of distributing the marbles. 504. in time O(M (n))?
Suppose you have m marbles and n jars. the output of your algorithm would be 0 1 1 2 2 2 505. recursive calls are not made when the part of the current object that has already been constructed cannot lead to a valid or optimal solution. 5 4 5 3 4 5 5 5 4 5 4 3 3 3 3 3 4 4 2 3 4 5 1 2 5 4 3 2 5 4 4 4 4 5 5 5 3 4 5 0 1 2 3 2 1 5 4 3 5 5 5 3 4 5 2 1 0
Prove that your algorithm for Problem 504 is correct. that is.
Sec. that a call to binary(n. then the recursion tree is pruned to prevent more ones from being put in. that is. how often are lines 2 and 3 executed)? Hence. 10. 509. 3. 4. 511.1–10.
. deduce the running time of the procedure.6.. Is it asymptotically optimal?
10. how often are lines 3 and 4 executed)? Hence. that a call to permute(n) processes all permutations of length n. all entries in U are assumed to be true. deduce the running time of the procedure. 5.4. and some do not. 510. procedure permute(m) comment process all permutations of length m if m = 0 then process(A) else for j := 1 to n do if U [j] then U [j]:=false A[m] := j.n] with nary strings.n].6
APPLICATIONS
The problems in this section are to be solved by reduction to one of the standard exhaustive search algorithms from Sections 10. 507. Initially. It ﬁlls an arrayA[1. The algorithm is used by calling permute(n). and prunes away nonpermutations by using an additional array U [1. does G contain a clique on r vertices? Devise an algorithm for the clique problem that runs in time .
Show that the algorithm is correct. How many assignments to array C are made (that is. U is a Boolean array. The clique problem is the following: Given a graph G and a natural number r. 2. 0) processes all binary strings of length n with k or fewer bits. Show that the algorithm is correct. with U [i] set to true if i has not yet been used in the permutation. Brieﬂy describe how to improve the running time by a factor O r2 n r of Ω(r). Is it asymptotically optimal?
The following is a backtracking algorithm for generating permutations.. How many assignments to array A are made (that is. Applications
123
Note that the pruning is done in line 3: If enough ones have been put into C already. 508. 6. that is. Some of them require backtracking. A clique is an induced complete subgraph. permute(m − 1) U [j]:=true
1.
j) to be the smallest number n such that every graph on n vertices has either a clique of size i or an independent set of size j. A peg may jump over its neighbor in the horizontal or vertical direction if its destination is unoccupied.
Chap. 10. has been a source of entertainment for over a century. The weight of a pairing of student i with student j is deﬁned to be P [i. j] · P [j. j. Design an algorithm for the pairing problem that runs in time proportional to the number of pairings.. if such a sequence of moves exists. Suppose you have n students in a class and wish to assign them into teams of two (you may assume that n is even). is called the (n2 − 1)puzzle.
516. based on an n × n grid. The peg that is jumped over is removed from the board. invented by Sam Loyd. j] contains the preference of student i for working with student j. ﬁnds R(i.n] in which P [i. Design an 2 algorithm for for ﬁnding closed knight’s tours that runs in time O(8n ). Pegs are moved as follows. The generalized HIQ puzzle has n(n + 4m) holes arranged in a symmetrical crossshape as shown in Figure 10. A peg (indicated by a square) is placed in every hole except the center one (indicated by a circle). Your algorithm should run in time O(n2 + 4k ). given i. The following puzzle is called HiQ. The 15puzzle. j ∈ IN. Figure 10. the start point is reachable from the ﬁnish point a single knight’s move). j) exists for all i. given some conﬁguration of the (n2 − 1)puzzle. i].2. The puzzle consists of 15 square tiles numbered 1 through 15. 1.3.124 512. arranged in rowmajor order in a square grid with a onetile gap. Devise an algorithm that. The aim is to produce a sequence of jumps that removes all pegs from the board except for one. for example. j). Design an algorithm that
513. as we have seen already. is a cyclic version of an open knight’s tour (that is. Design an algorithm that. The aim is to randomize the puzzle and then attempt to return it to the initial conﬁguration by repeatedly sliding an adjacent tile into the gap (see. You are given a board in which 33 holes have been drilled in the pattern shown in Figure 10. Analyze your algorithm.. where k is the number of moves required for solution of the puzzle.1).
515. The general form of this puzzle. You are given an integer array P [1. ﬁnds a sequence of moves that return the puzzle to the initial conﬁguration. Exhaustive Search
Deﬁne the Ramsey number R(i. Ramsey’s Theorem states that R(i. such that every square of the board is visited exactly once.
. Design an algorithm for ﬁnding open 2 knight’s tours that runs in time O(n2 8n ). An open knight’s tour is a sequence of n2 − 1 knight’s moves starting from some square of an n × n chessboard.
514. for any m and any odd n. The pairing problem is the problem of pairing each student with another student in such a way that the sum of the weights of the pairings is maximized. A closed knight’s tour.n. Analyze your algorithm.
.Sec. The nqueens problem (also known as the peaceful queens problem) is the problem of determining the number of ways that n queens can be placed on an n × n chessboard so that no queen threatens any other (for example.2.6. Let p = n(n + 4m).
The initial configuration
After peg jumps peg
Figure 10. 517. The initial conﬁguration of the HiQ puzzle (left). and the same conﬁguration after a single move (right). Applications
125
6 1 5 13 10 14 9
2 4 3 8 7 11 15 12
6 2 4 1 5 3 8 13 10 7 11 14 9 15 12
1 6 2 5 3 13 10 7 14 9 15
4 8 11 12
1 6 5 13 10 14 9
2 4 3 8 7 11 15 12
1 2 4 5 6 3 8 13 10 7 11 14 9 15 12
1 2 3 4 5 6 7 8 13 9 10 11 14 15 12
1 2 3 4 5 6 7 8 13 10 11 14 9 15 12
1 2 3 5 6 7 13 10 14 9 15
4 8 11 12
1 2 3 4 5 6 8 13 10 7 11 14 9 15 12
1 5 13 14
2 4 6 3 8 10 7 11 9 15 12
1 2 3 4 5 6 7 8 13 9 10 11 14 15 12
1 2 3 5 6 7 9 10 13 14 15
4 8 11 12
1 2 3 5 6 7 9 10 13 14 15
4 8 11 12
1 2 3 4 5 6 7 8 11 9 10 13 14 15 12
1 5 9 13
2 3 4 6 7 8 10 11 14 15 12
1 5 9 13
2 3 4 6 7 8 10 11 12 14 15
Figure 10. Devise an exhaustive search algorithm that solves the nqueens problem. and empty holes are indicated by a circle. Solving the 15puzzle.
solves the HiQ puzzle.4 shows one way of placing eight peaceful queens on an 8 × 8 chessboard).1. Figure 10. 10. Your algorithm should run in time O((4p)p−2 ). Pegs are indicated by a square.
Exhaustive Search
m
n
m
Figure 10.126
Chap. The generalized HiQ puzzle.3. or an ice cream cart can be reached by walking along only one street to the next intersection.
.
The IC Ice Cream Company plans to place ice cream carts at strategic intersections in Dallas so that for every street intersection. . either there is an ice cream cart there.4.
518. . Analyze your algorithm.
Figure 10. and can make a proﬁt if it pays at most $d in taxes. You are given a map of Dallas in the form of a graph with nodes numbered 1. the city will levy $ci in taxes. 10. Write a backtracking algorithm that outputs all of the possible locations for the k carts that require less than $d in taxes. For each ice cream cart at intersection i. The IC Ice Cream Company has k ice cream carts. n representing the intersections. . and undirected edges representing streets. 2. One way of placing eight peaceful queens on an 8 × 8 chessboard. .
v) ∈ E such that u ∈ U . either v ∈ U . the shaded vertices in the graph of Figure 10. determines whether G has a Hamiltonian cycle in time O(dn ). the output should be 1+2+3+4 1+2+7 1+3+6 1+4+5 1+9 2+3+5 2+8 3+7 4+6 10
520. If you were given a graph and asked to ﬁnd all Hamiltonian cycles in it. prints all Hamiltonian cycles in G.
.
521. The outdegree of a graph is the largest outdegree of its vertices. given a directed graph G. given a directed graph G.
Design a backtracking algorithm that inputs a natural number C. Devise an algorithm that. If G has n vertices. 10.
522. For example. the output should be 1+2+3 1+5 2+4 6 and if C = 10. Analyze the running time of your algorithm. If G has n vertices.6. Devise a backtracking algorithm that. if C = 6. and outputs all of the ways that a group of ascending positive numbers can be summed to give C. Analyze the running time of your algorithm. The outdegree of a vertex v in a graph is the number of edges directed out of v. your algorithm must run in time O(n!).
523. Applications
127
519. given a directed graph G of outdegree d. prints all Hamiltonian cycles in G. For example. E) is a set U ⊆ V of k vertices such that for all v ∈ V .
A dominating set of size k in a graph G = (V. or there exists an edge (u. Be sure to analyze the running time of your algorithm.Sec.5 form a dominating set.
Devise an algorithm that. which algorithm would you use. your algorithm must run in time O((n − 1)!). the algorithm from Problem 520 or the algorithm from Problem 522? Explain your answer.
given G and k.k = 0.
Design an algorithm that ﬁnds all n × n weighing matrices in time 2 O(n3 3n ).
Describe how to reduce the running time of your algorithm to
.k mj. Exhaustive Search
Figure 10.1 m2.1 m1. 0. k) in terms of n and k. . ﬁnds a dominating set of size k.
k=1
527. That m1.n . m1.2 where for all 1 ≤ j ≤ n and all 1 ≤ i < j.
A weighing matrix is an n × n matrix with scalar product of any two rows is zero. Analyze your algorithm. A dominating set (shaded vertices). Describe how to improve the running time of your algorithm to O(nD(n. . O(n3n ).
524.
Let D(n. k)).128
Chap.1 mn. . a matrix . if one exists. . k)). Devise an algorithm that.2 m2.5.
2
528. . Analyze the running time of your algorithm..n
525. .n .
526. Write a formula for D(n. . k) be the number of candidates for a dominating set in an exhaustive search algorithm. .
n
mi. Justify brieﬂy the running time of your modiﬁed algorithm. entries from {−1. 10. 1} such that the is. mn.2 M = mn. m2. . Your algorithm should run in time O(n2 D(n.
if n = 4. 531.n. Applications
129
Preference Matrices:
P 1 2 3 4
1 2 3 4 2 1 1 2 3 1 2 3 1 4 3 2 1 3 4 1 1
N 1 2 3 4
1 2 3 4 2 2 4 4 1 3 3 3 1 2 1 2 3 3 1 3 4 4 2 1 4 3
Candidate Matching:
Pilot:
Navigator: 1 Weight:
P[1. 1.2+1.
. Design an algorithm that.6. computes the length of the consecutive series of postage values that can legally be realized under these rules starting with the value 1.. 1. j] · N [j.. m = 5.3]N[3. You have an integer array P [1.n] and the value of m.4]N[4..4 = 12+6+3+8 = 29
Figure 10.. each a positive integer.6. j] is the preference of navigator i for pilot j.6 shows an example of the preference arrays. and the sum of the weights of the pairs in this matching: 529.n] in which P [i. The Fussytown Post Oﬃce has n diﬀerent denominations of stamps.. and the sum of the weights of the pairs in this matching (bottom). a candidate matching (center).n.3+2. then all postage values from 1 to 71 can be realized. given n diﬀerent postage stamp values in an array P [1. and 21. 4. and the stamps have face value 1. The postage stamp problem is deﬁned as follows. Fussytown regulations allow at most m stamps to be placed on any letter. The preference matrices P and N (top). a candidate matching. Your algorithm should run in time O(nm ).n] in which N [i. 530. For example. Figure 10.Sec. The ﬂightassignment problem is the problem of pairing each pilot with a navigator in such a way that the sum of the weights of the pairings is maximized.3]+P[4. 12. The weight of a pairing of pilot i with navigator j is deﬁned to be P [i. Design an algorithm for the ﬂightassignment problem that runs in time O((n + 1)!). Describe how to improve the running time of your algorithm from Problem 529 to O(n!).4+3.2]+P[3. Analyze your algorithm. i]. and an integer array N [1. j] is the preference of pilot i for navigator j.4] = 3. 10.
Suppose you are given n pilots and n navigators.2]N[2.1]N[1.1]+P[2.
The 4ominos. or more precisely. 534.n].
. 489. Suppose that A[i] = i for 1 ≤ i ≤ n. a call to evenpermute(n) • processes all permutations of the values in A[1. Trace the algorithm by hand to see what it generates when n = 1. computes the m postage stamp values that can realize the longest consecutive run of postage values starting at 1. Your induction hypothesis should look something like the following: When n is even.7
HINTS
487. and pretend that procedure process(A) prints the contents of A. Analyze your algorithm. The obvious choice of recurrence is quite diﬃcult to solve. Determining and describing this permutation is the diﬃcult part. given n and m. cycles A[1] through all possible values from 1 to n. Modify your algorithm to remove all rotations and reﬂections. and • ﬁnishes with the entries in A permuted in some order in which. The right choice can make it easy. 4. 10.. if repeated n − 1 times. This should give you some intuition as to how it works. 533. For example. Be very careful how you choose your recurrence relation. Exhaustive Search
Figure 10.7. 3.n]. Design an algorithm that runs in time O(3n ) to generate all nominos. and • ﬁnishes with the ﬁrst and last entries in A swapped.7 shows all of the 4ominos. 491..
532. what your induction hypothesis should be. When n is odd. 2. and how the basic structure of your proof should look.130
Chap. including all rotations and reﬂections.
Design an algorithm that. Figure 10.
A polyomino. See the hint for Problem 489. a call to oddpermute(n) • processes all permutations of the values in A[1. an nomino is a plane ﬁgure made up of n unitsize squares lined up on a grid.
10.
You may need to use the identity from Problem 23. Line 6 can be done in time O(n). See Problem 483.
. Reduce the problem to that of generating combinations. T (n) = (c + d)n! − d.
10. 527.Sec. for some c. Let T (n) be the running time of permute(n). 517. Think about dominating sets. Solutions
131
502. Use a minimalchange algorithm. 526.
Hence. 10. The following algorithm prints all of the Hamiltonian cycles of a graph G. 518. 528.8
SOLUTIONS
487. T (n) = nT (n − 1) + d(n − 1). 520.718 (where e is the base of the natural logarithm). Now suppose the hypothesis is true for n. You will need to use the fact that n i=1 i! = e − 1 ≈ 1.
T (n + 1) = = =
(n + 1)T (n) + dn (n + 1)((c + d)n! − d) + dn (c + d)(n + 1)! − d. since T (1) = c and (c + d)1! − d = c. 525. which is linear in the number of permutations.8. The running time gives away the technique to be used.718(n + 1)!. It is possible to solve this problem in time O(n!). and take advantage of Section 10. Use the algorithm from Problem 480. 510. and for n > 2. the running time is O(n!). Therefore.3. The answer is at most 1. The proof is by induction on n. We claim that for all n ≥ 1. Then. d ∈ IN. Reduce the problem to that of generating permutations. 520. T (1) = c. The claim is certainly true for n = 1.
j) ∈ E then missing:=missing+1 procedure iprocess(P. x. x. return(missing = 0)
.” We initialize the array and count when processing the ﬁrst permutation. 10. The solution is similar to the solution of Problem 520. j) D[i] := (i. and a count of missing edges. x. P [(y + 1) mod n]). P [(x + 1) mod n]). P [y]).n − 1] do process(P ) procedure process(P ) if hamiltonian(P ) then print(P ) function hamiltonian(P ):boolean ok:=true for i := 0 to n − 1 do ok:=ok ∧ (P [i].2. P [(i + 1) mod n]) ∈ E return(ok) We use an optimal permutation generation algorithm such as one of the algorithms from Section 10. called “missing. P [x]). y) procedure edge(i. iedge(P [x].n − 1]. A minimal change to P requires a minimal change to the array and the count. j) ∈ E if (i. the search takes time O(n · n!). y) then print(P ) function ihamiltonian(P. where D[i] is true iﬀ (P [i].2) to generate permutations. Exhaustive Search
for each permutation P [0.. but we use a minimalchange algorithm such as Heap’s algorithm (see Section 10. 521. x. Since function hamiltonian(P ) takes time O(n).132
Chap.. iedge(P [(y − 1) mod n]. The running time can be reduced to O(n!) by observing that all of the permutations can begin with vertex 1. y) if ihamiltonian(P. Procedure process is exactly as in the solution to Problem 520. P [(i + 1) mod n) ∈ E. for 0 ≤ i < n. iedge(P [y]. We keep a Boolean array D[0. y):boolean iedge(P [(x − 1) mod n]. missing:=0 P := ﬁrst permutation process(P ) for each permutation P in minimalchange order do Suppose P [x] and P [y] were just swapped iprocess(P.
Although an algorithm that runs in time O(n!) is considered very slow. Rothschild. The overhead for generating the permutations is O(n!) with a minimalchange algorithm such as Heap’s algorithm. this running time can be reduced by a factor of Ω(n) by observing that all of the permutations can begin with vertex 1. The algorithm generates the binary reﬂected Gray code (see also Problems 96 and 97). 517. It is not known exactly how many 8 × 8 tours exist. move to the one with the least number of unused legal moves out of it. j) ∈ E) then D[i]:=true. See also Problem 474. and Spencer [31]. 508. but empirical evidence indicates that their algorithm is impractical (Parberry [62]). See also Problems 48 and 369 (and associated comments).9
COMMENTS
478. j) ∈ E) then D[i]:=false. missing:=missing + 1 else if (not D[i]) ∧ ((i. You will ﬁnd a correctness proof there. 10. Therefore. The other n! − 1 permutations require O(1) time each to process. it
. which deal with closed knight’s tours.
10. 862 closed knight’s tours. The algorithm is adapted from Lam and Soicher [51]. 512. Hindrichs. Comments
133 procedure iedge(i. 16] appears to help both the exhaustive search and random walk algorithms ﬁnd solutions faster: When there is a choice of squares to move to. Chapter 7]) describe a neural network for ﬁnding closed knight’s tours. but there is an easier and clearer explanation. and Graham. 507. For more information on Ramsey numbers. 513.Sec. As before.9. Takefuji and Lee [79] (reprinted in Takefuji [78. Another interesting algorithm with a stronger minimalchange property is given by Eades and McKay [23]. See also Problem 475. the total running time is O(n + (n! − 1) + n!) = O(n!). A heuristic attributed to Warnsdorﬀ in 1823 by Conrad. consult Graham and Spencer [32]. missing:=missing − 1
The time needed to process the ﬁrst permutation is O(n). and Wegener [15. Rouse Ball and Coxeter [8] describe an interesting random walk algorithm that they attribute to Euler. j) if D[i] ∧ ((i. Morsy. in which case there are 9. Backtracking for n × n closed knight’s tours is only practical for n = 6 (there are no tours for smaller n). 497.
3 seconds 18. 10.624 time < 0.712 365. The algorithm was implemented in Berkeley Unix Pascal on a Sun Sparc 2.134 n 6 7 8 9 10 11 12 13 14 15 16 17 18 count 4 40 92 352 724 2.8 minutes 11.0 seconds 1.279.815.512 95.104 666. Bernhardsson [11]).184 14.1. Exhaustive Search
Table 10.8 days
Chap. Heuristics for ﬁnding large numbers of solutions (but not necessarily all of them) include Takefuji [78].6 minutes 1.1 hours 2.001 seconds < 0.
.001 seconds < 0.3 hours 9. 518.090.596 2. It is known that there is at least one solution to the nqueens problem for all n ≥ 4 (see.680 14.001 seconds 0.200 73. for example. The number of solutions to the nqueens problem for 6 ≤ n ≤ 18 and the running time required to exhaustively search for them using a naive exhaustive search algorithm.772. and Susic and Gu [77].1 shows the number of solutions to the nqueens problem and the running time required to ﬁnd them using Berkeley Unix Pascal on Sun Sparc 2.034 seconds 0.133 seconds 0. This problem was described to me by Mike Fellows.
is still usable in practice for very small n. Table 10.6 seconds 3.
its children’s children. if any.2). since an important issue in choosing a particular data structure is the availability of eﬃcient algorithms for creating. Missing leaves. and the unionﬁnd data structure. Secondly. since the proper organization of data is often the key to developing eﬃcient algorithms. you must understand algorithms in order to design data structures. Figure 11.1
HEAPS
A heap is a binary tree with the data stored in the nodes. you must understand data structures in order to design algorithms.
11. for example. It has two important properties: balance and structure. binary search trees. 2–3 trees. 535. Firstly.1. but I prefer to teach them together. are on the last level at the far right (see. Balance means that it is as much like a complete binary tree as possible. Figure 11.Chapter 11
Data Structures
Algorithms and data structures are obviously very closely related. etc. Structure means that the value in each parent is no greater than the values in its children (see. and accessing it. Prove that the value in each node is no greater than the values in its descendants (that is. here are some problems on advanced data structures. principally heaps.1). its children.)
Figure 11. Some schools separate algorithms and data structures into separate classes. AVL trees. for example. In case your algorithms class requires it. A balanced binary tree.
135
. updating.
Halving is the process of taking a set S of integers and separating it into two halfsized sets S1 . 9. Suppose a heap is stored in an array H in the standard fashion (see Problem 537). Prove that a list of the array elements H[1]. Data Structures
3 9 11 20 10 5 24
21 15 30 40 12
Figure 11. store its children in H[2k] and H[2k + 1].2. You are to draw the heap (as a binary tree) at each of the major steps in the construction.. 537. 12. 540. given parameters A. 8.
538. and for every node stored in H[k].
Write a procedure that. followed by (b) insert the value 0. 2. store the root in H[1]. 6.
536. Show that halving is equivalent to ﬁnding the median. A heap. that is:
543. 11. .136
Chap. as in heapsort. Suppose a heap is stored in an array H in the standard fashion. Devise a divideandconquer algorithm (see Chapter 7) for building a heap stored in an array H[1. H[2]. (a) Inserting the items in the order given into an empty heap.
541. .
539.2.
. 16.
Construct a heap containing the items 10. H[n] form a breadthﬁrst traversal of the heap.
Prove by induction that a heap with n vertices has exactly n/2 leaves. . where all of the values in S1 are smaller than all of the values in S2 . and i. n. That is. and redraw the heap and its array representation after each of the following operations have been applied in sequence: (a) deletemin. . and 1 ≤ i ≤ n.n] in O(n) time. where A is the array representation of an nelement heap. Prove that an nnode heap occupies the ﬁrst n contiguous entries of H.
542. (b) Building the heap from the bottom up. 1. Give the array representation heap shown in Figure 11. S2 . 3. deletes the element A[i] from the heap in O(log n) time.
which is processed in a similar fashion.Sec. Figure 11. and all of the values in one sibling are smaller than the values in the other (it doesn’t matter which is the “smaller” sibling). and the largest p values into d. Sequential algorithms for parallel heaps are fairly straightforward: An insert operation is processed by ﬁrst placing the p new items in the next unused node d. then the median can be found using T (n) + n/2 − 1 comparisons. . but with kary heaps for any k = 1. and the smallest p values into the other sibling. Deﬁne kary heaps and describe a spaceeﬃcient implementation in an array. Which k minimizes this factor? Why is k = 2 probably best anyway?
546. In particular. . Heaps
137
(a) If the median can be found using T (n) comparisons. For example.3 shows a parallel heap with p = 4. Heapsort works not only with binary heaps.
. Combine the values in d and its sibling (if it has one). Node r then becomes the dirty node. Combine the values in d and its parent r.1.. 545. and the deletemin operation deletes the p smallest values in the priority queue and returns them in no particular order. and deletemin and deletemax in O(1) time. . 544. Figure 11. What familiar sorting algorithm is kary heapsort for k = 1? For k > 1. and place the smallest p values into r. 547. In a kary heap. describe in detail how to get from any heap element to its parent and its k children. 3. then a set of size n can be halved with T (n) + n − 1 comparisons. Devise a data structure that supports the following operations on an nelement set: insert in time O(log n). but with p values per node. each node has at most k children. It also has the property that all of the values in a node are smaller than the values in its children. which we will call dirty. Write pseudocode for kary heapsort.
The standard heapsort algorithm uses binary heaps. 2. analyze the number of binary comparisons used by kary heapsort in terms of n and k. heaps in which each node has at most two children.
A parallel priority queue is a priority queue in which the insert operation inserts p values. 548.4 shows an insertion into a 4node parallel heap with p = 4. (b) If a set of size n can be halved using T (n) comparisons. and place the largest p values into the sibling that had the largest value between them. determining the coeﬃcient (depending on k) of the most signiﬁcant nterm. For example. A parallel heap is a data structure with the same tree structure as a heap. 11. that is.
2
AVL TREES
A binary search tree is a binary tree with the data stored in the nodes. it results in a parallel heap. heap.138
Chap. which is processed in a similar fashion. Node s then becomes the dirty node. 551. Data Structures
6 1 2 5 19 15 20 25 30 40
8
9
32 26 27 45
18 25 22 16
Figure 11. 11. 552. that is.3. and the smallest p values into the other sibling. The following two properties hold: • The value in each node is larger than the values in the nodes of its left subtree. Call the root dirty. that is. Combine the values in d and its sibling (if it has one). and place the smallest p values into d and the largest p values into s. and place the largest p values into the sibling that had the largest value between them. it results in a parallel
11. • The value in each node is smaller than the values in the nodes of its right subtree. 549.
. Let d be the dirty node. Prove that if the nodes of a binary search tree are printed in inorder. Show that the deletemin algorithm. Analyze the running time of the insert and deletemin algoShow that the insert algorithm is correct. A parallel heap with p = 4. rithms. they come out sorted.
A deletemin operation is processed by removing the values from the root and replacing them with the values from the last used node. 550. Combine the values in d with those of its smallest child s.
4. An insertion into a 4node parallel heap with p = 4. 11.3.Sec.23.10) 6 1 2 5 20 25 30 40 19 15
Create a new node 6 1 2 5 20 25 30 40 19 15
8
9
8
9
32 26 27 45 Combine with parent 6 1 2 5 10 3 20 23 50 30 25 40 19 15
32 26 27 45
50 3 10 23
Combine with sibling 6 1 2 5 10 3 20 23 25 26 27 30 5045 32 40 19 15
8
9
8
9
32 26 27 45
Combine with parent 3 1 2 10 6 20 23 25 26 27 30 5045 32 40 8 5 19 15
Combine with sibling 3 1 2 15 19 20 23 25 26 27 30 5045 32 40 5
9
10 8 6 9
Figure 11.2.
. The dirty node is shaded. AVL Trees
139
Insert (50. Heavily outlined nodes have been combined.
5.
An AVL tree is a binary search tree with one extra structure condition: For every node.
100 50 30 20 60 70 80 150 200
Figure 11. Operations insert. and findmin can be performed in time O(log n).
. delete. This diﬀerence in height is maintained during insertions and deletions using operations called single rotations and double rotations. Insert 5 into the AVL tree shown in Figure 11. 553. member. (the height of an AVL tree is the number of edges in the longest path from the root to a leaf).140
Chap. and explain why two rotations are necessary and suﬃcient. Data Structures
100 50 30 10 40 60 120 150 160
Figure 11.7. An AVL tree. 555.6.6.5. the magnitude of the diﬀerence in height between its left and right subtrees is at most one. Find an AVL tree for which the deletion of a node requires two single rotations. 11. Draw the tree. A second AVL tree. 556. Delete 30 from the AVL tree shown in Figure 11. Insert 55 into the AVL tree shown in Figure 11. indicate the node to be deleted. 554.
indicate the node to be deleted. 6.
. 7. Insert the following elements in sequence into an initially empty 2–3 tree: 5. Give a sequence of insertions and deletions that constructs the 2–3 trees shown in Figure 11. Show how such a tree is constructed. Operations insert.4404 log n. delete. Prove that the height of an AVL tree with n nodes is at most 1. The data are stored in the leaves in sorted order from left to right. 4. indicate the node to be deleted.7. Insertions and deletions are performed using a divideandconquer algorithm. 3.
561. and explain why two rotations are necessary and suﬃcient. the largest values in the left and center subtrees.
11.
Find an AVL tree for which the deletion of a node requires two double rotations. 560.3. for example. 9. Be sure to include all of the information that is kept in the internal nodes. and explain why n rotations are necessary and suﬃcient.3
2–3 TREES
A 2–3 tree in a tree in which every internal vertex has 2 or 3 children. there exists an AVL tree for which the deletion of a node requires n (single or double) rotations. Draw the tree. member and findmin can be performed in time O(log n). 11. Draw the 2–3 tree after every insertion and deletion operation.
557.
559. 1.
558. 2. A third AVL tree. Show the 2–3 tree after every insertion. Delete the element 3 from the resulting 2–3 tree. The internal nodes carry information to allow searching. 2–3 Trees
141
100 50 30 120 150 200 210
Figure 11. Show that for all n ∈ IN. and every path from the root to a leaf has the same length. 0. 8.8 (starting from the empty 2–3 tree each time).Sec.
11. one member per subset. . 564. and • union(x.
There is an algorithm for union and find operations that works in worstcase time O(log n) and amortized time O(log ∗ n). Use a tree: • • • • implemented with pointers and records. y): combine the two subsets that x and y are in. there is an array P [1.. Design an algorithm that can perform a sequence m union and find operations on a Universal set of n elements. in time O(m + n). n} initially partitioned into n disjoint subsets. each child has a pointer to its parent. set elements are stored in the nodes. used? Does this aﬀect the running time of the algorithm in the worst case? Does it aﬀect the amortized running time when path compression is
11. consisting of a sequence of unions followed by a sequence of finds. .5
APPLICATIONS
This section asks you to use your knowledge of data structures to design algorithms for new problems.
.n] with P [i] pointing to the node containing i. .4
THE UNIONFIND PROBLEM
Given a set {1. . There is a standard data structure using a tree. 2. Data Structures
(a)
3:5
(b)
2:5
(c)
2:4
1:2
4:5
6:7 6 7
1:2 1 2
3:4
6:7
1:2 1 2
3:4 3 4
5:6 5 6 7
1 2 3 4 5
3 4 5 6 7
Figure 11. 11. 562. 563. we want to perform the following operations: • find(x): return the name of the subset that x is in.142
Chap.8. Three 2–3 trees.
Suppose we modify the unionﬁnd algorithm so that the root of the shallower tree points to the root of the deeper tree (instead of the root of the smaller tree pointing to the root of the deeper tree).
T ) Insert item x into the set T .
567. delete.
All operations involving nelement sets are to take time O(log n). with no bin holding more than one kilogram. .
All operations on an n item set are to take time O(log n). x) position(S. member. Return true if x ∈ S. . but fragile bins. Your aim is to ﬁnd the smallest number of bins that will hold the n objects. . x) member(S. You are given n metal objects. Return the smallest element of T greater than x Set S to the union of S and T .
Design a data structure for a set in which insertions. 569. . Design an implementation of an Abstract Data Type consisting of a set with the following operations: insert(S. 11. T ) Insert item x into the set T . s } and T = {t1 .
566. Design an implementation of the following Abstract Data Type: a set with the operations: insert(x. . tm } be two sets of integers such that 1 ≤ si .5. tj ≤ n for all 1 ≤ i ≤ and 1 ≤ j ≤ m. Delete x from T . T ) next(x. Return true iﬀ x ∈ T . x) concatenate(S. . You may assume that the set elements are integers drawn from a ﬁnite set {1. false otherwise. Let S = {s1 .
. and next are to take time O(log n). t2 . T ) Insert x into the set S. . Delete the kth smallest element from T . 570. T ) member(x.Sec. Return the number of elements of S less than x. each weighing between zero and one kilogram. x) delete(S. . . . 568. and union is to take time O(n). Design an implementation of the following Abstract Data Type: a set with the operations: insert(x. Delete x from the set S. . The bin packing problem is the following. Applications
143
565. Return true iﬀ x ∈ T . 2. . n}. Design an algorithm for determining whether S = T in time O( + m). deletions. T ) delete(k. T ) union(S. T ) member(x. You also have a collection of large. T ) delete(x. assuming every element in S is smaller than every element of T . Set S to the union of S and T .
The operations insert. s2 . and membership queries can be processed in O(1) time in the worst case.
. inclusive. Return total toll for booths i through j. . Take each of the objects in the order in which they are given. x2 . k). wn and outputting the number of bins needed when using ﬁrstﬁt) in O(n log n) time. xn . j. 11.
. . Devise a data structure in which the following operations can be implemented in O(log n) time: insert(p. . Add $a to the toll for booths i through j. . . t2 . and group inversion is allowed for free. w2 . ·) is a group. Delete toll booth tk . j) update(i. Data Structures
The ﬁrstﬁt heuristic for the bin packing problem is the following. 576.144
Chap. . and “·” is associative. Design an algorithm that implements the ﬁrstﬁt heuristic (taking as input the n weights w1 . Design a data structure that uses O(n log n) space and answers queries in O(log n) time. ﬁnd the smallest value in xi . tn in which toll booth ti collects a toll of $ai . Design a data structure that can perform the following operations in O(log n) time. A is closed under “·”. an }. a) Insert a new toll booth after tk with toll $a. The Dallas Tollway uses a computerized toll collection system. The highway is modeled as as an ordered list of toll booths t1 . . For each object. . and place it into the ﬁrst bin in which it ﬁts. . where k is an integer key and p is an integer priority. a) delete(k) toll(i. j. . 571. . .
574.” 575. insert(k. . ·) is a semigroup.) Devise a data structure that allows you to compute ai · ai+1 · · · aj for any i ≤ j in only O(log n) semigroup operations. . Suppose (A. .
Suppose we are to support a set of ordered pairs (p. . (That is. inclusive.
You are to design a memoryeﬃcient data structure that is constructed from a sequence of n values x1 .
572. and can quickly answer queries of the form: “Given i. k) member(k) delete(k) Insert item with priority p and key k Return item of smallest priority among those with key ≤ k. 573. where A = {a1 . xj . . Design a data structure that uses O(n) space and answers queries in O(log n) time. for 1 ≤ i ≤ n. . Delete all items with key k. Reduce the running time in Problem 571 to O(1) when (A. . scan the bins in increasing order of remaining capacity.
570 You will need a data structure that allows you to ﬁnd the “ﬁrst bin in which it ﬁts” in time O(log n). N (1) = 2).. 11.
562. 537. Devise a recurrence relation for a lower bound for N (h) (start with N (0) = 1. It has two important properties: balance and structure. Let N (h) be the minimum number of nodes in an AVL tree of height h. A kary heap is a kary tree with the data stored in the nodes. Hints
145
577.6
HINTS
535. 551.
Use your solution to Problem 576 to design a data structure to quickly answer leastcommonancestor queries in a tree. store one item separately). Use two heaps A and B. the root of A will be the smallest value and the root of B will be the largest value.
11. Balance means that it is as much like a complete kary tree as possible. B a maxheap. Show that this recurrence has solution N (h) > √ 1+ 5 2
h
. That is.6. Then. if any. and using this data structure you must answer queries of the form: “What is the leastcommonancestor of tree vertices v and w?”
11. 559. your data structure will be constructed from a description of a tree.7
SOLUTIONS
545. 544. Prove it by induction on level. are on the last level at the far right. Prove it by induction on n. each containing n/2 elements (if n is odd. The diﬃcult part is the amortized analysis. A kary heap on n vertices can be eﬃciently implemented in storage using contiguous locations of a linear array A[1. Structure means that the value in each parent is no greater than the values in its children. the vertices are stored in level order. and ensure that for each 1 ≤ i ≤ n/2 . the vertices in it are stored in lefttoright
. and for any level.n]. Make A a minheap. See Problem 543. Missing leaves. It’s the standard unionﬁnd algorithm with path compression. A[i] ≤ B[i].Sec. such that the root of the heap is stored in A[1].
but you’ll seldom see a proof of it. N (0) = 1 and N (1) = 2. So we need (assuming c > 0) c2 − c − 1 ≤ 0.
. This problem is N Pcomplete (see Chapter 12 if you don’t know what this means). We claim that N (h) ≥ ch for some choice of c. with the extra modiﬁcation that the location of the leaf to be deleted is done using the new leafcount ﬁelds in the obvious way (recall that in a 2–3 tree. The claim is true for h ≤ 2 provided c < 2.8
COMMENTS
537. . middle. For h > 2. You’ll see this result in all competent data structures textbooks. by induction. It can be shown that the parent of A[m] is A[ (m + k − 2)/k ] and its children are A[km − k + 2].
567. and to take care of the new ﬁelds during node splitting. and for h ≥ 2. and right children. N (h) > N (h − 1) + N (h − 2) ≥ ch−1 + ch−2 . The largest value of c for which √ this is true is (1 + 5)/2. 11. . . N (h) > √ 1+ 5 2
h
.146
Chap. h< log n √ ≈ 1. n)]. A[min(km + 1. N (h) > N (h − 1) + N (h − 2). Therefore. We use a 2–3 tree augmented with extra ﬁelds in each internal node recording how many leaves are descendants of the left. Data Structures
order.4404 log n. Clearly. and so it is unlikely that there is a polynomialtime algorithm for ﬁnding an exact solution. Let N (h) be the minimum number of nodes in an AVL tree of height h. The insertion is done as normal. 559. the leaves are in ascending order from left to right).
What is the height of an AVL tree with n nodes? We know that n ≥ N (h) > Therefore. log((1 + 5)/2) √ 1+ 5 2
h
. The proof is by induction on h.
11. The deletion code is modiﬁed in a similar way. 570. by the induction hypothesis. with extra code to increment the leafcount of the nodes on the path from the root to the new leaf. The member operations are done as normal. .
Many of them use variants of the satisﬁability problem. problems that have a yes/no answer) that can be solved in polynomial time. then P = N P. P ⊆ N P.” That is.Chapter 12
N P completeness
Deﬁne P to be the set of decision problems (that is. where R ∈ P. Clearly. Then SAT is deﬁned as follows: SAT Instance: A Boolean formula F . N P is the set of existential questions that can be veriﬁed in polynomial time. But here are a few problems that I ﬁnd useful.
12. The truly dedicated student can practice by picking two random N Pcomplete problems from the long list in the back of Garey and Johnson [28] and attempting to prove that one is reducible to the other. and a Boolean formula is a conjunction of clauses. Deﬁne N P to be the set of decision problems of the following form.1
GENERAL
This section contains a few general questions about N Pcompleteness. The standard terminology is used here: A literal is a complemented or uncomplemented variable. But it is known that there are problems in N P with the property that if they are members of P. Listing problems on N Pcompleteness is a little bit redundant. y) ∈ R. c ∈ IN: “Given x. If x is a bit string. written A ≤m B. It is not known whether P = N P. if anything in N P is outside of P. if there is a polynomial time algorithm that maps p every instance x of A into an instance f (x) of B such that x ∈ A iﬀ f (x) ∈ B. They are called N Pcomplete problems. although the consensus of opinion is that probably P = N P. Note that the size of f (x) can be no greater than a polynomial of the size of x. That is. More speciﬁcally. does there exist y with y ≤ xc such that (x. if there is an algorithm that maps every instance x of A into an instance f (x) of B such that x ∈ A iﬀ f (x) ∈ B. a clause is a disjunction of literals. then they are. let x denote the number of bits in x. A problem A is polynomial time reducible to B. A problem A is reducible to B if an algorithm for B can be used to solve A. too. 147
.
Show that 2SAT ∈ P.2
COOK REDUCTIONS
The deﬁnition of “reduction” used in the preamble to this chapter is the one most commonly used in algorithms classes.
N P completeness
Question: Is there a truth assignment to the variables of F that makes F true? 578. 580.
Show that the following problem can be solved in polynomial time: STRONG 3SAT Instance: A Boolean formula F in conjunctive normal form with at most three literals per clause.
Show that the following problem can be solved in polynomial time: PARITY 3SAT Instance: A Boolean formula F in conjunctive normal form with at most three literals per clause. Question: Is there a truth assignment to the variables of F that makes at least two literals in each clause of F true?
582. 12. A problem A is polynomial time Turing reducible to B. Question: Is there a truth assignment to the variables of F that makes F true? Prove that DNF SAT ∈ P. that is. p p p C. Question: Is there a truth assignment to the variables of F that makes an odd number of literals in each clause of F true?
12.
2SAT is the satisﬁability problem with at most two literals per clause. it is called a polynomial time manyone reduction or a Karp reduction (after Karp [43]). A Boolean formula in disjunctive normal form is a disjunction of monoms.
581. A monom is a conjunction of literals. if there is an algorithm for B that calls the p
. Technically.148
Chap. then A 579. There is another deﬁnition of reducibility that you will meet in the literature if you read more about N Pcompleteness. Consider the following problem: DNF SAT Instance: A Boolean formula F in disjunctive normal form. written A ≤T B. if A ≤m B and B ≤m C. ≤m p Prove that “≤m ” is transitive.
Consider the following two versions of the satisﬁability problem. This type of reduction is sometimes called a Cook reduction after Cook [17]. if A ≤T B and B ≤T C. then B is N Pp complete. then p p p C. 585. and a positive integer B. Question: Is there a truth assignment to the variables of F that makes F true? SAT2 Instance: A Boolean formula F . and the second requires the construction of a solution. 12. Show that SAT2 ≤T SAT1. The term selfreducibility. A 586. 587. 584.Sec. Prove that “≤T ” is transitive. and this algorithm makes polynomially many calls to A and runs in polynomial time if the calls to A are not counted.
Prove that if B ≤T C. Cook Reductions
149
algorithm for A as a subroutine. if one exists. if one exists. is used to describe problems for which the opposite is true. A ≤T B. C ∈ N P. ≤T p Prove that if A ≤T B and B ∈ P. then C is p N Pcomplete. Clearly the decision problem is Cook reducible to the solution problem. 583.
. Output: A Hamiltonian cycle in G of cost at most B. p
Consider the following variants of the traveling salesperson problem: TSP1 Instance: A directed graph G with positive costs on the edges. coined by Mike Paterson. p Prove that if B ∈ N P and for all A ∈ N P.2. then A ∈ P. and a positive integer B. Question: Is there a Hamiltonian cycle in G of cost at most B? TSP2 Instance: A directed graph G with positive costs on the edges. The former is the decision problem for satisﬁability. SAT1 Instance: A Boolean formula F . Output: A truth assignment to the variables of F that makes F true. and B is N Pcomplete. that is.
P = N P. pick the tree with the smallest number of interior vertices. use the distributive law to construct an equivalent formula in disjunctive normal form.150
Chap. . n} such that S?
592. . Prove that TSP2 ≤T TSP1. 588. . p Prove that TSP3 ≤T TSP2. without using the fact that “≤T ” p p is transitive.
12. 593. 2. There are several incorrect proofs that P = N P announced every year. 590. . It is known that the dominating set problem (see also Problem 523) remains N Pcomplete even when the dominating set is required to be connected (see. a satisfying assignment for the new formula. Output: A Hamiltonian cycle in G of minimum cost. For example. Garey and Johnson [28]). For each vertex v of a graph G = (V. . The knapsack problem is deﬁned as follows: KNAPSACK Instance: Positive integers s1 . . and hence for the old formula. E) with n vertices and e edges. s2 .3
WHAT IS WRONG?
Scientists have been working since the early 1970s to either prove or disprove that P = N P.
i∈X
si =
. 12. for example. p Prove directly that TSP3 ≤T TSP1. It is known that 3SAT is N Pcomplete. and hence that P = N P. What is wrong with the following proofs that P = N P? 591. found in time O(ne + n2 ). sn . This shows that 3SAT ∈ P. Then. can be found in polynomial time. Hence. Since DNFSAT ∈ P (see Problem 580). if one exists. The interior nodes of a breadthﬁrst search tree form a connected dominating set.
N P completeness
TSP3 Instance: A directed graph G with positive costs on the edges. and a positive integer B. . . (x1 ∨ x2 ∨ x3 ) ∧ (x1 ∨ x2 ) = (x1 ∧ x2 ) ∨ (x2 ∧ x1 ) ∨ (x3 ∧ x1 ) ∨ (x3 ∧ x2 ). 589. These form a minimumsize connected dominating set. S Question: Does there exist X ⊆ {1. This open problem is rapidly gaining popularity as one of the leading mathematical open problems today (ranking with Fermat’s last theorem and the Reimann hypothesis). Given a Boolean formula in conjunctive normal form with at most three literals per clause. ﬁnd the breadthﬁrst search tree rooted at v in time O(n + e).
TM (n.) The ﬁrst part of the computation (which uses a Turing machine) has access to the ﬁrst part of the input only. where y encodes a Boolean formula on k variables. y ∈ IBm . The ﬁrst part of the input is a string of bits that is presented as input to a standard Turing machine in the usual manner. of a natural number k. iﬀ y is the encoding of a Boolean formula F with k variables. and hence P = N P. y ∈ IBm . However. A deterministic split Turing machine M is said to accept an input string x#y. Aho. y ∈ IBm . m)
=
A nondeterministic split Turing machine M is said to accept an input string x#y. in unary.
TM (n. where x ∈ IBn . where x ∈ IBn . The time complexity of M is deﬁned to be max{1} ∪ {t  there is an x#y where x ∈ IBn . where y encodes a Boolean formula on k variables. such that the computation of M on input x#1k takes time t}. for example. iﬀ y is the encoding of a Boolean formula F with k variables. (To refresh your memory about standard Turing machines see. The time taken by a computation of a deterministic split Turing machine M on input x#y is the number of steps taken by M on input x#1k . and Ullman [1]. TM (n. The time taken by a computation of a deterministic split Turing machine M on input x#y is the minimum over all accepting computations of M of the number of steps taken by M on input x#1k . Therefore. in Section 8. together with the representation. 594. The class
. and produces an output of exactly k bits. The time complexity of M is deﬁned to be max{t  there is an x#y where x ∈ IBn . y ∈ IBm . and some computation of M on input x#1k outputs a satisfying assignment for F .3. The second part of the input is the encoding of a Boolean formula using the standard encoding scheme from Garey and Johnson [28]. and M on input x#1k outputs a satisfying assignment for F . KNAPSACK ∈ P. m) ≤ p(n. such that M takes time t to accept x#1k }. What is Wrong?
151
It is known that KNAPSACK is N Pcomplete. A split Turing machine is a variant of the Turing machine in which the input and the computation are each split into two parts.Sec. Hopcroft. 12. m)
=
A split Turing machine M is said to run in polynomial time if there exists a polynomial p such that for all m.2 we saw a dynamic programming algorithm for the knapsack problem that runs in time O(nS). The Turing machine computes in the normal manner. m). and the second part of the computation (which uses Boolean formula evaluation) has access both to the second part of the input and to the output of the ﬁrst part of the computation. n ≥ 1.
Hence. and so splitP = splitN P. since the deterministic part of the computation has access only to the number of variables in F . Clearly. and the class splitN P is deﬁned to be the set of languages accepted by a nondeterministic split Turing machine in polynomial time. which it can do in time linear in k. this is a member of splitP.
N P completeness
splitP is then deﬁned to be the set of languages accepted by a deterministic split Turing machine in polynomial time. and S ∈ splitP. and append to every member of L a ﬁxed string encoding the identity formula. A task for an architecture is an inputoutput pair. The loading problem is the following: Given an architecture and a task set. and a task set is a ﬁnite set of tasks. The proof is as follows. any deterministic split Turing machine must make an error on one of them. ﬁnd functions for each of the gates that enable the circuit to give the correct output for each corresponding input in the task set. This implies that P = splitP. 596.
12.4
Circuits
An architecture is a ﬁnite combinational circuit (that is. it is easy to see that P = splitP. Both of these formulae are satisﬁable. S ∈ splitN P. However. a circuit constructed without feedback loops) with the gate functions left unspeciﬁed. To see that splitP ⊆ P. We claim that splitP = splitN P. decide whether there exist functions for each of the gates that enable the circuit to give the correct output for each corresponding input in the task set. Clearly. which is linear in the number of bits in y. merely need guess an output string of k bits. The problems in this section are from Parberry [63]. S cannot be in splitP. 595. Consider the language S that consists of the strings ε#y such that y is the encoding of a satisﬁable Boolean formula F . where k is the number of variables in F . P = splitP = splitN P = N P. Show that the loading problem is N Pcomplete. yet since there is no truth assignment that satisﬁes them both.152
Chap. complete. 12. note that a deterministic Turing machine can simulate a split Turing machine with polynomial overhead in time by simply simulating the Turing machine part and then evaluating the Boolean formula on the result obtained. Now. Show that the loading problem for circuits of depth 2 is N P
. The following questions deal with the decision problem version of the loading problem: Given an architecture and a task set. simply assume that L ∈ P is encoded in binary. and hence must give the same output for the twovariable Boolean formulae x1 ∧x2 and x1 ∧x2 . and it can be similarly shown that N P = splitN P. To see that P ⊆ splitP. since the Turing machine on input ε#1k . Thus. S ∈ splitN P.
598. All connections are directed. and selfloops are allowed. 12. Time is quantized. Question: Does M have a stable conﬁguration? 599.
12. Question: Is there a truth assignment for C in which each clause contains exactly one true literal? OneinThree 3SAT (O3SAT3) Instance: A list of clauses C in which each clause contains exactly three variables. 600. Show that SNN is N Pcomplete. and complement.
Show that the loading problem for circuits of fanin 3 is N PShow that the loading problem for circuits with nodefunction set consisting of disjunctions and conjunctions of literals is N Pcomplete even for circuits with only four nodes.
A cyclic circuit is a network of gates that may have cycles. Question: Is there a truth assignment for C in which each clause contains exactly one true variable?
. Consider the following problems: The Stable Conﬁguration Problem (SNN) Instance: A cyclic circuit M . and computes a new output. disjunction. Show that SNN is N Pcomplete for cyclic circuits of fanin 2. The gatefunctions are limited to are conjunction. one or more of the gates reads its inputs.5. and at each step.Sec. all uncomplemented. complete. The circuit is said to have converged to a stable conﬁguration if the output of all gates remains ﬁxed over time.5
ONEINTHREE 3SAT
Consider the following variants of OneinThree 3SAT: OneinThree 3SAT (O3SAT1) Instance: A list of clauses C in which each clause contains at most three literals. OneinThree 3SAT
153
597. Question: Is there a truth assignment for C in which each clause contains exactly one true literal? OneinThree 3SAT (O3SAT2) Instance: A list of clauses C in which each clause contains exactly three literals.
7
HINTS
594. Prove a Turing reduction to the problem “is there a factor less than M . The factorization problem is deﬁned as follows: FACTORIZATION Instance: A natural number. In the following problems. N .
12.
12. η : IN → IN.6
FACTORIZATION
Let pi denote the ith prime number. At least one of them is wrong.
The Smarandache function. Question: Is there a truth assignment for C in which each clause contains exactly one true variable? O3SAT1 is the original version of the problem studied by Schaefer [68]. 606. 601. It is up to you to discover which. 605. 1 2 k 604. Prove that if P = N P. Output: A list of pairs (pi . 12. and O3SAT3 is the version listed by Garey and Johnson [28]. then FACTORIZATION can be solved in polynomial time. Prove that O3SAT2 is N Pcomplete. then FACTORIZATION can be solved in polynomial time.154
Chap. is deﬁned as follows: η(k) is the smallest m ∈ IN such that k divides evenly into m!. Most of them are correct. then the Smarandache function can be computed in polynomial time.
N P completeness
Balanced OneinThree 3SAT (B3SAT) Instance: A list of clauses C in which each clause consists of three variables and every variable appears in exactly three clauses. Prove that O3SAT3 is N Pcomplete. 604. 603. Prove that if the Smarandache function can be computed in polynomial time. starting with p1 = 2. such that N = pa1 pa2 · · · pak . you may assume that O3SAT1 is N Pcomplete.”
. Prove that if FACTORIZATION can be solved in polynomial time. 602. Prove that B3SAT is N Pcomplete. ai ) for 1 ≤ i ≤ k. A lot of the details of this proof are missing.
the reduction is from O3SAT3. A solution to this problem can be found in Parberry [61]. A solution to this problem can be found in Parberry [60]. Lin and Vitter [53].8
SOLUTIONS
595–597. In a feverinduced dream in 1993 I hallucinated that I had succeeded in proving that P = N P . I awoke at 3 A. 594. Most textbooks (and classes) prove that if A ≤m B and B ∈ P. 12.9
COMMENTS
578. A solution to this problem can be found in Parberry [63]. and this was the proof. 40. 42].
. Chitturi in 1994. 603. 13]. consult also Judd [39.Sec. not O3SAT1 as requested. Solutions
155
12.
12. However. 599–600. so technically you’ll have to solve Problem 602 in order to complete this proof. p and omit the proof of this result saying only that it is “similar. and Blum and Rivest [12. 595. 598.8. and instantly knew that the proof was wrong (I was not that sick!). A solution to these problems can be found in Judd [41]. This problem came out of a conversation with B.M.” 591. For more information on the complexity of variants of the loading problem. then A ∈ P.
You’re on your own!
13. Show that n positive integers in the range 1 to k can be sorted in time O(n log k). or an exact match for the bolt.1
SORTING AND ORDER STATISTICS
This section contains questions on sorting and order statistics (the latter is a fancy name for “ﬁnd the k smallest element”). You are to match each bolt to its nut. but there is no way to compare two nuts together.
The nuts and bolts problem is deﬁned as follows. 156 Show that any algorithm for the nuts and bolts problem must take Ω(n log n) comparisons in the worst case. You are allowed to try a nut and bolt together.
. too small. That is.
608. any algorithm for ﬁnding the median of S in time T (n) can be used to design an O(T (n)) time algorithm for ﬁnding the kth smallest. Devise an algorithm for ﬁnding the k smallest elements of an unsorted set of n integers in time O(n + k log n).Chapter 13
Miscellaneous
Here are some miscellaneous problems. from which you can determine whether the nut is too large. whenever the following holds: T is monotone increasing and there exists a constant 0 < c < 1 such that T (n/2) < c · T (n) for n ∈ IN. and those for which part of the problem is to determine the algorithmic technique to be used. Show that ﬁnding the median of a set S and ﬁnding the kth smallest element are reducible to each other. and viceversa. and n corresponding nuts. Problems on lower bounds are mostly (but not exclusively) found in Section 13.2. 610.
609. 607. if S = n. You are given a collection of n bolts of diﬀerent widths. These are mainly upper bounds. or two bolts together. deﬁned to be those that do not necessarily ﬁt into the earlier chapters.
613. These are algorithms that store their input in memory. 620. without requiring any more comparisons to be performed. 612. Show that any comparisonbased algorithm for ﬁnding the median must use at least n − 1 comparisons. Prove that any comparisonbased algorithm that searches a sorted array must make at least Ω(log n) comparisons. Show that this can be done with only 2n − 2 comparisons. You may use Stirling’s approximation. Lower Bounds
157
611. 618. Suppose that instead of matching all of the nuts and bolts. The two most common techniques used are the adversary argument and the decision tree.2.
. Use decision trees to ﬁnd a lower bound of n + 0.2
LOWER BOUNDS
The lower bounds considered in this section are for what is known as comparisonbased algorithms.5 log n − 2.
617.
619. 615. Show that any comparisonbased algorithm for sorting can be modiﬁed to remove all duplicates without requiring any more comparisons to be performed. 616. √ n! ∼ 2πn(n/e)n . the input data are not used for computing an address or an index). 13.
Devise an algorithm for the nuts and bolts problem that runs in time O(n log n) on average .Sec. you wish to ﬁnd the smallest bolt and its corresponding nut. Show that any comparisonbased algorithm for ﬁnding the secondsmallest of n values can be extended to ﬁnd the smallest value also.
13.66 on the number of comparisons required to ﬁnd the median. Show a matching lower bound for the sorting problem of Problem 607 in a comparisonbased model. 614. Show that at least n + log n − 2 comparisons are necessary to ﬁnd the smallest and second smallest of a list of n items. and the only operations permitted on that data are comparisons and simple data movements (that is. Show that any comparisonbased algorithm for removing duplicates from a list of values must use Ω(n log n) comparisons.
n pij = 1. .158 621. and 13. given the adjacency matrix of G.3
GRAPH ALGORITHMS
This section contains general questions on graph algorithms. 622.3. v(i+1)modk ) ∈ E for all 0 ≤ i < k.
625. What does this say about the average case complexity of bubblesort?
13. The mean cost of c is deﬁned by µ(c) = 1 k
k−1
w(vi . Preprocessing of an nnode tree should take O(n) time.4. 13. E) be a directed graph with n vertices. Miscellaneous
Show that if each of the n! permutations is equally likely. (v.7. j ≤ n. vk−1 ) is a cycle. v2 .
Let G = (V. 8.
Chap. determines whether or not G has a sink in time O(n). (vi . j ≤ n. 0 ≤ pij ≤ 1. More graph algorithm problems can be found in Sections 2.10. Suppose c = (v0 . Suppose the cost of edge (v. 9. . The truck drivers generally ignore their instructions and drive about at random. Devise j=1 an algorithm that determines in time O(n3 ) the most probable path between cities i and j for all 1 ≤ i.2. 2. Devise an algorithm that. 9.
624. You may assume that for all 1 ≤ i. Design an algorithm that determines whether a graph with n vertices and e edges is triconnected in time O(ne). 7. and a few scattered in applications sections in various chapters. An ancestor query for a tree is a request for information about the ancestor relationship between two nodes in the tree. .4. that is.11. A sink is a vertex s ∈ V such that for all v ∈ V . The probability that a given truck driver in city i will choose to drive to city j is pij .4. and that for all 1 ≤ i ≤ n. then the average key is about n/3 places from its proper position in sorted order. v(i+1)modk ). 9. It can be viewed as evaluating the function: −1 if v is a proper ancestor of w 1 if w is a proper ancestor of v ancestor(v. w). E) be a directed labeled graph with n vertices and e edges. 623. Design and analyze an algorithm that takes a tree as input and preprocesses it so that ancestor queries can subsequently be answered in O(1) time.
i=0
. s) ∈ E. A graph is triconnected if there is no pair of vertices whose removal disconnects the graph. Let G = (V. w) = 0 otherwise. Suppose a set of truck drivers work for a shipping company that delivers commodities between n cities. w) is (v.
Show that if µ = 0. show that µ = min max δn (v) − δk (v) .
Let c be a cycle of cost zero. Graph Algorithms
159
Let µ be the cost of the minimum meancost cycle in G. Assume without loss of generality that every v ∈ V is reachable from a source vertex s ∈ V .
627.
Show that if we add a constant c to the cost of every edge of G.3. then there exists a vertex v on the minimum meancost cycle such that max δn (v) − δk (v) = 0. n−k
0≤k≤n−1
628. v be any pair of vertices on c. then G contains no negativecost cycles and for all v ∈ V . Show that if µ = 0. Hence. δ(v) =
0≤k≤n−1
min
δk (v).
Show that if µ = 0. and δk (v) be the cost of the cheapest path of length exactly k from s to v (if there is no path from s to v of length exactly k.
. 0≤k≤n−1 n−k max
631. n−k
v∈V 0≤k≤n−1
632. then min
v∈V
δn (v) − δk (v) = 0.
Give an O(ne) time algorithm to compute µ. 13. then δk (v) = ∞).
Show that if µ = 0. Suppose that the cost of the path from u to v along c is x. the minimum value of µc over all cycles c in G. Prove that δ(v) = δ(u) + x. Let δ(v) be the cost of the cheapest path from s to v.Sec. n−k
629.
0≤k≤n−1
630. that is. 626. and let u. then for all vertices v ∈ V . then µ is increased by exactly c. max δn (v) − δk (v) ≥ 0.
E) with source s. every vertex is on exactly one path).
. and the EdmondsKarp algorithm. v) ∈ E. Let G = (V. Suppose the capacity of a single edge is increased by one. sink t. 633.160
Chap. Give an O(n + e) algorithm for updating the maximum ﬂow. Give an O(n + e) algorithm for updating the maximum ﬂow. v = s. let the largest edge ﬂow be max f (u. where G has n vertices and e edges. 13. v) ∈ E is labeled with a capacity c(u.
635. u)
(if (u.
v∈V
The maximum ﬂow problem is to ﬁnd a ﬂow f that has maximum total ﬂow.
(u. respectively. and • for all v ∈ V .
Consider a ﬂow problem on a graph G = (V. Suppose we are given the maximum ﬂow in a ﬂow network G = (V. the Dinic algorithm. v) =
u∈V u∈V
f (v. we assume that c(u.
634. The total ﬂow of f is deﬁned to be f (v. E) with n vertices and e edges with all capacities bounded above by a polynomial in n.
Show that two diﬀerent maximum ﬂows in G can have diﬀerent largest edge ﬂows. t ∈ V be distinguished vertices called the source and sink. sink t. Let s. v). v) ∈ IN. 0 ≤ f (e) ≤ c(e).v)∈E
636. and integer capacities. and integer capacities. Design an algorithm to ﬁnd the minimum possible largest edge ﬂow over all maximum ﬂows in G in time O(ne2 log n).4
MAXIMUM FLOW
Suppose we are given a directed graph G = (V. Devise and analyze an eﬃcient algorithm to ﬁnd the smallest number of directed vertexdisjoint paths that cover all vertices (that is. For any given ﬂow. E) be a DAG. Suppose we are given the maximum ﬂow in a ﬂow network G = (V. v) = 0). f (u. E) in which each edge (u. v). Algorithms for the maximum ﬂow problem include the FordFulkerson algorithm. v) = f (u. where G has n vertices and e edges. t. s) − f (s. E) with source s. A ﬂow in G is a function f : E → IN such that the following two properties hold: • for all e ∈ E. Miscellaneous
13. Suppose the capacity of a single edge is decreased by one.
7. s. Conclude from the statements of Problems 638–639 that maxﬂowbyscaling can be implemented in time O(e2 log c). Show that multiplying two n × n matrices is equivalent to raising an n × n matrix to the fourth power. 638. 3. and integer capacities c(u. Some textbooks show that matrix multiplication is equivalent (under some extra conditions) to matrix inversion. 644. 13. v) ∈ E.
13. Matrix Reductions
161
Suppose we are given the maximum ﬂow in a ﬂow network G = (V. respectively.5.5
MATRIX REDUCTIONS
Consider two problems A and B that have time complexities TA (n) and TB (n). 637. multiplying two n×n matrices is equivalent to raising an n × n matrix to the kth power. Show that the residual capacity of a minimum cut of G is at most 2ke whenever line 4 is executed. Show that for all k ≥ 2. v) initialize ﬂow f to zero k := 2 log c while k ≥ 1 do while there exists an augmenting path p of capacity ≥ k do augment ﬂow f along p k := k/2 return(f ) Prove that maxﬂowbyscaling returns a maximum ﬂow. 5. Suppose G has n vertices and e edges. Prove that the inner while loop of lines 5–6 is executed O(e) times for each value of k. function maxﬂowbyscaling(G. 640. 1. 642. Show that multiplying two n × n matrices is equivalent to squaring an n × n matrix. 639. 2. t) c := max(u.v)∈E c(u. The following modiﬁed version of the standard FordFulkerson algorithm can be used to ﬁnd a maximum ﬂow in G.Sec. Show that multiplying two n × n matrices is equivalent to cubing an n × n matrix. Here are some more problems along this line: 641.
. 4. 643. 8. E) with source s. sink t. Problems A and B are said to be equivalent if TA (n) = Θ(TB (n)). 6. v) on each edge (u.
Chap. 650. Ak = A · Ak−1 . and for k > 0. (See Problem 645 for the deﬁnition of matrix closure. Show that multiplying two n × n matrices is equivalent to cubing an n × n lower triangular matrix. but you don’t know how far away or in which direction. multiplying two n × n matrices is equivalent to raising an n × n lower triangular matrix to the kth power. and to perform comparisons on the elements of the matrix. where A0 i=0 is the n × n identity matrix.
13. Show that there is an algorithm that enables you to ﬁnd the door by walking at most O(n) steps. 649.162 645. 13. 648.6
652. 647.)
646. (Note that closure is not deﬁned for all matrices. exceeding It’s Texas Chile Pepper day and you have a schedule of events that looks something like this:
653.
GENERAL
You are facing a high wall that stretches inﬁnitely in both directions. 651. Show that multiplying two n×n matrices is equivalent to multiplying two n × n lower triangular matrices. There is a door in the wall. Show that multiplying two n × n matrices is equivalent to squaring an n × n lower triangular matrix. It is pitch dark. Show that multiplying n × n matrices is equivalent to computing the closure of an n×n lower triangular matrix. Show that multiplying two n × n matrices is equivalent to raising an n × n lower triangular matrix to the fourth power.
. What is the constant multiple in the bigO bound for your algorithm? You are given an n × 2 matrix of integers. You are allowed to permute the rows of the matrix as monolithic items. Show that for all k ≥ 2. where“·” denotes matrix multiplication. but you have a very dim lighted candle that will enable you to see the door when you are right next to it. Miscellaneous
Deﬁne the closure of an n × n matrix A to be ∞ Ai . Your task is to ﬁnd an O(n log n) time algorithm that permutes the rows so that neither column of the matrix contains an increasing subsequence (not necessarily contiguous) of length √ n . where n is the number of steps that you would have taken if you knew where the door is and walked directly to it.
654.) Show that multiplying two n × n matrices is equivalent to computing the closure of an n × n matrix.
. Devise an O(n) time algorithm for determining the largest R ⊂ S such that f (R) = R.
Show that the worst case number of moves needed to solve the (n2 − 1)puzzle is at least n3 − O(n2 ). Design an algorithm that determines the size of the largest submatrix of ones in A in time O(n2 ). Let A be an n × n matrix of zeros and ones. How fast is it? 655. Design and analyze an algorithm that computes in time O(n) the method of dialing that involves moving your ﬁngers the smallest amount of total distance. Naturally. . For deﬁnitions. 0 ≤ k ≤ min(n − + 1. A submatrix S of A is any group of contiguous entries that forms a square. 13.j ∈ Z for all 1 ≤ i. General
163 8:30 9:30 9:50 9:00 – – – – . and that the eﬀort required to move a ﬁnger from one button to another is proportional to the Euclidean distance between them. called the (n2 − 1)puzzle. you want to attend as many events as you can. .
The following group of questions deals with the generalized version of the popular Sam Loyd 15puzzle. . that is.
. . If R ⊂ S.
657. 9:15 10:05 11:20 11:00 Pepper Pepper Pepper Pepper Pancake Party Eating Contest Personality Proﬁle Parade
The schedule lists the times at which each event begins and ends. Your algorithm must input a legal state of the puzzle and output a series of O(n3 ) moves that solves it. more formally. 659. The events may overlap. .
656. We assume that the two ﬁngers start out on the ∗ and # keys. Design an eﬃcient algorithm to ﬁnd the largest set of nonoverlapping events. 2. n} and f : S → S. whether an assignment of integers can be made to the variables so that all of the equations in E are satisﬁed. {A[i.6. m ≤ j ≤ m + k − 1} for some 1 ≤ . . . . deﬁne f (R) = {f (x)  x ∈ R}. We wish to compute the laziest way to dial an ndigit number on a standard pushbutton telephone using two ﬁngers. j ≤ n). Design an algorithm for the (n2 − 1)puzzle that runs in time O(n ). . Prove that your algorithm is correct. Let E be a set of m linear equations of the form xi = xj + cij over Z the variables x1 . see Problem 515.
658. j]  ≤ i ≤ + k − 1.Sec. n − m + 1). m ≤ n. S is said to have size k. Suppose S = {1. Devise an O(m) time algorithm for determining whether the equations in E are consistent.
3
660. xn (ci.
Show that if there are n ≥ 3 coins.
13. 666.
. the counterfeit coin can be found in three weighings. the counterfeit coin can be found in log3 2n weighings.7
HINTS
614. Show that if there are 39 coins. 662. 668. You are given a list of m statements of the form “i hates j. The counterfeit coin is indistinguishable from the real coins in all measurable characteristics except weight. then you do not want to seat i above or in the same balcony as j.
Suppose you are given a collection of gold coins that includes a clever counterfeit (all the rest are genuine). Give an algorithm that assigns the balconies (or says that it is not possible) in time O(m + n). the counterfeit coin can be found in four weighings. where x < y. which contradicts the correctness of the algorithm. then the search for any element a of the universe with x < a < y must also lead to that leaf. You have a scale with which you can compare the relative weight of coins. at least log3 2n weighings are needed to ﬁnd the counterfeit coin. 13.164 661. β < 1. otherwise i will throw popcorn at j instead of watching the play. However. 667. in the set leads to the same leaf of the comparison tree. Show that if there are 12 coins. Design and analyze an algorithm for ﬁnding the minimum number of balconies needed.
Chap. Show that for some constants 0 < α. Show that if the search for two diﬀerent elements x.” If i hates j. Your problem is to ﬁnd the counterfeit coin and determine whether it is lighter or heavier than the rest. Miscellaneous
Show that the average case number of moves needed to solve the (n2 − 1)puzzle is at least 2n3 /3 − O(n2 ). 663. Show that if there are n ≥ 3 coins. 664. y. a random legal conﬁguration of the (n2 − 1)puzzle requires at least αn3 moves to solve with 2 probability 1 − 1/e−βn .
Your job is to seat n rambunctious children in a theater with n balconies. 665. you do not know whether the counterfeit coin is heavier or lighter than the real thing.
. D denote moving the hole one place left. . The lower triangular property can actually make the problem easier. You will need to use Chernoﬀ bounds. k) that belongs in square (k. where f is the maximum ﬂow in G . Consider the EdmondsKarp algorithm for maximum ﬂow. t} ∪ {x1 . The Manhattan distance of a tile in square (i. Show that the minimum number of paths that cover V in G is equal to n − f . stopping when there is not enough space left to perform the maneuver. Use both properties from Problem 626. It can be shown that if the hole is in the lower right corner. vn }. 636. and down. Suppose the hole is immediately to the right of a square S. to move each square to its correct space. right. . and you’ve solved the problem! The best I can do is 5n3 − Ω(n2 ) moves. Find a lower bound for the average Manhattan distance.Sec. 641–645. 629. N. and others like it. 13. Make sure that the conﬁguration you come up with is solvable.7. . Form a network G = (V . xi )  1 ≤ i ≤ n} ∪ {(yi . . U. Use this sequence of moves. Suppose V = {v1 . as the solution to Problem 641 given later demonstrates. t)  1 ≤ i ≤ n} ∪ {(xi . the solvable states of the puzzle are exactly those that can be made with an even number of transpositions. V E = {s. ) is deﬁned to be i − k + j − . up. Use a combination of greedy and divideandconquer techniques. The cost of the path from v to u along c is −x. E ) as follows. . . Let L. 628. Hints
165
627. The Manhattan distance of a conﬁguration of the puzzle is deﬁned to be the sum of the Manhattan distances of its tiles. there do exist nontriangular solutions. Show that the Manhattan distance is a lower bound on the number of moves needed to solve the puzzle. . Show that a mincost path to any vertex on the minimum meancost cycle can be extended along the cycle to the next vertex on the cycle. 659. yn }. Let B(m. vj ) ∈ E}. xn } ∪ {y1 . . R. Consider Problems 647–651. 660. Then the sequence of moves U LDLU R moves S one square diagonally up and to the right. respectively. 662. Find a lower bound for the Manhattan distance of a random conﬁguration. Figure out how to ﬁnish it. and that there exists a conﬁguration that has Manhattan distance at least n3 − O(n2 ). if you think about it for long enough. On the other hand. The best value that I can get for α is 16/243. . p) be the probability of obtaining at least m successes
. .
The capacity of all edges is one. . 633. = {(s. yi )  (vi . 661.
p) ≤ e−β N p/2 . Actually. Horden [36] gives an interesting history of this puzzle and its variants. 659. Early work on the 15puzzle includes Johnson [38] and Storey [76].8
SOLUTIONS
641. This problem was suggested to the author by Arny Rosenberg.
. 2 If 0 ≤ β ≤ 1. 665. for example.
13. has AB has a submatrix. I’m not going to give you a solution. 653. 611.166
Chap. when squared. The nuts and bolts problem is from Rawlins [66]. Start by dividing the coins into three piles of four coins. 663. for descending) of length example. Pick two piles and compare their weights. It even appears in a science ﬁction novel by Piers Anthony [6. For k = 1. [4]. Interestingly. The minimum number of moves needed to solve the 15puzzle in the worst case is unknown. Angluin and Valiant [5] and Valiant and Brebner [80]): Let β = m/N p − 1. This algorithm is mentioned in Alon et al.9
COMMENTS
610. see. this is a relative of the Erd¨sSzekeres Theorem . but has been the subject of various papers in the AI literature. O(n2 )) time.
13. Then generalize your solution. See also Gardner [27]. 2. The trick is to embed A and B into a larger matrix that. Start by solving Problems 663 and 664. but an actual eﬃcient algorithm is known only for k = 1. 13. The solution can then be read oﬀ easily in optimal (that is. Chapter 16]. N.
663. There are many embeddings that work. which states that there exists o in every sequence of n integers a monotonic subsequence (either ascending or √ n (for more recent proofs of this theorem. The minimum number of moves to solve each legal permutation of the 8puzzle has been found by exhaustive search by Schoﬁeld [69]. then β(m. The following result is a wellknown consequence of the Chernoﬀ bounds (see. one can prove by a counting argument that one can so permute an n×k matrix. each with probability p of success. including 0 B A 0
2
=
AB 0
0 AB
. This problem is so popular with mathematicians that you ought to be able to ﬁnd one in the literature somewhere. Dijkstra [22]. Miscellaneous
out of N Bernoulli trials. and Seidenberg [73]).
13. Miller.9. Fleming and Oldﬁeld [57] and Korf [48]. Comments
167
including Michie. and they demonstrate an approximation algorithm that makes no more than a (fairly large) constant factor number of moves than necessary for any given legal conﬁguration. Ratner and Warmuth [65] have proved that the problem of determining the minimum length number of moves for any given legal conﬁguration of the (n2 − 1)puzzle is N Pcomplete. Kornhauser.
.Sec. and Spirakis [49] have shown an algorithm for the (n2 − 1)puzzle and its generalizations that always runs in time O(n3 ).
1974. M. Ullman. D. Hopcroft. D. Bentley. Fiat. Aho. Anthony. V. Baase. A. Foundations of Computer Science. Fast probabilistic algorithms for Hamiltonian circuits and matchings. Explicit solutions to the nqueens problem for all n. and J. 1986. AddisonWesley. University of Toronto Press. E. In Proceedings of the Ninth Annual ACM Symposium on Theory of Computing. 12th edition. [6] P. Computer Science Press. Ullman. Aho and J. Angluin and L. [5] D. ACM Press. Kannan.Bibliography
[1] A. 5th Annual Symposium on Discrete Algorithms. 1985. 1983. AddisonWesley. AddisonWesley. 2(2):7. Naor. Hopcroft. Programming Pearls. [3] A. M. AddisonWesley. The Design and Analysis of Computer Algorithms. 1991. 1994. S. and J. pages 30–41. E. and R. J. Bentley. [10] J. With a Tangled Skein. Aho. 1978. [11] B. W. pages 690–696. Mathematical Recreations and Essays. Blum. V. Coxeter. V. 1977. [9] J. M. volume 3 of Incarnations of Immortality. J. [4] N. [2] A. In Proc. 1992. [8] W. SIGART Bulletin. Ullman. AddisonWesley. Valiant. Computer Algorithms: Introduction to Design and Analysis. Alon. 1974. 168
. [7] S. Ballantine Books. Ostrovsky. Data Structures and Algorithms. 1988. Rouse Ball and H. Bernhardsson. S. More Programming Pearls: Confessions of a Coder. Matching nuts and bolts. D.
pages 10–14. Introduction to Algorithms. Leiserson. H. Bratley. [25] L. Sci. Graph Algorithms. February 1992. [15] A. Solution problematis ad geometriam situs pertinentis. [21] E. Dijkstra. Berlin. Hindrichs. Discrete Applied Mathematics.und endfeldern vollstaendig abzuschreiten. Prentice Hall. Conrad. Training a 3node neural network is NPcomplete. L. Some beautiful arguments using mathematical induction. 5(1):117–127. Algorithmics: Theory and Practice. In Neural Information Processing Systems 1. In Proceedings of the Third Annual ACM Symposium on Theory of Computing. pages 151–158. 1971. Morsy. W. 1980. 13:1–8. Blum and R. 1976. 1989. Brassard and P. Solution d’une question curieuse qui ne paroit soumise a aucune ` analyse. 16:276– 285. [26] S. H. H. Rivest. Journal of Symbolic Computation. [16] A. Matrix multiplication via arithmetic progressions. L. 1990. [23] P. pages 131–133. Information Processing Letters. The complexity of theorem proving procedures. schachbretter beliebiger groesse und zwischen beliebig vorgegebenen anfangs. [13] A. 1988. Rivest. Cook. Fibonacci Quarterly. Spektrum der Wissenschaft. 1994. 1984. and R. Wegener. 1736. T. [22] E. [18] D. Coppersmith and S. A Discipline of Programming. 1990. McKay.Bibliography
169
[12] A. Pitman. Eades and B. W. Hindrichs. Euler. Blum and R. Acta Informatica. MIT Press. Mem. Conrad. [14] G. 1979. and I. L. Rivest. Cull and J. Acad. [20] P. 1978. pages 310–337. Neural Networks. Morsy. [19] T. An algorithm for generating subsets of ﬁxed size with a strong minimal change property.
. Comentarii Academiae Scientarum Petropolitanae. Wegener. Training a 3node neural network is NPcomplete. 1759. Wie es dem springer gelang. Knight’s tour revisited. 1992. Euler. [24] L. and I. Solution of the knight’s Hamiltonian path problem on chessboards. Cormen. 8:128–140. Prentice Hall. C. pages 494–501. 50:125–134. Dijkstra. Winograd. T. [17] S. Even. Morgan Kaufmann. DeCurtins. E. 9:251–280. ACM Press. A.
Harel. E. 1988. Computer Science Press. 2(4):397–399. a [34] D. of the First International Conference on Neural Networks. Judd. S. R. Concrete Mathematics: A Foundation for Computer Science. Spencer. [35] B. In R. Judd. Freeman. AddisonWesley. Graham and J. L. PhD thesis. Sliding Piece Puzzles. E. 1989. Algorithmic Graph Theory. 1978. Rothschild. Neural Network Design and the Complexity of Learning. Complexity of Computer Computations. H. Horden.
. American Journal of Mathematics. Gardner. In Proc. Graham. S. Johnson. M. 1987. 4:177–192. [31] R. 1990. Neural Network Design and the Complexity of Learning. Cambridge University Press. 1990. H. Thatcher. MIT Press. A. H. L. S. 1972. W. pages 685–692. and J. John Wiley & Sons. Permutations by interchanges. Sahni. 1963. AddisonWesley. editors. R. New York. D. MA. Reducibility among combinatorial problems. Heap. Patashnik. 1985. Amherst. Dover. L. Knuth. W. 1986. [40] J. S. 1988. Graham. On the complexity of loading shallow neural networks.
Bibliography
[28] M. 1987. E. 1982. Oxford University Press. 1959. University of Massachusetts. H. [43] R. Miller and J. Notes on the 15 puzzle 1. Plenum Press. Mathematics for the Analysis of Algorithms. 1979. Scientiﬁc American. Karp. Learning in networks is hard. [36] L. Birkh¨user. [29] A. [32] R. Ramsey Theory.170 [27] M. [39] J. Computers and Intractability: A Guide to the Theory of NPCompleteness. Horowitz and S. L. Gibbons. Garey and D. July 1990. 6:293–294. B. and O. Spencer. M. [33] D. The Mathematical Puzzles of Sam Loyd. Ramsey theory. Journal of Complexity. [37] E. Fundamentals of Computer Algorithms. 1879. IEEE Computer Society Press. Greene and D. S. 263(1). [30] R. Judd. Judd. [38] W. E. Algorithmics: The Spirit of Computing. Knuth. [41] J. Computer Journal. Johnson. [42] J.
Fleming. [53] J. American Elsevier. Prentice Hall. 6:211–230. and applications. W. D. AddisonWesley. Knuth. pages 241–250. E. and unaided methods of solving a shortestroute problem. Shapiro. [45] D. Correctness. Design and Eﬃciency. Benjamin/Cummings. the diameter of permutation groups. The Design and Analysis of Algorithms. Kozen. [54] X. Knuth. Coordinating pebble motion on graphs. Machine Learning. interactive. Knuth. AddisonWesley.
. 1988. e e e [55] E. E. second edition. Spirakis. H. Michie. 25(8). G. H. Depthﬁrst iterative deepening: An optimal admissible tree search. 1991. 1992. Algorithms and Data Structures: Design. R´cr´ations Math´matiques. Kornhauser. 27(1):97–109. International Journal of Computer Mathematics. J. GauthierVillars. pages 245–255. [58] B. Analysis. Lam and L. 1989. V. [48] R. [49] D. [46] D. Harper Collins. E. Soicher. Oldﬁeld. [47] D. volume 3. Fundamental Algorithms. In 25th Annual Symposium on Foundations of Computer Science. Introduction to Algorithms: A Creative Approach. Korf. Michie. Communications of the ACM. H. 1982. IEEE Computer Society Press. volume 1 of The Art of Computer Programming.M. 1968. Machine Intelligence 3. 1981. E. Data Structures and Their Algorithms. [51] C. SpringerVerlag. 1893. Vitter. Artiﬁcial Intelligence. C. AddisonWesley. 1991.H. 1984. [50] D. Papadimitriou and K.Bibliography
171
[44] J. S. volume 1 of Algorithms from P to NP. [56] U. In D. 24:39–54. 1990. A comparison of heuristic. and P. 1973. Lucas. Manber. [59] C. H. Moret and H. second edition. Steiglitz. M. 1982. 1991. Sorting and Searching. Towers of Hanoi problem with arbitrary k ≥ 3 pegs. [57] D. E. Kingston. Combinatorial Optimization: Algorithms and Complexity. editor. volume 3 of The Art of Computer Programming. Three new combination algorithms with the minimal change property. Lu. AddisonWesley. Lin and J. Seminumerical Algorithms. Miller. AddisonWesley. and J. volume 2 of The Art of Computer Programming. 1985. [52] Lewis and Denenberg. 1973. Complexity results on learning by neural nets. G.
1978. L. [62] I. J. 1971. 1967. Brown. second edition edition. volume 506. [75] D. MIT Press. Smith. A. Technical Report CRPDC947. 1985. Computer Science Press. 1992. PWSKent. Rawlins. editors. D. Como puting. 1959. The complexity of satisﬁability problems. On the complexity of learning with a small number of nodes. [70] A.172
Bibliography
[60] I. [69] P. W. Parberry. D. 1990. Algorithms for touring knights. K. Parberry. 1983. In Proc. [65] D. On the computational complexity of optimal sorting network veriﬁcation. volume 3. Schaefer. pages 216–226. Collins and D. 1991. Michie. ACM Press. 1990. and Winston. Rawlins. Design and Analysis of Algorithms. J. 34. 64(5):325–332. Schnelle multiplikation grosser zahlen. American Elsevier. [74] J. Compared to What?: An Introduction to the Analysis of Algorithms. Strassen. Parberry. Seidenberg. May 1994. 1991. pages 252–269. How to Read and Do Proofs. Holt. Dept. Sch¨nhage and V. Sedgewick. A. Which rectangular chessboards have a knight’s tour? Mathematics Magazine. [73] A. and C. Algorithms. 1994.
. Circuit Complexity and Neural Networks. [68] T. 1992 International Joint Conference on Neural Networks. Machine Intelligence 1. 7:281–292. Solow. AddisonWesley. in Series Lecture Notes in Computer Science. pages 125–133. Journal of o the London Mathematical Society. [67] G. of Computer Sciences. The (n2 − 1)puzzle and related relocation problems. [71] A. Wiley. Ratner and M. Journal for Symbolic Computation. pages 893–898. [72] R. 1992. The Analysis of Algorithms. 1989. Compared to What? An Introduction to the Analysis of Algorithms. In N. In Proceedings of The Conference on Parallel Architectures and Languages Europe. SpringerVerlag. A simple proof of a theorem of Erd¨s and Szekeres. Computer Science Press. J. In Proceedings of the Tenth Annual ACM Symposium on Theory of Computing. [63] I. Warmuth. Rinehart. [61] I. Purdom Jr. Complete solution of the eight puzzle. Schwenk. Center for Research in Parallel and Distributed Computing. 10:11–137. [64] P. [66] G. Schoﬁeld. Parberry. University of North Texas. 1991.
Gu. [83] H. 1982.Bibliography
173
[76] W. 1992. Kluwer Academic Publishers.
. Neural network computing for knight’s tour problems. Neural Network Parallel Computing. SIGART Bulletin. Weiss. Algorithms and Complexity. International Journal of Computer Mathematics. On the towers of Hanoi problem with multiple spare pegs. 1992. E. A polynomial time algorithm for the nqueens problem. 52:17–22. A scheme for fast parallel communication. Susic and J. [81] J. Data Structures and Algorithm Analysis. SIAM Journal on Computing. 1879. S. 2(4):399–404. Storey. [77] R. Page. [80] L. C. J. 1986. 4(5):249–254. Prentice Hall. 1990. Neurocomputing. Benjamin/Cummings. Wilf. 1(3):7–11. 1992. Notes on the 15 puzzle 2. 1994. Brebner. A. [82] M. Takefuji and K. Veerasamy and I. Takefuji. American Journal of Mathematics. [78] Y. Lee. 11(2):350–361. G. Valiant and G. [79] Y.
145 time. 133 balance. 14 bolt. 123 operator. method of. 137 reﬂected Gray code. 78 bin packing problem. 25 areas. 118. 62. 135. O3SAT2. 74–75. see nuts and bolts Brahma. see satisfying assignment truth. 158. 135. 135. 81 addition. 166 2–3 tree. 164 bank. 97 architecture. 52. see B3SAT balcony. 150 traversal. 158 approximation algorithm. 152 acyclic. 116. 18. 165 AVL tree. 91 αpseudomedian. 124. 80 174
assignment. 167 arbitrage. 154 backtracking. see truth assignment average. 3 satisfying. 90. 82 matrix. 122–123. 136. 166 15puzzle. 157 airport. 166 biconnected. 79 2SAT. 85 tree. 55. 116. 164. 116. 121. 81 adjacency list. see pseudomedian amortized analysis. 50. 158 adversary. see O3SAT1. 24. 64. 61–66. 138 binomial. ix. 141. 135. 47. 145. 79 ascending. 148 3SAT. 59. 142 ancestor. 140–141. 126. 69. 86 breadthﬁrst search tree. 158
. 145. O3SAT3 parity. see method of areas arithmetic expression. 103 allpairs shortestpath. 136 bubblesort. 123 tree. 127. 135. see B3SAT oneinthree. 152 comparison. 6. 120. 79. 163. 143–144 binary. 138–140 string. 157. 146 B3SAT. 74. see strong 3SAT accept. 152 area. 145 balanced oneinthree 3SAT.Index
8puzzle. 19. 97 Bernoulli trial. 137 heap. see parity 3SAT strong. 77. 74. ix. 146 2colorable. 150 balanced. 151. 133 search. 25.
79. 18. 79 colorable. 163
capacity. 144. 150 contextfree grammar. 26. 165 Eulerian. 163 Dijkstra’s algorithm. 148. 140 dense. 118.Index
175 Hamiltonian. 115. 142. 2. 153. 82. 108. 136. 120–121. 19. see outdegree delete. 92 circle. 122. 26. 18. 150 component. 150 connected. 119 DAG. 19. 104. 124 circuit. 102–103 dimension. 2. 96. see Hamiltonian cycle cyclic. 78 dominating set. 165. 18. 161. 146 Chomsky Normal Form. 20. 26. 164. 148 dominating set. 154 clique. 133 closure. 92 continuous knapsack problem. 153. 26. 78. 90. 25. 152 tree. 147. 152–153 cyclic. 103. 12–14. 12. 166 chessboard. x. 150 divideandconquer. 82. 165 ceiling. 153 convex. 96. 78–79 spanning tree. 86. see cyclic circuit city. 137–139 disjunctive normal form. 160 decision problem. 141. 146 dial. 125 child. 79 combination. 137 deletemin. 103. 101–102 converge. 143. 128. 64 Erd¨sSzekeres Theorem. 59. 165 elementary operation. 157 degree. see connected dominating set double rotation. 14–18. 23. 12. 151 EdmondsKarp algorithm. 141. 81. 136–138. 136. 162 coin. 123 closed knight’s tour. 108 dirty. 145. 153 shift. 16. 140. 140. 80. 6. 98. 18. 146 deletemax. 110. 116. 16. 135. 124. 158. 116. 137. 102. 62. 135 connected. 96. 158 clause. 6. 150. 79. x. 16. 107. 67–87. 165 division. 20 circuit. 164 cycle. 20. 110 color. 141 dynamic programming. 131 Communications of the ACM. 160 directed spanning tree. 160. 18. see max cost mean. 127. 126. 48. see mean cost min. 131. 17 Dinic algorithm. 68. 106. 111. see Eulerian cycle
. 148–150 cost max. 101 depthﬁrst search. 110 out. 81. 25. 98. 123. 19. 80. 10 Chernoﬀ bound. 61 DNF SAT. 87– 100. see mincost counterfeit. 82. 101 density. 16. 148. 166 coinchanging problem. 18 Cook reduction. 166 o Euclidean distance. 109. 159. 149. 144. 157– 158 complete binary tree. 82. 78 descendant. 26. 5 comparisonbased algorithm. 147. 135–138. 86. 7. 160. 92. 124 problem. 124. 103.
160. 130. 26. 140. 66. x. 52. 160 maximum. 23. 91–92 FordFulkerson algorithm. 149. 124 induced subgraph. see maximum ﬂow Floyd’s Algorithm. 78. 126. 108 spanning. 126 increment. 10 ﬂow. 53. 35. 38. 117. 14. 106–110. 146 Journal of Algorithms. 38. 127. 137 height. 63 factorial. 49. 152 Fermat’s last theorem. 109. 93. Sam. 153 loading. 100. see zeroone knapsack problem knight. 141. 133 ﬁrstﬁt. see contextfree grammar graph. 41–46. see towers of Hanoi heap. 160 Gray code. see continuous knapsack problem zeroone. 81 exhaustive search. x. 152 programming. 97. 7. 111. ix. 57. 18 grammar. see open knight’s tour Kruskal’s algorithm. see programming language string processing. 100–115. x. 131. 150. 146 heuristic. see string processing language level. 165
. 97. 131. 124. 148 Richard M. 53. 161 forest. 26. 91. 132. 133 heapsort. 24 geometry. 16–18. see spanning forest Gauss. 133 Eulerian cycle. 51 horse. 80. 15. 124–125 Horner’s rule. 153. 145. 150 Fibonacci numbers. 12 knight’s tour closed. 16–17
Index
ice cream cart. 6. 20. 2. 102–104. 165 knapsack problem. 46. 51. 20. 18. 163 Manhattan distance. 16. 6. 55–58 Loyd. 147. 19. 87 time. 149.. 136. 2. 54. 145 literal. 123. 79. 64–66 ﬁrstﬁt heuristic. 165 halving. 152. x. 137 Hamiltonian cycle. Leonhard. 145. 1. 143–144 ﬂoor. 120. 127. x. 92. 104. 135–138.176 Euler. 79. 147 exponential. 150– 151 continuous. 51. 119. 81. 8–27. 145 Heap’s algorithm. 136. 133 greedy algorithm. 16 Hanoi. 16. 150 path. 52. 26. 150. 106–108 language. 26 hypercube. 131. 61. 47. 148. 61. 116–134. 124. 58. 2 exponentiation. 63 factorization. 20. 160. 81. 148. see ﬁrstﬁt heuristic HiQ. 61. 62. 49. 166 existential. 104. 5 Karp reduction. 136. 2. 154 feedback loop. 135. 146 independent set. see closed knight’s tour open. 63. 123 induction. 155 longest ascending subsequence. 92. 82. 99 loop invariant. 99. 82–85. 89. 5. 158.
62. 124. 142. 141. x. 87. 108 maximum ﬂow. 123. 99 parity 3SAT. 120. 59 production. 165 compression. 127 P. 133 nonterminal symbol. 22. see adjacency matrix inversion. 153–154 O3SAT3. see weighted median merge. see O3SAT1. x prune. 92. 88. 102. 125
marble. 118–120. 108. 19. 75–77 polygon. 137. 158–160. 122. 79. 98. 18 polynomial time. 80 MAXMIN. 144 queue. 134 path. 86. 125. see weighing matrix max cost spanning tree. 11–12. 132. 160–161. 123. 153–155 O3SAT2. 154 priority. 154 manyone reduction. 153–155 oneinthree 3SAT. 156–157. 81 method of areas. 97. 87. 121. 62 peaceful queens problem. 135. see Hamiltonian path patternmatching. 103. 110 open knight’s tour. 19 pivot. 103. 100 programming language. 92. 137. 102–108. 81 Prim’s algorithm. 117–121. 165 subsequence sum. 5. 122. 88. 98. 166 binary search tree. 148 reducible. 99. 137 parent. 155 N Pcomplete. 118. 148 Pascal. 130–133. 79. 97 weighing. 50. 19 priority queue. 145 Hamiltonian. 91. 75. 106–108 prime. 69. 106. 157 weighted. 122 matrix. 111. 68 mean cost. 161 multiplication. 104. 138 lines. 145. 94. 16. 158. 167 nqueens problem. 164. 146 parenthesize. 75. 36. 140. 103 parallel. 147. 38. 67. 147 Turing reduction. 76. 136. 25 mincost spanning tree. 148 (n2 − 1)puzzle. 3–5. 161– 162 product.Index
177 parallel algorithm. 129 predecessor. see parallel priority queue product rule. 87 adjacency. 123 pseudomedian. 113–116. x. 97. 166 O3SAT1. O3SAT2. 3. 146. 130 popcorn. ix heap. 76 queen. 110 outdegree. 100 N P. 122 permutation. 166 Pigeonhole Principle. 93. 147–152. 147–152. 149–152. 131–133 monom. 154. see nqueens problem perfect matching. 7. O3SAT3 oneprocessor scheduling problem. 163. 111 minimalchange. 90 tapestorage problem. 146–155 nuts and bolts. 155
. 148 polyomino. 110. 73–74. 50. 124 optimal. 137. 92. 164 postage stamp. 133–134 neural network. 142. 154. 18. 158 median. 156. 19.
see SAT satisfying assignment. 108 tree. see postage stamp Stirling’s approximation. 14. 124. see binary tree complete. 93 strong 3SAT. 117 sibling. 79. 152 telephone. 149 DNF. 104. 85. 151 savings certiﬁcate. 85 Ramsey number. 35. 149–150 treasury bond. 64. 110. 62. 80. 92 Texas Chile Pepper Day. 163 selfreducibility. 98. 158. 164 23. see AVL tree binary. see decision tree recursion. 150. 95. 130. 48. 144 tournament. 66. 101. 150 rooted tree. 145. 158
.178 quicksort. 74 stable conﬁguration. 19–21. 26. 140–142. 138. 104. 122. 165 toll. 150 remaindering. 158 triomino. 133 Ramsey’s Theorem. see complete binary tree decision. 144 sentinel. 100. 109. 124 recurrence relation. 162 tile. see 23 tree AVL. 157 Strassen’s algorithm. 103 shortestpaths. 12–14 truck. 75. 17–18. 149–150 semigroup. 55. 123 reﬂection. 65 root. 130 double. 147. see directed spanning tree maxcost. 136–138. x. 26. 124. 62. 73–74 string processing language. see depthﬁrst spanning tree directed. 160. 41–45. 150. 1. 153 stamp. 111. 102 depthﬁrst. 36. 92. 76. 12. 137. 138 single rotation. 159–161 spanning
Index
forest. 18. 140. 156–158 source. see maxcost spanning tree mincost. see recursion tree spanning. 66. 75–77. 141 singlesource longestpaths. 17. 141. 81. 142. 141. 102 sink. 137. 130 Reimann hypothesis. 77–78 travel agent. 55. 64. 50. 18. 146. 61 repeated substitution. 136. 79 triconnected. 161 Smarandache function. 81. 145 recursion tree. see single rotation salesperson traveling. 70. see traveling salesperson SAT. 75–77. 103 traveling salesperson. 158. 148 sum rule. see spanning tree triangle. 163 terminal symbol. 59 task. see mincost spanning tree squareroot. 37–45. 65–67. 98. 154 sort. see DNF SAT satisﬁability problem. 97 schedule. see double rotation single. 17 rotation. 97 tree. 145. 162. 16 towers of Hanoi.
145 vertex cover. 142. 110 warehouse.Index
179
truth assignment. 89
. 148. 135. 152–154 Turing machine. 128 weighted median. 95. 98 weighing matrix. 94. 80 zeroone knapsack problem. 151 unionﬁnd. 149.
Errata to “Problems on Algorithms”∗
Ian Parberry† Department of Computer Sciences University of North Texas April 7, 1998
This note contains corrections to Parberry [5]. An uptodate copy of this document and other useful information can be found on the WorldWide Web (Parberry [7]). I welcome further errata and comments from readers — I prefer electronic mail, but if you must use more primitive modes of communication, full contact information can be found in [6]. Then anyone who leaves behind him a written manual, and likewise anyone who receives it, in the belief that such writing will be clear and certain, must be exceedingly simpleminded. — Plato, Phaedrus
General Errata
Apparently I couldn’t make up my mind whether the keyword “return” in my algorithms should be in bold face or Roman type. It appears in bold face in Chapters 5, 7–9, and 13; in Roman face in Chapters 5, 6, and 10. Out of 71 occurrences, 39 are in bold face. A foolish consistency is the hobgoblin of little minds. — Ralph Waldo Emerson
Speciﬁc Errata
Page 3, Line 1: The ﬁrst comma should be a period. Page 6, Line 14: The second “but” should be deleted. Page 7, Last Paragraph: Wilf’s textbook is now out of print, and is very diﬃcult to obtain in its original form. Fortunately, the author has made it generally available in postscript form from http://www.cis.upenn.edu/~wilf/index.html.
Copyright c Ian Parberry. This Errata may be reproduced without fee provided that the copies are not made for proﬁt, and the full text, including the author information and this notice, are included. † Author’s address: Department of Computer Sciences, University of North Texas, P.O. Box 13886, Denton, TX 762036886, U.S.A. Electronic mail: ian@cs.unt.edu.
∗
1
Page 10, Problem 22: That should be induction on n ≥ 2. If n = 0, the right side of inequality doesn’t make sense, and if n = 1 equality holds. Page 12, Problem 47: This problem does not have a comment associated with it; the should be deleted. Page 16, Line 10: In the set V , the ﬁrst occurrence of v2 should be v1 . Page 16, Problem 67: I intended the graph here to be undirected and with no selfloops, but this did not make it into the deﬁnition. I need to add to the deﬁnition of a graph at the start of Section 2.10 the extra property that V is an ordered set and for all (u, v) ∈ E, u < v. Page 20, Hint to Problem 37: The comma at the end of the second sentence should be a period. Page 21, Solution to Problem 8: The fourth line of the multiline equation should begin with “=”, not “−”. Page 22, Solution to Problem 37, Line 5: Change “all values up to n − 1 cents” to “all values from 8 to n − 1 cents”. Page 25, Comment to Problem 13, Line 6: Change “by Problem 1” to “by Problem 11”. Page 35, Solution to Problem 133: On line 4, the denominator of the fraction should be 6, not 2. Page 40, Problem 241: This problem should ask you to show that 2n − 3 log n − 1 ≤ T (n) ≤ 2n − log n − 1. Page 40, Problem 242: That should be T (1) = 0, and T (n) = T ( n/2 )+T ( n/2 )+n−1 (otherwise it is inconsistent with the answer given on p. 41); it is the number of comparisons used by mergesort. Page 45, Line 3: “skip it is” should read “skip it if”. Page 52, Problem 266: In Line 1 of the algorithm, the Roman “n” in the return statement should be an italic “n”. Page 52, Problem 269: In Line 2 of the algorithm, the ﬁnal Roman “y” in the return statement should be an italic “y”. Page 56, Hint to Problem 258: The loop invariant is xj = y0 j−1 k, for all j ≥ 1. k=y Page 56, Solution to Problem 250: Line Line Line Line Line Line 4: “line 8” should read “line 7”. 6: “lines 3–7” should read “lines 3–6”. 14: “line 6” should read “line 5”. 15: Delete the parenthetical comment “(see Problem 93)”. 17: “line 5” should read “line 4”. 15: Insert “(see Problem 93)” immediately before the period.
Page 64, Hint to Problem 283: Insert a period after the problem number. Page 62, Problem 299: “line 3” should read “line 5”. Page 68, Problems 317–318: When I wrote Problem 317 I assumed that the standard recursive algorithm described at the start of Section 7.1 achieved this bound. But it doesn’t — it requires 5n/3 − 2 comparisons when n = 3 · 2k . To ﬁx it, simply ensure that the sizes of the two recursive splits are not both odd. (Thanks to Mike Paterson for pointing all this out). Problem 317 should be rewritten, or a hint added to help 2
make this clearer. The algorithm of Problem 318 almost meets the required bound. It makes 2 more comparisons than necessary, and should be rewritten slightly to save them (hint: consider the second and third comparisons). Page 68, Problem 320: The second question is not worded very clearly. The algorithm could be called upon to solve duplicate subproblems at the ﬁrst level of recursion, given suitable inputs (can you ﬁnd one?), but I was looking for more. There comes a point before the bottommost level of recursion at which the algorithm is obliged to solve duplicate subproblems. This is also the key to the algorithm in Problems 328330. Page 68, Problem 321: This problem has a comment, but is missing the . Page 68, Problem 322: The solution to this problem is a little bit more subtle than I ﬁrst thought. It begins: “Break the nbit number into n/m mbit numbers.” The remainder of the analysis from that point is straightforward, but in that ﬁrst step lies a small diﬃculty. It can be done by repeatedly shifting the large number to the right, removing single bits and reconstructing the pieces in m registers. One needs to keep two counters, one of which runs from 0 to m, and the other oﬀ which runs from 0 to n/m. Consider the second counter. A naive analysis would show that this requires O(n/m) increment operations, each of which takes O(log n − log m) bit operations. However, one needs to notice that the total is O(n/m) amortized time (see Problems 476, and 477). I need to increase the diﬃculty level of this problem and add a hint to this eﬀect. √ √ Page 74, Problem 340: Change n to N . Page 78, preamble to Section 7.7: Line 4 of the depthﬁrst search algorithm should read “mark w used”. Page 79, Problem 366: This problem should be deleted. There is no known way to ﬁnd triangles in time O(n + e), using depthﬁrst search or otherwise. Of course, it can be solved in time O(ne). This problem could be replaced with the much easier one of ﬁnding a cycle (of any size) in a directed graph in time O(n). Page 80, Problem 371: The array A should contain integers, not natural numbers. Otherwise the problem is trivial. Page 82, Hint to Problem 361: Delete the second adjacent occurrence of the word “in”. Page 85, Solution to Problem 369: Perhaps this should be called a comment, not a solution. Page 87, Preamble to Section 8.1: In Line 6 of the algorithm, the Roman “m” should be replaced by an italic “m”. Page 88, Problem 384, Line 3: The second occurrence of “M2 ” should be “Mi+2 ”. Page 88, Problem 390, Line 3: “M2 ” should be “Mi+2 ”. Page 90, Preamble to Section 8.3, Lines 11–12: The operation “member” should be “search” (two occurrences). Page 91, Preamble to Section 8.4, Line 4: “all pairs shortest paths” should be “allpairs shortestpaths”. Page 98, Problem 426: The function to be maximized should be: R[c1 , ci1 ] · R[ci1 , ci2 ] · · · R[cik−1 , cik ] · R[cik , c1 ]. Page 99, Hint to Problem 415: The last three lines are wrong, and should be deleted. Page 106, Problem 447: The stated proposition is wrong! Here is a counterexample. The 3
problem should be changed to “prove or disprove”.
2 2 1 1 4 3 1
2
Page 107, Problem 449: The running time should be e log k. Page 107, Problem 450: The third line of the putative variant of Prim’s algorithm should have a period after the line number. Page 117, Line 3: “line 8” should read “line 7”. Page 119, Preamble to Problems 489, 490: The line numbering in procedure permute(i) is totally wrong. The lines should of course be numbered consecutively. 2 Page 124, Problem 513, Line 4: The running time should be O(n8n ) in line 4. One of the two consecutive instances of the word “for” in the last line should be deleted. Page 129, Problems 529, 530: Both of these problems should be ﬂagged . A comment should also be added explaining that exhaustive search is unnecessary — the ﬂightassignment problem can be solved in polynomial time by reducing the problem to that of ﬁnding a maximumweight matching. For a polynomial time algorithm for the latter, see, for example, Gabow [3], and Gabox and Tarjan [4]. Page 131, Hint to Problem 510: The series should read ∞ 1/i! = e − 1 ≈ 1.718. i=1 Page 131, Hints to Problems 517, 518: These hints are listed in the wrong order. Page 131, Hint to Problem 518: A note should be added that the reader should read ahead to p. 127 for the deﬁnition of dominating set. Either that, or Problem 518 should be moved after Problem 526. Page 131, Solution to Problem 487, Line 1: The sentence “Line 6 can be done in time O(n).” should be replaced by the observation that the work in all of the other lines can be done in time O(n). A comment should be made that the tail of “ + d(n − 1)” in the recurrence for T (n) is used because it makes the solution very simple. If the obvious “ + dn” is used, then the recurrence becomes very diﬃcult to solve. Page 135, Preamble to Section 11.1: Heap operations have not been deﬁned. A heap is a set S in which the following operations can be performed: insert(x) insert x into S, that is, S := S ∪ {x} findmin return the smallest value in S deletemin return and delete the smallest value in S Page 135, Problem 535: The should come after the . Page 136–137, Problem 543: This problem should be moved to Section 11.5. Page 137, Problem 544: This problem should ask for insert, deletemin, and deletemax in O(log n) time, and findmin and findmax in O(1) time. Page 138, Problem 550: Insert “is correct” after the word “algorithm”. Page 140, Line 3: Capitalize the word “the” at the start of the parenthetical comment. 4
Page 141, Preamble to Section 11.3, Line 1: Change the ﬁrst occurrence of “in” to “is”. Page 142, Problem 562, Line 1: Insert “of” after the word “sequence”. Page 143, Problem 570: This problem has a hint and a comment, but lacks the appropriate icons . Page 144, Problem 573: Change “as as” to “as” in line 2. Page 145, Hint to Problem 544: This hint is misleading, and should be deleted. (Less signiﬁcantly, n/2 should be n/2 on lines 1 and 3). Page 145, Hint to Problem 570: Insert a period after the problem number. Page 150, Preamble to Section 12.3: “Riemann” is misspelled ‘Reimann” (also in the Index on p. 178). Fermat’s Last Theorem may have been proved recently. Page 155, Solutions: These aren’t really solutions, just bibliographic notes. Perhaps they should be moved to the Comments section. Page 155, Solutions to Problems 599–600: Change “this problem” to “these problems”. Page 157, Problem 620: This problem should be changed from ﬁnding the median to that of halving, which is dividing n values into two sets X, Y , of size n/2 and n/2 , respectively, such that for all x ∈ X and y ∈ Y , x < y. A comment should be added explaining that this problem is merely practice in using decision trees, since there is a 3n/2 lower bound using the adversary method due to Blum et al. [2]. Page 164, Problem 665 This problem is much harder than I thought. An upper bound of log3 n + 1 weighings is easy. A little more thought will get you log3 9n/4 weighings. The best algorithm known to me uses log3 2n + 2 weighings (see Aigner [1]). Page 171, references [46, 47]: For some reason BibTEX chose to list Knuth Vol. 3 before Vol. 2. Page 171, reference [52]: The authors’ initials are missing — H. R. Lewis and L. Denenberg. Page 172, references [66, 67]: These appear to be the same book. The correct reference is [67]. Page 172, reference [75]: The second occurrence of the word “Edition” should be deleted.
Acknowledgements
The author is very grateful to the following individuals who helpfully pointed out many of the above errors in the manuscript: Timothy Dimacchia, Sally Goldman, Narciso MartiOliet, G. Allen Morris III, Paul Newell, Mike Paterson, Steven Skiena, Mike Talbert, Steve Tate, and HungLi Tseng.
References
[1] M. Aigner. Combinatorial Search. WileyTeubner, 1988. [2] M. Blum, R. L. Floyd, V. Pratt, R. L. Rivest, and R. E. Tarjan. Time bounds for selection. Journal of Computer and System Sciences, 7:448–461, 1973. 5
unt. N.edu/ian.html. Tarjan. 23(2):221–234. Journal of the ACM. A WorldWide Web Document with URL http://hercule.[3] H. A WorldWide Web Document with URL http://hercule.edu/ian/books/poa. An eﬃcient implementation of Edmond’s algorithm for maximum matching on graphs. Faster scaling algorithms for general graphmatching problems. Prentice Hall. Created 1994. 1995. E. [4] H. Gabow and R. [5] I.
6
. Ian Parberry’s Home Page.csci. Parberry. 1991. Journal of the ACM. Parberry.csci. [7] I. Parberry. [6] I.unt. N. 1976. 38(4):815–853. Gabow. Problems on Algorithms. Created 1994. Problems on Algorithms.
Supplementary Problems
By Bill Gasarch
.
ﬁrst write an−1 in base n − 1. 4. (2) if player I removes 2 then player II removes 2. So a21 = x − 1 = 1097 − 1 = 1096. and Player II wins).” (b) Modify the game so that you can remove 1. Fill in the following sentence and justify: “Player I has a forced win iﬀ X(n). (Example game with n = 10: player I removes 3 (so there are 7 left). called player I and player II. 2. Player II has a FORCED WIN if. player II can win. k − 1. Let an = x − 1. no matter what player I does. player I removes 2 (so there are 3 left). player II removes 2 (so there are 5 left). 6.” (c) Let k ∈ N . 5. (EXAMPLE: if a20 = 1005 then I write 1005 in base 20 as 2T 5 = 2 · 202 + T · 20 + 5 · 1 where T is the symbol for ten.) For which natural numbers A will the sequence go to inﬁnity? 2. 2. .) (a) Fill in the following sentence and justify: “Player I has a forced win iﬀ X(n). Player II’s move is called a WINNING RESPONSE. Modify the game so that you can remove 1. In each move the player whose turn it is will remove 1. Then let x be the number that is that string of symbols if it was in base n. player II removes 3 (so there are 0 left. NO MATTER WHAT PLAYER II does. or 7 sticks.) Player I has a FORCED WIN if there is a move he can make as his ﬁrst move so that. . (2) To obtain an . So x = 2 · (21)2 + 10 · 21 + 5 · 1. player I can win the game. (Example: if n = 4 then player II has a forced win: if (1) player I removes 1. .Mathematical Induction
1
Games
1. Let A be any natural number. there is a move player II can make so that after that point. whatever player I does for the ﬁrst move. Player I will go ﬁrst. The player who removes the last toothpick wins. or k sticks.” 1
. This is called a WINNING FIRST MOVE.2. Here is a game: there are n toothpicks on a table. or 3 toothpicks from the table. We deﬁne a sequence as an as follows: (1) a10 = A. (3) if player I removes 3 then player II removes 1. There are two players. . 3. Fill in the following sentence and justify: “Player I has a forced win iﬀ X(n). player II removes 3. To specify a winning response you need to specify responses to all moves.
. Fill in the following sentence and prove it: “Player XXX can always win. for all but a ﬁnite number of n ∈ N . and we consider (a. (a) Consider the following game played between FILLER and EMPTYIER There is initially a box with a ﬁnite (nonempty) number of balls in it.” (d) Consider the same game as above except that the balls are labeled with ordered pairs of numbers. if it has property NOT(XXX) then the EMPTYIER can win. EMPTYIER can win. 3. . . or 4 sticks. (There may be many balls of the same rank. .” (e) Let a1 . EMPTYIER’s move consists of taking a ball from the box. a box could have 100 balls of rank 1. FILLER wants the box to never be empty. .) In every round EMPTYIER makes the ﬁrst move. b) to be of lower rank than (c. Show that there exists A ∈ N and X ⊆ {0. FILLER then counters be replacing the ball with a ﬁnite number of balls of lower rank. 888888888876234012 balls of rank 8.) EMPTYIER wants the box to be empty eventually. called its rank. however. Modify the game so that you can remove a1 . he could replace a ball of rank 1000 with 999999999999999 balls of rank 999. and 4 balls of rank 1. .(d) Let k ∈ N . d) if either a < c or a = c and b < d. . Fill in the following sentence and prove it: “Player XXX can always win.” 2
. . 2. . Fill in the following sentence and prove it: “Player XXX can always win. if n sticks are in the pile then Player I wins iﬀ (∃x ∈ X)[n ≡ x (mod A)].” (c) Consider the same game as above except that the balls are labeled with positive rationals. . (b) Consider the same game as above except that the balls are labeled with integers. . Show that no matter what the initial box of balls has in it. ak sticks. (For example. Fill in the following sentence and justify: “Player I has a forced win iﬀ X(n). For example. Each ball has a natural number on it.” (e) Fill in the following sentence and prove it: “If the game can be played with a set A with ordering ≤ such that the ordering has property XXX then the FILLER can win. ak ∈ N . and 4 of rank 1000. 20000 of rank 2. A− 1} such that. Modify the game so that you can remove 1. .
Consider the following sequence: T (n) ≤ 3T (n − 1) + 10. Consider the following sequence: T (n) ≤ T (n − 1) + 3 T (n − 2) + 2n. 5. √ √ Find a constant c such that T (n) ≤ cn3/2 .4. 3. T (1) = 1 . (Hint: n − 2 < n. T (0) = 0. Consider the following sequence: A(n) ≤ A( n/3 ) + A( n/5 ) + 2n. 2. Find a constant c such that T (n) ≤ cn . The goal is to take the last stone. but must take at least one stone. Consider the following game of “NIM”. Products. Player I can remove 1. Assume that the following two statements are known: 3
.) 4. Consider the following sequence: √ T (n) ≤ T (n − 2) + 5 n. The following is a winning strategy for the ﬁrst player: Take stones from the larger pile so as to make the two piles have equal size. At each turn a player may take as many stones as he or she likes from one (and only one) pile. Consider the following 2player game. There are two piles of stones of diﬀerent sizes. Summations. T (0) = 0. or 6 sticks. Players alternate turns.
2
Sequences. Prove that this really is a winning strategy using mathematical induction. T (0) = 0 . Let P (n) be a statement about an INTEGER n. T (0) = 0 . 5. and Recurrences
1. T (1) = 1 . There are n sticks on the table. Show that: if n ≡ 1 mod 2 then player I has a winning strategy. The player who removes the last stick WINS. Find a constant c such that A(n) ≤ cn.4.2 or 3 sticks. Show that T (n) ≤ 16n2 . Player II can remove 2.
Show that (∀n ∈ N )[an ≡ 1 mod 5]. P (q) is true. 10. No explanation is needed.(a) P (10). Assume that (∀n ≥ 5)[P (n − 1) → P (n + 1)] is known. 12. Let P (n) be some property of the natural number n (e. P (n) could be ‘217n ≡ 10 (mod 8)’). Let an be deﬁned as follows: a0 a1 a2 an = = = = 6 21 11 3an−1 + an−3 + 10n + 2
7. For which n ∈ N can we conclude P (n) is true? Express your answer clearly. Show that (∀n ≥ 1)[an ≡ 0 (mod 3)]. a0 = 3 a1 = 6 (∀n ≥ 3)[an = (an−1 + an−2 )2 + 6n] 13. Prove your claim by induction. Let an be deﬁned as below. Let P (n) be some property of the natural numbers. Find a function i=0 f (n) such that (∀n)[an ≤ f (n)]. Assume that P (10) and P (11) are known to be true. Assume that the following are known (1) For all primes q. Prove your result.g. 8. and (2) (∀n ∈ N )[P (n) → P (n2 )]. Deﬁne the sequence an by a0 = 1 and an = n−1 ai . (b) (∀k)[P (k) → P (k − 2)]. For which integers n do we know that P (n) is true? 6. 11. Find a number a such that the following is true: (∀n ∈ N. 4
. For which n can we conclude P (n) is true? Brieﬂy explain. n ≥ a)[n2 + 5n + 6 < 2n ]. Prove (∀n ∈ Z)[n4 + 1 ≡ 0 (mod 5) → n ≡ 0 (mod 5)].
9.
Assume that the following are known (a) P (−23). a2 = 13. (That is. a2 = 104. . Prove i=1 that your answer is correct by induction.) 18. A sequence b0 . a1 = 49. a1 = 4. b1 .) Show that (∀n ≥ 0)[an ∈ SQ]. and (∀n ≥ 3) an = (a Show that (∀n ≥ 0)[an ≡ 4 (mod 5)]. 19. (b) (∀z ∈ Z)[P (z) → P (z + 3)]. Let P (z) be some property of the integers. 15. Let a0 = 9.
. . a5 = 16. a1 = 12. (No justiﬁcation required. .
Let SQ = {n ∈ N  (∃x ∈ N )[n = x2 ]}. SQ is the set of all squares. and (∀n ≥ 2) [an = an−1 + an−3 − an−5 ] . Let an be deﬁned as follows: a0 = 10. a4 = 15. 17. a3 = 64. Describe the set of all a ∈ N such that the following is true: (n2 + 3n + a ≡ 0 (mod 5)) → (n ≡ 0 (mod 5)). Deﬁne a sequence via a1 = 4 and (∀n ≥ 2)[an = a2 ]. 5
n 2
(∀n ≥ 1)[an ≤ 4 · 3(3 ) ]. For which z can we conclude P (z) is true? (Express your answer clearly!) No justiﬁcation required. 16. a2 = 100. n−1 Show that 20. and (∀n ≥ 4) an = 25an−1 a
log n
]. Find a. is INCREASING if (∀x)(∀y)[x < y → bx < by ]. Let a0 = 4. b such that S(n) = an3 + bn.14. Let S(n) = n i(i + 1). a3 = 14.
n
)3 + a
√
n
+ 11 .
” No justiﬁcation required. T (n) ≤ dn log n. and (∀n ≥ 2)[an = 5an−1 + 20an−2 ]. No justiﬁcation required (but show work for partial credit). 27. b and c is it possible to ﬁnd a d such that for all n. a1 = 90. C ∈ N . Find POSITIVE constants A. . 26.
q (b) (∀q ∈ Q)[P (q) → P ( 2 )]. Solve the recurrence exactly. T (1) = 5. Then express the answer more simply using Onotation. B. Initial conditions T (0) = 0. 28. 23. Let P (q) be some property of the rational q. 29. NO PROOF REQUIRED. Consider the following recurrence. Let T (n) = 2T (n − 1) + T ( n ) + n. 24. 25. Write down the set A clearly but DO NOT use the notation “. T (n) = 7T (n − 1) − 12T (n − 2). Assume that the following are known (a) P (1). T (n) ≤ dn log n. For what values of a.
Let A be the set of all q such that we can conclude P (q) is true. 22. Use mathematical induction to show that 6
. . Find a value of d such that for all 4 4 n. Let T (n) ≤ T ( 3 n) + T ( 1 n) + 4n. Let a0 = 10. Find a simple function f such that T (n) = Θ(f (n)).21. Let T (n) be deﬁned by T (1) = 10 and T (n) = T (n − log n) + n. Prove by induction that if T is a tree then T is 2colorable. Prove what you can about this 2 recurrence. such that (∀n ≥ 0)[A · (C − 1)n ≤ an ≤ B · C n ]. Let T (n) ≤ T (an) + T (bn) + cn.
Consider the following code fragment. a. Assume every node of a tree with n nodes has either 0 or 3 children. Give a closedform solution for T (n) (i. Assume that Christmas has n days. How many leaves does the tree have? Prove your answer. no summations) by simplifying your summation. Express T (n) as a summation (actually three nested summations).e. you should do some research. 34. Simplify the following summation:
n n n−j
1
i=1 j=i k=1
32. Show your work. Let T (n) denote the number of times ‘foobar’ is printed as a function of n. 31.] 7
. Exactly how many presents did my “true love” send me? [If you do not understand this question. b.(a)
n
i(i + 1) =
i=1
n(n + 1)(n + 2) 3
(b)
n i=0
i2i−1 = (n − 1)2n − 1
30. Simplify the following summation:
n n 2j
1
i=1 j=n−i k=j
33. for i = 1 to n do for j = i to 2*i do for k = nj+1 to n do output ‘foobar’ a.
38. Consider the following code fragment. b. 39. Consider the following code fragment. exactly how many presents did my “true love” send during the twelve days of Christmas? 35. for i=1 to n step 2 do for j=1 to i do output ‘foobar’ Let T (n) denote the number of times ‘foobar’ is printed as a function of n. for i=1 to n do for j=i to 2*i do output ‘foobar’ Let T (n) denote the number of times ‘foobar’ is printed as a function of n. Show your work. Evaluate the sum 36. Show your work. Consider the following code fragment. Do not simplify the summation.
· 3k
37. Evaluate
n k=1 4 ∞ k=0 (3k
− 1)x3k for 0 < x < 1. (b) Simplify the summation. Be careful your “formula” works for n both even and odd. (a) Express T (n) as three nested summations. Express T (n) as two nested summations. for i=1 to n/2 do for j=i to ni do for k=1 to j do output ‘foobar’ Assume n is even. Express T (n) as a summation (actually two nested summations). a. 8
. Let T (n) denote the number of times ‘foobar’ is printed as a function of n. Simplify the summation. Using part (a).b.
c. How well can you do? 41. 46. Simplify your answer. Write down the “brute force” algorithm in pseudocode. d. [If you do this right. 9
. How many times is the inner loop executed? Write it using summations. Justify your work. for i=1 to n do for j=1 to lg i do print ‘foobar’
Let T (n) denote the number of times ‘foobar’ is printed as a function of n. (a) Express T (n) exactly as two nested summations. Approximate
n 3 k=1 k
using integrals. (c) Give good upper and lower bounds for T (n).] e. and T (x) is 0 if x is a negative integer. a. Make sure to get an upper and
n 4 k=1 3k−2 ∞ 1 k=1 k 3/2 . Write down an English description of the “brute force” algorithm for the “maximum contiguous rectangle problem”.40. the solution involves very little calculation. b. Given an m × n array of (positive and negative) numbers. 42. Consider the following code fragment. (b) Simplify down to one summation. ﬁnd the largest sum of values in a (contiguous) rectangle. 45. Approximate lower bound. Use integrals to give upper and lower bounds for √ 44. Find the solution for the recurrence relation T (n) = T (n − c) + k where both c and k are integer constants. One or two sentences should suﬃce. Appoximate n j=0 j using an integral. Find a better algorithm for the “maximum contiguous rectangle problem”. n is a natural number.
43.
10
. Assume n is a power of 4. Justify your answer. 55. Justify your answer. 54. Solve the recurrence T (n) = T (n − 1) + 3n − 2 (T (0) = 0) exactly. Show your calculations. Use mathematical induction to verify your solution. Assume n is a power of 2. Justify your answer. Consider the following recurrence. 52. Solve the recurrence T (n) = T (n/3) + 2 n > 1 0 otherwise
assuming n is a power of 3. Solve the recurrence T (n) = T (n − 2) + n 0 n>1 otherwise
assuming n is even. Show your calculations. b.47. 48. 49. Show your calculations. Solve the recurrence T (n) = 4T (n/3) + 2n − 1 (T (0) = 0) exactly. Justify your answer. Assume n is a power of 3. Solve the recurrence T (n) = 5T (n/2) + 2n − 1 (T (1) = 7) exactly. Justify your answer. Simplify as much as possible. Solve the recurrence exactly using the iteration method. 50. 53. deﬁned for n a power of 5: T (n) = 19 if n = 1 3T (n/5) + n − 4 otherwise
a. Solve the recurrence T (n) = 3T (n/4) + 2n − 1 (T (0) = 0) exactly. Solve the recurrence T (n) = T (n − 1) + 2n − 1 (T (0) = 0) exactly. Solve the recurrence T (n) = T (n/5) + 2 n > 1 3 otherwise
assuming n is a power of 5. 51.
Obtain exact solutions to the following two recurrences. T (n) = 3 if n = 1 2 + 1 otherwise 7T (n/2) + 4n
(b) Let n be a power of 4. a.56. (a) Let n be a power of 2. 58. Let n be a power of 4. b. T (n) = 2T (n/6) + 5n (T (1) = 7). Obtain exact solutions to the following two recurrences. a. deﬁned for n a power of 5: T (n) = 7 if n = 1 4T (n/5) + 2n − 3 otherwise
a. Assume n is a power of 6. T (n) = 5T (n/3) + 2n (T (1) = 7). T (n) = 3T (n/4) + 2n (T (0) = 0). Assume n is a power of 3. Obtain exact solutions to the following two recurrences. b. Use mathematical induction to verify your solution. T (n) = 4T (n/3) + 2n (T (0) = 0). Let n be a power of 2. Simplify as much as possible. 59. a. Consider the following recurrence. T (n) = b. T (n) = 4 if n = 1 3T (n/4) + 5n − 2 otherwise 11
. T (n) = 3 if n = 1 2T (n/4) + 4n otherwise 4 if n = 1 2 otherwise 5T (n/2) + 3n
60. 57. Solve the recurrence exactly using the iteration method. b. Obtain exact solutions to the following two recurrences.
Let T (n) = 2T (n/2) + 2 lg n if n > 1 0 if n = 1
k i=1 ri
= 1? What if
k i=1 ri
> 1?
a. Let a > 0 be a constant.e. (a) Let r1 . i. T (n) ≤ cn for some constant c. b. T (n) = 0 if n = 0 T ( n/2 ) + T ( n/4 ) + 3n otherwise
Find a constant c such that T (n) ≤ cn.
12
. b. and c. 63.e.) (a) Draw S0 .
i=1
Show that T (n) is linear. 0 < ri < 1. What can you say about c? (b) What if 64. . An S0 tree is a single node.61. (These are the trees that create the largest height for as few nodes as possible for the UNIONFIND algorithm with balancing. T (n) = 0 if n = 1 T ( 2n/3 ) + T ( n/6 ) + 5n otherwise
Use constructive induction to ﬁnd a constant c such that T (n) ≤ cn. Try to make c as small as possible. Use constructive induction to show that T (n) = an + b lg n + c. and S3 . i > 0. S1 . such that k ri < 1. An Si tree. Consider the following recurrence. is created by taking two Si−1 trees and making the root of one a child of the root of the other. rk be a list of positive fractions. You may also want to draw S4 for yourself. . r2 . . . Find constants a. 62. i. Let i=1 T (n) ≤
k
T ( ri n ) + an. Solve the recurrence using the iteration method. 65. S2 . Consider the following recurrence.
c. Justify your answer. 2. Write down a recurrence for how many nodes there are at depth d in Fn ? Note that there are two parameters. . b. (c) Make a conjecture for how many nodes are on level i of Sk .
13
. . Prove that your guess is correct using mathematical induction on your recurrence. An Fn+1 tree is created by taking two Fn trees and making the root of one the two trees the child of the other tree.(b) The nodes on level i are all nodes distance i from the root.) (d) Prove your conjecture. k (and thus have k + 1 levels). 66. Consider an F tree created as follows. a. Tree Sk will have nodes on levels 0. . Recall that the depth of a node is its distance from the root (so the root has depth 0. its children have level 1. etc. . Here are the ﬁrst few F trees. For S3 . How many nodes are there in an Fn tree.). d. and 3. (You may also want to look at S4 . 1. Guess a solution to your recurrence (by looking a few trees). how many nodes are on levels 0. An F0 tree is a single node.
6. (HINT: Express x as 1 + where 0 < < 1. For which real numbers x does x satisfy the following two conditions: (a) 1 < x < 2. 2.)
14
. Determine all real numbers following statement is true: r + = r . For which x is 4x = 4 x ? 5. For which positive reals x does x +
1 6
such that the
= x+
1 2
?
3. Find a number x such that ( x )2 < x2 . 4. Let r be a real number. For which real numbers x does the following hold: x +
1 3
= x+
1 2
.Floors and Ceilings 1. and (b) x 2 = x2 .
Let f be the function that takes a number x = dn dn−1 · · · d0 (these are the digits of x base 10) to the sum of the 2001 powers of the digits (e. or tuna ﬁsh. peanut butter and jelly. 5..Combinatorics 1.g. for any x. the last digit of 39489 is 9).} is ﬁnite. f (f (x)). Show that. . egg salad. Find a number x such that the statement a is TRUE and statement b is FALSE. (b) There must exist x + 1 people in the room that are from the same country. or a coconut. . Assume that there are 900 people in a room. Fruit is either an apple. (a) How many diﬀerent lunchs can Bill make for Carolyn ? (b) If Carolyn does not like to have egg salad with apple pie. or tuna ﬁsh with an apple. a brownie. a fruit. Find the largest x ∈ N such that the following statements is true: If A ⊆ N and A = 59 then there are at least x numbers in A that have the same last digit.. For this problem we express numbers in base 10. Every Sunday Bill makes lunches for Carolyn. or an apple pie. (a) There must exist x people in the room that are from the same country.g. The last digit of a number is the rightmost digit of the number (e. . Sandwich is either ham and cheese. f (327) = 32001 + 22001 + 72001 ). f (f (f (x))). Lunch consists of a sandwich. (c) Why does Bill call Carolyn his ‘gnilrad’ ? 3. the set {f (x). Brieﬂy explain WHY statement a is TRUE and WHY statement b is FALSE. Can you place 24 rooks on an 8 by 8 chessboard so that if you remove all but 4 then there will always be 2 that attack each other? How about 25? 4. Desert is either pretzels.
15
. Assume that there are at most 200 countries in the world. and a desert. an orange. then how many lunches can Bill make for Carolyn. 2.
2 (c) If the input i is one of { n + 1.” There are 4 × 13 = 52 such possible combinations. .H]? 3. You are given a deck of ﬁfty two cards which have printed on them a pair of symbols: an integer between 1 and 13. n. What is the average cost (i. followed by one of the letters “S. . . (b) If the input i is one of {1. Now what is the average cost of ﬁnding [11. 2. but otherwise all cards are randomly distributed. . n}. . . .Probability 1. . number of cards turned over) of ﬁnding the card [11.e. with probability 1 1≤i≤ n n 4 n 2 P (r = i) = n <i≤ n 4 2 1 n 2n 2 <i≤n For the algorithm below perform an average case analysis. and you may assume that you have one of each type. . handed to you face down. n} then P takes 2i steps 2
16
. .H]? (b) Suppose you know that the ﬁrst thirteen cards have one of the letters “H” or “D” on them. 2. n } then P takes i4 steps. if (r ≤ n/4) then perform 10 operations else if (r > n/4 and r ≤ n/2) then perform 20 operations else if (r > n/2 and r ≤ 3n/4) then perform 30 operations else perform n operations Perform an average case cost analysis of this algorithm. . n is large. . (a) Suppose the cards are randomly distributed and you turn them over one at a time.” “H.” or “C.” “D. Let P be a program that has the following behavior: (a) The only inputs are {1. and n is divisible by 2. . 2. . . . Let r be a random number taking on the values 1.
.Find a distribution on the inputs such that every input has a nonzero probability of occurring and the expected running time is θ(n17 ). P (4) = 4/21. . . n is large. and the probability that it in position i. with probability 1/2 that it is contained in (n/3.
(a) What is the probability that a toss of these two dice produces a sum of either seven or eleven? (b) What is the expected value of the sum of the values of the dice? (c) the product of the values of the dice? 5. whereas the worst and average case costs for binary search are log2 n and (log2 n)/2. For what values of p are we better oﬀ using linear search. and probability 1/4 that it is contained in either [1. we know that in general. . n}. Consider an algorithm that has the following behavior: if (a ≤ n/3) then perform i operations else if (a > n/3 and a ≤ 2n/3) then perform log3 n operations else perform n2 operations Perform an average case cost analysis of this algorithm. 3 17
. P (3) = 1/7. . 2. Let a be a random number taking on the values 1. n]. . . (b) If the input is one of {1. where the probabilities for the loaded die are P (1) = P (2) = P (5) = P (6) = 1/6. 2n/3]. . n } then P takes log n steps. Suppose we have one loaded die and one fair die. . . 2. . Let P be a program that has the following behavior: (a) The only inputs are {1. 4. . 6. Suppose instead that we have a special situation where the probability that the item we are seeking is ﬁrst in the list is p. the worst case cost for linear search is n. and its average cost is is (n + 1)/2. and n is divisible by 3. n. Given n numbers in a sorted list. 2 ≤ i ≤ n. n/3] or (2n/3. in the sense that the average cost for linear search is higher than that for binary search? 7. is (1 − p)/(n − 1).
is (1 − 2p)/(n − 2). 3 ≤ i ≤ n. . but do not simplify it. Let A[1.n] be a sorted list. For what values of p will the average case for linear search do better than log n? (Obtain an inequality involving p. .. but the expected time is θ(n). and the probability that it in position i. (c) Roll a 3 sided die and another 3sided die. and p3 of being 3. . with probability 1/2 that it is contained in (n/3.
(c) If the input is one of { n + 1. Consider an algorithm that has the following behavior: if (a ≤ n/3) then perform a log a operations else if (a > n/3 and a ≤ 2n/3) then perform log3 n operations else perform n2 operations Perform an average case cost analysis of this algorithm. . n} then P takes 2n steps 3 Find a probability distribution on the inputs such that every input has a nonzero probability of occurring. The ﬁnal answer can be in Θ notation. . (a) Roll a 6sided die and a 4sided die. and the probability that it is in the second position in the list is p. . Both dice are fair. Let a be a random number taking on the values 1. . Suppose that we have a special situation where the probability that the item we are seeking is ﬁrst in the list is p. Multiply the two values together. and probability 1/4 that it is in (2n/3. Consider the following experiments.) 9. n]. . Both dice are fair. given a number x.) 10. p2 of being 2. p2 . For each of them ﬁnd the expected value and the variance. 8. n/3]. 2n/3]. and p3 . The ﬁrst die has probability p1 of being 1.(d) If the input is one of { 2n + 1. . Add the two values together. without summations. see if it is on the list. . (b) Roll a 2sided die and a 3sided die. . and probability 1/4 that it is contained in [1. Add the two values together. . 2n } then P takes n steps 3 3
18
. n. (In this case E and V will be in terms of p1 . We are pondering writing a program that will. The second die is fair.
. such that the probability of picking i is pi = A(n)i. . (a) What is the expected run time if the distribution is uniform? You may use Θnotation. (b) If the input i is ≤ log n then P takes 2i steps (c) If the input i is such that log n < i ≤ (d) If the input i is such that i > Answer the following questions.. and (2) for i > 10 we have the probability that it is in position i is pi = 1−10p . . . Assume we pick numbers from the set {1.) 19
n 2 n 2
then P takes n2 steps
then P takes n3 steps. n is large. . . Prob(i) = A(n) (where i2 A(n) is some function of n). (Formally the sample space is {1. n}. What is A(n)? 13. and its average cost is (n + 1)/2.) 15. . . (In this distribution all elements must occur with nonzero probability. we know that in general. Suppose instead that we have a special situation where the probability that the ith item is sought is pi . p2 . . and (1) for i ≤ 10 we have pi = p.11. . .
. We are going to pick a number from {1. . Given n (n > 10) numbers in a sorted list. . . We are going to pick a number i out of S with the following probability distribution. you will need to look at exact results. . the worst case cost for linear search is n. . n} There are constants p1 . n}. and a quantity A(n). n} and the random variable is X(a) = a2 . (b) Find a distribution where the expected time is Θ(n2 ). .2 . 12. . . . Once we have this number i we compute X(i) = i3. 2. . 14. (a) Show A(n) = O(1). 2. Let P be a program that has the following behavior: (a) The only inputs are {1. Let S = {1. (b) Find E(X)? (Use Θ notation. Let n be a large number. . . Using a calculator ﬁnd values of n > 10 and p < 1 such that linear search performs better than log n average case. whereas the average case for binary search is (roughly) log n (its actually (log n)− c for some constant c).) Find a probability distribution such that E(X) = θ(log n). What is the average n−10 case cost of linear search with this probability distribution (the answer will be in terms of p). and after we get a number we square it. For this problem do NOT use Θnotation. n}.
Consider an ssided die. d. .) (b) Let a. What is the expected value of playing this game n times. Assume that f (i) = ib . c. and after we get a number we apply f to the result. Show that there exists a distribution such that E(X) = Θ(na ). . 2. (d) What is the variance and standard deviation of the sum of n (independent) rolls of this die. some of which do not use the assumption 1 < a << b and some of which do. .) 20
. Assume we pick numbers from the set {1. b. (c) What is the variance and standard deviation of one roll of this die. Let n be a large number. s. What is the expected value of playing this game once. (e) What is the probability of two rolls of this die summing to an integer t (2 ≤ t ≤ 2s)? (Give your answer as a function of s and t. n} and the random variable is X(i) = f (i). What is the variance of playing this game once n times. and each side has equal probability (1/s) of being rolled. .e. (There are several valid solutions to this problem. (Formally the sample space is {1. Let f be a function. a.) (a) What is the expected value if f (i) = i2 and the distribution is uniform? (Use Θ notation. b be reals such that 1 < a << b (a is much less than b). (b) What is the expected value of the sum of n (independent) rolls of this die. What is the standard deviation of playing this game once. e. 18. . What is the standard deviation of playing this game once n times. . . You win d dollars if the two dice land on the same value d (i. . . you throw double d’s). . (a) What is the expected value of one roll of this die. You play a game of throwing two six sided dice (with integer values 1. . f. . . . . n}. You lose a dollar if the two dice have diﬀerent values. where the sides are marked with the numbers 1.) 17. What is the variance of playing this game once. 6). .16.
What is the expected value? 21
. 22. and a 3 has probability r = 1−p−q of being rolled. What is (a) E[H]. 20. the variance of H? (c) the standard deviation of H? (d) the probability that H > 2? Show your calculations. 23. a 2 has probability q of being rolled. which may (or may not) be the same as the ﬁrst. What is the probability that initially there were more blue socks. Professor Pascal wakes up in the morning. a. He pulls two socks randomly out of the drawer. If it does not work he again tries a combination randomly. It turns out that they are both blue. He remembers that either he has six blue socks and four red socks in the drawer or he has four blue socks and six red socks in the drawer. A combination lock works by turning to the right three times and stopping a on particular integer from 1 to 60 (inclusive). A coin is tossed n times. (a) Assume you roll the dice once. Assume a lock has C combinations. A thief tries a combination randomly. and then turning to the right and stopping on a ﬁnal integer from 1 to 60. turning to the left two times and stopping on a particular integer from 1 to 60. On average.) 19.(f) What is the probability of two rolls of this die both being s given that you know that at least one of the two rolls was an s. Consider a threeside die where a 1 has probability p of being rolled. How many possible combinations are there. each time with an independent probability p of coming up heads and 1 − p of coming up tails. He keeps doing this until he ﬁnally opens the lock. but he cannot remember which (assume each is equally likely). Let H be the number of heads occurring. the expected number of heads? (b) V [H]. how many times will he try the lock? 21. (Note: The roll that was an s is NOT necessarily the ﬁrst roll.
a. At least one is a boy. What is the variance? c. The older one is a boy. Assume that (even within a family) children’s sexes are independent. What is the standard deviation? (b) Assume n independent such dice are rolled and their values are summed. Smith has two children. What is the expected value? b. Assume every child born has probability 1/2 of being a boy or a girl. What is the variance? c. (a) Mr. Johnson has three children. What is the standard deviation? Show your calculations. What is the probability that they are both boys? Why? (c) Mr. What is the probability that they are both boys? Why? (b) Mr. 24. Jones has two children.b. At least one is a boy. What is the probability that at least one is a girl? Why?
22
.
2. (Only consider the cases where 0 ≤ a. i=0 f4 (n) =
3. f2 (n) = ( n) log n. c are you better oﬀ reducing a? For which values of a. i=1 i f (n) = n log i. (a) Let a ≥ 0. c ≥ 10. f3 (n) = log2 ( n 2i ). Consider the recurrence T (n) = aT ( n ) + nc . f4 (n) = 3 12n 2 + 4n. n ik = Ank+1 + O(nk ). Let k be an integer such that k ≥ 1. Place the following functions into increasing asymptotic order.) 4. c are you better oﬀ reducing c? 23
. Find a function fa that does not involve summations such that n ai = θ(fa(n)). Let a.) (b) Let b ≥ 0. b. (a) (b) (c) (d) f (n) = n 1 . Find a function gb that does not involve summations such that n ib = θ(gb (n)). i=1 f (n) = log(n!). b. b. f2 (n) = n(log2 n)2 . For each of the following expressions f (n) ﬁnd a simple g(n) such that f (n) = Θ(g(n)).) (c) For what values of a and b does fa (n) = O(gb (n)). b. f1 (n) = n2 log2 n. Prove your result. Find the constant A such that the following holds. or c by 1. b. c are you better oﬀ reducing b? For which values of a. √ √ √ n f1 (n) = i=1 i. c be real numbers such that a.
n i i=0 2 .Asympototics 1. b. (The function gb might involve sevi=1 eral cases and might depend on b. If two or more of the functions are of the same asymptotic order then indicate this. b be real numbers such that a ≥ 0 and Let 0 ≤ b ≤ 1. i=1 i f (n) = n 1 . You want to make T (n) as small as possible. You are given the option of either reducing b a. b. Let a. f3 (n) = n log n. (The function fa might involve sevi=0 eral cases and might depend on a. i=1 6. For which values of a. Place the following functions into increasing asymptotic order. 5.
n i=1 i log i = Θ(f (n)). Let c > 0. (c) Does there exist a d such that.
13. (a) Find a function f such that (b) Find a simple function f such plain your answer.
Ex
12. c as above. 24
. for all a.7. (b) Show that if a + b = 1 then T (n) is bounded by O(n log n). Let a. b < 1. b. (a) Show that if a + b < 1 then T (n) is bounded by a linear function. b does the following hold?
n 2
n ia i=0 bi
= O(1). For each of the following expressions f (n) ﬁnd a simple g(n) such that f (n) = Θ(g(n)).
(b) For which constants a is it the case that 3 + 2n = O(an )? 11. (You should be able to prove your result by exhibiting the relevant parameters. T (n) = O(nd )? (Recall that Onotation just means that nd is an upper bound on T (n). Which of the following are true? (a)
n i i=1 3
= Θ(3n−1 ). and c > 0. Let a. Consider the recurrence T (n) = T ( n )+ T ( n )+ n. Let T be the function deﬁned by T (0) = 1 T (n) = 81T ( n ) + cn 3 Find a function g(n) such that T (n) = Θ(g(n)). Specify an inﬁnite number of values for (a. b) a b such that T (n) = Θ(n). but this is not required for the hw. (The function g may involve several cases depending on what c is. n i 2i i=1 5 + 3 .) (a) f (n) = (b) f (n) = (c) f (n) =
n 4 3 i=1 3i + 2i − 19i + 20. √ that n i=1 i log i = Θ(f (n)). b > 0. b be real numbers such that a. 8. n i i 19 i=1 3(4 ) + 2(3 ) − i + 20. (a) For what values of a. b. Let T (n) be deﬁned by T (2) = 4 and T (n) = T (an) + T (bn) + cn.) 9. The cases should be clearly and easily speciﬁed.) 10. c be numbers such that 0 ≤ a.
b be real numbers such that −100 ≤ a ≤ 100 and b > 0. Each case should be simple. b would reducing a by 1 reduce T (n)? For which values of a. For each of the following functions f ﬁnd a simple function g such that f (n) = Θ(g(n)). Consider the recurrence T (n) = aT ( n ) + n. (The function gb might involve several i=1 b cases and might depend on b. Let a. (The function fa might involve several i=0 i cases and might depend on a. (a) Find a function fa that does not involve summations such that n −a = Θ(f (n)).) 20. You are given the option of either reducing a or b b by 1. Let a. (The function f might involve several a a i=1 i cases and might depend on a. (c) For what values of a and b does fa (n) = O(gb (n))? (Only consider the cases where 0 ≤ a ≤ 1.) 16. and b ≥ 0. and b ≥ 0.) (d) For what values of a and b does gb (n) = O(fa (n))? (Only consider the cases where 0 ≤ a ≤ 1. Show that
= O(1).
∞ i5 i=0 i 43
14.(b) (c)
n i i=1 3 n i i=1 3
= Θ(3n ).) (b) Find a function gb that does not involve summations such that n i i=0 b = Θ(gb (n)). 19. √ 18. b be real numbers such that 0 ≤ a ≤ 1 and Let b ≥ 0. b be real numbers such a > 0 and b > 1. (a) Find a function fa that does not involve summations such that n2 7a = Θ(fa (n)). Let T (1) = 10 and T (n) = T ( n) + n.
15. b) such that T (n) = Θ(n). Let a. Let a. Each case should be simple. Consider the recurrence T (1) = 10. = Θ(3n+1 ). 25
. Specify an inﬁnite number of a b values for (a. b > 0. b would reducing b by 1 reduce T (n)? 17. Show that T (n) = O(n). You want to make T (n) as small as possible.) (b) Find a function gb that does not involve summations such that √ n 12i = Θ(gb (n)). T (n) = T ( n ) + T ( n ) + n. For which values of a. b be real numbers such that a.
) (d) For what values of a and b does fa (n) = Ω(gb (n))? (Only consider the cases a > 0 and −100 ≤ b ≤ 100. (The function gc.) (b) Find a function gb that does not involve summations such that n 5b i=0 i = Θ(gb (n)). NO PROOF REQUIRED. The cases should be clearly and easily speciﬁed.b may involve several cases depending on what a. Let a. (a) Find a function fa that does not involve summations such that n2 7i = Θ(f (n)).) 22. 7 Find a function gc. d ≥ 0. 26
.
Find a function ga. The function ga. Each case should be simple. list all correct ones. or Θ of B. one or more of these relations may hold for a given pair. b is. d be real numbers such that c. indicate whether A is O.d might involve several cases and might depend on c and d. Let c.99)n + n100 .) (c) For what values of a and b does fa (n) = O(gb (n))? (Only consider the cases a > 0 and −100 ≤ b ≤ 100. Each case should be simple. (The function gb might involve several cases and might depend on b.) 25. Each case should be simple.b should be as simple as possible.d ). Ω. Note that zero. 21. (c) f3 (n) = log(n20 ) + (log n)10 .d such that T (n) = Θ(gc. Consider the recurrence n T (n) = 49T ( ) + nc dn . (The function f might involve several a a i=1 a cases and might depend on a.(a) f1 (n) = (1000)2n + 4n . For each pair of expressions (A. (The function ga. Let a. √ (b) f2 (n) = n + n log n + n. o.b (n) such that T (n) = Θ(g(n)). b be real numbers such that a > 0 and −100 ≤ b ≤ 100. b be any real numbers Let T be the function deﬁned by
n
T (n) =
i=1
ia (log n)b . B) below. 23. (d) f4 (n) = (0.) 24. ω.
A B (a) n100 2n √ (b) (lg n)12 n √ (c) n ncos(πn/8) (d) 10n 100n lg n (e) n (lg n)n (f ) lg (n!) n lg n
27
.
or show that no such graph can exist. n2 ) does C(n1 . For each of the following combinations of properties either exhibit a connected graph on 10 vertices that exhibits these properties. Let Kn denote the complete graph with n vertices (i. Prove that in any (undirected) graph. then H ∈ A. Let C(n1 . n2 be natural numbers such that n1 ≥ 3 and n2 ≥ 3. Find a number a. and Cn2 . n2 ) is C(n1 . 0 < a < 1 such that the following statement is true “If G is a planar graph on n vertices then at least an vertices have degree ≤ 99. the number of vertices of odd degree is even. and connecting ALL the vertices of of Cn1 to ALL the vertices of Cn2 . n2 ) does C(n1 . 3. Prove rigorously that if a graph has three vertices of degree 1. 6. the cycle graph on n2 vertices. and let Kn denote the modiﬁcation of Kn in which one edge is removed. Let n1 . (a) When does Kn have an Euler path? (b) When does Kn have a Hamiltonian cycle? (c) What is the mininum number of colors needed for a proper coloring of Kn ? 2. n2 ) be the graph that is formed by taking Cn1 . Let A be a class of graphs with the following properties: (1) If G ∈ A and H is a subgraph of G. 7.Graph Problems 1.e. (2) If G ∈ A then G has a vertex of degree ≤ 10. then that graph does not have a Hamiltonian path. (a) For which values of (n1 . n2 ) have a Hamiltonian path? (c) For which values of (n1 . Show that every graph in A is 10colorable. n2 ) 5colorable but not 4colorable? 5.” 4. n2 ) have an Eulerian cycle? (b) For which values of (n1 . 28
. the cycle graph on n1 vertices. each pair of vertices contains one edge between them).
n − 1). n + 1} and edge set is the union of the sets {(1. Show that if G is a planar graph then at least half the vertices have degree ≤ 10. for any two cities x and y. . 3). (n − 1. Let Wn be the ‘wheel graph’ (pictured below). You want to minimize the number of roads you build. Find a number c such that the following is true:
29
.(a) Eulerian but not Hamiltonian. . (n − 2. (c) Hamiltonian and 2colorable. Show that every almostplanar graph is 5colorable (you may assume that every planar graph is 4colorable). . We only consider n ≥ 3. . n). For every n. 1)} and {(1. (b) Hamiltonian but not Eulerian. (n. (a) For which values of n is Wn 2colorable? (b) For which values of n is Wn 3colorable but not 2colorable? (c) For which values of n is Wn 4colorable but not 3colorable? (d) For which values of n is Wn 5colorable but not 4colorable? 14. 11. . 10. STATE this problem rigorously as a problem in graph theory. n. 2. . n + 1). . A graph is almostplanar if there exists a vertex v it such that if you remove that vertex then the graph is planar. . 2). 13. 9. . n + 1)}. Formally it has vertices {1. . there is a way to go from x to y that only passes through at most one other city. (2. (2. n + 1). . ﬁnd the maximal number of edges that a graph with n vertices can have and still be unconnected. 8. 12. Give an example of a planar graph with an Eulerian path that has 2 vertices of degree 5. (n. Consider the following real world problem: Given n cities you want to build a highway system such that. .
3 and K3.m have a cycle that includes every vertex exactly once. Deﬁne a notion of a kpartite graph. (n − 1.e. (c) For what values of n. (b) Find all values of n. m vertices (i.If G is a connected planar graph then at least at least have to have degree less than c. Phrase questions about kpartite graphs similar to those in the last questions.e. 1)}. V = {1.
3 4
of the vertices
You must prove that your answer is correct. For n. the left set has n vertices. (n. Let n ≥ 3. . . E) be the cycle graph (i. and for every left vertex a and right vertex b there is an edge between a and b). . . m does Kn.. (b) For what values of n. For any graphs G and H the graph G ⊕ H is formed by placing an edge between EVERY vertex of G and EVERY vertex of H. Show that if a graph on n vertices has ≥ n n edges then it must have a vertex of degree ≥ log n. m ∈ N let Kn.4 . (You may assume that n is large. n). 2). m does Kn. . 16. (c) Find all values of n.
30
. m such that Cn ⊕ Cm 6colorable but not 5colorable.) 18.. m such that Cn ⊕ Cm 4colorable but not 3colorable. You may assume that the number of vertices in G is divisible by 4. . n} and E = {(1. Let G be a bipartite graph with m nodes on the left side and n nodes on the right side.) (a) Find all values of n. What is the maximum number of edges G can have? What is the maximum number of edges G can have and still be disconnected? 19. Let Cn = (V.m be the complete bipartite graph on n. . 15. m such that Cn ⊕ Cm 5colorable but not 4colorable. .m have a path that includes every vertex exactly once. the right set has m vertices. (a) Draw K1. √ 17.
give a Θ(n2 )time algorithm to compute the adjacency list representation of G. (b) Find all possible schedules that assign teachers to classes so that all four classes are taught and every teacher teaches just one class. Let G = (V. (a) Model this scenario with a graph. Prove that in a graph G = (V.)
31
. (Hint: First show that v∈V deg(v) = 2m where m is the number of edges in G. 22. and the edge is directed towards the winner. If we take a complete undirected graph (a graph that has n vertices....20. Bill can teach 150 or 650.) 24. Let T be a weighted tree. 251. Give an eﬃcient algorithm to ﬁnd the f minimal spanning tree of a weighted tree.) A directed Hamilton path in G..n]. Edna can teach 150 or 251. An f minimal spanning tree is a spanning tree where the weights are w1 . Prove that every tournament has a Hamilton path. Diane can teach 451 or 650. Clyde can teach 150 or 451. (a) Assuming that G is represented by an adjacency matrix A[1. and 650 must be taught.n. we obtain a graph called a tournament. (Represent the addition of an element v to a list l using pseudocode by l ← l ∪ {v}.. E). . in which every two players play against each other. (c) Is there a reasonable assumption you can make to get the run time of part b down to O(E)? 23. is a directed path that includes every vertex of G exactly once. give a Θ(n2 )time algorithm to compute the adjacency matrix of G. The courses 150.n]. Let f be some function that is easily computed (it goes from Naturals to Naturals)..) (b) Assuming that G is represented by an adjacency list Adj[1. (Hint: Try Induction. (Think of n players playing a game. 21. E) be a directed graph. 451. 1. wm and m f (wi ) is i=1 minimal among all spanning trees. the number of vertices that have odd degree are even. and edges between all pairs of vertices) and direct all the edges in some arbitrary manner.
so that every customer gets to see the two movies they desire. Mk .25. and which movies should be televised on sunday. . (No edges between nodes in the same set are allowed. The question is: is there a schedule so that each movie is shown at most once? Design an eﬃcient algorithm to ﬁnd such a schedule if one exists? 32
. try your alg. There is also a set of customers. Give a rigorous.) Give an O(E + V ) algorithm that takes an input graph (represented in adjacency list form) and decides if the graph is bipartite. . undirected graph can be directed to produce a strongly connected directed graph. Movies are shown on saturday evening and sunday evening. with each one indicating the two movies they would like to see this weekend. Multiple movies may be screened at the same time. Hints: Topological sort. the algorithm should also produce the bipartition. An Euler Circuit for an undirected graph is a path which starts and ends at the same vertex and uses each edge exactly once.) 29. . M2 . If so. such that each edge in the graph goes between a vertex in A and a vertex in B. (Hint: Show that this can be done if and only if removing an edge leaves a connected graph. A bipartite graph is one whose vertex set can be partitioned into two sets A and B. (a) Prove that a connected. undirected graph has an Euler circuit if and only if every vertex has even degree. Using depth ﬁrst search. outline an algorithm that determines whether the edges of a connected. 28. If the graph is bipartite. 26. (Assume that customer i speciﬁes a subset Si of the two movies that he/she would like to see. on examples. (Hint: use BFS.) 27. (a) Give an O(V  + E) algorithm to ﬁnd an Euler circuit if one exists. Describe concisely and accurately a lineartime algorithm for solving the singlesource shortest paths problem in a weighted DAG. . Consider the following problem: There is a set of movies M1 . concise proof that your algorithm is correct. relaxation lemma.) You need to decide which movies should be televised on saturday. the algorithm outputs such an orientation of the edges.
v∈V
max δ(u. Hint: Use BFS or DFS. Give a simple example where this bound is met. (The length of a path is the number of edges on it.) 31. We would need k channels if k is the number of movies in this solution – hence if there is a solution in which each movie is shown at most once. 32. with the constraint that for each task t there is a set of tasks P [t] which must be completed before task t is begun. Show that the vertices of any nvertex binary tree can be partitioned into two disconnected sets A and B by removing a single edge such that A and B are at most 3n/4. Let G = (V. v) is the number of edges on the path from u to v). Give an O(n + e)time algorithm to compute a path in G that traverses each edge in E exactly once in each direction. 33. and show the correctness and analyze the running time of your algorithm.(One naive solution would be to show all movies on both days. we would like to ﬁnd it. Tasks may be performed simultaneously. Hint: work bottomup. acyclic graph with a vertex – the root – that has a path to all other vertices). v)
(where δ(u. item Suppose you are given a set of tasks T . The diameter of a tree T = (V. computes the minimum amount of time needed to complete all the tasks. Consider a rooted DAG (directed.
33
. where task t takes time D[t] from start to ﬁnish. DFSbased algorithm which. E) is given by
u. Let R = t∈T P [t]. or returns ∞ if it is not possible to meet the constraints. Describe (give pseudocode for) an O(T  + R)time. E) be an undirected graph. Describe an eﬃcient algorithm to compute the diameter of a tree. given D and P . Give a linear time (O(V + E)) algorithm to ﬁnd the length of the longest simple path from the root to each of the other vertices. so that each customer can see one desired movie on each day.) 30.
A pseudoforest in is a collection of vertexdisjoint pseudotrees. there exists a MST not containing (u.. E) be a directed graph.. 36.
35. give an O(n + e)time algorithm to compute an adjacency list representation of GR . there exists a MST containing (u. Describe and analyze a fast algorithm for ﬁnding a pseudoforest of maximum total edge weight in an undirected graph with nonnegative edge weights. v) is the lightest edge on a cut.34. Let G = (V. (a) Assuming that G is represented by an adjacency matrix A[1. FACT: There are algorithms for this problem that run a least as fast as the fastest known spanning tree algorithms. Give a simple example where this bound is met.
34
.n]. The reversal of G is a graph GR = (V. (b) Assuming that G is represented by an adjacency list Adj[1. v). 37. Show that the vertices of any nvertex binary tree can be partitioned into two disconnected sets A and B by removing a single edge such that A and B are at most 3n/4. E R ) where the directions of the edges have been reversed (i.e. 1.n. j)(j. v) is the heaviest edge on a cycle. A pseudotree in an undirected graph is a connected subgraph that is either a tree or a tree plus a single edge (so that it contains exactly one simple cycle).
a Show that if (u. give an O(n2 )time algorithm to compute the adjacency matrix AR for GR . E R = {(i.. v). i) ∈ E}).n]. b Show that if (u.
. what is
k∈Ij ∈S
(element associated to Ij ). and FINDMAX in O(1) steps. Let S = {p1 . (b) b) insert a point. Devise a Data Structure for S such that the following operations can be carried out quickly (a) a) which pair of points is closest together. . . y)  x0 ≤ x ≤ x1 . 2. ﬁnd the pair with the shortest distance between them. Let S be a set of n orthogonal lines in the plane (lines parallel to either the xaxis or yaxis). . . Assume they all have xcoordinate in the set {1. Devise a Data Structure for the following such that the following query will be easy: Given an orthogonal line (not in S). . Let S be a set of N intervals I1 . ﬁnd {(x. . . . Devise a structure that allows inserts and deletes in O(log n) steps and both FINDMAX and FINDMIN in O(1) steps. N } and y0 a natural number. . . . . Devise a data structure such that the following operation are O(log n).) 4. IN which have endpoints in {1. . Devise an (n log n) algorithm for the following problem: Given n points in the plane. x1 ∈ {1. 2. . . pn } be n points in the plane. Let S = {p1 . . with both coordinates in the set {1. . how many or p1 . . . 2. 2.
(b) Update by changing the value associated to an interval. Assume every interval in S has a natural number associated to it. . Let p1 . pn are in the rectangle. . The data 35
. .Data Structures 1. which of the n lines in S does it intersect?” 5. . (c) c) delete a point. . . (Hint: Sort the points on the xcoordinate and then store in some cleverly chosen data structure. Devise a Data Structure to answer the following type of query: Given x0 . N }. . pn be n points in the plane. . N }. . Recall that a heap allows inserts and deletes in O(log n) steps. . pn } be n points in the plane. n}. (a) Given k. y ≤ y0 } ∩ S. . 3. 2. Devise an O(n2 ) data structure for them such that the following type of query can be answered in O(1) steps: given a rectangle. . 6. . . 7. .
Picture an environment where updates and queries are coming in FAST. FINDMAX in O(1). . (c) DEUNION: Undoes the last union (d) BACKTRACK: Undoes all the unions going back to and including the largest one formed. 10. (b) FREQ:Σ+ →N. (c) the right subtree is a Minheap. 9. and a w in w4 starting in the 5th position. (a) FIND:Σ+ → Σ+ . INSERT. (My notes about the reference claim that INSERT can be done in O(log log n) steps — but I am not sure I believe that. 36
. Show how to implement FINDMIN. How well does it do. Devise a data structure that will accommodate the following operations in the time bounds listed (a) UNION(X. if the output is {(1. Devise a structure for a set on integers where the queries are just to see if some integer is in the structure. This sets X to X ∪ Y . (4. so you put the update itself in a buﬀer. Y ). a w in w1 starting in the 7th position. . (c) LOC:Σ+ → 2N×N . wk } where the wi are (very long) strings. 3). Let S = {w1 . (d) every leaf in the Maxheap is bigger that its corresponding leaf in the Minheap.g. FIND(w) returns the longest preﬁx of w that is a subword of some wi . and CREATE in O(n). 7). (b) the left subtree is a Maxheap. . e. FREQ(w) returns how often w occurs as a subword of any wi . (b) FIND(x). Devise a data structure that will support the following operations. and destroys Y . 5)} then there is a w in w1 starting in the 3rd position. LOC(w) returns the places w occurs. Let A be a set of n disjoint sets. (1. 11. A Deap is a data structure that is a tree such that (a) there is nothing at the root. . You might not have ANY time to do an update to the structure. and each query should take O(k + log log n). This returns where x is (recall that the sets are all disjoint and these operations will keep them that way). 8.structure should take up no more than O(N +n) space. and DELETEMIN and DELETEMAX in O(log n).
13. Look into variants. the more likely to move it up. (a) Merge two components. (b) Given edge. (c) Given a nonedge. Write pseudocode to implement this heuristic. then output the path.
37
. (a) Given two vertices x and y. (c) retrieve minimum edge in a component. if it hits one of them. 15. given an inﬁnite vertical line. Devise a Data Structure for a directed graph that can answer the following three types of queries.12. (b) locate which component a vertex is in. Devise a Data Structure for a weighted Directed Graph on n nodes that can do the following operations quickly. add it as an edge. Consider the movetofront heuristic where you decide whether or not to move the item up probabilistically: the more often the item has been accessed. delete it. Devise a data structure to store n horizontal lines so that you can tell. determine if there is a path between them. If so. 14. This should take time O(1) if there is no path. and O(LENGTH OF PATH) if there is a path. The data structure should take O(n2 ) space and be able to answer the query in O(1).
where U has n rows and columns and x and c are vectors of size n. Suppose you wish to solve the equation Ux = c for the unknown vector x. and d such the following statement holds when a. How many multiplications are required? Justify your answer.” (Try to make the conditions as general as possible. and d satisfy that condition: “If one could multiply 2 a × a matrices using b multiplications and c additions then one could multiply two n × n matrices in a better way than that shown in class. BEGIN Input(n) For i := 1 to n do [TAKES 5i STEPS] i := n While i ≥ begin SOMETHING THAT TAKES i STEPS i := i − 1 END 2. How fast can you multiply two n × n matrices (you may assume n is a power of 3). U . b. c.
n 2
do
38
. Is this better than the algorithm in class for n × n matrices? Give conditions on a.Analysis of Algorithms 1. For the following program ﬁnd a function f (n) such the runtime is Θ(f (n)). b. c. Assume you could multiply two 3 × 3 matrices in 18 multiplications and 54 additions. The result of applying Gaussian elimination to a square matrix is an upper triangular matrix.) 3.
Assume we could multiply two 3 digit numbers with k multiplications. How small does k have to be for this algorithm to be asymptotically faster than the O(nlg 3 ) algorithm? 2. do not worry about the constants) is a new “fancy” multiplication algorithm based on this fact? b. Devise an O(n log n) algorithm for the following problem: Given n points in the plane.
39
.e. Assume that additions and multiplications of numbers both take exactly one time unit.Divide and Conquer 1. 3. How small does k have to be for this algorithm to be asymptotically faster than the “standard” (O(n2 )) algorithm? c. How fast (just the order. (a) Exactly how many multiplications does the standard matrix multiplication algorithm use to multiply two n × n matrices? Exactly how many additions does it use? What is the exact time it uses to multiply two n × n matrices? (b) What is the exact time to multiply two n × n matrices using Strassen’s algorithm? (c) For what values of n is Strassen’s algorithm better than the standard algorithm? Justify your answer. a. i. ﬁnd the pair that is closed together.
so that we multiply A1 to Ai−1 in the best possible way. pn × qn . Let E(k. given keys k1 . . 3. (c) Find a recurrence for E(k. Look for the smallest dimension. pn . . to ﬁnd f . If an egg breaks then it cannot be reused. (a) Show that E(1. Let k. Show that this greedy algorithm does not ﬁnd the optimal way to multiply a chain of matrices.. . n). . and β to go right. Your goal is. but if it is dropped oﬀ the (f − 1)st ﬂoor it will not. Let α and β be constants. n) be the minimal number of eggdroppings that will always suﬃce.. kn which have probabilities of being searched p1 . In order the determine how to multiply A1 to Ai−1 and Ai to An we apply the algorithm recursively. Assume that in a tree it costs α to go left.) If an egg breaks when dropped from some ﬂoor then it also breaks if dropped from a higher ﬂoor. If an egg does not break when dropped from some ﬂoor then it also does not break if dropped from a lower ﬂoor. You are told that there is a ﬂoor f such that if an egg is dropped oﬀ the f th ﬂoor then it will break. Write a dynamic program to ﬁnd E(k. given k eggs. The (k. You would like to drop eggs as few times as possible. . and ﬁnally multiply the two resulting matrices. n) = n. n ∈ N .. n) = Θ(n k ). and parenthesize between matrices Ai−1 and Ai . How fast does it run? 2. Devise an algorithm that will.. . . n)egg problem is as follows: There is a building with n ﬂoors. Ai to An in the best possible way. An . . . have dimensions p1 × q1 . . How fast does it run? (d) Write a dynamic program that can be used to actually yield the optimal strategy for ﬁnding the ﬂoor f . Consider the following greedy algorithm for solving the chained matrix multiplication problem: Let the matrices A1 .Dynamic Programming 1. 40
1
. n). (It is possible that f = 1 so the egg always breaks. or f = n + 1 in which case the egg never breaks. say qi−1 and pi ... (b) Show that E(k. builds a tree with optimal worst case cost. The only operation you can perform is to drop an egg oﬀ some ﬂoor and see what happens.
)
h[6] w[6] H
W
(a) Write down a recurrence for book shelver’s problem. The height and width of the 6th book are shown to the right. d × 1. and W . given h[1. (b) Use your recurrence to develop an eﬃcient dynamic programming for solving the book shelver’s problem. (Notice that there is some unnecessarily wasted height on the third shelf. (c) Analyze the running time of your algorithm. and you may put as many books on each shelf as you like (up to the width of the book case). 1 × d. 41
. 5.. placed wherever you like (up to the height of the book case). and that books are stacked in the usual upright manner (i.e. what is the minimum height book case that can shelve all of these books. w[1. All book cases have width W . The books are placed on shelves in the book case. You may use as many shelves as you like. Write an eﬃcient algorithm to determine an order of evaluating the matrix product M1 × M2 × M3 ... × Mn so as to minimize the scalar multiplications in the case where each M is of dimension 1 × 1.n]. which must be placed in a library book case. but you may have any height you like.n]. Below shows an example of a shelving. Assume that these arrays are given by the books’ catalogue numbers (and the books MUST be shelved in this order). . Let h[1.. The bookshelver’s problem is. where h[i] denotes the height of the ith book and w[i] denotes the width of the ith book. . We are given a collection of n books.4. you CANNOT lay a book on its side)..n] and w[1. d × d for some ﬁxed d. Assume for simplicity that shelves take up no vertical height.n] be arrays.
[What happens if all entries are negative?] 7. What is its time? 9. 10. 14. consider the problem of ﬁnding the longest monotonically increasing subsequence of not necessarily contiguous entries in your array. but also determining the two endpoints. . and we wish to ﬁnd the maximum contiguous sum. 3. (d) Modify your algorithm to actually ﬁnd the longest increasing subsequence (not just its length). (a) Write down a recurrence for L[i]. 8. Assume that there are n numbers (some possibly negative) in a circle. . 5. 11. (b) Based on your recurrence. . Give an eﬃcient algorithm for solving this problem. Assume team A has probability p of winning each point and team B has probability q = 1−p of winning each point. What can you say about its running time? (c) Based on your recurrence. 13. 11. Consider the problem of not only ﬁnding the value of the maximum contiguous subsequence in an array. What is its time? 8. and we wish to ﬁnd the maximum contiguous sum on the circle. give a Θ(n2 ) dynamic programming algorithm for ﬁnding the length of the longest monotonically increasing subsequence. Given an array a[1. 9. HINT: Let L[i] be the length of the longest monotonically increasing sequence that ends at and uses element a[i]. 5.6. give a recursive program for ﬁnding the length of the longest monotonically increasing subsequence. then a longest monotonically increasing subsequence is 3. j) be the probability that team A will win the game given that it needs i points to win and team B needs j points. (e) Improve your algorithm so that it ﬁnds the length of the longest monotonically increasing subsequence in Θ(n lg n) time. Give an eﬃcient algorithm for solving this problem. . 8. For example. The goal of this problem is to ﬁnd an eﬃcient dynamic programming algorithm for solving this problem. Consider the problem of computing the probability of which team will win a game consisting of many points. Let P (i. if the entries are 10. 42
. Assume that there are n numbers (some possibly negative) in an array . Give a linear time algorithm for solving this problem. 14. n] of integers.
.. losses as 0. and bl of winning. (b) Use your recurrence to develop an eﬃcient dynamic programming algorithm for determining the probability that team A will win the game given that it needs i points and team B needs j points.i2 .i. Assume that there are g games left to play in the match and that the champion needs to win i games (which may end in a 1/2)..j2 ] is deﬁned recursively. wd . and losing playing white. and wl of winning. Not only are there wins and losses. The current champion retains the title in case the match is a tie. and losing playing black. We limit the decompositions to a special type called guillotine decompositions. (j + 1). An example of a guillotine decomposition into submatrices of one value is shown below. Wins count as 1. (c) Analyze the running time of your algorithm. We want to decompose this matrix into rectangular submatrices such that the entries in any submatrix are either all 0 or all 1. (a) Write down a recurrence for the probability that the champion retains the title.j] and A[i1 . bd . 12. then no further decomposition is possible.. The traditional world chess championship is a match of 24 games. j1 .j2 ] and A[(i + 1). The players take turns playing white and black.j2 ]. (c) Analyze its running time assuming that the match has n games.. (b) Based on your recurrence. White has an advantage.i2 . as follows. Once the two submatrices are formed.. j1 .(a) Write down a recurrence for P (i. drawing. j1 . A guillotine decomposition of a subarray A[i1 . give a dynamic programming to calculate the champion’s probability of retaining the title. 11.j2 ]. j1 . but some games end in a draw (tie). If j1 < j2 then select any j. and has probabilities bw . drawing. j1 ≤ j < j2 and split the array vertically just after column j. If both i1 < i2 and j1 < j2 then you are free to make either type of cut. i1 ≤ i < i2 and split the array horizontally just after row i. 43
.i2 . If i1 < i2 then select any i. has probabilities ww . creating two new subarrays A[i1 . they may be further decomposed by a guillotine decomposition. You are given an n × n matrix A whose entries are either 0 or 1. j).. creating two new subarrays A[i1 .i2 .. If i1 = i2 and j1 = j2 ... and draws as 1/2. because he moves ﬁrst. Assume the champion is white in the ﬁrst game.
For the last line. We want to print this paragraph neatly on a number of lines that hold a maximum of M characters each. Except for the last line. and word j ends in the rightmost position of the line. and derive its running time. ln . l2 . starting at the leftmost position of the line. It suﬃces to determine the minimum cost.) 13. Give a dynamic programming algorithm to determine the neatest way to print the words. j−i and the total cost of the line is the cube of this value multiplied times (j − i). so that there are at least two words on each line. not the actual printing method. You may assume that every word has length less than M/2. . . Justify the correctness of your algorithm. . Consider the problem of neatly printing a paragraph on a printer. we simply put one space between each word. and space is inserted between the words as evenly as possible.
44
.0 0 1 1 0
1 1 1 1 0
0 0 0 1 1
0 0 0 0 0
1 1 1 0 0
0 0 1 1 0
1 1 1 1 0
0 0 0 1 1
0 0 0 0 0
1 1 1 0 0
Write a dynamic programming algorithm to determine the MINIMUM number of guillotine cuts needed to decompose the matrix A into submatrices of one value. (Note: You do not have to output the actual cuts themselves. measured in characters. . Analyze the running time and space of your algorithm. Our criterion for “neatness” is as follows. We wish to minimize the sum of costs of all lines (except the last). The amount of extra space between any two words on this line is M − j lk k=i − 1. word i starts in the leftmost position of the line. The input text is a sequence of n words of lengths l1 . There MUST be at least one space between any two words on the same line. just the number of cuts. if a given line contains words i through j.
for example. Unfortunately you have a knapsack that only holds a total weight M . . xn } of n points on the real line and wish to cover them with unit length closed intervals. . Sort the edges of the graph in DECREASING order. . a cut treasure retains its fractional value (so. Assume that you are given a set {x1 . and then run the algorithm as before. (a) Describe a Θ(n lg n) time greedy algorithm to solve this problem. Give a GREEDY ALGORITHM that tries to ﬁnd a coloring of a graph that uses the least number of colors.Greedy Algorithms 1. perform an ExtractMax. (c) Improve the running time of your algorithm to Θ(n). True or False: This modiﬁed algorithms produces a Maximum Cost Spanning Tree. . Modify Dijkstra’s algorithm as follows. vn . Give a class of graphs for which your algorithm succeeds in ﬁnding the least number of colors needed. (1) Initialize all d[v] values to −∞. and (3) in relaxation. Inside there are n treasures with weights w1 . Write a greedy algorithm for the Traveling Salesperson Problem Find an inﬁnite set of graphs for which it ﬁnds the optimal solution (no proof required). 3. reverse all comparisons so that the shortest 45
. . (a) Describe an eﬃcient greedy algorithm that covers the points with as few intervals as possible. . . Modify Kruskal’s algorithm as follows. Give a class of graphs for which your algorithm always uses more colors then it has to. and if False then give a counterexample. a third of treasure i has weight wi /3 and value vi /3).) 2. given a weighted graph G. 5. 6. (b) Prove that your algorithm works correctly. 4. Fortunately there is a knife handy so that you can cut treasures if necessary. (2) rather than performing an ExtractMin. . . . . While walking on the beach one day you ﬁnd a treasure trove. ﬁnd the minimum cost hamiltonian cycle. . If True then prove your answer. (b) Prove that your algorithm works correctly. wn and values v1 . (The Traveling Salesperson problems is.
Suppose that your boss gives you a connected undirected graph G. The cost of an edge e is deﬁned to be 1/ce where ce is the cost of e in G. then the capacity of the path is deﬁned to be the minimum bandwidth of all the edges that belong to the path P . for each vertex v. (For extra credit: give a proof for your scheme. which is the same as G. Suppose we are given a graph G (connected. capacity(s. and analyze its running time. We deﬁne capacity(s. v) = maxP (s. 8. and if False then give a counterexample. and WHICH edges might possibly be inserted into the MST.) 9. or disprove. indicate whether 1) the MST cannot change 2) the MST can possibly change. v) is equal to the maximum capacity path from s to v. the Maximum cost spanning tree in G ? Prove or disprove.v) capacity(P ). State WHICH edges might possibly be deleted from the MST. For each of the following situations. where s is some ﬁxed source vertex.path estimate d[v] increases rather than decreases. Show that your algorithm is “correct”. (Design something that is no more than O(n2 ). True or False: This modiﬁed algorithm determines the cost of the longest simple path from the source to each vertex. Is P the longest simple path from s to v in G ? Prove. and you compute a minimum cost spanning tree T .) 46
. A good algorithm should recompute the MST by considering only those edges that might change. from s to v. Now he tells you that he wants to change the cost of an edge. except for the costs. v). (Essentially. 7.) Give an eﬃcient algorithm to compute capacity(s. If case 2) occurs then describe what changes might occur in the MST. Now we construct a new graph G . Assume that we have a network (a connected undirected graph) in which each edge ei has an associated bandwidth bi . If True then prove your answer. and with the right data structures takes O(m lg n) time. (a) Is the Minimum cost spanning tree in G. (b) Suppose that P is the shortest path from s to v in G. and HOW you would select the edges to be inserted/deleted. If we have a path P . Your job is to recompute the new minimum spanning tree (say T ). undirected) with costs on the edges (all costs > 0).
47
. the algorithm might get lucky and color it with as few colors as possible. (b) He increases the cost of an edge e that belongs to T . You do not need to design an algorithm.(a) He decreases the cost of an edge e that belongs to T . Then to consider every other node one at a time and color it “1” if it does not neighbor a node already colored “1”. If True then prove your answer. (a) How would you implement this algorithm eﬃciently? Be brief but clear. we desire to ﬁnd the minimum number of colors needed to color an undirected graph. (b) Prove that for every graph. 12. then it is not in any minimumweight spanning tree. perform an ExtractMax. 10. Continue in this fashion until all of the nodes in the graph are colored.) 11. (Hint: Recall the proof of Kruskal’s algorithm. edges E. then the minimum cost spanning tree is unique. but to simply state the edges that need to be considered. In the graph coloring problem. A coloring of an undirected graph is assignment of colors to the nodes so that no two neighboring nodes have the same color. and colors C. One possible greedy algorithm for this problem is to start with an arbitrary node and color it “1”. 13. reverse all comparisons so that the shortest path estimate d[v] increases rather than decreases. (c) He decreases the cost of an edge f that does not belong to T . What is your running time as a function of the number of vertices V . Prove the following lemma: If an edge of an undirected weighted graph is the unique most expensive edge on a cycle. (1) Initialize all d[v] values to −∞. and (3) in relaxation. (2) rather than performing an ExtractMin. Now do the same for all of the remaining nodes using the color “2”. Modify Dijkstra’s algorithm as follows. True or False: This modiﬁed algorithm determines the cost of the longest simple path from the source to each vertex. (d) He increases the cost of an edge f that does not belong to T . and if False then give a counterexample. Prove that if a connected undirected graph has distinct edge weights.
48
.(c) Prove that the algorithm can be arbitrarily bad. for every constant α > 0 the ratio between the number of colors used by the greedy algorithm and by the optimal algorithm could be at least α. In other words.
(a) Write an informal program that could implement this algorithm.Sorting and Selection 1..n] where the ﬁrst n − nα are sorted (though we know nothing about the remaining elements). and then merge the result with the third subset in sorted order. Assume that you have a subroutine that ﬁnds the minimum of n numbers in na steps. Assume that you have specialpurpose hardware that can take two sorted lists of equal size n and merge them in (n) steps. Given a set of n numbers. Write down a recurrence to √ describe the run time. divide it into three subsets of equal size. 3. Let A[1. O(n) steps). You may also assume that n is a power of three.n] be an array such that the ﬁrst n − n elements are already sorted (though we know nothing about the remaining elements. For what values of α.. merges them into one sorted list. Using this subroutine. 4. You may assume that you have available a procedure merge that merges two sorted lists of size n1 and n2 in time n1 + n2 .. (For your own enlightenment replace m with other functions and see what happens. Write down a recursive algorithm that uses this hardware to sort lists of length n. 6. A can be sorted in linear time (i. 49
. write an algorithm that sorts a list of n numbers. 0 ≤ α ≤ 1 is the following statement TRUE: Given an array A[1. For what values of a is the algorithm better than θ(n log n) (you may assume n is large and reason asymptotically). How fast does your algorithm run (you may use θnotation). √ and takes only m steps. Write an algorithm that will sort A in substantially better than n log n steps. sort each of the three subsets.) 2. Assume you have some kind of superhardware that. Consider the following variation of mergesort.e. √ 5. Devise a sorting algorithm using this hardware that performs substantially better than O(n log n). Let a be a number such that 0 ≤ a ≤ 1. (b) Analyze the cost of this algorithm. merge the ﬁrst two subsets in sorted order. when given two lists of length m that are sorted.
the running time of this program. Find a number c such that for all n ≥ 1. 10 cents. 9. Justify your answers. In American we have coins for 1 cent. T (n) ≤ cn.) For each of the following desired properties state whether you are better oﬀ using a heap or an ordered array. How fast can you multiply two numbers of n bits long. Assume you had a way of expressing the product of two numbers of n bits long by using a products of numbers that are n long. and ONE extra point. INSERT in O(log n) steps. and DELETE in O(n).7. (if n = 1 this takes 0 operations. For what values of a. Let P be a program that has the following format. 10.
n 3 n 4
Execute n operations
50
. along with b addition of c numbers. INSERT in O(log n).) Execute n operations Call P on input Call P on input end Write a recurrence for T (n).) 11. the greedy algorithm is optimal. c is this better than the algorithm in class. and 25 cents. (For your own enlightenment. and DELETE also in O(log n). the includes a 1 cent coin. b. etc. PROCEDURE P begin Input(n) (this takes 0 steps) if n = 1 then execute 4 operations and stop. Show that if you want x cents and the least amount of coins. come up with a structure that allows MAX and MIN in O(1) steps. (Heap is MAXheap. Exhibit of set of coin denominations for which greedy is not optimal. 8. You wish to store a set of n numbers in either a heap or a sorted array. 5 cents. or that it does not matter. has MAX element on top. Devise a structure that allows MAX and MIN in O(1) steps.
15.) 16. and swaps. (b) Want to be able to delete an element quickly. For what values of a. What is the worst case performance? What is the best case performance? 14.. Assume that you have hardware that can sort log n elements in ONE step. (d) Want to ﬁnd the MIN quickly. (b) The array is such that you have a good idea of where the median is.you can’t use even a little bit of extra space. (1) do an add or delete a times. c are much smaller than n. What is a LOWER BOUND on how well we can do? Explain your result. (d) Space is relevant.) 13. Rewrite 3 the pseudocode for QUICKSORT on page 154 so that it can use this MAGIC algorithm. Hence a variation of the decision tree model is appropriate. Assume that we are going to sort n numbers from the set {1. (c) Space is a crucial parameter. . (a) It is absolutely crucial that the array gets sorted quickly. the operation above.n] but that we have a MAGIC algorithm that sorts arrays of size ≤ n in n steps. Assume that we use RADIX sort. For each of the following scenarios indicate if you would use QUICKSORT or HEAPSORT and why. b. Assume that we are going to be sorting arrays A[1. Let f (n) be a function of n. (2) try to ﬁnd MAX b times. . (You must also take into consideration the time spent forming these structures. We can assume a. (3) try to ﬁnd MIN c times. c are we better oﬀ using a heap? For what values are we better oﬀ using a sorted array. 51
. in the course of time. but using a little extra space is okay. Find an inﬁnite number of functions f such that RADIX SORT performs substantially better than O(n log n). . We are thinking of sorting a list of n elements using this hardware. (You can assume that the only operations allowed are comparisons. . f (n)}.(a) Want to ﬁnd the MAX quickly. b. (c) Want to be able to form the structure quickly. Given n elements we are going to. 12.
20. (c) The elements to be sorted are numbers from the set {1. . The algorithm for bucket sort is geared to take inputs in the range [0. and bucket sort perform in time substantially better than O(n log n). Let p be a positive natural number.
52
. (2) Radix sort. (d) No bound is known on the elements to be sorted. . Assume that you can ﬁnd the median of a elements in b comparisons. Adjust the algorithm so it takes inputs that are natural numbers in {1. pn} and uses p buckets. 19. How might you adjust i the algorithm so that it performs well for this distribution? 18. n} and we have access to the actual bits of the number. . (a) The elements to be sorted are numbers from the set {1. Assume that you can ﬁnd the median of 7 elements in 12 comparisons. . Find values of a. but we need to guarantee fast performance. Discuss informally how this algorithm might behave if the inputs are chosen with the distribution P rob(x = i) = K (Where K is the appropriate constant). Try to make the bound as tight as possible. Brieﬂy explain why you chose the one you did. Use this in the lineartime median ﬁnding algorithm. Let a. Give an upper bound on the number of comparisons used (do NOT use Onotation). (b) We have access to a VERY FAST median ﬁnding algorithm that we can use as a subroutine. . 21. or COUNTING SORT. QUICKSORT (or variants). . RADIX SORT. For each of the following scenarios say whether you would use MERGESORT. . . Explain carefully why they are not. b such that you can ﬁnd medians in ≤ 4n comparisons. Try to make the bound as tight as possible. . . 1) (fractions allowed of course). .17. Give an upper bound on the number of comparisons used (do NOT use Onotation). Consider the following two statements: (1) Sorting n elements REQUIRES Ω(n log n) operations. Devise a lineartime MEDIAN FINDING algorithm that uses this. These statements seem to be in contradiction. . log n} but the only operations we are allowed to use are COMPARE and SWAP. b be numbers. counting sort.
(c) DELETE MAX. places the largest m into B[( m + 1). d(n) children). MAGIC does what PARTITION does in the usual QUICKSORT algorithm). Let d(n) be a function of n such that d(n) << n. determine whether it is in the structure). Assume you had some kind of superhardware that. QUICKSORT refers to the randomized version.m] (not 2 2 necessarily in order). Assume that this hardware works in md steps.22. Describe a sorting algorithm similar to QUICKSORT that uses MAGIC to sort A[1. which may have 1 or 2 children). Also. Write down a recursive algorithm that uses this hardware to sort lists of length n Write down a recurrence to describe the run time. (3) in Θ(n log n)? 26. given an array B[1... Assume that c ≥ 0. and takes only mc steps. merges them into one sorted list. (a) Give a scenario where RADIXSORT is better than MERGESORT.. b. and returns the index of the largest element in the ﬁrst partition (i. 53
. (d) FIND (given a key. and c no explanations are needed. which may have 1. Assume that n numbers are in a MAXheap. (b) INSERT.n].. How fast would the following operations be? You may use Θnotation. (b) Give a scenario where MERGESORT is better than QUICKSORT. .. when given two lists of length m that are sorted. (a) FIND MAX. We have special hardware labeled MAGIC that. Imagine that we redesigned MAXheaps so that instead of every internal node having 2 children (except possibly the rightmost internal node on the secondtolast level. Let d ≥ 0. In answering a.e. For what d. . (b) Show that if i is any constant then the ith largest element can be found in O(1) steps 24. 25. places the smallest m elements into B[1. (a) Show that the third largest element can be found in O(1) steps. they had d(n) children (except possibly the rightmost internal node on the secondtolast level.m]. will this sorting algorithm perform (1) much better than Θ(n log n). (2) much worse than Θ(n log n). . m ] (not 2 2 necessarily in order). For what values of c does this algorithm perform substantially better than O(n log n)? 23. .
’ (b) Find an inﬁnite number of such functions with diﬀerent orders of magnitude. √ and takes only m steps. ﬁnds the n rd largest element in n comparisons. not serious grading. merges them into one sorted list.) 30. Write down a recursive algorithm that uses this hardware to sort lists of length n. What is the lowest YOU can make c? (This last part is for speculation. Write down a recurrence to √ describe the run time. 1) using BUCKETSORT. similar to QUICKSORT. (We use the version in the book where there are n buckets and each individual one is sorted by INSERTION SORT. Assume that we have a MAGIC ALGORITHM that can. then BUCKETSORT is guaranteed to do substantially better than O(n log n).) 31. 27. We are going to be sorting numbers in the interval [0. 3 Use this to devise an algorithm that takes cn log n comparisons. We are going to be sorting numbers between 1 and na (log n)b . (For your own enlightenment replace m with other functions and see what happens.) (a) Find an increasing function f such that the following holds: ‘If when doing BUCKETSORT we are guaranteed that no bucket has more than f (n) elements in it. b ≥ 0. when given two lists of length m that are sorted. Assume that we have MAGIC HARDWARE that can ﬁnd the MAX of three elements in 1 step. b such that COUNTING SORT will perform substantially better than O(n log n). 32.
54
. Let a. 28. 29. Find all values of a. Assume you have some kind of superhardware that.(c) Give a scenario where QUICKSORT is better than RADIXSORT. (a) Give an algorithm for sorting (n elements) that uses this hardware. What is its running time? (b) Give a lower bound on how well any algorithm with this hardware can do. given an √ array of n elements. Assume that we have MAGIC HARDWARE that can sort k elements in 1 step.
(The function fa. (Note— there is no one ‘right answer’ to this problem.b such that T (n) = θ(fa. (b) How many comparisons does your algorithm take in the worst case? How many comparisons does your algorithm take in the 55
. b be real numbers such that a. . In some applications the lists one wants to sort are already ‘partially sorted’ Come up with a formal way to measure ‘how sorted’ a list is. Let a. What is its running time? (b) Give a lower bound on how well any algorithm with this hardware can do. Devise an algorithm that sorts A in substantially less than O(n log n). merges them in am mb steps. NO CODE). 33. 34. remove the root of the FEAP and insert the new element. Let a FEAP be just like a HEAP except that the internal nodes have FIVE children instead of TWO (only the right most internal node might have less than FIVE children). however whatever answer you give must be well thought out.b ). (a) Write an algorithm that uses this hardware to. Find a function fa.n]. It should match the upper bound.e.. Assume you have some kind of superhardware that..n] only has numbers from {1. when given 12 sorted lists of length m.) 35. Give an example of your algorithm at work.) You must describe your algorithm informally (i.. sort it. given an array A[1. Devise an algorithm that does very well on lists that are almost sorted (according to your deﬁnition). Your deﬁnition must be rigorous. (a) Give an algorithm that does the following: given a FEAP and a new element. Assume that the array A[1. .(a) Give an algorithm for MAX (of n elements) that uses this hardware. b does your algorithm run substantially faster than O(n log n)? 36. b ≥ 0.) (c) For what values of a. n2 } but that at most log log n of these numbers ever appear. (b) Let T (n) be the running time of your algorithm.b might involve several cases. . (When you are done you must have a FEAP again. .
b be real numbers such that a.m] returns the following: the ( m )th largest element. Also consider the model where you can ask k comparisons at a time. output some element larger than the median element.. Find functions gB (n) and gW (n) such that your algorithm takes gB (n) + θ(1) comparisons in the Best case. b ≥ 0. Let a. Assume 5 5 this hardware runs in time ma (log m)b . when given an array A[1. Assume you have some kind of superhardware that.) (b) Assume m is a constant.) 40. (Note that multiplicative constants are important.b ). (The function fa. 38. . 56
.) (a) Write an algorithm that uses this hardware to sort lists of length n. Consider the following problem: given a list of n elements. (Note that multiplicative constants are important. etc..) 39. (The end result is a heap that has that one element deleted. Find functions hB (n) and hW (n) such that your algorithm takes hB (n) + θ(1) comparisons in the Best case.b such that T (n) = θ(fa. ﬁnd and delete the mth largest element in the heap. . and the ( 4m )th largest element. . (You may assume √ that n and n1.) 37. (b) Let T (n) be the running time of your algorithm.) Devise an O(n) algorithm to sort A. (Formally the numbers above should be m . the ( 2m )th largest element. (a) Exhibit an O(log n) algorithm that will.best case? How many moves does it make in the worst case? How many moves does it make in the best case? (Do not ignore multiplicative constants. n1. given a heap on n elements and a number m with m ≤ log log n. .. Derive both upper and lower bounds for this problem in terms of the number of comparisons needed.2 are integers. 5 5 the ( 3m )th largest element. Find a function fa.2 } but √ that all numbers that appear are multiples of n. and gW (n) + θ(1) comparisons in the Worst case. but you may ignore this for the purposes of this 5 problem. and hW (n) + θ(1) comparisons in the Worst case.n] only has numbers from {1.) (c) Assume m is log log n.b might involve several cases and might depend on a and b. Assume that the array A[1.
Write an algorithm to ﬁnd an i such that A(i) = i (if one exists). using exactly r − p comparisons.41. < A(n). (HINT: Imagine you are running a tennis tournament and wish to ﬁnd the best and worst players. which should also be very brief. 42. Show that the professor is wrong by proving that the number of three way comparisons required to sort n elements is still Ω(n lg n). (b) Give an algorithm that uses substantially less than 2n comparisons. in which the control ﬂow of the program can split three ways after a single comparison ai : aj . Write an algorithm Partition(A.r. 44. Assume that we have MAGIC HARDWARE that can ﬁnd the 3rd largest of log n elements in 1 step. Prove that the elements in each row remain sorted. Give a lower bound on how well any algorithm with this hardware can do.p.s) to partition array A from p to r based on element A[s].) (c) Show that the algorithm presented in the last part is optimal. Show how to implement quicksort in worst case O(n lg n) time. Your algorithm should be high level and very brief. The professor claims that the Ω(n lg n) lower bound for sorting n numbers does not apply to his workstation. Now sort (in place) the elements in each column into increasing order. 47.
57
. Each row is given in increasing sorted order. Let hilo BE the problem of. 46. ﬁnd the smallest and largest elements on the list.) ? 43. given n numbers. according to whether ai < aj .) Supply the analysis. . (a) Give an algorithm to solve HILO with 2n − 3 comparisons. Let A be an array of positive integers (> 0). 45. ai = aj . . You may assume that n is even. (HINT: We need to choose the partition element carefully. where A(1) < A(2) < . What is the running time of your algorithm ? If the integers are allowed to be both positive and negative. Consider a rectangular array of distinct elements. then what is the running time of the algorithm (Hint: redesign a new algorithm for this part. or ai > aj .
can you ﬁnd its average case? 49. T (n) ≤ an. Pick two elements of the list. (You can let c be the number of comparisons needed to ﬁnd the median of 7 elements. Find a constant a such that. Partition based on both of the elements. Can you reﬁne your analysis of this version of quicksort? For example. a. The usual deterministic linear time (worst case) SELECT algorithm begins by breaking the set into groups of 5 elements each. Assume the partition elements always partition exactly into thirds. d. but instead of two children. Solve the recurrence.48. (c) Explain loosely (but clearly) how to extract the maximum element from the dary heap (and restore the heap). (a) How would you represent a dary heap in an array? (b) What is the height of a dary heap of n elements in terms of n and d. How many comparisons does it require? (d) How many comparisons does it take to sort? Just get the high order term exactly. Assume the partition elements always partition so exactly one quarter are to the left. Write high level code for this algorithm. and one quarter to the right. for all n. So the elements smaller than both are to the left. but show your calculations. Find a constant a such that. b. the elements in between are in the middle. T (n) ≤ an. A dary heap is like a binary heap. one half in the middle. the number of comparisons. the number of comparisons. Write a recurrence for T (n). How many comparisons and exchanges does the partition algorithm use? c. Assume that we changed the algorithm to break the set into groups of 7 elements each. a. 58
. for all n.) b. 50. and the elements larger than both are to the right. Write a recurrence for T (n). Write down the recurrence for the new running time. nodes have d children. Consider the following quicksortlike sorting algorithm. e.
59
.(e) What value(s) of d are optimal? Justify your answer. (g) Heapsort makes about n lg n moves. (a) Assume that the partition element ends up in position q. (a) Brieﬂy describe the algorithm in English. Devise an algorithm that ﬁnds the second largest of four elements. Starting from there look for the left most element that is larger and place it in the newly opened position on the right side. Take it out. Starting from there look for the right most element that is smaller and place it in the the newly opened position on the left side. What is the probability that an element originally to the left went to the right? (b) Assume that the partition element ends up in position q. 51. How does quicksort compare? 52. (e) Simplify the recurrence. Try to get a good value of a. We are going to derive the average number of moves for quicksort using a somewhat unusual partitioning algorithm. Finally. What is the expected number of elements originally to the left that go to the right? (c) Assume that the partition element ends up in position q. (b) Give a decision tree for it. put the partition element into the hole. We partition on the ﬁrst element. Look for the left most element that is larger and place it in the newly opened position on the right side. What is the expected number of moves? (d) Write a recurrence for the expected number of moves. assume you got the following recurrence: q2 2 n−1 T (q) + q − ] T (n) = [ n q=1 n Use constructive induction to show that T (n) ≤ an ln n. which is its ﬁnal position in the sorted array. but do not solve it. Look for the right most element that is smaller and place it in the ﬁrst position (which is the newly opened position on the left side). (f) To keep the calculations simple. Continue in this fashion until the pointers cross.
Show that T has depth n + lg n − O(1).53. Argue that Ti is essentially a decision tree to ﬁnd MAX on {x1 . Let i be such that 1 ≤ i ≤ n. Give upper and lower bounds for sorting using this component. In all of the arrangements. xn } − {xi }. 56. 54. for all i. iv. . . Y where each list keeps its original order. . . Show that. i. i.) (a) Let S(m. Let SECn be the problem of ﬁnding the second largest of n numbers.. whenever there is a comparison between xi and xj . In this problem you will ﬁnd S(m.) (b) Show that any Decision tree for M AXn has 2n−1 leaves. v. every branch has depth at least n − 1. eliminate the comparison by assuming that xi wins. Let Ti be formed as follows: take tree T and.e. Assume you had a special hardware component that could sort n numbers in one step. m ii. √ 55. n) be the number of arrangements of two lists X. Show that. Find a lower bound on the number of leaves in T by using the last two items. only comparisons are the main operation. iii. Show that S(m. We are going to derive a lower bound for merging two lists of sizes m. We assume the decision tree model. the leaves of Ti and the leaves of Tj are disjoint. (HINT: if only n − 2 comparisons are made then show that the maximum cannot be determined. n. Ti has 2n−2 leaves. ii. either the ﬁrst element of X is the smallest or the ﬁrst element of Y is the smallest (but 60
. for all i and j. Give upper and lower bounds for sorting using this component. Assume you had a special hardware component that could ﬁnd the √ minimum of n numbers in one step. (Assume all of the elements are distinct.n)=m+n using a combinatorial argument. (c) Let T be a Decision tree for SECn . n) two ways. Let M AXn be the problem of ﬁnding the max of n numbers. (a) Show that in any Decision Tree for M AXn . i.
Make your bound as exact as possible. etc. (b) Give an algorithm to sort in 2n/3 comparisons in the average case (assuming each of the n inputs is 0 or 1 with equal probability). For each comparison of two values x and y. A little looseness in your “proof” is acceptable here. (c) What is your answer when m = n (write it as simply as possible). (a) Give an algorithm to sort in n − 1 comparisons in the worst case. so that the ﬁrst group has the smallest n/k elements (not necessarily sorted). or x > y holds. 59. x = y. Show that your algorithm is optimal. the second group has the next smallest n/k elements (not necessarily sorted). (b) Give a lower bound for the number of comparisons needed to merge two lists. Justify your answer.) a. How many arrangements of the remaining elements are there? Assume it is the ﬁrst element of Y . where each element is within k positions of its correct sorted position (on either side).) (a) Give a lower bound on the number of comparisons it takes to sort the list. Simplify your answer using Stirling’s formula. Consider the problem of sorting a sequence of n 0’s and 1’s using comparisons. (b) Give an eﬃcient comparisonbased algorithm for sorting the list.
61
. (c) Compare your upper and lower bounds. How many arrangements of the remaining elements are there? Based on this information. How does it compare to the lower bound of 2n − 1. Show that m+n is a m solution to your recurrence. Assume you have a list of n elements. Derive a lower bound for the number of comparisons needed for solving this problem. (If k = 0 then the deck is sorted. the algorithm learns which of x < y. Consider the problem of partitioning n elements into k equal sized groups. (You may assume that k divides n. 58.not both). Assume it is the ﬁrst element of X. How many comparisons does it use in the worst case. n). 57. Show that your algorithm is optimal. write down a recurrence for S(m.
Design a linear (i. POT. (a) Assume you are given a number X and two sorted lists of n numbers each: A1 ≤ A2 ≤ · · · ≤ An and B1 ≤ B2 ≤ · · · ≤ Bn .n][1. OPT. the pair counting problem can be solved in o(n) time.
62
. na.n]. SUP. Justify the correctness and running time of your algorithm. 60. assume that you have access to a procedure Merge(A. This procedure runs in O(na + nb ) time.. Illustrate the operation of radix sort on the following list of English words: RUT. D[i][j] <= D[i][j+1]). How much preprocessing time do you use? How much storage is used? How much time is needed to answer one query (after preprocessing)? 61. such that each array is sorted (i.e. nb). B. TON. TOP. (b) Show how to preprocess the lists A and B (in any way that you like and producing any data structure that you like) so that given X.e. OUT. Suppose that you have a collection of n.nb] into a single sorted array C[1. D[1. See the ﬁgure below. Analyze its running time. 1dimensional arrays each of length n. The ﬁrst strategy for accomplishing this is called cascading merging. that merges two sorted arrays A[1. The ﬁnal merge is between an array of length (n − 1)n with D[n] to form the ﬁnal sorted array of length n2 . then merge this array with D[3] to form an array of length 3n. First merge D[1] with D[2] to form an array of length 2n.na] and B[1. O(n)) time algorithm to determine how many pairs (i. ROT. etc. We want to compute a single sorted array that contains all n2 elements. Give an eﬃcient algorithm for solving this problem. For this problem.. PUT 62. TOO.. c. Assume we also have access to unlimited dynamic array allocation. SUR.na+nb]... j) there are such that Ai + Bj ≤ X. TOR. C.b.
repeat the balanced merging on these arrays. For 1 ≤ i ≤ k there are mi copies of the ith value. Selection Sort can be thought of as a recursive algorithm as follows: Find the largest element and put it at the end of the list (to be sorted). (a) Show that n! m1 !m2 ! . (a) What is the total running time of cascading merging? Justify your answer. First merge pairs D[1] and D[2] together. . give an algorithm). 63. . Next.D[1] D[2] D[3] D[4]
❍❍ ✟✟❍❍❍ ✟✟ ❍ ✟✟ ✟✟✟ ✟ ✟✟
D[1] D[2] D[3] D[4]
❍❍ ✟✟❍❍❍ ❍ ✟✟ ❍❍✟✟ ✟✟
Cascading Merging
Balanced Merging
The second strategy is called balanced merging. The result is n/2 sorted arrays of size 2n each. (Note: You are required to show ONLY the lower bound. mk are.) Note that any algorithm for this problem does NOT know in advance what the values of m1 . . you do not need to show that this is always possible. resulting in n/4 arrays of size 4n each. This is repeated until there is 1 array of size n2 . and so on until merging D[n1] and D[n] together. 63
. . . D[3] and D[4] together. Assume that n is a power of 2. . m2 . For both parts (a) and (b) count only the time spent in the merging procedure. (b) What is the total running time of balanced merging? Justify your answer. 64. Derive the running time of your algorithm. mk ! comparisons are necessary to sort this list by a comparisonbased sorting algorithm. Recursively sort the remaining elements. Suppose you have a list of n numbers consisting of k ≤ n distinct values.e. O n + lg (b) Show that this many comparisons are suﬃcient to sort (i.
5) Compare each pair of elements vi and ui+1 (and exchange if vi > ui+1 ). (a) Write a recurrence for how many comparisons this merging algorithm uses. . y2 . n − 1]. . . y3 . n]. . . . . . . . Answer the following questions about the algorithm. y4 . . . . . v2 . . b. 3) Merge X and Y into V . . . u3 . c. b. 2) Merge X and Y into U . a. d. Assume n > 1 is a power of two. .) 1) Separate X and Y into even and odd indexed items. yn . 66. yn−1 and Y = y2 . Derive a recurrence for the exact number of comparisons the algorithm uses. xn . yn . . . 4) Interleave U and V into u1 . (c) Prove that this algorithm merges correctly. . Use mathematical induction to verify your solution. . Consider the following algorithm that merges two lists: X = x1 . c. recursively sort A[1 . un . v3 . vn . . Simplify as much as possible. x4 . just compare the two elements. . In order to sort A[1 . . n − 1] and then insert A[n] into the sorted array A[1 . . . (b) Solve the recurrence using the iteration method. v1 . Use the iteration method to solve the recurrence. . Give a recurrence for the number of comparisons this algorithm uses in the worstcase. Insertion sort can be expressed as a recursive procedure as follows. . . Write down the recursive version of Selection Sort in pseudocode. xn−1 and X = x2 . xn and Y = y1 . (For n = 1. u2 . What is the solution to the recurrence.a. Prove it using mathematical induction. . 65. Write the pseudocode for this (recursive) algorithm. .
64
. x2 . so X = x1 . and Y = y1 . x3 . . .
Consider an n × n array A containing integer elements (positive.. 68.e. Analyze its running time. then p − xM  < . and the elements if each column of A are in strictly decreasing order. (Hence there cannot be two zeroes in the same row or the same column. (Note: xM may be 0 or 1.) Describe an eﬃcient algorithm that counts the number of occurrences of the element 0 in A. Let f be a function deﬁned on the interval 0 to 1 (i. 0 ≤ x ≤ 1). (a) Write an algorithm to ﬁnd xM with error at most .e. Use instructions of the form y ← f (x) to indicate evaluation of f at the point x. and is strictly increasing for 0 ≤ x < xM and strictly decreasing for xM < x ≤ 1. if your algorithm’s answer is p. Count your evaluations as exactly as possible.) The only operation you can do with f is evaluate it at points in its interval. i. which achieves its maximum value at point xM . Assume that the elements in each row of A are in strictly increasing order. Try to use as few evaluations as possible. (b) How many evaluations does your algorithm do in the worst case?
65
.67. a point in the interval. and zero). negative.
Show that A1 is NPcomplete. Express the runtime for the algorithm for A in terms of p1 . ak . ak . k)  G has a vertex cover of size ≤ k}. Show that A2 can be solved in O(na ) where a < 10. .NPCompleteness 1. Let G be a graph where every edge is labelled with a diﬀerent Boolean variable. f12 takes time p12 (n). If it is known to be in P then give an algorithm for it that takes polynomial time. A2 = {G  G has a vertex cover of size ≤ 12}. . If it is not known to be in P then nothing more is required. (b) A2 = {(a1 . . p2 . . Let v be a vertex of G. . f12 (x)} are in B. . Show that if B ∈ P and A ≤ww B then A ∈ P. . N )  some subset of {a1 . ak } adds up to a number that is ≤ N }. . . . ak . (a) A1 = {(a1 . . . . . .v as v ranges over the vertices of G. . E) is a graph then a vertex cover for G is a set V ⊆ V such that for every edge in G one of the endpoints is in V . . 4. Let φG. Show that A1 ∈ NP. N )  some subset of {a1 . p12 and q. . . . determine if φG is satisﬁable. Show that A2 ∈ P. where n is the length of the input. Then state whether or not it is known to be in P. . Let φG be the AND of all the φG. . . . N )  some subset of {a1 . f12 such that each one can be computed in polynomial time and x ∈ A iﬀ at least 6 elements of {f1 (x). Is the following problem NPcomplete: Given a graph G that has edges labelled with diﬀerent Boolean variables. . 3. . . If G = (V. . . . . . . f2 takes time p2 (n). (c) A3 = {(a1 . . . ak } adds up to a number that is ≥ N } 5. . 66
. . Assume f1 takes time p1 (n). We deﬁne A ≤ww B if there are 12 functions f1 . .. 2.v be the exclusiveOR of all the edges that have v as an endpoint. For each of the following sets state whether it is in NP or not (no proof required). and the algorithm for B takes time q(n). . Let A1 = {(G. ak } adds up to exactly N }.
) f. Show that the decision version is in NP. E). Deﬁne a decision version of the stable set problem. d. does there exist a simple path that passes through every vertex of G?
67
. Show that you can solve the decision version for constant size stable sets in polynomial time. a. does there exist a simple path that passes through every vertex of G? The Directed Hamiltonian cycle problem (DHC) is: given a directed graph G = (V. With this. We deﬁne A ≤w B if there are 2 functions f1 . f2 takes time p2 (n). does there exist a simple path that passes through every vertex of G? The Hamiltonian cycle problem (HC) is: given an undirected graph G = (V. E). (Assume f1 takes time p1 (n). f2 such that each one can be computed in polynomial time and / x ∈ A iﬀ f1 (x) ∈ B and f2 (x) ∈ B. p2 . c. you should be able to express the running time for the algorithm for A in terms of p1 . A stable set of vertices in a graph do not have any edges between them. and the algorithm for B takes time q(n). Show that it is NPcomplete. e. The Hamiltonian path problem (HP) is: given an undirected graph G = (V. Prove that P = NP.) 7. Show that if you could solve the optimization version in polynomial time that you could also solve the decision version in polynomial time. E). does there exist a simple path that passes through every vertex of G? The Directed Hamiltonian path problem (DHP) is: given a directed graph G = (V.6. Show that if B ∈ P and A ≤w B then A ∈ P. b. and q. 9. E). 8. (Or alternatively show that it is polynomially equivalent to the clique problem. then P = coNP. HINT: Use CLIQUE. The optimization version of this problem is to ﬁnd a maximum stable set in a graph. Show that if you could solve the decision version in polynomial time that you could also solve the optimization version in polynomial time.
11. and there are no precedence constraints. (HINT on HP ≤p HC. with positive and negative weights w(e) on the edges e ∈ E. You could pick any problem to reduce from. PARTITION: Given a ﬁnite set A and a “size” s(a) (a positive integer) for each a ∈ A. and a “length” (x) for each task. DHC} show that P1 ≤p P2 . x and y. Is there a way of assigning the tasks to the three processors such that all the tasks are completed within the deadline D? A task can be scheduled on any processor. that are adjacent to the same vertices that v was.Show that all four of the problems above are in NP. Consider the problem DENSE SUBGRAPH: Given G. Is there a subset A ⊆ A such that a∈A s(a) = a∈A−A s(a) ? Now prove that the following SCHEDULING problem is NPcomplete: Given a set X of “tasks”. and an integer K.) 10. P2 from {HP. E). Let p(n) be a polynomial. does G contain a spanning tree with exactly K leaves ? (a) First prove that the problem is in NP by describing a “guessing” algorithm for it. E). Is there a simple cycle of zero weight in G? (Hint: Reduce PARTITION to ZERO CYCLE. Let A be an algorithm that accepts strings in a language L.) 68
. but in superpolynomial time if x is not in L. Replace some vertex v ∈ V by two new vertices. does it contain a subgraph H that has exactly K vertices and at least Y edges ? Prove that this problem is NPcomplete. 14. Assume that the following problem is NPcomplete. (Write out the algorithm in detail. Prove that the following ZERO CYCLE problem is NPcomplete: Given a simple directed graph G = (V. HC. and a “deadline” D. (Hint: ﬁrst prove the NPcompleteness of SCHEDULING with two processors. SPANNING TREE WITH A FIXED NUMBER OF LEAVES: Given a graph G(V. Assume that A runs in p(n) time if a given string x is in L. You will need to change the graph. What else do you need to do? Try to “force” any Hamiltonian path to visit these vertices ﬁrst and last. DHP.) 13. three processors. Prove that the following problem is NPcomplete. For all pairs of problems P1 . Give a polynomial time algorithm for recognizing strings in L.) 12.
going through each remaining vertex exactly once.(b) Now prove that the problem is NPhard. Subgraph Isomorphism: Given two undirected graphs G1 = (V1 . E1 ) and G2 = (V2 . in polynomial time returns 69
. v} ∈ E1 . and “False” otherwise. Show that both of the following two problems are NPcomplete. in polynomial time.) (b) How can you use the function CLIQUE to actually compute a clique that is largest in size ? (Try and solve the problem by asking the function CLIQUE as few questions as possible.) 18. the size of the largest clique by making.) 16. (Try and solve the problem by asking the function CLIQUE as few questions as possible. Note that a graph can have an exponential number of cycles. K) that returns “True” if G contains a clique of size at least K. Suppose we had a Boolean function SAT (F ). E) and an integer k is there a subset of vertices V ⊆ V of size k such that every cycle of G passes through a vertex in V ? Prove that CC is NPcomplete. The reduction is from vertex cover.) 17. {π(u). which given a Boolean formula F in conjunctive normal form. by reducing UNDIRECTED HAMILTONIAN PATH to it. (Hint: The proof that CC is in NP is tricky. and show that a known NPcomplete problem is reducible to this problem. is G1 is a subgraph of G2 ? More formally is there are 1–1 function π : V1 → V2 such that for all {u. 15. For each show that the problem is in NP. E2 ). The cycle cover problem (CC) is: given a directed graph G = (V. does G have a simple cycle of length greater than or equal to k? (Hint: Use Hamiltonian Cycle. (Hint: Use either CLIQUE or Hamiltonian Cycle. HAMILTONIAN PATH asks for a simple path that starts at some vertex and goes to some other vertex. Assume that you are given access to function CLIQUE(G. π(v)} ∈ E2 .) Longest Simple Cycle: Given an undirected graph G and an integer k. so the naive veriﬁcation algorithm will not work. (a) Use the function CLIQUE to determine.
Show that DOM is NPcomplete (Hint: Use the Vertex Cover problem. either v ∈ V or v is adjacent to an element of V . Show that P LAN AR − DOM is NPcomplete (Hint: Use the DOM problem. 19. E) is a graph then a Dominating Set is a set V ⊆ V such that.) (b) Let P LAN AR−DOM = {(G. k be a ﬁxed constant. (a) Show that the problem of testing if a graph is kcolorable is NPcomplete. (b) q − k − SAT be the set of all qCNF formulas that are satisﬁable with an assignment that has at most k variables set to TRUE. Let IN D−P LAN AR−DOMk = {G  G is planar and has a dominating set of size k Show that IN D − P LAN AR − DOMk can be solved in O(6k n) time where the constant does not depend on k. k)  G is planar and has a dominating set of size k}. Show that q − k − M ON O − SAT can be solved in O(2k n) where the constant does not depend on k.)
(d) Fix k. 21. (b) Show that if G is a graph and v is a node of degree ≤ d then G is d + 1colorable iﬀ G − {v} is d + 1colorable. Let qCNF be CNF where each clause has ≤ q literals. k)  G is planar and has a dominating set of size k tha Show that IN D − P LAN AR − DOM is NPcomplete (Hint: Use the P LAN AR − DOM problem. 70
. A formulas is MONOTONE if it has no negated variables. (a) Let DOM = {(G. k)  G has a dominating set of size k}. Show how to use this function to determine the actual variable assignment that makes the formula True. Show that q−k−SAT can be solved in O(2k n) where the constant does not depend on k. If G = (V. (a) Let q − k − M ON O − SAT be the set of all qCNF monotone formulas that are satisﬁable with an assignment that has at most k variables set to TRUE.)
(c) Let IN D−P LAN AR−DOM = {(G. (Hint: Use that a planar graph always has a vertex of degree ≤ 5. for all vertices v. (This is very unlikely since it would imply P = NP!) This function tells you nothing about how to set the variables to satisfy the formula.) 20. Let q.True if the function is satisﬁable and False otherwise.
(a) Let A = {G  either G or G has a Hamiltonian Cycle}. If G = (V. G is called the complement of G. Show that HAM CGC is NPcomplete. You may use the fact that HAM P is NPcomplete. 22. Show that determining if a graph is n − kcolorable can be done in O(2k n) time. a. E) be deﬁned by / (x. y) ∈ E iﬀ (x.(c) Fix k. (Let HAM P be the set of all (H. b) such that H has a Hamiltonian Path that starts at a and ends at b. (b) Make up and solve similar problems to the one above where you take some property of graphs and ask if the graph or its compliment has it.) 23. Either show A ∈ P or that A is NPcomplete. y) ∈ E.
71
. Let HAM CGC be the set of graphs G such that either G or G has a Hamiltonian cycle. E) is an undirected graph then let G = (V.