05758 Computer Algorithms Introduction to Design and Analysis THIRD EDITION . Sara Baase San Diego State University Allen Van Gelder University of California at Santa Cruz ‘yy ADDISON-WESLEY An imprint of Addison Wesley Longman, Inc. Reatling, Massachusetls # Menlo Pack, California * New York Harlow, Enyland nd + Don Mls, Ontario» Sney + wesio City Maclrid ®To Keith—always part of what I do To my parents, who were my first and most important teachers S.B. AVG.Assistant Editor: Jason Miranda . Composition/Art: Paul C. Abognostopoules, Windlall Software Copy Edtuor: Jona Flaherty Proofreader: Brooke Albright Cover ttustration: Ianetmarie Colby aver Design: Lynne Reed Menufacturing Coordinator: Timothy McDonald ‘Access the latest information abou Addison-Wesley titles from our World Wide Web site; wwsvaelonline com Many of the designations used by manufacturers and solters to distinguish their products are elaimed as ‘rademarks. Where those desigontions appear in this book, and Addison-Wesley was aware of a trademark: the designations have been printed in inital eaps or all caps, ‘The programs and applications presented in this book have been inchaded for their instructional value. They have been tested with care but ore not gvaranieed for any purpose. The publisher does net offer any warranties or representations. nos does it accept any liabilities with respect tothe programs or applications ‘This book was typeset in ZX on @ PC. The font fami used were Times, Optima, Lucida Sans, and ‘MathTime. h was printed on New Era Maite, Library of Congress Catsloging-In-Publlcation Data Basse, Sara, Computer algorithms / Sara Race, Allen Van Gelder, — 3rd ed. Pwd ISBN 0-201-61244.5 1. Computer algorithms. 1. Gelder, Alle Von, Ih. Title. QA769.A43B33 2000 519.7 —4e21 9914185, ce Copyright © 2000 by Addison Wesley Longman ‘All cights reserved, No port of this publication may be reproduced, stored in arevieval system, or transmitted, in any form oF by any menns,electionic, mechanical, photocopying. recording, or otherwise, withent the prior ‘writen permission of the publisher. Printed inthe United States of America 123456789 10-Ma—onm2010099Preface Purpose This book is intended for an upper-division or graduate course in algorithms. It has suff ent material to allow several choices of topics. ‘The purpose of the book is threefold, It is intended to teach algarithms for solving real problems that arise frequently in computer upplications, to teach basic principles and techniques of computational complexity (worst-case and average behavior, space usage, and lower bounds on the complexity of a problem), and to introduce the arens of NP- completeness and parallel algorithms. . Another of the book's aims, which is at least as importunt as teaching the subject matter, is to develop in the reader the habit of always responding to a new algorittun with the questions: How good is it? Is there a better way? Therefore, instead of presenting a series of complete, “pulled-out-of-a-hat” algorithms with analysis, the text often discusses ‘a problem first, considers one or more upproaches (o solving it (as a reader who sees the problem for the first time might), and then begins to develop un algorithm, analyzes it, and ‘modifies or rejects it until a satisfactory result is produced. (Altemnative approaches that are ultimately rejected ure also considered in the exercises; it is useful for the reader to know why they were rejected.) Questions such as: How can this be done more efficiently? What data structure would be usefut here? Which operations should we focus on to analyze this algorithm? How ‘must this variable (or data structure) be initialized? appeur frequently throughout the text. ‘Answers generally follow the questions, but we suggest readers pause before reading the ensuing text and think up their own answers, Learning is not a passive process. We hope readers will also leurn to be aware of how an algorithm uctually behaves ‘on various inputs—that is, Which branches are followed? What is the puttern of growth and shrinkage of stacks? How does presenting the input in different ways (e.., listing the vertices or edges of a graph in u diferent order) affect the behavior? Such questions ute raised in some of the exercises, but are not emplusized in the text because they require carefully going through the details of many examples. ‘Most of the algorithms presented ure of prictical use; we have chosen not to empha- size those with good asymptotic behavior that are poor for inputs of useful sizes (though ‘some important ones are included). Specific. algorithms were chosen for a variety of reasonsviti_ Preface including the importance of the problem, illustrating unalysis techniques, ilNistrating tech niques (¢-g., depth-first search) that give rise to numerous algorithms, and illustrating the development and improvement of techniques and algorithons (e.g., Union-Find programs), Prerequisites The book assumes familiarity with data structures such as linked lists, stacks, and trees, and prior exposure to recursion. However, we include a review, with specifications, for the standard data structures and some speciatized ones. We have also added a student-friendly review of recursion, Analysis of algorithms uses simple properties of logarithms and some calculus (dif- ferentiation to determine the asymptotic order of a function and integration to approximate summations), though virtually no calculus is used beyond Chapter 4. We find many stu- dents intimidated when they see the fitst log or integral sign because a year or more has passed since they had a calculus course, Readers will need only a few properties of logs ‘and a few integrals from first-semester calculus. Section 1.3 reviews some of the necessary mathematics, and Section 1,5.4 provides a practical guide. Algorithm Design Techniques Several important algorithm design techniques reappear in many algorithms. These in- Clude dividetind-conquer, greedy methods. depth-first search (for graphs), and dynamic Programming, This edition puts more emphasis on algorithm design techniques than did the second edition. Dynamic progtamming, as before, has its own chapter and depth-first Search is presented with many applications in the chapter on graph traversals (Chapter 7). Most chapters are organized by application area, rather than by design technique, so we provide here a list of places where you will find algorithms using divide-and-conquer and greedy techniques. The divide-and-conquer technique is described in Section 4.3, It is used in Binory Search (Section 1.6), most sorting methods (Chapter 4), median finding and the general selection problem (Section 5.4), binary search trees (Section 6.4), polynomial evaluation (Section 12.2), matrix multiplication (Section 12.3), the Fast Fourier Transform (Sec- tion 12.4), approximate graph coloring (Section 13.7). and, in a stightly different fori, for paralle! computation in Section 14.5. ‘ Greedy algorithms are used for finding minimum spanning trees and shortest paths in Chapter 8, and for various approximation algorithms for NP-hard optimization problems. such as bin packing. knapsack, graph coloring, and traveling salesperson (Sections 13.4 through 13.8), Changes from the Second Edition This edition has three new chapters and many new topics. Throughout the book, numerous sections have been extensively rewritten, A few topics froin the second edition have been moved to different chapters where we think they fit Better, We aclded nore than 100 new exercises, many bibliographic entries, and an appendix with Java examples. Chapters 2, 3, and 6 are virtually all new. LePreface Chapter 2 reviews abstract data types (ADTs) and inclules specifications for severel standard ADTs. The vole of abstiact data types in algorithm design is emphasized through- out the book, Chapter 3 reviews recursion and induction, emphasizing the connection between the {wo and their usefulness in designing and proving correctness of programs, The chapter also develops recursion trees, which provide a visual and intuitive representation of recur rence equations that arise in the analysis of recursive algorithms, Solutions for commonly occurring patterns are summarized so they are available for use in tater chapters. Chapter 6 covers hashing, red-black trees For balanced binary trees, advanced priority queves. and dynamic equivatence relations (Union-Find). The later topic was moved from a different chapter in the second edition. ‘We rewrote all algorithms in a Java-based pscudocode. Familiarity with Java is not Tequired: the algorithms can be read easily by anyone familiar with C or C++. Chapter | has an introduction to the Java-based pseudocovte We significantly expanded the section on mathematical tooks for algorithm analysis in Chapter I to provide a better review and reference for some of the mathemitics used in the book. The discussion of the asymptotic aider of functions in Sectioit £.5 was designed! {0 help students gain a better mastery of the concepis and techniques for dealing with ‘asymptotic order, We added rules, in informal fanguage, thal summarize the most common cases (Section 1.5.4), Chapter 4 contains an accelerated version of Heapsort in which the number of key comparisons is cut nearly in hulf, For Quicksort, we use the Hoare panition algorithm in the main text. Lomuto’s method is introrluced in an exercise. (This is reversed from the second edition.) We sptit the old graph chapter into two chapters, and changed the order of some topics. Chapter 7 concentrates on (linear time) traversal algorithms, ‘The presentation of depth-first search has been thoroughly revised to emphasize the general structure af the technique and show more applications. We added topological sorting and critical path analysis ts applications and because of their intrinsic value and their connection to dynamic Programming. Sharir’s algorithm, rather than Tarjan’s, is presented for strongly connected ‘components, Chapter 8 covers greedy algorithms for graph problems, The presentations of the Prim algorithm for minimuin spanning trees and the Dijkstra algorithm for shortest paths were rewritten to emphasize the roles of priarity quetes and to illustrate how the use of abstract data types can lead the designer to efficient implementations. The asymptotically optimal ‘(eu + 1 log m) implementation is mentioned, but is not covered in depth. We moved Kroskal's algorithin for minimum spanning trees to this chapter. The presentation of dynamic programming (Chapter 10) was substantially revised to emphasize a general approach to finding dynamic programming solutions. We addled a fnew application, a text-formatting problem, to reinforce the point that not all applications call for a two-dimensional array, We moved the approximate string matching application (which wies in this chapter in the second edition) to the sting matching chapter (Sec- tion 11.5). The exercises inclucle some other new application:Preface ‘Our teaching experience has pinpointed particular areus where students bad sitficulties with concepts related to P and NP (Chapter 13), particutasly nondeterministic algorithms and polynomial transformations, We rewrote souve definitions and examples to make the concepls clearer. We added a short section on approximation algorithms for the traveling salesperson problem sud a seetion on DNA computing. Instructors who used the second edition may particularly want to note that we changed some conventions and terminology (usually to conform to conmmon usage). Array indexes now offen begin at 0 instead of I. (ln some cases, where numbering fiom 1 was clearer, we left it that way.) We now ase the term depal rather than Jevel for the depth of a node ina wee. We use height instead of depit for the maximum depth of any node in a tree. In the second euition, a path in a graph was defined to be what is commonly called a simple arly, we use the ore general definition for part ia this edition and define simple path separately. A directed graph may now contain a self-edge. Exercises and Programs Some exercises ure somewhat open-ended. Kor example, one might ask for a good lower bound for the complexity of a problem, ruler shi asking students to show tha a given function is a lower bound, We did this for two reasons. One is to niake the form of the question move realistic; a solution must be discovered as well as verified. The other iy that it may be har for some students to prove the best known tower bound (or find the most” efficiemt algorithm for a problema), but there is still a range of solutions they cun offer to. show their mastery of the techniques studied, Some topics and interesting problems are introduced only in exercises, For example, wependent set problem For a tee is an exercise in Chapter 3, the maxiniuin subsequence sum problem is an exercise in Chapter 4, and the sink finding problem for @ graph is an exercise in Chapter 7. Several NP-complete problems are introduced in exercises in Chapter 13. “The abilities, background, and mathematical sophistication of students at different uni- versilies vary cousiderably, muking it difficult to decide exactly which exercises should be marked ("starred") as “hard.” We starred exercises that use more than minimal matheraat- ies, require substuntiat creativity, or require a long chain of reasoning. A few exercises have two stars, Some starred exercises have hints. The algorithms presented in this book are not prograns; that is, many details not important to the method or the analysis are omitted. Of course, students should know how to implement efficient algorithms in efficient, debugged programs. Many insivuctors may leach this course as a pure “theory” course without programming. For those who want to assign programming projects, most chapters include a list of programming assignments, “These ure brief suggestions that may need amplification by instructors wlio choose t0 use then, Selecting Topics for Your Course Clearly the amount of material and the particular setection of topics to cover depend on the particular nt population. We present sample outlines for two undergraduate aduate course.Preface xi This outline corresponds at San Diego State University proximately (o the senior-level course Sura Baase teaches 15-week semester with 3 hours per week of lecture. Chapter 1: The whole chapter is assigned as reading but I conceotrate on Sections 1.4 and L.5 in class. : Sections 2.1 through 2.4 assigned as reading. ions 3.1 through 3.4, 3.6, and 3.7 assigned as reuding with light cover- sections 4.1 through ’ : Seetions 5.1 through 5.2. 5.6, and some of 5.4. ections 7.1 through 7.4 and either 7.5 oF 7.6 and 7.7. 8.1 through 8.3 and brief mention of 8.4, Chapter 11: Sections 11.4 through 11.4, Chapter 13: Sections 13.1 through 43.5, 13.8, and 13.9. ‘The next outline is the junior-level course Allen Van Gelder teachdS af the University of California, Santa Cruz, in a 10-week quarter with 3.5 hours per week of lecture, Chapter 1: Sections 1.3 and 1.5, and remaining sections us reading. Chapter 2: Sections 2.1 through 2.3, and remaining sections as reading, Chapter 3: All sections are touched on; a lot is left for reading. Chapter 4: Sections 4.1 through 4.9. ly Section 5.4, the average Linear tine algorithm only, Chapter 6: Sections 6.4 through 6.6, Chapter 7: Sections 7.4 through 7.6. _ Chapter 8: The entire chapter. Chapter 9: Sections 9.1 through 9.4. Chapter 10: Possibly Sections 10.1 through 10.3, but usually no time. For the first-year graduate course at the University of California, Santa Cruz (also 10 weeks, 3.5 hours of lecture), the above material is compressed and the following additional topics are covered. Chapter $: The entire chapter. Chapter 6: The remainder of the chapter, with emphasis on amortized analysis. Chapter 10: The entire chapter. Chapter 13: Sections 13.4 through 13.3, and possibly Section 13.9. ‘The primary dependencies among chapters are shown in the following diagram with sofid lines; some secondary dependencies are indicated with dashed lines. A secondary dependency means that only a few topics in the carlier chapter are needed in the later chapter, of that only the more udvanced sections of the later chapter require the earlier one.Preface While material in Chapters 2 and 6 is important to have seen, a lot of it might have been covered in an earlier course. Some sections in Chapter 6 arc important for the more advanced parts of Chapter 8. We like to remind reacers of common themes or techniques, so we often refer back (o earlier sections; many of these references can be ignored if the earlier sections were not covered, Several chapters have a section on lower bounds, which benefits from the ideas and examples in Chapter 5, but the diagram does not show that dependency because many instructors do not cover lower bounds. We marked (“starred”) sections that contain more complicated mathematics or more complex or sophisticated arguments than most others, but only where the material is ot central to the book, We also starred one or two sections that contain optional digressions. We have not starred a few sections that we consider essential to a course for which the book is used, even though they contain a lot of mathematics. For examaple, at least some of the ‘material in Section 1.5 on the asymptotic growth rate of functions and in Section 3.7 on solutions of recurrence equations should be covered. Acknowledgments ‘We are happy to take this opportunity to thank the people who helped in big and small ways in the preparation of the third edition of this book. Sara Baase acknowledges the influence and inspiration of Dick Karp, who made the subject of computationat complexity exciting and beautiful in his superb lectures. Allen ‘Van Gelder acknowledges the insights gained from Bob Floyd, Don Knuth, Est Mayr, Vaughan Pratt, and Jeff Ullman: they all teach more than is “in the book.” Allen also ‘wishes to acknowledge colleagues David Helmbold for many discussions en how to present algorithms effectively and on fine points of many algorithms, and Charlie McDowell for help on many of the aspects of Java that are covered in this book's appendix. We thank Lila Kari for reading an carly draft of the section on DNA computing and answering our questions, OF course, we'd have nothing to write about without the many people who did the original research that provided the material we enjoy learning and passing on to new generations of students, We thank them for their work. In the years since the second edition appeared, several students and instructors who used the book sent in lists of errors, typos, and suggestions for changes. We don't have @ complete list of names, but we appreciate the time and thought that went into their letters The surveys and manuscript reviews obtained by Addison-Wesley were especially helpful. Our thanks to liana Bjorling-Sachs (Lafayette College), Mohammad B. Dadfar «Bowling Green State University}, Daniel Hirschberg (University of California at Irvine),| j ! | ‘ Preface Mitsunoti Ogihara (University of Rochester), R. W. Robinson (University of Georgia), Yaakov L, Varol (University of Nevada, Reno), William W. White (Southern Ilinois Uni- versity at Edwardsville), Dawn Wilkins (University of Mississippi), and Abdou Youssef (George Washington University). ‘We thank our editors at Addison-Wesley, Maite Suarez-Rivas and Karen Wernholm, for their confidence and patience in working with us on this project that often departed from standard production procedures and schedules. We thank Joan Flaherty for her painstak- ingly careful copy editing and valuable suggestions for improving the presentation. Brooke Albright’s caceful proofreading detected many errors that had survived earlier scrutiny: of ‘course, any that remain are the fault of the authors, ‘We thank Keith Mayers for assisting us in various ways. Sara thanks him for not seminding her too often that she broke her wedding vow to work less than seven days # week, Sara Base, San Diego, California fttp://www-rohan,sdsu.edu/facuty/baase Allen Van Gelder, Santa Cruz, California http://www.cse.ucse.edu/persannel/faculty/avg html June, 1999Contents Preface vii Analyzing Algorithms and Problems: : Principles and Examples 1 1.1 Intyoduction 2 1.2 Javaas an Algorithm Language 3 1.3 Mathematical Background 11 14 Analyzing Algorithms and Problems — 30 1.5. Classifying Functions by Their Asymptotic Growth Rates 43 16 Sewrehing an Ordered Array 53. Exercises 61 Notes and References 67 Data Abstraction and Basic Data Structures 69 2.1 Introduction 70 2.2 ADT Specification and Design Techniques 71 23 Elememury ADTs—Lisis and Trees 73 24 Stucks and Queues 86 2.5 ADTs for Dynamic Sets 89 Exercises 95 Notes and References 100 Recursion and Induction 101 3.1 Introduction 102 3.2 Recursive Procedures 102 3.3. What Isa Proof? 108 3.4 Induction Proofs 111 3.5. Proving Correctness of Procedures 118Contents xvii Programs 309. Notes and References 309 7 Graphs and Graph Traversals 313 7.1 Introduction 314 7.2 Definitions and Representations 314 7.3 Traversing Graphs 328 7.4 Deplh-First Search on Directed Graphs 336 7.5. Strongly Connected Components of a Directed Graph 357 7.6 Depth-First Search on Undirected Graphs 364 7.7 Biconnected Components of an Undirected Graph 366 Exercises 375 Programs 384 Notes and References 385 8 Graph Optimization Problems and Greedy Algorithms 387 8.1 Introduction 388 8.2 Prim’s Minimum Spanning Tree Algorithm — 388 83. Single-Source Shortest Paths 403 84° Kruskal’s Minimum Spanning Tree Algorithm — 412 Exercises 416 Programs 421 Notes and References 422 : 9 Transitive Closure, All-Pairs Shortest Paths 425 9.1. Introduction 426 9.2 The Transitive Closure of a Binary Relation 426 9.3 Warshall’s Algorithm for Transitive Closure 430, 9.4 All-Pairs Shortest Paths in Graphs 433 9.5 Computing Transitive Closure by Matrix Operations 436 9.6 Multiplying Bit Mavrices—Kronrod’s Algorithm 439 * Exercises 446 Programs 449 Notes and References 449 10 Dynamic Programming 451 10.1 Introduction 452 10.2 Subproblem Graphs and Their Traversal 453 10.1. Multiplying a Sequence of Matrices 457 | |xvi Contents 3.6 Recurrence Equations 130 3.7 Recursion Trees 134 + Exercises. 141 Notes and References 146, 4 Sorting . 4.t Introduction £50 4.2 Insertion Sort 151 4.3. Divide and Conquer 157 44° Quicksort 159 4.5 Merging Sorted Sequences 171 46 Mergesort 174 4.7 Lower Bounds for Sorting by Comparison of Keys 178 48 Heapson. 182 4.9 Comparison of Four Sorting Algorithms 197 4.10 4u Shetisont 197 Radix Soning 201 Exercises 206 Programs 221 Notes and References 221 5 Selection and Adversary Arguments SA 5.2 53 54 +55 56 Introduction 224 Finding max and min 226 Finding the Second-Largest Key 229 The Selection Problem 233 A Lower Bound for Finding the Median 238 Designing Against an Adversary 240 Exercises 242 Notes and References 246 6 Dynamic Sets and Searching 61 62 63 64 65 66 +67 Introduction 250 Array Doubling 250 Amortized Time Analysis 251 Red-Black Trees 253 Hashing 275 Dyna Priority Queues with a Decrease Key Operation 295 Exercises 302 Equivalence Relations and Union-Find Programs 283 149 223 249Contents 466 . an 10.6 Developing a Dynamic Programming Algorithm — 474 Exercises 475 Programs 481 Noles and References 482 11 String Matching 483 HLA Introduction 484 11.2. A Suaightforward Solution 485 11.3 ‘The Kuuth-Mortis-Pratt Algorithm — 487 11.4 The Boyer-Moore Algorithm — 495 11.5. Approximate String Matching 504 Exercises 508 Programs — $12 Notes und References 512 ° 12 Polynomials and Matrices 515 12.1 Introduction 516 12,2. Evaluating Polynomial Functions 516 12.3 Vector and Matrix Multiplication 522 #124 The Fust Fourier Transform and Convolution 528 Exercises 542 Programs 546 Notes and References 546 13 NP-complete Problems 547 13.) Inweduetion 548 13.2 Pand NP $48 13.3 NP-Complete Problems 559 13.4 Approximation Algorithms 570 135 Bin Packing $72 13.6 The Knapsack and Subset Sum Problems $77 13.7. Graph Coloring $81 13.8 The Traveling Salesperson Problem 589 13.9 Computing with DNA 592 Exercises 600 Notes and References 60814 Parallel Algorithms 14.1 Introduction 612 14.2. Parullclism, the PRAM, and Other Models 14.3. Some Simple PRAM Algorithms 616 144° Handling Write Conflicts 622 145. Merging and Sorting 624 146 . Finding Connected Components 621 14.7 A Lower Bound for Adding » Integers Exercises 643 Notes and References 647 A Java Examples and Techniques A. Introduction 650 A2 AJavaMain Program 651 A3° ASimple Input Library 656 ‘A4 Documenting Java Classes 658 AS Generic Order and the “Comparuble” Interface 641 e 659 AG Subclasses Extend ihe Capability of Their Superclass. A.1 Copy via the “Cloneable” Interface Bibliography Index 667 663 Contents xix 611 649 669 679Analyzing Algorithms and Problems: Principles and Examples + 1.1 Introduction 1.2 Javaas an Algorithm Language 1.3. Mathematical Background 1.4 Analyzing Algorithms and Problems 1.5 Classifying Functions by Their Asymptotic Growth Rates 1.6 Searching an Ordered Array11 Chapter 1 Anafyzing Algorithms and Problems: Principles and Examples Introduction . To say that a problem is solvable algorithmically means, informally, that a computer program can be written that will produce the correct answer for any input éf we Let it run fong enough and allow it as much storage space as il needs. tn the 1930s, before the advent ‘of computers, mathemitickuns worked very actively to formalize and study the notion of , which was then interpreted informally to mean a clearly specified set of simple instructions to be followed to solve a problem or compute a funtion. Various Format models of computation were devised and investigated. Much of the emphasis in the ely work in this field, called compuiability theory, was on describing or churacterizing those problems that could be solved algorithmically and on exhibiting some problems that could not be, One of the inportant negative results, established by Alan Turing, was the proof ‘of the unsolvability of the “halting problem.” The halting problem is to determine whether an arbitrary given algorithm (or computer program) will eventually halt (rather than, say, gel into an infinite loop) white working on a given input. There cannot exist a computer program that solves this problem. Although computability theory has obvious and fundamental implications for com- puter science, the knowledge that @ problem can (heoretieally be solved on a computer is not sufticien|Jo tetl us whether it is practical to do so, For example, a perfect chess-playing program could be written. This would not be a very difficult tisk; there are only a finite umber of ways to arrange the chess pieces on the boatd, and under certain rules a game ust (erminate alter a finite number of moves, The program could consider each of the computer's possible moves, each of its opponemt’s possible responses, each of its possi- ble respouses to those moves, and so on until each sequence of possible moves reaches an cud, Then since it knows the ullimace result of euch move, the computer can choose the best ane, The mumber of distinet arrungemens of pieces on the boaed that itis reasonable to consider (much less the number of sequences of moves) is roughly 10 by sume esti- mates, A program that examined them all would take several thousand years to run. Thus such a program has net been sun. Numerous problems with practical appfications can be solved—that is, programs can be written for them—but the Gine and storage requirements are mach tgo great for these is to be of practical use. Clearly the time and space requirements of a progeam are portance, They have become. therefore, the subject of theoretical study in the area of computer science called comautational complexity. One branch of this study, which is not covered ia this book, is concerned with setting up a formal and somewhat abstract theory of the compleaity of computable functions, (Solving a problem is equivalent to computing a function from the set of inputs to the set of ouiputs.) Axioms for measures of complexity have been formulated: they are basic and genesal enowh so that either the number of instructions executed or the number of slorige bits used by a program can be taken as a complexity measure. Using these axjoms, we can prove the existence of Jy complex problems and of problems for which there is no best program, ch of computational complexity studied in this book is concerned with an- alyzing specific problems and specific algorithms. This book is intended to help readers build a repertoire of classic algorithms to solve common problems, some general design1.2 1.2 Java as an Algorithm Language techniques, tools and principles for analyzing algorithms and problems, and methods of proving correctness. We will present, study, and analyze algorithms to solve a variety of problems for which computer programs are frequently used. We will analyze the amount of time the algorithins take (o execute, and we will also often analyze the amount of space used by the algorithms. In the course of describing algorithms for a variety of problems, ‘we will see that several algorithm design techniques often prove useful. Thus we will pause now and then to talk about some general techniques, such as divide-and-vonquer, greedy algorithms, depth-first search, and dynamic programming, We will also study the com> putational complexity of the problems themselves, that is, the time and space inherently required (0 solve the problem no matter what algorithm is died. We will study the class of NP-complete problens—problems for which uo efficient algorithms are known—and consider some heuristies for getting useful results. We will also describe an approach tor solving these problems using DNA instead of electronic computers, Finally, we will intro- duce the subject of algorithms for pavallel computers. {n the following sections we outline the alzorithim language, review some background and tools that will be used throughout the book, and illustrate the main concepts involved mnalyzing aw algorithm. ° Java as an Algorithm Language We chose Java as the algorithm language for this book by bulancing several criteria. The algorithins should be easy to read, We want 10 focus on the strategy and techniques of an al~ gorithm, aot declarations and syntax details of concern to a compiler. The language should support data abstraction and problem decomposition, to make il easy to express algorithmic ideas cleurly, The language should provide a practical pathway to implementation. Itshould be widely avaitable and provide support for program development. Actually implementing and running algorithms can enhance the student's understanding greatly, and should wot (urn into a frustrating bute with the compiler and debugger. Finally, because this book is teaching algorithms, not 2 programming language, it should be reasonably easy to fate an algorithim to a variety of languages that reuders might wish to use, und special language features should be minimized. ‘Java showed up well by several of our criteria, although we would not clvine it is ideal. 1t supports data abstraction naturally. Itis type-safe, meaning that objects of one type cannot be used in operations intended lora different type; arbitrary type conversions (called casts”) are not permitted, either, There is an explicit bootean type. so if one types (the assigament operator) when “==" (the equality operutor) was intended, the compiler catches it Java does not permit pointer manipulations, which are a frequent source of obscure ‘errors; in fact, pointers are hidden from the programmer and handled automatically bebind the scenes, At run time, Java checks for ou-oF-range array subscripts, and other incon- sistencies that might be other sources of obscure errors. Ht performs “gurbage collection,” which means that it recycles the storage space of ubjects that are no longer referenced; this takes a big burden of space management off the programmer,1.2 Java as an Algorithm Language Int getMin(PriorityQ pq) tells us that getMin takes one parameter of type (or elass) Priority and returns type int. Java has a few primitive types and all remaining types are called classes. The primitive types are logical (boolean) and numerical (byte. char, short, int, tong, float, and double} types. All classes (nonprimitive types) in Java are reference classes, Behind the scenes, variables declared in classes are “pointors”; their values are adresses. Instances of classes are called objects, Declaring a variable does not create an object. Generally, objects are created with a “new” operator, which returns a reference to the new object. The data fields of an object are called instance fields in object-oriented terminology. The binary dot operator is used to access instance fields of an object. Example 1.1. Creating and accessing Java objects For this example, let’s assume that date information has the following nested logical structure: * year + number + isleap + month + day That is, using informal terminology. year is a compound attribute that consists of the boolean attribute isLeap and the integer attribute number, while month and day are simple integer attributes, To reflect this nested stracture, we have to define two classes in Java, one for the whole date and another for the year field. Assume we choose the names Date and Year, respectively, for these classes. Then we would declare number and isLeap as instance fields in the Year class ane declare year, month, and day as instance fictds in the Date class. Moreover, we would most likely define Year as an innes class of Date. ‘The syntax is shown i Figure 1.1 class Date { public Year year; public int month; public int day; public static class Year t public int number; public boolean isLeap; ? Figure 1.1 Java syntax for the Date class with an inner Year chissChapter 1 Analyzing Algor ms and Problems: Principles and Examples ‘On the downside, Java has many of the same terse, cryptic syntax featurewof C. The object structure may force inefficiencies in time and space. Many Java constructs require greater verbosity than other languages, stich as C, for instance. ‘Although Java has many specialized features, the algorithms presented in this book avoid most of them, in the interest of being language-incependent. In fact, some steps within an algorithm may be stated in pseudocode for easier readability. This section de- scribes a small subset of Java that we use for the book, and the pseudocode conventions that we use to improve readability of the algorithms, The Java-specific Appendix A gives some additional implementation details for readers who want (0 get a Java program running, but these dotails are not pertinent to understanding the main text. 1.2.1. A Usable Subset of fava A thorough acquaintance with Java is not important 10 understand the algorithms in this text. This section gives a brief overview of the Java features that do appear, for those readers who wish (o follow the implementation issues closely. In some cases we point out object- oriented features of Java that might be used, but which we avoid so that the text can be fairly language-independent: this is mainly for the benefit of readers who are familiar with some other object-oriented language, such as C++, but who are not completely familiar with Java, A sample Java “main program” appears in Appendix A. Many books are available for in-depth coverage of the Java language, Readers who are well acquainted with Java will undoubtedly notice many instances in which some nice Java feature could have been used. However, the concepts behind the algorithms do net require any special features, and we want these concepts to be easy to grasp and apply in a variety of languages, so we leave it to the readers, once they have Brasped the concepts, to tailor the implementations to th Tanguage. Readers familiar with C syntax will recognize many similarities in Java syntax: Blocks are delimited by curly braces, “{" and °°; square brackets, “f” and “J”, enclose array indexes. As in C and C+. a two-dimensional array is reatly a one-dimensional array ‘whose clements are themselves one-dimensional arrays, so (wo pairs of square brackets are needed to access an element, as in “matrixil[}". Opermors "==", " are the keyboard versions of the mathematical relational operates "= examples use the “#4” and“=~" operators 0 iacemen an deren but never use them embedded in other expressions. There are also the operators "+ adopted from C, For example, ptaq, /+Add qtop. +/ Y= x; // Subtract x from y. As just illustrated, comuments extend from °//™ to end-of-line, or from “/#" to "#/". as in CH. ‘ Funezion headers normally look the same in Java as in C. The function header specifies the parameter ixpe signature in parentheses after the Function name; it specifies the rerun Axpe before the function name, The combination of return type and parameter type signature is called the function's fill type signature, or protonype. ThusChapter 1 Analyzing Algorithms and Problems: Principles and Examples Without the public keyword, the instance fields woutd not be accessibte outside of the Date and Year classes; for simplicity, we make them public here. The reason for declaring the inner class, Year, (o be static is so we can create an instance of Year that is not associated with any particular Date object. All inner classes will be static in this book. Suppose we have created a Date object that is referenced by variable dueDate, To access the instance fietd year in this object, the dot operator is used, us in “dueDate.year.” It the instance field is in a cluss (as opposed to being in a primitive type), then further dot operators access its instance fields, as in “dueDate.year.isLeap” The assignment statement copies only the reference, or address, of an object in a " ure usually written, instead of their keyboard versions, Relational operators are used on types where the meaning is clear, such as String, even though this would be invalid syntax in Java, 6. Keywords, which are either reserved words or stundard parts of Java, are set in this font: int, String. Comments are set in this font. Code statements and program variable names are set in this font. However, pseudocode statements are set in the regular font of the text, like this sentence. Occasional departures from this scheme occur when we are making & specific poimt about the Java language. Mathematical Background We use a variety of mathematical concepts, tools, and techniques in this book. Most should already be familiar to you, although a few might be new. This section collects therm 10 provide a ready reference, as well as a brief review. Proof concepts are covered in greuter depth in Chapter 3. 1.3.1 Sets, Tuples, and Relations is section provides informal definitions and a few elementary properties of se selaied concepts, A set is a collection of distinel elements that we wish to real ds a single object. Usually the elements are of the sume “type” a and have some uddilional common W1.3 Mathematical Background follows: The first element of the sequence can he chosen from any element of S, so there are n choices. Then the second element of the sequence can be chosen fram any remaining element of 5, so there are (11 — 1) choices for this, and so on until k elements are chosen, (if k > 1 it is impossible to make K distinct choices, so the result is 0.) Therefore there are n(n — 1) +++ (n= & + 1) distinet sequences of distinct elements. But we saw that a specific set of & elements can he represented us k! sequences. So the number of distinct subsets of k, drawn from a set of n, is. m\ neko )asabapachs Coys ( Gane Since every subset must have some size from 0 through 7, we arrive at the identity zy () a2. 1.2) =o forn> k20. (1) Tuples and the Cross Product A tuple is a finite sequence whose elements often do not have the same type. For example, in a two-dimensional plane, a point can be represented by the ordered pair (x, y). If itis a geometric plane, x and y are both “length” But if it is a plot of running time vs. problem size, then ¥ might be seconds and x might be an integer. Short tuples have special names: pair, triple, quadruple. quintuple, and so on, In the context of “tuple” these are unclerstood to be ordered; in other contexts “pai” might mean “set of two” instead of “sequence of two," and so on. A k-uple is a mple of & elemen ‘The cmss product of two sets, say S and 7’ is the set of pairs that can be formed by choosing an element of $ as te first element of the tuple and an clement of F as the second, In mathematical notation we have SxTHiG lees yeT}. 0.3) Therefore |S x T] = [S47 1. It often happens that 5 and 7 are the same set. but this is not necessary. We can define the iterated cross product to produce longer tuples. For example, ST x U isthe set of all triples formed hy taking aa element of S, followed by an clement of 7, followed by an clement of U. Relations and Functions A relation is sinaply some subset of a (possibly iterated) cross product. This subset might be finite or infinite, and can be empty or the entire cross product. The most important case is a binary relation, which is simply some subset of a simple cross product. We are all familiar with many examples of binary relations, such as “less than” on the reals, Letting R denote the set of all reals, the “less than” relation can be defined formally as {Oxy} fr ER. y € Ry x < yl). As we see, this is a subset of R x R. As another example. if P isthe set of all people, then P x P is the sct of all pairs of people, We can define “parent of as (x. y) such that x is a parent of y, “ancestor of” as (4, ») stich that x is an ancestor of y, and these are subsets of P x P, . 13Chapter 1 Analyzing Algorithens and Problems: Principles and Examples properties that make it useful to think of them as one object, The notation ¢ ¢ S is read “element ¢ is a member of set 5” of, briefly, “e is in $.” Notice that ¢ and S'are different types in this case. Por example, if ¢ is an integer, S is.a ser of integers, which is different from being an integer A particular sot is defined by listing or describing its elements between a pair of curty braces. Examples of this notation are Si = {a,b,c}, Sa= {1 [x is an integer power of 2}, Spe (hens). ‘The expression for Sp is read “the set of all clements x such that x is an integer power of 2" The |" symbol is read “such that” in this context. Sometimes a coton (*:") is used in this place. The ellipsis“, .." may be used when the implicit elements are clear, Tf all clements of one set, S), are also in another set, S>, then 5; is said to be a subset Of Sz and S3 is said to be a superset of S;. The notations are Si C Sp and Sz D S;. To denote that Sy is a subset of S2 and is not equal to Sp, we write §| C Sz oF $2 > S). Its important not to confuse “e” with "C." The former means “is an element in” and the latter means “is a set of elements contained within.” The enppry set, denoted by @, has no elements, so it is a subset of every set. A set has no inherent order. Thus, in the above examples, 5; could have been defined as {b,¢, a}4ind $3 could have been defined as {é f 1 J and x > 0, log, x (read “log to the base b of x”) is that real number L such that bE = x; that is, log, x is the power to which & must be raised to getx. ‘The following properties of logarithms foltow easily from the definition. Lemma 1. Let.x and y be arbitrary positive real numbers, let a be any real number, and et > Lande > 1 be real numbers. logy is a strictly increasing function, that is, if > y, then tog, x > log, y. log, is a one-to-one function, that is, if log, x = log, y, then x = y, log, 1 =0. log, bf =a. log, (xy) = logy x + log, ». logy (x#) =a log, x. Hat a yl, PIAA ae ‘To convert from one base to another: tog,.x = (log, x)/(logy ¢). ve the log to the base 2 is used most often in computational complexity, there is a special notatis “tgs that is, 1g x = logy x. The natural logarithm (log to the basc ¢) is denoted by “In”; that is, Inx = log. When log() is used without any base being, mentioned, it means the statement is true for any base, Sometimes the logarithm function is applied lo itself. The notation Ig Ig(x) means Ig(le(z)). The notation Ig"??(x) means p applications, so Ig'?"(x) is the same as Ig le(x). Note that 1g)(65536) =: 2, which is quite different from (1g(65536))? = 4096. 151.3 Mathematical Background with “heads” facing up or with “tails” facing up. We let s1 = ‘heads’ and s) = ‘tails’ and assign Pr(s\) = 1/2 and Pr(sy) = 1/2. (If someone objects because the coin could land on its edge, we may let 5 = ‘edge’ and define Pr(sy) = 0. However, with a finite number of events, an event of probability zero can be ignored, so such elemtentary events are not usu- ally defined.) If a six-sided die is thrown, there are six poss 1 = “the die tands with side number { facing up." and Pr(s,) = 1/6, In general, if there are & possible outcomes and each is considered equally likely, then we let Pr(si) = 1/k for each i, Often, there is no reason to assume all outcomes are equally likely; primarily, this is an assumption used in examples or used because there.is no data to support a better assumption, If the experiment involves several objects, then an elementary event must take into 2ocount what is observed about all of them. For example, if two dice, A and B, are thrown, then the event “A lands with side 1 facing up” is not an elementary event because there are several outcomes associated with B. In this case, the elementary events would be 5; = “die A lands with side j facing up and die B lands with side j facing wp.” for | 3.32. The derivative. of In¢x) is 1/x. Using part 8 of Lemma 11, the derivative of Ig(x) is Ig(e)/x. Permutations A permutation of 1 distinct objects is a sequence that contains each object once. Let $= (51,52, +Sn}- Note that the elements of § are ordered by their indexes; that is, 51 is the first element, «2 the second, and so on. A permutation of 5 is a one-to-one function a from the set {1, 2...) onto itself. We think of = as rearranging S by moving the ith element, sifto the 2()th position. We may describe 2 simply by listing its values, that is, (r(1), (2). .... (a). For example, for n = 5, m = (4, 3, 1, 5,2) rearranges the elements of $ as follows: $3. $5, $2.51. 54. The number of permutations of n distinct objects is 1!, To see this, observe that the first element can be moved to any of the m positions; then that position is filled and the second element can be moved to any of the — J remaining positions; the third element can be moved to any of the remaining m — 2 positions, and so on, So the total number of possible rearrangements is n x (n ~ 1) x (1 =2) x... x2 x Laat Probability Suppose that in a given situation an event, or experiment, may have any one, and only one, of & outcomes, 51,52, »St- These outcomes are called elementary events, The set of all elementary events is called the universe and is denoted U. With Each outcome 5; we associate a real number Pr(s;), called the probability of s;, such that O 0, log(x) for x > 0, and ¢. Less familiar monotonic functions are [x] and [x], showing that monotonic functions need not be continuous. An antimonotonic example is 1/x for x > 0. Definition 1.8 Linear interpolation function ‘The linear interpolation of a given function f(x) between two points « and v, u < v, is the function defined by 21.3 Mathematical Background 2. A function f(a) defined on imegers is convex if and onty if, for any n,0 +1, n+2, Slat SEL) + fOr4D). In words, (7 + 1) is at most the average of f(n) and f(a +2). Lemma 1.4 summarizes several useful properties of monotonicity and convexity. It states that functions defined only on the integers can be extended to the reals by linear interpolation, preserving properties of monotonicity and convexily. Also, some properties involving derivatives are stated. The proofs are in Exercises 117 through 1,19. Lemma 14 1. Let f(n) be defined only on integers. Let f*(x) be the extension of f to the reals by linear interpolation between consecutive integers (see Figure 1.3b). a. f (2) is monotonic if and only if f*(x) is monotonic. b. f(2) is convex if and only if f(x) is convex, 2. I the first derivative of (x) exists and is nonnegative, then f(x} is monotonic. iW the first derivative of f(x) exists and is monotonic, then f(x) is convex. 4, If the second derivative of f (x) exists and is nonnegative, then f(x) is convex. (This follows from parts 2 and 3.) 0 e Summations Using Integration Several summations that arise often in the analysis of algorithms can be approximated (or bounded from above or below) using integration, First, let us review some useful integration formulas: fe drs peg [anton 1). , : \ (LIS) ‘ etl nen) = tH f at inte) d= pnt Inn) ay if f(x) is monotonic (or nondecreasing), then b +t S@drsT Os} feeds (1.16) ina Similarly, if f(x) is antimonotonic (or nonincrensing), then aa b b f fdx s Pi) sf fous. (ain 2524 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples fo (a) Linear interpolation {b) Extension of f(1) to f*(x) Figure 1.3 Illustrations for convexity discussion: The function f is different in parts (a) and {b). In part (b), f*(2) is conver. @- Df) +O—wWfe) Lyal) = wow TIW+G = 1 LO=LO = py yn fP=, iy vom uae that is, the straight-line segment joining f(u) and f(v) (see Figure 1.3a), Definition 1.9 Convex functions A function f(x) is said to be convex if forall u mga-alge. Since Ige < 1.443, Dileiz aign — 1.4430 (1.18) as of the previous example, bul with more precise mathermathies, it is possible to derive Stirting's formula giving bounds for nt: (2) vam O0 (1.20) then ASC then A+CS84D then aA sab1.3 Mathematical Background bbe more like the logical form, and then ask yourself if it mewns the same thing in natural language. For example, “Any person must hreathe to live” might be the sentence you start with, Possible rephrasings are "For all peaple x must breathe to live and “For some person x, x must breathe to five.” Which means the same as the original sentence? Negating a Quantified Statement, Counterexamples What is necessary (0 prove that a general statement, say ¥e(A(x) => B(x)), is false? We can use the foregoing identities to clarify the goal. The first thing to realize is that it is nov necessary to prove Wx(A(x) = ~B(x)). This is tod strong a statement, The nogation of ¥r(ACx) => BCD) is ~Cex(ACe) => B(x})), which can be put throngh a series of transformations: A(¥x(A(X) => BLX))) is logically equivalent to 3x-(AC) = BEX) logically equivalent to Ax(—AGr) v BOX) (1.26) logically equivalent to Ax(AGx) A 7BQ))- Tn words, if we can exhibit some object x for which A(x) is true and B (3) is false, then we have proven that ¥x(A(x) => B(x) is false, Sucl an object (x) is called a counterexample. Contrapositives When trying to prove a statement, it is offen convenient to manipttate it into a logically equivalent form. One such form is the comrapositive, The contrapasitive of A => B is (-B) =+ (7A), Equation (1.21) allows us to verity that the eonrapasicie of an implication is true exactly when the implication itself is true: Axe Bis logically equivalent to (4B) => (>A). 27 Sometimes, proving the contrapositive of a statement is called “proof hy contra but “proof by contraposition” is a more acctirate description, The genuine “proof by con- tradiction” is described next. Proof by Contradiction Suppose the goal is to prove a statement of the form A => B.A genuine praf by eantradic- tion adlds an adklitional hypothesis of +B. and then proves B itself. That is, (A AB) => B is the Full statement that is proved. The following identity justifies this method: A= B- islogicully equivatentto (AA7B) = B. 1.28) ‘A genuine proof by contradiction is rare in algorithm analysis, However, Exercise 1.24 calls for onc. Most so-called proofs by contradiction are actually proofs by contraposition. Rules of Inference So far we have seen numerous pairs of logically equivalent statements. oF logical identities: ‘One statement is true if and only if the second statement is true. Klentities are “reversible: 2928 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples 1.3.3 Elements of Logic . Logic is a system for formalizing natural language statements so that we can reason more accurately. The simplest statements are called atomic formulas, More complex statements can be built up through the use of fogieal connectives. Examples of atomic formulas are 4 > 32° "4.2 is am integer,” and “x + 1 > x.” Notice that a logical statement need not be truc. The objective of a proof is to show that a logical statement is true. The mast familiar logicat connectives are “A” (and). "V" (or), and "+" (not), which are also called Boolean operators. The truth value of a complex statement is derived fram the truth values of its atomic formulas, according to rules for the connectives. Let A and B bbe logical statements. Then, 1, AA Bis true if and only if A is true and & is true: 2. AY Bis true if and only if A is true of B is true, or both, 3, 7A is true if and only if A is false. Another important connective for reasoning is called “implies,” which we denote with the symbol “=>”, (The symbol “+” is also seen.) The statement A => B is read os “A implics BY or “if A then B.” (Notice that this statement has no “else” clause.) The “implies” oparator can be represented with a combination of other operators, according to the following identity: A => Bis logically equivalentto. —A vB. (121) This can be verified by checking alt combinations of truth assignments to A and B. Another useful set of identities are called DeMorgan’s laws: (AA B) is logically equivalem to AAV —B, (122) “(AV B) is logically equivalent to A AB. (1.23) rs. Quan Another important kind of logical connective is the qumrifier. The symbol Vs is called the universal quantifier and is read “for all x." while the symbol Ax is called the existential quantifier and is read “there exists x." These connectives can be applied to statements that ‘contain the variable x, The statement ¥x P(x) is true if and only if P(x) is true for ali x. The statement 3x P(x) is tre if and only if P(x) is true for some value of x. Most frequently. a universally quantified statement is conditional: Wx (A(x) =» B(x)), This ean be read "For all x such that A(x) holds, B(x) holds.” ‘Quantified statements obey a variation on DeMorgan's laws: VrA(x) is logically equivalent to -3x(=A(x)), (1.24) 3xA(x) is logically equivalent i “Va(A(x))- (1.25) Sometimes the translation from natural Language into a quantified statement is trouble- some. People don’t speak in the stilted language of logic, usually. We need to realize that “for any 2” usually means “for all x," although “any” and “some” are often interchangeable in normal speech. The best guideline is to try rephrasing a sentence in matural language to30 Chapter 1 Analyzing Algorithms and Problems: Principles and Examples Most proofs ive directed at “irreversible” combinations of statements, however The com plete statement 10 be proved is of the form “if kxpotteses, then conelasion.” The reversal, “it conclusion, then hypotheses” is often of true. Logical identities are not flexible enough o prove such “if-then” statements, In these situations, we need rales of inference. A rule of inference is a generat pattern that allows us to draw some new conclusion from a set of given statements, It can be stated, “If we know By, ..., Be, then we can conclude C2" where Bi... Bas and C ure logical statements in their own right, Heve are a few well-known rules: Mowe know then we can conclude B and BC c (1.29) A+B und BSC Ave (1.30) Bac md BSC OC a3) Some of the these rules are known by their Greek of Latin names, Equation (1.29) is modus is sy/fogisin, and Equation (1.31) is the rule of cases. These rules ‘are not independent; in Exercise 1.21 you will prove the rule of cases using other rules of inference und logical identities. , Analyzing Algorithms and Problems We analyze algorithms with the intention of improving them, if possible, and for choosing mong several available fora problem. We will use the following criteria: Convetness Amount of work done Amount of space used Simplicity, etarity Optimality yRepe We will discuss each of these criteria at length and give several examples of their appli- cation, When considering the optimatity of algorithm uroduce techniques for estublishing lower bounds on the complexity of problems, 1.4.1 Correctness ‘There are three nxajor steps involved in establishing the correctness of an algorithm, First, belive we can even attempt to determine whether an ulgorithm is correct, we must have a clear understanding of what “correct” means, We need a precise statement about the characteristics of the inpuis it is expected to work on (called the preconditions), and what result it is to produce for each input (calted the pastcenditions). Then we can try to prove statements about the relitionsbips between the input and the output, that is, that if the. preconditions are sitisfied, the postconditions will be true when the algorithm terminates. “There are two aspects 40 an algorithm: the sulution method! and the sequence of instructions for carrying it out, that is, its implementation, Establishing the correctness of1.4 Analyzing Algorithms and Problems the method and/or formulas used may be easy or niay require a long sequence of lemmas and theorems about the objects on which the algorithm works (e.g., graphs, permutations, matrices). For example, the vatidity of the Gauss elimination method for solving systems of linear equations depends on a number of theorems in linear algebra. Some of the methods used in algorithms in this book are not obviously correet; they must be justified by theorems. Once the method is established, we implement it in a program. Ifan algorithm is fairly short and straightforward, we generally use some informal means of convincing ourselves that the various parts do what we expect thei to do. We may-check some details carefully (e.g. initial and final values of loop counters), and hand-simulate the algorithm on a few small examples. None of this proves that itis correct, but informal techniques may suffice for small programs, More formal techniques, such as loop invariants, may be used to verify correctness of parts of programs. Section 3.3 expands upon this topic. ‘Most progranis written outside of classes are very large and very complex. To prove, the correctness of a large program, we can ity (0 break the program down into smaller modules; show that, if all of the smaller modules do their jobs properly, then the whole program is correct; and then prove that each of the modules is correct. This task is made easier if (it may be more accurate 10 say, “This task is possible only if”) algorithons and programs are written in modules that are largely independent and can be verified separately. This is one of the many strong arguments for structured, modular programming, Most of the algorithms presented in this book are the small segments from which large programs ‘are built, $0 we will not deal with the difficulties of proving the correctness of very long algorithms or programs. We will not always do formal proofs of correctness in this book, though we will give arguments or explanations to justify complex or tricky parts of algorithms. Correctness can be proved, though indeed for tong and complex programs it is a formidable task. In Chapter 3 we will introduce some techniques to help make proofs more manageable. 1.4.2. Amount of Work Done How shall we measure the amount of work dane by an algorithm? The measure we chaose should aid in comparing two algorithms for the sume problem so that we can determine whether one is more efficient than the other. It would be handy if our measure of work ‘gave some indication of how the actual execution times of the two algorithms compare, but we will not use execution time as a measuce of work for a number of reasons, First, of course, it vasies with the computer used, and we don’t want to develop a theory for ‘one particular computer. We may instead count alt the instructions or statements executed by a program, but this'measure still has several of the other faults of execution tinne. ft is highly dependent on the programming language used and on the programmer's style. It would also require that we spend time and effort writing and debugging programs for each algorithm to be studied, We want a measure of work that tells us something about the eificiency of the method used by an algorithm independent of not only the computer, programming Janguage, and programmer, but also of the many implementation details, overhead (or “bookkeeping” operations) such as incrementing loop indexes, computing 31