This action might not be possible to undo. Are you sure you want to continue?
Michelle Kuttel & Sonia Berman
(mkuttel | email@example.com)
Analysis of Algorithms
How good is the algorithm?
Correctness Time efficiency Space efficiency Simplicity
Does there exist a better algorithm?
Lower bounds Optimality
Analysis of Algorithms
Correctness (Is it guaranteed to produce a correct correspondence between inputs and outputs?) Time efficiency (How fast does it run?) Space efficiency (How much extra space does it require?) Optimality (Is this provably the best possible solution?)
Theoretical analysis Empirical analysis Visualisation
Can analyse efficiency with respect to either:
• how fast • easier to improve I often say when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge of it is of a meagre and unsatisfactory kind: it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be. .
• how big • less important than before
can be studied in precise quantitative terms
- Lord Kelvin (1824-1907)
Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size Basic operation: the operation that contributes most towards the running time of the algorithm input size Theoretical analysis of time efficiency T(n) ≈ copC(n) running time execution time for basic operation number of times basic operation is executed 5 .
Classical Problems Problem Search for key in list of n items Matrix Multiplication Compute an Graph problem Input size measure Number of items in list n Dimensions of matrices n #vertices and/or edges Basic operation Key comparison Floating point multiplication Floating point multiplication Visiting a vertex or traversing an edge 6 .
Objectives To outline a general analytic framework To introduce the conceptual tools of: Asymptotic Notations Base Efficiency Classes To cover Techniques for the Analysis of Algorithms: Mathematical (non-recursive and recursive) Empirical Visualisation 7 .
Best-. as need to make assumptions about the possible inputs of size n 8 .and Worst-cases For some algorithms efficiency depends on type of input: Worst case: • W(n) – max over inputs of size n Best case: • B(n) – min over inputs of size n Average case: • A(n) – “avg” over inputs of size n • Number of times the basic operation will be executed on typical input • NOT the average of worst and best case • INSTEAD expected number of ops considered as a random variable under some assumption about the probability distribution of all possible inputs of size n • most difficult to estimate. Average.
Tarjan . the total time for a sequence n of such operations is always significantly better than the worst-case efficiency multiplied by n Amortized efficiency: amortized time defined as the time of an operation averaged over a worst-case sequence of operations 9 . E.recipient of 1986 Turing Award [TARJAN87] Though in some situations a single operation can be expensive.Amortized Efficiency From a data structure perspective. the total time of a sequence of operations is important Real-time apps are an exception R.
Exercise: Sequential search Problem: Given a list of n elements and a search key K. if any exists Algorithm: Scan the list and compare successive elements with K until either a matching element is found (successful search) or the list is exhausted (unsuccessful search) Calculate the: Worst case? Best case? Average case? 10 . find an element equal to K.
..g.Types of Formulas for Operation Counts Exact formula e. C(n) ≈ cn2 11 .g. C(n) ≈ 0. C(n) = n(n-1)/2 Formula indicating order of growth with specific multiplicative constant e.5 n2 Formula indicating order of growth with unknown multiplicative constant e..g.
Order of Growth Most important: Order of growth within a constant multiple as n→∞ Example: How much faster will algorithm run on a computer that is twice as fast? How much longer does it take to solve problem of double input size? 12 .
Asymptotic Notations A way of comparing functions that ignores constant factors and small input sizes O(g(n)): Class of functions f(n) that grow no faster than g(n) f(n) ≤ c g(n) for all n ≥ n0 Ω(g(n)): Class of functions f(n) that grow at least as fast as g(n) f(n) ≥ c g(n) for all n ≥ n0 θ(g(n)): Class of functions f(n) that grow at the same rate as g(n) c2 g(n) ≤ f(n) ≤ c1 g(n) for all n ≥ n0 13 .
Big-Oh: t(n) ∈ O(g(n)) 14 .
Big-Omega: t(n) ∈ Ω(g(n)) 15 .
Big-Theta: t(n) ∈ θ(g(n)) 16 .
Establishing Order: Using Definitions f(n) is O(g(n)) if growth of f(n) ≤ growth of g (n) (within constant multiple) There exists a positive constant c and nonnegative integer n0 such that f(n) ≤ c g(n) for every n ≥ n0 This needs to be mathematically provable Examples: 10n is O(2n2) because 10n < 2n2 for n > 5 5n+20 is O(10n) because 5n+20 < 10n for n > 4 17 .
vs.Establishing Order: Using Limits limn→∞ t(n)/g(n) = 0 implies order of t(n) < order of g(n) c implies order of t(n) = order of g(n) ∞ implies order of t(n) > order of g(n) Example: 10n vs. n2 logc n 18 . 2n2 limn→∞ 10n/2n2 = 5 limn→∞ 1/n = 0 Exercises: n(n+1)/2 logb n vs.
order log n < order n (actually little-o) 19 .L’Hôpital’s rule If limn→∞ f(n) = limn → ∞ g(n) = ∞ and the derivatives f '. g' exist. n limn→∞ log n / n = limn→∞ (1/n)/1 = 0 So. Then limn→∞ f(n) / g(n) = limn → ∞ f '(n) / g'(n) Example: log n vs.
Basic Asymptotic Efficiency Classes 1 log n n n log n n2 n3 2n n! constant logarithmic linear n log n quadratic cubic exponential factorial Outside best-case. quicksort Typically two embedded loops Typically three embedded loops Algorithms that generate all subsets of an n-element list Algorithms that generate all permutations of an n-element list 20 .. e. few examples Algorithms that decrease by a constant Algorithms that scan an n-sized list Algorithms that divide and conquer.g.
Principal change in the second edition is the chapter on iterative improvement will do the simplex method also included more puzzles 21 21 .
Objectives To outline a general analytic framework To introduce the conceptual tools of: Asymptotic Notations Base Efficiency Classes To cover Techniques for the Analysis of Algorithms: Mathematical (non-recursive and recursive) Empirical Visualisation 22 .
Applying the general framework to analyse the efficiency of non-recursive algorithms Decide on parameter n indicating input size Identify algorithm’s basic operation Determine worst. 5. average. 2. 4. and best case for input of size n Set up summation for C(n) reflecting algorithm’s loop structure Simplify summation using standard formulas 23 .Analysing Non-recursive Algorithms 1. 3.
.. this is ignoring estimates of the probability of a successful search. 24 11 24 . However. which will on average look at half the list..Sequential Search: Average Efficiency Previously.. assumed a successful search. as we will see..
.#comparisons is n Cave(n)=[1.(1-p) =p/n[1+2+..p/n] + n.p/n+...+n.+n] +n(1-p) =p/n.p/n+2. n(n+1)/2 +n(1-p) = p(n+1)/2 +n(1-p) 25 25 . number of comparisons is i unsuccessful search: probability (1-p).Sequential Search: Average Efficiency Assumptions: probability of a successful search = p probability of first match occurring at position i is the same for every i successful search: probability of first match occurring at ith position of the list is p/n for every i.
Will it’s efficiency differ from the efficiency of the classic sequential search? (Aside: Please DO the questions .1 (3) Consider a variation of the sequential search. that scans a list to return the number of occurrences of a given search key in the list.Question 2.it’s the only way to test whether you’re learning anything) 26 26 .
Examples: Analysing nonrecursive algorithms Matrix multiplication Multiply two square matrices of order n using dot product of rows and columns Selection sort Find smallest element in remainder of list and swap with current element Insertion sort Assume sub-list is sorted and insert current element Mystery Algorithm Exercise 27 .
Best case = worst case = average case 28 .Matrix Multiplication = 1. n = matrix order 2. Basic op = multiplication or addition 3.
Selection Sort - n = number of list elements basic op = comparison best case = worst case = average case 29 .
Insertion Sort - n = number of list elements basic op = comparision best case != worst case != average case best case: A[j] > v executed only once on each iteration 30 .
. j] ≠ A[j. i] return false return true What does this algorithm compute? What is its basic operation? What is its efficiency class? Suggest an improvement and analyse your improvement 31 ..n-1.Exercise: Mystery Algorithm // input: a matrix A[0. 0.n-1] of real numbers for i =0 to n-1 do for j = i to n-1 do if A[i.
Analysing Recursive Algorithms 32 32 .
Puzzle 2.3 (11): von Neumann Neighbourhood How many one-by-one squares are generated by the algorithm that starts with a single square and on each of its n iterations adds squares all around the outside? von Neumann neighborhoods for ranges r==0. 33 33 . 1. 2. and 3 are illustrated above.
Recap: Basic asymptotic efficiency classes? 34 34 .
Factorial: a Recursive Function Definition: n ! = 1*2*…*(n-1)*n Recursive form for n!: F(n) = F(n-1) * n. F(0) = 1 // F(n): calculate the factorial n! of a number n F(n): if n = 0 f←1 else f ← F(n-1) * n return f Need to find the number of multiplications M(n) required to compute n! 35 .
no mults when n = 0] Must differentiate the recurrence relation for the factorial calculation from the recurrence relation for the number of basic operations 36 .. M(n) = M(n-1) + 1 for n > 0 Initial Condition: The terminating step E.. M(0) = 0 [call stops when n = 0.g.g.Recurrence Relations Definition: An equation or inequality that describes a function in terms of its value on smaller inputs Recurrence: The recursive step E.
Recurrence Solution for n! Need an analytic solution for M(n) Solved by Method of Backward Substitution: M(n) = M(n-1) + 1 Substitute M(n-1) = M(n-2) + 1 ⇒ M(n) = [M(n-2) + 1] +1 = M(n-2) + 2 Substitute M(n-2) = M(n-3) + 1 ⇒ M(n) = [M(n-3) + 1] + 2 = M(n-3) + 3 Pattern: Ultimately: M(n) = M(n-i) + i M(n) = M(n-n)+n = M(0) + n = n 37 .
average. count recursive calls OR 5. If basic operation can vary based on input. worst. Decide on parameter n indicating input size 2. (a) Solve the recurrence to obtain a closed form (b) ascertain the order of magnitude of the solution (see Appendix B) 38 . and best case must be investigated separately 4. Identify algorithm’s basic operation 3. (a) Set up a recurrence relation and initial condition(s) for C(n)-the number of times the basic operation will be executed for an input of size n OR (b) Alternatively.Plan for Analysing Recursive Algorithms 1.
39 39 . (using Peg 1 as holder).Towers of Hanoi Recursive solution: to move n>1 disks from Peg 1 to Peg 3 (using Peg 2 as holder): • first move n-1 disks from Peg 1 to Peg 2 (using Peg 3 as holder) • then move largest directly to Peg 3 • then move n-1 from Peg 2 to Peg 3.
..Recursive Solution for Towers of Hanoi basic operation: moving a disk recurrence relation: M(n)=M(n-1)+ 1 + M(n-1) for n >1 M(1) = 1 substitute in M(n-1) = 2M(n-2)+1 .. M(n)= 2iM(n-i)+2i-1 substitute in M(1)=1 (i=n-1) M(n)= 2n-1 40 40 ..
Tree of recursive calls When algorithm makes more than a single call to itself. useful to construct a tree of recursive calls can count the number of nodes in the tree to get the total number of recursive calls 41 41 .
an algorithm is a step-by-step method for doing a job. This formulaic style of thinking can itself be a useful tool for businesses. As the amount of data on everything from shopping habits to media consumption increases and as customers choose more personalisation. Buy a copy and they help a logistics ﬁrm to decide on the best delivery route. Algorithms can take many forms. of interest only to dome-headed mathematicians. The speed and processing power of computers mean that algorithms can execute tasks with blinding speed using vast amounts of data. From analysing credit-card transactions to deciding how to stack supermarket shelves.” 42 explains Andrew Herbert. Britain. 42 . Algorithms can. But computers have made algorithms far more valuable to companies. who runs Microsoft Research in Cambridge.Algorithms Business by numbers Sep 13th 2007 From The Economist print edition Consumers and companies increasingly depend on a hidden mathematical world Illustration by Gillian Blease ALGORITHMS sound scary. boss of Autonomy. a computing ﬁrm that uses algorithms to make sense of unstructured data. Ring to check your order's progress and more algorithms spring into action to determine the quickest connection to and through a call-centre. In fact they have become the instruction manuals for a host of routine consumer transactions. algorithms now underpin a large amount of everyday life. At its core. “A computer program is a written encoding of an algorithm. These can be prosaic—a recipe is an algorithm for preparing a meal—or they can be anything but: the decision-tree posters that hang on hospital walls and which help doctors work out what is wrong with a patient from his symptoms are called medical algorithms. algorithms will only become more important. much like the rigour of good project-management. Browse for a book on Amazon.” says Mike Lynch. “No human being can work fast enough to process all the data available at a certain scale.com and algorithms generate recommendations for other titles to buy. Their pervasiveness reﬂects the application of novel computing power to the age-old complexities of business.
Iterative Methods for Solving Recurrences Method of Forward Substitution: Starting from the initial condition generate the first few terms Look for a pattern expressible as a closed formula Check validity by direct substitution or induction Limited because pattern is hard to spot Method of Backward Substitution: Express x(n-1) successively as a function of x(n-2)..i = base condition Surprisingly successful See Appendix B for more. x(n-3). 43 .. … Derive x(n) as a function of x(n-i) Substitute n .
5.Example: Fibonacci Numbers The Fibonacci sequence: 0. 21. 13. 34. … Describes the growth pattern of Rabbits. 1. just ask Australia! Fibonacci recurrence: F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1 44 . 2. Which is exponential. 8. 1. 3.
assuming an efficient way of computing matrix 45 .Computing Fibonacci Numbers Algorithm Alternatives: Definition-based recursive algorithm Nonrecursive brute-force algorithm Explicit formula algorithm: F(n) = 1/√5 φn where φ is the golden ratio (1 + √5) / 2 Logarithmic algorithm based on formula: F(n-1) F(n) = 0 1 1 1 n F(n) F(n+1) powers for n≥1.
Empirical Analysis of Time Efficiency Sometimes a mathematical analysis is difficult for even simple algorithms (limited applicability) Alternative is to measure algorithm execution PLAN: 1. hypothesising the efficiency class? 2.. comparing efficiency of different algorithms or implementations. Understand the experiment’s purpose Are we checking the accuracy of a theoretical result.g. milliseconds) OR Count actual number of basic operations 46 . Decide on an efficiency measure Use physical units of time (e.
Patterned inputs or a Combination Certain problems already have benchmark inputs 4. Generate a Sample Set of Inputs 6. Decide on Characteristics of the Input Sample Can choose Pseudorandom inputs.Plan for Empirical Analysis 3. Implement the Algorithm 5. Run the Algorithm and Record the Empirical Results 7. Analyze the Data Regression Analysis is often used to fit a function to the scatterplot of results 47 .
odd and even disks in Towers of Hanoi Attributes of Information Visualisation apply overview. 48 .Algorithm Visualisation Definition: Algorithm Visualisation seeks to convey useful algorithm information through the use of images Flavours: Static algorithm visualisation Algorithm animation Some new insights: e.g. detail on demand.. zoom and filter.
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue listening from where you left off, or restart the preview.