6 views

Uploaded by m1u2n3n4i5

asymptotic notations

asymptotic notations

Attribution Non-Commercial (BY-NC)

- Java Source Code
- GATE+TIFR April 2016
- Relational Algebra & Syncsort
- First Course in Algorithms Through Puzzles
- CS302FinalGuide_2
- openqc
- Solutions 4
- Line Segment Intersect
- big-oh
- Chapter 2
- L-02 Complexity Analysis.ppt
- Lecture 1
- Ps2 Solutions
- 1_ Logs KEY
- 475228_634176862880063750
- Roetteler All
- Algorithm Analysis
- Eksponencijalne i Logaritamske Jednacine
- mj
- 151704-2150703-ADA

You are on page 1of 14

Agenda: Worst/Best/Avg case analysis InsertionSort example Loop invariant Asymptotic Notations Textbook pages: 15-27, 41-57

InsertionSort(A) **sort A[1..n] in place for j 2 to n do key A[j ] **insert A[j ] into sorted sublist A[1..j 1] ij1 while (i > 0 and A[i] > key ) do A[i + 1] A[i] ii1 A[i + 1] key

InsertionSort(A) for j 2 to n do key A[j ] ij1 while (i > 0 and A[i] > key ) do A[i + 1] A[i] ii1 A[i + 1] key cost c1 c2 c3 c4

j =2 n

times n n1 n1

n

tj (tj 1)

j =2 n

c5 c6

j =2

(tj 1) n1

c7

n

j =2

tj

Assumptions: (Uniform cost) RAM Key comparison (KC) happens: i > 0 and A[i] > key (minors: loop counter increment operation, copy) RAM running time proportional to the number of KC Best case (BC) What is the best case? already sorted One KC for each j , and so Worst case (WC) What is the worst case? reverse sorted j KC for xed j , and so Average case (AC)

n j j =2 n 1 j =2

=n1

n(n+1) 2

Average case: always ask average over what input distribution? Unless stated otherwise, assume each possible input equiprobable Uniform distribution Here, each of the possible inputs equiprobable (why?)

Key observation: equiprobable inputs imply for each key, rank among keys so far is equiprobable e.g., when j = 4, expected number of KC is

1+2+3+4 4

j

= 2 .5

n

i=1

j +1 2

j+1 1 = 2 2

n+1

=

j =3

1 2

(n + 1)(n + 2) 3 2

n2 + 3n 4 4

j =2

We will ignore the constant factors; we also only care about the dominant term, here it is n2 .

Claim: At the start of each iteration of the for loop, the subarray A[1..j 1] consists of the elements originally in A[1..j 1] and in sorted order. Proof of claim. initialization: j = 2 maintenance: j j + 1 termination: j = n + 1

Common points initialization vs. base step maintenance vs. inductive step Dierence termination vs. innite

Why correctness? Always a good idea to verify correctness Becoming more common in industry This course: a simple introduction to correctness proofs When loop is involved, use loop invariant (and induction) When recursion is involved, use induction Loop invariant (LI) Initialization: Maintenance: next? Termination #1: correctness? Termination #2: Insert sort LI: At start of for loop, keys initially in A[1..j 1] are in A[1..j 1] and sorted. Initialization: A[1..1] is trivially sorted does LI hold 1st time through? if LI holds one time, does LI hold the upon completion, LI implies does loop terminate?

Maintenance: none from A[1..j ] moves beyond j ; sorted Termination #1: A[1..n] is sorted upon completion, j = n + 1 and by LI

Termination #2: for loop counter j increases by 1 at a time, and no change inside the loop 6

Assume LI holds when j = k and so A[1] A[2] . . . A[k 1] The for loop body contains another while loop. Use another LI. LI2: let A [1..n] denote the list at start of while loop. Then each time execution reaches start of while loop: A[1..i + 1] = A [1..i + 1] A[i + 2..j ] = A [i + 1..j 1] Prove LI2 (exercise) Using LI2, prove LI Hint: when LI2 terminates, either i = 0 or A[i] key (the latter implies either i = j 1 or A[i + 1] > key ).

Analysis of algorithms becomes analysis of functions: e.g., f (n) denotes the WC running time of insertion sort g (n) denotes the WC running time of merge sort f (n) = c1 n2 + c2 n + c3 g (n) = c4 n log n Which algorithm is preferred (runs faster)? To simplify algorithm analysis, want function notation which indicates rate of growth (a.k.a., order of complexity)

roughly, The set of functions which, as n gets large, grow no faster than a constant times f (n). precisely, (or mathematically) The set of functions {h(n) : N R} such that for each h(n), there are constants c0 R+ and n0 N such that h(n) c0 f (n) for all n > n0 . examples: h(n) = 3n3 + 10n + 1000 log n O(n3 ) h(n) = 3n3 + 10n + 1000 log n O(n4 ) h(n) = 5n , n2 , n 10120 O(n2 ) 120 n > 10 8

Denitions:

O(f (n)) is the set of functions h(n) that roughly, grow no faster than f (n), namely c0 , n0 , such that h(n) c0 f (n) for all n n0 (f (n)) is the set of functions h(n) that roughly, grow at least as fast as f (n), namely c0 , n0 , such that h(n) c0 f (n) for all n n0 (f (n)) is the set of functions h(n) that roughly, grow at the same rate as f (n), namely c0 , c1 , n0 , such that c0 f (n) h(n) c1 f (n) for all n n0 (f (n)) = O(f (n)) (f (n)) o(f (n)) is the set of functions h(n) that roughly, grow slower than f (n), namely

h(n) limn f =0 (n)

(f (n)) is the set of functions h(n) that roughly, grow faster than f (n), namely

h(n) limn f = (n)

Warning:

the textbook overloads = Textbook uses g (n) = O(f (n)) Incorrect !!! Because O(f (n)) is a set of functions. Correct: g (n) O(f (n)) You should use the correct notations.

Examples: which of the following belongs to O(n3 ), (n3 ), (n3 ), o(n3 ), (n3 ) ?

1. f1 (n) = 19n 2. f2 (n) = 77n2 3. f3 (n) = 6n3 + n2 log n 4. f4 (n) = 11n4

10

Answers:

1. 2. 3. 4. f1 (n) = 19n f2 (n) = 77n2 f3 (n) = 6n3 + n2 log n f4 (n) = 11n4

f1 , f2 , f3 O(n3 ) f1 (n) 19n3 , for all n 0 c0 = 19, n0 = 0 f2 (n) 77n3 , for all n 0 c0 = 77, n0 = 0 f3 (n) 6n3 + n2 n, for all n 1, since log n n if f4 (n) c0 n3 , then n f3 , f4 (n3 ) f3 (n) 6n3 , for all n 1, since n2 log n 0 f4 (n) 11n3 , for all n 0 f3 (n3 ) why? f1 , f2 o(n3 )

n f1 (n): limn 19 = limn 19 =0 n3 n2 n f2 (n): limn 77 = limn 77 =0 n3 n

2

c0 11

no such n0 exists

f3 (n): limn 6n

+n2 log n n3

4

= limn 6 +

log n n

=6

f4 (n3 ) 11

logarithm review:

Denition of logb n (b, n > 0): blogb n = n logb n as a function in n: increasing, one-to-one logb 1 = 0 logb xp = p logb x logb (xy ) = logb x + logb y xlogb y = y logb x logb x = (logb c)(logc x)

ln n = loge n (natural logarithm) lg n = log2 n (base 2, binary) (logb n) = (log{whatever

d dx positive} n)

= (log n)

ln x =

1 x

h(n) O(f (n)) if and only if f (n) (h(n))

h(n) = ... limit rules: limn f (n)

. . . , then h (f ), (f ) . . . 0 < k < , then h (f ) . . . 0, then h O(f ), o(f ) LH opitals rules: if limn h(n) = , limn f (n) = , and h (n), f (n) exist, then h (n) h(n) = lim n f (n) n f (n) lim

n 1 e.g., limn ln = limn n =0 n

Still, we have h(n) O(n2 ), h(n) (1), etc. O(), (), (), o(), () JUST useful asymptotic notations

13

n n . e

Example: The following functions are ordered in increasing order of growth (each is in big-Oh of next one). Those in the same group are in big-Theta of each other.

{n1/ log n ,

1} ,

ln ln n},

log n,

ln n, {n2 , en ,

n

log n

( 2)log n , {(log n)

{n log n, }, 3 2

log(n!)},

n

(log n)!,

log n

log log n

n.2n ,

(n!)2 ,

14

- Java Source CodeUploaded byDarko Jakovleski
- GATE+TIFR April 2016Uploaded byAshish Kumar
- Relational Algebra & SyncsortUploaded byvengangaboy
- First Course in Algorithms Through PuzzlesUploaded byjorgecarranza
- CS302FinalGuide_2Uploaded byJuly Zerg
- openqcUploaded bySrikar Varadaraj
- Solutions 4Uploaded byamit
- Line Segment IntersectUploaded bywanting252
- big-ohUploaded byHien Tran
- Chapter 2Uploaded bytenpointer
- L-02 Complexity Analysis.pptUploaded bycatarac19
- Lecture 1Uploaded byArslan Muhammad
- Ps2 SolutionsUploaded byOliver Gannon
- 1_ Logs KEYUploaded byjessica padilla
- 475228_634176862880063750Uploaded bypv28
- Roetteler AllUploaded bycharles luis
- Algorithm AnalysisUploaded bybeenagodbin
- Eksponencijalne i Logaritamske JednacineUploaded byAnonymous WkbmWCa8M
- mjUploaded byMatt John Evangelista Loresco
- 151704-2150703-ADAUploaded byInnoce Jempth
- ChecklistUploaded byChathumi Lelwala
- An Optimal Algorithm for Approximate Nearest. Neighbor Searching in Fixed DimensionsUploaded bylerhlerh
- SDT Exam MS Sept 2016 - FinalUploaded byRoshan Bc
- MIT15_082JF10_lec01.pptUploaded byeviroyer
- Convex HullUploaded byশুভ্রা দেবনাথ
- next1.docxUploaded bybhatiaharryjassi
- thuat toan k-treeUploaded bydiephtv
- excel-missing-values.pdfUploaded byseptian
- Comparison Chart.pdfUploaded byRyan Mills
- APIReferenceUploaded byPeter Mtz

- Life After Loss Grieving With Hope(1)Uploaded bym1u2n3n4i5
- optimality of naive bayes.pdfUploaded bym1u2n3n4i5
- ShinyUploaded bym1u2n3n4i5
- unit-IV.pptUploaded bym1u2n3n4i5
- 02DataUploaded bym1u2n3n4i5
- DWDMUploaded bym1u2n3n4i5
- New Text Document (2)Uploaded bym1u2n3n4i5
- AcdUploaded bym1u2n3n4i5
- SongUploaded bym1u2n3n4i5
- Ecm-oops Through JavaUploaded bym1u2n3n4i5
- 10.1.1.94.3837[1]Uploaded bym1u2n3n4i5

- SortingUploaded byAishwarya
- Python Lab Manual FinalUploaded bySARANYA.R MIT-AP/CSE
- Cmpt 125 Study NotesUploaded byMonique Lai
- Tut+6Uploaded byThu Vu
- Assignment 1 FrontsheetUploaded byChấn Nguyễn
- MCA06Uploaded bySrinivasan Ramachandran
- Proposal of a Two Way Sorting Algorithm and Performance Comparison with Existing AlgorithmsUploaded byBilly Bryan
- Computer Science Notes – Problem Solving TechniquiesUploaded bygautham28
- Top Data Structure Interview Questions and Answers Are BelowUploaded byArchana Sinha
- DSA quizUploaded byHenryHai Nguyen
- Data Structures.docxUploaded byRitesh Singh
- Appendix F - Sorting- A Deeper LookUploaded byZoltan Loboda
- Elementary Sorts CoursenotesUploaded bybsitler
- Comparison and Enhancement of Sorting AlgorithmsUploaded byEditor IJRITCC
- AlgorithmUploaded bykaloy33
- wipro cUploaded byapi-3844691
- Internal Sorting MethodsUploaded byKeith Tanaka Magaka
- Data Str Lab Manual Bit 232Uploaded byJnaneswar Reddy
- DataStructure PracticalsUploaded byTurtle17
- alorithm basicsUploaded byankit pandey
- comp231_2013_2014_worksheet_01Uploaded byKadir
- Problem Set 2 SolUploaded bysristisagar
- Bubble SortUploaded byRongeet Sodhi
- bca iv sem ada qbUploaded byapi-285363432
- Chapter 3 - Simple Sorting and SearchingUploaded byshemsedin shukre
- Selection Sort Insertion Sort Bottomup Merge Chapter 1Uploaded bySahil Arora
- Question CS301 for Final 2010Uploaded bycs619finalproject.com
- UNIT 8Uploaded bybadboy_rockss
- Cpds Previous Papers Questions 2012 HarshaUploaded byHarsha Vardhan
- c queUploaded bygceramesh