13 views

Uploaded by FarizParahitaRoestam

- Dynamic Problems (Fundamentals of Algorithms)
- hwk1
- Asymptotic Notation.pdf
- PTTKGT
- 15-bigO
- Big-O Algorithm Complexity Cheat Sheet
- Substitution Method
- General Math DLL for SHS - (more dll at depedtambayanph.blogspot.com) Q1, Week 10.xlsx
- Algo Analysis and Time Complexity
- Algorithm for graph isomorphism with polinomial complexity
- Daa Ppt 28th Jan
- IVIC-SmFunct
- MIT18_310F13_Ch18
- A Faster algorithm for Calculating Hypervolume.pdf
- Optimal Solutions for the Temporal Precedence Problem
- Mathematical Analysis - List 2
- Causal Inference
- Dcap407 Data Structure
- samworth12st
- Computational Complexity

You are on page 1of 6

P. Danziger

Comparing Algorithms

We have seen that in many cases we would like to compare two algorithms. Generally, the eciency of an algorithm can be guaged by how long it takes to run as a function of the size of the input. For example the eciency of a graph algorithm might be measured as a function of the number of vertices or edges of the input graph. So, the Eulerian algorithm nds an Eulerian circuit after roughly m steps, where m is the number of edges in the input graph. When considering the eciency of an algorithm we always consider the worst case. The Hamiltonian circuit problem can be solved in roughly n! steps, by considering all possible circuits. We might be lucky and discover a Hamiltonian circuit on the rst try, but we assume the worst case. We now consider what we mean by roughly. We say that two functions f and g are of thee same order if they behave similarly for large values of n, i.e. f (n) g(n) for large n. Consider the functions f (n) = n2 and g(n) = n2 + 2n + 3 n 1 n2 1 2 n + 2n + 3 6 2n 10 100 123 20 400 443 50 100 200 2500 10000 40000 2603 10203 40403 1030 1060 300 90000 90603 400 160000 160803 500 250000 251003 1000 1000000 1002003 10301

1.1

Big O

Denition 1 Given a function f : R R, or the corresponding sequnce an = f (n). 1. (f ) = functions that are of equal or greater order than f . All functions that are larger than some constant multiple of |f (x)| for suciently large values of x. (f ) = {g(x) | A, a R+ such that A|f (x)| |g(x)| for all x > a} 2. O(f ) = functions that are of equal or less order than f . All functions that are smaller than some constant multiple of |f (x)| for suciently large values of x. O(f ) = {g(x) | A, a R+ such that |g(x)| A|f (x)| for all x > a} 3. (x) = functions that are of the same order as f . All functions that are always between two constant multiples of |f (x)| for suciently large values of x. (f ) = {g(x) | A, B, a R+ such that A|f (x)| |g(x)| B|f (x)| for all x > a} 1

Big O Notation

P. Danziger

4. o(f ) = functions that are of strictly lesser order than f . o(f ) = O(f ) (f ) If g O(f ) we say g is of order f , many authors abuse notation by writing g = O(f ). Alternately, we can dene order by using the notion of the limit of the ratio of sequences tending to a value. Denition 2 Given a function f : R R, or the corresponding sequnce an = f (n). 1. (f ). (f ) = 2. O(f ) O(f ) = 3. (f ) (f ) = 4. o(f ) o(f ) = g(x)

x

g(x)

lim

g(x)

lim

g(x)

lim

lim

g (f ) means that g is of equal or greater order than f . g (f ) means that f and g are roughly the same order of magnitude. g O(f ) means that g is of lesser or equal magnitude than f . g o(f ) means that g is of lesser magnitude than f . Big O is by far the most commonly used and it is very common to say f O(g) (f is at most order g) when what is really meant is that f (g) (f and g are the same order). Example 3 Show that 2n o(3n ). 2n Consider lim n = lim n n 3 3n 3 (Also lim n = lim n 2 n 2

2 3

Theorem 4 (9.2.1) Let f, g, h and k be real valued functions dened on the same set of nonnegative reals, then: 1. (f ) O(f ) = (f ). 2. f (g) g O(f ). 2

Big O Notation

P. Danziger

3. (Reexivity of O) f O(f ), f (f ) and f (f ). 4. (Transitivity of O) If f O(g) and g O(h) then f O(h). 5. If f O(g) and c R {0} then c f (x) O(g) 6. If f O(h) and g O(k) then f (x) + g(x) O(G(x)) where G(x) = max(|h(x)|, |k(x)|). 7. If f (x) O(h) and g O(k) then f (x) g(x) O(h(x) k(x)). In fact a more useful rule than 6 is: 8. If f O(h) and g O(h) then f (x) + g(x) O(h). 9. If f o(g) then g (f ) (f ) 10. o(f ) O(f ).

1.2

Types of Order

The orders form a hierarchy in the sense that if g is in a lower member of the hierarchy it is in all higher members. This is essentially the statement of transitivity (4). The following is a list of common types of orders and their names: Notation Name O(1) Constant O(log(n)) Logarithmic O(log(log(n)) Double logarithmic (iterative logarithmic) o(n) Sublinear O(n) Linear O(n log(n)) Loglinear, Linearithmic, Quasilinear or Supralinear 2 O(n ) Quadratic O(n3 ) Cubic c O(n ) Polynomial (dierent class for each c > 1) n O(c ) Exponential (dierent class for each c > 1) O(n!) Factorial n O(n ) - (Yuck!) Rules 5 and 8 are particularly useful for nding which class a function belongs. Example 5 1. 2n + n2 O(2n ) 2. 2n3 + 3n2 2n + 5 is O(n3 ) (cubic). 3. (2n3 + 3n2 2n + 5) 3 is O(n) (linear). In general a polynomial is of the order of its highest term. 3

1

Big O Notation

P. Danziger

Theorem 6 Given a polynomial f , if the highest power of f is xr : If r < s then f o(xs ) O(xs ). If r = s then f (xs ) O(xs ). If r > s then f (xs ). Proof: Let f be a polynomial with highest power xr . So f (x) = ar xr + ar1 xr1 + . . . + a1 x + a0 . Now let s R and consider f (x) = lim x xs = ar xr + ar1 xr1 + . . . + a1 x + a0 lim x xs

x

0 r<s ar r = s = r>s

1.3

Algorithms

The primary use of order notation in computer science is to compare the eciency of algorithms. Big O notation is especially useful when analyzing the eciency of algorithms. In this case n is the size of the input and f (n) is the running time of the algorithm relative to input size. Suppose that we have two algorithms to solve a problem, we may compare them by comparing their orders. In the following table we suppose that a linear algorithm can do a problem up to size 1000 in 1 second. We then use this to calculate how large a problem this algorithm could handle in 1 minute and 1 hour. We also calculate the largest tractable problem for other algorithms with the given speeds. Running time n n log(n) n2 n3 2n Maximum problem size 1 sec 1 min. 1 hr. 4 1000 6 10 3.6 106 140 4893 2 105 31 244 1897 10 39 153 9 15 21 4

Big O Notation

P. Danziger

In the following table we suppose that the next generation of computers are 10 times faster than the current ones. We then calculate how much bigger the largest tractable problem is for the given algorithm speed. Running time n n log(n) n2 n3 2n After 10 speedup 10 10 3.16 2.15 +3.3

We will need to manipulate some equations involving series. Theorem 8 Given two series f (i) and g(i) and a constant a:

n n n

1.

i=1 n

(f (i) + g(i)) =

i=1 n

f (i) +

i=1

g(i)

2.

i=1

af (i) = a

i=1

f (i)

i.e. The operation of summation from the set of series to R is linear. Proof: Let f (i) and g(i) be two series and a be a constant.

n

1.

i=1

(f (i) + g(i)) = (f (1) + g(1)) + (f (2) + g(2)) + . . . + (f (n) + g(n)) = [f (1) + f (2) + f (3) + . . . + f (n)] + [g(1) + g(2) + g(3) + . . . + g(n)]

n n

=

i=1 n

f (i) +

i=1

g(i)

2.

i=1

af (i)

n

= a

i=1

f (i)

n i=1 a n i=1 i n 2 i=1 i

Big O Notation

P. Danziger

When confronted with a series n f (i) involving powers of i we rst collect coecients in f (i) i=1 so that it is expressed as a sum of powers of i, f (i) = a0 + a1 i + a2 i2 + . . . + an in . Now we use the formulas for powers of summing i. We use pk (n) to represent the formula giving n ik . i.e. i=1 n ik = pk (n). i=1 n i=1 f (i) n 2 n = i=1 (a0 + a1 i + a2 i + . . . + an i ) n n n n 2 n = i=1 a0 + i=1 a1 i + i=1 a2 i + . . . + i=1 an i 1 = na0 + a1 1 n(n + 1)i + 6 a2 n(n + 1)(2n + 1) + . . . + an pn (n) 2 We then simplify by collecting powers of n.

- Dynamic Problems (Fundamentals of Algorithms)Uploaded byVyanshu Arora
- hwk1Uploaded by007wasr
- Asymptotic Notation.pdfUploaded byJuan Marcelo Chamorro Avendaño
- PTTKGTUploaded bykhai
- 15-bigOUploaded byDiponegoro Muhammad Khan
- Big-O Algorithm Complexity Cheat SheetUploaded byNilesh
- Substitution MethodUploaded byamukhopadhyay
- General Math DLL for SHS - (more dll at depedtambayanph.blogspot.com) Q1, Week 10.xlsxUploaded byMelanio
- Algo Analysis and Time ComplexityUploaded bylavamgmca
- Algorithm for graph isomorphism with polinomial complexityUploaded byUros
- Daa Ppt 28th JanUploaded byccatty
- IVIC-SmFunctUploaded byAnonymous 0U9j6BLllB
- MIT18_310F13_Ch18Uploaded bybhar
- A Faster algorithm for Calculating Hypervolume.pdfUploaded byIvan Hernandez
- Optimal Solutions for the Temporal Precedence ProblemUploaded byJamesHowson
- Mathematical Analysis - List 2Uploaded byslowmotiony
- Causal InferenceUploaded bygschiro
- Dcap407 Data StructureUploaded byJagat Mishra
- samworth12stUploaded bySanjay Chouhan
- Computational ComplexityUploaded byseunnuga93
- TTPUploaded byNguyễn Việt Phương
- Week 3 - Lecture 1Uploaded bySayantan Ghosh
- Various Heuristic techniques used for problem solving : A ReviewUploaded bySOMNATH MAJI
- MELJUN CORTES Automata Lecture Graphs 1Uploaded byMELJUN CORTES, MBA,MPA
- Lauth, B. - Inductive Inference in the Limit of Empirically Adequate TheoriesUploaded byRicardo Rom
- CMIIUploaded byGustavo Bravo
- Ch 2 ArraysUploaded bythe_summit
- Re Configurations of Polygonal StructuresUploaded byNick Watson
- UBICC Manuscript 323-2-323Uploaded byUbiquitous Computing and Communication Journal
- 39 definition of a functionUploaded byapi-299265916

- ApacheHVACUploaded byRaffaele
- energy in the ecosystem webquestUploaded byapi-327342342
- ALGORITMO GENETICOUploaded byAlvaro Portela DelaRosa
- 2_NCL -PU Grout.pdfUploaded byKanda Dinda
- Non Destructive TestingUploaded byAhmad Hanafi Saharuddin
- Effectiveness of Pandora Advetising on Business Performance MetricsUploaded byOscar Bueno
- HVAC BOQ R1- 29-08-2013 (2)Uploaded byVishwajit Kale
- TQM IEM_Reliability (1)Uploaded byShailesh Gupta
- Break Out of or Exit Different Types of Loops in VBA Macros in ExcelUploaded byiksyot
- rfm02Uploaded bylityroby
- QP-QAM06Uploaded byRavi Nookala
- 92101638 AP Payment Process NotesUploaded bymymle1
- final pptUploaded bySaurabh Kumar
- SIPOC TemplateUploaded byfabianbeck
- 3 guide-Uploaded byAl7amdlellah
- performance appraisalUploaded byRonak Gupta
- EkgUploaded byShivana Seeram
- Pro-Social Reinforcement PwrPointUploaded bymusdave
- Fire Safety Green BuildingsUploaded byferdi1974
- T11557Uploaded byRaúl Frodden
- Diff Eq's Odd AnswersUploaded byBrad Stalder
- PRC Lab.docxUploaded byHannah Joy Pansacala
- IRCEELT 2018 ProceedingsUploaded byდავით იაკობიძე
- AD-D-01 Land Surveying and Mapping GuideUploaded byAnonymous zmYUcSP
- 15C1FinlUploaded byfast nspeedy
- timer circuitUploaded byMamon Ghosh
- AltivarUploaded byTaryo
- DSKI_Part_IUploaded byMuhammad Mahfud Rosyidi
- Barton - Rock MechanicsUploaded byKlenher Madariaga Hancco
- Professional Practice CompiledUploaded byMAND33P