Professional Documents
Culture Documents
Data Structures
and Algorithms
1
Algorithms
Analysis of Algorithms 3
Efficiency ?
• Execution time …
• Memory …
• Simplicity ….
Analysis of Algorithms 4
Running Time
Analysis of Algorithms 5
Running time
best case
average case
worst case
120
100
Running Time
80
60
40
20
0
1000 2000 3000 4000
Input Size
Analysis of Algorithms 6
Analysis of Algorithms
T(n)
• Running Time
• Upper Bounds n=4
• Lower Bounds
• Examples
• Mathematical
facts Input Algorithm Output
Analysis of Algorithms 7
Average Case vs. Worst Case
Running Time of an algorithm
3 ms
}
average-case?
best-case
2 ms
1 ms
Analysis of Algorithms 8
A B C D E F G
Input
Measuring the Running Time
Analysis of Algorithms 9
Measuring the Running Time
• Approach 1: Experimental Study
• The measurement of the experiment can
be like this… best case
average case
worst case
120
100
Running Time
80
60
40
20
0
Analysis of Algorithms 1000 2000 3000 4000
Input Size
Beyond Experimental Studies
Analysis of Algorithms 11
Theoretical Analysis
Analysis of Algorithms 12
Analysis of Algorithms
• Examples:
– calling a method and returning from a method
– arithmetic operations (e.g. addition)
– comparing two numbers, etc.
13
Pseudo-code
Analysis of Algorithms 14
Pseudo-code
Analysis of Algorithms 15
Pseudo-code
• Programming element:
– decision: if.. Then.. [else..]
– Loop while: while…do
– Loop repeat: repeat … until..
– Loop for: for.. Do
– Vector index: A[i]
• methods:
– call: object method (args)
– return: return value
Analysis of Algorithms 16
Example:
currentMax ¬ A[0]
for i ¬ 1 to n -1 do
if currentMax < A[i] then
currentMax ¬ A[i]
return currentMax
Analysis of Algorithms 17
A
5 13 4 7 6 2 3 8 1 2
currentMax
currentMax ¬ A[0]
for i ¬ 1 to n -1 do
if currentMax < A[i] then
currentMax ¬ A[i]
return currentMax
Analysis of Algorithms 18
A
5 13 4 7 6 2 3 8 1 2
currentMax 5
currentMax ¬ A[0]
for i ¬ 1 to n -1 do
if currentMax < A[i] then
currentMax ¬ A[i]
return currentMax
Analysis of Algorithms 19
A
5 13 4 7 6 2 3 8 1 2
currentMax 5
currentMax ¬ A[0]
for i ¬ 1 to n -1 do
if currentMax < A[i] then
currentMax ¬ A[i]
return currentMax
Analysis of Algorithms 20
What are the primitive operations to count
A
5 13 4 7 6 2 3 8 1 2
Comparisons
Assignments to
currentMax
currentMax 13
currentMax ¬ A[0]
for i ¬ 1 to n -1 do
if currentMax < A[i] then
currentMax ¬ A[i]
return currentMax
Analysis of Algorithms 21
currentMax ¬ A[0] 1 assignment
for i ¬ 1 to n -1 do
if currentMax < A[i] then
n-1 comparisons
currentMax ¬ A[i]
n-1 assignments (worst case)
return currentMax
5 7 8 10 11 12 14 16 17 20
Analysis of Algorithms 22
In the best case ?
15 1 12 3 9 7 6 4 2 11
return currentMax
Analysis of Algorithms 23
Summarizing:
Analysis of Algorithms 24
Another Example
i¬0 1 assignment
while (A[i] ¹ element)
sizeA checks & assignment
i ¬ i+1
(if we are not lucky)
return i
Worst Case
Analysis of Algorithms 25
Upper Bound
f(n)
n0
Analysis of Algorithms 26n
prove that f(n)≤ c g(n) for some n ≥ n0
An Example
f(n) = 60n2 + 5n + 1 g(n) = n2
≤ prove that f(n)≤ c n2
= 66n2
c = 66 n0 = 1
f(n) ≤ c n2 n ≥ n0
Analysis of Algorithms 27
On the other hand…
Analysis of Algorithms 28
n0 n
O(1) < O(log n) < O(n) < O(n log n) < O(n2) <
O(n3) < O(2n) … remember !!
n2
log(n)
n0 n
Analysis of Algorithms 29
n0 n
Analysis of Algorithms 30
Analysis of Algorithms 31
Analysis of Algorithms 32
O(1) < O(log n) < O(n) < O(n log n) < O(n2) <
n
O(n ) < O(2 ) … remember !!
3
Analysis of Algorithms 33
n= 2 16 256 1024
log log n 0 2 3 3.32
log n 1 4 8 10
n 2 16 256 1024
n log n 2 64 448 10 200
n2 4 256 65 500 1.05 * 106
Analysis of Algorithms 34
Asymptotic Notation (cont.)
Analysis of Algorithms 35
Theorem:
If g(n) is O(f(n)) , then for any constant
c >0
g(n) is also O(c f(n))
Theorem:
O(f(n) + g(n)) = O(max(f(n), g(n)))
Ex 1:
2n3 + 3n2 = O (max(2n3, 3n2))
= O(2n3) = O(n3)
Ex 2:
n2 + 3 log n – 7 = O(max(n2, 3 log n – 7))
= O(n2)
Analysis of Algorithms 36
Simple Big Oh Rule:
7n-3 is O(n)
Analysis of Algorithms 37
Other Big Oh Rules:
Analysis of Algorithms 38
Asymptotic Notation
(terminology)
quadratic:
1E+18
O(n2)
T (n )
1E+16
cubic: O(n3) 1E+14
1E+12
polynomial: O(nk), k ≥ 1 1E+10
1E+8
exponential: O(an), n > 1 1E+6
1E+4
1E+2
1E+0
1E+0 1E+2 1E+4 1E+6 1E+8 1E+10
n
Analysis of Algorithms 39
Asymptotic Analysis and execution
time
• Use the Big-O notation
– to indicate the number of primitive operations
executed according to the entry size
• For example, we say that algorithm arrayMax
has an execution time O(n)
• While comparing the asymptotic execution
times
– O(log n) is better than O(n)
– O(n) is better than O(n2)
– log n << n-2 << n << n log n << n2 << n3 << 2n
Analysis of Algorithms 40
Asymptotic Analysis and execution
time
• Use the Big-O notation
– to indicate the number of primitive operations
executed according to the entry size
• For example, we say that algorithm arrayMax
has an execution time O(n)
• While comparing the asymptotic execution
times
– O(log n) is better than O(n)
– O(n) is better than O(n2)
– log n << n << n log n << n2 << n3 << 2n
Analysis of Algorithms 41
Example of Asymptotic Analysis
Analysis of Algorithms 42
Example of Asymptotic Analysis
Algorithm prefixAverages1(X, n)
Analysis of Algorithms 44
Another Example
f(n)
c • g(n)
Analysis of Algorithms n0 46
n
Tight Bound
… is big theta …
g(n) is Q(f(n))
<===>
if g(n) Î O(f(n))
AND
f(n) Î O(g(n))
Therefore:
f(n) is O(n2)
AND
f(n) is Ω(n2)
f(n) is Q(n2)
Analysis of Algorithms 48
Intuition for Asymptotic
Notation
Big-Oh
– f(n) is O(g(n)) if f(n) is asymptotically
less than or equal to g(n)
big-Omega
– f(n) is W(g(n)) if f(n) is asymptotically
greater than or equal to g(n)
big-Theta
– f(n) is Q(g(n)) if f(n) is asymptotically
equal to g(n)
Analysis of Algorithms 49
Math You Need to Review
Logarithms and Exponents
properties of logarithms:
logb(xy) = logbx + logby
logb (x/y) = logbx - logby
logbxa = alogbx
logba= logxa/logxb
properties of exponentials:
a(b+c) = aba c
abc = (ab)c
ab /ac = a(b-c)
b = a logab
bc = a c*logab
Analysis of Algorithms 50
More Math to Review
2S = nd + nd + nd + …+ nd
= (n+1) nd
S = d/2 n(n+1)
Analysis of Algorithms
for d=1, S = 1/2 n(n+1) 52
More Math to Review