Professional Documents
Culture Documents
2.04.algorithm Analysis
2.04.algorithm Analysis
Algorithm Analysis
Douglas
Douglas Wilhelm
Wilhelm Harder,
Harder, M.Math.
M.Math. LEL
LEL
Department
Department of of Electrical
Electrical and
and Computer
Computer Engineering
Engineering
University
University of
of Waterloo
Waterloo
Waterloo,
Waterloo, Ontario,
Ontario, Canada
Canada
ece.uwaterloo.ca
ece.uwaterloo.ca
dwharder@alumni.uwaterloo.ca
dwharder@alumni.uwaterloo.ca
©
© 2006-2013
2006-2013 by
by Douglas
Douglas Wilhelm
Wilhelm Harder.
Harder. Some
Some rights
rights reserved.
reserved.
Asymptotic Analysis
2
Outline
Motivation
Motivation
Motivation
However:
– Merge sort will require twice and 10 times as much memory
– Quick sort will require one or four additional memory locations
– Heap sort will not require any additional memory
Asymptotic Analysis
6
Motivation
Motivation
Motivation
If we are storing objects which are not related, the hash table has, in
many cases, optimal characteristics:
– Many operations are Q(1)
– I.e., the run times are independent of the number of objects being
stored
Motivation
Machine Instructions
Machine Instructions
Machine Instructions
Machine Instructions
Operators
Operators
Blocks of Operations
Each operation runs in Q(1) time and therefore any fixed number
of operations also run in Q(1) time, for example:
// Swap variables a and b
int tmp = a;
a = b;
b = tmp;
Blocks of Operations
Blocks in Sequence
To calculate the total run time, add the entries: Q(1 + n + 1) = Q(n)
Asymptotic Analysis
19
Blocks in Sequence
Q(1)
return *this;
}
if ( capacity != A.capacity ) {
delete [] column_index;
delete [] off_diagonal;
capacity = A.capacity;
column_index = new int[capacity]; Q(1) Q(1 + 1 + min(M, N) + M + n + 1)
}
off_diagonal = new double[capacity];
= Q(M + n)
for ( int i = 0; i < minMN; ++i ) {
diagonal[i] = A.diagonal[i];
} Q(min(M, N))
for ( int i = 0; i <= M; ++i ) {
row_index[i] = A.row_index[i];
}
Q(M) Note that min(M, N) ≤ M but we
for ( int i = 0; i < A.size(); ++i ) { cannot say anything about M and n
column_index[i] = A.column_index[i];
off_diagonal[i] = A.off_diagonal[i];
} Q(n)
return *this;
}
Q(1)
Asymptotic Analysis
20
Blocks in Sequence
Control Statements
Control Statements
Control Statements
Given
if ( condition ) {
// true body
} else {
// false body
}
Control Statements
Control Statements
return max;
}
Asymptotic Analysis
26
Analysis of Statements
Condition-controlled Loops
Condition-controlled Loops
Condition-controlled Loops
For example,
for ( int i = 0; i < n; ++i ) {
// ...
}
Condition-controlled Loops
If the body does not depend on the variable (in this example, i),
then the run time of
for ( int i = 0; i < n; ++i ) {
// code which is Theta(f(n))
}
is Q(n f(n))
If the body is O(f(n)), then the run time of the loop is O(n f(n))
Asymptotic Analysis
31
Condition-controlled Loops
For example,
int sum = 0;
for ( int i = 0; i < n; ++i ) {
sum += 1; Theta(1)
}
Condition-controlled Loops
Conditional Statements
void Disjoint_sets::clear() {
if ( sets == n ) {
return; Q(1)
}
max_height = 0;
num_disjoint_sets = n; Q(1)
for ( int i = 0; i < n; ++i ) {
parent[i] = i;
Q(n)
tree_height[i] = 0;
}
} Q(1)
(1) sets n
Tclear (n)
(n) otherwise
Asymptotic Analysis
35
If the body does depends on the variable (in this example, i), then
the run time of
for ( int i = 0; i < n; ++i ) {
// code which is Theta(f(i,n))
}
n 1
is 1 1 f i, n and if the body is
i 0
O(f(i, n)), the result is
n 1
O 1 1 f i, n
i 0
Asymptotic Analysis
36
For example,
int sum = 0;
for ( int i = 0; i < n; ++i ) {
for ( int j = 0; j < i; ++j ) {
sum += i + j;
}
}
n 1 n 1
n n 1
1 1 i 1 n i 1 n n
2
i 0 i 0 2
Asymptotic Analysis
37
As another example:
int sum = 0;
for ( int i = 0; i < n; ++i ) {
for ( int j = 0; j < i; ++j ) {
for ( int k = 0; k < j; ++k ) {
sum += i + j + k;
}
}
}
Control Statements
switch( i ) {
case 1: /* do stuff */ break;
case 2: /* do other stuff */ break;
case 3: /* do even more stuff */ break;
case 4: /* well, do stuff */ break;
case 5: /* tired yet? */ break;
default: /* do default stuff */
}
Asymptotic Analysis
39
Control Statements
if ( i == 1 ) { /* do stuff */ }
else if ( i == 2 ) { /* do other stuff */ }
else if ( i == 3 ) { /* do even more stuff */ }
else if ( i == 4 ) { /* well, do stuff */ }
else if ( i == 5 ) { /* tired yet? */ }
else { /* do default stuff */ }
Asymptotic Analysis
40
Control Statements
Question:
Why would you introduce something into
programming language which is redundant?
Control Statements
First, you may recall that the cases must be actual values, either:
– integers
– characters
Control Statements
Then the compiler simply makes a jump size based on the variable,
jumping ahead either 0, 24, 48, 72, ..., or 240 instructions
Asymptotic Analysis
43
Serial Statements
If the first block of code is O(f(n)) and the second is O(g(n)), then the
run time of two blocks of code is
O( f(n) + g(n) )
which usually (for algorithms not including function calls) simplifies
to one or the other
Asymptotic Analysis
44
Serial Statements
What is the proper means of describing the run time of these two
algorithms?
Asymptotic Analysis
45
Serial Statements
Searching for the maximum entry requires that each element in the
array be examined, thus, it must run in Q(n) time
Searching for a particular entry may end earlier: for example, the
first entry we are searching for may be the one we are looking for,
thus, it runs in O(n) time
Asymptotic Analysis
46
Serial Statements
Therefore:
– if the leading term is big-Q, then the result must be big-Q, otherwise
– if the leading term is big-O, we can say the result is big-O
For example,
O(n) + O(n2) + O(n4) = O(n + n2 + n4) = O(n4)
O(n) + Q(n2) = Q(n2)
O(n2) + Q(n) = O(n2)
O(n2) + Q(n2) = Q(n2)
Asymptotic Analysis
47
Functions
Functions
Functions
Functions
That is, it is impossible for any function call to have a zero run time
Asymptotic Analysis
51
Functions
Because the run time of any function is at least O(1), we will include
the time required to both call and return from the function in the run
time
Asymptotic Analysis
52
Functions
if ( m == n ) {
return;
}
Tset_union= 2Tfind + Q(1)
--num_disjoint_sets;
Recursive Functions
Recursive Functions
Recursive Functions
Θ(1) n 1
T! ( n )
T! ( n 1) Θ(1) n 1
We don’t have to worry about the time of the conditional (Q(1)) nor is
there a probability involved with the conditional statement
Asymptotic Analysis
56
Recursive Functions
Recursive Functions
Thus, to find the run time of the factorial function, we need to solve
T!(n) = T!(n – 1) + 1 T!(1) = 1
Recursive Functions
Recursive Functions
If k = n – 1 then
T!(n) = T!(n – (n – 1)) + n – 1
= T!(1) + n – 1
=1+n–1=n
Recursive Functions
Recursive Functions
We could:
– go through the list and find the largest item
– swap the last entry in the list with that largest item
– then, go on and sort the rest of the array
Recursive Functions
int tmp = array[n - 1]; // swap the largest entry with the last
array[n - 1] = array[posn];
array[posn] = tmp;
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
n
1n( n1 ) 1
2
> expand( % );
1 1 2
n n
2 2
Asymptotic Analysis
73
Recursive Functions
T( n ) T( n 1) n
T( n 2) ( n 1) n
T( n 2) n ( n 1)
T( n 3) n ( n 1) ( n 2)
n n n
n( n 1)
T(1)
i 2
i 1
i 2
i
i 1
i
2
Asymptotic Analysis
74
Recursive Functions
Recursive Functions
Recursive Functions
Recursive Functions
T( n ) T( 2 k 1)
2k 1 1
T 1
2
T( 2 k 1 1) 1
2 k 1 1 1
T 11
2
T( 2 k 2 1) 2
Asymptotic Analysis
78
Recursive Functions
T( 2 k 1 1) 1
2 k 1 1 1
T 11
2
T( 2 k 2 1) 2
T( 2 k 3 1) 3
Asymptotic Analysis
79
Recursive Functions
T( n ) T( 2 k 1)
T( 2 k ( k 1) 1) k 1
T(1) k 1 k
because T(1) = 1
Asymptotic Analysis
80
Recursive Functions
f n
However, recall that f(n) = Q(g(n)) if lim c for 0 c
n g n
1
lg n 1 n 1 ln 2 n 1
lim lim lim
n ln n n 1 n n 1 ln 2 ln 2
n
Thus, T(n) = Q(lg(n + 1)) = Q (ln(n))
Asymptotic Analysis
81
Cases
Cases
Cases
Assume the case we are looking for is in the list and equally likely
distributed
If the list is of size n, then there is a 1/n chance of it being in the kth
location
Thus, we sum
1 n 1 n(n 1) n 1
n k 1
k
n 2
2
which is O(n)
Asymptotic Analysis
84
Cases
We could write:
n
i i
k 1 2
k
k ?
k 1 2
Asymptotic Analysis
85
Cases
You’re not expected to know this for the final, however, for interest:
Cases
return max;
}
Asymptotic Analysis
87
Cases
Cases
Thus, given a sub-list of size k, the probability that any one element
is the largest is 1/k
n 1 n
1 1
i 0
i 1
i 1
i
?
Asymptotic Analysis
89
Cases
Cases
Cases
From calculus:
n n 1
1 1 n 1
0 x 1 dx
1
x
dx ln( x) 1 ln(n 1) ln(1) ln(n 1)
Cases
Cases
Cases
Cases
return max;
}
n 1 1
is 1 1 1 n ln(n) n
i 1 i 1
Asymptotic Analysis
96
Summary
Summary
References
Wikipedia, https://en.wikipedia.org/wiki/Mathematical_induction
These slides are provided for the ECE 250 Algorithms and Data Structures course. The
material in it reflects Douglas W. Harder’s best judgment in light of the information available to
him at the time of preparation. Any reliance on these course slides by any party for any other
purpose are the responsibility of such parties. Douglas W. Harder accepts no responsibility for
damages, if any, suffered by any party as a result of decisions made or actions based on these
course slides for any other purpose than that for which it was intended.