1 Algorithm Analysis

We analyse algorithms, rather than

problems. A problem can be solved
with several algorithms, some are
more efficient than others.
Why need algorithm
analysis?
 Just writing a syntax-error-free program is not
enough. We need to know whether the
algorithm is correct or not, i.e. whether it can
give correct answers for the inputs.

 If the program is run on a large data set, the

running time becomes an issue. We want to
know how the algorithm performs when the
input size is large. The program may be
computationally inefficient, or may need lots
of memory. We analyze the resources that the
algorithm requires: memory, and computation
time. We can also compare algorithms without
 How to Analyse an Algorithm?
3

 To analyse an algorithm; determine the

amount of resources (such as time and
storage) necessary to execute it.
 Most algorithms are designed to work
with inputs of arbitrary length.
 Usually the efficiency or running time of
an algorithm is stated as a function
relating the input length to the number
of steps (time complexity) or storage
locations (space complexity).
Analysis of Algorithms
 Analysis of Algorithm refers to
calculating or guessing resources
needful for the algorithm.
 Resources means computer memory,
processing time etc.
 In all these factors, time is most
important because the program
developed should be fast enough.
Analysis of Algorithms
 Time complexity
 The amount of time that an algorithm needs to
run to completion
 Space complexity
 The amount of memory an algorithm needs to
run
 We will occasionally look at space
complexity, but we are mostly interested in
time complexity in this course.
 Thus in this course the better algorithm is
the one which runs faster (has smaller time
complexity)
6
Running Time of An
Algorithm
Running time of an algorithm is
stated as a function relating the
input length to the number of steps
(time complexity) or storage
locations (space complexity).
Factors Affecting Running
Time
 There are several factors affecting the
running time
 Computer
 Compiler
 Algorithm
 Input to the algorithm

 The content of the input affects the

running time typically, the input size
(number of items in the input) is the main
consideration
 E.g. sorting problem ⇒ the number of
Running time of an
Algorithm
 Running time of an algorithm depends upon the
input size and nature of input.
 Most algorithms transform input objects into
output objects

 The running time of an algorithm typically grows

with the input size
 idea: analyze running time as a function of

input size
Running Time of an
Algorithm
 Even on inputs of the same size, running
time can be very different
 Example: algorithm that finds the first
prime number in an array by scanning it left
to right
 Idea: analyze running time in the
 Best case
 Worst case
 Average case
Finding Running Time of an
Algorithm
 Running time is measured by number of
steps/primitive operations performed

 We use RAM (Random Access Model), in which

each operation (e.g. +, -, x, /,=) and each
memory access take one run-time unit. Loops and
functions can take multiple time units.

 Count the number of basic operations of an

algorithm
 Read, write, compare, assign, jump, arithmetic

operations (increment, decrement, add,

subtract, multiply, divide), open, close, logical
Example

 Lines 1 and 4 count for one unit each.

 Line 3: executed N times, each time four units.
 Line 2: (1 for initialization, N+1 for all tests, N for all
increments) total 2N + 2.
 total time units: 6N + 4. (⇒ O(N), will be discussed
later.)
Growth Rate of the Algorithm
Running Time
 Exact running time f(N) is difficult to find for some cases.
It is easier to find the upper and lower bounds of f(N).
 Now, we find the orders of growth of f(N) for large N.
13
Asymptotic Notation - Big-
Oh
Big Oh notation describes the
limiting behaviour of a function
when the argument tends towards a
particular value or infinity, usually in
terms of simpler functions
Asymptotic Notation - Big-Oh

 If f(N) and g(N) are two complexity functions, we

say
f(N) = O(g(N))
(read "f(N) is order g(N)", or "f(N) is big-O of
g(N)")

 There are positive constants c and n0 such that

f(N) ≤ c g(N) when N ≥ n0
 The growth rate of f(N) is less than or equal to the
growth rate of g(N). In other words, f(N) grows no
faster than g(N) for large N.
 g(N) is an upper bound on f(N).
Big-Oh Example
 Let f(N) =2N 2 Then
 N 4 ) (loose bound)
f(N) = O(
 N 3 ) (loose bound)
f(N) = O(
 N 2 ) (It is the best answer and the
f(N) = O(
bound is asymptotically tight.)

2
 N
O( ): reads “order N-squared” or “Big-
Oh N-squared”.
Big-Oh Notation Rules
 When considering the growth rate of a function
using Big-Oh notation,
 we ignore the lower order terms
 We ignore the coefficients of the highest-order term; and
 we don’t need to specify the base of logarithm
 Note that changing the base from one constant to
another changes the value of the logarithm by only a
constant factor

 If T1(N) = O(f(N)) and T2(N) = O(g(N)), then

 T1(N) + T2(N) = O(f(N) + g(N))
or max(O(f(N)), O(g(N))),
 T1(N) * T2(N) = O(f(N) * g(N))
Big-Oh Example (2)
// Input: int A[N], array of N integers
// Output: Sum of all numbers in array A

int Sum(int A[], int N){

int s=0; 1
for (int i=0; i< N; i++)
2 3 4
s = s + A[i];
5 6 7
return s;
} 1,2,8: Once
8 3,4,5,6,7: Once per each iteration
of for loop, N iteration
Total: 5N + 3
The complexity function of the
algorithm is : f(N) = 5N +3
Big-Oh Example (2)

 Estimated running time for different values of N:

N = 10 => 53 steps
N = 100 => 503 steps
N = 1,000 => 5003 steps
N = 1,000,000 => 5,000,003 steps
As N grows, the number of steps grow in linear
proportion to N for this function “Sum”

 What Dominates in Previous Example?

What about the +3 and 5 in 5N+3?
 As N gets large, the +3 becomes insignificant
 5 is inaccurate, as different operations require varying amounts
of time and also does not have any significant importance
Asymptotic Complexity of Example

 What term in the previous complexity function dominates?

 What about the 5 in 5n+3? What about the +3?
 As N gets large, the +3 becomes insignificant
 The 5 is inaccurate as different operations require varying
amounts of time
 What is fundamental is that the time is linear in N.

 Asymptotic Complexity: As N gets large, ignore all lower

order terms and concentrate on the highest order term
only:
 Drop lower order terms such as +3
 Drop the constant coefficient of the highest order term such as
5.
More Big-Oh Example
2
 N 2 N
/ 2 – 3N = O( )
(We throw away leading constants and low-order terms.)

 1+4N = O(N)
3
 7N 2 + 10N + 3 =NO(
2 N O(
)= ) (loose bound)
 log10 N  log 2 N / log 2 10  O(log 2 N)  O(log N)

 log N + N = O(N)
To prove that mathematically: Example:

2 2
Consider f(n) = n – 3n + 6. Then f(n)n= O(
3
2 )n= O( )
n0
Try some values of c and find out the corresponding which satisfies
the condition.
2
n
1. f(n) = O( )
(a) Suppose2 we choose c =
2 2:
n n
2 – 3n + 6 ≤ 2
– 3n + 6 ≤ 0
n≥2 n0
So we can see that if we choose c = 2 and = 2, the condition is satisfied.
(b) Suppose
n 2 we choose c n=2 3:
2 n 2+ 6 ≤ 3
– 3n
+ 3n - 6 ≥ 0
n ≤ -4.37(ignored) or n ≥n1.37
0

So we can see that if we choosenc0 = 3 and = 1.37, the condition is satisfied .

• There are other values of c and which satisfy the condition .
To prove that mathematically: Example:

2. f(n) = O(n3)
Suppose we choose c = 1:
2 n 2 + 6 ≤ n3
– 3n
3
n+2
2
–n3n + 6 ≤ 0
n≥2
So we can see that if we choose c n
=0 1 and = 2, the
condition is satisfied.
* There are other values of c and n0 which satisfy the
condition.
Performance Classification
f(n) Classification
1 Constant: run time is fixed, and does not depend upon n. Most instructions are
executed once, or only a few times, regardless of the amount of information being
processed
log n Logarithmic: when n increases, so does run time, but much slower. Common in
programs which solve large problems by transforming them into smaller problems.

n Linear: run time varies directly with n. Typically, a small amount of processing is
done on each element.

n log n When n doubles, run time slightly more than doubles. Common in programs which
break a problem down into smaller sub-problems, solves them independently, then
combines solutions
n2 Quadratic: when n doubles, runtime increases fourfold. Practical only for small
problems; typically the program processes all pairs of input (e.g. in a double nested
loop).
n3 Cubic: when n doubles, runtime increases eightfold

2n Exponential: when n doubles, run time squares. This is often the result of a natural,
“brute force” solution.
Effect of Input Size

What happens if we double the input size N?

Nlog2N 5N N log2N N2 2N
8 3 40 24 64 256
16 4 80 64 256 65536
32 5 160 160 1024 ~109
64 6 320 384 4096 ~1019
128 7 640 896 16384 ~1038
256 8 1280 2048 65536 ~1076