Analysis of Algorithms

Functions

NYU Abu Dhabi

1/8
Functions Used
I Constant Function:
f (n) = c where n ∈ N and c is fixed.

– C takes some specific value such as c = 5 or c = 103 etc.

– f (n) = c models the number of steps needed to carry out most
machine language primitives.

I Logarithmic Function:
f (n) = logb n where n > 0 and b > 1.

– b - constant base of the logarithm

– x = logb n if and only if b x = n
– log3 27 = 3 and log4 (64) = 3
– by convention: log n = log2 n.
2/8
Properties of the logarithmic function

1. logb ac = logb a + logb c

2. logb a/c = logb a − logb c
3. logb ac = c logb a
4. logb a = (logd a)/(logd b) [change of base formula].
5. b logd a = alogd b
– log n3 = 3 log n
– log 2n = n log 2 = n
– log4 n = log n/ log 4 = log n/2
– 2log n = nlog 2 = n1 = n

3/8
Functions Used

I Linear Function:
f (n) = n, n ∈ N.

– this is the best time we can hope for when n elements are not
in memory yet. Need at least linear time to “read in” the input.

I N-log-N Function:
f (n) = n log n where n > 0

– This function grows a little faster than the linear and much
slower than the quadratic!
– There are O(n log n) sorting algorithms (mergesort, heapsort).

4/8
Functions Used

I Quadratic Function:
f (n) = n2
– Usually models the behavior of two nested for-loops where the
loop-control variables increase to n.
– For example if the number of operations in the inner loop
increases by 1 at each iteration, then we have:
1 + 2 + 3 + ... + (n − 1) + n = n(n + 1)/2 operations

I Cubic Function:
f (n) = n3
– Appears rather infrequently but it does. Classic example:
All-Pairs Shortest Paths.

5/8
Functions Used
I Polynomials:
f (n) = a0 + a1 n + a2 n2 + a3 n3 + · · · + ad nd
where a0 , a1 , a2 . . . ad are constants, with ad 6= 0, termed
coefficients
– A computation that takes at most polynomial time in the
input size is termed efficient.
– d indicates the degree of polynomial.
– All previous functions are bounded by a polynomial.
I Exponential Function:
f (n) = b n where b > 1 is the base and n the exponent
– Hallmark of inefficiency, infeasible computation already for
small n.

– Geometric Sums: 1 + a + a2 + · · · + an = (an+1 − 1)/(a − 1)

– If a = 2 → 1 + 2 + 4 + 8 + ... + 2n−1 = 2n − 1
6/8
Analysis of Algorithms
Two main complexity measures of interest: time and space
(memory). We focus on time.
Running time may be influenced by several parameters such as:
CPU clock, RAM available, PL used etc.

These factors are implementation-dependent and do not help us

compare the quality of different algorithms for the same problem.
Algorithm design comes first, implementation second.

We characterize an algorithm based on its running time T (n) as

a function of the input size n.

Understanding the structure of the problem is what drives the

design of an efficient algorithm. Being an expert in C++ or Python
is irrelevant at the design stage.
7/8
How to measure the running time
1 Try to estimate the number of steps (basic primitive
operations) executed by the algorithm.
2 Each basic (primitive) operation is assumed to take one unit
of time.
3 Consider the properties of all possible inputs in the process!

Point 1 suggests: obtain an upper bound on the number of steps.

The exact number varies anyway based on the input.

Point 2 suggests: abstract away implementation details.

Point 3 suggests: go out of your way to identify the weaknesses of

your design.
8/8