DS Notes How Are You

Data Structures and Algorithms Made Easy
Chapter 2
www.CareerMonk.com
ANALYSIS OF ALGORITHMS
Introduction
This chapter will create the base for remaining all chapters. The objective of this chapter is to
tell you the importance of analysis of algorithms, their notations, relationships and solving as
many problems as possible. We first concentrate on understanding the importance of analysis
and then slowly move towards analyzing the algorithms with different notations and finally,
the problems. After completion of this chapter you will get the confidence in finding the
complexities of any given algorithm (especially recursive functions).
What is algorithm?
Just to understand better, let us consider the problem of preparing an omelet. For preparing
omelet, general steps which we follow are:
1) Get the frying pan.
2) Get the oil.
a. Do we have oil?
i. If yes, put it in the pan.
ii. If no, do we want to buy oil?
1. If yes, then go out and buy.
2. If no, we can terminate.
3) Turn on the stove.
4) Etc...
The above steps says that, for a given problem (in this case, preparing an omelet), giving step
by step procedure for solving it. So, the definition of algorithm can be given as:
An algorithm is the stepstep-byby-step instructions to a given problem.
One important thing to remember while writing the algorithms is, we do not have to prove
each step. Also, all algorithms involve the basic elements which we have discussed in
chapter.
66 Analysis of Algorithms | CareerMonk Publications
www.CareerMonk.com
Why Analysis of Algorithms?

lgorithms?
If we want to go from city to city . There can be many ways of doing this: by flight,
by bus, by train and also by cycle. Depending on the availability and convenience we choose
the one which suits us. Similarly, in computer science there can be multiple algorithms exist
for solving the same problem (for example, sorting problem has lot of algorithms like
insertion sort, selection sort, quick sort and many more). Algorithm analysis helps us to
determine which of them is efficient in terms of time and space consumed.
Goal of Analysis of Algorithms?

The goal of is to compare algorithms (or solutions) mainly in terms
of running time but also in terms of other factors (e.g., memory, developer's effort etc.)
What is Running Time Analysis?

nalysis?
Its the process of determining how processing time increases as the size of the problem
increases. Input size is number of elements in the input and depending on the problem type
the input may be of different types. In general, we encounter the following types of inputs.
Size of an array
Polynomial degree
Number of elements in a matrix
Number of bits in the binary representation of the input
Vertices and edges in a graph
How to Compare Algorithms?

To compare algorithms, let us define some .
Execution times?
as execution times are specific to a particular computer.
Number of statements executed?
since the number of statements varies with the programming language
as well as the style of the individual programmer.
Ideal Solution?
Solution
www.CareerMonk.com
Let us assume that we expressed running time of given algorithm as a function of the input
size (i.e., ). We can compare these different functions corresponding to running times
and this kind of comparison is independent of machine time, programming style, etc..
What is Rate of Growth?

The rate at which the running time increases as a function of input is called rate of growth.
Let us assume that you went to a shop for buying a car and a cycle. If your friend sees you
there and asks what you are buying then in general we say . This is because cost
of car is too big compared to cost of cycle ( approximating the cost of cycle to cost of car).
Total Cost = cost_of_car + cost_of_cycle
Total Cost cost_of_car (approximation)
For the above example, we can represent the cost of car and cost of cycle in terms of function
and for a given function we ignore the low order terms that are relatively insignificant (for
large value of input size, ). As an example in the below case, , 2" , 100 and 500 are the
individual costs of some function and we approximate it to . Since, is the highest rate of
growth.
& 2" & 100 & 500
Commonly used Rate of Growths

Below is the some list of rate of growths which come across in remaining chapters.
Time complexity Name
'1
Example
Constant
Adding an element to the front of a linked list
'
Logarithmic
Finding an element in a sorted array
'
Linear
Finding an element in an unsorted array
'
Linear Logarithmic Sorting n items by divide-and-conquer-Mergesort
'
'2)
Quadratic
Shortest path between two nodes in a graph
Cubic
Exponential
Matrix Multiplication
The Towers of Hanoi problem
'
"
(
Below diagram shows the relationship between different rates of growth.
www.CareerMonk.com

2"
!
D
e
c
r
e
a
s
i
n
g
4)
2)
"
log
log !

2./0 )
"
1
R
a
t
e
s
O
f
G
r
o
w
t
h
log log
1
Types of Analysis
Assume that we have an algorithm for a problem and want to know on what inputs the
algorithm is taking less time (performing well) and on what inputs the algorithm is taking
huge time.
We have already seen that an algorithm can be represented in the form of an expression.
That means we represent the algorithm with multiple expressions: one for case where it is
taking the less time and other for case where it is taking the more time. In general the first
www.CareerMonk.com
case is called the best case for the algorithm and second case is called the worst case for the
algorithm.
To analyze an algorithm we need some kind of syntax and that forms the base for asymptotic
analysis/notation. There are three types of analysis:
Worst case
o Defines the input on which the algorithm is taking huge time.
o Input is the one for which the algorithm runs the slower.
Best case
o Defines the input on which the algorithm is taking lowest time.
o Input is the one for which the algorithm runs the fastest.
Average case
o Provides a prediction about the running time of the algorithm
o Assumes that the input is random
Lower Bound <= Average Time <= Upper Bound
For a given algorithm, we can represent best case, worst case, average case analysis in the
form of expressions. As an example, let be the function which represents the given
algorithm. Then, the following expression for best case and worst case.
= " & 500, for worst case
= & 100 & 500 , for best case
Similarly, for average case too. The expression defines the inputs with which the algorithm
takes the average running time (or memory).
Asymptotic notation?
Having the expressions for best case, average case and worst case, to find the upper bounds
and lower bounds for each of them: for all the three cases we need to identify the upper
bound, lower bounds. In order to represent these upper bound and lower bounds we need
some syntax and that creates the base for following discussion.
For the below discussions, let us assume that the given algorithm is represented in the form
of function .
BigBig-O notation
www.CareerMonk.com
This notation gives the upper bound of the given function. Generally we represent it
as = '. That means, at larger values of , the upper bound of is .
For example, if = & 100" & 10 & 50 is the given algorithm, then is .
That means gives the maximum rate of growth for at larger values of .
Now, let us see the ' Gnotation with little more detail. ' Gnotation defined as ' =
H: there exist positive constants and nJ such that 0 K K for all L J M.
is an asymptotic tight upper bound for . Our objective is to give some rate of
growth which is greater than given algorithms rate of growth .
In general, we do not consider lower values of . That means the rate of growth at lower
values of is not important. In the below figure, J is the point from which we consider
the rate of growths for a given algorithm. Below J the rate of growths may be different.
Rate of Growth

Input Size,
J
BigBig-O Visualization
'
3 & 100,
100, 2
G 1, 3, .
'1
100,1000,
200,1,20, .
'
5, 3 G 100,
2 G 1, 100, 100,
.
'"
, 5 G 10, 100,
" G 2 & 1, 5,
.
"
' is the set of functions with smaller or same order of growth as . For example,
' " includes '1, ', ' etc..
www.CareerMonk.com
Note: Analyze the algorithms at larger values of only. What this says is, below J we do
not care for rates of growth.
BigBig-O examples
ExampleExample-1 Find upper bound for = 3 & 8
Solution: 3 & 8 K 4, for all L 1
3 & 8 = ' with c = 4 and nJ = 8
ExampleExample-2 Find upper bound for = " & 1

Solution: " & 1 K 2" , for all L 1
" & 1 = '" with c = 2 and nJ = 1
ExampleExample-3 Find upper bound for = & 100" & 50
Solution: & 100" & 50 K 2 , for all L 1

& 100" & 50 = ' with c = 2 and nJ = 100
ExampleExample-4 Find upper bound for = 2( G 2"
Solution: 2( G 2" K 2( , for all L 1

2( G 2" = '2( with c = 2 and J = 1
ExampleExample-5 Find upper bound for =

Solution: K " , for all L 1
= '" with c = 1 and J = 1
ExampleExample-6 Find upper bound for = 410

Solution: 410 K 410, for all L 1
100 = '1 with c = 1 and nJ = 1
No uniqueness?
There is no unique set of values for J and in proving the asymptotic bounds. Let us
consider, 100 & 5 = '" . For this function there are multiple J and .
www.CareerMonk.com
Solution1: 100 & 5 K 100 & = 101 K 101" for all L 5, J = 5 and =
101 is a solution.
Solution2: 100 & 5 K 100 & 5 = 105 K 105" for all L 1, J = 1 and
= 105 is also a solution.
OmegaOmega- notation
Similar to above discussion, this notation gives the tighter lower bound of the given
algorithm and we represent it as = . That means, at larger values of , the
tighter lower bound of is .
For example, if = 100" & 10 & 50, is " .
The notation as be defined as = H: there exist positive constants and
nJ such that 0 K K for all L 0 M. is an asymptotic lower bound for
. is the set of functions with smaller or same order of growth as .
Rate of Growth
Examples

J

Input Size,
ExampleExample-1 Find lower bound for = 5"
Solution: , nJ Such that: 0 5" 5 " = 1 and nJ = 1

5" = with = 1 and J = 1
ExampleExample-2 Prove = 100 & 5 R "
Solution: c, nJ Such that: 0 " 100 & 5

100 & 5 100 & 5 1 = 105
www.CareerMonk.com

" 105 105 0
Since is positive 105 0 105/
Contradiction: cannot be smaller than a constant
ExampleExample-3 = 2, ( = " , =
ThetaTheta- notation
This notation decides whether the upper and lower bounds of a given function are same or
not. The average running time of algorithm is always between lower bound and upper
bound. If the upper bound (') and lower bound () gives the same result then V notation
will also have the same rate of growth. As an example, let us assume that = 10 & is
the expression. Then, its tight upper bound is '. The rate of growth in best case
is = '. In this case, rate of growths in best case and worst are same. As a result, the
average case will also be same.
Note: For a given function (algorithm), if the rate of growths (bounds) for ' and are not
same then the rate of growth V case may not be same.
c"
Rate of Growth

cY
Input Size,
J
Now let us see the definition of V notation. It is defined as X = H: there exist
positive constants Y , " and nJ such that 0 K Y K K " for all L 0 M.
is an asymptotic tight bound for . V is the set of functions with the same
order of growth as .
Examples
ExampleExample-1 Find bound for =
)Z
"
G"
www.CareerMonk.com

Solution:
)Z
[
)Z
"
)Z
"
G K " , for all, n 1

)
"
G " = " with Y = 1/5, Y = 1 and nJ = 1

)
ExampleExample-2 Prove R V"
Solution: Y " K K " " only holds for: K 1/Y

R "
ExampleExample-3 Prove 6( V"
Solution: Y n2 6( " " only holds for: K " /6

6( R "
ExampleExample-4 Prove R V
Solution: Y K K " log " L
./0 )
, L n0 Impossible
Important Notes
For each of the analysis (best case, worst case and average) we try to give upper bound (')
and lower bound () and average running time V. Also, from the above examples, it should
also be clear that, for a given function (algorithm) getting upper bound (') and lower bound
() and average running time (V) may not be possible always.
For example, if we are discussing the best case of an algorithm, then we try to give upper
bound (') and lower bound () and average running time (V).
In the remaining chapters we generally concentrate on upper bound (') because knowing
lower bound () of an algorithm is of no practical importance and we use notation if upper
bound (') and lower bound () are same.
Why we call Asymptotic Analysis?

Analysis?
From the above discussion (for all the three notations: worst case, best case and average case),
we can easily understand that, in every case for a given function we are trying to find
other function which approximates at higher values of . That means, is also
a curve which approximates at higher values of . In mathematics we call such curve as
www.CareerMonk.com
] . In other terms, is the asymptotic curve for . For this reason, we
call our analysis as ] .
Guidelines for asymptotic analysis?

There are some general rules to help us in determining the running time of an algorithm.
Below are few of them.
1) Loops
Loops: The running time of a loop is, at most, the running time of the statements inside
the loop (including tests) multiplied by the number of iterations.
// executes times
for (i=1; i<=n; i++)
{
m = m + 2; // constant time, c
}
Total time = a constant ^ = = '.
2) Nested loops:
loops: Analyze from inside out. Total running time is the product of the sizes of
all the loops.
//outer loop executed n times
for (i=1; i<=n; i++)
{
// inner loop executed n times
for (j=1; j<=n; j++)
{
k = k+1; //constant time
}
}
Total time = ^ ^ = " = '" .
3) Consecutive statements:
statements: Add the time complexities of each statement.
x = x +1; //constant time
// executed n times
for (i=1; i<=n; i++)
www.CareerMonk.com
{
m = m + 2; //constant time
}
//outer loop executed n times
for (i=1; i<=n; i++)
{
//inner loop executed n times
for (j=1; j<=n; j++)
{
k = k+1; //constant time
}
}
Total time = J & Y & " " = ' " .
4) If
If--thenthen-else statements:
statements: Worst-case running time: the test, plus either the then part or
the else part (whichever is the larger).
//test: constant
if (length ( ) != otherStack. length ( ) )
{
return false; //then part: constant
}
else
{
// else part: (constant + constant) * n
for (int n = 0; n < length( ); n++)
{
// another if : constant + constant (no else part)
if (!list[n].equals(otherStack.list[n]))
//constant
return false;
}
}
Total time = J & Y & " & ( _ = '.
5) Logarithmic complexity:
complexity: An algorithm is ' if it takes a constant time to cut the
problem size by a fraction (usually by ).
www.CareerMonk.com
As an example let us consider the following program:

for (i=1; i<=n;)
{
i = i*2;
}
If we observe carefully, the value of i is doubling every time. That means, initially = 1,
in next step = 2, and in subsequent steps = 4, 8 and so on.
Let us say the loop is executing some a times. That means at a G step 2b = n and we
come out of loop. So, if we take logarithm on both sides,
c2b d =
2 =
= //if we assume base-2
So the total time = '.
Note: Similarly, for the below case also the worst case rate of growth is '. That means,
the same discussion holds good for decreasing sequence also.
for (i=n; i<=1;)
{
i = i/2;
}
Another example algorithm (binary search): finding a word in a dictionary of pages
Look at the centre point in the dictionary

Is word towards left or right of centre?
Repeat process with left or right part of dictionary until the word is found
Properties of notations
Transitivity: = and = = . Valid for O
and also.
Reflexivity: = . Valid for O and also.
Symmetry: = if and only if = .
www.CareerMonk.com
Transpose symmetry: = O if and only if = .
Commonly used logarithms and summations

f g = f
= h)
f = f &
i = i
=
f
= f

Logarithms
jklm = f jklm pq =

n
n
jklo
m
jklo
Arithmetic series
)
r s = 1 & 2 & t& =
uvY
Geometric series
& 1
2
xn{1 G 1
r = 1 & x & x & =
x R 1
xG1
n
kv1
Harmonic series
)
1
1
1
r = 1 & & & log
s
2

ivY
)
Other important formulae

r log s
ivY
)
r s | = 1| & 2| & t & |
ivY
1
|{Y
]&1
Master Theorem for Divide and Conquer

In all efficient divide and conquer algorithms we will divide the problem into subproblems,
each of which is some part of the original problem, and then perform some additional work
to compute the final answer. As an example, if we consider merge sort [for details, refer
} chapter], it operates on two problems, each of which is half the size of the original,
and then uses ' additional work for merging. This gives the running time equation:
www.CareerMonk.com

~ = 2~ & '
2
The following theorem can be used to determine the running time of divide and conquer
algorithms. For a given program or algorithm, first we try to find the recurrence relation for
the problem. If the recurrence is of below form then we directly give the answer without
fully solving it.
If the reccurrence is of the form ~ = ~ & Xs| , where L 1, 1, s L 0
)
p
and ] is a real number, then we can directly give the answer as:
If a
2) If a =
a.
b.
c.
3) If a <
a.
b.
1)
b , then ~ = Xcn./0 d

b
If ] G1, then ~ = Xcn./0 |{Y d
If ] = G1, then ~ = Xcn./0 loglognd
If ] < G1, then ~ = Xcn./0 sd

b
If ] L 0, then ~ = Xcn | d
If ] < 0, then ~ = 'cn d
Problems on Master Theorem for Divide and Conquer

For each of the following recurrences, give an expression for the runtime ~ if the
recurrence can be solved with the Master Theorem. Otherwise, indicate that the Master
Theorem does not apply.
ProblemProblem-1
~ = 3~ /2 & "
ProblemProblem-2
~ = 4~ /2 & "
ProblemProblem-3
~ = ~/2 & 2)
ProblemProblem-4
~ = 2) ~/2 & )
Solution:
Solution: ~ = 3~ /2 & " = ~ = X" (Master Theorem Case 3.a)
Solution:
Solution: ~ = 4~ /2 & " = ~ = X" (Master Theorem Case 2.a)
Solution:
Solution: ~ = ~/2 & 2) = X2) (Master Theorem Case 3.a)
Solution:
Solution: ~ = 2) ~/2 & ) => Does not apply ( is not constant)
ProblemProblem-5
~ = 16~/4 &
ProblemProblem-6
~ = 2~/2 &
ProblemProblem-7
~ = 2~/2 & /
ProblemProblem-8
~ = 2~ /4 & J.[Y
ProblemProblem-9
~ = 0.5~/2 & 1/
www.CareerMonk.com
Solution:
Solution: ~ = 16~ /4 & = ~ = X" (Master Theorem Case 1)
Solution:
Solution: ~ = 2~/2 & = ~ = X" (Master Theorem Case 2.a)
Solution:
Solution: ~ = 2~/2 & / = ~ = Xloglogn (Master Theorem Case
2.b)
Solution:
Solution: ~ = 2~/4 & J.[Y = ~ = 'J.[Y (Master Theorem Case 3.b)
Solution:
Solution: ~ = 0.5~/2 & 1/ = Does not apply ( < 1)
ProblemProblem-10 ~ = 6~ /3 & "
Solution:
Solution: ~ = 6~/3 & " = ~ = X" (Master Theorem Case 3.a)
ProblemProblem-11 ~ = 64~/8 G "
Solution:
Solution: ~ = 64~/8 G " = Does not apply (function is not positive)
ProblemProblem-12 ~ = 7~/3 & "
Solution
Solution: ~ = 7~/3 & " = ~ = X" (Master Theorem Case 3.as)
ProblemProblem-13 ~ = 4~ /2 &
Solution:
Solution: ~ = 4~/2 & = ~ = X" (Master Theorem Case 1)
ProblemProblem-14 ~ = ~/2 & 2 G
Solution:
Solution: ~ = ~/2 & 2 G = Does not apply. We are in Case 3, but the
regularity condition is violated. (Consider = 2k, where s is odd and arbitrarily large. For
any such choice of , you can show that = 3/2, thereby violating the regularity
condition.)
ProblemProblem-15 ~ = 16~ /4 & !

www.CareerMonk.com
Solution:
Solution: ~ = 16~ /4 & ! = ~ = X! (Master Theorem Case 3.a)
Solution:
Solution: ~ = 2~ /2 & = ~ = X (Master Theorem Case 1)
Solution:
Solution: ~ = 3~ /2 & = ~ = X jkl( (Master Theorem Case 1)
Solution:
Solution: ~ = 3~ /3 & = ~ = X (Master Theorem Case 1)
Solution:
Solution: ~ = 4~ /2 & = ~ = X " (Master Theorem Case 1)
Solution:
Solution: ~ = 3~ /4 & = ~ = X (Master Theorem Case 3.a)
ProblemProblem-21 ~ = 3~ /3 & /2
Solution:
olution: ~ = 3~ /3 & /2 = ~ = X (Master Theorem Case 2.a)
Master Theorem for Subtract and Conquer Recurrences

Let ~ be a function defined on positive , and having the property
~ =
c,
aTn G b & fn,
if K 1
if 1
for some constants , 0, 0, s L 0, and function . If is in 'i , then
Ocn d,
{Y
~ = Ocn d,

O
n
a
,
if a < 1
if a = 1
if a 1
Variant of subtraction and conquer master theorem

theorem
The solution to the equation ~ = ~ & ~1 G & , where 0 < < 1 and
0 are constants, is '.
www.CareerMonk.com
Problems on Algorithms
Algorithms Analysis
Note: From the following problems, try to understand in what cases we get different
complexities (', ', ' etc..).
ProblemProblem-22 Find the complexity of the below recurrence:

~ =
3~ G 1, 0,
1,

Solution: Let us try to solve this function with substitution.

~ = 3~ G 1
~ = 3c3~ G 2d = 3" ~ G 2

~ = 3" 3~ G 3
.
.
~ = 3) ~ G = 3) ~0 = 3)
This clearly shows that the complexity of this function is '3) .
Note: We can directly use the } master theorem for this problem.
ProblemProblem-23 Find the complexity of the below recurrence:

~ =
2~ G 1 G 1, 0,
1,

Solution:
Solution: Let us try to solve this function with substitution.
~ = 2~ G 1 G 1
~ = 22~ G 2 G 1 G 1 = 2" ~ G 2 G 2 G 1
~ = 2" 2~ G 3 G 2 G 1 G 1 = 2( ~ G 4 G 2" G 2Y G 2J

~ = 2) ~ G G 2)Y G 2)" G 2)( . 2" G 2Y G 2J
~ = 2) G 2)Y G 2)" G 2)( . 2" G 2Y G 2J
~ = 2) G 2) G 1 : 2)Y & 2)" & t & 2J = 2)

~ = 1
So the complexity is '1. Note that while the recurrence relation looks exponential the
solution to the recurrence relation here gives a different result.
www.CareerMonk.com
ProblemProblem-24 What is the running time of the following function (specified as a function of
the input value n)
void Function(int n)
{
int i=1 ;
int s=1 ;
while( s n)
{
i++ ;
s= s+i ;
print(\*");
}
}
Solution:
Solution: Consider the comments in below function:
void Function (int n)
{
int i=1 ;
int s=1 ;
// s is increasing not at rate 1 but i
while( s n)
{
i++ ;
s= s+i ;
print(*");
}
}
We can define the terms according to the relation = Y & i. The value of increases
by one for each iteration. So the value contained in at the iteration is the sum of the
first positive integers. If s is the total number of iterations taken by the program, then the
while loop terminates once.
1 & 2 & ...& s =
ss & 1
L s = '.
2
ProblemProblem-25 Find the complexity of the function given below.

{
www.CareerMonk.com
int i, count =0;;

for(i=1; i*i<=n; i++)
count++;
}
{
int i, count =0;;
for(i=1; i*i<=n; i++)
count++;
}
In the above function the loop will end, if " K ~ = '. The reasoning is same
as that of Problem-24.
ProblemProblem-26 What is the complexity of the below program:

void function(int n)
{
int i, j, k , count =0;
for(i=n/2; i<=n; i++)
for(j=1; j + n/2<=n; j= j++)
for(k=1; k<=n; k= k * 2)
count++;
}
Solution:
Solution: Let us consider the comments in the following function.
{
//outer loop execute n/2 times
for(i=n/2; i<=n; i++)
//Middle loop executes n/2 times
for(j=1; j + n/2<=n; j= j++)
//outer loop execute logn times
for(k=1; k<=n; k= k * 2)
count++;
}
The complexity of the above function is '" .
www.CareerMonk.com
ProblemProblem-27 What is the complexity of the below program:

{
for(i=n/2; i<=n; i++)
for(j=1; j<=n; j= 2 * j)
for(k=1; k<=n; k= k * 2)
count++;
}
Solution:
{
//outer loop execute n/2 times
for(i=n/2; i<=n; i++)
//Middle loop executes logn times
for(j=1; j<=n; j= 2 * j)
//outer loop execute logn times
for(k=1; k<=n; k= k*2)
count++;
}
The complexity of the above function is '" .
ProblemProblem-28 Find the complexity of the below program.

function( int n )
{
if ( n == 1 ) return ;
for( i = 1 ; i <= n ; i + + )
{
for( j= 1 ; j <= n ; j + + )
{
print(*" ) ;
break;
}
}
}
function( int n )
www.CareerMonk.com
{
//constant time
//outer loop execute n times
for( i = 1 ; i <= n ; i + + )
{
// inner loop executes only time due to break statement.
for( j= 1 ; j <= n ; j + + )
{
print(*" ) ;
break;
}
}
}
The complexity of the above function is '. Even though the inner loop is bounded by n,
but due to the break statement it is executing only once.
ProblemProblem-29 Write a recursive function for the running time ~ of the function function,
whose code is below. Prove using the iterative method that ~ = X" .
function( int n )
{
for( i = 1 ; i <= n ; i + + )
for( j = 1 ; j <= n ; j + + )
print(*" ) ;
function( n-3 );
}
Solution:
function (int n)
{
//constant time
//outer loop execute n times
for( i = 1 ; i <= n ; i + + )
//inner loop executes n times
for( j = 1 ; j <= n ; j + + )
www.CareerMonk.com
//constant time
print(*" ) ;
function( n-3 );
}
The recurrence for this code is clearly T = ~ G 3 & " for some constant 0
since each call prints out " asterisks and calls itself recursively on n - 3. Using the iterative
method we get:
~ = ~ G 3 & "
Using the } master theorem, we get ~ = X( .
ProblemProblem-30 Determine X bounds for the following recurrence relation:

~ = 2~ &
2
Solution:
Solution: Using Divide and Conquer master theorem, we get ' " .
ProblemProblem-31 Determine X bounds for the following recurrence
~ = ~/2 & ~/4 & ~/8 &

Solution:: Substituting in the recurrence equation, we get:
Solution

& 2 _ & 3 _ &
2
4
8
K s _ , where k is a constant.
~ K 1 _
ProblemProblem-32 Determine bounds for the following recurrence relation:

~ = ~/2 & 7
Solution:
Solution: Using Master Theorem we get X.
ProblemProblem-33 Prove that the running time of the code below is .
Read(int n);
{
int k = 1 ;
while( k < n )
k = 3k;
}
www.CareerMonk.com
Solution:
Solution: The while loop will terminate once the value of s is greater than or equal to the
value of . Since each loop the value of s is being multiplied by 3, if i is the number of
iterations, then s has the value of 3i after i iterations. That is the loop is terminated upon
reaching i iterations when 3 n i log ( , which shows that = .
ProblemProblem-34 Solve the following recurrence.
1,
= 1
~ =
~ G 1 & G 1, L 2
Solution: By iteration:
~ = ~ G 2 & G 1 G 2 & G 1
~ = ~1 & r G 1

vY
)
~ = ~1 & r G r

vY
"
vY
& 12 & 1 & 1

~ = 1 &
G
6
2
~ = (
Note: We can directly use the } master theorem for this problem.
ProblemProblem-35 Consider the following program:

Fib[n]
if (n==0) then return 0
else if (n==1) then return 1
else return Fib[n-1]+Fib[n-2]
Solution: The recurrence relation for running time of this program is
~ = ~ G 1 & ~ G 2 & .
Notice ~ has two recurrence calls indicating a binary tree. Each step recursively calls the
program for only reduced by 1 and 2, so the depth of the recurrence tree is '. The
number of leaves at depth is 2 since this is a full binary tree, and each leaf takes at least
'1 computation for the constant factor. So, the running time is clearly exponential in .
www.CareerMonk.com
ProblemProblem-36 Running time of following program?

function(n)
{
for( i = 1 ; i n ; i + + )
for( j = 1 ; j n ; j+ = i )
print( * ) ;
}
Solution:
function (n)
{
//this loop executes n times
for( i = 1 ; i n ; i + + )
//this loop executes j times with j increase by the rate of i
for( j = 1 ; j n ; j+ = i )
print( * ) ;
}
Its running time is ^ = 'Z since the inner loop is same as that of G
24].
ProblemProblem-37 What is the complexity of )vY ?
Solution:
Solution: Using the logarithmic property that, f = f & . We can see that this
problem is equivalent to
)
r = 1 & 2 & t &

vY
= 1 ^ 2 ^ ^
= !
K )
K
This shows that that the time complexity = '.

function of the input value )? First write the recurrence formula and then find its
complexity.
function(int n)
{
if (n 1)
ProblemProblem-38 What is the running time of the following recursive function (specified as a
www.CareerMonk.com

return ;
int i;
for (i=1 ; i 3; i++ )
f( );
}
Solution:
olution: Consider the comments in below function:
function (int n)
{
//constant time
if (n 1)
return ;
int i;
//this loop executes with recursive loop of value
for (i=1 ; i 3; i++ )
f(();
We can assume that for asymptotical analysis k = k = k for every integer s L 1. The
)
recurrence for this code is ~ = 3~ ( & X1.
Using master theorem, we get ~ = X.
function of the input value ? First write a recurrence formula, and show its solution
using induction.
function(int n)
{
if (n 1)
return;
for (i=1 ; i 3 ; i++ )
function (n 1).
}
ProblemProblem-39 What is the running time of the following recursive function (specified as a
Solution:
function (int n)
{
//constant time
if (n 1)
www.CareerMonk.com
return;
//this loop executes 3 times with recursive call of n-1 value
for (i=1 ; i 3 ; i++ )
function (n 1).
}
The requires constant time (call it '1). With the ], we neglect the
loop overhead and only count the three times that the function is called recursively. This
implies a time complexity recurrence
~ = , K 1;
= & 3~ G 1, 1.
Now we use repeated substitution to guess at the solution when we substitute k times:
~ = & 3~ G 1
Using the } master theorem, we get ~ = X3) .
ProblemProblem-40 Write a recursion formula for the running time ~ of the function f, whose
code is below. What is the running time of , as a function of ?
function (int n)
{
if (n 1)
return;
int i = 1 ;
for(i = 1; i < n; i + +)
print(*);
function ( 0.8n ) ;
}
Solution:
function (int n)
{
//constant time
if (n 1)
return;
//constant time
int i = 1 ;
// this loop executes times with constant time loop
www.CareerMonk.com

for(i = 1; i < n; i + +)
print(*);
//recursive call with 0.8n
function ( 0.8n ) ;
}
The recurrence for this piece of code is ~ = ~.8 & '
4
~ = ~ & X
5
4
~ = ~ & X
5
Applying master theorem, we get ~ = '.
ProblemProblem-41 Find the complexity of the following recurrence.

~ = 2~ &
Solution: The given recurrence is not in the master theorem form. So we try to convert this
master theorem format. For that let use assume that = 2
If we apply logarithm on both side, we get, = 2 =

Now, the given function becomes,
~ = ~2 = 2~c2 d & = 2~ 2 Z & .
To make it simple we assume } = ~2 } = ~2 Z } = 2} &
"
Applying the master theorem would result } = '

If we substitute = back, ~ = } = ' .

~ = ~ & 1
Solution: We apply the same logic as that of Problem-41 and we get

} = } & 1
2
"
www.CareerMonk.com

Applying the master theorem would result } = '.
Substituting = , gives ~ = } = '.

~ = 2~ & 1
Solution: Applying the logic of Problem-41, gives

} = 2} & 1
2
Using the master theorem results } = 'cjklZ d = '.

Substituting = gives ~ = '.
Z
ProblemProblem-44 Find the complexity of the below function.

int Function (int n)
{
if (n <= 2)
return 1;
else
return (Function (floor(sqrt(n))) + 1);
}
int Function (int n)
{
//constant time
if (n <= 2)
return 1;
else
// executes + 1 times
return (Function (floor(sqrt(n))) + 1);
}
For the above function, the recurrence function can be given as: ~ = ~ & 1. And,
this is same as that of Problem-42.
www.CareerMonk.com
ProblemProblem-45 Analyze the running time of the following recursive procedure as a function of
n. You may assume that each assignment or division takes unit time.
{
if ( n < 2 )
return;
else
counter = 0;
for i = 1 to 8 do
)
function ( );
"
for I =1 to ( do
counter = counter + 1;
}
Solution: Consider the comments in below function and let us refer to the running time of
function (n) as T(n).
{
//constant time
if ( n < 2 )
return;
else
counter = 0;
// this loop executes 8 times with n value half in every call
for i = 1 to 8 do
)
function ( );
"
// this loop executes ( times with constant time loop

for I =1 to ( do
counter = counter + 1;
}
~ can be defined as follows:
~ = 1 < 2,

= 8~ & ( & 1 .
2
Using the master theorem gives, ~ = XjklZ log = ( log n.
www.CareerMonk.com

temp = 1
repeat
for i = 1 to n
temp = temp + 1;
)
n = ;
until n 1
"

//const time
temp = 1
repeat
// this loops executes n times
for i = 1 to n
temp = temp + 1;
)
//recursive call with value
n = ";
)
"
until n 1
The recurrence for this function is ~ = ~ &
Using master theorem, we get, ~ = '.
)
"

function(int n)
{
for( i = 1 ; i n ; i + + )
for( j = 1 ; j n ; j * = 2 )
print( * ) ;
}
function(int n)
{
// this loops executes n times
for( i = 1 ; i n ; i + + )
// this loops executes logn times from our logarithms
//guideline
www.CareerMonk.com
for( j = 1 ; j n ; j * = 2 )
print( * ) ;
}
Complexity of above program is : '.

function(int n)
{
for( i = 1 ; i n/3 ; i + + )
for( j = 1 ; j n ; j += 4 )
print( * ) ;
}
function(int n)
{
// this loops executes n/3 times
for( i = 1 ; i n/3 ; i + + )
// this loops executes n/4 times
for( j = 1 ; j n ; j += 4)
print( * ) ;
}
The time complexity of this program is : '" .

{
if(n 1)
return;
if (n > 1)
{
print ("*");
function( " );
function( " );
www.CareerMonk.com

{
//constant time
if(n 1)
return;
if (n > 1)
{
//constant time
print ("*");
//recursion with n/2 value
function( n/2 );
//recursion with n/2 value
function( n/2 );
}
}
The recurrence for this function is: ~ = 2~ " & 1
Using master theorem, we get ~ = '.
Note:
Note: With these examples, I hope you got an understanding of analysis of algorithms. In this
chapter, try to understand when do we get different complexities like: ', ',
', '" , '2) etc so that it will be clear when we refer them in remaining chapters.

DS Notes How Are You

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DS Notes How Are You

Uploaded by

Copyright:

Available Formats

Data Structures and Algorithms Made Easy

66 Analysis of Algorithms | CareerMonk Publications

Data Structures and Algorithms Made Easy

Why Analysis of Algorithms?

Goal of Analysis of Algorithms?

What is Running Time Analysis?

How to Compare Algorithms?

    as execution times are specific to a particular computer.

Number of statements executed?

67 Analysis of Algorithms | CareerMonk Publications

Data Structures and Algorithms Made Easy

What is Rate of Growth?

Commonly used Rate of Growths

Adding an element to the front of a linked list

Finding an element in a sorted array

Finding an element in an unsorted array

Linear Logarithmic Sorting n items by divide-and-conquer-Mergesort

Shortest path between two nodes in a graph

Below diagram shows the relationship between different rates of growth.

68 Analysis of Algorithms | CareerMonk Publications

Data Structures and Algorithms Made Easy

Data Structures and Algorithms Made Easy

Data Structures and Algorithms Made Easy

Data Structures and Algorithms Made Easy

ExampleExample-2 Find upper bound for  = " & 1

ExampleExample-3 Find upper bound for  = & 100" & 50

Solution: & 100" & 50 K 2 , for all L 1

Solution: 2( G 2" K 2( , for all L 1

ExampleExample-5 Find upper bound for  =

ExampleExample-6 Find upper bound for  = 410

Data Structures and Algorithms Made Easy

ExampleExample-1 Find lower bound for  = 5"

Solution: , nJ Such that: 0  5"  5 "  = 1 and nJ = 1

Solution: c, nJ Such that: 0 " 100 & 5

Data Structures and Algorithms Made Easy

ExampleExample-3 = 2, ( = " , = 

74 Analysis of Algorithms | CareerMonk Publications

Data Structures and Algorithms Made Easy

G K " , for all, n 1

G " = "  with Y = 1/5, Y = 1 and nJ = 1

ExampleExample-2 Prove R V" 

Solution: Y " K K " " only holds for: K 1/Y

ExampleExample-3 Prove 6( V" 

Solution: Y n2 6( " " only holds for: K " /6

Solution: Y  K K " log " L

Why we call Asymptotic Analysis?

75 Analysis of Algorithms | CareerMonk Publications

Data Structures and Algorithms Made Easy

Guidelines for asymptotic analysis?

Data Structures and Algorithms Made Easy

Data Structures and Algorithms Made Easy

As an example let us consider the following program:

So the total time = '.

Look at the centre point in the dictionary

Data Structures and Algorithms Made Easy

Transpose symmetry:  = O if and only if  = .

Commonly used logarithms and summations

jklm = f jklm pq =

r s = 1 & 2 & t& =

Other important formulae

r s | = 1| & 2| & t & | 

Master Theorem for Divide and Conquer

79 Analysis of Algorithms | CareerMonk Publications

as execution times are specific to a particular computer.

ExampleExample-2 Find upper bound for = " & 1

ExampleExample-3 Find upper bound for = & 100" & 50

ExampleExample-5 Find upper bound for =

ExampleExample-6 Find upper bound for = 410

ExampleExample-1 Find lower bound for = 5"

Solution: , nJ Such that: 0 5" 5 " = 1 and nJ = 1

Solution: c, nJ Such that: 0 " 100 & 5

ExampleExample-3 = 2, ( = " , =

G " = " with Y = 1/5, Y = 1 and nJ = 1

ExampleExample-2 Prove R V"

Solution: Y " K K " " only holds for: K 1/Y

ExampleExample-3 Prove 6( V"

Solution: Y n2 6( " " only holds for: K " /6

Solution: Y K K " log " L

So the total time = '.

Transpose symmetry: = O if and only if = .

jklm = f jklm pq =

r s | = 1| & 2| & t & |

b , then ~ = Xcn./0 d

If ] G1, then ~ = Xcn./0 |{Y d

If ] = G1, then ~ = Xcn./0 loglognd

If ] < G1, then ~ = Xcn./0 sd

~ = 3~ /2 & "

~ = 4~ /2 & "

~ = ~/2 & 2)

~ = 2) ~/2 & )

~ = 16~/4 &

~ = 2~/2 &

~ = 2~/2 & /

~ = 2~ /4 & J.[Y

~ = 0.5~/2 & 1/

ProblemProblem-10 ~ = 6~ /3 & "

ProblemProblem-11 ~ = 64~/8 G "

ProblemProblem-12 ~ = 7~/3 & "

ProblemProblem-13 ~ = 4~ /2 &

ProblemProblem-14 ~ = ~/2 & 2 G

ProblemProblem-15 ~ = 16~ /4 & !

ProblemProblem-16 ~ = 2~ /2 &

ProblemProblem-17 ~ = 3~ /2 &

ProblemProblem-18 ~ = 3~ /3 &

ProblemProblem-19 ~ = 4~ /2 &

ProblemProblem-20 ~ = 3~ /4 &

ProblemProblem-21 ~ = 3~ /3 & /2

for some constants , 0, 0, s L 0, and function . If is in 'i , then

~ = 3c3~ G 2d = 3" ~ G 2

~ = 3) ~ G = 3) ~0 = 3)

This clearly shows that the complexity of this function is '3) .

~ = 22~ G 2 G 1 G 1 = 2" ~ G 2 G 2 G 1

~ = 2" 2~ G 3 G 2 G 1 G 1 = 2( ~ G 4 G 2" G 2Y G 2J

~ = 2) G 2) G 1 : 2)Y & 2)" & t & 2J = 2)

~ = ~/2 & ~/4 & ~/8 &

~ = ~ G 2 & G 1 G 2 & G 1

~ = ~1 & r G 1

~ = ~1 & r G r

& 12 & 1 & 1

ProblemProblem-37 What is the complexity of )vY ?