DAA Unit-I

Introduction and Mathematical foundations UNIT-1
1. Introduction:
Algorithm: The word algorithm came from the name of a Persian mathematician Abu Jafar
Mohammed Ibn Musa Al Khowarizmi (ninth century). An algorithm is simply s set of rules used to
perform some calculations either by hand or more usually on a machine (computer).
Definition: An algorithm is a finite set of instructions that accomplishes a particular task. Another
definition is a sequence of unambiguous instructions for solving a problem i.e, for obtaining a required
output for any legitimate (genuine) input in a finite amount of time.
In addition all algorithms must satisfy the following properties (criteria / characteristics).
1. Input: zero or more quantities are externally supplied as input.
Consider Fibonacci numbers program, here aim of the problem is to display ten Fibonacci
numbers. No input is required; in the problem itself this is clearly mentioned as ten Fibonacci values. So
zero items required for input.
Another problem is displaying given numbers of evens, so user should accept how many evens
required. Based on the user input the number of evens is to be displayed. So, one data item is required as
input.
2. Output: At least one quantity is produced by given algorithm as output.
In the case of Fibonacci numbers program after executing the program, first ten Fibonacci values
displayed as output.
In second case, based on user input it should display given number of evens. An input of negative
number is wrong, should display proper error message as output. So this program displays at least one
output as error message, or number if outputs that show given number of steps.
3. Definiteness: Each instruction is clear and unambiguous i.e. each step must be easy to understand
and convey only a single meaning.
4. Effectiveness: each instruction must be very basic, so that it can be carried out by something.
This step is common in both Fibonacci and primes. For example, if user enters a negative numbers
as input in evens, if you have a step like
Step: If N < 0 then
Go to ERROR
A wrong instruction given as go to ERROR, those kinds of instructions should not be there in an
algorithm.
5. Finiteness: If we can trace out the instructions of an algorithm then for all cases, the algorithm
terminate after a finite number of steps.
Either in the case of Fibonacci or even numbers problem should be solved in some number of
steps. For example, continuous display or Fibonacci series without termination leads to abnormal
termination.
Criteria for Algorithms

Input: Zero or more inputs
Output: At least one output.
Finiteness: N number of steps.
Definiteness: Clear algorithm step.
Effectiveness: A carried out step.
Blog: anilkumarprathipati.wordpress.com Page 1

2. Process for Design and analysis of algorithms:

Understand the Problem
Decide on: Computational means,

Algo. Design technique
Design an Algorithm
No
Prove Correctness
Yes
Analyze the Algorithm No
Yes
Code the algorithm

Fig: Algorithm Design and Analysis Process
1. Understand the problem: This is very crucial phase. If we did any mistake in this step the entire
algorithm becomes wrong. So before designing an algorithm is to understand the problem first.
2. Solution as an algorithm (Exact vs approximation solving): Solve the problem exactly if possible.
Even though some problems are solvable by exact method, but they are not faster when compared to
approximation method. So in that situation we will use approximation method.
3. Algorithm techniques: In this we will use different design techniques like,
i) Divide-and-conquer
ii) Greedy method
iii) Dynamic programming
iv) Backtracking
v) Branch and bound…. etc.,
4. Prove correctness: once algorithm has been specified, next we have to prove its correctness.
Usually testing is used for proving correctness.
5. Analyze an algorithm: Analyzing an algorithm means studying the algorithm behavior ie.,
calculating the time complexity and space complexity. If the time complexity of algorithm is more then
we will use one more designing technique such that time complexity should be minimum.
6. Coding an algorithm: after completion of all phases successfully then we will code an algorithm.
Coding should not depend on any program language. We will use general notation (pseudo-code) and
English language statement. Ultimately algorithms are implemented as computer programs.
3. Types of Algorithms:
There are many types of algorithm but the most fundamental types of algorithm are:
1. Recursive algorithms
2. Dynamic programming algorithm
3. Backtracking algorithm
4. Divide and conquer algorithm
5. Greedy algorithm
6. Brute Force algorithm
7. Randomized algorithm

1) Simple recursive algorithm:

Solves the base case directly and then recurs with a simpler or easier input every time (A base
value is set at the starting for which the algorithm terminates).
It is use to solve the problems which can be broken into simpler or smaller problems of same type.
Example:
To find factorial using recursion, here is the pseudo code:
Fact(x)
If x is 0 /*0 is the base value and x is 0 is base case*/
return 1
return (x*Fact(x-1)) /* breaks the problem into small problems*/
2) Dynamic programming algorithm:
A dynamic programming algorithm (also known as dynamic optimization algorithm)
remembers the past result and uses them to find new result means it solve complex problems by breaking
it down into a collection of simpler sub-problems, then solving each of those sub-problems only once, and
storing their solution for future use instead of re-computing their solutions again.
Example:
Fibonacci sequence, here is the pseudo code:
Fib(n)
if n=0
return 0
else
prev_Fib=0,curr_Fib=1
repeat n-1 times /*if n=0 it will skip*/
next_Fib=prev_Fib+curr_Fib
prev_Fib=curr_Fib
curr_Fib=new_Fib
return curr_Fib
3) Backtracking algorithm:
Let's say we have a problem A and we divided it into three smaller problems B, C and D.
Now it may be the case that the solution to A does not depend on all the three sub-problems, in fact we
don't even know on which one it depends.
Let's take a situation. Suppose you are standing in front of three tunnels, one of which is
having a bag of gold at its end, but you don't know which one. So you'll try all three. First go in tunnel 1,
if that is not the one, then come out of it, and go into tunnel 2, and again if that is not the one, come out of
it and go into tunnel 3.
So basically in backtracking we attempt solving a sub-problem, and if we don't reach the
desired solution, then undo whatever we did for solving that sub-problem, and try solving another sub-
problem.
Example:
Queens Problem
4) Divide and conquer algorithm:
Divide and conquer consist of two parts first of all it divides the problems into smaller sub-problems
of the same type and solve them recursively and then combine them to form the solution of the original
problem.
Example:
Quick sort, Merge sort
5) Greedy algorithm:
Greedy algorithm is an algorithm that solves the problem by taking optimal solution at the local level
(without regards for any consequences) with the hope of finding optimal solution at the global level.
Greedy algorithm is used to find the optimal solution but it is not necessary that you will definitely
find the optimal solution by following this algorithm.
Example:
Huffman tree, counting money
6) Brute force algorithm:
A brute force algorithm simply tries all the possibilities until a satisfactory solution is found.
Such types of algorithm are also used to find the optimal (best) solution as it checks all the possible
solutions.
And also used for finding a satisfactory solution (not the best), simply stop as soon as a solution of the
problem is found.
Example:
Exact string matching algorithm
7) Randomized algorithm:
A randomized algorithm uses a random number at least once during the computation to make a
decision.
Example:
Quick Sort
As we use random number to choose the pivot point.
4. Specification of Algorithm:
There are various ways by which we can specify an algorithm.
Natural Language
Flowchart
Algorithm
Pseudocode
Program
It is very easy to specify an algorithm using natural language. But many times specification of
algorithm by using natural language is not clear, and may require brief description. Such a specification
creates difficulty, while actually implementing it (difficulty in converting into source code).
Hence many programmers prefer to have specification of algorithm by means of pseudo-code, which
is very near to programming language, but it is not easily understood by normal users.
Another way of representing the algorithm is by flow chart. Flow chart is a graphical representation
of an algorithm, but flowchart method work well only if the algorithm is small and simple.
One more specification is Program, which changes from one programming language to another
programming language.
5. Examples:
A) Air Travel:
So, we have an airline; Barbet airlines, which serves several cities in the country. And of course,
although it serves several cities, it does not really connect all these cities directly. Only some of the cities
are connected by direct flights. And for other pair of cities you have to take a hopping flight. You have to
go via an intermediate city. So, our first goal is to compute every pair of cities, which are actually
connected by this network served by this airline. So, how do we find out all pairs of cities A, B? Such that
A and B are connected by a sequence of flights.

There are about ten cities; Delhi, Varanasi, Ahmadabad, down to Trivandrum in the south. And
you have some arrows indicating the flights. Now, some of these flights are in one direction. You can go
from Delhi to Varanasi, but you cannot come back directly to Delhi. You must go to Ahmadabad and then
come back. So, this is quite common.
Some pairs of important cities; like in this case Mumbai and Delhi might be connected by flights
in both directions or even Mumbai and Calcutta. And so now we have these ten cities. We want to know
is it really possible go from say Varanasi to Trivandrum or is it not possible, is it possible to go from
Hyderabad to Delhi or is not possible. So, in this case what we really would like to know is the structure
of this network. The map itself is not relevant. We just need to know how many cities are there, which are
on the network and how are they connected by the flights. So, the picture below which has the circles and
arrows represents this network. And these cities are the circles and the flights are the arrows and the
arrow heads indicates the direction. So, if there is an arrow head in one direction, it is a one directional
flight; if there is an arrow head in both ends, its means it is a bidirectional flight. Ignore the actual names
of the cities. So, we can call them 1, 2, 3, 4, 5or a, b, c, d, e or whatever and solve the problem. So, this
kind of a picture is called graph. We will study the graphs more formally when we come to this module in
our course. It is as shown below.
Now, in terms of efficiency we have to look at what are the things that determine the complexity of a
problem. It is fairly obvious in this particular case that if we have more cities, the problem is more

complicated. So the number of cities, which we can call N, is certainly one parameter which determines
how complicated the algorithm is going to be, or not how complicated the algorithm is going to be, or
rather how long it is going to take to run. The other question which determines how complex the network
is this how many direct flights there are...Obviously there are fewer flights, there are fewer places, which
can be connected and we have to explore fewer possibilities.
So from this, it follows that computing the paths depends on both N and F. So, we will not have an
algorithm which will always take twenty steps. It will have to depend some number of steps depending on
N and F. Now what is this dependency, how does it grow? If N doubles, does our algorithm take two
times more times? Does it takes four times more times? If N term grows to factor often, does it takes ten
times more or hundred times more time?
The other question related to this is given this dependency on N and F what realistic size of
networks can we handle? If the airline grows to twenty flights, we will still have be able to give our
answer in the reasonable time. Remember that this kind of an answer is typically required when
somebody is making an online booking or something. And you will not allowed to reply in a few seconds,
right, it is not enough to come back after an hour and say “yes, there is a flight from Trivandrum to
Calcutta”. So, what is the limit of our efficiency? Can we scale this algorithm to cover airlines, multiple
airlines? So, we have a website which actually sells. Across all airlines I can take you from place A to
place B. That depends on how larger value of N and F we can handle.
B) To sort phone numbers:

Let assume you want sort our telephone directory one billion phone numbers. And you
have two types Algorithms as:
1) Navie based Algorithm (like brute force) ==> O(n2)
2) Best Algorithm ==> O(n log n)
Coming to System Performance (Speed), CPU with speed (1 G Hz) 109 operations per second,
Problem is to sort around 10 9 Phone numbers.
By using Algorithm -1, What is the total time it takes?
Where ’n’ is the input number, here 109 therefore time complexity will be calculated by using the
O(n2) formulae.
i.e. Time complexity = (Total Number of operations) / (CPU Speed)
≈ (109) 2 operations / 109 operations per second
≈ 1018 / 109 Seconds
≈ 109 Seconds
≈ 16,666,666.67 Minutes
≈ 277,777.78 Hours
≈ 11,574.07 Days
≈ 385.80 Months
≈ 32.15 Years (approx).
By using Algorithm -2, What is the total time it takes?

≈ 10 9 * log 2 10 9 / 109 Seconds
≈ 109 * 29.9 / 10 9 Seconds
≈ 29.9 Seconds
Q): Which Algorithm do you choose? By default Algorithm-2.

C) For Video Gaming:
So, supposing we are playing a video game, right. So, this might be one of the action type games
where there are objects moving around the screen and we have to know, identify certain object, shoot
them down, and capture them and whatever. So, let us assume that as part of the game in order to
compute the score, it has to periodically find out the closest pair of objects on the screen. So, now, how
do you find the closest pair of objects among group of objects? Well, of course, you can take every pair of
them, find the distance between each pair and then take the smallest one, right. So, this will been an n
squared algorithm, right. So, you compute the distance between any two objects and then after doing this
for every pair you take the smallest value. Now, it turns out, that there is a clever algorithm, again, which
takes time n log n. So, what is this distinction between n-square and n log n in this context?
Now, on a modern kind of gaming computer with a large display it is not unreasonable to assume, that
we could have a resolution of say, 2500 by 1500 pixel. If we have one of these reasonable size 20 inch
monitors, we could easily get this kind of resolutions. So, we will have about 3.75 millions points on the
screen. Now, we have some objects placed at some of these points.
So, let us assume that we have five lakh objects, five hundred thousand objects placed on the screen.
So, if you were to now compute the pair of objects among these five hundred thousand, which are closest
to each other and we use the naïve n squared algorithm, then you would expect to take 25 10 into the 10
steps because that is 5 into 10 to the 5 whole square. 25 into 10 to the 10 is 2500 second, which is around
40 minutes.
Now, if you are playing a game, an action game in which reflexes determine your score, obviously,
each update cannot take 40 minutes. That would not be an effective game. On
the other hand, you can easily check, that if you do n log n calculation for 5 into 10 to the 5, you take
something like 10 to the 6 or 10 to the 7 seconds. So, this will be a fraction of second, 1-10th or 1-100th
of the second, which is well below your human response time. So, it will be, essentially, instantaneously.
So, we move from a game, which is hopelessly slow to one in which it can really test your reflexes as a
human.
6. Performance Analysis:
Performance analysis or analysis of algorithms refers to the task of determining the efficiency of an
algorithm i.e. how much computing time and storage are required to run (or execute). This analysis of
algorithm helps in judging the value of one algorithm over another.
To judge an algorithm, particularly two things are taken into consideration
1. Space complexity
2. Time complexity.
Space Complexity:
The space complexity of an algorithm (program) is the amount of memory it needs to run to
completion. The space needed by an algorithm has the following components.
1. Instruction Space.
2. Data Space.
3. Environment Stack Space.
Instruction Space: Instruction space is the space needed to store the compiled version of the program
instructions. The amount of instruction space that is needed depends on factors such as-
i). The compiler used to compile the program into machine code.
ii). The compiler options in effect at the time of compilation.
iii). The target computer, i.,e computer on which the algorithm run.
Note that, one compiler may produce less code as compared to another compiler, when the same
program is compiled by these two.

Data Space: Data space is the space needed to store all constant and variable values. Data space has
two components.
i). Space needed by constants, for example 0, 1, 2.134.
ii). Space needed by dynamically allocated objects such as arrays, structures, classes.
Environmental Stack Space: Environmental stack space is used during execution of functions. Each
time function is involved the following data are saved as the environmental stack.
i). The return address.
ii). Value of local variables.
iii). Value of formal parameters in the function being invoked.
Environmental stack space is mainly used in recursive functions. Thus, the space requirement of any
program p may therefore be written as
Space complexity S(P) = C + Sp (Instance characteristics).
This equation shows that the total space needed by a program is divided into two parts.
 Fixed space requirements(C) is independent of instance characteristics of the inputs and outputs.
- Instruction space
- Space for simple variables, fixed-size structure variables, constants.
 A variable space requirements (SP(1)) dependent on instance characteristics 1.
- This part includes dynamically allocated space and the recursion stack space.
Example of instance character is:
Examples: 1
Algorithm REC (float x, float y, float z)
{
Return (X + Y +Y * Z + (X + Y +Z)) /(X+ Y) + 4.0;
}
In the above algorithm, there are no instance characteristics and the space needed by X, Y, Z is
independent of instance characteristics, therefore we can write,
S(XYZ) =3+0=3
One space each for X, Y and Z
Space complexity is O(1).
Examples: 2
Algorithm ADD ( float [], int n)
{
sum = 0.0;
for i=1 to n do
sum=sum+X[i];
return sum; }
Here, atleast n words since X must be large enough to hold the n elements to be summed. Here the
problem instances is characterized by n, the number of elements to be summed. So, we can write,
S(ADD) =3+n
3-one each for n, I and sum
Where n- is for array X[],
Space complexity is O(n).
Time Complexity:
The time complexity of an algorithm is the amount of compile time it needs to run to completion. We
can measure time complexity of an algorithm in two approaches
1. Priori analysis or compile time
2. Posteriori analysis or run (execution) time.

In priori analysis before the algorithm is executed we will analyze the behavior of the algorithm. A
priori analysis concentrates on determining the order if execution of statements.
In Posteriori analysis while the algorithm is executed we measure the execution time. Posteriori
analysis gives accurate values but it is very costly.
As we know that the compile time does not depend on the size of the input. Hence, we will confine
ourselves to consider only the run-time which depends on the size of the input and this run-time is
denoted by TP(n). Hence
Time complexity T(P) = C + TP(n).
The time (T(P)) taken by a program P is the sum of the compile time and execution time. The
compile time does not depend on the instance characteristics, so we concentrate on the runtime of a
program. This runtime is denoted by tp (instance characteristics).
The following equation determines the number of addition, subtraction, multiplication, division
compares, loads stores and so on, that would be made by the code for p.
tp(n) = CaADD(n)+ CsSUB(n)+ CmMUL(n)+ CdDIV(n)+……………..
where n denotes instance characteristics, and Ca, Cs, Cm, Cd and so on…..
As denote the time needed for an addition, subtraction, multiplication, division and so on, and ADD,
SUB, MUL, DIV and so on, are functions whose values are the number of additions, subtractions,
multiplications, divisions and so on. But this method is an impossible task to find out time complexity.
Another method is step count. By using step count, we can determine the number if steps needed by a
program to solve a particular problem in 2 ways.
Method 1: introduce a global variable “count”, which is initialized to zero. So each time a statement
in the signal program is executed, count is incremented by the step count of that statement.
Example: Algorithm sum with count statement added
count:=0;
Algorithm Sum(a, n) Algorithm Sum(a,n)
{ {
s:=0; s:=0;
count:=count+1;
for i:=1 to n do for i:=1 to n do
{ {
count:=count +1;
s:=s+a[i]; s:=s+a[i];
count:=count+1;
} }
count:=count+1; //for last time of for loop
count:=count+1; //for return statement
return s; return s;
} }
Thus the total number of steps are 2n+3
Method 2: The second method to determine the step count of an algorithm is to build a table in which
we list the total number of steps contributed by each statement.

Statement S/e Frequency Total steps

Ex:
1. Algorithm Sum(a, n) 0 - 0
2. { 0 - 0
3. s:=0; 1 1 1
4. for i:=1 to n do 1 n+1 n+1
5. s:=s+a[i]; 1 n n
6. return s; 1 1 1
7. } 0 - 0
Total 2n+3 steps
The S/e (steps per execution) of a statement is the amount by which the count changes as a result of
the execution of that statement. The frequency determines the total number of times each statement is
executed.
Complexity of Algorithms:
1. Best Case: Inputs are provided in such a way that the minimum time is required to process them.
2. Average Case: The amount of time the algorithm takes on an average set of inputs.
3. Worst Case: The amount of time the algorithm takes on the worst possible set of inputs.
Example: Linear Search
3 4 5 6 7 9 10 12 15
A 1 2 3 4 5 6 7 8 9
Best Case: If we want to search an element 3, whether it is present in the array or not. First, A(1) is
compared with 3, match occurs. So the number of comparisons is only one. It is observed that search
takes minimum number of comparisons, so it comes under best case.
Time complexity is O(1).
Average Case: If we want to search an element 7, whether it is present in the array or not.
First, A(1) is compared with 7 i,.e, (3=7), no match occurs. Next, compare A(2) and 7, no match
occurs.
Compare A(3) and A(4) with 7, no match occurs. Up to now 4 comparisons takes place. Now
compare A(5) and 7 (i.,e, 7=7), so match occurs. The number of comparisons is 5. It is observed that
search takes average number of comparisons. So it comes under average case.
Note: If there are n elements, then we require n/2 comparisons.
. n
. . Time complexity is O = O(n) (we neglect constant)
2
Worst Case: If we want to search an element 15, whether it is present in the array or not.
First, A(1) is compared with 15 (i.,e, 3=15), no match occurs. Continue this process until either
element is found or the list is exhausted. The element is found at 9th comparison. So number of
comparisons are 9.
Time complexity is O(n).
Note: If the element is not found in array, then we have to search entire array, so it comes under worst
case.

7. Asymptotic Notation:
Accurate measurement of time complexity is possible with asymptotic notation. Asymptotic
complexity gives an idea of how rapidly the space requirement or time requirement grow as problem size
increase. When there is a computing device that can execute 1000 complex operations per second. The
size of the problem is that can be solved in a second or minute or an hour by algorithms of different
asymptotic complexity. In general asymptotic complexity is a measure of algorithm not problem. Usually
the complexity of an algorithm is as a function relating the input length to the number of steps (time
complexity) or storage location (space complexity). For example, the running time is expressed as a
function of the input size ‘n’ as follows.
f(n)=n4+100n2+10 n+50 (running time)
There are four important asymptotic notations.
1. Big oh notation (O)
2. Omega notation ().
3. Theta notation ()
Let f(n) and g(n) are two non-negative functions.
Big oh notation:
Big oh notation is denoted by ‘O’. it is used to describe the efficiency of an algorithm. It is used to
represent the upper bound of an algorithms running time. Using Big O notation, we can give largest
amount of time taken by the algorithm to complete.
Definition: Let f(n) and g(n) be the two non-negative functions. We say that f(n) is said to be O(g(n))
if and only if there exists a positive constant ‘c’ and ‘n0‘ such that,
f(n)c*g(n) for all non-negative values of n, where n≥n0.
Here, g(n) is the upper bound for f(n).
4
Ex: Let f(n) = 2n + 5n2 + 2n +3
< 2n4 + 5n4 + 2n4 +3n4 c*g(n)
< (2+5+2+3)n4
4
< 12n . f(n)
. 4
.. f(n)=12n
This implies g(n)=n4, n > 1
.
. . c=12 and n0 =1
.
. . f(n)=O(n4)
n0
The above definition states that the function ‘f’ is almost ‘c’ times the function ‘g’ when ‘n’ is greater
than or equal to n0.
This notion provides an upper bound for the function ‘f’ i.,e, the function g(n) is an upper bound on
the value of f(n) for all n, where n≥ n0.
Big omega notation:
Big omega notation is denoted by ‘’. It is used to represent the lower bound of an algorithms
running time. Using big omega notation we can give shortest amount of time taken by the algorithm to
complete.
Definition: The function f(n)= (g(n)) (read as for of n is omega of g of n) if and only if there exist
positive constants ‘c’ and ‘n0’ such that,
f(n) ≥ c*g(n) for all n, n≥n0

Example:
4
f(n)
Let f(n) = 2n + 5n2 + 2n +3
> 2n4 (for example as n ,
lower order oterms c*g(n)
are insignificant)
. 4
. . f(n) > 2n , n >1
. 4
.. g(n)=n , c=2 and n 0 =1
. 4
.. f(n)= (n )
n0
Big Theta notation:
The big theta notation is denoted by ‘’. It is in between the upper bound and lower bound of an
algorithms running time.
Definition: Let f(n) and g(n) be the two non-negetive functions. We say that f(n) is said to be (g(n))
if and only if there exists a positive constants ‘c1’ and ‘c2’, such that,
c1g(n)  f(n)  c2g((n) for all non-negative values n, where n ≥ n0.
The above definition states that the function f(n) lies between ‘c1’times the function g(n) and ‘c2’,
times the function g(n) where ‘c1’ and ‘c2’ are positive constants.
This notation provides both lower and upper bounds for the function f(n) i.,e, g(n) is both lower and
upper bounds on the value of f(n), for large n. in other words theta notation says that f(n) is both O(g(n))
and (g(n)) for all n, where n≥n0.
This function f(n) = (g(n)) iff g(n) is both upper and lower bound an f(n).
Example:
c2*g(n)
f(n) = 2n4+ 5n2
+ 2n +3
f(n)
2n4 2n4 + 5n2 + 2n +3 12n4
c1*g(n)
2n4 f(n) 12n4 , n 1
.
.. g(n) = n4
.
.. c1=2, c2=12 and n0=1
.
.. f(n)=(n4)
n0 n
Quick Sort
Best Case O(n log n)
Average Case O(n log n)
Worst Case O(n2)

8. Properties of asymptotic notations:

Asymptotic Notations Properties are:
1. General Properties
2. Reflexive Properties
3. Transitive Properties
4. Symmetric Properties
5. Transpose Symmetric Properties
As we have gone through the definition of these three notations (Big-O, Omega-Q, Theta-Θ) in our
previous article. Now let’s discuss some important properties of those notations.
General Properties:
If f(n) is O(g(n)) then a*f(n) is also O(g(n)) ; where a is a constant.
Example:
f(n) = 2n²+5 is O(n²)
then 7*f(n) = 7(2n²+5)
= 14n²+35 is also O(n²)
Similarly, this property satisfies both Θ and Ω notation. We can say

If f(n) is Θ(g(n)) then a*f(n) is also Θ(g(n)); where a is a constant.
If f(n) is Ω (g(n)) then a*f(n) is also Ω (g(n)); where a is a constant.
Reflexive Properties:
If f(n) is given then f(n) is O(f(n)).
Example: f(n) = n² ; O(n²) i.e O(f(n))
Similarly, this property satisfies both Θ and Ω notation. We can say

If f(n) is given then f(n) is Θ(f(n)).
If f(n) is given then f(n) is Ω (f(n)).
Transitive Properties :
If f(n) is O(g(n)) and g(n) is O(h(n)) then f(n) = O(h(n)) .
Example: if f(n) = n , g(n) = n² and h(n)=n³
n is O(n²) and n² is O(n³) then n is O(n³)
Similarly this property satisfies for both Θ and Ω notation. We can say
If f(n) is Θ(g(n)) and g(n) is Θ(h(n)) then f(n) = Θ(h(n)) .
If f(n) is Ω (g(n)) and g(n) is Ω (h(n)) then f(n) = Ω (h(n))
Symmetric Properties :
If f(n) is Θ(g(n)) then g(n) is Θ(f(n)) .
Example: f(n) = n² and g(n) = n² then f(n) = Θ(n²) and g(n) = Θ(n²)
This property only satisfies for Θ notation.
Transpose Symmetric Properties :

If f(n) is O(g(n)) then g(n) is Ω (f(n)).
Example: f(n) = n , g(n) = n² then n is O(n²) and n² is Ω (n)
This property only satisfies for O and Ω notations.
Some More Properties :

1. If f(n) = O(g(n)) and f(n) = Ω(g(n)) then f(n) = Θ(g(n))
2. If f(n) = O(g(n)) and d(n)=O(e(n))
then f(n) + d(n) = O( max( g(n), e(n) ))
Example: f(n) = n i.e O(n)

d(n) = n² i.e O(n²)
then f(n) + d(n) = n + n² i.e O(n²)
3.If f(n)=O(g(n)) and d(n)=O(e(n))

then f(n) * d(n) = O( g(n) * e(n) )
Example: f(n) = n i.e O(n)

d(n) = n² i.e O(n²)
then f(n) * d(n) = n * n² = n³ i.e O(n³)
9. Recursion Relation:
The process in which a function calls itself directly or indirectly is called a recursion.
The corresponding function is called a recursive function. Using a recursive algorithm, certain problems
can be solved quite easily.
Examples of such problems are Towers of Hanoi (TOH), Inorder/Preorder/Postorder Tree

Traversals, DFS of Graph, etc…
In recursion, a function α either calls itself directly or calls a function β that in turn calls the original
function α. The function α is called recursive function.
Example − a function calling itself.
int function(int value)
{
if(value < 1)
return;
function(value - 1);
printf("%d ",value);
}
Example − a function that calls another function which in turn calls it again.
int function1(int value1)
{
if(value1 < 1)
return; function2(value1 - 1); printf("%d ",value1);
}
int function2(int value2)
{
function1(value2);
}
Adv:
• Reduce time complexity.
• Adds clarity to code.

• Reduce time to debug.

Dis-adv:
• Uses more space.
• Recursion can be slow.
A recursive method must have at least one base, or stopping, case. Otherwise go to an infinite loop.
In recursion, number of parameters are never change, code never change only the parameter value
changes.
A recurrence is an equation or inequality that describes a function in terms of its values on smaller inputs.
To solve a Recurrence Relation means to obtain a function defined on the natural numbers that satisfy the
recurrence.
For Example, the Worst Case Running Time T(n) of the MERGE SORT Procedures is described by the
recurrence.
T (n) = θ (1) if n=1
2T + θ (n) if n>1
There are four methods for solving Recurrence:

1. Substitution Method
2. Iteration Method
3. Recursion Tree Method
4. Master Method
A. Substitution Method:
The Substitution Method consists of two main steps:

1. Guess the Solution.
2. Use the mathematical induction to find the boundary condition and shows that the guess is correct.
For Example1 Solve the equation by Substitution Method.
T (n) = T +n
We have to show that it is asymptotically bound by O (log n).
Solution:
For T (n) = O (log n)
We have to show that for some constant c
1. T (n) ≤c logn.
Put this in given Recurrence Equation.
T (n) ≤c log +1
≤c log + 1 = c logn-clog2 2+1

≤c logn for c≥1
Thus T (n) =O logn.

Example2 Consider the Recurrence
T (n) = 2T + n n>1
Find an Asymptotic bound on T.
Solution:
We guess the solution is O (n (logn)).Thus for constant 'c'.
T (n) ≤c n logn
Put this in given Recurrence Equation.
Now,
T (n) ≤2c log +n

≤cnlogn-cnlog2+n
=cn logn-n (clog2-1)
≤cn logn for (c≥1)
Thus T (n) = O (n logn).
B. Iteration Methods:
It means to expand the recurrence and express it as a summation of terms of n and initial condition.
Example1: Consider the Recurrence
1. T (n) = 1 if n=1
2. = 2T (n-1) if n>1
Solution:
T (n) = 2T (n-1)
= 2[2T (n-2)] = 22T (n-2)
= 4[2T (n-3)] = 23T (n-3)
= 8[2T (n-4)] = 24T (n-4) (Eq.1)
Repeat the procedure for i times
T (n) = 2i T (n-i)
Put n-i=1 or i= n-1 in (Eq.1)
T (n) = 2 n-1 T (1)
= 2 n-1 .1 {T (1) =1 .....given}
= 2 n-1
Example2: Consider the Recurrence
1. T (n) = T (n-1) +1 and T (1) = θ (1).
Solution:
T (n) = T (n-1) +1
= (T (n-2) +1) +1 = (T (n-3) +1) +1+1
= T (n-4) +4 = T (n-5) +1+4
= T (n-5) +5= T (n-k) + k
Where k = n-1
T (n-k) = T (1) = θ (1)
T (n) = θ (1) + (n-1) = 1+n-1=n= θ (n).

C. Recursion Tree Method

1. Recursion Tree Method is a pictorial representation of an iteration method which is in the form of a
tree where at each level nodes are expanded.
2. In general, we consider the second term in recurrence as root.
3. It is useful when the divide & Conquer algorithm is used.
4. It is sometimes difficult to come up with a good guess. In Recursion tree, each root and child represents
the cost of a single subproblem.
5. We sum the costs within each of the levels of the tree to obtain a set of pre-level costs and then sum all
pre-level costs to determine the total cost of all levels of the recursion.
6. A Recursion Tree is best used to generate a good guess, which can be verified by the Substitution
Method.
Example 1
Consider T (n) = 2T + n2
We have to obtain the asymptotic bound using recursion tree method.
Solution: The Recursion tree for the above recurrence is

Example 2: Consider the following recurrence
T (n) = 4T +n
Obtain the asymptotic bound using recursion tree method.
Solution: The recursion trees for the above recurrence

Example 3: Consider the following recurrence
Obtain the asymptotic bound using recursion tree method.

Solution: The given Recurrence has the following recursion tree
When we add the values across the levels of the recursion trees, we get a value of n for every level. The
longest path from the root to leaf is

D. Master Method
The master method applies to recurrences of the form
Where:
o n is the size of the problem.
o a is the number of subproblems in the recursion.
o n/b is the size of each subproblem. (Here it is assumed that all subproblems are essentially the
same size.)
o f (n) is the sum of the work done outside the recursive calls, which includes the sum of dividing
the problem and the sum of combining the solutions to the subproblems.
o It is not possible always bound the function according to the requirement, so we make three cases
which will tell us what kind of bound we can apply on the function.
The Idea of Master theorem:
Master theorem Cases:

Example-1: T(n) = √2T(n/2) + logn

Solution-
We compare the given recurrence relation with T(n) = aT(n/b) + θ (nklogpn).
Then, we have-
a = √2
b=2
k=0
p=1
Now, a = √2 = 1.414 and bk = 20 = 1.
Clearly, a > bk.
So, we follow case-1.
So, we have-
T(n) = θ (nlogba)
T(n) = θ (nlog2√2)
T(n) = θ (n1/2)
Thus,
T(n) = θ (√n)
Example-2: T(n) = 3T(n/3) + n/2

Solution-
Then, we have-
a=3
b=3
k=1
p=0
Now, a = 3 and bk = 31 = 3.
Clearly, a = bk.
Since p = 0, so we have-
T(n) = θ (nlogba.logp+1 n)
T(n) = θ (nlog33.log0+1 n)
T(n) = θ (n1.log1n)
Thus,
T(n) = θ (nlogn)
Example-3: T(n) = 2T(n/2) + nlogn

Solution-
Then, we have-
a=2
b=2
k=1
p=1
Now, a = 2 and bk = 21 = 2.
Clearly, a = bk.
T(n) = θ (nlogba.logp+1 n)
T(n) = θ (nlog22.log1+1 n)
Thus,
T(n) = θ (nlog 2n)
Example-4: T(n) = 3T(n/2) + n2

Solution-
Then, we have-
a=3
b=2
k=2
p=0
Now, a = 3 and bk = 22 = 4.
Clearly, a < bk.
T(n) = θ (nklogpn)
T(n) = θ (n2log0n)
Thus,
T(n) = θ (n2)
Example-5: T(n) = 2T(n/4) + n0.51

Solution-
Then, we have-
a=2
b=4
k = 0.51
p=0
Now, a = 2 and bk = 40.51 = 2.0279.
Clearly, a < bk.
T(n) = θ (nklogpn)
T(n) = θ (n0.51log0n)
Thus,
T(n) = θ (n0.51)
Note: The master theorem cannot be used if:
• T(n) is not monotone.
• eg. T(n) = sin n
• f(n) is not a polynomial.
• eg. f(n) = 2 n
• T(n) = nT(n/3) + f(n)
• a is not a constant.
• eg. a = 2n
• a<1

10. Amortized Analysis:

Not just consider one operation, but a sequence of operations on a given data structure. Average cost over
a sequence of operations.
Amortized analysis has no involvement of probability. Average performance on a sequence of operations,
even some operation is expensive. Guarantee average performance of each operation among the sequence
in worst case.
In amortized analysis, average or worst case measure is not input dependent.
Example-1:
Suppose there is a shop, in which there are 500 item and the cost of each of these item is rupees one, and
there is another item in the shop (call it the 501 item) and the cost of this item is rupees 500 (very large).
Total there are 501 items.
Suppose, want to buy all these 501. Don't know how much money should take within the shop in
order to buy all these 501 items.
According to Asymptotic Analysis:
That the maximum price of any item in the shop is rupees 500 and the number of items in the shop is
501.
501 * 500 = 2 50 500.
According to Amortize Analysis: 500 *1 + 1 *500 = 1 000
Should follow to Asymptotic or Amortized?
Ans: Must follow the Amortized.
Example-2:
If Stack allows three operations as:
• Push
• Pop
• Multipop(k)
Note: Multipop(k) means pop k number of elements from the stack in a single time.
Its functionality is like: false if k>n;
true otherwise.
To perform the operation
• push will take O(1) and
• pop also will take O(1) time.
• multipop(k), where k is equal to equal to n. In this case the time complexity will be O(n).
According to Asymptotic Analysis :
The worst case time complexity is the time complexity of order of n and suppose doing it for n
operations.
So the time complexity: n *O(n) = O(n2).
According to Amortized Analysis :
The first multi-pop operation will perform is O(n), then the second operation perform is multi-pop of n, is
this possible?
No. first multipop operation was popped out all the elements
So the next multi-pop operation again is not possible to perform.
Actually need to do push in those number of elements again, n operations of push. Now perform this
multipop again over.
So the time complexity: 1*n + n*1 + n = O(n).

Example-3:
The idea of dynamic array is to increase the size of the table whenever it becomes full. Following are the
steps to follow when the table becomes full.
1) Allocate memory for larger table size, typically twice the old table.
2) Copy the contents of the old table to a new table.
3) Free the old table.
If the table has space available, we simply insert a new item in the available space.
According to Asymptotic Analysis:

The worst case time complexity is the time complexity of order of n and supposes doing it for n
operations.
So the time complexity: n *O(n) = O(n2).
According to Amortize Analysis:

Three Methods of Amortized Analysis

1. Aggregate analysis:
• Total cost of n operations / n, i.e. T(n) / n
2. Accounting method:
• Assign each type of operation an (different) amortized cost
• overcharge some operations,
• store the overcharge as credit on specific objects,
• then use the credit for compensation for some later operations.
3. Potential method:
• Same as accounting method
• But store the credit as “potential energy” and as a whole.
• p( i )=amortized( i )-actual( i )+p( i-1 )
• If we sum equation (2) for 1≤i≤n we get
P (n)-p (0) ≥0
Example: Let assume we pay $50 for each month other than March, June, September, and December
$100 for every June, September. calculate cost by using aggregate, accounting and potential method .
1. Aggregate Method: 900/12  75

2. Accounting method:
From the above table we see that using any cost less than $75 will result in p(n)-p(0)≤0.
The amortized cost must be ≥ 75. If the amortized cost ≤ 75 then only the condition p(n)-p(0)<=0.
3. Potential method :
To the given problem we start with the potential function as:
P (n) =0 n mod 12=0
P (n) =25 n mod 12=1 or 3
P (n) =50 n mod 12=4, 6, 2
P (n) =75 n mod 12=5, 7, 9
P (n) =100 n mod 12=8, 10
P (n) =125 n mod 12=4
From these function the amortized cost for operation is evaluated for
amortized( i )=p( i )-p( i-1) +actual( i ).

DAA Unit-I

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

DAA Unit-I

Uploaded by

Copyright:

Available Formats

Introduction and Mathematical foundations UNIT-1

Criteria for Algorithms

Blog: anilkumarprathipati.wordpress.com Page 1

2. Process for Design and analysis of algorithms:

Decide on: Computational means,

Code the algorithm

Blog: anilkumarprathipati.wordpress.com Page 2

1) Simple recursive algorithm:

Blog: anilkumarprathipati.wordpress.com Page 4

Blog: anilkumarprathipati.wordpress.com Page 5

B) To sort phone numbers:

By using Algorithm -2, What is the total time it takes?

Blog: anilkumarprathipati.wordpress.com Page 6

C) For Video Gaming:

Blog: anilkumarprathipati.wordpress.com Page 7

Blog: anilkumarprathipati.wordpress.com Page 8

Blog: anilkumarprathipati.wordpress.com Page 9

Statement S/e Frequency Total steps

4. for i:=1 to n do 1 n+1 n+1

Total 2n+3 steps

Blog: anilkumarprathipati.wordpress.com Page 10

Blog: anilkumarprathipati.wordpress.com Page 11

Best Case O(n log n)

Average Case O(n log n)

Worst Case O(n2)

Blog: anilkumarprathipati.wordpress.com Page 12

8. Properties of asymptotic notations:

Similarly, this property satisfies both Θ and Ω notation. We can say

Similarly, this property satisfies both Θ and Ω notation. We can say

Transpose Symmetric Properties :

This property only satisfies for O and Ω notations.

Some More Properties :

Example: f(n) = n i.e O(n)

3.If f(n)=O(g(n)) and d(n)=O(e(n))

Example: f(n) = n i.e O(n)

Examples of such problems are Towers of Hanoi (TOH), Inorder/Preorder/Postorder Tree

Blog: anilkumarprathipati.wordpress.com Page 14

• Reduce time to debug.

There are four methods for solving Recurrence:

The Substitution Method consists of two main steps:

For Example1 Solve the equation by Substitution Method.

≤c log + 1 = c logn-clog2 2+1

Blog: anilkumarprathipati.wordpress.com Page 15

Example2 Consider the Recurrence

T (n) ≤2c log +n

Repeat the procedure for i times

Blog: anilkumarprathipati.wordpress.com Page 16

C. Recursion Tree Method

Blog: anilkumarprathipati.wordpress.com Page 17

Example 2: Consider the following recurrence

Blog: anilkumarprathipati.wordpress.com Page 18

Example 3: Consider the following recurrence

Obtain the asymptotic bound using recursion tree method.

Blog: anilkumarprathipati.wordpress.com Page 19

Master theorem Cases:

Blog: anilkumarprathipati.wordpress.com Page 20

Example-1: T(n) = √2T(n/2) + logn

Example-2: T(n) = 3T(n/3) + n/2

Example-3: T(n) = 2T(n/2) + nlogn

Example-4: T(n) = 3T(n/2) + n2

Example-5: T(n) = 2T(n/4) + n0.51

Blog: anilkumarprathipati.wordpress.com Page 22

10. Amortized Analysis:

Blog: anilkumarprathipati.wordpress.com Page 23