Rahul Singh (AP -SS)

Design and Analysis of Algorithms (DAA)-

CSEG 2021

Total Credits 3
Evaluation--- Quiz + Assignment + Class test

Rahul Singh
AI&ML Cluster
SOCS
Rahul Singh (AP -SS)
 Objective

1. To understand the necessity of the algorithm design.

2. To write the algorithm to solve a problem.

3. To analyze the performance of the algorithm.

4. To implement the algorithm in C/C++.

Rahul Singh (AP -SS)

 Backtracking
• Backtracking is a very important skill set to solve recursive solutions. Recursive
functions are those that calls itself more than once.
• It uses the Brute force search to solve the problem, and the brute force search says that
for the given problem, we try to make all the possible solutions and pick out the best
solution from all the desired solutions.
• This rule is also followed in dynamic programming, but dynamic programming is used
for solving optimization problems.
• Backtracking is not used in solving optimization problems. Backtracking is used when
we have multiple solutions, and we require all those solutions.

Rahul Singh (AP -SS)

 Backtracking
• Backtracking name itself suggests that we are going back and coming forward; if it
satisfies the condition, then return success, else we go back again.
• Backtracking is like trying different paths, and when you hit a dead end, you backtrack to
the last choice and try a different route.
• It is commonly used in situations where you need to explore multiple possibilities to
solve a problem, like searching for a path in a maze or solving puzzles like Sudoku.

Rahul Singh (AP -SS)

 When to use a Backtracking algorithm?
• A piece of sufficient information is not available to make the best choice, so we use the

backtracking strategy to try out all the possible solutions.

• Each decision leads to a new set of choices. Then again, we backtrack to make new

decisions. In this case, we need to use the backtracking strategy.

Types of Backtracking Problems

Decision Problems: Here, we search for a feasible solution.

Optimization Problems: For this type, we search for the best solution.

Enumeration Problems: We find set of all possible feasible solutions to the problems of this
type. Rahul Singh (AP -SS)
How does Backtracking works?

Rahul Singh (AP -SS)

 Applications of Backtracking
• N-queen problem

• Sum of subset problem

• Graph coloring

Rahul Singh (AP -SS)

Difference between the Backtracking and Recursion
– Recursion is a technique that calls the same function again and again until you reach the
base case. Backtracking is an algorithm that finds all the possible solutions and selects the
desired solution from the given set of solutions.

– Complexity Analysis of Backtracking: Backtracking algorithm is purely brute force

therefore in terms of time complexity, it performs very poorly.
• Exponential (O(K^N))

• Factorial (O(N!))

Rahul Singh (AP -SS)

 N-Queens Problem
– N - Queens problem is to place n - queens in such a manner on an n x n chessboard that no
queens attack each other by being in the same row, column or diagonal.
– It can be seen that for n =1, the problem has a trivial solution, and no solution exists for n
=2 and n =3. So first we will consider the 4 queens problem and then generate it to n -
queens problem.

– Given a 4 x 4 chessboard and number the rows and column of the chessboard 1
through 4.

Rahul Singh (AP -SS)

 N-Queens Problem
– Since, we have to place 4 queens such as q1 q2 q3 and q4 on the chessboard, such that no
two queens attack each other. In such a conditional each queen must be placed on a
different row, i.e., we put queen "i" on row “i”.
– we place queen q1 in the very first acceptable position (1, 1). Next, we put queen q2 so that
both these queens do not attack each other. We find that if we place q2 in column 1 and 2,
then the dead end is encountered. Thus the first acceptable position for q2 in column 3, i.e.
(2, 3) but then no position is left for placing queen 'q3' safely.
– So we backtrack one step and place the queen 'q2' in (2, 4), the next best possible solution.
Then we obtain the position for placing 'q3' which is (3, 2). But later this position also leads
to a dead end, and no place is found where 'q4' can be placed safely.
Rahul Singh (AP -SS)
 N-Queens Problem
– Then we have to backtrack till 'q1' and place it to (1, 2) and then all other queens are placed
safely by moving q2 to (2, 4), q3 to (3, 1) and q4 to (4, 3). That is, we get the solution (2, 4,
1, 3). This is one possible solution for the 4-queens problem. For another possible solution,
the whole method is repeated for all partial solutions. The other solutions for 4 - queens
problems is (3, 1, 4, 2) i.e.

Rahul Singh (AP -SS)

 N-Queens Problem
– The implicit tree for 4 - queen problem for a solution (2, 4, 1, 3) is as follows:

Rahul Singh (AP -SS)

 N-Queens Problem
– Fig shows the complete state space for 4 - queens problem. But we can use backtracking
method to generate the necessary node and stop if the next node violates the rule, i.e., if two
queens are attacking.

It can be seen
that all the
solutions to the 4
queens problem
can be
represented as 4 -
tuples (x1, x2, x3,
x4) where xi
represents the
column on which
queen "qi" is
placed.
Rahul Singh (AP -SS)
 N-Queens Problem
– If two queens are placed at position (i, j) and (k, l).
– Then they are on same diagonal only if (i - j) = k - l or i + j = k + l.
– The first equation implies that j - l = i - k.
– The second equation implies that j - l = k - i.
– Therefore, two queens lie on the duplicate diagonal if and only if |j-l|=|i-k|

• Step 1 − Start from 1st position in the array.

• Step 2 − Place queens in the board and check. Do,
• Step 2.1 − After placing the queen, mark the position as a part of the solution and then recursively check
if this will lead to a solution.
• Step 2.2 − Now, if placing the queen doesn’t lead to a solution and trackback and go to step (a) and
place queens to other rows.
• Step 2.3 − If placing queen returns a lead to solution return TRUE.
• Step 3 − If all queens are placed return TRUE.
• Step 4 − If all rows are tried and no solution is found, return FALSE.
Rahul Singh (AP -SS)
 Subset Sum Problem
– It is one of the most important problems in complexity theory. The problem is given an A
set of integers a1, a2,…., an upto n integers. The question arises that is there a non-empty
subset such that the sum of the subset is given as M integer?. For example, the set is given
as [5, 2, 1, 3, 9], and the sum of the subset is 9; the answer is YES as the sum of the subset
[5, 3, 1] is equal to 9. This is an NP-complete problem again. It is the special case of
knapsack

Rahul Singh (AP -SS)

 Subset Sum Problem
– Example 1:

Input: set[] = {4, 16, 5, 23, 12}, sum = 9

Output = true
Subset {4, 5} has the sum equal to 9.
– Example 2:

Input: set[] = {2, 3, 5, 6, 8, 10}, sum = 10

Output = true
There are three possible subsets that have the sum equal to 10.
Subset1: {5, 2, 3}
Subset2: {2, 8}
Subset3: {10}
Rahul Singh (AP -SS)
 Subset Sum Problem
– There are two ways of solving the subset problem:
– Dynamic programming
– Backtracking

Rahul Singh (AP -SS)

 Branch and bound
– Branch and bound is one of the techniques used for problem solving.
– It is similar to the backtracking since it also uses the state space tree.
– It is used for solving the optimization problems and minimization problems.
– If we have given a maximization problem then we can convert it using the Branch and
bound technique by simply converting the problem into a maximization problem.

Rahul Singh (AP -SS)

 Branch -n bound vs Backtracking
Backtracking Branch and bound
Backtracking is a problem-solving technique so it Branch n bound is a problem-solving technique so it solves the optimization
solves the decision problem. problem.
When we find the solution using backtracking then When we find the solution using Branch n bound then it provides a better
some bad choices can be made. solution so there are no chances of making a bad choice.
Backtracking uses a Depth first search. It is not necessary that branch n bound uses Depth first search. It can even
use a Breadth-first search and best-first search.
The state space tree is searched until the solution of The state space tree needs to be searched completely as the optimum
the problem is obtained. solution can be present anywhere in the state space tree.
In backtracking, all the possible solutions are tried. In branch and bound, based on search; bounding values are calculated.
If the solution does not satisfy the constraint, then According to the bounding values, we either stop there or extend.
we backtrack and look for another solution.
Applications of backtracking are n-Queens problem, Applications of branch and bound are knapsack problem, travelling
Sum of subset. salesman problem, etc.
Backtracking is more efficient than the Branch and Branch n bound is less efficient.
bound.
Backtracking solves the given problem by first Branch and bound solves the given problem by dividing the problem into
finding the solution of the subproblem and then two atleast subproblems.
recursively solves the other problems based on the
solution of the first subproblem.

Rahul Singh (AP -SS)

 Branch and bound: Traveling Salesman Problem
Follow my class as per the instruction

Rahul Singh (AP -SS)

 Non- Comparison Sorting
Complete: Lower Bounds For Sorting, Counting Sort, Radix Sort, bucket sort

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
– NP class Problem: - The set of all decision-based problems came into the division of
NP Problems who can't be solved or produced an output within polynomial time but
verified in the polynomial time. NP class contains P class as a subset. NP problems
being hard to solve.
• The term "NP" does not mean "not polynomial." Originally, the term meant "non-deterministic
polynomial. It means according to the one input number of output will be produced.

– P class Problem: - The set of decision-based problems come into the division of P
Problems who can be solved or produced an output within polynomial time. P problems
being easy to solve

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
– Polynomial time: - If we produce an output according to the given input within a
specific amount of time such as O(nc) c is constant. This is known as Polynomial time.

– Non-Polynomial time: - If we produce an output according to the given input but there
are no time constraints (O (Kn) k is constant) is known as Non-Polynomial time. But
yes output will produce but time is not fixed yet.

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
– Definition of Decision Based Problem: - A problem is called a decision problem if its
output is a simple "yes" or "no" (or you may need this of this as true/false, 0/1,
accept/reject.) We will phrase many optimization problems as decision problems. For
example, Greedy method, D.P., given a graph G= (V, E) if there exists any Hamiltonian
cycle.
– Definition of NP-hard class: - Here you to satisfy the following points to come into the
division of NP-hard
• If we can solve this problem in polynomial time, then we can solve all NP problems in polynomial time
• If you convert the issue into one form to another form within the polynomial time

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
– Definition of NP-complete class: - A problem is in NP-complete, if
• It is in NP and

• It is NP-hard

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
– Polynomial Time Reduction: We say that Decision Problem L1 is Polynomial time
Reducible to decision Problem L2 (L1≤p L2) if there is a polynomial time computation
function f such that of all x, xϵL1 if and only if x ϵ L2.
– Relation of P and NP classes
– P contains in NP
– P=NP

– Observe that P contains in NP. In other words, if we can solve a problem in polynomial time, we can indeed verify
the solution in polynomial time. More formally, we do not need to see a certificate (there is no need to specify the
vertex/intermediate of the specific path) to solve the problem; we can explain it in polynomial time anyway.
– However, it is not known whether P = NP. It seems you can verify and produce an output of the set of decision-based
problems in NP classes in a polynomial time which is impossible because according to the definition of NP classes
you can verify the solution within the polynomial time. So this relation can never be held.
Rahul Singh (AP -SS)
 NP-Hard and NP-Complete problem
– NP-Completeness

A decision problem L is NP-Hard if

L' ≤p L for all L' ϵ NP.
Definition: L is NP-complete if
L ϵ NP and
L' ≤ p L for some known NP-complete problem L.' Given this formal definition, the
complexity classes are:
P: is the set of decision problems that are solvable in polynomial time.

Rahul Singh (AP -SS)

 NP-Hard and NP-Complete problem
NP: is the set of decision problems that can be verified in polynomial time.
NP-Hard: L is NP-hard if for all L' ϵ NP, L' ≤p L. Thus if we can solve L in polynomial time,
we can solve all NP problems in polynomial time.
NP-Complete L is NP-complete if

L ϵ NP and
L is NP-hard
If any NP-complete problem is solvable in polynomial time, then every NP-Complete
problem is also solvable in polynomial time. Conversely, if we can prove that any NP-
Complete problem cannot be solved in polynomial time, every NP-Complete problem cannot
be solvable in polynomial time. Rahul Singh (AP -SS)
 NP-Hard and NP-Complete problem
Reductions
Concept: - If the solution of NPC problem does not exist then the conversion from one
NPC problem to another NPC problem within the polynomial time. For this, you need the
concept of reduction. If a solution of the one NPC problem exists within the polynomial
time, then the rest of the problem can also give the solution in polynomial time (but it's
hard to believe). For this, you need the concept of reduction.
Example: - Suppose there are two problems, A and B. You know that it is impossible to
solve problem A in polynomial time. You want to prove that B cannot be solved in
polynomial time. So you can convert the problem A into problem B in polynomial time.
Rahul Singh (AP -SS)
 References

1. Thomas H. Cormen (2009) Introduction to Algorithm (Third Edition), The MIT

Press. ISBN: 978-0-262-03384-8
2. Rajesh K. Shukla (2015) Analysis and Design of Algorithms: A Beginner's
Approach,Wiley, ISBN-10: 8126554770

Rahul Singh (AP -SS)

Rahul Singh (AP -SS)