Unit-3: Divide and Conquer Algorithms

Analysis and Design of Algorithms (ADA)
GTU # 3150703
Unit-3:
Divide and Conquer
Algorithms
Computer Engineering Department

Darshan Institute of Engineering & Technology, Rajkot
gopi.sanghani@darshan.ac.in
9825621471
 Outline
Looping
 Introduction to Recurrence Equation
 Different methods to solve recurrence
 Divide and Conquer Technique
 Multiplying large Integers Problem
 Problem Solving using divide and conquer
algorithm –
 Binary Search
 Sorting (Merge Sort, Quick Sort)
 Matrix Multiplication
 Exponential
Introduction
 Many algorithms (divide and conquer) are recursive in nature.
 When we analyze them, we get a recurrence relation for time complexity.
 We get running time as a function of (input size) and we get the running time on inputs of
smaller sizes.
 A recurrence is a recursive description of a function, or a description of a function in terms of
itself.
 A recurrence relation recursively defines a sequence where the next term is a function of the
previous terms.
Dr. Gopi Sanghani #3150703 (ADA)  Unit 3 – Divide & Conquer Algorithms 4
Methods to Solve Recurrence
 Substitution
 Homogeneous (characteristic equation)
 Inhomogeneous
 Master method
 Recurrence tree
 Intelligent guess work
 Change of variable
 Range transformations
Substitution Method – Example 1
 We make a guess for the solution and then we use mathematical induction to prove the guess
is correct or incorrect.
Time to solve the
Example 1: instance of size 𝑛 − 1
 Replacing by to solve and

Time the , we can write following equations.
instance of size 𝑛
2
 Substituting equation in and equation in we have now,

4
 From above, we can write the general form as,
 Suppose, if we take then,
 Rewrite the equation,

 Now, replace by and
 Substitute the values of and
 In general,
 Suppose if we take then,
Substitution Method Exercises
 Solve the following recurrences using substitution method.
and
Homogeneous Recurrence
 Recurrence equation
 The equation of degree in is called the characteristic equation of the recurrence,
 Which can be factorized as,
 The solution of recurrence is given as,

𝒌
𝒏
𝒏 𝒊 𝒊
𝒊 𝟏
Homogeneous Recurrence – Example 1 : Fibonacci Series
 Fibonacci series Iterative Algorithm
Function fibiter(n)
i ← 1; j ← 0;
for k ← 1 to n do
j ← i + j;
i ← j – i;
return j
 Analysis of Iterative Algorithm: If we count all arithmetic operations at unit cost; the
instructions inside for loop take constant time . The time taken by the for loop is bounded
above by , .,
Case 1
 If the value of is large, then time needed to execute addition operation increases linearly with
the length of operand.
 At the end of 𝑡ℎ iteration, the value of and will be 𝒌 𝟏 and 𝒌.
 As per De Moivre’s formula the size of 𝒌 is in .
 So, 𝒕𝒉 iteration takes time in . let be some constant such that this time is bounded
above by for all .
 The time taken by fibiter algorithm is bounded above by,
Case 2
 Recursive Algorithm for Fibonacci series,
Function fibrec(n)
if n < 2 then return n
else return fibrec (n – 1) + fibrec (n – 2)
 The recurrence equation of above algorithm is given as,
 The recurrence can be re-written as,
 The characteristic polynomial is,
 Find the roots of characteristic polynomial, 𝟐
 The roots are, Here, and
and
𝒌
 The general solution is therefore of the form, 𝒏
𝒏 𝒏 𝒏 𝒊 𝒊
𝒏 𝟏 𝟏 𝟐 𝟐
𝒊 𝟏
 Substituting initial values and
 Solving these equations, we obtain

and
 Substituting the values of roots and constants in general solution,
𝒏 𝒏
𝒏 𝟏 𝟏 𝟐 𝟐
𝒏
𝒏
 Time taken for recursive Fibonacci algorithm grows Exponentially.
Example 2 : Tower of Hanoi
tower 1 tower 2 tower 3
tower 1 tower 2 tower 3
 The number of movements of a ring required in the tower of Hanoi problem is given by,
 The equation can be written as,

Inhomogeneous equation
 To convert it into a homogeneous equation, multiply with and replace by ,
 Solving equations (1) and (2), we have now
 The characteristic polynomial is, 𝒕 𝒎 − 𝟑𝒕 𝒎 − 𝟏 + 𝟐𝒕(𝒎 − 𝟐) = 𝟎
𝟐
 Whose roots are,

𝟏 and 𝟐
 The general solution is therefore of the form,
𝒎 𝒎
𝒎 𝟏 𝟐
 Substituting initial values and
𝟎 𝟏 𝟐
𝟏 𝟏 𝟐
 Solving these linear equations we get 𝟏 and 𝟐

 Therefore, time complexity of tower of Hanoi problem is given as,
𝒎 𝒎
Homogeneous Recurrence Exercises
 Solve the following recurrences
Master Theorem
 The master theorem is a cookbook method for solving recurrences.
Time to divide &
 Suppose you have a recurrence of the form recombine
Number of sub- Time required to

problems solve a sub-problem
 This recurrence would arise in the analysis of a recursive algorithm.

 When input size is large, the problem is divided up into sub-problems each of size .
Sub-problems are solved recursively and results are recombined.
 The work to split the problem into sub-problems and recombine the results is .
Master Theorem – Example 1
 There are three cases:
 Example 1:
 Here . So, =
 Also, =
 Case 2 applies:
Dr. Gopi Sanghani #3150703 (ADA)  Unit 1 – Basics of Algorithms and Mathematics 21
 Example 2:
 Here . So,

 Case 2 applies: the solution is 𝒍𝒐𝒈𝒃 𝒂
 Example 3:
 Here . So, 𝑏 and 2

 So, 2 is in
 Case 1 applies: 𝟐
Master Theorem Exercises
 Example 4: 2
 Example 5: 3
 Example 6: (Summer 17, Summer 19)

 Example 7: (Summer 17)
 Example 8: 3 (Winter 18)
 Example 9: (Winter 19)
Recurrence Tree Method
 In recurrence tree, each node represents the cost of a single sub-problem in the set of
recursive function invocations.
 We sum the costs within each level of the tree to obtain a set of per level costs.
 Then we sum the all the per level costs to determine the total cost of all levels of the recursion.
 Here while solving recurrences, we divide the problem into sub-problems of equal size.
 E.g., where and is a given function.
 is the cost of splitting or combining the sub problems.
𝒏⁄𝒃 𝒏⁄𝒃
Example 1:
The recursion tree for this recurrence is  When we add the values across the levels of
the recursion tree, we get a value of for
every level.
 The bottom level has nodes, each
contributing the cost
 We have
𝟐
𝒍𝒐𝒈𝟐 𝒏 𝟏
𝒍𝒐𝒈 𝒏
𝒊 𝟎
Example 2:
The recursion tree for this recurrence is  When we add the values across the levels of
the recursion tree, we get a value of for
every level.
/
/
𝑛 ⁄3 2𝑛⁄3
𝟑 𝟑 /𝟐 𝟑/𝟐
1𝑛 2𝑛 1 2𝑛 2 2𝑛
33 33 3 3 3 3
Example 3: 𝟐
The recursion tree for this recurrence is  Sub-problem size at level is
 Cost of problem at level Is
 Total cost,
𝒍𝒐𝒈𝟐 𝒏 𝟏 𝒊
𝟐
𝒊 𝟎
𝒊
𝟐
𝒊 𝟎
Recurrence Tree Method - Exercises
 Example 1:
 Example 2: 2
 Example 3: 2
 Example 4:
Introduction
 Many useful algorithms are recursive in structure: to solve a given problem, they call
themselves recursively one or more times.
 These algorithms typically follow a divide-and-conquer approach:
 The divide-and-conquer approach involves three steps at each level of the recursion:
1. Divide: Break the problem into several sub problems that are similar to the original problem but smaller in
size.
2. Conquer: Solve the sub problems recursively. If the sub problem sizes are small enough, just solve the sub
problems in a straightforward manner.
3. Combine: Combine these solutions to create a solution to the original problem.
D&C Running Time Analysis
 The running-time analysis of such divide-and-conquer (D&C) algorithms is almost automatic.
 Let be the time required by D&C on instances of size .
 The total time taken by this divide-and-conquer algorithm is given by recurrence equation,
 The solution of equation is given as,
where is the power of in
Introduction
 Binary Search is an extremely well-known instance of divide-and-conquer approach.
 Let be an array of increasing sorted order; that is whenever
.
 Let be some number. The problem consists of finding in the array if it is there.
 If is not in the array, then we want to find the position where it might be inserted.
Binary Search Example
Input: sorted array of integer values.
1 3 7 9 11 32 52 74 90
Step 1:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
Find approximate midpoint

Step 2:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
IsIs < =midpoint

midpointvalue?
value?YES.
No.
Step 3:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
Search for the target in the area before midpoint.

Step 4:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
Find approximate midpoint

Step 5:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
> value of midpoint? YES.

Step 6:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
Search for the in the area after midpoint.
Step 7:
1 2 3 4 5 6 7 8 9
1 3 7 9 11 32 52 74 90
Find approximate midpoint.

Is = midpoint value? YES.
Binary Search – Iterative Algorithm
Algorithm: Function biniter(T[1,…,n], x)
if x > T[n] then return n+1
i ← 1; i 3
j ← n; 6
while i < j do
7
k ← (i + j ) ÷ 2
k 11
if x ≤ T [k] then j ← k
else i ← k + 1 32
return i 33
j 53
Binary Search – Recursive Algorithm
Algorithm: Function binsearch(T[1,…,n], x)
if n = 0 or x > T[n] then return n + 1
else return binrec(T[1,…,n], x)
Function binrec(T[i,…,j], x)
if i = j then return i
k ← (i + j) ÷ 2
if x ≤ T[k] then
return binrec(T[i,…,k],x)
else return binrec(T[k + 1,…,j], x)
Binary Search - Analysis
 Let be the time required for a call on binrec where is the
number of elements still under consideration in the search.
 The recurrence equation is given as,
 Comparing this to the general template for divide and conquer algorithm,
 The complexity of binary search is
Binary Search – Examples
1. Demonstrate binary search algorithm and find the element in the following array. [3 /
4]
2. Explain binary search algorithm and find the element in the following array. [7]
3. Let be a sorted array of distinct integers. Give an algorithm that can find an index
such that and provided such an index exists. Prove that your
algorithm takes time in in the worst case.
Multiplying Large Integers – Introduction
 Multiplying two digit large integers using divide and conquer method.
 Example: Multiplication of by .
1. Convert both the numbers into same length nos. and split each operand into two parts:
2. We can write as,

2 𝟐 𝟐
2
Multiplying Large Integers – Example 1
 Now, the required product can be computed as,
 The above procedure still needs four half-size multiplications:
 The computation of can be done as,
 Only one multiplication is required instead of two.

Additional terms
𝟒 𝟐
 Now we can compute the required product as follows:
Multiplying Large Integers – Analysis
 981 × 1234 can be reduced to three multiplications of two-figure numbers (09∙12, 81∙34
90∙46) together with a certain number of shifts, additions and subtractions.
 Reducing four multiplications to three will enable us to cut 25% of the computing time required
for large multiplications.
 We obtain an algorithm that can multiply two -figure numbers in a time,
( )= ( / ) + ( ),
 Solving it gives,
𝒍𝒈𝟑
 Example: Multiply with using divide & conquer method.
 Solution using D&C
Step 1:
Step 2: Calculate and
0
𝟒 𝟐
Introduction
 Merge Sort is an example of divide and conquer algorithm.
 It is based on the idea of breaking down a list into several sub-lists until each sub list consists
of a single element.
 Merging those sub lists in a manner that results into a sorted list.
 Procedure
 Divide the unsorted list into N sub lists, each containing 1 element
 Take adjacent pairs of two singleton lists and merge them to form a list of 2 elements. N will now convert
into N/2 lists of size 2
 Repeat the process till a single sorted list of all the elements is obtained
Merge Sort – Example
Unsorted Array
724 521 2 98 529 31 189 451
1 2 3 4 5 6 7 8
Step 1: Split the selected array

1 2 3 4 5 6 7 8
724 521 2 98 529 31 189 451
724 521 2 98 529 31 189 451

1 2 3 4 1 2 3 4
Merge Sort – Example
Select the left subarray and Split Select the right subarray and Split
1 2 3 4 1 2 3 4
724 521 2 98 529 31 189 451
1 2 1 2 Split 1 2 1 2
724 521 2 98 529 31 189 451
1 1 1 1 1 1 1 1
724 521 2 98 529 31 189 451
521 724 2 98 31 529 189 451

Merge
2 98 521 724 31 189 451 529
2 31 98 189 451 521 529 724

Merge Sort – Algorithm
Procedure: mergesort(T[1,…,n]) Procedure:
merge(U[1,…,m+1],V[1,…,n+1],T[1,…,m+n])
if n is sufficiently small then
insert(T) i ← 1;
else j ← 1;
array U[1,…,1+n/2],V[1,…,1+n/2] U[m+1], V[n+1] ← ∞;
U[1,…,n/2] ← T[1,…,n/2] for k ← 1 to m + n do
V[1,…,n/2] ← T[n/2+1,…,n] if U[i] < V[j]
mergesort(U[1,…,n/2]) then T[k] ← U[i];
mergesort(V[1,…,n/2]) i ← i + 1;
merge(U, V, T) else T[k] ← V[j];
j ← j + 1;
Merge Sort - Analysis
 Let be the time taken by this algorithm to sort an array of elements.
 Separating into takes linear time; also takes linear time.
where .
 Applying the general case,
 Since 𝑘 the second case applies so,

 Time complexity of merge sort is 𝒌 𝒌
𝒌 𝒌
𝒍𝒐𝒈𝒃 𝒍 𝒌
Introduction
 Quick sort chooses the first element as a pivot element, a lower bound is the first index and an
upper bound is the last index.
 The array is then partitioned on either side of the pivot.
 Elements are moved so that, those greater than the pivot are shifted to its right whereas the
others are shifted to its left.
 Each Partition is internally sorted recursively.
Pivot
Element
0 1 2 3 4 5 6 7 8 9
42 23 74 11 65 58 94 36 99 87
LB UB
Quick Sort - Example
Procedure pivot(T[i,…,j]; var l) 0 1 2 3 4 5 6 7 8 9
p ← T[i] 42 23 74 11 65 58 94 36 99 87
k ← i; l ← j+1 k l
Repeat
k ← k+1 until T[k] > p or k ≥ j Swap
Repeat 42 23 36
74 11 65 58 94 74
36 99 87
l ← l-1 until T[l] ≤ p k l
While k < l do
Swap
Swap T[k] and T[l]
Repeat k ← k+1 until T[k] > p 11
42 23 36 42
11 65 58 94 74 99 87
Repeat l ← l-1 until T[l] ≤ p k l
Swap T[i] and T[l]
LB = 0, UB = 9
p = 42
k = 0, l = 10
p ← T[i] 11 23 36 42 65 58 94 74 99 87
k ← i; l ← j+1 LB UB
Repeat
k ← k+1 until T[k] > p or k ≥ j 11 23 36
Repeat
k l
l ← l-1 until T[l] ≤ p
While k < l do LB UB
Swap T[k] and T[l] 11 23 36 42 65 58 94 74 99 87
Repeat k ← k+1 until T[k] > p
Repeat l ← l-1 until T[l] ≤ p
Swap T[i] and T[l]
23 36
k l
11 23 36 42 65 58 94 74 99 87
LB UB
p ← T[i] 11 23 36 42 65 58 94 74 99 87
k ← i; l ← j+1
Swap
Repeat
k ← k+1 until T[k] > p or k ≥ j 65 65
58 58 94 74 99 87
Repeat
k l
l ← l-1 until T[l] ≤ p
While k < l do
Swap T[k] and T[l] 58 65 94 74 99 87
Swap T[i] and T[l] LB UB
11 23 36 42 58 65 94 74 99 87
LB UB
Procedure pivot(T[i,…,j]; var l) Swap
p ← T[i] 94 74 99
87 87
99
k ← i; l ← j+1 k l
Repeat
Swap
k ← k+1 until T[k] > p or k ≥ j
Repeat
94 74 94
87 87 99
l ← l-1 until T[l] ≤ p k l
While k < l do LB UB
Swap T[k] and T[l] Swap
Repeat k ← k+1 until T[k] > p 74 87
87 74 94 99
Swap T[i] and T[l] k l
11 23 36 42 58 65 74 87 94 99
Quick Sort - Algorithm
Procedure: quicksort(T[i,…,j]) Procedure: pivot(T[i,…,j]; var l)
{Sorts subarray T[i,…,j] into p ← T[i]
ascending order}
k ← i
if j – i is sufficiently small
then insert (T[i,…,j]) l ← j + 1
else repeat k ← k+1 until T[k] > p or k ≥ j
pivot(T[i,…,j],l) repeat l ← l-1 until T[l] ≤ p
quicksort(T[i,…,l - 1]) while k < l do
quicksort(T[l+1,…,j] Swap T[k] and T[l]
Swap T[i] and T[l]
Quick Sort Algorithm – Analysis
1. Worst Case
 Running time depends on which element is chosen as key or pivot element.
 The worst case behavior for quick sort occurs when the array is partitioned into one sub-array with 𝒏 − 𝟏
elements and the other with 𝟎 element.
 In this case, the recurrence will be,
2. Best Case
 Occurs when partition produces sub-problems each of size n/2.
 Recurrence equation:
Quick Sort Algorithm – Analysis
3. Average Case
 Average case running time is much closer to the best case.
 If suppose the partitioning algorithm produces a 9:1 proportional split the recurrence will be,
Quick Sort - Examples
 Sort the following array in ascending order using quick sort algorithm.
1. 5, 3, 8, 9, 1, 7, 0, 2, 6, 4
2. 3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8, 9
3. 9, 7, 5, 11, 12, 2, 14, 3, 10, 6
Matrix Multiplication
 Multiply following two matrices. Count how many scalar multiplications are required.
 To multiply matrices, total 3 scalar multiplications are required.
Matrix Multiplication
 In general, and are two matrices to be multiplied.
and
 Computing each entry in the product takes multiplications and there are 𝟐 entries for a total
of 𝟑 .
Strassen’s Algorithm for Matrix Multiplication
 Consider the problem of multiplying two matrices.
 Strassen’s devised a better method which has the same basic method as the multiplication of
long integers.
 The main idea is to save one multiplication on a small problem and then use recursion.
Strassen’s Algorithm for Matrix Multiplication
𝐴 𝐴 𝐵 𝐵
𝐴= and 𝐵 =
Step 1 Step 2 Step 3 𝐴 𝐴 𝐵 𝐵
Final Answer:
Where,
All above
operations No multiplication is
involve only one required here.
multiplication.
Strassen’s Algorithm - Analysis
 It is therefore possible to multiply two matrices using only seven scalar multiplications.
 Let be the time needed to multiply two matrices by recursive use of equations.
Where 2
 The general equation applies with and .
 Since the third case applies and

 Since , it is possible to multiply two matrices in a time 𝟐.𝟖𝟏 .
𝒌 𝒌
𝒌 𝒌
𝒍𝒐𝒈𝒃 𝒍 𝒌
Exponentiation - Sequential
 Let and be two integers. We wish to compute the exponentiation 𝒏
.
 Algorithm using Sequential Approach:
function exposeq(a, n)
r ← a
for i ← 1 to n - 1 do
r ← a * r
return r
 This algorithm takes a time in since the instruction is executed exactly

times, provided the multiplications are counted as elementary operations.
 But to handle larger operands, we must consider the time required for each multiplication.
 Let is the size of operand .
 Therefore, the multiplication performed the 𝒕𝒉 time round the loop concerns an integer of size
and an integer whose size is between and , which takes a time between
and
and and suppose

The body of loop executes 𝑡ℎ time as,
here times multiplication is already done so 𝟗
The size of in the 10th iteration will be between i.e.,

between 10-10+1 10
 The total time spent multiplying when computing an with exposeq is therefore,
𝟐 𝟐
 If we use the divide-and-conquer multiplication algorithm,

𝒍𝒈𝟑 𝟐
Exponentiation – D & C
 Suppose, we want to compute 𝟏𝟎
 We can write as,
10 5 2 4 2 2 2 2
 In general,
 Algorithm using Divide & Conquer Approach:
function expoDC(a, n)
if n = 1 then return a
if n is even then return [expoDC(a, n/2)]2
return a * expoDC(a, n - 1)
Exponentiation – D & C
Number of operations performed by

the algorithm is given by,
Time taken by the

algorithm is given by,
Solving it gives, 𝒍𝒈𝟑 𝒍𝒈𝟑
function expoDC(a, n)
if n = 1 then return a
if n is even then return [expoDC(a, n/2)]2
return a * expoDC(a, n - 1)
Exponentiation – Summary
Multiplication
Classic D&C
exposeq 𝟐 𝟐 𝒍𝒈𝟑 𝟐
expoDC 𝟐 𝟐 𝒍𝒈𝟑 𝒍𝒈𝟑

Unit-3: Divide and Conquer Algorithms

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Unit-3: Divide and Conquer Algorithms

Uploaded by

Copyright:

Available Formats

Analysis and Design of Algorithms (ADA)

Computer Engineering Department

 Replacing by to solve and

 Substituting equation in and equation in we have now,

 From above, we can write the general form as,

 Suppose, if we take then,

 Rewrite the equation,

 Substitute the values of and

 Suppose if we take then,

 The equation of degree in is called the characteristic equation of the recurrence,

 Which can be factorized as,

 The solution of recurrence is given as,

 The recurrence equation of above algorithm is given as,

 The recurrence can be re-written as,

 The characteristic polynomial is,

 The roots are, Here, and

 Solving these equations, we obtain

 Time taken for recursive Fibonacci algorithm grows Exponentially.

tower 1 tower 2 tower 3

 The equation can be written as,

 To convert it into a homogeneous equation, multiply with and replace by ,

 Solving equations (1) and (2), we have now

 Whose roots are,

 Solving these linear equations we get 𝟏 and 𝟐

Number of sub- Time required to

 This recurrence would arise in the analysis of a recursive algorithm.

 There are three cases:

 There are three cases:

 There are three cases:

 Example 6: (Summer 17, Summer 19)

 Example 9: (Winter 19)

 The solution of equation is given as,

where is the power of in

Input: sorted array of integer values.

Find approximate midpoint

IsIs < =midpoint

Search for the target in the area before midpoint.

Find approximate midpoint

> value of midpoint? YES.

Search for the in the area after midpoint.

Find approximate midpoint.

 The complexity of binary search is

2. We can write as,

 The above procedure still needs four half-size multiplications:

 The computation of can be done as,

 Only one multiplication is required instead of two.

 Now we can compute the required product as follows:

Step 2: Calculate and

Step 1: Split the selected array

724 521 2 98 529 31 189 451

521 724 2 98 31 529 189 451

2 31 98 189 451 521 529 724

 Applying the general case,

 Since 𝑘 the second case applies so,

 To multiply matrices, total 3 scalar multiplications are required.

 The general equation applies with and .

 Since the third case applies and

 This algorithm takes a time in since the instruction is executed exactly

and and suppose

here times multiplication is already done so 𝟗

The size of in the 10th iteration will be between i.e.,

 If we use the divide-and-conquer multiplication algorithm,