Algorithm

DIVIDE AND CONQUER

ALGORITHM
At the end of the module the students are expected to:

Understand the meaning and use of Divide and conquer

algorithm

List different applications of Divide and conquer

algorithm
In divide and conquer approach, the problem in hand, is
divided into smaller sub-problems and then each problem is
solved independently. When we keep on dividing the
subproblems into even smaller sub-problems, we may
eventually reach a stage where no more division is possible.

Those "atomic" smallest possible sub-problem (fractions) are

solved. The solution of all sub-problems is finally merged in
order to obtain the solution of an original problem.
Divide/Break

This step involves breaking the problem into smaller sub-

problems. Sub-problems should represent a part of the
original problem. This step generally takes a recursive
approach to divide the problem until no sub-problem is further
divisible. At this stage, sub-problems become atomic in
nature but still represent some part of the actual problem.
Conquer/Solve
This step receives a lot of smaller sub-problems to be solved.
Generally, at this level, the problems are considered 'solved' on
their own.

Merge/Combine
When the smaller sub-problems are solved, this stage recursively
combines them until they formulate a solution of the original
problem. This algorithmic approach works recursively and
conquer & merge steps works so close that they appear as one.
The following computer algorithms are based on divide-and-
conquer programming approach −
• Merge Sort
• Quick Sort
• Binary Search
• Strassen's Matrix Multiplication
• Closest pair (points)
There are various ways available to solve any computer problem,
but the mentioned are a good example of divide and conquer
approach.
Merge sort
Merge sort is a sorting technique based on divide and
conquer technique. With worst-case time complexity being
Ο(n log n), it is one of the most respected algorithms.

Merge sort first divides the array into equal halves and then
combines them in a sorted manner.
To understand merge sort, we take an unsorted array as the
following −

We know that merge sort first divides the whole array iteratively
into equal halves unless the atomic values are achieved. We see
here that an array of 8 items is divided into two arrays of size 4.
This does not change the sequence of appearance of items
in the original. Now we divide these two arrays into halves.

We further divide these arrays and we achieve atomic value

which can no more be divided.
Now, we combine them in exactly the same manner as they were broken
down. Please note the color codes given to these lists.

We first compare the element for each list and then combine them into
another list in a sorted manner. We see that 14 and 33 are in sorted
positions. We compare 27 and 10 and in the target list of 2 values we put 10
first, followed by 27. We change the order of 19 and 35 whereas 42 and 44
are placed sequentially.
In the next iteration of the combining phase, we compare lists
of two data values, and merge them into a list of found data
values placing all in a sorted order.

After the final merging, the list should look like this −
Merge sort keeps on dividing the list into equal halves until it can
no more be divided. By definition, if it is only one element in the
list, it is sorted. Then, merge sort combines the smaller sorted
lists keeping the new list sorted too.

Step 1 − if it is only one element in the list it is already

sorted, return.
Step 2 − divide the list recursively into two halves until it can
no more be divided.
Step 3 − merge the smaller lists into new list in sorted order.
We shall now see the
pseudocodes for merge
sort functions. As our
algorithms point out two
main functions − divide
& merge.
Merge sort works with
recursion and we shall
see our implementation
in the same way.
Program Implementation
Merge Sort
Quick Sort
Quick sort is a highly efficient sorting algorithm and is based on
partitioning of array of data into smaller arrays. A large array is
partitioned into two arrays one of which holds values smaller than
the specified value, say pivot, based on which the partition is
made and another array holds values greater than the pivot
value.

Quicksort partitions an array and then calls itself recursively twice

to sort the two resulting subarrays..
The quicksort technique is done by separating the list into two
parts. Initially, a pivot element is chosen by partitioning algorithm.
The left part of the pivot holds the smaller values than the pivot,
and right part holds the larger value. After partitioning, each
separate lists are partitioned using the same procedure

The complexity of Quicksort Technique

Time Complexity: O(n log n) for best case and average case,
O(n^2) for the worst case.
Space Complexity: O(log n)
Following animated representation explains how to find the pivot value in an
array.

The pivot value divides the list into two parts. And recursively, we find the
pivot for each sub-lists until all lists contains only one element.
Based on our understanding of partitioning in quick sort, we will now try to write
an algorithm for it, which is as follows.

Step 1 − Choose the highest index value has pivot

Step 2 − Take two variables to point left and right of the list excluding pivot
Step 3 − left points to the low index
Step 4 − right points to the high
Step 5 − while value at left is less than pivot move right
Step 6 − while value at right is greater than pivot move left
Step 7 − if both step 5 and step 6 does not match swap left and right
Step 8 − if left ≥ right, the point where they met is new pivot
Using pivot algorithm recursively, we end up with smaller possible
partitions.

Each partition is then processed for quick sort. We define

recursive algorithm for quicksort as follows −

Step 1 − Make the right-most index value pivot

Step 2 − partition the array using pivot value
Step 3 − quicksort left partition recursively
Step 4 − quicksort right partition recursively
To get more into it, let see the pseudocode for quick sort
algorithm −
Program Implementation
Quick Sort
Binary Search
Binary search is a fast search algorithm with run-time complexity of Ο(log n).
This search algorithm works on the principle of divide and conquer. For this
algorithm to work properly, the data collection should be in the sorted form.

Binary search looks for a particular item by comparing the middle most item
of the collection. If a match occurs, then the index of item is returned. If the
middle item is greater than the item, then the item is searched in the sub-
array to the left of the middle item. Otherwise, the item is searched for in the
sub-array to the right of the middle item. This process continues on the sub-
array as well until the size of the subarray reduces to zero.
For a binary search to work, it is mandatory for the target
array to be sorted. We shall learn the process of binary
search with a pictorial example. The following is our sorted
array and let us assume that we need to search the location
of value 31 using binary search.
First, we shall determine half of the array by using this
formula − mid = low + (high - low) / 2

Here it is, 0 + (9 - 0 ) / 2 = 4 (integer value of 4.5). So, 4 is

the mid of the array.
Now we compare the value stored at location 4, with the
value being searched, i.e. 31. We find that the value at
location 4 is 27, which is not a match. As the value is greater
than 27 and we have a sorted array, so we also know that the
target value must be in the upper portion of the array.
We change our low to mid + 1 and find the new mid value
again.

low = mid + 1 mid = low + (high - low) / 2

Our new mid is 7 now. We compare the value stored at

location 7 with our target value 31.
The value stored at location 7 is not a match, rather it is more
than what we are looking for. So, the value must be in the
lower part from this location.
Hence, we calculate the mid again. This time it is 5.

We compare the value stored at location 5 with our target value.

We find that it is a match.

We conclude that the target value 31 is stored at location 5.

Binary search halves the searchable items and thus reduces the count of comparisons to be made to
very less numbers.
The pseudocode of binary
search algorithms should
look like this −
Program Implementation
Binary Search
Strassen’s Matrix
Multiplication
In linear algebra, the Strassen algorithm, named after Volker
Strassen, is an algorithm for matrix multiplication. It is faster than
the standard matrix multiplication algorithm and is useful in
practice for large matrices, but would be slower than the fastest
known algorithms for extremely large matrices.

Strassen's algorithm works for any ring, such as plus/multiply, but

not all semirings, such as min-plus or boolean algebra, where the
naive algorithm still works, and so called combinatorial matrix
multiplication.
Volker Strassen first published this algorithm in 1969 and
proved that the n3 general matrix multiplication algorithm
wasn't optimal.

The Strassen algorithm is only slightly better than that, but

its publication resulted in much more research about matrix
multiplication that led to faster approaches, such as
the Coppersmith-Winograd algorithm.
Strassen in 1969 which gives an overview that how we can
find the multiplication of two 2*2 dimension matrix by the
brute-force algorithm. But by using divide and conquer
technique the overall complexity for multiplication two
matrices is reduced.

This happens by decreasing the total number if multiplication

performed at the expenses of a slight increase in the number
of addition
For multiplying the two 2*2 dimension matrices Strassen's used
some formulas in which there are seven multiplication and
eighteen addition, subtraction, and in brute force algorithm, there
is eight multiplication and four addition.

The utility of Strassen's formula is shown by its asymptotic

superiority when order n of matrix reaches infinity. Let us consider
two matrices A and B, n*n dimension, where n is a power of two.
It can be observed that we can contain four n/2*n/2 submatrices
from A, B and their product C. C is the resultant matrix
of A and B.
There are some procedures:

1. Divide a matrix of order of 2*2 recursively till we get the matrix

of 2*2.
2. Use the previous set of formulas to carry out 2*2 matrix
multiplication.
3. In this eight multiplication and four additions, subtraction are
performed.
4. Combined the result of two matrixes to find the final product or
final matrix.
Following is simple Divide and Conquer
method to multiply two square
matrices.

1) Divide matrices A and B in 4 sub-

matrices of size N/2 x N/2 as shown in
the below diagram.
2) Calculate following values
recursively. ae + bg, af + bh, ce + dg
and cf + dh.
In the above method, we do 8
multiplications for matrices of size N/2 x
N/2 and 4 additions. Addition of two
matrices takes O(N2) time. So the time
complexity can be written as T(N) =
8T(N/2) + O(N2)
In the above divide and conquer method, the main
component for high time complexity is 8 recursive calls. The
idea of Strassen’s method is to reduce the number of
recursive calls to 7. Strassen’s method is similar to above
simple divide and conquer method in the sense that this
method also divide matrices to sub-matrices of size N/2 x
N/2 as shown in the above diagram, but in Strassen’s
method, the four sub-matrices of result are calculated using
following formulae
1) The constants used in Strassen’s method are high and for
a typical application Naive method works better.
2) For Sparse matrices, there are better methods especially
designed for them.
3) The submatrices in recursion take extra space.
4) Because of the limited precision of computer arithmetic on
noninteger values, larger errors accumulate in Strassen’s
algorithm than in Naive Method.
The basic idea behind
Strassen's algorithm is
to split A & B into 8
submatricies and then
recursively compute the
submatricies of C.

This strategy is called Divide

and Conquer.
 The above strategy is the basic O(N^3) strategy.
We then use these
results to compute C's
submatricies.
Using the Master Theorem with T(n) = 8T(n/2) + O(n^2) we still get a
runtime of O(n^3).

Strassen's insight was that we don't actually need 8 recursive calls to

complete this process. We can finish the call with 7 recursive calls and a
little bit of addition and subtraction.

Strassen's 7 calls are as follows: Our new matrix C's new quadrants
1. Divide matrices A and B in 4 sub-matrices of size N/2 x N/2
as shown in the above diagram.

2. Calculate the 7 matrix multiplications recursively.

3. Compute the submatricies of C.

4. Combine these submatricies into our new matrix C

Worst case time complexity: Θ(n^2.8074)

Best case time complexity: Θ(1)

Space complexity: Θ(logn)

 Program Implementation
Strassen Matrix Multiplication Algorithm
Closest pair
The closest pair of points problem or closest
pair problem is a problem of computational
geometry: given n points in metric space, find a pair
of points with the smallest distance between them.

The closest pair problem for points in the Euclidean

plane was among the first geometric problems that
were treated at the origins of the systematic study
of the computational complexity of geometric
algorithms.
A naive algorithm of finding distances between all pairs of points in a space
of dimension d and selecting the minimum requires O(n2) time. It turns out
that the problem may be solved in O(n log n) time in a Euclidean
space or Lp space of fixed dimension d.

In the algebraic decision tree model of computation, the O(n log n) algorithm
is optimal, by a reduction from the element uniqueness problem. In the
computational model that assumes that the floor function is computable in
constant time the problem can be solved in O(n log log n) time. If we allow
randomization to be used together with the floor function, the problem can
be solved in O(n) time.
Following are the detailed steps of a O(n (Logn)^2) algortihm.

Input: An array of n points P[]

Output: The smallest distance between two points in the

given array.

As a pre-processing step, the input array is sorted according

to x coordinates.
1) Find the middle point in the sorted array, we can
take P[n/2] as middle point.

2) Divide the given array in two halves. The first subarray

contains points from P[0] to P[n/2]. The second subarray
contains points from P[n/2+1] to P[n-1].
3) Recursively find the smallest
distances in both subarrays. Let the
distances be dl and dr. Find the
minimum of dl and dr. Let the
minimum be d.
4) From the above 3 steps, we have an
upper bound d of minimum distance. Now
we need to consider the pairs such that
one point in pair is from the left half and
the other is from the right half. Consider
the vertical line passing through P[n/2]
and find all points whose x coordinate is
closer than d to the middle vertical line.
Build an array strip[] of all such points.
5) Sort the array strip[] according to y coordinates. This step is
O(nLogn). It can be optimized to O(n) by recursively sorting and
merging.
6) Find the smallest distance in strip[]. This is tricky. From the first
look, it seems to be a O(n^2) step, but it is actually O(n). It can be
proved geometrically that for every point in the strip, we only
need to check at most 7 points after it (note that strip is sorted
according to Y coordinate).
7) Finally return the minimum of d and distance calculated in the
above step (step 6)
Let Time complexity of above algorithm be T(n). Let us
assume that we use a O(nLogn) sorting algorithm. The above
algorithm divides all points in two sets and recursively calls
for two sets. After dividing, it finds the strip in O(n) time, sorts
the strip in O(nLogn) time and finally finds the closest points
in strip in O(n) time. So T(n) can expressed as follows
T(n) = 2T(n/2) + O(n) + O(nLogn) + O(n)
T(n) = 2T(n/2) + O(nLogn)
T(n) = T(n x Logn x Logn)
1) Time complexity can be improved to O(nLogn) by
optimizing step 5 of the above algorithm. We will soon be
discussing the optimized solution in a separate post.
2) The code finds smallest distance. It can be easily modified
to find the points with the smallest distance.
3) The code uses quick sort which can be O(n^2) in the worst
case. To have the upper bound as O(n (Logn)^2), a O(nLogn)
sorting algorithm like merge sort or heap sort can be used
Program Implementation
Closest Pair of Points