Divide and Conqure

Divide and conquer
Rutal Mahajan
S.N.P.I.T.R.C., Umrakh
Rutal.mahajan@gmail.com
02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 1

Divide & Conquer
Contains following major phases:
1. Breaking the problem into several sub-problems that are
similar to the original problem but smaller in size,
2. Solve the sub-problem recursively (successively and
independently), and then
3. Combine these solutions to sub problems to create a solution
to the original problem.

Divide & Conquer

Merge sort
• The key to Merge Sort is merging two sorted lists into
one, such that if you have two lists X (x1x2…xm)
and Y(y1y2…yn) the resulting list is Z(z1z2…zm+n)
• Merge sort: O (n lgn) in worst case

Merge sort
• Merge sort Algorithm
MergeSort(arr[], left, right)
If right > left
1. Find the middle point to divide the array into two halves:
middle m = (left+right)/2
2. Call mergeSort for first half:
Call mergeSort(arr, left, m)
3. Call mergeSort for second half:
Call mergeSort(arr, m+1, right)
4. Merge the two halves sorted in step 2 and 3:
Call merge(arr, left, m, right)

MergeSort (Example) - 1

MergeSort (Example) – 1 contd.





















MergeSort (Example) – 2
merge part
14 23 45 98 6 33 42 67

14 23 45 98 6 33 42 67
Merge
14 23 45 98 6 33 42 67
Merge
14 23 45 98 6 33 42 67
6 14
Merge
14 23 45 98 6 33 42 67
6 14 23
Merge
14 23 45 98 6 33 42 67
6 14 23 33
Merge
14 23 45 98 6 33 42 67
6 14 23 33 42
Merge
14 23 45 98 6 33 42 67
6 14 23 33 42 45
Merge
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67
Merge
14 23 45 98 6 33 42 67
6 14 23 33 42 45 67 98
Merge
• Example 3: given two sorted sub arrays
L1 = { 3 10 23 54 } L2 = { 1 5 25 75}
merge(L1, L2) = { 1 3 5 10 23 25 54 75}
Merge sort
X: Y:
3 10 23 54 1 5 25 75
Result:

cont.
X: Y:
3 10 23 54 5 25 75
Result:
1

cont.
X: Y:
10 23 54 5 25 75
Result:
1 3

cont.
X: Y:
10 23 54 25 75
Result:
1 3 5

cont.
X: Y:
23 54 25 75
Result:
1 3 5 10

cont.
X: Y:
54 25 75
Result:
1 3 5 10 23

cont.
X: Y:
54 75
Result:
1 3 5 10 23 25

cont.
X: Y:
75
Result:
1 3 5 10 23 25 54

cont.
X: Y:
Result:
1 3 5 10 23 25 54 75

Analysis of Merge Sort
If we look at the Merge Sort algorithm (from last class) more closely, we
notice that the running time on an array of size n accrues as follows:
1. First, we spend time O(1) for computing m.

2. Then, we make two recursive calls to Merge Sort, with arrays of sizes
⌊(n − 1)/2⌋ and ⌈(n − 1)/2⌉.
3. Finally, we call Merge. Merge goes through the two sub arrays with one
loop, always increasing one of i and j. Thus, it takes time Θ(n).

Quick sort
Divide:
Partition the array A[p….r] in to two sub arrays A[p...q-1] and A[q+1...r] such that
each element of A[p…q-1] < = A[q]. q is an index from which partitioning is done.
Conquer:
Sort the two subarrays A[p...q-1] and A[q+1...r] by recursive calls to quick sort.
Combine:
Since the sub arrays are sorted in place, no work is needed to combine them: entire
array A[p...r] is sorted now.
….

Advantages:
1. in place sorting
2. O(nlogn) time for average case sorting
3. Efficient in practice
Disadvantages:
1. not stable when two elements are same
2. O(n2) time for worst case sorting
3. Efficient in practice

Algorithm
Partition takes place in O(n) time

Algorithm
Given an array of n elements (e.g. integers):
If array only contains one element, return

Else
pick one element to use as pivot.
Partition elements into two sub-arrays:
Elements less than or equal to pivot
Elements greater than pivot
Quicksort () two sub-arrays
Return results
End if
Partition takes place in O(n) time

pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

1. While data[i] <= data[pivot]
++i
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
2. While data[j] > data[pivot]
--j
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i > j
swap data[i] and data[j]
pivot_index = 0 40 20 10 80 60 50 7 30 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If I < j
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
4. When i and j crossed each other
compare data[j]with data[pivot]
swap if required.
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
swap if required. Go to step 1.
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
swap if required. Go to step 1.
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
4. While j > i, go to 1.
5. When I and j crossed each other and compare data[j] with data[pivot]
Swap data[j] and data[pivot]
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 60 50 7 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
5. Swap data[j] and data[pivot]
pivot_index = 0 40 20 10 30 7 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

++i
--j
3. If i < j
5. Swap data[j] and data[pivot]
pivot_index = 4 7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

Partition Result
7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
<= data[pivot] > data[pivot]

Recursion: Quicksort Sub-arrays
7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• What is best case running time?

Quicksort Analysis
distributed.
– Recursion:
1. Partition splits array in two sub-arrays of size n/2
2. Quicksort each sub-array

Quicksort Analysis
distributed.
– Recursion:
– Depth of recursion tree?

Quicksort Analysis
distributed.
– Recursion:
– Depth of recursion tree? O(log2n)

Quicksort Analysis
distributed.
– Recursion:
– Number of times in partition() called?

Quicksort Analysis
distributed.
– Recursion:
– Number of times in partition() called? O(n)

Quicksort Analysis
distributed.
• Best case running time: O(n log2n)

Quicksort Analysis
distributed.
• Worst case running time?

Quicksort: Worst Case
• Assume first element is chosen as pivot.
• Assume we get array that is already in order:
pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
i j

pivot_index = 0 2 4 10 12 13 50 57 63 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]

Quicksort Analysis
distributed.
– Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
– Depth of recursion tree?

Quicksort Analysis
distributed.
– Recursion:
– Depth of recursion tree? O(n)

Quicksort Analysis
distributed.
– Recursion:
– Number of accesses per partition?

Quicksort Analysis
distributed.
– Recursion:
– Number of accesses per partition? O(n)

Quicksort Analysis
distributed.
• Worst case running time: O(n2)!!!

Quicksort Analysis
distributed.
• Worst case running time: O(n2)!!!
• What can we do to avoid worst case?

Pick median value of three elements from data array:
data[0], data[n/2], and data[n-1].
Use this median value as pivot.

Quick Sort Example
1. (5 3 1 9 8 2 4 7) take 5 as a key
2. (36 15 40 1 60 20 55 25 50 20)
hint. take key =A[10] i.e 20
02-Oct-17 Sorting Algorithms(DAA B.E.5th semester) by Rutal Mahajan 100

Integer Multiplication Using Divide
and Conquer
(Karastuba algorithm)

A n- bit number
B n- bit number
Result= Multiply A and B.
E.g. 12345 * 6789
What is the running time of conventional algorithm?? O (n2)

Divide and Conquer method for dividing a number
Divide each number into two halves.

X= x1Bm + x0
Y= y1Bm + y0
Then…
XY = (x1Bm + x0) y1Bm + y0
= x1 y1B2m + (x0 y1 + x1 y0)Bm + x0 y0
Run time…
T(n) = 4T (n/2) +O(n)

Procedure Karastuba (X, Y )
Input: X, Y : n-digit integers.
Output: the product X ∗ Y .
Comment: We assume n = 2k, by prefixing X, Y with zeros if necessary.
1. if n = 1 then use multiplication table to find result := X ∗ Y

2. else split X, Y in half:
3. X =: 10n/2X1 + X0
4. Y =: 10n/2Y1 + Y0
Comment: X1, X0, Y1, Y0 each have n/2 digits.
5. U := karastuba(X1, Y1)
6. V := karastuba (X0, Y0)
7. W := karastuba (X1 − X0, Y1 − Y0)
8. Z := U + V − W
9. result := 10nU + 10n/2Z + V
Comment: So U = X1∗Y1, V = X2∗Y2, W = (X1−X2)∗(Y1−Y2),and therefore
result = X1 ∗ Y2 + X2 ∗ Y1. Finally we conclude that
result =10nX1 ∗ Y1 + 10n/2(X1 ∗ Y2 + X2 ∗ Y1) + X2 ∗ Y2 = X ∗ Y .
10. return result
Karastuba algorithm requires only 3 sub- problems
X= x1Bm + x0
Y= y1Bm + y0
Then…
XY = (x1Bm + x0) y1Bm + y0
= x1 y1B2m + (x0 y1 + x1 y0)Bm + x0 y0
= Z2 B2m + Z1(x0 y1 + x1 y0)Bm + Z0
Z2 = x1 y1
Z1 = x1 y0 + x0 y1
Z0 = x0 y0
Run time…
T(n) = 3T (n/2) +O(n)
So after solving it we get, T(n) = O(nlog3)
Strassen’s matrix multiplication
algorithm

Basic matrix multiplication algorithm
Suppose we want to multiply two matrices of size N

x N: for example A x B = C.
C11 = a11b11 + a12b21

C12 = a11b12 + a12b22
C21 = a21b11 + a22b21
C22 = a21b12 + a22b22
2x2 matrix multiplication can be
accomplished in 8 multiplication.(2log28 =23)

Basic matrix multiplication algorithm
void matrix_mult (){

code
for (i = 1; i <= N; i++) {
for (j = 1; j <= N; j++) {
compute Ci,j;
}
}}
N
Ci , j   ai ,k bk , j
Time analysis from k 1
code: N N N
Thus T ( N )   c  cN 3  O( N 3 )
i 1 j 1 k 1

Basic matrix multiplication operations
Analysis
Time Analysis from algorithm:
1. line 3 take constant time.

2. lines 12–15 is θ (n2)
(adding two n/2 x n/2)
matrices takes time n2/4 =
O (n2). )
3. There are 8 recursive
calls lines 4–11.
So

Strassen’s matrix multiplication
• Strassen showed that 2x2 matrix multiplication

can be accomplished in 7 multiplication and 18
additions or subtractions. .(2log27 =22.807)
• This reduce can be done by Divide and Conquer

Approach.

Divide and Conquer matrix multiplication
A  B = R
A0 A1 B0 B1 A0B0+A1B2 A0B1+A1B3
 =
A2 A3 B2 B3 A2B0+A3B2 A2B1+A3B3
•Divide matrices into sub-matrices: A0 , A1, A2 etc

•Use blocked matrix multiply equations
•Recursively multiply sub-matrices
Divide and Conquer matrix multiplication
a0  b0 = a0  b0
• Terminate recursion with a simple base case

Strassen’s matrix multiplication algorithm
P1 = (A11+ A22)(B11+B22) C11 = P1 + P4 - P5 + P7

P2 = (A21 + A22) * B11 C12 = P3 + P5
P3 = A11 * (B12 - B22) C21 = P2 + P4
P4 = A22 * (B21 - B11) C22 = P1 + P3 - P2 + P6
P5 = (A11 + A12) * B22
P6 = (A21 - A11) * (B11 + B12)
P7 = (A12 - A22) * (B21 + B22)
Time analysis:
1. line 3 take constant time.
2. lines 11–14 takes θ(n2).
3. 7 recursive calls lines 4-10

Divide and Conqure

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Divide and Conqure

Uploaded by

Copyright:

Available Formats

Divide and conquer

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 1

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 2

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 3

• Merge sort: O (n lgn) in worst case

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 4

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 5

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 6

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 7

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 8

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 9

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 10

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 11

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 12

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 13

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 14

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 15

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 16

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 17

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 18

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 19

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 20

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 21

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 22

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 23

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 24

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 25

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 26

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 27

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 28

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 39

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 40

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 41

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 42

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 43

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 44

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 45

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 46

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 47

1. First, we spend time O(1) for computing m.

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 48

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 49

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 50

Partition takes place in O(n) time

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 51

If array only contains one element, return

Partition takes place in O(n) time

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 52

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 53

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 54

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 55

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 56

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 57

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 58

[0] [1] [2] [3] [4] [5] [6] [7] [8]

02-Oct-17 Divide and Conquer (DAA B.E.5th semester) by Rutal Mahajan 59

[0] [1] [2] [3] [4] [5] [6] [7] [8]