Design and Analysis

of Algorithms
(CSE 373)

Lecture 3_4
Divide and Conquer (Mergesort)
 Divide and Conquer
Recursive in structure

Divide the problem into independent sub-problems that are

similar to the original but smaller in size
Conquer the sub-problems by solving them recursively. If they
are small enough, just solve them in a straightforward manner.
This can be done by reducing the problem until it reaches the
base case, which is the solution.
Combine the solutions of the sub-problems to create a solution
to the original problem
 Example: Merge Sort
Sorting Problem: Sort a sequence of n elements into non-
decreasing order.

Divide: Divide the n-element sequence to be sorted into

two subsequences of n/2 elements each

Conquer: Sort the two subsequences recursively using

merge sort.

Combine: Merge the two sorted subsequences to

produce the sorted answer.
Merge Sort – Example
Original Sequence Sorted Sequence
18 26 32 6 43 15 9 1 1 6 9 15 18 26 32 43

18 26 32 6 43 15 9 1 6 18 26 32 1 9 15 43
43

18 26 32 6 43 15 9 1 18 26 6 32 15 43 1 9

18 26 32 6 43 15 9 1 18 26 32 6 43 15 9 1

18 26 32 6 43 15 9 1
98 23 45 14 6 67 33 42
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23

23 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

98 23 45 14

23 98 14 45

14 23 45 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14

23 98 14 45

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67

23 98 14 45 6 67

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67

Merge
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

Merge
98 23 45 14 6 67 33 42

6 14 23 33 42 45 67 98
Merge-Sort (A, p, r)
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers

MergeSort (A, p, r) // sort A[p..r] by divide & conquer

1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]

Initial Call: MergeSort(A, 1, n)

Merging two sorted subsequeces

6 18 56 62 1 9 15 43
Merging two sorted subsequeces
Unsorted

6 18 56 62 1 9 15 43

Sorted Sorted
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging
Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
6 18 56 62 1 9 15 43
Merging

1 6 9 15 18 43 56 62
Left half

Right half

Minimum between first elements in both halves

Merging two sorted subsequeces
1 6 9 15 18 43 56 62
Merging

1 6 9 15 18 43 56 62
Left half

Right half

Minimum between first elements in both halves

Merge – Example
A … 61 86 26
8 32 1 32
9 26 9 42 43 …
k k k k k k k k k

L 6 8 26 32  R 1 9 42 43 
i i i i i j j j j j

Comp 122
Merging two sorted subsequeces
Merge(A, p, q, r)
1 n1  q – p + 1
2 n2  r – q Input: Array containing
3 for i  1 to n1 sorted subarrays A[p..q] and
4 do L[i]  A[p + i – 1] A[q+1..r].
5 for j  1 to n2 Output: Merged sorted
6 do R[j]  A[q + j] subarray in A[p..r].
7 L[n1+1]  
8 R[n2+1]  
9 i1
10 j1
11 for k p to r Sentinels, to avoid having to
12 do if L[i]  R[j] check if either subarray is
13 then A[k]  L[i]
fully copied at each step.
14 ii+1
15 else A[k]  R[j]
16 jj+1
Time complexity of Merge
Merge(A, p, q, r) //Let r-p+1 = n
1 n1  q – p + 1 //O(1)
2 n2  r – q //O(1)
Input: Array containing
3 for i  1 to n1 //O(q-p+1)
sorted subarrays A[p..q] and
4 do L[i]  A[p + i – 1] A[q+1..r].
5 for j  1 to n2 //O(r-q)
6 do R[j]  A[q + j]
Output: Merged sorted
subarray in A[p..r].
7 L[n1+1]  
8 R[n2+1]  
9 i1
10 j1
11 for k p to r //O(r-p+1) = O(n)
12 do if L[i]  R[j]
13 then A[k]  L[i]
14 ii+1
15 else A[k]  R[j]
16 jj+1
//Total time: O(n)
Merge Sort
MergeSort (A, p, r) // sort A[p..r] by divide & conquer
1 if p < r
2 then q  (p+r)/2 //divide
3 MergeSort (A, p, q) //conquer
4 MergeSort (A, q+1, r) //conquer
5 Merge (A, p, q, r) //combine: merge A[p..q] with A[q+1..r]

Initial Call: MergeSort(A, 1, n)

Analysis of Merge Sort
Statement Cost
MergeSort (A, p, r) //initial call: MergeSort(A,1,n) T(n) [let]
1 if p < r
2 then q  (p+r)/2
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r)

So T(n) = O(1) when n = 1, and 2T(n/2) + O(n) when

n>1
Analysis of Merge Sort
Statement Cost
MergeSort (A, p, r) T(n), to sort n elements
1 if p < r O(1)
2 then q  (p+r)/2 O(1)
3 MergeSort (A, p, q) T(n/2), to sort n/2 elements
4 MergeSort (A, q+1, r) T(n/2), to sort n/2 elements
5 Merge (A, p, q, r) O(n)

So T(n) = O(1) when n = 1, and 2T(n/2) + O(n)

when n > 1
Analysis of Merge Sort
1. The divide step takes constant time, regardless of the
subarray size. After all, the divide step just computes
the midpoint q of the indices p and r. Recall that in big-
O notation, we indicate constant time by O(1).
2. The conquer step, where we recursively sort two
subarrays of approximately n/2 elements each, takes
some amount of time, but we'll account for that time
when we consider the subproblems.
3. The combine step merges a total of n elements, taking
O(n) time.

let's say that the divide and combine steps

together take cn time for some constant c
Analysis of Merge Sort
Elements to sort /
Recursion-tree Method
Recursion Trees
•Show successive expansions of recurrences using trees.

•Keep track of the time spent on the subproblems of a divide and

conquer algorithm.

•Help organize the algebraic bookkeeping necessary to solve a

recurrence.
Recursion Tree – Example
Running time of Merge Sort:
T(n) = O(1) if n = 1
T(n) = 2T(n/2) + O(n) if n > 1

Rewrite the recurrence as

T(n) = c if n = 1
T(n) = 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and
time per array element for the divide and
combine steps.
Recursion Tree for Merge Sort

For the original problem,
we have a cost of cn, plus
two subproblems each of
size (n/2) and running time
T(n/2).
Each of the size n/2 problems has
a cost of cn/2 plus two
subproblems, each costing T(n/4).
cn

cn
Cost of divide
and merge.
cn/2 cn/2

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.

cn cn

cn/2 cn/2 cn

log2n + 1

cn/4 cn/4 cn/4 cn/4 cn

c c c c c c cn
Total : cnlog2n + cn
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn •Each level has total cost cn.

•Each time we go down one level,

the number of subproblems
cn/2 cn/2
doubles, but the cost per
subproblem halves  cost per
level remains the same.
cn/4 cn/4 cn/4 cn/4
•There are lg n + 1 levels, height is
lg n. (Assuming n is a power of 2.)

•Total cost = sum of costs at each

c c c c c c
level = (lg n + 1)cn = cnlgn + cn =
O(n lgn).
Analysis: solving recurrence
 All cases have same efficiency: O(n log n)
T(n) = 2T(n/2) + O(n), and O(n)=cn
n Since n=2k, we have k=log2 n
T (n)  2T ( )  cn
2
n
n cn T (n)  2 T ( k )  kn
k
 2(2T ( )  )  cn 2
4 2
 n  n log n
n
 4T ( )  2cn  O (n log n)
4
n cn
 4(2T ( )  )  2cn
8 4
n
 8T ( )  3cn
8
n
 2 k T ( k )  kn
2