9/24/22

MERGE SORT

LECTURE - 03
Dr. Ragini Karwayun

DIVIDE-AND-CONQUER
• There are many ways to design an algorithm.
• Insertion sort uses incremental approach.
• An alternative approach “Divide and Conquer” is used to design sorting problem,
which yields a much less running time in worst case.
Divide and Conquer Approach
It involves three steps at each level of recursion:
• Divide: divide the n-element sequence into two subproblems of n/2 elements each.
• Conquer: sort the two subsequences recursively using merge sort. If the length of a
sequence is 1, do nothing since it is already in order.
• Combine: merge the two sorted subsequences to produce the sorted answer.

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

1
 9/24/22

MERGE SORT—DIVIDE-AND-CONQUER
• Merge sort algorithm closely follows the divide and conquer approach.

• Divide: divide the n-element sequence into two subproblems of n/2 elements each.
• Conquer: sort the two subsequences recursively using merge sort. If the length of a
sequence is 1, do nothing since it is already in order.
• Combine: merge the two sorted subsequences to produce the sorted answer.

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

MERGE SORT—FUNCTIONALITY
• Merge sort algorithm is based on following approach.
• Given a sequence of n elements a[1],a[2],…….a[n], the general idea is to split them into
two sets a[1],a[2],…a[⎿n/2⏌] and a[⎿n/2⏌+1]…..a[n].
• Each set is individually sorted, and the resulting sorted sequences are merged to
produce a single sorted sequence of n elements.
• Merge is the key operation in merge sort.
• Suppose the (sub)sequence(s) are stored in the array a. Moreover, a[p..q] and a[q+1..r]
are two sorted subsequences.
• Merge(a,p,q,r) will merge the two subsequences into sorted sequence a[p..r]
• Merge(a,p,q,r) takes Q(r-p+1).

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

2
 9/24/22

MERGE-SORT(A,P,R)
Merge-sort(a,p,r)
1. If p < r
2. Then q ¬ ë(p+r)/2û
3. Merge-sort(a,p,q)
4. Merge-sort(a,q+1,r)
5. Merge(a,p,q,r)

Initial call to merge-sort(a,1,n)

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

MERGE(A, P, Q, R)
MERGE(A, p, q, r)
1 n1 ← q − p + 1
2 n2 ←r − q
3 create arrays L[1 . . n1 + 1] and R[1 . . n2 + 1]
4 for i ← 1 to n1
5 L[i ] ← a[p + i − 1]
6 for j ← 1 to n2
7 R[ j ]← a[q + j ]
8 i←1
11 j←1
12 for k ← p to r
13 if L[i ] ≤ R[ j ]
14 a[k] ← l[i ]
15 i←i+1
16 else a[k] ← R[ j ]
17 j←j+1

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

3
 9/24/22

TRACING MERGE-SORT
A = [13,20,17,6,12,22] 13 20 17 6 12 22 MERGESORT[1,6]
13 20 17 MERGESORT[1,3]
13 20 MERGESORT[1,2]
13 MERGESORT[1,1]
20 MERGESORT[2,2]
13 20 MERGE[1,1,2]
17 MERGESORT[3,3]
13 17 20 MERGE[1,2,3]
6 12 22 MERGESORT[4,6]
6 12 MERGESORT[4,5]
6 MERGESORT[4,4]
12 MERGESORT[5,5]
6 12 MERGE[4,4,5]
22 MERGESORT[6,6]
6 12 22 MERGE[4,5,5]
6 12 13 17 20 22 MERGE[1,3,6]

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

EXAMPLE
Draw only the function call merge sort when array is of size 10
1,10

1,5 6,10

1,3 4,5 6,8 9,10

1,2 3,3 4,4 5,5 6,7 8,8 9,9 10,10

1,1 2,2 6,6 7,7

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

4
 9/24/22

MERGE-SORT TREE
• An execution of merge-sort is depicted by a binary tree
• Each node represents a recursive call of merge-sort and stores
• Unsorted sequence before the execution and its partition
• Sorted sequence at the end of the execution
• The root is the initial call
• The leaves are calls on subsequences of size 0 or 1

7 2 ½9 4 ® 2 4 7 9

7 ½2 ® 2 7 9 ½4 ® 4 9

7®7 2®2 9®9 4®4

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

10

5
 9/24/22

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

11

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

7½2

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

12

6
 9/24/22

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

7½2

7®7

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

13

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

7½2

7®7 2®2

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

14

7
 9/24/22

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

7½2®2 7

7®7 2®2

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

15

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4

7½2®2 7 9½4 ® 4 9

7®7 2®2 9®9 4®4

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

16

8
 9/24/22

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1

7 2½9 4® 2 4 7 9 3 8 6 1 ® 1 3 6 8

7½2®2 7 9½4 ® 4 9 3 8 ® 3 8 6 1 ® 1 6

9®9 4®4 3®3 8®8 6®6 1®1

7®7 2®2

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

17

EXAMPLE - PARTITION

7 2 9 4½3 8 6 1 ® 1 2 3 4 6 7 8 9

7 2½9 4® 2 4 7 9 3 8 6 1 ® 1 3 6 8

7½2®2 7 9½4 ® 4 9 3 8 ® 3 8 6 1 ® 1 6

9®9 4®4 3®3 8®8 6®6 1®1

7®7 2®2

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

18

9
 9/24/22

SORTING EXAMPLE2
Sort the array [14, 10, 4, 8, 2, 12, 6, 0] in the ascending order
Divide

14, 10, 4, 8, 2, 12, 6,0

Merge-sort

14, 10, 4, 8 2, 12, 6,0

Merge-sort Merge-sort

14, 10 4, 8 2, 12 6, 0
Merge-sort Merge-sort Merge-sort Merge-sort

14 10 4 8 2 12 6 0

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

19

SORTING EXAMPLE2
Recursion and Conquer

0, 2, 4, 6, 8, 10, 12, 14

Merge

4, 8, 10, 14 0, 2, 6, 12

Merge Merge

10, 14 4, 8 2, 12 0, 6
Merge Merge Merge Merge

14 10 4 8 2 12 6 0

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

20

10
 9/24/22

ANALYSIS OF DIVIDE-AND-CONQUER
Described by recursive equation
Suppose T(n) is the running time on a problem of size n.

T(n) = Q(1) if n£nc

= aT(n/b)+D(n)+C(n) if n> nc

Where a: number of subproblems

n/b: size of each subproblem
D(n): cost of divide operation
C(n): cost of combination operation

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

21

ANALYSIS OF MERGE-SORT
Divide: D(n) = Q(1)
Conquer: a=2,b=2, so 2T(n/2)
Combine: C(n) = Q(n)
T(n) = Q(1) if n=1
2T(n/2)+ Q(n) if n>1

T(n) = c if n=1
= 2T(n/2)+ cn if n>1

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

22

11
 9/24/22

SOLVING THE RECURRENCE

T(n) = c if n=1
= 2T(n/2)+ cn if n>1

• We draw a recursion tree which shows successive expansions of the recurrence.

• For the original problem, we have a cost of cn, plus the two subproblems, each
costing T(n/2)
cn

T(n/2) T(n/2)

• For each of the size n/2 we have a cost cn/2, plus two subproblems, each costing
T(n/4)
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

23

SOLVING THE RECURRENCE (CONT.)

cn

cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

• We continue expanding each node in the tree by breaking it into its constituent parts
as determined by the recurrence, until the problem sizes get down to 1, each with a
cost of c.
• To calculate the total cost of the algorithm T(n), we need to calculate the number of
levels and cost of each level

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

24

12
 9/24/22

SOLVING THE RECURRENCE (CONT.)

cn cn

cn/2 cn/2 cn

lg n cn/4 cn/4 cn/4 cn/4 cn

.
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
C(1) C(1) C(1) C(1) C(1) C(1) C(1) C(1) cn

Total cost = cnlgn +cn

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

25

PROOF
ØFind the cost of each level à Each level has cost cn.
ØThe top most level has cost cn.
ØThe next level has 2 subproblems each contributing cost cn/2 à cn/2 + cn/2 = cn
ØThe next level has 4 subproblems, each costing cost cn/4 à cn/4+cn/4+cn/4+cn/4 = cn
ØEach time we go down one level, the number of subproblems doubles but the cost per
subproblem halves.
ØSo, cost per level stays the same. Level i below the top has 2i nodes, each contributing a
cost of c(n/2i)
ØTherefore Total cost = 2i c(n/2i) = cn

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

26

13
 9/24/22

PROOF
ØFind the number of levels
ØAs we go down one level each time the number of subproblems doubles.
ØWe keep on dividing the problem until the boundary condition reaches, i.e., a constant
time is needed to run.
ØFor the ith level we will have n/2i subproblems.
ØWe will reach the boundary when n/2i = 1 à n=2i. à i = lg2n = lg n
ØSo, height of the tree is equal to lg n.
ØNumber of levels = height +1 = lgn + 1

ØTherefore, total cost = cost per level * Number of levels

T(n) = cn * (lg n +1) = cnlgn + cn = ⍬(nlgn)

IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun

27

14