9/24/22
MERGE SORT
LECTURE - 03
Dr. Ragini Karwayun
DIVIDE-AND-CONQUER
• There are many ways to design an algorithm.
• Insertion sort uses incremental approach.
• An alternative approach “Divide and Conquer” is used to design sorting problem,
which yields a much less running time in worst case.
Divide and Conquer Approach
It involves three steps at each level of recursion:
• Divide: divide the n-element sequence into two subproblems of n/2 elements each.
• Conquer: sort the two subsequences recursively using merge sort. If the length of a
sequence is 1, do nothing since it is already in order.
• Combine: merge the two sorted subsequences to produce the sorted answer.
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
1
9/24/22
MERGE SORT—DIVIDE-AND-CONQUER
• Merge sort algorithm closely follows the divide and conquer approach.
• Divide: divide the n-element sequence into two subproblems of n/2 elements each.
• Conquer: sort the two subsequences recursively using merge sort. If the length of a
sequence is 1, do nothing since it is already in order.
• Combine: merge the two sorted subsequences to produce the sorted answer.
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
MERGE SORT—FUNCTIONALITY
• Merge sort algorithm is based on following approach.
• Given a sequence of n elements a[1],a[2],…….a[n], the general idea is to split them into
two sets a[1],a[2],…a[⎿n/2⏌] and a[⎿n/2⏌+1]…..a[n].
• Each set is individually sorted, and the resulting sorted sequences are merged to
produce a single sorted sequence of n elements.
• Merge is the key operation in merge sort.
• Suppose the (sub)sequence(s) are stored in the array a. Moreover, a[p..q] and a[q+1..r]
are two sorted subsequences.
• Merge(a,p,q,r) will merge the two subsequences into sorted sequence a[p..r]
• Merge(a,p,q,r) takes Q(r-p+1).
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
2
9/24/22
MERGE-SORT(A,P,R)
Merge-sort(a,p,r)
1. If p < r
2. Then q ¬ ë(p+r)/2û
3. Merge-sort(a,p,q)
4. Merge-sort(a,q+1,r)
5. Merge(a,p,q,r)
Initial call to merge-sort(a,1,n)
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
MERGE(A, P, Q, R)
MERGE(A, p, q, r)
1 n1 ← q − p + 1
2 n2 ←r − q
3 create arrays L[1 . . n1 + 1] and R[1 . . n2 + 1]
4 for i ← 1 to n1
5 L[i ] ← a[p + i − 1]
6 for j ← 1 to n2
7 R[ j ]← a[q + j ]
8 i←1
11 j←1
12 for k ← p to r
13 if L[i ] ≤ R[ j ]
14 a[k] ← l[i ]
15 i←i+1
16 else a[k] ← R[ j ]
17 j←j+1
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
3
9/24/22
TRACING MERGE-SORT
A = [13,20,17,6,12,22] 13 20 17 6 12 22 MERGESORT[1,6]
13 20 17 MERGESORT[1,3]
13 20 MERGESORT[1,2]
13 MERGESORT[1,1]
20 MERGESORT[2,2]
13 20 MERGE[1,1,2]
17 MERGESORT[3,3]
13 17 20 MERGE[1,2,3]
6 12 22 MERGESORT[4,6]
6 12 MERGESORT[4,5]
6 MERGESORT[4,4]
12 MERGESORT[5,5]
6 12 MERGE[4,4,5]
22 MERGESORT[6,6]
6 12 22 MERGE[4,5,5]
6 12 13 17 20 22 MERGE[1,3,6]
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
EXAMPLE
Draw only the function call merge sort when array is of size 10
1,10
1,5 6,10
1,3 4,5 6,8 9,10
1,2 3,3 4,4 5,5 6,7 8,8 9,9 10,10
1,1 2,2 6,6 7,7
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
4
9/24/22
MERGE-SORT TREE
• An execution of merge-sort is depicted by a binary tree
• Each node represents a recursive call of merge-sort and stores
• Unsorted sequence before the execution and its partition
• Sorted sequence at the end of the execution
• The root is the initial call
• The leaves are calls on subsequences of size 0 or 1
7 2 ½9 4 ® 2 4 7 9
7 ½2 ® 2 7 9 ½4 ® 4 9
7®7 2®2 9®9 4®4
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
10
5
9/24/22
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
11
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
7½2
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
12
6
9/24/22
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
7½2
7®7
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
13
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
7½2
7®7 2®2
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
14
7
9/24/22
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
7½2®2 7
7®7 2®2
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
15
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4
7½2®2 7 9½4 ® 4 9
7®7 2®2 9®9 4®4
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
16
8
9/24/22
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1
7 2½9 4® 2 4 7 9 3 8 6 1 ® 1 3 6 8
7½2®2 7 9½4 ® 4 9 3 8 ® 3 8 6 1 ® 1 6
9®9 4®4 3®3 8®8 6®6 1®1
7®7 2®2
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
17
EXAMPLE - PARTITION
7 2 9 4½3 8 6 1 ® 1 2 3 4 6 7 8 9
7 2½9 4® 2 4 7 9 3 8 6 1 ® 1 3 6 8
7½2®2 7 9½4 ® 4 9 3 8 ® 3 8 6 1 ® 1 6
9®9 4®4 3®3 8®8 6®6 1®1
7®7 2®2
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
18
9
9/24/22
SORTING EXAMPLE2
Sort the array [14, 10, 4, 8, 2, 12, 6, 0] in the ascending order
Divide
14, 10, 4, 8, 2, 12, 6,0
Merge-sort
14, 10, 4, 8 2, 12, 6,0
Merge-sort Merge-sort
14, 10 4, 8 2, 12 6, 0
Merge-sort Merge-sort Merge-sort Merge-sort
14 10 4 8 2 12 6 0
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
19
SORTING EXAMPLE2
Recursion and Conquer
0, 2, 4, 6, 8, 10, 12, 14
Merge
4, 8, 10, 14 0, 2, 6, 12
Merge Merge
10, 14 4, 8 2, 12 0, 6
Merge Merge Merge Merge
14 10 4 8 2 12 6 0
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
20
10
9/24/22
ANALYSIS OF DIVIDE-AND-CONQUER
Described by recursive equation
Suppose T(n) is the running time on a problem of size n.
T(n) = Q(1) if n£nc
= aT(n/b)+D(n)+C(n) if n> nc
Where a: number of subproblems
n/b: size of each subproblem
D(n): cost of divide operation
C(n): cost of combination operation
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
21
ANALYSIS OF MERGE-SORT
Divide: D(n) = Q(1)
Conquer: a=2,b=2, so 2T(n/2)
Combine: C(n) = Q(n)
T(n) = Q(1) if n=1
2T(n/2)+ Q(n) if n>1
T(n) = c if n=1
= 2T(n/2)+ cn if n>1
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
22
11
9/24/22
SOLVING THE RECURRENCE
T(n) = c if n=1
= 2T(n/2)+ cn if n>1
• We draw a recursion tree which shows successive expansions of the recurrence.
• For the original problem, we have a cost of cn, plus the two subproblems, each
costing T(n/2)
cn
T(n/2) T(n/2)
• For each of the size n/2 we have a cost cn/2, plus two subproblems, each costing
T(n/4)
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
23
SOLVING THE RECURRENCE (CONT.)
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)
• We continue expanding each node in the tree by breaking it into its constituent parts
as determined by the recurrence, until the problem sizes get down to 1, each with a
cost of c.
• To calculate the total cost of the algorithm T(n), we need to calculate the number of
levels and cost of each level
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
24
12
9/24/22
SOLVING THE RECURRENCE (CONT.)
cn cn
cn/2 cn/2 cn
lg n cn/4 cn/4 cn/4 cn/4 cn
.
. . . . . . . . .
. . . . . . . . .
. . . . . . . . .
C(1) C(1) C(1) C(1) C(1) C(1) C(1) C(1) cn
Total cost = cnlgn +cn
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
25
PROOF
ØFind the cost of each level à Each level has cost cn.
ØThe top most level has cost cn.
ØThe next level has 2 subproblems each contributing cost cn/2 à cn/2 + cn/2 = cn
ØThe next level has 4 subproblems, each costing cost cn/4 à cn/4+cn/4+cn/4+cn/4 = cn
ØEach time we go down one level, the number of subproblems doubles but the cost per
subproblem halves.
ØSo, cost per level stays the same. Level i below the top has 2i nodes, each contributing a
cost of c(n/2i)
ØTherefore Total cost = 2i c(n/2i) = cn
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
26
13
9/24/22
PROOF
ØFind the number of levels
ØAs we go down one level each time the number of subproblems doubles.
ØWe keep on dividing the problem until the boundary condition reaches, i.e., a constant
time is needed to run.
ØFor the ith level we will have n/2i subproblems.
ØWe will reach the boundary when n/2i = 1 à n=2i. à i = lg2n = lg n
ØSo, height of the tree is equal to lg n.
ØNumber of levels = height +1 = lgn + 1
ØTherefore, total cost = cost per level * Number of levels
T(n) = cn * (lg n +1) = cnlgn + cn = ⍬(nlgn)
IPEC KCS-503. Design and Analysis of Algorithms Dr. Ragini Karwayun
27
14
