Data Structures and

Algorithms - II

• Text Books:
• T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to
Algorithms, MIT Press, 3/e, 2009.
• J. Kleinberg and E. Tardos, Algorithm Design, Pearson, 2006.
• S. Sahni, Data Structures, Algorithms, and Applications in C++, Silicon
Press, 2/e, 2005.
 Good Algorithm
• For the same problem, there can be infinitely many
algorithms
• Good algorithm means efficient algorithm
• Efficient algorithm:
– The algorithm with smaller running time (time
complexity) and takes less memory (space complexity)
• These measures of efficiency are seen as a function of
input size
• The time taken by the program will increases
with the increase in the input size
• Measuring running time:
– Experimental study
• Correctness of program only on the sample inputs

4
 General Methodology for Analysing
Running Time
• Using of high level description of the algorithm, instead of
testing one of its implementations
• This helps in evaluating the efficiency of any algorithm (as
a function of input size) independent of the hardware and
software requirement
• This analysis helps algorithm to scale automatically with
varying input size
• High level description specify the key aspect of algorithm
• High level description is given in the form of pseudo-code
• Identify the primitive operations in the pseudo-code
– Data movement statements, control statements, arithmetic
and logical operation statements
• Count the number primitive operations executed by the
algorithm

5
Insertion Sort
 Insertion Sort
INPUT : A[1..n], array of integers
OUTPUT: Rearrangement of A such that A[1]≤A[2] ≤… ≤A[n]
INSERTION-SORT (A)
1. for j = 2 to n
2. key = A[j]
3. // Insert A[j] into the sorted sequence A[1 … j - 1]
4. i=j-1
5. while i > 0 and A[i] > key
6. A[i+1] = A[i]
7. i=i-1
8. A[i + 1] = key

10
Analysis of Insertion Sort Algorithm

n
n 1

n 1
n

t
j 2
j

 (t
j 2
j  1)
n

 (t
j 2
j  1)

n 1

15
 Analysis of Insertion Sort Algorithm
• Total Time T(n):
n  n   n 
T ( n)  c1n  c2 (n  1)  c3 (n  1)  c4  t j  c5   (t j  1)   c6   (t j  1)   c7 (n  1)
j 2  j 2   j 2 

• This can be reorganized as

 n   n 
T (n)  (c1  c2  c3  c7 )n  c4   t j   (c5  c6 )   (t j  1)   (c2  c3  c7 )
 j 2   j 2 

16
 Analysis of Insertion Sort Algorithm
• Best case: t j  1 Linear time
T ( n)  (c1  c2  c3  c4  c7 )n  (c2  c3  c7 )

T (n)  a1n  a2 a1 and a2 are constants

• Worst case: t j  j Quadratic time

  c  c  c  c c c 
T (n)   c1  c2  c3  c7   4 5 6   n   4 5 6  n 2  (c2  c3  c4  c5  c6  c7 )
  2   2 

T (n)  b1n  b2 n 2  b3 b1 , b2 and b3 are constants

j
• Average case: t j  Quadratic time
2
 c 3  c c c   c c c  
T ( n)   c1  c2  c3  4  c7  c5  c6   n   4 5 6  n 2   c2  c3   4 5 6 
 7c
 4 4   4    2  

T (n)  A1n  A2 n 2  A3 A1 , A2 and A3 are constants

17
 Readings
• Read Chapter 1 and 2 in Corman book
• Read Chapter 2 in Sahani book

18
Divide and Conquer Method

19
 Merge Sort
• Let A be the set of n numbers to be sorted
• Divide:
– Split A into two sets A1 and A2 with roughly equal size

• Conquer:
– Sort A1 and A2 recursively using merge sort

• Combine:
– Merge A1 and A2 into sorted sequence A

21
 Pseudo-code for Merge Sort
MERGE-SORT (A, p, r)
1. if p < r then
2. q = (p + r)/2
3. MERGE-SORT (A, p, q)
4. MERGE-SORT (A, q+1, r)
5. MERGE (A, p, q, r)

• Initial call: MERGE-SORT (A, 1, n)

MERGE (A, p, q, r)
• Take smallest of the two top most elements of sorted
subsequence A1 = A[p … q] and A2 = A[q+1 … r]
– Put it into the resulting sequence
• Repeat this until both the subsequences are empty
– Each time append the smallest element in the resulting
sequence
• Copy the resulting sequence into A[p … r]
22
 Merging Two Sequences
MERGE (A, p, q, r)
MERGE (A, 1, 4, 8)
1. n1 = q - p + 1 p = 1; q = 4; r = 8
2. n2 = r - q
k k k k k k k k k
3. A1[n1+1] = ∞ 1 2 3 4 5 6 7 8

4. A2[n2+1] = ∞ A 1 2 2 3 4 5 6 7
5. i = 1 i i i i i j j j j j
6. j = 1 1 2 3 4 5 1 2 3 4 5
7. for k = p to r A 1 2 4 5 7 ∞ A2 1 2 3 6 ∞
8. if A1[i] ≤ A2[j]
9. A[k] = A1[i]
cn
10. i=i+1
11. else
12. A[k] = A2[j]
13. j=j+1
23
Analysis of Divide and Conquer Method
• Recurrence relations:
– Function that described in terms of its values in previous
points on small inputs
– E.g.: f(n) = f(n - 1) + c for n > 0 and f(1)=0
• T(n) : Time taken to solve the problem of size n
• k : Number pieces the problem is divided
• l: Number of factors the problem is divided
• General recurrence relation for divide and conquer
method:

Intial condition if n  1
T ( n)  n
k T    Time for Divide  Time for Combine if n  1
l

42
 Analysis of Merge Sort
• T(n) : Time taken to solve the problem of size n
• k : Number pieces the problem is divided = 2
• l: Number of factors the problem is divided = 2
• Recurrence relation for merge sort:
1 if n  1
T (n)  n
2 T    cn if n  1
2

• Running time of merge sort: T (n)  c1 n log 2 n  c2 n

– Here, c1 and c2 are some constants

43
 Quick Sort
• It is a quick algorithm i.e. it is fast in practice
– It has small number of constants
• It is an in-place sorting algorithm
• Let A be the set of n numbers to be sorted
• Divide:
– Partition the array A into two parts, one is left part and
another is right part
– Every element in the left part is lower than every
element in the right part
• Conquer:
– Sort the left and right parts recursively using quick sort
• Combine:
– It is a trivial case since the sorting done in-place

44
 Pseudo-code for Quick Sort
QUICK-SORT (A, p, r)
1. if p < r then
2. q = PARTITION (A, p, r)
3. QUICK-SORT (A, p, q)
4. QUICK-SORT (A, q+1, r)

• Initial call: QUICK-SORT (A, 1, n)

q = PARTITION (A, p, r)
• Partitioning is done around a pivot element
• From the array A, take one of the elements as pivot
• Everything smaller than the pivot element will be in left
part
• Everything larger than the pivot element will be in right
part

45
 Procedure for Partition
PARTITION (A, p, r)
q = PARTITION (A, 1, 8)
1. pivot = A[r] // Pivot
p = 1; r = 8
2. i = p - 1
3. j = r + 1 pivot = 3
4. while TRUE
5. do j j j j j j
6. j=j-1 1 2 3 4 5 6 7 8

7. while A[j] > pivot A 53 22 2

4 7 1 6 4
1 7 2 5
3
8. do i i i i i i
9. i=i+1
10. while A[i] ≤ pivot
11. if j > i
12. exchange A[i] with A[j] Running time = cn
13. else if j = i
14. return j - 1
15 else
16 return j
46
 Procedure for Partition
QUICK-SORT (A, p, r)
1. if p < r then

2. q = PARTITION (A, p, r)
3. QUICK-SORT (A, p, q)
4. QUICK-SORT (A, q+1, r)
pivot = 3
q
1 2 3 4 5 6 7 8

A 53 22 24 7 1 6 4
1 7 2 5
3

47
 Procedure for Partition
QUICK-SORT (A, p, r)
1. if p < r then

2. q = PARTITION (A, p, r)
3. QUICK-SORT (A, p, q)
4. 1 QUICK-SORT
2 3 4 (A,
5 q+1,
6 r) 7 8

A 53 22 24 7 1 6 4
1 7 2 5
3
pivot = 1
q q q q q
1 2 3 4 5 6 7 8

A 1 2 2 1
3 3
i i i

48
 Procedure for Partition
QUICK-SORT (A, p, r)
1. if p < r then

2. q = PARTITION (A, p, r)
3. QUICK-SORT (A, p, q)
4. QUICK-SORT (A, q+1, r)
1 2 3 4 5 6 7 8

A 53 22 24 7 1 6 4
1 7 2 5
3
1 2 3 4 5 6 7 8

A 1 2 2 1
3 3
1 2 3 4 5 6 7 8

A 1

49
 Procedure for Partition
QUICK-SORT (A, p, r)
1. if p < r then

2. q = PARTITION (A, p, r)
3. QUICK-SORT (A, p, q)
4. QUICK-SORT (A, q+1, r)
1 2 3 4 5 6 7 8

A 53 22 24 7 1 6 4
1 7 2 5
3
1 2 3 4 5 6 7 8

A 1 2 2 1
3 3
1 2 3 4 5 6 7 8

A 1
pivot = 3
q q q
1 2 3 4 5 6 7 8

A 1 22 2 3
3
i i i i 50
 Procedure for Partition
QUICK-SORT (A, p, r) pivot = 2
q q q
1. if p < r then 1 2 3 4 5 6 7 8

2. q = PARTITION (A, p, r) A 1 2 2
3. QUICK-SORT (A, p, q) i i i
4. QUICK-SORT (A, q+1, r)
1 2 3 4 5 6 7 8

A 53 22 24 7 1 6 4
1 7 2 5
3
1 2 3 4 5 6 7 8

A 1 2 2 1
3 3
1 2 3 4 5 6 7 8

A 1
1 2 3 4 5 6 7 8

A 1 22 2 3
3

51
 Procedure for Partition
1 2 3 4 5 6 7 8
QUICK-SORT (A, p, r)
1. if p < r then A 1 2 2
1 2 3 4 5 6 7 8
2. q = PARTITION (A, p, r)
A 1 2
3. QUICK-SORT (A, p, q)
1 2 3 4 5 6 7 8
4. QUICK-SORT (A, q+1, r)
1 2 3 4 5 6 7 8
A 1 2 2
A 53 22 24 7 1 6 4
1 7 2 5
3 1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
A 1 2 2 3
A 1 2 2 1
3 3
pivot = 5
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
A 1 A 5 6
1 2 2 3 7 4 4
6 5
7
1 2 3 4 5 6 7 8

A 1 22 2 3
3

52
 Procedure for Partition
1 2 3 4 5 6 7 8
QUICK-SORT (A, p, r)
1. if p < r then A 1 2 2
1 2 3 4 5 6 7 8
2. q = PARTITION (A, p, r)
A 1 2
3. QUICK-SORT (A, p, q)
1 2 3 4 5 6 7 8
4. QUICK-SORT (A, q+1, r)
1 2 3 4 5 6 7 8
A 1 2 2
A 53 22 24 7 1 6 4
1 7 2 5
3 1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
A 1 2 2 3
A 1 2 2 1
3 3 1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
A 1 2 2 3 7
5 6
4 4
6 5
7
A 1 1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
A 1 2 2 3 4 5 4
6 5
7
A 1 22 2 3
3 1 2 3 4 5 6 7 8

A 1 2 2 3 4 5 6 7
53
 Analysis of Quick Sort
• Time taken by the quick sort depends on which element is
used for partitioning
• Best case: Recurrence for the running time is
1 if n = 1
T ( n) = T (n)  c1 n log 2 n  c2 n
n
2 T   +cn if n  1
2
• Worst case:
– One side gets one element and other side gets (n-1) elements
1 if n = 1
T ( n) = T (n)  c1 n 2  c2 n
T  n  1 +T (1) + cn if n  1

• Balanced/Average case: Alternate between equal partition

and skewed partition
n
T (n)  2 U   + cn
2 T ( n)  c1 n log 2 n  c2 n
U (n)  T  n  1 +T (1) + cn
54
 Readings
• Read Chapter 7 in Corman book
• Read Chapter 5 in Kleinberg book

55