Data Structures and Algorithms

Prof. Ganesh Ramakrishnan,

Prof. Ajit Diwan,
Prof. D.B. Phatak

Department of Computer Science and Engineering

IIT Bombay

Session: Sorted List-Based(Insertion Sort)

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Sorted List-based (Insertion Sort)

 inserti (e, P ): Scan list and insert in order to maintain increasing order of list P
I

1 + 2 + . . . n = O(n2 )

 delete(P, m): Remove the (min) element m at the beginning of the list P
I

O(n)

 O(n2 ) overall

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Array-based in-place version of Insertion Sort

Algorithm Insertion-Sort(S)
Input: Array S
Output: Array S sorted in increasing order
i = 1, n = length(S)
repeat
1. k = S[i]
//Insert k into the sorted array S[1 . . . i 1]
2. j = i 1
while j > 0 and S[j] > k do
1. S[j + 1] = S[j]
2. j = j 1
end while
S[j + 1] = k
until i > n
Figure: In-place Insertion Sort

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Insertion Sort & Loop Invariant

 Algorithm maintains following loop invariant:

I

At start of each iteration of ith repeat loop, the subarray S[1 . . . i 1] consists of
the elements of the old S[1 . . . i 1] but sorted in ascending order

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Analysis of in-place Insertion Sort

Algorithm Insertion-Sort(S)
Input: Array S
Output: Array S sorted in increasing order
c1
1 times
i = 1, n = length(S) =
repeat
c2
n 1 times
1. k = S[i] =
//Insert k into the sorted array S[1 . . . i 1]
c3
n 1 times
2. j = i 1 =
while j > 0 and S[j] > k do
n
X
c4

(wi 1) times
1. S[j + 1] = S[j] =
i=2

2. j = j 1

c5

n
X

(wi 1) times

i=2

end while

c6

n
X

wi times

i=2

S[j + 1] = k
until i > n =

=
c8

c7
n 1 times
n times

Figure: wi is number of iterations of the inner while loop.

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak
Sorting

IIT Bombay

Analysis of in-place Insertion Sort

Algorithm Insertion-Sort(S)
Input: Array S
Output: Array S sorted in increasing order
c1
1 times
i = 1, n = length(S) =
repeat
c2
n 1 times
1. k = S[i] =
//Insert k into the sorted array S[1 . . . i 1]
c3
n 1 times
2. j = i 1 =
while j > 0 and S[j] > k do
n
X
c4

(wi 1) times
1. S[j + 1] = S[j] =
i=2

2. j = j 1

c5

n
X

(wi 1) times

i=2

end while

c6

n
X

wi times

i=2

S[j + 1] = k
until i > n =

=
c8

c7
n 1 times
n times

Figure: wi is number of iterations ofPthe inner while loop.

Pn
Pn
n
T (n) = c1 +c2 (n1)+c3 (n1)+c4 i=2 (wi 1)+c5 i=2 (wi 1)+c6 i=2 wi +c7 (n1)+c8 n
Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak
Sorting

IIT Bombay

Analysis of in-place Insertion Sort

 T (n)
P = c1 + c2 (n 1) + c3 (n 1) + c4
c6 ni=2 wi + c7 (n 1) + c8 n
 Best Case Running Time:

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

Pn

i=2 (wi

1) + c5

Pn

i=2 (wi

1) +

IIT Bombay

Analysis of in-place Insertion Sort

 T (n)
P = c1 + c2 (n 1) + c3 (n 1) + c4
c6 ni=2 wi + c7 (n 1) + c8 n
 Best Case Running Time:
I
I

i=2 (wi

1) + c5

Pn

i=2 (wi

1) +

If S[1 . . . n] is already sorted, wi = 1 for i = 2 . . . n

= T (n) = c1 + c2 (n 1) + c3 (n 1) + c6 (n 1) + c7 (n 1) + c8 n =
c1 + n(c2 + c3 + c6 + c7 + c8 ) (c2 + c3 + c6 + c7 ) = O(n)

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

Pn

IIT Bombay

Analysis of in-place Insertion Sort

 T (n)
P = c1 + c2 (n 1) + c3 (n 1) + c4
c6 ni=2 wi + c7 (n 1) + c8 n
 Best Case Running Time:
I
I

Pn

i=2 (wi

1) + c5

Pn

i=2 (wi

1) +

If S[1 . . . n] is already sorted, wi = 1 for i = 2 . . . n

= T (n) = c1 + c2 (n 1) + c3 (n 1) + c6 (n 1) + c7 (n 1) + c8 n =
c1 + n(c2 + c3 + c6 + c7 + c8 ) (c2 + c3 + c6 + c7 ) = O(n)

 Worst Case Running Time:

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Analysis of in-place Insertion Sort

 T (n)
P = c1 + c2 (n 1) + c3 (n 1) + c4
c6 ni=2 wi + c7 (n 1) + c8 n
 Best Case Running Time:
I
I

Pn

i=2 (wi

1) + c5

Pn

i=2 (wi

1) +

If S[1 . . . n] is already sorted, wi = 1 for i = 2 . . . n

= T (n) = c1 + c2 (n 1) + c3 (n 1) + c6 (n 1) + c7 (n 1) + c8 n =
c1 + n(c2 + c3 + c6 + c7 + c8 ) (c2 + c3 + c6 + c7 ) = O(n)

 Worst Case Running Time:

I
I

If S[1 . . . n] is sorted in reverse order, wi = i 

for i = 2. . . n 

= T (n) = c1 + c2 (n 1) + c3 (n 1) + c4 n(n1)
+ c5 n(n1)
+
2
2


c6 n(n+1)
1 + c7 (n 1) + c8 n = O(n2 )
2

 How about the average case?

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak
Sorting

IIT Bombay

Average Complexity of in-place Insertion Sort

 A swap in insertion sort: S[j + 1] = S[j] followed by S[j + 1] = k

 A swap occurs for each inversion:
I

An ordered pair of indices (i, j) into S is an inversion if i < j but S[i] > S[j]

 Number of operations T (n) = time to scan each element + number of inversions

 I = number of inversions T (n) = O(n + I)

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Average Complexity of in-place Insertion Sort

 A swap in insertion sort: S[j + 1] = S[j] followed by S[j + 1] = k

 A swap occurs for each inversion:
I

An ordered pair of indices (i, j) into S is an inversion if i < j but S[i] > S[j]

 Number of operations T (n) = time to scan each element + number of inversions

 I = number of inversions T (n) = O(n + I)
 Claim: Average number of inversions in a list of n distinct elements is

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

n(n1)
4

IIT Bombay

Average number of Inversions

 Let Sr be the reverse of S.
 A pair of elements e1 and e2 will be inverted in exactly one of S and Sr
n(n1)
2
n(n1)
n!
2
2

 Total number of such pairs inverted in one of S or Sr =

 Total number of inversions across all permutations =

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Average number of Inversions

 Let Sr be the reverse of S.
 A pair of elements e1 and e2 will be inverted in exactly one of S and Sr
n(n1)
2
n!
inversions across all permutations = 2 n(n1)
2
of inversions I across all permutations = T otal
n! =

 Total number of such pairs inverted in one of S or Sr =

 Total number of
 Average number
n(n1)
n!
2
2

n!

n(n1)
4

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Average number of Inversions

 Let Sr be the reverse of S.
 A pair of elements e1 and e2 will be inverted in exactly one of S and Sr
n(n1)
2
n!
inversions across all permutations = 2 n(n1)
2
of inversions I across all permutations = T otal
n! =

 Total number of such pairs inverted in one of S or Sr =

 Total number of
 Average number
n(n1)
n!
2
2

n!

n(n1)
4

 Average T (n) =
I

n(n1)
4

+ n = O(n2 )

True about ANY swap based algorithm, since swap removes exactly one inversion in
a single step.

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

Summary Analysis of in-place Insertion Sort

 Best Case Running Time: S[1 . . . n] is already sorted = T (n) = O(n)

 Worst Case Running Time: S[1 . . . n] is sorted in reverse order =
T (n) = O(n2 )
 Average Case Running Time: T (n) = O(n2 )
 Can an alternative algorithm give better average and worst-case running times?

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay

 Next Session: Heap Based Heap Sort

Thank you

Prof. Ganesh Ramakrishnan, Prof. Ajit Diwan, Prof. D.B. Phatak

Sorting

IIT Bombay