Scribd
Upload
156 views19 pages

Recursion and Space Complexity in Sorting

The document discusses space complexity of sorting algorithms, specifically insertion sort and merge sort. It explains that insertion sort has O(1) space complexity since function calls are not recursive, while merge sort has O(N) space complexity due to recursive function calls requiring stack space. The depth of recursion is logN for merge sort, resulting in O(N) space overall. Finally, it shows how a recursive divide-and-conquer algorithm like merge sort can be implemented iteratively to avoid stack space usage.

Uploaded by

SIMRAN SEHGAL
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
156 views19 pages

Recursion and Space Complexity in Sorting

The document discusses space complexity of sorting algorithms, specifically insertion sort and merge sort. It explains that insertion sort has O(1) space complexity since function calls are not recursive, while merge sort has O(N) space complexity due to recursive function calls requiring stack space. The depth of recursion is logN for merge sort, resulting in O(N) space overall. Finally, it shows how a recursive divide-and-conquer algorithm like merge sort can be implemented iteratively to avoid stack space usage.

Uploaded by

SIMRAN SEHGAL
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

CS/IS F211 Data Structures & Algorithms

2/8/2017
Sundar B.
REVIEW: TOP DOWN DESIGN

CSIS, BITS, Pilani


Space Complexity:
Function Call Overhead: Stack Space, Recursion
1
Recursion vs. Iteration:
Divide-and-Conquer Designs
2/8/2017
Sundar B.
FUNCTION CALL OVERHEAD
- STACK SPACE AND RECURSION

CSIS, BITS, Pilani


2
SPACE COMPLEXITY OF SORTING ALGORITHMS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {


if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// Time Complexity: O(N*N) }
// Space Complexity: O(1) // Time Complexity: O(N*N)
// Space Complexity: O(N)
3
Incorrect?
SPACE COMPLEXITY – COUNTING CALL FRAMES
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {


if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// Space Complexity: }
/* How many function calls (i.e. // Space Complexity:
stack frames) in sequence?
/* How many function calls (i.e.
*/
stack frames) in sequence? */
4
COUNTING CALL FRAMES – NON-RECURSIVE CALLS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {


if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
Call frames are allocated and
deallocated (before next call) }
i.e. maximum additional
space is O(1)
5
COUNTING CALL FRAMES – RECURSIVE CALLS
Insertion Sorting Merge Sorting

mergeSort(A, st, en) {


insertSort(A, n) { if (en-st < 1) return;
if (n>1) { mid=floor((st+en)/2);
mergeSort(A, st, mid);
insertSort(A,n-1);
mergeSort(A, mid+1,en);
insertInOrder(A[n-1], A, n-1);
mergeIn(A, st, mid, en);
}
}
} /* How many recursive calls in
/* How many recursive calls in sequence?
sequence? i.e. what is the depth of recursion (in
i.e. what is the depth of recursion divide-and-conquer) ?*/
(in divide-and-conquer) ?*/
COUNTING CALL FRAMES – RECURSIVE CALLS
Insertion Sorting Merge Sorting

insertSort(A, n) { mergeSort(A, st, en) {


if (n>1) { if (en-st < 1) return;
insertSort(A,n-1); mid=floor((st+en)/2);
insertInOrder(A[n-1], A, n-1); mergeSort(A, st, mid);
} mergeSort(A, mid+1,en);
} mergeIn(A, st, mid, en);
// depth of recursion: N }
// Space Complexity: O(N) // depth of recursion: log2N
// Space Complexity:
// N + logN which is O(N)
SPACE COMPLEXITY AND RECURSION
 Typically, stack space (i.e. number of call frames)
 is not dependent on input size except for recursive procedures.

2/8/2017
 e.g.
 for i = 1 to N { f(N); }

Sundar B.
 where N is input size, and f does not call other procedures

 uses O(1) space;


 but time overhead (due to function calls) is O(N) – Why?

 e.g.

CSIS, BITS, Pilani


 for i = 1 to N { f(N); }
 where N is input size, and neither f nor its descendants in the call

sequence are recursive.


 uses O(k) space
 where k is the length of the call sequence starting at f.

8
2/8/2017
Sundar B.
DIVIDE-AND-CONQUER
- RECURSIVE VS. ITERATIVE ALGORITHMS

CSIS, BITS, Pilani


9
DIVIDE-AND-CONQUER: SPACE COMPLEXITY

2/8/2017
 Divide-and-Conquer designs translate naturally to
recursive algorithms:

Sundar B.
 but recursive algorithms incur stack space overhead.
 Solutions?
 Can we translate Divide-and-Conquer designs to iterative
algorithms?

CSIS, BITS, Pilani


 Can we rewrite recursive algorithms using iteration?

10
RECURSIVE VS. ITERATIVE ALGORITHMS – INSERTION SORTING
 Problem: Sort, in-place, a list of N elements.
 Assume list is stored as an array A[0], A[1], … A[n-1]

2/8/2017
 Design : Divide-And-Conquer
 Sub-problem: Sort a list of size N-1 (A[0], A[1],…A[n-2])

Sundar B.
 Combination: Insert A[n-1] in order (i.e. in the right position)
 Termination: when size is <= 1.

insertInOrd(A[n-2],A,n-2)

CSIS, BITS, Pilani


Sort(A, n)
Sort(A,n-1) Sort(A,n-2) … Sort (A,1)

insertInOrd(A[n-1],A,n-1) insertInOrd(A[1],A,1)

insertSort(A, n) {
11
if (n>1) { insertSort(A,n-1); insertInOrder(A[n-1], A, n-1); }
}
RECURSIVE VS. ITERATIVE ALGORITHMS – INSERTION SORTING
insertInOrd(A[n-2],A,n-2)

Sort(A, n)
…

2/8/2017
Sort(A,n-1) Sort(A,n-2) Sort (A,1)

insertInOrd(A[n-1],A,n-1)

Sundar B.
insertInOrd(A[1],A,1)

 Iterative Algorithm
Can be inlined.
insertionSort(A,n) {

CSIS, BITS, Pilani


for j := 1 to n-1 insertInOrder(A[j],A,j) ; Exercise:
} Find out option in gcc.
// Pre-condition: (length(A) > last) & forall j: 0<=j<last-1 --> A[j]<=A[j+1]
insertInOrder(v,A,last) {
j := last;
while (j>0 && v<A[j]) { A[j] := A[j-1]; j--; }
A[j] := v;
} 12
// Post-condition: forall j: 0<=j<last --> A[j]<=A[j+1]
DIVIDE-AND-CONQUER DESIGN – MERGE SORTING

Sort : A[0] to A[N-1]

M
Sort : A[0] to A[N/2] Sort : A[N/2+1] to A[N-1]

Sort : A[0] to A[N/4 ] Sort A[N/4  +1] to A[N/2]

…
MERGE SORTING - DESIGN Merge 1 pair of lists
S [N] of size N/2 each

S[N/2] M S [N/2]

S[N/4] S[N/4] S[N/4] S[N/4]


M M
Merge 2 pairs of lists of size N/4 each

S[1] S[1] S[1] S[1] ... S[1] S[1] S[1] S[1]

M M M M
14
Merge N/2 pairs of lists of size 1 each
MERGESORT – ITERATIVE ALGORITHM VER.0.5
S [N]

S[N/2] M slSz=N/2 S [N/2]

S[N/4] S[N/4] S[N/4] S[N/4]


M M
slSz=N/4

for (slSz=1; slSz <= maxSlSz; slSz*=2) {


merge N/(2*slSz) pairs of lists of size slSz each
}

S[1] S[1] S[1] S[1] ... S[1] S[1] S[1] S[1]

M M M M
slSz=1 15
MERGE SORT – ITERATIVE ALGORITHM VER.0.7
for (slSz=1; slSz<=maxSlSz; slSz*=2) {
//Merging N/(2*slSz) pairs of lists of size slSz each

2/8/2017
for (st1=0; st1<=last; st1+=2*slSz) {

Sundar B.
st2=st1+slSz; en2=st2+slSz-1;
//merging ls[st1..st2-1] with ls[st2..en2]
// and putting the result back in ls[st1..en2]

CSIS, BITS, Pilani


mergeIn(ls, st1, st2, en2);
}
} // Limitation:
// Assumes (sub)lists are of equal size when merging
// i.e. this works only on lists of size=2k for some k>0 16
MERGE SORT – ITERATIVE ALGORITHM VER.0.8
for (slSz=1; slSz<=maxSlSz; slSz*=2) {
for (st1=0; st1<=last; st1=st1+2*slSz) {

2/8/2017
st2=st1+slSz; en2=st2+slSz-1;
if (st2 > last) { // Handle uneven-sized sublists

Sundar B.
en2 = last;
if (st2-1 > last) { st2=last+1; }
}
mergeIn(ls, st1, st2, en2);

CSIS, BITS, Pilani


}
}
// maxSlSize has to be computed such that
// if 2^(k-1) < n <= 2^k, then maxSlSz = 2^(k-1)
17
MERGE SORT – ITERATIVE ALGORITHM VER.0.9
for(maxSlSz=1;n>maxSlSz;maxSlSz*=2) ;
/* Postcondition: maxSlSz/2 < n <= maxSlSz */
maxSlSz /= 2;

2/8/2017
/* Postcondition: maxSlSz < n <= 2*maxSlSz */
for (slSz=1; slSz<=maxSlSz; slSz*=2) {

Sundar B.
for (st1=0; st1<=last; st1=st1+2*slSz) {
st2=st1+slSz; en2=st2+slSz-1;
if (st2 > last) { // Handle uneven-sized sublists
en2 = last;

CSIS, BITS, Pilani


if (st2-1 > last) { st2=last+1; }
}
mergeIn(ls, st1, st2, en2);
}
}
/* mergeIn uses a temp. array to merge and copy back to ls */ 18
MERGE SORT – ITERATIVE ALGORITHM VER.1.0

2/8/2017
 Algorithm same as ver. 0.9 but, mergeIn uses two arrays, ls1
and ls2:

Sundar B.
 in odd iterations of outer loop:
 ls1 is an array of sorted sublists:

 merged into temporary array ls2

 in even iterations of outer loop:

CSIS, BITS, Pilani


 ls2 is an array or sorted sublists

 merged into temporary array ls1

 Exercise:
 Write mergeIn to meet this requirement.
 Modify mergeSort suitably. 19

You might also like