Data Structures and Algorithms

(ESO207A)

Lecture 18:
• Solving recurrences
that occur frequently in
… the analysis of algorithms.

1
 Homework from the last class

• Suitably modify the Merge procedure to compute .

2
 Relook
Merging A[] and A[]

Sorted Sorted

A x

𝒊 >x
𝒋 𝒌

3
 Pesudo-code for Merging two sorted arrays

Merge(A,,C)
 ;  ;  0;
While( ≤ and )
{ If(A[]< A[]) { C[]  A[]; ++; ++ }
Else { C[]  A[];++; ++ }
}
While( ≤ ) { C[]  A[]; ++; ++ }
While( ≤ ) { C[]  A[]; ++; ++ }
return C ;

We shall make just a slight change in the above

pseudo-code to achieve our main objective of
computing . If you understood the animation shown
in the previous slide, can you guess it now ? 4
 Pesudo-code for
Merging and counting inversions
Merge_and_CountInversion(A,,C)
 ;  ;  0;
 0;
While( ≤ and )
{ If(A[]< A[]) { C[]  A[]; ++; ++ }
Else { C[]  A[];++; ++
?  + ? ; ();
}
}
While( ≤ ) { C[]  A[]; ++; ++ }
Nothing extra is
While( ≤ ) { C[]  A[]; ++; ++ } needed here.
return ;

5
 Counting Inversions
Final algorithm based on divide & conquer
Sort_and_CountInversion(A)
{ If () return 0;
else
{ ()/2;
 Sort_and_CountInversion (A,);
T()
 Sort_and_CountInversion (A,);
Create a temporary array C[ ]
 Merge_and_CountInversion(A,,C);
Copy C[] to A[]; O()
return + + ;
}
}

6
 Counting Inversions
Final algorithm based on divide & conquer
Time complexity analysis:
If = 1,
T() = c for some constant c
If > 1,
T() = c + 2 T(/2)
= O( log )

Theorem: There is a divide and conquer based algorithm

for computing the number of inversions in an array of size .
The running time of the algorithm is O( log ).

7
Commonly occurring recurrences

T()
T()
T()=T()
T()
=T() +=+=T(/2)
==T() +T()
+=24T()
+3T()
+T()
2T()
T()+ T()

8
Methods for solving Recurrences

commonly occuring in
algorithm analysis

9
 Methods for solving common Recurrences

• Unfolding the recurrence.

• Guessing the solution and then proving by induction.

• A General solution for a large class of recurrences (Master theorem)

10
 Solving a recurrence by unfolding

Let T(1) = 1,
T() = + 4 T(/2) for >1, where is some positive constant

Solving the recurrence for T() by unfolding (expanding)

T() = + T(/)
= + + T(/)
= + + T(/)
= +2 +
= +2 +

A geometric increasing series with log terms and common ratio 2

= O()

11
 Solving a recurrence by unfolding

Let T(1) = 1,
T() = + 4 T(/2) for >1, where is some positive constant

Solving the recurrence for T() by unfolding (expanding)

T() = + T(/)
= + + T(/)
= + + T(/)
= +2 +
= +2 +

= O()

12
 Solving a recurrence by guessing and
then proving by induction
T(1) = It looks similar/identical to the
T() = 2T(/2) + recurrence of merge sort. So
we guess
Guess: T() ≤ + for some constants and . T() = O( log )
Proof by induction:
Base case: holds true if ≥
Induction hypothesis: T() ≤ + for all <
To prove: T() ≤ +
These inequalities
Proof: T() = 2T(/2) + can be satisfied
≤ 2( + ) + // by induction hypothesis simultaneously
by selecting
= - +2 +
=
= + +(+ - ) =+
≤ + if ≥ +

Hence T() ≤ ( + for all value of .

13
So T() = )
 Solving a recurrence by guessing and
then proving by induction
Key points:

• You have to make a right guess (past experience may help)

• What if your guess is too loose ?

• Be careful in the induction step.

14
A General Method for solving a large class of
Recurrences

15
 Solving a large class of recurrences

T(1) = 1,
T() = f() + T(/)
Where
• and are constants and >1
• f() is a multiplicative function:
f() = f()f()

AIM : To solve T() for =

16
 Warm-up
f() is a multiplicative function:
f() = f()f()
f() = ? 1
f() = ? f ( 𝑎) 𝒊
f() = ? 1/f (𝑛)
Example of a multiplicative function : f() =

Question: Can you express as power of ?

Answer:

17
 Solving a slightly general class of recurrences

T() = f() + T(/)

= f() + f(/) + T(/)
= f() + f(/) + f(/) + T(/)
= …
= f() + f(/) + … + f(/) + … + f(/) +T(1)

=() +
… after rearranging …
= +
… continued to the next page …

18
T() = +
(since is multiplicative)
= +
= +
= +
= +

= + A geometric series

Case 1: = , T() = ??
𝑎𝑘 (𝑘+1) ¿ O( 𝒂 𝐥𝐨𝐠 𝒏 log𝑏 𝑛) ¿O( 𝑛𝐥𝐨𝐠 𝒂 log 𝑏 𝑛)
𝒃 𝒃

19
T() = +
(since is multiplicative)
= +
= +
= +
= + For < , the sum of this
series is bounded by
= ?O(1)
= +

Case 2: < , T() = ??

+O(1) = O) = O(f(𝑛))

20
T() = +
(since is multiplicative)
= +
= +
= + For > , the sum of this
= + series is equal to

= +

Case 3: > , T() = ??

+ =)

21
 Three cases
T() = +

Case 1: = ,
T() = )

Case 2: < ,
T() =

Case 3: > ,
T() = )

22
 Master theorem
T(1) = 1,
T() = f() + a T(/b) where f is multiplicative.
There are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

23
Examples

24
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 1: T()= + 4 T(/2)

This is case 3

Solution: T()= ??
O( 𝑛2 )

25
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 2: T()= + 4 T(/2)

This is case 1

Solution: T()= ?? O (𝑛𝟐 log 𝑛)

2

26
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 3: T()= + 4 T(/2)

This is case 2

Solution: T()= ??
 O ( 𝑛3 )

27
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 4: T()= 2 + 3 T(/2)

Solution: T()= ?? We can not apply master theorem directly

since f() = 2 is not multiplicative.

28
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 4: T()= 2 + 3 T(/2)

Solution: G()= T()/2 This is case 3

 G()= + 3 G(/2)
 G(n) = 1.58
T(n) = .
=   O (𝑛 )
29
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 4: T()= 2 + 3 T(/2)

Solution: G()= T()/2
 G()= + 3 G(/2)
 G(n) = 1.58
T(n) = .
=   O (𝑛 )
30
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 6: T() = T() + c

Solution: T()= ?? We can not apply master theorem directly

since T() <> T(/) for any constant .

31
 Solving T() = T() + c using the method of
unfolding
T() = c + T()
Can you guess the
= c + c+ T() number of terms in
= c + c+ c + T() this series ?

= c + c+ … c + … + T(1)

A series which is decreasing at a rate

⌈loglog𝑛⌉
faster than any geometric series

= O()

32
 Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )

Example 5: T()= + 2 T(/2)

Solution: T()= ?? We Using theapply

can not method of “unfolding”,
master theorem since
it can
f() be
= isshown that T() = .
not multiplicative.

33
 Homework # 1
Solve the following recurrences systematically (if possible by various
methods). Assume that T() =1 for all these recurrences.
• T() = 1 + 2 T(/2)
• T() = + 2 T(/2)
• T() = + 7 T(/3)
• T() = + 2T(/2)
• T() = 1 + T(/5)
• T() = + 2 T(/4)
• T() = 1 + T()
• T() = + T(/10)
• T() = + T(/4)

34
 Homework # 2
Find error in the following reasoning.
For the recurrence T(1) = , and T() = 2T(/2) + ,

one guesses T() = O()

Proposed (wrong)proof by induction:

Induction hypothesis: T() ≤ for all

T() = 2T(/2) +
≤ 2() + // by induction hypothesis
= +
= O()

35
 Proof of correctness

Algorithm for majority element

36
 The Algorithm at the end of th iteration
𝒊 −𝟏

𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
P() : isisaamajority
majority
is a majority elementelement
element
of of {,…count times…,,A[],…,A[]}
of{A[],A[],…,A[]}
{A[],A[],…,A[]}
… ?…
Question: What is P() ?
Answer: is a majority element of {,…count times…,}

37
We need count > 0 at the end of the while loop for drawing this inference.
 The Algorithm at the end of th iteration
𝒊 −𝟏

𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
P() : After cancelling pairs of distinct element, {,…count times…,} remains.

So we have multiple assertions that hold.

38
 The Algorithm at the end of th iteration
𝒊 −𝟏

𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
1. If A[],…,A[] have a majority element, it must be .
2. is the majority element of {,…count times…,,A[],…,A[]}
3. After cancelling pairs of distinct elements from A[],…,A[],
the multiset {,…count times…,} remains.
39
 Conclusion
Which of these assertions will suffice as proof of correctness ?
Do we need any other lemma ?
What about the following lemma ?
Lemma: If there is a majority element, say , after any sequence of
cancelling pairs of distinct elements from A, a nonzero copies of will always survive.

Do you use this lemma to establish validity of one of the assertions ?

[You surely do not require it at least for one assertion].

Can you use this lemma and one of the assertions to prove correctness of algorithm ?

Is there an alternate way to prove the correctness of the algorithm without P() ?

Ponder over these questions, without fea,r without aversion 40

Proof of correctness of an algorithm is a delicate and nontrivial concept.

Take your time to internalize it. Do not worry or feel anxious.

[No question will be asked in the mid sem exam on the proof of correctness.]

But you can not avoid it 

It does not require any formula.

Just formalizing the right understanding of the algorithm.

Best wishes 

41