Algorithm Analysis and Design

Master Theorem
Master Theorem
• Master method is based on the master theorem.

Source: CLRS
Master Theorem

• What does it all mean?

• T(n) is the running time on a problem of size n.
• If we divide the problem into a subproblems, each of which is 1/b the
size of the original problem. (a does not have to equal b).
• It takes T(n/b) time to solve one subproblem, and a.T(n/b) to solve all
of them.
• f(n) represents the additional time to (first) divide the problem and
(then) to combine the solutions to the subproblems.
Master Theorem

Example 1: T(n) = 9T(n/3) + n

a = 9, b = 3, 𝑓 𝑛 = 𝑛
𝑛log𝑏 𝑎 = 𝑛2
𝑓 𝑛 = 𝑛 = 𝑛2−1
We have Case 1. So, T(n) = 𝜃(𝑛log𝑏 𝑎 ) = 𝜃(𝑛2 )
Master Theorem

Example 2: T(n) = T(2n/3) + 1

a = 1, b = 3/2, 𝑓 𝑛 = 1
𝑛log𝑏 𝑎 = 𝑛0 = 1
𝑓 𝑛 =1
We have Case 2. So, T(n) = 𝜃 𝑛log𝑏 𝑎 lg 𝑛 = 𝜃(lg 𝑛) = 𝜃 log 2 𝑛
Master Theorem

Example 3: T(n) = 3T(n/4) + n lg n

a = 3, b = 4, 𝑓 𝑛 = 𝑛 lg 𝑛
𝑛log𝑏 𝑎 = 𝑛0.793
𝑓 𝑛 = 𝑛 lg 𝑛 = Ω(𝑛)
We might have Case 3. Is a.f(n/b) ≤ c.f(n) for some c < 1 and large n?
Is 3f(n/4) ≤ f(n)? 3(n/4) lg (n/4) ≤ 3(n/4) lg n = (3/4) n lg n. c = 3/4.
So, we have Case 3. T(n) = 𝜃 𝑓(𝑛) = 𝜃(𝑛 lg 𝑛)
Master Theorem

Example 4: T(n) = 2T(n/2) + n lg n

a = 2, b = 2, 𝑓 𝑛 = 𝑛 lg 𝑛
𝑛log𝑏 𝑎 = 𝑛1 = 𝑛
𝑓 𝑛 = 𝑛 lg 𝑛 = Ω(𝑛)
We might have Case 3. Is a.f(n/b) ≤ c.f(n) for some c < 1 and large n?
Is 2f(n/2) ≤ f(n)? 2(n/2) lg (n/2) = n lg (n/2) = n(lg n – 1) = n lg n – n.
Is n lg n – n ≤ 0.999 n lg n (for example)? Not after a certain n.
So, we don’t have Case 3. Master theorem does not apply.
Master Theorem
Example 5: Finally, let’s construct the recurrence for binary search.
• At each step, we divide problem into two pieces, but proceed with
only one of these pieces. So, a = 1.
• The size of a piece is half the size of the previous piece. So, b = 2.
• Dividing takes constant time. There is no combining. So, f(n) = θ(1).
T(n) = T(n/2) + 1
𝑛log𝑏 𝑎 = 𝑛0 = 1
𝑓 𝑛 = θ(1)
We have Case 2. So, T(n) = 𝜃 𝑛log𝑏 𝑎 lg 𝑛 = 𝜃(lg 𝑛)
References

• Adapted from CENG368 lecture slides of Mehmet

Demirci