Master Theorem

01/11/2022 SATPAL SINGH KUSHWAHA 1

 Course Information
• Course Handout
• Communicate through eMail
• Office hours
• To be communicated
• Grading policy
• Will be communicated as per university guidelines

01/11/2022 SATPAL SINGH KUSHWAHA 2

 Syllabus
• Introduction: Fundamentals of Algorithms, Important Problem Types,
Analysis of algorithm efficiency. Analysis Framework: Asymptotic Notations
and Basic Efficiency Classes. Mathematical Analysis of Nonrecursive and
Recursive Algorithms: Brute force Techniques, Divide and Conquer. Decrease
and Conquer: Insertion Sort, Depth First Search, Breadth First Search,
Topological Sorting. Transform and Conquer: Presorting, BST, Heapsort. Space
and Time tradeoffs: Input Enhancement in String Matching. Dynamic
Programming: Warshall's and Floyd's Algorithms, The Knapsack Problem.
Greedy Techniques: Prim's, Kruskal's and Dijkstra's Algorithm, Huffman Trees.
Coping with limitations of algorithmic power. Backtracking: nQueens problem,
Hamiltonian Circuit Problem, Subset Sum Problem. Branch and Bound:
Assignment Problem, Knapsack Problem, TSP. P, NP, and NP-complete
Problems.

01/11/2022 3
 More Information
• Textbook
• Introduction to Algorithms 3rd ,Cormen,
Leiserson, Rivest and Stein, The MIT Press,
• Fundamentals of Computer Algorithms,
2nd, Sartaj Sahni, Ellis Horowitz,
Sanguthevar Rajasekaran
• Others
• Introduction to Design & Analysis
Computer Algorithm 3rd,
Sara Baase, Allen Van Gelder, Adison-
Wesley, 2000.
• Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice
Hall, 2004.
• Introduction to The Design and Analysis of Algorithms 2nd
Edition, Anany Levitin, Adison-Wesley, 2007.
SATPAL SINGH KUSHWAHA
01/11/2022 4
 Course Objectives
• CS1501.1 Analyse the running times of algorithms using asymptotic analysis.
• CS1501.2 Demonstrate and Design algorithms using divide-and-
conquer paradigm to solve business problems hence enhance skills.
• CS1501.3 Illustrate the concept of greedy and dynamic-programming approach
to solve real life problems to enhance entrepreneurship capabilities.
• CS1501.4 Demonstrate the concept of backtracking and branch &
bound algorithms.
• CS1501.5 Synthesize and analyse various advanced algorithms concept such as
graphs, string matching, approximation algorithms and complexity classes to
enhance employability.

01/11/2022 SATPAL SINGH KUSHWAHA 5

 Master’s method
• “Cookbook” for solving recurrences of the form:
n
T (n)  aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Idea: compare f(n) with nlogba

• f(n) is asymptotically smaller or larger than nlogba by a

polynomial factor n

• f(n) is asymptotically equal with nlogba 6

 Master’s method
• “Cookbook” for solving recurrences of the form:

n
T (n)  aT    f (n)
b
where, a ≥ 1, b > 1, and f(n) > 0

Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)

Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)

Case 3: if f(n) = (nlogba +) for some  > 0, and if

af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:

T(n) = (f(n))
regularity condition
7
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
#leaves = ah
…

= alogbn nlogbaT (1)

T (1)
= nlogba
01/11/2022 SATPAL SINGH KUSHWAHA 8
Three common cases
Compare f (n) with nlogba:
1. f (n) = O(nlogba – e) for some constant e > 0.
• f (n) grows polynomially slower than nlogba
(by an ne factor).
Solution: T(n) = Q(nlogba) .

01/11/2022 SATPAL SINGH KUSHWAHA 9

Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
CASE 1: The weight increases
…

geometrically from the root to the nlogbaT (1)

T (1) leaves. The leaves hold a constant
fraction of the total weight. Q(nlogba)
01/11/2022 SATPAL SINGH KUSHWAHA 10
Three common cases
Compare f (n) with nlogba:

2. f (n) = Q(nlogba lgkn) for some constant k ³ 0.

• f (n) and nlogba grow asymptotically at
similar rates.
Solution: T(n) = Q(nlogba lgk+1n) .

01/11/2022 SATPAL SINGH KUSHWAHA 11

Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
…

CASE 2: (k = 0) The weight

is approximately the same on nlogbaT (1)
T (1)
each of the logbn levels.
Q(nlogbalg n)
01/11/2022 SATPAL SINGH KUSHWAHA 12
Three common cases (cont.)
Compare f (n) with nlogba:
3. f (n) = W(nlogba + e) for some constant e > 0.
• f (n) grows polynomially faster than nlogba (by an ne
factor),
and f (n) satisfies the
regularity condition
that a f (n/b) £ c f (n) for some constant c < 1.
Solution: T(n) = Q( f (n) ) .

01/11/2022 SATPAL SINGH KUSHWAHA 13

Regularity Condition:
Imagine the recurrence aT(n/b)+f(n) as a tree. The third case covers the scenario
when the root node dominates the running time asymptotically, i.e. most of the work is
being done in measly node on top of the recurrence tree. Then the running time is

Θ(f(n)).
To make sure the root actually does more work you need the

af(n/b)≤cf(n)
This says that f(n) (the amount of work done in the root) needs to be at least as big as

the sum of the work done in the lower levels. (The recurrence is called aa times on
n/b of the input.)
Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
CASE 3: The weight decreases
…

geometrically from the root to the nlogbaT (1)

T (1) leaves. The root holds a constant
fraction of the total weight. Q( f (n))
01/11/2022 SATPAL SINGH KUSHWAHA 15
Examples
Ex. T(n) = 4T(n/2) + n
a = 4, b = 2  nlogba = n2; f (n) = n.
CASE 1: f (n) = O(n2 – e) for e = 1.
 T(n) = Q(n2).

Ex. T(n) = 4T(n/2) + n2

a = 4, b = 2  nlogba = n2; f (n) = n2.
CASE 2: f (n) = Q(n2lg0n), that is, k = 0.
 T(n) = Q(n2lg n).
01/11/2022 SATPAL SINGH KUSHWAHA 17
Examples
Ex. T(n) = 4T(n/2) + n3
a = 4, b = 2  nlogba = n2; f (n) = n3.
CASE 3: f (n) = W(n2 + e) for e = 1
and 4(cn/2)3 £ cn3 (reg. cond.) for c = 1/2.
 T(n) = Q(n3).

Ex. T(n) = 4T(n/2) + n2/lg n

a = 4, b = 2  nlogba = n2; f (n) = n2/lg n.
Master method does not apply

01/11/2022 SATPAL SINGH KUSHWAHA 18

 Examples

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)
19
 Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
 f(n) = (n1+) Case 3  verify regularity cond.
a f(n/b) ≤ c f(n)
 2 n2/4 ≤ c n2  c = ½ is a solution (c<1)
 T(n) = (n2)

20
 Examples (cont.)

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare n with f(n) = n1/2

 f(n) = O(n1-) Case 1

 T(n) = (n)

21
 Examples

T(n) = 3T(n/4) + nlgn

a = 3, b = 4, log43 = 0.793

Compare n0.793 with f(n) = nlgn

f(n) = (nlog43+) Case 3

Check regularity condition:

3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n), c=3/4

T(n) = (nlgn)
22
 Examples

T(n) = 2T(n/2) + nlgn

a = 2, b = 2, log22 = 1

• Compare n with f(n) = nlgn

– seems like case 3 should apply

• f(n) must be polynomially larger by a factor of n

• In this case it is only larger by a factor of lgn

23
 Examples

T(n) = 8T(n/4) + n2lgn

a = 8, b = 4, log48 = 2

• Compare n2 with f(n) = n2lgn

– seems like case 3 should apply

• f(n) must be polynomially larger by a factor of n

• In this case it is only larger by a factor of lgn

24
 Exercises
1. T(n) = 3T(n/5) + n

• 2. T(n) = 2T(n/2) - n

• 3. T(n) = 0.5T(n/2) + 1

• 4. T(n) = T(n/2) + n
But regularity condition not satisfied
So no solution using master theorem

• 5. T(n) = T(n/2) + 1