Analysis and Design of Algorithms

Divide and Conquer

What is Divide and Conquer?

A Paradigm to Designing of
Algorithms with Theoretical
Methodology for Analysis

2
 Divide-and-Conquer Steps
 Divide: the problem into smaller instances of the
same problem

 Conquer: the sub-problems by solving them

recursively. For small enough sub-problems
 Solution is straightforward
 Recurrence is not really needed

 Combine: the solutions of sub-problems to

achieve the solution of original problem
 Divide and Conquer Procedure
Problem DIVIDE

Sub-Problem Sub-Problem

Sub-Problem Sub-Problem Sub-Problem Sub-Problem

Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem

CONQUER
Sol. To Sol. To Sol. To Sol. To Sol. To Sol. To Sol. To Sol. To
Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem Sub-Problem

Sol to Sol to Sol to Sol to

Sub-Problem Sub-Problem Sub-Problem Sub-Problem

Sol. to Sol. to
Sub-Problem Sub-Problem
Problem Solved COMBINE

4
 Merge Sort
 Divide the n-elements unsorted sequence into
sub-problems of n/2 each

 Conquer by solving sub-problems recursively until

the sub-problem that has a straightforward solution
is reached

 Combine all sub-problems until the problem

solution is achieved

5
 Merge Sort Example
14 2 33 7 5 12 54 9

14 2 33 7 5 12 54 9

14 2 33 7 5 12 54 9

14 2 33 7 5 12 54 9

2 14 7 33 5 12 9 54

2 7 14 33 5 9 12 54

2 5 7 9 12 14 33 54

6
7
 Merge-Sort Analysis

. . . . .

… …

Time Complexity to Sort n Elements

= / + / + +
Computation-Time of Sorting n-elements is decomposed into:
 Time to Divide the Problem into 2 sub-problems ( )
 Computations-Time (Conquer) for Sub-problem 1 ( / )
 Computations-Time (Conquer) for Sub-problem 2 ( / )
 Time to Combine Solutions of Both Sub-problems ( )
8
 Merge-Sort Complexity Decomposition
 Divide: A simple process of identifying the middle
of n (which is known)  =

 Conquer: if we know then we know / .

They could be solved recursively

 Combine:
 This is the obstacle for now to solve for
 It depends on the algorithm followed in the combining
step
 In Merge-sort =

9
Merge-Sort Recurrence Formula

=
=
⁄ + + >

=
=
⁄ + >

=
=
⁄ + >
10
Recursion Tree of Merge-Sort
=
=
⁄ + >

/ /

/ /

/ / / /
/ / / /

11
Recursion Tree of Merge-Sort

/ /
+

/ / / /

: × +
≡
⇒ =

12
Recursion Tree for = / +

/ / / / / /

13
 Recursion Tree for = / +

/ / /
+

:
⇒ =?
14
 Recursion Sol for = / +
 Number of Levels is +
 Number of Leaf Nodes is =

= + + + ⋯+ + ×

= + + + ⋯+ +

−
= +
Note:
− =
=
= ×
= ×
=
= = =

15
 Master Method
Based on Master Theorem is the Cookbook for
solving Recurrences of the form
= × +
where
≥1
>1
>0

16
 Recursion Tree Re-visited

= × +

/
= × + = × +

⇒ = + +

17
 One-Level further

= + +

⇒ = + + +

18
 As the Recurrence Bottoms Out
Level #0

Level #1

Level #2

Level #h

= 1 + + ⋯+ +

19
 Leading to …
Level #0

Level #1

Level #2

Level #h

⇒ = × 1 +

20
 How many Levels?
Level #0

Level #1

Level #2

Level #h

At Level #h

= 1 ⇒ ℎ = log
21
 Time Complexity becomes
Level #0

Level #1

Level #2

Level #h

⇒ = × 1 +

22
 Time Complexity becomes
Level #0

Level #1

Level #2

Level #h

⇒ = × 1 +

23
 Determining Order of Growth of T(n)

= × 1 +

Determined by comparison between order of growth

of against order of growth of
 is asymptotically smaller than by a
polynomial factor
 is asymptotically equal with
 is asymptotically larger than by a
polynomial factor

24
 The Master Theorem
Let ≥ 1, > 1 be constants, let be a function and let
be defined on non-negative integers by the recurrence relation

= × +
could be bounded asymptotically in three cases

Case 1: if = for some > 0, then: =

Case 2: if =Θ , then: =

Case 3: if =Ω for some > 0, and if ≤ for some

constant 0 < < 1 and large , then: =

25
 The Master Theorem & Order of Growth

= × 1 +

Determined by comparison between order of growth of

against order of growth of
 is asymptotically smaller than
⇒ =
 is asymptotically equal with
⇒ =
 is asymptotically larger than
⇒ =

26
 Examples

=2× +
2
 = 2, = 2 ⇒ log =1
 =
Compare = with =

=Θ (Case 2)

= =

27
 Examples

=2× +
2
 = 2, = 2 ⇒ log =1
 =
Compare = with =

=Ω (Case 3)
Condition: ≤ ⇒2 ≤ ,0 < = < 1 is a solution

= =

28
 Examples

=2× +
2
 = 2, = 2 ⇒ log =1
 = .
Compare = with = .

=O (Case 1)

= =

29
 Examples

=3× +
4
 = 3, = 4 ⇒ log = 0.793
 =
Compare = . with =

=Ω . (Case 3)
Condition: ≤ ⇒3 ≤ ,0 < = < 1 is a solution

= =

30