http://eclass.aueb.

gr/courses/INF161/
2016 - I.

Divide and Conquer (D&C)

- 2016 - . - 04 - DIVIDE & CONQUER I 1

Divide & Conquer
() -
~
: ( ) -
,
() -

AN
:
-

2
- 2016 - . - 04 - DIVIDE & CONQUER I
Binary Search
binS (A[i..j], k)
: [1..n],
k
: H k ,
nil
if i>j: return nil
m= (i j ) / 2 O(1)
if k=A[m]: return m
If k>A[m]: return binS(A[m+1..j],k) T(n/2)
else: return binS(A[i, m-1],k)

: binS(A[1..n],k)

1, if n 1
T ( n) (n) O(log n)
T (n / 2) O(1), if n 1
- 2016 - . - 04 - DIVIDE & CONQUER I 3
Integer Multiplication
2 n bits

**********
x **********
**********
**********
**********
**********
**********
********** (n2 )
**********
**********
*********
*********

*******************

- 2016 - . - 04 - DIVIDE & CONQUER I 4

 Integer Multiplication
Divide and Conquer n/2 bits n/2 bits
x= a b = a 2n/2 + b

y= c d = c 2n/2 + d

x y = (a 2n/2 + b) (c 2n/2 + d) = a c 2n + (a d +b c) 2n/2 + b d

IntMult1(x,y)
: x, y n bits
: xy
if n=1: return xy
a= n/2 leftmost bits of x, b= n/2 rightmost bits of x
c= n/2 leftmost bits of y, d= n/2 rightmost bits of y
P1 = IntMult1(a,c), P2 = IntMult1(a,d)
P3 = IntMult1(b,c), P4 = IntMult1(b,d)
return P1 2n + (P2 + P3)2n/2 + P4

- 2016 - . - 04 - DIVIDE & CONQUER I 5

Integer Multiplication
IntMult1(x,y)
: x, y n bits
: xy
if n=1: return xy
a= n/2 leftmost bits of x, b= n/2 rightmost bits of x
c= n/2 leftmost bits of y, d= n/2 rightmost bits of y
P1 = IntMult1(a,c), P2 = IntMult1(a,d)
P3 = IntMult1(b,c), P4 = IntMult1(b,d) 4 T(n/2)
return P1 2n + (P2 + P3) 2n/2 + P4

4 n/2 bits 4(n/2)
2 2 O(n)
3 n bits O(n)
1, if n 1
T ( n) (n) O(n 2 )
4T (n / 2) O(n), if n 1

- 2016 - . - 04 - DIVIDE & CONQUER I 6
 Integer Multiplication
Divide and Conquer n/2 bits n/2 bits
x= a b = a 2n/2 + b

y= c d = c 2n/2 + d

x y = (a 2n/2 + b) (c 2n/2 + d) = a c 2n + (a d +b c) 2n/2 + b d

Gauss equation : ad bc (a b)(c d ) ac bd

xy = (a 2n/2 + b) (c 2n/2 + d)
= ac 2n + (ad +bc) 2n/2 + bd
= ac 2n + [(a+b)(c+d) ac bd] 2n/2 + bd P1 = ac
P2 = bd
= P1 2n + [P3-P1P2] 2n/2 + P2
P3 = (a+b)(c+d)
[Karatsuba, 1962]
- 2016 - . - 04 - DIVIDE & CONQUER I 7
 Integer Multiplication
IntMult2(x,y)
: x, y n bits
: xy
if n=1: return xy O(n)
a= n/2 leftmost bits of x, b= n/2 rightmost bits of x
c= n/2 leftmost bits of y, d= n/2 rightmost bits of y
P1 = IntMult2(a,c), P2 = IntMult2(b,d) 3 T(n/2)
P3 = IntMult2((a+b),(c+d))
return P1 2n + (P3 -P1 P2)2n/2 + P2 O(n)

3 n/2 bits (n/2)
2 2 O(n)
6 n bits O(n)

1, if n 1
T ( n) (n) O(n log 2 3 ) O(n1.59 )
3T (n / 2) O(n), if n 1
T
- 2016 - . - 04 - DIVIDE & CONQUER I 8
Matrix Multiplication
: X Y: n x n
Z=X Y

n
Z ij X ik Ykj Complexity: O(n3)
k 1

9
- 2016 - . - 04 - DIVIDE & CONQUER I
Matrix Multiplication
Divide and Conquer

A, B,C,D,E,F,G,H, AE+BG, AF+BH,CE+DG,CF+DH: matrices n/2 x n/2

8 n/2 x n/2 8 (n/2)

4 n/2 x n/2 O(n2)

1, if n 1 3
T ( n) 2
( n ) O ( n )
8T (n / 2) O(n ), if n 1

- 2016 - . - 04 - DIVIDE & CONQUER I 10
Matrix Multiplication: Strassens Algorithm

7 n/2 x n/2
18 n/2 x n/2
[Strassen, 1969]

11
- 2016 - . - 04 - DIVIDE & CONQUER I
 Matrix Multiplication: Strassens Algorithm

7 n/2 x n/2 7 (n/2)
18 n/2 x n/2 O(n2)

1, if n 1 log 2 7 2.808
T ( n) 2
( n ) O ( n ) O ( n )
7T (n / 2) O (n ), if n 1

? ! O(n2.376) !

For n=100
n3 is 1,000,000
n2.808 is 413,048
n2.376 is 56,494

12
- 2016 - . - 04 - DIVIDE & CONQUER I
Master* Theorem
n d
If T (n) aT ( b ) n , a 0, b 1, d 0,
and n b m (m log b n),
then :

O(n d ), if d log b a d
(b a)

T (n) O(n d log b n), if d log b a (b d a)
O(n logb a ), if d log a d
(b a)
b

d log b a
Compare n ~ n
- 2016 - . - 04 - DIVIDE & CONQUER I 13
 Proof of Master* Theorem
n d
T (n) aT ( ) n b
2 n n d d n b m (m log b n)
a T ( ) a( ) n
b2 b

a3T ( bn3 ) a 2 ( bn2 )d a( bn )d nd

m m1 d n d d
a T( ) a n
bm
( bm1 ) a( ) n
n
b
m m
k n d d a k
a ( ) T (n) n
bk ( bd
)
k 0 k 0

- 2016 - . - 04 - DIVIDE & CONQUER I 14

 Proof of Master* Theorem

n b m (m log b n)

Level k : a k subproblems, each of size n / b k

d k
n a
Work for level k : a k nd
k
d
b b
m k
a
Total work for all levels: T (n) n d d
k 0 b

15
- 2016 - . - 04 - DIVIDE & CONQUER I
Proof of Master* Theorem
m
d k
T ( n) n (
k 0
a
bd
)
a
Geometric series with ratio
d
b
Three cases:

a a a
i) d 1 (ii ) d 1 (iii ) d 1
b b b

16
- 2016 - . - 04 - DIVIDE & CONQUER I
 Proof of Master* Theorem
a
Case i: d 1 ( d log b a )
b
m
d 1 k d d k d
T ( n) n ( ) n ( ) n
a
bd a
O ( n ) a
bd
k 0 k 0 1 bd

Case ii:
a
d
1 (d log b a ) m log b n
b

m log b n
d k d d d
T ( n) n (
k 0
a
bd
) n 1 n
k 0
log b n O(n log b n)

- 2016 - . - 04 - DIVIDE & CONQUER I 17

 Proof of Master* Theorem
a
Case iii: x d 1 (d log b a) bm n
b
m log b n

m m 1
d k d x 1 d m
T ( n) n (
k 0
a
bd
) n
x 1
O(n x )

d a m n d m
a n d m
a
O n d O dm O d
b b n

m log b n log b a
O(a ) O(a ) O(n )

- 2016 - . - 04 - DIVIDE & CONQUER I 18

Master* Theorem - Examples
T(n) = 9T(n/3) + n
a=9, b=3, logb a = log3 9=2
d=1 < 2= logb a
case (iii) applies: T ( n) O n O(n )
log b a 2

T(n) = T(2n/3) + 1
a=1, b=3/2, logb a = log3/2 1=0
d=0 = logb a
case (ii) applies: T ( n ) O n 0

log 3 / 2 n O(logn)
T(n) = 5T(n/25) + n2
a=5, b=25, logb a = log255=0.5
d=2 > 0.5 = logb a
d 2
case (i) applies: T (n) O n O(n )
- 2016 - . - 04 - DIVIDE & CONQUER I 19
 Master Theorem
If T(n) = aT(n/b) + f(n) then




n log b a
, if f ( n ) O n
log b a

a 1
b 1

T (n) n logb a log n , if f (n) n logb a
0
c 1
f (n) ,
if f ( n) n logb a
and af (n / b) cf (n)

for large n
- 2016 - . - 04 - DIVIDE & CONQUER I 20