Algorithms and programming Divide and conquer

Paradigms for Problem

Solving: Divide and Conquer
The “divide and conquer” Divide

paradigm  starting from a problem of size n generate

a independent problems of size n/b.
Conquer
 solve the “elementary” problems

Combine
 build an overall solution by combining the
partial solutions.
Recursive implementation.

Complexity: recursion equation

Termination T(n) = D(n) + a T(n/b) + C(n) n>c
condition
T(n) = (1) nc
Solve(Problem): “a” subproblems,
 If problem elementary:
each “b” times Express T(n) in terms of execution time for
smaller smaller inputs:
 Solution = Trivial_solve(Problem)
 D(n): cost of dividing
 else:
recursive  a: number of subproblems generated in
 Subproblem1,2,3,…,a = Divide(Problem) ;
call the divide step
 For each Subproblemi:
 n/b: size of each subproblem
 Subsolutioni = Solve(Subproblemi) ;
 Return Solution = Combine(Subsolution1,2,3,…,a) ;  C(n): cost of ricombination
 (1): cost of the elementary solution

Example

T(n)
Binary search:
Solve(Problem): (1)  D(n) = (1), C(n) = (1)
 If problem elementary:
 a = 1, b = 2
 Solution = Trivial_solve(Problem)
D(n)  Recursion equation:
 else: a subproblems
T(n) = T(n/2) + 1 n>1
 Subproblem1,2,3,…,a = Divide(Problem) ;
For each Subproblemi: T(1) = 1 n=1
 T(n/b)
 Subsolutioni = Solve(Subproblemi) ;  Solution: T(n) = O(logn)
 Return Solution = Combine(Subsolution1,2,3,…,a) ;
C(n)
Algorithms and programming Divide and conquer

Divide and conquer example:

Divide and conquer paradigm: limits The Hanoi Towers
 Assumption: independent subproblems  Initial configuration:
Computations repeated for shared subproblems

• 3 shafts, 3 disks of decreasing size on
first shaft
FIB5 FIB5  Final configuration:
FIB3 FIB4 FIB4  3 disks on third shaft

FIB1 FIB2 FIB3 FIB2  Rules

FIB0 FIB1 FIB1 FIB2

FIB3
• access only to the top disk
FIB0 FIB1
FIB2 • on each disk only smaller disks
FIB0 FIB1
FIB1 FIB0  Generalization: n disks and k shafts.

Example of solution Divide and conquer strategy

0 2 0 1
 Initial problem: move n disks from 0 to 2
0 1 2 0 1 2
 Reduction to subproblems:
2 1 0 2 • n-1 disks from 0 to 1, 2 temporary
0 1 2 0 1 2 storage
1 0 1 2 • last disk from 0 to 2
• n-1 disks from 1 to 2, 0 temporary
0 1 2 0 1 2
storage
0 2  Termination condition: move just 1 disk.

0 1 2 0 1 2

 0, 1, 2: shafts 0, 1, 2
  large disk Problem 000 222 decomposed into 3
subproblems:
  medium disk
1. medium and small disks from 0 to 1 000 011
  small disk
2. large disk from 0 to 2 011 211
 0 means small disk on shaft 0, 2 means
large disk on shaft 2, etc. 3. Medium and small disks from 1 to 2. 211 222
 state 011
 state transition
Algorithms and programming Divide and conquer

Recursion tree Complexity

000 222  Divide: consider n-1 disks

D(n)=(1)
 Solve: solve 2 subproblems whose size is
000 011 011 211 211 222
n-1 each
2T(n-1)
000 002 002 012 012 011 211 210 210 220 220 222  Termination: move 1 disk

(1)
 Combine: no action

C(n) = (1)

Recursive sorting
algorithms
Recurrence equation :
T(n) = 2T(n-1) + 1 n>1
T(1) = 1 n=1
Solution: T(n) = O(2n)

Merge Sort

Recursion
 Division:
 merge sort on left subarray A[p..q]
• 2 subarrays (left and right) with respect
 merge sort on right subarray A[q+1..r]
to the array’s middle element.
 Termination condition: with 1 (p=r) or

p r
0 (p>r) elements the array is sorted
 Ricombination:

 Merge 2 sorted subarrays into one

p q q+1 r sorted array.

Algorithms and programming Divide and conquer

Example
Recursive division: Ricombination:

12 6 4 5 9 2 3 1
12 6 4 5 9 2 3 1
12 6 4 5 9 2 3 1
6 12 4 5 2 9 1 3

12 6 4 5 9 2 3 1 4 5 6 12 1 2 3 9

12 6 4 5 9 2 3 1 1 2 3 4 5 6 9 12

Analysis
Assumption: n = 2k.  Intuitively:
 Divide: divide array in 2 D(n)=(1)
 Solve: solve 2 subproblems of size n/2 each
2T(n/2)
log2 n

 Termination: simple test (1)

 Combine: based on Merge C(n) = (n)
 C(n) + D(n) = (n)
 Recursion equation:
T(n) = 2T(n/2) + n n>1
Recursion levels: log2 n Operations at each level: n
T(1) = 1 n=1
Total operations: n log2 n
 Solution: T(n) = O(n log2 n)

Quicksort

 In place Recursion
 Non stable  quicksort on the left subarray A[p..q]
 Division:
 quicksort on the right subarray A[q+1..r]
• partition array A[p..r] in 2 subarrays:
 termination condition: if the array has
 given a pivot element x

 A[p..q] contains all elements < x

just 1 element, it is sorted
 A[q+1..r] contains all elements  x  Ricombination:
 division doesn’t necessarily halve the array  none.
Algorithms and programming Divide and conquer

Partition Partitioning example

p r
 Pivot x = A[p] A 5 3 2 6 4 1 3 7 x 5
 Find A[i] and A[j], i.e. 2 elements not in the i j
proper partition A 5 3 2 6 4 1 3 7
 descending loop on j until we find an
i j
element smaller than the pivot x
 ascending loop on i until we find an A 3 3 2 6 4 1 5 7
element larger than or equal to the pivot x i j
 Swap A[i] and A[j] A
 Repeat while i < j 3 3 2 1 4 6 5 7
 T(n) = (n).
q j i

Quicksort example Analysis

A 5 3 2 6 4 1 3 7 x 5 Efficiency depends on how balanced are the

partitions.
3 3 2 1 4 x 3 6 5 7 x 6 At each step partition returns:
 in the worst case a subarray with n-1
1 2 x 1 3 3 4 x 3 5 6 7 x 6 elements and a subarray with 1 element
 in the best case 2 subarrays with n/2
1 2 3 3 4 x 3 6 7
elements
 in the average case 2 subarrays of different
3 4
sizes.
A 1 2 3 3 4 5 6 7
Balancing depends on the choice of the pivot.

Worst case Best case

 Worst case: pivot = minimum or

maximum (monotone array)
 Recursion equation:  Recursion equation:
T(n) = T(n-1) + n n>=2 T(n) = 2T(n/2) + n n>=2
T(1) = 1 n=1 T(1) = 1 n=1
 Solution: T(n) = O(n2)  Solution: T(n) = O(n log2 n)
Algorithms and programming Divide and conquer

Average case

n n
 Provided we are not in the worst case,
though partitions may be strongly n/10 9n/10 n
unbalanced, average case = best case log10 n n/100 9n/100 9n/100 81n/100 n
 Example: at each step generate 2
partitions, the first one with 9/10 n and the log10/9 n
second one with n/10 elements.
1
1 n
T(n) = O(n log n)

Pivot selection

 Random element: generate a random

number i with p  i  r, then swap A[0] and
Backtracking
A[i], use A[0] as pivot
 Middle element: x  A[(p+r)/2]

 Select average between min and max

 Select median of 3 elements picked randomly

in array
…

Backtracking Backtracking: applications

Given a problem:
 Count the number of valid solutions
Not a true programming paradigm (like the
greedy one), since no general scheme exists.  Enumerate all the valid solutions:

Rather, it is an algorithmic technique aiming  anagrams of a word

at examining all the possible valid solutions  possible combinations of elements

within a given search space. taken from a set S
A solution is said to be valid when it satisfies  Find at least one valid solution (search
the problem constraints. problem)
 Find, in the set of valid solutions, the
optimal one according to some criterions
(optimization problem)
Algorithms and programming Divide and conquer

Backtracking: basic principles Backtracking: basic principles

 Assume that it is necessary to take a
sequence of decisions, each one among a
set of possible choices, without any
information guiding the process
 Every decision leads to a new set of
possible choices
 Some decision sequences lead to a valid
problem solution, some others do not
 A brute-force approach tries all the
possible combinations of choices, thus
exploring the whole space
 Conceptually, a solution can be thought as
an array S[1..n] recording the sequence of
choices which have been taken
 At each step, a partial solution S[1..k] is
the starting point:
 if S[1..k] represents a valid solution, the search
succeeds (stop recurring)
 otherwise:
 if it is possible to make a choice k+1,
extend S[1..k] to S[1..k+1] and recur
 undo the last choice (backtrack) and restart
from S[1..k] with another choice

Backtracking: basic principles Example #1: combinations

All the valid (and not valid as well) solutions
of a problem are thus enumerated by brute-
force, exploring the whole solution space,
represented as a decision tree:
 The root is the initial (empty) solution
 Internal nodes represent partial solutions
 Leaves are the possible full solutions
Given a set S of N integer values, generate all
the combinations (without repetitions) of M
numbers taken from S
The number of combinations is N!/((N-M)!*M!)
Examples:
 S = {1, 2, 3, 4}, M=2: 6 combinations

{1,2} {1,3} {1,4} {2,3} {2,4} {3,4}

 S = {1, 2, 3, 4, 5}, M=4: 5 combinations

{1,2,3,4} {1,2,3,5} {1,2,4,5} {1,3,4,5} {2,3,4,5}

S 1 2 3 4 N=4 M=3
Example #1: combinations
 Array v: it represents the set S start =0
 Array c: it represents the (current) combination pos = 0 1 2 3 4
i=0 i=1 i=2 i=3
 At each step (pos), a new element from v is
added to c (in position pos)
start =1 start =2 start =3
 The element to be added to c is “selected” by pos = 1 1 2 1 3 2 3 2 4
1 4 3 4
scanning v through an index i i=1 i=2 i=3 i=2 i=3 i=3

 In order to skip repeated combinations, it is

necessary to avoid the selection of previously start =2 start = 3 start =3
pos = 2 1 2 3 1 2 4 1 3 4 2 3 4
added values: the value of the i index gives i=2 i=3 i=3 i=3
the starting position (start) for the following
element selections
Algorithms and programming Divide and conquer

int comb(int *v, int n, int *c, int m, int comb(int *v, int n, int *c, int m,
int start, int pos, int count) int start, int pos, int count)
{ {
int i; int i;
if (pos >= m) { if (pos >= m) { termination:
for (i=0; i<m; i++) for (i=0; i<m; i++)
display solution
printf("%d ", c[i]); printf("%d ", c[i]);
printf("\n"); printf("\n");
return count+1; return count+1;
} }
for (i=start; i<n; i++) { for (i=start; i<n; i++) {
c[pos] = v[i]; c[pos] = v[i];
count = comb(v, n, c, m, i+1, pos+1, count); count = comb(v, n, c, m, i+1, pos+1, count);
} }
return count; return count;
} }

int comb(int *v, int n, int *c, int m, int comb(int *v, int n, int *c, int m,
int start, int pos, int count) int start, int pos, int count)
{ {
int i; int i;
if (pos >= m) { if (pos >= m) {
for (i=0; i<m; i++) for (i=0; i<m; i++)
printf("%d ", c[i]); printf("%d ", c[i]);
printf("\n"); printf("\n"); c[pos] filled with all v values
return count+1; return count+1; from the start position
stop recursion
} }
for (i=start; i<n; i++) { for (i=start; i<n; i++) {
c[pos] = v[i]; c[pos] = v[i];
count = comb(v, n, c, m, i+1, pos+1, count); count = comb(v, n, c, m, i+1, pos+1, count);
} }
return count; return count;
} }

Example #1: combinations

int comb(int *v, int n, int *c, int m,  Array c: its elements contain the information on
int start, int pos, int count) whether the corresponding v elements belong
{
int i; to the (current) combination or they don’t
if (pos >= m) {
for (i=0; i<m; i++)  At each step (i), both possibilities for the
printf("%d ", c[i]); corresponding element from v are considered
printf("\n");
return count+1;  Keep track of the total number (tot) of values
}
for (i=start; i<n; i++) { added to the (current) combination
c[pos] = v[i];
count = comb(v, n, c, m, i+1, pos+1, count);  Adding a new value to the combination makes
} sense only if tot < m
return count;
}
recur over i+1 (start) and pos+1 (pos)
 Not adding a new value to the combination
makes sense only if tot+(n-i) > m
Algorithms and programming Divide and conquer

void comb(int *v, int n, int *c, int m, int i, int tot) void comb(int *v, int n, int *c, int m, int i, int tot)
{ {
int j; int j;
if (i >= n) { if (i >= n) {
for (j=0; j<n; j++) for (j=0; j<n; j++)
termination:
if (c[j] != 0) if (c[j] != 0)
display solution
printf("%d ", v[j]); printf("%d ", v[j]);
printf("\n"); printf("\n");
return; return;
} }
if (n-i+tot > m) { if (n-i+tot > m) {
c[i] = 0; comb(v, n, c, m, i+1, tot); c[i] = 0; comb(v, n, c, m, i+1, tot);
} }
if (tot < m) { if (tot < m) {
c[i] = 1; comb(v, n, c, m, i+1, tot+1); c[i] = 1; comb(v, n, c, m, i+1, tot+1);
} }
} }

void comb(int *v, int n, int *c, int m, int i, int tot) void comb(int *v, int n, int *c, int m, int i, int tot)
{ {
int j; int j;
if (i >= n) { if (i >= n) {
for (j=0; j<n; j++) for (j=0; j<n; j++)
if (c[j] != 0) if (c[j] != 0)
printf("%d ", v[j]); printf("%d ", v[j]);
printf("\n"); printf("\n");
check wether we can skip the current element
return; return;
stop recursion
} }
if (n-i+tot > m) { if (n-i+tot > m) {
c[i] = 0; comb(v, n, c, m, i+1, tot); c[i] = 0; comb(v, n, c, m, i+1, tot);
} }
if (tot < m) { if (tot < m) {
c[i] = 1; comb(v, n, c, m, i+1, tot+1); c[i] = 1; comb(v, n, c, m, i+1, tot+1);
} }
} }

void comb(int *v, int n, int *c, int m, int i, int tot) void comb(int *v, int n, int *c, int m, int i, int tot)
{ {
int j; int j;
if (i >= n) { if (i >= n) {
for (j=0; j<n; j++) for (j=0; j<n; j++)
if (c[j] != 0) if (c[j] != 0)
printf("%d ", v[j]); printf("%d ", v[j]);
printf("\n"); printf("\n");
skip the current element recur over the next element
return; return;
} }
if (n-i+tot > m) { if (n-i+tot > m) {
c[i] = 0; comb(v, n, c, m, i+1, tot); c[i] = 0; comb(v, n, c, m, i+1, tot);
} }
if (tot < m) { if (tot < m) {
c[i] = 1; comb(v, n, c, m, i+1, tot+1); c[i] = 1; comb(v, n, c, m, i+1, tot+1);
} }
} }
Algorithms and programming Divide and conquer

void comb(int *v, int n, int *c, int m, int i, int tot) void comb(int *v, int n, int *c, int m, int i, int tot)
{ {
int j; int j;
if (i >= n) { if (i >= n) {
for (j=0; j<n; j++) for (j=0; j<n; j++)
if (c[j] != 0) if (c[j] != 0)
printf("%d ", v[j]); printf("%d ", v[j]);
printf("\n"); printf("\n");
return; return;
} }
if (n-i+tot > m) { if (n-i+tot > m) {
check n,
c[i] = 0; comb(v, wether
c, we
m, can addtot);
i+1, the current element addm,
c[i] = 0; comb(v, n, c, thei+1,
current element
tot);
} }
if (tot < m) { if (tot < m) {
c[i] = 1; comb(v, n, c, m, i+1, tot+1); c[i] = 1; comb(v, n, c, m, i+1, tot+1);
} }
} }

S 1 2 3 4 N=4 M=2

void comb(int *v, int n, int *c, int m, int i, int tot)
{
int j;
if (i >= n) { n-i+tot >m
for (j=0; j<n; j++) 4–0+0 >2
if (c[j] != 0)
printf("%d ", v[j]);
printf("\n");
return;
}
if (n-i+tot > m) { recur over the next element
c[i] = 0; comb(v, n, c, m, i+1, tot);
}
if (tot < m) {
c[i] = 1; comb(v, n, c, m, i+1, tot+1);
}
}

0 0

n-i+tot >m
4–1+0 >2 00

n-i+tot >m
4–2+0 >2
Algorithms and programming Divide and conquer

0 0

00 00

tot < m
0<2 0 01
n-i+tot >m
4–3+1 >2

0 0

00 00

00 1 0 01

tot < m
1<2 0 01 1
{1,2}

0 0

00 tot < m 00 01
0<2
n-i+tot >m
00 1 0 01 4–2+1 >2

00 11 0 01 1
Algorithms and programming Divide and conquer

0 0

00 01 00 01

00 1 01 0 0 01 01 0

n-i+tot >m tot < m

0 0 1 1 4–3+1 >2 0 01 1 1<2

0 0

00 01 00 01

tot < m
00 1 01 0 0 01 01 0
1<2

00 11 0 10 1 0 01 1 01 01
{1,3}

0 0

00 01 00 01

00 1 01 0 01 1 0 01 01 0 0 11

n-i+tot >m
00 11 0 10 1 4–3+2 >2 0 01 1 01 01 01 10
{2,3}
Algorithms and programming Divide and conquer

0<2

0
4-1+1 < 2 1 1<2

00 01 10 11
1<2 4-2+2< 2 2<2
00 1 01 0 01 1 10 0 1 01 1 10
4-3+1 < 2 1<2 4-3+2< 2 2 < 2 4-3+2< 2 2<2
tot < m
00 11 0 10 1 0 11 0 2<2 10 01 1 01 0 1 10 0
{1,4} {2,4} {3,4}

Example #2: permutations Example #2: permutations

Given a set S of N integer values, generate all  Array v: it represents the set S
their possible permutations  Array p: it represents the (current) permutation
The number of permutations is N!  At each step (pos), a new element from v is
Example: added to p (in position pos)
 S = {1, 2, 3} : 6 permutations  The element to be added to p is “selected” by
{1,2,3} {1,3,2} {2,1,3} {2,3,1} {3,1,2} {3,2,1} completely scanning v through an index i
 S = {1, 2, 3, 4} : 24 permutations  In order to skip repetitions, the used values are
{1,2,3,4} {1,2,4,3} {1,3,2,4} {1,3,4,2} {1,4,2,3} {1,4,3,2} marked with a flag (array m): however, after a
{2,1,3,4} {2,1,4,3} {2,3,1,4} {2,3,4,1} {2,4,1,3} {2,4,3,1} value has been used, it is necessary to unmark
{3,1,2,4} {3,1,4,2} {3,2,1,4} {3,2,4,1} {3,4,1,2} {3,4,2,1} it (backtrack) before considering the next
{4,1,2,3} {4,1,3,2} {4,2,1,3} {4,2,3,1} {4,3,1,2} {4,3,2,1} element in the v array

void perm(int *v, int *p, int *m, int n, int pos) void perm(int *v, int *p, int *m, int n, int pos)
{ {
int i; int i;
if (pos >= n) { if (pos >= n) {
for (i=0; i<n; i++) printf("%d ", p[i]); for (i=0; i<n; i++) printf("%d ", p[i]);
printf("\n"); printf("\n");
return; return; termination:
} } display solution
for (i=0; i<n; i++) { for (i=0; i<n; i++) {
if (m[i] == 0) { if (m[i] == 0) {
m[i] = 1; m[i] = 1;
p[pos] = v[i]; p[pos] = v[i];
perm(v, p, m, n, pos+1); perm(v, p, m, n, pos+1);
m[i] = 0; m[i] = 0;
} }
} }
} }
Algorithms and programming Divide and conquer

void perm(int *v, int *p, int *m, int n, int pos) void perm(int *v, int *p, int *m, int n, int pos)
{ {
int i; int i;
if (pos >= n) { if (pos >= n) {
for (i=0; i<n; i++) printf("%d ", p[i]); for (i=0; i<n; i++) printf("%d ", p[i]);
printf("\n"); printf("\n");
return; return; iterate over
stop recursion all values in v
} }
for (i=0; i<n; i++) { for (i=0; i<n; i++) {
if (m[i] == 0) { if (m[i] == 0) {
m[i] = 1; m[i] = 1;
p[pos] = v[i]; p[pos] = v[i];
perm(v, p, m, n, pos+1); perm(v, p, m, n, pos+1);
m[i] = 0; m[i] = 0;
} }
} }
} }

void perm(int *v, int *p, int *m, int n, int pos) void perm(int *v, int *p, int *m, int n, int pos)
{ {
int i; int i;
if (pos >= n) { if (pos >= n) {
for (i=0; i<n; i++) printf("%d ", p[i]); for (i=0; i<n; i++) printf("%d ", p[i]);
printf("\n"); printf("\n");
return; return;
} }
for (i=0; i<n; i++) { if a value has not been used yet for (i=0; i<n; i++) {
if (m[i] == 0) { if (m[i] == 0) { mark it
m[i] = 1; m[i] = 1;
p[pos] = v[i]; p[pos] = v[i];
perm(v, p, m, n, pos+1); perm(v, p, m, n, pos+1);
m[i] = 0; m[i] = 0;
} }
} }
} }

void perm(int *v, int *p, int *m, int n, int pos) void perm(int *v, int *p, int *m, int n, int pos)
{ {
int i; int i;
if (pos >= n) { if (pos >= n) {
for (i=0; i<n; i++) printf("%d ", p[i]); for (i=0; i<n; i++) printf("%d ", p[i]);
printf("\n"); printf("\n");
return; return;
} }
for (i=0; i<n; i++) { for (i=0; i<n; i++) {
if (m[i] == 0) { if (m[i] == 0) { recur over the next position
m[i] = 1; use it m[i] = 1;
p[pos] = v[i]; p[pos] = v[i];
perm(v, p, m, n, pos+1); perm(v, p, m, n, pos+1);
m[i] = 0; m[i] = 0;
} }
} }
} }
Algorithms and programming Divide and conquer

void perm(int *v, int *p, int *m, int n, int pos) 0 0 0 array m 1 2 3
{ ? ? ? array p
int i; array v
if (pos >= n) {
for (i=0; i<n; i++) printf("%d ", p[i]);
printf("\n"); 1 0 0 0 1 0 0 0 1
return; 1 ? ? 2 ? ? 3 ? ?
}
for (i=0; i<n; i++) {
if (m[i] == 0) {
m[i] = 1; 1 1 0 1 0 1 1 1 0 0 1 1 1 0 1 0 1 1
p[pos] = v[i]; 1 2 ? 1 3 ? 2 1 ? 2 3 ? 3 1 ? 3 2 ?
perm(v, p, m, n, pos+1);
m[i] = 0;
} unmark the used value 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
}
}
1 2 3 1 3 2 2 1 3 2 3 1 3 1 2 3 2 1

Example #3: sum of sub-sets Example #3: sum of sub-sets

Given a set S of positive integer values and an  Array v: it represents the (sorted) set S
integer number N, generate all the sub-sets Y of  Array m: it “flags” the values in S which have
S such that the sum of the elements in Y is been currently taken
equal to N  Two integer variables (ps and t) represent the
sum of the selected elements and the sum of
Example: the elements still to consider, respectively
 At each step (i), a new element from v is
 S = {1, 2, 4, 6}, N = 7
analyzed:
result = {{1, 2, 4}, {1, 6}}
 The element can be added if: ps+v[i] <= sum
 The element can be skipped if: ps+t >= sum

void sumset(int *v,int *m,int i,int ps,int t,int sum)

{}
{
int j;
if (ps == sum) { {1} {2} {4} {6}
printf("{ ");
for (j=0;j<i;j++)
if (m[j]) {1,2} {1,4} {1,6} {2,4} {2,6} {4,6}
printf("%d ",v[j]);
printf("}\n");
return; {1,2,4} {1,2,6} {1,4,6} {2,4,6}
}
if (ps+v[i]<=sum && ps+t>=sum){
m[i]=1;
Node OK
sumset(v, m, i+1, ps+v[i], t-v[i], sum);
m[i]=0; “promising” choice Node KO
sumset(v, m, i+1, ps, t-v[i], sum);
}
} not “promising” choice Undecided node
Algorithms and programming Divide and conquer

Search and optimization problems

Search and optimization  A problem is said to be a search problem when
all its solutions are equivalent: stop recursion
problems as soon as one solution is found
 The returned value of the function is often used for
this purpose
 A problem is said to be an optimization problem
when its solutions are NOT equivalent (they
can be ranked according to some criterions):
completely (recursively) explore the solution
space, keeping track of the best solution
 Two “variables”, representing the current and the
(temporarily) best solutions are necessary

Example #1: map coloring Example #1: map coloring

 Given a map consisting of several distinct  The map can be represented as a graph,
regions, color each region in such a way where every region corresponds to a vertex
that any two adjacent regions have a and edges connect the adjacent regions
different color  Regions numbered with integer values

 It is possible to prove that 4 colors are starting from 0, edges represented through
enough in order to achieve the goal above the adjacency matrix
for any “planar” map int map[MAX][MAX] = {
{ 0, 1, 1, 0, 1, 1, 0},
{ 1, 0, 0, 0, 1, 1, 1}, 0 1
{ 1, 0, 0, 1, 1, 1, 1}, 4
{ 0, 0, 1, 0, 1, 0, 1}, 2 3
{ 1, 1, 1, 1, 0, 0, 1},
{ 1, 1, 1, 0, 0, 0, 1}, 6
5
{ 0, 1, 1, 1, 1, 1, 0}
};

Example #1: map coloring

 The color of every region is recorded in the int colorMap(int map[][MAX],int n,int r,int *color)
color array {
int i;
 All regions are recursively considered. For if (r >= n)
each region a color is decided (by trying all return 1;
the possibilities) in such a way that it is for (i=RED; i<=BLUE; i++) {
different from the adjacent regions: if (check(map, n, r, i, color)) {
color[r] = i;
 If a valid color is found, recur on the next region if (colorMap(map, n, r+1, color))
 If no valid color is found, terminate recursion return 1;
with a failure result color[r] = NONE;
}
 If all regions have been enumerated, }
terminate the recursion with a success return 0;
result }
Algorithms and programming Divide and conquer

Example #2: the 8 queens problem The 8 queens: resolution

 Empty 8x8 chessboard, 8 queens  Strategy
 Place all the queens on the chessboard in ● Recursively place one queen onto an

such a way that none of them can eat any empty cell
of the other ones ● Termination (all queens placed): check
whether the queens’ configuration is valid
 Optimization

● Recursively place one queen only onto a

Solution free (not “eatable”) cell
● Termination (all queens placed): no
check needed (solution by construction)

The 8 queens: further optimization The 8 queens: further optimization

 15 diagonals:  15 reverse diagonals:
● identified by a constant sum of the ● identified by a constant difference of
indices of their cells (i+j) the indices of their cells (i-j)
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

0 8 0
1 9 1 0
2 10 2 1
3 11 3 2
4 12 4 3
5 13 5 4
6 14 6 5
7 7 6
0 1 2 3 4 5 6 7 14 13 12 11 10 9 8 7

The 8 queens: further optimization The 8 queens: further optimization

 ith
row (implicitly) occupied by the ith queen  set_queen(i, j, …): place the ith queen on
column j, marking the proper elements of
 queen[i] = index of the column where the ith
arrays col, diag and revdiag
queen has been placed
 rem_queen(i, j, …): remove the ith queen
 the ith queen, placed in column j, can eat from column j, cleaning the proper cells of
any other queen on: arrays col, diag and revdiag
• the jth column: col[j]  place_recur(n, …): recursively place the nth

• the (i+j)th diagonal: diag[i+j] queen in row n (all columns are tried).
Termination: all queens have been placed.
• the (i-j)th reverse diagonal: revdiag[i-j+7]
Algorithms and programming
Example #3: chemical substances
An industrial process consists of 10 phases, for each
of which a chemical substance, chosen within a set,
must be used.
 Every substance can be used for a specific phase
only, and it is characterized by a price
 Every substance may be incompatible with other
substances that can be used in the process (in
other phases), so that using a substance makes it
impossible the use of all the substances which are
incompatible with it (in all the previous and
subsequent phases)
Example #3: chemical substances
Input file format:
 First row provides the total number of substances
 For every substance, there are:
 one line containing the substance name, its price, the phase
in which it can be used and the number of substances with
which it is incompatible (among the ones previously
reported in the file itself), according to the format:
<name> <price> <phase> <number of incompatibilities>
as many lines as the number of incompatible substances,

each of which provides the name of one of the incompatible
substances
Example #3: chemical substances
Data structures:
 Substances
 single set containing all the substances OR separated
sets (one set specifying only the substances suitable for a
given phase)
 Incompatibilities
 Graph! (vertices = substances, edges = incompatibilities)
Representation by means of an adjacency matrix or an
adjacency list
 Solutions (current and optimal)
 Two arrays (size?)
Divide and conquer
Example #3: chemical substances
The program:
 must load the information about the available
substances (price, limited to value 1000, phase
and list of incompatible substances)
 must determine the chemical substance to be
used for each phase, in such a way that all the
choosen substances are each other compatible
 must minimize the overall process cost (sum of
all the costs of the selected substances)
Example #3: chemical substances
Resolution algorithm:
 Enumerate (recursively) all the possible solutions:
the recursive function at level i manages only the
ith phase of the process: it has to try ALL the
substances suitable for that phase (preventing
incompatibilities), every time activating the
completion of the subsequent phases.
 Keep track of the best solution (minimum cost)
obtained while enumerating the possible
configurations  two (identical) data structures
needed: “current” and “optimal” solutions.
Whenever a new (current) solution is found, it is
compared against the optimal solution, possibly
updating the latter one if the former one is better.
Example #3: chemical substances
Pseudo-code for the recursive function:
explore(…, int p) // p represents the current phase
// termination
if all the phases have already been processed
check current solution against the optimal one
return
// recursive step
for each substance s usable in phase p
if s is compatible with the previously selected subst.
select s
explore(…, p+1)
deselect s
Algorithms and programming Divide and conquer

Example #4: suppliers Example #4: suppliers

Two text files provide information about:
 (suppliers.txt) a set of available products: for
each product the set of firms selling it is also
specified
 (purchases.txt) a sub-set of products (among the
ones given in the first file) which must be bought
The program must select the minimal sub-set of
suppliers able to provide all the products specified in
the second file.
“suppliers.txt” file format
 first row giving the total number of suppliers ns and
products np
 ns subsequent rows: suppliers’ names (one per row)
 remaining lines: list of products; for the ith product
there is one line specifying its name and the number
nsi of suppliers selling it, then nsi lines containing
the names of such firms
“purchase.txt” file format
 at most np lines, each one giving the name of a
product

Example #4: suppliers Example #4: suppliers

suppliers.txt purchase.txt
3 10 Resolution algorithm:
supplier1 productD  Enumerate (recursively) all the possible solutions:
supplier2 productA the recursive function at level i manages only the
supplier3 productC ith product to buy: it has to try ALL the suppliers
productA 2 … selling it, activating then the same process for the
supplier1 subsequent products.
supplier3  Keep track of the optimal solution (minimal
productB 1 cardinality of the selected suppliers’ set) obtained
supplier2 while enumerating the possible configurations 
… two (identical) data structures needed: “current”
productF 3 and “optimal” solutions. Whenever a new
supplier3 (current) solution is found, it is compared against
supplier1 the optimal solution, possibly updating the latter
supplier2 one if the former one is better.
…

Example #4: suppliers Example #4: suppliers

Pseudocode for the recursive function:
Data structures: explore(…, int p) // p represents the current purchase
 Suppliers and products // termination
if all the purchases have already been processed
 arrays of structures, containing their names and other
check current solution against the optimal one
additional information
return
 Correspondences between products and suppliers // recursive step
 Graph! (vertices = suppliers/products, edges = selling for each supplier s of the current purchase
relationships) Representation by means of an adjacency if s was not already selected
matrix or an adjacency list
select s
 Solutions (current and optimal) explore(…, p+1)
 Two arrays (size?) deselect s
else
explore(…, p+1)