BackTracking Algorithms

Topics

 What is Backtracking
 N-Queens Problem
 Sum of Subsets
 Graph Coloring
 Hamiltonian Circuits
 Other Problems

2
 Algorithm Design

Human
Problems Result

Input Data Output Data

Structures Processing Structures

Computer
Algorithms
3
Algorithm Design …
For a problem? What is an Optimal Solution?

• Minimum CPU time

• Minimum memory
Example: Given 4 numbers, sort it to nonincreasing order.
Method 1: Sequential comparison
1. Find the largest (3 comparisons)
2. Find the second largest (2 comparisons)
3. Find the third largest (1 comparisons)
4. Find the fourth largest
A total of 6 comparisons

4
Algorithm Design …
For a problem? What is an Optimal Solution?

• Minimum CPU time

• Minimum memory
Example: Given 4 numbers, sort it to nonincreasing order.
Method 2: Somewhat clever method
a1 a2 a3 a4 a2 a3
(4 comparisons)

a2 a4 a3
(5 comparisons)
a2 a3 a1 a3
a4
5
a2 a3 or a1
 Backtracking Problems

 Find your way through the well-known maze of

hedges by Hampton Court Palace in England? Until
you reached a dead end.

 0-1 Knapsack problem – exponential time

complexity.

 N-Queens problem.

6
Backtracking
 Suppose you have to make a series of
decisions, among various choices, where
– You don’t have enough information to know what
to choose
– Each decision leads to a new set of choices
– Some sequence of choices (possibly more than
one) may be a solution to your problem
 Backtracking is a methodical way of trying out
various sequences of decisions, until you find
one that “works”
7
 Introduction

 Backtracking is used to solve problems in which a sequence

of objects is chosen from a specified set so that the sequence
satisfies some criterion.
 Backtracking is a modified depth-first search of a
tree.
 Backtracking involves only a tree search.
 Backtracking is the procedure whereby, after
determining that a node can lead to nothing but dead
nodes, we go back (“backtrack”) to the node’s parent
and proceed with the search on the next child.
8
 Introduction …

 We call a node nonpromising if when visiting the node

we determine that it cannot possibly lead to a solution.
Otherwise, we call it promising.
 In summary, backtracking consists of
– Doing a depth-first search of a state space tree,
– Checking whether each node is promising, and, if it is
nonpromising, backtracking to the node’s parent.
 This is called pruning the state space tree, and the
subtree consisting of the visited nodes is called the
pruned state space tree.
9
Solving a maze

 Given a maze, find a path from start to finish

 At each intersection, you have to decide between three or fewer
choices:
– Go straight
– Go left
– Go right
 You don’t have enough information to choose correctly
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not) lead to a
solution
 Many types of maze problem can be solved with backtracking
10
 Coloring a map

 You wish to color a map with

not more than four colors
– red, yellow, green, blue
 Adjacent countries must be in
different colors
 You don’t have enough information to choose colors
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not)
lead to a solution
 Many coloring problems can be solved with
backtracking 11
Solving a puzzle

 In this puzzle, all holes but one are filled

with white pegs
 You can jump over one peg with
another
 Jumped pegs are removed
 The object is to remove all but the last peg
 You don’t have enough information to jump correctly
 Each choice leads to another set of choices
 One or more sequences of choices may (or may not)
lead to a solution
 Many kinds of puzzle can be solved with backtracking 12
 Backtracking (animation)

? dead end
dead end
dead end
?
start ? ? dead end
dead end
?
success!

13
N-Queens Problem

 Try to place N queens on an N * N board

such that none of the queens can attack
another queen.
 Remember that queens can move
horizontally, vertically, or diagonally any
distance.
 Let’s consider the 8 queen example…

14
The 8-Queens Example

15
 0 1 2 3 4 5 6 7
0
1
2
3
4
5
6
7

16
A t ( 4 , 4 ) a t t a c k f r o m ( 0 ,0 ) A t ( 5 ,4 ) a t t a c k f r o m ( 2 ,1 ) A t ( 6 , 4 ) a t t a c k f r o m ( 4 ,2 ) A t ( 7 ,4 ) s u c c e s s

0 1 2 3 4 5 6 7
0
1
2
3
4
5
6

7 17
Let’s look at it run

18
 Terminology I

A tree is composed of nodes

There are three kinds of

nodes:
The (one) root node
Internal nodes
Backtracking can be thought of
Leaf nodes as searching a tree for a
particular “goal” leaf node 19
Terminology II

 Each non-leaf node in a tree is a parent of one or

more other nodes (its children)
 Each node in the tree, other than the root, has
exactly one parent parent
Usually, however,
we draw our trees
downward, with
parent
the root at the top
children children
20
Real and virtual trees

 There is a type of data structure called a tree

– But we are not using it here
 If we diagram the sequence of choices we
make, the diagram looks like a tree
– In fact, we did just this a couple of slides ago
– Our backtracking algorithm “sweeps out a tree” in
“problem space”

21
The backtracking algorithm

 Backtracking is really quite simple--we “explore”

each node, as follows:
 To “explore” node N:
1. If N is a goal node, return “success”
2. If N is a leaf node, return “failure”
3. For each child C of N,
3.1. Explore C
3.1.1. If C was successful, return “success”
4. Return “failure”
22
 Sum-of-Subsets problem

 Recall the thief and the 0-1 Knapsack problem.

 The goal is to maximize the total value of the stolen
items while not making the total weight exceed W.
 If we sort the weights in nondecreasing order before
doing the search, there is an obvious sign telling us that
a node is nonpromising.

23
 Sum-of-Subsets problem …

 Let total be the total weight of the remaining weights, a

node at the ith level is nonpromising if
weight + total > W

24
Example

 Say that our weight values are 5, 3, 2, 4, 1

 W is 8
 We could have
– 5+3
– 5+2+1
– 4+3+1
 We want to find a sequence of values that
satisfies the criteria of adding up to W

25
Tree Space
 Visualize a tree in which the children of the root
indicate whether or not value has been picked (left is
picked, right is not picked)
 Sort the values in non-decreasing order so the
lightest value left is next on list
 Weight is the sum of the weights that have been
included at level i
 Let weight be the sum of the weights that have been
included up to a node at level i. Then, a node at the
ith level is nonpromising if
weight + wi+1 > W
26
 Sum-of-Subsets problem …

 Example: Show the pruned state space tree when

backtracking is used with n = 4, W = 13, and w1 = 3,
w2 = 4, w3 = 5, and w4 = 6. Identify those
nonpromising nodes.

27
Full example: Map coloring

 The Four Color Theorem states that any map on

a plane can be colored with no more than four
colors, so that no two countries with a common
border are the same color
 For most maps, finding a legal coloring is easy
 For some maps, it can be fairly difficult to find a
legal coloring
 We will develop a complete Java program to
solve this problem 28
Data structures

 We need a data structure that is easy to work with,

and supports:
– Setting a color for each country
– For each country, finding all adjacent countries
 We can do this with two arrays
– An array of “colors”, where countryColor[i] is the color of
the ith country
– A ragged array of adjacent countries, where map[i][j] is the
jth country adjacent to country i
 Example: map[5][3]==8 means the 3th country adjacent to
country 5 is country 8 29
Creating the map 0 1
4
2 3

int map[][]; 6
5

void createMap() {
map = new int[7][];
map[0] = new int[] { 1, 4, 2, 5 };
map[1] = new int[] { 0, 4, 6, 5 };
map[2] = new int[] { 0, 4, 3, 6, 5 };
map[3] = new int[] { 2, 4, 6 };
map[4] = new int[] { 0, 1, 6, 3, 2 };
map[5] = new int[] { 2, 6, 1, 0 };
map[6] = new int[] { 2, 3, 4, 1, 5 }; 30
}
Setting the initial colors

static final int NONE = 0;

static final int RED = 1;
static final int YELLOW = 2;
static final int GREEN = 3;
static final int BLUE = 4;

int mapColors[] = { NONE, NONE, NONE, NONE,

NONE, NONE, NONE };

31
The main program

(The name of the enclosing class is ColoredMap)

public static void main(String args[]) {

ColoredMap m = new ColoredMap();
m.createMap();
boolean result = m.explore(0, RED);
System.out.println(result);
m.printMap();
}
32
 The backtracking method

boolean explore(int country, int color) {

if (country >= map.length) return true;
if (okToColor(country, color)) {
mapColors[country] = color;
for (int i = RED; i <= BLUE; i++) {
if (explore(country + 1, i)) return true;
}
}
return false;
}
33
Checking if a color can be used

boolean okToColor(int country, int color) {

for (int i = 0; i < map[country].length; i++) {
int ithAdjCountry = map[country][i];
if (mapColors[ithAdjCountry] == color) {
return false;
}
}
return true;
}
34
Printing the results

void printMap() {
for (int i = 0; i < mapColors.length; i++) {
System.out.print("map[" + i + "] is ");
switch (mapColors[i]) {
case NONE: System.out.println("none"); break;
case RED: System.out.println("red"); break;
case YELLOW: System.out.println("yellow"); break;
case GREEN: System.out.println("green"); break;
case BLUE: System.out.println("blue"); break;
}
} 35
}
Recap
 We went through all the countries recursively, starting
with country zero
 At each country we had to decide a color
– It had to be different from all adjacent countries
– If we could not find a legal color, we reported failure
– If we could find a color, we used it and recurred with
the next country
– If we ran out of countries (colored them all), we
reported success
 When we returned from the topmost call, we were done36
Hamiltonian Circuits Problem

 Hamiltonian circuit (tour) of a graph is a path

that starts at a given vertex, visits each
vertex in the graph exactly once, and ends at
the starting vertex.

37
State Space Tree

 Put the starting vertex at level 0 in the tree

 At level 1, create a child node for the root
node for each remaining vertex that is
adjacent to the first vertex.
 At each node in level 2, create a child node
for each of the adjacent vertices that are not
in the path from the root to this vertex, and
so on.
38
Example
18
C F

3 15 4 10

12 5 19
A B D H

6 5 4
22
E G

39
Backtracking Algorithms

How can a computer play

the game?

xx Remember Deep Blue?

The tic-tac-toe game

40
Backtracking Algorithms(1,1)C
0 1 2

0
x (0,0)H (0,1)H (0,2)H (1,0)H...

1 (0,0)C, (0,1)C, (1,0)C...

2
x (0,1)H, (1,0)H, …, (2,2)H

: Computer
x : Human
The tic-tac-toe game
(0,1)C, (1,0)C, (1,2)C, (2,0)C...

41
 Backtracking Algorithms

3 missionaries and 2 cannibals want to cross the river

Condition:
1. A boat can take one or two (must include a missionary)
2. At any time, on either bank, the number of missionaries
must not be less than the number of cannibals.
42
Backtracking Search

Essentially a simplified depth-first

algorithm using recursion

43
Backtracking Search
(3 variables)

Assignment = {}
44
Backtracking Search
(3 variables)

X1

v11

Assignment = {(X1,v11)}
45
Backtracking Search
(3 variables)

X1

v11

X3

v31

Assignment = {(X1,v11), (X3,v31)}

46
Backtracking Search
(3 variables)

X1

v11
Then, the search algorithm
X3 backtracks to the previous
variable and tries another value
v31

X2
Assume that no value of X2
leads to a valid assignment

Assignment = {(X1,v11), (X3,v31)}

47
Backtracking Search
(3 variables)

X1

v11

X3

v31 v32

X2

Assignment = {(X1,v11), (X3,v32)}

48
Backtracking Search
(3 variables)

X1
The search algorithm
backtracks to the previous
v11 variable (X3) and tries
another value. But assume
X3 that X3 has only two
possible values. The
v31 v32
algorithm backtracks to X1
X2 X2
Assume again that no value of
X2 leads to a valid assignment

Assignment = {(X1,v11), (X3,v32)}

49
Backtracking Search
(3 variables)

X1

v11 v12

X3

v31 v32

X2 X2

Assignment = {(X1,v12)}
50
Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2

Assignment = {(X1,v12), (X2,v21)}

51
Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2 The algorithm need not consider

the variables in the same order in
this sub-tree as in the other

Assignment = {(X1,v12), (X2,v21)}

52
Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

X2 X2 X3

v32

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

53
Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

The algorithm need
X2 X2 X3 not consider the values
of X3 in the same order
v32
in this sub-tree

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

54
Backtracking Search
(3 variables)

X1

v11 v12

X3 X2

v31 v32 v21

Since there are only
X2 X2 X3 three variables, the
assignment is complete
v32

Assignment = {(X1,v12), (X2,v21), (X3,v32)}

55
 [This recursive algorithm keeps too much data in memory.
An iterative version could save memory (left as an exercise)]

Backtracking Algorithm

CSP-BACKTRACKING(A)
1. If assignment A is complete then return A
2. X  select a variable not in A
3. D  select an ordering on the domain of X
4. For each value v in D do
a. Add (Xv) to A
b. If A is valid then
i. result  CSP-BACKTRACKING(A)
ii. If result  failure then return result
5. Return failure

Call CSP-BACKTRACKING({})

56
Map Coloring
{}

WA=red WA=green WA=blue

WA=red WA=red
NT=green NT=blue
NT
Q
WA
WA=red WA=red
NT=green NT=green SA NSW
Q=red Q=blue V

T 57
 Chapter Summary

 Backtracking is an algorithm design technique for

solving problems in which the number of choices
grows at least exponentially with their instant size.
 This approach makes it possible to solve many large
instances of NP-hard problems in an acceptable
amount of time.
 The technique constructs a pruned state space
tree.
 Backtracking constructs its state-space tree in the
depth-first search fashion in the majority of its
applications.
58