Backtracking is a form of recursion.

The usual scenario is that you are faced with a number of options, and you must choose one of these. After you make your choice you will get a new set of options; just what set of options you
get depends on what choice you made. This procedure is repeated over and over until you reach a final state. If you made a good sequence of choices, your final state is a goal state; if you
didn't, it isn't.

Conceptually, you start at the root of a tree; the tree probably has some good leaves and some bad leaves, though it may be that the leaves are all good or all bad. You want to get to a good
leaf. At each node, beginning with the root, you choose one of its children to move to, and you keep this up until you get to a leaf.

Suppose you get to a bad leaf. You can backtrack to continue the search for a good leaf by revoking your most recent choice, and trying out the next option in that set of options. If you run out of
options, revoke the choice that got you here, and try another choice at that node. If you end up at the root with no options left, there are no good leaves to be found.

This needs an example.

Starting at Root, your options are A and B. You choose A.

At A, your options are C and D. You choose C.
C is bad. Go back to A.
At A, you have already tried C, and it failed. Try D.
D is bad. Go back to A.
At A, you have no options left to try. Go back to Root.
At Root, you have already tried A. Try B.
At B, your options are E and F. Try E.
E is good. Congratulations!

In this example we drew a picture of a tree. The tree is an abstract model of the possible sequences of choices we could make. There is also a data structure called a tree, but usually we don't
have a data structure to tell us what choices we have. (If we do have an actual tree data structure, backtracking on it is called depth-first tree searching.)

The backtracking algorithm.

Here is the algorithm (in pseudocode) for doing backtracking from a given node n:

boolean solve(Node n) {
if n is a leaf node {
if the leaf is a goal node, return true
else return false
} else {
for each child c of n {
if solve(c) succeeds, return true
}
return false
}
}

Notice that the algorithm is expressed as a boolean function. This is essential to understanding the algorithm. If solve(n) is true, that means node n is part of a solution--that is, node n is one of
the nodes on a path from the root to some goal node. We say that n is solvable. If solve(n) is false, then there is no path that includes n to any goal node.

How does this work?

If any child of n is solvable, then n is solvable.

If no child of n is solvable, then n is not solvable.

Hence, to decide whether any non-leaf node n is solvable (part of a path to a goal node), all you have to do is test whether any child of n is solvable. This is done recursively, on each child of n.
In the above code, this is done by the lines

for each child c of n {

if solve(c) succeeds, return true
}
return false

Eventually the recursion will "bottom" out at a leaf node. If the leaf node is a goal node, it is solvable; if the leaf node is not a goal node, it is not solvable. This is our base case. In the above
code, this is done by the lines

if n is a leaf node {
if the leaf is a goal node, return true
else return false
}

The backtracking algorithm is simple but important. You should understand it thoroughly. Another way of stating it is as follows:
To search a tree:

If the tree consists of a single leaf, test whether it is a goal node,

Otherwise, search the subtrees until you find one containing a goal node, or until you have searched them all unsuccessfully.

Non-recursive backtracking, using a stack

Backtracking is a rather typical recursive algorithm, and any recursive algorithm can be rewritten as a stack algorithm. In fact, that is how your recursive algorithms are translated into machine or
assembly language.

boolean solve(Node n) {
put node n on the stack;
while the stack is not empty {
if the node at the top of the stack is a leaf {
if it is a goal node, return true
else pop it off the stack
}
else {
if the node at the top of the stack has untried children
push the next untried child onto the stack
else pop the node off the stack

}
return false
}

Starting from the root, the only nodes that can be pushed onto the stack are the children of the node currently on the top of the stack, and these are only pushed on one child at a time; hence,
the nodes on the stack at all times describe a valid path in the tree. Nodes are removed from the stack only when it is known that they have no goal nodes among their descendents. Therefore,
if the root node gets removed (making the stack empty), there must have been no goal nodes at all, and no solution to the problem.

When the stack algorithm terminates successfully, the nodes on the stack form (in reverse order) a path from the root to a goal node.
Similarly, when the recursive algorithm finds a goal node, the path information is embodied (in reverse order) in the sequence of recursive calls. Thus as the recursion unwinds, the path can be
recovered one node at a time, by (for instance) printing the node at the current level, or storing it in an array.

Here is the recursive backtracking algorithm, modified slightly to print (in reverse order) the nodes along the successful path:

boolean solve(Node n) {
if n is a leaf node {
if the leaf is a goal node {
print n
return true
}
else return false
} else {
for each child c of n {
if solve(c) succeeds {
print n
return true
}
}
return false
}
}

Keeping backtracking simple

All of these versions of the backtracking algorithm are pretty simple, but when applied to a real problem, they can get pretty cluttered up with details. Even determining whether the node is a leaf
can be complex: for example, if the path represents a series of moves in a chess endgame problem, the leaves are the checkmate and stalemate solutions.

To keep the program clean, therefore, tests like this should be buried in methods. In a chess game, for example, you could test whether a node is a leaf by writing a gameOver method (or you
could even call it isLeaf). This method would encapsulate all the ugly details of figuring out whether any possible moves remain.

Notice that the backtracking altorithms require us to keep track, for each node on the current path, which of its children have been tried already (so we don't have to try them again). In the above
code we made this look simple, by just saying for each child c of n. In reality, it may be difficult to figure out what the possible children are, and there may be no obvious way to step through
them. In chess, for example, a node can represent one arrangement of pieces on a chessboard, and each child of that node can represent the arrangement after some piece has made a legal
move. How do you find these children, and how do you keep track of which ones you've already examined?

The most straightforward way to keep track of which children of the node have been tried is as follows: Upon initial entry to the node (that is, when you first get there from above), make a list of
all its children. As you try each child, take it off the list. When the list is empty, there are no remaining untried children, and you can return "failure." This is a simple approach, but it may require
quite a lot of additional work.

There is an easier way to keep track of which children have been tried, if you can define an ordering on the children. If there is an ordering, and you know which child you just tried, you can
determine which child to try next.

For example, you might be able to number the children 1 through n, and try them in numerical order. Then, if you have just tried child k, you know that you have already tried children 1 through
k-1, and you have not yet tried children k+1 through n. Or, if you are trying to color a map with just four colors, you can always try red first, then yellow, then green, then blue. If child yellow fails,
you know to try child green next. If you are searching a maze, you can try choices in the order left, straight, right (or perhaps north, east, south, west).

It isn't always easy to find a simple way to order the children of a node. In the chess game example, you might number your pieces (or perhaps the squares of the board) and try them in
numerical order; but in addition each piece may also have several moves, and these must also be ordered.

You can probably find some way to order the children of a node. If the ordering scheme is simple enough, you should use it; but if it is too cumbersome, you are better off keeping a list of untried
children.

Example: TreeSearch

For starters, let's do the simplest possible example of backtracking, which is searching an actual tree. We will also use the simplest kind of tree, a binary tree.

A binary tree is a data structure composed of nodes. One node is designated as the root node. Each node can reference (point to) zero, one, or two other nodes, which are called its children.
The children are referred to as the left child and/or the right child. All nodes are reachable (by one or more steps) from the root node, and there are no cycles. For our purposes, although this is
not part of the definition of a binary tree, we will say that a node might or might not be a goal node, and will contain its name. The first example in this paper (which we repeat here) shows a
binary tree.

Here's a definition of the BinaryTree class:

public class BinaryTree {

BinaryTree leftChild = null;
BinaryTree rightChild = null;
boolean isGoalNode = false;
String name;

BinaryTree(String name, BinaryTree left, BinaryTree right, boolean isGoalNode) {

this.name = name;
leftChild = left;
rightChild = right;
this.isGoalNode = isGoalNode;
}
}

Next we will create a TreeSearch class, and in it we will define a method makeTree() which constructs the above binary tree.

static BinaryTree makeTree() {

BinaryTree root, a, b, c, d, e, f;
c = new BinaryTree("C", null, null, false);
d = new BinaryTree("D", null, null, false);
e = new BinaryTree("E", null, null, true);
f = new BinaryTree("F", null, null, false);
a = new BinaryTree("A", c, d, false);
b = new BinaryTree("B", e, f, false);
root = new BinaryTree("Root", a, b, false);
return root;
}

Here's a main program to create a binary tree and try to solve it:

public static void main(String args[]) {

BinaryTree tree = makeTree();
System.out.println(solvable(tree));
}
And finally, here's the recursive backtracking routine to "solve" the binary tree by finding a goal node.

static boolean solvable(BinaryTree node) {

/* 1 */ if (node == null) return false;
/* 2 */ if (node.isGoalNode) return true;
/* 3 */ if (solvable(node.leftChild)) return true;
/* 4 */ if (solvable(node.rightChild)) return true;
/* 5 */ return false;
}

Here's what the numbered lines are doing:

If we are given a null node, it's not solvable. This statement is so that we can call this method with the children of a node, without first checking whether those children actually exist.
If the node we are given is a goal node, return success.
See if the left child of node is solvable, and if so, conclude that node is solvable. We will only get to this line if node is non-null and is not a goal node, says to
Do the same thing for the right child.
Since neither child of node is solvable, node itself is not solvable.

This program runs correctly and produces the unenlightening result true.

Each time we ask for another node, we have to check if it is null. In the above we put that check as the first thing in solvable. An alternative would be to check first whether each child exists, and
recur only if they do. Here's that alternative version:

static boolean solvable(BinaryTree node) {

if (node.isGoalNode) return true;
if (node.leftChild != null && solvable(node.leftChild)) return true;
if (node.rightChild != null && solvable(node.rightChild)) return true;
return false;
}

I think the first version is simpler, but the second version is slightly more efficient.

What are the children?

One of the things that simplifies the above binary tree search is that, at each choice point, you can ignore all the previous choices. Previous choices don't give you any information about what
you should do next; as far as you know, both the left and the right child are possible solutions. In many problems, however, you may be able to eliminate children immediately, without recursion.

Consider, for example, the problem of four-coloring a map. It is a theorem of mathematics that any map on a plane, no matter how convoluted the countries are, can be colored with at most four
colors, so that no two countries that share a border are the same color.

To color a map, you choose a color for the first country, then a color for the second country, and so on, until all countries are colored. There are two ways to do this:

Method 1. Try each of the four possible colors, and recur. When you run out of countries, check whether you are at a goal node.
Method 2. Try only those colors that have not already been used for an adjacent country, and recur. If and when you run out of countries, you have successfully colored the map.

Let's apply each of these two methods to the problem of coloring a checkerboard. This should be easily solvable; after all, a checkerboard only needs two colors.

In both methods, the colors are represented by integers, from RED=1 to BLUE=4. We define the following helper methods. The helper method code isn't displayed here because it's not
important for understanding the method that does the backtracking.

boolean mapIsOK()
Used by method 1 to check (at a leaf node) whether the entire map is colored correctly.

boolean okToColor(int row, int column, int color)

Used by method 2 to check, at every node, whether there is an adjacent node already colored with the given color.

int[] nextRowAndColumn(int row, int column)

Used by both methods to find the next "country" (actually, the row and column of the next square on the checkerboard).

Here's the code for method 1:

boolean explore1(int row, int column, int color) {

if (row >= NUM_ROWS) return mapIsOK();
map[row][column] = color;
for (int nextColor = RED; nextColor <= BLUE; nextColor++) {
int[] next = nextRowAndColumn(row, column);
if (explore1(next[0], next[1], nextColor)) return true;
}
return false;
}

And here's the code for method 2:

boolean explore2(int row, int column, int color) {

if (row >= NUM_ROWS) return true;
if (okToColor(row, column, color)) {
map[row][column] = color;
for (int nextColor = RED; nextColor <= BLUE; nextColor++) {
int[] next = nextRowAndColumn(row, column);
if (explore2(next[0], next[1], nextColor)) return true;
}
}
return false;
}

Those appear pretty similar, and you might think they are equally good. However, the timing information suggests otherwise:

2 by 3 map 3 by 3 map 3 by 4 map

Method 1: 60 ms. 940 ms. 60530 ms. (1 minute)
Method 2: 0ms. 0 ms. 0 ms

The zeros in the above table indicate times too short to measure (less than 1 millisecond). Why this huge difference? Either of these methods could have exponential growth. Eliminating a node
automatically eliminates all of its descendents, and this will often prevent exponential growth. Conversely, by waiting to check until a leaf node is reached, exponential growth is practically
guaranteed. If there is any way to eliminate children (reduce the set of choices), do so!

Debugging techniques

Often our first try at a program doesn't work, and we need to debug it. Debuggers are helpful, but sometimes we need to fall back on inserting print statements. There are some simple tricks to
making effective use of print statements. These tricks can be applied to any program, but are especially useful when you are trying to debug recursive routines.

Trick #1: Indent when you print method entries and exits. Often, the best debugging technique is to print every method call and return (or at least the most important ones). You probably want to
print, for each method, what parameters it came in with, and what value it leaves with. However, if you just print a long list of these, it's hard to match up method exits with their corresponding
entries. Indenting to show the level of nesting can help.

Trick #2: Use specialized print methods for debugging. Don't clutter up your actual code more than you must. Also, remember that code inserted for debugging purposes can itself contain bugs,
or (in the worst case) can affect the results, so be very careful with it.

Here's our debugging code. For this trivial program, there's almost more debugging code than actual code, but in larger programs the proportions will be better.
 static String indent = ""; if (node.isGoalNode) return yes(node);
if (solvable(node.leftChild)) return yes(node);
static String name(BinaryTree node) { if (solvable(node.rightChild)) return yes(node);
if (node == null) return null; return no(node);
else return node.name; }
}
And we get these results:
static void enter(BinaryTree node) {
System.out.println(indent + "Entering solvable(" + name(node) + ")"); Entering solvable(Root)
indent = indent + "| "; | Entering solvable(A)
} | | Entering solvable(C)
| | | Entering solvable(null)
static boolean yes(BinaryTree node) { | | | solvable(null) returns false
indent = indent.substring(3); | | | Entering solvable(null)
System.out.println(indent + "solvable(" + name(node) + ") returns true"); | | | solvable(null) returns false
return true; | | solvable(C) returns false
} | | Entering solvable(D)
| | | Entering solvable(null)
static boolean no(BinaryTree node) { | | | solvable(null) returns false
indent = indent.substring(3); | | | Entering solvable(null)
System.out.println(indent + "solvable(" + name(node) + ") returns false"); | | | solvable(null) returns false
return false; | | solvable(D) returns false
} | solvable(A) returns false
| Entering solvable(B)
To use this code, we modify solvable as follows: | | Entering solvable(E)
| | solvable(E) returns true
static boolean solvable(BinaryTree node) { | solvable(B) returns true
enter(node); solvable(Root) returns true
if (node == null) return no(node); true

Trick #3: Never discard your debugging statements. Writing debugging statements is programming, too. Often it's as much work to debug the debugging statements as it is to debug the actual
program. Once your program is working, why throw this code away?

Obviously, you don't want to print out all this debugging information from a program you are ready to submit (or to turn over to your manager). You could comment out your debugging calls, but
that can be a lot of work. What's more, in the above example, you would have to replace every return(yes(node)) with return(true), and every return(no(node)) with return false. With all these
changes, you might introduce new bugs into your program.

The simple solution is to make your debugging statements conditional. For example,

static final boolean debugging = false; System.out.println(indent + "solvable(" + name(node) + ") returns true");
}
static void enter(BinaryTree node) { return true;
if (debugging) { }
System.out.println(indent + "Entering solvable(" + name(node) + ")");
indent = indent + "| "; static boolean no(BinaryTree node) {
} if (debugging) {
} indent = indent.substring(3);
System.out.println(indent + "solvable(" + name(node) + ") returns false");
static boolean yes(BinaryTree node) { }
if (debugging) { return false;
indent = indent.substring(3); }

In industry, actual programs often have multiple flags to control different aspects of debugging. Don't worry too much about making your code larger; modern compilers will notice that since the
variable debugging is final, it can never be true, and the controlled code will be discarded.

Trick #4: Create an Exception. If an Exception is thrown, you can get information about just where it happened by sending it the message printStackTrace(PrintStream). Since an Exception is an
object like any other, you can create and throw your own Exceptions. However, Java programmers don't always realize that you can create an Exception without throwing it. For example, the
following code

new Exception("Checkpoint Charlie").printStackTrace(System.out);

will print out a message something like this, and the program will then continue normally. That is, the above code just acts like a print statement.

java.lang.Exception: Checkpoint Charlie

at TreeSearch.solvable(TreeSearch.java:53)
at TreeSearch.solvable(TreeSearch.java:57)
at TreeSearch.main(TreeSearch.java:72)
at __SHELL38.run(__SHELL38.java:16)
at bluej.runtime.ExecServer.suspendExecution(Unknown Source)

Example: Cindy's Puzzle

I call the following puzzle "Cindy's puzzle" for historical reasons. You have some number n of black marbles and the same number of white marbles, and you have a playing board which consists
simply of a line of 2n+1 spaces to put the marbles in. Start with the black marbles all at one end (say, the left), the white marbles all at the other end, and a free space in between.
The goal is to reverse the positions of the marbles:

The black marbles can only move to the right, and the white marbles can only move to the left (no backing up). At each move, a marble can either:

Move one space ahead, if that space is clear, or

Jump ahead over exactly one marble of the opposite color, if the space just beyond that marble is clear.

For example, you could make the following sequence of moves:

Starting position:

Black moves ahead:

White jumps:

Black moves ahead:

Black jumps:

White moves ahead:

Stuck!

Now to the program. The main program will initialize the board, and call a recursive backtracking routine to attempt to solve the puzzle. The backtracking routine will either succeed and print out
a winning path, or it will fail, and the main program will have to print out the bad news.

The backtracking method is named solvable and returns a boolean. In solvable we shall need to check whether we are at a leaf, which in this case means a position from which no further moves
are possible. This isn't so easy.

Each possible move will result in a new board position, and these new board positions are the children of the current board position. Hence to find the children of a node (that is, of a board
position), we need only find the possible moves from that node. Remember that it is also highly desirable to find an ordering on these possible moves.

Here it is time to stop and take thought. To make progress, we must analyze the game to some extent. Probably a number of approaches would work, and what follows is based on the way I
worked it out. If you were to program this puzzle, you might find a different but equally valid approach.

First, notice that if a marble has a move, that move is unique: if it can move ahead one square, then it cannot jump. If it can jump, it cannot move ahead one square. This suggests that, to find
the possible moves, we might assign numbers to the marbles, and check each marble in turn. When we have looked at all the marbles, we have looked at all the possible moves. This would
require having a table to keep track of where each marble is, or else somehow "marking" each marble with its number and searching the board each time to find the marble we want. Neither
alternative is very attractive.

Next, notice that for a given board position, each marble occupies a unique space. Hence, instead of talking about moving a particular marble, we can talk about moving the marble in a
particular space. If a move is possible from a given space, then that must be the only move possible from that space, because if the marble in that space has a move, it is unique. There is a
slight complication because not every space contains a marble, but at least the spaces (unlike the marbles) stay in one place.

Now we have a simpler ordering of moves to use in our program. Just check, in order, the 2n+1 spaces of the board. For each space, either zero or one moves is possible. With this
understanding, we can write a boolean method canMove(int[] board, int position) which determines whether a move is possible from the given position:

If the position is empty, no move is possible;

If the position contains a black marble, the method checks for a move or jump to the right;
If the position contains a white marble, the method checks for a move or jump to the left.

We write another method int[] makeMove(int[] oldBoard, int position) that will take a board and a position, make a move from that position, and return as its value a new board. (We could write
this somewhat more efficiently by changing the old board, rather than creating a new one, but here we are more concerned with simplicity.) In technical jargon, makeMove is "applicative" rather
than "mutative."

With these methods, our central backtracking method can be written as follows:

boolean solvable(int[] board) {

if (puzzleSolved(board)) {
return true;
}
for (int position = 0; position < BOARD_SIZE; position++) {
if (canMove(board, position)) {
int[] newBoard = makeMove(board, position);
if (solvable(newBoard)) {
printBoard(newBoard);
return true;
}
}
}
return false;
}

Along with canMove and makeMove, we are using methods puzzleSolved and printBoard with meanings that should be obvious.

Here is some output from the program:

16. WHITE WHITE WHITE _____ BLACK BLACK BLACK

15. WHITE WHITE WHITE BLACK _____ BLACK BLACK
14. WHITE WHITE _____ BLACK WHITE BLACK BLACK
13. WHITE _____ WHITE BLACK WHITE BLACK BLACK
12. WHITE BLACK WHITE _____ WHITE BLACK BLACK
11. WHITE BLACK WHITE BLACK WHITE _____ BLACK
10. WHITE BLACK WHITE BLACK WHITE BLACK _____
9. WHITE BLACK WHITE BLACK _____ BLACK WHITE
8. WHITE BLACK _____ BLACK WHITE BLACK WHITE
7. _____ BLACK WHITE BLACK WHITE BLACK WHITE
6. BLACK _____ WHITE BLACK WHITE BLACK WHITE
5. BLACK BLACK WHITE _____ WHITE BLACK WHITE
4. BLACK BLACK WHITE BLACK WHITE _____ WHITE
3. BLACK BLACK WHITE BLACK _____ WHITE WHITE
2. BLACK BLACK _____ BLACK WHITE WHITE WHITE
1. BLACK BLACK BLACK _____ WHITE WHITE WHITE

Notice that the solution is given in reverse order: BLACK starts out on the left and WHITE on the right, as in the last line. I've added line numbers to the actual output in order to emphasize this
point. Backtracking always produces its results (sequence of choices) in reverse order; it is up to you, the programmer, to reverse the results again to get them in the correct order.
=========================================
Backtracking requires JavaScript (MSIE recommended)

A mouseclick on any card belonging to the array on the right side ("pool") moves that card into the next empty field of the left array ("game-area").

A single mouseclick on any card in the game-area causes a quarter turn of the card counterclockwise.

A double mouseclick on any card in the game-area moves the card back to the original place in the pool.

Aim: all cards should be placed in the game-area such that all neighbours fit, i.e. 12 unicolored letters "A" should appear.

After activation of the option "automatically" a backtracking-algorithm will run down for search of a solution. It may be interrupted by activation of the option "by mouseclick". When a solution is
found, the algorithm will end and in that case a search for another solution may be continued with the button "new solution". The delay (in milliseconds) regulates the frequency of the image
output.
================
Backtracking
From Wikipedia, the free encyclopedia
Jump to: navigation, search

Backtracking is a general algorithm for finding all (or some) solutions to some computational problem, that incrementally builds candidates to the solutions, and abandons each partial candidate
c ("backtracks") as soon as it determines that c cannot possibly be completed to a valid solution.[1][2][3]

The classic textbook example of the use of backtracking is the eight queens puzzle, that asks for all arrangements of eight queens on a standard chessboard so that no queen attacks any other.
In the common backtracking approach, the partial candidates are arrangements of k queens in the first k rows of the board, all in different rows and columns. Any partial solution that contains
two mutually attacking queens can be abandoned, since it cannot possibly be completed to a valid solution.

Backtracking can be applied only for problems which admit the concept of a "partial candidate solution" and a relatively quick test of whether it can possibly be completed to a valid solution. It is
useless, for example, for locating a given value in an unordered table. When it is applicable, however, backtracking is often much faster than brute force enumeration of all complete candidates,
since it can eliminate a large number of candidates with a single test.

Backtracking is an important tool for solving constraint satisfaction problems, such as crosswords, verbal arithmetic, Sudoku, and many other puzzles. It is often the most convenient (if not the
most efficient[citation needed]) technique for parsing, for the knapsack problem and other combinatorial optimization problems. It is also the basis of the so-called logic programming languages
such as Icon, Planner and Prolog. Backtracking is also utilized in the (diff) difference engine for the MediaWiki software.

Backtracking depends on user-given "black box procedures" that define the problem to be solved, the nature of the partial candidates, and how they are extended into complete candidates. It is
therefore a metaheuristic rather than a specific algorithm – although, unlike many other meta-heuristics, it is guaranteed to find all solutions to a finite problem in a bounded amount of time.

The term "backtrack" was coined by American mathematician D. H. Lehmer in the 1950s.[4] The pioneer string-processing language SNOBOL (1962) may have been the first to provide a built-in
general backtracking facility.
Contents

1 Description of the method

1.1 Pseudocode
1.2 Usage considerations
1.3 Early stopping variants
2 Examples
2.1 Constraint satisfaction
3 See also
4 Notes
5 References
6 External links

[edit] Description of the method

The backtracking algorithm enumerates a set of partial candidates that, in principle, could be completed in various ways to give all the possible solutions to the given problem. The completion is
done incrementally, by a sequence of candidate extension steps.

Conceptually, the partial candidates are the nodes of a tree structure, the potential search tree. Each partial candidate is the parent of the candidates that differ from it by a single extension step;
the leaves of the tree are the partial candidates that cannot be extended any further.

The backtracking algorithm traverses this search tree recursively, from the root down, in depth-first order. At each node c, the algorithm checks whether c can be completed to a valid solution. If
it cannot, the whole sub-tree rooted at c is skipped (pruned). Otherwise, the algorithm (1) checks whether c itself is a valid solution, and if so reports it to the user; and (2) recursively enumerates
all sub-trees of c. The two tests and the children of each node are defined by user-given procedures.

Therefore, the actual search tree that is traversed by the algorithm is only a part of the potential tree. The total cost of the algorithm is the number of nodes of the actual tree times the cost of
obtaining and processing each node. This fact should be considered when choosing the potential search tree and implementing the pruning test.
[edit] Pseudocode

In order to apply backtracking to a specific class of problems, one must provide the data P for the particular instance of the problem that is to be solved, and six procedural parameters, root,
reject, accept, first, next, and output. These procedures should take the instance data P as a parameter and should do the following:

root(P): return the partial candidate at the root of the search tree.
reject(P,c): return true only if the partial candidate c is not worth completing.
accept(P,c): return true if c is a solution of P, and false otherwise.
first(P,c): generate the first extension of candidate c.
next(P,s): generate the next alternative extension of a candidate, after the extension s.
output(P,c): use the solution c of P, as appropriate to the application.

The backtracking algorithm reduces then to the call bt(root(P)), where bt is the following recursive procedure:

procedure bt(c)
if reject(P,c) then return
if accept(P,c) then output(P,c)
s ← first(P,c)
while s ≠ Λ do
bt(s)
s ← next(P,s)

[edit] Usage considerations

The reject procedure should be a boolean-valued function that returns true only if it is certain that no possible extension of c is a valid solution for P. If the procedure cannot reach a definite
conclusion, it should return false. An incorrect true result may cause the bt procedure to miss some valid solutions. The procedure may assume that reject(P,t) returned false for every ancestor t
of c in the search tree.

On the other hand, the efficiency of the backtracking algorithm depends on reject returning true for candidates that are as close to the root as possible. If reject always returns false, the
algorithm will still find all solutions, but it will be equivalent to a brute-force search.

The accept procedure should return true if c is a complete and valid solution for the problem instance P, and false otherwise. It may assume that the partial candidate c and all its ancestors in
the tree have passed the reject test.

Note that the general pseudo-code above does not assume that the valid solutions are always leaves of the potential search tree. In other words, it admits the possibility that a valid solution for
P can be further extended to yield other valid solutions.

The first and next procedures are used by the backtracking algorithm to enumerate the children of a node c of the tree, that is, the candidates that differ from c by a single extension step. The
call first(P,c) should yield the first child of c, in some order; and the call next(P,s) should return the next sibling of node s, in that order. Both functions should return a distinctive "null" candidate,
denoted here by 'Λ', if the requested child does not exist.

Together, the root, first, and next functions define the set of partial candidates and the potential search tree. They should be chosen so that every solution of P occurs somewhere in the tree, and
no partial candidate occurs more than once. Moreover, they should admit an efficient and effective reject predicate.
[edit] Early stopping variants

The pseudo-code above will call output for all candidates that are a solution to the given instance P. The algorithm is easily modified to stop after finding the first solution, or a specified number
of solutions; or after testing a specified number of partial candidates, or after spending a given amount of CPU time.
[edit] Examples

Typical examples are

Puzzles such as eight queens puzzle, crosswords, verbal arithmetic, Sudoku, Peg Solitaire
combinatorial optimization problems such as parsing and the knapsack problem
logic programming languages such as Icon, Planner and Prolog, which use backtracking internally to generate answers.
Backtracking is also utilized in the "diff" (version comparing) engine for the MediaWiki software.

Below is an example for the constraint satisfaction problem:

[edit] Constraint satisfaction

The general constraint satisfaction problem consists in finding a list of integers x = (x[1],x[2], ..., x[n]), each in some range {1, 2, ..., m}, that satisfies some arbitrary constraint (boolean function)
F.

For this class of problems, the instance data P would be the integers m and n, and the predicate F. In a typical backtracking solution to this problem, one could define a partial candidate as a list
of integers c = (c[1],c[2], ... c[k]), for any k between 0 and n, that are to be assigned to the first k variables x[1],x[2], ..., x[k]). The root candidate would then be the empty list (). The first and next
procedures would then be

function first(P,c)
k ← length(c)
if k = n
then return Λ
else return (c[1], c[2], ..., c[k], 1)

function next(P,s)
k ← length(s)
if s[k] = m
then return Λ
else return (s[1], s[2], ..., s[k-1], 1 + s[k])

Here "length(c)" is the number of elements in the list c.

The call reject(P,c) should return true if the constraint F cannot be satisfied by any list of n integers that begins with the k elements of c. For backtracking to be effective, there must be a way to
detect this situation, at least for some candidates c, without enumerating all those mn-k n-tuples.

For example, if F is the conjunction of several boolean predicates, F = F[1] \wedge F[2] \wedge \cdots \wedge F[p], and each F[i] depends only on a small subset of the variables x[1], ..., x[n],
then the reject procedure could simply check the terms F[i] that depend only on variables x[1], ..., x[k], and return true if any of those terms returns false. In fact, reject needs only check those
terms that do depend on x[k], since the terms that depend only on x[1], ..., x[k-1] will have been tested further up in the search tree.

Assuming that reject is implemented as above, then accept(P,c) needs only check whether c is complete, that is, whether it has n elements.

It is generally better to order the list of variables so that it begins with the most critical ones (i.e. the ones with fewest value options, or which have a greater impact on subsequent choices).

One could also allow the next function to choose which variable should be assigned when extending a partial candidate, based on the values of the variables already assigned by it. Further
improvements can be obtained by the technique of constraint propagation.

In addition to retaining minimal recovery values used in backing up, backtracking implementations commonly keep a variable trail, to record value change history. An efficient implementation will
avoid creating a variable trail entry between two successive changes when there is no choice point, as the backtracking will erase all of the changes as a single operation.

An alternative to the variable trail is to keep a timestamp of when the last change was made to the variable. The timestamp is compared to the timestamp of a choice point. If the choice point
has an associated time later than that of the variable, it is unnecessary to revert the variable when the choice point is backtracked, as it was changed before the choice point occurred.

==================================
Backtracking Algorithms
19.1 Solution Spaces

Backtracking is a refinement of the brute force approach, which systematically searches for a solution to a problem among all available options. It does so by assuming that the solutions are
represented by vectors (v1, ..., vm) of values and by traversing, in a depth first manner, the domains of the vectors until the solutions are found.

When invoked, the algorithm starts with an empty vector. At each stage it extends the partial vector with a new value. Upon reaching a partial vector (v1, ..., vi) which can’t represent a partial
solution, the algorithm backtracks by removing the trailing value from the vector, and then proceeds by trying to extend the vector with alternative values.

ALGORITHM try(v1,...,vi)
IF (v1,...,vi) is a solution THEN RETURN (v1,...,vi)
FOR each v DO
IF (v1,...,vi,v) is acceptable vector THEN
sol = try(v1,...,vi,v)
IF sol != () THEN RETURN sol
END
END
RETURN ()

If Si is the domain of vi, then S1 × ... × Sm is the solution space of the problem. The validity criteria used in checking for acceptable vectors determines what portion of that space needs to be
searched, and so it also determines the resources required by the algorithm.

The traversal of the solution space can be represented by a depth-first traversal of a tree. The tree itself is rarely entirely stored by the algorithm in discourse; instead just a path toward a root is
stored, to enable the backtracking.

• |------------------| v1 : |-•||--------S-------•|-|- |-----|-----------1-----|------| v2 :|•----|•|S||--•|- |•----|•|S||--•|- |----|||||2-----| |----|||||2-----| ... ...... ... ... ... ... ...... ... ... ...
19.2 Traveling Salesperson

The problem assumes a set of n cities, and a salesperson which needs to visit each city exactly once and return to the base city at the end. The solution should provide a route of minimal
length.

The route (a, b, d, c) is the shortest one for the following one, and its length is 51.

do-----12-|co | ||| ||| | | ||| | 15|20 9|18 ao| -----|o 10 b

The traveling salesperson problem is an NP-hard problem, and so no polynomial time algorithm is available for it. Given an instance G = (V, E) the backtracking algorithm may search for a
vector of cities (v1, ..., v|V |) which represents the best route.

The validity criteria may just check for number of cities in of the routes, pruning out routes longer than |V |. In such a case, the algorithm needs to investigate |V ||V | vectors from the solution
space.

bo---- | ---- | ---ao |----- co-

• |---------------------------------| |----a------| |----b------| |-----c-----| |-a-| |b--| |c-| |-a-| |b--| |c--||-a-| |-b-| |c--| a b ca b c a b c a b c a b ca b c a b c a b ca b c

On the other hand, the validity criteria may check for repetition of cities, in which case the number of vectors reduces to |V |!.

•| |-------| |a| b-| c-| b c ac|a|b| c b ca b a

19.3 The Queens Problem

Consider a n by n chess board, and the problem of placing n queens on the board without the queens threatening one another.

|-|-o|-|-o|-|-o|-|--| |-|--|-|--|-|--|-|--| |-|--|o|-o|o|--|-|--| |o|-o|o|-o|o|-o|o|o-| |-|--|o|-o|o|--|-|--| |-|-o|-|-o|-|-o|-|--| |o|--|-|-o|-|--|o|--| | | | | o| | | |o |-|--|-|-o|-|--|-|--| --------------------

The solution space is {1, 2, 3, ..., n}n. The backtracking algorithm may record the columns where the different queens are positioned. Trying all vectors (p1, ..., pn) implies nn cases. Noticing that
all the queens must reside in different columns reduces the number of cases to n!.

For the latter case, the root of the traversal tree has degree n, the children have degree n - 1, the grand children degree n - 2, and so forth.

||-|| ||-|| |-------------------| 1 2 3 •|-|| ||•|| ||-•| ||-|| ||-|| ||-|| |----| |----| |----| |1,2||1,3||2,1||2,3||3,1|-3,2-| |••|||•|•||••|||-••||•|•|-|••-| --|----|----|-- -|----|----|--- 1,2,3 1,3,2 2,1,3 2,3,1 3,1,2 3,2,1 |•||||•||||-•|||-•|||-|•|-||•-|
|-••||-••||•|•||•|•||••||-••|-| --------------- ---------------

Checking for threatening positions along the way my further reduce the number of visited configurations.

||-|| ||-|| |-------------------| 1 2 3 •|-|| ||•|| ||-•| ||-|| ||-|| ||-|| |----| |----| |----| |1,2||1,3||2,1||2,3||3,1|-3,2-| |••|||•|•||••|||-••||•|•|-|••-| -------|------- ------|-------- 1,3,2 3,1,2 |•||| |-|•|- |-••| |••||- ----- -----
19.4 Convex Hull (Graham’s Scan)

The problem asks to construct the shortest polygon which encloses a given set of points on a plan. Intuitively, the polygon can be determined by placing a rubber around the set.

• |--- | ---- |• --- | --- •|-----• ---- ----------•

Determine an extreme point with the largest x coordinate. Sort the points in order of increasing angles, relatively to the extreme point.

• b --- ---- •-- ---- c -------- •----d-•-------- e ---------• a

Traverse the points in the sorted order, adding them to the partial solution upon making a turn of less than 180 degrees, and backtracking when making a larger turn.

b •|- |---- |c ---- • ---- •d --- •e ---a • b •|-- ||||- |c ----- •---- ---- e -•d ---- • -•a b •|-- | ---- •c |||- --| d|--- e-----• - ---- • ||||- -•a b •|-- | ---- •c- ---- --- d --- e • --- • -•a •b |--- | --- •c ---- d ---- •e •
---- •a •b |--- ||||--- •c - --- ||||- d --- •e • ---- •a •b- |--- | ---- •c ---- •d --- •e ---a • •b- ---- c ---- • --- •d ---- •e ---a • b •-- ||-|-- | c ---- |• ---- e •d --- • ---a • b •--- | --- | c --- |• d---- e|| • ----
•---------------•a

The algorithm requires O(n log n) time for sorting the points, and O(n) time to select an appropriate subset.
19.5 Generating Permutations

A permutation can be obtained by selecting an element in the given set and recursively permuting the remaining elements.

{ ai,P(a1,...,ai-1,ai+1,...,aN) if N > 1 P(a1,...,aN) = aN if N = 1

--|--|--|-| |a|b-|c-d-| a|------------b------------c-------------d --|--|--|-| ---|-|--|--| ---|--|-|--| --|--|--|-| |-|b-|c-d-| |a-|-|c-|d-| |a-|b-|-|d-| |a|b-|c-|-|

At each stage of the permutation process, the given set of elements consists of two parts: a subset of values that already have been processed, and a subset that still needs to be processed.
This logical seperation can be physically realized by exchanging, in the i’th step, the i’th value with the value being chosen at that stage. That approaches leaves the first subset in the first i
locations of the outcome.

--|--|--|-| |a|b-|c-d-| --|--|------------|--------------------------|--|-| a||b |c d | |b a |c |d | |c||b |a|d | |d|b |c |a| -----|--------------------------|---- ----------- --|--|--|-| ---|-|--|--| ---|--|-|--| b-|a-|c-d-| |b-|c|a-|d-| |b-|d-|c|a-|
---|--|------------|--|-| |b-|c-a-|d-| b-|c-|d-|a| | b-|c-|d-|a| |-|--|--|-|

permute(i)
if i == N output A[N]
else
for j = i to N do
swap(A[i], A[j])
permute(i+1)
swap(A[i], A[j])
=========================================