Backtracking

&
Branch and Bound

Backtracking
Suppose you have to make a series of
decisions, among various choices, where
You dont have enough information to know
what to choose
Each decision leads to a new set of choices
Some sequence of choices (possibly more
than one) may be a solution to your problem

Backtracking is a methodical way of

trying out various sequences of
decisions, until you find one that works

Backtracking Algorithm
Based on depth-first recursive search
Approach
1. Tests whether solution has been found
2. If found solution, return it
3. Else for each choice that can be made
a) Make that choice
b) Recur
c) If recursion returns a solution, return it

4. If no choices remain, return failure

Some times called search tree

3

Backtracking
Performs a depth-first traversal of a
tree
Continues until it reaches a node that
is non-viable or non-promising
Prunes the sub tree rooted at this
node and continues the depth-first
traversal of the tree

Backtracking Algorithm Example

Find path through maze
Start at beginning of maze
If at exit, return true
Else for each step from current location
Recursively find path
Return with first successful step
Return false if all steps fail

Backtracking
Backtracking is a technique used to solve problems
with a large search space, by systematically trying
and eliminating possibilities.
A standard example of backtracking would be
going through a maze.
At some point in a maze, you might have two options of
which direction to go:

Portion B

n
o
i
t
Portion A
c
n
Ju

Backtracking
strategy would

be to try going through Junc

Portion A of the
maze.
If

you get stuck before

you find your way out,
then you "backtrack"
to the junction.

At

this point in time

you know that Portion
A will NOT lead you
out of the maze,
so

you then start

searching in Portion B

n
o
i
t
Portion B

Portion A

One

Backtracking
Clearly, at a single
junction you could have
even more than 2 choices.
The backtracking strategy
says to try each choice,
one after the other,
if you ever get stuck,
"backtrack" to the
junction and try the next
choice.

If you try all choices and

never found a way out,
then there IS no solution
to the maze.

Ju

on
i
t
nc

C
B

Backtracking Eight Queens Problem

Find an arrangement of 8
queens on a single chess
board such that no two
queens are attacking one
another.
In chess, queens can move
all the way down any row,
column or diagonal (so long
as no pieces are in the way).
Due to the first two restrictions,
it's clear that each row and
column of the board will have
exactly one queen.

Backtracking Eight Queens Problem

The backtracking strategy is
as follows:
1) Place a queen on the first
available square in row 1.
2) Move onto the next row,
placing a queen on the first
available square there (that
doesn't conflict with the
previously placed queens).
3) Continue in this fashion until
either:
a) you have solved the problem, or
b) you get stuck.

When you get stuck, remove the

queens that got you there, until you
get to a row where there is another
valid square to try.

Q
Q

Q
Q

Continue

Animated Example:
http://www.hbmeyer.de/bac
ktrack/achtdamen/eight.ht
m#up

Backtracking Eight Queens

Problem
When we carry out backtracking,
an easy way to visualize what is
going on is a tree that shows all
the different possibilities that
have been tried.
On the board we will show a
visual representation of solving
the 4 Queens problem (placing 4
queens on a 4x4 board where no
two attack one another).

Backtracking Eight Queens

Problem
The neat thing about coding up
backtracking, is that it can be done
recursively, without having to do all
the bookkeeping at once.
Instead, the stack or recursive calls does
most of the bookkeeping
(ie, keeping track of which queens we've
placed, and which combinations we've
tried so far, etc.)

perm[] - stores a valid permutation of queens from index 0 to location-1.

location the column we are placing the next queen
usedList[] keeps track of the rows in which the queens have already been
placed.

void solveItRec(int perm[], int location, struct onesquare usedList[]) {

if (location == SIZE) {
printSol(perm);
}
for (int i=0; i<SIZE; i++) {

Found a solution to the problem, so

print it!

Loop through possible rows to place this

queen.
Only try this row if it hasnt been
if (usedList[i] == false) {
used
Check if this position conflicts with any
previous queens on the diagonal
if (!conflict(perm, location, i)) {
perm[location] = i;
usedList[i] = true;
solveItRec(perm, location+1, usedList);
usedList[i] = false;
}
}

}
}

1) mark the queen in this

row
2) mark the row as used
3) solve the next column
location recursively
4) un-mark the row as used,
so we can get ALL
possible valid solutions.

Backtracking 8 queens problem - Analysis

Another possible brute-force algorithm is generate the
permutations of the numbers 1 through 8 (of which there are 8!
= 40,320),
and uses the elements of each permutation as indices to place a queen
on each row.
Then it rejects those boards with diagonal attacking positions.

The backtracking algorithm, is a slight improvement on the

permutation method,
constructs the search tree by considering one row of the board at a
time, eliminating most non-solution board positions at a very early
stage in their construction.
Because it rejects row and diagonal attacks even on incomplete boards,
it examines only 15,720 possible queen placements.

A further improvement which examines only 5,508 possible

queen placements is to combine the permutation based method
with the early pruning method:
The permutations are generated depth-first, and the search space is
pruned if the partial permutation produces a diagonal attack

Sudoku and Backtracking

Another common puzzle that can be solved by
backtracking is a Sudoku puzzle.
The basic idea behind the solution is as follows:
1) Scan the board to look for an empty square that could take on
the fewest possible values based on the simple game
constraints.
2) If you find a square that can only be one possible value, fill it in
with that one value and continue the algorithm.
3) If no such square exists, place one of the possible numbers for
that square in the number and repeat the process.
4) If you ever get stuck, erase the last number placed and see if
there are other possible choices for that slot and try those
next.

Mazes and Backtracking

A final example of something that can be solved
using backtracking is a maze.
From your start point, you will iterate through each
possible starting move.
From there, you recursively move forward.
If you ever get stuck, the recursion takes you back to
where you were, and you try the next possible move.

In dealing with a maze, to make sure you don't try

too many possibilities,
one should mark which locations in the maze have
been visited already so that no location in the maze
gets visited twice.
(If a place has already been visited, there is no point in
trying to reach the end of the maze from there again.

Example: N-Queens Problem

Given an N x N sized
chess board
Objective: Place N queens
on the board so that no
queens are in danger

 One option would be to generate a

tree of every possible board layout
This would be an expensive way to
find a solution

Backtracking
Backtracking prunes
entire sub trees if
their root node is
not a viable solution
The algorithm will
backtrack up the
tree to search for
other possible
solutions

Efficiency of Backtracking
This given a significant advantage
over an exhaustive search of the tree
for the average problem
Worst case: Algorithm tries every
path, traversing the entire search
space as in an exhaustive search

Branch and Bound

Where backtracking uses a depthfirst search with pruning, the branch
and bound algorithm uses a breadthfirst search with pruning
Branch and bound uses a queue as
an auxiliary data structure

The Branch and Bound Algorithm

Starting by considering the root node
and applying a lower-bounding and
upper-bounding procedure to it
If the bounds match, then an optimal
solution has been found and the
algorithm is finished
If they do not match, then algorithm
runs on the child nodes

Example:
The Traveling Salesman Problem
Branch and bound can be used to solve the TSP
using a priority queue as an auxiliary data structure
An example is the problem with a directed graph
given by this adjacency matrix:

Traveling Salesman Problem

The problem starts at vertex 1
The initial bound for the minimum
tour is the sum of the minimum
outgoing edges from each vertex
Vertex 1
= 4
Vertex 2
= 7
Vertex 3
= 4
Vertex 4
= 2
Vertex 5
= 4

min (14, 4, 10, 20)

min (14, 7, 8, 7)
min (4, 5, 7, 16)
min (11, 7, 9, 2)
min (18, 7, 17, 4)

Bound

Traveling Salesman Problem

Next, the bound for the node for the
partial tour from 1 to 2 is calculated
using the formula:
Bound = Length from 1 to 2 + sum of min
outgoing edges for vertices 2 to 5 = 14
+ (7 + 4 + 2 + 4) = 31

Traveling Salesman Problem

The node is added to the priority
queue
The node with the lowest bound is
then removed
This calculation for the bound for the
node of the partial tours is repeated
on this node
The process ends when the priority
queue is empty

Traveling Salesman Problem

The final results
of this example
are in this tree:
The
accompanying
number for each
node is the order
it was removed in

Efficiency of Branch and

Bound
In many types of problems, branch
and bound is faster than branching,
due to the use of a breadth-first
search instead of a depth-first search
The worst case scenario is the same,
as it will still visit every node in the
tree