Search

www.math.sc.chula.ac.th/~jaruloj/C681/6_BB.ppt
 Outline
• Problem space/ State space
• Exhaustive search
• Depth-first search
• Breadth-first search
• Backtracking
• Branch-and-bound

Jaruloj Chongstitvatana Search 2

Problem Space
or State Space
 Problem Space

• General problem statement

• Given a problem instance P,
find answer A=(a1, a2, …, an) such that
the criteria C(A, P) is satisfied.

• Problem space of P is the set of all

possible answers A.

Jaruloj Chongstitvatana Search 4

 Example: Shortest path
• Given a graph G=(V,E), and nodes u and v, find
the shortest path between u and v.

• General problem statement

• Given a graph G and nodes u and v,
find the path (u, n1, n2, …, nk, v), and
(u, n1, n2,…, nk, v) is the shortest path between u and v.

• Problem space
• Set of all possible path between u and v.
• {(u, n1, n2, …, nk, v)| ni is in V, for 1≤i≤k}.

Jaruloj Chongstitvatana Search 5

 Example: 0/1 Knapsack
• Given a set S of n objects, where the object i has
value vi and weight wi , and a knapsack with
weight capacity C, find the maximum of value of
objects in S which can be put in the knapsack.

• General problem statement

• Given vi and wi , for 1 ≤ i ≤ n ,
find the set K such that for each i in K, 1 ≤ i ≤ n,
Σ vi is maximum while Σ wi ≤ C.
i∈K i∈K

• Problem space
• Any subset of S.

Jaruloj Chongstitvatana Search 6

 Example: n Queens
• Given an nxn board, find the n squares in
the board to put n queens so that no pair
can attack.

• General problem statement

• Find (p1, p2, …, pn) where pi = (xi, yi) is a
square on the board, where there is no pair
(xi, yi) and (xj, yj) such that xi = xj or yi = yj.

• Problem space
• A set of any n positions on the board.

Jaruloj Chongstitvatana Search 7

Exhaustive Search
 Exhaustive Search
• Generate every possible answer
• Test each answer with the constraint to find the
correct answer
• Inefficient because the number of answers in the
problem space can be exponential.
• Examples:
• Shortest path
• n! paths to be considered, where n is the number of nodes.
• 0/1 Knapsack
• 2n selections, where n is the number of objects.
• n Queens
• n2!/n! (n2-n)!

Jaruloj Chongstitvatana Search 9

 State-Space Tree
• Let (a1, a2, …, an) be a possible answer.
• Suppose ai is either 0 or 1, for 1 ≤ i ≤ nใ

(?, …, ?)

(0, ?, …, ?) (1, ?, …, ?)

(0, 0, ?, …, ?) (0, 1, ?, …, ?) (1, 0,?, …, ?) (1, 1, ?, …, ?)

(0,0,0, …, ?) (0,0,1, …, ?) (0,1,0, …, ?) (0,1,1, …, ?)

Jaruloj Chongstitvatana Search 10

 State-Space Tree: Shortest Path

8 (1)
3 5
2
5 2 (1,2) (1,3)
1 -6 1
-1
2 (1,2,3) (1,2,4) (1,3,4) (1,3,5)
2 4
(1,2,3,4) (1,2,3,5) (1,2,4,5) (1,3,4,5)

(1,2,3,4,5)

Jaruloj Chongstitvatana Search 11

 Generating Possible Paths
• Let {1,2,3, …, n} be a set of nodes and E[i][j] is
the weight of the edge between node i and j.

path(p)
last = last node in the path p
for next = 1 to n
do np = p
if next is not in np and E[last][next] != 0
then np = np || next
path(np)
else return

Jaruloj Chongstitvatana Search 12

 State-Space Tree : 0/1 Knapsack
• Given a set of objects o1, …, o5.
{}

{1} {2} {3} {4} {5}

{1,2} {1,3} {1,4} {1,5}

{1,2,3} {1,2,4} {1,2,5} {1,3,4} {1,3,5} {1,4,5}

{1,2,3,4} {1,2,3,5} {1,2,4,5} {1,3,4, 5}

{1,2,3,4,5}
Jaruloj Chongstitvatana Search 13
 State-Space Tree : n Queen
Q Q
Q Q

Q Q

Q
Q
Q

Q
Q
Q

Q
Q
Q

Q
Q
Q
Q
Q
Q
Q Q
Q
Q
Q
Q Q
Jaruloj Chongstitvatana Search 14
 Depth-first Search
• Traverse the tree from root until a leaf is
reached.
• Then, traverse back up to visited the next
unvisited node.

depthFirst(v)
visited[v] = 1
for each node k adjacent to v
do if not visited[k]
then depthFirst(k)

Jaruloj Chongstitvatana Search 15

 0/1 Knapsack: Depth-first Search
Global: maxV=0 maxSack={}

DFknapsack(sack, unchosen)
for each object p in unchosen
do unchosen=unchosen-{p}
sack=sack U {p}
val=evaluate(sack)
if unchosen is empty
► A leaf is reached.
then maxV=max(maxV, val)
if maxV=val then maxSack=sack
return
else DFknapsack(sack, unchosen)
return

Jaruloj Chongstitvatana Search 16

 Breadth-first Search
• Traverse the tree from root until the nodes of the
same depth are all visited.
• Then, visited the node in the next level.

breadthFirst(v)
Q = empty queue
enqueue(Q, v) visited[v] = 1
while not empty (Q)
do u = dequeue(Q)
for each node k adjacent to u
do if not visited[k]
then visited[k] = true
enqueue(Q, k)
Jaruloj Chongstitvatana Search 17
 0/1 Knapsack: Breadth-first Search
BFknapsack
Q = empty queue maxV=0
sack = { }
unchosen = set of all objects
enqueue(Q, <sack, unchosen>)
while not empty (Q)
do <sack, unchosen> = dequeue(Q)
if unchosen is not empty
then for each object p in unchosen
do enqueue(Q,<sackU{p}, unchosen-{p}>)
else maxV = max(maxV, evaluate(sack))
if maxV=evaluate(sack) then maxSack = sack

Jaruloj Chongstitvatana Search 18

Backtracking
 Backtracking
• Reduce the search by cutting down some
branches in the tree

Jaruloj Chongstitvatana Search 20

 0/1 Knapsack: Backtracking
Sack {}
Node

Current weight, current value 0,0

{1} {2} {3} {4}

2,5 1,4 3,8 2,7

{1,2} {1,3} {1,4} {2,3} {2,4} {3,4}

3,9 5,13 4,12 4,12 3,11 5,15
object weight value
{1,2,3} {1,2,4} {2,3,4}
1 2 5
6, 17 5, 16 6,19
2 1 4
3 3 8
4 2 7
Capacity = 5
Jaruloj Chongstitvatana Search 21
 0/1 Knapsack: Backtracking
BTknapsack(sack, unchosen)
for each object p in unchosen
do unchosen=unchosen-{p}
if p can be put in sack then sack = sack U {p}
► Backtracking occurs when p cannot be put in sack.
val=evaluate(sack)
if unchosen is empty
► A leaf is reached.
then maxV=max(maxV, val)
maxSack=sack
return
else BTknapsack(sack, unchosen)
return

Jaruloj Chongstitvatana Search 22

Branch-and-Bound
 Branch-and-bound
• Use for optimization problems
• An optimal solution is a feasible solution
with the optimal value of the objective
function.
• Search in state-space trees can be pruned
by using bound.

Jaruloj Chongstitvatana Search 24

 State-Space Tree with Bound

State
bound
State State State
bound bound bound

State State State State

bound bound bound bound

State State State

bound bound bound

Jaruloj Chongstitvatana Search 25

 Branch and Bound
From a node N:
• If the bound of N is not better than the
current overall bound, terminate the
search from N.
• Otherwise,
• If N is a leaf node
• If its bound is better than the current overall
bound, update the overall bound and terminate the
search from N.
• Otherwise, terminate the search from N.
• Otherwise, search each child of N.

Jaruloj Chongstitvatana Search 26

 0/1 Knapsack: Branch-and-Bound
Global: OvBound=0
BBknapsack(sack, unchosen)
if bound(sack, unchosen)>OvBound
► Estimated bound can be better than the overall bound.
then for each object p in unchosen
do unchosen=unchosen-{p}
if p can be put in sack then sack = sack U {p}
► Backtracking occurs when p cannot be put in sack.
val=evaluate(sack)
if unchosen is empty
► A leaf is reached.
then maxV = max(maxV, val)
maxSack = sack
OvBound = max(evaluate(sack), OvBound)
return
else ► A leaf is not reached.
BBknapsack(sack, unchosen)
return

Jaruloj Chongstitvatana Search 27

 0/1 Knapsack: Estimated bound
• Current value
•

Jaruloj Chongstitvatana Search 28

 0/1 Knapsack: Branch-and-bound
Sack estimated bound {} 20
Node

Current weight, current value 0,0

{1}17 {2}18 {3}16 {4}19

2,5 1,4 3,8 2,7

{1,2}16 {1,3} 13 {2,3}15.5 {2,4}16.3 {3,4}15

3,9 5,13 4,12 3,11 5,15
object weight value ratio
{1,2,4} 16
1 2 5 2.5
5, 16 Overall bound 16
0
2 1 4 4
3 3 8 2.67
4 2 7 3.5
Capacity = 5
Jaruloj Chongstitvatana Search 29