Brute Force Approaches

Techniques for finding optimal solutions to

hard problems relatively quickly
Backtracking
Branch and Bound
Combining efficient solutions with brute
force approaches
Other issues

The Knapsack Problem

Input
Capacity K
n items with weights wi and values vi

Goal
Output a set of items S such that
the sum of weights of items in S is at most K
and the sum of values of items in S is maximized

Greedy Does Not Work

What are possible greedy strategies?
Highest Density First
Highest Value First
Lowest Weight First

Prove these are not optimal for the 0-1

knapsack problem.

Multi-constraint Knapsack
Input
Capacity K1 and K2
n items with integer weights wi, size si, and values vi

Goal
Output a set of items S such that
the sum of weights of items in S is at most K1
the sum of sizes of items in S is at most K 2
and the sum of values of items in S is maximized

Can we use dynamic programming?

Situation where dynamic

programming does not work
What if our problem cant be described with integers?
wA = 2

vA = $40

wB =

vB = $50

wC = 1.98

vC = $100

wD = 5

vD = $95

wE = 3

vE = $30

We have to resort to brute force.

Brute Force
Generate all possible solutions
With n items, there are 2n solutions to be
generated
Check each to see if they satisfy the constraint
Save maximum solution that satisfies constraint

Can be represented as a tree

Brute Force: Branching

A

In

Out

D
E

D
E

D
E

Weight = 15.12
Value = $315

D
E

D
E

Weight = 8.98
Value = $235

D
E

D
E

D
E

Weight = 9.98
Value = $225

Backtracking
In the tree representation, we can think of the
previous algorithm doing a DFS in the tree
If we reach a point where a solution no longer is
feasible, there is no need to continue exploring
We can backtrack from this point and potentially
cut off much of the tree and many of the solutions
In the given example, backtracking would be
much more effective if we had even more items or
a smaller knapsack capacity

2, $40
, $50

Backtracking
A

In

1.98, $100
5, $95
3, $30

Out

D
E

D
E

D
E

D
E

D
E

Weight = 8.98
Value = $235

D
E

D
E

D
E

Weight = 9.98
Value = $225

Branch and Bound

We can backtrack if we know the best possible
solution in current subtree is worse than current
best solution obtained so far.
Estimates for improvement given current ordering

A down can give $315

B down -> $275
C down -> $225
D down -> $125
E down -> $30

2, $40
, $50

Branch and Bound

A

In

D
E

Weight = 7.12
Value = $190

3, $30

C
D

5, $95

Out

1.98, $100

C
D

E
Weight = 8.98
Value = $235

D
E

C
D

Generating a good bound

The key to branch and bound is
having a good initial bound.
How might we generate a good
initial bound?

2, $40
, $50

Order of items

1.98, $100
5, $95
3, $30

Since there is no fixed ordering of

items, is there a better way to order the
input items?
Highest weight first
Generate infeasible solutions quickly

Highest density first

Generate good solution quickly

Which is better depends on input

D: 5, $95
B: , $50

Heaviest on Top
D

In
B

C
Weight = 8.14
Value = $145

A
C

C: 1.98, $100

Out

(Max remaining = $220)

E
A

A: 2, $40

E
A

E: 3, $30

A
C

Weight = 10.0
Value = $165

Weight = 9.98
Value = $225

Weight = 8.98
Value = $235

C: 1.98, $100
A: 2, $40

Best Density on Top

C

In
A

B
E

D
B

B
E

Weight = 8.98
Value = $235

B
E

B: , $50
E: 3, $30

Out
A

D: 5, $95

Greedy and Brute Force

Suppose half the inputs to our knapsack
problem all have the same weight.
If all inputs had the same weight, we could
implement a greedy solution
However, since half do not, we cannot use
greedy alone to find optimal solution

Combine brute-force approach with greedy

to find optimal solution more quickly.

The Breakdown...
F1

In

Out

F2

F2

F3
F4

F3
F4

F5 F5 F5 F5

F4

F3
F4

F4

F5 F5

F5 F5

F3
F4

Greedy
on half!

F4
F5 F5

F4

Algorithm
Backtrack on half the inputs
At leafs, apply greedy strategy on the other
half of the inputs
Comparison
Pure brute force: O(2n)
Combination: O(n2n/2)

How much better?

Assumptions
Suppose n=50
We can test 1,000,000 solutions/second.

250 would take over 35 years

225 can be generated in half an hour
Plus marginal time to generate greedy solution
for each of the 225 solutions

Another combination
Suppose half the inputs to our knapsack
problem have small integral weights/values
while the other half have real
weights/values.
How can we combine approaches to solve
this efficiently?

The n-Queens Problem

Input
Positive integer n

Task
Place n queens on an n by n chessboard so that
no two queens attack each other (on same row,
column, diagonal), or report that this is
impossible

Solve the n-queens problem for n = 1, 2, 3

n=4
Pure brute force search
16 squares on a 4 by 4 chessboard
Leads to 16 * 15 * 14 * 13 = 43,680 possible
solutions

Improvements
At most one queen per row: 44 = 256 possible
solutions
Backtracking: If two queens already attack
before final queen placed, backtrack

Larger values of n
n=8
At most one queen per row: 88 = 16,777,216
Early backtracking: 2057 nodes total
Time to find first solution: 114 nodes

n=12
At most one queen per row: 1212
Early backtracking: 856,189 nodes total
Time to find first solution: 262 nodes

The Problem
In the U.S. navy, the SEALS are each specially trained in
a wide variety of skills so that small teams can handle a
multitude of missions.
If there are k different skills needed for a mission, and n
SEAL members that can be assigned to the team, find the
smallest team that will cover all of the required skills.
Andersen knows hand-to-hand, first aid, and camouflage
Butler knows hand-to-hand and snares
Cunningham knows hand-to-hand
Douglas knows hand-to-hand, sniping, diplomacy, and snares
Eckers knows first-aid, sniping, and diplomacy

Greedy Algorithm
What is the obvious greedy algorithm for
this problem?
Find a counter-example to the optimality of
greedy for this problem.

Brute Force Approach

What is the brute-force approach?
How can we simplify the problem as far as
possible in polynomial time?
How can we set bounds on the solutions?
When will we need to do backtracking?
What order should we test the potential members
in when branching?