Heuristic Search

1
 Heuristic Search Techniques
• Direct techniques (blind search) are not
always possible (they require too much time
or memory).

• Weak techniques can be effective if applied

correctly on the right kinds of tasks.

– Typically require domain specific information.

2
 Informed (Heuristic) Search
Strategies
• There is a whole family of Best-First Search
algorithms with different evaluation functions
– Each has a heuristic function h(n)

• h(n) = estimated cost of the cheapest path from

node n to a goal node

• Example: in route planning the estimate of the cost

of the cheapest path might be the straight line
distance between two cities
January 31, 2006 AI: Chapter 4: Informed Search and 3
Exploration
 Informed (Heuristic) Search
Strategies
• Informed Search – a strategy that uses problem-
specific knowledge beyond the definition of the
problem itself
• Best-First Search – an algorithm in which a node is
selected for expansion based on an evaluation
function f(n)
– Traditionally the node with the lowest evaluation
function is selected
– Not an accurate name…expanding the best node first
would be a straight march to the goal.
– Choose the node that appears to be the best
January 31, 2006 AI: Chapter 4: Informed Search and 4
Exploration
 Example: 8 Puzzle

1 2 3 1 2 3
7 8 4 8 4
6 5 7 6 5
5
1 23 GOAL 1 23
8 4 7 84
7 65 6 5
up
left right

1 23 1 23 1 23
7 84 7 84 7 4
65 6 5 6 85

Which move is best?

6
 8 Puzzle Heuristics
• Blind search techniques used an arbitrary
ordering (priority) of operations.
• Heuristic search techniques make use of
domain specific information - a heuristic.
• What heurisitic(s) can we use to decide
which 8-puzzle move is “best” (worth
considering first).

7
 8 Puzzle Heuristics
• For now - we just want to establish some
ordering to the possible moves (the values
of our heuristic does not matter as long as it
ranks the moves).
• Later - we will worry about the actual
values returned by the heuristic function.

8
 A Simple 8-puzzle heuristic
• Number of tiles in the correct position.
– The higher the number the better.
– Easy to compute (fast and takes little memory).
– Probably the simplest possible heuristic.

9
 Another approach
• Number of tiles in the incorrect position.
– This can also be considered a lower bound on the
number of moves from a solution!
– The “best” move is the one with the lowest number
returned by the heuristic.
– Is this heuristic more than a heuristic (is it always
correct?).
• Given any 2 states, does it always order them properly
with respect to the minimum number of moves away from
a solution?
10
1 23 GOAL 1 23
8 4 7 84
7 65 6 5
up
left right

1 23 1 23 1 23
7 84 7 84 7 4
65 6 5 6 85
h=2 h=4 h=3

11
 Another 8-puzzle heuristic
• Count how far away (how many tile
movements) each tile is from it’s correct
position.
• Sum up this count over all the tiles.
• This is another estimate on the number of
moves away from a solution.

12
1 23 GOAL 1 23
8 4 7 84
7 65 6 5
up
left right

1 23 1 23 1 23
7 84 7 84 7 4
65 6 5 6 85
h=2 h=4 h=4

13
 Techniques
• There are a variety of search techniques that
rely on the estimate provided by a heuristic
function.
• In all cases - the quality (accuracy) of the
heuristic is important in real-life application
of the technique!

14
 Generate-and-test
• Very simple strategy - just keep guessing.

do while goal not accomplished

generate a possible solution
test solution to see if it is a goal

• Heuristics may be used to determine the

specific rules for solution generation.
15
 Example - Traveling Salesman
Problem (TSP)
• Traveler needs to visit n cities.
• Know the distance between each pair of
cities.
• Want to know the shortest route that visits
all the cities once.
• n=80 will take millions of years to solve
exhaustively!
16
 TSP Example

A 6 B
1 2
5 3

D 4 C

17
 Generate-and-test Example
• TSP - generation of possible
solutions is done in
lexicographical order of A B C D
cities:
1. A - B - C - D B C D
2. A - B - D - C
3. A - C - B - D C D B D C B
4. A - C - D - B
...
D C D B B C
18
 A Quick Review
• g(n) = cost from the initial state to the
current state n

• h(n) = estimated cost of the cheapest path

from node n to a goal node

• f(n) = evaluation function to select a node

for expansion (usually the lowest cost node)
January 31, 2006 AI: Chapter 4: Informed Search and 19
Exploration
 Greedy Best-First Search
• Greedy Best-First search tries to expand the node that is
closest to the goal assuming it will lead to a solution
quickly
– f(n) = h(n)
– aka “Greedy Search”

• Implementation
– expand the “most desirable” node into the fringe queue
– sort the queue in decreasing order of desirability

• Example: consider the straight-line distance heuristic hSLD

– Expand the node that appears to be closest to the goal

January 31, 2006 AI: Chapter 4: Informed Search and 20

Exploration
 Greedy Best-First Search

January 31, 2006 AI: Chapter 4: Informed Search and 21

Exploration
 Greedy Best-First Search
• hSLD(In(Arid)) = 366

• Notice that the values of hSLD cannot be

computed from the problem itself

• It takes some experience to know that h SLD

is correlated with actual road distances
– Therefore a useful heuristic
January 31, 2006 AI: Chapter 4: Informed Search and 22
Exploration
 Greedy Best-First Search

January 31, 2006 AI: Chapter 4: Informed Search and 23

Exploration
 Greedy Best-First Search

January 31, 2006 AI: Chapter 4: Informed Search and 24

Exploration
 Greedy Best-First Search

January 31, 2006 AI: Chapter 4: Informed Search and 25

Exploration
 Greedy Best-First Search
• Complete
– No, GBFS can get stuck in loops (e.g. bouncing back
and forth between cities)
• Time
– O(bm) but a good heuristic can have dramatic
improvement
• Space
– O(bm) – keeps all the nodes in memory
• Optimal
– No!
January 31, 2006 AI: Chapter 4: Informed Search and 26
Exploration
 A Quick Review - Again
• g(n) = cost from the initial state to the
current state n

• h(n) = estimated cost of the cheapest path

from node n to a goal node

• f(n) = evaluation function to select a node

for expansion (usually the lowest cost node)
January 31, 2006 AI: Chapter 4: Informed Search and 27
Exploration
 Hill Climbing
• Variation on generate-and-test:
– generation of next state depends on feedback
from the test procedure.
– Test now includes a heuristic function that
provides a guess as to how good each possible
state is.
• There are a number of ways to use the
information returned by the test procedure.
28
 Simple Hill Climbing
• Use heuristic to move only to states that are
better than the current state.

• Always move to better state when possible.

• The process ends when all operators have

been applied and none of the resulting states
are better than the current state.
29
 Simple Hill Climbing
Function Optimization

y = f(x)
y

x 30
 Potential Problems with
Simple Hill Climbing
• Will terminate when at local optimum.

• The order of application of operators can

make a big difference.

• Can’t see past a single move in the state

space.

31
 Simple Hill Climbing Example
• TSP - define state space as the set of all
possible tours.

• Operators exchange the position of adjacent

cities within the current tour.

• Heuristic function is the length of a tour.

32
 TSP Hill Climb State Space
ABCD Initial State
ABCD
Swap 1,2 Swap 2,3
Swap 3,4 Swap 4,1

BACD
BACD ACBD
ACBD ABDC
ABDC DBCA
DBCA

Swap 1,2 Swap 3,4

Swap 2,3 Swap 4,1

CABD
CABD ABCD
ABCD ACDB
ACDB DCBA
DCBA

33
 Steepest-Ascent Hill Climbing
• A variation on simple hill climbing.
• Instead of moving to the first state that is
better, move to the best possible state that is
one move away.
• The order of operators does not matter.

• Not just climbing to a better state, climbing

up the steepest slope.
34
 Hill Climbing Termination
• Local Optimum: all neighboring states are
worse or the same.

• Plateau - all neighboring states are the same

as the current state.

• Ridge - local optimum that is caused by

inability to apply 2 operators at once.
35
 Heuristic Dependence
• Hill climbing is based on the value assigned
to states by the heuristic function.
• The heuristic used by a hill climbing
algorithm does not need to be a static
function of a single state.
• The heuristic can look ahead many states,
or can use other means to arrive at a value
for a state.
36
 Best-First Search
• Combines the advantages of Breadth-First
and Depth-First searchs.
– DFS: follows a single path, don’t need to
generate all competing paths.
– BFS: doesn’t get caught in loops or dead-end-
paths.
• Best First Search: explore the most
promising path seen so far.
37
 Best-First Search (cont.)
While goal not reached:
1. Generate all potential successor states and add to a
list of states.
2. Pick the best state in the list and go to it.

• Similar to steepest-ascent, but don’t throw

away states that are not chosen.

38
 Simulated Annealing
• Based on physical process of annealing a
metal to get the best (minimal energy) state.
• Hill climbing with a twist:
– allow some moves downhill (to worse states)
– start out allowing large downhill moves (to
much worse states) and gradually allow only
small downhill moves.

39
 Simulated Annealing (cont.)
• The search initially jumps around a lot,
exploring many regions of the state space.

• The jumping is gradually reduced and the

search becomes a simple hill climb (search
for local optimum).

40
 Simulated Annealing

6
2 4
3 5
1

41
 A* Algorithm (a sure test topic)
• The A* algorithm uses a modified
evaluation function and a Best-First search.

• A* minimizes the total path cost.

• Under the right conditions A* provides the

cheapest cost solution in the optimal time!
42
 A* evaluation function
• The evaluation function f is an estimate of
the value of a node x given by:
f(x) = g(x) + h’(x)
• g(x) is the cost to get from the start state to
state x.
• h’(x) is the estimated cost to get from state
x to the goal state (the heuristic).
43
 Modified State Evaluation
• Value of each state is a combination of:
– the cost of the path to the state
– estimated cost of reaching a goal from the state.
• The idea is to use the path to a state to
determine (partially) the rank of the state
when compared to other states.
• This doesn’t make sense for DFS or BFS,
but is useful for Best-First Search.
44
 Why we need modified
evaluation
• Consider a best-first search that generates the
same state many times.
• Which of the paths leading to the state is the
best ?

• Recall that often the path to a goal is the

answer (for example, the water jug problem)
45
 A* Algorithm
• The general idea is:
– Best First Search with the modified evaluation
function.
– h’(x) is an estimate of the number of steps from
state x to a goal state.
– loops are avoided - we don’t expand the same
state twice.
– Information about the path to the goal state is
retained.
46
 A* Algorithm
1. Create a priority queue of search nodes (initially the start
state). Priority is determined by the function f )
2. While queue not empty and goal not found:
get best state x from the queue.
If x is not goal state:
generate all possible children of x (and save
path information with each node).
Apply f to each new node and add to queue.
Remove duplicates from queue (using f to pick
the best).
47
 Example - Maze

START GOAL

48
 Example - Maze

START GOAL

49
A* Optimality and Completeness
• If the heuristic function h’ is admissible the
algorithm will find the optimal (shortest
path) to the solution in the minimum number
of steps possible (no optimal algorithm can
do better given the same heuristic).
• An admissible heuristic is one that never
overestimates the cost of getting from a state
to the goal state (is pessimistic).
50
 Admissible Heuristics
• Given an admissible heuristic h’, path
length to each state given by g, and the
actual path length from any state to the goal
given by a function h.
• We can prove that the solution found by A*
is the optimal solution.

51
 A* Optimality Proof
• Assume A* finds the (suboptimal) goal G2
and the optimal goal is G.
• Since h’ is admissible: h’(G2)=h’(G)=0
• Since G2 is not optimal: f(G2) > f(G).
• At some point during the search some node n
on the optimal path to G is not expanded. We
know:
f(n)  f(G)
52
 Proof (cont.)
• We also know node n is not expanded
before G2, so:
f(G2)  f(n)
• Combining these we know:
f(G2)  f(G)
• This is a contradiction ! (G2 can’t be
suboptimal).
53
 root (start state)

G G2

54
 A* Example
Towers of Hanoi
Little Disk Peg 1 Peg 3
Peg 2

Big Disk

• Move both disks on to Peg 3

• Never put the big disk on top the little disk

55
 Relaxed Problems
• A Relaxed Problem is a problem with fewer
restrictions on the actions
– The cost of an optimal solution to a relaxed
problem is an admissible heuristic for the original
problem

• Key point: The optimal solution of a relaxed

problem is no greater than the optimal solution
of the real problem
January 31, 2006 AI: Chapter 4: Informed Search and 56
Exploration
 Relaxed Problems
• Example: 8-puzzle
– Consider only getting tiles 1, 2, 3, and 4 into
place

– If the rules are relaxed such that a tile can move

anywhere then h1(n) gives the shortest solution
– If the rules are relaxed such that a tile can move
to any adjacent square then h2(n) gives the
shortest solution
January 31, 2006 AI: Chapter 4: Informed Search and 57
Exploration
 Relaxed Problems
• Store sub-problem solutions in a database
– # patterns is much smaller than the search space
– Generate database by working backwards from
the solution
– If multiple sub-problems apply, take the max
– If multiple disjoint sub-problems apply,
heuristics can be added

January 31, 2006 AI: Chapter 4: Informed Search and 58

Exploration
 Learning Heuristics From
Experience
• h(n) is an estimate cost of the solution beginning at
state n
• How can an agent construct such a function?
• Experience!
– Have the agent solve many instances of the problem and
store the actual cost of h(n) at some state n
– Learn from the features of a state that are relevant to the
solution, rather than the state itself
• Generate “many” states with a given feature and determine the
average distance
• Combine the information from multiple features
– h(n) = c(1)*x1(n) + c(2)*x2(n) + … where x1, x2, … are features

January 31, 2006 AI: Chapter 4: Informed Search and 59

Exploration
 Optimization Problems
• Instead of considering the whole state space,
consider only the current state
• Limits necessary memory; paths not retained
• Amenable to large or continuous (infinite)
state spaces where exhaustive search
algorithms are not possible
• Local search algorithms can’t backtrack

January 31, 2006 AI: Chapter 4: Informed Search and 60

Exploration
 Local Search Algorithms
• They are useful for solving optimization problems
– Aim is to find a best state according to an objective
function

• Many optimization problems do not fit the

standard search model outlined in chapter 3
– E.g. There is no goal test or path cost in Darwinian
evolution
• State space landscape
January 31, 2006 AI: Chapter 4: Informed Search and 61
Exploration
 Optimization Problems
• Given measure of goodness (of fit)
– Find optimal parameters (e.g correspondences)
– That maximize goodness measure (or minimize
badness measure)

• Optimization techniques
– Direct (closed-form)
– Search (generate-test)
– Heuristic search (e.g Hill Climbing)
– Genetic Algorithm
January 31, 2006 AI: Chapter 4: Informed Search and 62
Exploration
 Direct Optimization
• The slope of a function at the maximum or minimum is 0
– Function is neither growing nor shrinking
– True at global, but also local extreme points

• Find where the slope is zero and you find extrema!

• (If you have the equation, use calculus (first derivative=0)

January 31, 2006 AI: Chapter 4: Informed Search and 63

Exploration