Lec4 4

CS321: Principles of Artificial Intelligence
Principles of Artificial
Intelligence
Course Code CS321
Faculty of Computing and Information Technology

Computer Science Department
Jan, 2022
These slides are based on lecture notes of the book’s author

&
King Saud University course materials
Lecturer:Wedad Al-Sorori
Search 2022 1/90
Informed (Heuristic)
Search Algorithms
Search 2022 2/90

Chapter Objectives
• At the end of this chapter, the student should be able to:

• Distinguish between various types of informed search algorithms.
• Trace the solution path depending on the chosen strategy.
• Compare between informed search algorithms based on evaluation dimensions.
Search 2022 3/90

Introduction (1/7)
• What we have seen so far are all those

search algorithms that operate with out
any domain knowledge, and try to solve Search everywhere!!
the problem in brute-force fashion.
• Such methods are insufficient for most
problems specially when the problem is
large and complex.
• Now we will look into another type of
search techniques called informed
search.
Search 2022 4/90

Introduction (2/7)
• With knowledge, one can search the state space as if he was given “hints”
when exploring a maze.
• Heuristic information in search = Hints
• Leads to dramatic speed up in efficiency.
A
B C D E
F G H I J
i.e. search only in K L M N

this subtree!!
O
Search 2022 5/90

Introduction (3/7)
• In any search problem where there are at most b choices at each node and a depth
of d at the goal node, a naive search algorithm would have to, in the worst case,
search around O(bd) nodes before finding a solution (Exponential Time
Complexity).
• Heuristics improve the efficiency of search algorithms by reducing the effective

branching factor from b to (ideally) a low constant b* such that
• 1 =< b* << b
Search 2022 6/90

Heuristic Functions
Introduction (4/7)
• A heuristic function is a function f(n) that gives an estimation on the “cost” of

getting from node n to the goal state – so that the node with the least cost
among all possible choices can be selected for expansion first.
• Three approaches to defining f:
• f measures the value of the current state (its “goodness”)
• f measures the estimated cost of getting to the goal from the current state:
• f(n) = h(n) where h(n) = an estimate of the cost to get from n to a goal
• f measures the estimated cost of getting to the goal state from the current state
and the cost of the existing path to it. Often, in this case, we decompose f:
• f(n) = g(n) + h(n) where g(n) = the cost to get to n (from initial state)
Search 2022 7/90

Introduction (5/7)
• Approach 1: f Measures the Value of the Current State

• Usually the case when solving optimization problems
• Finding a state such that the value of the metric f is optimized
• Often, in these cases, f could be a weighted sum of a set of

component values:
• N-Queens
• Example: the number of queens under attack …
Search 2022 8/90

Introduction (6/7)
• Approach 2: f Measures the Cost to the Goal

• A state X would be better than a state Y if the estimated cost of getting
from X to the goal is lower than that of Y – because X would be closer to
the goal than Y
• 8–Puzzle
• h1: The number of misplaced tiles (squares with number).
• h2: The sum of the distances of the tiles from their goal positions.
Search 2022 9/90

Introduction (7/7)
• Approach 3: f measures the total cost of the solution path

(Admissible Heuristic Functions)
• A heuristic function f(n) = g(n) + h(n) is admissible if h(n) never overestimates the
cost to reach the goal.
• Admissible heuristics are “optimistic”: “the cost is not that much …”
• However, g(n) is the exact cost to reach node n from the initial state.
• Therefore, f(n) never over-estimate the true cost to reach the goal state through
node n.
• Theorem: A search is optimal if h(n) is admissible.
• I.e. The search using h(n) returns an optimal solution.
• Given h2(n) > h1(n) for all n, it’s always more efficient to use h2(n).
• h2 is more realistic than h1 (more informed), though both are optimistic.
Search 2022 10/90

Informed Search Strategies (1/1)
• Best First Search

• Greedy Search
• A*
• Iterative Deepening A* (IDA*)
• Hill Climbing
Search 2022 11/90

Best First Search (BFS) (1/3)

Best-FS tries to minimize some estimated measure (evaluation function) of
the cost of the solution.
• Idea: use an evaluation function f(n) for each node
• estimate of "desirability"
• Expand most desirable unexpanded node
• Implementation:
Order the nodes in fringe in decreasing order of desirability
• Then the Best-FS may follow two strategies:
− Greedy Search
− A* Search
• Best-FS specializes to BFS when f (n) = h(n) and to UFS when f (n) = g(n).
Search 2022 12/90
• Greedy Best first search

• Expands the node closest to the goal, i.e., f (n) = h(n).
• “Always chooses the successor node with the best f value” where f(n) = h(n)
• We choose the one that is nearest to the final state among all possible choices
• A* search
• Expands the node on the least cost solution path, i.e., f (n) = g(n) + h(n).
• Best first search using an “admissible” heuristic function f that takes into account the
current cost g
• Always returns the optimal solution path
Search 2022 13/90

function BEST-FIRST-SEARCH (problem, eval-fn)

returns a solution sequence, or failure
queuing-fn = a function that sorts nodes by eval-fn

return GENERIC-SEARCH (problem,queuing-fn)
Search 2022 14/90

Informed Search Strategies (1/1)
• Best First Search

• Greedy Search
• A*
• Iterative Deepening A* (IDA*)
• Hill Climbing
Search 2022 15/90

Greedy Search (1/7)
Start State Heuristic: h(n)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I f(n) = hSLD (n) = straight-line distance heuristic
Search 2022 16/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
Search 2022 17/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
f(n) = hSLD (n) = straight-line distance heuristic
Search 2022 18/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 19/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 20/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 21/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 22/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
Search 2022 23/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 24/90
Greedy Search (2/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
Search 2022 25/90
Greedy Search (3/7)
Start
A
Search 2022 26/90

Greedy Search (3/7)
Start
A 75
118
[329] C 140 [374] B
[253] E
Search 2022 27/90

Greedy Search (3/7)
Start
A 75
118
[329] C 140 [374] B
[253] E
80 99
[193] G F [178]
[366] A
Search 2022 28/90

Greedy Search (3/7)
Start
A 75
118
[329] C 140 [374] B
[253] E
80 99
[193] G F [178]
[366] A
211
[253] E I [0]
Goal
Search 2022 29/90
Greedy Search (3/7)
Start
A 75
118
[329] C 140 [374] B
[253] E
80 99
[193] G F [178]
[366] A
211
Path cost(A-E-F-I) = 253 + 178 + 0 = 431 [253] E I [0]

dist(A-E-F-I) = 140 + 99 + 211 = 450
Goal
Search 2022 30/90
Greedy Search (4/7)

A 75
118 A 366
140 B B 374
C
111 C 329
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I f(n) = h (n) = straight-line distance heuristic
Goal dist(A-E-G-H-I) =140+80+97+101=418
Search 2022 31/90
Greedy Search (5/7)

A 75
118 A 366
140 B B 374
C
111 ** C 250
E
D D 244
80 99
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I
f(n) = h (n) = straight-line distance heuristic
Goal
Search 2022 32/90
Greedy Search (6/7)
Start
A
Search 2022 33/90

Greedy Search (6/7)
Start
A 75
118
[250] C 140 [374] B
[253] E
Search 2022 34/90

Greedy Search (6/7)
Start
A 75
118
[250] C 140 [374] B
111 [253] E
[244] D
Search 2022 35/90

Greedy Search (6/7)
Start
A 75
118
[250] C 140 [374] B
111 [253] E
[244] D
Infinite Branch !
[250] C
Search 2022 36/90

Greedy Search (6/7)
Start
A 75
118
[250] C 140 [374] B
111 [253] E
[244] D
Infinite Branch !
[250] C
[244] D
Search 2022 37/90

Greedy Search (6/7)
Start
A 75
118
[250] C 140 [374] B
111 [253] E
[244] D
Infinite Branch !
[250] C
[244] D
Search 2022 38/90

Greedy Search
⚫ Complete? No – can get stuck in loops, e.g., Iasi → Neamt → Iasi →

Neamt →
⚫ Time? O(bm), but a good heuristic can give dramatic improvement
⚫ Space? O(bm) -- keeps all nodes in memory
⚫ Optimal? No
Search 2022 39/90

Greedy Search (7/7)
Start • Greedy search is not optimal.

A 75
118
140 B • Greedy search is incomplete without
C
111 systematic checking of repeated states.
E
D 80 99 • In the worst case, the time and space
complexity of greedy search are both
G F
O(bm), Where b is the branching factor
97
and m the maximum path length
H 211
101
I
Search 2022 40/90
Informed Search Strategies

A* Search
eval-fn: f(n)=g(n)+h(n)
Search 2022 41/90

A* (A Star)
• Greedy Search minimizes a heuristic h(n) which is an estimated cost
from a node n to the goal state. Greedy Search is efficient but it is
not optimal nor complete.
• Uniform Cost Search minimizes the cost g(n) from the initial state to
n. UCS is optimal and complete but not efficient.
• New Strategy: Combine Greedy Search and UCS to get an efficient

algorithm which is complete and optimal.
Search 2022 42/90

A* (A Star)
• A* uses a heuristic function which combines g(n) and h(n): f(n) =

g(n) + h(n)
• g(n) is the exact cost to reach node n from the initial state.
• h(n) is an estimation of the remaining cost to reach the goal.
Search 2022 43/90

A* (A Star)
g(n)
f(n) = g(n)+h(n) n
h(n)
Search 2022 44/90

A* Search
Start
A 75 State Heuristic: h(n)
118
A 366
140 B
C B 374
111 C 329
E
D 80 99 D 244
E 253
G F F 178
97 G 193
H 211 H 98
101 I 0
I f(n) = g(n) + h (n)
Goal
g(n): is the exact cost to reach node n from the initial state.
Search 2022 45/90
A* Search: Tree Search
A Start
Search 2022 46/90

A Start
118 75
140
C E [393] B [449]
[447]
Search 2022 47/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
Search 2022 48/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[415] H
Search 2022 49/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[415] H
101
I
Goal [418]
Search 2022 50/90
A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[415] H I [450]
101
Goal I [418]
Search 2022 51/90
A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[415] H I [450]
101
Goal I [418]
Search 2022 52/90
A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[415] H I [450]
101
I
Goal [418]
Search 2022 53/90
A* with f() not Admissible

h() overestimates the cost to reach the goal state
Search 2022 54/90

A* Search: h not admissible ! State Heuristic: h(n)

Start A 366
A 75
118 B 374
140 B
C C 329
111 E D 244
D 80 99 E 253
F 178
G F
G 193
97
H 138
H 211 I 0
101 f(n) = g(n) + h (n) – (H-I) Overestimated
I
Goal g(n): is the exact cost to reach node n from the initial state.
Search 2022 55/90
A Start
Search 2022 56/90

A Start
118 75
140
C E [393] B [449]
[447]
Search 2022 57/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
Search 2022 58/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[455] H
Search 2022 59/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[413] G F [417]
97
[455] H I [450]
Goal
Search 2022 60/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[473] D [413] G F [417]
97
[455] H I [450]
Goal
Search 2022 61/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[473] D [413] G F [417]
97
[455] H I [450]
Goal
Search 2022 62/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[473] D [413] G F [417]
97
[455] H I [450]
Goal
Search 2022 63/90

A Start
118 75
140
C E [393] B [449]
[447]
80 99
[473] D [413] G F [417]
97
[455] H I [450]
Goal
A* not optimal !!!
Search 2022 64/90

A* Algorithm
A* with systematic checking for repeated states …
Search 2022 65/90

A* Algorithm
1. Search queue Q is empty.

2. Place the start state s in Q with f value h(s).
3. If Q is empty, return failure.
4. Take node n from Q with lowest f value.
(Keep Q sorted by f values and pick the first element).
5. If n is a goal node, stop and return solution.
6. Generate successors of node n.
7. For each successor n’ of n do:
a) Compute f(n’) = g(n) + cost(n,n’) + h(n’).
b) If n’ is new (never generated before), add n’ to Q.
c) If node n’ is already in Q with a higher f value, replace it with current f(n’) and place it in
sorted order in Q.
End for
8. Go back to step 3.
Search 2022 66/90

A* Algorithm
⚫ Complete? Yes (unless there are infinitely many nodes with f ≤ f(G) )
⚫ Time? Exponential
⚫ Space? Keeps all nodes in memory
⚫ Optimal? Yes
Search 2022 67/90

A* Search: Analysis
Start
A 75
118
140 B
C • A* is complete except if there is an infinity of nodes with f
111 E
< f(G).
D 80 99 • A* is optimal if heuristic h is admissible.
• Time complexity depends on the quality of heuristic but is
G F still exponential.
97 • For space complexity, A* keeps all nodes in memory. A*

has worst case O(bd) space complexity, but an iterative
H 211 deepening version is possible (IDA*).
101
I
Goal
Search 2022 68/90
Informed Search Strategies

Iterative Deepening A*
Search 2022 69/90

Iterative Deepening A*:IDA*
• Use f(N) = g(N) + h(N) with admissible and consistent h
• Each iteration is depth-first with cutoff on the value of f of

expanded nodes
Search 2022 70/90

Consistent Heuristic
• The admissible heuristic h is consistent (or satisfies the
monotone restriction) if for every node N and every
successor N’ of N:
N
h(N)  c(N,N’) + h(N’) c(N,N’)

N’ h(N)
(triangular inequality) h(N’)
• A consistent heuristic is admissible.
71
Search 2022 71/90
IDA* Algorithm
• In the first iteration, we determine a “f-cost limit” – cut-off value
f(n0) = g(n0) + h(n0) = h(n0), where n0 is the start node.
• We expand nodes using the depth-first algorithm and backtrack whenever f(n) for an expanded
node n exceeds the cut-off value.
• If this search does not succeed, determine the lowest f-value among the nodes that were visited
but not expanded.
• Use this f-value as the new limit value – cut-off value and do another depth-first search.
• Repeat this procedure until a goal node is found.
Search 2022 72/90

8-Puzzle f(N) = g(N) + h(N)

with h(N) = number of misplaced tiles
4
Cutoff=4
6
Search 2022 73/90

4
Cutoff=4 4
6 6
Search 2022 74/90
8-Puzzle
f(N) = g(N) + h(N)
4
Cutoff=4 4 5
6 6
Search 2022 75/90

4
Cutoff=4 4 5
6 6
Search 2022 76/90

6 5
4
Cutoff=4 4 5
6 6
Search 2022 77/90
8-Puzzle
f(N) = g(N) + h(N)
4
Cutoff=5
6
Search 2022 78/90
THANKS
Search 2022 79/90

Lec4 4

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec4 4

Uploaded by

Copyright:

Available Formats

CS321: Principles of Artificial Intelligence

Faculty of Computing and Information Technology

These slides are based on lecture notes of the book’s author

Search 2022 2/90

• At the end of this chapter, the student should be able to:

• Trace the solution path depending on the chosen strategy.

• Compare between informed search algorithms based on evaluation dimensions.

Search 2022 3/90

• What we have seen so far are all those

Search 2022 4/90

i.e. search only in K L M N

Search 2022 5/90

• Heuristics improve the efficiency of search algorithms by reducing the effective

Search 2022 6/90

• A heuristic function is a function f(n) that gives an estimation on the “cost” of

• Three approaches to defining f:

• f measures the value of the current state (its “goodness”)

Search 2022 7/90

• Approach 1: f Measures the Value of the Current State

• Often, in these cases, f could be a weighted sum of a set of

Search 2022 8/90

• Approach 2: f Measures the Cost to the Goal

Search 2022 9/90

• Approach 3: f measures the total cost of the solution path

Search 2022 10/90

Informed Search Strategies (1/1)

• Best First Search

Search 2022 11/90

Best First Search (BFS) (1/3)

Best First Search (BFS) (2/3)

• Greedy Best first search

Search 2022 13/90

Best First Search (BFS) (3/3)

function BEST-FIRST-SEARCH (problem, eval-fn)

queuing-fn = a function that sorts nodes by eval-fn

Search 2022 14/90

Informed Search Strategies (1/1)

• Best First Search

Search 2022 15/90

Greedy Search (1/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (2/7)

Start State Heuristic: h(n)

Greedy Search (3/7)

Search 2022 26/90

Greedy Search (3/7)