Searching

Techniques
 Searching
• Searching is the process of looking for
the solution to a goal state through a set
of possibilities.

• There are three conditions of Searching:

Current state –specify where one is.
Goal state (solution state) –Specify
final state
Cost of obtaining the solution.
 Searching strategies requirements
• A good searching method should have the following
Properties.
1. Saves time: Reach the goal with less time.
2.Saves memory: Reach the goal with less state-
space.
3.Complete: Guaranteed to reach the goal.
4. Finite: search Must end either by reaching a goal
or fail to reach a goal.
5.Optimal : should return the best (cheapest)
solution
6. Simple: The method must be as simple as possible
and does not contain complex calculations that affect
the efficiency and speed
 Search

• We have some actions that can change the state of the

world
• Change induced by an action perfectly predictable
• Try to come up with a sequence of actions that will lead
us to a goal state
• May want to minimize number of actions
• More generally, may want to minimize total cost of actions
• Do not need to execute actions in real life while
searching for solution!
 A simple example:
traveling on a graph

C 2
B 9
2
3 F goal state
start state A D E
4
3 4
 Searching for a solution

C 2
B 9
2
3 F goal state
start state A D
3
 Search tree
state = A,
cost = 0

state = B, state = D,
cost = 3 cost = 3

state = C, state = F,
cost = 5 cost = 12

goal state!
state = A,
cost = 7

search tree nodes and states are not the same thing!
 Full search tree
state = A,
cost = 0

state = B, state = D,
cost = 3 cost = 3

state = C, state = F, state = E,

cost = 5 cost = 12 cost = 7

goal state!
state = A,
state = F,
cost = 7
cost = 11

goal state!
state = B, state = D,

.. cost = 10
.. cost = 10

. .
 Changing the goal:
want to visit all vertices on the graph

C 2
B 9
2
3 F
A E
D 4
3 4

need a different definition of a state

“currently at A, also visited B, C already”
large number of states: n*2n-1
could turn these into a graph, but…
 Full search tree
state = A, {}
cost = 0

state = B, {A} state = D, {A}

cost = 3 cost = 3

state = F, {A, B}
state = C, {A, B} state = E, {A, D}
cost = 12
cost = 5 cost = 7

state = A, {B, C}
state = F, {A, D, E}
cost = 7
cost = 11

state = D, {A, B, C}
state = B, {A, C}
cost = 10
What would happen if the
.. cost = 10
.. goal were to visit every
. . location twice?
 Key concepts in search
• Set of states that we can be in
• Including an initial state…
• … and goal states (equivalently, a goal test)
• For every state, a set of actions that we can take
• Each action results in a new state
• Typically defined by successor function
• Given a state, produces all states that can be reached from it
• Cost function that determines the cost of each action
(or path = sequence of actions)
• Solution: path from initial state to a goal state
• Optimal solution: solution with minimal cost
 8-puzzle

1 2 1 2 3
4 5 3 4 5 6
7 8 6 7 8
goal state
8-puzzle
1 2
4 5 3
7 8 6

1 2 1 2 1 5 2
4 5 3 4 5 3 4 3
7 8 6 7 8 6 7 8 6

.. ..
. .
 Generic search algorithm
• Fringe = set of nodes generated but not expanded

• fringe := {initial state}

• loop:
• if fringe empty, declare failure
• choose and remove a node v from fringe
• check if v’s state s is a goal state; if so, declare success
• if not, expand v, insert resulting nodes into fringe

• Key question in search: Which of the generated nodes

do we expand next?
Search methods categories
There are two main category of
searching methods.
1. Uninformed search methods
2. Informed search methods.
Uninformed
search
methods
 Uninformed Search Methods
• Methods that do not use any specific knowledge
about the goal. Also known as Blind search methods
• Examples:
• Depth-first search
• Breadth-first search
• Non deterministic search
• Iterative deepening search
• Bi-directional search
• Uniform cost search
 Characteristics of Uninformed
Search
Blind or uninformed search has the following characteristics.

1. No heuristics : It has no knowledge provided about the about

domain or direction of the goal
4. No preferences: Blind Searches have no preference as to which
state (node) that is expanded next.
5. Ordered search : The search is characterised by the order in
which nodes are expanded.
6. Inefficient: Blind search consumes a lot of time and memory
 Depth-first search
Involves expanding the tree as deep as possible
and returning to upper levels when needed.

S
• Select a child
• convention: left-to-right

A • Repeatedly go to next
child, as long as possible.
B

C E • Return to left-over
alternatives (higher-up)
D F only when needed.
G
Depth-first search
 Implementing depth-first search
• Fringe can be maintained as a Last-In-First-Out (LIFO) structure
(aka. a stack)
• Also easy to implement recursively:
• DFS(node)
• If goal(node) return solution(node);
• For each successor of node
• Return DFS(successor) unless it is failure;
• Return failure;
Depth-first pseudocode:
1. QUEUE <-- path only containing the root;

2. WHILE QUEUE is not empty

AND goal is not reached

DO remove the first path from the QUEUE;

create new paths (to all children);
reject the new paths with loops;
add the new paths to front of QUEUE;

3. IF goal reached
THEN success;
ELSE failure;
 Example:
• Consider the following map:

A 4 B 4 C
3
S 5 5 G
4 D E F 3
2 4

Use depth first method to find a path from s to G

 Trace of depth-first
• (S) S removed, (SA,SD) computed and added
• (SA, SD) SA removed, (SAB,SAD,SAS) computed,
(SAB,SAD) added
• (SAB,SAD,SD) SAB removed, (SABA,SABC,SABE) computed,
(SABC,SABE) added
• (SABC,SABE,SAD,SD) SABC removed, (SABCB) computed,
nothing added
• (SABE,SAD,SD) SABE removed, (SABEB,SABED,SABEF)
computed, (SABED,SABEF)added
 Depth-first algorithm:

• (SABED,SABEF,SAD,SD) SABED removed,

(SABEDS,SABEDA.SABEDE) computed,
nothing added
• (SABEF,SAD,SD) SABEF removed, (SABEFE,SABEFG)
computed, (SABEFG) added
• (SABEFG,SAD,SD) goal is reached: reports success
 Evaluating Depth-first search :

Advantage
1. Completeness- Depth first is Complete for FINITE number of
nodes.

Disadvantages
1.Speed- IF the search space contains very deep branches
without solution, THEN Depth-first may waste much time in
them.

2. Optimality - Depth first does NOT find the shortest path.

 2. Breadth-first Search
• Expand the tree layer by layer, progressing
in depth.
In other words,
• Expand root node first
• Expand all nodes at level 1 before expanding
level 2
OR
• Expand all nodes at level d before expanding
nodes at level d+1
 2. Breadth-first search

S • Move
downwards,
A D level by level,
until goal is
B A
reached.
D E

C E E B B F

D F B F C E A C G

G C G F
G
Breadth-first search
 Properties of breadth-first search
• Nodes are expanded in the same order in which they are
generated
• Fringe can be maintained as a First-In-First-Out (FIFO) queue
• BFS is complete: if a solution exists, one will be found
• BFS finds a shallowest solution
• Not necessarily an optimal solution
• If every node has b successors (the branching factor), first
solution is at depth d, then fringe size will be at least bd at some
point
• This much space (and time) required
 2. Breadth-first: Pseudo code

1. QUEUE <-- path only containing the root;

2. WHILE QUEUE is not empty

AND goal is not reached

DO remove the first path from the QUEUE;

create new paths (to all children);
reject the new paths with loops;
add the new paths to back of QUEUE;

3. IF goal reached ONLY

THEN success; DIFFERENCE !
ELSE failure;
 2. Breadth-first: Example
• Consider the following map:

A 4 B 4 C
3
S 5 5 G
4 D E F 3
2 4

Use breadth first algorithm to find a path from S to G

 Breadth-first:
Example
(S) S removed, (SA,SD) computed and added
(SA, SD) SA removed, (SAB,SAD,SAS) computed,
(SAB,SAD) added
(SD,SAB,SAD) SD removed, (SDA,SDE,SDS) computed,
(SDA,SDE) added
(SAB,SAD,SDA,SDE) SAB removed, (SABA,SABE,SABC)
computed, (SABE,SABC) added
(SAD,SDA,SDE,SABE,SABC) SAD removed, (SADS,SADA, SADE)
computed, (SADE) added
etc, until QUEUE contains:
(SABED,SABEF,SADEB,SADEF,SDABC,SDABE,SDEBA,SDEBC, SDEFG)
goal is reached: reports success
 Evaluating breadth-first
Advantages
1.Complete: guarantees to find solution even
without our loop-checking.
2.Optimal : Always finds the shortest path (solution).

Limitations
1 .Memory demanding : it Is Very demanding on
memory.

2. Speed: Very slow since If a goal node is found on

depth m of the tree, all nodes up till that depth
 3. Non-deterministic search:
1. QUEUE <-- path only containing the root;
2. 2. WHILE QUEUE is not empty
AND goal is not reached

DO remove the first path from the QUEUE;

create new paths (to all children);
reject the new paths with loops;
add the new paths in random places in QUEUE;

3. IF goal reached
THEN success;
ELSE failure;
4. Iterative deepening Search
•Restrict a depth-first search to a
fixed depth.

•If no path was found, increase

the depth and restart the search.

•Also called Depth-limited search

4. Iterative deepening Search
1. DEPTH <-- <some natural number>
QUEUE <-- path only containing the root;

2. WHILE QUEUE is not empty

AND goal is not reached

DO remove the first path from the QUEUE;

IF path has length smaller than DEPTH
create new paths (to all children);
reject the new paths with loops;
add the new paths to front of QUEUE;
3. IF goal reached
THEN success;
ELSE failure;
5. Bi-directional Search
•Compute the tree from the start
node and from a goal node, until
these meet.

•It involves searching from two

directions

•Hence the name bi-directional

 Bi-directional search
• IF you are able to EXPLICITLY describe the GOAL
state, AND you have BOTH rules for FORWARD
reasoning AND BACKWARD reasoning:

Goal
Start
Bi-directional pseudocode:
1. QUEUE1 <-- path only containing the root;
QUEUE2 <-- path only containing the goal;

2. WHILE both QUEUEi are not empty

AND QUEUE1 and QUEUE2 do NOT share a state

DO remove their first paths;

create their new paths (to all children);
reject their new paths with loops;
add their new paths to back;

3. IF QUEUE1 and QUEUE2 share a state

THEN success;
ELSE failure;
 Uniform cost search

• This methods works by expanding the

lowest cost node on the fringe (edge)
where the cost is the accumulated
cost, g(n).

• breadth first search is a uniform cost

search with g(n) = DEPTH(n).

• Is also called uniformed best-first

 Uniform cost search

S
3 4 • At each step,
select the node
4
4 A 3 5 D with the lowest
5 2 accumulated
cost.
B 7 D 8 A9 E 6
4 5 5 4
5 4
C 11 E 12 E 13 B 13 B 11 F 10
3
D F B F C E A C G 13

G C G F
G
 Uniform cost search
• Consider the following problem.
• 
•

Question: use the following algorithms find the route from S to G;

1. Depth first
2. Breath first
3. Bi-directional
2. Uniform cost search
 Uniform cost search: Example
• That is S is the initial state and G is the goal state.
• A is the first node to be expanded at level 1.
• (SA,SB,SC)

• This leads to the following tree

 Uniform cost search: Example
• The path cost of A is the cheapest, so it is
expanded next.
• Remove SA (SAG,SB,SC)
• 11 5 15
• This gives the following tree.
Uniform cost search:
Example
• The algorithm order the queue by the path cost so the node
with cost 11 will be behind node B in the queue.
Node B (being the cheapest) is now expanded, giving
• (SB, SAG,SC) Remove SB (SBG)
• 5 11 15 10
Uniform cost search: Example
• A goal state (G) will now be at the front of the
queue. Therefore the search will end and the
path SBG will be returned.
• ( SBG,SAG,SC)
• 10, 11, 15

• Solution: SBG
Tutorial 1.
• Consider the following map
Uniform cost search exercise
Use the following to search the
path from A to p.
•1 uniform cost search method
•2. depth first
•3. breadth first
•4. bi-directional search
•3. iterative depth limited
 Exercise
A 4 B 4 C
3
S 5 5 G
4 D E F 3
2 4
•Use the following blind search methods
•To find the path from S to G in the above map.
•1. uniform cost search method
•2. bi-directional search method
•3. iterative depth limited search
Informed Searching
Techniques
Concept of informed (heuristic) search
• Involves exploring the node that is most likely to be nearest to
a goal state.
• There is no guarantee that the heuristic provided will get you
closer to a goal state than any other.
• Also called informed search
heuristics Example
• An office (goal) in a city building

• Knowledge
• Building name e.g. KICC, NSSF?
• Wing,
• floor no
• Street etc

• Some streets in Nairobi are named in alphabetical order

heuristics Example

• Example:
Patient visit the doctor and specify symptoms.
• Knowledge : fever, nausea, headache, …
Knowledge (heuristics) example
• Example:
• Climbing a hill in thick Fog
• Heuristic function checks the change in altitude in 4 directions;
• the strongest increase is the direction in which to move next.
Heuristic (informed) Searches - Characteristics
1. They require some domain knowledge (heuristics)
2. Usually more efficient than blind searches
3. There is no guarantee that the next node to be expanded
is the best node
Heuristic Searches - Why Use?
Advantage:
They are efficient in terms of both time and space.
Unlike blind search methods that are too resource intensive
(inefficient)
Heuristic search
Disadvantages:
1. There is no guarantee that the heuristic function will return a
value that ensures that a particular node is closer to a goal
state than any other node.
2. heuristics may not be available
Heuristic functions
• Heuristic function is a function that tells us how close we are
to a goal state
• That is, the heuristic function give an estimate of how close
the current state is to the goal state.
Uniform cost vs heuristic functions
• Uniform cost method calculates the cheapest cost so far
(accumulated cost).
• This cost is denoted by g(n)
• On the other hand, heuristic function tries to estimate how
close we are from the current state to the goal state
• This estimate is denoted by h(n)
Heuristic Functions Example
• We can develop a heuristic function that helps us to find a
solution to the Romanian route finding problem.
• One possible heuristic is based on the straight line distance (as
the crow flies) between two cities
Heuristic Functions Example
Heuristic Functions Example
• The route to your final destination may take a winding road so,
although the Straight Line Distance is shorter, the actual
distance to travel is longer.
• E.g SLD for arad to sibiu =140
• But the actual distance might be longer than 140.
• This is because you might follow a wind road to Sibiu

• But as a general strategy using the SLD heuristic is likely to find

a solution faster than a blind search.
 Example 2: road map
• Imagine the problem of finding a route on a road map and that
the NET below is the road map:

A 4 B 4 C
3
S 5 5 G
4 D E F 3
2 4
• Define f(T) = the straight-line distance from T
to G A 10.4 B 6.7 C 4
11 The estimate
S
G can be wrong!
8.9
D E 6.9 F 3
Heuristic Search Methods
• Methods that use a heuristic function to provide specific
knowledge about the problem:
• Examples:
• Hill climbing method
• Greedy search method
• A* search Method
Hill climbing
• This method tries to find a point in the search space that is
better than all the others.
• That is a solution state of better quality than all the others.
• The search Moves in direction of best value
• The best value can be the highest or the lowest value.
• It depends on the search problem
Hill climbing
• Hill climbing is a combination of depth-first and heuristic
functions
Hill climbing algorithm
1. Pick a random point in the search space.
2. Consider all the neighbours of the current state.
3. Choose the neighbour with the best quality and move to that
state.
4. Repeat 2 thru 4 until all the neighbouring states are of lower
quality.
5. Return the current state as the solution state.
 Hill climbing example1

• Example: using the straight-line distance:

• Perform depth-first, BUT:
S • instead of left-to-right
selection,
10.4
A
8.9
D • FIRST select the child with
the best heuristic value

10.4
A 6.9
E

6.7 B F 3.0

G
Problems of hill climbing

• Local maxima
• Plateau
• Ridges
local maxima problem
• When we reach a position where there are no better
neighbours, it is not a guarantee that we have found the best
solution state. It might be a small hillock instead of at the top
of the mountain rang

Foothills:

Local
maximum
local maxima problem
• This small hillock (peak) is called local maxima.
• It has no better neighbours but there are better solution states
elsewhere in the search space.
Hill climbing algorithm problems
local maxima problem
• One solution to this problem is to conduct a multi-start hill-
climb
• This consists of a series of hill climbing searches, each one
starting at a random point in the search space.
• The best result is chosen from all the searches and is returned
as the solution.
2. Plateau
• This is an area where the search space is flat so that all
neighbours return the same evaluation.
• The solution of this problem is to conduct a random walk or
jump until you find an area that starts to give better quality
solutions
2. Plateau
• If there are no neighbours of better quality it is assumed that
we are on the top of table top mountain

Plateaus
Ridges:
• When the search has reached on top of a ridge ,all directions fixed
in advance move downwards from the current position surface.

The solution of this problem is to set new rules, which are more directly
targeted to the goal, irrespective of the value returned .

Ridges
Heuristic Searches - Greedy search
• So named because it takes the biggest “bite” it can out of the
problem.
It expands the node that is judged to be closest to the goal
state.
• To make this judgement it uses the heuristic function, h(n).

in other words,
• Always expand the heuristically best nodes first.
 Greedy search, or heuristic best-first search:

S 30

10 40
1 20 • At each step, select

3
the node with the
best (in this case:
lowest) heuristic
35 15 value.

2 27 18

25 45 The ‘open’
nodes
 Greedy search algorithm:

1. QUEUE <-- path only containing the root;

2. WHILE QUEUE is not empty

AND goal is not reached

DO remove the first path from the QUEUE;

create new paths (to all children);
reject the new paths with loops;
add the new paths and sort the entire QUEUE;
(HEURISTIC)
3. IF goal reached
THEN success;
ELSE failure;
Greedy Search Limitations

• It is possible to get stuck in an infinite loop, unless you check for

repeated states
• It is not optimal
• It is not complete
A* search Algorithm
• This is a combination of Greedy search and Uniform cost search. - to
compliment one another
• This search method minimises the cost to the goal using an heuristic
function, h(n).
• Greedy search can considerably cut the search time but it is neither
optimal nor complete.
• By comparison uniform cost search minimises the cost of the path
so far, g(n).
• Uniform cost search is both optimal and complete but can be very
inefficient.
Road map example:
Re-introduce the costs of paths in the
NET
A 4 B 4 C
3
S 5 5
G
4 D 2 E F 3
4
3 S 4
4 A D
5 5 2
B D A E
4 5 2 4 5 4
C E E B B F
2 4 5 4 4 5 4 4 3
D F B F C E A C G
3 4 3 4
G C G F
3
G
 A look at Uniform cost search = uniformed
best-first

S
3 4
4
4 A 3 5 D • At each step,
5 2 select the node
B 7 D 8 A 9 E 6 with the lowest
4 5 5 4 accumulated
5 4
cost.
C 11 E 12 E 13 B 13 B 11 F 10
3
D F B F C E A C G 13

G C G F
G
 Now incorporate heuristic estimates
• Replace the ‘accumulated cost’ in the ‘Uniform cost search’ by a function:

f(path) = cost(path) + h(endpoint_path)

fn g(n) h(n)
• where:
cost(path) .g(n) = the accumulated cost of the
partial path
h(T) = a heuristic estimate of the cost
remaining
from T to a goal node

f(path) = an estimate of the cost of a path

extending the current path to reach a goal.
Heuristic Searches - A* Algorithm

• Combines the cost so far and the heuristic estimated cost to the
goal. That is fn = g(n) + h(n)
This gives us estimated cost of the cheapest solution through path n

• It can be proved to be optimal and complete provided that the

heuristic is admissible.
That is the heuristic must never over estimate the cost to reach the
goal.
 Example: road map
• Imagine the problem of finding a route on a road map. The paths
distances between nodes define g(n):

4 4
3 A B C
S 5 5
G
4 D 2 E F 3
4
• Define h(n) = the straight-line distance from
node to G A 10.4 B 6.7 C 4
11
S
G
8.9
D E 6.9 F 3
 S
3 + 10.4 = 13.4 A D 4 + 8.9 = 12.9

S
A 13.4 D
9 + 10.4 = 19.4 A E 6 + 6.9 = 12.9
X
S
A 13.4 D
A E
X B F 10 + 3.0 = 13.0
11 + 6.7 = 17.7
S
A 13.4 D
A
X
E STOP!
17.7 B F

G 13 + 0.0 = 13.0
Summary
• Blind Search is very expensive
• A Search with info (heuristics) is more efficient
END