Beyond Classical Search

Bùi Tiến Lên

2022
Contents

1. Hill-climbing Search

2. Simulated Annealing

3. Local Beam Search

4. Genetic Algorithms

5. Local Search in Continuous Spaces

6. Searching With Nondeterministic Actions

 Optimization problems
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • In many optimization problems, path is irrelevant; the goal state itself is the
Genetic
Algorithms
solution
Local Search in • “Pure optimization” problems
Continuous
Spaces • All states have an objective function
Searching With • Goal is to find state with max (or min) objective value
Nondeterminis-
tic Actions • Does not quite fit into path-cost/goal-state formulation
• Local search can do quite well on these problems.
• Then state space = set of “complete” configurations
• find optimal configuration, e.g., TSP
• find configuration satisfying constraints, e.g., timetable

3
 Local search algorithms
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Local search
Genetic
Algorithms
• Keep track of single current state
Local Search in
• Move only to neighboring states
Continuous
Spaces
• Ignore paths
Searching With • Advantages:
Nondeterminis-
tic Actions • Use very little memory
• Can often find reasonable solutions in large or infinite (continuous) state
spaces.

4
 State-space landscape
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• An useful landscape has both “location” (defined by the state) and
Genetic “elevation” (defined by the value of the heuristic cost function or objective
function)
Algorithms

Local Search in
Continuous objective function
Spaces global maximum

Searching With
Nondeterminis-
tic Actions shoulder

local maximum
“flat” local maximum

state space
current
state

Figure 1: A one-dimensional state-space landscape in which elevation corresponds to the

objective function
5
Hill-climbing Search
 Hill-climbing Search
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • The hill-climbing search algorithm (steepest-ascent version) is simply a
Genetic
Algorithms
loop that continually moves in the direction of increasing value (uphill). It
Local Search in
terminates when it reaches a “peak” where no neighbor has a higher value.
Continuous
Spaces
• The algorithm does not maintain a search tree, so the data structure for the
Searching With current node need only record the state and the value of the objective
function.
Nondeterminis-
tic Actions

• Hill climbing does not look ahead beyond the immediate neighbors of the
current state

7
 Algorithm
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
function Hill-Climbing(problem)
Genetic
Algorithms
returns a state that is a local maximum
Local Search in
current ← Make-Node(problem.Initial-State)
Continuous loop do
Spaces
neighbor ← a highest-valued successor of current
Searching With
Nondeterminis- if neighbor .Value ≤ current.Value then return current.State
tic Actions current ← neighbor

• At each step the current node is replaced by the best neighbor; in this
version, that means the neighbor with the highest Value, but if a heuristic
cost estimate h is used, we would find the neighbor with the lowest h.
• Hill-climbing algorithms typically choose randomly among the set of best
successors if there is more than one.

8
 Illustration
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Find a state with the highest value.
Genetic
Algorithms
a 10
Local Search in
Continuous c 8
Spaces b 9
Searching With
Nondeterminis- f 15
tic Actions e 6
d 1 j 12
i 7
h 5

g 3
l 6

k 4

9
 Example
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• Find a state with the highest value in a grid problem.
Genetic
Algorithms

Local Search in
Continuous
Spaces

Searching With
Nondeterminis-
tic Actions

10
 Knapsack Problem
Hill-climbing
Search

Simulated
Annealing

?
Local Beam
Search

Genetic $4 kg
Algorithms 12

$2 g
Local Search in 2k
Continuous 15 kg
Spaces

Searching With $2 g
1k
Nondeterminis-
tic Actions $1 1 kg

$10 g
4k

• Configurations: Any combination of objects inside the knapsack

• Initial configuration: Empty knapsack
• Actions: Put objects in and take objects from the knapsack
• Best configuration: A configuration with max valuei
P
11
 8-queens Problem
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Complete-state formulation
Genetic
Algorithms
• All 8 queens on the board, one per column
Local Search in • Successor function
Continuous
Spaces • Move a single queen to another square in the same column 8 × 7 = 56
Searching With
Nondeterminis-
successors
tic Actions
• Heuristic cost function h(n)
• The number of pairs of queens that are attacking each other
• Global minimum has h(n) = 0

12
 8-queens Problem (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
18 12 14 13 13 12 14 14
Genetic
Algorithms 14 16 13 15 12 14 12 16
Local Search in
14 12 18 13 15 12 14 14
Continuous
Spaces 15 14 14 13 16 13 16
Searching With 14 17 15 14 16 16
Nondeterminis-
tic Actions 17 16 18 15 15
18 14 15 15 14 16
14 14 13 17 12 14 12 18

(a) (b)

Figure 2: (a) An 8-queens state with heuristic cost estimate h = 17, showing the value of
h for each possible successor obtained by moving a queen within its column. The best
moves are marked. (b) A local minimum in the 8-queens state space; the state has h = 1
but every successor has a higher cost. It takes just 5 steps from state (a) to state (b)
13
 Problems
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Hill climbing often gets stuck for the following reasons:
Genetic
Algorithms
• Local maxima: a local maximum is a peak that is higher than each of
Local Search in its neighboring states but lower than the global maximum
Continuous
Spaces
• Ridges: ridges result in a sequence of local maxima that is very difficult
Searching With for greedy algorithms to navigate.
Nondeterminis-
tic Actions
• Plateaux: a plateau is a flat area of the state-space landscape.
• In each case, the algorithm reaches a point at which no progress is being
made. Starting from a randomly generated 8-queens state, hill climbing gets
stuck 86% of the time, solving only 14% of problem instances.
• It takes 4 steps on average for each successful instance and 3 for each
failure

14
 Possible solution
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • If no uphill (downhill) moves, allow sideways moves in hope that algorithm
Genetic
Algorithms
can escape local maximum
Local Search in • A limit on the possible number of sideways moves required to avoid infinite
Continuous
Spaces loops
Searching With • 8-queens problem
Nondeterminis-
tic Actions • Allow sideways moves up to 100 → percentage of problem instances
solved raises from 14 to 94%
• It takes 21 steps on average for each successful instance and 64 for each
failure

15
 Variants of hill climbing
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Stochastic hill climbing
Genetic
Algorithms
• Choose at random from among the uphill moves with a probability of
Local Search in selection varied with the moves’ steepness
Continuous
Spaces
• Usually converge more slowly than steepest ascent, but find better
Searching With solutions in some cases
Nondeterminis-
tic Actions • First-choice hill climbing
• Generate successors randomly until better than the current state
• Good strategy when a state has many successors (e.g., thousands)

16
 Variants of hill climbing (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Random-restart hill climbing: “If at first you don’t succeed, try, try again.”
Genetic
Algorithms
• A series of hill-climbing searches from randomly generated initial states
Local Search in until a goal is found.
Continuous
Spaces
• If each hill-climbing search has a probability p of success, then the
Searching With expected number of restart 1/p
Nondeterminis-
tic Actions • Random-walk hill climbing: At each step do one of the two
• Greedy: With probability p move to the neighbor with largest value
• Random: With probability 1 − p move to a random neighbor

17
Simulated Annealing
 Simulated Annealing
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Simulated Annealing = physics inspired twist on random walk
Genetic
Algorithms
• A metal alloy or dissolution is heated at high temperatures and progressively
Local Search in cooled in a controlled way
Continuous
Spaces • If the cooling process is adequate the minimal state of energy of the system is
Searching With
Nondeterminis-
achieved (global minimum)
tic Actions

19
 Algorithm
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
function Simulated-Annealing(problem,schedule)
Genetic
Algorithms
returns a solution state
Local Search in
inputs: problem, a problem
Continuous schedule, a mapping from time to "temperature"
Spaces
current ← Make-Node(problem.Initial-State)
Searching With
Nondeterminis- for t = 1 to ∞ do
tic Actions T ← schedule(t)
if T = 0 then return current
next ← a randomly selected successor of current
∆E ← next.Value − current.Value
if ∆E > 0 then current ← next
else current ← next only with probability e T
∆E

20
 Physical Interpretation of Simulated Annealing
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • A Physical Analogy:
Genetic
Algorithms
• imagine letting a ball roll downhill on the function surface →this is like
Local Search in hill-climbing (for minimization)
Continuous
Spaces
• now imagine shaking the surface, while the ball rolls, gradually reducing
Searching With the amount of shaking → this is like simulated annealing
Nondeterminis-
tic Actions • Annealing = physical process of cooling a liquid or metal until particles
achieve a certain frozen crystal state
• free variables are like particles
• seek “low energy” (high quality) configuration
• slowly reducing temp. T with particles moving around randomly

21
 TSP Problem
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Given a list of cities and the distances
Genetic
Algorithms
• What is the shortest possible route {(x1 , y1 ), ..., (xn , yn )} that visits each city
Local Search in exactly once and returns to the origin city?
Continuous
Spaces

Searching With
Nondeterminis-
tic Actions

22
 TSP Problem
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Intial a list of cities
Genetic • An energy function (sum of the distance among the cities, following the order
Algorithms

Local Search in
in the list)
Continuous
n p
f = (xi − xi+1 )2 + (yi − yi+1 )2 (1)
X
Spaces

Searching With
Nondeterminis-
i=1
tic Actions • Action: swap two cities
{1, 5, 2, 3, 4} {1, 5, 3, 2, 4}

1 2 1 2

Swap(2,3)
5 5
3 3

4 4

23
Local Beam Search
 Local Beam Search
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search Keep track of k states rather than just one (extreme reaction to memory
Genetic problems)
Algorithms
• Begin with k randomly generated states
Local Search in
Continuous
Spaces
• At each step, all successors of all k tates are generated
Searching With • If any of successors is goal → finished
Nondeterminis-
tic Actions • Else select k best from successors and repeat
S

a c d b

e g f i j h

k l m n p

25
 Local Beam Search (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• Not the same as k random-start searches run in
Genetic
Algorithms parallel!
Local Search in
Continuous
• In its simplest form, local beam search can suffer
Spaces from a lack of diversity among the k states
Searching With
Nondeterminis- • They can quickly become concentrated in a
tic Actions
small region of the state space → an expensive
version of hill climbing
• Stochastic beam search
• Choose k successors at random, with the
probability of choosing a given successor being
an increasing function of its value

26
Genetic Algorithms
 Genetic algorithms
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• A genetic algorithm (or GA) is a variant of stochastic beam search in which
Genetic successor states are generated by combining two parent states rather than by
modifying a single state.
Algorithms

Local Search in
Continuous • The analogy to natural selection is the same as in stochastic beam search,
Spaces
except that now we are dealing with sexual rather than asexual reproduction.
Searching With
Nondeterminis-
tic Actions

28
 Genetic algorithms (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • The GA begins with a set of k randomly generated states, called the
Genetic
Algorithms
population.
Local Search in • Each state is rated by the objective function, called the fitness function.
Continuous
Spaces • Higher values for better state
Searching With
Nondeterminis-
• Produce the next generation by “simulated evolution”
tic Actions
• Random selection
• Crossover
• Random mutation

29
 Algorithm
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search function Genetic-Algorithm(population,Fitness-Fn) returns an individual
Genetic inputs: population, a set of individuals
Algorithms Fitness-Fn, a function that measures the fitness of an individual
Local Search in repeat
Continuous new_population ← ∅
Spaces for i = 1 to SIZE(population) do
Searching With x ← Random-Selection(population,Fitness-Fn)
Nondeterminis- y ← Random-Selection(population,Fitness-Fn)
tic Actions
child ← Reproduce(x,y)
if (small random probability) then child ← Mutate(child)
add child to new_population
population ← new_population
until some individual is fit enough, or enough time has elapsed
return the best individual in population, according to Fitness-Fn
function Reproduce(x, y) returns an individual
inputs: x,y, parent individuals
n ← Length(x);
c ← random number from 1 to n
return Append(SubString(x, 1, c),SubString(y, c + 1, n))

30
 Illustration
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• Representation of Individuals
Genetic • Each state, or individual, is represented as a string over a finite alphabet
– most commonly, a string of 0s and 1s.
Algorithms

Local Search in
Continuous • Alternatively, the state could be represented as 8 digits, each in the
Spaces
range from 1 to 8.
Searching With
Nondeterminis-
tic Actions 24748552 24 31% 32752411 32748552 32748152
32752411 23 29% 24748552 24752411 24752411
24415124 20 26% 32752411 32752124 32252124
32543213 11 14% 24415124 24415411 24415417

(a) (b) (c) (d) (e)

Initial Population Fitness Function Selection Crossover Mutation

Figure 3: The genetic algorithm, illustrated for digit strings representing 8-queens states.
The initial population in (a) is ranked by the fitness function in (b), resulting in pairs for
mating in (c). They produce offspring in (d), which are subject to mutation in (e). 31
 Illustration (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search

Genetic
Algorithms

Local Search in
Continuous + =
Spaces

Searching With
Nondeterminis-
tic Actions

Figure 4: The 8-queens states corresponding to the first two parents in Figure 3(c) and
the first offspring in Figure 3(d). The shaded columns are lost in the crossover step and
the unshaded columns are retained.

32
Local Search in Continuous Spaces
 Optimization of Continuous Functions
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search

Genetic • Discrete environments

Algorithms

Local Search in
• Use hill-climbing
Continuous
Spaces

Searching With
Nondeterminis-
tic Actions

• Continuous environments
• Use gradient descent Objective function f(x,y)
sin(x) * cos(y)
0.5
0
1 -0.5
0.5
0
-0.5
-1
3
2
1
-3 -2 0
-1 0 -1
1 2 -2
3 -3
34
 Example
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Suppose we want to site three airports in Romania:
Genetic • 6-D state space defined by (x1 , y2 ), (x2 , y2 ), (x3 , y3 )
Algorithms
• Objective function f (x) = f (x1 , y2 , x2 , y2 , x3 , y3 ) = sum of squared
Local Search in
Continuous distances from each city to nearest airport
Spaces
Oradea
Searching With 71
Neamt
Nondeterminis-
tic Actions Zerind 151
((x2 , y2 ) 87
75
Iasi
Arad
140
92
Sibiu
99 Fagaras
((x3 , y3 )
118
((x1 , y1 )
Vaslui
80
Rimnicu Vilcea
Timisoara
142
111 Pitesti 211
Lugoj 97
70 98
85 Hirsova
Mehadia 146 101 Urziceni
75 138 86
Bucharest
Drobeta 120
90
Craiova Eforie
Giurgiu

35
 Gradient descent
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Discretization methods turn continuous space into discrete space
Genetic
Algorithms
• E.g., empirical gradient considers ±δ change in each coordinate
Local Search in • Gradient descent methods using
Continuous
Spaces  
Searching With
∂f ∂f ∂f ∂f ∂f ∂f
Nondeterminis- ∇f = , , , , ,
tic Actions ∂x1 ∂y1 ∂x2 ∂y2 ∂x3 ∂y3
• Init x ← x 0
• To reduce f , update x
x ← x − η∇f
where η is learning rate (0 < η < 1)

36
 Gradient descent (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search • Newton–Raphson methods update x
Genetic

x ← x − ηH −1 ∇f
Algorithms

Local Search in
Continuous
Spaces
where H is a Hesian matrix
Searching With
Nondeterminis- ∂2f ∂2f ∂2f
 
tic Actions
∂x1 ∂x1 ∂x1 ∂x2 ··· ∂x1 ∂xn
∂2f ∂2f ∂2f
···
 
H= ∂x2 ∂x1
..
∂x2 ∂x2
..
∂x2 ∂xn
..
 
..

. . . .
 
 
∂2f ∂2f ∂2f
∂xn ∂x1 ∂xn ∂x2 ··· ∂xn ∂xn

37
 Gradient descent (cont.)
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search

Genetic
Algorithms

x0
Local Search in
Continuous

x1
Spaces
x2
Searching With
Nondeterminis-
tic Actions

Figure 5: xi approach to the optimal point

38
Searching With Nondeterministic Actions
 The erratic vacuum world
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search In the erratic vacuum world, the Suck action works as follows:
Genetic • When applied to a dirty square the action cleans the square and sometimes
Algorithms

Local Search in
cleans up dirt in an adjacent square, too.
Continuous
Spaces
• When applied to a clean square the action sometimes deposits dirt on the
Searching With carpet.
Nondeterminis-
tic Actions
1 2

3 4

5 6

7 8

Figure 6: The eight possible states of the vacuum world; states 7 and 8 are goal states.

40
 AND–OR search trees
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search
• In this problem, a Results function that returns a set of possible outcome
Genetic states
Algorithms
• The solution to the problem is not a sequence but a contingency plan (also
Local Search in
Continuous known as a strategy) that specifies what to do depending on what percepts
Spaces
are received
Searching With
Nondeterminis-
1

tic Actions
Suck Right

7 5 2

GOAL Suck Right Left Suck

5 1 6 1 8 4

LOOP LOOP Suck Left LOOP GOAL

8 5

GOAL LOOP

Figure 7: The first two levels of the search tree for the erratic vacuum world. State nodes
are OR nodes where some action must be chosen. At the AND nodes, shown as circles,
every outcome must be handled, as indicated by the arc linking the outgoing branches.
The solution found is shown in bold lines. 41
 Alogrithm
Hill-climbing
Search

Simulated
Annealing

Local Beam
Search function And-Or-Graph-Search(problem) returns a conditional plan, or failure
Or-Search(problem.Initial-State, problem, ∅)
Genetic
Algorithms function Or-Search(state, problem, path) returns a conditional plan, or failure
if problem.Goal-Test(state) then return the empty plan
Local Search in if state is on path then return failure
Continuous for each action in problem.Actions(state) do
Spaces plan ← And-Search(Results(state, action), problem, [state | path])
if plan 6= failure then return [action | plan]
Searching With
return failure
Nondeterminis-
tic Actions function And-Search(states, problem, path) returns a conditional plan, or failure
for each si in states do
plani ← Or-Search(si , problem, path)
if plani = failure then return failure
return [if s1 then plan1
else if s2 then plan2
else if ...
else if sn−1 then plann−1
else plann ]

42
References

Goodfellow, I., Bengio, Y., and Courville, A. (2016).

Deep learning.
MIT press.
Lê, B. and Tô, V. (2014).
Cở sở trí tuệ nhân tạo.
Nhà xuất bản Khoa học và Kỹ thuật.
Nguyen, T. (2018).
Artificial intelligence slides.
Technical report, HCMC University of Sciences.
Russell, S. and Norvig, P. (2016).
Artificial intelligence: a modern approach.
Pearson Education Limited.