Lecture Note

Artificial Intelligence “Search”

Professor dr. Hiroyuki Iida

JAIST

2012-11-8 1
 A* search algorithm
http://en.wikipedia.org/wiki/A*_search_algorithm

In computer science, A* (pronounced "A star") is a

computer algorithm that is widely used in pathfinding and
graph traversal, the process of plotting an efficiently
traversable path between points, called nodes. Noted for its
performance and accuracy, it enjoys widespread use.
Peter Hart, Nils Nilsson, and Bertram Raphael first described
the algorithm in 1968[1]. It is an extension of Edsger Dijkstra's
1959 algorithm. A* achieves better performance (with respect
to time) by using heuristics.
Q3-1. What is Dijkstra's algorithm?
http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm

2
 Description
A* uses a best-first search and finds the least-cost path
from a given initial node to one goal node (out of one or
more possible goals).
It uses a distance-plus-cost heuristic function (usually
denoted f(x)) to determine the order in which the search
visits nodes in the tree. The distance-plus-cost heuristic is
a sum of two functions:
　・ the path-cost function, which is the cost from the
starting node to the current node (usually denoted g(x))
　・ and an admissible "heuristic estimate" of the distance
to
　　 the goal (usually denoted h(x)
Q3-2. What is the best-first search?
http://en.wikipedia.org/wiki/Best-first_search

3
 The h(x) part of the f(x) function must be an
admissible heuristic; that is, it must not overestimate the
distance to the goal. Thus, for an application like routing,
h(x) might represent the straight-line distance to the goal,
since that is physically the smallest possible distance
between any two points or nodes.
If the heuristic h satisfies the additional condition
for every edge x, y of the graph (where d
denotes the length of that edge), then h is called
monotone, or consistent. In such a case, A* can be
implemented more efficiently—roughly speaking, no node
needs to be processed more than once (see closed set
below)—and A* is equivalent to running Dijkstra's algorithm
with the reduced cost d'(x,y): = d(x,y) − h(x) + h(y).
Note that A* has been generalized into a bidirectional
heuristic search algorithm; see bidirectional search.
4
 History of A*
In 1964 Nils Nilsson invented a heuristic based approach to
increase the speed of Dijkstra's algorithm. This algorithm was
called A1.

In 1967 Bertram Raphael made dramatic improvements upon

this algorithm, but failed to show optimality. He called this
algorithm A2.

Then in 1968 Peter E. Hart introduced an argument that proved

A2 was optimal when using a consistent heuristic with only minor
changes. His proof of the algorithm also included a section that
showed that the new A2 algorithm was the best algorithm
possible given the conditions. He thus named the new algorithm
in Kleene star syntax to be the algorithm that starts with A and
includes all possible version numbers or A*. [2]
5
 AND/OR-Tree Search
Using shogi as a testbed for search
(endgame solver)

derived from a collaboration with Makoto Sakura

6
A traditional shōgi-ban (shogi board) displaying a set of koma (pieces).
The pieces on the far side are turned to show their promoted values.
The stands on either side are komadai used to hold captured pieces.
The board itself is raised for the comfort of players seated on tatami
mats (background), and is hollowed underneath to produce a pleasing
sound when the pieces are moved
7
Game tree and AND/OR Tree
http://en.wikipedia.org/wiki/Game_tree

If you're looking for game tree as it's used in game theory (not
combinatorial game theory), please see Extensive-form game.
In combinatorial game theory, a game tree is a directed graph whose
nodes are positions in a game and whose edges are moves. The
complete game tree for a game is the game tree starting at the initial
position and containing all possible moves from each position.

The first two ply of the game tree for tic-tac-toe.

The diagram shows the first two levels, or ply, in the game tree for tic-
tac-toe. We consider all the rotations and reflections of positions as
being equivalent, so the first player has three choices of move: in the
center, at the edge, or in the corner. The second player has two choices
for the reply if the first player played in the center, otherwise five
choices. And so on.
8
 Overview
• Tsume-shogi (mating search) problem is a
good application for AND/OR-tree search
algorithms
• Algorithms for solving tsume-shogi
– ID, AO*, PN search, PN*, PDS and DF-PN
– Equivalent relations between these algorithms
• Tsume-shogi in actual shogi endgame
– Find a relatively long-step winning sequence
Q3-3. What is the difference between minimax
game tree search and AND/OR-tree search?
9
 Main rules of tsume-shogi
• Two agents: attacker and defender
• Attacker has to continue the check move sequences.
• Attacker must mate in the shortest sequence.
• Defender must select a move which makes the
mating sequence as long as possible.
• The length of the solution sequence is not given in
advance.
• See also [Seo, Iida and Uiterwijk, 2001]

Q3-4. What is the PN* algorithm?

Artificial Intelligence
Volume 129, Issues 1-2, June 2001, pages 253-277
10
 Why tsume-shogi is a good application
for AND/OR-tree search?

• Tsume-shogi has an enormous search space, so

simple brute-force search is not adequate.
• A tsume-shogi problem has a unique solution.
• There are various types of problems, including those
very hard to solve and those having a very long
solution sequence.
• Many high-performance tsume-shogi solving
programs have been developed, so the performance
of a program can be measured and compared.
11
Examples of solving tsume-shogi(1)

• An endgame position from GM’s game.

(mate in 17 plies, relatively not so long)
– ID solved in 22.6 seconds.
– PN solved in 7.93 seconds.
– PDS solved in 6.11 seconds.
– PN* solved in 4.87 seconds.
• See the detail (machine spec, size of transposition
table, etc.) in [Sakuta and Iida, ACG9].

12
Examples of solving tsume-shogi(2)
• An endgame position from GM’s game.
(mate in 37 plies, relatively long in actual
games)
– ID was unable to solve.
– PN was unable to solve.
– PN* solved in 1000 seconds.
– PDS solved in 757 seconds.
• See the detail (machine spec, size of
transposition table, etc.) in [Sakuta and Iida,
ACG9].
13
 Why proof-number search?
• Best-first search algorithms such as proof-number (PN)
search work well when solving endgame positions in
chess-like games. (also other games: GO, LOA, etc.)
• However, it is hard, even for PN search, to solve an
endgame problem with very long sequences because
of its inherent memory limits.
• Recently, innovative depth-first search algorithms
based on proof/disproof numbers were developed
(PN*, PDS and DF-PN) for solving tsume-shogi
problems, among which one with 1525 steps.
• These depth-first algorithms behave like best-first
search.
14
 AND/OR-tree search
Best-first search
AO* (Nilsson) and its variants
Minimax tree
A* for OR Graph CN-search (McAllester, 1988)
(Schaeffer, 1990)
conspiracy number

PN-search (Allis,1994)
proof number
disproof
number 15
 PN-search (Allis, 1994)
• PN (proof number) search is a best-first AND/OR tree-search algorithm.
• Inspired by conspiracy-number search algorithm (McAllester, 1988).
• Before starting the search, a search goal is defined. E.g., try to mate.
The evaluation of a node returns one of three values: true, false, or
unknown. The evaluation is seen from the point of view of the player to
move in the root position. The value true indicates that the player to
move in the root position can achieve the goal, while false indicates that
the goal is unreachable.
• A node is proved if its value has been established to be true, whereas
the node is disproved if its value has been determined to be false.
• A node is solved as soon as it has been proved or disproved.
• A tree is solved (proved or disprove) if its root is solved.
• The goal of PN search is to solve a tree.

16
 PN-search (continue)
• Has high ability for solving AND/OR trees.
• However, it expands the whole search tree in working
memory and require much memory.
• This means that there is a severe memory restriction
for hard problems.
• The detail of PN search [Allis et al., 1994]
• PN^2 search (Breuker et al., 1999)
• Would it be possible to transform a best-first PN search
into a depth-first search?

17
 PN* (Seo, 1995)
• is recursive iterative-deepening depth-first
search algorithm using proof numbers as a
criterion to develop the frontier nodes.
• PN* starts searching by setting the proof-
number threshold to 1 and expands the
root node. If the proof number of the node
under consideration exceeds the threshold,
PN* stops node expansion at this node.
When all frontier nodes have proof
numbers exceeding the proof-number
threshold, the search tree is discarded, the
threshold is incremented and a new search
starts.
• This iterative searching is enhanced by
storing the expanded nodes and their
properties in the transposition table.

18
 PN* (continue)
• As the threshold is getting larger, node
expansion proceeds in such a way that nodes
having relatively small proof numbers are
expanded first, like in best-first search, without
the disadvantage of the memory requirement.
• PN* uses the method of recursive iterative
deepening. At an AND node the threshold
values assigned to the child OR nodes are
iteratively increased from 1 to their current
maximum.
• Enhanced by dynamic evaluation, efficient
successor ordering and pruning by
dependency relations.
• The detail of PN* [Seo, Iida and Uiterwijk,
2001]

19
 Recursive iterative deepening
iterative deepening at root node best-first search (e.g. pn-search)
threshold=2

threshold=3
frontier nodes
threshold=4

recursive iterative deepening iterative-deepening depth-first search

depth=1
threshold=2
depth=2 or 3

threshold=3 depth=3 or 5

threshold=4 depth=4 or 7

depth=5 or 9

20
 Equivalence (1): AO* and PN* 　
(Nagai, 1999)
• PN* (depth-first search) algorithm behaves the
same as AO* (best-first search) in the meaning
that PN* always expands a most-proving node.
• Most proving node (in the context of AO*) is a
leaf node selected by tracing from the root in the
following way.
• For each OR node, trace the child with minimum
proof number.
• For each AND node, trace any child with non-zero
proof number.

21
 Tsume-shogi solver: SEO
• based on the PN* algorithm
• excellent management of transposition table
• uses Dominance Relation of positions
• omits the useless moves of dropping pieces
• other domain-specific features
• solved “Microcosmos”, the problem with the
longest steps, i.e., 1525 plies (1997)

22
 PDS (Nagai,1998)
• PDS (Proof-number and Disproof-number Search ) is a
straight extension of PN*, using both proof numbers
and disproof numbers. PDS uses two thresholds in
searching, the proof-number threshold and the
disproof-number threshold.
• If the proof number or the disproof number exceeds
the threshold at a certain node, PDS stops node
expansion at this node. When PDS fails the root node
expansion, it increases the either threshold value and
restarts the searching.
• Note that the basic concept of PDS differs from PN
search, i.e., asymptotically equivalent to PN search.

23
 DF-PN (Nagai,1999)
• Depth-first version of PN-search
• DF-PN first sets, at the root node, the
thresholds of both proof number and
disproof number to a certain large value
(∞). Then, the threshold values are
distributed among the descendant nodes.
• At every node, recursive iterative
deepening is performed (cf. only OR
nodes in PN*)
24
 Equivalence (2): PN and DF-PN
(Nagai, 1999)
• DF-PN (depth-first search) algorithm behaves the
same as PN (best-first search) in the meaning that
DF-PN always expands a most-proving node.
• Most proving node (in the context of PN) is a leaf
node selected by tracing from the root in the
following way.
• For each OR node, trace the child with minimum
proof number.
• For each AND node, trace the child with minimum
disproof number.
25
 Tsume-shogi solver: NAGAI
• based on the DF-PN algorithm
• efficient use of transposition table
– SmallTreeGC and SmallTreeReplacement (Nagai, 99)
• pruning by dependency relations (positions and
proof/disproof numbers)
– More efficient than the solver SEO
• GHI problem (cf. Breuker et al., CG98)
• Proof pieces (Seo, 99) and disproof pieces
• detecting a DAG to avoid the double counting of
proof numbers

26
Performance of PN*, PDS and PN Search
(Sakuta and Iida, ACG9, 1999)
• Experiments in two domains:
– tsume-shogi (derived from GM’s games), and
– 6x6 othello (positions on PV)
• Comparison shows that
– In both, PDS outperforms PN*.
– In othello, PN search variants do not show
remarkable advantage over depth-first search.
– In shogi, PN search variants show remarkable
advantage over depth-first search.
• Forcing move sequences and sudden
termination.
• Disproving ability is important as well as
proving. 27
Disproving ability compared in
tsume-shogi
PN* PDS PN-search iterative
deepening
ex1p ○ ○ ○ ○
0.6 0.9 0.3 28.5
ex1d × ○ ○ ×
1.5 12.1
ex2p ○ ○ ○ ○
0.7 0.6 0.6 0.6

ex2d × ○ ○ ×
1.0 4.1

28
Four algorithms (equivalent)
Proof number Proof number and
disproof number

Best-first search AO* PN search

Depth-first search PN* Df-pn

29
 More detail comparison
Iterative
deepening
PN PN* PDS DF-PN

behaviour depth-first best-first depth-first depth-first depth-first

transposition yes basically indispensable indispensable indispensable

table
no
conspiracy no proof & proof proof & proof &
number
disproof disproof disproof
iterative depth no proof proof & proof &
deepening
threshold disproof disproof
recursive no no at OR at AND/OR at AND/OR
iterative
nodes nodes nodes
deepening
iteration at the yes yes proof proof & no
root node
threshold disproof
30
Advantages of depth-first-type algorithm
• If a best-first algorithm and a depth-first
algorithm behave in the same way, it is certainly
better to adopt the depth-first one.
– IDA* and A*, MT-SSS* and SSS*, and
– MTD(f) for minimax trees [Plaat et al., 1996]

• Many enhancements can be used:

– History heuristics, transposition table, iterative
deepening, and so on.
• Depth-first algorithm uses less memory space
– Search speed is often faster in practice

31
 Summary of experimental
performance
• PN* solved most hard problems,
among which one with 1525
steps. But, it was unable to solve
10 hard problems with long steps
over 300.
• PDS basically outperforms PN*
in solving long-step problems.
• DF-PN have solved all hard-
problems.

32
 Application to actual
endgames in shogi
• Advances of AND/OR tree search
algorithms enable a computer to:
– Find a mate under tournament time constraints,
at interior nodes (close to the root).
– Reuse the outcome of mating search in the
successive searches.
– Find a brink-mate (forcing mate threat)
– Play a high-level endgame (mating race).

33
 Conclusions
• Depth-first searches using proof/disproof
numbers have recently been developed.
– Basically depth-first search but behaves in best-first
manner like proof number search (PN search).
– Memory requirement is much less than best-first.
• DF-PN have solved all hard-problems of tsume-
shogi with long steps over 300.
• These algorithms are now used in shogi playing
programs.
– High-level mating race in the endgame.

Q3-5. What do you think any possibility of

applying the AND/OR tree search algorithm
to the composition of mating problems?
34