Optimized AO* Algorithm for AND-OR graph.
1st Rishavh Srivastava 2nd Robin Bisht
Department of Computer Science
NIT Uttarakhand
rishavh.cse17@nituk.ac.in
Abstract—The objective of this paper is to propose and
optimize AO* algorithm with a depth limited search. AO* is
one of the major AND-OR graph search algorithms. According
to AO* algorithm it is not exploring all the solution paths once it
has got the solution. Here the modified method is providing better
heuristic value for the solution if the search space is unstable. It
is also considering interacting sub problems in the in the search
space considering the loop structures.
Index Terms—AND-OR graph, best first search, depth limited
search, AO*, interacting sub problems, heuristic.
I. I NTRODUCTION
One of the most widely-used frameworks for problem-
solving in artificial intelligence is state-space search. A
state-space search problem is defined by a set of states, a set
of actions (or operators) that map states to successor states, a
start state, and a set of goal states. The objective is to find a
sequence of actions that transforms the start state into a goal
state, and also optimizes some measure of the quality of the
solution.
A form of graph or tree used in problem solving and problem
decomposition. The nodes of the graph represent states or
goals and their successors are labeled as either AND or OR
branches. The AND successors are subgoals that must all
be achieved to satisfy the parent goal, while OR branches
indicate alternative subgoals, any one of which could satisfy
the parent goal.The AND-OR graph (or tree) is useful for
representing the solution of problems that can solved by
decomposing them into a set of smaller problems, all of
which must then be solved. This decomposition, or reduction,
generates arcs that we call AND arcs. One AND arc may
point to any number of successor nodes, all of which must
be solved in order for the arc to point to a solution. Just as
in an OR graph, several arcs may emerge from a single node,
indicating a variety of ways in which the original problem
might be solved. This is why the structure is called not
simply an AND-graph but rather an AND-OR graph (which
also happens to be an AND-OR tree).
Department of Computer Science
NIT Uttarakhand
robin.cse17@nituk.ac.in
P
Q R S
T U
Fig. 1: An AND-OR Graph
Definition of AND-OR graph
Given an initial problem P0 and set of problem solving
methods of the form:
P if P1 and . . . and Pn
the associated and-or graph is a set of labelled nodes such
that:
1) The root of the tree is a node labelled by P0 .
2) For every node N labelled by a problem or sub-problem
P and for every method of the form: P if P1 and . . . and
Pn, there exists a set of children nodes: N1 , . . . , Nn of
the node N, such that each node Ni is labelled by Pi .
The nodes are conjoined by an arc, to distinguish them
from children of N that might be associated with other
methods.
A node N, labelled by a problem P, is a success node if there
is a method of the form P if nothing (i.e., P is a ”fact”). The
node is a failure node if there is no method for solving P.
If all of the children of a node N, conjoined by the same arc,
are success nodes, then the node N is also a success node.
Otherwise the node is a failure node.
In an AND-OR graph, AO* algorithm is an efficient
method to explore a solution path. AO* algorithm works
mainly based on two phases. First phase will find a heuristic
value for nodes and arcs in a particular level. The changes in
the values of nodes will be propagated back in the next phase.
There are situations like a non-promising node eventually
become a promising one. AO* algorithm will not give the
best solution in such cases. Here a new method is introduced
to consider such nodes by performing a depth limited search
before fixing the heuristic value. Also the proposed method
is handling interacting sub problems which is not handled by
AO* algorithm .
A. Heuristic Search Algortihms
1) Hill Climbing
Hill climbing algorithm is a local search algorithm which
continuously moves in the direction of increasing eleva-
tion/value to find the peak of the mountain or best solution to
the problem. It terminates when it reaches a peak value where
no neighbor has a higher value. It is a technique which is used
for optimizing the mathematical problems. One of the widely
discussed examples of Hill climbing algorithm is Traveling-
salesman Problem in which we need to minimize the distance
traveled by the salesman. It is also called greedy local search
as it only looks to its good immediate neighbor state and
not beyond that. A node of hill climbing algorithm has two
components which are state and value. Hill Climbing is mostly
used when a good heuristic is available. In this algorithm, we
don’t need to maintain and handle the search tree or graph as
it only keeps a single current state. Features of Hill Climbing
include:
1) Generate and text variant.
2) Greedy Approach
3) No BackTracking
2) Best First Search
This search uses an evaluation function which indicates
the distance of the goal node from the current node. The
evaluation function of goal node is zero. This search guar-
antees completeness but not optimality. may follow infinite
path if heuristic rates each state on such a path as the best
option. Most reasonable heuristics will not cause this problem
however. The worst case time complexity is still O(bm ) where
m is the maximum depth. Since must maintain a queue of all
unexpanded states, space-complexity is also O(bm ). However,
a good heuristic will avoid this worst-case behavior for most
problems.
3) A* search algorithm
A* Search algorithm is one of the best and popular tech-
nique used in path-finding and graph traversals. Informally
speaking, A* Search algorithms, unlike other traversal tech-
niques, it has “brains”. What it means is that it is really a
smart algorithm which separates it from the other conventional
algorithms. This fact is cleared in detail in below sections.
And it is also worth mentioning that many games and web-
based maps use this algorithm to find the shortest path very
efficiently (approximation). A* algorithm is based on two con-
ventional algorithms, namely Dijkstra’s algorithm and Greedy
Best-First-Search algorithm,
It employs a ”heuristic estimate” h(x) that gives an estimate of
the best route that goes through that node. It visits the nodes in
order of this heuristic estimate. It follows the approach of best
first search. The secret to its success is that it combines the
pieces of information that Dijkstra algorithm uses (favoring
vertices that are close to the starting point) and information
that Best- First-Search uses (favoring vertices that are close
to the goal). In the standard terminology used when talking
about A*, g(n)represents the exact cost of the path from the
starting point to any vertex n, and h(n) represents the heuristic
estimated cost from vertex n to the goal[7]. The A* algorithm
depends on evaluating the best next step in searching for a
path. This is done by evaluating each of the possible next
steps against a heuristic to give a value that can be used to
sort the list and hence determine the most likely next step. As
you can imagine this makes choosing a good heuristic for your
map and game really important to getting good path finding
performance. F (n) =g(n)+h(n)
B. General AO* Algorithm
In order to find solution in an AND-OR graph, AO*
algorithm works well similar to best first search with an
ability to handle the AND arc appropriately. The algorithm
finds an optimal path from initial node by propagating the
results like solution and change in heuristic value to the
ancestors as in algorithm.
Algorithm::Basic AO*
1) Initialize the graph to the starting node.
2) Loop until the starting node is labeled SOLVED or until
its cost goes above FUTILITY:
a) Traverse the graph, starting at the initial node and
following the current best path and accumulate the
set of nodes that are on that path and have not yet
been expanded or labelled as solved.
b) Pick one of these unexpanded nodes and expand it.
If there are no successors, assign FUTILITY as the
value of this node. Otherwise, add its successors to
the graph and for each of them compute f ’. If f ’
of any node is 0, mark that node as SOLVED.
c) Change the f ’ estimate of the newly expanded
node to reflect the new information provided by
its successors. Propagate this change backward
through the graph till the initial node. If any node
contains a successor arc whose descendants are all
solved, label the node itself as SOLVED.
A
12 11
6 B C D 10
4
G H E F
5 7 4 4
Fig. 2: Search According To AO*
Fig.1 shows a solution according to AO* algorithm.Here
path through node D is giving solution with a value 11 since
arc containing B and C is not optimal, i.e are giving value 12.
 II. M OTIVATION
AND-OR graphs have always been an area of interest for
researchers which are very less explored. AND-OR graph
is an arrangement where two siblings in a graph represent
two different actions which are complimentary and equally
important to solve the purpose. Therefore, any one of them
cannot be chosen. Both the nodes are to be considered and
solved.
In AI plannig problems involving uncertainity and feedback
can be naturally formulated as AND-OR graph. This calls
for the use of AO* algorithm. One cannot deny the fact that
currently, we either have only AO* algorithm or modified
versions of this algorithm. Though less used and even less
heard of, this algorithm can be used to solve many problems
like Travelling Salesman Problem [3]. Also on applying AO*
algorithm for TSP, we can obtain the optimal solution as the
least cost Hamiltonian route. On various integer input range,
the proposed AO* procedure can provide the solution for
travelling salesman problem.
This results in the speculation of how one can use AND-
OR graph and apply AO* algorithm to generate results.
AO* algorithm has a lot of potential and using it can prove
beneficial in a lot of problems that does not ask for an optimal
path, rather looks to find solution fast.
III. E XISTING W ORK
AO* algorithm has undergone many modifications and
transformation in recent years to make it more facilitating for
the use of AND-OR graph. Various researchers has now been
longing to use AND-OR graph as problem planning method
and AO* algorithm to find the solution. As we know, AO*
not gives an optimal solution for the problem and that made
it less productive for the programmers in the past. But now
researchers have came up with their variant of AO* algorithm
so as to use it in the mainstream problem solving procedures.
Following are the modifications and improvement over AO*
algortithm that various researchers have provided.
A. LAO*
LAO* algorithm [4], that is a heuristic search algorithm
that find solutions with loop.It can be used to solve Markov
Decision Problem and it shares the advantage heuristics search
has over dynamic programming for other classes of problems.
Given a start state, it can find an optimal solution without
evalutating the entire state space.Thus, it provides a useful
alternative to dynamic programming algorithms for MDPs
such as value iteration and policy iteration.
It is straightforward to represent an indefinite-horizon MDP
as an AND/OR graph. We let each state of the AND/OR
graph correspond to a state of the MDP and we let each k-
connector correspond to an action with k possible outcomes.
In reviewing MDPs and AND/OR graph search, we used the
same notation for their transition functions, cost functions,
and optimality equations, to help clarify the close relationship
between these two frameworks. A difference is that a solution
to an indefinite-horizon MDP contains loops (i.e., it allows
the same state to be visited more than once). Loops express
indefinitehorizon behavior. The classic AO* algorithm can
only solve problems that have acyclic solutions because the
backwards induction algorithm used in its cost revision step
assumes an acyclic solution. The key step in generalizing AO*
to create LAO* is to recognize that the cost revision step of
AO* is a dynamic programming algorithm, and to generalize
this step appropriately. Instead of using backwards induction,
state costs can be updated by using a dynamic programming
algorithm for indefinite-horizon MDPs, such as policy iteration
or value iteration.
The cost revision step of LAO* can be performed using either
policy iteration or value iteration. An advantage of using policy
iteration is that it can compute an exact cost for each state of
the explicit graph after a finite number of iterations, based on
the heuristic estimates at the tip states. When value iteration is
used, convergence to exact state costs is asymptotic. However,
this disadvantage is usually offset by the improved efficiency
of value iteration for larger problems.
Given a start state and an admissible heuristic to guide forward
expansion of a partial solution, LAO* can find an optimal
solution without evaluating the entire state space. However, it
only enjoys this advantage for problems for which an optimal
solution visits a fraction of the state space, beginning from
the start state.It would be useful to have a way of identifying
problems for which LAO* is effective. There does not appear
to be an easy answer to this question. Nevertheless, we make
a few remarks in this direction.
B. Hybrid AO*
Another manipulation is the HAO* algorithm [5], which is
a generalization of the AO* algorithm that performs search in
a hybrid state space that is modeled d using both discrete and
continuous state variables. The search algorithm leverages
knowledge of the starting state to focus computational effort
on the relevant parts of the state space. We claim that this
approach is especially effective when resource limitations
contribute to reachability constraints. Experimental results
show its effectiveness in the domain that motivates our
research – automated planning for planetary exploration
rovers.
They considered a Markov decision process (MDP) with
both continuous and discrete state variables (also called a
hybrid MDP (Guestrin et al. 2004) or Generalized State
MDP (Younes and Simmons 2004)). Each state corresponds
to an assignment to a set of state variables. These variables
may be discrete or continuous. Continuous variables typically
represent resources, where one possible type of resource is
time. Discrete variables model other aspects of the state,
including (in our application) the set of goals achieved so far
by the rover. (Keeping track of already-achieved goals ensures
a Markovian reward structure, since we reward achievement
of a goal only if it was not achieved in the past.) Although
our models typically contain multiple discrete variables, this
plays no role in the description of our algorithm, and so, for
notational convenience, they model the discrete component
as a single variable n.
A simple way of understanding HAO* is as an AO* variant
where states with identical discrete component are expanded
in unison. HAO* works with two graphs:
1) The explicit graph describes all the states that have been
generated so far and the AND/OR edges that connect
them. The nodes of the explicit graph are stored in two
lists: OPEN and CLOSED.
2) The greedy policy (or partial solution) graph, denoted
GREEDY in the algorithms, is a sub-graph of the explicit
graph describing the current optimal policy.
It is the first algorithm to deal with: limited continuous
resources, uncertainty, and oversubscription planning. They
developed a sophisticated reachability analysis involving con-
tinuous variables that could be useful for heuristic search
algorithms at large
C. MAO*
One lesser known variant of AO* algorithm is MAO* al-
gorithm, Modification to AO* [6]. The locla maxima problem
is the problem that is a hindrance to the popularity of AO*
algorithm. Many time a problem is not converted to AO*
alogrithm, the sole reason being A* is a better algorithm as
it definitely gives a optimal solutio when compared to AO*
algrithm.
To overcome the problem of local maxima, it is proposed
that whenever an AND arc is encountered between two nodes,
compare the weight of the two nodes. The node with the higher
weight should be made a child of the node with higher weight.
This algorithm generates correct solution as it overcomes the
problem of the local maxima. By employing this algorithm,
the space complexity remains same. The time complexity
increases slightly by a fraction of nano seconds. But in today’s
world, with the changing technologies and the commencement
of high speed processors, this fraction of time is quite negli-
gible.
IV. P ROPOSED W ORK
When performing AO* algorithm some non promising
nodes will be underestimated (Fig. 3). Once solution is found
AO* is not looking into that. Here in Fig. 3 arc containing
nodes B and C is becoming a promising on further exploration.
So node A is getting a better score of 10.
A
10 11
4 B C D 10
4
G 3 H E F
7 4 4
G
2
Fig. 3: Search Tree with better solution
The algorithm can be optimised by measuring the goodness
of a node with a look ahead. In figure 3 node G is further
estimated and found that I is giving a solution with better
heuristic value. If there a look ahead on node B this would
have been explored. Also node C may or may not give better
value if explored further. This look ahead is implemented
efficiently using a depth limited search on nodes examined
by step 2(b) in the basic AO* algorithm.
Now we have to delve deeper and explore the ranges that
are required to convert the proposed methodology into a
functioning alogrithm.
A. Depth Limited Search
Basically Depth Limited Search(DLS) [2] is only a Depth
First Search(DFS) with a depth limit. In a recursive method
for DFS each recursive call is performed only if depth of
the search tree is less than depth limit. But there is some
change in the DLS algorithm for AND-OR graphs since
arcs are there. The non promising nodes can be explored
efficiently by the DLS method [2]. So this modification will
propagate back a stable result to ancestors. In the figure
1 search with a depth limit 3 will give a solution with score 10.
Algorithm:
1) The start node or node 1 is added to the beginning of
the stack.
2) Then it is marked as visited and if node 1 is not the goal
node in the search, then we push second node 2 on top
of the stack.
3) Next, we mark it as visited and check if node 2 is the
goal node or not.
4) If node 2 is not found to be the goal node then we push
node 4 on top of the stack.
5) Now we search in the same depth limit and move along
depth-wise to check for the goal nodes.
6) If Node 4 is also not found to be the goal node and
depth limit is found to be reached then we retrace back
to nearest nodes that remain unvisited or unexplored.
7) Then we push them into the stack and mark them visited.
 8) We continue to perform these steps in iterative ways 2) Loop with CURRENT until the starting node is labelled
unless the goal node is reached or until all nodes within SOLVED or until its cost goes above FUTILITY:
depth limit have been explored for the goal. a) If depth limit has reached return node’s or arc’s
heuristic value.
B. Interacting Sub Problems b) Explore successor nodes and arcs of CURRENT.
Interacting sub problem is one of the limitation to the If there are no successors, assign FUTILITY as the
algorithm. Here in Fig. 4 nodes B and C are interacting. But value of this node and exit.
C finds that D optimal compared to B for solving it. This is c) Otherwise loop for each SUCCESSOR node or arc
an overhead for finding the solution since both B and D has of CURRENT.
to be solved separately. i) find unexpanded successors of the SUCCES-
SOR of CURRENT by referring Explored List
A if CURRENT is in Explored List.
ii) make recursive calls for the algorithm with
9
unexpanded successors SUCCESSOR as the
4 B C 3 starting node.
iii) if SUCCESSOR is an arc check for loops based
on the parental information for finding interact-
ing sub problems obtained from Explored List.
D Evaluate the score for the loop and assign to
2 SUCCESSOR if it is better than that of previous
step.
Fig. 4: Solution for Interacting Sub Problems in AO* iv) if SUCCESSOR is giving better value change
Considering the interaction the cost on A will get reduced the f estimate of the CURRENT node to reflect
to 7 (Fig. 5) instead of 9 without considering the node D. the new information provided by its successors.
Here B has to be done only once under the arc. So B and C v) if SUCCESSOR is giving better value call it
altogether taking a cost of 5 and the cost of the arc is 2 which the BEST SUCC.
will bring the total cost to 7. vi) add or update CURRENT and the successor in
Explored List with relevant information.
A d) Propagate this change backward through the graph
till the initial node. If any node contains a successor
7 arc whose descendants are all solved, label the
node itself as SOLVED.
4 B C 5
e) Make the BEST SUCC as the CURRENT node.
The OAO* algorithm is finding a solution according to best
first search where each node or arc performing a depth
D limited search. The Explored List is keeping an updated list
2 of explored nodes so that DLS overhead is reduced. Here the
search is started only from the unexpanded nodes because the
Fig. 5: Solution for Interacting Sub Problems in OAO* other explored nodes are already evaluated which reduces the
search overhead. The interacting sub problems are identified
C. Optimised AO*(OAO*) Algorithm
by checking loops [7] making use of parental information in
The optimization of AO* algorithm can done based the the Explored List. If loops are there the evaluation is done as
two aspects as discussed above. One is considering the non in Fig. 5. This is giving more realistic solution to problems.
promising nodes and the other is considering interacting sub
problems. The algorithm 2 is explaining the depth limited V. A NALYSIS
search on nodes which will consider the non promising nodes The Optimised AO* algorithm is always giving a better
also. An additional data structure in the form of Explored List result with new concepts like depth limited search and evalua-
containing explored nodes with the information about the tion of loop for interacting sub problems. This is verified with
parent, list of unexpanded successors obtained through depth evaluation of graphs shown in Fig. 3 and Fig. 5. The OAO*
limited search, depth and evaluated score is required for the algorithm is evaluating lesser number of nodes compared to
algorithm. AO* with the a look up in Explored List. So the explored
Algorithm OAO* nodes are not evaluated further which improves time com-
Data Structure : Explored List for explored nodes. plexity even though slight compromise with space complexity
1) Initialize the graph with the starting node as CURRENT is required. Here the algorithm offers a better consistent trace
node. for the solution by the look ahead based on a depth limited
search.
These improvement, however, could not advance the usage of
AO* algorithm to cyclic graph and thus only work in absence
of any loop. Nevertheless, the finding done by our algorithm,
i.e, OAO* algorithm has paved a way to find a optimal path
in a state space search in AND-OR graph. That all said is
just theoretical and proposed and we have yet to program
this algorithm in a programming language, most preferably
MATLAB. But the theoretical concept of our algorithm has
removed the vulnerability of AO* algorithm and thus it has
now came to ranks with A* algorithm, which is the most
famous of state space search algorithm present. The next
test that this algorithm is to underatke is fight with the A*
algorithm for finding optimal solution of cyclic graph for state
space search. The heuristic used will have a impact in both.
R EFERENCES
[1] Artificial Intelligence by Elaine Rich and Kevin Knight, McGraw Hill
(1991).
[2] Richard E. korf, Depth limited search for real time problem solving,
The journal of real time systems,2,1990,7-24.
[3] Shraddha Ramteke, Mrs. Deepty Dubey, AO* algorithm for solving
travelling salesman problem.
[4] Eric A. Hansen, Shlomo Zilberstein, LAO*: A heuristic search algorithm
that finds solutions with loops
[5] Emmanuel Benazera Ronen Brafman, Nicolas Meuleau, An AO* Algo-
rithm for Planning with Continuous Resources
[6] Dr. Mahesh Kumar,Shikha Aggarwal,An AO* Algorithm for Planning
with Continuous Resources
[7] Eric A. Hansen, Shlomo Zilberstein, LAO*: A heuristic search algorithm
that finds solutions with loops, Artificial Intelligence, 129, 2001, 35–62.
[8] Ethan Burns, Sofia Lemons, Wheeler Ruml, Rong Zhou, Best-First
Heuristic Search for Multicore Machines, Journal of Artificial Intelli-
gence Research, 39, 2010, 689–743.

C ONCLUSION
The AO* algorithm, as stated previously has many flaws
that kept it away from the mainstream usage and most of
the academic syllabus. Unlike A* algorithm, which finds an
optimal path, AO* suffered from the problem of leaving the
search after it found a solution and surely does not care for
optimal solution. AO* algorithm, thus is considered unstable.
But as we have included, there have been variations of the
algorithm and those are improving the results obatined and
making AO* algorithm optimal.
Among the above mentioned modifications of the algorithm,
we propose our OAO* algorithm that is short for Optimal
AO* algorithm. The OAO* algorithm has taken care of major
drawbacks basic AO* algorithm. The proposed algorithm gives
a stable heuristic search by avoiding underestimation of non
promising nodes and overestimation of promising nodes. The
algorithm has handled the problem of interacting sub problems
to give better heuristic search. Also the algorithm is giving
more consistent search path compared to AO* algorithm.
As we have seen that LAO* algorithm is able to find solution
in a cyclic graph whereas normal AO* algorithm is only able to
find solution in acyclic graph, though not optimal. Our OAO*
algorithm is able to find an optimal solution in cyclic graph,
an improvement over normal AO* algorithm, but it still lack
the approach to find the solution in a cyclic graph. What next
we can do is to merge the two algorithm so as to advance the
working of our algorithm so that cyclic graph, when made to
AND-OR graph, will be able to find the optimal solution.

VI. ACKNOWLEDGMENT
We are grateful to everyone who has diretly or indirectly
helped us in completing this research paper. We are very
thankful to Dr. Abhimanyu Kumar, who gave us the op-
portunity to go beyond our institute syllabus and find what
other technique are present outside the course.There are many
researchers whose works helped us in this research paper.
This project work has undertaken a comprehensive study of
Artificial Intelligence and Graph Theory ideas and has helped
us in our academics also.